[PATCH 6.6.y] spi: spi-cadence-quadspi: Fix pm runtime unbalance