[PATCH 5.10 079/299] soundwire: cadence: only prepare attached devices on clock stop