[PATCH 6.17 632/849] net: phy: dp83640: improve phydev and driver removal handling