From: Jason Gunthorpe jgg@nvidia.com
[ Upstream commit 4aa1615268a8ac2b20096211d3f9ac53874067d7 ]
rxe will hold a refcount on the IB device as long as CQ objects exist, this causes destruction of a rxe device to hang if the CQ pool has any cached CQs since they are being destroyed after the refcount must go to zero.
Treat the CQ pool like a client and create/destroy it before/after all other clients. No users of CQ pool can exist past a client remove call.
Link: https://lore.kernel.org/r/e8a240aa-9e9b-3dca-062f-9130b787f29b@acm.org Fixes: c7ff819aefea ("RDMA/core: Introduce shared CQ pool API") Tested-by: Bart Van Assche bvanassche@acm.org Tested-by: Yi Zhang yi.zhang@redhat.com Signed-off-by: Bart Van Assche bvanassche@acm.org Signed-off-by: Jason Gunthorpe jgg@nvidia.com Signed-off-by: Sasha Levin sashal@kernel.org --- drivers/infiniband/core/device.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index eadba29432dd7..abcfe4dc1284f 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -1282,6 +1282,8 @@ static void disable_device(struct ib_device *device) remove_client_context(device, cid); }
+ ib_cq_pool_destroy(device); + /* Pairs with refcount_set in enable_device */ ib_device_put(device); wait_for_completion(&device->unreg_completion); @@ -1325,6 +1327,8 @@ static int enable_device_and_get(struct ib_device *device) goto out; }
+ ib_cq_pool_init(device); + down_read(&clients_rwsem); xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) { ret = add_client_context(device, client); @@ -1397,7 +1401,6 @@ int ib_register_device(struct ib_device *device, const char *name) goto dev_cleanup; }
- ib_cq_pool_init(device); ret = enable_device_and_get(device); dev_set_uevent_suppress(&device->dev, false); /* Mark for userspace that device is ready */ @@ -1452,7 +1455,6 @@ static void __ib_unregister_device(struct ib_device *ib_dev) goto out;
disable_device(ib_dev); - ib_cq_pool_destroy(ib_dev);
/* Expedite removing unregistered pointers from the hash table */ free_netdevs(ib_dev);