From: Daniel Wagner wagi@kernel.org
[ Upstream commit b71cbcf7d170e51148d5467820ae8a72febcb651 ]
nvme_fc_ctrl_put can acquire the rport lock when freeing the ctrl object:
nvme_fc_ctrl_put nvme_fc_ctrl_free spin_lock_irqsave(rport->lock)
Thus we can't hold the rport lock when calling nvme_fc_ctrl_put.
Justin suggested use the safe list iterator variant because nvme_fc_ctrl_put will also modify the rport->list.
Cc: Justin Tee justin.tee@broadcom.com Reviewed-by: Christoph Hellwig hch@lst.de Signed-off-by: Daniel Wagner wagi@kernel.org Signed-off-by: Keith Busch kbusch@kernel.org Signed-off-by: Sasha Levin sashal@kernel.org ---
LLM Generated explanations, may be completely bogus:
This shows the affected function was introduced in v5.8-rc1 (`14fd1e98afafc`), meaning this deadlock bug has existed since **Linux 5.8** and affects all stable kernels from 5.8 onwards (5.10.y, 5.15.y, 6.1.y, 6.6.y, etc.).
### SUMMARY
**What the commit fixes:** A **deadlock bug** in the NVMe-FC (Fibre Channel) driver where `nvme_fc_match_disconn_ls()` holds `rport->lock` while calling `nvme_fc_ctrl_put()`. When the reference count hits zero, `nvme_fc_ctrl_free()` tries to acquire the same lock, causing a deadlock.
**Stable kernel criteria:** | Criterion | Assessment | |-----------|------------| | Obviously correct | ✅ Uses canonical lock drop/reacquire pattern | | Fixes real bug | ✅ Deadlock - system hang | | Important issue | ✅ Deadlocks in storage paths are critical | | Small and contained | ✅ ~6 lines in one function | | No new features | ✅ Pure bug fix | | Expert reviewed | ✅ Christoph Hellwig |
**Risk vs Benefit:** - **Risk:** LOW - The fix uses a well-established kernel pattern (`list_for_each_entry_safe` + lock release/reacquire) - **Benefit:** HIGH - Prevents deadlock in NVMe-FC storage driver used in enterprise environments
**Concerns:** - No explicit `Cc: stable` tag, but this is not required for obvious bug fixes - No `Fixes:` tag, but we've identified the bug exists since v5.8 - The fix should apply cleanly to any kernel with the affected function (5.8+)
### CONCLUSION
This commit fixes a clear deadlock bug in the NVMe-FC driver that has existed since Linux 5.8. The fix is: - Small and surgical (only ~6 lines changed) - Uses well-understood, standard kernel locking patterns - Has been reviewed by a respected kernel developer (Christoph Hellwig) - Signed off by the NVMe maintainer (Keith Busch) - Affects enterprise storage users who rely on NVMe over Fibre Channel
Deadlocks in storage drivers are serious issues that warrant stable backporting. The minimal scope and established fix pattern make this a low-risk, high-value backport.
**YES**
drivers/nvme/host/fc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 2c903729b0b90..8324230c53719 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -1468,14 +1468,14 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport, { struct fcnvme_ls_disconnect_assoc_rqst *rqst = &lsop->rqstbuf->rq_dis_assoc; - struct nvme_fc_ctrl *ctrl, *ret = NULL; + struct nvme_fc_ctrl *ctrl, *tmp, *ret = NULL; struct nvmefc_ls_rcv_op *oldls = NULL; u64 association_id = be64_to_cpu(rqst->associd.association_id); unsigned long flags;
spin_lock_irqsave(&rport->lock, flags);
- list_for_each_entry(ctrl, &rport->ctrl_list, ctrl_list) { + list_for_each_entry_safe(ctrl, tmp, &rport->ctrl_list, ctrl_list) { if (!nvme_fc_ctrl_get(ctrl)) continue; spin_lock(&ctrl->lock); @@ -1488,7 +1488,9 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport, if (ret) /* leave the ctrl get reference */ break; + spin_unlock_irqrestore(&rport->lock, flags); nvme_fc_ctrl_put(ctrl); + spin_lock_irqsave(&rport->lock, flags); }
spin_unlock_irqrestore(&rport->lock, flags);