Hi Allison,
On 10 Sep 2025, at 20:47, Allison Henderson allison.henderson@oracle.com wrote:
On Wed, 2025-09-10 at 13:04 +0200, Håkon Bugge wrote:
We need to increment i_fastreg_wrs before we bail out from rds_ib_post_reg_frmr().
We have a fixed budget of how many FRWR operations that can be outstanding using the dedicated QP used for memory registrations and de-registrations. This budget is enforced by the atomic_t i_fastreg_wrs. If we bail out early in rds_ib_post_reg_frmr(), we will "leak" the possibility of posting an FRWR operation, and if that accumulates, no FRWR operation can be carried out.
Hi Håkon,
This sounds much clearer, thank you!
Fixes: 1659185fb4d0 ("RDS: IB: Support Fastreg MR (FRMR) memory registration mode") Fixes: 3a2886cca703 ("net/rds: Keep track of and wait for FRWR segments in use upon shutdown") Cc: stable@vger.kernel.org Signed-off-by: Håkon Bugge haakon.bugge@oracle.com
v2 -> v3:
- Amended commit message
- Removed indentation of this section
- Fixing error path from ib_post_send()
v1 -> v2: Added Cc: stable@vger.kernel.org
net/rds/ib_frmr.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/net/rds/ib_frmr.c b/net/rds/ib_frmr.c index 28c1b00221780..395a99b5a65ca 100644 --- a/net/rds/ib_frmr.c +++ b/net/rds/ib_frmr.c @@ -133,12 +133,15 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr)
ret = ib_map_mr_sg_zbva(frmr->mr, ibmr->sg, ibmr->sg_dma_len, &off, PAGE_SIZE);
- if (unlikely(ret != ibmr->sg_dma_len))
- return ret < 0 ? ret : -EINVAL;
- if (unlikely(ret != ibmr->sg_dma_len)) {
- ret = ret < 0 ? ret : -EINVAL;
- goto out_inc;
- }
- if (cmpxchg(&frmr->fr_state,
FRMR_IS_FREE, FRMR_IS_INUSE) != FRMR_IS_FREE)- return -EBUSY;
- if (cmpxchg(&frmr->fr_state, FRMR_IS_FREE, FRMR_IS_INUSE) != FRMR_IS_FREE) {
- ret = -EBUSY;
- goto out_inc;
- }
atomic_inc(&ibmr->ic->i_fastreg_inuse_count);
@@ -166,11 +169,10 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr) /* Failure here can be because of -ENOMEM as well */ rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_STALE);
- atomic_inc(&ibmr->ic->i_fastreg_wrs);
if (printk_ratelimit()) pr_warn("RDS/IB: %s returned error(%d)\n", __func__, ret);
- goto out;
Just one nit: This was the only place the out label was used isnt it? If so, lets go ahead and clear it out.
That is indeed true!
- goto out_inc;
}
/* Wait for the registration to complete in order to prevent an invalid @@ -178,9 +180,11 @@ static int rds_ib_post_reg_frmr(struct rds_ib_mr *ibmr)
- being accessed while registration is still pending.
*/ wait_event(frmr->fr_reg_done, !frmr->fr_reg);
out:
With that fixed, you can go ahead and add my rvb: Reviewed-by: Allison Henderson allison.henderson@oracle.com
Will do, thanks for the r-b!
Håkon
Thank you!
- return ret;
+out_inc:
- atomic_inc(&ibmr->ic->i_fastreg_wrs);
return ret; }