Hello Neil, Hello Trond,
On Do, Sep 08, 2022 at 01:09:29 +0000, Trond Myklebust wrote:
On Thu, 2022-09-08 at 10:58 +1000, NeilBrown wrote:
On Thu, 08 Sep 2022, Eugeniu Rosca wrote:
Hello all,
On Mo, Apr 18, 2022 at 02:10:03 +0200, Greg Kroah-Hartman wrote:
From: NeilBrown neilb@suse.de
commit 3848e96edf4788f772d83990022fa7023a233d83 upstream.
xprt_destory() claims XPRT_LOCKED and then calls del_timer_sync(). Both xprt_unlock_connect() and xprt_release() call ->release_xprt() which drops XPRT_LOCKED and *then* xprt_schedule_autodisconnect() which calls mod_timer().
This may result in mod_timer() being called *after* del_timer_sync(). When this happens, the timer may fire long after the xprt has been freed, and run_timer_softirq() will probably crash.
The pairing of ->release_xprt() and xprt_schedule_autodisconnect() is always called under ->transport_lock. So if we take -
transport_lock to
call del_timer_sync(), we can be sure that mod_timer() will run first (if it runs at all).
Cc: stable@vger.kernel.org Signed-off-by: NeilBrown neilb@suse.de Signed-off-by: Trond Myklebust https://urldefense.proofpoint.com/v2/url?u=http-3A__trond.myklebust-40hammerspace.com&d=DwIGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=SAhjP5GOmrADp1v_EE5jWoSuMlYCIt9gKduw-DCBPLs&m=HrAc1Ouz3sNPrdBv5QBgh7SToNI8M0iGJyDPgOTT5AE&s=cNzW7c7t83SH27ck7hxvneR9awrt17JualiCD6TZtdI&e= Signed-off-by: Greg Kroah-Hartman gregkh@linuxfoundation.org
net/sunrpc/xprt.c | 7 +++++++ 1 file changed, 7 insertions(+)
--- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1520,7 +1520,14 @@ static void xprt_destroy(struct rpc_xprt */ wait_on_bit_lock(&xprt->state, XPRT_LOCKED, TASK_UNINTERRUPTIBLE);
/** xprt_schedule_autodisconnect() can run afterXPRT_LOCKED
* is cleared. We use ->transport_lock to ensure themod_timer()
* can only run *before* del_time_sync(), never after.*/spin_lock(&xprt->transport_lock); del_timer_sync(&xprt->timer);spin_unlock(&xprt->transport_lock);I think it is sufficient to change the to spin_{,un}lock_bh() in older kernels. The spinlock call need to match other uses of the same lock.
Agreed. On older kernels, the xprt->transport_lock served the same purpose, but it had to take a bh-safe spinlock in order to avoid certain races with the socket callbacks. Since then, a number of changes to both the socket layer and the SUNRPC code have made it possible to eliminate bh-safe requirement.
Can you confirm doing that removes the problem?
Your proposal [*] seems to resolve the issue for me.
Any chance to get a stable patch, to which I will gladly provide the Reviewed-by/Tested-by signatures?
NeilBrown
[*] diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index e7d55d63d4f1..7f9b94acf597 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1525,9 +1525,9 @@ static void xprt_destroy(struct rpc_xprt *xprt) * is cleared. We use ->transport_lock to ensure the mod_timer() * can only run *before* del_time_sync(), never after. */ - spin_lock(&xprt->transport_lock); + spin_lock_bh(&xprt->transport_lock); del_timer_sync(&xprt->timer); - spin_unlock(&xprt->transport_lock); + spin_unlock_bh(&xprt->transport_lock);
/* * Destroy sockets etc from the system workqueue so they can
Best Regards, Eugeniu Rosca