On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57c
These versions don't have the TCP fix backported. Please do so.
Thanks, Siddh
On Tue, Sep 03, 2024 at 11:56:17AM +0000, Siddh Raman Pant wrote:
On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57c
These versions don't have the TCP fix backported. Please do so.
What fix backported exactly to where? Please be more specific. Better yet, please provide working, and tested, backports.
confused,
greg k-h
On Tue, Sep 3, 2024 at 2:07 PM gregkh@linuxfoundation.org gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 11:56:17AM +0000, Siddh Raman Pant wrote:
On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57cThese versions don't have the TCP fix backported. Please do so.
What fix backported exactly to where? Please be more specific. Better yet, please provide working, and tested, backports.
commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 Author: Stanislav Fomichev sdf@fomichev.me Date: Wed Nov 8 13:13:25 2023 -0800
net: set SOCK_RCU_FREE before inserting socket into hashtable ... Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF")
It seems 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 has not been pushed to 5.10 or 5.4 lts
Stanislav mentioned a WARN_ONCE() being hit, I presume we could push the patch to 5.10 and 5.4.
I guess this was skipped because of a merge conflict.
On Tue, Sep 03, 2024 at 02:53:57PM +0200, Eric Dumazet wrote:
On Tue, Sep 3, 2024 at 2:07 PM gregkh@linuxfoundation.org gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 11:56:17AM +0000, Siddh Raman Pant wrote:
On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57cThese versions don't have the TCP fix backported. Please do so.
What fix backported exactly to where? Please be more specific. Better yet, please provide working, and tested, backports.
commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 Author: Stanislav Fomichev sdf@fomichev.me Date: Wed Nov 8 13:13:25 2023 -0800
net: set SOCK_RCU_FREE before inserting socket into hashtable... Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF")
It seems 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 has not been pushed to 5.10 or 5.4 lts
Stanislav mentioned a WARN_ONCE() being hit, I presume we could push the patch to 5.10 and 5.4.
I guess this was skipped because of a merge conflict.
Yes, the commit does not apply, we need someone to send a working backport for us to be able to take it.
Siddh, can you please do this?
thanks,
greg k-h
On Tue, Sep 03 2024 at 18:28:14 +0530, gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 02:53:57PM +0200, Eric Dumazet wrote:
On Tue, Sep 3, 2024 at 2:07 PM gregkh@linuxfoundation.org gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 11:56:17AM +0000, Siddh Raman Pant wrote:
On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57cThese versions don't have the TCP fix backported. Please do so.
What fix backported exactly to where? Please be more specific. Better yet, please provide working, and tested, backports.
commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 Author: Stanislav Fomichev sdf@fomichev.me Date: Wed Nov 8 13:13:25 2023 -0800
net: set SOCK_RCU_FREE before inserting socket into hashtable... Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF")
It seems 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 has not been pushed to 5.10 or 5.4 lts
Stanislav mentioned a WARN_ONCE() being hit, I presume we could push the patch to 5.10 and 5.4.
I guess this was skipped because of a merge conflict.
Yes, the commit does not apply, we need someone to send a working backport for us to be able to take it.
Siddh, can you please do this?
Sure.
I see there are Stable-dep commits too, but the seem unrelated and require some commits from another feature patchset. Do I need to backport them too?
Thanks, Siddh
On Wed, Sep 04, 2024 at 11:26:36AM +0000, Siddh Raman Pant wrote:
On Tue, Sep 03 2024 at 18:28:14 +0530, gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 02:53:57PM +0200, Eric Dumazet wrote:
On Tue, Sep 3, 2024 at 2:07 PM gregkh@linuxfoundation.org gregkh@linuxfoundation.org wrote:
On Tue, Sep 03, 2024 at 11:56:17AM +0000, Siddh Raman Pant wrote:
On Mon, 29 Jul 2024 16:32:36 +0200, Greg Kroah-Hartman wrote:
In the Linux kernel, the following vulnerability has been resolved:
udp: Set SOCK_RCU_FREE earlier in udp_lib_get_port().
[...]
We had the same bug in TCP and fixed it in commit 871019b22d1b ("net: set SOCK_RCU_FREE before inserting socket into hashtable").
Let's apply the same fix for UDP.
[...]
The Linux kernel CVE team has assigned CVE-2024-41041 to this issue.
Affected and fixed versions
Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.4.280 with commit 7a67c4e47626 Issue introduced in 4.20 with commit 6acc9b432e67 and fixed in 5.10.222 with commit 9f965684c57cThese versions don't have the TCP fix backported. Please do so.
What fix backported exactly to where? Please be more specific. Better yet, please provide working, and tested, backports.
commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 Author: Stanislav Fomichev sdf@fomichev.me Date: Wed Nov 8 13:13:25 2023 -0800
net: set SOCK_RCU_FREE before inserting socket into hashtable... Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF")
It seems 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 has not been pushed to 5.10 or 5.4 lts
Stanislav mentioned a WARN_ONCE() being hit, I presume we could push the patch to 5.10 and 5.4.
I guess this was skipped because of a merge conflict.
Yes, the commit does not apply, we need someone to send a working backport for us to be able to take it.
Siddh, can you please do this?
Sure.
I see there are Stable-dep commits too, but the seem unrelated and require some commits from another feature patchset. Do I need to backport them too?
Do what you think you need to do :)
[ Upstream commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 ]
We've started to see the following kernel traces:
WARNING: CPU: 83 PID: 0 at net/core/filter.c:6641 sk_lookup+0x1bd/0x1d0
Call Trace: <IRQ> __bpf_skc_lookup+0x10d/0x120 bpf_sk_lookup+0x48/0xd0 bpf_sk_lookup_tcp+0x19/0x20 bpf_prog_<redacted>+0x37c/0x16a3 cls_bpf_classify+0x205/0x2e0 tcf_classify+0x92/0x160 __netif_receive_skb_core+0xe52/0xf10 __netif_receive_skb_list_core+0x96/0x2b0 napi_complete_done+0x7b5/0xb70 <redacted>_poll+0x94/0xb0 net_rx_action+0x163/0x1d70 __do_softirq+0xdc/0x32e asm_call_irq_on_stack+0x12/0x20 </IRQ> do_softirq_own_stack+0x36/0x50 do_softirq+0x44/0x70
__inet_hash can race with lockless (rcu) readers on the other cpus:
__inet_hash __sk_nulls_add_node_rcu <- (bpf triggers here) sock_set_flag(SOCK_RCU_FREE)
Let's move the SOCK_RCU_FREE part up a bit, before we are inserting the socket into hashtables. Note, that the race is really harmless; the bpf callers are handling this situation (where listener socket doesn't have SOCK_RCU_FREE set) correctly, so the only annoyance is a WARN_ONCE.
More details from Eric regarding SOCK_RCU_FREE timeline:
Commit 3b24d854cb35 ("tcp/dccp: do not touch listener sk_refcnt under synflood") added SOCK_RCU_FREE. At that time, the precise location of sock_set_flag(sk, SOCK_RCU_FREE) did not matter, because the thread calling __inet_hash() owns a reference on sk. SOCK_RCU_FREE was only tested at dismantle time.
Commit 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF") started checking SOCK_RCU_FREE _after_ the lookup to infer whether the refcount has been taken care of.
Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF") Reviewed-by: Eric Dumazet edumazet@google.com Signed-off-by: Stanislav Fomichev sdf@google.com Reviewed-by: Kuniyuki Iwashima kuniyu@amazon.com Signed-off-by: David S. Miller davem@davemloft.net [Resolved conflict for 5.10 and below.] Signed-off-by: Siddh Raman Pant siddh.raman.pant@oracle.com --- net/ipv4/inet_hashtables.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 56deddeac1b0..0fb5d758264f 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -653,6 +653,7 @@ int __inet_hash(struct sock *sk, struct sock *osk) if (err) goto unlock; } + sock_set_flag(sk, SOCK_RCU_FREE); if (IS_ENABLED(CONFIG_IPV6) && sk->sk_reuseport && sk->sk_family == AF_INET6) __sk_nulls_add_node_tail_rcu(sk, &ilb->nulls_head); @@ -660,7 +661,6 @@ int __inet_hash(struct sock *sk, struct sock *osk) __sk_nulls_add_node_rcu(sk, &ilb->nulls_head); inet_hash2(hashinfo, sk); ilb->count++; - sock_set_flag(sk, SOCK_RCU_FREE); sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1); unlock: spin_unlock(&ilb->lock);
On Wed, Sep 04, 2024 at 01:06:45PM +0000, Siddh Raman Pant wrote:
[ Upstream commit 871019b22d1bcc9fab2d1feba1b9a564acbb6e99 ]
We've started to see the following kernel traces:
WARNING: CPU: 83 PID: 0 at net/core/filter.c:6641 sk_lookup+0x1bd/0x1d0
Call Trace:
<IRQ> __bpf_skc_lookup+0x10d/0x120 bpf_sk_lookup+0x48/0xd0 bpf_sk_lookup_tcp+0x19/0x20 bpf_prog_<redacted>+0x37c/0x16a3 cls_bpf_classify+0x205/0x2e0 tcf_classify+0x92/0x160 __netif_receive_skb_core+0xe52/0xf10 __netif_receive_skb_list_core+0x96/0x2b0 napi_complete_done+0x7b5/0xb70 <redacted>_poll+0x94/0xb0 net_rx_action+0x163/0x1d70 __do_softirq+0xdc/0x32e asm_call_irq_on_stack+0x12/0x20 </IRQ> do_softirq_own_stack+0x36/0x50 do_softirq+0x44/0x70
__inet_hash can race with lockless (rcu) readers on the other cpus:
__inet_hash __sk_nulls_add_node_rcu <- (bpf triggers here) sock_set_flag(SOCK_RCU_FREE)
Let's move the SOCK_RCU_FREE part up a bit, before we are inserting the socket into hashtables. Note, that the race is really harmless; the bpf callers are handling this situation (where listener socket doesn't have SOCK_RCU_FREE set) correctly, so the only annoyance is a WARN_ONCE.
More details from Eric regarding SOCK_RCU_FREE timeline:
Commit 3b24d854cb35 ("tcp/dccp: do not touch listener sk_refcnt under synflood") added SOCK_RCU_FREE. At that time, the precise location of sock_set_flag(sk, SOCK_RCU_FREE) did not matter, because the thread calling __inet_hash() owns a reference on sk. SOCK_RCU_FREE was only tested at dismantle time.
Commit 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF") started checking SOCK_RCU_FREE _after_ the lookup to infer whether the refcount has been taken care of.
Fixes: 6acc9b432e67 ("bpf: Add helper to retrieve socket in BPF") Reviewed-by: Eric Dumazet edumazet@google.com Signed-off-by: Stanislav Fomichev sdf@google.com Reviewed-by: Kuniyuki Iwashima kuniyu@amazon.com Signed-off-by: David S. Miller davem@davemloft.net [Resolved conflict for 5.10 and below.] Signed-off-by: Siddh Raman Pant siddh.raman.pant@oracle.com
net/ipv4/inet_hashtables.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Now queued up, thanks.
greg k-h
linux-stable-mirror@lists.linaro.org
-
Eric Dumazet -
gregkh＠linuxfoundation.org -
Siddh Raman Pant