Fix for TPROXY initialization of fastreuse flag. upstream patch was backported by Greg K-H to 4.14 and higher stable version.
see also https://lore.kernel.org/stable/20200818072007.GA9254@kroah.com/
code in inet_csk_get_port for 4.9 (and 4.4) is different from the upstream version, so these backports had to be adapted (a bit)
Tim Froidcoeur (2): net: refactor bind_bucket fastreuse into helper net: initialize fastreuse on inet_inherit_port
include/net/inet_connection_sock.h | 4 ++++ net/ipv4/inet_connection_sock.c | 37 ++++++++++++++++++++---------- net/ipv4/inet_hashtables.c | 1 + 3 files changed, 30 insertions(+), 12 deletions(-)
-- 2.25.1
[ Upstream commit 62ffc589abb176821662efc4525ee4ac0b9c3894 ]
Refactor the fastreuse update code in inet_csk_get_port into a small helper function that can be called from other places.
Acked-by: Matthieu Baerts matthieu.baerts@tessares.net Signed-off-by: Tim Froidcoeur tim.froidcoeur@tessares.net Signed-off-by: David S. Miller davem@davemloft.net Signed-off-by: Tim Froidcoeur tim.froidcoeur@tessares.net --- include/net/inet_connection_sock.h | 4 ++++ net/ipv4/inet_connection_sock.c | 37 ++++++++++++++++++++---------- 2 files changed, 29 insertions(+), 12 deletions(-)
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h index 146054ceea8e..5bb56ebf3c9f 100644 --- a/include/net/inet_connection_sock.h +++ b/include/net/inet_connection_sock.h @@ -319,5 +319,9 @@ int inet_csk_compat_getsockopt(struct sock *sk, int level, int optname, int inet_csk_compat_setsockopt(struct sock *sk, int level, int optname, char __user *optval, unsigned int optlen);
+/* update the fast reuse flag when adding a socket */ +void inet_csk_update_fastreuse(struct inet_bind_bucket *tb, + struct sock *sk); + struct dst_entry *inet_csk_update_pmtu(struct sock *sk, u32 mtu); #endif /* _INET_CONNECTION_SOCK_H */ diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 1bcbb7399fe6..5a0352ccadd3 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -89,6 +89,28 @@ int inet_csk_bind_conflict(const struct sock *sk, } EXPORT_SYMBOL_GPL(inet_csk_bind_conflict);
+void inet_csk_update_fastreuse(struct inet_bind_bucket *tb, + struct sock *sk) +{ + kuid_t uid = sock_i_uid(sk); + bool reuse = sk->sk_reuse && sk->sk_state != TCP_LISTEN; + + if (!hlist_empty(&tb->owners)) { + if (!reuse) + tb->fastreuse = 0; + if (!sk->sk_reuseport || !uid_eq(tb->fastuid, uid)) + tb->fastreuseport = 0; + } else { + tb->fastreuse = reuse; + if (sk->sk_reuseport) { + tb->fastreuseport = 1; + tb->fastuid = uid; + } else { + tb->fastreuseport = 0; + } + } +} + /* Obtain a reference to a local port for the given sock, * if snum is zero it means select any available local port. * We try to allocate an odd port (and leave even ports for connect()) @@ -218,19 +240,10 @@ int inet_csk_get_port(struct sock *sk, unsigned short snum) } goto fail_unlock; } - if (!reuse) - tb->fastreuse = 0; - if (!sk->sk_reuseport || !uid_eq(tb->fastuid, uid)) - tb->fastreuseport = 0; - } else { - tb->fastreuse = reuse; - if (sk->sk_reuseport) { - tb->fastreuseport = 1; - tb->fastuid = uid; - } else { - tb->fastreuseport = 0; - } } + + inet_csk_update_fastreuse(tb, sk); + success: if (!inet_csk(sk)->icsk_bind_hash) inet_bind_hash(sk, tb, port);
[ Upstream commit d76f3351cea2d927fdf70dd7c06898235035e84e ]
In the case of TPROXY, bind_conflict optimizations for SO_REUSEADDR or SO_REUSEPORT are broken, possibly resulting in O(n) instead of O(1) bind behaviour or in the incorrect reuse of a bind.
the kernel keeps track for each bind_bucket if all sockets in the bind_bucket support SO_REUSEADDR or SO_REUSEPORT in two fastreuse flags. These flags allow skipping the costly bind_conflict check when possible (meaning when all sockets have the proper SO_REUSE option).
For every socket added to a bind_bucket, these flags need to be updated. As soon as a socket that does not support reuse is added, the flag is set to false and will never go back to true, unless the bind_bucket is deleted.
Note that there is no mechanism to re-evaluate these flags when a socket is removed (this might make sense when removing a socket that would not allow reuse; this leaves room for a future patch).
For this optimization to work, it is mandatory that these flags are properly initialized and updated.
When a child socket is created from a listen socket in __inet_inherit_port, the TPROXY case could create a new bind bucket without properly initializing these flags, thus preventing the optimization to work. Alternatively, a socket not allowing reuse could be added to an existing bind bucket without updating the flags, causing bind_conflict to never be called as it should.
Call inet_csk_update_fastreuse when __inet_inherit_port decides to create a new bind_bucket or use a different bind_bucket than the one of the listen socket.
Fixes: 093d282321da ("tproxy: fix hash locking issue when using port redirection in __inet_inherit_port()") Acked-by: Matthieu Baerts matthieu.baerts@tessares.net Signed-off-by: Tim Froidcoeur tim.froidcoeur@tessares.net Signed-off-by: David S. Miller davem@davemloft.net Signed-off-by: Tim Froidcoeur tim.froidcoeur@tessares.net --- net/ipv4/inet_hashtables.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 4bf542f4d980..887633870763 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -163,6 +163,7 @@ int __inet_inherit_port(const struct sock *sk, struct sock *child) return -ENOMEM; } } + inet_csk_update_fastreuse(tb, child); } inet_bind_hash(child, tb, port); spin_unlock(&head->lock);
On Wed, Sep 02, 2020 at 04:05:43PM +0200, Tim Froidcoeur wrote:
Fix for TPROXY initialization of fastreuse flag. upstream patch was backported by Greg K-H to 4.14 and higher stable version.
see also https://lore.kernel.org/stable/20200818072007.GA9254@kroah.com/
code in inet_csk_get_port for 4.9 (and 4.4) is different from the upstream version, so these backports had to be adapted (a bit)
Tim Froidcoeur (2): net: refactor bind_bucket fastreuse into helper net: initialize fastreuse on inet_inherit_port
include/net/inet_connection_sock.h | 4 ++++ net/ipv4/inet_connection_sock.c | 37 ++++++++++++++++++++---------- net/ipv4/inet_hashtables.c | 1 + 3 files changed, 30 insertions(+), 12 deletions(-)
-- 2.25.1
Both now queued up, thanks.
greg k-h
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Tim Froidcoeur