From: Eric Dumazet edumazet@google.com
[ Upstream commit 38ec4944b593fd90c5ef42aaaa53e66ae5769d04 ]
After commit 0f6925b3e8da ("virtio_net: Do not pull payload in skb->head") Guenter Roeck reported one failure in his tests using sh architecture.
After much debugging, we have been able to spot silent unaligned accesses in inet_gro_receive()
The issue at hand is that upper networking stacks assume their header is word-aligned. Low level drivers are supposed to reserve NET_IP_ALIGN bytes before the Ethernet header to make that happen.
This patch hardens skb_gro_reset_offset() to not allow frag0 fast-path if the fragment is not properly aligned.
Some arches like x86, arm64 and powerpc do not care and define NET_IP_ALIGN as 0, this extra check will be a NOP for them.
Note that if frag0 is not used, GRO will call pskb_may_pull() as many times as needed to pull network and transport headers.
[ Backport note ]
A small conflict has been reported:
++<<<<<<< HEAD + if (skb_mac_header(skb) == skb_tail_pointer(skb) && + pinfo->nr_frags && + !PageHighMem(skb_frag_page(frag0))) { ++======= + if (!skb_headlen(skb) && pinfo->nr_frags && + !PageHighMem(skb_frag_page(frag0)) && + (!NET_IP_ALIGN || !(skb_frag_off(frag0) & 3))) { ++>>>>>>> 38ec4944b593 (gro: ensure frag0 meets IP header alignment)
This is expected because older kernels are missing commit 8aef998df3979 ("net: core: allow fast GRO for skbs with Ethernet header in head"). This commit modifies the beginning of the 'if' statement.
The resolution was easy: the patch we want to backport here is adding new conditions to the 'if' statement:
&& (!NET_IP_ALIGN || !(skb_frag_off(frag0) & 3))
We simply append these new conditions to it on older kernels.
Another issue had to be resolved: skb_frag_off() is used in this patch we want to backport but this function is not defined in kernels < 5.4. It has then been extracted and imported from commit 7240b60c98d63 ("linux: Add skb_frag_t page_offset accessors").
Fixes: 0f6925b3e8da ("virtio_net: Do not pull payload in skb->head") Fixes: 78a478d0efd9 ("gro: Inline skb_gro_header and cache frag0 virtual address") Signed-off-by: Eric Dumazet edumazet@google.com Reported-by: Guenter Roeck linux@roeck-us.net Cc: Xuan Zhuo xuanzhuo@linux.alibaba.com Cc: "Michael S. Tsirkin" mst@redhat.com Cc: Jason Wang jasowang@redhat.com Acked-by: Michael S. Tsirkin mst@redhat.com Tested-by: Guenter Roeck linux@roeck-us.net Signed-off-by: David S. Miller davem@davemloft.net Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- include/linux/skbuff.h | 9 +++++++++ net/core/dev.c | 3 ++- 2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 06176ef2a842..5f2e6451ece5 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -2788,6 +2788,15 @@ static inline void skb_propagate_pfmemalloc(struct page *page, skb->pfmemalloc = true; }
+/** + * skb_frag_off() - Returns the offset of a skb fragment + * @frag: the paged fragment + */ +static inline unsigned int skb_frag_off(const skb_frag_t *frag) +{ + return frag->page_offset; +} + /** * skb_frag_page - retrieve the page referred to by a paged fragment * @frag: the paged fragment diff --git a/net/core/dev.c b/net/core/dev.c index 722ae0b57f3f..a6798117bb1a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5400,7 +5400,8 @@ static void skb_gro_reset_offset(struct sk_buff *skb)
if (skb_mac_header(skb) == skb_tail_pointer(skb) && pinfo->nr_frags && - !PageHighMem(skb_frag_page(frag0))) { + !PageHighMem(skb_frag_page(frag0)) && + (!NET_IP_ALIGN || !(skb_frag_off(frag0) & 3))) { NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0); NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int, skb_frag_size(frag0),