BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el):
LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32
Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above.
Verify the __retain macro with readelf by checking object flags for 'R':
$ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific)
Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot lkp@intel.com Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar Tony.Ambardar@gmail.com --- include/linux/btf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..7c3e40c3295e 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -82,7 +82,7 @@ * as to avoid issues such as the compiler inlining or eliding either a static * kfunc, or a global kfunc in an LTO build. */ -#define __bpf_kfunc __used noinline +#define __bpf_kfunc __used __retain noinline
#define __bpf_kfunc_start_defs() \ __diag_push(); \
On Mon, Jun 03, 2024 at 10:23:16PM -0700, Tony Ambardar wrote:
BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el):
LD vmlinux BTFIDS vmlinuxWARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32
Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above.
Verify the __retain macro with readelf by checking object flags for 'R':
$ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific)
Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot lkp@intel.com Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar Tony.Ambardar@gmail.com
tested on mips64 cross build and the warnings are gone and related functions are in the vmlinux
patchset looks good to me
Tested-by: Jiri Olsa jolsa@kernel.org Reviewed-by: Jiri Olsa jolsa@kernel.org
thanks, jirka
include/linux/btf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..7c3e40c3295e 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -82,7 +82,7 @@
- as to avoid issues such as the compiler inlining or eliding either a static
- kfunc, or a global kfunc in an LTO build.
*/ -#define __bpf_kfunc __used noinline +#define __bpf_kfunc __used __retain noinline #define __bpf_kfunc_start_defs() \ __diag_push(); \ -- 2.34.1
Hi Tony,
On Mon, 3 Jun 2024, Tony Ambardar wrote:
BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el):
LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32
Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above.
Verify the __retain macro with readelf by checking object flags for 'R':
$ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific)
Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot lkp@intel.com Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar Tony.Ambardar@gmail.com
Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in v6.10-rc5.
This is causing build failures on ARM with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y:
net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11859 | { | ^ net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11872 | { | ^ net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11885 | { | ^ net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11906 | { | ^ net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] 12092 | { | ^ net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] 713 | { | ^ net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] 736 | { | ^ net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] 769 | { | ^ [...]
My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04).
--- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -82,7 +82,7 @@
- as to avoid issues such as the compiler inlining or eliding either a static
- kfunc, or a global kfunc in an LTO build.
*/ -#define __bpf_kfunc __used noinline +#define __bpf_kfunc __used __retain noinline
#define __bpf_kfunc_start_defs() \ __diag_push(); \
Gr{oetje,eeting}s,
Geert
-- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
On Tue, Jun 25, 2024 at 12:46:48PM +0200, Geert Uytterhoeven wrote:
Hi Tony,
On Mon, 3 Jun 2024, Tony Ambardar wrote:
BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el):
LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32
Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above.
Verify the __retain macro with readelf by checking object flags for 'R':
$ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific)
Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot lkp@intel.com Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar Tony.Ambardar@gmail.com
Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in v6.10-rc5.
This is causing build failures on ARM with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y:
net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11859 | { | ^ net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11872 | { | ^ net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11885 | { | ^ net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11906 | { | ^ net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] 12092 | { | ^ net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] 713 | { | ^ net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] 736 | { | ^ net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] 769 | { | ^ [...]My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04).
hum, so it'd mean __has_attribute(__retain__) returns true while gcc still ignores the retain attribute.. like in this bug which seems similar: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587 but not sure how it got fixed.. any chance you can upgrade gcc and retest?
jirka
--- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -82,7 +82,7 @@
- as to avoid issues such as the compiler inlining or eliding either a static
- kfunc, or a global kfunc in an LTO build.
*/ -#define __bpf_kfunc __used noinline +#define __bpf_kfunc __used __retain noinline
#define __bpf_kfunc_start_defs() \ __diag_push(); \
Gr{oetje,eeting}s,
Geert-- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Hi Jiri,
On Wed, Jun 26, 2024 at 11:52 AM Jiri Olsa olsajiri@gmail.com wrote:
On Tue, Jun 25, 2024 at 12:46:48PM +0200, Geert Uytterhoeven wrote:
On Mon, 3 Jun 2024, Tony Ambardar wrote:
BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el):
LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32
Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above.
Verify the __retain macro with readelf by checking object flags for 'R':
$ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific)
Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot lkp@intel.com Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar Tony.Ambardar@gmail.com
Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in v6.10-rc5.
This is causing build failures on ARM with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y:
net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11859 | { | ^ net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11872 | { | ^ net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11885 | { | ^ net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11906 | { | ^ net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] 12092 | { | ^ net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] 713 | { | ^ net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] 736 | { | ^ net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] 769 | { | ^ [...]My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04).
hum, so it'd mean __has_attribute(__retain__) returns true while gcc still ignores the retain attribute.. like in this bug which seems similar: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587 but not sure how it got fixed.. any chance you can upgrade gcc and retest?
Indeed, __has_attribute(__retain__) returns true, while the attribute is not supported.
My test program:
cat > /tmp/a.c <<EOF #if __has_attribute(__retain__) #warning __retain__ OK #else #warning No __retain__ #endif
int x __attribute__((__retain__)); EOF
$ arm-linux-gnueabihf-gcc-11 -c /tmp/a.c # gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04))
/tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~ /tmp/a.c:7:1: warning: ‘retain’ attribute ignored [-Wattributes] 7 | int x __attribute__((__retain__)); | ^~~
Oops
$ arm-linux-gnueabihf-gcc-12 -c /tmp/a.c # gcc version 12.3.0 (Ubuntu 12.3.0-1ubuntu1~22.04) /tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~
Fixed
It works fine with the native gcc-11:
$ gcc-11 -c /tmp/a.c # gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04) /tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~
I gave it a try on all installed gcc-11 compilers.
/usr/bin/aarch64-linux-gnu-gcc-11 /usr/bin/alpha-linux-gnu-gcc-11 /usr/bin/arm-linux-gnueabi-gcc-11 /usr/bin/arm-linux-gnueabihf-gcc-11 /usr/bin/hppa64-linux-gnu-gcc-11 /usr/bin/hppa-linux-gnu-gcc-11 /usr/bin/m68k-linux-gnu-gcc-11 /usr/bin/powerpc64le-linux-gnu-gcc-11 /usr/bin/powerpc64-linux-gnu-gcc-11 /usr/bin/powerpc-linux-gnu-gcc-11 /usr/bin/riscv64-linux-gnu-gcc-11 /usr/bin/s390x-linux-gnu-gcc-11 /usr/bin/sh4-linux-gnu-gcc-11 /usr/bin/sparc64-linux-gnu-gcc-11 /usr/bin/x86_64-linux-gnu-gcc-11 /usr/bin/x86_64-linux-gnux32-gcc-11
All of them failed (incl. x32), except for the native x86_64-linux-gnu-gcc-11.
It works fine with all installed gcc-12 compilers (arm-linux-gnueabihf-gcc-12, m68k-linux-gnu-gcc-12, x86_64-linux-gnu-gcc-12).
With gcc-9, the absence of __retain__ is detected correctly.
Gr{oetje,eeting}s,
Geert
linux-stable-mirror@lists.linaro.org
-
Geert Uytterhoeven -
Jiri Olsa -
Tony Ambardar