[PATCH bpf-next v10 0/3] libbpf: fix USDT SIB argument handling causing unrecognized register error

List overview All Threads
Download

newer

older

[PATCHv2 net-next 0/3] bonding:...

[PATCH v5 0/7] Add support for...

Jiawei Zhao

14 Aug 2025 14 Aug '25

4:07 p.m.

When using GCC on x86-64 to compile an usdt prog with -O1 or higher optimization, the compiler will generate SIB addressing mode for global array and PC-relative addressing mode for global variable, e.g. "1@-96(%rbp,%rax,8)" and "-1@4+t1(%rip)".

The current USDT implementation in libbpf cannot parse these two formats, causing `bpf_program__attach_usdt()` to fail with -ENOENT (unrecognized register).

This patch series adds support for SIB addressing mode in USDT probes. The main changes include: - add correct handling logic for SIB-addressed arguments in `parse_usdt_arg`. - add an usdt_o2 test case to cover SIB addressing mode.

Testing shows that the SIB probe correctly generates 8@(%rcx,%rax,8) argument spec and passes all validation checks.

The modification history of this patch series: Change since v1: - refactor the code to make it more readable - modify the commit message to explain why and how

Change since v2: - fix the `scale` uninitialized error

Change since v3: - force -O2 optimization for usdt.test.o to generate SIB addressing usdt and pass all test cases.

Change since v4: - split the patch into two parts, one for the fix and the other for the test

Change since v5: - Only enable optimization for x86 architecture to generate SIB addressing usdt argument spec.

Change since v6: - Add an usdt_o2 test case to cover SIB addressing mode. - Reinstate the usdt.c test case.

Change since v7: - Refactor modifications to __bpf_usdt_arg_spec to avoid increasing its size, achieving better compatibility - Fix some minor code style issues - Refactor the usdt_o2 test case, removing semaphore and adding GCC attribute to force -O2 optimization

Change since v8: - Refactor the usdt_o2 test case, using assembly to force SIB addressing mode.

Change since v9: - Only enable the usdt_o2 test case on x86_64 and i386 architectures since the SIB addressing mode is only supported on x86_64 and i386.

Jiawei Zhao (3): libbpf: fix USDT SIB argument handling causing unrecognized register error selftests/bpf: Add an usdt_o2 test case in selftests to cover SIB handling logic selftests/bpf: make usdt_o2 reliably generate SIB USDT arg spec

tools/lib/bpf/usdt.bpf.h | 54 ++++++++++++- tools/lib/bpf/usdt.c | 61 ++++++++++++-- tools/testing/selftests/bpf/Makefile | 1 + .../selftests/bpf/prog_tests/usdt_o2.c | 79 +++++++++++++++++++ .../selftests/bpf/progs/test_usdt_o2.c | 37 +++++++++ 5 files changed, 225 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/usdt_o2.c create mode 100644 tools/testing/selftests/bpf/progs/test_usdt_o2.c

-- 2.43.0

Show replies by date

Jiawei Zhao

14 Aug 14 Aug

4:07 p.m.

New subject: [PATCH bpf-next v10 1/3] libbpf: fix USDT SIB argument handling causing unrecognized register error

On x86-64, USDT arguments can be specified using Scale-Index-Base (SIB) addressing, e.g. "1@-96(%rbp,%rax,8)". The current USDT implementation in libbpf cannot parse this format, causing `bpf_program__attach_usdt()` to fail with -ENOENT (unrecognized register).

This patch fixes this by implementing the necessary changes: - add correct handling for SIB-addressed arguments in `bpf_usdt_arg`. - add adaptive support to `__bpf_usdt_arg_type` and `__bpf_usdt_arg_spec` to represent SIB addressing parameters.

Signed-off-by: Jiawei Zhao phoenix500526@163.com --- tools/lib/bpf/usdt.bpf.h | 54 +++++++++++++++++++++++++++++++++-- tools/lib/bpf/usdt.c | 61 ++++++++++++++++++++++++++++++++++++---- 2 files changed, 108 insertions(+), 7 deletions(-)

diff --git a/tools/lib/bpf/usdt.bpf.h b/tools/lib/bpf/usdt.bpf.h index 2a7865c8e3fe..1e6de0afe65d 100644 --- a/tools/lib/bpf/usdt.bpf.h +++ b/tools/lib/bpf/usdt.bpf.h @@ -34,13 +34,31 @@ enum __bpf_usdt_arg_type { BPF_USDT_ARG_CONST, BPF_USDT_ARG_REG, BPF_USDT_ARG_REG_DEREF, + BPF_USDT_ARG_SIB, };

+/* + * To preserve overall layout and avoid growing this struct while adding SIB + * extras, we keep 4 bytes worth of space after val_off: + * + * - arg_type is stored as a single byte (values from enum below) + * - idx_packed is a 16-bit field packing idx_reg_off (high 12 bits) + * and scale shift (low 4 bits, i.e., scale = 1 << shift) + * - reserved is one spare byte for future use + * + * This keeps the offset of reg_off identical to the historical layout + * (val_off:8 + 4 bytes here), ensuring backwards/forwards compatibility for + * non-SIB modes that only rely on val_off/arg_type/reg_off/... offsets. + */ struct __bpf_usdt_arg_spec { /* u64 scalar interpreted depending on arg_type, see below */ __u64 val_off; /* arg location case, see bpf_usdt_arg() for details */ - enum __bpf_usdt_arg_type arg_type; + __u8 arg_type; + /* packed: [15:4] idx_reg_off, [3:0] scale_shift */ + __u16 idx_packed; + /* reserved for future use, keeps reg_off offset stable */ + __u8 reserved; /* offset of referenced register within struct pt_regs */ short reg_off; /* whether arg should be interpreted as signed value */ @@ -52,6 +70,10 @@ struct __bpf_usdt_arg_spec { char arg_bitshift; };

+/* Helpers to (un)pack SIB extras from idx_packed without relying on bitfields. */ +#define USDT_IDX_OFF(packed) ((packed) >> 4) +#define USDT_IDX_SCALE_SHIFT(packed) ((packed) & 0x000f) + /* should match USDT_MAX_ARG_CNT in usdt.c exactly */ #define BPF_USDT_MAX_ARG_CNT 12 struct __bpf_usdt_spec { @@ -149,8 +171,9 @@ int bpf_usdt_arg(struct pt_regs *ctx, __u64 arg_num, long *res) { struct __bpf_usdt_spec *spec; struct __bpf_usdt_arg_spec *arg_spec; - unsigned long val; + unsigned long val, idx; int err, spec_id; + int idx_off = 0, scale = 0;

*res = 0;

@@ -202,6 +225,33 @@ int bpf_usdt_arg(struct pt_regs *ctx, __u64 arg_num, long *res) return err; #if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ val >>= arg_spec->arg_bitshift; +#endif + break; + case BPF_USDT_ARG_SIB: + /* Arg is in memory addressed by SIB (Scale-Index-Base) mode + * (e.g., "-1@-96(%rbp,%rax,8)" in USDT arg spec). Register + * is identified like with BPF_USDT_ARG_SIB case, the offset + * is in arg_spec->val_off, the scale factor is in arg_spec->scale. + * Firstly, we fetch the base register contents and the index + * register contents from pt_regs. Secondly, we multiply the + * index register contents by the scale factor, then add the + * base address and the offset to get the final address. Finally, + * we do another user-space probe read to fetch argument value + * itself. + */ + idx_off = USDT_IDX_OFF(arg_spec->idx_packed); + scale = 1UL << USDT_IDX_SCALE_SHIFT(arg_spec->idx_packed); + err = bpf_probe_read_kernel(&val, sizeof(val), (void *)ctx + arg_spec->reg_off); + if (err) + return err; + err = bpf_probe_read_kernel(&idx, sizeof(idx), (void *)ctx + idx_off); + if (err) + return err; + err = bpf_probe_read_user(&val, sizeof(val), (void *)val + idx * scale + arg_spec->val_off); + if (err) + return err; +#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + val >>= arg_spec->arg_bitshift; #endif break; default: diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c index 3373b9d45ac4..0a6499f654fc 100644 --- a/tools/lib/bpf/usdt.c +++ b/tools/lib/bpf/usdt.c @@ -200,12 +200,15 @@ enum usdt_arg_type { USDT_ARG_CONST, USDT_ARG_REG, USDT_ARG_REG_DEREF, + USDT_ARG_SIB, };

/* should match exactly struct __bpf_usdt_arg_spec from usdt.bpf.h */ struct usdt_arg_spec { __u64 val_off; - enum usdt_arg_type arg_type; + __u8 arg_type; /* enum value stored as u8 */ + __u16 idx_packed; /* [15:4]=idx_reg_off, [3:0]=scale_shift */ + __u8 reserved; /* keep reg_off offset stable */ short reg_off; bool arg_signed; char arg_bitshift; @@ -214,6 +217,10 @@ struct usdt_arg_spec { /* should match BPF_USDT_MAX_ARG_CNT in usdt.bpf.h */ #define USDT_MAX_ARG_CNT 12

+/* Helpers to (un)pack SIB extras from idx_packed without relying on bitfields. */ +#define BPF_USDT_IDX_PACK(idx_off, scale_shift) \ + ((__u16)(((__u16)((idx_off) & 0x0fff)) << 4) | (__u16)((scale_shift) & 0x000f)) + /* should match struct __bpf_usdt_spec from usdt.bpf.h */ struct usdt_spec { struct usdt_arg_spec args[USDT_MAX_ARG_CNT]; @@ -1283,11 +1290,54 @@ static int calc_pt_regs_off(const char *reg_name)

static int parse_usdt_arg(const char *arg_str, int arg_num, struct usdt_arg_spec *arg, int *arg_sz) { - char reg_name[16]; - int len, reg_off; - long off; + char reg_name[16] = {0}, idx_reg_name[16] = {0}; + int len, reg_off, idx_reg_off, scale = 1; + long off = 0; + __u16 scale_shift; + + if (sscanf(arg_str, " %d @ %ld ( %%%15[^,] , %%%15[^,] , %d ) %n", + arg_sz, &off, reg_name, idx_reg_name, &scale, &len) == 5 || + sscanf(arg_str, " %d @ ( %%%15[^,] , %%%15[^,] , %d ) %n", + arg_sz, reg_name, idx_reg_name, &scale, &len) == 4 || + sscanf(arg_str, " %d @ %ld ( %%%15[^,] , %%%15[^)] ) %n", + arg_sz, &off, reg_name, idx_reg_name, &len) == 4 || + sscanf(arg_str, " %d @ ( %%%15[^,] , %%%15[^)] ) %n", + arg_sz, reg_name, idx_reg_name, &len) == 3 + ) { + /* + * Scale Index Base case: + * 1@-96(%rbp,%rax,8) + * 1@(%rbp,%rax,8) + * 1@-96(%rbp,%rax) + * 1@(%rbp,%rax) + */ + arg->arg_type = USDT_ARG_SIB; + arg->val_off = off;

- if (sscanf(arg_str, " %d @ %ld ( %%%15[^)] ) %n", arg_sz, &off, reg_name, &len) == 3) { + reg_off = calc_pt_regs_off(reg_name); + if (reg_off < 0) + return reg_off; + arg->reg_off = reg_off; + + idx_reg_off = calc_pt_regs_off(idx_reg_name); + if (idx_reg_off < 0) + return idx_reg_off; + /* pack idx_reg_off and scale shift (scale in {1,2,4,8}) */ + if (scale == 1) + scale_shift = 0; + else if (scale == 2) + scale_shift = 1; + else if (scale == 4) + scale_shift = 2; + else if (scale == 8) + scale_shift = 3; + else { + pr_warn("usdt: invalid SIB scale %d, expected 1,2,4,8; defaulting to 1\n", scale); + return -EINVAL; + } + arg->idx_packed = BPF_USDT_IDX_PACK(idx_reg_off, scale_shift); + } else if (sscanf(arg_str, " %d @ %ld ( %%%15[^)] ) %n", + arg_sz, &off, reg_name, &len) == 3) { /* Memory dereference case, e.g., -4@-20(%rbp) */ arg->arg_type = USDT_ARG_REG_DEREF; arg->val_off = off; @@ -1306,6 +1356,7 @@ static int parse_usdt_arg(const char *arg_str, int arg_num, struct usdt_arg_spec } else if (sscanf(arg_str, " %d @ %%%15s %n", arg_sz, reg_name, &len) == 2) { /* Register read case, e.g., -4@%eax */ arg->arg_type = USDT_ARG_REG; + /* register read has no memory offset */ arg->val_off = 0;

reg_off = calc_pt_regs_off(reg_name);

-- 2.43.0

Jiawei Zhao

4:07 p.m.

New subject: [PATCH bpf-next v10 2/3] selftests/bpf: Add an usdt_o2 test case in selftests to cover SIB handling logic

In this patch: - add usdt_o2 test case to cover SIB addressing usdt argument spec handling logic

Signed-off-by: Jiawei Zhao phoenix500526@163.com --- tools/testing/selftests/bpf/Makefile | 1 + .../selftests/bpf/prog_tests/usdt_o2.c | 69 +++++++++++++++++++ .../selftests/bpf/progs/test_usdt_o2.c | 37 ++++++++++ 3 files changed, 107 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/usdt_o2.c create mode 100644 tools/testing/selftests/bpf/progs/test_usdt_o2.c

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 4863106034df..24ff1a329625 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -760,6 +760,7 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built) TRUNNER_BPF_CFLAGS := $(eval $(call DEFINE_TEST_RUNNER,test_maps))

+ # Define test_verifier test runner. # It is much simpler than test_maps/test_progs and sufficiently different from # them (e.g., test.h is using completely pattern), that it's worth just diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c new file mode 100644 index 000000000000..f02dcf5188ab --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2025 Jiawei Zhao phoenix500526@163.com. */ +#include <test_progs.h> + +#include "../sdt.h" +#include "test_usdt_o2.skel.h" + +#if defined(__GNUC__) && !defined(__clang__) +__attribute__((optimize("O2"))) +#endif + +#define test_value 0xFEDCBA9876543210ULL +#define SEC(name) __attribute__((section(name), used)) + +int lets_test_this(int); +static volatile __u64 array[1] = {test_value}; + +static __always_inline void trigger_func(void) +{ + /* Base address + offset + (index * scale) */ + for (volatile int i = 0; i <= 0; i++) + STAP_PROBE1(test, usdt1, array[i]); +} + +static void basic_sib_usdt(void) +{ + LIBBPF_OPTS(bpf_usdt_opts, opts); + struct test_usdt_o2 *skel; + struct test_usdt_o2__bss *bss; + int err; + + skel = test_usdt_o2__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + bss = skel->bss; + bss->my_pid = getpid(); + + err = test_usdt_o2__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto cleanup; + + /* usdt1 won't be auto-attached */ + opts.usdt_cookie = 0xcafedeadbeeffeed; + skel->links.usdt1 = bpf_program__attach_usdt(skel->progs.usdt1, + 0 /*self*/, "/proc/self/exe", + "test", "usdt1", &opts); + if (!ASSERT_OK_PTR(skel->links.usdt1, "usdt1_link")) + goto cleanup; + + trigger_func(); + + ASSERT_EQ(bss->usdt1_called, 1, "usdt1_called"); + ASSERT_EQ(bss->usdt1_cookie, 0xcafedeadbeeffeed, "usdt1_cookie"); + ASSERT_EQ(bss->usdt1_arg_cnt, 1, "usdt1_arg_cnt"); + ASSERT_EQ(bss->usdt1_arg, test_value, "usdt1_arg"); + ASSERT_EQ(bss->usdt1_arg_ret, 0, "usdt1_arg_ret"); + ASSERT_EQ(bss->usdt1_arg_size, sizeof(array[0]), "usdt1_arg_size"); + +cleanup: + test_usdt_o2__destroy(skel); +} + + + +void test_usdt_o2(void) +{ + basic_sib_usdt(); +} diff --git a/tools/testing/selftests/bpf/progs/test_usdt_o2.c b/tools/testing/selftests/bpf/progs/test_usdt_o2.c new file mode 100644 index 000000000000..14602aa54578 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_usdt_o2.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/usdt.bpf.h> + +int my_pid; + +int usdt1_called; +u64 usdt1_cookie; +int usdt1_arg_cnt; +int usdt1_arg_ret; +u64 usdt1_arg; +int usdt1_arg_size; + +SEC("usdt") +int usdt1(struct pt_regs *ctx) +{ + long tmp; + + if (my_pid != (bpf_get_current_pid_tgid() >> 32)) + return 0; + + __sync_fetch_and_add(&usdt1_called, 1); + + usdt1_cookie = bpf_usdt_cookie(ctx); + usdt1_arg_cnt = bpf_usdt_arg_cnt(ctx); + + usdt1_arg_ret = bpf_usdt_arg(ctx, 0, &tmp); + usdt1_arg = (u64)tmp; + usdt1_arg_size = bpf_usdt_arg_size(ctx, 0); + + return 0; +} + +char _license[] SEC("license") = "GPL";

-- 2.43.0

Jiawei Zhao

4:07 p.m.

New subject: [PATCH bpf-next v10 3/3] selftests/bpf: make usdt_o2 reliably generate SIB USDT arg spec

usdt_o2 is intended to exercise the SIB (Scale-Index-Base) argument handling in libbpf's USDT path. With GCC 13 this reliably produced a SIB-form argument (e.g. 8@(%rdx,%rax,8)), but with newer GCC (e.g. 15) the compiler frequently optimizes the probe argument into a plain register (e.g. 8@%rax) or a stack slot, so the test stops covering the SIB code path and becomes flaky across toolchains.

Force a SIB memory operand in the probe by: * placing the base pointer into %rdx and the index into %rax using an empty inline asm with output constraints ("=d", "=a") and matching inputs * immediately passing base[idx] to STAP_PROBE1. * only enable on x86 platform.

This makes the compiler encode the operand as SIB (base + index8), which in .note.stapsdt shows up as 8@(%rdx,%rax,8) regardless of GCC version. A memory clobber and noinline prevent reordering/re-allocation around the probe site.

This change is x86_64-specific and does not alter program semantics; it only stabilizes the USDT argument shape so the test consistently validates SIB handling. Clang historically prefers stack temporaries for such operands, but the selftests build with GCC, and this keeps behavior stable across GCC versions without introducing a separate .S file.

Signed-off-by: Jiawei Zhao phoenix500526@163.com --- .../selftests/bpf/prog_tests/usdt_o2.c | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c index f02dcf5188ab..e46d5743ad24 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -15,11 +15,19 @@ __attribute__((optimize("O2"))) int lets_test_this(int); static volatile __u64 array[1] = {test_value};

-static __always_inline void trigger_func(void) +static noinline void trigger_func(void) { +#if defined(__x86_64__) || defined(__i386__) /* Base address + offset + (index * scale) */ - for (volatile int i = 0; i <= 0; i++) - STAP_PROBE1(test, usdt1, array[i]); + /* Force SIB addressing with inline assembly */ + const __u64 *base; + __u32 idx; + /* binding base to %rdx and idx to %rax */ + asm volatile("" : "=d"(base), "=a"(idx) : "0"(array), "1"((__u32)0) : "memory"); + STAP_PROBE1(test, usdt1, base[idx]); +#else + STAP_PROBE1(test, usdt1, array[0]); +#endif }

static void basic_sib_usdt(void) @@ -61,9 +69,11 @@ static void basic_sib_usdt(void) test_usdt_o2__destroy(skel); }

- - void test_usdt_o2(void) { +#if !defined(__x86_64__) && !defined(__i386__) + test__skip(); + return; +#endif basic_sib_usdt(); }

-- 2.43.0

Jiri Olsa

15 Aug 15 Aug

12:44 p.m.

New subject: [PATCH bpf-next v10 3/3] selftests/bpf: make usdt_o2 reliably generate SIB USDT arg spec

On Thu, Aug 14, 2025 at 04:07:39PM +0000, Jiawei Zhao wrote:

...

usdt_o2 is intended to exercise the SIB (Scale-Index-Base) argument handling in libbpf's USDT path. With GCC 13 this reliably produced a SIB-form argument (e.g. 8@(%rdx,%rax,8)), but with newer GCC (e.g. 15) the compiler frequently optimizes the probe argument into a plain register (e.g. 8@%rax) or a stack slot, so the test stops covering the SIB code path and becomes flaky across toolchains.

Force a SIB memory operand in the probe by:

placing the base pointer into %rdx and the index into %rax using an empty inline asm with output constraints ("=d", "=a") and matching inputs

immediately passing base[idx] to STAP_PROBE1.

only enable on x86 platform.

This makes the compiler encode the operand as SIB (base + index8), which in .note.stapsdt shows up as 8@(%rdx,%rax,8) regardless of GCC version. A memory clobber and noinline prevent reordering/re-allocation around the probe site.

This change is x86_64-specific and does not alter program semantics; it only stabilizes the USDT argument shape so the test consistently validates SIB handling. Clang historically prefers stack temporaries for such operands, but the selftests build with GCC, and this keeps behavior stable across GCC versions without introducing a separate .S file.

Signed-off-by: Jiawei Zhao phoenix500526@163.com

.../selftests/bpf/prog_tests/usdt_o2.c | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c index f02dcf5188ab..e46d5743ad24 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -15,11 +15,19 @@ __attribute__((optimize("O2"))) int lets_test_this(int); static volatile __u64 array[1] = {test_value}; -static __always_inline void trigger_func(void) +static noinline void trigger_func(void) { +#if defined(__x86_64__) || defined(__i386__) /* Base address + offset + (index * scale) */
for (volatile int i = 0; i <= 0; i++)
STAP_PROBE1(test, usdt1, array[i]);
/* Force SIB addressing with inline assembly */

const __u64 *base;

__u32 idx;

/* binding base to %rdx and idx to %rax */

asm volatile("" : "=d"(base), "=a"(idx) : "0"(array), "1"((__u32)0) : "memory");

STAP_PROBE1(test, usdt1, base[idx]);

hum, I still end up with

stapsdt 0x0000002a NT_STAPSDT (SystemTap probe descriptors) Provider: test Name: usdt1 Location: 0x00000000007674c9, Base: 0x00000000035bc698, Semaphore: 0x0000000000000000 Arguments: 8@%rax

disasm being:

static noinline void trigger_func(void) { 76749f: 55 push %rbp 7674a0: 48 89 e5 mov %rsp,%rbp /* Base address + offset + (index * scale) */ /* Force SIB addressing with inline assembly */ const __u64 *base; __u32 idx; /* binding base to %rdx and idx to %rax */ asm volatile("" : "=d"(base), "=a"(idx) : "0"(array), "1"((__u32)0) : "memory"); 7674a3: ba 20 49 9c 03 mov $0x39c4920,%edx 7674a8: b8 00 00 00 00 mov $0x0,%eax 7674ad: 48 89 55 f8 mov %rdx,-0x8(%rbp) 7674b1: 89 45 f4 mov %eax,-0xc(%rbp) STAP_PROBE1(test, usdt1, base[idx]); 7674b4: 8b 45 f4 mov -0xc(%rbp),%eax 7674b7: 48 8d 14 c5 00 00 00 lea 0x0(,%rax,8),%rdx 7674be: 00 7674bf: 48 8b 45 f8 mov -0x8(%rbp),%rax 7674c3: 48 01 d0 add %rdx,%rax 7674c6: 48 8b 00 mov (%rax),%rax 7674c9: 90 nop #else STAP_PROBE1(test, usdt1, array[0]); #endif } 7674ca: 90 nop 7674cb: 5d pop %rbp 7674cc: c3 ret

I wonder we could also try to bring in Andrii's usdt.h [1] and overload usdt arguments like outlined in the hack below (full code in [1])

we will probably need smarter and sustainable change, but you I guess you get the idea

jirka

[1] https://github.com/anakryiko/usdt [2] git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git usdt_hack --- diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c index e46d5743ad24..7bb098c37de5 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -4,6 +4,8 @@

#include "../sdt.h" #include "test_usdt_o2.skel.h" +#define USDT_ARGS ".asciz "(,%%rax,8)"\n" +#include "usdt.h"

#if defined(__GNUC__) && !defined(__clang__) __attribute__((optimize("O2"))) @@ -28,6 +30,7 @@ static noinline void trigger_func(void) #else STAP_PROBE1(test, usdt1, array[0]); #endif + USDT(krava, test1, 1, 2); }

static void basic_sib_usdt(void) diff --git a/tools/testing/selftests/bpf/usdt.h b/tools/testing/selftests/bpf/usdt.h index 549d1f774810..960ebd6aa88b 100644 --- a/tools/testing/selftests/bpf/usdt.h +++ b/tools/testing/selftests/bpf/usdt.h @@ -403,6 +403,10 @@ struct usdt_sema { volatile unsigned short active; }; __asm__ __volatile__ ("" :: "m" (sema)); #endif

+#ifndef USDT_ARGS +#define USDT_ARGS __usdt_asm_args(__VA_ARGS__) +#endif + /* main USDT definition (nop and .note.stapsdt metadata) */ #define __usdt_probe(group, name, sema_def, sema, ...) do { \ sema_def(sema) \ @@ -418,7 +422,7 @@ struct usdt_sema { volatile unsigned short active; }; __usdt_asm1( __usdt_asm_addr sema) \ __usdt_asm_strz(group) \ __usdt_asm_strz(name) \ - __usdt_asm_args(__VA_ARGS__) \ + USDT_ARGS \ __usdt_asm1( .ascii "\0") \ __usdt_asm1(994: .balign 4) \ __usdt_asm1( .popsection) \

赵佳炜

16 Aug 16 Aug

7:04 a.m.

New subject: [PATCH bpf-next v10 3/3] selftests/bpf: make usdt_o2 reliably generate SIB USDT arg spec

Hi, Jiri. Sorry for my oversight, the optimized compile condition didn't work properly.

Could you try to replace the `__attribute__((optimize("O2")))` with `#pragma GCC optimize("O2")` in usdt_o2.c and verify it one more time? I guess it'll help. In fact, the usdt1 argument spec generated by the `__attribute__((optimize("O2")))` on my machine was `8@%rax`, not `8@(%rdx,%rax,8)`.

For more detail:

...

#if defined(__GNUC__) && !defined(__clang__)

__attribute__((optimize("O2")))

+#if (defined(__GNUC__) && !defined(__clang__)) +#pragma GCC optimize("O2") +#else +#pragma message("non-gcc compiler: the correct probes might not be installed") +#endif

Thanks

At 2025-08-15 20:44:32, "Jiri Olsa" olsajiri@gmail.com wrote:

...

On Thu, Aug 14, 2025 at 04:07:39PM +0000, Jiawei Zhao wrote:

...
usdt_o2 is intended to exercise the SIB (Scale-Index-Base) argument handling in libbpf's USDT path. With GCC 13 this reliably produced a SIB-form argument (e.g. 8@(%rdx,%rax,8)), but with newer GCC (e.g. 15) the compiler frequently optimizes the probe argument into a plain register (e.g. 8@%rax) or a stack slot, so the test stops covering the SIB code path and becomes flaky across toolchains.

Force a SIB memory operand in the probe by:

placing the base pointer into %rdx and the index into %rax using an empty inline asm with output constraints ("=d", "=a") and matching inputs

immediately passing base[idx] to STAP_PROBE1.

only enable on x86 platform.

This makes the compiler encode the operand as SIB (base + index8), which in .note.stapsdt shows up as 8@(%rdx,%rax,8) regardless of GCC version. A memory clobber and noinline prevent reordering/re-allocation around the probe site.

This change is x86_64-specific and does not alter program semantics; it only stabilizes the USDT argument shape so the test consistently validates SIB handling. Clang historically prefers stack temporaries for such operands, but the selftests build with GCC, and this keeps behavior stable across GCC versions without introducing a separate .S file.

Signed-off-by: Jiawei Zhao phoenix500526@163.com

.../selftests/bpf/prog_tests/usdt_o2.c | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c index f02dcf5188ab..e46d5743ad24 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -15,11 +15,19 @@ __attribute__((optimize("O2"))) int lets_test_this(int); static volatile __u64 array[1] = {test_value}; -static __always_inline void trigger_func(void) +static noinline void trigger_func(void) { +#if defined(__x86_64__) || defined(__i386__) /* Base address + offset + (index * scale) */
for (volatile int i = 0; i <= 0; i++)
STAP_PROBE1(test, usdt1, array[i]);
/* Force SIB addressing with inline assembly */

const __u64 *base;

__u32 idx;

/* binding base to %rdx and idx to %rax */

asm volatile("" : "=d"(base), "=a"(idx) : "0"(array), "1"((__u32)0) : "memory");

STAP_PROBE1(test, usdt1, base[idx]);
hum, I still end up with
 stapsdt              0x0000002a       NT_STAPSDT (SystemTap probe descriptors)
   Provider: test
   Name: usdt1
   Location: 0x00000000007674c9, Base: 0x00000000035bc698, Semaphore: 0x0000000000000000
   Arguments: 8@%rax
disasm being:

static noinline void trigger_func(void) { 76749f: 55 push %rbp 7674a0: 48 89 e5 mov %rsp,%rbp /* Base address + offset + (index * scale) */ /* Force SIB addressing with inline assembly */ const __u64 *base; __u32 idx; /* binding base to %rdx and idx to %rax */ asm volatile("" : "=d"(base), "=a"(idx) : "0"(array), "1"((__u32)0) : "memory"); 7674a3: ba 20 49 9c 03 mov $0x39c4920,%edx 7674a8: b8 00 00 00 00 mov $0x0,%eax 7674ad: 48 89 55 f8 mov %rdx,-0x8(%rbp) 7674b1: 89 45 f4 mov %eax,-0xc(%rbp) STAP_PROBE1(test, usdt1, base[idx]); 7674b4: 8b 45 f4 mov -0xc(%rbp),%eax 7674b7: 48 8d 14 c5 00 00 00 lea 0x0(,%rax,8),%rdx 7674be: 00 7674bf: 48 8b 45 f8 mov -0x8(%rbp),%rax 7674c3: 48 01 d0 add %rdx,%rax 7674c6: 48 8b 00 mov (%rax),%rax 7674c9: 90 nop #else STAP_PROBE1(test, usdt1, array[0]); #endif } 7674ca: 90 nop 7674cb: 5d pop %rbp 7674cc: c3 ret

I wonder we could also try to bring in Andrii's usdt.h [1] and overload usdt arguments like outlined in the hack below (full code in [1])

we will probably need smarter and sustainable change, but you I guess you get the idea

jirka

[1] https://github.com/anakryiko/usdt [2] git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git usdt_hack

diff --git a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c index e46d5743ad24..7bb098c37de5 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt_o2.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt_o2.c @@ -4,6 +4,8 @@

#include "../sdt.h" #include "test_usdt_o2.skel.h" +#define USDT_ARGS ".asciz "(,%%rax,8)"\n" +#include "usdt.h"

#if defined(__GNUC__) && !defined(__clang__) __attribute__((optimize("O2"))) @@ -28,6 +30,7 @@ static noinline void trigger_func(void) #else STAP_PROBE1(test, usdt1, array[0]); #endif

USDT(krava, test1, 1, 2);

}

static void basic_sib_usdt(void) diff --git a/tools/testing/selftests/bpf/usdt.h b/tools/testing/selftests/bpf/usdt.h index 549d1f774810..960ebd6aa88b 100644 --- a/tools/testing/selftests/bpf/usdt.h +++ b/tools/testing/selftests/bpf/usdt.h @@ -403,6 +403,10 @@ struct usdt_sema { volatile unsigned short active; }; __asm__ __volatile__ ("" :: "m" (sema)); #endif

+#ifndef USDT_ARGS +#define USDT_ARGS __usdt_asm_args(__VA_ARGS__) +#endif

/* main USDT definition (nop and .note.stapsdt metadata) */ #define __usdt_probe(group, name, sema_def, sema, ...) do { \ sema_def(sema) \ @@ -418,7 +422,7 @@ struct usdt_sema { volatile unsigned short active; }; __usdt_asm1( __usdt_asm_addr sema) \ __usdt_asm_strz(group) \ __usdt_asm_strz(name) \

__usdt_asm_args(__VA_ARGS__) \

USDT_ARGS \ __usdt_asm1( .ascii "\0") \ __usdt_asm1(994: .balign 4) \ __usdt_asm1( .popsection) \

Jiri Olsa

18 Aug 18 Aug

6:56 a.m.

New subject: [PATCH bpf-next v10 3/3] selftests/bpf: make usdt_o2 reliably generate SIB USDT arg spec

On Sat, Aug 16, 2025 at 03:04:17PM +0800, 赵佳炜 wrote:

...

Hi, Jiri. Sorry for my oversight, the optimized compile condition didn't work properly.

Could you try to replace the `__attribute__((optimize("O2")))` with `#pragma GCC optimize("O2")` in usdt_o2.c and verify it one more time? I guess it'll help. In fact, the usdt1 argument spec generated by the `__attribute__((optimize("O2")))` on my machine was `8@%rax`, not `8@(%rdx,%rax,8)`.

For more detail:

...

#if defined(__GNUC__) && !defined(__clang__)

__attribute__((optimize("O2")))

+#if (defined(__GNUC__) && !defined(__clang__)) +#pragma GCC optimize("O2") +#else +#pragma message("non-gcc compiler: the correct probes might not be installed") +#endif

Thanks

yep, that helped

stapsdt 0x00000033 NT_STAPSDT (SystemTap probe descriptors) Provider: test Name: usdt1 Location: 0x00000000007674d9, Base: 0x00000000035bc698, Semaphore: 0x0000000000000000 Arguments: 8@(%rdx,%rax,8)

thanks, jirka

days inactive

days old

linux-kselftest-mirror@lists.linaro.org

6 comments

participants

tags (0)

participants (3)

Jiawei Zhao
Jiri Olsa
赵佳炜