Since commit b3cf07851b6c ("arm64: entry: Switch to generic IRQ entry"), _TIF_WORK_MASK is never used, so remove it.

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/include/asm/thread_info.h | 6 ------ 1 file changed, 6 deletions(-)

diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index f241b8601ebd..ff4998fa1844 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -106,12 +106,6 @@ void arch_setup_new_exec(void); #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_TSC_SIGSEGV (1 << TIF_TSC_SIGSEGV)

-#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY | \ - _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_MTE_ASYNC_FAULT | \ - _TIF_NOTIFY_SIGNAL | _TIF_SIGPENDING | \ - _TIF_PATCH_PENDING) - #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ _TIF_SYSCALL_EMU)

The generic entry abort the syscall_trace_enter() sequence if ptrace_report_syscall_entry() errors out, but arm64 not.

As the ptrace_report_syscall_entry() comment said, the calling arch code should abort the system call and must prevent normal entry so no system call is made if ptrace_report_syscall_entry() return nonzero.

In preparation for moving arm64 over to the generic entry code, return early if ptrace_report_syscall_entry() encounters an error.

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/kernel/ptrace.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 4532e9831856..3926ce6aa268 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2346,15 +2346,18 @@ static __always_inline unsigned long ptrace_save_reg(struct pt_regs *regs, return saved_reg; }

-static void report_syscall_entry(struct pt_regs *regs) +static int report_syscall_entry(struct pt_regs *regs) { unsigned long saved_reg; - int regno; + int regno, ret;

saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_ENTER, &regno); - if (ptrace_report_syscall_entry(regs)) + ret = ptrace_report_syscall_entry(regs); + if (ret) forget_syscall(regs); regs->regs[regno] = saved_reg; + + return ret; }

static void report_syscall_exit(struct pt_regs *regs) @@ -2381,10 +2384,11 @@ static void report_syscall_exit(struct pt_regs *regs) int syscall_trace_enter(struct pt_regs *regs) { unsigned long flags = read_thread_flags(); + int ret;

if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) { - report_syscall_entry(regs); - if (flags & _TIF_SYSCALL_EMU) + ret = report_syscall_entry(regs); + if (ret || (flags & _TIF_SYSCALL_EMU)) return NO_SYSCALL; }

commit a9f3a74a29af ("entry: Provide generic syscall exit function") introduce generic syscall exit function and call rseq_syscall() before audit_syscall_exit() and arch_syscall_exit_tracehook().

And commit b74406f37737 ("arm: Add syscall detection for restartable sequences") add rseq support for arm32, which also call rseq_syscall() before audit_syscall_exit() and tracehook_report_syscall().

However, commit 409d5db49867c ("arm64: rseq: Implement backend rseq calls and select HAVE_RSEQ") implement arm64 rseq and call rseq_syscall() after audit_syscall_exit() and tracehook_report_syscall().

So compared to the generic entry and arm32 code, arm64 terminates the process a bit later if the syscall is issued within a restartable sequence.

But as commit b74406f37737 ("arm: Add syscall detection for restartable sequences") said, syscalls are not allowed inside restartable sequences, so should call rseq_syscall() at the very beginning of system call exiting path for CONFIG_DEBUG_RSEQ=y kernel. This could help us to detect whether there is a syscall issued inside restartable sequences.

It makes sense to raise SIGSEGV via rseq_syscall() before auditing and ptrace syscall exit, because this guarantees that the process is already in an error state with SIGSEGV pending when those later steps run. Although it makes no practical difference to signal delivery (signals are processed at the very end in arm64_exit_to_user_mode()), the ordering is more logical: detect and flag the error first, then proceed with the remaining work.

To make it more reasonable and in preparation for moving arm64 over to the generic entry code, move rseq_syscall() ahead before audit_syscall_exit().

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/kernel/ptrace.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index b962cbcca11d..8b0a46aac51f 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2416,6 +2416,8 @@ int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags)

void syscall_trace_exit(struct pt_regs *regs, unsigned long flags) { + rseq_syscall(regs); + audit_syscall_exit(regs);

if (flags & _TIF_SYSCALL_TRACEPOINT) @@ -2423,8 +2425,6 @@ void syscall_trace_exit(struct pt_regs *regs, unsigned long flags)

if (flags & (_TIF_SYSCALL_TRACE | _TIF_SINGLESTEP)) report_syscall_exit(regs); - - rseq_syscall(regs); }

/*

As syscall_exit_work() do not handle seccomp, so not check _TIF_SECCOMP for syscall_exit_work().

And as the man manual of PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP said, "For PTRACE_SYSEMU, continue and stop on entry to the next system call, which will not be executed. For PTRACE_SYSEMU_SINGLESTEP, do the same but also singlestep if not a system call.". So only the syscall entry need to be reported for SYSCALL_EMU, so not check _TIF_SYSCALL_EMU for syscall_exit_work().

After this, audit_syscall_exit() and report_syscall_exit() will no longer be called if only SECCOMP and/or SYSCALL_EMU is set.

And remove has_syscall_work() by the way as currently it is only used in el0_svc_common().

This is another preparation for moving arm64 over to the generic entry code.

Link：https://man7.org/linux/man-pages/man2/ptrace.2.html Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/include/asm/syscall.h | 5 ----- arch/arm64/include/asm/thread_info.h | 3 +++ arch/arm64/kernel/ptrace.c | 2 +- arch/arm64/kernel/syscall.c | 2 +- 4 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/syscall.h b/arch/arm64/include/asm/syscall.h index ccdaa805aa52..9f2aa9c57560 100644 --- a/arch/arm64/include/asm/syscall.h +++ b/arch/arm64/include/asm/syscall.h @@ -120,11 +120,6 @@ static inline int syscall_get_arch(struct task_struct *task) return AUDIT_ARCH_AARCH64; }

-static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags); void syscall_exit_to_user_mode_prepare(struct pt_regs *regs);

diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index ff4998fa1844..8d308ef64612 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -110,6 +110,9 @@ void arch_setup_new_exec(void); _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ _TIF_SYSCALL_EMU)

+#define _TIF_SYSCALL_EXIT_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ + _TIF_SYSCALL_TRACEPOINT) + #ifdef CONFIG_SHADOW_CALL_STACK #define INIT_SCS \ .scs_base = init_shadow_call_stack, \ diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 25fdf320d73a..3e233968efa2 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2431,7 +2431,7 @@ void syscall_exit_to_user_mode_prepare(struct pt_regs *regs)

rseq_syscall(regs);

- if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP) + if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP) syscall_exit_work(regs, flags); }

diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 65021d0f49e1..881fd261e8cb 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -101,7 +101,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, return; }

- if (has_syscall_work(flags)) { + if (unlikely(flags & _TIF_SYSCALL_WORK)) { /* * The de-facto standard way to skip a system call using ptrace * is to set the system call to -1 (NO_SYSCALL) and set x0 to a

The generic entry expand secure_computing() in place and call __secure_computing() directly.

In order to switch to the generic entry for arm64, refactor secure_computing() for syscall_trace_enter().

No functional changes.

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/kernel/ptrace.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 25111c121b5e..c6dee8a355e5 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2392,8 +2392,11 @@ int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags) }

/* Do the secure computing after ptrace; failures should be fast. */ - if (secure_computing() == -1) - return NO_SYSCALL; + if (flags & _TIF_SECCOMP) { + ret = __secure_computing(); + if (ret == -1) + return NO_SYSCALL; + }

/* Either of the above might have changed the syscall number */ syscall = syscall_get_nr(current, regs); @@ -2411,7 +2414,7 @@ int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags) audit_syscall_entry(syscall, regs->orig_x0, regs->regs[1], regs->regs[2], regs->regs[3]);

- return syscall; + return ret ? : syscall; }

static inline bool report_single_step(unsigned long flags)

In the generic entry code, the beginning of syscall_exit_to_user_mode_work() can be reused on arm64 so it makes sense to split it.

In preparation for moving arm64 over to the generic entry code, split out syscall_exit_to_user_mode_prepare() helper from syscall_exit_to_user_mode_work().

No functional changes.

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Reviewed-by: Thomas Gleixner tglx@linutronix.de Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- include/linux/entry-common.h | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)

diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h index 7177436f0f9e..cd6dacb2d8bf 100644 --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -137,20 +137,11 @@ static __always_inline long syscall_enter_from_user_mode(struct pt_regs *regs, l */ void syscall_exit_work(struct pt_regs *regs, unsigned long work);

-/** - * syscall_exit_to_user_mode_work - Handle work before returning to user mode - * @regs: Pointer to currents pt_regs - * - * Same as step 1 and 2 of syscall_exit_to_user_mode() but without calling - * exit_to_user_mode() to perform the final transition to user mode. - * - * Calling convention is the same as for syscall_exit_to_user_mode() and it - * returns with all work handled and interrupts disabled. The caller must - * invoke exit_to_user_mode() before actually switching to user mode to - * make the final state transitions. Interrupts must stay disabled between - * return from this function and the invocation of exit_to_user_mode(). +/* + * Syscall specific exit to user mode preparation. Runs with interrupts + * enabled. */ -static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs) +static __always_inline void syscall_exit_to_user_mode_prepare(struct pt_regs *regs) { unsigned long work = READ_ONCE(current_thread_info()->syscall_work); unsigned long nr = syscall_get_nr(current, regs); @@ -171,6 +162,24 @@ static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs) */ if (unlikely(work & SYSCALL_WORK_EXIT)) syscall_exit_work(regs, work); +} + +/** + * syscall_exit_to_user_mode_work - Handle work before returning to user mode + * @regs: Pointer to currents pt_regs + * + * Same as step 1 and 2 of syscall_exit_to_user_mode() but without calling + * exit_to_user_mode() to perform the final transition to user mode. + * + * Calling convention is the same as for syscall_exit_to_user_mode() and it + * returns with all work handled and interrupts disabled. The caller must + * invoke exit_to_user_mode() before actually switching to user mode to + * make the final state transitions. Interrupts must stay disabled between + * return from this function and the invocation of exit_to_user_mode(). + */ +static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs) +{ + syscall_exit_to_user_mode_prepare(regs); local_irq_disable_exit_to_user(); exit_to_user_mode_prepare(regs); }

Currently, x86, Riscv, Loongarch use the generic entry which makes maintainers' work easier and codes more elegant. arm64 has already switched to the generic IRQ entry, so completely convert arm64 to use the generic entry infrastructure from kernel/entry/*.

The changes are below: - Remove TIF_SYSCALL_* flag.

- Remove _TIF_SYSCALL_WORK and has_syscall_work(), as _TIF_SYSCALL_WORK is equal with SYSCALL_WORK_ENTER.

- Implement arch_ptrace_report_syscall_entry/exit() with report_syscall_entry/exit() to do arm64-specific save/restore during syscall entry/exit.

- Add "ARCH_SYSCALL_WORK_EXIT" to be defined as "_TIF_SECCOMP | _TIF_SYSCALL_EMU" to keep the arm64 behaviour unchanged.

- Remove arm64 syscall_trace_enter(), syscall_exit_to_user_mode_prepare(), and related sub-functions including syscall_exit_work() and syscall_enter_audit(), by calling generic entry's functions with similar functionality.

- Implement arch_syscall_is_vdso_sigreturn() to support "Syscall User Dispatch".

Suggested-by: Kevin Brodsky kevin.brodsky@arm.com Suggested-by: Mark Rutland mark.rutland@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/entry-common.h | 76 +++++++++++++ arch/arm64/include/asm/syscall.h | 19 +++- arch/arm64/include/asm/thread_info.h | 19 +--- arch/arm64/kernel/debug-monitors.c | 7 ++ arch/arm64/kernel/ptrace.c | 153 -------------------------- arch/arm64/kernel/signal.c | 2 +- arch/arm64/kernel/syscall.c | 6 +- 8 files changed, 107 insertions(+), 177 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6663ffd23f25..1463ff15d67a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -152,9 +152,9 @@ config ARM64 select GENERIC_CPU_DEVICES select GENERIC_CPU_VULNERABILITIES select GENERIC_EARLY_IOREMAP + select GENERIC_ENTRY select GENERIC_IDLE_POLL_SETUP select GENERIC_IOREMAP - select GENERIC_IRQ_ENTRY select GENERIC_IRQ_IPI select GENERIC_IRQ_KEXEC_CLEAR_VM_FORWARD select GENERIC_IRQ_PROBE diff --git a/arch/arm64/include/asm/entry-common.h b/arch/arm64/include/asm/entry-common.h index cab8cd78f693..d8bf4bf342e8 100644 --- a/arch/arm64/include/asm/entry-common.h +++ b/arch/arm64/include/asm/entry-common.h @@ -3,14 +3,21 @@ #ifndef _ASM_ARM64_ENTRY_COMMON_H #define _ASM_ARM64_ENTRY_COMMON_H

+#include <linux/ptrace.h> #include <linux/thread_info.h>

+#include <asm/compat.h> #include <asm/cpufeature.h> #include <asm/daifflags.h> #include <asm/fpsimd.h> #include <asm/mte.h> #include <asm/stacktrace.h>

+enum ptrace_syscall_dir { + PTRACE_SYSCALL_ENTER = 0, + PTRACE_SYSCALL_EXIT, +}; + #define ARCH_EXIT_TO_USER_MODE_WORK (_TIF_MTE_ASYNC_FAULT | _TIF_FOREIGN_FPSTATE)

static __always_inline void arch_exit_to_user_mode_work(struct pt_regs *regs, @@ -54,4 +61,73 @@ static inline bool arch_irqentry_exit_need_resched(void)

#define arch_irqentry_exit_need_resched arch_irqentry_exit_need_resched

+static __always_inline unsigned long ptrace_save_reg(struct pt_regs *regs, + enum ptrace_syscall_dir dir, + int *regno) +{ + unsigned long saved_reg; + + /* + * We have some ABI weirdness here in the way that we handle syscall + * exit stops because we indicate whether or not the stop has been + * signalled from syscall entry or syscall exit by clobbering a general + * purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee + * and restoring its old value after the stop. This means that: + * + * - Any writes by the tracer to this register during the stop are + * ignored/discarded. + * + * - The actual value of the register is not available during the stop, + * so the tracer cannot save it and restore it later. + * + * - Syscall stops behave differently to seccomp and pseudo-step traps + * (the latter do not nobble any registers). + */ + *regno = (is_compat_task() ? 12 : 7); + saved_reg = regs->regs[*regno]; + regs->regs[*regno] = dir; + + return saved_reg; +} + +static __always_inline int arch_ptrace_report_syscall_entry(struct pt_regs *regs) +{ + unsigned long saved_reg; + int regno, ret; + + saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_ENTER, &regno); + ret = ptrace_report_syscall_entry(regs); + if (ret) + forget_syscall(regs); + regs->regs[regno] = saved_reg; + + return ret; +} + +#define arch_ptrace_report_syscall_entry arch_ptrace_report_syscall_entry + +static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs, + int step) +{ + unsigned long saved_reg; + int regno; + + saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_EXIT, &regno); + if (!step) { + ptrace_report_syscall_exit(regs, 0); + regs->regs[regno] = saved_reg; + } else { + regs->regs[regno] = saved_reg; + + /* + * Signal a pseudo-step exception since we are stepping but + * tracer modifications to the registers may have rewound the + * state machine. + */ + ptrace_report_syscall_exit(regs, 1); + } +} + +#define arch_ptrace_report_syscall_exit arch_ptrace_report_syscall_exit + #endif /* _ASM_ARM64_ENTRY_COMMON_H */ diff --git a/arch/arm64/include/asm/syscall.h b/arch/arm64/include/asm/syscall.h index 9f2aa9c57560..240d45735cc5 100644 --- a/arch/arm64/include/asm/syscall.h +++ b/arch/arm64/include/asm/syscall.h @@ -9,6 +9,9 @@ #include <linux/compat.h> #include <linux/err.h>

+#include <asm/compat.h> +#include <asm/vdso.h> + typedef long (*syscall_fn_t)(const struct pt_regs *regs);

extern const syscall_fn_t sys_call_table[]; @@ -120,7 +123,19 @@ static inline int syscall_get_arch(struct task_struct *task) return AUDIT_ARCH_AARCH64; }

-int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags); -void syscall_exit_to_user_mode_prepare(struct pt_regs *regs); +static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) +{ + unsigned long sigtramp; + +#ifdef CONFIG_COMPAT + if (is_compat_task()) { + unsigned long sigpage = (unsigned long)current->mm->context.sigpage; + + return regs->pc >= sigpage && regs->pc < (sigpage + PAGE_SIZE); + } +#endif + sigtramp = (unsigned long)VDSO_SYMBOL(current->mm->context.vdso, sigtramp); + return regs->pc == (sigtramp + 8); +}

#endif /* __ASM_SYSCALL_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 8d308ef64612..0c083be23018 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -43,6 +43,7 @@ struct thread_info { void *scs_sp; #endif u32 cpu; + unsigned long syscall_work; /* SYSCALL_WORK_ flags */ };

#define thread_saved_pc(tsk) \ @@ -65,11 +66,6 @@ void arch_setup_new_exec(void); #define TIF_UPROBE 5 /* uprobe breakpoint or singlestep */ #define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */ #define TIF_NOTIFY_SIGNAL 7 /* signal notifications exist */ -#define TIF_SYSCALL_TRACE 8 /* syscall trace active */ -#define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ -#define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ -#define TIF_SECCOMP 11 /* syscall secure computing */ -#define TIF_SYSCALL_EMU 12 /* syscall emulation active */ #define TIF_PATCH_PENDING 13 /* pending live patching update */ #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_FREEZE 19 @@ -92,27 +88,14 @@ void arch_setup_new_exec(void); #define _TIF_NEED_RESCHED_LAZY (1 << TIF_NEED_RESCHED_LAZY) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE) -#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) -#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) -#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) -#define _TIF_SECCOMP (1 << TIF_SECCOMP) -#define _TIF_SYSCALL_EMU (1 << TIF_SYSCALL_EMU) #define _TIF_PATCH_PENDING (1 << TIF_PATCH_PENDING) #define _TIF_UPROBE (1 << TIF_UPROBE) -#define _TIF_SINGLESTEP (1 << TIF_SINGLESTEP) #define _TIF_32BIT (1 << TIF_32BIT) #define _TIF_SVE (1 << TIF_SVE) #define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_TSC_SIGSEGV (1 << TIF_TSC_SIGSEGV)

-#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ - _TIF_SYSCALL_EMU) - -#define _TIF_SYSCALL_EXIT_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT) - #ifdef CONFIG_SHADOW_CALL_STACK #define INIT_SCS \ .scs_base = init_shadow_call_stack, \ diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 29307642f4c9..e67643a70405 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -385,11 +385,18 @@ void user_enable_single_step(struct task_struct *task)

if (!test_and_set_ti_thread_flag(ti, TIF_SINGLESTEP)) set_regs_spsr_ss(task_pt_regs(task)); + + /* + * Ensure that a trap is triggered once stepping out of a system + * call prior to executing any user instruction. + */ + set_task_syscall_work(task, SYSCALL_EXIT_TRAP); } NOKPROBE_SYMBOL(user_enable_single_step);

void user_disable_single_step(struct task_struct *task) { clear_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP); + clear_task_syscall_work(task, SYSCALL_EXIT_TRAP); } NOKPROBE_SYMBOL(user_disable_single_step); diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 36e7ba3f38a5..cfbb6e409fcf 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -8,7 +8,6 @@ * Copyright (C) 2012 ARM Ltd. */

-#include <linux/audit.h> #include <linux/compat.h> #include <linux/kernel.h> #include <linux/sched/signal.h> @@ -18,7 +17,6 @@ #include <linux/smp.h> #include <linux/ptrace.h> #include <linux/user.h> -#include <linux/seccomp.h> #include <linux/security.h> #include <linux/init.h> #include <linux/signal.h> @@ -28,7 +26,6 @@ #include <linux/hw_breakpoint.h> #include <linux/regset.h> #include <linux/elf.h> -#include <linux/rseq.h>

#include <asm/compat.h> #include <asm/cpufeature.h> @@ -38,13 +35,9 @@ #include <asm/mte.h> #include <asm/pointer_auth.h> #include <asm/stacktrace.h> -#include <asm/syscall.h> #include <asm/traps.h> #include <asm/system_misc.h>

-#define CREATE_TRACE_POINTS -#include <trace/events/syscalls.h> - struct pt_regs_offset { const char *name; int offset; @@ -2312,152 +2305,6 @@ long arch_ptrace(struct task_struct *child, long request, return ptrace_request(child, request, addr, data); }

-enum ptrace_syscall_dir { - PTRACE_SYSCALL_ENTER = 0, - PTRACE_SYSCALL_EXIT, -}; - -static __always_inline unsigned long ptrace_save_reg(struct pt_regs *regs, - enum ptrace_syscall_dir dir, - int *regno) -{ - unsigned long saved_reg; - - /* - * We have some ABI weirdness here in the way that we handle syscall - * exit stops because we indicate whether or not the stop has been - * signalled from syscall entry or syscall exit by clobbering a general - * purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee - * and restoring its old value after the stop. This means that: - * - * - Any writes by the tracer to this register during the stop are - * ignored/discarded. - * - * - The actual value of the register is not available during the stop, - * so the tracer cannot save it and restore it later. - * - * - Syscall stops behave differently to seccomp and pseudo-step traps - * (the latter do not nobble any registers). - */ - *regno = (is_compat_task() ? 12 : 7); - saved_reg = regs->regs[*regno]; - regs->regs[*regno] = dir; - - return saved_reg; -} - -static int report_syscall_entry(struct pt_regs *regs) -{ - unsigned long saved_reg; - int regno, ret; - - saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_ENTER, &regno); - ret = ptrace_report_syscall_entry(regs); - if (ret) - forget_syscall(regs); - regs->regs[regno] = saved_reg; - - return ret; -} - -static void report_syscall_exit(struct pt_regs *regs) -{ - unsigned long saved_reg; - int regno; - - saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_EXIT, &regno); - if (!test_thread_flag(TIF_SINGLESTEP)) { - ptrace_report_syscall_exit(regs, 0); - regs->regs[regno] = saved_reg; - } else { - regs->regs[regno] = saved_reg; - - /* - * Signal a pseudo-step exception since we are stepping but - * tracer modifications to the registers may have rewound the - * state machine. - */ - ptrace_report_syscall_exit(regs, 1); - } -} - -static inline void syscall_enter_audit(struct pt_regs *regs, long syscall) -{ - if (unlikely(audit_context())) { - unsigned long args[6]; - - syscall_get_arguments(current, regs, args); - audit_syscall_entry(syscall, args[0], args[1], args[2], args[3]); - } -} - -int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags) -{ - int ret; - - if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) { - ret = report_syscall_entry(regs); - if (ret || (flags & _TIF_SYSCALL_EMU)) - return NO_SYSCALL; - } - - /* Do the secure computing after ptrace; failures should be fast. */ - if (flags & _TIF_SECCOMP) { - ret = __secure_computing(); - if (ret == -1) - return NO_SYSCALL; - } - - /* Either of the above might have changed the syscall number */ - syscall = syscall_get_nr(current, regs); - - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) { - trace_sys_enter(regs, syscall); - - /* - * Probes or BPF hooks in the tracepoint may have changed the - * system call number as well. - */ - syscall = syscall_get_nr(current, regs); - } - - syscall_enter_audit(regs, syscall); - - return ret ? : syscall; -} - -static inline bool report_single_step(unsigned long flags) -{ - if (flags & _TIF_SYSCALL_EMU) - return false; - - return flags & _TIF_SINGLESTEP; -} - -static void syscall_exit_work(struct pt_regs *regs, unsigned long flags) -{ - bool step; - - audit_syscall_exit(regs); - - if (flags & _TIF_SYSCALL_TRACEPOINT) - trace_sys_exit(regs, syscall_get_return_value(current, regs)); - - step = report_single_step(flags); - if (step || flags & _TIF_SYSCALL_TRACE) - report_syscall_exit(regs); -} - -void syscall_exit_to_user_mode_prepare(struct pt_regs *regs) -{ - unsigned long flags = read_thread_flags(); - - rseq_syscall(regs); - - if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP) - syscall_exit_work(regs, flags); -} - /* * SPSR_ELx bits which are always architecturally RES0 per ARM DDI 0487D.a. * We permit userspace to set SSBS (AArch64 bit 12, AArch32 bit 23) which is diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 1110eeb21f57..d3ec1892b3c7 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -8,8 +8,8 @@

#include <linux/cache.h> #include <linux/compat.h> +#include <linux/entry-common.h> #include <linux/errno.h> -#include <linux/irq-entry-common.h> #include <linux/kernel.h> #include <linux/signal.h> #include <linux/freezer.h> diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 881fd261e8cb..47e193a1cfff 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -2,6 +2,7 @@

#include <linux/compiler.h> #include <linux/context_tracking.h> +#include <linux/entry-common.h> #include <linux/errno.h> #include <linux/nospec.h> #include <linux/ptrace.h> @@ -68,6 +69,7 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno, static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, const syscall_fn_t syscall_table[]) { + unsigned long work = READ_ONCE(current_thread_info()->syscall_work); unsigned long flags = read_thread_flags();

regs->orig_x0 = regs->regs[0]; @@ -101,7 +103,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, return; }

- if (unlikely(flags & _TIF_SYSCALL_WORK)) { + if (unlikely(work & SYSCALL_WORK_ENTER)) { /* * The de-facto standard way to skip a system call using ptrace * is to set the system call to -1 (NO_SYSCALL) and set x0 to a @@ -119,7 +121,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, */ if (scno == NO_SYSCALL) syscall_set_return_value(current, regs, -ENOSYS, 0); - scno = syscall_trace_enter(regs, scno, flags); + scno = syscall_trace_enter(regs, scno, work); if (scno == NO_SYSCALL) goto trace_exit; }

After switch arm64 to Generic Entry, a new hotspot syscall_exit_work() appeared because syscall_exit_work() is no longer inlined. so inline syscall_exit_work(), and it has 2.6% performance uplift on perf bench basic syscall on kunpeng920 as below.

| Metric | W/O this patch | With this patch | Change | | ---------- | -------------- | --------------- | --------- | | Total time | 2.171 [sec] | 2.114 [sec] | ↓2.6% | | usecs/op | 0.217192 | 0.211453 | ↓2.6% | | ops/sec | 4,604,225 | 4,729,178 | ↑2.7% |

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com --- include/linux/entry-common.h | 63 ++++++++++++++++++++++++++++++++++- kernel/entry/syscall-common.c | 59 ++------------------------------ 2 files changed, 64 insertions(+), 58 deletions(-)

diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h index cd6dacb2d8bf..2f84377fb016 100644 --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -2,6 +2,7 @@ #ifndef __LINUX_ENTRYCOMMON_H #define __LINUX_ENTRYCOMMON_H

+#include <linux/audit.h> #include <linux/irq-entry-common.h> #include <linux/ptrace.h> #include <linux/seccomp.h> @@ -128,6 +129,41 @@ static __always_inline long syscall_enter_from_user_mode(struct pt_regs *regs, l return ret; }

+/* + * If SYSCALL_EMU is set, then the only reason to report is when + * SINGLESTEP is set (i.e. PTRACE_SYSEMU_SINGLESTEP). This syscall + * instruction has been already reported in syscall_enter_from_user_mode(). + */ +static __always_inline bool report_single_step(unsigned long work) +{ + if (work & SYSCALL_WORK_SYSCALL_EMU) + return false; + + return work & SYSCALL_WORK_SYSCALL_EXIT_TRAP; +} + +/** + * arch_ptrace_report_syscall_exit - Architecture specific + * ptrace_report_syscall_exit. + * + * Invoked from syscall_exit_work() to wrap ptrace_report_syscall_exit(). + * + * The main purpose is to support arch-specific ptrace_report_syscall_exit + * implementation. + */ +static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs, + int step); + +#ifndef arch_ptrace_report_syscall_exit +static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs, + int step) +{ + ptrace_report_syscall_exit(regs, step); +} +#endif + +void __trace_sys_exit(struct pt_regs *regs, long ret); + /** * syscall_exit_work - Handle work before returning to user mode * @regs: Pointer to current pt_regs @@ -135,7 +171,32 @@ static __always_inline long syscall_enter_from_user_mode(struct pt_regs *regs, l * * Do one-time syscall specific work. */ -void syscall_exit_work(struct pt_regs *regs, unsigned long work); +static __always_inline void syscall_exit_work(struct pt_regs *regs, unsigned long work) +{ + bool step; + + /* + * If the syscall was rolled back due to syscall user dispatching, + * then the tracers below are not invoked for the same reason as + * the entry side was not invoked in syscall_trace_enter(): The ABI + * of these syscalls is unknown. + */ + if (work & SYSCALL_WORK_SYSCALL_USER_DISPATCH) { + if (unlikely(current->syscall_dispatch.on_dispatch)) { + current->syscall_dispatch.on_dispatch = false; + return; + } + } + + audit_syscall_exit(regs); + + if (work & SYSCALL_WORK_SYSCALL_TRACEPOINT) + __trace_sys_exit(regs, syscall_get_return_value(current, regs)); + + step = report_single_step(work); + if (step || work & SYSCALL_WORK_SYSCALL_TRACE) + arch_ptrace_report_syscall_exit(regs, step); +}

/* * Syscall specific exit to user mode preparation. Runs with interrupts diff --git a/kernel/entry/syscall-common.c b/kernel/entry/syscall-common.c index 27310e611567..1636f49c58d2 100644 --- a/kernel/entry/syscall-common.c +++ b/kernel/entry/syscall-common.c @@ -90,62 +90,7 @@ noinstr void syscall_enter_from_user_mode_prepare(struct pt_regs *regs) instrumentation_end(); }

-/* - * If SYSCALL_EMU is set, then the only reason to report is when - * SINGLESTEP is set (i.e. PTRACE_SYSEMU_SINGLESTEP). This syscall - * instruction has been already reported in syscall_enter_from_user_mode(). - */ -static inline bool report_single_step(unsigned long work) -{ - if (work & SYSCALL_WORK_SYSCALL_EMU) - return false; - - return work & SYSCALL_WORK_SYSCALL_EXIT_TRAP; -} - -/** - * arch_ptrace_report_syscall_exit - Architecture specific - * ptrace_report_syscall_exit. - * - * Invoked from syscall_exit_work() to wrap ptrace_report_syscall_exit(). - * - * The main purpose is to support arch-specific ptrace_report_syscall_exit - * implementation. - */ -static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs, - int step); - -#ifndef arch_ptrace_report_syscall_exit -static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs, - int step) +void __trace_sys_exit(struct pt_regs *regs, long ret) { - ptrace_report_syscall_exit(regs, step); -} -#endif - -void syscall_exit_work(struct pt_regs *regs, unsigned long work) -{ - bool step; - - /* - * If the syscall was rolled back due to syscall user dispatching, - * then the tracers below are not invoked for the same reason as - * the entry side was not invoked in syscall_trace_enter(): The ABI - * of these syscalls is unknown. - */ - if (work & SYSCALL_WORK_SYSCALL_USER_DISPATCH) { - if (unlikely(current->syscall_dispatch.on_dispatch)) { - current->syscall_dispatch.on_dispatch = false; - return; - } - } - - audit_syscall_exit(regs); - - if (work & SYSCALL_WORK_SYSCALL_TRACEPOINT) - trace_sys_exit(regs, syscall_get_return_value(current, regs)); - - step = report_single_step(work); - if (step || work & SYSCALL_WORK_SYSCALL_TRACE) - arch_ptrace_report_syscall_exit(regs, step); + trace_sys_exit(regs, ret); }

On 2025/12/9 21:48, Kevin Brodsky wrote:

On 04/12/2025 09:21, Jinjie Ruan wrote:

...

After switch arm64 to Generic Entry, a new hotspot syscall_exit_work() appeared because syscall_exit_work() is no longer inlined. so inline

Before this series the call to syscall_trace_exit() in el0_svc_common() could not be inlined, so "no longer inlined" doesn't seem to be accurate.

I think the original "syscall_trace_exit()" is on an equal footing with new introduced syscall_exit_to_user_mode_prepare() which is now inlined.

...

syscall_exit_work(), and it has 2.6% performance uplift on perf bench basic syscall on kunpeng920 as below.

That seems strange. syscall_exit_work() is only called if some flag in SYSCALL_WORK_EXIT is set, which means that we're doing something special like tracing. That shouldn't be the case when running a simple perf bench syscall.

Also worth nothing that its counterpart (syscall_trace_enter())) is not currently inlined, the asymmetry would have to be justified.

Will check the "syscall_trace_enter" inline performance impact.

...

| Metric     | W/O this patch | With this patch | Change    |
| ---------- | -------------- | --------------- | --------- |
| Total time | 2.171 [sec]    | 2.114 [sec]     |  ↓2.6%    |
| usecs/op   | 0.217192       | 0.211453        |  ↓2.6%    |
| ops/sec    | 4,604,225      | 4,729,178       |  ↑2.7%    |

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

include/linux/entry-common.h | 63 ++++++++++++++++++++++++++++++++++- kernel/entry/syscall-common.c | 59 ++------------------------------

These changes are purely generic, surely all architectures using GENERIC_ENTRY should get similar benefits (assuming LTO isn't used)?

It would be great if someone could help test the performance differences.

...

2 files changed, 64 insertions(+), 58 deletions(-)

diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h index cd6dacb2d8bf..2f84377fb016 100644 --- a/include/linux/entry-common.h +++ b/include/linux/entry-common.h @@ -2,6 +2,7 @@ #ifndef __LINUX_ENTRYCOMMON_H #define __LINUX_ENTRYCOMMON_H +#include <linux/audit.h> #include <linux/irq-entry-common.h> #include <linux/ptrace.h> #include <linux/seccomp.h> @@ -128,6 +129,41 @@ static __always_inline long syscall_enter_from_user_mode(struct pt_regs *regs, l return ret; } +/*

• • If SYSCALL_EMU is set, then the only reason to report is when
• • SINGLESTEP is set (i.e. PTRACE_SYSEMU_SINGLESTEP). This syscall
• • instruction has been already reported in syscall_enter_from_user_mode().
• */

+static __always_inline bool report_single_step(unsigned long work) +{

• if (work & SYSCALL_WORK_SYSCALL_EMU)

• return false;
  

• return work & SYSCALL_WORK_SYSCALL_EXIT_TRAP;

+}

+/**

• • arch_ptrace_report_syscall_exit - Architecture specific

• • 		     ptrace_report_syscall_exit.
    

• • Invoked from syscall_exit_work() to wrap ptrace_report_syscall_exit().
• • The main purpose is to support arch-specific ptrace_report_syscall_exit
• • implementation.
• */

+static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs,

• 					    int step);
  

+#ifndef arch_ptrace_report_syscall_exit +static __always_inline void arch_ptrace_report_syscall_exit(struct pt_regs *regs,

• 					    int step)
  

+{

• ptrace_report_syscall_exit(regs, step);

+} +#endif

If we want syscall_exit_work() to be inline, then why would we define this hook in syscall-common.c in patch 12? Might as well define both hooks in entry-common.h right away and avoid some noise here.

Make sense.

• Kevin

...

[...]

On 04/12/25 1:51 PM, Jinjie Ruan wrote:

Since commit b3cf07851b6c ("arm64: entry: Switch to generic IRQ entry"), _TIF_WORK_MASK is never used, so remove it.

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

---

arch/arm64/include/asm/thread_info.h | 6 ------ 1 file changed, 6 deletions(-)

diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index f241b8601ebd..ff4998fa1844 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -106,12 +106,6 @@ void arch_setup_new_exec(void); #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_TSC_SIGSEGV (1 << TIF_TSC_SIGSEGV) -#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY | \

• 		 _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \
  

• 		 _TIF_UPROBE | _TIF_MTE_ASYNC_FAULT | \
  

• 		 _TIF_NOTIFY_SIGNAL | _TIF_SIGPENDING | \
  

• 		 _TIF_PATCH_PENDING)
  

#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ _TIF_SYSCALL_EMU)

There are no more _TIF_WORK_MASK users left on arm64 platform.

Reviewed-by: Anshuman Khandual anshuman.khandual@arm.com

On 04/12/25 1:51 PM, Jinjie Ruan wrote:

The generic entry abort the syscall_trace_enter() sequence if ptrace_report_syscall_entry() errors out, but arm64 not.

As the ptrace_report_syscall_entry() comment said, the calling arch code should abort the system call and must prevent normal entry so no system call is made if ptrace_report_syscall_entry() return nonzero.

In preparation for moving arm64 over to the generic entry code, return early if ptrace_report_syscall_entry() encounters an error.

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

arch/arm64/kernel/ptrace.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 4532e9831856..3926ce6aa268 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2346,15 +2346,18 @@ static __always_inline unsigned long ptrace_save_reg(struct pt_regs *regs, return saved_reg; } -static void report_syscall_entry(struct pt_regs *regs) +static int report_syscall_entry(struct pt_regs *regs) { unsigned long saved_reg;

• int regno;

• int regno, ret;

saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_ENTER, &regno);

• if (ptrace_report_syscall_entry(regs))

• ret = ptrace_report_syscall_entry(regs);
• if (ret) forget_syscall(regs); regs->regs[regno] = saved_reg;
• return ret;

} static void report_syscall_exit(struct pt_regs *regs) @@ -2381,10 +2384,11 @@ static void report_syscall_exit(struct pt_regs *regs) int syscall_trace_enter(struct pt_regs *regs) { unsigned long flags = read_thread_flags();

• int ret;

if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) {

• report_syscall_entry(regs);
  

• if (flags & _TIF_SYSCALL_EMU)
  

• ret = report_syscall_entry(regs);
  

• if (ret || (flags & _TIF_SYSCALL_EMU))
  

A small nit.

Could report_syscall_entry(regs) be checked directly in the 'if' conditional code block thus dropping new 'ret' altogether ?

	return NO_SYSCALL;

}

On 2025/12/8 13:48, Anshuman Khandual wrote:

On 04/12/25 1:51 PM, Jinjie Ruan wrote:

...

The generic syscall entry code has the following form, which use the input syscall work flag and syscall number:

| syscall_trace_enter(struct pt_regs *regs, long syscall, | unsigned long work) | | syscall_exit_work(struct pt_regs *regs, unsigned long work)

In preparation for moving arm64 over to the generic entry code, refactor syscall_trace_enter/exit() to also pass thread flags, and get syscall number by syscall_get_nr() helper.

No functional changes.

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

---

arch/arm64/include/asm/syscall.h | 4 ++-- arch/arm64/kernel/ptrace.c | 25 ++++++++++++++++--------- arch/arm64/kernel/syscall.c | 5 +++-- 3 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/syscall.h b/arch/arm64/include/asm/syscall.h index 5e4c7fc44f73..cef1d9ce6e1b 100644 --- a/arch/arm64/include/asm/syscall.h +++ b/arch/arm64/include/asm/syscall.h @@ -120,7 +120,7 @@ static inline int syscall_get_arch(struct task_struct *task) return AUDIT_ARCH_AARCH64; } -int syscall_trace_enter(struct pt_regs *regs); -void syscall_trace_exit(struct pt_regs *regs); +int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags); +void syscall_trace_exit(struct pt_regs *regs, unsigned long flags); #endif /* __ASM_SYSCALL_H */ diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 3926ce6aa268..b962cbcca11d 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2381,9 +2381,8 @@ static void report_syscall_exit(struct pt_regs *regs) } } -int syscall_trace_enter(struct pt_regs *regs) +int syscall_trace_enter(struct pt_regs *regs, long syscall, unsigned long flags) {

• unsigned long flags = read_thread_flags(); int ret;

if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) { @@ -2396,19 +2395,27 @@ int syscall_trace_enter(struct pt_regs *regs) if (secure_computing() == -1) return NO_SYSCALL;

• if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))

• trace_sys_enter(regs, regs->syscallno);
  

• /* Either of the above might have changed the syscall number */
• syscall = syscall_get_nr(current, regs);

• audit_syscall_entry(regs->syscallno, regs->orig_x0, regs->regs[1],

• if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) {

• trace_sys_enter(regs, syscall);
  

• /*
  

•  * Probes or BPF hooks in the tracepoint may have changed the
  

•  * system call number as well.
  

•  */
  

•  syscall = syscall_get_nr(current, regs);
  

• }
• audit_syscall_entry(syscall, regs->orig_x0, regs->regs[1], regs->regs[2], regs->regs[3]);

• return regs->syscallno;

• return syscall;

regs->syscallno still remains unchanged even if syscall is evaluated to be different ?

No, arm64 syscall_get_nr() returns regs->syscallno, use syscall_get_nr() helper here to get the local "syscall" is to stay consistent with the generic entry, without any semantic change.

23 static inline int syscall_get_nr(struct task_struct *task, 24 >------->------->------->------- struct pt_regs *regs) 25 { 26 >-------return regs->syscallno; 27 }

...

} -void syscall_trace_exit(struct pt_regs *regs) +void syscall_trace_exit(struct pt_regs *regs, unsigned long flags) {

• unsigned long flags = read_thread_flags();
• audit_syscall_exit(regs);

if (flags & _TIF_SYSCALL_TRACEPOINT) diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index aba7ca6bca2d..ec31f82d2e9f 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -124,7 +124,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, */ if (scno == NO_SYSCALL) syscall_set_return_value(current, regs, -ENOSYS, 0);

• scno = syscall_trace_enter(regs);
  

• scno = syscall_trace_enter(regs, scno, flags);
  

  if (scno == NO_SYSCALL) goto trace_exit; }

@@ -143,7 +143,8 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, } trace_exit:

• syscall_trace_exit(regs);

• flags = read_thread_flags();
• syscall_trace_exit(regs, flags);

} void do_el0_svc(struct pt_regs *regs)

On 04/12/2025 09:21, Jinjie Ruan wrote:

The generic entry abort the syscall_trace_enter() sequence if ptrace_report_syscall_entry() errors out, but arm64 not.

As the ptrace_report_syscall_entry() comment said, the calling arch code should abort the system call and must prevent normal entry so no system call is made if ptrace_report_syscall_entry() return nonzero.

This patch is now in the right position, which means that arm64 does abort the syscall already without this patch. What we're changing here is that the tracing sequence is interrupted. The comment you're referring to says nothing about tracing so I'm not sure it helps to refer to it.

Otherwise:

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com

In preparation for moving arm64 over to the generic entry code, return early if ptrace_report_syscall_entry() encounters an error.

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

arch/arm64/kernel/ptrace.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 4532e9831856..3926ce6aa268 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -2346,15 +2346,18 @@ static __always_inline unsigned long ptrace_save_reg(struct pt_regs *regs, return saved_reg; } -static void report_syscall_entry(struct pt_regs *regs) +static int report_syscall_entry(struct pt_regs *regs) { unsigned long saved_reg;

• int regno;

• int regno, ret;

saved_reg = ptrace_save_reg(regs, PTRACE_SYSCALL_ENTER, &regno);

• if (ptrace_report_syscall_entry(regs))

• ret = ptrace_report_syscall_entry(regs);
• if (ret) forget_syscall(regs); regs->regs[regno] = saved_reg;
• return ret;

} static void report_syscall_exit(struct pt_regs *regs) @@ -2381,10 +2384,11 @@ static void report_syscall_exit(struct pt_regs *regs) int syscall_trace_enter(struct pt_regs *regs) { unsigned long flags = read_thread_flags();

• int ret;

if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) {

• report_syscall_entry(regs);
  

• if (flags & _TIF_SYSCALL_EMU)
  

• ret = report_syscall_entry(regs);
  

• if (ret || (flags & _TIF_SYSCALL_EMU))
  return NO_SYSCALL;
  

  }

On 04/12/2025 09:21, Jinjie Ruan wrote:

After switch arm64 to Generic Entry, the compiler no longer inlines

Did it inline it before this series?

el0_svc_common() into do_el0_svc(). So inline el0_svc_common() and it has 1% performance uplift on perf bench basic syscall on kunpeng920 as below.

| Metric | W/O this patch | With this patch | Change | | ---------- | -------------- | --------------- | --------- | | Total time | 2.195 [sec] | 2.171 [sec] | ↓1.1% | | usecs/op | 0.219575 | 0.217192 | ↓1.1% | | ops/sec | 4,554,260 | 4,604,225 | ↑1.1% |

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

I think this is sensible - do_el0_svc() is clearly hot and the small increase in code size is completely justified. It also removes a performance regression when enabling CONFIG_COMPAT (without it el0_svc_common() has only one caller so it should be inlined regardless).

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com

---

arch/arm64/kernel/syscall.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 47e193a1cfff..5aa51da9ec25 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -66,8 +66,8 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno, choose_random_kstack_offset(get_random_u16()); } -static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,

• 	   const syscall_fn_t syscall_table[])
  

+static __always_inline void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,

• 			   const syscall_fn_t syscall_table[])
  

{ unsigned long work = READ_ONCE(current_thread_info()->syscall_work); unsigned long flags = read_thread_flags();

On 2025/12/9 21:48, Kevin Brodsky wrote:

On 04/12/2025 09:21, Jinjie Ruan wrote:

...

After switch arm64 to Generic Entry, the compiler no longer inlines

Did it inline it before this series?

Yes, as below

<do_el0_svc>: d503201f nop d503201f nop d503233f paciasp a9be7bfd stp x29, x30, [sp, #-32]! 910003fd mov x29, sp a90153f3 stp x19, x20, [sp, #16] aa0003f3 mov x19, x0 d5384114 mrs x20, sp_el0 f9400001 ldr x1, [x0] f9400282 ldr x2, [x20] f9008801 str x1, [x0, #272] f9402001 ldr x1, [x0, #64] b9011801 str w1, [x0, #280] 373001e2 tbnz w2, #6, ffff80008002c0f0 <do_el0_svc+0x70> f278105f tst x2, #0x1f00 54000261 b.ne ffff80008002c108 <do_el0_svc+0x88> // b.any 52803ac2 mov w2, #0x1d6 // #470 97ffffb1 bl ffff80008002bf88 <invoke_syscall.constprop.0> f9400280 ldr x0, [x20] 92783400 and x0, x0, #0x3fff00 926bdc00 and x0, x0, #0xffffffffffe01fff b4000060 cbz x0, ffff80008002c0e0 <do_el0_svc+0x60> aa1303e0 mov x0, x19 97ffc835 bl ffff80008001e1b0 <syscall_trace_exit> a94153f3 ldp x19, x20, [sp, #16] a8c27bfd ldp x29, x30, [sp], #32 d50323bf autiasp d65f03c0 ret 92804000 mov x0, #0xfffffffffffffdff // #-513 f9000260 str x0, [x19] a94153f3 ldp x19, x20, [sp, #16] a8c27bfd ldp x29, x30, [sp], #32 d50323bf autiasp d65f03c0 ret 3100043f cmn w1, #0x1 54000140 b.eq ffff80008002c134 <do_el0_svc+0xb4> // b.none aa1303e0 mov x0, x19 97ffc7c1 bl ffff80008001e018 <syscall_trace_enter> 2a0003e1 mov w1, w0 3100041f cmn w0, #0x1 54fffdc0 b.eq ffff80008002c0d8 <do_el0_svc+0x58> // b.none aa1303e0 mov x0, x19 52803ac2 mov w2, #0x1d6 // #470 97ffff97 bl ffff80008002bf88 <invoke_syscall.constprop.0> 17ffffea b ffff80008002c0d8 <do_el0_svc+0x58> 928004a0 mov x0, #0xffffffffffffffda // #-38 f9000260 str x0, [x19] 17fffff5 b ffff80008002c110 <do_el0_svc+0x90> d53cd044 mrs x4, tpidr_el2 d53cd040 mrs x0, tpidr_el2 d53cd041 mrs x1, tpidr_el2 00000000 udf #0 d503201f nop d503201f nop

...

el0_svc_common() into do_el0_svc(). So inline el0_svc_common() and it has 1% performance uplift on perf bench basic syscall on kunpeng920 as below.

| Metric | W/O this patch | With this patch | Change | | ---------- | -------------- | --------------- | --------- | | Total time | 2.195 [sec] | 2.171 [sec] | ↓1.1% | | usecs/op | 0.219575 | 0.217192 | ↓1.1% | | ops/sec | 4,554,260 | 4,604,225 | ↑1.1% |

Signed-off-by: Jinjie Ruan ruanjinjie@huawei.com

I think this is sensible - do_el0_svc() is clearly hot and the small increase in code size is completely justified. It also removes a performance regression when enabling CONFIG_COMPAT (without it el0_svc_common() has only one caller so it should be inlined regardless).

Reviewed-by: Kevin Brodsky kevin.brodsky@arm.com

...

---

arch/arm64/kernel/syscall.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 47e193a1cfff..5aa51da9ec25 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -66,8 +66,8 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno, choose_random_kstack_offset(get_random_u16()); } -static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,

• 	   const syscall_fn_t syscall_table[])
  

+static __always_inline void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,

• 			   const syscall_fn_t syscall_table[])
  

{ unsigned long work = READ_ONCE(current_thread_info()->syscall_work); unsigned long flags = read_thread_flags();