March 2019 - Linux-kselftest-mirror

Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

by Karim Yaghmour

Hi Alan, On 3/18/19 2:57 PM, Alan Cox wrote: >> I think the compressed tarball is much simpler/easier overall. If >> someone really wants the filesystem, they just uncompress it into a >> tmpfs mount. It's much less moving kernel code to worry about. > > There is an even simpler approach. The people who want this for whatever > strange reason are Android folks. Android lives on flash, so all they have > to do is put the headers in a flash file system that is updated with the > kernel and mount it wherever they like. Simple matter of a bit of > devicetree no ? The Android use-case scenario might indeed have been the one to best crystallize the requirement, but that doesn't mean that all use-cases of eBPF wouldn't benefit from this -- which in fact they would, see instructions here for example on the need for kernel headers: https://github.com/iovisor/bcc/blob/master/INSTALL.md Just a quick $PREFERRED_SEARCH_ENGINE search returns interesting exchanges such as this one taken from a discussion thread on LWN covering an introduction to eBPF: https://lwn.net/Articles/741348/ Effectively this person had to be hand-held in understanding that they needed a good set of kernel headers to make the tool work. Their comment after being shown that this was needed was: "It looks like the headers package you need on Ubuntu is linux-headers-$(uname -r), which contains the entire kernel source tree, and is specific to the running kernel." Surely having Joel's patch in the kernel would obviate the issue for all Linux kernel users, not just Android. Cheers, -- Karim Yaghmour CEO - Opersys inc. / www.opersys.com http://twitter.com/karimyaghmour

5 years, 9 months

1
0
0 0

selftests: bpf: test_align Failed to find match 1: R1=ctx(id=0,off=0,imm=0) func#0 @0

by Naresh Kamboju

selftests: bpf: test_align failed on all devices ( arm64, armv7, x86_64 and i386) running Linux -next kernel 5.1.0-rc1 tag: next-20190318. Here is the log from arm64, selftests: bpf: test_align Test 0: mov ... Failed to find match 1: R1=ctx(id=0,off=0,imm=0) func#0 @0 0: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R10=fp0,call_-1 0: (b7) r3 = 2 1: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv2 R10=fp0,call_-1 1: (b7) r3 = 4 2: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv4 R10=fp0,call_-1 2: (b7) r3 = 8 3: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv8 R10=fp0,call_-1 3: (b7) r3 = 16 4: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv16 R10=fp0,call_-1 4: (b7) r3 = 32 5: R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv32 R10=fp0,call_-1 5: (b7) r0 = 0 6: R0_w=inv0 R1=ctx(id=0 ref_obj_id=0,off=0,imm=0) R3_w=inv32 R10=fp0,call_-1 6: (95) exit processed 7 insns (limit 131072), stack depth 0 FAIL Test 1: shift ... Failed to find match 1: R1=ctx(id=0,off=0,imm=0) func#0 @0 ... processed 6 insns (limit 131072), stack depth 0 FAIL ... processed 22 insns (limit 131072), stack depth 0 FAIL Results: 0 pass 12 fail not ok 1..7 selftests: bpf: test_align [FAIL] selftests: bpf_test_align [FAIL] Test results comparison, https://qa-reports.linaro.org/lkft/linux-next-oe/tests/kselftest/bpf_test_a… Test complete log ( and you might notice large number of bpf test failure) https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190318/testru… Good linux -next tag: next-20190306 Bad linux -next tag: next-20190318 git branch master git commit b9998194591467dc562c23ecefb63af4eff7530b git describe next-20190318 git repo https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git Best regards Naresh Kamboju

5 years, 9 months

3
2
0 0

Re: [PATCH v11 09/14] kernel, arm64: untag user pointers in prctl_set_mm*

by Andrey Konovalov

On Sat, Mar 16, 2019 at 8:32 PM kbuild test robot <lkp(a)intel.com> wrote: > > Hi Andrey, > > Thank you for the patch! Yet something to improve: > > [auto build test ERROR on linus/master] > [also build test ERROR on v5.0 next-20190306] > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] > > url: https://github.com/0day-ci/linux/commits/Andrey-Konovalov/uaccess-add-untag… > config: x86_64-randconfig-x012-201911 (attached as .config) > compiler: gcc-7 (Debian 7.3.0-1) 7.3.0 > reproduce: > # save the attached .config to linux build tree > make ARCH=x86_64 > > All errors (new ones prefixed by >>): > > kernel/sys.c: In function 'prctl_set_mm_map': > >> kernel/sys.c:1996:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->start_code = untagged_addr(prctl_map.start_code); > ^~ > kernel/sys.c:1997:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->end_code = untagged_addr(prctl_map.end_code); > ^~ > kernel/sys.c:1998:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->start_data = untagged_addr(prctl_map.start_data); > ^~ > kernel/sys.c:1999:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->end_data = untagged_addr(prctl_map.end_data); > ^~ > kernel/sys.c:2000:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->start_brk = untagged_addr(prctl_map.start_brk); > ^~ > kernel/sys.c:2001:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->brk = untagged_addr(prctl_map.brk); > ^~ > kernel/sys.c:2002:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->start_stack = untagged_addr(prctl_map.start_stack); > ^~ > kernel/sys.c:2003:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->arg_start = untagged_addr(prctl_map.arg_start); > ^~ > kernel/sys.c:2004:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->arg_end = untagged_addr(prctl_map.arg_end); > ^~ > kernel/sys.c:2005:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->env_start = untagged_addr(prctl_map.env_start); > ^~ > kernel/sys.c:2006:11: error: invalid type argument of '->' (have 'struct prctl_mm_map') > prctl_map->env_end = untagged_addr(prctl_map.env_end); > ^~ > > vim +1996 kernel/sys.c Right, I didn't have the related config options enabled when I did the testing... > > 1974 > 1975 #ifdef CONFIG_CHECKPOINT_RESTORE > 1976 static int prctl_set_mm_map(int opt, const void __user *addr, unsigned long data_size) > 1977 { > 1978 struct prctl_mm_map prctl_map = { .exe_fd = (u32)-1, }; > 1979 unsigned long user_auxv[AT_VECTOR_SIZE]; > 1980 struct mm_struct *mm = current->mm; > 1981 int error; > 1982 > 1983 BUILD_BUG_ON(sizeof(user_auxv) != sizeof(mm->saved_auxv)); > 1984 BUILD_BUG_ON(sizeof(struct prctl_mm_map) > 256); > 1985 > 1986 if (opt == PR_SET_MM_MAP_SIZE) > 1987 return put_user((unsigned int)sizeof(prctl_map), > 1988 (unsigned int __user *)addr); > 1989 > 1990 if (data_size != sizeof(prctl_map)) > 1991 return -EINVAL; > 1992 > 1993 if (copy_from_user(&prctl_map, addr, sizeof(prctl_map))) > 1994 return -EFAULT; > 1995 > > 1996 prctl_map->start_code = untagged_addr(prctl_map.start_code); > 1997 prctl_map->end_code = untagged_addr(prctl_map.end_code); > 1998 prctl_map->start_data = untagged_addr(prctl_map.start_data); > 1999 prctl_map->end_data = untagged_addr(prctl_map.end_data); > 2000 prctl_map->start_brk = untagged_addr(prctl_map.start_brk); > 2001 prctl_map->brk = untagged_addr(prctl_map.brk); > 2002 prctl_map->start_stack = untagged_addr(prctl_map.start_stack); > 2003 prctl_map->arg_start = untagged_addr(prctl_map.arg_start); > 2004 prctl_map->arg_end = untagged_addr(prctl_map.arg_end); > 2005 prctl_map->env_start = untagged_addr(prctl_map.env_start); > 2006 prctl_map->env_end = untagged_addr(prctl_map.env_end); > 2007 > 2008 error = validate_prctl_map(&prctl_map); > 2009 if (error) > 2010 return error; > 2011 > 2012 if (prctl_map.auxv_size) { > 2013 memset(user_auxv, 0, sizeof(user_auxv)); > 2014 if (copy_from_user(user_auxv, > 2015 (const void __user *)prctl_map.auxv, > 2016 prctl_map.auxv_size)) > 2017 return -EFAULT; > 2018 > 2019 /* Last entry must be AT_NULL as specification requires */ > 2020 user_auxv[AT_VECTOR_SIZE - 2] = AT_NULL; > 2021 user_auxv[AT_VECTOR_SIZE - 1] = AT_NULL; > 2022 } > 2023 > 2024 if (prctl_map.exe_fd != (u32)-1) { > 2025 error = prctl_set_mm_exe_file(mm, prctl_map.exe_fd); > 2026 if (error) > 2027 return error; > 2028 } > 2029 > 2030 /* > 2031 * arg_lock protects concurent updates but we still need mmap_sem for > 2032 * read to exclude races with sys_brk. > 2033 */ > 2034 down_read(&mm->mmap_sem); > 2035 > 2036 /* > 2037 * We don't validate if these members are pointing to > 2038 * real present VMAs because application may have correspond > 2039 * VMAs already unmapped and kernel uses these members for statistics > 2040 * output in procfs mostly, except > 2041 * > 2042 * - @start_brk/@brk which are used in do_brk but kernel lookups > 2043 * for VMAs when updating these memvers so anything wrong written > 2044 * here cause kernel to swear at userspace program but won't lead > 2045 * to any problem in kernel itself > 2046 */ > 2047 > 2048 spin_lock(&mm->arg_lock); > 2049 mm->start_code = prctl_map.start_code; > 2050 mm->end_code = prctl_map.end_code; > 2051 mm->start_data = prctl_map.start_data; > 2052 mm->end_data = prctl_map.end_data; > 2053 mm->start_brk = prctl_map.start_brk; > 2054 mm->brk = prctl_map.brk; > 2055 mm->start_stack = prctl_map.start_stack; > 2056 mm->arg_start = prctl_map.arg_start; > 2057 mm->arg_end = prctl_map.arg_end; > 2058 mm->env_start = prctl_map.env_start; > 2059 mm->env_end = prctl_map.env_end; > 2060 spin_unlock(&mm->arg_lock); > 2061 > 2062 /* > 2063 * Note this update of @saved_auxv is lockless thus > 2064 * if someone reads this member in procfs while we're > 2065 * updating -- it may get partly updated results. It's > 2066 * known and acceptable trade off: we leave it as is to > 2067 * not introduce additional locks here making the kernel > 2068 * more complex. > 2069 */ > 2070 if (prctl_map.auxv_size) > 2071 memcpy(mm->saved_auxv, user_auxv, sizeof(user_auxv)); > 2072 > 2073 up_read(&mm->mmap_sem); > 2074 return 0; > 2075 } > 2076 #endif /* CONFIG_CHECKPOINT_RESTORE */ > 2077 > > --- > 0-DAY kernel test infrastructure Open Source Technology Center > https://lists.01.org/pipermail/kbuild-all Intel Corporation

5 years, 9 months

1
0
0 0

[PATCH v9 6/9] kselftests: cgroup: add freezer controller self-tests

by Roman Gushchin

This patch implements 9 tests for the freezer controller for cgroup v2: 1) a simple test, which aims to freeze and unfreeze a cgroup with 100 processes 2) a more complicated tree test, which creates a hierarchy of cgroups, puts some processes in some cgroups, and tries to freeze and unfreeze different parts of the subtree 3) a forkbomb test: the test aims to freeze a forkbomb running in a cgroup, kill all tasks in the cgroup and remove the cgroup without the unfreezing. 4) rmdir test: the test creates two nested cgroups, freezes the parent one, checks that the child can be successfully removed, and a new child can be created 5) migration tests: the test checks migration of a task between frozen cgroups: from a frozen to a running, from a running to a frozen, and from a frozen to a frozen. 6) ptrace test: the test checks that it's possible to attach to a process in a frozen cgroup, get some information and detach, and the cgroup will remain frozen. 7) stopped test: the test checks that it's possible to freeze a cgroup with a stopped task 8) ptraced test: the test checks that it's possible to freeze a cgroup with a ptraced task 9) vfork test: the test checks that it's possible to freeze a cgroup with a parent process waiting for the child process in vfork() Expected output: $ ./test_freezer ok 1 test_cgfreezer_simple ok 2 test_cgfreezer_tree ok 3 test_cgfreezer_forkbomb ok 4 test_cgrreezer_rmdir ok 5 test_cgfreezer_migrate ok 6 test_cgfreezer_ptrace ok 7 test_cgfreezer_stopped ok 8 test_cgfreezer_ptraced ok 9 test_cgfreezer_vfork Signed-off-by: Roman Gushchin <guro(a)fb.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: Tejun Heo <tj(a)kernel.org> Cc: kernel-team(a)fb.com Cc: linux-kselftest(a)vger.kernel.org --- tools/testing/selftests/cgroup/.gitignore | 1 + tools/testing/selftests/cgroup/Makefile | 2 + tools/testing/selftests/cgroup/cgroup_util.c | 54 +- tools/testing/selftests/cgroup/cgroup_util.h | 5 + tools/testing/selftests/cgroup/test_freezer.c | 851 ++++++++++++++++++ 5 files changed, 912 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/cgroup/test_freezer.c diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore index adacda50a4b2..7f9835624793 100644 --- a/tools/testing/selftests/cgroup/.gitignore +++ b/tools/testing/selftests/cgroup/.gitignore @@ -1,2 +1,3 @@ test_memcontrol test_core +test_freezer diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index 23fbaa4a9630..8d369b6a2069 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -5,8 +5,10 @@ all: TEST_GEN_PROGS = test_memcontrol TEST_GEN_PROGS += test_core +TEST_GEN_PROGS += test_freezer include ../lib.mk $(OUTPUT)/test_memcontrol: cgroup_util.c $(OUTPUT)/test_core: cgroup_util.c +$(OUTPUT)/test_freezer: cgroup_util.c diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index eba06f94433b..4c223266299a 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -74,6 +74,16 @@ char *cg_name_indexed(const char *root, const char *name, int index) return ret; } +char *cg_control(const char *cgroup, const char *control) +{ + size_t len = strlen(cgroup) + strlen(control) + 2; + char *ret = malloc(len); + + snprintf(ret, len, "%s/%s", cgroup, control); + + return ret; +} + int cg_read(const char *cgroup, const char *control, char *buf, size_t len) { char path[PATH_MAX]; @@ -196,7 +206,32 @@ int cg_create(const char *cgroup) return mkdir(cgroup, 0644); } -static int cg_killall(const char *cgroup) +int cg_wait_for_proc_count(const char *cgroup, int count) +{ + char buf[10 * PAGE_SIZE] = {0}; + int attempts; + char *ptr; + + for (attempts = 10; attempts >= 0; attempts--) { + int nr = 0; + + if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf))) + break; + + for (ptr = buf; *ptr; ptr++) + if (*ptr == '\n') + nr++; + + if (nr >= count) + return 0; + + usleep(100000); + } + + return -1; +} + +int cg_killall(const char *cgroup) { char buf[PAGE_SIZE]; char *ptr = buf; @@ -238,6 +273,14 @@ int cg_destroy(const char *cgroup) return ret; } +int cg_enter(const char *cgroup, int pid) +{ + char pidbuf[64]; + + snprintf(pidbuf, sizeof(pidbuf), "%d", pid); + return cg_write(cgroup, "cgroup.procs", pidbuf); +} + int cg_enter_current(const char *cgroup) { char pidbuf[64]; @@ -367,3 +410,12 @@ int set_oom_adj_score(int pid, int score) close(fd); return 0; } + +char proc_read_text(int pid, const char *item, char *buf, size_t size) +{ + char path[PATH_MAX]; + + snprintf(path, sizeof(path), "/proc/%d/%s", pid, item); + + return read_text(path, buf, size); +} diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h index 9ac8b7958f83..c72f28046bfa 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.h +++ b/tools/testing/selftests/cgroup/cgroup_util.h @@ -18,6 +18,7 @@ static inline int values_close(long a, long b, int err) extern int cg_find_unified_root(char *root, size_t len); extern char *cg_name(const char *root, const char *name); extern char *cg_name_indexed(const char *root, const char *name, int index); +extern char *cg_control(const char *cgroup, const char *control); extern int cg_create(const char *cgroup); extern int cg_destroy(const char *cgroup); extern int cg_read(const char *cgroup, const char *control, @@ -32,6 +33,7 @@ extern int cg_write(const char *cgroup, const char *control, char *buf); extern int cg_run(const char *cgroup, int (*fn)(const char *cgroup, void *arg), void *arg); +extern int cg_enter(const char *cgroup, int pid); extern int cg_enter_current(const char *cgroup); extern int cg_run_nowait(const char *cgroup, int (*fn)(const char *cgroup, void *arg), @@ -41,3 +43,6 @@ extern int alloc_pagecache(int fd, size_t size); extern int alloc_anon(const char *cgroup, void *arg); extern int is_swap_enabled(void); extern int set_oom_adj_score(int pid, int score); +extern int cg_wait_for_proc_count(const char *cgroup, int count); +extern int cg_killall(const char *cgroup); +extern char proc_read_text(int pid, const char *item, char *buf, size_t size); diff --git a/tools/testing/selftests/cgroup/test_freezer.c b/tools/testing/selftests/cgroup/test_freezer.c new file mode 100644 index 000000000000..2bfddb6d6d3b --- /dev/null +++ b/tools/testing/selftests/cgroup/test_freezer.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include <stdbool.h> +#include <linux/limits.h> +#include <sys/ptrace.h> +#include <sys/types.h> +#include <sys/mman.h> +#include <unistd.h> +#include <stdio.h> +#include <errno.h> +#include <poll.h> +#include <stdlib.h> +#include <sys/inotify.h> +#include <string.h> +#include <sys/types.h> +#include <sys/wait.h> + +#include "../kselftest.h" +#include "cgroup_util.h" + +#define DEBUG +#ifdef DEBUG +#define debug(args...) fprintf(stderr, args) +#else +#define debug(args...) +#endif + +/* + * Check if the cgroup is frozen by looking at the cgroup.events::frozen value. + */ +static int cg_check_frozen(const char *cgroup, bool frozen) +{ + if (frozen) { + if (cg_read_strstr(cgroup, "cgroup.events", "frozen 1") != 0) { + debug("Cgroup %s isn't frozen\n", cgroup); + return -1; + } + } else { + /* + * Check the cgroup.events::frozen value. + */ + if (cg_read_strstr(cgroup, "cgroup.events", "frozen 0") != 0) { + debug("Cgroup %s is frozen\n", cgroup); + return -1; + } + } + + return 0; +} + +/* + * Freeze the given cgroup. + */ +static int cg_freeze_nowait(const char *cgroup, bool freeze) +{ + return cg_write(cgroup, "cgroup.freeze", freeze ? "1" : "0"); +} + +/* + * Prepare for waiting on cgroup.events file. + */ +static int cg_prepare_for_wait(const char *cgroup) +{ + int fd, ret = -1; + + fd = inotify_init1(0); + if (fd == -1) { + debug("Error: inotify_init1() failed\n"); + return fd; + } + + ret = inotify_add_watch(fd, cg_control(cgroup, "cgroup.events"), + IN_MODIFY); + if (ret == -1) { + debug("Error: inotify_add_watch() failed\n"); + close(fd); + } + + return fd; +} + +/* + * Wait for an event. If there are no events for 10 seconds, + * treat this an error. + */ +static int cg_wait_for(int fd) +{ + int ret = -1; + struct pollfd fds = { + .fd = fd, + .events = POLLIN, + }; + + while (true) { + ret = poll(&fds, 1, 10000); + + if (ret == -1) { + if (errno == EINTR) + continue; + debug("Error: poll() failed\n"); + break; + } + + if (ret > 0 && fds.revents & POLLIN) { + ret = 0; + break; + } + } + + return ret; +} + +/* + * Attach a task to the given cgroup and wait for a cgroup frozen event. + * All transient events (e.g. populated) are ignored. + */ +static int cg_enter_and_wait_for_frozen(const char *cgroup, int pid, + bool frozen) +{ + int fd, ret = -1; + int attempts; + + fd = cg_prepare_for_wait(cgroup); + if (fd < 0) + return fd; + + ret = cg_enter(cgroup, pid); + if (ret) + goto out; + + for (attempts = 0; attempts < 10; attempts++) { + ret = cg_wait_for(fd); + if (ret) + break; + + ret = cg_check_frozen(cgroup, frozen); + if (ret) + continue; + } + +out: + close(fd); + return ret; +} + +/* + * Freeze the given cgroup and wait for the inotify signal. + * If there are no events in 10 seconds, treat this as an error. + * Then check that the cgroup is in the desired state. + */ +static int cg_freeze_wait(const char *cgroup, bool freeze) +{ + int fd, ret = -1; + + fd = cg_prepare_for_wait(cgroup); + if (fd < 0) + return fd; + + ret = cg_freeze_nowait(cgroup, freeze); + if (ret) { + debug("Error: cg_freeze_nowait() failed\n"); + goto out; + } + + ret = cg_wait_for(fd); + if (ret) + goto out; + + ret = cg_check_frozen(cgroup, freeze); +out: + close(fd); + return ret; +} + +/* + * A simple process running in a sleep loop until being + * re-parented. + */ +static int child_fn(const char *cgroup, void *arg) +{ + int ppid = getppid(); + + while (getppid() == ppid) + usleep(1000); + + return getppid() == ppid; +} + +/* + * A simple test for the cgroup freezer: populated the cgroup with 100 + * running processes and freeze it. Then unfreeze it. Then it kills all + * processes and destroys the cgroup. + */ +static int test_cgfreezer_simple(const char *root) +{ + int ret = KSFT_FAIL; + char *cgroup = NULL; + int i; + + cgroup = cg_name(root, "cg_test_simple"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + for (i = 0; i < 100; i++) + cg_run_nowait(cgroup, child_fn, NULL); + + if (cg_wait_for_proc_count(cgroup, 100)) + goto cleanup; + + if (cg_check_frozen(cgroup, false)) + goto cleanup; + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + if (cg_freeze_wait(cgroup, false)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +/* + * The test creates the following hierarchy: + * A + * / / \ \ + * B E I K + * /\ | + * C D F + * | + * G + * | + * H + * + * with a process in C, H and 3 processes in K. + * Then it tries to freeze and unfreeze the whole tree. + */ +static int test_cgfreezer_tree(const char *root) +{ + char *cgroup[10] = {0}; + int ret = KSFT_FAIL; + int i; + + cgroup[0] = cg_name(root, "cg_test_tree_A"); + if (!cgroup[0]) + goto cleanup; + + cgroup[1] = cg_name(cgroup[0], "B"); + if (!cgroup[1]) + goto cleanup; + + cgroup[2] = cg_name(cgroup[1], "C"); + if (!cgroup[2]) + goto cleanup; + + cgroup[3] = cg_name(cgroup[1], "D"); + if (!cgroup[3]) + goto cleanup; + + cgroup[4] = cg_name(cgroup[0], "E"); + if (!cgroup[4]) + goto cleanup; + + cgroup[5] = cg_name(cgroup[4], "F"); + if (!cgroup[5]) + goto cleanup; + + cgroup[6] = cg_name(cgroup[5], "G"); + if (!cgroup[6]) + goto cleanup; + + cgroup[7] = cg_name(cgroup[6], "H"); + if (!cgroup[7]) + goto cleanup; + + cgroup[8] = cg_name(cgroup[0], "I"); + if (!cgroup[8]) + goto cleanup; + + cgroup[9] = cg_name(cgroup[0], "K"); + if (!cgroup[9]) + goto cleanup; + + for (i = 0; i < 10; i++) + if (cg_create(cgroup[i])) + goto cleanup; + + cg_run_nowait(cgroup[2], child_fn, NULL); + cg_run_nowait(cgroup[7], child_fn, NULL); + cg_run_nowait(cgroup[9], child_fn, NULL); + cg_run_nowait(cgroup[9], child_fn, NULL); + cg_run_nowait(cgroup[9], child_fn, NULL); + + /* + * Wait until all child processes will enter + * corresponding cgroups. + */ + + if (cg_wait_for_proc_count(cgroup[2], 1) || + cg_wait_for_proc_count(cgroup[7], 1) || + cg_wait_for_proc_count(cgroup[9], 3)) + goto cleanup; + + /* + * Freeze B. + */ + if (cg_freeze_wait(cgroup[1], true)) + goto cleanup; + + /* + * Freeze F. + */ + if (cg_freeze_wait(cgroup[5], true)) + goto cleanup; + + /* + * Freeze G. + */ + if (cg_freeze_wait(cgroup[6], true)) + goto cleanup; + + /* + * Check that A and E are not frozen. + */ + if (cg_check_frozen(cgroup[0], false)) + goto cleanup; + + if (cg_check_frozen(cgroup[4], false)) + goto cleanup; + + /* + * Freeze A. Check that A, B and E are frozen. + */ + if (cg_freeze_wait(cgroup[0], true)) + goto cleanup; + + if (cg_check_frozen(cgroup[1], true)) + goto cleanup; + + if (cg_check_frozen(cgroup[4], true)) + goto cleanup; + + /* + * Unfreeze B, F and G + */ + if (cg_freeze_nowait(cgroup[1], false)) + goto cleanup; + + if (cg_freeze_nowait(cgroup[5], false)) + goto cleanup; + + if (cg_freeze_nowait(cgroup[6], false)) + goto cleanup; + + /* + * Check that C and H are still frozen. + */ + if (cg_check_frozen(cgroup[2], true)) + goto cleanup; + + if (cg_check_frozen(cgroup[7], true)) + goto cleanup; + + /* + * Unfreeze A. Check that A, C and K are not frozen. + */ + if (cg_freeze_wait(cgroup[0], false)) + goto cleanup; + + if (cg_check_frozen(cgroup[2], false)) + goto cleanup; + + if (cg_check_frozen(cgroup[9], false)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + for (i = 9; i >= 0 && cgroup[i]; i--) { + cg_destroy(cgroup[i]); + free(cgroup[i]); + } + + return ret; +} + +/* + * A fork bomb emulator. + */ +static int forkbomb_fn(const char *cgroup, void *arg) +{ + int ppid; + + fork(); + fork(); + + ppid = getppid(); + + while (getppid() == ppid) + usleep(1000); + + return getppid() == ppid; +} + +/* + * The test runs a fork bomb in a cgroup and tries to freeze it. + * Then it kills all processes and checks that cgroup isn't populated + * anymore. + */ +static int test_cgfreezer_forkbomb(const char *root) +{ + int ret = KSFT_FAIL; + char *cgroup = NULL; + + cgroup = cg_name(root, "cg_forkbomb_test"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + cg_run_nowait(cgroup, forkbomb_fn, NULL); + + usleep(100000); + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + if (cg_killall(cgroup)) + goto cleanup; + + if (cg_wait_for_proc_count(cgroup, 0)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +/* + * The test creates two nested cgroups, freezes the parent + * and removes the child. Then it checks that the parent cgroup + * remains frozen and it's possible to create a new child + * without unfreezing. The new child is frozen too. + */ +static int test_cgfreezer_rmdir(const char *root) +{ + int ret = KSFT_FAIL; + char *parent, *child = NULL; + + parent = cg_name(root, "cg_test_rmdir_A"); + if (!parent) + goto cleanup; + + child = cg_name(parent, "cg_test_rmdir_B"); + if (!child) + goto cleanup; + + if (cg_create(parent)) + goto cleanup; + + if (cg_create(child)) + goto cleanup; + + if (cg_freeze_wait(parent, true)) + goto cleanup; + + if (cg_destroy(child)) + goto cleanup; + + if (cg_check_frozen(parent, true)) + goto cleanup; + + if (cg_create(child)) + goto cleanup; + + if (cg_check_frozen(child, true)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (child) + cg_destroy(child); + free(child); + if (parent) + cg_destroy(parent); + free(parent); + return ret; +} + +/* + * The test creates two cgroups: A and B, runs a process in A + * and performs several migrations: + * 1) A (running) -> B (frozen) + * 2) B (frozen) -> A (running) + * 3) A (frozen) -> B (frozen) + * + * On each step it checks the actual state of both cgroups. + */ +static int test_cgfreezer_migrate(const char *root) +{ + int ret = KSFT_FAIL; + char *cgroup[2] = {0}; + int pid; + + cgroup[0] = cg_name(root, "cg_test_migrate_A"); + if (!cgroup[0]) + goto cleanup; + + cgroup[1] = cg_name(root, "cg_test_migrate_B"); + if (!cgroup[1]) + goto cleanup; + + if (cg_create(cgroup[0])) + goto cleanup; + + if (cg_create(cgroup[1])) + goto cleanup; + + pid = cg_run_nowait(cgroup[0], child_fn, NULL); + if (pid < 0) + goto cleanup; + + if (cg_wait_for_proc_count(cgroup[0], 1)) + goto cleanup; + + /* + * Migrate from A (running) to B (frozen) + */ + if (cg_freeze_wait(cgroup[1], true)) + goto cleanup; + + if (cg_enter_and_wait_for_frozen(cgroup[1], pid, true)) + goto cleanup; + + if (cg_check_frozen(cgroup[0], false)) + goto cleanup; + + /* + * Migrate from B (frozen) to A (running) + */ + if (cg_enter_and_wait_for_frozen(cgroup[0], pid, false)) + goto cleanup; + + if (cg_check_frozen(cgroup[1], true)) + goto cleanup; + + /* + * Migrate from A (frozen) to B (frozen) + */ + if (cg_freeze_wait(cgroup[0], true)) + goto cleanup; + + if (cg_enter_and_wait_for_frozen(cgroup[1], pid, true)) + goto cleanup; + + if (cg_check_frozen(cgroup[0], true)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup[0]) + cg_destroy(cgroup[0]); + free(cgroup[0]); + if (cgroup[1]) + cg_destroy(cgroup[1]); + free(cgroup[1]); + return ret; +} + +/* + * The test checks that ptrace works with a tracing process in a frozen cgroup. + */ +static int test_cgfreezer_ptrace(const char *root) +{ + int ret = KSFT_FAIL; + char *cgroup = NULL; + siginfo_t siginfo; + int pid; + + cgroup = cg_name(root, "cg_test_ptrace"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + pid = cg_run_nowait(cgroup, child_fn, NULL); + if (pid < 0) + goto cleanup; + + if (cg_wait_for_proc_count(cgroup, 1)) + goto cleanup; + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + if (ptrace(PTRACE_SEIZE, pid, NULL, NULL)) + goto cleanup; + + if (ptrace(PTRACE_INTERRUPT, pid, NULL, NULL)) + goto cleanup; + + waitpid(pid, NULL, 0); + + /* + * Cgroup has to remain frozen, however the test task + * is in traced state. + */ + if (cg_check_frozen(cgroup, true)) + goto cleanup; + + if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &siginfo)) + goto cleanup; + + if (ptrace(PTRACE_DETACH, pid, NULL, NULL)) + goto cleanup; + + if (cg_check_frozen(cgroup, true)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +/* + * Check if the process is stopped. + */ +static int proc_check_stopped(int pid) +{ + char buf[PAGE_SIZE]; + int len; + + len = proc_read_text(pid, "stat", buf, sizeof(buf)); + if (len == -1) { + debug("Can't get %d stat\n", pid); + return -1; + } + + if (strstr(buf, "(test_freezer) T ") == NULL) { + debug("Process %d in the unexpected state: %s\n", pid, buf); + return -1; + } + + return 0; +} + +/* + * Test that it's possible to freeze a cgroup with a stopped process. + */ +static int test_cgfreezer_stopped(const char *root) +{ + int pid, ret = KSFT_FAIL; + char *cgroup = NULL; + + cgroup = cg_name(root, "cg_test_stopped"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + pid = cg_run_nowait(cgroup, child_fn, NULL); + + if (cg_wait_for_proc_count(cgroup, 1)) + goto cleanup; + + if (kill(pid, SIGSTOP)) + goto cleanup; + + if (cg_check_frozen(cgroup, false)) + goto cleanup; + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + if (cg_freeze_wait(cgroup, false)) + goto cleanup; + + if (proc_check_stopped(pid)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +/* + * Test that it's possible to freeze a cgroup with a ptraced process. + */ +static int test_cgfreezer_ptraced(const char *root) +{ + int pid, ret = KSFT_FAIL; + char *cgroup = NULL; + siginfo_t siginfo; + + cgroup = cg_name(root, "cg_test_ptraced"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + pid = cg_run_nowait(cgroup, child_fn, NULL); + + if (cg_wait_for_proc_count(cgroup, 1)) + goto cleanup; + + if (ptrace(PTRACE_SEIZE, pid, NULL, NULL)) + goto cleanup; + + if (ptrace(PTRACE_INTERRUPT, pid, NULL, NULL)) + goto cleanup; + + waitpid(pid, NULL, 0); + + if (cg_check_frozen(cgroup, false)) + goto cleanup; + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + /* + * cg_check_frozen(cgroup, true) will fail here, + * because the task in in the TRACEd state. + */ + if (cg_freeze_wait(cgroup, false)) + goto cleanup; + + if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &siginfo)) + goto cleanup; + + if (ptrace(PTRACE_DETACH, pid, NULL, NULL)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +static int vfork_fn(const char *cgroup, void *arg) +{ + int pid = vfork(); + + if (pid == 0) + while (true) + sleep(1); + + return pid; +} + +/* + * Test that it's possible to freeze a cgroup with a process, + * which called vfork() and is waiting for a child. + */ +static int test_cgfreezer_vfork(const char *root) +{ + int ret = KSFT_FAIL; + char *cgroup = NULL; + + cgroup = cg_name(root, "cg_test_vfork"); + if (!cgroup) + goto cleanup; + + if (cg_create(cgroup)) + goto cleanup; + + cg_run_nowait(cgroup, vfork_fn, NULL); + + if (cg_wait_for_proc_count(cgroup, 2)) + goto cleanup; + + if (cg_freeze_wait(cgroup, true)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (cgroup) + cg_destroy(cgroup); + free(cgroup); + return ret; +} + +#define T(x) { x, #x } +struct cgfreezer_test { + int (*fn)(const char *root); + const char *name; +} tests[] = { + T(test_cgfreezer_simple), + T(test_cgfreezer_tree), + T(test_cgfreezer_forkbomb), + T(test_cgfreezer_rmdir), + T(test_cgfreezer_migrate), + T(test_cgfreezer_ptrace), + T(test_cgfreezer_stopped), + T(test_cgfreezer_ptraced), + T(test_cgfreezer_vfork), +}; +#undef T + +int main(int argc, char *argv[]) +{ + char root[PATH_MAX]; + int i, ret = EXIT_SUCCESS; + + if (cg_find_unified_root(root, sizeof(root))) + ksft_exit_skip("cgroup v2 isn't mounted\n"); + for (i = 0; i < ARRAY_SIZE(tests); i++) { + switch (tests[i].fn(root)) { + case KSFT_PASS: + ksft_test_result_pass("%s\n", tests[i].name); + break; + case KSFT_SKIP: + ksft_test_result_skip("%s\n", tests[i].name); + break; + default: + ret = EXIT_FAILURE; + ksft_test_result_fail("%s\n", tests[i].name); + break; + } + } + + return ret; +} -- 2.20.1

5 years, 9 months

1
0
0 0

[PATCH v9 5/9] kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy()

by Roman Gushchin

If the cgroup destruction races with an exit() of a belonging process(es), cg_kill_all() may fail. It's not a good reason to make cg_destroy() fail and leave the cgroup in place, potentially causing next test runs to fail. Signed-off-by: Roman Gushchin <guro(a)fb.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: Tejun Heo <tj(a)kernel.org> Cc: kernel-team(a)fb.com Cc: linux-kselftest(a)vger.kernel.org --- tools/testing/selftests/cgroup/cgroup_util.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index 14c9fe284806..eba06f94433b 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -227,9 +227,7 @@ int cg_destroy(const char *cgroup) retry: ret = rmdir(cgroup); if (ret && errno == EBUSY) { - ret = cg_killall(cgroup); - if (ret) - return ret; + cg_killall(cgroup); usleep(100); goto retry; } -- 2.20.1

5 years, 9 months

1
0
0 0

[PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

by Joel Fernandes (Google)

Introduce in-kernel headers and other artifacts which are made available as an archive through proc (/proc/kheaders.tar.xz file). This archive makes it possible to build kernel modules, run eBPF programs, and other tracing programs that need to extend the kernel for tracing purposes without any dependency on the file system having headers and build artifacts. On Android and embedded systems, it is common to switch kernels but not have kernel headers available on the file system. Raw kernel headers also cannot be copied into the filesystem like they can be on other distros, due to licensing and other issues. There's no linux-headers package on Android. Further once a different kernel is booted, any headers stored on the file system will no longer be useful. By storing the headers as a compressed archive within the kernel, we can avoid these issues that have been a hindrance for a long time. The feature is also buildable as a module just in case the user desires it not being part of the kernel image. This makes it possible to load and unload the headers on demand. A tracing program, or a kernel module builder can load the module, do its operations, and then unload the module to save kernel memory. The total memory needed is 3.8MB. The code to read the headers is based on /proc/config.gz code and uses the same technique to embed the headers. To build a module, the below steps have been tested on an x86 machine: modprobe kheaders rm -rf $HOME/headers mkdir -p $HOME/headers tar -xvf /proc/kheaders.tar.xz -C $HOME/headers >/dev/null cd my-kernel-module make -C $HOME/headers M=$(pwd) modules rmmod kheaders Additional notes: (1) external modules must be built on the same arch as the host that built vmlinux. This can be done either in a qemu emulated chroot on the target, or natively. This is due to host arch dependency of kernel scripts. (2) A limitation of module building with this is, since Module.symvers is not available in the archive due to a cyclic dependency with building of the archive into the kernel or module binaries, the modules built using the archive will not contain symbol versioning (modversion). This is usually not an issue since the idea of this patch is to build a kernel module on the fly and load it into the same kernel. An appropriate warning is already printed by the kernel to alert the user of modules not having modversions when built using the archive. For building with modversions, the user can use traditional header packages. For our tracing usecases, we build modules on the fly with this so it is not a concern. (3) I have left IKHD_ST and IKHD_ED markers as is to facilitate future patches that would extract the headers from a kernel or module image. Signed-off-by: Joel Fernandes (Google) <joel(a)joelfernandes.org> --- Changes since v3: - Blank tar was being generated because of a one line I forgot to push. It is updated now. - Added module.lds since arm64 needs it to build modules. Changes since v2: (Thanks to Masahiro Yamada for several excellent suggestions) - Added support for out of tree builds. - Added incremental build support bringing down build time of incremental builds from 50 seconds to 5 seconds. - Fixed various small nits / cleanups. - clean ups to kheaders.c pointed by Alexey Dobriyan. - Fixed MODULE_LICENSE in test module and kheaders.c - Dropped Module.symvers from archive due to circular dependency. Changes since v1: - removed IKH_EXTRA variable, not needed (Masahiro Yamada) - small fix ups to selftest - added target to main Makefile etc - added MODULE_LICENSE to test module - made selftest more quiet Changes since RFC: Both changes bring size down to 3.8MB: - use xz for compression - strip comments except SPDX lines - Call out the module name in Kconfig - Also added selftests in second patch to ensure headers are always working. Other notes: By the way I still see this error (without the patch) when doing a clean build: Makefile:594: include/config/auto.conf: No such file or directory It appears to be because of commit 0a16d2e8cb7e ("kbuild: use 'include' directive to load auto.conf from top Makefile") Documentation/dontdiff | 1 + init/Kconfig | 11 ++++++ kernel/.gitignore | 3 ++ kernel/Makefile | 37 +++++++++++++++++++ kernel/kheaders.c | 72 ++++++++++++++++++++++++++++++++++++ scripts/gen_ikh_data.sh | 78 +++++++++++++++++++++++++++++++++++++++ scripts/strip-comments.pl | 8 ++++ 7 files changed, 210 insertions(+) create mode 100644 kernel/kheaders.c create mode 100755 scripts/gen_ikh_data.sh create mode 100755 scripts/strip-comments.pl diff --git a/Documentation/dontdiff b/Documentation/dontdiff index 2228fcc8e29f..05a2319ee2a2 100644 --- a/Documentation/dontdiff +++ b/Documentation/dontdiff @@ -151,6 +151,7 @@ int8.c kallsyms kconfig keywords.c +kheaders_data.h* ksym.c* ksym.h* kxgettext diff --git a/init/Kconfig b/init/Kconfig index c9386a365eea..63ff0990ae55 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -563,6 +563,17 @@ config IKCONFIG_PROC This option enables access to the kernel configuration file through /proc/config.gz. +config IKHEADERS_PROC + tristate "Enable kernel header artifacts through /proc/kheaders.tar.xz" + select BUILD_BIN2C + depends on PROC_FS + help + This option enables access to the kernel header and other artifacts that + are generated during the build process. These can be used to build kernel + modules, and other in-kernel programs such as those generated by eBPF + and systemtap tools. If you build the headers as a module, a module + called kheaders.ko is built which can be loaded to get access to them. + config LOG_BUF_SHIFT int "Kernel log buffer size (16 => 64KB, 17 => 128KB)" range 12 25 diff --git a/kernel/.gitignore b/kernel/.gitignore index b3097bde4e9c..484018945e93 100644 --- a/kernel/.gitignore +++ b/kernel/.gitignore @@ -3,5 +3,8 @@ # config_data.h config_data.gz +kheaders.md5 +kheaders_data.h +kheaders_data.tar.xz timeconst.h hz.bc diff --git a/kernel/Makefile b/kernel/Makefile index 6aa7543bcdb2..240685a6b638 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -70,6 +70,7 @@ obj-$(CONFIG_UTS_NS) += utsname.o obj-$(CONFIG_USER_NS) += user_namespace.o obj-$(CONFIG_PID_NS) += pid_namespace.o obj-$(CONFIG_IKCONFIG) += configs.o +obj-$(CONFIG_IKHEADERS_PROC) += kheaders.o obj-$(CONFIG_SMP) += stop_machine.o obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o obj-$(CONFIG_AUDIT) += audit.o auditfilter.o @@ -130,3 +131,39 @@ filechk_ikconfiggz = \ targets += config_data.h $(obj)/config_data.h: $(obj)/config_data.gz FORCE $(call filechk,ikconfiggz) + +# Build a list of in-kernel headers for building kernel modules +ikh_file_list := include/ +ikh_file_list += arch/$(SRCARCH)/Makefile +ikh_file_list += arch/$(SRCARCH)/include/ +ikh_file_list += arch/$(SRCARCH)/kernel/module.lds +ikh_file_list += scripts/ +ikh_file_list += Makefile + +# Things we need from the $objtree. "OBJDIR" is for the gen_ikh_data.sh +# script to identify that this comes from the $objtree directory +ikh_file_list += OBJDIR/scripts/ +ikh_file_list += OBJDIR/include/ +ikh_file_list += OBJDIR/arch/$(SRCARCH)/include/ +ifeq ($(CONFIG_STACK_VALIDATION), y) +ikh_file_list += OBJDIR/tools/objtool/objtool +endif + +$(obj)/kheaders.o: $(obj)/kheaders_data.h + +targets += kheaders_data.tar.xz + +quiet_cmd_genikh = GEN $(obj)/kheaders_data.tar.xz +cmd_genikh = $(srctree)/scripts/gen_ikh_data.sh $@ $(ikh_file_list) +$(obj)/kheaders_data.tar.xz: FORCE + $(call cmd,genikh) + +filechk_ikheadersxz = \ + echo "static const char kernel_headers_data[] __used = KH_MAGIC_START"; \ + cat $< | scripts/bin2c; \ + echo "KH_MAGIC_END;" + +targets += kheaders_data.h +targets += kheaders.md5 +$(obj)/kheaders_data.h: $(obj)/kheaders_data.tar.xz FORCE + $(call filechk,ikheadersxz) diff --git a/kernel/kheaders.c b/kernel/kheaders.c new file mode 100644 index 000000000000..46a6358301e5 --- /dev/null +++ b/kernel/kheaders.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * kernel/kheaders.c + * Provide headers and artifacts needed to build kernel modules. + * (Borrowed code from kernel/configs.c) + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/proc_fs.h> +#include <linux/init.h> +#include <linux/uaccess.h> + +/* + * Define kernel_headers_data and kernel_headers_data_size, which contains the + * compressed kernel headers. The file is first compressed with xz and then + * bounded by two eight byte magic numbers to allow extraction from a binary + * kernel image: + * + * IKHD_ST + * <image> + * IKHD_ED + */ +#define KH_MAGIC_START "IKHD_ST" +#define KH_MAGIC_END "IKHD_ED" +#include "kheaders_data.h" + + +#define KH_MAGIC_SIZE (sizeof(KH_MAGIC_START) - 1) +#define kernel_headers_data_size \ + (sizeof(kernel_headers_data) - 1 - KH_MAGIC_SIZE * 2) + +static ssize_t +ikheaders_read_current(struct file *file, char __user *buf, + size_t len, loff_t *offset) +{ + return simple_read_from_buffer(buf, len, offset, + kernel_headers_data + KH_MAGIC_SIZE, + kernel_headers_data_size); +} + +static const struct file_operations ikheaders_file_ops = { + .read = ikheaders_read_current, + .llseek = default_llseek, +}; + +static int __init ikheaders_init(void) +{ + struct proc_dir_entry *entry; + + /* create the current headers file */ + entry = proc_create("kheaders.tar.xz", S_IRUGO, NULL, + &ikheaders_file_ops); + if (!entry) + return -ENOMEM; + + proc_set_size(entry, kernel_headers_data_size); + + return 0; +} + +static void __exit ikheaders_cleanup(void) +{ + remove_proc_entry("kheaders.tar.xz", NULL); +} + +module_init(ikheaders_init); +module_exit(ikheaders_cleanup); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Joel Fernandes"); +MODULE_DESCRIPTION("Echo the kernel header artifacts used to build the kernel"); diff --git a/scripts/gen_ikh_data.sh b/scripts/gen_ikh_data.sh new file mode 100755 index 000000000000..1fa5628fcc30 --- /dev/null +++ b/scripts/gen_ikh_data.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +spath="$(dirname "$(readlink -f "$0")")" +kroot="$spath/.." +outdir="$(pwd)" +tarfile=$1 +cpio_dir=$outdir/$tarfile.tmp + +file_list=${@:2} + +src_file_list="" +for f in $file_list; do + src_file_list="$src_file_list $(echo $f | grep -v OBJDIR)" +done + +obj_file_list="" +for f in $file_list; do + f=$(echo $f | grep OBJDIR | sed -e 's/OBJDIR\///g') + obj_file_list="$obj_file_list $f"; +done + +# Support incremental builds by skipping archive generation +# if timestamps of files being archived are not changed. + +# This block is useful for debugging the incremental builds. +# Uncomment it for debugging. +# iter=1 +# if [ ! -f /tmp/iter ]; then echo 1 > /tmp/iter; +# else; iter=$(($(cat /tmp/iter) + 1)); fi +# find $src_file_list -type f | xargs ls -lR > /tmp/src-ls-$iter +# find $obj_file_list -type f | xargs ls -lR > /tmp/obj-ls-$iter + +# modules.order and include/generated/compile.h are ignored because these are +# touched even when none of the source files changed. This causes pointless +# regeneration, so let us ignore them for md5 calculation. +pushd $kroot > /dev/null +src_files_md5="$(find $src_file_list -type f ! -name modules.order | + grep -v "include/generated/compile.h" | + xargs ls -lR | md5sum | cut -d ' ' -f1)" +popd > /dev/null +obj_files_md5="$(find $obj_file_list -type f ! -name modules.order | + grep -v "include/generated/compile.h" | + xargs ls -lR | md5sum | cut -d ' ' -f1)" + +if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi +if [ -f kernel/kheaders.md5 ] && + [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] && + [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] && + [ "$(cat kernel/kheaders.md5|tail -1)" == "$tarfile_md5" ]; then + exit +fi + +rm -rf $cpio_dir +mkdir $cpio_dir + +pushd $kroot > /dev/null +for f in $src_file_list; + do find "$f" ! -name "*.c" ! -name "*.o" ! -name "*.cmd" ! -name ".*"; +done | cpio --quiet -pd $cpio_dir +popd > /dev/null + +# The second CPIO can complain if files already exist which can +# happen with out of tree builds. Just silence CPIO for now. +for f in $obj_file_list; + do find "$f" ! -name "*.c" ! -name "*.o" ! -name "*.cmd" ! -name ".*"; +done | cpio --quiet -pd $cpio_dir >/dev/null 2>&1 + +find $cpio_dir -type f -print0 | + xargs -0 -P8 -n1 -I {} sh -c "$spath/strip-comments.pl {}" + +tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null + +echo "$src_files_md5" > kernel/kheaders.md5 +echo "$obj_files_md5" >> kernel/kheaders.md5 +echo "$(md5sum $tarfile | cut -d ' ' -f1)" >> kernel/kheaders.md5 + +rm -rf $cpio_dir diff --git a/scripts/strip-comments.pl b/scripts/strip-comments.pl new file mode 100755 index 000000000000..f8ada87c5802 --- /dev/null +++ b/scripts/strip-comments.pl @@ -0,0 +1,8 @@ +#!/usr/bin/perl -pi +# SPDX-License-Identifier: GPL-2.0 + +# This script removes /**/ comments from a file, unless such comments +# contain "SPDX". It is used when building compressed in-kernel headers. + +BEGIN {undef $/;} +s/\/\*((?!SPDX).)*?\*\///smg; -- 2.21.0.352.gf09ad66450-goog

5 years, 9 months

16
45
0 0

[PATCH v4 0/8] selftests/kexec: add kexec tests

by Mimi Zohar

The kernel may be configured or an IMA policy specified on the boot command line requiring the kexec kernel image signature to be verified. At runtime a custom IMA policy may be loaded, replacing the policy specified on the boot command line. In addition, the arch specific policy rules are dynamically defined based on the secure boot mode that may require the kernel image signature to be verified. The kernel image may have a PE signature, an IMA signature, or both. In addition, there are two kexec syscalls - kexec_load and kexec_file_load - but only the kexec_file_load syscall can verify signatures. These kexec selftests verify that only properly signed kernel images are loaded as required, based on the kernel config, the secure boot mode, and the IMA runtime policy. Loading a kernel image or kernel module requires root privileges. To run just the KEXEC selftests: sudo make TARGETS=kexec kselftest Changelog v4: - Moved the kexec tests to selftests/kexec, as requested by Dave Young. - Removed the kernel module selftest from this patch set. - Rewritten cover letter, removing reference to kernel modules. Changelog v3: - Updated tests based on Petr's review, including the defining a common test to check for root privileges. - Modified config, removing the CONFIG_KEXEC_VERIFY_SIG requirement. - Updated the SPDX license to GPL-2.0 based on Shuah's review. - Updated the secureboot mode test to check the SetupMode as well, based on David Young's review. Mimi Zohar (7): selftests/kexec: move the IMA kexec_load selftest to selftests/kexec selftests/kexec: cleanup the kexec selftest selftests/kexec: define a set of common functions selftests/kexec: define common logging functions kselftest/kexec: define "require_root_privileges" selftests/kexec: kexec_file_load syscall test selftests/kexec: check kexec_load and kexec_file_load are enabled Petr Vorel (1): selftests/kexec: Add missing '=y' to config options tools/testing/selftests/Makefile | 2 +- tools/testing/selftests/ima/Makefile | 11 -- tools/testing/selftests/ima/config | 4 - tools/testing/selftests/ima/test_kexec_load.sh | 54 ------ tools/testing/selftests/kexec/Makefile | 12 ++ tools/testing/selftests/kexec/config | 3 + tools/testing/selftests/kexec/kexec_common_lib.sh | 175 ++++++++++++++++++ .../selftests/kexec/test_kexec_file_load.sh | 195 +++++++++++++++++++++ tools/testing/selftests/kexec/test_kexec_load.sh | 39 +++++ 9 files changed, 425 insertions(+), 70 deletions(-) delete mode 100644 tools/testing/selftests/ima/Makefile delete mode 100644 tools/testing/selftests/ima/config delete mode 100755 tools/testing/selftests/ima/test_kexec_load.sh create mode 100644 tools/testing/selftests/kexec/Makefile create mode 100644 tools/testing/selftests/kexec/config create mode 100755 tools/testing/selftests/kexec/kexec_common_lib.sh create mode 100755 tools/testing/selftests/kexec/test_kexec_file_load.sh create mode 100755 tools/testing/selftests/kexec/test_kexec_load.sh -- 2.7.5

5 years, 9 months

2
9
0 0

[PATCH v2] selftests/x86: Support Atom for syscall_arg_fault test

by Tong Bo

Atom-based CPUs trigger stack fault when invoke 32-bit SYSENTER instruction with invalid register values. So we also need SIGBUS handling in this case. Following is assembly when the fault exception happens. (gdb) disassemble $eip Dump of assembler code for function __kernel_vsyscall: 0xf7fd8fe0 <+0>: push %ecx 0xf7fd8fe1 <+1>: push %edx 0xf7fd8fe2 <+2>: push %ebp 0xf7fd8fe3 <+3>: mov %esp,%ebp 0xf7fd8fe5 <+5>: sysenter 0xf7fd8fe7 <+7>: int $0x80 => 0xf7fd8fe9 <+9>: pop %ebp 0xf7fd8fea <+10>: pop %edx 0xf7fd8feb <+11>: pop %ecx 0xf7fd8fec <+12>: ret End of assembler dump. According to Intel SDM, this could also be a Stack Segment Fault(#SS, 12), except a normal Page Fault(#PF, 14). Especially, in section 6.9 of Vol.3A, both stack and page faults are within the 10th(lowest priority) class, and as it said, "exceptions within each class are implementation-dependent and may vary from processor to processor". It's expected for processors like Intel Atom to trigger stack fault(SIGBUS), while we get page fault(SIGSEGV) from common Core processors. Signed-off-by: Tong Bo <bo.tong(a)intel.com> --- tools/testing/selftests/x86/syscall_arg_fault.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/x86/syscall_arg_fault.c b/tools/testing/selftests/x86/syscall_arg_fault.c index 7db4fc9..810180a 100644 --- a/tools/testing/selftests/x86/syscall_arg_fault.c +++ b/tools/testing/selftests/x86/syscall_arg_fault.c @@ -43,7 +43,7 @@ static sigjmp_buf jmpbuf; static volatile sig_atomic_t n_errs; -static void sigsegv(int sig, siginfo_t *info, void *ctx_void) +static void sigsegv_or_sigbus(int sig, siginfo_t *info, void *ctx_void) { ucontext_t *ctx = (ucontext_t*)ctx_void; @@ -73,7 +73,12 @@ int main() if (sigaltstack(&stack, NULL) != 0) err(1, "sigaltstack"); - sethandler(SIGSEGV, sigsegv, SA_ONSTACK); + sethandler(SIGSEGV, sigsegv_or_sigbus, SA_ONSTACK); + /* The actual exception can vary. On Atom CPUs, we get #SS + * instead of #PF when the vDSO fails to access the stack when + * ESP is too close to 2^32, and #SS causes SIGBUS. + */ + sethandler(SIGBUS, sigsegv_or_sigbus, SA_ONSTACK); sethandler(SIGILL, sigill, SA_ONSTACK); /* -- 2.7.4

5 years, 9 months

2
1
0 0

[RFC PATCH 2/2] rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs

by Mathieu Desnoyers

The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers. Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs. Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers(a)efficios.com> CC: Thomas Gleixner <tglx(a)linutronix.de> CC: Joel Fernandes <joelaf(a)google.com> CC: Peter Zijlstra <peterz(a)infradead.org> CC: Catalin Marinas <catalin.marinas(a)arm.com> CC: Dave Watson <davejwatson(a)fb.com> CC: Will Deacon <will.deacon(a)arm.com> CC: Shuah Khan <shuah(a)kernel.org> CC: Andi Kleen <andi(a)firstfloor.org> CC: linux-kselftest(a)vger.kernel.org CC: "H . Peter Anvin" <hpa(a)zytor.com> CC: Chris Lameter <cl(a)linux.com> CC: Russell King <linux(a)arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages(a)gmail.com> CC: "Paul E . McKenney" <paulmck(a)linux.vnet.ibm.com> CC: Paul Turner <pjt(a)google.com> CC: Boqun Feng <boqun.feng(a)gmail.com> CC: Josh Triplett <josh(a)joshtriplett.org> CC: Steven Rostedt <rostedt(a)goodmis.org> CC: Ben Maurer <bmaurer(a)fb.com> CC: linux-api(a)vger.kernel.org CC: Andy Lutomirski <luto(a)amacapital.net> CC: Andrew Morton <akpm(a)linux-foundation.org> CC: Linus Torvalds <torvalds(a)linux-foundation.org> --- tools/testing/selftests/rseq/rseq-arm.h | 32 +++++++++++++++++-------------- tools/testing/selftests/rseq/rseq-arm64.h | 9 ++++++--- tools/testing/selftests/rseq/rseq-mips.h | 32 +++++++++++++++++-------------- tools/testing/selftests/rseq/rseq-ppc.h | 22 +++++++++++++-------- tools/testing/selftests/rseq/rseq-s390.h | 18 +++++++++++------ tools/testing/selftests/rseq/rseq-x86.h | 19 ++++++++++++------ 6 files changed, 81 insertions(+), 51 deletions(-) diff --git a/tools/testing/selftests/rseq/rseq-arm.h b/tools/testing/selftests/rseq/rseq-arm.h index 17e8d231943a..5f262c54364f 100644 --- a/tools/testing/selftests/rseq/rseq-arm.h +++ b/tools/testing/selftests/rseq/rseq-arm.h @@ -30,24 +30,28 @@ do { \ #include "rseq-skip.h" #else /* !RSEQ_SKIP_FASTPATH */ -#define __RSEQ_ASM_DEFINE_TABLE(version, flags, start_ip, \ +#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip, \ post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ + __rseq_str(label) ":\n\t" \ ".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".word " __rseq_str(label) "b, 0x0\n\t" \ ".popsection\n\t" -#define RSEQ_ASM_DEFINE_TABLE(start_ip, post_commit_ip, abort_ip) \ - __RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip, \ +#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \ + __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) /* * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ @@ -99,7 +103,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -166,7 +170,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -237,7 +241,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) #endif @@ -292,7 +296,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -367,7 +371,7 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -443,7 +447,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -527,7 +531,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -651,7 +655,7 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) diff --git a/tools/testing/selftests/rseq/rseq-arm64.h b/tools/testing/selftests/rseq/rseq-arm64.h index 2079f71e0ca2..b41a2a48e965 100644 --- a/tools/testing/selftests/rseq/rseq-arm64.h +++ b/tools/testing/selftests/rseq/rseq-arm64.h @@ -82,13 +82,16 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip, \ post_commit_offset, abort_ip) \ - " .pushsection __rseq_table, \"aw\"\n" \ + " .pushsection __rseq_cs, \"aw\"\n" \ " .balign 32\n" \ __rseq_str(label) ":\n" \ " .long " __rseq_str(version) ", " __rseq_str(flags) "\n" \ " .quad " __rseq_str(start_ip) ", " \ __rseq_str(post_commit_offset) ", " \ __rseq_str(abort_ip) "\n" \ + " .popsection\n\t" \ + " .pushsection __rseq_cs_ptr_array, \"aw\"\n" \ + " .quad " __rseq_str(label) "b\n" \ " .popsection\n" #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \ @@ -99,8 +102,8 @@ do { \ * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ diff --git a/tools/testing/selftests/rseq/rseq-mips.h b/tools/testing/selftests/rseq/rseq-mips.h index 25d10ff54769..fe3eabcdcbe5 100644 --- a/tools/testing/selftests/rseq/rseq-mips.h +++ b/tools/testing/selftests/rseq/rseq-mips.h @@ -54,26 +54,30 @@ do { \ # error unsupported _MIPS_SZLONG #endif -#define __RSEQ_ASM_DEFINE_TABLE(version, flags, start_ip, \ +#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip, \ post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ + __rseq_str(label) ":\n\t" \ ".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ LONG " " U32_U64_PAD(__rseq_str(start_ip)) "\n\t" \ LONG " " U32_U64_PAD(__rseq_str(post_commit_offset)) "\n\t" \ LONG " " U32_U64_PAD(__rseq_str(abort_ip)) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + LONG " " U32_U64_PAD(__rseq_str(label) "b") "\n\t" \ ".popsection\n\t" -#define RSEQ_ASM_DEFINE_TABLE(start_ip, post_commit_ip, abort_ip) \ - __RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip, \ +#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \ + __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) /* * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ @@ -127,7 +131,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -192,7 +196,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -261,7 +265,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) #endif @@ -316,7 +320,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -389,7 +393,7 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -463,7 +467,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -543,7 +547,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) @@ -664,7 +668,7 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( - RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */ RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) #ifdef RSEQ_COMPARE_TWICE RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) diff --git a/tools/testing/selftests/rseq/rseq-ppc.h b/tools/testing/selftests/rseq/rseq-ppc.h index 24f95649d71e..9df18487fa9f 100644 --- a/tools/testing/selftests/rseq/rseq-ppc.h +++ b/tools/testing/selftests/rseq/rseq-ppc.h @@ -33,8 +33,8 @@ do { \ #else /* !RSEQ_SKIP_FASTPATH */ /* - * The __rseq_table section can be used by debuggers to better handle - * single-stepping through the restartable critical sections. + * The __rseq_cs_ptr_array and __rseq_cs sections can be used by debuggers to + * better handle single-stepping through the restartable critical sections. */ #ifdef __PPC64__ @@ -46,11 +46,14 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".quad " __rseq_str(label) "b\n\t" \ ".popsection\n\t" #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ @@ -67,8 +70,8 @@ do { \ * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ @@ -85,20 +88,23 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ /* 32-bit only supported on BE */ \ ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".long 0x0, " __rseq_str(label) "b\n\t" \ ".popsection\n\t" /* * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ diff --git a/tools/testing/selftests/rseq/rseq-s390.h b/tools/testing/selftests/rseq/rseq-s390.h index b8b5b6f900af..fbb97815d71c 100644 --- a/tools/testing/selftests/rseq/rseq-s390.h +++ b/tools/testing/selftests/rseq/rseq-s390.h @@ -37,19 +37,22 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".quad " __rseq_str(label) "b\n\t" \ ".popsection\n\t" /* * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ @@ -61,19 +64,22 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".long 0x0, " __rseq_str(label) "b\n\t" \ ".popsection\n\t" /* * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ diff --git a/tools/testing/selftests/rseq/rseq-x86.h b/tools/testing/selftests/rseq/rseq-x86.h index 5857570d1c51..2d4887b5d3f0 100644 --- a/tools/testing/selftests/rseq/rseq-x86.h +++ b/tools/testing/selftests/rseq/rseq-x86.h @@ -37,13 +37,17 @@ do { \ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".quad " __rseq_str(label) "b\n\t" \ ".popsection\n\t" + #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \ __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) @@ -52,8 +56,8 @@ do { \ * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ @@ -562,11 +566,14 @@ do { \ */ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ start_ip, post_commit_offset, abort_ip) \ - ".pushsection __rseq_table, \"aw\"\n\t" \ + ".pushsection __rseq_cs, \"aw\"\n\t" \ ".balign 32\n\t" \ __rseq_str(label) ":\n\t" \ ".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \ ".long " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \ + ".long " __rseq_str(label) "b, 0x0\n\t" \ ".popsection\n\t" #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \ @@ -577,8 +584,8 @@ do { \ * Exit points of a rseq critical section consist of all instructions outside * of the critical section where a critical section can either branch to or * reach through the normal course of its execution. The abort IP and the - * post-commit IP are already part of the __rseq_table section and should not - * be explicitly defined as additional exit points. Knowing all exit points is + * post-commit IP are already part of the __rseq_cs section and should not be + * explicitly defined as additional exit points. Knowing all exit points is * useful to assist debuggers stepping over the critical section. */ #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ -- 2.11.0

5 years, 9 months

1
0
0 0

[RFC PATCH 1/2] rseq/selftests: Add __rseq_exit_point_array section for debuggers

by Mathieu Desnoyers

Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers(a)efficios.com> CC: Thomas Gleixner <tglx(a)linutronix.de> CC: Joel Fernandes <joelaf(a)google.com> CC: Peter Zijlstra <peterz(a)infradead.org> CC: Catalin Marinas <catalin.marinas(a)arm.com> CC: Dave Watson <davejwatson(a)fb.com> CC: Will Deacon <will.deacon(a)arm.com> CC: Shuah Khan <shuah(a)kernel.org> CC: Andi Kleen <andi(a)firstfloor.org> CC: linux-kselftest(a)vger.kernel.org CC: "H . Peter Anvin" <hpa(a)zytor.com> CC: Chris Lameter <cl(a)linux.com> CC: Russell King <linux(a)arm.linux.org.uk> CC: Michael Kerrisk <mtk.manpages(a)gmail.com> CC: "Paul E . McKenney" <paulmck(a)linux.vnet.ibm.com> CC: Paul Turner <pjt(a)google.com> CC: Boqun Feng <boqun.feng(a)gmail.com> CC: Josh Triplett <josh(a)joshtriplett.org> CC: Steven Rostedt <rostedt(a)goodmis.org> CC: Ben Maurer <bmaurer(a)fb.com> CC: linux-api(a)vger.kernel.org CC: Andy Lutomirski <luto(a)amacapital.net> CC: Andrew Morton <akpm(a)linux-foundation.org> CC: Linus Torvalds <torvalds(a)linux-foundation.org> --- tools/testing/selftests/rseq/rseq-arm.h | 52 +++++++++++++++++ tools/testing/selftests/rseq/rseq-arm64.h | 52 +++++++++++++++++ tools/testing/selftests/rseq/rseq-mips.h | 53 +++++++++++++++++ tools/testing/selftests/rseq/rseq-ppc.h | 66 ++++++++++++++++++++++ tools/testing/selftests/rseq/rseq-s390.h | 55 ++++++++++++++++++ tools/testing/selftests/rseq/rseq-x86.h | 94 +++++++++++++++++++++++++++++++ 6 files changed, 372 insertions(+) diff --git a/tools/testing/selftests/rseq/rseq-arm.h b/tools/testing/selftests/rseq/rseq-arm.h index 3cea19877227..17e8d231943a 100644 --- a/tools/testing/selftests/rseq/rseq-arm.h +++ b/tools/testing/selftests/rseq/rseq-arm.h @@ -42,6 +42,19 @@ do { \ __RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) ", 0x0\n\t" \ + ".popsection\n\t" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ "adr r0, " __rseq_str(cs_label) "\n\t" \ @@ -87,6 +100,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -149,6 +167,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -215,6 +238,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -267,6 +293,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -337,6 +368,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -408,6 +444,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -486,6 +528,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif "str %[src], %[rseq_scratch0]\n\t" "str %[dst], %[rseq_scratch1]\n\t" "str %[len], %[rseq_scratch2]\n\t" @@ -605,6 +652,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif "str %[src], %[rseq_scratch0]\n\t" "str %[dst], %[rseq_scratch1]\n\t" "str %[len], %[rseq_scratch2]\n\t" diff --git a/tools/testing/selftests/rseq/rseq-arm64.h b/tools/testing/selftests/rseq/rseq-arm64.h index 954f34671ca6..2079f71e0ca2 100644 --- a/tools/testing/selftests/rseq/rseq-arm64.h +++ b/tools/testing/selftests/rseq/rseq-arm64.h @@ -95,6 +95,19 @@ do { \ __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + " .pushsection __rseq_exit_point_array, \"aw\"\n" \ + " .quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n" \ + " .popsection\n" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ " adrp " RSEQ_ASM_TMP_REG ", " __rseq_str(cs_label) "\n" \ @@ -182,6 +195,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -231,6 +249,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -282,6 +305,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -325,6 +351,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -379,6 +410,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -433,6 +469,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error3]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -490,6 +532,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) @@ -545,6 +592,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) RSEQ_INJECT_ASM(3) diff --git a/tools/testing/selftests/rseq/rseq-mips.h b/tools/testing/selftests/rseq/rseq-mips.h index 7f48ecf46994..25d10ff54769 100644 --- a/tools/testing/selftests/rseq/rseq-mips.h +++ b/tools/testing/selftests/rseq/rseq-mips.h @@ -68,6 +68,20 @@ do { \ __RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + LONG " " U32_U64_PAD(__rseq_str(start_ip)) "\n\t" \ + LONG " " U32_U64_PAD(__rseq_str(exit_ip)) "\n\t" \ + ".popsection\n\t" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ LONG_LA " $4, " __rseq_str(cs_label) "\n\t" \ @@ -114,6 +128,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -174,6 +193,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -238,6 +262,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -290,6 +317,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -358,6 +390,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -427,6 +464,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -501,6 +544,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif LONG_S " %[src], %[rseq_scratch0]\n\t" LONG_S " %[dst], %[rseq_scratch1]\n\t" LONG_S " %[len], %[rseq_scratch2]\n\t" @@ -617,6 +665,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, rseq_workaround_gcc_asm_size_guess(); __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif LONG_S " %[src], %[rseq_scratch0]\n\t" LONG_S " %[dst], %[rseq_scratch1]\n\t" LONG_S " %[len], %[rseq_scratch2]\n\t" diff --git a/tools/testing/selftests/rseq/rseq-ppc.h b/tools/testing/selftests/rseq/rseq-ppc.h index 52630c9f42be..24f95649d71e 100644 --- a/tools/testing/selftests/rseq/rseq-ppc.h +++ b/tools/testing/selftests/rseq/rseq-ppc.h @@ -63,6 +63,19 @@ do { \ "std %%r17, %[" __rseq_str(rseq_cs) "]\n\t" \ __rseq_str(label) ":\n\t" +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n\t" \ + ".popsection\n\t" + #else /* #ifdef __PPC64__ */ #define STORE_WORD "stw " @@ -80,6 +93,20 @@ do { \ ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \ ".popsection\n\t" +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + /* 32-bit only supported on BE */ \ + ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) "\n\t" \ + ".popsection\n\t" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ "lis %%r17, (" __rseq_str(cs_label) ")@ha\n\t" \ @@ -169,6 +196,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -224,6 +256,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -286,6 +323,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -337,6 +377,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -400,6 +445,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -465,6 +515,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) /* cmp cpuid */ @@ -532,6 +588,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* setup for mempcy */ "mr %%r19, %[len]\n\t" "mr %%r20, %[src]\n\t" @@ -601,6 +662,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* setup for mempcy */ "mr %%r19, %[len]\n\t" "mr %%r20, %[src]\n\t" diff --git a/tools/testing/selftests/rseq/rseq-s390.h b/tools/testing/selftests/rseq/rseq-s390.h index 1069e85258ce..b8b5b6f900af 100644 --- a/tools/testing/selftests/rseq/rseq-s390.h +++ b/tools/testing/selftests/rseq/rseq-s390.h @@ -44,6 +44,19 @@ do { \ ".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \ ".popsection\n\t" +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n\t" \ + ".popsection\n\t" + #elif __s390__ #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, \ @@ -55,6 +68,19 @@ do { \ ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \ ".popsection\n\t" +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) "\n\t" \ + ".popsection\n\t" + #define LONG_L "l" #define LONG_S "st" #define LONG_LT_R "ltr" @@ -102,6 +128,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -160,6 +191,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -220,6 +256,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -268,6 +307,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -339,6 +383,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -407,6 +457,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif LONG_S " %[src], %[rseq_scratch0]\n\t" LONG_S " %[dst], %[rseq_scratch1]\n\t" LONG_S " %[len], %[rseq_scratch2]\n\t" diff --git a/tools/testing/selftests/rseq/rseq-x86.h b/tools/testing/selftests/rseq/rseq-x86.h index 089410a314e9..5857570d1c51 100644 --- a/tools/testing/selftests/rseq/rseq-x86.h +++ b/tools/testing/selftests/rseq/rseq-x86.h @@ -48,6 +48,19 @@ do { \ __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n\t" \ + ".popsection\n\t" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ "leaq " __rseq_str(cs_label) "(%%rip), %%rax\n\t" \ @@ -83,6 +96,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -140,6 +158,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -199,6 +222,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -244,6 +270,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -314,6 +345,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -381,6 +418,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif "movq %[src], %[rseq_scratch0]\n\t" "movq %[dst], %[rseq_scratch1]\n\t" "movq %[len], %[rseq_scratch2]\n\t" @@ -531,6 +573,19 @@ do { \ __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ (post_commit_ip - start_ip), abort_ip) +/* + * Exit points of a rseq critical section consist of all instructions outside + * of the critical section where a critical section can either branch to or + * reach through the normal course of its execution. The abort IP and the + * post-commit IP are already part of the __rseq_table section and should not + * be explicitly defined as additional exit points. Knowing all exit points is + * useful to assist debuggers stepping over the critical section. + */ +#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n\t" \ + ".long " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) ", 0x0\n\t" \ + ".popsection\n\t" + #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs) \ RSEQ_INJECT_ASM(1) \ "movl $" __rseq_str(cs_label) ", %[rseq_cs]\n\t" \ @@ -565,6 +620,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -622,6 +682,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -681,6 +746,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu) __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -726,6 +794,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -788,6 +861,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -853,6 +931,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3]) +#endif /* Start rseq by storing table entry pointer into rseq_cs. */ RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs) RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) @@ -922,6 +1006,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif "movl %[src], %[rseq_scratch0]\n\t" "movl %[dst], %[rseq_scratch1]\n\t" "movl %[len], %[rseq_scratch2]\n\t" @@ -1030,6 +1119,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect, __asm__ __volatile__ goto ( RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */ + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2]) +#endif "movl %[src], %[rseq_scratch0]\n\t" "movl %[dst], %[rseq_scratch1]\n\t" "movl %[len], %[rseq_scratch2]\n\t" -- 2.11.0

5 years, 9 months

1
0
0 0