January 2020 - Linux-kselftest-mirror

[PATCH bpf-next v2 1/4] selftests: bpf: use a temporary file in test_sockmap

by Lorenz Bauer

Use a proper temporary file for sendpage tests. This means that running the tests doesn't clutter the working directory, and allows running the test on read-only filesystems. Signed-off-by: Lorenz Bauer <lmb(a)cloudflare.com> Fixes: 16962b2404ac ("bpf: sockmap, add selftests") --- tools/testing/selftests/bpf/test_sockmap.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/bpf/test_sockmap.c b/tools/testing/selftests/bpf/test_sockmap.c index 4a851513c842..779e11da979c 100644 --- a/tools/testing/selftests/bpf/test_sockmap.c +++ b/tools/testing/selftests/bpf/test_sockmap.c @@ -331,7 +331,7 @@ static int msg_loop_sendpage(int fd, int iov_length, int cnt, FILE *file; int i, fp; - file = fopen(".sendpage_tst.tmp", "w+"); + file = tmpfile(); if (!file) { perror("create file for sendpage"); return 1; @@ -340,13 +340,8 @@ static int msg_loop_sendpage(int fd, int iov_length, int cnt, fwrite(&k, sizeof(char), 1, file); fflush(file); fseek(file, 0, SEEK_SET); - fclose(file); - fp = open(".sendpage_tst.tmp", O_RDONLY); - if (fp < 0) { - perror("reopen file for sendpage"); - return 1; - } + fp = fileno(file); clock_gettime(CLOCK_MONOTONIC, &s->start); for (i = 0; i < cnt; i++) { @@ -354,11 +349,11 @@ static int msg_loop_sendpage(int fd, int iov_length, int cnt, if (!drop && sent < 0) { perror("send loop error"); - close(fp); + fclose(file); return sent; } else if (drop && sent >= 0) { printf("sendpage loop error expected: %i\n", sent); - close(fp); + fclose(file); return -EIO; } @@ -366,7 +361,7 @@ static int msg_loop_sendpage(int fd, int iov_length, int cnt, s->bytes_sent += sent; } clock_gettime(CLOCK_MONOTONIC, &s->end); - close(fp); + fclose(file); return 0; } -- 2.20.1

4 years, 3 months

1
0
0 0

[PATCH bpf 4/4] selftests: bpf: reset global state between reuseport test runs

by Lorenz Bauer

Currently, there is a lot of false positives if a single reuseport test fails. This is because expected_results and the result map are not cleared. Zero both after individual test runs, which fixes the mentioned false positives. Signed-off-by: Lorenz Bauer <lmb(a)cloudflare.com> --- .../selftests/bpf/prog_tests/select_reuseport.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c index 09a536af139a..0bab8b1ca1c3 100644 --- a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c +++ b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c @@ -699,7 +699,19 @@ static void setup_per_test(int type, sa_family_t family, bool inany, static void cleanup_per_test(bool no_inner_map) { - int i, err; + int i, err, zero = 0; + + memset(expected_results, 0, sizeof(expected_results)); + + for (i = 0; i < NR_RESULTS; i++) { + err = bpf_map_update_elem(result_map, &i, &zero, BPF_ANY); + RET_IF(err, "reset elem in result_map", + "i:%u err:%d errno:%d\n", i, err, errno); + } + + err = bpf_map_update_elem(linum_map, &zero, &zero, BPF_ANY); + RET_IF(err, "reset line number in linum_map", "err:%d errno:%d\n", + err, errno); for (i = 0; i < REUSEPORT_ARRAY_SIZE; i++) close(sk_fds[i]); -- 2.20.1

4 years, 3 months

2
1
0 0

[PATCH kunit 0/3] kunit: add debugfs representation to show results/run tests

by Alan Maguire

When kunit tests are run on native (i.e. non-UML) environments, the results of test execution are often intermixed with dmesg output. This patch series attempts to solve this by providing a debugfs representation of the results of the last test run, available as /sys/kernel/debug/kunit/<testsuite>/results In addition, we provide a way to re-run the tests and show results via /sys/kernel/debug/kunit/<testsuite>/run Alan Maguire (3): kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display kunit: add "run" debugfs file to run suites, display results kunit: update documentation to describe debugfs representation Documentation/dev-tools/kunit/usage.rst | 20 +++++ include/kunit/test.h | 21 +++-- lib/kunit/Makefile | 3 +- lib/kunit/debugfs.c | 145 ++++++++++++++++++++++++++++++++ lib/kunit/debugfs.h | 11 +++ lib/kunit/test.c | 88 ++++++++++++++----- 6 files changed, 260 insertions(+), 28 deletions(-) create mode 100644 lib/kunit/debugfs.c create mode 100644 lib/kunit/debugfs.h -- 1.8.3.1

4 years, 3 months

3
5
0 0

[PATCH v9 1/8] hugetlb_cgroup: Add hugetlb_cgroup reservation counter

by Mina Almasry

These counters will track hugetlb reservations rather than hugetlb memory faulted in. This patch only adds the counter, following patches add the charging and uncharging of the counter. This is patch 1 of an 8 patch series. Problem: Currently tasks attempting to allocate more hugetlb memory than is available get a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1]. However, if a task attempts to allocate hugetlb memory only more than its hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call, but will SIGBUS the task when it attempts to fault the memory in. We have developers interested in using hugetlb_cgroups, and they have expressed dissatisfaction regarding this behavior. We'd like to improve this behavior such that tasks violating the hugetlb_cgroup limits get an error on mmap/shmget time, rather than getting SIGBUS'd when they try to fault the excess memory in. The underlying problem is that today's hugetlb_cgroup accounting happens at hugetlb memory *fault* time, rather than at *reservation* time. Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and the offending task gets SIGBUS'd. Proposed Solution: A new page counter named hugetlb.xMB.reservation_[limit|usage]_in_bytes. This counter has slightly different semantics than hugetlb.xMB.[limit|usage]_in_bytes: - While usage_in_bytes tracks all *faulted* hugetlb memory, reservation_usage_in_bytes tracks all *reserved* hugetlb memory and hugetlb memory faulted in without a prior reservation. - If a task attempts to reserve more memory than limit_in_bytes allows, the kernel will allow it to do so. But if a task attempts to reserve more memory than reservation_limit_in_bytes, the kernel will fail this reservation. This proposal is implemented in this patch series, with tests to verify functionality and show the usage. We also added cgroup-v2 support to hugetlb_cgroup so that the new use cases can be extended to v2. Alternatives considered: 1. A new cgroup, instead of only a new page_counter attached to the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code duplication with hugetlb_cgroup. Keeping hugetlb related page counters under hugetlb_cgroup seemed cleaner as well. 2. Instead of adding a new counter, we considered adding a sysctl that modifies the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at reservation time rather than fault time. Adding a new page_counter seems better as userspace could, if it wants, choose to enforce different cgroups differently: one via limit_in_bytes, and another via reservation_limit_in_bytes. This could be very useful if you're transitioning how hugetlb memory is partitioned on your system one cgroup at a time, for example. Also, someone may find usage for both limit_in_bytes and reservation_limit_in_bytes concurrently, and this approach gives them the option to do so. Testing: - Added tests passing. - Used libhugetlbfs for regression testing. [1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html Signed-off-by: Mina Almasry <almasrymina(a)google.com> Acked-by: Hillf Danton <hdanton(a)sina.com> --- include/linux/hugetlb.h | 4 +- mm/hugetlb_cgroup.c | 116 +++++++++++++++++++++++++++++++++++----- 2 files changed, 106 insertions(+), 14 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1e897e4168ac1..dea6143aa0685 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -432,8 +432,8 @@ struct hstate { unsigned int surplus_huge_pages_node[MAX_NUMNODES]; #ifdef CONFIG_CGROUP_HUGETLB /* cgroup control files */ - struct cftype cgroup_files_dfl[5]; - struct cftype cgroup_files_legacy[5]; + struct cftype cgroup_files_dfl[7]; + struct cftype cgroup_files_legacy[9]; #endif char name[HSTATE_NAME_LEN]; }; diff --git a/mm/hugetlb_cgroup.c b/mm/hugetlb_cgroup.c index e434b05416c68..35415af9ed26f 100644 --- a/mm/hugetlb_cgroup.c +++ b/mm/hugetlb_cgroup.c @@ -36,6 +36,11 @@ struct hugetlb_cgroup { */ struct page_counter hugepage[HUGE_MAX_HSTATE]; + /* + * the counter to account for hugepage reservations from hugetlb. + */ + struct page_counter reserved_hugepage[HUGE_MAX_HSTATE]; + atomic_long_t events[HUGE_MAX_HSTATE][HUGETLB_NR_MEMORY_EVENTS]; atomic_long_t events_local[HUGE_MAX_HSTATE][HUGETLB_NR_MEMORY_EVENTS]; @@ -55,6 +60,14 @@ struct hugetlb_cgroup { static struct hugetlb_cgroup *root_h_cgroup __read_mostly; +static inline struct page_counter * +hugetlb_cgroup_get_counter(struct hugetlb_cgroup *h_cg, int idx, bool reserved) +{ + if (reserved) + return &h_cg->reserved_hugepage[idx]; + return &h_cg->hugepage[idx]; +} + static inline struct hugetlb_cgroup *hugetlb_cgroup_from_css(struct cgroup_subsys_state *s) { @@ -295,28 +308,42 @@ void hugetlb_cgroup_uncharge_cgroup(int idx, unsigned long nr_pages, enum { RES_USAGE, + RES_RESERVATION_USAGE, RES_LIMIT, + RES_RESERVATION_LIMIT, RES_MAX_USAGE, + RES_RESERVATION_MAX_USAGE, RES_FAILCNT, + RES_RESERVATION_FAILCNT, }; static u64 hugetlb_cgroup_read_u64(struct cgroup_subsys_state *css, struct cftype *cft) { struct page_counter *counter; + struct page_counter *reserved_counter; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(css); counter = &h_cg->hugepage[MEMFILE_IDX(cft->private)]; + reserved_counter = &h_cg->reserved_hugepage[MEMFILE_IDX(cft->private)]; switch (MEMFILE_ATTR(cft->private)) { case RES_USAGE: return (u64)page_counter_read(counter) * PAGE_SIZE; + case RES_RESERVATION_USAGE: + return (u64)page_counter_read(reserved_counter) * PAGE_SIZE; case RES_LIMIT: return (u64)counter->max * PAGE_SIZE; + case RES_RESERVATION_LIMIT: + return (u64)reserved_counter->max * PAGE_SIZE; case RES_MAX_USAGE: return (u64)counter->watermark * PAGE_SIZE; + case RES_RESERVATION_MAX_USAGE: + return (u64)reserved_counter->watermark * PAGE_SIZE; case RES_FAILCNT: return counter->failcnt; + case RES_RESERVATION_FAILCNT: + return reserved_counter->failcnt; default: BUG(); } @@ -338,10 +365,16 @@ static int hugetlb_cgroup_read_u64_max(struct seq_file *seq, void *v) 1 << huge_page_order(&hstates[idx])); switch (MEMFILE_ATTR(cft->private)) { + case RES_RESERVATION_USAGE: + counter = &h_cg->reserved_hugepage[idx]; + /* Fall through. */ case RES_USAGE: val = (u64)page_counter_read(counter); seq_printf(seq, "%llu\n", val * PAGE_SIZE); break; + case RES_RESERVATION_LIMIT: + counter = &h_cg->reserved_hugepage[idx]; + /* Fall through. */ case RES_LIMIT: val = (u64)counter->max; if (val == limit) @@ -365,6 +398,7 @@ static ssize_t hugetlb_cgroup_write(struct kernfs_open_file *of, int ret, idx; unsigned long nr_pages; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of)); + bool reserved = false; if (hugetlb_cgroup_is_root(h_cg)) /* Can't set limit on root */ return -EINVAL; @@ -378,9 +412,14 @@ static ssize_t hugetlb_cgroup_write(struct kernfs_open_file *of, nr_pages = round_down(nr_pages, 1 << huge_page_order(&hstates[idx])); switch (MEMFILE_ATTR(of_cft(of)->private)) { + case RES_RESERVATION_LIMIT: + reserved = true; + /* Fall through. */ case RES_LIMIT: mutex_lock(&hugetlb_limit_mutex); - ret = page_counter_set_max(&h_cg->hugepage[idx], nr_pages); + ret = page_counter_set_max(hugetlb_cgroup_get_counter(h_cg, idx, + reserved), + nr_pages); mutex_unlock(&hugetlb_limit_mutex); break; default: @@ -406,18 +445,26 @@ static ssize_t hugetlb_cgroup_reset(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { int ret = 0; - struct page_counter *counter; + struct page_counter *counter, *reserved_counter; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of)); counter = &h_cg->hugepage[MEMFILE_IDX(of_cft(of)->private)]; + reserved_counter = + &h_cg->reserved_hugepage[MEMFILE_IDX(of_cft(of)->private)]; switch (MEMFILE_ATTR(of_cft(of)->private)) { case RES_MAX_USAGE: page_counter_reset_watermark(counter); break; + case RES_RESERVATION_MAX_USAGE: + page_counter_reset_watermark(reserved_counter); + break; case RES_FAILCNT: counter->failcnt = 0; break; + case RES_RESERVATION_FAILCNT: + reserved_counter->failcnt = 0; + break; default: ret = -EINVAL; break; @@ -472,7 +519,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) struct hstate *h = &hstates[idx]; /* format the size */ - mem_fmt(buf, 32, huge_page_size(h)); + mem_fmt(buf, sizeof(buf), huge_page_size(h)); /* Add the limit file */ cft = &h->cgroup_files_dfl[0]; @@ -482,15 +529,30 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->write = hugetlb_cgroup_write_dfl; cft->flags = CFTYPE_NOT_ON_ROOT; - /* Add the current usage file */ + /* Add the reservation limit file */ cft = &h->cgroup_files_dfl[1]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.reservation_max", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_LIMIT); + cft->seq_show = hugetlb_cgroup_read_u64_max; + cft->write = hugetlb_cgroup_write_dfl; + cft->flags = CFTYPE_NOT_ON_ROOT; + + /* Add the current usage file */ + cft = &h->cgroup_files_dfl[2]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.current", buf); cft->private = MEMFILE_PRIVATE(idx, RES_USAGE); cft->seq_show = hugetlb_cgroup_read_u64_max; cft->flags = CFTYPE_NOT_ON_ROOT; + /* Add the current reservation usage file */ + cft = &h->cgroup_files_dfl[3]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.reservation_current", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_USAGE); + cft->seq_show = hugetlb_cgroup_read_u64_max; + cft->flags = CFTYPE_NOT_ON_ROOT; + /* Add the events file */ - cft = &h->cgroup_files_dfl[2]; + cft = &h->cgroup_files_dfl[4]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.events", buf); cft->private = MEMFILE_PRIVATE(idx, 0); cft->seq_show = hugetlb_events_show; @@ -498,7 +560,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->flags = CFTYPE_NOT_ON_ROOT; /* Add the events.local file */ - cft = &h->cgroup_files_dfl[3]; + cft = &h->cgroup_files_dfl[5]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.events.local", buf); cft->private = MEMFILE_PRIVATE(idx, 0); cft->seq_show = hugetlb_events_local_show; @@ -507,7 +569,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->flags = CFTYPE_NOT_ON_ROOT; /* NULL terminate the last cft */ - cft = &h->cgroup_files_dfl[4]; + cft = &h->cgroup_files_dfl[6]; memset(cft, 0, sizeof(*cft)); WARN_ON(cgroup_add_dfl_cftypes(&hugetlb_cgrp_subsys, @@ -530,28 +592,58 @@ static void __init __hugetlb_cgroup_file_legacy_init(int idx) cft->read_u64 = hugetlb_cgroup_read_u64; cft->write = hugetlb_cgroup_write_legacy; - /* Add the usage file */ + /* Add the reservation limit file */ cft = &h->cgroup_files_legacy[1]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.reservation_limit_in_bytes", + buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_LIMIT); + cft->read_u64 = hugetlb_cgroup_read_u64; + cft->write = hugetlb_cgroup_write_legacy; + + /* Add the usage file */ + cft = &h->cgroup_files_legacy[2]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.usage_in_bytes", buf); cft->private = MEMFILE_PRIVATE(idx, RES_USAGE); cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the reservation usage file */ + cft = &h->cgroup_files_legacy[3]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.reservation_usage_in_bytes", + buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_USAGE); + cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the MAX usage file */ - cft = &h->cgroup_files_legacy[2]; + cft = &h->cgroup_files_legacy[4]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.max_usage_in_bytes", buf); cft->private = MEMFILE_PRIVATE(idx, RES_MAX_USAGE); cft->write = hugetlb_cgroup_reset; cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the MAX reservation usage file */ + cft = &h->cgroup_files_legacy[5]; + snprintf(cft->name, MAX_CFTYPE_NAME, + "%s.reservation_max_usage_in_bytes", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_MAX_USAGE); + cft->write = hugetlb_cgroup_reset; + cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the failcntfile */ - cft = &h->cgroup_files_legacy[3]; + cft = &h->cgroup_files_legacy[6]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.failcnt", buf); - cft->private = MEMFILE_PRIVATE(idx, RES_FAILCNT); + cft->private = MEMFILE_PRIVATE(idx, RES_FAILCNT); + cft->write = hugetlb_cgroup_reset; + cft->read_u64 = hugetlb_cgroup_read_u64; + + /* Add the reservation failcntfile */ + cft = &h->cgroup_files_legacy[7]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.reservation_failcnt", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RESERVATION_FAILCNT); cft->write = hugetlb_cgroup_reset; cft->read_u64 = hugetlb_cgroup_read_u64; /* NULL terminate the last cft */ - cft = &h->cgroup_files_legacy[4]; + cft = &h->cgroup_files_legacy[8]; memset(cft, 0, sizeof(*cft)); WARN_ON(cgroup_add_legacy_cftypes(&hugetlb_cgrp_subsys, -- 2.24.1.735.g03f4e72817-goog

4 years, 3 months

4
26
0 0

[PATCH 1/3] selftests/x86: Build with LDFLAGS and LDLIBS

by Daniel Díaz

During cross-compilation, it was discovered that LDFLAGS and LDLIBS were not being used while building binaries, leading to defaults which were not necessarily correct. OpenEmbedded reported this kind of problem: ERROR: QA Issue: No GNU_HASH in the ELF binary [...], didn't pass LDFLAGS? Signed-off-by: Daniel Díaz <daniel.diaz(a)linaro.org> --- tools/testing/selftests/x86/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/x86/Makefile b/tools/testing/selftests/x86/Makefile index 5d49bfec1e9a..f8360454eafd 100644 --- a/tools/testing/selftests/x86/Makefile +++ b/tools/testing/selftests/x86/Makefile @@ -71,10 +71,10 @@ all_64: $(BINARIES_64) EXTRA_CLEAN := $(BINARIES_32) $(BINARIES_64) $(BINARIES_32): $(OUTPUT)/%_32: %.c - $(CC) -m32 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl -lm + $(CC) -m32 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -lrt -ldl -lm $(BINARIES_64): $(OUTPUT)/%_64: %.c - $(CC) -m64 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl + $(CC) -m64 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -lrt -ldl # x86_64 users should be encouraged to install 32-bit libraries ifeq ($(CAN_BUILD_I386)$(CAN_BUILD_X86_64),01) -- 2.20.1

4 years, 3 months

5
6
0 0

Re: [RFC PATCH v1] pin_on_cpu: Introduce thread CPU pinning system call

by Mathieu Desnoyers

(replying again as plain text for mailing lists) ----- On Jan 22, 2020, at 10:44 AM, Jan Ziak 0xe2.0x9a.0x9b(a)gmail.com wrote: > Hello > I would like to note that this does not help userspace to express dynamic > scheduling relationships among processes/threads such as "do not run processes > A and B on the same core" or "run processes A and B on cores sharing the same > L2 cache". Indeed, this is not what this system call is trying to solve. Does the name "pin_on_cpu" lead to confusion here ? I thought that cgroups was already the mechanism taking care of this kind of requirement. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com

4 years, 3 months

1
0
0 0

Re: [RFC PATCH v1] pin_on_cpu: Introduce thread CPU pinning system call

by Jan Ziak

Hello I would like to note that this does not help userspace to express dynamic scheduling relationships among processes/threads such as "do not run processes A and B on the same core" or "run processes A and B on cores sharing the same L2 cache". Sincerely Jan

4 years, 3 months

1
0
0 0

[PATCH v5 6/6] selftests/cgroup: add tests for cloning into cgroups

by Christian Brauner

Expand the cgroup test-suite to include tests for CLONE_INTO_CGROUP. This adds the following tests: - CLONE_INTO_CGROUP manages to clone a process directly into a correctly delegated cgroup - CLONE_INTO_CGROUP fails to clone a process into a cgroup that has been removed after we've opened an fd to it - CLONE_INTO_CGROUP fails to clone a process into an invalid domain cgroup - CLONE_INTO_CGROUP adheres to the no internal process constraint - CLONE_INTO_CGROUP works with the freezer feature Cc: Tejun Heo <tj(a)kernel.org> Cc: Shuah Khan <shuah(a)kernel.org> Cc: cgroups(a)vger.kernel.org Cc: linux-kselftest(a)vger.kernel.org Acked-by: Roman Gushchin <guro(a)fb.com> Signed-off-by: Christian Brauner <christian.brauner(a)ubuntu.com> --- /* v1 */ Link: https://lore.kernel.org/r/20191218173516.7875-4-christian.brauner@ubuntu.com /* v2 */ Link: https://lore.kernel.org/r/20191223061504.28716-4-christian.brauner@ubuntu.c… unchanged /* v3 */ Link: https://lore.kernel.org/r/20200117002143.15559-6-christian.brauner@ubuntu.c… unchanged /* v4 */ Link: https://lore.kernel.org/r/20200117181219.14542-7-christian.brauner@ubuntu.c… unchanged /* v5 */ unchanged - Christian Brauner <christian.brauner(a)ubuntu.com>: - add Acked-by: Roman Gushchin <guro(a)fb.com> Signed-off-by: Christian Brauner <christian.brauner(a)ubuntu.com> --- tools/testing/selftests/cgroup/Makefile | 6 +- tools/testing/selftests/cgroup/cgroup_util.c | 126 ++++++++++++++++++ tools/testing/selftests/cgroup/cgroup_util.h | 4 + tools/testing/selftests/cgroup/test_core.c | 64 +++++++++ .../selftests/clone3/clone3_selftests.h | 19 ++- 5 files changed, 214 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index 66aafe1f5746..967f268fde74 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -11,6 +11,6 @@ TEST_GEN_PROGS += test_freezer include ../lib.mk -$(OUTPUT)/test_memcontrol: cgroup_util.c -$(OUTPUT)/test_core: cgroup_util.c -$(OUTPUT)/test_freezer: cgroup_util.c +$(OUTPUT)/test_memcontrol: cgroup_util.c ../clone3/clone3_selftests.h +$(OUTPUT)/test_core: cgroup_util.c ../clone3/clone3_selftests.h +$(OUTPUT)/test_freezer: cgroup_util.c ../clone3/clone3_selftests.h diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index 8f7131dcf1ff..8a637ca7d73a 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -15,6 +15,7 @@ #include <unistd.h> #include "cgroup_util.h" +#include "../clone3/clone3_selftests.h" static ssize_t read_text(const char *path, char *buf, size_t max_len) { @@ -331,12 +332,112 @@ int cg_run(const char *cgroup, } } +pid_t clone_into_cgroup(int cgroup_fd) +{ +#ifdef CLONE_ARGS_SIZE_VER2 + pid_t pid; + + struct clone_args args = { + .flags = CLONE_INTO_CGROUP, + .exit_signal = SIGCHLD, + .cgroup = cgroup_fd, + }; + + pid = sys_clone3(&args, sizeof(struct clone_args)); + /* + * Verify that this is a genuine test failure: + * ENOSYS -> clone3() not available + * E2BIG -> CLONE_INTO_CGROUP not available + */ + if (pid < 0 && (errno == ENOSYS || errno == E2BIG)) + goto pretend_enosys; + + return pid; + +pretend_enosys: +#endif + errno = ENOSYS; + return -ENOSYS; +} + +int clone_reap(pid_t pid, int options) +{ + int ret; + siginfo_t info = { + .si_signo = 0, + }; + +again: + ret = waitid(P_PID, pid, &info, options | __WALL | __WNOTHREAD); + if (ret < 0) { + if (errno == EINTR) + goto again; + return -1; + } + + if (options & WEXITED) { + if (WIFEXITED(info.si_status)) + return WEXITSTATUS(info.si_status); + } + + if (options & WSTOPPED) { + if (WIFSTOPPED(info.si_status)) + return WSTOPSIG(info.si_status); + } + + if (options & WCONTINUED) { + if (WIFCONTINUED(info.si_status)) + return 0; + } + + return -1; +} + +int dirfd_open_opath(const char *dir) +{ + return open(dir, O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW | O_PATH); +} + +#define close_prot_errno(fd) \ + if (fd >= 0) { \ + int _e_ = errno; \ + close(fd); \ + errno = _e_; \ + } + +static int clone_into_cgroup_run_nowait(const char *cgroup, + int (*fn)(const char *cgroup, void *arg), + void *arg) +{ + int cgroup_fd; + pid_t pid; + + cgroup_fd = dirfd_open_opath(cgroup); + if (cgroup_fd < 0) + return -1; + + pid = clone_into_cgroup(cgroup_fd); + close_prot_errno(cgroup_fd); + if (pid == 0) + exit(fn(cgroup, arg)); + + return pid; +} + int cg_run_nowait(const char *cgroup, int (*fn)(const char *cgroup, void *arg), void *arg) { int pid; + pid = clone_into_cgroup_run_nowait(cgroup, fn, arg); + if (pid > 0) + return pid; + + /* Genuine test failure. */ + if (pid < 0 && errno != ENOSYS) + return -1; + pid = fork(); if (pid == 0) { char buf[64]; @@ -450,3 +551,28 @@ int proc_read_strstr(int pid, bool thread, const char *item, const char *needle) return strstr(buf, needle) ? 0 : -1; } + +int clone_into_cgroup_run_wait(const char *cgroup) +{ + int cgroup_fd; + pid_t pid; + + cgroup_fd = dirfd_open_opath(cgroup); + if (cgroup_fd < 0) + return -1; + + pid = clone_into_cgroup(cgroup_fd); + close_prot_errno(cgroup_fd); + if (pid < 0) + return -1; + + if (pid == 0) + exit(EXIT_SUCCESS); + + /* + * We don't care whether this fails. We only care whether the initial + * clone succeeded. + */ + (void)clone_reap(pid, WEXITED); + return 0; +} diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h index 49c54fbdb229..5a1305dd1f0b 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.h +++ b/tools/testing/selftests/cgroup/cgroup_util.h @@ -50,3 +50,7 @@ extern int cg_wait_for_proc_count(const char *cgroup, int count); extern int cg_killall(const char *cgroup); extern ssize_t proc_read_text(int pid, bool thread, const char *item, char *buf, size_t size); extern int proc_read_strstr(int pid, bool thread, const char *item, const char *needle); +extern pid_t clone_into_cgroup(int cgroup_fd); +extern int clone_reap(pid_t pid, int options); +extern int clone_into_cgroup_run_wait(const char *cgroup); +extern int dirfd_open_opath(const char *dir); diff --git a/tools/testing/selftests/cgroup/test_core.c b/tools/testing/selftests/cgroup/test_core.c index c5ca669feb2b..96e016ccafe0 100644 --- a/tools/testing/selftests/cgroup/test_core.c +++ b/tools/testing/selftests/cgroup/test_core.c @@ -25,8 +25,11 @@ static int test_cgcore_populated(const char *root) { int ret = KSFT_FAIL; + int err; char *cg_test_a = NULL, *cg_test_b = NULL; char *cg_test_c = NULL, *cg_test_d = NULL; + int cgroup_fd = -EBADF; + pid_t pid; cg_test_a = cg_name(root, "cg_test_a"); cg_test_b = cg_name(root, "cg_test_a/cg_test_b"); @@ -78,6 +81,52 @@ static int test_cgcore_populated(const char *root) if (cg_read_strcmp(cg_test_d, "cgroup.events", "populated 0\n")) goto cleanup; + /* Test that we can directly clone into a new cgroup. */ + cgroup_fd = dirfd_open_opath(cg_test_d); + if (cgroup_fd < 0) + goto cleanup; + + pid = clone_into_cgroup(cgroup_fd); + if (pid < 0) { + if (errno == ENOSYS) + goto cleanup_pass; + goto cleanup; + } + + if (pid == 0) { + if (raise(SIGSTOP)) + exit(EXIT_FAILURE); + exit(EXIT_SUCCESS); + } + + err = cg_read_strcmp(cg_test_d, "cgroup.events", "populated 1\n"); + + (void)clone_reap(pid, WSTOPPED); + (void)kill(pid, SIGCONT); + (void)clone_reap(pid, WEXITED); + + if (err) + goto cleanup; + + if (cg_read_strcmp(cg_test_d, "cgroup.events", "populated 0\n")) + goto cleanup; + + /* Remove cgroup. */ + if (cg_test_d) { + cg_destroy(cg_test_d); + free(cg_test_d); + cg_test_d = NULL; + } + + pid = clone_into_cgroup(cgroup_fd); + if (pid < 0) + goto cleanup_pass; + if (pid == 0) + exit(EXIT_SUCCESS); + (void)clone_reap(pid, WEXITED); + goto cleanup; + +cleanup_pass: ret = KSFT_PASS; cleanup: @@ -93,6 +142,8 @@ static int test_cgcore_populated(const char *root) free(cg_test_c); free(cg_test_b); free(cg_test_a); + if (cgroup_fd >= 0) + close(cgroup_fd); return ret; } @@ -136,6 +187,16 @@ static int test_cgcore_invalid_domain(const char *root) if (errno != EOPNOTSUPP) goto cleanup; + if (!clone_into_cgroup_run_wait(child)) + goto cleanup; + + if (errno == ENOSYS) + goto cleanup_pass; + + if (errno != EOPNOTSUPP) + goto cleanup; + +cleanup_pass: ret = KSFT_PASS; cleanup: @@ -345,6 +406,9 @@ static int test_cgcore_internal_process_constraint(const char *root) if (!cg_enter_current(parent)) goto cleanup; + if (!clone_into_cgroup_run_wait(parent)) + goto cleanup; + ret = KSFT_PASS; cleanup: diff --git a/tools/testing/selftests/clone3/clone3_selftests.h b/tools/testing/selftests/clone3/clone3_selftests.h index a3f2c8ad8bcc..91c1a78ddb39 100644 --- a/tools/testing/selftests/clone3/clone3_selftests.h +++ b/tools/testing/selftests/clone3/clone3_selftests.h @@ -5,12 +5,24 @@ #define _GNU_SOURCE #include <sched.h> +#include <linux/sched.h> +#include <linux/types.h> #include <stdint.h> #include <syscall.h> -#include <linux/types.h> +#include <sys/wait.h> + +#include "../kselftest.h" #define ptr_to_u64(ptr) ((__u64)((uintptr_t)(ptr))) +#ifndef CLONE_INTO_CGROUP +#define CLONE_INTO_CGROUP 0x200000000ULL /* Clone into a specific cgroup given the right permissions. */ +#endif + +#ifndef CLONE_ARGS_SIZE_VER0 +#define CLONE_ARGS_SIZE_VER0 64 +#endif + #ifndef __NR_clone3 #define __NR_clone3 -1 struct clone_args { @@ -22,10 +34,13 @@ struct clone_args { __aligned_u64 stack; __aligned_u64 stack_size; __aligned_u64 tls; +#define CLONE_ARGS_SIZE_VER1 80 __aligned_u64 set_tid; __aligned_u64 set_tid_size; +#define CLONE_ARGS_SIZE_VER2 88 + __aligned_u64 cgroup; }; -#endif +#endif /* __NR_clone3 */ static pid_t sys_clone3(struct clone_args *args, size_t size) { -- 2.25.0

4 years, 3 months

1
0
0 0

[PATCH bpf-next v5 00/11] tools: Use consistent libbpf include paths everywhere

by Toke Høiland-Jørgensen

We are currently being somewhat inconsistent with the libbpf include paths, which makes it difficult to move files from the kernel into an external libbpf-using project without adjusting include paths. Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a requirement for building against libbpf before, and indeed the libbpf pkg-config file doesn't include it. So let's make all libbpf includes across the kernel tree use the bpf/ prefix in their includes. Since bpftool skeleton generation emits code with a libbpf include, this also ensures that those can be used in existing external projects using the regular pkg-config include path. This turns out to be a somewhat invasive change in the number of files touched; however, the actual changes to files are fairly trivial (most of them are simply made with 'sed'). The series is split to make the change for one tool subdir at a time, while trying not to break the build along the way. It is structured like this: - Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing the include paths. - Patch 4-8: Change the include directives to use the bpf/ prefix, and updates Makefiles to make sure tools/lib/ is part of the include path, but without removing tools/lib/bpf - Patch 9-11: Remove tools/lib/bpf from include paths to make sure we don't inadvertently re-introduce includes without the bpf/ prefix. Changelog: v5: - Combine the libbpf build rules in selftests Makefile (using Andrii's suggestion for a make rule). - Re-use self-tests libbpf build for runqslower (new patch 10) - Formatting fixes v4: - Move runqslower error on missing BTF into make rule - Make sure we don't always force a rebuild selftests - Rebase on latest bpf-next (dropping patch 11) v3: - Don't add the kernel build dir to the runqslower Makefile, pass it in from selftests instead. - Use libbpf's 'make install_headers' in selftests instead of trying to generate bpf_helper_defs.h in-place (to also work on read-only filesystems). - Use a scratch builddir for both libbpf and bpftool when building in selftests. - Revert bpf_helpers.h to quoted include instead of angled include with a bpf/ prefix. - Fix a few style nits from Andrii v2: - Do a full cleanup of libbpf includes instead of just changing the bpf_helper_defs.h include. --- Toke Høiland-Jørgensen (11): samples/bpf: Don't try to remove user's homedir on clean tools/bpf/runqslower: Fix override option for VMLINUX_BTF selftests: Pass VMLINUX_BTF to runqslower Makefile tools/runqslower: Use consistent include paths for libbpf selftests: Use consistent include paths for libbpf bpftool: Use consistent include paths for libbpf perf: Use consistent include paths for libbpf samples/bpf: Use consistent include paths for libbpf tools/runqslower: Remove tools/lib/bpf from include path runsqslower: Support user-specified libbpf include and object paths selftests: Refactor build to remove tools/lib/bpf from include path samples/bpf/Makefile | 5 +- samples/bpf/cpustat_kern.c | 2 - samples/bpf/fds_example.c | 2 - samples/bpf/hbm.c | 4 + samples/bpf/hbm_kern.h | 4 + samples/bpf/ibumad_kern.c | 2 - samples/bpf/ibumad_user.c | 2 - samples/bpf/lathist_kern.c | 2 - samples/bpf/lwt_len_hist_kern.c | 2 - samples/bpf/map_perf_test_kern.c | 4 + samples/bpf/offwaketime_kern.c | 4 + samples/bpf/offwaketime_user.c | 2 - samples/bpf/parse_ldabs.c | 2 - samples/bpf/parse_simple.c | 2 - samples/bpf/parse_varlen.c | 2 - samples/bpf/sampleip_kern.c | 4 + samples/bpf/sampleip_user.c | 2 - samples/bpf/sock_flags_kern.c | 2 - samples/bpf/sockex1_kern.c | 2 - samples/bpf/sockex1_user.c | 2 - samples/bpf/sockex2_kern.c | 2 - samples/bpf/sockex2_user.c | 2 - samples/bpf/sockex3_kern.c | 2 - samples/bpf/spintest_kern.c | 4 + samples/bpf/spintest_user.c | 2 - samples/bpf/syscall_tp_kern.c | 2 - samples/bpf/task_fd_query_kern.c | 2 - samples/bpf/task_fd_query_user.c | 2 - samples/bpf/tc_l2_redirect_kern.c | 2 - samples/bpf/tcbpf1_kern.c | 2 - samples/bpf/tcp_basertt_kern.c | 4 + samples/bpf/tcp_bufs_kern.c | 4 + samples/bpf/tcp_clamp_kern.c | 4 + samples/bpf/tcp_cong_kern.c | 4 + samples/bpf/tcp_dumpstats_kern.c | 4 + samples/bpf/tcp_iw_kern.c | 4 + samples/bpf/tcp_rwnd_kern.c | 4 + samples/bpf/tcp_synrto_kern.c | 4 + samples/bpf/tcp_tos_reflect_kern.c | 4 + samples/bpf/test_cgrp2_tc_kern.c | 2 - samples/bpf/test_current_task_under_cgroup_kern.c | 2 - samples/bpf/test_lwt_bpf.c | 2 - samples/bpf/test_map_in_map_kern.c | 4 + samples/bpf/test_overhead_kprobe_kern.c | 4 + samples/bpf/test_overhead_raw_tp_kern.c | 2 - samples/bpf/test_overhead_tp_kern.c | 2 - samples/bpf/test_probe_write_user_kern.c | 4 + samples/bpf/trace_event_kern.c | 4 + samples/bpf/trace_event_user.c | 2 - samples/bpf/trace_output_kern.c | 2 - samples/bpf/trace_output_user.c | 2 - samples/bpf/tracex1_kern.c | 4 + samples/bpf/tracex2_kern.c | 4 + samples/bpf/tracex3_kern.c | 4 + samples/bpf/tracex4_kern.c | 4 + samples/bpf/tracex5_kern.c | 4 + samples/bpf/tracex6_kern.c | 2 - samples/bpf/tracex7_kern.c | 2 - samples/bpf/xdp1_kern.c | 2 - samples/bpf/xdp1_user.c | 4 + samples/bpf/xdp2_kern.c | 2 - samples/bpf/xdp2skb_meta_kern.c | 2 - samples/bpf/xdp_adjust_tail_kern.c | 2 - samples/bpf/xdp_adjust_tail_user.c | 4 + samples/bpf/xdp_fwd_kern.c | 2 - samples/bpf/xdp_fwd_user.c | 2 - samples/bpf/xdp_monitor_kern.c | 2 - samples/bpf/xdp_redirect_cpu_kern.c | 2 - samples/bpf/xdp_redirect_cpu_user.c | 2 - samples/bpf/xdp_redirect_kern.c | 2 - samples/bpf/xdp_redirect_map_kern.c | 2 - samples/bpf/xdp_redirect_map_user.c | 2 - samples/bpf/xdp_redirect_user.c | 2 - samples/bpf/xdp_router_ipv4_kern.c | 2 - samples/bpf/xdp_router_ipv4_user.c | 2 - samples/bpf/xdp_rxq_info_kern.c | 2 - samples/bpf/xdp_rxq_info_user.c | 4 + samples/bpf/xdp_sample_pkts_kern.c | 2 - samples/bpf/xdp_sample_pkts_user.c | 2 - samples/bpf/xdp_tx_iptunnel_kern.c | 2 - samples/bpf/xdp_tx_iptunnel_user.c | 2 - samples/bpf/xdpsock_kern.c | 2 - samples/bpf/xdpsock_user.c | 6 +- tools/bpf/bpftool/Documentation/bpftool-gen.rst | 2 - tools/bpf/bpftool/Makefile | 2 - tools/bpf/bpftool/btf.c | 8 +- tools/bpf/bpftool/btf_dumper.c | 2 - tools/bpf/bpftool/cgroup.c | 2 - tools/bpf/bpftool/common.c | 4 + tools/bpf/bpftool/feature.c | 4 + tools/bpf/bpftool/gen.c | 10 ++- tools/bpf/bpftool/jit_disasm.c | 2 - tools/bpf/bpftool/main.c | 4 + tools/bpf/bpftool/map.c | 4 + tools/bpf/bpftool/map_perf_ring.c | 4 + tools/bpf/bpftool/net.c | 8 +- tools/bpf/bpftool/netlink_dumper.c | 4 + tools/bpf/bpftool/perf.c | 2 - tools/bpf/bpftool/prog.c | 6 +- tools/bpf/bpftool/xlated_dumper.c | 2 - tools/bpf/runqslower/Makefile | 36 ++++++----- tools/bpf/runqslower/runqslower.bpf.c | 2 - tools/bpf/runqslower/runqslower.c | 4 + tools/perf/examples/bpf/5sec.c | 2 - tools/perf/examples/bpf/empty.c | 2 - tools/perf/examples/bpf/sys_enter_openat.c | 2 - tools/perf/include/bpf/pid_filter.h | 2 - tools/perf/include/bpf/stdio.h | 2 - tools/perf/include/bpf/unistd.h | 2 - tools/testing/selftests/bpf/.gitignore | 4 - tools/testing/selftests/bpf/Makefile | 65 ++++++++++---------- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 4 + tools/testing/selftests/bpf/bpf_trace_helpers.h | 2 - tools/testing/selftests/bpf/bpf_util.h | 2 - tools/testing/selftests/bpf/prog_tests/cpu_mask.c | 2 - .../testing/selftests/bpf/prog_tests/perf_buffer.c | 2 - tools/testing/selftests/bpf/progs/bpf_dctcp.c | 4 + tools/testing/selftests/bpf/progs/bpf_flow.c | 4 + tools/testing/selftests/bpf/progs/connect4_prog.c | 4 + tools/testing/selftests/bpf/progs/connect6_prog.c | 4 + tools/testing/selftests/bpf/progs/dev_cgroup.c | 2 - tools/testing/selftests/bpf/progs/fentry_test.c | 2 - tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c | 2 - .../selftests/bpf/progs/fexit_bpf2bpf_simple.c | 2 - tools/testing/selftests/bpf/progs/fexit_test.c | 2 - .../selftests/bpf/progs/get_cgroup_id_kern.c | 2 - tools/testing/selftests/bpf/progs/kfree_skb.c | 4 + tools/testing/selftests/bpf/progs/loop1.c | 4 + tools/testing/selftests/bpf/progs/loop2.c | 4 + tools/testing/selftests/bpf/progs/loop3.c | 4 + tools/testing/selftests/bpf/progs/loop4.c | 2 - tools/testing/selftests/bpf/progs/loop5.c | 2 - tools/testing/selftests/bpf/progs/netcnt_prog.c | 2 - tools/testing/selftests/bpf/progs/pyperf.h | 2 - .../testing/selftests/bpf/progs/sample_map_ret0.c | 2 - tools/testing/selftests/bpf/progs/sendmsg4_prog.c | 4 + tools/testing/selftests/bpf/progs/sendmsg6_prog.c | 4 + .../selftests/bpf/progs/socket_cookie_prog.c | 4 + .../selftests/bpf/progs/sockmap_parse_prog.c | 4 + .../selftests/bpf/progs/sockmap_tcp_msg_prog.c | 4 + .../selftests/bpf/progs/sockmap_verdict_prog.c | 4 + .../testing/selftests/bpf/progs/sockopt_inherit.c | 2 - tools/testing/selftests/bpf/progs/sockopt_multi.c | 2 - tools/testing/selftests/bpf/progs/sockopt_sk.c | 2 - tools/testing/selftests/bpf/progs/strobemeta.h | 2 - tools/testing/selftests/bpf/progs/tailcall1.c | 2 - tools/testing/selftests/bpf/progs/tailcall2.c | 2 - tools/testing/selftests/bpf/progs/tailcall3.c | 2 - tools/testing/selftests/bpf/progs/tailcall4.c | 2 - tools/testing/selftests/bpf/progs/tailcall5.c | 2 - tools/testing/selftests/bpf/progs/tcp_rtt.c | 2 - .../testing/selftests/bpf/progs/test_adjust_tail.c | 2 - .../selftests/bpf/progs/test_attach_probe.c | 2 - tools/testing/selftests/bpf/progs/test_btf_haskv.c | 2 - tools/testing/selftests/bpf/progs/test_btf_newkv.c | 2 - tools/testing/selftests/bpf/progs/test_btf_nokv.c | 2 - .../testing/selftests/bpf/progs/test_core_extern.c | 2 - .../selftests/bpf/progs/test_core_reloc_arrays.c | 4 + .../bpf/progs/test_core_reloc_bitfields_direct.c | 4 + .../bpf/progs/test_core_reloc_bitfields_probed.c | 4 + .../bpf/progs/test_core_reloc_existence.c | 4 + .../selftests/bpf/progs/test_core_reloc_flavors.c | 4 + .../selftests/bpf/progs/test_core_reloc_ints.c | 4 + .../selftests/bpf/progs/test_core_reloc_kernel.c | 4 + .../selftests/bpf/progs/test_core_reloc_misc.c | 4 + .../selftests/bpf/progs/test_core_reloc_mods.c | 4 + .../selftests/bpf/progs/test_core_reloc_nesting.c | 4 + .../bpf/progs/test_core_reloc_primitives.c | 4 + .../bpf/progs/test_core_reloc_ptr_as_arr.c | 4 + .../selftests/bpf/progs/test_core_reloc_size.c | 4 + .../selftests/bpf/progs/test_get_stack_rawtp.c | 2 - .../testing/selftests/bpf/progs/test_global_data.c | 2 - .../selftests/bpf/progs/test_global_func1.c | 2 - .../selftests/bpf/progs/test_global_func3.c | 2 - .../selftests/bpf/progs/test_global_func5.c | 2 - .../selftests/bpf/progs/test_global_func6.c | 2 - .../selftests/bpf/progs/test_global_func7.c | 2 - tools/testing/selftests/bpf/progs/test_l4lb.c | 4 + .../selftests/bpf/progs/test_l4lb_noinline.c | 4 + .../selftests/bpf/progs/test_lirc_mode2_kern.c | 2 - .../selftests/bpf/progs/test_lwt_ip_encap.c | 4 + .../selftests/bpf/progs/test_lwt_seg6local.c | 4 + .../testing/selftests/bpf/progs/test_map_in_map.c | 2 - tools/testing/selftests/bpf/progs/test_map_lock.c | 2 - tools/testing/selftests/bpf/progs/test_mmap.c | 2 - tools/testing/selftests/bpf/progs/test_obj_id.c | 2 - tools/testing/selftests/bpf/progs/test_overhead.c | 4 + .../testing/selftests/bpf/progs/test_perf_buffer.c | 2 - tools/testing/selftests/bpf/progs/test_pinning.c | 2 - .../selftests/bpf/progs/test_pinning_invalid.c | 2 - .../testing/selftests/bpf/progs/test_pkt_access.c | 4 + .../selftests/bpf/progs/test_pkt_md_access.c | 2 - .../testing/selftests/bpf/progs/test_probe_user.c | 4 + .../selftests/bpf/progs/test_queue_stack_map.h | 2 - .../testing/selftests/bpf/progs/test_rdonly_maps.c | 2 - tools/testing/selftests/bpf/progs/test_seg6_loop.c | 4 + .../bpf/progs/test_select_reuseport_kern.c | 4 + .../selftests/bpf/progs/test_send_signal_kern.c | 2 - .../selftests/bpf/progs/test_sk_lookup_kern.c | 4 + .../selftests/bpf/progs/test_skb_cgroup_id_kern.c | 2 - tools/testing/selftests/bpf/progs/test_skb_ctx.c | 2 - tools/testing/selftests/bpf/progs/test_skeleton.c | 2 - .../selftests/bpf/progs/test_sock_fields_kern.c | 4 + tools/testing/selftests/bpf/progs/test_spin_lock.c | 2 - .../selftests/bpf/progs/test_stacktrace_build_id.c | 2 - .../selftests/bpf/progs/test_stacktrace_map.c | 2 - .../selftests/bpf/progs/test_sysctl_loop1.c | 2 - .../selftests/bpf/progs/test_sysctl_loop2.c | 2 - .../testing/selftests/bpf/progs/test_sysctl_prog.c | 2 - tools/testing/selftests/bpf/progs/test_tc_edt.c | 4 + tools/testing/selftests/bpf/progs/test_tc_tunnel.c | 4 + .../bpf/progs/test_tcp_check_syncookie_kern.c | 4 + .../testing/selftests/bpf/progs/test_tcp_estats.c | 2 - .../testing/selftests/bpf/progs/test_tcpbpf_kern.c | 4 + .../selftests/bpf/progs/test_tcpnotify_kern.c | 4 + .../testing/selftests/bpf/progs/test_tracepoint.c | 2 - .../testing/selftests/bpf/progs/test_tunnel_kern.c | 4 + .../selftests/bpf/progs/test_verif_scale1.c | 2 - .../selftests/bpf/progs/test_verif_scale2.c | 2 - .../selftests/bpf/progs/test_verif_scale3.c | 2 - tools/testing/selftests/bpf/progs/test_xdp.c | 4 + .../testing/selftests/bpf/progs/test_xdp_bpf2bpf.c | 2 - tools/testing/selftests/bpf/progs/test_xdp_loop.c | 4 + tools/testing/selftests/bpf/progs/test_xdp_meta.c | 2 - .../selftests/bpf/progs/test_xdp_noinline.c | 4 + .../selftests/bpf/progs/test_xdp_redirect.c | 2 - tools/testing/selftests/bpf/progs/test_xdp_vlan.c | 4 + tools/testing/selftests/bpf/progs/xdp_dummy.c | 2 - .../testing/selftests/bpf/progs/xdp_redirect_map.c | 2 - tools/testing/selftests/bpf/progs/xdp_tx.c | 2 - tools/testing/selftests/bpf/progs/xdping_kern.c | 4 + tools/testing/selftests/bpf/test_cpp.cpp | 6 +- tools/testing/selftests/bpf/test_hashmap.c | 2 - tools/testing/selftests/bpf/test_progs.h | 2 - tools/testing/selftests/bpf/test_sock.c | 2 - tools/testing/selftests/bpf/test_sockmap_kern.h | 4 + tools/testing/selftests/bpf/test_sysctl.c | 2 - tools/testing/selftests/bpf/trace_helpers.h | 2 - 238 files changed, 392 insertions(+), 388 deletions(-)

4 years, 3 months

3
15
0 0

[PATCH bpf-next v4 00/10] tools: Use consistent libbpf include paths everywhere

by Toke Høiland-Jørgensen

We are currently being somewhat inconsistent with the libbpf include paths, which makes it difficult to move files from the kernel into an external libbpf-using project without adjusting include paths. Having the bpf/ subdir of $INCLUDEDIR in the include path has never been a requirement for building against libbpf before, and indeed the libbpf pkg-config file doesn't include it. So let's make all libbpf includes across the kernel tree use the bpf/ prefix in their includes. Since bpftool skeleton generation emits code with a libbpf include, this also ensures that those can be used in existing external projects using the regular pkg-config include path. This turns out to be a somewhat invasive change in the number of files touched; however, the actual changes to files are fairly trivial (most of them are simply made with 'sed'). The series is split to make the change for one tool subdir at a time, while trying not to break the build along the way. It is structured like this: - Patch 1-3: Trivial fixes to Makefiles for issues I discovered while changing the include paths. - Patch 4-8: Change the include directives to use the bpf/ prefix, and updates Makefiles to make sure tools/lib/ is part of the include path, but without removing tools/lib/bpf - Patch 9-10: Remove tools/lib/bpf from include paths to make sure we don't inadvertently re-introduce includes without the bpf/ prefix. Changelog: v4: - Move runqslower error on missing BTF into make rule - Make sure we don't always force a rebuild selftests - Rebase on latest bpf-next (dropping patch 11) v3: - Don't add the kernel build dir to the runqslower Makefile, pass it in from selftests instead. - Use libbpf's 'make install_headers' in selftests instead of trying to generate bpf_helper_defs.h in-place (to also work on read-only filesystems). - Use a scratch builddir for both libbpf and bpftool when building in selftests. - Revert bpf_helpers.h to quoted include instead of angled include with a bpf/ prefix. - Fix a few style nits from Andrii v2: - Do a full cleanup of libbpf includes instead of just changing the bpf_helper_defs.h include. --- Toke Høiland-Jørgensen (10): samples/bpf: Don't try to remove user's homedir on clean tools/bpf/runqslower: Fix override option for VMLINUX_BTF selftests: Pass VMLINUX_BTF to runqslower Makefile tools/runqslower: Use consistent include paths for libbpf selftests: Use consistent include paths for libbpf bpftool: Use consistent include paths for libbpf perf: Use consistent include paths for libbpf samples/bpf: Use consistent include paths for libbpf selftests: Remove tools/lib/bpf from include path tools/runqslower: Remove tools/lib/bpf from include path samples/bpf/Makefile | 5 +- samples/bpf/cpustat_kern.c | 2 - samples/bpf/fds_example.c | 2 - samples/bpf/hbm.c | 4 + samples/bpf/hbm_kern.h | 4 + samples/bpf/ibumad_kern.c | 2 - samples/bpf/ibumad_user.c | 2 - samples/bpf/lathist_kern.c | 2 - samples/bpf/lwt_len_hist_kern.c | 2 - samples/bpf/map_perf_test_kern.c | 4 + samples/bpf/offwaketime_kern.c | 4 + samples/bpf/offwaketime_user.c | 2 - samples/bpf/parse_ldabs.c | 2 - samples/bpf/parse_simple.c | 2 - samples/bpf/parse_varlen.c | 2 - samples/bpf/sampleip_kern.c | 4 + samples/bpf/sampleip_user.c | 2 - samples/bpf/sock_flags_kern.c | 2 - samples/bpf/sockex1_kern.c | 2 - samples/bpf/sockex1_user.c | 2 - samples/bpf/sockex2_kern.c | 2 - samples/bpf/sockex2_user.c | 2 - samples/bpf/sockex3_kern.c | 2 - samples/bpf/spintest_kern.c | 4 + samples/bpf/spintest_user.c | 2 - samples/bpf/syscall_tp_kern.c | 2 - samples/bpf/task_fd_query_kern.c | 2 - samples/bpf/task_fd_query_user.c | 2 - samples/bpf/tc_l2_redirect_kern.c | 2 - samples/bpf/tcbpf1_kern.c | 2 - samples/bpf/tcp_basertt_kern.c | 4 + samples/bpf/tcp_bufs_kern.c | 4 + samples/bpf/tcp_clamp_kern.c | 4 + samples/bpf/tcp_cong_kern.c | 4 + samples/bpf/tcp_dumpstats_kern.c | 4 + samples/bpf/tcp_iw_kern.c | 4 + samples/bpf/tcp_rwnd_kern.c | 4 + samples/bpf/tcp_synrto_kern.c | 4 + samples/bpf/tcp_tos_reflect_kern.c | 4 + samples/bpf/test_cgrp2_tc_kern.c | 2 - samples/bpf/test_current_task_under_cgroup_kern.c | 2 - samples/bpf/test_lwt_bpf.c | 2 - samples/bpf/test_map_in_map_kern.c | 4 + samples/bpf/test_overhead_kprobe_kern.c | 4 + samples/bpf/test_overhead_raw_tp_kern.c | 2 - samples/bpf/test_overhead_tp_kern.c | 2 - samples/bpf/test_probe_write_user_kern.c | 4 + samples/bpf/trace_event_kern.c | 4 + samples/bpf/trace_event_user.c | 2 - samples/bpf/trace_output_kern.c | 2 - samples/bpf/trace_output_user.c | 2 - samples/bpf/tracex1_kern.c | 4 + samples/bpf/tracex2_kern.c | 4 + samples/bpf/tracex3_kern.c | 4 + samples/bpf/tracex4_kern.c | 4 + samples/bpf/tracex5_kern.c | 4 + samples/bpf/tracex6_kern.c | 2 - samples/bpf/tracex7_kern.c | 2 - samples/bpf/xdp1_kern.c | 2 - samples/bpf/xdp1_user.c | 4 + samples/bpf/xdp2_kern.c | 2 - samples/bpf/xdp2skb_meta_kern.c | 2 - samples/bpf/xdp_adjust_tail_kern.c | 2 - samples/bpf/xdp_adjust_tail_user.c | 4 + samples/bpf/xdp_fwd_kern.c | 2 - samples/bpf/xdp_fwd_user.c | 2 - samples/bpf/xdp_monitor_kern.c | 2 - samples/bpf/xdp_redirect_cpu_kern.c | 2 - samples/bpf/xdp_redirect_cpu_user.c | 2 - samples/bpf/xdp_redirect_kern.c | 2 - samples/bpf/xdp_redirect_map_kern.c | 2 - samples/bpf/xdp_redirect_map_user.c | 2 - samples/bpf/xdp_redirect_user.c | 2 - samples/bpf/xdp_router_ipv4_kern.c | 2 - samples/bpf/xdp_router_ipv4_user.c | 2 - samples/bpf/xdp_rxq_info_kern.c | 2 - samples/bpf/xdp_rxq_info_user.c | 4 + samples/bpf/xdp_sample_pkts_kern.c | 2 - samples/bpf/xdp_sample_pkts_user.c | 2 - samples/bpf/xdp_tx_iptunnel_kern.c | 2 - samples/bpf/xdp_tx_iptunnel_user.c | 2 - samples/bpf/xdpsock_kern.c | 2 - samples/bpf/xdpsock_user.c | 6 +- tools/bpf/bpftool/Documentation/bpftool-gen.rst | 2 - tools/bpf/bpftool/Makefile | 2 - tools/bpf/bpftool/btf.c | 8 +-- tools/bpf/bpftool/btf_dumper.c | 2 - tools/bpf/bpftool/cgroup.c | 2 - tools/bpf/bpftool/common.c | 4 + tools/bpf/bpftool/feature.c | 4 + tools/bpf/bpftool/gen.c | 10 ++-- tools/bpf/bpftool/jit_disasm.c | 2 - tools/bpf/bpftool/main.c | 4 + tools/bpf/bpftool/map.c | 4 + tools/bpf/bpftool/map_perf_ring.c | 4 + tools/bpf/bpftool/net.c | 8 +-- tools/bpf/bpftool/netlink_dumper.c | 4 + tools/bpf/bpftool/perf.c | 2 - tools/bpf/bpftool/prog.c | 6 +- tools/bpf/bpftool/xlated_dumper.c | 2 - tools/bpf/runqslower/Makefile | 23 ++++---- tools/bpf/runqslower/runqslower.bpf.c | 2 - tools/bpf/runqslower/runqslower.c | 4 + tools/perf/examples/bpf/5sec.c | 2 - tools/perf/examples/bpf/empty.c | 2 - tools/perf/examples/bpf/sys_enter_openat.c | 2 - tools/perf/include/bpf/pid_filter.h | 2 - tools/perf/include/bpf/stdio.h | 2 - tools/perf/include/bpf/unistd.h | 2 - tools/testing/selftests/bpf/.gitignore | 2 - tools/testing/selftests/bpf/Makefile | 57 +++++++++++++------- tools/testing/selftests/bpf/bpf_tcp_helpers.h | 4 + tools/testing/selftests/bpf/bpf_trace_helpers.h | 2 - tools/testing/selftests/bpf/bpf_util.h | 2 - tools/testing/selftests/bpf/prog_tests/cpu_mask.c | 2 - .../testing/selftests/bpf/prog_tests/perf_buffer.c | 2 - tools/testing/selftests/bpf/progs/bpf_dctcp.c | 4 + tools/testing/selftests/bpf/progs/bpf_flow.c | 4 + tools/testing/selftests/bpf/progs/connect4_prog.c | 4 + tools/testing/selftests/bpf/progs/connect6_prog.c | 4 + tools/testing/selftests/bpf/progs/dev_cgroup.c | 2 - tools/testing/selftests/bpf/progs/fentry_test.c | 2 - tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c | 2 - .../selftests/bpf/progs/fexit_bpf2bpf_simple.c | 2 - tools/testing/selftests/bpf/progs/fexit_test.c | 2 - .../selftests/bpf/progs/get_cgroup_id_kern.c | 2 - tools/testing/selftests/bpf/progs/kfree_skb.c | 4 + tools/testing/selftests/bpf/progs/loop1.c | 4 + tools/testing/selftests/bpf/progs/loop2.c | 4 + tools/testing/selftests/bpf/progs/loop3.c | 4 + tools/testing/selftests/bpf/progs/loop4.c | 2 - tools/testing/selftests/bpf/progs/loop5.c | 2 - tools/testing/selftests/bpf/progs/netcnt_prog.c | 2 - tools/testing/selftests/bpf/progs/pyperf.h | 2 - .../testing/selftests/bpf/progs/sample_map_ret0.c | 2 - tools/testing/selftests/bpf/progs/sendmsg4_prog.c | 4 + tools/testing/selftests/bpf/progs/sendmsg6_prog.c | 4 + .../selftests/bpf/progs/socket_cookie_prog.c | 4 + .../selftests/bpf/progs/sockmap_parse_prog.c | 4 + .../selftests/bpf/progs/sockmap_tcp_msg_prog.c | 4 + .../selftests/bpf/progs/sockmap_verdict_prog.c | 4 + .../testing/selftests/bpf/progs/sockopt_inherit.c | 2 - tools/testing/selftests/bpf/progs/sockopt_multi.c | 2 - tools/testing/selftests/bpf/progs/sockopt_sk.c | 2 - tools/testing/selftests/bpf/progs/strobemeta.h | 2 - tools/testing/selftests/bpf/progs/tailcall1.c | 2 - tools/testing/selftests/bpf/progs/tailcall2.c | 2 - tools/testing/selftests/bpf/progs/tailcall3.c | 2 - tools/testing/selftests/bpf/progs/tailcall4.c | 2 - tools/testing/selftests/bpf/progs/tailcall5.c | 2 - tools/testing/selftests/bpf/progs/tcp_rtt.c | 2 - .../testing/selftests/bpf/progs/test_adjust_tail.c | 2 - .../selftests/bpf/progs/test_attach_probe.c | 2 - tools/testing/selftests/bpf/progs/test_btf_haskv.c | 2 - tools/testing/selftests/bpf/progs/test_btf_newkv.c | 2 - tools/testing/selftests/bpf/progs/test_btf_nokv.c | 2 - .../testing/selftests/bpf/progs/test_core_extern.c | 2 - .../selftests/bpf/progs/test_core_reloc_arrays.c | 4 + .../bpf/progs/test_core_reloc_bitfields_direct.c | 4 + .../bpf/progs/test_core_reloc_bitfields_probed.c | 4 + .../bpf/progs/test_core_reloc_existence.c | 4 + .../selftests/bpf/progs/test_core_reloc_flavors.c | 4 + .../selftests/bpf/progs/test_core_reloc_ints.c | 4 + .../selftests/bpf/progs/test_core_reloc_kernel.c | 4 + .../selftests/bpf/progs/test_core_reloc_misc.c | 4 + .../selftests/bpf/progs/test_core_reloc_mods.c | 4 + .../selftests/bpf/progs/test_core_reloc_nesting.c | 4 + .../bpf/progs/test_core_reloc_primitives.c | 4 + .../bpf/progs/test_core_reloc_ptr_as_arr.c | 4 + .../selftests/bpf/progs/test_core_reloc_size.c | 4 + .../selftests/bpf/progs/test_get_stack_rawtp.c | 2 - .../testing/selftests/bpf/progs/test_global_data.c | 2 - .../selftests/bpf/progs/test_global_func1.c | 2 - .../selftests/bpf/progs/test_global_func3.c | 2 - .../selftests/bpf/progs/test_global_func5.c | 2 - .../selftests/bpf/progs/test_global_func6.c | 2 - .../selftests/bpf/progs/test_global_func7.c | 2 - tools/testing/selftests/bpf/progs/test_l4lb.c | 4 + .../selftests/bpf/progs/test_l4lb_noinline.c | 4 + .../selftests/bpf/progs/test_lirc_mode2_kern.c | 2 - .../selftests/bpf/progs/test_lwt_ip_encap.c | 4 + .../selftests/bpf/progs/test_lwt_seg6local.c | 4 + .../testing/selftests/bpf/progs/test_map_in_map.c | 2 - tools/testing/selftests/bpf/progs/test_map_lock.c | 2 - tools/testing/selftests/bpf/progs/test_mmap.c | 2 - tools/testing/selftests/bpf/progs/test_obj_id.c | 2 - tools/testing/selftests/bpf/progs/test_overhead.c | 4 + .../testing/selftests/bpf/progs/test_perf_buffer.c | 2 - tools/testing/selftests/bpf/progs/test_pinning.c | 2 - .../selftests/bpf/progs/test_pinning_invalid.c | 2 - .../testing/selftests/bpf/progs/test_pkt_access.c | 4 + .../selftests/bpf/progs/test_pkt_md_access.c | 2 - .../testing/selftests/bpf/progs/test_probe_user.c | 4 + .../selftests/bpf/progs/test_queue_stack_map.h | 2 - .../testing/selftests/bpf/progs/test_rdonly_maps.c | 2 - tools/testing/selftests/bpf/progs/test_seg6_loop.c | 4 + .../bpf/progs/test_select_reuseport_kern.c | 4 + .../selftests/bpf/progs/test_send_signal_kern.c | 2 - .../selftests/bpf/progs/test_sk_lookup_kern.c | 4 + .../selftests/bpf/progs/test_skb_cgroup_id_kern.c | 2 - tools/testing/selftests/bpf/progs/test_skb_ctx.c | 2 - tools/testing/selftests/bpf/progs/test_skeleton.c | 2 - .../selftests/bpf/progs/test_sock_fields_kern.c | 4 + tools/testing/selftests/bpf/progs/test_spin_lock.c | 2 - .../selftests/bpf/progs/test_stacktrace_build_id.c | 2 - .../selftests/bpf/progs/test_stacktrace_map.c | 2 - .../selftests/bpf/progs/test_sysctl_loop1.c | 2 - .../selftests/bpf/progs/test_sysctl_loop2.c | 2 - .../testing/selftests/bpf/progs/test_sysctl_prog.c | 2 - tools/testing/selftests/bpf/progs/test_tc_edt.c | 4 + tools/testing/selftests/bpf/progs/test_tc_tunnel.c | 4 + .../bpf/progs/test_tcp_check_syncookie_kern.c | 4 + .../testing/selftests/bpf/progs/test_tcp_estats.c | 2 - .../testing/selftests/bpf/progs/test_tcpbpf_kern.c | 4 + .../selftests/bpf/progs/test_tcpnotify_kern.c | 4 + .../testing/selftests/bpf/progs/test_tracepoint.c | 2 - .../testing/selftests/bpf/progs/test_tunnel_kern.c | 4 + .../selftests/bpf/progs/test_verif_scale1.c | 2 - .../selftests/bpf/progs/test_verif_scale2.c | 2 - .../selftests/bpf/progs/test_verif_scale3.c | 2 - tools/testing/selftests/bpf/progs/test_xdp.c | 4 + .../testing/selftests/bpf/progs/test_xdp_bpf2bpf.c | 2 - tools/testing/selftests/bpf/progs/test_xdp_loop.c | 4 + tools/testing/selftests/bpf/progs/test_xdp_meta.c | 2 - .../selftests/bpf/progs/test_xdp_noinline.c | 4 + .../selftests/bpf/progs/test_xdp_redirect.c | 2 - tools/testing/selftests/bpf/progs/test_xdp_vlan.c | 4 + tools/testing/selftests/bpf/progs/xdp_dummy.c | 2 - .../testing/selftests/bpf/progs/xdp_redirect_map.c | 2 - tools/testing/selftests/bpf/progs/xdp_tx.c | 2 - tools/testing/selftests/bpf/progs/xdping_kern.c | 4 + tools/testing/selftests/bpf/test_cpp.cpp | 6 +- tools/testing/selftests/bpf/test_hashmap.c | 2 - tools/testing/selftests/bpf/test_progs.h | 2 - tools/testing/selftests/bpf/test_sock.c | 2 - tools/testing/selftests/bpf/test_sockmap_kern.h | 4 + tools/testing/selftests/bpf/test_sysctl.c | 2 - tools/testing/selftests/bpf/trace_helpers.h | 2 - 238 files changed, 388 insertions(+), 369 deletions(-)

4 years, 3 months

2
19
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror January 2020