- Linux-kselftest-mirror - lists.linaro.org

[PATCH v2 0/2] firmware: selftest for request_firmware_into_buf

by Scott Branden

This patch series adds kernel selftest of request_firmware_into_buf. The API was added to the kernel previously untested. Changes from v1: - Dropped demonstration patch for a race condition discovered while testing request_firmare_into_buf. The new test exposes a kernel opps with the firmware fallback mechanism that may be fixed separate from these tests. - minor whitespace formatting in patch - added Ack's - added "s" in commit message (changed selftest: to selftests:) Scott Branden (2): test_firmware: add support for request_firmware_into_buf selftests: firmware: Add request_firmware_into_buf tests lib/test_firmware.c | 50 +++++++++++++++- .../selftests/firmware/fw_filesystem.sh | 57 ++++++++++++++++++- tools/testing/selftests/firmware/fw_lib.sh | 11 ++++ 3 files changed, 114 insertions(+), 4 deletions(-) -- 2.17.1

6 years, 3 months

1
2
0 0

[PATCH] selftests: net: add missing NFT_FWD_NETDEV to config

by Anders Roxell

When running xfrm_policy.sh we see the following # sysctl cannot stat /proc/sys/net/ipv4/conf/eth1/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv4/conf/eth1/forwarding # # sysctl cannot stat /proc/sys/net/ipv4/conf/veth0/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv4/conf/veth0/forwarding # # sysctl cannot stat /proc/sys/net/ipv4/conf/eth1/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv4/conf/eth1/forwarding # # sysctl cannot stat /proc/sys/net/ipv4/conf/veth0/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv4/conf/veth0/forwarding # # sysctl cannot stat /proc/sys/net/ipv6/conf/eth1/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv6/conf/eth1/forwarding # # sysctl cannot stat /proc/sys/net/ipv6/conf/veth0/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv6/conf/veth0/forwarding # # sysctl cannot stat /proc/sys/net/ipv6/conf/eth1/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv6/conf/eth1/forwarding # # sysctl cannot stat /proc/sys/net/ipv6/conf/veth0/forwarding No such file or directory cannot: stat_/proc/sys/net/ipv6/conf/veth0/forwarding # # modprobe FATAL Module ip_tables not found in directory /lib/modules/5.3.0-rc5-next-20190820+ FATAL: Module_ip_tables # # iptables v1.6.2 can't initialize iptables table `filter' Table does not exist (do you need to insmod?) v1.6.2: can't_initialize # Rework to enable CONFIG_NF_TABLES_NETDEV and CONFIG_NFT_FWD_NETDEV. Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org> --- tools/testing/selftests/net/config | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/config b/tools/testing/selftests/net/config index b8503a8119b0..e30b0ae5d474 100644 --- a/tools/testing/selftests/net/config +++ b/tools/testing/selftests/net/config @@ -29,3 +29,5 @@ CONFIG_NET_SCH_FQ=m CONFIG_NET_SCH_ETF=m CONFIG_TEST_BLACKHOLE_DEV=m CONFIG_KALLSYMS=y +CONFIG_NF_TABLES_NETDEV=y +CONFIG_NFT_FWD_NETDEV=m -- 2.20.1

6 years, 3 months

2
1
0 0

[PATCH] selftests: bpf: install files test_xdp_vlan.sh

by Anders Roxell

When ./test_xdp_vlan_mode_generic.sh runs it complains that it can't find file test_xdp_vlan.sh. # selftests: bpf: test_xdp_vlan_mode_generic.sh # ./test_xdp_vlan_mode_generic.sh: line 9: ./test_xdp_vlan.sh: No such file or directory Rework so that test_xdp_vlan.sh gets installed, added to the variable TEST_PROGS_EXTENDED. Fixes: d35661fcf95d ("selftests/bpf: add wrapper scripts for test_xdp_vlan.sh") Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org> --- tools/testing/selftests/bpf/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 1faad0c3c3c9..d7968e20463c 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -68,7 +68,8 @@ TEST_PROGS := test_kmod.sh \ TEST_PROGS_EXTENDED := with_addr.sh \ with_tunnels.sh \ tcp_client.py \ - tcp_server.py + tcp_server.py \ + test_xdp_vlan.sh # Compile but not part of 'make run_tests' TEST_GEN_PROGS_EXTENDED = test_libbpf_open test_sock_addr test_skb_cgroup_id_user \ -- 2.20.1

6 years, 3 months

3
2
0 0

[PATCH] selftests: bpf: add config fragment BPF_JIT

by Anders Roxell

When running test_kmod.sh the following shows up # sysctl cannot stat /proc/sys/net/core/bpf_jit_enable No such file or directory cannot: stat_/proc/sys/net/core/bpf_jit_enable # # sysctl cannot stat /proc/sys/net/core/bpf_jit_harden No such file or directory cannot: stat_/proc/sys/net/core/bpf_jit_harden # Rework to enable CONFIG_BPF_JIT to solve "No such file or directory" Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org> --- tools/testing/selftests/bpf/config | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config index f7a0744db31e..5dc109f4c097 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -34,3 +34,4 @@ CONFIG_NET_MPLS_GSO=m CONFIG_MPLS_ROUTING=m CONFIG_MPLS_IPTUNNEL=m CONFIG_IPV6_SIT=m +CONFIG_BPF_JIT=y -- 2.20.1

6 years, 3 months

2
1
0 0

[PATCH v13 00/18] kunit: introduce KUnit, the Linux kernel unit testing framework

by Brendan Higgins

## TL;DR This revision addresses comments from Stephen and Bjorn Helgaas. Most changes are pretty minor stuff that doesn't affect the API in anyway. One significant change, however, is that I added support for freeing kunit_resource managed resources before the test case is finished via kunit_resource_destroy(). Additionally, Bjorn pointed out that I broke KUnit on certain configurations (like the default one for x86, whoops). Based on Stephen's feedback on the previous change, I think we are pretty close. I am not expecting any significant changes from here on out. ## Background This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel. Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM (however, KUnit still allows you to run tests on test machines or in VMs if you want[1]) and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in about a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded). KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more. ### What's so special about unit testing? A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code. ### Is KUnit trying to replace other testing frameworks for the kernel? No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed. ### More information on KUnit There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here[2]. Additionally for convenience, I have applied these patches to a branch[3]. The repo may be cloned with: git clone https://kunit.googlesource.com/linux This patchset is on the kunit/rfc/v5.3/v13 branch. ## Changes Since Last Version - Added support for freeing kunit_resources (KUnit managed resources) via kunit_resource_destroy() as suggested by Stephen. - Promoted WARN() after __noreturn function to BUG() in "[PATCH v13 09/18] kunit: test: add support for test abort" as suggested by Stephen. - Dropped concept of death test since I am not actually using it yet as pointed out by Stephen. - Replaced usage of warn_slowpath_fmt with WARN in kunit_do_assertion since warn_slowpath_fmt is not available on some build configurations, as pointed out by Bjorn. - Lots of other minor changes suggested by Stephen. [1] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kuni… [2] https://google.github.io/kunit-docs/third_party/kernel/docs/ [3] https://kunit.googlesource.com/linux/+/kunit/rfc/v5.3/v13 -- 2.23.0.rc1.153.gdeed80330f-goog

6 years, 3 months

3
30
0 0

general protection fault in tls_setsockopt

by syzbot

Hello, syzbot found the following crash on: HEAD commit: a131c2bf Merge tag 'acpi-5.3-rc1-2' of git://git.kernel.or.. git tree: net-next console output: https://syzkaller.appspot.com/x/log.txt?x=1603e9c0600000 kernel config: https://syzkaller.appspot.com/x/.config?x=8bff73c5ba9e876 dashboard link: https://syzkaller.appspot.com/bug?extid=23d9570edec63669d890 compiler: gcc (GCC) 9.0.0 20181231 (experimental) syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13560870600000 The bug was bisected to: commit 7c85c448e7d74c4ddd759440a2141eab663567cf Author: Prashant Bhole <bhole_prashant_q7(a)lab.ntt.co.jp> Date: Tue Oct 9 01:04:54 2018 +0000 selftests/bpf: test_verifier, check bpf_map_lookup_elem access in bpf prog bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=17000114600000 final crash: https://syzkaller.appspot.com/x/report.txt?x=14800114600000 console output: https://syzkaller.appspot.com/x/log.txt?x=10800114600000 IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+23d9570edec63669d890(a)syzkaller.appspotmail.com Fixes: 7c85c448e7d7 ("selftests/bpf: test_verifier, check bpf_map_lookup_elem access in bpf prog") kasan: CONFIG_KASAN_INLINE enabled kasan: GPF could be caused by NULL-ptr deref or user memory access general protection fault: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 9098 Comm: syz-executor.2 Not tainted 5.2.0+ #65 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:tls_setsockopt+0x87/0x8c0 net/tls/tls_main.c:592 Code: 80 3c 02 00 0f 85 a5 07 00 00 49 8b 84 24 b0 06 00 00 48 ba 00 00 00 00 00 fc ff df 48 8d b8 88 00 00 00 48 89 f9 48 c1 e9 03 <80> 3c 11 00 0f 85 85 07 00 00 41 89 d8 4c 89 f1 44 89 ea 44 89 fe RSP: 0018:ffff88809b2e7d30 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000011 RDX: dffffc0000000000 RSI: ffffffff86186a84 RDI: 0000000000000088 RBP: ffff88809b2e7d80 R08: ffff888098896480 R09: ffff888098896d08 R10: ffffed1015d06c83 R11: ffff8880ae83641b R12: ffff888099d86d00 R13: 000000000000001f R14: 0000000020000340 R15: 0000000000000006 FS: 00007f29b1f6b700(0000) GS:ffff8880ae800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffb5492d518 CR3: 00000000a49ea000 CR4: 00000000001406f0 Call Trace: sock_common_setsockopt+0x94/0xd0 net/core/sock.c:3130 __sys_setsockopt+0x253/0x4b0 net/socket.c:2080 __do_sys_setsockopt net/socket.c:2096 [inline] __se_sys_setsockopt net/socket.c:2093 [inline] __x64_sys_setsockopt+0xbe/0x150 net/socket.c:2093 do_syscall_64+0xfd/0x680 arch/x86/entry/common.c:296 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x459819 Code: fd b7 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 cb b7 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007f29b1f6ac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000036 RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000000459819 RDX: 000000000000001f RSI: 0000000000000006 RDI: 0000000000000005 RBP: 000000000075bf20 R08: 0000000000000004 R09: 0000000000000000 R10: 0000000020000340 R11: 0000000000000246 R12: 00007f29b1f6b6d4 R13: 00000000004c7cb9 R14: 00000000004dd878 R15: 00000000ffffffff Modules linked in: ---[ end trace 1e30f09ab6b57d8c ]--- RIP: 0010:tls_setsockopt+0x87/0x8c0 net/tls/tls_main.c:592 Code: 80 3c 02 00 0f 85 a5 07 00 00 49 8b 84 24 b0 06 00 00 48 ba 00 00 00 00 00 fc ff df 48 8d b8 88 00 00 00 48 89 f9 48 c1 e9 03 <80> 3c 11 00 0f 85 85 07 00 00 41 89 d8 4c 89 f1 44 89 ea 44 89 fe RSP: 0018:ffff88809b2e7d30 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000011 RDX: dffffc0000000000 RSI: ffffffff86186a84 RDI: 0000000000000088 RBP: ffff88809b2e7d80 R08: ffff888098896480 R09: ffff888098896d08 R10: ffffed1015d06c83 R11: ffff8880ae83641b R12: ffff888099d86d00 R13: 000000000000001f R14: 0000000020000340 R15: 0000000000000006 FS: 00007f29b1f6b700(0000) GS:ffff8880ae800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000007126b4 CR3: 00000000a49ea000 CR4: 00000000001406f0 --- This bug is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller(a)googlegroups.com. syzbot will keep track of this bug report. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. For information about bisection process see: https://goo.gl/tpsmEJ#bisection syzbot can test patches for this bug, for details see: https://goo.gl/tpsmEJ#testing-patches

6 years, 3 months

2
1
0 0

[PATCH] selftests: use "$(MAKE)" instead of "make"

by Ilya Leoshkevich

When doing "make kselftest TARGETS=bpf -j12", bpf progs end up being compiled sequentially and thus slowly. The reason is that parent make (tools/testing/selftests/Makefile) does not share its jobserver with child make (tools/testing/selftests/bpf/Makefile), therefore the latter runs with -j1. Change all instances of "make" to "$(MAKE)", so that the whole make hierarchy runs using a single jobserver. Signed-off-by: Ilya Leoshkevich <iii(a)linux.ibm.com> --- I tested this with: make kselftest make -C tools/testing/selftests make -C tools/testing/selftests/bpf Unfortunately, in my setup a lot of tests fail for a number of reasons. However, this change does not make it worse. tools/testing/selftests/Makefile | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 25b43a8c2b15..c3feccb99ff5 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -126,9 +126,9 @@ endif # in the default INSTALL_HDR_PATH usr/include. khdr: ifeq (1,$(DEFAULT_INSTALL_HDR_PATH)) - make --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install + $(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install else - make --no-builtin-rules INSTALL_HDR_PATH=$$BUILD/usr \ + $(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$BUILD/usr \ ARCH=$(ARCH) -C $(top_srcdir) headers_install endif @@ -136,35 +136,35 @@ all: khdr @for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ mkdir $$BUILD_TARGET -p; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET;\ done; run_tests: all @for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET run_tests;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET run_tests;\ done; hotplug: @for TARGET in $(TARGETS_HOTPLUG); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET;\ done; run_hotplug: hotplug @for TARGET in $(TARGETS_HOTPLUG); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET run_full_test;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET run_full_test;\ done; clean_hotplug: @for TARGET in $(TARGETS_HOTPLUG); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET clean;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET clean;\ done; run_pstore_crash: - make -C pstore run_crash + $(MAKE) -C pstore run_crash # Use $BUILD as the default install root. $BUILD points to the # right output location for the following cases: @@ -184,7 +184,7 @@ ifdef INSTALL_PATH install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/ @for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \ done; @# Ask all targets to emit their test scripts @@ -203,7 +203,7 @@ ifdef INSTALL_PATH echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \ echo "cd $$TARGET" >> $(ALL_SCRIPT); \ echo -n "run_many" >> $(ALL_SCRIPT); \ - make -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \ + $(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \ echo "" >> $(ALL_SCRIPT); \ echo "cd \$$ROOT" >> $(ALL_SCRIPT); \ done; @@ -216,7 +216,7 @@ endif clean: @for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ - make OUTPUT=$$BUILD_TARGET -C $$TARGET clean;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET clean;\ done; .PHONY: khdr all run_tests hotplug run_hotplug clean_hotplug run_pstore_crash install clean -- 2.21.0

6 years, 3 months

2
1
0 0

[PATCH ARM] selftests, arm64: fix uninitialized symbol in tags_test.c

by Andrey Konovalov

Fix tagged_ptr not being initialized when TBI is not enabled. Dan Carpenter <dan.carpenter(a)oracle.com> Signed-off-by: Andrey Konovalov <andreyknvl(a)google.com> --- tools/testing/selftests/arm64/tags_test.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/arm64/tags_test.c b/tools/testing/selftests/arm64/tags_test.c index 22a1b266e373..5701163460ef 100644 --- a/tools/testing/selftests/arm64/tags_test.c +++ b/tools/testing/selftests/arm64/tags_test.c @@ -14,15 +14,17 @@ int main(void) { static int tbi_enabled = 0; - struct utsname *ptr, *tagged_ptr; + unsigned long tag = 0; + struct utsname *ptr; int err; if (prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE, 0, 0, 0) == 0) tbi_enabled = 1; ptr = (struct utsname *)malloc(sizeof(*ptr)); if (tbi_enabled) - tagged_ptr = (struct utsname *)SET_TAG(ptr, 0x42); - err = uname(tagged_ptr); + tag = 0x42; + ptr = (struct utsname *)SET_TAG(ptr, tag); + err = uname(ptr); free(ptr); return err; -- 2.23.0.rc1.153.gdeed80330f-goog

6 years, 3 months

2
5
0 0

[bug report] selftests, arm64: add a selftest for passing tagged pointers to kernel

by Dan Carpenter

Hello Andrey Konovalov, The patch 9ce1263033cd: "selftests, arm64: add a selftest for passing tagged pointers to kernel" from Jul 23, 2019, leads to the following static checker warning: ./tools/testing/selftests/arm64/tags_test.c:25 main() error: uninitialized symbol 'tagged_ptr'. tools/testing/selftests/arm64/tags_test.c 14 int main(void) 15 { 16 static int tbi_enabled = 0; 17 struct utsname *ptr, *tagged_ptr; ^^^^^^^^^^ 18 int err; 19 20 if (prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE, 0, 0, 0) == 0) 21 tbi_enabled = 1; 22 ptr = (struct utsname *)malloc(sizeof(*ptr)); 23 if (tbi_enabled) 24 tagged_ptr = (struct utsname *)SET_TAG(ptr, 0x42); ^^^^^^^^^^^^^^^ No else path. 25 err = uname(tagged_ptr); 26 free(ptr); 27 28 return err; 29 } regards, dan carpenter

6 years, 3 months

2
1
0 0

[RFC PATCH v2 0/5] hugetlb_cgroup: Add hugetlb_cgroup reservation limits

by Mina Almasry

Problem: Currently tasks attempting to allocate more hugetlb memory than is available get a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1]. However, if a task attempts to allocate hugetlb memory only more than its hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call, but will SIGBUS the task when it attempts to fault the memory in. We have developers interested in using hugetlb_cgroups, and they have expressed dissatisfaction regarding this behavior. We'd like to improve this behavior such that tasks violating the hugetlb_cgroup limits get an error on mmap/shmget time, rather than getting SIGBUS'd when they try to fault the excess memory in. The underlying problem is that today's hugetlb_cgroup accounting happens at hugetlb memory *fault* time, rather than at *reservation* time. Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and the offending task gets SIGBUS'd. Proposed Solution: A new page counter named hugetlb.xMB.reservation_[limit|usage]_in_bytes. This counter has slightly different semantics than hugetlb.xMB.[limit|usage]_in_bytes: - While usage_in_bytes tracks all *faulted* hugetlb memory, reservation_usage_in_bytes tracks all *reserved* hugetlb memory. - If a task attempts to reserve more memory than limit_in_bytes allows, the kernel will allow it to do so. But if a task attempts to reserve more memory than reservation_limit_in_bytes, the kernel will fail this reservation. This proposal is implemented in this patch, with tests to verify functionality and show the usage. Alternatives considered: 1. A new cgroup, instead of only a new page_counter attached to the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code duplication with hugetlb_cgroup. Keeping hugetlb related page counters under hugetlb_cgroup seemed cleaner as well. 2. Instead of adding a new counter, we considered adding a sysctl that modifies the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at reservation time rather than fault time. Adding a new page_counter seems better as userspace could, if it wants, choose to enforce different cgroups differently: one via limit_in_bytes, and another via reservation_limit_in_bytes. This could be very useful if you're transitioning how hugetlb memory is partitioned on your system one cgroup at a time, for example. Also, someone may find usage for both limit_in_bytes and reservation_limit_in_bytes concurrently, and this approach gives them the option to do so. Caveats: 1. This support is implemented for cgroups-v1. I have not tried hugetlb_cgroups with cgroups v2, and AFAICT it's not supported yet. This is largely because we use cgroups-v1 for now. If required, I can add hugetlb_cgroup support to cgroups v2 in this patch or a follow up. 2. Most complicated bit of this patch I believe is: where to store the pointer to the hugetlb_cgroup to uncharge at unreservation time? Normally the cgroup pointers hang off the struct page. But, with hugetlb_cgroup reservations, one task can reserve a specific page and another task may fault it in (I believe), so storing the pointer in struct page is not appropriate. Proposed approach here is to store the pointer in the resv_map. See patch for details. Signed-off-by: Mina Almasry <almasrymina(a)google.com> [1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html Changes in v2: - Split the patch into a 5 patch series. - Fixed patch subject. Mina Almasry (5): hugetlb_cgroup: Add hugetlb_cgroup reservation counter hugetlb_cgroup: add interface for charge/uncharge hugetlb reservations hugetlb_cgroup: add reservation accounting for private mappings hugetlb_cgroup: add accounting for shared mappings hugetlb_cgroup: Add hugetlb_cgroup reservation tests include/linux/hugetlb.h | 10 +- include/linux/hugetlb_cgroup.h | 19 +- mm/hugetlb.c | 256 ++++++++-- mm/hugetlb_cgroup.c | 153 +++++- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 4 + .../selftests/vm/charge_reserved_hugetlb.sh | 438 ++++++++++++++++++ .../selftests/vm/write_hugetlb_memory.sh | 22 + .../testing/selftests/vm/write_to_hugetlbfs.c | 252 ++++++++++ 9 files changed, 1087 insertions(+), 68 deletions(-) create mode 100755 tools/testing/selftests/vm/charge_reserved_hugetlb.sh create mode 100644 tools/testing/selftests/vm/write_hugetlb_memory.sh create mode 100644 tools/testing/selftests/vm/write_to_hugetlbfs.c -- 2.23.0.rc1.153.gdeed80330f-goog

6 years, 4 months

2
17
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror