April 2024 - Linux-kselftest-mirror

[PATCH v3 00/29] riscv control-flow integrity for usermode

by Deepak Gupta

Sending out v3 for cpu assisted riscv user mode control flow integrity. v2 [9] was sent a week ago for this riscv usermode control flow integrity enabling. RFC patchset was (v1) early this year (January) [7]. changes in v3 -------------- envcfg: logic to pick up base envcfg had a bug where `ENVCFG_CBZE` could have been picked on per task basis, even though CPU didn't implement it. Fixed in this series. dt-bindings: As suggested, split into separate commit. fixed the messaging that spec is in public review arch_is_shadow_stack change: arch_is_shadow_stack changed to vma_is_shadow_stack hwprobe: zicfiss / zicfilp if present will get enumerated in hwprobe selftests: As suggested, added object and binary filenames to .gitignore Selftest binary anyways need to be compiled with cfi enabled compiler which will make sure that landing pad and shadow stack are enabled. Thus removed separate enable/disable tests. Cleaned up tests a bit. changes in v2 --------------- As part of testing effort, compiled a rootfs with shadow stack and landing pad enabled (libraries and binaries) and booted to shell. As part of long running tests, I have been able to run some spec 2006 benchmarks [8] (here link is provided only for list of benchmarks that were tested for long running tests, excel sheet provided here actually is for some static stats like code size growth on spec binaries). Thus converting from RFC to regular patchset. Securing control-flow integrity for usermode requires following - Securing forward control flow : All callsites must reach reach a target that they actually intend to reach. - Securing backward control flow : All function returns must return to location where they were called from. This patch series use riscv cpu extension `zicfilp` [2] to secure forward control flow and `zicfiss` [2] to secure backward control flow. `zicfilp` enforces that all indirect calls or jmps must land on a landing pad instr and label embedded in landing pad instr must match a value programmed in `x7` register (at callsite via compiler). `zicfiss` introduces shadow stack which can only be writeable via shadow stack instructions (sspush and ssamoswap) and thus can't be tampered with via inadvertent stores. More details about extension can be read from [2] and there are details in documentation as well (in this patch series). Using config `CONFIG_RISCV_USER_CFI`, kernel support for riscv control flow integrity for user mode programs can be compiled in the kernel. Enabling of control flow integrity for user programs is left to user runtime (specifically expected from dynamic loader). There has been a lot of earlier discussion on the enabling topic around x86 shadow stack enabling [3, 4, 5] and overall consensus had been to let dynamic loader (or usermode) to decide for enabling the feature. This patch series introduces arch agnostic `prctls` to enable shadow stack and indirect branch tracking. And implements them on riscv. arm64 is expected to implement shadow stack part of these arch agnostic `prctls` [6] Changes since last time *********************** Spec changes ------------ - Forward cfi spec has become much simpler. `lpad` instruction is pseudo for `auipc rd, <20bit_imm>`. `lpad` checks x7 against 20bit embedded in instr. Thus label width is 20bit. - Shadow stack management instructions are reduced to sspush - to push x1/x5 on shadow stack sspopchk - pops from shadow stack and comapres with x1/x5. ssamoswap - atomically swap value on shadow stack. rdssp - reads current shadow stack pointer - Shadow stack accesses on readonly memory always raise AMO/store page fault. `sspopchk` is load but if underlying page is readonly, it'll raise a store page fault. It simplifies hardware and kernel for COW handling for shadow stack pages. - riscv defines a new exception type `software check exception` and control flow violations raise software check exception. - enabling controls for shadow stack and landing are in xenvcfg CSR and controls lower privilege mode enabling. As an example senvcfg controls enabling for U and menvcfg controls enabling for S mode. core mm shadow stack enabling ----------------------------- Shadow stack for x86 usermode are now in mainline and thus this patch series builds on top of that for arch-agnostic mm related changes. Big thanks and shout out to Rick Edgecombe for that. selftests --------- Created some minimal selftests to test the patch series. [1] - https://lore.kernel.org/lkml/20230213045351.3945824-1-debug@rivosinc.com/ [2] - https://github.com/riscv/riscv-cfi [3] - https://lore.kernel.org/lkml/ZWHcBq0bJ+15eeKs@finisterre.sirena.org.uk/T/#m… [4] - https://lore.kernel.org/all/20220130211838.8382-1-rick.p.edgecombe@intel.co… [5] - https://lore.kernel.org/lkml/CAHk-=wgP5mk3poVeejw16Asbid0ghDt4okHnWaWKLBkRh… [6] - https://lore.kernel.org/linux-mm/20231122-arm64-gcs-v7-2-201c483bd775@kerne… [7] - https://lore.kernel.org/lkml/20240125062739.1339782-1-debug@rivosinc.com/ [8] - https://docs.google.com/spreadsheets/d/1_cHGH4ctNVvFRiS7hW9dEGKtXLAJ3aX4Z_i… [9] - https://lore.kernel.org/lkml/20240329044459.3990638-1-debug@rivosinc.com/

7 months, 1 week

9
81
0 0

[PATCH v10 0/5] Introduce mseal

by jeffxu＠chromium.org

From: Jeff Xu <jeffxu(a)chromium.org> This is V10 version, it rebases v9 patch to 6.9.rc3. We also applied and tested mseal() in chrome and chromebook. ------------------------------------------------------------------ This patchset proposes a new mseal() syscall for the Linux kernel. In a nutshell, mseal() protects the VMAs of a given virtual memory range against modifications, such as changes to their permission bits. Modern CPUs support memory permissions, such as the read/write (RW) and no-execute (NX) bits. Linux has supported NX since the release of kernel version 2.6.8 in August 2004 [1]. The memory permission feature improves the security stance on memory corruption bugs, as an attacker cannot simply write to arbitrary memory and point the code to it. The memory must be marked with the X bit, or else an exception will occur. Internally, the kernel maintains the memory permissions in a data structure called VMA (vm_area_struct). mseal() additionally protects the VMA itself against modifications of the selected seal type. Memory sealing is useful to mitigate memory corruption issues where a corrupted pointer is passed to a memory management system. For example, such an attacker primitive can break control-flow integrity guarantees since read-only memory that is supposed to be trusted can become writable or .text pages can get remapped. Memory sealing can automatically be applied by the runtime loader to seal .text and .rodata pages and applications can additionally seal security critical data at runtime. A similar feature already exists in the XNU kernel with the VM_FLAGS_PERMANENT [3] flag and on OpenBSD with the mimmutable syscall [4]. Also, Chrome wants to adopt this feature for their CFI work [2] and this patchset has been designed to be compatible with the Chrome use case. Two system calls are involved in sealing the map: mmap() and mseal(). The new mseal() is an syscall on 64 bit CPU, and with following signature: int mseal(void addr, size_t len, unsigned long flags) addr/len: memory range. flags: reserved. mseal() blocks following operations for the given memory range. 1> Unmapping, moving to another location, and shrinking the size, via munmap() and mremap(), can leave an empty space, therefore can be replaced with a VMA with a new set of attributes. 2> Moving or expanding a different VMA into the current location, via mremap(). 3> Modifying a VMA via mmap(MAP_FIXED). 4> Size expansion, via mremap(), does not appear to pose any specific risks to sealed VMAs. It is included anyway because the use case is unclear. In any case, users can rely on merging to expand a sealed VMA. 5> mprotect() and pkey_mprotect(). 6> Some destructive madvice() behaviors (e.g. MADV_DONTNEED) for anonymous memory, when users don't have write permission to the memory. Those behaviors can alter region contents by discarding pages, effectively a memset(0) for anonymous memory. The idea that inspired this patch comes from Stephen Röttger’s work in V8 CFI [5]. Chrome browser in ChromeOS will be the first user of this API. Indeed, the Chrome browser has very specific requirements for sealing, which are distinct from those of most applications. For example, in the case of libc, sealing is only applied to read-only (RO) or read-execute (RX) memory segments (such as .text and .RELRO) to prevent them from becoming writable, the lifetime of those mappings are tied to the lifetime of the process. Chrome wants to seal two large address space reservations that are managed by different allocators. The memory is mapped RW- and RWX respectively but write access to it is restricted using pkeys (or in the future ARM permission overlay extensions). The lifetime of those mappings are not tied to the lifetime of the process, therefore, while the memory is sealed, the allocators still need to free or discard the unused memory. For example, with madvise(DONTNEED). However, always allowing madvise(DONTNEED) on this range poses a security risk. For example if a jump instruction crosses a page boundary and the second page gets discarded, it will overwrite the target bytes with zeros and change the control flow. Checking write-permission before the discard operation allows us to control when the operation is valid. In this case, the madvise will only succeed if the executing thread has PKEY write permissions and PKRU changes are protected in software by control-flow integrity. Although the initial version of this patch series is targeting the Chrome browser as its first user, it became evident during upstream discussions that we would also want to ensure that the patch set eventually is a complete solution for memory sealing and compatible with other use cases. The specific scenario currently in mind is glibc's use case of loading and sealing ELF executables. To this end, Stephen is working on a change to glibc to add sealing support to the dynamic linker, which will seal all non-writable segments at startup. Once this work is completed, all applications will be able to automatically benefit from these new protections. In closing, I would like to formally acknowledge the valuable contributions received during the RFC process, which were instrumental in shaping this patch: Jann Horn: raising awareness and providing valuable insights on the destructive madvise operations. Liam R. Howlett: perf optimization. Linus Torvalds: assisting in defining system call signature and scope. Theo de Raadt: sharing the experiences and insight gained from implementing mimmutable() in OpenBSD. MM perf benchmarks ================== This patch adds a loop in the mprotect/munmap/madvise(DONTNEED) to check the VMAs’ sealing flag, so that no partial update can be made, when any segment within the given memory range is sealed. To measure the performance impact of this loop, two tests are developed. [8] The first is measuring the time taken for a particular system call, by using clock_gettime(CLOCK_MONOTONIC). The second is using PERF_COUNT_HW_REF_CPU_CYCLES (exclude user space). Both tests have similar results. The tests have roughly below sequence: for (i = 0; i < 1000, i++) create 1000 mappings (1 page per VMA) start the sampling for (j = 0; j < 1000, j++) mprotect one mapping stop and save the sample delete 1000 mappings calculates all samples. Below tests are performed on Intel(R) Pentium(R) Gold 7505 @ 2.00GHz, 4G memory, Chromebook. Based on the latest upstream code: The first test (measuring time) syscall__ vmas t t_mseal delta_ns per_vma % munmap__ 1 909 944 35 35 104% munmap__ 2 1398 1502 104 52 107% munmap__ 4 2444 2594 149 37 106% munmap__ 8 4029 4323 293 37 107% munmap__ 16 6647 6935 288 18 104% munmap__ 32 11811 12398 587 18 105% mprotect 1 439 465 26 26 106% mprotect 2 1659 1745 86 43 105% mprotect 4 3747 3889 142 36 104% mprotect 8 6755 6969 215 27 103% mprotect 16 13748 14144 396 25 103% mprotect 32 27827 28969 1142 36 104% madvise_ 1 240 262 22 22 109% madvise_ 2 366 442 76 38 121% madvise_ 4 623 751 128 32 121% madvise_ 8 1110 1324 215 27 119% madvise_ 16 2127 2451 324 20 115% madvise_ 32 4109 4642 534 17 113% The second test (measuring cpu cycle) syscall__ vmas cpu cmseal delta_cpu per_vma % munmap__ 1 1790 1890 100 100 106% munmap__ 2 2819 3033 214 107 108% munmap__ 4 4959 5271 312 78 106% munmap__ 8 8262 8745 483 60 106% munmap__ 16 13099 14116 1017 64 108% munmap__ 32 23221 24785 1565 49 107% mprotect 1 906 967 62 62 107% mprotect 2 3019 3203 184 92 106% mprotect 4 6149 6569 420 105 107% mprotect 8 9978 10524 545 68 105% mprotect 16 20448 21427 979 61 105% mprotect 32 40972 42935 1963 61 105% madvise_ 1 434 497 63 63 115% madvise_ 2 752 899 147 74 120% madvise_ 4 1313 1513 200 50 115% madvise_ 8 2271 2627 356 44 116% madvise_ 16 4312 4883 571 36 113% madvise_ 32 8376 9319 943 29 111% Based on the result, for 6.8 kernel, sealing check adds 20-40 nano seconds, or around 50-100 CPU cycles, per VMA. In addition, I applied the sealing to 5.10 kernel: The first test (measuring time) syscall__ vmas t tmseal delta_ns per_vma % munmap__ 1 357 390 33 33 109% munmap__ 2 442 463 21 11 105% munmap__ 4 614 634 20 5 103% munmap__ 8 1017 1137 120 15 112% munmap__ 16 1889 2153 263 16 114% munmap__ 32 4109 4088 -21 -1 99% mprotect 1 235 227 -7 -7 97% mprotect 2 495 464 -30 -15 94% mprotect 4 741 764 24 6 103% mprotect 8 1434 1437 2 0 100% mprotect 16 2958 2991 33 2 101% mprotect 32 6431 6608 177 6 103% madvise_ 1 191 208 16 16 109% madvise_ 2 300 324 24 12 108% madvise_ 4 450 473 23 6 105% madvise_ 8 753 806 53 7 107% madvise_ 16 1467 1592 125 8 108% madvise_ 32 2795 3405 610 19 122% The second test (measuring cpu cycle) syscall__ nbr_vma cpu cmseal delta_cpu per_vma % munmap__ 1 684 715 31 31 105% munmap__ 2 861 898 38 19 104% munmap__ 4 1183 1235 51 13 104% munmap__ 8 1999 2045 46 6 102% munmap__ 16 3839 3816 -23 -1 99% munmap__ 32 7672 7887 216 7 103% mprotect 1 397 443 46 46 112% mprotect 2 738 788 50 25 107% mprotect 4 1221 1256 35 9 103% mprotect 8 2356 2429 72 9 103% mprotect 16 4961 4935 -26 -2 99% mprotect 32 9882 10172 291 9 103% madvise_ 1 351 380 29 29 108% madvise_ 2 565 615 49 25 109% madvise_ 4 872 933 61 15 107% madvise_ 8 1508 1640 132 16 109% madvise_ 16 3078 3323 245 15 108% madvise_ 32 5893 6704 811 25 114% For 5.10 kernel, sealing check adds 0-15 ns in time, or 10-30 CPU cycles, there is even decrease in some cases. It might be interesting to compare 5.10 and 6.8 kernel The first test (measuring time) syscall__ vmas t_5_10 t_6_8 delta_ns per_vma % munmap__ 1 357 909 552 552 254% munmap__ 2 442 1398 956 478 316% munmap__ 4 614 2444 1830 458 398% munmap__ 8 1017 4029 3012 377 396% munmap__ 16 1889 6647 4758 297 352% munmap__ 32 4109 11811 7702 241 287% mprotect 1 235 439 204 204 187% mprotect 2 495 1659 1164 582 335% mprotect 4 741 3747 3006 752 506% mprotect 8 1434 6755 5320 665 471% mprotect 16 2958 13748 10790 674 465% mprotect 32 6431 27827 21397 669 433% madvise_ 1 191 240 49 49 125% madvise_ 2 300 366 67 33 122% madvise_ 4 450 623 173 43 138% madvise_ 8 753 1110 357 45 147% madvise_ 16 1467 2127 660 41 145% madvise_ 32 2795 4109 1314 41 147% The second test (measuring cpu cycle) syscall__ vmas cpu_5_10 c_6_8 delta_cpu per_vma % munmap__ 1 684 1790 1106 1106 262% munmap__ 2 861 2819 1958 979 327% munmap__ 4 1183 4959 3776 944 419% munmap__ 8 1999 8262 6263 783 413% munmap__ 16 3839 13099 9260 579 341% munmap__ 32 7672 23221 15549 486 303% mprotect 1 397 906 509 509 228% mprotect 2 738 3019 2281 1140 409% mprotect 4 1221 6149 4929 1232 504% mprotect 8 2356 9978 7622 953 423% mprotect 16 4961 20448 15487 968 412% mprotect 32 9882 40972 31091 972 415% madvise_ 1 351 434 82 82 123% madvise_ 2 565 752 186 93 133% madvise_ 4 872 1313 442 110 151% madvise_ 8 1508 2271 763 95 151% madvise_ 16 3078 4312 1234 77 140% madvise_ 32 5893 8376 2483 78 142% From 5.10 to 6.8 munmap: added 250-550 ns in time, or 500-1100 in cpu cycle, per vma. mprotect: added 200-750 ns in time, or 500-1200 in cpu cycle, per vma. madvise: added 33-50 ns in time, or 70-110 in cpu cycle, per vma. In comparison to mseal, which adds 20-40 ns or 50-100 CPU cycles, the increase from 5.10 to 6.8 is significantly larger, approximately ten times greater for munmap and mprotect. When I discuss the mm performance with Brian Makin, an engineer worked on performance, it was brought to my attention that such a performance benchmarks, which measuring millions of mm syscall in a tight loop, may not accurately reflect real-world scenarios, such as that of a database service. Also this is tested using a single HW and ChromeOS, the data from another HW or distribution might be different. It might be best to take this data with a grain of salt. Change history: =============== V10: - rebase to 6.9.rc3 (no code change, resolve conflict only) - Stephen Röttger applied mseal() in Chrome code, and I tested it on chromebook, the mseal() is working as designed. V9: - remove mmap(PROT_SEAL) and mmap(MAP_SEALABLE) (Linus, Theo de Raadt) - Update mseal_test to check for prot bit (Liam R. Howlett) - Update documentation to give more detail on sealing check (Liam R. Howlett) - Add seal_elf test. - Add performance measure data. - mseal_test: fix arm build. https://lore.kernel.org/all/20240214151130.616240-1-jeffxu@chromium.org/ V8: - perf optimization in mmap. (Liam R. Howlett) - add one testcase (test_seal_zero_address) - Update mseal.rst to add note for MAP_SEALABLE. https://lore.kernel.org/lkml/20240131175027.3287009-1-jeffxu@chromium.org/ V7: - fix index.rst (Randy Dunlap) - fix arm build (Randy Dunlap) - return EPERM for blocked operations (Theo de Raadt) https://lore.kernel.org/linux-mm/20240122152905.2220849-2-jeffxu@chromium.o… V6: - Drop RFC from subject, Given Linus's general approval. - Adjust syscall number for mseal (main Jan.11/2024) - Code style fix (Matthew Wilcox) - selftest: use ksft macros (Muhammad Usama Anjum) - Document fix. (Randy Dunlap) https://lore.kernel.org/all/20240111234142.2944934-1-jeffxu@chromium.org/ V5: - fix build issue in mseal-Wire-up-mseal-syscall (Suggested by Linus Torvalds, and Greg KH) - updates on selftest. https://lore.kernel.org/lkml/20240109154547.1839886-1-jeffxu@chromium.org/#r V4: (Suggested by Linus Torvalds) - new signature: mseal(start,len,flags) - 32 bit is not supported. vm_seal is removed, use vm_flags instead. - single bit in vm_flags for sealed state. - CONFIG_MSEAL kernel config is removed. - single bit of PROT_SEAL in the "Prot" field of mmap(). Other changes: - update selftest (Suggested by Muhammad Usama Anjum) - update documentation. https://lore.kernel.org/all/20240104185138.169307-1-jeffxu@chromium.org/ V3: - Abandon per-syscall approach, (Suggested by Linus Torvalds). - Organize sealing types around their functionality, such as MM_SEAL_BASE, MM_SEAL_PROT_PKEY. - Extend the scope of sealing from calls originated in userspace to both kernel and userspace. (Suggested by Linus Torvalds) - Add seal type support in mmap(). (Suggested by Pedro Falcato) - Add a new sealing type: MM_SEAL_DISCARD_RO_ANON to prevent destructive operations of madvise. (Suggested by Jann Horn and Stephen Röttger) - Make sealed VMAs mergeable. (Suggested by Jann Horn) - Add MAP_SEALABLE to mmap() - Add documentation - mseal.rst https://lore.kernel.org/linux-mm/20231212231706.2680890-2-jeffxu@chromium.o… v2: Use _BITUL to define MM_SEAL_XX type. Use unsigned long for seal type in sys_mseal() and other functions. Remove internal VM_SEAL_XX type and convert_user_seal_type(). Remove MM_ACTION_XX type. Remove caller_origin(ON_BEHALF_OF_XX) and replace with sealing bitmask. Add more comments in code. Add a detailed commit message. https://lore.kernel.org/lkml/20231017090815.1067790-1-jeffxu@chromium.org/ v1: https://lore.kernel.org/lkml/20231016143828.647848-1-jeffxu@chromium.org/ ---------------------------------------------------------------- [1] https://kernelnewbies.org/Linux_2_6_8 [2] https://v8.dev/blog/control-flow-integrity [3] https://github.com/apple-oss-distributions/xnu/blob/1031c584a5e37aff177559b… [4] https://man.openbsd.org/mimmutable.2 [5] https://docs.google.com/document/d/1O2jwK4dxI3nRcOJuPYkonhTkNQfbmwdvxQMyXge… [6] https://lore.kernel.org/lkml/CAG48ez3ShUYey+ZAFsU2i1RpQn0a5eOs2hzQ426Fkcgnf… [7] https://lore.kernel.org/lkml/20230515130553.2311248-1-jeffxu@chromium.org/ [8] https://github.com/peaktocreek/mmperf Jeff Xu (5): mseal: Wire up mseal syscall mseal: add mseal syscall selftest mm/mseal memory sealing mseal:add documentation selftest mm/mseal read-only elf memory segment Documentation/userspace-api/index.rst | 1 + Documentation/userspace-api/mseal.rst | 199 ++ arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + include/linux/syscalls.h | 1 + include/uapi/asm-generic/unistd.h | 5 +- kernel/sys_ni.c | 1 + mm/Makefile | 4 + mm/internal.h | 37 + mm/madvise.c | 12 + mm/mmap.c | 31 +- mm/mprotect.c | 10 + mm/mremap.c | 31 + mm/mseal.c | 307 ++++ tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 2 + tools/testing/selftests/mm/mseal_test.c | 1836 +++++++++++++++++++ tools/testing/selftests/mm/seal_elf.c | 183 ++ 33 files changed, 2678 insertions(+), 3 deletions(-) create mode 100644 Documentation/userspace-api/mseal.rst create mode 100644 mm/mseal.c create mode 100644 tools/testing/selftests/mm/mseal_test.c create mode 100644 tools/testing/selftests/mm/seal_elf.c -- 2.44.0.683.g7961c838ac-goog

7 months, 1 week

15
54
0 0

selftests: clone3: clone3_cap_checkpoint_restore fails - Could not set CAP_CHECKPOINT_RESTORE

by Naresh Kamboju

selftests clone3 cap_checkpoint_restore fails on all devices. Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> [ 97.198602] audit: type=1701 audit(1651167820.383:12): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=1732 comm=\"clone3_cap_chec\" exe=\"/opt/kselftests/default-in-kernel/clone3/clone3_cap_checkpoint_restore\" sig=6 res=1 # selftests: clone3: clone3_cap_checkpoint_restore # TAP version 13 # 1..1 # # Starting 1 tests from 1 test cases. # # RUN global.clone3_cap_checkpoint_restore ... # # clone3_cap_checkpoint_restore.c:155:clone3_cap_checkpoint_restore:Child has PID 1733 # # clone3() syscall supported # cap_set_proc: Operation not permitted # # clone3_cap_checkpoint_restore.c:164:clone3_cap_checkpoint_restore:Expected set_capability() (-1) == 0 (0) # # clone3_cap_checkpoint_restore.c:165:clone3_cap_checkpoint_restore:Could not set CAP_CHECKPOINT_RESTORE # # clone3_cap_checkpoint_restore: Test terminated by assertion # # FAIL global.clone3_cap_checkpoint_restore # not ok 1 global.clone3_cap_checkpoint_restore # # FAILED: 0 / 1 tests passed. # # Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0 not ok 4 selftests: clone3: clone3_cap_checkpoint_restore # exit=1 Test details links, https://lkft.validation.linaro.org/scheduler/job/5812724#L2074 Test results comparison link, https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221107/te… https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221107/te… https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221107/te… metadata: git_ref: master git_repo: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next git_sha: d8e87774068af213ab5b058b1b114dc397b577aa git_describe: next-20221107 kernel_version: 6.1.0-rc3 kernel-config: https://builds.tuxbuild.com/2HChVOSNxUpvfAfLTnaK7zyTdUi/config build-url: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next/-/pipelines/687092361 artifact-location: https://builds.tuxbuild.com/2HChVOSNxUpvfAfLTnaK7zyTdUi toolchain: gcc-11 -- Linaro LKFT https://lkft.linaro.org

7 months, 1 week

3
3
0 0

[PATCH 0/3] kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing

by Helen Koike

Dear Kernel Community, This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a basic test pipeline triggered by code pushes to a GitLab-CI instance. This initial version includes static checks (checkpatch and smatch for now) and build tests across various architectures and configurations. It leverages an integrated cache for efficient build times and introduces a flexible 'scenarios' mechanism for subsystem-specific extensions. tl;dr: check this video to see a quick demo: https://youtu.be/TWiTjhjOuzg, but don't forget to check the "Motivation for this work" below. Your feedback, whether a simple thumbs up or down, is crucial to determine if it is worthwhile to pursue this initiative. GitLab is an Open Source platform that includes integrated CI/CD. The pipeline provided in this patch is designed to work out-of-the-box with any GitLab instance, including the gitlab.com Free Tier. If you reach the limits of the Free Tier, consider using community instances like https://gitlab.freedesktop.org/. Alternatively, you can set up a local runner for more flexibility. The bootstrap-gitlab-runner.sh script included with this patch simplifies this process, enabling you to run tests on your preferred infrastructure, including your own machine. For detailed information, please refer to the documentation included in the patch, or check the rendered version here: https://koike.pages.collabora.com/-/linux/-/jobs/298498/artifacts/artifacts… . Motivation for this Work ======================== We all know tests are a major topic in the community, so let's mention the specificities of this approach: 1. **Built-in User Interface:** GitLab CI/CD is growing in popularity and has an user-friendly interface. Our experience with the upstream DRM-CI in the kernel tree (see this blog post [https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-…] ) has provided insights into how such a system can benefit the wider community. 2. **Distributed Infrastructure:** The proposed GitLab-CI pipeline is designed with a distributed infrastructure model, being possible to run in any gitlab instance. 3. **Reduce regressions:** Fostering a culture where people habitually run validated tests and post their results can prevent many issues in post-merge tests. 4. **Collaborative Testing Environment:** The kernel community is already engaged in numerous testing efforts, including various GitLab-CI pipelines such as DRM-CI, which I maintain, along with other solutions like KernelCI and BPF-CI. This proposal is designed to further stimulate contributions to the evolving testing landscape. Our goal is to establish a comprehensive suite of common tools and files. 5. **Ownership of QA:** Discrepancies between kernel code and outdated tests often lead to misattributed failures, complicating regression tracking. This issue, often arising from neglected or deprioritized test updates, creates uncertainty about the source of failures. Adopting an "always green pipeline" approach, as detailed in this patch's documentation, encourages timely maintenance and validation of tests. This ensures that testing accurately reflects the current state of the kernel, thereby improving the effectiveness of our QA processes. Additionally, if we discover that this method isn't working for us, we can easily remove it from the codebase, as it is primarily contained within the ci/ folder. Future Work =========== **Expanding Static Checks:** We have the opportunity to integrate a variety of static analysis tools, including: - dtbs_checks - sparse - yamllint - dt-doc-validate - coccicheck **Adding Userspace Tests on VMs:** To further our testing, we can implement userspace tests that run on virtual machines (VMs), such as: - kselftests - kunit tests - Subsystem-specific tests, customizable in the scenarios. **Leveraging External Test Labs:** We can extend our testing to external labs, similar to what DRM-CI currently does. This includes: - Lava labs - Bare metal labs - Using KernelCI-provided labs **Other integrations** - Submit results to KCIDB **Lightweight Implementation for All Developers:** We aim to design these tests to be lightweight, ensuring developers with limited computing resources can still run essential tests. Resource-intensive tests can be set to trigger manually, rather than automatically, to accommodate diverse development environments. Chat Discussions ================ For those interested in further discussions: **Join Our Slack Channel:** We have a Slack channel, #gitlab-ci, on the KernelCI Slack instance https://kernelci.slack.com/ . Feel free to join and contribute to the conversation. The KernelCI team has weekly calls where we also discuss the GitLab-CI pipeline. **Acknowledgments:** A special thanks to Nikolai Kondrashov, Tales da Aparecida - both from Red Hat - and KernelCI community for their valuable feedback and support in this proposal. I eagerly await your thoughts and suggestions on this initiative. Also, if you want to see this initiave move faster, we are happy to discuss funding options. Best regards, Helen Koike Helen Koike (3): kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing kci-gitlab: Add documentation kci-gitlab: docs: Add images .gitlab-ci.yml | 2 + Documentation/ci/gitlab-ci/gitlab-ci.rst | 404 ++++++++++++++++++ .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 159752 bytes .../gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 91675 bytes .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes Documentation/index.rst | 7 + MAINTAINERS | 9 + ci/gitlab-ci/bootstrap-gitlab-runner.sh | 55 +++ ci/gitlab-ci/ci-scripts/build-docs.sh | 35 ++ ci/gitlab-ci/ci-scripts/build-kernel.sh | 35 ++ ci/gitlab-ci/ci-scripts/ici-functions.sh | 104 +++++ ci/gitlab-ci/ci-scripts/install-smatch.sh | 13 + .../ci-scripts/parse_commit_message.sh | 27 ++ ci/gitlab-ci/ci-scripts/run-checkpatch.sh | 19 + ci/gitlab-ci/ci-scripts/run-smatch.sh | 45 ++ ci/gitlab-ci/docker-compose.yaml | 18 + ci/gitlab-ci/linux.code-workspace | 11 + ci/gitlab-ci/yml/build.yml | 43 ++ ci/gitlab-ci/yml/cache.yml | 26 ++ ci/gitlab-ci/yml/container.yml | 36 ++ ci/gitlab-ci/yml/gitlab-ci.yml | 71 +++ ci/gitlab-ci/yml/kernel-combinations.yml | 18 + ci/gitlab-ci/yml/scenarios.yml | 12 + ci/gitlab-ci/yml/scenarios/file-systems.yml | 21 + ci/gitlab-ci/yml/scenarios/media.yml | 21 + ci/gitlab-ci/yml/scenarios/network.yml | 21 + ci/gitlab-ci/yml/static-checks.yml | 21 + 29 files changed, 1074 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 Documentation/ci/gitlab-ci/gitlab-ci.rst create mode 100644 Documentation/ci/gitlab-ci/images/job-matrix.png create mode 100644 Documentation/ci/gitlab-ci/images/new-project-runner.png create mode 100644 Documentation/ci/gitlab-ci/images/pipelines-on-push.png create mode 100644 Documentation/ci/gitlab-ci/images/the-pipeline.png create mode 100644 Documentation/ci/gitlab-ci/images/variables.png create mode 100755 ci/gitlab-ci/bootstrap-gitlab-runner.sh create mode 100755 ci/gitlab-ci/ci-scripts/build-docs.sh create mode 100755 ci/gitlab-ci/ci-scripts/build-kernel.sh create mode 100644 ci/gitlab-ci/ci-scripts/ici-functions.sh create mode 100755 ci/gitlab-ci/ci-scripts/install-smatch.sh create mode 100755 ci/gitlab-ci/ci-scripts/parse_commit_message.sh create mode 100755 ci/gitlab-ci/ci-scripts/run-checkpatch.sh create mode 100755 ci/gitlab-ci/ci-scripts/run-smatch.sh create mode 100644 ci/gitlab-ci/docker-compose.yaml create mode 100644 ci/gitlab-ci/linux.code-workspace create mode 100644 ci/gitlab-ci/yml/build.yml create mode 100644 ci/gitlab-ci/yml/cache.yml create mode 100644 ci/gitlab-ci/yml/container.yml create mode 100644 ci/gitlab-ci/yml/gitlab-ci.yml create mode 100644 ci/gitlab-ci/yml/kernel-combinations.yml create mode 100644 ci/gitlab-ci/yml/scenarios.yml create mode 100644 ci/gitlab-ci/yml/scenarios/file-systems.yml create mode 100644 ci/gitlab-ci/yml/scenarios/media.yml create mode 100644 ci/gitlab-ci/yml/scenarios/network.yml create mode 100644 ci/gitlab-ci/yml/static-checks.yml -- 2.40.1

7 months, 1 week

18
57
0 0

[PATCH] riscv: selftests: Add hwprobe binaries to .gitignore

by Charlie Jenkins

The cbo and which-cpu hwprobe selftests leave their artifacts in the kernel tree and end up being tracked by git. Add the binaries to the hwprobe selftest .gitignore so this no longer happens. Signed-off-by: Charlie Jenkins <charlie(a)rivosinc.com> Fixes: a29e2a48afe3 ("RISC-V: selftests: Add CBO tests") Fixes: ef7d6abb2cf5 ("RISC-V: selftests: Add which-cpus hwprobe test") --- tools/testing/selftests/riscv/hwprobe/.gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/riscv/hwprobe/.gitignore b/tools/testing/selftests/riscv/hwprobe/.gitignore index 8113dc3bdd03..6e384e80ea1a 100644 --- a/tools/testing/selftests/riscv/hwprobe/.gitignore +++ b/tools/testing/selftests/riscv/hwprobe/.gitignore @@ -1 +1,3 @@ hwprobe +cbo +which-cpus --- base-commit: ed30a4a51bb196781c8058073ea720133a65596f change-id: 20240425-gitignore_hwprobe_artifacts-fb0f2cd3509c -- - Charlie

7 months, 1 week

4
3
0 0

[PATCH] riscv: selftests: Add signal handling vector tests

by Charlie Jenkins

Add two tests to check vector save/restore when a signal is received during a vector routine. One test ensures that a value is not clobbered during signal handling. The other verifies that vector registers modified in the signal handler are properly reflected when the signal handling is complete. Signed-off-by: Charlie Jenkins <charlie(a)rivosinc.com> --- These tests came about to highlight the bug fixed in https://lore.kernel.org/lkml/20240403072638.567446-1-bjorn@kernel.org/ and will only pass with that fix applied. --- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/sigreturn/.gitignore | 1 + tools/testing/selftests/riscv/sigreturn/Makefile | 12 ++++ .../testing/selftests/riscv/sigreturn/sigreturn.c | 82 ++++++++++++++++++++++ 4 files changed, 96 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftests/riscv/Makefile index 4a9ff515a3a0..7ce03d832b64 100644 --- a/tools/testing/selftests/riscv/Makefile +++ b/tools/testing/selftests/riscv/Makefile @@ -5,7 +5,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),riscv)) -RISCV_SUBTARGETS ?= hwprobe vector mm +RISCV_SUBTARGETS ?= hwprobe vector mm sigreturn else RISCV_SUBTARGETS := endif diff --git a/tools/testing/selftests/riscv/sigreturn/.gitignore b/tools/testing/selftests/riscv/sigreturn/.gitignore new file mode 100644 index 000000000000..35002b8ae780 --- /dev/null +++ b/tools/testing/selftests/riscv/sigreturn/.gitignore @@ -0,0 +1 @@ +sigreturn diff --git a/tools/testing/selftests/riscv/sigreturn/Makefile b/tools/testing/selftests/riscv/sigreturn/Makefile new file mode 100644 index 000000000000..eb8bac9279a8 --- /dev/null +++ b/tools/testing/selftests/riscv/sigreturn/Makefile @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2021 ARM Limited +# Originally tools/testing/arm64/abi/Makefile + +CFLAGS += -I$(top_srcdir)/tools/include + +TEST_GEN_PROGS := sigreturn + +include ../../lib.mk + +$(OUTPUT)/sigreturn: sigreturn.c + $(CC) -static -o$@ $(CFLAGS) $(LDFLAGS) $^ diff --git a/tools/testing/selftests/riscv/sigreturn/sigreturn.c b/tools/testing/selftests/riscv/sigreturn/sigreturn.c new file mode 100644 index 000000000000..62397d5934f1 --- /dev/null +++ b/tools/testing/selftests/riscv/sigreturn/sigreturn.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <signal.h> +#include <stdio.h> +#include <stdlib.h> +#include <ucontext.h> +#include <linux/ptrace.h> +#include "../../kselftest_harness.h" + +#define RISCV_V_MAGIC 0x53465457 +#define DEFAULT_VALUE 2 +#define SIGNAL_HANDLER_OVERRIDE 3 + +static void simple_handle(int sig_no, siginfo_t *info, void *vcontext) +{ + ucontext_t *context = vcontext; + + context->uc_mcontext.__gregs[REG_PC] = context->uc_mcontext.__gregs[REG_PC] + 4; +} + +static void vector_override(int sig_no, siginfo_t *info, void *vcontext) +{ + ucontext_t *context = vcontext; + + // vector state + struct __riscv_extra_ext_header *ext; + struct __riscv_v_ext_state *v_ext_state; + + /* Find the vector context. */ + ext = (void *)(&context->uc_mcontext.__fpregs); + if (ext->hdr.magic != RISCV_V_MAGIC) { + fprintf(stderr, "bad vector magic: %x\n", ext->hdr.magic); + abort(); + } + + v_ext_state = (void *)((char *)(ext) + sizeof(*ext)); + + *(int *)v_ext_state->datap = SIGNAL_HANDLER_OVERRIDE; + + context->uc_mcontext.__gregs[REG_PC] = context->uc_mcontext.__gregs[REG_PC] + 4; +} + +static int vector_sigreturn(int data, void (*handler)(int, siginfo_t *, void *)) +{ + int after_sigreturn; + struct sigaction sig_action = { + .sa_sigaction = handler, + .sa_flags = SA_SIGINFO + }; + + sigaction(SIGSEGV, &sig_action, 0); + + asm(".option push \n\ + .option arch, +v \n\ + vsetivli x0, 1, e32, ta, ma \n\ + vmv.s.x v0, %1 \n\ + # Generate SIGSEGV \n\ + lw a0, 0(x0) \n\ + vmv.x.s %0, v0 \n\ + .option pop" : "=r" (after_sigreturn) : "r" (data)); + + return after_sigreturn; +} + +TEST(vector_restore) +{ + int result; + + result = vector_sigreturn(DEFAULT_VALUE, &simple_handle); + + EXPECT_EQ(DEFAULT_VALUE, result); +} + +TEST(vector_restore_signal_handler_override) +{ + int result; + + result = vector_sigreturn(DEFAULT_VALUE, &vector_override); + + EXPECT_EQ(SIGNAL_HANDLER_OVERRIDE, result); +} + +TEST_HARNESS_MAIN --- base-commit: 4cece764965020c22cff7665b18a012006359095 change-id: 20240403-vector_sigreturn_tests-8118f0ac54fa -- - Charlie

7 months, 1 week

6
6
0 0

[PATCH 1/2] KVM: selftests: Fix nx_huge_pages_test for default_hugepagesz=1G

by Dongsheng Zhang

From: donsheng <dongsheng.x.zhang(a)intel.com> If the host was booted with the "default_hugepagesz=1G" kernel command-line parameter, running the NX hugepage test will fail with error "Invalid argument" at the TEST_ASSERT line in kvm_util.c's __vm_mem_region_delete() function: static void __vm_mem_region_delete(struct kvm_vm *vm, struct userspace_mem_region *region, bool unlink) { int ret; ... ret = munmap(region->mmap_start, region->mmap_size); TEST_ASSERT(!ret, __KVM_SYSCALL_ERROR("munmap()", ret)); ... } NX hugepage test creates a VM with a data slot of 6M size backed with huge pages. If the default hugetlb page size is set to 1G, calling mmap() with MAP_HUGETLB and a length of 6M will succeed but calling its matching munmap() will fail. Documentation/admin-guide/mm/hugetlbpage.rst specifies this behavior: "Syscalls that operate on memory backed by hugetlb pages only have their lengths aligned to the native page size of the processor; they will normally fail with errno set to EINVAL or exclude hugetlb pages that extend beyond the length if not hugepage aligned. For example, munmap(2) will fail if memory is backed by a hugetlb page and the length is smaller than the hugepage size." Explicitly use MAP_HUGE_2MB in conjunction with MAP_HUGETLB to fix the issue. Signed-off-by: donsheng <dongsheng.x.zhang(a)intel.com> Suggested-by: Zide Chen <zide.chen(a)intel.com> --- tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c index 17bbb96fc4df..146e9033e206 100644 --- a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c +++ b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c @@ -129,7 +129,7 @@ void run_test(int reclaim_period_ms, bool disable_nx_huge_pages, vcpu = vm_vcpu_add(vm, 0, guest_code); - vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB, + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB, HPAGE_GPA, HPAGE_SLOT, HPAGE_SLOT_NPAGES, 0); -- 2.43.0

7 months, 1 week

2
2
0 0

[syzbot] WARNING in btrfs_free_reserved_data_space_noquota

by syzbot

Hello, syzbot found the following issue on: HEAD commit: b7b275e60bcd Linux 6.1-rc7 git tree: upstream console+strace: https://syzkaller.appspot.com/x/log.txt?x=158a7b73880000 kernel config: https://syzkaller.appspot.com/x/.config?x=2325e409a9a893e1 dashboard link: https://syzkaller.appspot.com/bug?extid=adec8406ad17413d4c06 compiler: Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=169ccb75880000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17bf7153880000 Downloadable assets: disk image: https://storage.googleapis.com/syzbot-assets/525233126d34/disk-b7b275e6.raw… vmlinux: https://storage.googleapis.com/syzbot-assets/e8299bf41400/vmlinux-b7b275e6.… kernel image: https://storage.googleapis.com/syzbot-assets/eebf691dbf6f/bzImage-b7b275e6.… mounted in repro: https://storage.googleapis.com/syzbot-assets/5423c2d2ad62/mount_0.gz The issue was bisected to: commit c814bf958926ff45a9c1e899bd001006ab6cfbae Author: ye xingchen <ye.xingchen(a)zte.com.cn> Date: Tue Aug 16 10:51:06 2022 +0000 powerpc/selftests: Use timersub() for gettimeofday() bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=118c3d03880000 final oops: https://syzkaller.appspot.com/x/report.txt?x=138c3d03880000 console output: https://syzkaller.appspot.com/x/log.txt?x=158c3d03880000 IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+adec8406ad17413d4c06(a)syzkaller.appspotmail.com Fixes: c814bf958926 ("powerpc/selftests: Use timersub() for gettimeofday()") RDX: 0000000000000001 RSI: 0000000020000280 RDI: 0000000000000005 RBP: 00007ffd32e91c70 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000008000000 R11: 0000000000000246 R12: 0000000000000006 R13: 00007ffd32e91cb0 R14: 00007ffd32e91c90 R15: 0000000000000006 </TASK> ------------[ cut here ]------------ WARNING: CPU: 1 PID: 3764 at fs/btrfs/space-info.h:122 btrfs_space_info_free_bytes_may_use fs/btrfs/space-info.h:154 [inline] WARNING: CPU: 1 PID: 3764 at fs/btrfs/space-info.h:122 btrfs_free_reserved_data_space_noquota+0x219/0x2b0 fs/btrfs/delalloc-space.c:179 Modules linked in: CPU: 1 PID: 3764 Comm: syz-executor759 Not tainted 6.1.0-rc7-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022 RIP: 0010:btrfs_space_info_update_bytes_may_use fs/btrfs/space-info.h:122 [inline] RIP: 0010:btrfs_space_info_free_bytes_may_use fs/btrfs/space-info.h:154 [inline] RIP: 0010:btrfs_free_reserved_data_space_noquota+0x219/0x2b0 fs/btrfs/delalloc-space.c:179 Code: 2f 00 74 08 4c 89 ef e8 b5 98 32 fe 49 8b 5d 00 48 89 df 4c 8b 74 24 08 4c 89 f6 e8 21 81 de fd 4c 39 f3 73 16 e8 d7 7e de fd <0f> 0b 31 db 4c 8b 34 24 41 80 3c 2f 00 75 8c eb 92 e8 c1 7e de fd RSP: 0018:ffffc9000443f410 EFLAGS: 00010293 RAX: ffffffff83ac1919 RBX: 00000000005cb000 RCX: ffff888027989d40 RDX: 0000000000000000 RSI: 0000000000800000 RDI: 00000000005cb000 RBP: dffffc0000000000 R08: ffffffff83ac190f R09: fffffbfff1cebe0e R10: fffffbfff1cebe0e R11: 1ffffffff1cebe0d R12: ffff8880774f3800 R13: ffff8880774f3860 R14: 0000000000800000 R15: 1ffff1100ee9e70c FS: 0000555555aaa300(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f0d98f20140 CR3: 0000000025ccf000 CR4: 00000000003506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> btrfs_free_reserved_data_space+0x9d/0xd0 fs/btrfs/delalloc-space.c:199 btrfs_dio_iomap_begin+0x8f7/0x1070 fs/btrfs/inode.c:7762 iomap_iter+0x606/0x8a0 fs/iomap/iter.c:74 __iomap_dio_rw+0xd91/0x20d0 fs/iomap/direct-io.c:601 btrfs_dio_write+0x9c/0xe0 fs/btrfs/inode.c:8094 btrfs_direct_write fs/btrfs/file.c:1835 [inline] btrfs_do_write_iter+0x871/0x1260 fs/btrfs/file.c:1980 do_iter_write+0x6c2/0xc20 fs/read_write.c:861 vfs_writev fs/read_write.c:934 [inline] do_pwritev+0x200/0x350 fs/read_write.c:1031 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f0d98ea8ea9 Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 41 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007ffd32e91c38 EFLAGS: 00000246 ORIG_RAX: 0000000000000148 RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f0d98ea8ea9 RDX: 0000000000000001 RSI: 0000000020000280 RDI: 0000000000000005 RBP: 00007ffd32e91c70 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000008000000 R11: 0000000000000246 R12: 0000000000000006 R13: 00007ffd32e91cb0 R14: 00007ffd32e91c90 R15: 0000000000000006 </TASK> --- This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller(a)googlegroups.com. syzbot will keep track of this issue. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. For information about bisection process see: https://goo.gl/tpsmEJ#bisection syzbot can test patches for this issue, for details see: https://goo.gl/tpsmEJ#testing-patches

7 months, 2 weeks

3
3
0 0

[PATCH v6] lib: add basic KUnit test for lib/math

by Daniel Latypov

Add basic test coverage for files that don't require any config options: * part of math.h (what seem to be the most commonly used macros) * gcd.c * lcm.c * int_sqrt.c * reciprocal_div.c (Ignored int_pow.c since it's a simple textbook algorithm.) These tests aren't particularly interesting, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- Changes since v5: * add in test cases for roundup/rounddown * address misc comments from David Changes since v4: * add in test cases for some math.h macros (abs, round_up/round_down, div_round_down/closest) * use parameterized testing less to keep things terser Changes since v3: * fix `checkpatch.pl --strict` warnings * add test cases for gcd(0,0) and lcm(0,0) * minor: don't test both gcd(a,b) and gcd(b,a) when a == b Changes since v2: mv math_test.c => math_kunit.c Changes since v1: * Rebase and rewrite to use the new parameterized testing support. * misc: fix overflow in literal and inline int_sqrt format string. * related: commit 1f0e943df68a ("Documentation: kunit: provide guidance for testing many inputs") was merged explaining the patterns shown here. * there's an in-flight patch to update it for parameterized testing. --- lib/math/Kconfig | 12 ++ lib/math/Makefile | 2 + lib/math/math_kunit.c | 291 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 305 insertions(+) create mode 100644 lib/math/math_kunit.c diff --git a/lib/math/Kconfig b/lib/math/Kconfig index f19bc9734fa7..a974d4db0f9c 100644 --- a/lib/math/Kconfig +++ b/lib/math/Kconfig @@ -15,3 +15,15 @@ config PRIME_NUMBERS config RATIONAL bool + +config MATH_KUNIT_TEST + tristate "KUnit test for lib/math and math.h" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS + help + This builds unit tests for lib/math and math.h. + + For more information on KUnit and unit tests in general, please refer + to the KUnit documentation in Documentation/dev-tools/kunit/. + + If unsure, say N. diff --git a/lib/math/Makefile b/lib/math/Makefile index be6909e943bd..30abb7a8d564 100644 --- a/lib/math/Makefile +++ b/lib/math/Makefile @@ -4,3 +4,5 @@ obj-y += div64.o gcd.o lcm.o int_pow.o int_sqrt.o reciprocal_div.o obj-$(CONFIG_CORDIC) += cordic.o obj-$(CONFIG_PRIME_NUMBERS) += prime_numbers.o obj-$(CONFIG_RATIONAL) += rational.o + +obj-$(CONFIG_MATH_KUNIT_TEST) += math_kunit.o diff --git a/lib/math/math_kunit.c b/lib/math/math_kunit.c new file mode 100644 index 000000000000..556c23b17c3c --- /dev/null +++ b/lib/math/math_kunit.c @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Simple KUnit suite for math helper funcs that are always enabled. + * + * Copyright (C) 2020, Google LLC. + * Author: Daniel Latypov <dlatypov(a)google.com> + */ + +#include <kunit/test.h> +#include <linux/gcd.h> +#include <linux/kernel.h> +#include <linux/lcm.h> +#include <linux/reciprocal_div.h> + +static void abs_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, abs((char)0), (char)0); + KUNIT_EXPECT_EQ(test, abs((char)42), (char)42); + KUNIT_EXPECT_EQ(test, abs((char)-42), (char)42); + + /* The expression in the macro is actually promoted to an int. */ + KUNIT_EXPECT_EQ(test, abs((short)0), 0); + KUNIT_EXPECT_EQ(test, abs((short)42), 42); + KUNIT_EXPECT_EQ(test, abs((short)-42), 42); + + KUNIT_EXPECT_EQ(test, abs(0), 0); + KUNIT_EXPECT_EQ(test, abs(42), 42); + KUNIT_EXPECT_EQ(test, abs(-42), 42); + + KUNIT_EXPECT_EQ(test, abs(0L), 0L); + KUNIT_EXPECT_EQ(test, abs(42L), 42L); + KUNIT_EXPECT_EQ(test, abs(-42L), 42L); + + KUNIT_EXPECT_EQ(test, abs(0LL), 0LL); + KUNIT_EXPECT_EQ(test, abs(42LL), 42LL); + KUNIT_EXPECT_EQ(test, abs(-42LL), 42LL); + + /* Unsigned types get casted to signed. */ + KUNIT_EXPECT_EQ(test, abs(0ULL), 0LL); + KUNIT_EXPECT_EQ(test, abs(42ULL), 42LL); +} + +static void int_sqrt_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, int_sqrt(0UL), 0UL); + KUNIT_EXPECT_EQ(test, int_sqrt(1UL), 1UL); + KUNIT_EXPECT_EQ(test, int_sqrt(4UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(5UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(8UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(1UL << 30), 1UL << 15); +} + +static void round_up_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, round_up(0, 1), 0); + KUNIT_EXPECT_EQ(test, round_up(1, 2), 2); + KUNIT_EXPECT_EQ(test, round_up(3, 2), 4); + KUNIT_EXPECT_EQ(test, round_up((1 << 30) - 1, 2), 1 << 30); + KUNIT_EXPECT_EQ(test, round_up((1 << 30) - 1, 1 << 29), 1 << 30); +} + +static void round_down_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, round_down(0, 1), 0); + KUNIT_EXPECT_EQ(test, round_down(1, 2), 0); + KUNIT_EXPECT_EQ(test, round_down(3, 2), 2); + KUNIT_EXPECT_EQ(test, round_down((1 << 30) - 1, 2), (1 << 30) - 2); + KUNIT_EXPECT_EQ(test, round_down((1 << 30) - 1, 1 << 29), 1 << 29); +} + +/* These versions can round to numbers that aren't a power of two */ +static void roundup_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, roundup(0, 1), 0); + KUNIT_EXPECT_EQ(test, roundup(1, 2), 2); + KUNIT_EXPECT_EQ(test, roundup(3, 2), 4); + KUNIT_EXPECT_EQ(test, roundup((1 << 30) - 1, 2), 1 << 30); + KUNIT_EXPECT_EQ(test, roundup((1 << 30) - 1, 1 << 29), 1 << 30); + + KUNIT_EXPECT_EQ(test, roundup(3, 2), 4); + KUNIT_EXPECT_EQ(test, roundup(4, 3), 6); +} + +static void rounddown_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, rounddown(0, 1), 0); + KUNIT_EXPECT_EQ(test, rounddown(1, 2), 0); + KUNIT_EXPECT_EQ(test, rounddown(3, 2), 2); + KUNIT_EXPECT_EQ(test, rounddown((1 << 30) - 1, 2), (1 << 30) - 2); + KUNIT_EXPECT_EQ(test, rounddown((1 << 30) - 1, 1 << 29), 1 << 29); + + KUNIT_EXPECT_EQ(test, rounddown(3, 2), 2); + KUNIT_EXPECT_EQ(test, rounddown(4, 3), 3); +} + +static void div_round_up_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(0, 1), 0); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(20, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 10), 3); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 20), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 99), 1); +} + +static void div_round_closest_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(0, 1), 0); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(20, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(21, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(25, 10), 3); +} + +/* Generic test case for unsigned long inputs. */ +struct test_case { + unsigned long a, b; + unsigned long result; +}; + +static struct test_case gcd_cases[] = { + { + .a = 0, .b = 0, + .result = 0, + }, + { + .a = 0, .b = 1, + .result = 1, + }, + { + .a = 2, .b = 2, + .result = 2, + }, + { + .a = 2, .b = 4, + .result = 2, + }, + { + .a = 3, .b = 5, + .result = 1, + }, + { + .a = 3 * 9, .b = 3 * 5, + .result = 3, + }, + { + .a = 3 * 5 * 7, .b = 3 * 5 * 11, + .result = 15, + }, + { + .a = 1 << 21, + .b = (1 << 21) - 1, + .result = 1, + }, +}; + +KUNIT_ARRAY_PARAM(gcd, gcd_cases, NULL); + +static void gcd_test(struct kunit *test) +{ + const char *message_fmt = "gcd(%lu, %lu)"; + const struct test_case *test_param = test->param_value; + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + gcd(test_param->a, test_param->b), + message_fmt, test_param->a, + test_param->b); + + if (test_param->a == test_param->b) + return; + + /* gcd(a,b) == gcd(b,a) */ + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + gcd(test_param->b, test_param->a), + message_fmt, test_param->b, + test_param->a); +} + +static struct test_case lcm_cases[] = { + { + .a = 0, .b = 0, + .result = 0, + }, + { + .a = 0, .b = 1, + .result = 0, + }, + { + .a = 1, .b = 2, + .result = 2, + }, + { + .a = 2, .b = 2, + .result = 2, + }, + { + .a = 3 * 5, .b = 3 * 7, + .result = 3 * 5 * 7, + }, +}; + +KUNIT_ARRAY_PARAM(lcm, lcm_cases, NULL); + +static void lcm_test(struct kunit *test) +{ + const char *message_fmt = "lcm(%lu, %lu)"; + const struct test_case *test_param = test->param_value; + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + lcm(test_param->a, test_param->b), + message_fmt, test_param->a, + test_param->b); + + if (test_param->a == test_param->b) + return; + + /* lcm(a,b) == lcm(b,a) */ + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + lcm(test_param->b, test_param->a), + message_fmt, test_param->b, + test_param->a); +} + +struct u32_test_case { + u32 a, b; + u32 result; +}; + +static struct u32_test_case reciprocal_div_cases[] = { + { + .a = 0, .b = 1, + .result = 0, + }, + { + .a = 42, .b = 20, + .result = 2, + }, + { + .a = 42, .b = 9999, + .result = 0, + }, + { + .a = (1 << 16), .b = (1 << 14), + .result = 1 << 2, + }, +}; + +KUNIT_ARRAY_PARAM(reciprocal_div, reciprocal_div_cases, NULL); + +static void reciprocal_div_test(struct kunit *test) +{ + const struct u32_test_case *test_param = test->param_value; + struct reciprocal_value rv = reciprocal_value(test_param->b); + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + reciprocal_divide(test_param->a, rv), + "reciprocal_divide(%u, %u)", + test_param->a, test_param->b); +} + +static void reciprocal_scale_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, reciprocal_scale(0u, 100), 0u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u, 100), 0u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u << 4, 1 << 28), 1u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u << 16, 1 << 28), 1u << 12); + KUNIT_EXPECT_EQ(test, reciprocal_scale(~0u, 1 << 28), (1u << 28) - 1); +} + +static struct kunit_case math_test_cases[] = { + KUNIT_CASE(abs_test), + KUNIT_CASE(int_sqrt_test), + KUNIT_CASE(round_up_test), + KUNIT_CASE(round_down_test), + KUNIT_CASE(roundup_test), + KUNIT_CASE(rounddown_test), + KUNIT_CASE(div_round_up_test), + KUNIT_CASE(div_round_closest_test), + KUNIT_CASE_PARAM(gcd_test, gcd_gen_params), + KUNIT_CASE_PARAM(lcm_test, lcm_gen_params), + KUNIT_CASE_PARAM(reciprocal_div_test, reciprocal_div_gen_params), + KUNIT_CASE(reciprocal_scale_test), + {} +}; + +static struct kunit_suite math_test_suite = { + .name = "lib-math", + .test_cases = math_test_cases, +}; + +kunit_test_suites(&math_test_suite); + +MODULE_LICENSE("GPL v2"); base-commit: 7e25f40eab52c57ff6772d27d2aef3640a3237d7 -- 2.31.1.368.gbe11c130af-goog

7 months, 2 weeks

5
5
0 0

[PATCH v4 00/30] NT synchronization primitive driver

by Elizabeth Figura

This patch series implements a new char misc driver, /dev/ntsync, which is used to implement Windows NT synchronization primitives. NT synchronization primitives are unique in that the wait functions both are vectored, operate on multiple types of object with different behaviour (mutex, semaphore, event), and affect the state of the objects they wait on. This model is not compatible with existing kernel synchronization objects or interfaces, and therefore the ntsync driver implements its own wait queues and locking. Hence I would like to request review from someone familiar with locking to make sure that the usage of low-level kernel primitives is correct and that the wait queues work as intended, and to that end I've CC'd the locking maintainers. == Background == The Wine project emulates the Windows API in user space. One particular part of that API, namely the NT synchronization primitives, have historically been implemented via RPC to a dedicated "kernel" process. However, more recent applications use these APIs more strenuously, and the overhead of RPC has become a bottleneck. The NT synchronization APIs are too complex to implement on top of existing primitives without sacrificing correctness. Certain operations, such as NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require direct control over the underlying wait queue, and implementing a wait queue sufficiently robust for Wine in user space is not possible. This proposed driver, therefore, implements the problematic interfaces directly in the Linux kernel. This driver was presented at Linux Plumbers Conference 2023. For those further interested in the history of synchronization in Wine and past attempts to solve this problem in user space, a recording of the presentation can be viewed here: https://www.youtube.com/watch?v=NjU4nyWyhU8 == Performance == The gain in performance varies wildly depending on the application in question and the user's hardware. For some games NT synchronization is not a bottleneck and no change can be observed, but for others frame rate improvements of 50 to 150 percent are not atypical. The following table lists frame rate measurements from a variety of games on a variety of hardware, taken by users Dmitry Skvortsov, FuzzyQuils, OnMars, and myself: Game Upstream ntsync improvement =========================================================================== Anger Foot 69 99 43% Call of Juarez 99.8 224.1 125% Dirt 3 110.6 860.7 678% Forza Horizon 5 108 160 48% Lara Croft: Temple of Osiris 141 326 131% Metro 2033 164.4 199.2 21% Resident Evil 2 26 77 196% The Crew 26 51 96% Tiny Tina's Wonderlands 130 360 177% Total War Saga: Troy 109 146 34% =========================================================================== == Patches == The intended semantics of the patches are broadly intended to match those of the corresponding Windows functions. For those not already familiar with the Windows functions (or their undocumented behaviour), patch 27/27 provides a detailed specification, and individual patches also include a brief description of the API they are implementing. The patches making use of this driver in Wine can be retrieved or browsed here: https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5 == Implementation == Some aspects of the implementation may deserve particular comment: * In the interest of performance, each object is governed only by a single spinlock. However, NTSYNC_IOC_WAIT_ALL requires that the state of multiple objects be changed as a single atomic operation. In order to achieve this, we first take a device-wide lock ("wait_all_lock") any time we are going to lock more than one object at a time. The maximum number of objects that can be used in a vectored wait, and therefore the maximum that can be locked simultaneously, is 64. This number is NT's own limit. The acquisition of multiple spinlocks will degrade performance. This is a conscious choice, however. Wait-for-all is known to be a very rare operation in practice, especially with counts that approach the maximum, and it is the intent of the ntsync driver to optimize wait-for-any at the expense of wait-for-all as much as possible. * NT mutexes are tied to their threads on an OS level, and the kernel includes builtin support for "robust" mutexes. In order to keep the ntsync driver self-contained and avoid touching more code than necessary, it does not hook into task exit nor use pids. Instead, the user space emulator is expected to manage thread IDs and pass them as an argument to any relevant functions; this is the "owner" field of ntsync_wait_args and ntsync_mutex_args. When the emulator detects that a thread dies, it should therefore call NTSYNC_IOC_MUTEX_KILL on any open mutexes. * ntsync is module-capable mostly because there was nothing preventing it, and because it aided development. It is not a hard requirement, though. == Previous versions == Changes from v3: * Add .gitignore and use KHDR_INCLUDES in selftest build files, per Muhammad Usama Anjum. * Try to explain why we are rolling our own primitives a little better, per Greg Kroah-Hartman. * Link to v3: https://lore.kernel.org/lkml/20240329000621.148791-1-zfigura@codeweavers.co… * Link to v2: https://lore.kernel.org/lkml/20240219223833.95710-1-zfigura@codeweavers.com/ * Link to v1: https://lore.kernel.org/lkml/20240214233645.9273-1-zfigura@codeweavers.com/ * Link to RFC v2: https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura@codeweavers.com/ * Link to RFC v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura@codeweavers.com/ Elizabeth Figura (27): ntsync: Introduce NTSYNC_IOC_WAIT_ANY. ntsync: Introduce NTSYNC_IOC_WAIT_ALL. ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX. ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK. ntsync: Introduce NTSYNC_IOC_MUTEX_KILL. ntsync: Introduce NTSYNC_IOC_CREATE_EVENT. ntsync: Introduce NTSYNC_IOC_EVENT_SET. ntsync: Introduce NTSYNC_IOC_EVENT_RESET. ntsync: Introduce NTSYNC_IOC_EVENT_PULSE. ntsync: Introduce NTSYNC_IOC_SEM_READ. ntsync: Introduce NTSYNC_IOC_MUTEX_READ. ntsync: Introduce NTSYNC_IOC_EVENT_READ. ntsync: Introduce alertable waits. selftests: ntsync: Add some tests for semaphore state. selftests: ntsync: Add some tests for mutex state. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for manual-reset event state. selftests: ntsync: Add some tests for auto-reset event state. selftests: ntsync: Add some tests for wakeup signaling with events. selftests: ntsync: Add tests for alertable waits. selftests: ntsync: Add some tests for wakeup signaling via alerts. selftests: ntsync: Add a stress test for contended waits. maintainers: Add an entry for ntsync. docs: ntsync: Add documentation for the ntsync uAPI. Documentation/userspace-api/index.rst | 1 + Documentation/userspace-api/ntsync.rst | 399 +++++ MAINTAINERS | 9 + drivers/misc/ntsync.c | 925 ++++++++++- include/uapi/linux/ntsync.h | 39 + tools/testing/selftests/Makefile | 1 + .../selftests/drivers/ntsync/.gitignore | 1 + .../testing/selftests/drivers/ntsync/Makefile | 7 + tools/testing/selftests/drivers/ntsync/config | 1 + .../testing/selftests/drivers/ntsync/ntsync.c | 1407 +++++++++++++++++ 10 files changed, 2786 insertions(+), 4 deletions(-) create mode 100644 Documentation/userspace-api/ntsync.rst create mode 100644 tools/testing/selftests/drivers/ntsync/.gitignore create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile create mode 100644 tools/testing/selftests/drivers/ntsync/config create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c base-commit: ebbc1a4789c666846b9854ef845a37a64879e5f9 -- 2.43.0

7 months, 2 weeks

4
54
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror April 2024