July 2023 - Linux-kselftest-mirror

[PATCH v8 0/4] RISC-V: mm: Make SV48 the default address space

by Charlie Jenkins

Make sv48 the default address space for mmap as some applications currently depend on this assumption. Users can now select a desired address space using a non-zero hint address to mmap. Previously, requesting the default address space from mmap by passing zero as the hint address would result in using the largest address space possible. Some applications depend on empty bits in the virtual address space, like Go and Java, so this patch provides more flexibility for application developers. -Charlie --- v8: - Fix RV32 and the RV32 compat mode of RV64 - Extract out addr and base from the mmap macros v7: - Changing RLIMIT_STACK inside of an executing program does not trigger arch_pick_mmap_layout(), so rewrite tests to change RLIMIT_STACK from a script before executing tests. RLIMIT_STACK of infinity forces bottomup mmap allocation. - Make arch_get_mmap_base macro more readible by extracting out the rnd calculation. - Use MMAP_MIN_VA_BITS in TASK_UNMAPPED_BASE to support case when mmap attempts to allocate address smaller than DEFAULT_MAP_WINDOW. - Fix incorrect wording in documentation. v6: - Rebase onto the correct base v5: - Minor wording change in documentation - Change some parenthesis in arch_get_mmap_ macros - Added case for addr==0 in arch_get_mmap_ because without this, programs would crash if RLIMIT_STACK was modified before executing the program. This was tested using the libhugetlbfs tests. v4: - Split testcases/document patch into test cases, in-code documentation, and formal documentation patches - Modified the mmap_base macro to be more legible and better represent memory layout - Fixed documentation to better reflect the implmentation - Renamed DEFAULT_VA_BITS to MMAP_VA_BITS - Added additional test case for rlimit changes --- Charlie Jenkins (4): RISC-V: mm: Restrict address space for sv39,sv48,sv57 RISC-V: mm: Add tests for RISC-V mm RISC-V: mm: Update pgtable comment documentation RISC-V: mm: Document mmap changes Documentation/riscv/vm-layout.rst | 22 +++++++ arch/riscv/include/asm/elf.h | 2 +- arch/riscv/include/asm/pgtable.h | 28 ++++++-- arch/riscv/include/asm/processor.h | 52 +++++++++++++-- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/mm/.gitignore | 2 + tools/testing/selftests/riscv/mm/Makefile | 15 +++++ .../riscv/mm/testcases/mmap_bottomup.c | 35 ++++++++++ .../riscv/mm/testcases/mmap_default.c | 35 ++++++++++ .../selftests/riscv/mm/testcases/mmap_test.h | 64 +++++++++++++++++++ .../selftests/riscv/mm/testcases/run_mmap.sh | 12 ++++ 11 files changed, 257 insertions(+), 12 deletions(-) create mode 100644 tools/testing/selftests/riscv/mm/.gitignore create mode 100644 tools/testing/selftests/riscv/mm/Makefile create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_bottomup.c create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_default.c create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap_test.h create mode 100755 tools/testing/selftests/riscv/mm/testcases/run_mmap.sh -- 2.41.0

1 year, 4 months

2
9
0 0

[PATCH v5 00/19] selftests/resctrl: Fixes and cleanups

by Ilpo Järvinen

Here is a series with some fixes and cleanups to resctrl selftests. v5: - Improve changelogs - Close fd_lm only in cat_val() - Improve unmount error handling v4: - Move resctrlfs (unconditional) umount after resctrl fs support check v3: - Don't include rewritten CAT test into this series! - Tweak wildcard style in Makefile - Fix many changelog typos, remove some wrong claims, and generally improve them. - Add fix to PARENT_EXIT() to unmount resctrl FS - Add unmounting resctrl FS before starting any tests - Add fix for buf leak - Add fix for perf fd closing - Split mount/remount/umount patches differently - Use size_t and %zu for span - Keep MBM print as MB, only internally use span in bytes - Drop start_buf global from fill_buf v2 (was sent with CAT test rewrite which is no longer included in v3): - Rebased on top of next to solve the conflicts - Added 2 patches related to resctrl FS mount/umount (fix + cleanup) - Consistently use "alloc" in cache_alloc_size() - CAT test error handling tweaked - Remove a spurious newline change from the CAT patch - Small improvements to changelogs Ilpo Järvinen (19): selftests/resctrl: Add resctrl.h into build deps selftests/resctrl: Don't leak buffer in fill_cache() selftests/resctrl: Unmount resctrl FS if child fails to run benchmark selftests/resctrl: Close perf value read fd on errors selftests/resctrl: Unmount resctrl FS before starting the first test selftests/resctrl: Move resctrl FS mount/umount to higher level selftests/resctrl: Refactor remount_resctrl(bool mum_resctrlfs) to mount_resctrl() selftests/resctrl: Remove mum_resctrlfs from struct resctrl_val_param selftests/resctrl: Convert span to size_t selftests/resctrl: Express span internally in bytes selftests/resctrl: Remove duplicated preparation for span arg selftests/resctrl: Remove "malloc_and_init_memory" param from run_fill_buf() selftests/resctrl: Remove unnecessary startptr global from fill_buf selftests/resctrl: Improve parameter consistency in fill_buf selftests/resctrl: Don't pass test name to fill_buf selftests/resctrl: Don't use variable argument list for ->setup() selftests/resctrl: Move CAT/CMT test global vars to function they are used in selftests/resctrl: Pass the real number of tests to show_cache_info() selftests/resctrl: Remove test type checks from cat_val() tools/testing/selftests/resctrl/Makefile | 2 +- tools/testing/selftests/resctrl/cache.c | 66 +++++++------- tools/testing/selftests/resctrl/cat_test.c | 28 ++---- tools/testing/selftests/resctrl/cmt_test.c | 29 ++----- tools/testing/selftests/resctrl/fill_buf.c | 87 +++++++------------ tools/testing/selftests/resctrl/mba_test.c | 9 +- tools/testing/selftests/resctrl/mbm_test.c | 17 ++-- tools/testing/selftests/resctrl/resctrl.h | 17 ++-- .../testing/selftests/resctrl/resctrl_tests.c | 83 ++++++++++++------ tools/testing/selftests/resctrl/resctrl_val.c | 7 +- tools/testing/selftests/resctrl/resctrlfs.c | 64 +++++++------- 11 files changed, 178 insertions(+), 231 deletions(-) -- 2.30.2

1 year, 4 months

5
28
0 0

[PATCH v4 00/19] selftests/resctrl: Fixes and cleanups

by Ilpo Järvinen

Here is a series with some fixes and cleanups to resctrl selftests. Only has a minor change in code ordering in main() compared with v3. v4: - Move resctrlfs (unconditional) umount after resctrl fs support check v3: - Don't include rewritten CAT test into this series! - Tweak wildcard style in Makefile - Fix many changelog typos, remove some wrong claims, and generally improve them. - Add fix to PARENT_EXIT() to unmount resctrl FS - Add unmounting resctrl FS before starting any tests - Add fix for buf leak - Add fix for perf fd closing - Split mount/remount/umount patches differently - Use size_t and %zu for span - Keep MBM print as MB, only internally use span in bytes - Drop start_buf global from fill_buf v2 (was sent with CAT test rewrite which is no longer included in v3): - Rebased on top of next to solve the conflicts - Added 2 patches related to resctrl FS mount/umount (fix + cleanup) - Consistently use "alloc" in cache_alloc_size() - CAT test error handling tweaked - Remove a spurious newline change from the CAT patch - Small improvements to changelogs Ilpo Järvinen (19): selftests/resctrl: Add resctrl.h into build deps selftests/resctrl: Don't leak buffer in fill_cache() selftests/resctrl: Unmount resctrl FS if child fails to run benchmark selftests/resctrl: Close perf value read fd on errors selftests/resctrl: Unmount resctrl FS before starting the first test selftests/resctrl: Move resctrl FS mount/umount to higher level selftests/resctrl: Refactor remount_resctrl(bool mum_resctrlfs) to mount_resctrl() selftests/resctrl: Remove mum_resctrlfs from struct resctrl_val_param selftests/resctrl: Convert span to size_t selftests/resctrl: Express span internally in bytes selftests/resctrl: Remove duplicated preparation for span arg selftests/resctrl: Remove "malloc_and_init_memory" param from run_fill_buf() selftests/resctrl: Remove unnecessary startptr global from fill_buf selftests/resctrl: Improve parameter consistency in fill_buf selftests/resctrl: Don't pass test name to fill_buf selftests/resctrl: Don't use variable argument list for ->setup() selftests/resctrl: Move CAT/CMT test global vars to function they are used in selftests/resctrl: Pass the real number of tests to show_cache_info() selftests/resctrl: Remove test type checks from cat_val() tools/testing/selftests/resctrl/Makefile | 2 +- tools/testing/selftests/resctrl/cache.c | 64 +++++++------- tools/testing/selftests/resctrl/cat_test.c | 28 ++---- tools/testing/selftests/resctrl/cmt_test.c | 29 ++----- tools/testing/selftests/resctrl/fill_buf.c | 87 +++++++------------ tools/testing/selftests/resctrl/mba_test.c | 9 +- tools/testing/selftests/resctrl/mbm_test.c | 17 ++-- tools/testing/selftests/resctrl/resctrl.h | 17 ++-- .../testing/selftests/resctrl/resctrl_tests.c | 82 +++++++++++------ tools/testing/selftests/resctrl/resctrl_val.c | 7 +- tools/testing/selftests/resctrl/resctrlfs.c | 57 ++++++------ 11 files changed, 169 insertions(+), 230 deletions(-) -- 2.30.2

1 year, 4 months

3
55
0 0

[PATCH v6 00/15] tools/nolibc: add a new syscall helper

by Zhangjin Wu

Hi, Willy Here is the v6 of the __sysret series [1], applies your suggestions. additionally, the sbrk() also uses the __sysret helper. These patches are tested (together with the coming v4 selftests/nolibc patches) for all of the supported architectures: arch/board | result ------------|------------ arm/vexpress-a9 | 142 test(s) passed, 1 skipped, 0 failed. arm/virt | 142 test(s) passed, 1 skipped, 0 failed. aarch64/virt | 142 test(s) passed, 1 skipped, 0 failed. ppc/g3beige | not supported ppc/ppce500 | not supported i386/pc | 142 test(s) passed, 1 skipped, 0 failed. x86_64/pc | 142 test(s) passed, 1 skipped, 0 failed. mipsel/malta | 142 test(s) passed, 1 skipped, 0 failed. loongarch64/virt | 142 test(s) passed, 1 skipped, 0 failed. riscv64/virt | 142 test(s) passed, 1 skipped, 0 failed. riscv32/virt | 0 test(s) passed, 0 skipped, 0 failed. s390x/s390-ccw-virtio | 142 test(s) passed, 1 skipped, 0 failed. Changes from v5 --> v6: * tools/nolibc: arch-*.h: fix up code indent errors toolc/nolibc: arch-*.h: clean up whitespaces after __asm__ Fix up the code indent errors and whitespaces between __asm__ and volatile. The post-whitespaces are reserved as before. * tools/nolibc: arch-loongarch.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST tools/nolibc: arch-mips.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST Add _NOLIBC_ prefix for SYSCALL_CLOBBERLIST. * tools/nolibc: add missing my_syscall6() for mips Use post-whitespaces instead of post-tab. The above 4 patches are preparation for this one. * tools/nolibc: __sysret: support syscalls who return a pointer Add comments about the new errno range [-MAX_ERRNOR, -1], add ref to the musl and glibc. * tools/nolibc: clean up mmap() routine Comment the MAP_FAILED return info. * tools/nolibc: clean up sbrk() routine New patch, applies __sysret() helper too and also fixes up an error reported by scripts/checkpatch.pl. * selftests/nolibc: export argv0 for some tests selftests/nolibc: prepare: create /dev/zero Prepare /dev/zero and argv0 for mmap test cases. * selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER selftests/nolibc: add sbrk_0 to test current brk getting No change. * selftests/nolibc: add mmap_bad test case selftests/nolibc: add munmap_bad test case selftests/nolibc: add mmap_munmap_good test case Split the first two out to standalone patches. Add /dev/zero and argv0 to the file list and assigns a file_size manually for /dev/zero. Best regards, Zhangjin --- [1]: https://lore.kernel.org/lkml/cover.1687957589.git.falcon@tinylab.org/ Zhangjin Wu (15): tools/nolibc: arch-*.h: fix up code indent errors toolc/nolibc: arch-*.h: clean up whitespaces after __asm__ tools/nolibc: arch-loongarch.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST tools/nolibc: arch-mips.h: shrink with _NOLIBC_SYSCALL_CLOBBERLIST tools/nolibc: add missing my_syscall6() for mips tools/nolibc: __sysret: support syscalls who return a pointer tools/nolibc: clean up mmap() routine tools/nolibc: clean up sbrk() routine selftests/nolibc: export argv0 for some tests selftests/nolibc: prepare: create /dev/zero selftests/nolibc: add EXPECT_PTREQ, EXPECT_PTRNE and EXPECT_PTRER selftests/nolibc: add sbrk_0 to test current brk getting selftests/nolibc: add mmap_bad test case selftests/nolibc: add munmap_bad test case selftests/nolibc: add mmap_munmap_good test case tools/include/nolibc/arch-aarch64.h | 28 ++-- tools/include/nolibc/arch-arm.h | 28 ++-- tools/include/nolibc/arch-i386.h | 24 ++-- tools/include/nolibc/arch-loongarch.h | 37 +++--- tools/include/nolibc/arch-mips.h | 73 +++++++---- tools/include/nolibc/arch-riscv.h | 14 +- tools/include/nolibc/arch-s390.h | 14 +- tools/include/nolibc/arch-x86_64.h | 28 ++-- tools/include/nolibc/nolibc.h | 9 +- tools/include/nolibc/sys.h | 55 ++++---- tools/include/nolibc/types.h | 6 + tools/testing/selftests/nolibc/nolibc-test.c | 129 ++++++++++++++++++- 12 files changed, 292 insertions(+), 153 deletions(-) -- 2.25.1

1 year, 4 months

3
22
0 0

[PATCH 0/1] Possible bug in zram on ppc64le on vfat

by Petr Vorel

Hi all, following bug is trying to workaround an error on ppc64le, where zram01.sh LTP test (there is also kernel selftest tools/testing/selftests/zram/zram01.sh, but LTP test got further updates) has often mem_used_total 0 although zram is already filled. Patch tries to repeatedly read /sys/block/zram*/mm_stat for 1 sec, waiting for mem_used_total > 0. The question if this is expected and should be workarounded or a bug which should be fixed. REPRODUCE THE ISSUE Quickest way to install only zram tests and their dependencies: make autotools && ./configure && for i in testcases/lib/ testcases/kernel/device-drivers/zram/; do cd $i && make -j$(getconf _NPROCESSORS_ONLN) && make install && cd -; done Run the test (only on vfat) PATH="/opt/ltp/testcases/bin:$PATH" LTP_SINGLE_FS_TYPE=vfat zram01.sh Petr Vorel (1): zram01.sh: Workaround division by 0 on vfat on ppc64le .../kernel/device-drivers/zram/zram01.sh | 27 +++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) -- 2.38.0

1 year, 4 months

5
19
0 0

[PATCH v26 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v26:* - Code re-structurring and API changes in PAGEMAP_IOCTL *Changes in v25*: - Do proper filtering on hole as well (hole got missed earlier) *Changes in v24*: - Rebase on top of next-20230710 - Place WP markers in case of hole as well *Changes in v23*: - Set vec_buf_index in loop only when vec_buf_index is set - Return -EFAULT instead of -EINVAL if vec is NULL - Correctly return the walk ending address to the page granularity *Changes in v22*: - Interface change: - Replace [start start + len) with [start, end) - Return the ending address of the address walk in start *Changes in v21*: - Abort walk instead of returning error if WP is to be performed on partial hugetlb *Changes in v20* - Correct PAGE_IS_FILE and add PAGE_IS_PFNZERO *Changes in v19* - Minor changes and interface updates *Changes in v18* - Rebase on top of next-20230613 - Minor updates *Changes in v17* - Rebase on top of next-20230606 - Minor improvements in PAGEMAP_SCAN IOCTL patch *Changes in v16* - Fix a corner case - Add exclusive PM_SCAN_OP_WP back *Changes in v15* - Build fix (Add missed build fix in RESEND) *Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() and ResetWriteWatch() syscalls [1]. The GetWriteWatch() retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirty feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 64 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 653 ++++++++ fs/userfaultfd.c | 26 +- include/linux/hugetlb.h | 1 + include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 58 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 34 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 58 + tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1485 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 16 files changed, 2457 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c -- 2.39.2

1 year, 4 months

4
19
0 0

[PATCH v4 0/6] kselfest/arm64: Fix a SVE memcpy() issue and use tools/include

by Mark Brown

Will noticed that with newer toolchains memcpy() ends up being implemented with SVE instructions, breaking the signals tests when in streaming mode. We fixed this by using an open coded version of OPTIMZER_HIDE_VAR(), but in the process it was noticed that some of the selftests are using the tools/include headers and it might be nice to share things there. We also have a custom compiler.h in the BTI tests. Update the tools/include headers to have what we need, pull them into the arm64 selftests build and make use of them in the signals and BTI tests. Since the resulting changes are a bit invasive for a fix we keep an initial patch using the open coding, updating and replacing that later. Signed-off-by: Mark Brown <broonie(a)kernel.org> --- Changes in v4: - Roll in a refactoring to include and use the tools/include headers. - Link to v3: https://lore.kernel.org/r/20230720-arm64-signal-memcpy-fix-v3-1-08aed2385d6… Changes in v3: - Open code OPTIMISER_HIDE_VAR() instead of the memory clobber. - Link to v2: https://lore.kernel.org/r/20230712-arm64-signal-memcpy-fix-v2-1-494f7025caf… Changes in v2: - Rebase onto v6.5-rc1. - Link to v1: https://lore.kernel.org/r/20230628-arm64-signal-memcpy-fix-v1-1-db3e0300829… --- Mark Brown (6): kselftest/arm64: Exit streaming mode after collecting signal context tools compiler.h: Add OPTIMIZER_HIDE_VAR() tools include: Add some common function attributes kselftest/arm64: Make the tools/include headers available kselftest/arm64: Use shared OPTIMZER_HIDE_VAR() definiton kselftest/arm64: Use the tools/include compiler.h rather than our own tools/include/linux/compiler.h | 18 +++++++++++++++ tools/testing/selftests/arm64/Makefile | 2 ++ tools/testing/selftests/arm64/bti/compiler.h | 21 ----------------- tools/testing/selftests/arm64/bti/system.c | 4 +--- tools/testing/selftests/arm64/bti/system.h | 4 ++-- tools/testing/selftests/arm64/bti/test.c | 1 - .../selftests/arm64/signal/test_signals_utils.h | 27 +++++++++++++++++++++- 7 files changed, 49 insertions(+), 28 deletions(-) --- base-commit: 6eaae198076080886b9e7d57f4ae06fa782f90ef change-id: 20230628-arm64-signal-memcpy-fix-7de3b3c8fa10 Best regards, -- Mark Brown <broonie(a)kernel.org>

1 year, 4 months

2
7
0 0

[PATCH v3 0/3] kunit: Expose some built-in features to modules

by Janusz Krzysztofik

Submit the top-level headers also from the kunit test module notifier initialization callback, so external tools that are parsing dmesg for kunit test output are able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list", so external tools can obtain a list of test cases to be executed in advance and can make a better job on assigning kernel messages interleaved with kunit output to specific tests. Use test filtering functions in kunit module notifier callback functions, so external tools are able to execute individual test cases from kunit test modules in order to still better isolate their potential impact on kernel messages that appear interleaved with output from other tests. v3: Fix CONFIG_GLOB, required by filtering fuctions, not selected when building as a module. v2: Fix new name of a structure moved to kunit namespace not updated across all uses. Janusz Krzysztofik (3): kunit: Report the count of test suites in a module kunit: Make 'list' action available to kunit test modules kunit: Allow kunit test modules to use test filtering include/kunit/test.h | 14 +++++++++++ lib/kunit/Kconfig | 2 +- lib/kunit/executor.c | 57 +++++++++++++++++++++++++------------------- lib/kunit/test.c | 57 +++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 104 insertions(+), 26 deletions(-) -- 2.41.0

1 year, 4 months

3
10
0 0

[PATCH v1 1/1] test_firmware: prevent race conditions by a correct implementation of locking

by Mirsad Todorovac

NOTE: This patch is tested against 5.4 stable NOTE: This is a patch for the 5.4 stable branch, not for the torvalds tree. The torvalds tree, and stable tree 5.10, 5.15, 6.1 and 6.4 branches were fixed in the separate commit ID 4acfe3dfde68 ("test_firmware: prevent race conditions by a correct implementation of locking") which was incompatible with 5.4 Dan Carpenter spotted a race condition in a couple of situations like these in the test_firmware driver: static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { u8 val; int ret; ret = kstrtou8(buf, 10, &val); if (ret) return ret; mutex_lock(&test_fw_mutex); *(u8 *)cfg = val; mutex_unlock(&test_fw_mutex); /* Always return full write size even if we didn't consume all */ return size; } static ssize_t config_num_requests_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { int rc; mutex_lock(&test_fw_mutex); if (test_fw_config->reqs) { pr_err("Must call release_all_firmware prior to changing config\n"); rc = -EINVAL; mutex_unlock(&test_fw_mutex); goto out; } mutex_unlock(&test_fw_mutex); // NOTE: HERE is the race!!! Function can be preempted! // test_fw_config->reqs can change between the release of // the lock about and acquire of the lock in the // test_dev_config_update_u8() rc = test_dev_config_update_u8(buf, count, &test_fw_config->num_requests); out: return rc; } static ssize_t config_read_fw_idx_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { return test_dev_config_update_u8(buf, count, &test_fw_config->read_fw_idx); } The function test_dev_config_update_u8() is called from both the locked and the unlocked context, function config_num_requests_store() and config_read_fw_idx_store() which can both be called asynchronously as they are driver's methods, while test_dev_config_update_u8() and siblings change their argument pointed to by u8 *cfg or similar pointer. To avoid deadlock on test_fw_mutex, the lock is dropped before calling test_dev_config_update_u8() and re-acquired within test_dev_config_update_u8() itself, but alas this creates a race condition. Having two locks wouldn't assure a race-proof mutual exclusion. This situation is best avoided by the introduction of a new, unlocked function __test_dev_config_update_u8() which can be called from the locked context and reducing test_dev_config_update_u8() to: static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { int ret; mutex_lock(&test_fw_mutex); ret = __test_dev_config_update_u8(buf, size, cfg); mutex_unlock(&test_fw_mutex); return ret; } doing the locking and calling the unlocked primitive, which enables both locked and unlocked versions without duplication of code. Fixes: c92316bf8e948 ("test_firmware: add batched firmware tests") Cc: Luis R. Rodriguez <mcgrof(a)kernel.org> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Cc: Russ Weight <russell.h.weight(a)intel.com> Cc: Takashi Iwai <tiwai(a)suse.de> Cc: Tianfei Zhang <tianfei.zhang(a)intel.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: Colin Ian King <colin.i.king(a)gmail.com> Cc: Randy Dunlap <rdunlap(a)infradead.org> Cc: linux-kselftest(a)vger.kernel.org Cc: stable(a)vger.kernel.org # v5.4 Suggested-by: Dan Carpenter <error27(a)gmail.com> Link: https://lore.kernel.org/r/20230509084746.48259-1-mirsad.todorovac@alu.unizg… Signed-off-by: Mirsad Todorovac <mirsad.todorovac(a)alu.unizg.hr> --- lib/test_firmware.c | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/lib/test_firmware.c b/lib/test_firmware.c index 38553944e967..92d7195d5b5b 100644 --- a/lib/test_firmware.c +++ b/lib/test_firmware.c @@ -301,16 +301,26 @@ static ssize_t config_test_show_str(char *dst, return len; } -static int test_dev_config_update_bool(const char *buf, size_t size, - bool *cfg) +static inline int __test_dev_config_update_bool(const char *buf, size_t size, + bool *cfg) { int ret; - mutex_lock(&test_fw_mutex); if (strtobool(buf, cfg) < 0) ret = -EINVAL; else ret = size; + + return ret; +} + +static int test_dev_config_update_bool(const char *buf, size_t size, + bool *cfg) +{ + int ret; + + mutex_lock(&test_fw_mutex); + ret = __test_dev_config_update_bool(buf, size, cfg); mutex_unlock(&test_fw_mutex); return ret; @@ -340,7 +350,7 @@ static ssize_t test_dev_config_show_int(char *buf, int cfg) return snprintf(buf, PAGE_SIZE, "%d\n", val); } -static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) +static inline int __test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { int ret; long new; @@ -352,14 +362,23 @@ static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) if (new > U8_MAX) return -EINVAL; - mutex_lock(&test_fw_mutex); *(u8 *)cfg = new; - mutex_unlock(&test_fw_mutex); /* Always return full write size even if we didn't consume all */ return size; } +static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) +{ + int ret; + + mutex_lock(&test_fw_mutex); + ret = __test_dev_config_update_u8(buf, size, cfg); + mutex_unlock(&test_fw_mutex); + + return ret; +} + static ssize_t test_dev_config_show_u8(char *buf, u8 cfg) { u8 val; @@ -392,10 +411,10 @@ static ssize_t config_num_requests_store(struct device *dev, mutex_unlock(&test_fw_mutex); goto out; } - mutex_unlock(&test_fw_mutex); - rc = test_dev_config_update_u8(buf, count, - &test_fw_config->num_requests); + rc = __test_dev_config_update_u8(buf, count, + &test_fw_config->num_requests); + mutex_unlock(&test_fw_mutex); out: return rc; -- 2.34.1

1 year, 4 months

2
4
0 0

[PATCH] selftests/riscv: fix potential build failure during the "emit_tests" step

by John Hubbard

The riscv selftests (which were modeled after the arm64 selftests) are improperly declaring the "emit_tests" target to depend upon the "all" target. This approach, when combined with commit 9fc96c7c19df ("selftests: error out if kernel header files are not yet built"), has caused build failures [1] on arm64, and is likely to cause similar failures for riscv. To fix this, simply remove the unnecessary "all" dependency from the emit_tests target. The dependency is still effectively honored, because again, invocation is via "install", which also depends upon "all". An alternative approach would be to harden the emit_tests target so that it can depend upon "all", but that's a lot more complicated and hard to get right, and doesn't seem worth it, especially given that emit_tests should probably not be overridden at all. [1] https://lore.kernel.org/20230710-kselftest-fix-arm64-v1-1-48e872844f25@kern… Fixes: 9fc96c7c19df ("selftests: error out if kernel header files are not yet built") Signed-off-by: John Hubbard <jhubbard(a)nvidia.com> --- Andrew, With this, and with my arm64 fix [2] that you've already put into mm-unstable, you should be able to safely drop commit 819187ab8741 ("selftests: fix arm64 test installation"). [2] https://lore.kernel.org/20230711005629.2547838-1-jhubbard@nvidia.com thanks, John Hubbard tools/testing/selftests/riscv/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftests/riscv/Makefile index 9dd629cc86aa..f4b3d5c9af5b 100644 --- a/tools/testing/selftests/riscv/Makefile +++ b/tools/testing/selftests/riscv/Makefile @@ -43,7 +43,7 @@ run_tests: all done # Avoid any output on non riscv on emit_tests -emit_tests: all +emit_tests: @for DIR in $(RISCV_SUBTARGETS); do \ BUILD_TARGET=$(OUTPUT)/$$DIR; \ $(MAKE) OUTPUT=$$BUILD_TARGET -C $$DIR $@; \ base-commit: 3f01e9fed8454dcd89727016c3e5b2fbb8f8e50c prerequisite-patch-id: 37c92f7425689ff069fb83996a25cd98e78d7242 -- 2.41.0

1 year, 4 months

5
4
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror July 2023