- Linux-kselftest-mirror - lists.linaro.org

[PATCH v32 0/6] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v31 and v32*: - Minor updates *Changes in v30*: - Rebase on top of next-20230815 - Minor nitpicks *Changes in v29:* - Polish IOCTL and improve documentation *Changes in v28:* - Fix walk_end and add 17 test cases in selftests patch *Changes in v27:* - Handle review comments and minor improvements - Add performance improvement patch on top with test for easy review *Changes in v26:* - Code re-structurring and API changes in PAGEMAP_IOCTL *Changes in v25*: - Do proper filtering on hole as well (hole got missed earlier) *Changes in v24*: - Rebase on top of next-20230710 - Place WP markers in case of hole as well *Changes in v23*: - Set vec_buf_index in loop only when vec_buf_index is set - Return -EFAULT instead of -EINVAL if vec is NULL - Correctly return the walk ending address to the page granularity *Changes in v22*: - Interface change: - Replace [start start + len) with [start, end) - Return the ending address of the address walk in start *Changes in v21*: - Abort walk instead of returning error if WP is to be performed on partial hugetlb *Changes in v20* - Correct PAGE_IS_FILE and add PAGE_IS_PFNZERO *Changes in v19* - Minor changes and interface updates *Changes in v18* - Rebase on top of next-20230613 - Minor updates *Changes in v17* - Rebase on top of next-20230606 - Minor improvements in PAGEMAP_SCAN IOCTL patch *Changes in v16* - Fix a corner case - Add exclusive PM_SCAN_OP_WP back *Changes in v15* - Build fix (Add missed build fix in RESEND) *Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() and ResetWriteWatch() syscalls [1]. The GetWriteWatch() retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirty feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (5): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs fs/proc/task_mmu: Add fast paths to get/clear PAGE_IS_WRITTEN flag tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 89 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 705 ++++++++ fs/userfaultfd.c | 26 +- include/linux/hugetlb.h | 1 + include/linux/userfaultfd_k.h | 28 +- include/uapi/linux/fs.h | 59 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 34 +- mm/memory.c | 28 +- tools/include/uapi/linux/fs.h | 59 + tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1660 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 16 files changed, 2719 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c -- 2.40.1

2 years

4
11
0 0

clang-17: selftests: exec/load_address_4096 clang: error: argument unused during compilation: '-pie' [-Werror,-Wunused-command-line-argument]

by Naresh Kamboju

Recently we have updated the toolchain clang-17 version. While building selftests exec following warnings / errors noticed on the Linux next with clang-17. But pass with gcc-13. Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> exec/load_address_4096 clang: error: argument unused during compilation: '-pie' [-Werror,-Wunused-command-line-argument] Links: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… - https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20230821/te… Steps to reproduce: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… tuxmake --runtime podman --target-arch x86_64 --toolchain clang-17 --kconfig https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… LLVM=1 LLVM_IAS=1 debugkernel cpupower headers kernel kselftest modules -- Linaro LKFT https://lkft.linaro.org

2 years

1
0
0 0

clang: error: cannot specify -o when generating multiple output files

by Naresh Kamboju

Recently we have updated the toolchain clang-17 version. While building selftests filesystems/binderfs, openat2, resctrl and x86 following warnings / errors noticed on the Linux next with clang-17. But pass with gcc-13. Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> Build logs: ======== binderfs_test clang: error: cannot specify -o when generating multiple output files ... openat2/openat2_test clang: error: cannot specify -o when generating multiple output files ... resctrl/resctrl_tests clang: error: cannot specify -o when generating multiple output files ... make[4]: Entering directory 'tools/testing/selftests/x86' clang --target=x86_64-linux-gnu -fintegrated-as -Werror=unknown-warning-option -Werror=ignored-optimization-argument -Werror=option-ignored -Werror=unused-command-line-argument --target=x86_64-linux-gnu -fintegrated-as -m64 -o /home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest/x86/single_step_syscall_64 -O2 -g -std=gnu99 -pthread -Wall -isystem /home/tuxbuild/.cache/tuxmake/builds/1/build/usr/include -no-pie -DCAN_BUILD_64 single_step_syscall.c helpers.h -lrt -ldl clang: error: cannot specify -o when generating multiple output files make[4]: *** [Makefile:78: /home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest/x86/single_step_syscall_64] Error 1 Links: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… - https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20230821/te… - https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20230821/te… - Steps to reproduce: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… tuxmake --runtime podman --target-arch x86_64 --toolchain clang-17 --kconfig https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… LLVM=1 LLVM_IAS=1 debugkernel cpupower headers kernel kselftest modules -- Linaro LKFT https://lkft.linaro.org

2 years

1
0
0 0

clang-17: selftests: vdso_standalone_test_x86.c:(.text+0x1e6): undefined reference to `memcpy'

by Naresh Kamboju

Recently we have updated toolchain clang-17 version. While building selftests vdso following warnings / errors noticed on the Linux next with clang-17. but pass with gcc-13. Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> make[4]: Entering directory 'tools/testing/selftests/vDSO' clang --target=x86_64-linux-gnu -fintegrated-as -Werror=unknown-warning-option -Werror=ignored-optimization-argument -Werror=option-ignored -Werror=unused-command-line-argument --target=x86_64-linux-gnu -fintegrated-as -std=gnu99 -nostdlib -fno-asynchronous-unwind-tables -fno-stack-protector \ vdso_standalone_test_x86.c parse_vdso.c \ -o /vDSO/vdso_standalone_test_x86 vdso_standalone_test_x86.c:73:16: warning: unknown attribute 'externally_visible' ignored [-Wunknown-attributes] 73 | __attribute__((externally_visible)) void c_main(void **stack) | ^~~~~~~~~~~~~~~~~~ 1 warning generated. parse_vdso.c:65:9: warning: using the result of an assignment as a condition without parentheses [-Wparentheses] 65 | if (g = h & 0xf0000000) | ~~^~~~~~~~~~~~~~~~ parse_vdso.c:65:9: note: place parentheses around the assignment to silence this warning 65 | if (g = h & 0xf0000000) | ^ | ( ) parse_vdso.c:65:9: note: use '==' to turn this assignment into an equality comparison 65 | if (g = h & 0xf0000000) | ^ | == parse_vdso.c:206:22: warning: passing 'const char *' to parameter of type 'const unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign] 206 | ver_hash = elf_hash(version); | ^~~~~~~ parse_vdso.c:59:52: note: passing argument to parameter 'name' here 59 | static unsigned long elf_hash(const unsigned char *name) | ^ parse_vdso.c:207:46: warning: passing 'const char *' to parameter of type 'const unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign] 207 | ELF(Word) chain = vdso_info.bucket[elf_hash(name) % vdso_info.nbucket]; | ^~~~ parse_vdso.c:59:52: note: passing argument to parameter 'name' here 59 | static unsigned long elf_hash(const unsigned char *name) | ^ 3 warnings generated. /usr/bin/x86_64-linux-gnu-ld: /tmp/vdso_standalone_test_x86-31b09f.o: in function `c_main': vdso_standalone_test_x86.c:(.text+0x1e6): undefined reference to `memcpy' clang: error: linker command failed with exit code 1 (use -v to see invocation) make[4]: Leaving directory 'tools/testing/selftests/vDSO' Links: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… - https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20230821/te… Steps to reproduce: - https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… tuxmake --runtime podman --target-arch x86_64 --toolchain clang-17 --kconfig https://storage.tuxsuite.com/public/linaro/lkft/builds/2UHhSg0TPLhjp9Uq9EFc… LLVM=1 LLVM_IAS=1 debugkernel cpupower headers kernel kselftest modules -- Linaro LKFT https://lkft.linaro.org

2 years

1
0
0 0

[PATCH v2 0/7] selftests/resctrl: Rework benchmark command handling

by Ilpo Järvinen

The benchmark command handling (-b) in resctrl selftests is overly complicated code. This series turns the benchmark command immutable to preserve it for all selftests and improves benchmark command related error handling. This series also ends up removing the strcpy() calls which were pointed out earlier. v2: - Added argument length check into patch 1/7 - Updated also -b line in help message. - Document -b argument related "algorithm" - Use asprintf() to convert defined constant int to string - Improved changelog texts - Added \n to ksft_exit_fail_msg() call messages. - Print DEFAULT_SPAN with %u instead of %zu to avoid need to cast it Ilpo Järvinen (7): selftests/resctrl: Ensure the benchmark commands fits to its array selftests/resctrl: Correct benchmark command help selftests/resctrl: Remove bw_report and bm_type from main() selftests/resctrl: Simplify span lifetime selftests/resctrl: Make benchmark command const and build it with pointers selftests/resctrl: Remove ben_count variable selftests/resctrl: Cleanup benchmark argument parsing tools/testing/selftests/resctrl/cache.c | 5 +- tools/testing/selftests/resctrl/cat_test.c | 13 +- tools/testing/selftests/resctrl/cmt_test.c | 34 +++-- tools/testing/selftests/resctrl/mba_test.c | 4 +- tools/testing/selftests/resctrl/mbm_test.c | 7 +- tools/testing/selftests/resctrl/resctrl.h | 19 ++- .../testing/selftests/resctrl/resctrl_tests.c | 122 ++++++++++-------- tools/testing/selftests/resctrl/resctrl_val.c | 10 +- 8 files changed, 125 insertions(+), 89 deletions(-) -- 2.30.2

2 years

1
7
0 0

[PATCH bpf-next v5] selftests/bpf: trace_helpers.c: optimize kallsyms cache

by Rong Tao

From: Rong Tao <rongtao(a)cestc.cn> Static ksyms often have problems because the number of symbols exceeds the MAX_SYMS limit. Like changing the MAX_SYMS from 300000 to 400000 in commit e76a014334a6("selftests/bpf: Bump and validate MAX_SYMS") solves the problem somewhat, but it's not the perfect way. This commit uses dynamic memory allocation, which completely solves the problem caused by the limitation of the number of kallsyms. Acked-by: Stanislav Fomichev <sdf(a)google.com> Signed-off-by: Rong Tao <rongtao(a)cestc.cn> --- v5: Release the allocated memory once the load_kallsyms_refresh() upon error given it's dynamically allocated. v4: https://lore.kernel.org/lkml/tencent_59C74613113F0C728524B2A82FE5540A5E09@q… Make sure most cases we don't need the realloc() path to begin with, and check strdup() return value. v3: https://lore.kernel.org/lkml/tencent_50B4B2622FE7546A5FF9464310650C008509@q… Do not use structs and judge ksyms__add_symbol function return value. v2: https://lore.kernel.org/lkml/tencent_B655EE5E5D463110D70CD2846AB3262EED09@q… Do the usual len/capacity scheme here to amortize the cost of realloc, and don't free symbols. v1: https://lore.kernel.org/lkml/tencent_AB461510B10CD484E0B2F62E3754165F2909@q… --- tools/testing/selftests/bpf/trace_helpers.c | 62 +++++++++++++++++---- 1 file changed, 52 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/selftests/bpf/trace_helpers.c index f83d9f65c65b..0053ba22f0cb 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -18,10 +18,47 @@ #define TRACEFS_PIPE "/sys/kernel/tracing/trace_pipe" #define DEBUGFS_PIPE "/sys/kernel/debug/tracing/trace_pipe" -#define MAX_SYMS 400000 -static struct ksym syms[MAX_SYMS]; +static struct ksym *syms; +static int sym_cap; static int sym_cnt; +static int ksyms__add_symbol(const char *name, unsigned long addr) +{ + void *tmp; + unsigned int new_cap; + + if (sym_cnt + 1 > sym_cap) { + new_cap = sym_cap * 4 / 3; + tmp = realloc(syms, sizeof(struct ksym) * new_cap); + if (!tmp) + return -ENOMEM; + syms = tmp; + sym_cap = new_cap; + } + + tmp = strdup(name); + if (!tmp) + return -ENOMEM; + syms[sym_cnt].addr = addr; + syms[sym_cnt].name = tmp; + + sym_cnt++; + + return 0; +} + +static void ksyms__free(void) +{ + unsigned int i; + + if (!syms) + return; + + for (i = 0; i < sym_cnt; i++) + free(syms[i].name); + free(syms); +} + static int ksym_cmp(const void *p1, const void *p2) { return ((struct ksym *)p1)->addr - ((struct ksym *)p2)->addr; @@ -33,9 +70,14 @@ int load_kallsyms_refresh(void) char func[256], buf[256]; char symbol; void *addr; - int i = 0; + int ret; + /* Make sure most cases we don't need the realloc() path to begin with */ + sym_cap = 400000; sym_cnt = 0; + syms = malloc(sizeof(struct ksym) * sym_cap); + if (!syms) + return -ENOMEM; f = fopen("/proc/kallsyms", "r"); if (!f) @@ -46,17 +88,17 @@ int load_kallsyms_refresh(void) break; if (!addr) continue; - if (i >= MAX_SYMS) - return -EFBIG; - - syms[i].addr = (long) addr; - syms[i].name = strdup(func); - i++; + ret = ksyms__add_symbol(func, (unsigned long)addr); + if (ret) + goto error; } fclose(f); - sym_cnt = i; qsort(syms, sym_cnt, sizeof(struct ksym), ksym_cmp); return 0; + +error: + ksyms__free(); + return ret; } int load_kallsyms(void) -- 2.39.3

2 years

3
8
0 0

[PATCH v4 00/12] Add Intel VT-d nested translation

by Yi Liu

This is to add Intel VT-d nested translation based on IOMMUFD nesting infrastructure. As the iommufd nesting infrastructure series[1], iommu core supports new ops to report iommu hardware information, allocate domains with user data and invalidate stage-1 IOTLB when there is mapping changed in stage-1 page table. The data required in the three paths are vendor-specific, so 1) IOMMU_HWPT_TYPE_VTD_S1 is defined for the Intel VT-d stage-1 page table, it will be used in the stage-1 domain allocation and IOTLB syncing path. struct iommu_hwpt_vtd_s1 is defined to pass user_data for the Intel VT-d stage-1 domain allocation. struct iommu_hwpt_vtd_s1_invalidate is defined to pass the data for the Intel VT-d stage-1 IOTLB invalidation. 2) IOMMU_HW_INFO_TYPE_INTEL_VTD and struct iommu_hw_info_vtd are defined to report iommu hardware information for Intel VT-d. With above IOMMUFD extensions, the intel iommu driver implements the three paths to support nested translation. The first Intel platform supporting nested translation is Sapphire Rapids which, unfortunately, has a hardware errata [2] requiring special treatment. This errata happens when a stage-1 page table page (either level) is located in a stage-2 read-only region. In that case the IOMMU hardware may ignore the stage-2 RO permission and still set the A/D bit in stage-1 page table entries during page table walking. A flag IOMMU_HW_INFO_VTD_ERRATA_772415_SPR17 is introduced to report this errata to userspace. With that restriction the user should either disable nested translation to favor RO stage-2 mappings or ensure no RO stage-2 mapping to enable nested translation. Intel-iommu driver is armed with necessary checks to prevent such mix in patch12 of this series. Qemu currently does add RO mappings though. The vfio agent in Qemu simply maps all valid regions in the GPA address space which certainly includes RO regions e.g. vbios. In reality we don't know a usage relying on DMA reads from the BIOS region. Hence finding a way to skip RO regions (e.g. via a discard manager) in Qemu might be an acceptable tradeoff. The actual change needs more discussion in Qemu community. For now we just hacked Qemu to test. Complete code can be found in [3], corresponding QEMU could can be found in [4]. [1] https://lore.kernel.org/linux-iommu/20230724110406.107212-1-yi.l.liu@intel.… [2] https://www.intel.com/content/www/us/en/content-details/772415/content-deta… [3] https://github.com/yiliu1765/iommufd/tree/iommufd_nesting [4] https://github.com/yiliu1765/qemu/tree/wip/iommufd_rfcv4_nesting Change log: v4: - Remove ascii art tables (Jason) - Drop EMT (Tina, Jason) - Drop MTS and related definitions (Kevin) - Rename macro IOMMU_VTD_PGTBL_ to IOMMU_VTD_S1_ (Kevin) - Rename struct iommu_hwpt_intel_vtd_ to iommu_hwpt_vtd_ (Kevin) - Rename struct iommu_hwpt_intel_vtd to iommu_hwpt_vtd_s1 (Kevin) - Put the vendor specific hwpt alloc data structure before enuma iommu_hwpt_type (Kevin) - Do not trim the higher page levels of S2 domain in nested domain attachment as the S2 domain may have been used independently. (Kevin) - Remove the first-stage pgd check against the maximum address of s2_domain as hw can check it anyhow. It makes sense to check every pfns used in the stage-1 page table. But it cannot make it. So just leave it to hw. (Kevin) - Split the iotlb flush part into an order of uapi, helper and callback implementation (Kevin) - Change the policy of VT-d nesting errata, disallow RO mapping once a domain is used as parent domain of a nested domain. This removes the nested_users counting. (Kevin) - Minor fix for "make htmldocs" v3: https://lore.kernel.org/linux-iommu/20230511145110.27707-1-yi.l.liu@intel.c… - Further split the patches into an order of adding helpers for nested domain, iotlb flush, nested domain attachment and nested domain allocation callback, then report the hw_info to userspace. - Add batch support in cache invalidation from userspace - Disallow nested translation usage if RO mappings exists in stage-2 domain due to errata on readonly mappings on Sapphire Rapids platform. v2: https://lore.kernel.org/linux-iommu/20230309082207.612346-1-yi.l.liu@intel.… - The iommufd infrastructure is split to be separate series. v1: https://lore.kernel.org/linux-iommu/20230209043153.14964-1-yi.l.liu@intel.c… Regards, Yi Liu Lu Baolu (5): iommu/vt-d: Extend dmar_domain to support nested domain iommu/vt-d: Add helper for nested domain allocation iommu/vt-d: Add helper to setup pasid nested translation iommu/vt-d: Add nested domain allocation iommu/vt-d: Disallow nesting on domains with read-only mappings Yi Liu (7): iommufd: Add data structure for Intel VT-d stage-1 domain allocation iommu/vt-d: Make domain attach helpers to be extern iommu/vt-d: Set the nested domain to a device iommufd: Add data structure for Intel VT-d stage-1 cache invalidation iommu/vt-d: Make iotlb flush helpers to be extern iommu/vt-d: Add iotlb flush for nested domain iommu/vt-d: Implement hw_info for iommu capability query drivers/iommu/intel/Makefile | 2 +- drivers/iommu/intel/iommu.c | 80 +++++++++++++--- drivers/iommu/intel/iommu.h | 55 +++++++++-- drivers/iommu/intel/nested.c | 174 +++++++++++++++++++++++++++++++++++ drivers/iommu/intel/pasid.c | 127 +++++++++++++++++++++++++ drivers/iommu/intel/pasid.h | 2 + drivers/iommu/iommufd/main.c | 6 ++ include/linux/iommu.h | 1 + include/uapi/linux/iommufd.h | 124 +++++++++++++++++++++++++ 9 files changed, 549 insertions(+), 22 deletions(-) create mode 100644 drivers/iommu/intel/nested.c -- 2.34.1

2 years

6
74
0 0

[RFC] tools/nolibc: x86_64: Make it compile with -pie

by Vitaly Chikunov

Use RIP-relative addressing when setting `environ` and `_auxv` in startup code. Some toolchains have `-pie` enabled by default. On them or when -pie is specified manually gcc produces error like this: ld: /tmp/cci0uPcR.o: relocation R_X86_64_32S against symbol `environ' can not be used when making a PIE object; recompile with -fPIE ld: failed to set dynamic section sizes: bad value This is because asm() startup code accesses there pointers with absolute addressing. This may inspire others to fix the problem for other architectures too. Signed-off-by: Vitaly Chikunov <vt(a)altlinux.org> --- tools/include/nolibc/arch-x86_64.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/include/nolibc/arch-x86_64.h b/tools/include/nolibc/arch-x86_64.h index 6fc4d8392742..a6be44b333ce 100644 --- a/tools/include/nolibc/arch-x86_64.h +++ b/tools/include/nolibc/arch-x86_64.h @@ -199,14 +199,14 @@ void __attribute__((weak,noreturn,optimize("omit-frame-pointer"))) __no_stack_pr "pop %rdi\n" /* argc (first arg, %rdi) */ "mov %rsp, %rsi\n" /* argv[] (second arg, %rsi) */ "lea 8(%rsi,%rdi,8),%rdx\n" /* then a NULL then envp (third arg, %rdx) */ - "mov %rdx, environ\n" /* save environ */ + "mov %rdx, environ(%rip)\n" /* save environ */ "xor %ebp, %ebp\n" /* zero the stack frame */ "mov %rdx, %rax\n" /* search for auxv (follows NULL after last env) */ "0:\n" "add $8, %rax\n" /* search for auxv using rax, it follows the */ "cmp -8(%rax), %rbp\n" /* ... NULL after last env (rbp is zero here) */ "jnz 0b\n" - "mov %rax, _auxv\n" /* save it into _auxv */ + "mov %rax, _auxv(%rip)\n" /* save it into _auxv */ "and $-16, %rsp\n" /* x86 ABI : esp must be 16-byte aligned before call */ "call main\n" /* main() returns the status code, we'll exit with it. */ "mov %eax, %edi\n" /* retrieve exit code (32 bit) */ -- 2.33.8

2 years

2
2
0 0

[PATCH v1] selftests/net: Add log.txt and tools to .gitignore

by Anh Tuan Phan

Update .gitignore to untrack tools directory and log.txt. "tools" is generated in "selftests/net/Makefile" and log.txt is generated in "selftests/net/gro.sh" when executing run_all_tests. Signed-off-by: Anh Tuan Phan <tuananhlfc(a)gmail.com> --- tools/testing/selftests/net/.gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/.gitignore b/tools/testing/selftests/net/.gitignore index 501854a89cc0..2f9d378edec3 100644 --- a/tools/testing/selftests/net/.gitignore +++ b/tools/testing/selftests/net/.gitignore @@ -15,6 +15,7 @@ ip_local_port_range ipsec ipv6_flowlabel ipv6_flowlabel_mgr +log.txt msg_zerocopy nettest psock_fanout @@ -45,6 +46,7 @@ test_unix_oob timestamping tls toeplitz +tools tun txring_overwrite txtimestamp -- 2.34.1

2 years

2
1
0 0

selftests: net/af_unix test_unix_oob [FAILED]

by Mirsad Todorovac

Hi all, In the kernel 6.5-rc5 build on Ubuntu 22.04 LTS (jammy jellyfish) on a Ryzen 7950 assembled box, vanilla torvalds tree kernel, the test test_unix_oob unexpectedly fails: # selftests: net/af_unix: test_unix_oob # Test 2 failed, sigurg 23 len 63 OOB % It is this code: /* Test 2: * Verify that the first OOB is over written by * the 2nd one and the first OOB is returned as * part of the read, and sigurg is received. */ wait_for_data(pfd, POLLIN | POLLPRI); len = 0; while (len < 70) len = recv(pfd, buf, 1024, MSG_PEEK); len = read_data(pfd, buf, 1024); read_oob(pfd, &oob); if (!signal_recvd || len != 127 || oob != '#') { fprintf(stderr, "Test 2 failed, sigurg %d len %d OOB %c\n", signal_recvd, len, oob); die(1); } In 6.5-rc4, this test was OK, so it might mean we have a regression? marvin@defiant:~/linux/kernel/linux_torvalds$ grep test_unix_oob ../kselftest-6.5-rc4-1.log /net/af_unix/test_unix_oob # selftests: net/af_unix: test_unix_oob ok 2 selftests: net/af_unix: test_unix_oob marvin@defiant:~/linux/kernel/linux_torvalds$ Hope this helps. NOTE: the kernel is vanilla torvalds tree, only "dirty" because the selftests were modified. Kind regards, Mirsad Todorovac

2 years

2
5
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror