January 2023 - Linux-kselftest-mirror

[PATCH v7 00/14] KVM: mm: fd-based approach for supporting KVM guest private memory

by Chao Peng

This is the v7 of this series which tries to implement the fd-based KVM guest private memory. The patches are based on latest kvm/queue branch commit: b9b71f43683a (kvm/queue) KVM: x86/mmu: Buffer nested MMU split_desc_cache only by default capacity Introduction ------------ In general this patch series introduce fd-based memslot which provides guest memory through memory file descriptor fd[offset,size] instead of hva/size. The fd can be created from a supported memory filesystem like tmpfs/hugetlbfs etc. which we refer as memory backing store. KVM and the the memory backing store exchange callbacks when such memslot gets created. At runtime KVM will call into callbacks provided by the backing store to get the pfn with the fd+offset. Memory backing store will also call into KVM callbacks when userspace punch hole on the fd to notify KVM to unmap secondary MMU page table entries. Comparing to existing hva-based memslot, this new type of memslot allows guest memory unmapped from host userspace like QEMU and even the kernel itself, therefore reduce attack surface and prevent bugs. Based on this fd-based memslot, we can build guest private memory that is going to be used in confidential computing environments such as Intel TDX and AMD SEV. When supported, the memory backing store can provide more enforcement on the fd and KVM can use a single memslot to hold both the private and shared part of the guest memory. mm extension --------------------- Introduces new MFD_INACCESSIBLE flag for memfd_create(), the file created with these flags cannot read(), write() or mmap() etc via normal MMU operations. The file content can only be used with the newly introduced memfile_notifier extension. The memfile_notifier extension provides two sets of callbacks for KVM to interact with the memory backing store: - memfile_notifier_ops: callbacks for memory backing store to notify KVM when memory gets invalidated. - backing store callbacks: callbacks for KVM to call into memory backing store to request memory pages for guest private memory. The memfile_notifier extension also provides APIs for memory backing store to register/unregister itself and to trigger the notifier when the bookmarked memory gets invalidated. The patchset also introduces a new memfd seal F_SEAL_AUTO_ALLOCATE to prevent double allocation caused by unintentional guest when we only have a single side of the shared/private memfds effective. memslot extension ----------------- Add the private fd and the fd offset to existing 'shared' memslot so that both private/shared guest memory can live in one single memslot. A page in the memslot is either private or shared. Whether a guest page is private or shared is maintained through reusing existing SEV ioctls KVM_MEMORY_ENCRYPT_{UN,}REG_REGION. Test ---- To test the new functionalities of this patch TDX patchset is needed. Since TDX patchset has not been merged so I did two kinds of test: - Regresion test on kvm/queue (this patchset) Most new code are not covered. Code also in below repo: https://github.com/chao-p/linux/tree/privmem-v7 - New Funational test on latest TDX code The patch is rebased to latest TDX code and tested the new funcationalities. See below repos: Linux: https://github.com/chao-p/linux/tree/privmem-v7-tdx QEMU: https://github.com/chao-p/qemu/tree/privmem-v7 An example QEMU command line for TDX test: -object tdx-guest,id=tdx,debug=off,sept-ve-disable=off \ -machine confidential-guest-support=tdx \ -object memory-backend-memfd-private,id=ram1,size=${mem} \ -machine memory-backend=ram1 Changelog ---------- v7: - Move the private/shared info from backing store to KVM. - Introduce F_SEAL_AUTO_ALLOCATE to avoid double allocation. - Rework on the sync mechanism between zap/page fault paths. - Addressed other comments in v6. v6: - Re-organzied patch for both mm/KVM parts. - Added flags for memfile_notifier so its consumers can state their features and memory backing store can check against these flags. - Put a backing store reference in the memfile_notifier and move pfn_ops into backing store. - Only support boot time backing store register. - Overall KVM part improvement suggested by Sean and some others. v5: - Removed userspace visible F_SEAL_INACCESSIBLE, instead using an in-kernel flag (SHM_F_INACCESSIBLE for shmem). Private fd can only be created by MFD_INACCESSIBLE. - Introduced new APIs for backing store to register itself to memfile_notifier instead of direct function call. - Added the accounting and restriction for MFD_INACCESSIBLE memory. - Added KVM API doc for new memslot extensions and man page for the new MFD_INACCESSIBLE flag. - Removed the overlap check for mapping the same file+offset into multiple gfns due to perf consideration, warned in document. - Addressed other comments in v4. v4: - Decoupled the callbacks between KVM/mm from memfd and use new name 'memfile_notifier'. - Supported register multiple memslots to the same backing store. - Added per-memslot pfn_ops instead of per-system. - Reworked the invalidation part. - Improved new KVM uAPIs (private memslot extension and memory error) per Sean's suggestions. - Addressed many other minor fixes for comments from v3. v3: - Added locking protection when calling invalidate_page_range/fallocate callbacks. - Changed memslot structure to keep use useraddr for shared memory. - Re-organized F_SEAL_INACCESSIBLE and MEMFD_OPS. - Added MFD_INACCESSIBLE flag to force F_SEAL_INACCESSIBLE. - Commit message improvement. - Many small fixes for comments from the last version. Links to previous discussions ----------------------------- [1] Original design proposal: https://lkml.kernel.org/kvm/20210824005248.200037-1-seanjc@google.com/ [2] Updated proposal and RFC patch v1: https://lkml.kernel.org/linux-fsdevel/20211111141352.26311-1-chao.p.peng@li… [3] Patch v5: https://lkml.org/lkml/2022/5/19/861 Chao Peng (12): mm: Add F_SEAL_AUTO_ALLOCATE seal to memfd selftests/memfd: Add tests for F_SEAL_AUTO_ALLOCATE mm: Introduce memfile_notifier mm/memfd: Introduce MFD_INACCESSIBLE flag KVM: Rename KVM_PRIVATE_MEM_SLOTS to KVM_INTERNAL_MEM_SLOTS KVM: Use gfn instead of hva for mmu_notifier_retry KVM: Rename mmu_notifier_* KVM: Extend the memslot to support fd-based private memory KVM: Add KVM_EXIT_MEMORY_FAULT exit KVM: Register/unregister the guest private memory regions KVM: Handle page fault for private memory KVM: Enable and expose KVM_MEM_PRIVATE Kirill A. Shutemov (1): mm/shmem: Support memfile_notifier Documentation/virt/kvm/api.rst | 77 +++++- arch/arm64/kvm/mmu.c | 8 +- arch/mips/include/asm/kvm_host.h | 2 +- arch/mips/kvm/mmu.c | 10 +- arch/powerpc/include/asm/kvm_book3s_64.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_host.c | 4 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 +- arch/powerpc/kvm/book3s_hv_nested.c | 2 +- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 8 +- arch/powerpc/kvm/e500_mmu_host.c | 4 +- arch/riscv/kvm/mmu.c | 4 +- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/Kconfig | 3 + arch/x86/kvm/mmu.h | 2 - arch/x86/kvm/mmu/mmu.c | 74 +++++- arch/x86/kvm/mmu/mmu_internal.h | 18 ++ arch/x86/kvm/mmu/mmutrace.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 4 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 105 +++++--- include/linux/memfile_notifier.h | 91 +++++++ include/linux/shmem_fs.h | 2 + include/uapi/linux/fcntl.h | 1 + include/uapi/linux/kvm.h | 37 +++ include/uapi/linux/memfd.h | 1 + mm/Kconfig | 4 + mm/Makefile | 1 + mm/memfd.c | 18 +- mm/memfile_notifier.c | 123 ++++++++++ mm/shmem.c | 125 +++++++++- tools/testing/selftests/memfd/memfd_test.c | 166 +++++++++++++ virt/kvm/Kconfig | 3 + virt/kvm/kvm_main.c | 272 ++++++++++++++++++--- virt/kvm/pfncache.c | 14 +- 35 files changed, 1074 insertions(+), 127 deletions(-) create mode 100644 include/linux/memfile_notifier.h create mode 100644 mm/memfile_notifier.c -- 2.25.1

1 year

23
171
0 0

[PATCH -next v2 0/6] landlock: add chmod and chown support

by Xiu Jianfeng

v2: * abstract walk_to_visible_parent() helper * chmod and chown rights only take affect on directory's context * add testcase for fchmodat/lchown/fchownat * fix other review issues Xiu Jianfeng (6): landlock: expand access_mask_t to u32 type landlock: abstract walk_to_visible_parent() helper landlock: add chmod and chown support landlock/selftests: add selftests for chmod and chown landlock/samples: add chmod and chown support landlock: update chmod and chown support in document Documentation/userspace-api/landlock.rst | 9 +- include/uapi/linux/landlock.h | 10 +- samples/landlock/sandboxer.c | 13 +- security/landlock/fs.c | 110 ++++++-- security/landlock/limits.h | 2 +- security/landlock/ruleset.h | 2 +- security/landlock/syscalls.c | 2 +- tools/testing/selftests/landlock/base_test.c | 2 +- tools/testing/selftests/landlock/fs_test.c | 267 ++++++++++++++++++- 9 files changed, 386 insertions(+), 31 deletions(-) -- 2.17.1

1 year

4
30
0 0

[PATCH v2] selftests/kselftest/runner/run_one(): Allow running non-executable files

by SeongJae Park

From: SeongJae Park <sjpark(a)amazon.de> When running a test program, 'run_one()' checks if the program has the execution permission and fails if it doesn't. However, it's easy to mistakenly missing the permission, as some common tools like 'diff' don't support the permission change well[1]. Compared to that, making mistakes in the test program's path would only rare, as those are explicitly listed in 'TEST_PROGS'. Therefore, it might make more sense to resolve the situation on our own and run the program. For the reason, this commit makes the test program runner function to still print the warning message but try parsing the interpreter of the program and explicitly run it with the interpreter, in the case. [1] https://lore.kernel.org/mm-commits/YRJisBs9AunccCD4@kroah.com/ Suggested-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: SeongJae Park <sjpark(a)amazon.de> --- Changes from v1 (https://lore.kernel.org/linux-kselftest/20210810140459.23990-1-sj38.park@gm…) - Parse and use the interpreter instead of changing the file tools/testing/selftests/kselftest/runner.sh | 28 +++++++++++++-------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index cc9c846585f0..a9ba782d8ca0 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -33,9 +33,9 @@ tap_timeout() { # Make sure tests will time out if utility is available. if [ -x /usr/bin/timeout ] ; then - /usr/bin/timeout --foreground "$kselftest_timeout" "$1" + /usr/bin/timeout --foreground "$kselftest_timeout" $1 else - "$1" + $1 fi } @@ -65,17 +65,25 @@ run_one() TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST" echo "# $TEST_HDR_MSG" - if [ ! -x "$TEST" ]; then - echo -n "# Warning: file $TEST is " - if [ ! -e "$TEST" ]; then - echo "missing!" - else - echo "not executable, correct this." - fi + if [ ! -e "$TEST" ]; then + echo "# Warning: file $TEST is missing!" echo "not ok $test_num $TEST_HDR_MSG" else + cmd="./$BASENAME_TEST" + if [ ! -x "$TEST" ]; then + echo "# Warning: file $TEST is not executable" + + if [ $(head -n 1 "$TEST" | cut -c -2) = "#!" ] + then + interpreter=$(head -n 1 "$TEST" | cut -c 3-) + cmd="$interpreter ./$BASENAME_TEST" + else + echo "not ok $test_num $TEST_HDR_MSG" + return + fi + fi cd `dirname $TEST` > /dev/null - ((((( tap_timeout ./$BASENAME_TEST 2>&1; echo $? >&3) | + ((((( tap_timeout "$cmd" 2>&1; echo $? >&3) | tap_prefix >&4) 3>&1) | (read xs; exit $xs)) 4>>"$logfile" && echo "ok $test_num $TEST_HDR_MSG") || -- 2.17.1

1 year, 1 month

3
6
0 0

[PATCH v2] selftests: netfilter: fix a build error on openSUSE

by Geliang Tang

This patch fixed the following build error on openSUSE Leap 15.3: ======================================================================= gcc nf-queue.c -lmnl -o tools/testing/selftests/netfilter/nf-queue nf-queue.c:13:10: fatal error: libmnl/libmnl.h: No such file or directory #include <libmnl/libmnl.h> ^~~~~~~~~~~~~~~~~ compilation terminated. ======================================================================= It is because libmnl.h is put in the directory of "/usr/include/libmnl/libmnl/" on openSUSE, not "/usr/include/libmnl/": > rpm -ql libmnl-devel /usr/include/libmnl /usr/include/libmnl/libmnl /usr/include/libmnl/libmnl/libmnl.h /usr/lib64/libmnl.so /usr/lib64/pkgconfig/libmnl.pc Suggested-by: Kai Liu <kai.liu(a)suse.com> Signed-off-by: Geliang Tang <geliang.tang(a)suse.com> --- v2: - use pkg-config --- tools/testing/selftests/netfilter/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/netfilter/Makefile b/tools/testing/selftests/netfilter/Makefile index e4f845dd942b..8136c1fab7ab 100644 --- a/tools/testing/selftests/netfilter/Makefile +++ b/tools/testing/selftests/netfilter/Makefile @@ -8,6 +8,7 @@ TEST_PROGS := nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh \ ipip-conntrack-mtu.sh conntrack_tcp_unreplied.sh \ conntrack_vrf.sh nft_synproxy.sh +CFLAGS += $(shell pkg-config --cflags libmnl 2>/dev/null || echo "-I/usr/include/libmnl") LDLIBS = -lmnl TEST_GEN_FILES = nf-queue -- 2.34.1

1 year, 1 month

5
9
0 0

[PATCH v2] kselftest/alsa: Increase kselftest timeout

by Nícolas F. R. A. Prado

The default timeout for kselftests is 45 seconds, but that isn't enough time to run pcm-test when there are many PCMs on the device, nor for mixer-test when slower control buses and fancier CODECs are present. As data points, running pcm-test on mt8192-asurada-spherion takes about 1m15s, and mixer-test on rk3399-gru-kevin takes about 2m. Set the timeout to 4 minutes to allow both pcm-test and mixer-test to run to completion with some slack. Reviewed-by: Mark Brown <broonie(a)kernel.org> Signed-off-by: Nícolas F. R. A. Prado <nfraprado(a)collabora.com> --- Changes in v2: - Reduced timeout from 10 to 4 minutes - Tweaked commit message to also mention mixer-test and run time for mixer-test on rk3399-gru-kevin tools/testing/selftests/alsa/settings | 1 + 1 file changed, 1 insertion(+) create mode 100644 tools/testing/selftests/alsa/settings diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings new file mode 100644 index 000000000000..b478e684846a --- /dev/null +++ b/tools/testing/selftests/alsa/settings @@ -0,0 +1 @@ +timeout=240 -- 2.39.0

1 year, 2 months

3
3
0 0

[PATCH v1 1/1] .gitignore: Unignore .kunitconfig

by Andy Shevchenko

There are almost dozen of .kunitconfig files that are ignored but tracked. Unignore them. Signed-off-by: Andy Shevchenko <andriy.shevchenko(a)linux.intel.com> --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 22984d22d29e..e4f2ba0be516 100644 --- a/.gitignore +++ b/.gitignore @@ -100,6 +100,7 @@ modules.order !.get_maintainer.ignore !.gitattributes !.gitignore +!.kunitconfig !.mailmap !.rustfmt.toml -- 2.39.0

1 year, 2 months

3
7
0 0

[PATCH v2 0/2] selftests/x86: AMX-related test improvements

by Chang S. Bae

Sorry for the delay in this update. Changes from v1: * Improve the skip message along with the changelog massage (Suah Khan). * Simplify the feature support check (Suah Khan). === Cover Letter === A couple of test updates are included: * With the STRICT_SIGALTSTACK_SIZE option [1,2], the kernel's altstack check becomes stringent. The x86 sigaltstack test is ignorant about this. Adjust the test now. This check was established [3] to ensure every AMX task's altstack is sufficient (regardless of that option) [4]. * The AMX test wrongly fails on non-AMX machines. Fix the code to skip the test instead. The series is available in this repository: git://github.com/intel/amx-linux.git selftest [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arc… [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Doc… [3] 3aac3ebea08f ("x86/signal: Implement sigaltstack size validation") [4] 4b7ca609a33d ("x86/signal: Use fpu::__state_user_size for sigalt stack validation") Chang S. Bae (2): selftests/x86/signal: Adjust the test to the kernel's altstack check selftests/x86/amx: Fix the test to avoid failure when AMX is unavailable tools/testing/selftests/x86/amx.c | 31 ++++++++--------------- tools/testing/selftests/x86/sigaltstack.c | 14 +++++++++- 2 files changed, 23 insertions(+), 22 deletions(-) base-commit: 32346491ddf24599decca06190ebca03ff9de7f8 -- 2.17.1

1 year, 2 months

1
6
0 0

[PATCH v2 00/41] RPCSEC GSS krb5 enhancements

by Chuck Lever

The purpose of this series is to improve/harden the security provided by the Linux kernel's RPCSEC GSS Kerberos 5 mechanism. There are lots of clean-ups in this series, but the pertinent feature is the addition of a clean deprecation path for the DES- and SHA1-based encryption types in accordance with Internet BCPs. This series disables DES-based enctypes by default, provides a mechanism for disabling SHA1-based enctypes, and introduces two modern enctypes that do not use deprecated crypto algorithms. Not only does that improve security for Kerberos 5 users, but it also prepares SunRPC for eventually switching to a shared common kernel Kerberos 5 implementation, which surely will not implement any deprecated encryption types (in particular, DES-based ones). Today, MIT supports both of the newly-introduced enctypes, but Heimdal does not appear to. Thus distributions can enable and disable kernel enctype support to match the set of enctypes supported in their user space Kerberos libraries. Scott has been kicking the tires -- we've found no regressions with the current SHA1-based enctypes, while the new ones are disabled by default until we have an opportunity for interop testing. The KUnit tests for the new enctypes pass and this implementation successfully interoperates with itself using these enctypes. Therefore I believe it to be safe to merge. When this series gets merged, the Linux NFS community should select and announce a date-certain for removal of SunRPC's DES-based enctype code. --- Changes since v1: - Addressed Simo's NAK on "SUNRPC: Improve Kerberos confounder generation" - Added Cc: linux-kselftest@ for review of the KUnit-related patches Chuck Lever (41): SUNRPC: Add header ifdefs to linux/sunrpc/gss_krb5.h SUNRPC: Remove .blocksize field from struct gss_krb5_enctype SUNRPC: Remove .conflen field from struct gss_krb5_enctype SUNRPC: Improve Kerberos confounder generation SUNRPC: Obscure Kerberos session key SUNRPC: Refactor set-up for aux_cipher SUNRPC: Obscure Kerberos encryption keys SUNRPC: Obscure Kerberos signing keys SUNRPC: Obscure Kerberos integrity keys SUNRPC: Refactor the GSS-API Per Message calls in the Kerberos mechanism SUNRPC: Remove another switch on ctx->enctype SUNRPC: Add /proc/net/rpc/gss_krb5_enctypes file NFSD: Replace /proc/fs/nfsd/supported_krb5_enctypes with a symlink SUNRPC: Replace KRB5_SUPPORTED_ENCTYPES macro SUNRPC: Enable rpcsec_gss_krb5.ko to be built without CRYPTO_DES SUNRPC: Remove ->encrypt and ->decrypt methods from struct gss_krb5_enctype SUNRPC: Rename .encrypt_v2 and .decrypt_v2 methods SUNRPC: Hoist KDF into struct gss_krb5_enctype SUNRPC: Clean up cipher set up for v1 encryption types SUNRPC: Parametrize the key length passed to context_v2_alloc_cipher() SUNRPC: Add new subkey length fields SUNRPC: Refactor CBC with CTS into helpers SUNRPC: Add gk5e definitions for RFC 8009 encryption types SUNRPC: Add KDF-HMAC-SHA2 SUNRPC: Add RFC 8009 encryption and decryption functions SUNRPC: Advertise support for RFC 8009 encryption types SUNRPC: Support the Camellia enctypes SUNRPC: Add KDF_FEEDBACK_CMAC SUNRPC: Advertise support for the Camellia encryption types SUNRPC: Move remaining internal definitions to gss_krb5_internal.h SUNRPC: Add KUnit tests for rpcsec_krb5.ko SUNRPC: Export get_gss_krb5_enctype() SUNRPC: Add KUnit tests RFC 3961 Key Derivation SUNRPC: Add Kunit tests for RFC 3962-defined encryption/decryption SUNRPC: Add KDF KUnit tests for the RFC 6803 encryption types SUNRPC: Add checksum KUnit tests for the RFC 6803 encryption types SUNRPC: Add encryption KUnit tests for the RFC 6803 encryption types SUNRPC: Add KDF-HMAC-SHA2 Kunit tests SUNRPC: Add RFC 8009 checksum KUnit tests SUNRPC: Add RFC 8009 encryption KUnit tests SUNRPC: Add encryption self-tests fs/nfsd/nfsctl.c | 74 +- include/linux/sunrpc/gss_krb5.h | 196 +-- include/linux/sunrpc/gss_krb5_enctypes.h | 41 - net/sunrpc/.kunitconfig | 30 + net/sunrpc/Kconfig | 96 +- net/sunrpc/auth_gss/Makefile | 2 + net/sunrpc/auth_gss/auth_gss.c | 17 + net/sunrpc/auth_gss/gss_krb5_crypto.c | 656 +++++-- net/sunrpc/auth_gss/gss_krb5_internal.h | 232 +++ net/sunrpc/auth_gss/gss_krb5_keys.c | 416 ++++- net/sunrpc/auth_gss/gss_krb5_mech.c | 730 +++++--- net/sunrpc/auth_gss/gss_krb5_seal.c | 122 +- net/sunrpc/auth_gss/gss_krb5_seqnum.c | 2 + net/sunrpc/auth_gss/gss_krb5_test.c | 2040 ++++++++++++++++++++++ net/sunrpc/auth_gss/gss_krb5_unseal.c | 63 +- net/sunrpc/auth_gss/gss_krb5_wrap.c | 124 +- net/sunrpc/auth_gss/svcauth_gss.c | 65 + 17 files changed, 4001 insertions(+), 905 deletions(-) delete mode 100644 include/linux/sunrpc/gss_krb5_enctypes.h create mode 100644 net/sunrpc/.kunitconfig create mode 100644 net/sunrpc/auth_gss/gss_krb5_internal.h create mode 100644 net/sunrpc/auth_gss/gss_krb5_test.c -- Chuck Lever

1 year, 2 months

8
64
0 0

[PATCH mm-unstable v1] selftests/vm: cow: Add COW tests for collapsing of PTE-mapped anon THP

by David Hildenbrand

Currently, anonymous PTE-mapped THPs cannot be collapsed in-place: collapsing (e.g., via MADV_COLLAPSE) implies allocating a fresh THP and mapping that new THP via a PMD: as it's a fresh anon THP, it will get the exclusive flag set on the head page and everybody is happy. However, if the kernel would ever support in-place collapse of anonymous THPs (replacing a page table mapping each sub-page of a THP via PTEs with a single PMD mapping the complete THP), exclusivity information stored for each sub-page would have to be collapsed accordingly: (1) All PTEs map !exclusive anon sub-pages: the in-place collapsed THP must not not have the exclusive flag set on the head page mapped by the PMD. This is the easiest case to handle ("simply don't set any exclusive flags"). (2) All PTEs map exclusive anon sub-pages: when collapsing, we have to clear the exclusive flag from all tail pages and only leave the exclusive flag set for the head page. Otherwise, fork() after collapse would not clear the exclusive flags from the tail pages and we'd be in trouble once PTE-mapping the shared THP when writing to shared tail pages that still have the exclusive flag set. This would effectively revert what the PTE-mapping code does when propagating the exclusive flag to all sub-pages. (3) PTEs map a mixture of exclusive and !exclusive anon sub-pages (can happen e.g., due to MADV_DONTFORK before fork()). We must not collapse the THP in-place, otherwise bad things may happen: the exclusive flags of sub-pages would get ignored and the exclusive flag of the head page would get used instead. Now that we have MADV_COLLAPSE in place to trigger collapsing a THP, let's add some test cases that would bail out early, if we'd voluntarily/accidantially unlock in-place collapse for anon THPs and forget about taking proper care of exclusive flags. Running the test on a kernel with MADV_COLLAPSE support: # [INFO] Anonymous THP tests # [RUN] Basic COW after fork() when collapsing before fork() ok 169 No leak from parent into child # [RUN] Basic COW after fork() when collapsing after fork() (fully shared) ok 170 # SKIP MADV_COLLAPSE failed: Invalid argument # [RUN] Basic COW after fork() when collapsing after fork() (lower shared) ok 171 No leak from parent into child # [RUN] Basic COW after fork() when collapsing after fork() (upper shared) ok 172 No leak from parent into child For now, MADV_COLLAPSE always seems to fail if all PTEs map shared sub-pages. Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Shuah Khan <shuah(a)kernel.org> Cc: Hugh Dickins <hughd(a)google.com> Cc: Peter Xu <peterx(a)redhat.com> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: Nadav Amit <nadav.amit(a)gmail.com> Cc: Zach O'Keefe <zokeefe(a)google.com> Cc: Andrea Arcangeli <aarcange(a)redhat.com> Signed-off-by: David Hildenbrand <david(a)redhat.com> --- A patch from Hugh made me explore the wonderful world of in-place collapse of THP, and I was briefly concerned that it would apply to anon THP as well. After thinking about it a bit, I decided to add test cases, to better be safe than sorry in any case, and to document how PG_anon_exclusive is to be handled in that case. --- tools/testing/selftests/vm/cow.c | 228 +++++++++++++++++++++++++++++++ 1 file changed, 228 insertions(+) diff --git a/tools/testing/selftests/vm/cow.c b/tools/testing/selftests/vm/cow.c index 26f6ea3079e2..16216d893d96 100644 --- a/tools/testing/selftests/vm/cow.c +++ b/tools/testing/selftests/vm/cow.c @@ -30,6 +30,10 @@ #include "../kselftest.h" #include "vm_util.h" +#ifndef MADV_COLLAPSE +#define MADV_COLLAPSE 25 +#endif + static size_t pagesize; static int pagemap_fd; static size_t thpsize; @@ -1178,6 +1182,228 @@ static int tests_per_anon_test_case(void) return tests; } +enum anon_thp_collapse_test { + ANON_THP_COLLAPSE_UNSHARED, + ANON_THP_COLLAPSE_FULLY_SHARED, + ANON_THP_COLLAPSE_LOWER_SHARED, + ANON_THP_COLLAPSE_UPPER_SHARED, +}; + +static void do_test_anon_thp_collapse(char *mem, size_t size, + enum anon_thp_collapse_test test) +{ + struct comm_pipes comm_pipes; + char buf; + int ret; + + ret = setup_comm_pipes(&comm_pipes); + if (ret) { + ksft_test_result_fail("pipe() failed\n"); + return; + } + + /* + * Trigger PTE-mapping the THP by temporarily mapping a single subpage + * R/O, such that we can try collapsing it later. + */ + ret = mprotect(mem + pagesize, pagesize, PROT_READ); + if (ret) { + ksft_test_result_fail("mprotect() failed\n"); + goto close_comm_pipes; + } + ret = mprotect(mem + pagesize, pagesize, PROT_READ | PROT_WRITE); + if (ret) { + ksft_test_result_fail("mprotect() failed\n"); + goto close_comm_pipes; + } + + switch (test) { + case ANON_THP_COLLAPSE_UNSHARED: + /* Collapse before actually COW-sharing the page. */ + ret = madvise(mem, size, MADV_COLLAPSE); + if (ret) { + ksft_test_result_skip("MADV_COLLAPSE failed: %s\n", + strerror(errno)); + goto close_comm_pipes; + } + break; + case ANON_THP_COLLAPSE_FULLY_SHARED: + /* COW-share the full PTE-mapped THP. */ + break; + case ANON_THP_COLLAPSE_LOWER_SHARED: + /* Don't COW-share the upper part of the THP. */ + ret = madvise(mem + size / 2, size / 2, MADV_DONTFORK); + if (ret) { + ksft_test_result_fail("MADV_DONTFORK failed\n"); + goto close_comm_pipes; + } + break; + case ANON_THP_COLLAPSE_UPPER_SHARED: + /* Don't COW-share the lower part of the THP. */ + ret = madvise(mem, size / 2, MADV_DONTFORK); + if (ret) { + ksft_test_result_fail("MADV_DONTFORK failed\n"); + goto close_comm_pipes; + } + break; + default: + assert(false); + } + + ret = fork(); + if (ret < 0) { + ksft_test_result_fail("fork() failed\n"); + goto close_comm_pipes; + } else if (!ret) { + switch (test) { + case ANON_THP_COLLAPSE_UNSHARED: + case ANON_THP_COLLAPSE_FULLY_SHARED: + exit(child_memcmp_fn(mem, size, &comm_pipes)); + break; + case ANON_THP_COLLAPSE_LOWER_SHARED: + exit(child_memcmp_fn(mem, size / 2, &comm_pipes)); + break; + case ANON_THP_COLLAPSE_UPPER_SHARED: + exit(child_memcmp_fn(mem + size / 2, size / 2, + &comm_pipes)); + break; + default: + assert(false); + } + } + + while (read(comm_pipes.child_ready[0], &buf, 1) != 1) + ; + + switch (test) { + case ANON_THP_COLLAPSE_UNSHARED: + break; + case ANON_THP_COLLAPSE_UPPER_SHARED: + case ANON_THP_COLLAPSE_LOWER_SHARED: + /* + * Revert MADV_DONTFORK such that we merge the VMAs and are + * able to actually collapse. + */ + ret = madvise(mem, size, MADV_DOFORK); + if (ret) { + ksft_test_result_fail("MADV_DOFORK failed\n"); + write(comm_pipes.parent_ready[1], "0", 1); + wait(&ret); + goto close_comm_pipes; + } + /* FALLTHROUGH */ + case ANON_THP_COLLAPSE_FULLY_SHARED: + /* Collapse before anyone modified the COW-shared page. */ + ret = madvise(mem, size, MADV_COLLAPSE); + if (ret) { + ksft_test_result_skip("MADV_COLLAPSE failed: %s\n", + strerror(errno)); + write(comm_pipes.parent_ready[1], "0", 1); + wait(&ret); + goto close_comm_pipes; + } + break; + default: + assert(false); + } + + /* Modify the page. */ + memset(mem, 0xff, size); + write(comm_pipes.parent_ready[1], "0", 1); + + wait(&ret); + if (WIFEXITED(ret)) + ret = WEXITSTATUS(ret); + else + ret = -EINVAL; + + ksft_test_result(!ret, "No leak from parent into child\n"); +close_comm_pipes: + close_comm_pipes(&comm_pipes); +} + +static void test_anon_thp_collapse_unshared(char *mem, size_t size) +{ + do_test_anon_thp_collapse(mem, size, ANON_THP_COLLAPSE_UNSHARED); +} + +static void test_anon_thp_collapse_fully_shared(char *mem, size_t size) +{ + do_test_anon_thp_collapse(mem, size, ANON_THP_COLLAPSE_FULLY_SHARED); +} + +static void test_anon_thp_collapse_lower_shared(char *mem, size_t size) +{ + do_test_anon_thp_collapse(mem, size, ANON_THP_COLLAPSE_LOWER_SHARED); +} + +static void test_anon_thp_collapse_upper_shared(char *mem, size_t size) +{ + do_test_anon_thp_collapse(mem, size, ANON_THP_COLLAPSE_UPPER_SHARED); +} + +/* + * Test cases that are specific to anonymous THP: pages in private mappings + * that may get shared via COW during fork(). + */ +static const struct test_case anon_thp_test_cases[] = { + /* + * Basic COW test for fork() without any GUP when collapsing a THP + * before fork(). + * + * Re-mapping a PTE-mapped anon THP using a single PMD ("in-place + * collapse") might easily get COW handling wrong when not collapsing + * exclusivity information properly. + */ + { + "Basic COW after fork() when collapsing before fork()", + test_anon_thp_collapse_unshared, + }, + /* Basic COW test, but collapse after COW-sharing a full THP. */ + { + "Basic COW after fork() when collapsing after fork() (fully shared)", + test_anon_thp_collapse_fully_shared, + }, + /* + * Basic COW test, but collapse after COW-sharing the lower half of a + * THP. + */ + { + "Basic COW after fork() when collapsing after fork() (lower shared)", + test_anon_thp_collapse_lower_shared, + }, + /* + * Basic COW test, but collapse after COW-sharing the upper half of a + * THP. + */ + { + "Basic COW after fork() when collapsing after fork() (upper shared)", + test_anon_thp_collapse_upper_shared, + }, +}; + +static void run_anon_thp_test_cases(void) +{ + int i; + + if (!thpsize) + return; + + ksft_print_msg("[INFO] Anonymous THP tests\n"); + + for (i = 0; i < ARRAY_SIZE(anon_thp_test_cases); i++) { + struct test_case const *test_case = &anon_thp_test_cases[i]; + + ksft_print_msg("[RUN] %s\n", test_case->desc); + do_run_with_thp(test_case->fn, THP_RUN_PMD); + } +} + +static int tests_per_anon_thp_test_case(void) +{ + return thpsize ? 1 : 0; +} + typedef void (*non_anon_test_fn)(char *mem, const char *smem, size_t size); static void test_cow(char *mem, const char *smem, size_t size) @@ -1518,6 +1744,7 @@ int main(int argc, char **argv) ksft_print_header(); ksft_set_plan(ARRAY_SIZE(anon_test_cases) * tests_per_anon_test_case() + + ARRAY_SIZE(anon_thp_test_cases) * tests_per_anon_thp_test_case() + ARRAY_SIZE(non_anon_test_cases) * tests_per_non_anon_test_case()); gup_fd = open("/sys/kernel/debug/gup_test", O_RDWR); @@ -1526,6 +1753,7 @@ int main(int argc, char **argv) ksft_exit_fail_msg("opening pagemap failed\n"); run_anon_test_cases(); + run_anon_thp_test_cases(); run_non_anon_test_cases(); err = ksft_get_fail_cnt(); -- 2.39.0

1 year, 2 months

2
2
0 0

[PATCH] KVM: selftest: Add dependency rules in makefile for C source code

by Yu Zhang

Currently, KVM selftests have to run "make clean && make" to rebuild the entire test suite each time a header file is modified. Define "-MD" as an EXTRA_CFLAGS, so we can generate the dependency rules for each target object, whose prerequisites contains the source file and the included header files as well. And including those dependency files in KVM selftests' makefile will release us from such annoyance. Signed-off-by: Yu Zhang <yu.c.zhang(a)linux.intel.com> --- tools/testing/selftests/kvm/Makefile | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 1750f91dd936..b329e0d1a460 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -180,6 +180,8 @@ TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(ARCH_DIR)) TEST_GEN_PROGS_EXTENDED += $(TEST_GEN_PROGS_EXTENDED_$(ARCH_DIR)) LIBKVM += $(LIBKVM_$(ARCH_DIR)) +OVERRIDE_TARGETS = 1 + # lib.mak defines $(OUTPUT), prepends $(OUTPUT)/ to $(TEST_GEN_PROGS), and most # importantly defines, i.e. overwrites, $(CC) (unless `make -e` or `make CC=`, # which causes the environment variable to override the makefile). @@ -198,9 +200,11 @@ CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \ -fno-builtin-memcmp -fno-builtin-memcpy -fno-builtin-memset \ -fno-stack-protector -fno-PIE -I$(LINUX_TOOL_INCLUDE) \ -I$(LINUX_TOOL_ARCH_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude \ - -I$(<D) -Iinclude/$(ARCH_DIR) -I ../rseq -I.. $(EXTRA_CFLAGS) \ + -I$(<D) -Iinclude/$(ARCH_DIR) -I ../rseq -I.. \ $(KHDR_INCLUDES) +EXTRA_CFLAGS += -MD + no-pie-option := $(call try-run, echo 'int main(void) { return 0; }' | \ $(CC) -Werror $(CFLAGS) -no-pie -x c - -o "$$TMP", -no-pie) @@ -218,11 +222,22 @@ LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S)) LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING)) LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) -EXTRA_CLEAN += $(LIBKVM_OBJS) cscope.* +TEST_GEN_OBJ = $(patsubst %, %.o, $(TEST_GEN_PROGS)) +TEST_GEN_OBJ += $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED)) +TEST_DEP_FILES = $(patsubst %.o, %.d, $(TEST_GEN_OBJ)) +TEST_DEP_FILES += $(patsubst %.o, %.d, $(LIBKVM_OBJS)) +-include $(TEST_DEP_FILES) + +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): %: %.o + $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) $< $(LIBKVM_OBJS) $(LDLIBS) -o $@ +$(TEST_GEN_OBJ): %.o: %.c + $(CC) $(CFLAGS) $(CPPFLAGS) $(EXTRA_CFLAGS) $(TARGET_ARCH) -c $< -o $@ + +EXTRA_CLEAN += $(LIBKVM_OBJS) $(TEST_DEP_FILES) $(TEST_GEN_OBJ) cscope.* x := $(shell mkdir -p $(sort $(dir $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ)))) $(LIBKVM_C_OBJ): $(OUTPUT)/%.o: %.c - $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ + $(CC) $(CFLAGS) $(CPPFLAGS) $(EXTRA_CFLAGS) $(TARGET_ARCH) -c $< -o $@ $(LIBKVM_S_OBJ): $(OUTPUT)/%.o: %.S $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ -- 2.25.1

1 year, 2 months

2
2
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror January 2023