- Linux-kselftest-mirror - lists.linaro.org

[PATCH v3 0/4] powerpc/selftests: Add Power10 2nd DAWR selftests

by Ravi Bangoria

Power10 introduced 2nd watchpoint (DAWR). ISA 3.1, Book 3S, Ch 9 - 'Debug Facilities' covers the feature in detail. Kernel patches to enable the 2nd DAWR are already in[1], including kvm enablement[2]. These patches adds selftests for 2nd DAWR. [1]: https://git.kernel.org/torvalds/c/deb2bd9bcc8428d4b65b6ba640ba8b57c1b20b17 [2]: https://git.kernel.org/torvalds/c/bd1de1a0e6eff4bde5ceae969673b85b8446fd6a v2: https://lore.kernel.org/r/20210407054938.312857-1-ravi.bangoria@linux.ibm.c… v2->v3: - Fixed some checkpatch warnings v1: https://lore.kernel.org/r/20200723102058.312282-1-ravi.bangoria@linux.ibm.c… v1->v2: - Kvm patches are already upstream - Rebased selftests to powerpc/next Ravi Bangoria (4): powerpc/selftests/ptrace-hwbreak: Add testcases for 2nd DAWR powerpc/selftests/perf-hwbreak: Coalesce event creation code powerpc/selftests/perf-hwbreak: Add testcases for 2nd DAWR powerpc/selftests: Add selftest to test concurrent perf/ptrace events .../selftests/powerpc/ptrace/.gitignore | 1 + .../testing/selftests/powerpc/ptrace/Makefile | 2 +- .../selftests/powerpc/ptrace/perf-hwbreak.c | 631 +++++++++++++++-- .../selftests/powerpc/ptrace/ptrace-hwbreak.c | 79 +++ .../powerpc/ptrace/ptrace-perf-hwbreak.c | 659 ++++++++++++++++++ 5 files changed, 1330 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/powerpc/ptrace/ptrace-perf-hwbreak.c -- 2.27.0

4 years, 2 months

2
5
0 0

[RFC PATCH v2 0/2] x86/speculation: Add finer control for when to issue IBPB

by Anand K Mistry

It is documented in Documentation/admin-guide/hw-vuln/spectre.rst, that disabling indirect branch speculation for a user-space process creates more overhead and cause it to run slower. The performance hit varies by CPU, but on the AMD A4-9120C and A6-9220C CPUs, a simple ping-pong using pipes between two processes runs ~10x slower when disabling IB speculation. Patch 2, included in this RFC but not intended for commit, is a simple program that demonstrates this issue. Running on a A4-9120C without IB speculation disabled, each process ping-pong takes ~7us: localhost ~ # taskset 1 /usr/local/bin/test ... iters: 262144, t: 1936300, iter/sec: 135383, us/iter: 7 But when IB speculation is disabled, that number increases significantly: localhost ~ # taskset 1 /usr/local/bin/test d ... iters: 16384, t: 1500518, iter/sec: 10918, us/iter: 91 Although this test is a worst-case scenario, we can also consider a real situation: an audio server (i.e. pulse). If we imagine a low-latency capture, with 10ms packets and a concurrent task on the same CPU (i.e. video encoding, for a video call), the audio server will preempt the CPU at a rate of 100HZ. At 91us overhead per preemption (switching to and from the audio process), that's 0.9% overhead for one process doing preemption. In real-world testing (on a A4-9120C), I've seen 9% of CPU used by IBPB when doing a 2-person video call. With this patch, the number of IBPBs issued can be reduced to the minimum necessary, only when there's a potential attacker->victim process switch. Running on the same A4-9120C device, this patch reduces the performance hit of IBPB by ~half, as expected: localhost ~ # taskset 1 /usr/local/bin/test ds ... iters: 32768, t: 1824043, iter/sec: 17964, us/iter: 55 It should be noted, CPUs from multiple vendors experience a performance hit due to IBPB. I also tested a Intel i3-8130U which sees a noticable (~2x) increase in process switch time due to IBPB. IB spec enabled: localhost ~ # taskset 1 /usr/local/bin/test ... iters: 262144, t: 1210821us, iter/sec: 216501, us/iter: 4 IB spec disabled: localhost ~ # taskset 1 /usr/local/bin/test d ... iters: 131072, t: 1257583us, iter/sec: 104225, us/iter: 9 Open questions: - There are a significant number of task flags, which also now reaches the limit of the 'long' on 32-bit systems. Should the 'mode' flags be stored somewhere else? - Having x86-specific flags in linux/sched.h feels wrong. However, this is the mechanism for doing atomic flag updates. Is there an alternate approach? Open tasks: - Documentation - Naming Changes in v2: - Make flag per-process using prctl(). Anand K Mistry (2): x86/speculation: Allow per-process control of when to issue IBPB selftests: Benchmark for the cost of disabling IB speculation arch/x86/include/asm/thread_info.h | 4 + arch/x86/kernel/cpu/bugs.c | 56 +++++++++ arch/x86/kernel/process.c | 10 ++ arch/x86/mm/tlb.c | 51 ++++++-- include/linux/sched.h | 10 ++ include/uapi/linux/prctl.h | 5 + .../testing/selftests/ib_spec/ib_spec_bench.c | 109 ++++++++++++++++++ 7 files changed, 236 insertions(+), 9 deletions(-) create mode 100644 tools/testing/selftests/ib_spec/ib_spec_bench.c -- 2.31.1.498.g6c1eba8ee3d-goog

4 years, 2 months

1
1
0 0

Running kunit_tool on unclean trees

by Marcelo Schmitt

Hi, a friend and I were chasing bug 205219 [1] listed in Bugzilla. We step into something a little bit different when trying to reproduce the buggy behavior. In our try, compilation failed with a message form make asking us to clean the source tree. We couldn't run kunit_tool after compiling the kernel for x86, as described by Ted in the discussion pointed out by the bug report. Steps to reproduce: 0) Run kunit_tool $ ./tools/testing/kunit/kunit.py run Works fine with a clean tree. 1) Compile the kernel for some architecture (we did it for x86_64). 2) Run kunit_tool again $ ./tools/testing/kunit/kunit.py run Fails with a message form make asking us to clean the source tree. Removing the clean source tree check from the top-level Makefile gives us a similar error to what was described in the bug report. We see that after running `git clean -fdx` kunit_tool runs nicely again. However, this is not a real solution since some kernel binaries are erased by git. We also had a look into the commit messages of Masahiro Yamada but couldn't quite grasp why the check for the tree to be clean was added. We could invest more time in this issue but actually don't know how to proceed. We'd be glad to receive any comment about it. We could also try something else if it's a too hard issue for beginners. [1]: https://bugzilla.kernel.org/show_bug.cgi?id=205219 Best Regards, Marcelo

4 years, 2 months

3
3
0 0

[PATCH v3] KVM: x86: Fix KVM_GET_CPUID2 ioctl to return cpuid entries count

by Valeriy Vdovin

KVM_GET_CPUID2 kvm ioctl is not very well documented, but the way it is implemented in function kvm_vcpu_ioctl_get_cpuid2 suggests that even at error path it will try to return number of entries to the caller. But The dispatcher kvm vcpu ioctl dispatcher code in kvm_arch_vcpu_ioctl ignores any output from this function if it sees the error return code. It's very explicit by the code that it was designed to receive some small number of entries to return E2BIG along with the corrected number. This lost logic in the dispatcher code has been restored by removing the lines that check for function return code and skip if error is found. Without it, the ioctl caller will see both the number of entries and the correct error. In selftests relevant function vcpu_get_cpuid has also been modified to utilize the number of cpuid entries returned along with errno E2BIG. Signed-off-by: Valeriy Vdovin <valeriy.vdovin(a)virtuozzo.com> --- v2: - Added description to documentation of KVM_GET_CPUID2. - Copy back nent only if E2BIG is returned. - Fixed error code sign. Documentation/virt/kvm/api.rst | 81 ++++++++++++------- arch/x86/kvm/x86.c | 11 ++- .../selftests/kvm/lib/x86_64/processor.c | 20 +++-- 3 files changed, 73 insertions(+), 39 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 245d80581f15..c7cfe4b9614e 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -711,7 +711,34 @@ resulting CPUID configuration through KVM_GET_CPUID2 in case. }; -4.21 KVM_SET_SIGNAL_MASK +4.21 KVM_GET_CPUID2 +------------------ + +:Capability: basic +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_cpuid (in/out) +:Returns: 0 on success, -1 on error + +Returns a full list of cpuid entries that are supported by this vcpu and were +previously set by KVM_SET_CPUID/KVM_SET_CPUID2. + +The userspace must specify the number of cpuid entries it is ready to accept +from the kernel in the 'nent' field of 'struct kmv_cpuid'. + +The kernel will try to return all the cpuid entries it has in the response. +If the userspace nent value is too small for the full response, the kernel will +set the error code to -E2BIG, set the same 'nent' field to the actual number of +cpuid_entries and return without writing back any entries to the userspace. +The userspace can thus implement a two-call sequence, where the first call is +made with nent set to 0 to read the number of entries from the kernel and +use this response to allocate enough memory for a full response for the second +call. + +The call cal also return with error code -EFAULT in case of other errors. + + +4.22 KVM_SET_SIGNAL_MASK ------------------------ :Capability: basic @@ -737,7 +764,7 @@ signal mask. }; -4.22 KVM_GET_FPU +4.23 KVM_GET_FPU ---------------- :Capability: basic @@ -766,7 +793,7 @@ Reads the floating point state from the vcpu. }; -4.23 KVM_SET_FPU +4.24 KVM_SET_FPU ---------------- :Capability: basic @@ -795,7 +822,7 @@ Writes the floating point state to the vcpu. }; -4.24 KVM_CREATE_IRQCHIP +4.25 KVM_CREATE_IRQCHIP ----------------------- :Capability: KVM_CAP_IRQCHIP, KVM_CAP_S390_IRQCHIP (s390) @@ -817,7 +844,7 @@ Note that on s390 the KVM_CAP_S390_IRQCHIP vm capability needs to be enabled before KVM_CREATE_IRQCHIP can be used. -4.25 KVM_IRQ_LINE +4.26 KVM_IRQ_LINE ----------------- :Capability: KVM_CAP_IRQCHIP @@ -886,7 +913,7 @@ be used for a userspace interrupt controller. }; -4.26 KVM_GET_IRQCHIP +4.27 KVM_GET_IRQCHIP -------------------- :Capability: KVM_CAP_IRQCHIP @@ -911,7 +938,7 @@ KVM_CREATE_IRQCHIP into a buffer provided by the caller. }; -4.27 KVM_SET_IRQCHIP +4.28 KVM_SET_IRQCHIP -------------------- :Capability: KVM_CAP_IRQCHIP @@ -936,7 +963,7 @@ KVM_CREATE_IRQCHIP from a buffer provided by the caller. }; -4.28 KVM_XEN_HVM_CONFIG +4.29 KVM_XEN_HVM_CONFIG ----------------------- :Capability: KVM_CAP_XEN_HVM @@ -972,7 +999,7 @@ fields must be zero. No other flags are currently valid in the struct kvm_xen_hvm_config. -4.29 KVM_GET_CLOCK +4.30 KVM_GET_CLOCK ------------------ :Capability: KVM_CAP_ADJUST_CLOCK @@ -1005,7 +1032,7 @@ TSC is not stable. }; -4.30 KVM_SET_CLOCK +4.31 KVM_SET_CLOCK ------------------ :Capability: KVM_CAP_ADJUST_CLOCK @@ -1027,7 +1054,7 @@ such as migration. }; -4.31 KVM_GET_VCPU_EVENTS +4.32 KVM_GET_VCPU_EVENTS ------------------------ :Capability: KVM_CAP_VCPU_EVENTS @@ -1146,7 +1173,7 @@ directly to the virtual CPU). __u32 reserved[12]; }; -4.32 KVM_SET_VCPU_EVENTS +4.33 KVM_SET_VCPU_EVENTS ------------------------ :Capability: KVM_CAP_VCPU_EVENTS @@ -1209,7 +1236,7 @@ exceptions by manipulating individual registers using the KVM_SET_ONE_REG API. See KVM_GET_VCPU_EVENTS for the data structure. -4.33 KVM_GET_DEBUGREGS +4.34 KVM_GET_DEBUGREGS ---------------------- :Capability: KVM_CAP_DEBUGREGS @@ -1231,7 +1258,7 @@ Reads debug registers from the vcpu. }; -4.34 KVM_SET_DEBUGREGS +4.35 KVM_SET_DEBUGREGS ---------------------- :Capability: KVM_CAP_DEBUGREGS @@ -1246,7 +1273,7 @@ See KVM_GET_DEBUGREGS for the data structure. The flags field is unused yet and must be cleared on entry. -4.35 KVM_SET_USER_MEMORY_REGION +4.36 KVM_SET_USER_MEMORY_REGION ------------------------------- :Capability: KVM_CAP_USER_MEMORY @@ -1315,7 +1342,7 @@ The KVM_SET_MEMORY_REGION does not allow fine grained control over memory allocation and is deprecated. -4.36 KVM_SET_TSS_ADDR +4.37 KVM_SET_TSS_ADDR --------------------- :Capability: KVM_CAP_SET_TSS_ADDR @@ -1335,7 +1362,7 @@ because of a quirk in the virtualization implementation (see the internals documentation when it pops into existence). -4.37 KVM_ENABLE_CAP +4.38 KVM_ENABLE_CAP ------------------- :Capability: KVM_CAP_ENABLE_CAP @@ -1390,7 +1417,7 @@ function properly, this is the place to put them. The vcpu ioctl should be used for vcpu-specific capabilities, the vm ioctl for vm-wide capabilities. -4.38 KVM_GET_MP_STATE +4.39 KVM_GET_MP_STATE --------------------- :Capability: KVM_CAP_MP_STATE @@ -1438,7 +1465,7 @@ For arm/arm64: The only states that are valid are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not. -4.39 KVM_SET_MP_STATE +4.40 KVM_SET_MP_STATE --------------------- :Capability: KVM_CAP_MP_STATE @@ -1460,7 +1487,7 @@ For arm/arm64: The only states that are valid are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not. -4.40 KVM_SET_IDENTITY_MAP_ADDR +4.41 KVM_SET_IDENTITY_MAP_ADDR ------------------------------ :Capability: KVM_CAP_SET_IDENTITY_MAP_ADDR @@ -1484,7 +1511,7 @@ documentation when it pops into existence). Fails if any VCPU has already been created. -4.41 KVM_SET_BOOT_CPU_ID +4.42 KVM_SET_BOOT_CPU_ID ------------------------ :Capability: KVM_CAP_SET_BOOT_CPU_ID @@ -1499,7 +1526,7 @@ is vcpu 0. This ioctl has to be called before vcpu creation, otherwise it will return EBUSY error. -4.42 KVM_GET_XSAVE +4.43 KVM_GET_XSAVE ------------------ :Capability: KVM_CAP_XSAVE @@ -1518,7 +1545,7 @@ otherwise it will return EBUSY error. This ioctl would copy current vcpu's xsave struct to the userspace. -4.43 KVM_SET_XSAVE +4.44 KVM_SET_XSAVE ------------------ :Capability: KVM_CAP_XSAVE @@ -1537,7 +1564,7 @@ This ioctl would copy current vcpu's xsave struct to the userspace. This ioctl would copy userspace's xsave struct to the kernel. -4.44 KVM_GET_XCRS +4.45 KVM_GET_XCRS ----------------- :Capability: KVM_CAP_XCRS @@ -1564,7 +1591,7 @@ This ioctl would copy userspace's xsave struct to the kernel. This ioctl would copy current vcpu's xcrs to the userspace. -4.45 KVM_SET_XCRS +4.46 KVM_SET_XCRS ----------------- :Capability: KVM_CAP_XCRS @@ -1591,7 +1618,7 @@ This ioctl would copy current vcpu's xcrs to the userspace. This ioctl would set vcpu's xcr to the value userspace specified. -4.46 KVM_GET_SUPPORTED_CPUID +4.47 KVM_GET_SUPPORTED_CPUID ---------------------------- :Capability: KVM_CAP_EXT_CPUID @@ -1676,7 +1703,7 @@ if that returns true and you use KVM_CREATE_IRQCHIP, or if you emulate the feature in userspace, then you can enable the feature for KVM_SET_CPUID2. -4.47 KVM_PPC_GET_PVINFO +4.48 KVM_PPC_GET_PVINFO ----------------------- :Capability: KVM_CAP_PPC_GET_PVINFO diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index efc7a82ab140..3f941b1f4e78 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4773,14 +4773,17 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r = -EFAULT; if (copy_from_user(&cpuid, cpuid_arg, sizeof(cpuid))) goto out; + r = kvm_vcpu_ioctl_get_cpuid2(vcpu, &cpuid, cpuid_arg->entries); - if (r) + + if (r && r != -E2BIG) goto out; - r = -EFAULT; - if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) + + if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) { + r = -EFAULT; goto out; - r = 0; + } break; } case KVM_GET_MSRS: { diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index a8906e60a108..a412b39ad791 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -727,17 +727,21 @@ struct kvm_cpuid2 *vcpu_get_cpuid(struct kvm_vm *vm, uint32_t vcpuid) cpuid = allocate_kvm_cpuid2(); max_ent = cpuid->nent; + cpuid->nent = 0; - for (cpuid->nent = 1; cpuid->nent <= max_ent; cpuid->nent++) { - rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); - if (!rc) - break; + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); + TEST_ASSERT(rc == -1 && errno == E2BIG, + "KVM_GET_CPUID2 should return E2BIG: %d %d", + rc, errno); - TEST_ASSERT(rc == -1 && errno == E2BIG, - "KVM_GET_CPUID2 should either succeed or give E2BIG: %d %d", - rc, errno); - } + TEST_ASSERT(cpuid->nent, + "KVM_GET_CPUID2 failed to set cpuid->nent with E2BIG"); + + TEST_ASSERT(cpuid->nent < max_ent, + "KVM_GET_CPUID2 has %d entries, expected maximum: %d", + cpuid->nent, max_ent); + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); TEST_ASSERT(rc == 0, "KVM_GET_CPUID2 failed, rc: %i errno: %i", rc, errno); -- 2.17.1

4 years, 2 months

1
0
0 0

[PATCH v2] KVM: x86: Fix KVM_GET_CPUID2 ioctl to return cpuid entries count

by Valeriy Vdovin

KVM_GET_CPUID2 kvm ioctl is not very well documented, but the way it is implemented in function kvm_vcpu_ioctl_get_cpuid2 suggests that even at error path it will try to return number of entries to the caller. But The dispatcher kvm vcpu ioctl dispatcher code in kvm_arch_vcpu_ioctl ignores any output from this function if it sees the error return code. It's very explicit by the code that it was designed to receive some small number of entries to return E2BIG along with the corrected number. This lost logic in the dispatcher code has been restored by removing the lines that check for function return code and skip if error is found. Without it, the ioctl caller will see both the number of entries and the correct error. In selftests relevant function vcpu_get_cpuid has also been modified to utilize the number of cpuid entries returned along with errno E2BIG. Signed-off-by: Valeriy Vdovin <valeriy.vdovin(a)virtuozzo.com> --- v2: - Added description to documentation of KVM_GET_CPUID2. - Copy back nent only if E2BIG is returned. Documentation/virt/kvm/api.rst | 81 ++++++++++++------- arch/x86/kvm/x86.c | 11 ++- .../selftests/kvm/lib/x86_64/processor.c | 20 +++-- 3 files changed, 73 insertions(+), 39 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 245d80581f15..c7cfe4b9614e 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -711,7 +711,34 @@ resulting CPUID configuration through KVM_GET_CPUID2 in case. }; -4.21 KVM_SET_SIGNAL_MASK +4.21 KVM_GET_CPUID2 +------------------ + +:Capability: basic +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_cpuid (in/out) +:Returns: 0 on success, -1 on error + +Returns a full list of cpuid entries that are supported by this vcpu and were +previously set by KVM_SET_CPUID/KVM_SET_CPUID2. + +The userspace must specify the number of cpuid entries it is ready to accept +from the kernel in the 'nent' field of 'struct kmv_cpuid'. + +The kernel will try to return all the cpuid entries it has in the response. +If the userspace nent value is too small for the full response, the kernel will +set the error code to -E2BIG, set the same 'nent' field to the actual number of +cpuid_entries and return without writing back any entries to the userspace. +The userspace can thus implement a two-call sequence, where the first call is +made with nent set to 0 to read the number of entries from the kernel and +use this response to allocate enough memory for a full response for the second +call. + +The call cal also return with error code -EFAULT in case of other errors. + + +4.22 KVM_SET_SIGNAL_MASK ------------------------ :Capability: basic @@ -737,7 +764,7 @@ signal mask. }; -4.22 KVM_GET_FPU +4.23 KVM_GET_FPU ---------------- :Capability: basic @@ -766,7 +793,7 @@ Reads the floating point state from the vcpu. }; -4.23 KVM_SET_FPU +4.24 KVM_SET_FPU ---------------- :Capability: basic @@ -795,7 +822,7 @@ Writes the floating point state to the vcpu. }; -4.24 KVM_CREATE_IRQCHIP +4.25 KVM_CREATE_IRQCHIP ----------------------- :Capability: KVM_CAP_IRQCHIP, KVM_CAP_S390_IRQCHIP (s390) @@ -817,7 +844,7 @@ Note that on s390 the KVM_CAP_S390_IRQCHIP vm capability needs to be enabled before KVM_CREATE_IRQCHIP can be used. -4.25 KVM_IRQ_LINE +4.26 KVM_IRQ_LINE ----------------- :Capability: KVM_CAP_IRQCHIP @@ -886,7 +913,7 @@ be used for a userspace interrupt controller. }; -4.26 KVM_GET_IRQCHIP +4.27 KVM_GET_IRQCHIP -------------------- :Capability: KVM_CAP_IRQCHIP @@ -911,7 +938,7 @@ KVM_CREATE_IRQCHIP into a buffer provided by the caller. }; -4.27 KVM_SET_IRQCHIP +4.28 KVM_SET_IRQCHIP -------------------- :Capability: KVM_CAP_IRQCHIP @@ -936,7 +963,7 @@ KVM_CREATE_IRQCHIP from a buffer provided by the caller. }; -4.28 KVM_XEN_HVM_CONFIG +4.29 KVM_XEN_HVM_CONFIG ----------------------- :Capability: KVM_CAP_XEN_HVM @@ -972,7 +999,7 @@ fields must be zero. No other flags are currently valid in the struct kvm_xen_hvm_config. -4.29 KVM_GET_CLOCK +4.30 KVM_GET_CLOCK ------------------ :Capability: KVM_CAP_ADJUST_CLOCK @@ -1005,7 +1032,7 @@ TSC is not stable. }; -4.30 KVM_SET_CLOCK +4.31 KVM_SET_CLOCK ------------------ :Capability: KVM_CAP_ADJUST_CLOCK @@ -1027,7 +1054,7 @@ such as migration. }; -4.31 KVM_GET_VCPU_EVENTS +4.32 KVM_GET_VCPU_EVENTS ------------------------ :Capability: KVM_CAP_VCPU_EVENTS @@ -1146,7 +1173,7 @@ directly to the virtual CPU). __u32 reserved[12]; }; -4.32 KVM_SET_VCPU_EVENTS +4.33 KVM_SET_VCPU_EVENTS ------------------------ :Capability: KVM_CAP_VCPU_EVENTS @@ -1209,7 +1236,7 @@ exceptions by manipulating individual registers using the KVM_SET_ONE_REG API. See KVM_GET_VCPU_EVENTS for the data structure. -4.33 KVM_GET_DEBUGREGS +4.34 KVM_GET_DEBUGREGS ---------------------- :Capability: KVM_CAP_DEBUGREGS @@ -1231,7 +1258,7 @@ Reads debug registers from the vcpu. }; -4.34 KVM_SET_DEBUGREGS +4.35 KVM_SET_DEBUGREGS ---------------------- :Capability: KVM_CAP_DEBUGREGS @@ -1246,7 +1273,7 @@ See KVM_GET_DEBUGREGS for the data structure. The flags field is unused yet and must be cleared on entry. -4.35 KVM_SET_USER_MEMORY_REGION +4.36 KVM_SET_USER_MEMORY_REGION ------------------------------- :Capability: KVM_CAP_USER_MEMORY @@ -1315,7 +1342,7 @@ The KVM_SET_MEMORY_REGION does not allow fine grained control over memory allocation and is deprecated. -4.36 KVM_SET_TSS_ADDR +4.37 KVM_SET_TSS_ADDR --------------------- :Capability: KVM_CAP_SET_TSS_ADDR @@ -1335,7 +1362,7 @@ because of a quirk in the virtualization implementation (see the internals documentation when it pops into existence). -4.37 KVM_ENABLE_CAP +4.38 KVM_ENABLE_CAP ------------------- :Capability: KVM_CAP_ENABLE_CAP @@ -1390,7 +1417,7 @@ function properly, this is the place to put them. The vcpu ioctl should be used for vcpu-specific capabilities, the vm ioctl for vm-wide capabilities. -4.38 KVM_GET_MP_STATE +4.39 KVM_GET_MP_STATE --------------------- :Capability: KVM_CAP_MP_STATE @@ -1438,7 +1465,7 @@ For arm/arm64: The only states that are valid are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not. -4.39 KVM_SET_MP_STATE +4.40 KVM_SET_MP_STATE --------------------- :Capability: KVM_CAP_MP_STATE @@ -1460,7 +1487,7 @@ For arm/arm64: The only states that are valid are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not. -4.40 KVM_SET_IDENTITY_MAP_ADDR +4.41 KVM_SET_IDENTITY_MAP_ADDR ------------------------------ :Capability: KVM_CAP_SET_IDENTITY_MAP_ADDR @@ -1484,7 +1511,7 @@ documentation when it pops into existence). Fails if any VCPU has already been created. -4.41 KVM_SET_BOOT_CPU_ID +4.42 KVM_SET_BOOT_CPU_ID ------------------------ :Capability: KVM_CAP_SET_BOOT_CPU_ID @@ -1499,7 +1526,7 @@ is vcpu 0. This ioctl has to be called before vcpu creation, otherwise it will return EBUSY error. -4.42 KVM_GET_XSAVE +4.43 KVM_GET_XSAVE ------------------ :Capability: KVM_CAP_XSAVE @@ -1518,7 +1545,7 @@ otherwise it will return EBUSY error. This ioctl would copy current vcpu's xsave struct to the userspace. -4.43 KVM_SET_XSAVE +4.44 KVM_SET_XSAVE ------------------ :Capability: KVM_CAP_XSAVE @@ -1537,7 +1564,7 @@ This ioctl would copy current vcpu's xsave struct to the userspace. This ioctl would copy userspace's xsave struct to the kernel. -4.44 KVM_GET_XCRS +4.45 KVM_GET_XCRS ----------------- :Capability: KVM_CAP_XCRS @@ -1564,7 +1591,7 @@ This ioctl would copy userspace's xsave struct to the kernel. This ioctl would copy current vcpu's xcrs to the userspace. -4.45 KVM_SET_XCRS +4.46 KVM_SET_XCRS ----------------- :Capability: KVM_CAP_XCRS @@ -1591,7 +1618,7 @@ This ioctl would copy current vcpu's xcrs to the userspace. This ioctl would set vcpu's xcr to the value userspace specified. -4.46 KVM_GET_SUPPORTED_CPUID +4.47 KVM_GET_SUPPORTED_CPUID ---------------------------- :Capability: KVM_CAP_EXT_CPUID @@ -1676,7 +1703,7 @@ if that returns true and you use KVM_CREATE_IRQCHIP, or if you emulate the feature in userspace, then you can enable the feature for KVM_SET_CPUID2. -4.47 KVM_PPC_GET_PVINFO +4.48 KVM_PPC_GET_PVINFO ----------------------- :Capability: KVM_CAP_PPC_GET_PVINFO diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index efc7a82ab140..fa9bb6b751c6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4773,14 +4773,17 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r = -EFAULT; if (copy_from_user(&cpuid, cpuid_arg, sizeof(cpuid))) goto out; + r = kvm_vcpu_ioctl_get_cpuid2(vcpu, &cpuid, cpuid_arg->entries); - if (r) + + if (r && r != E2BIG) goto out; - r = -EFAULT; - if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) + + if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) { + r = -EFAULT; goto out; - r = 0; + } break; } case KVM_GET_MSRS: { diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index a8906e60a108..a412b39ad791 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -727,17 +727,21 @@ struct kvm_cpuid2 *vcpu_get_cpuid(struct kvm_vm *vm, uint32_t vcpuid) cpuid = allocate_kvm_cpuid2(); max_ent = cpuid->nent; + cpuid->nent = 0; - for (cpuid->nent = 1; cpuid->nent <= max_ent; cpuid->nent++) { - rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); - if (!rc) - break; + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); + TEST_ASSERT(rc == -1 && errno == E2BIG, + "KVM_GET_CPUID2 should return E2BIG: %d %d", + rc, errno); - TEST_ASSERT(rc == -1 && errno == E2BIG, - "KVM_GET_CPUID2 should either succeed or give E2BIG: %d %d", - rc, errno); - } + TEST_ASSERT(cpuid->nent, + "KVM_GET_CPUID2 failed to set cpuid->nent with E2BIG"); + + TEST_ASSERT(cpuid->nent < max_ent, + "KVM_GET_CPUID2 has %d entries, expected maximum: %d", + cpuid->nent, max_ent); + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); TEST_ASSERT(rc == 0, "KVM_GET_CPUID2 failed, rc: %i errno: %i", rc, errno); -- 2.17.1

4 years, 2 months

1
0
0 0

[PATCH] KVM: x86: Fix KVM_GET_CPUID2 ioctl to return cpuid entries count

by Valeriy Vdovin

KVM_GET_CPUID2 kvm ioctl is not very well documented, but the way it is implemented in function kvm_vcpu_ioctl_get_cpuid2 suggests that even at error path it will try to return number of entries to the caller. But The dispatcher kvm vcpu ioctl dispatcher code in kvm_arch_vcpu_ioctl ignores any output from this function if it sees the error return code. It's very explicit by the code that it was designed to receive some small number of entries to return E2BIG along with the corrected number. This lost logic in the dispatcher code has been restored by removing the lines that check for function return code and skip if error is found. Without it, the ioctl caller will see both the number of entries and the correct error. In selftests relevant function vcpu_get_cpuid has also been modified to utilize the number of cpuid entries returned along with errno E2BIG. Signed-off-by: Valeriy Vdovin <valeriy.vdovin(a)virtuozzo.com> --- arch/x86/kvm/x86.c | 10 +++++----- .../selftests/kvm/lib/x86_64/processor.c | 20 +++++++++++-------- 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index efc7a82ab140..df8a3e44e722 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4773,14 +4773,14 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r = -EFAULT; if (copy_from_user(&cpuid, cpuid_arg, sizeof(cpuid))) goto out; + r = kvm_vcpu_ioctl_get_cpuid2(vcpu, &cpuid, cpuid_arg->entries); - if (r) - goto out; - r = -EFAULT; - if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) + + if (copy_to_user(cpuid_arg, &cpuid, sizeof(cpuid))) { + r = -EFAULT; goto out; - r = 0; + } break; } case KVM_GET_MSRS: { diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index a8906e60a108..a412b39ad791 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -727,17 +727,21 @@ struct kvm_cpuid2 *vcpu_get_cpuid(struct kvm_vm *vm, uint32_t vcpuid) cpuid = allocate_kvm_cpuid2(); max_ent = cpuid->nent; + cpuid->nent = 0; - for (cpuid->nent = 1; cpuid->nent <= max_ent; cpuid->nent++) { - rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); - if (!rc) - break; + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); + TEST_ASSERT(rc == -1 && errno == E2BIG, + "KVM_GET_CPUID2 should return E2BIG: %d %d", + rc, errno); - TEST_ASSERT(rc == -1 && errno == E2BIG, - "KVM_GET_CPUID2 should either succeed or give E2BIG: %d %d", - rc, errno); - } + TEST_ASSERT(cpuid->nent, + "KVM_GET_CPUID2 failed to set cpuid->nent with E2BIG"); + + TEST_ASSERT(cpuid->nent < max_ent, + "KVM_GET_CPUID2 has %d entries, expected maximum: %d", + cpuid->nent, max_ent); + rc = ioctl(vcpu->fd, KVM_GET_CPUID2, cpuid); TEST_ASSERT(rc == 0, "KVM_GET_CPUID2 failed, rc: %i errno: %i", rc, errno); -- 2.17.1

4 years, 2 months

3
3
0 0

[GIT PULL] KUnit update for Linux 5.13-rc1

by Shuah Khan

Hi Linus, Please pull the following KUnit update for Linux 5.13-rc1. This KUnit update for Linux 5.13-rc1 consists of several fixes and new feature to support failure from dynamic analysis tools such as UBSAN and fake ops for testing. - a fake ops struct for testing a "free" function to complain if it was called with an invalid argument, or caught a double-free. Most return void and have no normal means of signalling failure (e.g. super_operations, iommu_ops, etc.). diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit a38fd8748464831584a19438cbb3082b5a2dab15: Linux 5.12-rc2 (2021-03-05 17:33:41 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-kunit-5.13-rc1 for you to fetch changes up to de2fcb3e62013738f22bbb42cbd757d9a242574e: Documentation: kunit: add tips for using current->kunit_test (2021-04-07 16:40:37 -0600) ---------------------------------------------------------------- linux-kselftest-kunit-5.13-rc1 This KUnit update for Linux 5.13-rc1 consists of several fixes and new feature to support failure from dynamic analysis tools such as UBSAN and fake ops for testing. - a fake ops struct for testing a "free" function to complain if it was called with an invalid argument, or caught a double-free. Most return void and have no normal means of signalling failure (e.g. super_operations, iommu_ops, etc.). ---------------------------------------------------------------- Daniel Latypov (4): kunit: make KUNIT_EXPECT_STREQ() quote values, don't print literals kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment kunit: fix -Wunused-function warning for __kunit_fail_current_test Documentation: kunit: add tips for using current->kunit_test Lucas Stankus (1): kunit: Match parenthesis alignment to improve code readability Uriel Guajardo (1): kunit: support failure from dynamic analysis tools Documentation/dev-tools/kunit/tips.rst | 78 +++++++++++++++++++++++++++++++++- include/kunit/test-bug.h | 29 +++++++++++++ lib/kunit/.kunitconfig | 3 ++ lib/kunit/assert.c | 61 ++++++++++++++++++-------- lib/kunit/test.c | 39 +++++++++++++++-- tools/testing/kunit/kunit.py | 4 +- tools/testing/kunit/kunit_kernel.py | 2 + tools/testing/kunit/kunit_tool_test.py | 6 +++ 8 files changed, 198 insertions(+), 24 deletions(-) create mode 100644 include/kunit/test-bug.h create mode 100644 lib/kunit/.kunitconfig ----------------------------------------------------------------

4 years, 2 months

2
1
0 0

[GIT PULL] Kselftest update for Linux 5.13-rc1

by Shuah Khan

Hi Linus, Please pull the following Kselftest update for Linux 5.13-rc1. This Kselftest update for Linux 5.13-rc1 consists of: - fixes and updates to resctrl test from Fenghua Yu and Reinette Chatre - fixes to Kselftest documentation, framework - minor spelling correction in timers test diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit a38fd8748464831584a19438cbb3082b5a2dab15: Linux 5.12-rc2 (2021-03-05 17:33:41 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-next-5.13-rc1 for you to fetch changes up to e75074781f1735c1976bc551e29ccf2ba9a4b17f: selftests/resctrl: Change a few printed messages (2021-04-07 16:37:49 -0600) ---------------------------------------------------------------- linux-kselftest-next-5.13-rc1 This Kselftest update for Linux 5.13-rc1 consists of: - fixes and updates to resctrl test from Fenghua Yu and Reinette Chatre - fixes to Kselftest documentation, framework - minor spelling correction in timers test ---------------------------------------------------------------- Antonio Terceiro (1): Documentation: kselftest: fix path to test module files Colin Ian King (1): selftests/timers: Fix spelling mistake "clocksourc" -> "clocksource" Fenghua Yu (20): selftests/resctrl: Enable gcc checks to detect buffer overflows selftests/resctrl: Fix compilation issues for global variables selftests/resctrl: Fix compilation issues for other global variables selftests/resctrl: Clean up resctrl features check selftests/resctrl: Fix missing options "-n" and "-p" selftests/resctrl: Rename CQM test as CMT test selftests/resctrl: Call kselftest APIs to log test results selftests/resctrl: Share show_cache_info() by CAT and CMT tests selftests/resctrl: Add config dependencies selftests/resctrl: Check for resctrl mount point only if resctrl FS is supported selftests/resctrl: Use resctrl/info for feature detection selftests/resctrl: Fix MBA/MBM results reporting format selftests/resctrl: Don't hard code value of "no_of_bits" variable selftests/resctrl: Modularize resctrl test suite main() function selftests/resctrl: Skip the test if requested resctrl feature is not supported selftests/resctrl: Fix unmount resctrl FS selftests/resctrl: Fix incorrect parsing of iMC counters selftests/resctrl: Fix checking for < 0 for unsigned values selftests/resctrl: Create .gitignore to include resctrl_tests selftests/resctrl: Change a few printed messages Ilya Leoshkevich (1): selftests: fix prepending $(OUTPUT) to $(TEST_PROGS) Reinette Chatre (2): selftests/resctrl: Ensure sibling CPU is not same as original CPU selftests/resctrl: Fix a printed message Documentation/dev-tools/kselftest.rst | 4 +- tools/testing/selftests/lib.mk | 3 +- tools/testing/selftests/resctrl/.gitignore | 2 + tools/testing/selftests/resctrl/Makefile | 2 +- tools/testing/selftests/resctrl/README | 4 +- tools/testing/selftests/resctrl/cache.c | 52 ++++++- tools/testing/selftests/resctrl/cat_test.c | 57 +++---- .../selftests/resctrl/{cqm_test.c => cmt_test.c} | 75 +++------- tools/testing/selftests/resctrl/config | 2 + tools/testing/selftests/resctrl/fill_buf.c | 4 +- tools/testing/selftests/resctrl/mba_test.c | 43 +++--- tools/testing/selftests/resctrl/mbm_test.c | 42 +++--- tools/testing/selftests/resctrl/resctrl.h | 29 +++- tools/testing/selftests/resctrl/resctrl_tests.c | 163 ++++++++++++++------- tools/testing/selftests/resctrl/resctrl_val.c | 95 +++++++----- tools/testing/selftests/resctrl/resctrlfs.c | 134 ++++++++++------- .../testing/selftests/timers/clocksource-switch.c | 2 +- 17 files changed, 413 insertions(+), 300 deletions(-) create mode 100644 tools/testing/selftests/resctrl/.gitignore rename tools/testing/selftests/resctrl/{cqm_test.c => cmt_test.c} (56%) create mode 100644 tools/testing/selftests/resctrl/config ----------------------------------------------------------------

4 years, 2 months

2
1
0 0

[PATCH bpf-next v5 0/2] add batched ops for percpu array

by Pedro Tammela

This patchset introduces batched operations for the per-cpu variant of the array map. Also updates the batch ops test for arrays. v4 -> v5: - Revert removal of percpu macros v3 -> v4: - Prefer 'calloc()' over 'malloc()' on batch ops tests - Add missing static keyword in a couple of test functions - 'offset' to 'cpu_offset' as suggested by Martin v2 -> v3: - Remove percpu macros as suggested by Andrii - Update tests that used the per cpu macros v1 -> v2: - Amended a more descriptive commit message Pedro Tammela (2): bpf: add batched ops support for percpu array bpf: selftests: update array map tests for per-cpu batched ops kernel/bpf/arraymap.c | 2 + .../bpf/map_tests/array_map_batch_ops.c | 104 +++++++++++++----- 2 files changed, 77 insertions(+), 29 deletions(-) -- 2.25.1

4 years, 2 months

2
3
0 0

[PATCH v4 00/10] userfaultfd: add minor fault handling for shmem

by Axel Rasmussen

Base ==== This series is based on (and therefore should apply cleanly to) the tag "v5.12-rc7-mmots-2021-04-11-20-49", additionally with Peter's selftest cleanup series applied first: https://lore.kernel.org/patchwork/cover/1412450/ Changelog ========= v3->v4: - Fix handling of the shmem private mcopy case. Previously, I had (incorrectly) assumed that !vma_is_anonymous() was equivalent to "the page will be in the page cache". But, in this case we have an optimization where we allocate a new *anonymous* page. So, use a new "bool page_in_cache" instead, which checks if page->mapping is set. Correct several places with this new check. [Hugh] - Fix calling mm_counter() before page_add_..._rmap(). [Hugh] - When modifying shmem_mcopy_atomic_pte() to use the new install_pte() helper, just use lru_cache_add_inactive_or_unevictable(), no need to branch and maybe use lru_cache_add(). [Hugh] - De-pluralize mcopy_atomic_install_pte(s). [Hugh] - Make "writable" a bool, and initialize consistently. [Hugh] v2->v3: - Picked up {Reviewed,Acked}-by's. - Reorder commits: introduce CONTINUE before MINOR registration. [Hugh, Peter] - Don't try to {unlock,put}_page an xarray value in shmem_getpage_gfp. [Hugh] - Move enum mcopy_atomic_mode forward declare out of CONFIG_HUGETLB_PAGE. [Hugh] - Keep mistakenly removed UFFD_USER_MODE_ONLY in selftest. [Peter] - Cleanup context management in self test (make clear implicit, remove unneeded return values now that we have err()). [Peter] - Correct dst_pte argument to dst_pmd in shmem_mcopy_atomic_pte macro. [Hugh] - Mention the new shmem support feature in documentation. [Hugh] v1->v2: - Pick up Reviewed-by's. - Don't swapin page when a minor fault occurs. Notice that it needs to be swapped in, and just immediately fire the minor fault. Let a future CONTINUE deal with swapping in the page. [Peter] - Clarify comment about i_size checks in mm/userfaultfd.c. [Peter] - Only forward declare once (out of #ifdef) in hugetlb.h. [Peter] Changes since [2]: - Squash the fixes ([2]) in with the original series ([1]). This makes reviewing easier, as we no longer have to sift through deltas undoing what we had done before. [Hugh, Peter] - Modify shmem_mcopy_atomic_pte() to use the new mcopy_atomic_install_ptes() helper, reducing code duplication. [Hugh] - Properly trigger handle_userfault() in the shmem_swapin_page() case. [Hugh] - Use shmem_getpage() instead of find_lock_page() to lookup the existing page in for continue. This properly deals with swapped-out pages. [Hugh] - Unconditionally pte_mkdirty() for anon memory (as before). [Peter] - Don't include userfaultfd_k.h in either hugetlb.h or shmem_fs.h. [Hugh] - Add comment for UFFD_FEATURE_MINOR_SHMEM (to match _HUGETLBFS). [Hugh] - Fix some small cleanup issues (parens, reworded conditionals, reduced plumbing of some parameters, simplify labels/gotos, ...). [Hugh, Peter] Overview ======== See the series which added minor faults for hugetlbfs [3] for a detailed overview of minor fault handling in general. This series adds the same support for shmem-backed areas. This series is structured as follows: - Commits 1 and 2 are cleanups. - Commits 3 and 4 implement the new feature (minor fault handling for shmem). - Commits 5, 6, 7, 8 update the userfaultfd selftest to exercise the feature. - Commit 9 is one final cleanup, modifying an existing code path to re-use a new helper we've introduced. We rely on the selftest to show that this change doesn't break anything. - Commit 10 is a small documentation update to reflect the new changes. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra <lokeshgidra(a)google.com>) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." [1] https://lore.kernel.org/patchwork/cover/1388144/ [2] https://lore.kernel.org/patchwork/patch/1408161/ [3] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen… Axel Rasmussen (10): userfaultfd/hugetlbfs: avoid including userfaultfd_k.h in hugetlb.h userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte userfaultfd/shmem: support UFFDIO_CONTINUE for shmem userfaultfd/shmem: support minor fault registration for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support userfaultfd/shmem: modify shmem_mcopy_atomic_pte to use install_pte() userfaultfd: update documentation to mention shmem minor faults Documentation/admin-guide/mm/userfaultfd.rst | 3 +- fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 4 +- include/linux/shmem_fs.h | 17 +- include/linux/userfaultfd_k.h | 5 + include/uapi/linux/userfaultfd.h | 7 +- mm/hugetlb.c | 1 + mm/memory.c | 8 +- mm/shmem.c | 115 +++----- mm/userfaultfd.c | 175 ++++++++---- tools/testing/selftests/vm/userfaultfd.c | 274 ++++++++++++------- 11 files changed, 364 insertions(+), 251 deletions(-) -- 2.31.1.368.gbe11c130af-goog

4 years, 2 months

3
27
0 0

[PATCH 0/2] selftests: futex: Improve timeout testing

by André Almeida

This small series expands futex timeout selftests by checking if all operations that allows timeouts works as expected. When some version of Thomas' series "futex: Bugfixes and FUTEX_LOCK_PI2"[0] get merged, I'll add the new rules to the timeout test. This test should be used to check for regressions when modifying the timeout path or changing the interface. Additionally, fix a bug in the Makefile that can be found when compiling selftests with new operations, like the one defined at [0] or from the futex2 patchset. [0] https://lore.kernel.org/lkml/20210422194417.866740847@linutronix.de/ André Almeida (2): selftests: futex: Correctly include headers dirs selftests: futex: Expand timeout test .../selftests/futex/functional/Makefile | 3 +- .../futex/functional/futex_wait_timeout.c | 126 +++++++++++++++--- 2 files changed, 112 insertions(+), 17 deletions(-) -- 2.31.1

4 years, 2 months

1
2
0 0

[PATCH] selftests: kvm: Fix the check of return value

by Zhenzhong Duan

In vm_vcpu_rm() and kvm_vm_release(), a stale return value is checked in TEST_ASSERT macro. Fix it by assigning variable ret with correct return value. Signed-off-by: Zhenzhong Duan <zhenzhong.duan(a)intel.com> --- tools/testing/selftests/kvm/lib/kvm_util.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index b8849a1aca79..53d3a7eb0d47 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -514,7 +514,7 @@ static void vm_vcpu_rm(struct kvm_vm *vm, struct vcpu *vcpu) ret = munmap(vcpu->state, vcpu_mmap_sz()); TEST_ASSERT(ret == 0, "munmap of VCPU fd failed, rc: %i " "errno: %i", ret, errno); - close(vcpu->fd); + ret = close(vcpu->fd); TEST_ASSERT(ret == 0, "Close of VCPU fd failed, rc: %i " "errno: %i", ret, errno); @@ -534,7 +534,7 @@ void kvm_vm_release(struct kvm_vm *vmp) TEST_ASSERT(ret == 0, "Close of vm fd failed,\n" " vmp->fd: %i rc: %i errno: %i", vmp->fd, ret, errno); - close(vmp->kvm_fd); + ret = close(vmp->kvm_fd); TEST_ASSERT(ret == 0, "Close of /dev/kvm fd failed,\n" " vmp->kvm_fd: %i rc: %i errno: %i", vmp->kvm_fd, ret, errno); } -- 2.25.1

4 years, 2 months

2
1
0 0

[RFC v4 0/2] CPU-Idle latency selftest framework

by Pratik Rajesh Sampat

Changelog v3-->v4 Based on review comments by Doug Smythies, 1. Parsing the thread_siblings_list for CPU topology information to correctly identify the cores the test should run on in default(quick) mode. 2. The source CPU to source CPU interaction in the IPI test will always result in a lower latency and cause a bias in the average, hence avoid adding the latency to be averaged for same cpu IPIs. The latency will still be displayed in the detailed logs. RFC v3: https://lkml.org/lkml/2021/4/4/31 --- A kernel module + userspace driver to estimate the wakeup latency caused by going into stop states. The motivation behind this program is to find significant deviations behind advertised latency and residency values. The patchset measures latencies for two kinds of events. IPIs and Timers As this is a software-only mechanism, there will additional latencies of the kernel-firmware-hardware interactions. To account for that, the program also measures a baseline latency on a 100 percent loaded CPU and the latencies achieved must be in view relative to that. To achieve this, we introduce a kernel module and expose its control knobs through the debugfs interface that the selftests can engage with. The kernel module provides the following interfaces within /sys/kernel/debug/latency_test/ for, IPI test: ipi_cpu_dest = Destination CPU for the IPI ipi_cpu_src = Origin of the IPI ipi_latency_ns = Measured latency time in ns Timeout test: timeout_cpu_src = CPU on which the timer to be queued timeout_expected_ns = Timer duration timeout_diff_ns = Difference of actual duration vs expected timer Sample output on a POWER9 system is as follows: # --IPI Latency Test--- # Baseline Average IPI latency(ns): 3114 # Observed Average IPI latency(ns) - State0: 3265 # Observed Average IPI latency(ns) - State1: 3507 # Observed Average IPI latency(ns) - State2: 3739 # Observed Average IPI latency(ns) - State3: 3807 # Observed Average IPI latency(ns) - State4: 17070 # Observed Average IPI latency(ns) - State5: 1038174 # Observed Average IPI latency(ns) - State6: 1068784 # # --Timeout Latency Test-- # Baseline Average timeout diff(ns): 1420 # Observed Average timeout diff(ns) - State0: 1640 # Observed Average timeout diff(ns) - State1: 1764 # Observed Average timeout diff(ns) - State2: 1715 # Observed Average timeout diff(ns) - State3: 1845 # Observed Average timeout diff(ns) - State4: 16581 # Observed Average timeout diff(ns) - State5: 939977 # Observed Average timeout diff(ns) - State6: 1073024 Things to keep in mind: 1. This kernel module + bash driver does not guarantee idleness on a core when the IPI and the Timer is armed. It only invokes sleep and hopes that the core is idle once the IPI/Timer is invoked onto it. Hence this program must be run on a completely idle system for best results 2. Even on a completely idle system, there maybe book-keeping tasks or jitter tasks that can run on the core we want idle. This can create outliers in the latency measurement. Thankfully, these outliers should be large enough to easily weed them out. 3. A userspace only selftest variant was also sent out as RFC based on suggestions over the previous patchset to simply the kernel complexeity. However, a userspace only approach had more noise in the latency measurement due to userspace-kernel interactions which led to run to run variance and a lesser accurate test. Another downside of the nature of a userspace program is that it takes orders of magnitude longer to complete a full system test compared to the kernel framework. RFC patch: https://lkml.org/lkml/2020/9/2/356 4. For Intel Systems, the Timer based latencies don't exactly give out the measure of idle latencies. This is because of a hardware optimization mechanism that pre-arms a CPU when a timer is set to wakeup. That doesn't make this metric useless for Intel systems, it just means that is measuring IPI/Timer responding latency rather than idle wakeup latencies. (Source: https://lkml.org/lkml/2020/9/2/610) For solution to this problem, a hardware based latency analyzer is devised by Artem Bityutskiy from Intel. https://youtu.be/Opk92aQyvt0?t=8266 https://intel.github.io/wult/ Pratik Rajesh Sampat (2): cpuidle: Extract IPI based and timer based wakeup latency from idle states selftest/cpuidle: Add support for cpuidle latency measurement drivers/cpuidle/Makefile | 1 + drivers/cpuidle/test-cpuidle_latency.c | 157 ++++++++ lib/Kconfig.debug | 10 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/cpuidle/Makefile | 6 + tools/testing/selftests/cpuidle/cpuidle.sh | 402 +++++++++++++++++++++ tools/testing/selftests/cpuidle/settings | 2 + 7 files changed, 579 insertions(+) create mode 100644 drivers/cpuidle/test-cpuidle_latency.c create mode 100644 tools/testing/selftests/cpuidle/Makefile create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh create mode 100644 tools/testing/selftests/cpuidle/settings -- 2.17.1

4 years, 2 months

2
5
0 0

[PATCH] selftests/net: bump timeout to 5 minutes

by Po-Hsu Lin

We found that with the latest mainline kernel (5.12.0-051200rc8) on some KVM instances / bare-metal systems, the following tests will take longer than the kselftest framework default timeout (45 seconds) to run and thus got terminated with TIMEOUT error: * xfrm_policy.sh - took about 2m20s * pmtu.sh - took about 3m5s * udpgso_bench.sh - took about 60s Bump the timeout setting to 5 minutes to allow them have a chance to finish. https://bugs.launchpad.net/bugs/1856010 Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com> --- tools/testing/selftests/net/Makefile | 2 ++ tools/testing/selftests/net/settings | 1 + 2 files changed, 3 insertions(+) create mode 100644 tools/testing/selftests/net/settings diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index 25f198b..2be4670 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -37,6 +37,8 @@ TEST_GEN_FILES += ipsec TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls +TEST_FILES := settings + KSFT_KHDR_INSTALL := 1 include ../lib.mk diff --git a/tools/testing/selftests/net/settings b/tools/testing/selftests/net/settings new file mode 100644 index 0000000..694d707 --- /dev/null +++ b/tools/testing/selftests/net/settings @@ -0,0 +1 @@ +timeout=300 -- 2.7.4

4 years, 2 months

2
1
0 0

[PATCH] selftests/bpf: fix warning comparing pointer to 0

by Jiapeng Chong

Fix the following coccicheck warning: ./tools/testing/selftests/bpf/progs/fentry_test.c:76:15-16: WARNING comparing pointer to 0. Reported-by: Abaci Robot <abaci(a)linux.alibaba.com> Signed-off-by: Jiapeng Chong <jiapeng.chong(a)linux.alibaba.com> --- tools/testing/selftests/bpf/progs/fentry_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/progs/fentry_test.c b/tools/testing/selftests/bpf/progs/fentry_test.c index 52a550d..d4247d6 100644 --- a/tools/testing/selftests/bpf/progs/fentry_test.c +++ b/tools/testing/selftests/bpf/progs/fentry_test.c @@ -73,7 +73,7 @@ int BPF_PROG(test7, struct bpf_fentry_test_t *arg) SEC("fentry/bpf_fentry_test8") int BPF_PROG(test8, struct bpf_fentry_test_t *arg) { - if (arg->a == 0) + if (!arg->a) test8_result = 1; return 0; } -- 1.8.3.1

4 years, 2 months

4
3
0 0

[PATCH v34 00/13] Landlock LSM

by Mickaël Salaün

Hi, This updated patch series adds a new patch on top of the previous ones. It brings a new flag to landlock_create_ruleset(2) that enables efficient and simple backward compatibility checks for future evolutions of Landlock (e.g. new access-control rights). Indeed, it is important to help user space to follow a best-effort security. This new flag is not strictly useful for applications using the current Landlock features but it will be useful when applications developed for newer kernels will be run on older kernels (e.g. the current one). Here is an example of a (work in progress) library using this information to provide a nice backward compatible API: https://github.com/landlock-lsm/rust-landlock The SLOC count is 1331 for security/landlock/ and 2626 for tools/testing/selftest/landlock/ . Test coverage for security/landlock/ is 93.6% of lines: https://landlock.io/linux-lcov/landlock-v34/security/landlock/index.html The code not covered only deals with internal kernel errors (e.g. memory allocation), race conditions and safety checks that should not be triggered. This series is being fuzzed by syzkaller (covering internal kernel errors) that now supports Landlock: https://github.com/google/syzkaller/pull/2380 syzkaller coverage reached 72% (ci-upstream-linux-next-kasan-gce-root): https://syzkaller.appspot.com/upstream The HTML documentation is available here: https://landlock.io/linux-doc/landlock-v34/userspace-api/landlock.html This series can be applied on top of v5.12-rc3 . This can be tested with CONFIG_SECURITY_LANDLOCK, CONFIG_SAMPLE_LANDLOCK and by prepending "landlock," to CONFIG_LSM. This patch series can be found in a Git repository here: https://github.com/landlock-lsm/linux/commits/landlock-v34 This patch series seems ready for upstream and I would really appreciate final reviews. Landlock LSM ============ The goal of Landlock is to enable to restrict ambient rights (e.g. global filesystem access) for a set of processes. Because Landlock is a stackable LSM [1], it makes possible to create safe security sandboxes as new security layers in addition to the existing system-wide access-controls. This kind of sandbox is expected to help mitigate the security impact of bugs or unexpected/malicious behaviors in user-space applications. Landlock empowers any process, including unprivileged ones, to securely restrict themselves. Landlock is inspired by seccomp-bpf but instead of filtering syscalls and their raw arguments, a Landlock rule can restrict the use of kernel objects like file hierarchies, according to the kernel semantic. Landlock also takes inspiration from other OS sandbox mechanisms: XNU Sandbox, FreeBSD Capsicum or OpenBSD Pledge/Unveil. In this current form, Landlock misses some access-control features. This enables to minimize this patch series and ease review. This series still addresses multiple use cases, especially with the combined use of seccomp-bpf: applications with built-in sandboxing, init systems, security sandbox tools and security-oriented APIs [2]. [1] https://lore.kernel.org/lkml/50db058a-7dde-441b-a7f9-f6837fe8b69f@schaufler… [2] https://lore.kernel.org/lkml/f646e1c7-33cf-333f-070c-0a40ad0468cd@digikod.n… Previous versions: v33: https://lore.kernel.org/lkml/20210407160726.542794-1-mic@digikod.net/ v32: https://lore.kernel.org/lkml/20210401205208.2756565-1-mic@digikod.net/ v31: https://lore.kernel.org/lkml/20210324191520.125779-1-mic@digikod.net/ v30: https://lore.kernel.org/lkml/20210316204252.427806-1-mic@digikod.net/ v29: https://lore.kernel.org/lkml/20210225190614.2181147-1-mic@digikod.net/ v28: https://lore.kernel.org/lkml/20210202162710.657398-1-mic@digikod.net/ v27: https://lore.kernel.org/lkml/20210121205119.793296-1-mic@digikod.net/ v26: https://lore.kernel.org/lkml/20201209192839.1396820-1-mic@digikod.net/ v25: https://lore.kernel.org/lkml/20201201192322.213239-1-mic@digikod.net/ v24: https://lore.kernel.org/lkml/20201112205141.775752-1-mic@digikod.net/ v23: https://lore.kernel.org/lkml/20201103182109.1014179-1-mic@digikod.net/ v22: https://lore.kernel.org/lkml/20201027200358.557003-1-mic@digikod.net/ v21: https://lore.kernel.org/lkml/20201008153103.1155388-1-mic@digikod.net/ v20: https://lore.kernel.org/lkml/20200802215903.91936-1-mic@digikod.net/ v19: https://lore.kernel.org/lkml/20200707180955.53024-1-mic@digikod.net/ v18: https://lore.kernel.org/lkml/20200526205322.23465-1-mic@digikod.net/ v17: https://lore.kernel.org/lkml/20200511192156.1618284-1-mic@digikod.net/ v16: https://lore.kernel.org/lkml/20200416103955.145757-1-mic@digikod.net/ v15: https://lore.kernel.org/lkml/20200326202731.693608-1-mic@digikod.net/ v14: https://lore.kernel.org/lkml/20200224160215.4136-1-mic@digikod.net/ v13: https://lore.kernel.org/lkml/20191104172146.30797-1-mic@digikod.net/ v12: https://lore.kernel.org/lkml/20191031164445.29426-1-mic@digikod.net/ v11: https://lore.kernel.org/lkml/20191029171505.6650-1-mic@digikod.net/ v10: https://lore.kernel.org/lkml/20190721213116.23476-1-mic@digikod.net/ v9: https://lore.kernel.org/lkml/20190625215239.11136-1-mic@digikod.net/ v8: https://lore.kernel.org/lkml/20180227004121.3633-1-mic@digikod.net/ v7: https://lore.kernel.org/lkml/20170821000933.13024-1-mic@digikod.net/ v6: https://lore.kernel.org/lkml/20170328234650.19695-1-mic@digikod.net/ v5: https://lore.kernel.org/lkml/20170222012632.4196-1-mic@digikod.net/ v4: https://lore.kernel.org/lkml/20161026065654.19166-1-mic@digikod.net/ v3: https://lore.kernel.org/lkml/20160914072415.26021-1-mic@digikod.net/ v2: https://lore.kernel.org/lkml/1472121165-29071-1-git-send-email-mic@digikod.… v1: https://lore.kernel.org/kernel-hardening/1458784008-16277-1-git-send-email-… Casey Schaufler (1): LSM: Infrastructure management of the superblock Mickaël Salaün (12): landlock: Add object management landlock: Add ruleset and domain management landlock: Set up the security framework and manage credentials landlock: Add ptrace restrictions fs,security: Add sb_delete hook landlock: Support filesystem access-control landlock: Add syscall implementations arch: Wire up Landlock syscalls selftests/landlock: Add user space tests samples/landlock: Add a sandbox manager example landlock: Add user and kernel documentation landlock: Enable user space to infer supported features Documentation/security/index.rst | 1 + Documentation/security/landlock.rst | 85 + Documentation/userspace-api/index.rst | 1 + Documentation/userspace-api/landlock.rst | 311 ++ MAINTAINERS | 15 + arch/Kconfig | 7 + arch/alpha/kernel/syscalls/syscall.tbl | 3 + arch/arm/tools/syscall.tbl | 3 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 6 + arch/ia64/kernel/syscalls/syscall.tbl | 3 + arch/m68k/kernel/syscalls/syscall.tbl | 3 + arch/microblaze/kernel/syscalls/syscall.tbl | 3 + arch/mips/kernel/syscalls/syscall_n32.tbl | 3 + arch/mips/kernel/syscalls/syscall_n64.tbl | 3 + arch/mips/kernel/syscalls/syscall_o32.tbl | 3 + arch/parisc/kernel/syscalls/syscall.tbl | 3 + arch/powerpc/kernel/syscalls/syscall.tbl | 3 + arch/s390/kernel/syscalls/syscall.tbl | 3 + arch/sh/kernel/syscalls/syscall.tbl | 3 + arch/sparc/kernel/syscalls/syscall.tbl | 3 + arch/um/Kconfig | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 3 + arch/x86/entry/syscalls/syscall_64.tbl | 3 + arch/xtensa/kernel/syscalls/syscall.tbl | 3 + fs/super.c | 1 + include/linux/lsm_hook_defs.h | 1 + include/linux/lsm_hooks.h | 4 + include/linux/security.h | 4 + include/linux/syscalls.h | 7 + include/uapi/asm-generic/unistd.h | 8 +- include/uapi/linux/landlock.h | 137 + kernel/sys_ni.c | 5 + samples/Kconfig | 7 + samples/Makefile | 1 + samples/landlock/.gitignore | 1 + samples/landlock/Makefile | 13 + samples/landlock/sandboxer.c | 238 ++ security/Kconfig | 11 +- security/Makefile | 2 + security/landlock/Kconfig | 21 + security/landlock/Makefile | 4 + security/landlock/common.h | 20 + security/landlock/cred.c | 46 + security/landlock/cred.h | 58 + security/landlock/fs.c | 692 ++++ security/landlock/fs.h | 70 + security/landlock/limits.h | 21 + security/landlock/object.c | 67 + security/landlock/object.h | 91 + security/landlock/ptrace.c | 120 + security/landlock/ptrace.h | 14 + security/landlock/ruleset.c | 473 +++ security/landlock/ruleset.h | 165 + security/landlock/setup.c | 40 + security/landlock/setup.h | 18 + security/landlock/syscalls.c | 451 +++ security/security.c | 51 +- security/selinux/hooks.c | 58 +- security/selinux/include/objsec.h | 6 + security/selinux/ss/services.c | 3 +- security/smack/smack.h | 6 + security/smack/smack_lsm.c | 35 +- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/landlock/.gitignore | 2 + tools/testing/selftests/landlock/Makefile | 24 + tools/testing/selftests/landlock/base_test.c | 266 ++ tools/testing/selftests/landlock/common.h | 183 ++ tools/testing/selftests/landlock/config | 7 + tools/testing/selftests/landlock/fs_test.c | 2791 +++++++++++++++++ .../testing/selftests/landlock/ptrace_test.c | 337 ++ tools/testing/selftests/landlock/true.c | 5 + 72 files changed, 6986 insertions(+), 77 deletions(-) create mode 100644 Documentation/security/landlock.rst create mode 100644 Documentation/userspace-api/landlock.rst create mode 100644 include/uapi/linux/landlock.h create mode 100644 samples/landlock/.gitignore create mode 100644 samples/landlock/Makefile create mode 100644 samples/landlock/sandboxer.c create mode 100644 security/landlock/Kconfig create mode 100644 security/landlock/Makefile create mode 100644 security/landlock/common.h create mode 100644 security/landlock/cred.c create mode 100644 security/landlock/cred.h create mode 100644 security/landlock/fs.c create mode 100644 security/landlock/fs.h create mode 100644 security/landlock/limits.h create mode 100644 security/landlock/object.c create mode 100644 security/landlock/object.h create mode 100644 security/landlock/ptrace.c create mode 100644 security/landlock/ptrace.h create mode 100644 security/landlock/ruleset.c create mode 100644 security/landlock/ruleset.h create mode 100644 security/landlock/setup.c create mode 100644 security/landlock/setup.h create mode 100644 security/landlock/syscalls.c create mode 100644 tools/testing/selftests/landlock/.gitignore create mode 100644 tools/testing/selftests/landlock/Makefile create mode 100644 tools/testing/selftests/landlock/base_test.c create mode 100644 tools/testing/selftests/landlock/common.h create mode 100644 tools/testing/selftests/landlock/config create mode 100644 tools/testing/selftests/landlock/fs_test.c create mode 100644 tools/testing/selftests/landlock/ptrace_test.c create mode 100644 tools/testing/selftests/landlock/true.c base-commit: 1e28eed17697bcf343c6743f0028cc3b5dd88bf0 -- 2.31.1

4 years, 2 months

2
15
0 0

[PATCH 1/2] kunit: introduce kunit_kmalloc_array/kunit_kcalloc() helpers

by Daniel Latypov

Add in: * kunit_kmalloc_array() and wire up kunit_kmalloc() to be a special case of it. * kunit_kcalloc() for symmetry with kunit_kzalloc() This should using KUnit more natural by making it more similar to the existing *alloc() APIs. And while we shouldn't necessarily be writing unit tests where overflow should be a concern, it can't hurt to be safe. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- include/kunit/test.h | 36 ++++++++++++++++++++++++++++++++---- lib/kunit/test.c | 22 ++++++++++++---------- 2 files changed, 44 insertions(+), 14 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 49601c4b98b8..7fa0de4af977 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -577,16 +577,30 @@ static inline int kunit_destroy_named_resource(struct kunit *test, void kunit_remove_resource(struct kunit *test, struct kunit_resource *res); /** - * kunit_kmalloc() - Like kmalloc() except the allocation is *test managed*. + * kunit_kmalloc_array() - Like kmalloc_array() except the allocation is *test managed*. * @test: The test context object. + * @n: number of elements. * @size: The size in bytes of the desired memory. * @gfp: flags passed to underlying kmalloc(). * - * Just like `kmalloc(...)`, except the allocation is managed by the test case + * Just like `kmalloc_array(...)`, except the allocation is managed by the test case * and is automatically cleaned up after the test case concludes. See &struct * kunit_resource for more information. */ -void *kunit_kmalloc(struct kunit *test, size_t size, gfp_t gfp); +void *kunit_kmalloc_array(struct kunit *test, size_t n, size_t size, gfp_t flags); + +/** + * kunit_kmalloc() - Like kmalloc() except the allocation is *test managed*. + * @test: The test context object. + * @size: The size in bytes of the desired memory. + * @gfp: flags passed to underlying kmalloc(). + * + * See kmalloc() and kunit_kmalloc_array() for more information. + */ +static inline void *kunit_kmalloc(struct kunit *test, size_t size, gfp_t gfp) +{ + return kunit_kmalloc_array(test, 1, size, gfp); +} /** * kunit_kfree() - Like kfree except for allocations managed by KUnit. @@ -601,13 +615,27 @@ void kunit_kfree(struct kunit *test, const void *ptr); * @size: The size in bytes of the desired memory. * @gfp: flags passed to underlying kmalloc(). * - * See kzalloc() and kunit_kmalloc() for more information. + * See kzalloc() and kunit_kmalloc_array() for more information. */ static inline void *kunit_kzalloc(struct kunit *test, size_t size, gfp_t gfp) { return kunit_kmalloc(test, size, gfp | __GFP_ZERO); } +/** + * kunit_kzalloc() - Just like kunit_kmalloc_array(), but zeroes the allocation. + * @test: The test context object. + * @n: number of elements. + * @size: The size in bytes of the desired memory. + * @gfp: flags passed to underlying kmalloc(). + * + * See kcalloc() and kunit_kmalloc_array() for more information. + */ +static inline void *kunit_kcalloc(struct kunit *test, size_t n, size_t size, gfp_t flags) +{ + return kunit_kmalloc_array(test, n, size, flags | __GFP_ZERO); +} + void kunit_cleanup(struct kunit *test); void kunit_log_append(char *log, const char *fmt, ...); diff --git a/lib/kunit/test.c b/lib/kunit/test.c index ec9494e914ef..052fccf69eef 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -540,41 +540,43 @@ int kunit_destroy_resource(struct kunit *test, kunit_resource_match_t match, } EXPORT_SYMBOL_GPL(kunit_destroy_resource); -struct kunit_kmalloc_params { +struct kunit_kmalloc_array_params { + size_t n; size_t size; gfp_t gfp; }; -static int kunit_kmalloc_init(struct kunit_resource *res, void *context) +static int kunit_kmalloc_array_init(struct kunit_resource *res, void *context) { - struct kunit_kmalloc_params *params = context; + struct kunit_kmalloc_array_params *params = context; - res->data = kmalloc(params->size, params->gfp); + res->data = kmalloc_array(params->n, params->size, params->gfp); if (!res->data) return -ENOMEM; return 0; } -static void kunit_kmalloc_free(struct kunit_resource *res) +static void kunit_kmalloc_array_free(struct kunit_resource *res) { kfree(res->data); } -void *kunit_kmalloc(struct kunit *test, size_t size, gfp_t gfp) +void *kunit_kmalloc_array(struct kunit *test, size_t n, size_t size, gfp_t gfp) { - struct kunit_kmalloc_params params = { + struct kunit_kmalloc_array_params params = { .size = size, + .n = n, .gfp = gfp }; return kunit_alloc_resource(test, - kunit_kmalloc_init, - kunit_kmalloc_free, + kunit_kmalloc_array_init, + kunit_kmalloc_array_free, gfp, &params); } -EXPORT_SYMBOL_GPL(kunit_kmalloc); +EXPORT_SYMBOL_GPL(kunit_kmalloc_array); void kunit_kfree(struct kunit *test, const void *ptr) { base-commit: 16fc44d6387e260f4932e9248b985837324705d8 -- 2.31.1.498.g6c1eba8ee3d-goog

4 years, 2 months

3
3
0 0

[PATCH] selftests/bpf: fix warning comparing pointer to 0

by Jiapeng Chong

Fix the following coccicheck warning: ./tools/testing/selftests/bpf/progs/fentry_test.c:76:15-16: WARNING comparing pointer to 0. Reported-by: Abaci Robot <abaci(a)linux.alibaba.com> Signed-off-by: Jiapeng Chong <jiapeng.chong(a)linux.alibaba.com> --- tools/testing/selftests/bpf/progs/fentry_test.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/fentry_test.c b/tools/testing/selftests/bpf/progs/fentry_test.c index 5f645fd..d4247d6 100644 --- a/tools/testing/selftests/bpf/progs/fentry_test.c +++ b/tools/testing/selftests/bpf/progs/fentry_test.c @@ -64,7 +64,7 @@ struct bpf_fentry_test_t { SEC("fentry/bpf_fentry_test7") int BPF_PROG(test7, struct bpf_fentry_test_t *arg) { - if (arg == 0) + if (!arg) test7_result = 1; return 0; } @@ -73,7 +73,7 @@ int BPF_PROG(test7, struct bpf_fentry_test_t *arg) SEC("fentry/bpf_fentry_test8") int BPF_PROG(test8, struct bpf_fentry_test_t *arg) { - if (arg->a == 0) + if (!arg->a) test8_result = 1; return 0; } -- 1.8.3.1

4 years, 2 months

2
2
0 0

[PATCH v8 6/6] selftest/x86/signal: Include test cases for validating sigaltstack

by Chang S. Bae

The test measures the kernel's signal delivery with different (enough vs. insufficient) stack sizes. Signed-off-by: Chang S. Bae <chang.seok.bae(a)intel.com> Reviewed-by: Len Brown <len.brown(a)intel.com> Cc: x86(a)kernel.org Cc: linux-kselftest(a)vger.kernel.org Cc: linux-kernel(a)vger.kernel.org --- Changes from v3: * Revised test messages again (Borislav Petkov) Changes from v2: * Revised test messages (Borislav Petkov) --- tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/sigaltstack.c | 128 ++++++++++++++++++++++ 2 files changed, 129 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/x86/sigaltstack.c diff --git a/tools/testing/selftests/x86/Makefile b/tools/testing/selftests/x86/Makefile index 333980375bc7..65bba2ae86ee 100644 --- a/tools/testing/selftests/x86/Makefile +++ b/tools/testing/selftests/x86/Makefile @@ -13,7 +13,7 @@ CAN_BUILD_WITH_NOPIE := $(shell ./check_cc.sh $(CC) trivial_program.c -no-pie) TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt test_mremap_vdso \ check_initial_reg_state sigreturn iopl ioperm \ test_vsyscall mov_ss_trap \ - syscall_arg_fault fsgsbase_restore + syscall_arg_fault fsgsbase_restore sigaltstack TARGETS_C_32BIT_ONLY := entry_from_vm86 test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ vdso_restorer diff --git a/tools/testing/selftests/x86/sigaltstack.c b/tools/testing/selftests/x86/sigaltstack.c new file mode 100644 index 000000000000..f689af75e979 --- /dev/null +++ b/tools/testing/selftests/x86/sigaltstack.c @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#define _GNU_SOURCE +#include <signal.h> +#include <stdio.h> +#include <stdbool.h> +#include <string.h> +#include <err.h> +#include <errno.h> +#include <limits.h> +#include <sys/mman.h> +#include <sys/auxv.h> +#include <sys/prctl.h> +#include <sys/resource.h> +#include <setjmp.h> + +/* sigaltstack()-enforced minimum stack */ +#define ENFORCED_MINSIGSTKSZ 2048 + +#ifndef AT_MINSIGSTKSZ +# define AT_MINSIGSTKSZ 51 +#endif + +static int nerrs; + +static bool sigalrm_expected; + +static unsigned long at_minstack_size; + +static void sethandler(int sig, void (*handler)(int, siginfo_t *, void *), + int flags) +{ + struct sigaction sa; + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = handler; + sa.sa_flags = SA_SIGINFO | flags; + sigemptyset(&sa.sa_mask); + if (sigaction(sig, &sa, 0)) + err(1, "sigaction"); +} + +static void clearhandler(int sig) +{ + struct sigaction sa; + + memset(&sa, 0, sizeof(sa)); + sa.sa_handler = SIG_DFL; + sigemptyset(&sa.sa_mask); + if (sigaction(sig, &sa, 0)) + err(1, "sigaction"); +} + +static int setup_altstack(void *start, unsigned long size) +{ + stack_t ss; + + memset(&ss, 0, sizeof(ss)); + ss.ss_size = size; + ss.ss_sp = start; + + return sigaltstack(&ss, NULL); +} + +static jmp_buf jmpbuf; + +static void sigsegv(int sig, siginfo_t *info, void *ctx_void) +{ + if (sigalrm_expected) { + printf("[FAIL]\tWrong signal delivered: SIGSEGV (expected SIGALRM)."); + nerrs++; + } else { + printf("[OK]\tSIGSEGV signal delivered.\n"); + } + + siglongjmp(jmpbuf, 1); +} + +static void sigalrm(int sig, siginfo_t *info, void *ctx_void) +{ + if (!sigalrm_expected) { + printf("[FAIL]\tWrong signal delivered: SIGALRM (expected SIGSEGV)."); + nerrs++; + } else { + printf("[OK]\tSIGALRM signal delivered.\n"); + } +} + +static void test_sigaltstack(void *altstack, unsigned long size) +{ + if (setup_altstack(altstack, size)) + err(1, "sigaltstack()"); + + sigalrm_expected = (size > at_minstack_size) ? true : false; + + sethandler(SIGSEGV, sigsegv, 0); + sethandler(SIGALRM, sigalrm, SA_ONSTACK); + + if (!sigsetjmp(jmpbuf, 1)) { + printf("[RUN]\tTest an alternate signal stack of %ssufficient size.\n", + sigalrm_expected ? "" : "in"); + printf("\tRaise SIGALRM. %s is expected to be delivered.\n", + sigalrm_expected ? "It" : "SIGSEGV"); + raise(SIGALRM); + } + + clearhandler(SIGALRM); + clearhandler(SIGSEGV); +} + +int main(void) +{ + void *altstack; + + at_minstack_size = getauxval(AT_MINSIGSTKSZ); + + altstack = mmap(NULL, at_minstack_size + SIGSTKSZ, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0); + if (altstack == MAP_FAILED) + err(1, "mmap()"); + + if ((ENFORCED_MINSIGSTKSZ + 1) < at_minstack_size) + test_sigaltstack(altstack, ENFORCED_MINSIGSTKSZ + 1); + + test_sigaltstack(altstack, at_minstack_size + SIGSTKSZ); + + return nerrs == 0 ? 0 : 1; +} -- 2.17.1

4 years, 2 months

1
0
0 0

[PATCH v8 4/6] selftest/sigaltstack: Use the AT_MINSIGSTKSZ aux vector if available

by Chang S. Bae

The SIGSTKSZ constant may not represent enough stack size in some architectures as the hardware state size grows. Use getauxval(AT_MINSIGSTKSZ) to increase the stack size. Signed-off-by: Chang S. Bae <chang.seok.bae(a)intel.com> Reviewed-by: Len Brown <len.brown(a)intel.com> Cc: linux-kselftest(a)vger.kernel.org Cc: linux-kernel(a)vger.kernel.org --- Changes from v5: * Added as a new patch. --- tools/testing/selftests/sigaltstack/sas.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/sigaltstack/sas.c b/tools/testing/selftests/sigaltstack/sas.c index 8934a3766d20..c53b070755b6 100644 --- a/tools/testing/selftests/sigaltstack/sas.c +++ b/tools/testing/selftests/sigaltstack/sas.c @@ -17,6 +17,7 @@ #include <string.h> #include <assert.h> #include <errno.h> +#include <sys/auxv.h> #include "../kselftest.h" @@ -24,6 +25,11 @@ #define SS_AUTODISARM (1U << 31) #endif +#ifndef AT_MINSIGSTKSZ +#define AT_MINSIGSTKSZ 51 +#endif + +static unsigned int stack_size; static void *sstack, *ustack; static ucontext_t uc, sc; static const char *msg = "[OK]\tStack preserved"; @@ -47,7 +53,7 @@ void my_usr1(int sig, siginfo_t *si, void *u) #endif if (sp < (unsigned long)sstack || - sp >= (unsigned long)sstack + SIGSTKSZ) { + sp >= (unsigned long)sstack + stack_size) { ksft_exit_fail_msg("SP is not on sigaltstack\n"); } /* put some data on stack. other sighandler will try to overwrite it */ @@ -108,6 +114,10 @@ int main(void) stack_t stk; int err; + /* Make sure more than the required minimum. */ + stack_size = getauxval(AT_MINSIGSTKSZ) + SIGSTKSZ; + ksft_print_msg("[NOTE]\tthe stack size is %lu\n", stack_size); + ksft_print_header(); ksft_set_plan(3); @@ -117,7 +127,7 @@ int main(void) sigaction(SIGUSR1, &act, NULL); act.sa_sigaction = my_usr2; sigaction(SIGUSR2, &act, NULL); - sstack = mmap(NULL, SIGSTKSZ, PROT_READ | PROT_WRITE, + sstack = mmap(NULL, stack_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0); if (sstack == MAP_FAILED) { ksft_exit_fail_msg("mmap() - %s\n", strerror(errno)); @@ -139,7 +149,7 @@ int main(void) } stk.ss_sp = sstack; - stk.ss_size = SIGSTKSZ; + stk.ss_size = stack_size; stk.ss_flags = SS_ONSTACK | SS_AUTODISARM; err = sigaltstack(&stk, NULL); if (err) { @@ -161,7 +171,7 @@ int main(void) } } - ustack = mmap(NULL, SIGSTKSZ, PROT_READ | PROT_WRITE, + ustack = mmap(NULL, stack_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0); if (ustack == MAP_FAILED) { ksft_exit_fail_msg("mmap() - %s\n", strerror(errno)); @@ -170,7 +180,7 @@ int main(void) getcontext(&uc); uc.uc_link = NULL; uc.uc_stack.ss_sp = ustack; - uc.uc_stack.ss_size = SIGSTKSZ; + uc.uc_stack.ss_size = stack_size; makecontext(&uc, switch_fn, 0); raise(SIGUSR1); -- 2.17.1

4 years, 2 months

1
0
0 0

[PATCH] selftests: timens: Fix gettime_perf to work on powerpc

by Christophe Leroy

On powerpc: - VDSO library is named linux-vdso32.so.1 or linux-vdso64.so.1 - clock_gettime is named __kernel_clock_gettime() Ensure gettime_perf tries these names before giving up. Signed-off-by: Christophe Leroy <christophe.leroy(a)csgroup.eu> --- tools/testing/selftests/timens/gettime_perf.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/testing/selftests/timens/gettime_perf.c b/tools/testing/selftests/timens/gettime_perf.c index 7bf841a3967b..6b13dc277724 100644 --- a/tools/testing/selftests/timens/gettime_perf.c +++ b/tools/testing/selftests/timens/gettime_perf.c @@ -25,12 +25,20 @@ static void fill_function_pointers(void) if (!vdso) vdso = dlopen("linux-gate.so.1", RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); + if (!vdso) + vdso = dlopen("linux-vdso32.so.1", + RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); + if (!vdso) + vdso = dlopen("linux-vdso64.so.1", + RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); if (!vdso) { pr_err("[WARN]\tfailed to find vDSO\n"); return; } vdso_clock_gettime = (vgettime_t)dlsym(vdso, "__vdso_clock_gettime"); + if (!vdso_clock_gettime) + vdso_clock_gettime = (vgettime_t)dlsym(vdso, "__kernel_clock_gettime"); if (!vdso_clock_gettime) pr_err("Warning: failed to find clock_gettime in vDSO\n"); -- 2.25.0

4 years, 2 months

2
1
0 0

[PATCH v3] Documentation: dev-tools: Add Testing Overview

by David Gow

The kernel now has a number of testing and debugging tools, and we've seen a bit of confusion about what the differences between them are. Add a basic documentation outlining the testing tools, when to use each, and how they interact. This is a pretty quick overview rather than the idealised "kernel testing guide" that'd probably be optimal, but given the number of times questions like "When do you use KUnit and when do you use Kselftest?" are being asked, it seemed worth at least having something. Hopefully this can form the basis for more detailed documentation later. Signed-off-by: David Gow <davidgow(a)google.com> Reviewed-by: Marco Elver <elver(a)google.com> Reviewed-by: Daniel Latypov <dlatypov(a)google.com> --- Thanks again. Assuming no-one has any objections, I think this is good to go. -- David Changes since v2: https://lore.kernel.org/linux-kselftest/20210414081428.337494-1-davidgow@go… - A few typo fixes (Thanks Daniel) - Reworded description of dynamic analysis tools. - Updated dev-tools index page to not use ':doc:' syntax, but to provide a path instead. - Added Marco and Daniel's Reviewed-by tags. Changes since v1: https://lore.kernel.org/linux-kselftest/20210410070529.4113432-1-davidgow@g… - Note KUnit's speed and that one should provide selftests for syscalls - Mention lockdep as a Dynamic Analysis Tool - Refer to "Dynamic Analysis Tools" instead of "Sanitizers" - A number of minor formatting tweaks and rewordings for clarity Documentation/dev-tools/index.rst | 4 + Documentation/dev-tools/testing-overview.rst | 117 +++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100644 Documentation/dev-tools/testing-overview.rst diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 1b1cf4f5c9d9..929d916ffd4c 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -7,6 +7,9 @@ be used to work on the kernel. For now, the documents have been pulled together without any significant effort to integrate them into a coherent whole; patches welcome! +A brief overview of testing-specific tools can be found in +Documentation/dev-tools/testing-overview.rst + .. class:: toc-title Table of contents @@ -14,6 +17,7 @@ whole; patches welcome! .. toctree:: :maxdepth: 2 + testing-overview coccinelle sparse kcov diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst new file mode 100644 index 000000000000..b5b46709969c --- /dev/null +++ b/Documentation/dev-tools/testing-overview.rst @@ -0,0 +1,117 @@ +.. SPDX-License-Identifier: GPL-2.0 + +==================== +Kernel Testing Guide +==================== + + +There are a number of different tools for testing the Linux kernel, so knowing +when to use each of them can be a challenge. This document provides a rough +overview of their differences, and how they fit together. + + +Writing and Running Tests +========================= + +The bulk of kernel tests are written using either the kselftest or KUnit +frameworks. These both provide infrastructure to help make running tests and +groups of tests easier, as well as providing helpers to aid in writing new +tests. + +If you're looking to verify the behaviour of the Kernel — particularly specific +parts of the kernel — then you'll want to use KUnit or kselftest. + + +The Difference Between KUnit and kselftest +------------------------------------------ + +KUnit (Documentation/dev-tools/kunit/index.rst) is an entirely in-kernel system +for "white box" testing: because test code is part of the kernel, it can access +internal structures and functions which aren't exposed to userspace. + +KUnit tests therefore are best written against small, self-contained parts +of the kernel, which can be tested in isolation. This aligns well with the +concept of 'unit' testing. + +For example, a KUnit test might test an individual kernel function (or even a +single codepath through a function, such as an error handling case), rather +than a feature as a whole. + +This also makes KUnit tests very fast to build and run, allowing them to be +run frequently as part of the development process. + +There is a KUnit test style guide which may give further pointers in +Documentation/dev-tools/kunit/style.rst + + +kselftest (Documentation/dev-tools/kselftest.rst), on the other hand, is +largely implemented in userspace, and tests are normal userspace scripts or +programs. + +This makes it easier to write more complicated tests, or tests which need to +manipulate the overall system state more (e.g., spawning processes, etc.). +However, it's not possible to call kernel functions directly from kselftest. +This means that only kernel functionality which is exposed to userspace somehow +(e.g. by a syscall, device, filesystem, etc.) can be tested with kselftest. To +work around this, some tests include a companion kernel module which exposes +more information or functionality. If a test runs mostly or entirely within the +kernel, however, KUnit may be the more appropriate tool. + +kselftest is therefore suited well to tests of whole features, as these will +expose an interface to userspace, which can be tested, but not implementation +details. This aligns well with 'system' or 'end-to-end' testing. + +For example, all new system calls should be accompanied by kselftest tests. + +Code Coverage Tools +=================== + +The Linux Kernel supports two different code coverage measurement tools. These +can be used to verify that a test is executing particular functions or lines +of code. This is useful for determining how much of the kernel is being tested, +and for finding corner-cases which are not covered by the appropriate test. + +:doc:`gcov` is GCC's coverage testing tool, which can be used with the kernel +to get global or per-module coverage. Unlike KCOV, it does not record per-task +coverage. Coverage data can be read from debugfs, and interpreted using the +usual gcov tooling. + +:doc:`kcov` is a feature which can be built in to the kernel to allow +capturing coverage on a per-task level. It's therefore useful for fuzzing and +other situations where information about code executed during, for example, a +single syscall is useful. + + +Dynamic Analysis Tools +====================== + +The kernel also supports a number of dynamic analysis tools, which attempt to +detect classes of issues when they occur in a running kernel. These typically +each look for a different class of bugs, such as invalid memory accesses, +concurrency issues such as data races, or other undefined behaviour like +integer overflows. + +Some of these tools are listed below: + +* kmemleak detects possible memory leaks. See + Documentation/dev-tools/kmemleak.rst +* KASAN detects invalid memory accesses such as out-of-bounds and + use-after-free errors. See Documentation/dev-tools/kasan.rst +* UBSAN detects behaviour that is undefined by the C standard, like integer + overflows. See Documentation/dev-tools/ubsan.rst +* KCSAN detects data races. See Documentation/dev-tools/kcsan.rst +* KFENCE is a low-overhead detector of memory issues, which is much faster than + KASAN and can be used in production. See Documentation/dev-tools/kfence.rst +* lockdep is a locking correctness validator. See + Documentation/locking/lockdep-design.rst +* There are several other pieces of debug instrumentation in the kernel, many + of which can be found in lib/Kconfig.debug + +These tools tend to test the kernel as a whole, and do not "pass" like +kselftest or KUnit tests. They can be combined with KUnit or kselftest by +running tests on a kernel with these tools enabled: you can then be sure +that none of these errors are occurring during the test. + +Some of these tools integrate with KUnit or kselftest and will +automatically fail tests if an issue is detected. + -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

3
3
0 0

[PATCH][next] selftests: add missing void in MREMAP_DONTUNMAP selftest function prototypes

by Colin King

From: Colin Ian King <colin.king(a)canonical.com> There are a few function prototypes that are missing a void parameter, fix this by adding it in. Signed-off-by: Colin Ian King <colin.king(a)canonical.com> --- tools/testing/selftests/vm/mremap_dontunmap.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/vm/mremap_dontunmap.c b/tools/testing/selftests/vm/mremap_dontunmap.c index f01dc4a85b0b..78baaf0e85d9 100644 --- a/tools/testing/selftests/vm/mremap_dontunmap.c +++ b/tools/testing/selftests/vm/mremap_dontunmap.c @@ -42,7 +42,7 @@ static void dump_maps(void) // Try a simple operation for to "test" for kernel support this prevents // reporting tests as failed when it's run on an older kernel. -static int kernel_support_for_mremap_dontunmap() +static int kernel_support_for_mremap_dontunmap(void) { int ret = 0; unsigned long num_pages = 1; @@ -95,7 +95,7 @@ static int check_region_contains_byte(void *addr, unsigned long size, char byte) // this test validates that MREMAP_DONTUNMAP moves the pagetables while leaving // the source mapping mapped. -static void mremap_dontunmap_simple() +static void mremap_dontunmap_simple(void) { unsigned long num_pages = 5; @@ -128,7 +128,7 @@ static void mremap_dontunmap_simple() } // This test validates that MREMAP_DONTUNMAP on a shared mapping works as expected. -static void mremap_dontunmap_simple_shmem() +static void mremap_dontunmap_simple_shmem(void) { unsigned long num_pages = 5; @@ -181,7 +181,7 @@ static void mremap_dontunmap_simple_shmem() // This test validates MREMAP_DONTUNMAP will move page tables to a specific // destination using MREMAP_FIXED, also while validating that the source // remains intact. -static void mremap_dontunmap_simple_fixed() +static void mremap_dontunmap_simple_fixed(void) { unsigned long num_pages = 5; @@ -226,7 +226,7 @@ static void mremap_dontunmap_simple_fixed() // This test validates that we can MREMAP_DONTUNMAP for a portion of an // existing mapping. -static void mremap_dontunmap_partial_mapping() +static void mremap_dontunmap_partial_mapping(void) { /* * source mapping: -- 2.30.2

4 years, 2 months

1
0
0 0

[PATCH v3 00/10] userfaultfd: add minor fault handling for shmem

by Axel Rasmussen

Base ==== This series is based on (and therefore should apply cleanly to) the tag "v5.12-rc7-mmots-2021-04-11-20-49", additionally with Peter's selftest cleanup series applied first: https://lore.kernel.org/patchwork/cover/1412450/ Changelog ========= v2->v3: - Picked up {Reviewed,Acked}-by's. - Reorder commits: introduce CONTINUE before MINOR registration. [Hugh, Peter] - Don't try to {unlock,put}_page an xarray value in shmem_getpage_gfp. [Hugh] - Move enum mcopy_atomic_mode forward declare out of CONFIG_HUGETLB_PAGE. [Hugh] - Keep mistakenly removed UFFD_USER_MODE_ONLY in selftest. [Peter] - Cleanup context management in self test (make clear implicit, remove unneeded return values now that we have err()). [Peter] - Correct dst_pte argument to dst_pmd in shmem_mcopy_atomic_pte macro. [Hugh] - Mention the new shmem support feature in documentation. [Hugh] v1->v2: - Pick up Reviewed-by's. - Don't swapin page when a minor fault occurs. Notice that it needs to be swapped in, and just immediately fire the minor fault. Let a future CONTINUE deal with swapping in the page. [Peter] - Clarify comment about i_size checks in mm/userfaultfd.c. [Peter] - Only forward declare once (out of #ifdef) in hugetlb.h. [Peter] Changes since [2]: - Squash the fixes ([2]) in with the original series ([1]). This makes reviewing easier, as we no longer have to sift through deltas undoing what we had done before. [Hugh, Peter] - Modify shmem_mcopy_atomic_pte() to use the new mcopy_atomic_install_ptes() helper, reducing code duplication. [Hugh] - Properly trigger handle_userfault() in the shmem_swapin_page() case. [Hugh] - Use shmem_getpage() instead of find_lock_page() to lookup the existing page in for continue. This properly deals with swapped-out pages. [Hugh] - Unconditionally pte_mkdirty() for anon memory (as before). [Peter] - Don't include userfaultfd_k.h in either hugetlb.h or shmem_fs.h. [Hugh] - Add comment for UFFD_FEATURE_MINOR_SHMEM (to match _HUGETLBFS). [Hugh] - Fix some small cleanup issues (parens, reworded conditionals, reduced plumbing of some parameters, simplify labels/gotos, ...). [Hugh, Peter] Overview ======== See the series which added minor faults for hugetlbfs [3] for a detailed overview of minor fault handling in general. This series adds the same support for shmem-backed areas. This series is structured as follows: - Commits 1 and 2 are cleanups. - Commits 3 and 4 implement the new feature (minor fault handling for shmem). - Commits 5, 6, 7, 8 update the userfaultfd selftest to exercise the feature. - Commit 9 is one final cleanup, modifying an existing code path to re-use a new helper we've introduced. We rely on the selftest to show that this change doesn't break anything. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra <lokeshgidra(a)google.com>) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." [1] https://lore.kernel.org/patchwork/cover/1388144/ [2] https://lore.kernel.org/patchwork/patch/1408161/ [3] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen… Axel Rasmussen (10): userfaultfd/hugetlbfs: avoid including userfaultfd_k.h in hugetlb.h userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte userfaultfd/shmem: support UFFDIO_CONTINUE for shmem userfaultfd/shmem: support minor fault registration for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support userfaultfd/shmem: modify shmem_mcopy_atomic_pte to use install_ptes userfaultfd: update documentation to mention shmem minor faults Documentation/admin-guide/mm/userfaultfd.rst | 3 +- fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 4 +- include/linux/shmem_fs.h | 17 +- include/linux/userfaultfd_k.h | 5 + include/uapi/linux/userfaultfd.h | 7 +- mm/hugetlb.c | 1 + mm/memory.c | 8 +- mm/shmem.c | 114 +++----- mm/userfaultfd.c | 183 +++++++++---- tools/testing/selftests/vm/userfaultfd.c | 274 ++++++++++++------- 11 files changed, 371 insertions(+), 251 deletions(-) -- 2.31.1.368.gbe11c130af-goog

4 years, 2 months

2
12
0 0

[PATCH v2 0/9] userfaultfd: add minor fault handling for shmem

by Axel Rasmussen

Base ==== This series is based on (and therefore should apply cleanly to) the tag "v5.12-rc7-mmots-2021-04-11-20-49", additionally with Peter's selftest cleanup series applied *first*: https://lore.kernel.org/patchwork/cover/1412450/ Changelog ========= v1->v2: - Pick up Reviewed-by's. - Don't swapin page when a minor fault occurs. Notice that it needs to be swapped in, and just immediately fire the minor fault. Let a future CONTINUE deal with swapping in the page. [Peter] - Clarify comment about i_size checks in mm/userfaultfd.c. [Peter] - Only forward declare once (out of #ifdef) in hugetlb.h. [Peter] Changes since [2]: - Squash the fixes ([2]) in with the original series ([1]). This makes reviewing easier, as we no longer have to sift through deltas undoing what we had done before. [Hugh, Peter] - Modify shmem_mcopy_atomic_pte() to use the new mcopy_atomic_install_ptes() helper, reducing code duplication. [Hugh] - Properly trigger handle_userfault() in the shmem_swapin_page() case. [Hugh] - Use shmem_getpage() instead of find_lock_page() to lookup the existing page in for continue. This properly deals with swapped-out pages. [Hugh] - Unconditionally pte_mkdirty() for anon memory (as before). [Peter] - Don't include userfaultfd_k.h in either hugetlb.h or shmem_fs.h. [Hugh] - Add comment for UFFD_FEATURE_MINOR_SHMEM (to match _HUGETLBFS). [Hugh] - Fix some small cleanup issues (parens, reworded conditionals, reduced plumbing of some parameters, simplify labels/gotos, ...). [Hugh, Peter] Overview ======== See the series which added minor faults for hugetlbfs [3] for a detailed overview of minor fault handling in general. This series adds the same support for shmem-backed areas. This series is structured as follows: - Commits 1 and 2 are cleanups. - Commits 3 and 4 implement the new feature (minor fault handling for shmem). - Commits 5, 6, 7, 8 update the userfaultfd selftest to exercise the feature. - Commit 9 is one final cleanup, modifying an existing code path to re-use a new helper we've introduced. We rely on the selftest to show that this change doesn't break anything. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra <lokeshgidra(a)google.com>) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." [1] https://lore.kernel.org/patchwork/cover/1388144/ [2] https://lore.kernel.org/patchwork/patch/1408161/ [3] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen… Axel Rasmussen (9): userfaultfd/hugetlbfs: avoid including userfaultfd_k.h in hugetlb.h userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte userfaultfd/shmem: support minor fault registration for shmem userfaultfd/shmem: support UFFDIO_CONTINUE for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support userfaultfd/shmem: modify shmem_mcopy_atomic_pte to use install_ptes fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 4 +- include/linux/shmem_fs.h | 15 +- include/linux/userfaultfd_k.h | 5 + include/uapi/linux/userfaultfd.h | 7 +- mm/hugetlb.c | 1 + mm/memory.c | 8 +- mm/shmem.c | 112 +++------ mm/userfaultfd.c | 183 ++++++++++----- tools/testing/selftests/vm/userfaultfd.c | 280 +++++++++++++++-------- 10 files changed, 377 insertions(+), 244 deletions(-) -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

3
22
0 0

[PATCH v2 0/2] secretmem: optimize page_is_secretmem()

by Mike Rapoport

From: Mike Rapoport <rppt(a)linux.ibm.com> Hi, This is an updated version of page_is_secretmem() changes. This is based on v5.12-rc7-mmots-2021-04-15-16-28. @Andrew, please let me know if you'd like me to rebase it differently or resend the entire set. v2: * move the check for secretmem page in gup_pte_range after we get a reference to the page, per Matthew. Mike Rapoport (2): secretmem/gup: don't check if page is secretmem without reference secretmem: optimize page_is_secretmem() include/linux/secretmem.h | 26 +++++++++++++++++++++++++- mm/gup.c | 6 +++--- mm/secretmem.c | 12 +----------- 3 files changed, 29 insertions(+), 15 deletions(-) -- 2.28.0

4 years, 2 months

3
7
0 0

[PATCH] secretmem: optimize page_is_secretmem()

by Mike Rapoport

From: Mike Rapoport <rppt(a)linux.ibm.com> Kernel test robot reported -4.2% regression of will-it-scale.per_thread_ops due to commit "mm: introduce memfd_secret system call to create "secret" memory areas". The perf profile of the test indicated that the regression is caused by page_is_secretmem() called from gup_pte_range() (inlined by gup_pgd_range): 27.76 +2.5 30.23 perf-profile.children.cycles-pp.gup_pgd_range 0.00 +3.2 3.19 ± 2% perf-profile.children.cycles-pp.page_mapping 0.00 +3.7 3.66 ± 2% perf-profile.children.cycles-pp.page_is_secretmem Further analysis showed that the slow down happens because neither page_is_secretmem() nor page_mapping() are not inline and moreover, multiple page flags checks in page_mapping() involve calling compound_head() several times for the same page. Make page_is_secretmem() inline and replace page_mapping() with page flag checks that do not imply page-to-head conversion. Reported-by: kernel test robot <oliver.sang(a)intel.com> Signed-off-by: Mike Rapoport <rppt(a)linux.ibm.com> --- @Andrew, The patch is vs v5.12-rc7-mmots-2021-04-15-16-28, I'd appreciate if it would be added as a fixup to the memfd_secret series. include/linux/secretmem.h | 26 +++++++++++++++++++++++++- mm/secretmem.c | 12 +----------- 2 files changed, 26 insertions(+), 12 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index 907a6734059c..b842b38cbeb1 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -4,8 +4,32 @@ #ifdef CONFIG_SECRETMEM +extern const struct address_space_operations secretmem_aops; + +static inline bool page_is_secretmem(struct page *page) +{ + struct address_space *mapping; + + /* + * Using page_mapping() is quite slow because of the actual call + * instruction and repeated compound_head(page) inside the + * page_mapping() function. + * We know that secretmem pages are not compound and LRU so we can + * save a couple of cycles here. + */ + if (PageCompound(page) || !PageLRU(page)) + return false; + + mapping = (struct address_space *) + ((unsigned long)page->mapping & ~PAGE_MAPPING_FLAGS); + + if (mapping != page->mapping) + return false; + + return page->mapping->a_ops == &secretmem_aops; +} + bool vma_is_secretmem(struct vm_area_struct *vma); -bool page_is_secretmem(struct page *page); bool secretmem_active(void); #else diff --git a/mm/secretmem.c b/mm/secretmem.c index 3b1ba3991964..0bcd15e1b549 100644 --- a/mm/secretmem.c +++ b/mm/secretmem.c @@ -151,22 +151,12 @@ static void secretmem_freepage(struct page *page) clear_highpage(page); } -static const struct address_space_operations secretmem_aops = { +const struct address_space_operations secretmem_aops = { .freepage = secretmem_freepage, .migratepage = secretmem_migratepage, .isolate_page = secretmem_isolate_page, }; -bool page_is_secretmem(struct page *page) -{ - struct address_space *mapping = page_mapping(page); - - if (!mapping) - return false; - - return mapping->a_ops == &secretmem_aops; -} - static struct vfsmount *secretmem_mnt; static struct file *secretmem_file_create(unsigned long flags) -- 2.28.0

4 years, 2 months

3
11
0 0

[PATCH bpf-next v4 0/3] add batched ops for percpu array

by Pedro Tammela

This patchset introduces batched operations for the per-cpu variant of the array map. It also removes the percpu macros from 'bpf_util.h'. This change was suggested by Andrii in a earlier iteration of this patchset. The tests were updated to reflect all the new changes. v3 -> v4: - Prefer 'calloc()' over 'malloc()' on batch ops tests - Add missing static keyword in a couple of test functions - 'offset' to 'cpu_offset' as suggested by Martin v2 -> v3: - Remove percpu macros as suggested by Andrii - Update tests that used the per cpu macros v1 -> v2: - Amended a more descriptive commit message Pedro Tammela (3): bpf: add batched ops support for percpu array bpf: selftests: remove percpu macros from bpf_util.h bpf: selftests: update array map tests for per-cpu batched ops kernel/bpf/arraymap.c | 2 + tools/testing/selftests/bpf/bpf_util.h | 7 -- .../bpf/map_tests/array_map_batch_ops.c | 104 +++++++++++++----- .../bpf/map_tests/htab_map_batch_ops.c | 87 +++++++-------- .../selftests/bpf/prog_tests/map_init.c | 9 +- tools/testing/selftests/bpf/test_maps.c | 84 ++++++++------ 6 files changed, 173 insertions(+), 120 deletions(-) -- 2.25.1

4 years, 2 months

3
5
0 0

[PATCH v2 5/5] selftests/vm: add test for MADV_POPULATE_(READ|WRITE)

by David Hildenbrand

Let's add a simple test for MADV_POPULATE_READ and MADV_POPULATE_WRITE, verifying some error handling, that population works, and that softdirty tracking works as expected. For now, limit the test to private anonymous memory. Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Arnd Bergmann <arnd(a)arndb.de> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Oscar Salvador <osalvador(a)suse.de> Cc: Matthew Wilcox (Oracle) <willy(a)infradead.org> Cc: Andrea Arcangeli <aarcange(a)redhat.com> Cc: Minchan Kim <minchan(a)kernel.org> Cc: Jann Horn <jannh(a)google.com> Cc: Jason Gunthorpe <jgg(a)ziepe.ca> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Hugh Dickins <hughd(a)google.com> Cc: Rik van Riel <riel(a)surriel.com> Cc: Michael S. Tsirkin <mst(a)redhat.com> Cc: Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: Richard Henderson <rth(a)twiddle.net> Cc: Ivan Kokshaysky <ink(a)jurassic.park.msu.ru> Cc: Matt Turner <mattst88(a)gmail.com> Cc: Thomas Bogendoerfer <tsbogend(a)alpha.franken.de> Cc: "James E.J. Bottomley" <James.Bottomley(a)HansenPartnership.com> Cc: Helge Deller <deller(a)gmx.de> Cc: Chris Zankel <chris(a)zankel.net> Cc: Max Filippov <jcmvbkbc(a)gmail.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: Peter Xu <peterx(a)redhat.com> Cc: Rolf Eike Beer <eike-kernel(a)sf-tec.de> Cc: Shuah Khan <shuah(a)kernel.org> Cc: linux-alpha(a)vger.kernel.org Cc: linux-mips(a)vger.kernel.org Cc: linux-parisc(a)vger.kernel.org Cc: linux-xtensa(a)linux-xtensa.org Cc: linux-arch(a)vger.kernel.org Cc: linux-kselftest(a)vger.kernel.org Cc: Linux API <linux-api(a)vger.kernel.org> Signed-off-by: David Hildenbrand <david(a)redhat.com> --- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 1 + tools/testing/selftests/vm/madv_populate.c | 342 +++++++++++++++++++++ tools/testing/selftests/vm/run_vmtests.sh | 16 + 4 files changed, 360 insertions(+) create mode 100644 tools/testing/selftests/vm/madv_populate.c diff --git a/tools/testing/selftests/vm/.gitignore b/tools/testing/selftests/vm/.gitignore index b4fc0148360e..c9a5dd1adf7d 100644 --- a/tools/testing/selftests/vm/.gitignore +++ b/tools/testing/selftests/vm/.gitignore @@ -24,3 +24,4 @@ hmm-tests local_config.* protection_keys_32 protection_keys_64 +madv_populate diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile index 8b0cd421ebd3..04b6650c1924 100644 --- a/tools/testing/selftests/vm/Makefile +++ b/tools/testing/selftests/vm/Makefile @@ -42,6 +42,7 @@ TEST_GEN_FILES += on-fault-limit TEST_GEN_FILES += thuge-gen TEST_GEN_FILES += transhuge-stress TEST_GEN_FILES += userfaultfd +TEST_GEN_FILES += madv_populate ifeq ($(MACHINE),x86_64) CAN_BUILD_I386 := $(shell ./../x86/check_cc.sh $(CC) ../x86/trivial_32bit_program.c -m32) diff --git a/tools/testing/selftests/vm/madv_populate.c b/tools/testing/selftests/vm/madv_populate.c new file mode 100644 index 000000000000..b959e4ebdad4 --- /dev/null +++ b/tools/testing/selftests/vm/madv_populate.c @@ -0,0 +1,342 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * MADV_POPULATE_READ and MADV_POPULATE_WRITE tests + * + * Copyright 2021, Red Hat, Inc. + * + * Author(s): David Hildenbrand <david(a)redhat.com> + */ +#define _GNU_SOURCE +#include <stdlib.h> +#include <string.h> +#include <stdbool.h> +#include <stdint.h> +#include <unistd.h> +#include <errno.h> +#include <fcntl.h> +#include <sys/mman.h> + +#include "../kselftest.h" + +#if defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) + +/* + * For now, we're using 2 MiB of private anonymous memory for all tests. + */ +#define SIZE (2 * 1024 * 1024) + +static size_t pagesize; + +static uint64_t pagemap_get_entry(int fd, char *start) +{ + const unsigned long pfn = (unsigned long)start / pagesize; + uint64_t entry; + int ret; + + ret = pread(fd, &entry, sizeof(entry), pfn * sizeof(entry)); + if (ret != sizeof(entry)) + ksft_exit_fail_msg("reading pagemap failed\n"); + return entry; +} + +static bool pagemap_is_populated(int fd, char *start) +{ + uint64_t entry = pagemap_get_entry(fd, start); + + /* Present or swapped. */ + return entry & 0xc000000000000000ull; +} + +static bool pagemap_is_softdirty(int fd, char *start) +{ + uint64_t entry = pagemap_get_entry(fd, start); + + return entry & 0x0080000000000000ull; +} + +static void sense_support(void) +{ + char *addr; + int ret; + + addr = mmap(0, pagesize, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (!addr) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, pagesize, MADV_POPULATE_READ); + if (ret) + ksft_exit_skip("MADV_POPULATE_READ is not available\n"); + + ret = madvise(addr, pagesize, MADV_POPULATE_WRITE); + if (ret) + ksft_exit_skip("MADV_POPULATE_WRITE is not available\n"); + + munmap(addr, pagesize); +} + +static void test_prot_read(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ with PROT_READ\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == EINVAL, + "MADV_POPULATE_WRITE with PROT_READ\n"); + + munmap(addr, SIZE); +} + +static void test_prot_write(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == EINVAL, + "MADV_POPULATE_READ with PROT_WRITE\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE with PROT_WRITE\n"); + + munmap(addr, SIZE); +} + +static void test_holes(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ret = munmap(addr + pagesize, pagesize); + if (ret) + ksft_exit_fail_msg("munmap failed\n"); + + /* Hole in the middle */ + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes in the middle\n"); + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes in the middle\n"); + + /* Hole at end */ + ret = madvise(addr, 2 * pagesize, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes at the end\n"); + ret = madvise(addr, 2 * pagesize, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes at the end\n"); + + /* Hole at beginning */ + ret = madvise(addr + pagesize, pagesize, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes at the beginning\n"); + ret = madvise(addr + pagesize, pagesize, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes at the beginning\n"); + + munmap(addr, SIZE); +} + +static bool range_is_populated(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (!pagemap_is_populated(fd, start)) + ret = false; + close(fd); + return ret; +} + +static bool range_is_not_populated(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (pagemap_is_populated(fd, start)) + ret = false; + close(fd); + return ret; +} + +static void test_populate_read(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ksft_test_result(range_is_not_populated(addr, SIZE), + "range initially not populated\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ\n"); + ksft_test_result(range_is_populated(addr, SIZE), + "range is populated\n"); + + munmap(addr, SIZE); +} + +static void test_populate_write(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ksft_test_result(range_is_not_populated(addr, SIZE), + "range initially not populated\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE\n"); + ksft_test_result(range_is_populated(addr, SIZE), + "range is populated\n"); + + munmap(addr, SIZE); +} + +static bool range_is_softdirty(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (!pagemap_is_softdirty(fd, start)) + ret = false; + close(fd); + return ret; +} + +static bool range_is_not_softdirty(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (pagemap_is_softdirty(fd, start)) + ret = false; + close(fd); + return ret; +} + +static void clear_softdirty(void) +{ + int fd = open("/proc/self/clear_refs", O_WRONLY); + const char *ctrl = "4"; + int ret; + + if (fd < 0) + ksft_exit_fail_msg("opening clear_refs failed\n"); + ret = write(fd, ctrl, strlen(ctrl)); + if (ret != strlen(ctrl)) + ksft_exit_fail_msg("writing clear_refs failed\n"); + close(fd); +} + +static void test_softdirty(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + /* Clear any softdirty bits. */ + clear_softdirty(); + ksft_test_result(range_is_not_softdirty(addr, SIZE), + "range is not softdirty\n"); + + /* Populating READ should set softdirty. */ + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ\n"); + ksft_test_result(range_is_not_softdirty(addr, SIZE), + "range is not softdirty\n"); + + /* Populating WRITE should set softdirty. */ + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE\n"); + ksft_test_result(range_is_softdirty(addr, SIZE), + "range is softdirty\n"); + + munmap(addr, SIZE); +} + +int main(int argc, char **argv) +{ + int err; + + pagesize = getpagesize(); + + ksft_print_header(); + ksft_set_plan(21); + + sense_support(); + test_prot_read(); + test_prot_write(); + test_holes(); + test_populate_read(); + test_populate_write(); + test_softdirty(); + + err = ksft_get_fail_cnt(); + if (err) + ksft_exit_fail_msg("%d out of %d tests failed\n", + err, ksft_test_num()); + return ksft_exit_pass(); +} + +#else /* defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) */ + +#warning "missing MADV_POPULATE_READ or MADV_POPULATE_WRITE definition" + +int main(int argc, char **argv) +{ + ksft_print_header(); + ksft_exit_skip("MADV_POPULATE_READ or MADV_POPULATE_WRITE not defined\n"); +} + +#endif /* defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) */ diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh index e953f3cd9664..955782d138ab 100755 --- a/tools/testing/selftests/vm/run_vmtests.sh +++ b/tools/testing/selftests/vm/run_vmtests.sh @@ -346,4 +346,20 @@ else exitcode=1 fi +echo "--------------------------------------------------------" +echo "running MADV_POPULATE_READ and MADV_POPULATE_WRITE tests" +echo "--------------------------------------------------------" +./madv_populate +ret_val=$? + +if [ $ret_val -eq 0 ]; then + echo "[PASS]" +elif [ $ret_val -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip +else + echo "[FAIL]" + exitcode=1 +fi + exit $exitcode -- 2.30.2

4 years, 2 months

1
0
0 0

[PATCH v2 4/5] selftests/vm: add protection_keys_32 / protection_keys_64 to gitignore

by David Hildenbrand

We missed to add two binaries to gitignore. Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Oscar Salvador <osalvador(a)suse.de> Cc: Jason Gunthorpe <jgg(a)ziepe.ca> Cc: Peter Xu <peterx(a)redhat.com> Cc: Ram Pai <linuxram(a)us.ibm.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: linux-kselftest(a)vger.kernel.org Signed-off-by: David Hildenbrand <david(a)redhat.com> --- tools/testing/selftests/vm/.gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/vm/.gitignore b/tools/testing/selftests/vm/.gitignore index 9a35c3f6a557..b4fc0148360e 100644 --- a/tools/testing/selftests/vm/.gitignore +++ b/tools/testing/selftests/vm/.gitignore @@ -22,3 +22,5 @@ map_fixed_noreplace write_to_hugetlbfs hmm-tests local_config.* +protection_keys_32 +protection_keys_64 -- 2.30.2

4 years, 2 months

1
0
0 0

[PATCH v2 3/5] MAINTAINERS: add tools/testing/selftests/vm/ to MEMORY MANAGEMENT

by David Hildenbrand

MEMORY MANAGEMENT seems to be a good fit. Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Oscar Salvador <osalvador(a)suse.de> Cc: Jason Gunthorpe <jgg(a)ziepe.ca> Cc: Peter Xu <peterx(a)redhat.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: linux-kselftest(a)vger.kernel.org Signed-off-by: David Hildenbrand <david(a)redhat.com> --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 9450e052f1b1..cd267d218e08 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -11566,6 +11566,7 @@ F: include/linux/mm.h F: include/linux/mmzone.h F: include/linux/vmalloc.h F: mm/ +F: tools/testing/selftests/vm/ MEMORY TECHNOLOGY DEVICES (MTD) M: Miquel Raynal <miquel.raynal(a)bootlin.com> -- 2.30.2

4 years, 2 months

1
0
0 0

[PATCH v2 0/4] KVM statistics data fd-based binary interface

by Jing Zhang

This patchset provides a file descriptor for every VM and VCPU to read KVM statistics data in binary format. It is meant to provide a lightweight, flexible, scalable and efficient lock-free solution for user space telemetry applications to pull the statistics data periodically for large scale systems. The pulling frequency could be as high as a few times per second. In this patchset, every statistics data are treated to have some attributes as below: * architecture dependent or common * VM statistics data or VCPU statistics data * type: cumulative, instantaneous, * unit: none for simple counter, nanosecond, microsecond, millisecond, second, Byte, KiByte, MiByte, GiByte. Clock Cycles Since no lock/synchronization is used, the consistency between all the statistics data is not guaranteed. That means not all statistics data are read out at the exact same time, since the statistics date are still being updated by KVM subsystems while they are read out. --- * v1 -> v2 - Use ARRAY_SIZE to count the number of stats descriptors - Fix missing `size` field initialization in macro STATS_DESC [1] https://lore.kernel.org/kvm/20210402224359.2297157-1-jingzhangos@google.com --- Jing Zhang (4): KVM: stats: Separate common stats from architecture specific ones KVM: stats: Add fd-based API to read binary stats data KVM: stats: Add documentation for statistics data binary interface KVM: selftests: Add selftest for KVM statistics data binary interface Documentation/virt/kvm/api.rst | 169 ++++++++ arch/arm64/include/asm/kvm_host.h | 9 +- arch/arm64/kvm/guest.c | 42 +- arch/mips/include/asm/kvm_host.h | 9 +- arch/mips/kvm/mips.c | 67 +++- arch/powerpc/include/asm/kvm_host.h | 9 +- arch/powerpc/kvm/book3s.c | 68 +++- arch/powerpc/kvm/book3s_hv.c | 12 +- arch/powerpc/kvm/book3s_pr.c | 2 +- arch/powerpc/kvm/book3s_pr_papr.c | 2 +- arch/powerpc/kvm/booke.c | 63 ++- arch/s390/include/asm/kvm_host.h | 9 +- arch/s390/kvm/kvm-s390.c | 133 ++++++- arch/x86/include/asm/kvm_host.h | 9 +- arch/x86/kvm/x86.c | 71 +++- include/linux/kvm_host.h | 132 ++++++- include/linux/kvm_types.h | 12 + include/uapi/linux/kvm.h | 48 +++ tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 3 + .../testing/selftests/kvm/include/kvm_util.h | 3 + .../selftests/kvm/kvm_bin_form_stats.c | 370 ++++++++++++++++++ tools/testing/selftests/kvm/lib/kvm_util.c | 11 + virt/kvm/kvm_main.c | 237 ++++++++++- 24 files changed, 1401 insertions(+), 90 deletions(-) create mode 100644 tools/testing/selftests/kvm/kvm_bin_form_stats.c base-commit: f96be2deac9bca3ef5a2b0b66b71fcef8bad586d -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

3
8
0 0

[PATCH] KVM: selftests: Tweak time measurement flag in kvm_page_table_test

by Yanan Wang

Also use CLOCK_MONOTONIC flag to get time in kvm_page_table_test.c, since that's what all the kvm/selftests do currently. And this will be consistent with function timespec_elapsed() in test_util.c. Signed-off-by: Yanan Wang <wangyanan55(a)huawei.com> --- tools/testing/selftests/kvm/kvm_page_table_test.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kvm/kvm_page_table_test.c b/tools/testing/selftests/kvm/kvm_page_table_test.c index 1c4753fff19e..d7847fba47a8 100644 --- a/tools/testing/selftests/kvm/kvm_page_table_test.c +++ b/tools/testing/selftests/kvm/kvm_page_table_test.c @@ -212,7 +212,7 @@ static void *vcpu_worker(void *data) if (READ_ONCE(host_quit)) return NULL; - clock_gettime(CLOCK_MONOTONIC_RAW, &start); + clock_gettime(CLOCK_MONOTONIC, &start); ret = _vcpu_run(vm, vcpu_id); ts_diff = timespec_elapsed(start); @@ -390,7 +390,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) /* Test the stage of KVM creating mappings */ *current_stage = KVM_CREATE_MAPPINGS; - clock_gettime(CLOCK_MONOTONIC_RAW, &start); + clock_gettime(CLOCK_MONOTONIC, &start); vcpus_complete_new_stage(*current_stage); ts_diff = timespec_elapsed(start); @@ -403,7 +403,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) *current_stage = KVM_UPDATE_MAPPINGS; - clock_gettime(CLOCK_MONOTONIC_RAW, &start); + clock_gettime(CLOCK_MONOTONIC, &start); vcpus_complete_new_stage(*current_stage); ts_diff = timespec_elapsed(start); @@ -415,7 +415,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) *current_stage = KVM_ADJUST_MAPPINGS; - clock_gettime(CLOCK_MONOTONIC_RAW, &start); + clock_gettime(CLOCK_MONOTONIC, &start); vcpus_complete_new_stage(*current_stage); ts_diff = timespec_elapsed(start); -- 2.23.0

4 years, 2 months

1
0
0 0

[PATCH v6 00/10] KVM: selftests: some improvement and a new test for kvm page table

by Yanan Wang

Hi, This v6 series can mainly include two parts. Rebased on kvm queue branch: https://git.kernel.org/pub/scm/virt/kvm/kvm.git/log/?h=queue In the first part, all the known hugetlb backing src types specified with different hugepage sizes are listed, so that we can specify use of hugetlb source of the exact granularity that we want, instead of the system default ones. And as all the known hugetlb page sizes are listed, it's appropriate for all architectures. Besides, a helper that can get granularity of different backing src types(anonumous/thp/hugetlb) is added, so that we can use the accurate backing src granularity for kinds of alignment or guest memory accessing of vcpus. In the second part, a new test is added: This test is added to serve as a performance tester and a bug reproducer for kvm page table code (GPA->HPA mappings), it gives guidance for the people trying to make some improvement for kvm. And the following explains what we can exactly do through this test. The function guest_code() can cover the conditions where a single vcpu or multiple vcpus access guest pages within the same memory region, in three VM stages(before dirty logging, during dirty logging, after dirty logging). Besides, the backing src memory type(ANONYMOUS/THP/HUGETLB) of the tested memory region can be specified by users, which means normal page mappings or block mappings can be chosen by users to be created in the test. If ANONYMOUS memory is specified, kvm will create normal page mappings for the tested memory region before dirty logging, and update attributes of the page mappings from RO to RW during dirty logging. If THP/HUGETLB memory is specified, kvm will create block mappings for the tested memory region before dirty logging, and split the blcok mappings into normal page mappings during dirty logging, and coalesce the page mappings back into block mappings after dirty logging is stopped. So in summary, as a performance tester, this test can present the performance of kvm creating/updating normal page mappings, or the performance of kvm creating/splitting/recovering block mappings, through execution time. When we need to coalesce the page mappings back to block mappings after dirty logging is stopped, we have to firstly invalidate *all* the TLB entries for the page mappings right before installation of the block entry, because a TLB conflict abort error could occur if we can't invalidate the TLB entries fully. We have hit this TLB conflict twice on aarch64 software implementation and fixed it. As this test can imulate process from dirty logging enabled to dirty logging stopped of a VM with block mappings, so it can also reproduce this TLB conflict abort due to inadequate TLB invalidation when coalescing tables. Links about the TLB conflict abort: https://lore.kernel.org/lkml/20201201201034.116760-3-wangyanan55@huawei.com/ --- Change logs: v5->v6: - Address Andrew Jones's comments for v5 series - Add Andrew Jones's R-b tags in some patches - Rebased on newest kvm/queue tree - v5: https://lore.kernel.org/lkml/20210323135231.24948-1-wangyanan55@huawei.com/ v4->v5: - Use synchronization(sem_wait) for time measurement - Add a new patch about TEST_ASSERT(patch 4) - Address Andrew Jones's comments for v4 series - Add Andrew Jones's R-b tags in some patches - v4: https://lore.kernel.org/lkml/20210302125751.19080-1-wangyanan55@huawei.com/ v3->v4: - Add a helper to get system default hugetlb page size - Add tags of Reviewed-by of Ben in the patches - v3: https://lore.kernel.org/lkml/20210301065916.11484-1-wangyanan55@huawei.com/ v2->v3: - Add tags of Suggested-by, Reviewed-by in the patches - Add a generic micro to get hugetlb page sizes - Some changes for suggestions about v2 series - v2: https://lore.kernel.org/lkml/20210225055940.18748-1-wangyanan55@huawei.com/ v1->v2: - Add a patch to sync header files - Add helpers to get granularity of different backing src types - Some changes for suggestions about v1 series - v1: https://lore.kernel.org/lkml/20210208090841.333724-1-wangyanan55@huawei.com/ --- Yanan Wang (10): tools headers: sync headers of asm-generic/hugetlb_encode.h mm/hugetlb: Add a macro to get HUGETLB page sizes for mmap KVM: selftests: Use flag CLOCK_MONOTONIC_RAW for timing KVM: selftests: Print the errno besides error-string in TEST_ASSERT KVM: selftests: Make a generic helper to get vm guest mode strings KVM: selftests: Add a helper to get system configured THP page size KVM: selftests: Add a helper to get system default hugetlb page size KVM: selftests: List all hugetlb src types specified with page sizes KVM: selftests: Adapt vm_userspace_mem_region_add to new helpers KVM: selftests: Add a test for kvm page table code include/uapi/linux/mman.h | 2 + tools/include/asm-generic/hugetlb_encode.h | 3 + tools/include/uapi/linux/mman.h | 2 + tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 3 + .../selftests/kvm/demand_paging_test.c | 8 +- .../selftests/kvm/dirty_log_perf_test.c | 14 +- .../testing/selftests/kvm/include/kvm_util.h | 4 +- .../testing/selftests/kvm/include/test_util.h | 21 +- .../selftests/kvm/kvm_page_table_test.c | 506 ++++++++++++++++++ tools/testing/selftests/kvm/lib/assert.c | 4 +- tools/testing/selftests/kvm/lib/kvm_util.c | 59 +- tools/testing/selftests/kvm/lib/test_util.c | 163 +++++- tools/testing/selftests/kvm/steal_time.c | 4 +- 14 files changed, 733 insertions(+), 61 deletions(-) create mode 100644 tools/testing/selftests/kvm/kvm_page_table_test.c -- 2.23.0

4 years, 2 months

4
16
0 0

[PATCH v2] tools: do not include scripts/Kbuild.include

by Masahiro Yamada

Since commit d9f4ff50d2aa ("kbuild: spilt cc-option and friends to scripts/Makefile.compiler"), some kselftests fail to build. The tools/ directory opted out Kbuild, and went in a different direction. They copy any kind of files to the tools/ directory in order to do whatever they want in their world. tools/build/Build.include mimics scripts/Kbuild.include, but some tool Makefiles included the Kbuild one to import a feature that is missing in tools/build/Build.include: - Commit ec04aa3ae87b ("tools/thermal: tmon: use "-fstack-protector" only if supported") included scripts/Kbuild.include from tools/thermal/tmon/Makefile to import the cc-option macro. - Commit c2390f16fc5b ("selftests: kvm: fix for compilers that do not support -no-pie") included scripts/Kbuild.include from tools/testing/selftests/kvm/Makefile to import the try-run macro. - Commit 9cae4ace80ef ("selftests/bpf: do not ignore clang failures") included scripts/Kbuild.include from tools/testing/selftests/bpf/Makefile to import the .DELETE_ON_ERROR target. - Commit 0695f8bca93e ("selftests/powerpc: Handle Makefile for unrecognized option") included scripts/Kbuild.include from tools/testing/selftests/powerpc/pmu/ebb/Makefile to import the try-run macro. Copy what they need into tools/build/Build.include, and make them include it instead of scripts/Kbuild.include. Link: https://lore.kernel.org/lkml/86dadf33-70f7-a5ac-cb8c-64966d2f45a1@linux.ibm… Fixes: d9f4ff50d2aa ("kbuild: spilt cc-option and friends to scripts/Makefile.compiler") Reported-by: Janosch Frank <frankja(a)linux.ibm.com> Reported-by: Christian Borntraeger <borntraeger(a)de.ibm.com> Signed-off-by: Masahiro Yamada <masahiroy(a)kernel.org> --- Changes in v2: - copy macros to tools/build/BUild.include tools/build/Build.include | 24 +++++++++++++++++++ tools/testing/selftests/bpf/Makefile | 2 +- tools/testing/selftests/kvm/Makefile | 2 +- .../selftests/powerpc/pmu/ebb/Makefile | 2 +- tools/thermal/tmon/Makefile | 2 +- 5 files changed, 28 insertions(+), 4 deletions(-) diff --git a/tools/build/Build.include b/tools/build/Build.include index 585486e40995..2cf3b1bde86e 100644 --- a/tools/build/Build.include +++ b/tools/build/Build.include @@ -100,3 +100,27 @@ cxx_flags = -Wp,-MD,$(depfile) -Wp,-MT,$@ $(CXXFLAGS) -D"BUILD_STR(s)=\#s" $(CXX ## HOSTCC C flags host_c_flags = -Wp,-MD,$(depfile) -Wp,-MT,$@ $(KBUILD_HOSTCFLAGS) -D"BUILD_STR(s)=\#s" $(HOSTCFLAGS_$(basetarget).o) $(HOSTCFLAGS_$(obj)) + +# output directory for tests below +TMPOUT = .tmp_$$$$ + +# try-run +# Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise) +# Exit code chooses option. "$$TMP" serves as a temporary file and is +# automatically cleaned up. +try-run = $(shell set -e; \ + TMP=$(TMPOUT)/tmp; \ + mkdir -p $(TMPOUT); \ + trap "rm -rf $(TMPOUT)" EXIT; \ + if ($(1)) >/dev/null 2>&1; \ + then echo "$(2)"; \ + else echo "$(3)"; \ + fi) + +# cc-option +# Usage: cflags-y += $(call cc-option,-march=winchip-c6,-march=i586) +cc-option = $(call try-run, \ + $(CC) -Werror $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2)) + +# delete partially updated (i.e. corrupted) files on error +.DELETE_ON_ERROR: diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 044bfdcf5b74..17a5cdf48d37 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -include ../../../../scripts/Kbuild.include +include ../../../build/Build.include include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index a6d61f451f88..5ef141f265bd 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -include ../../../../scripts/Kbuild.include +include ../../../build/Build.include all: diff --git a/tools/testing/selftests/powerpc/pmu/ebb/Makefile b/tools/testing/selftests/powerpc/pmu/ebb/Makefile index af3df79d8163..c5ecb4634094 100644 --- a/tools/testing/selftests/powerpc/pmu/ebb/Makefile +++ b/tools/testing/selftests/powerpc/pmu/ebb/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -include ../../../../../../scripts/Kbuild.include +include ../../../../../build/Build.include noarg: $(MAKE) -C ../../ diff --git a/tools/thermal/tmon/Makefile b/tools/thermal/tmon/Makefile index 59e417ec3e13..9db867df7679 100644 --- a/tools/thermal/tmon/Makefile +++ b/tools/thermal/tmon/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 # We need this for the "cc-option" macro. -include ../../../scripts/Kbuild.include +include ../../build/Build.include VERSION = 1.0 -- 2.27.0

4 years, 2 months

4
4
0 0

[PATCH v2] Documentation: kunit: Update kunit_tool page

by David Gow

The kunit_tool documentation page was pretty minimal, and a bit outdated. Update it and flesh it out a bit. In particular, - Mention that .kunitconfig is now in the build directory - Describe the use of --kunitconfig to specify a different config framgent - Mention the split functionality (i.e., commands other than 'run') - Describe --raw_output and kunit.py parse - Mention the globbing support - Provide a quick overview of other options, including --build_dir and --alltests Note that this does overlap a little with the new running_tips page. I don't think it's a problem having both: this page is supposed to be a bit more of a reference, rather than a list of useful tips, so the fact that they both describe the same features isn't a problem. Signed-off-by: David Gow <davidgow(a)google.com> Reviewed-by: Daniel Latypov <dlatypov(a)google.com> --- Adopted the changes from Daniel. Changes since v1: https://lore.kernel.org/linux-kselftest/20210416034036.797727-1-davidgow@go… - Mention that the default build directory is '.kunit' when discussing '.kunitconfig' files. - Reword the discussion of 'CONFIG_KUNIT_ALL_TESTS' under '--alltests' Documentation/dev-tools/kunit/kunit-tool.rst | 140 +++++++++++++++++-- 1 file changed, 132 insertions(+), 8 deletions(-) diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst index 29ae2fee8123..4247b7420e3b 100644 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ b/Documentation/dev-tools/kunit/kunit-tool.rst @@ -22,14 +22,19 @@ not require any virtualization support: it is just a regular program. What is a .kunitconfig? ======================= -It's just a defconfig that kunit_tool looks for in the base directory. -kunit_tool uses it to generate a .config as you might expect. In addition, it -verifies that the generated .config contains the CONFIG options in the -.kunitconfig; the reason it does this is so that it is easy to be sure that a -CONFIG that enables a test actually ends up in the .config. +It's just a defconfig that kunit_tool looks for in the build directory +(``.kunit`` by default). kunit_tool uses it to generate a .config as you might +expect. In addition, it verifies that the generated .config contains the CONFIG +options in the .kunitconfig; the reason it does this is so that it is easy to +be sure that a CONFIG that enables a test actually ends up in the .config. -How do I use kunit_tool? -======================== +It's also possible to pass a separate .kunitconfig fragment to kunit_tool, +which is useful if you have several different groups of tests you wish +to run independently, or if you want to use pre-defined test configs for +certain subsystems. + +Getting Started with kunit_tool +=============================== If a kunitconfig is present at the root directory, all you have to do is: @@ -48,10 +53,129 @@ However, you most likely want to use it with the following options: .. note:: This command will work even without a .kunitconfig file: if no - .kunitconfig is present, a default one will be used instead. + .kunitconfig is present, a default one will be used instead. + +If you wish to use a different .kunitconfig file (such as one provided for +testing a particular subsystem), you can pass it as an option. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig For a list of all the flags supported by kunit_tool, you can run: .. code-block:: bash ./tools/testing/kunit/kunit.py run --help + +Configuring, Building, and Running Tests +======================================== + +It's also possible to run just parts of the KUnit build process independently, +which is useful if you want to make manual changes to part of the process. + +A .config can be generated from a .kunitconfig by using the ``config`` argument +when running kunit_tool: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py config + +Similarly, if you just want to build a KUnit kernel from the current .config, +you can use the ``build`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py build + +And, if you already have a built UML kernel with built-in KUnit tests, you can +run the kernel and display the test results with the ``exec`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py exec + +The ``run`` command which is discussed above is equivalent to running all three +of these in sequence. + +All of these commands accept a number of optional command-line arguments. The +``--help`` flag will give a complete list of these, or keep reading this page +for a guide to some of the more useful ones. + +Parsing Test Results +==================== + +KUnit tests output their results in TAP (Test Anything Protocol) format. +kunit_tool will, when running tests, parse this output and print a summary +which is much more pleasant to read. If you wish to look at the raw test +results in TAP format, you can pass the ``--raw_output`` argument. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --raw_output + +.. note:: + The raw output from test runs may contain other, non-KUnit kernel log + lines. + +If you have KUnit results in their raw TAP format, you can parse them and print +the human-readable summary with the ``parse`` command for kunit_tool. This +accepts a filename for an argument, or will read from standard input. + +.. code-block:: bash + + # Reading from a file + ./tools/testing/kunit/kunit.py parse /var/log/dmesg + # Reading from stdin + dmesg | ./tools/testing/kunit/kunit.py parse + +This is very useful if you wish to run tests in a configuration not supported +by kunit_tool (such as on real hardware, or an unsupported architecture). + +Filtering Tests +=============== + +It's possible to run only a subset of the tests built into a kernel by passing +a filter to the ``exec`` or ``run`` commands. For example, if you only wanted +to run KUnit resource tests, you could use: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run 'kunit-resource*' + +This uses the standard glob format for wildcards. + +Other Useful Options +==================== + +kunit_tool has a number of other command-line arguments which can be useful +when adapting it to fit your environment or needs. + +Some of the more useful ones are: + +``--help`` + Lists all of the available options. Note that different commands + (``config``, ``build``, ``run``, etc) will have different supported + options. Place ``--help`` before the command to list common options, + and after the command for options specific to that command. + +``--build_dir`` + Specifies the build directory that kunit_tool will use. This is where + the .kunitconfig file is located, as well as where the .config and + compiled kernel will be placed. Defaults to ``.kunit``. + +``--make_options`` + Specifies additional options to pass to ``make`` when compiling a + kernel (with the ``build`` or ``run`` commands). For example, to enable + compiler warnings, you can pass ``--make_options W=1``. + +``--alltests`` + Builds a UML kernel with all config options enabled using ``make + allyesconfig``. This allows you to run as many tests as is possible, + but is very slow and prone to breakage as new options are added or + modified. In most cases, enabling all tests which have satisfied + dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your + .kunitconfig is preferable. + +There are several other options (and new ones are often added), so do check +``--help`` if you're looking for something not mentioned here. -- 2.31.1.368.gbe11c130af-goog

4 years, 2 months

1
0
0 0

[PATCH] Documentation: kunit: Update kunit_tool page

by David Gow

The kunit_tool documentation page was pretty minimal, and a bit outdated. Update it and flesh it out a bit. In particular, - Mention that .kunitconfig is now in the build directory - Describe the use of --kunitconfig to specify a different config framgent - Mention the split functionality (i.e., commands other than 'run') - Describe --raw_output and kunit.py parse - Mention the globbing support - Provide a quick overview of other options, including --build_dir and --alltests Note that this does overlap a little with the new running_tips page. I don't think it's a problem having both: this page is supposed to be a bit more of a reference, rather than a list of useful tips, so the fact that they both describe the same features isn't a problem. Signed-off-by: David Gow <davidgow(a)google.com> --- Documentation/dev-tools/kunit/kunit-tool.rst | 132 ++++++++++++++++++- 1 file changed, 128 insertions(+), 4 deletions(-) diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst index 29ae2fee8123..0b45affcd65c 100644 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ b/Documentation/dev-tools/kunit/kunit-tool.rst @@ -22,14 +22,19 @@ not require any virtualization support: it is just a regular program. What is a .kunitconfig? ======================= -It's just a defconfig that kunit_tool looks for in the base directory. +It's just a defconfig that kunit_tool looks for in the build directory. kunit_tool uses it to generate a .config as you might expect. In addition, it verifies that the generated .config contains the CONFIG options in the .kunitconfig; the reason it does this is so that it is easy to be sure that a CONFIG that enables a test actually ends up in the .config. -How do I use kunit_tool? -======================== +It's also possible to pass a separate .kunitconfig fragment to kunit_tool, +which is useful if you have several different groups of tests you wish +to run independently, or if you want to use pre-defined test configs for +certain subsystems. + +Getting Started with kunit_tool +=============================== If a kunitconfig is present at the root directory, all you have to do is: @@ -48,10 +53,129 @@ However, you most likely want to use it with the following options: .. note:: This command will work even without a .kunitconfig file: if no - .kunitconfig is present, a default one will be used instead. + .kunitconfig is present, a default one will be used instead. + +If you wish to use a different .kunitconfig file (such as one provided for +testing a particular subsystem), you can pass it as an option. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig For a list of all the flags supported by kunit_tool, you can run: .. code-block:: bash ./tools/testing/kunit/kunit.py run --help + +Configuring, Building, and Running Tests +======================================== + +It's also possible to run just parts of the KUnit build process independently, +which is useful if you want to make manual changes to part of the process. + +A .config can be generated from a .kunitconfig by using the ``config`` argument +when running kunit_tool: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py config + +Similarly, if you just want to build a KUnit kernel from the current .config, +you can use the ``build`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py build + +And, if you already have a built UML kernel with built-in KUnit tests, you can +run the kernel and display the test results with the ``exec`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py exec + +The ``run`` command which is discussed above is equivalent to running all three +of these in sequence. + +All of these commands accept a number of optional command-line arguments. The +``--help`` flag will give a complete list of these, or keep reading this page +for a guide to some of the more useful ones. + +Parsing Test Results +==================== + +KUnit tests output their results in TAP (Test Anything Protocol) format. +kunit_tool will, when running tests, parse this output and print a summary +which is much more pleasant to read. If you wish to look at the raw test +results in TAP format, you can pass the ``--raw_output`` argument. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --raw_output + +.. note:: + The raw output from test runs may contain other, non-KUnit kernel log + lines. + +If you have KUnit results in their raw TAP format, you can parse them and print +the human-readable summary with the ``parse`` command for kunit_tool. This +accepts a filename for an argument, or will read from standard input. + +.. code-block:: bash + + # Reading from a file + ./tools/testing/kunit/kunit.py parse /var/log/dmesg + # Reading from stdin + dmesg | ./tools/testing/kunit/kunit.py parse + +This is very useful if you wish to run tests in a configuration not supported +by kunit_tool (such as on real hardware, or an unsupported architecture). + +Filtering Tests +=============== + +It's possible to run only a subset of the tests built into a kernel by passing +a filter to the ``exec`` or ``run`` commands. For example, if you only wanted +to run KUnit resource tests, you could use: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run 'kunit-resource*' + +This uses the standard glob format for wildcards. + +Other Useful Options +==================== + +kunit_tool has a number of other command-line arguments which can be useful +when adapting it to fit your environment or needs. + +Some of the more useful ones are: + +``--help`` + Lists all of the available options. Note that different commands + (``config``, ``build``, ``run``, etc) will have different supported + options. Place ``--help`` before the command to list common options, + and after the command for options specific to that command. + +``--build_dir`` + Specifies the build directory that kunit_tool will use. This is where + the .kunitconfig file is located, as well as where the .config and + compiled kernel will be placed. Defaults to ``.kunit``. + +``--make_options`` + Specifies additional options to pass to ``make`` when compiling a + kernel (with the ``build`` or ``run`` commands). For example, to enable + compiler warnings, you can pass ``--make_options W=1``. + +``--alltests`` + Builds a UML kernel with all config options enabled using + ``make allyesconfig``. This allows you to run as many tests as is + possible, but is very slow and prone to breakage as new options are + added or modified. Most people should add ``CONFIG_KUNIT_ALL_TESTS=1`` + to their .kunitconfig instead if they wish to run "all tests". + + +There are several other options (and new ones are often added), so do check +``--help`` if you're looking for something not mentioned here. -- 2.31.1.368.gbe11c130af-goog

4 years, 2 months

2
1
0 0

[PATCH V6 00/10] PKS: Add Protection Key Supervisor support

by ira.weiny＠intel.com

From: Ira Weiny <ira.weiny(a)intel.com> Introduce a new page protection mechanism for supervisor pages, Protection Key Supervisor (PKS). Generally PKS enables protections on 'domains' of supervisor pages to limit supervisor mode access to pages beyond the normal paging protections. PKS works in a similar fashion to user space pkeys, PKU. As with PKU, supervisor pkeys are checked in addition to normal paging protections and Access or Writes can be disabled via a MSR update without TLB flushes when permissions change. Also like PKU, a page mapping is assigned to a domain by setting pkey bits in the page table entry for that mapping. Access is controlled through a PKRS register which is updated via WRMSR/RDMSR. XSAVE is not supported for the PKRS MSR. Therefore the implementation saves/restores the MSR across context switches and during exceptions. Nested exceptions are supported by each exception getting a new PKS state. For consistent behavior with current paging protections, pkey 0 is reserved and configured to allow full access via the pkey mechanism, thus preserving the default paging protections on mappings with the default pkey value of 0. Other keys, (1-15) are allocated by an allocator which prepares us for key contention from day one. Kernel users should be prepared for the allocator to fail either because of key exhaustion or due to PKS not being supported on the CPU instance. The following are key attributes of PKS. 1) Fast switching of permissions 1a) Prevents access without page table manipulations 1b) No TLB flushes required 2) Works on a per thread basis PKS is available with 4 and 5 level paging. Like PKRU it consumes 4 bits from the PTE to store the pkey within the entry. All code to support PKS is configured via ARCH_ENABLE_SUPERVISOR_PKEYS which is designed to only be turned on when a user is configured on in the kernel. Those users must depend on ARCH_HAS_SUPERVISOR_PKEYS to properly work with other architectures which do not yet support PKS. Originally this series was submitted as part of a large patch set which converted the kmap call sites.[1] Many follow on discussions revealed a few problems. The first of which was that some callers leak a kmap mapping across threads rather than containing it to a critical section. Attempts were made to see if these 'global kmaps' could be supported.[2] However, supporting global kmaps had many problems. Work is being done in parallel on converting as many kmap calls to the new kmap_local_page().[3] Changes from V5 [6] From Dave Hansen Remove 'we' from comments Changes from V4 [5] From kernel test robot <lkp(a)intel.com> Fix i386 build: pks_init_task not found Move MSR_IA32_PKRS and INIT_PKRS_VALUE into patch 5 where they are first 'used'. (Technically nothing is 'used' until the final test patch. But review wise this is much cleaner.) From Sean Christoperson Add documentation details on what happens if the pkey is violated Change cpu_feature_enabled to be in WARN_ON check Clean up commit message of patch 6 [1] https://lore.kernel.org/lkml/20201009195033.3208459-1-ira.weiny@intel.com/ [2] https://lore.kernel.org/lkml/87mtycqcjf.fsf@nanos.tec.linutronix.de/ [3] https://lore.kernel.org/lkml/20210128061503.1496847-1-ira.weiny@intel.com/ https://lore.kernel.org/lkml/20210210062221.3023586-1-ira.weiny@intel.com/ https://lore.kernel.org/lkml/20210205170030.856723-1-ira.weiny@intel.com/ https://lore.kernel.org/lkml/20210217024826.3466046-1-ira.weiny@intel.com/ [4] https://lore.kernel.org/lkml/20201106232908.364581-1-ira.weiny@intel.com/ [5] https://lore.kernel.org/lkml/20210322053020.2287058-1-ira.weiny@intel.com/ [6] https://lore.kernel.org/lkml/20210331191405.341999-1-ira.weiny@intel.com/ Fenghua Yu (1): x86/pks: Add PKS kernel API Ira Weiny (9): x86/pkeys: Create pkeys_common.h x86/fpu: Refactor arch_set_user_pkey_access() for PKS support x86/pks: Add additional PKEY helper macros x86/pks: Add PKS defines and Kconfig options x86/pks: Add PKS setup code x86/fault: Adjust WARN_ON for PKey fault x86/pks: Preserve the PKRS MSR on context switch x86/entry: Preserve PKRS MSR across exceptions x86/pks: Add PKS test code Documentation/core-api/protection-keys.rst | 112 +++- arch/x86/Kconfig | 1 + arch/x86/entry/calling.h | 26 + arch/x86/entry/common.c | 57 ++ arch/x86/entry/entry_64.S | 22 +- arch/x86/entry/entry_64_compat.S | 6 +- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/pgtable.h | 15 +- arch/x86/include/asm/pgtable_types.h | 12 + arch/x86/include/asm/pkeys.h | 4 + arch/x86/include/asm/pkeys_common.h | 34 + arch/x86/include/asm/pks.h | 54 ++ arch/x86/include/asm/processor-flags.h | 2 + arch/x86/include/asm/processor.h | 47 +- arch/x86/include/uapi/asm/processor-flags.h | 2 + arch/x86/kernel/cpu/common.c | 2 + arch/x86/kernel/fpu/xstate.c | 22 +- arch/x86/kernel/head_64.S | 7 +- arch/x86/kernel/process.c | 3 + arch/x86/kernel/process_64.c | 2 + arch/x86/mm/fault.c | 30 +- arch/x86/mm/pkeys.c | 218 +++++- include/linux/pgtable.h | 4 + include/linux/pkeys.h | 34 + kernel/entry/common.c | 14 +- lib/Kconfig.debug | 11 + lib/Makefile | 3 + lib/pks/Makefile | 3 + lib/pks/pks_test.c | 694 ++++++++++++++++++++ mm/Kconfig | 5 + tools/testing/selftests/x86/Makefile | 3 +- tools/testing/selftests/x86/test_pks.c | 149 +++++ 34 files changed, 1527 insertions(+), 81 deletions(-) create mode 100644 arch/x86/include/asm/pkeys_common.h create mode 100644 arch/x86/include/asm/pks.h create mode 100644 lib/pks/Makefile create mode 100644 lib/pks/pks_test.c create mode 100644 tools/testing/selftests/x86/test_pks.c -- 2.28.0.rc0.12.gb6a658bd00c9

4 years, 2 months

2
11
0 0

[PATCH v2] kunit: add unit test for filtering suites by names

by Daniel Latypov

This adds unit tests for kunit_filter_subsuite() and kunit_filter_suites(). Note: what the executor means by "subsuite" is the array of suites corresponding to each test file. This patch lightly refactors executor.c to avoid the use of global variables to make it testable. It also includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- v1 -> v2: * Fix missing free in kfree_subsuites_at_end() --- lib/kunit/executor.c | 26 ++++---- lib/kunit/executor_test.c | 133 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 148 insertions(+), 11 deletions(-) create mode 100644 lib/kunit/executor_test.c diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 15832ed44668..96a4ae983786 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -19,7 +19,7 @@ MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites run at boot-time, e.g. list*"); static struct kunit_suite * const * -kunit_filter_subsuite(struct kunit_suite * const * const subsuite) +kunit_filter_subsuite(struct kunit_suite * const * const subsuite, const char *filter_glob) { int i, n = 0; struct kunit_suite **filtered; @@ -52,19 +52,14 @@ struct suite_set { struct kunit_suite * const * const *end; }; -static struct suite_set kunit_filter_suites(void) +static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, + const char *filter_glob) { int i; struct kunit_suite * const **copy, * const *filtered_subsuite; struct suite_set filtered; - const size_t max = __kunit_suites_end - __kunit_suites_start; - - if (!filter_glob) { - filtered.start = __kunit_suites_start; - filtered.end = __kunit_suites_end; - return filtered; - } + const size_t max = suite_set->end - suite_set->start; copy = kmalloc_array(max, sizeof(*filtered.start), GFP_KERNEL); filtered.start = copy; @@ -74,7 +69,7 @@ static struct suite_set kunit_filter_suites(void) } for (i = 0; i < max; ++i) { - filtered_subsuite = kunit_filter_subsuite(__kunit_suites_start[i]); + filtered_subsuite = kunit_filter_subsuite(suite_set->start[i], filter_glob); if (filtered_subsuite) *copy++ = filtered_subsuite; } @@ -98,8 +93,13 @@ static void kunit_print_tap_header(struct suite_set *suite_set) int kunit_run_all_tests(void) { struct kunit_suite * const * const *suites; + struct suite_set suite_set = { + .start = __kunit_suites_start, + .end = __kunit_suites_end, + }; - struct suite_set suite_set = kunit_filter_suites(); + if (filter_glob) + suite_set = kunit_filter_suites(&suite_set, filter_glob); kunit_print_tap_header(&suite_set); @@ -115,4 +115,8 @@ int kunit_run_all_tests(void) return 0; } +#if IS_BUILTIN(CONFIG_KUNIT_TEST) +#include "executor_test.c" +#endif + #endif /* IS_BUILTIN(CONFIG_KUNIT) */ diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c new file mode 100644 index 000000000000..cdbe54b16501 --- /dev/null +++ b/lib/kunit/executor_test.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for the KUnit executor. + * + * Copyright (C) 2021, Google LLC. + * Author: Daniel Latypov <dlatypov(a)google.com> + */ + +#include <kunit/test.h> + +static void kfree_at_end(struct kunit *test, const void *to_free); +static struct kunit_suite *alloc_fake_suite(struct kunit *test, + const char *suite_name); + +static void filter_subsuite_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL, NULL}; + struct kunit_suite * const *filtered; + + subsuite[0] = alloc_fake_suite(test, "suite1"); + subsuite[1] = alloc_fake_suite(test, "suite2"); + + /* Want: suite1, suite2, NULL -> suite2, NULL */ + filtered = kunit_filter_subsuite(subsuite, "suite2*"); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered); + kfree_at_end(test, filtered); + + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered[0]); + KUNIT_EXPECT_STREQ(test, (const char *)filtered[0]->name, "suite2"); + + KUNIT_EXPECT_FALSE(test, filtered[1]); +} + +static void filter_subsuite_to_empty_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL, NULL}; + struct kunit_suite * const *filtered; + + subsuite[0] = alloc_fake_suite(test, "suite1"); + subsuite[1] = alloc_fake_suite(test, "suite2"); + + filtered = kunit_filter_subsuite(subsuite, "not_found"); + kfree_at_end(test, filtered); /* just in case */ + + KUNIT_EXPECT_FALSE_MSG(test, filtered, + "should be NULL to indicate no match"); +} + +static void kfree_subsuites_at_end(struct kunit *test, struct suite_set *suite_set) +{ + struct kunit_suite * const * const *suites; + + kfree_at_end(test, suite_set->start); + for (suites = suite_set->start; suites < suite_set->end; suites++) + kfree_at_end(test, *suites); +} + +static void filter_suites_test(struct kunit *test) +{ + /* Suites per-file are stored as a NULL terminated array */ + struct kunit_suite *subsuites[2][2] = { + {NULL, NULL}, + {NULL, NULL}, + }; + /* Match the memory layout of suite_set */ + struct kunit_suite * const * const suites[2] = { + subsuites[0], subsuites[1], + }; + + const struct suite_set suite_set = { + .start = suites, + .end = suites + 2, + }; + struct suite_set filtered = {.start = NULL, .end = NULL}; + + /* Emulate two files, each having one suite */ + subsuites[0][0] = alloc_fake_suite(test, "suite0"); + subsuites[1][0] = alloc_fake_suite(test, "suite1"); + + /* Filter out suite1 */ + filtered = kunit_filter_suites(&suite_set, "suite0"); + kfree_subsuites_at_end(test, &filtered); /* let us use ASSERTs without leaking */ + KUNIT_ASSERT_EQ(test, filtered.end - filtered.start, (ptrdiff_t)1); + + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered.start); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered.start[0]); + KUNIT_EXPECT_STREQ(test, (const char *)filtered.start[0][0]->name, "suite0"); +} + +static struct kunit_case executor_test_cases[] = { + KUNIT_CASE(filter_subsuite_test), + KUNIT_CASE(filter_subsuite_to_empty_test), + KUNIT_CASE(filter_suites_test), + {} +}; + +static struct kunit_suite executor_test_suite = { + .name = "kunit_executor_test", + .test_cases = executor_test_cases, +}; + +kunit_test_suites(&executor_test_suite); + +/* Test helpers */ + +static void kfree_res_free(struct kunit_resource *res) +{ + kfree(res->data); +} + +/* Use the resource API to register a call to kfree(to_free). + * Since we never actually use the resource, it's safe to use on const data. + */ +static void kfree_at_end(struct kunit *test, const void *to_free) +{ + /* kfree() handles NULL already, but avoid allocating a no-op cleanup. */ + if (IS_ERR_OR_NULL(to_free)) + return; + kunit_alloc_and_get_resource(test, NULL, kfree_res_free, GFP_KERNEL, + (void *)to_free); +} + +static struct kunit_suite *alloc_fake_suite(struct kunit *test, + const char *suite_name) +{ + struct kunit_suite *suite; + + /* We normally never expect to allocate suites, hence the non-const cast. */ + suite = kunit_kzalloc(test, sizeof(*suite), GFP_KERNEL); + strncpy((char *)suite->name, suite_name, sizeof(suite->name) - 1); + + return suite; +} base-commit: 151501160401e2dc669ea7dac2c599b53f220c33 -- 2.31.1.368.gbe11c130af-goog

4 years, 2 months

1
0
0 0

[PATCH 2/2] tools: do not include scripts/Kbuild.include

by Masahiro Yamada

Since commit d9f4ff50d2aa ("kbuild: spilt cc-option and friends to scripts/Makefile.compiler"), some kselftests fail to build. The tools/ directory opted out Kbuild, and went in a different direction. They copy any kind of files to the tools/ directory in order to do whatever they want to do in their world. tools/build/Build.include mimics scripts/Kbuild.include, but some tool Makefiles included the Kbuild one to import a feature that is missing in tools/build/Build.include: - Commit ec04aa3ae87b ("tools/thermal: tmon: use "-fstack-protector" only if supported") included scripts/Kbuild.include from tools/thermal/tmon/Makefile to import the cc-option macro. - Commit c2390f16fc5b ("selftests: kvm: fix for compilers that do not support -no-pie") included scripts/Kbuild.include from tools/testing/selftests/kvm/Makefile to import the try-run macro. - Commit 9cae4ace80ef ("selftests/bpf: do not ignore clang failures") included scripts/Kbuild.include from tools/testing/selftests/bpf/Makefile to import the .DELETE_ON_ERROR target. - Commit 0695f8bca93e ("selftests/powerpc: Handle Makefile for unrecognized option") included scripts/Kbuild.include from tools/testing/selftests/powerpc/pmu/ebb/Makefile to import the try-run macro. Copy what they want there, and stop including scripts/Kbuild.include from the tool Makefiles. Link: https://lore.kernel.org/lkml/86dadf33-70f7-a5ac-cb8c-64966d2f45a1@linux.ibm… Fixes: d9f4ff50d2aa ("kbuild: spilt cc-option and friends to scripts/Makefile.compiler") Reported-by: Janosch Frank <frankja(a)linux.ibm.com> Reported-by: Christian Borntraeger <borntraeger(a)de.ibm.com> Signed-off-by: Masahiro Yamada <masahiroy(a)kernel.org> --- tools/testing/selftests/bpf/Makefile | 3 ++- tools/testing/selftests/kvm/Makefile | 12 +++++++++++- .../selftests/powerpc/pmu/ebb/Makefile | 11 ++++++++++- tools/thermal/tmon/Makefile | 19 +++++++++++++++++-- 4 files changed, 40 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 044bfdcf5b74..d872b9f41543 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -1,5 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -include ../../../../scripts/Kbuild.include include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include @@ -476,3 +475,5 @@ EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature \ $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc bpf_testmod.ko) + +.DELETE_ON_ERROR: diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index a6d61f451f88..8b45bc417d83 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -1,5 +1,15 @@ # SPDX-License-Identifier: GPL-2.0-only -include ../../../../scripts/Kbuild.include + +TMPOUT = .tmp_$$$$ + +try-run = $(shell set -e; \ + TMP=$(TMPOUT)/tmp; \ + mkdir -p $(TMPOUT); \ + trap "rm -rf $(TMPOUT)" EXIT; \ + if ($(1)) >/dev/null 2>&1; \ + then echo "$(2)"; \ + else echo "$(3)"; \ + fi) all: diff --git a/tools/testing/selftests/powerpc/pmu/ebb/Makefile b/tools/testing/selftests/powerpc/pmu/ebb/Makefile index af3df79d8163..d5d3e869df93 100644 --- a/tools/testing/selftests/powerpc/pmu/ebb/Makefile +++ b/tools/testing/selftests/powerpc/pmu/ebb/Makefile @@ -1,5 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -include ../../../../../../scripts/Kbuild.include noarg: $(MAKE) -C ../../ @@ -8,6 +7,16 @@ noarg: CFLAGS += -m64 TMPOUT = $(OUTPUT)/TMPDIR/ + +try-run = $(shell set -e; \ + TMP=$(TMPOUT)/tmp; \ + mkdir -p $(TMPOUT); \ + trap "rm -rf $(TMPOUT)" EXIT; \ + if ($(1)) >/dev/null 2>&1; \ + then echo "$(2)"; \ + else echo "$(3)"; \ + fi) + # Toolchains may build PIE by default which breaks the assembly no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \ $(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -no-pie -x c - -o "$$TMP", -no-pie) diff --git a/tools/thermal/tmon/Makefile b/tools/thermal/tmon/Makefile index 59e417ec3e13..92a683e4866c 100644 --- a/tools/thermal/tmon/Makefile +++ b/tools/thermal/tmon/Makefile @@ -1,6 +1,21 @@ # SPDX-License-Identifier: GPL-2.0 -# We need this for the "cc-option" macro. -include ../../../scripts/Kbuild.include + +TMPOUT = .tmp_$$$$ + +try-run = $(shell set -e; \ + TMP=$(TMPOUT)/tmp; \ + mkdir -p $(TMPOUT); \ + trap "rm -rf $(TMPOUT)" EXIT; \ + if ($(1)) >/dev/null 2>&1; \ + then echo "$(2)"; \ + else echo "$(3)"; \ + fi) + +__cc-option = $(call try-run,\ + $(1) -Werror $(2) $(3) -c -x c /dev/null -o "$$TMP",$(3),$(4)) + +cc-option = $(call __cc-option, $(CC),\ + $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS),$(1),$(2)) VERSION = 1.0 -- 2.27.0

4 years, 2 months

3
6
0 0

[PATCH bpf-next 0/5] libbpf: Add TC-BPF API

by Kumar Kartikeya Dwivedi

This series adds support to libbpf for attaching SCHED_CLS and SCHED_ACT bpf programs to their respective tc attach points. Currently, a user needs to shell out to the tc command line for add, change, replace, and del operations, which is not ideal. Some of the features that have been omitted for the CLS API: * TCA_BPF_POLICE Support for adding police actions to filter has been omitted for now. * TCA_RATE Support for packet rate estimator has been omitted for now. * Attaching actions directly to the classifier This allows the attached actions to be bound to classifier and get auto detached when it is deleted. It translates to 'bind' refcount in the kernel internally. They run after a successful classification from the SCHED_CLS prog. Support for this can be added later, but has been omitted for now, primarily because direct-action mode provides a better alternative. A high level TC-BPF API is also provided, and currently only supports attach and destroy operations. These functions return a pointer to a bpf_link object. When falling back to the low level API, the link must be disconnected to take over its ownership. It can be released using bpf_link__destroy, which will also cause the filter/action to be detached if not disconnected. The individual commits contain a general API summary and examples. Kumar Kartikeya Dwivedi (5): tools pkt_cls.h: sync with kernel sources libbpf: add helpers for preparing netlink attributes libbpf: add low level TC-BPF API libbpf: add high level TC-BPF API libbpf: add selftests for TC-BPF API tools/include/uapi/linux/pkt_cls.h | 174 +++- tools/lib/bpf/libbpf.c | 110 ++- tools/lib/bpf/libbpf.h | 133 ++++ tools/lib/bpf/libbpf.map | 17 + tools/lib/bpf/netlink.c | 752 +++++++++++++++++- tools/lib/bpf/nlattr.h | 43 + .../selftests/bpf/prog_tests/test_tc_bpf.c | 261 ++++++ .../selftests/bpf/progs/test_tc_bpf_kern.c | 18 + 8 files changed, 1476 insertions(+), 32 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_tc_bpf.c create mode 100644 tools/testing/selftests/bpf/progs/test_tc_bpf_kern.c -- 2.30.2

4 years, 2 months

6
54
0 0

[PATCH v4] Documentation: kunit: add tips for running KUnit

by Daniel Latypov

This is long overdue. There are several things that aren't nailed down (in-tree .kunitconfig's), or partially broken (GCOV on UML), but having them documented, warts and all, is better than having nothing. This covers a bunch of the more recent features * kunit_filter_glob * kunit.py run --kunitconfig * slightly more detail on building tests as modules * CONFIG_KUNIT_DEBUGFS By my count, the only headline features now not mentioned are the KASAN integration and KernelCI json output support (kunit.py run --json). And then it also discusses how to get code coverage reports under UML and non-UML since this is a question people have repeatedly asked. Non-UML coverage collection is no different from normal, but we should probably explicitly call this out. As for UML, I was able to get it working again with two small hacks.* E.g. with CONFIG_KUNIT=y && CONFIG_KUNIT_ALL_TESTS=y Overall coverage rate: lines......: 15.1% (18294 of 120776 lines) functions..: 16.8% (1860 of 11050 functions) Note: this doesn't document --alltests since this is not stable yet. Hopefully being run more frequently as part of KernelCI will help... *Using gcc/gcov-6 and not using uml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhiggins(a)google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- v3 -> v4: * update instructions on how to remove uml_abort() call v2 -> v3: * Suggest --make_options=CC=/usr/bin/gcc-6 instead of manually editing kunit_kernel.py v1 -> v2: Fix typos, drop --alltests, changed wordiing on config fragments. --- Documentation/dev-tools/kunit/index.rst | 1 + .../dev-tools/kunit/running_tips.rst | 259 ++++++++++++++++++ Documentation/dev-tools/kunit/start.rst | 2 + 3 files changed, 262 insertions(+) create mode 100644 Documentation/dev-tools/kunit/running_tips.rst diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index 848478838347..7f7cf8d2ab20 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -14,6 +14,7 @@ KUnit - Unit Testing for the Linux Kernel style faq tips + running_tips What is KUnit? ============== diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst new file mode 100644 index 000000000000..7d99386cf94a --- /dev/null +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -0,0 +1,259 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================ +Tips For Running KUnit Tests +============================ + +Using ``kunit.py run`` ("kunit tool") +===================================== + +Running from any directory +-------------------------- + +It can be handy to create a bash function like: + +.. code-block:: bash + + function run_kunit() { + ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run $@ ) + } + +.. note:: + Early versions of ``kunit.py`` (before 5.6) didn't work unless run from + the kernel root, hence the use of a subshell and ``cd``. + +Running a subset of tests +------------------------- + +``kunit.py run`` accepts an optional glob argument to filter tests. Currently +this only matches against suite names, but this may change in the future. + +Say that we wanted to run the sysctl tests, we could do so via: + +.. code-block:: bash + + $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig + $ ./tools/testing/kunit/kunit.py run 'sysctl*' + +We're paying the cost of building more tests than we need this way, but it's +easier than fiddling with ``.kunitconfig`` files or commenting out +``kunit_suite``'s. + +However, if we wanted to define a set of tests in a less ad hoc way, the next +tip is useful. + +Defining a set of tests +----------------------- + +``kunit.py run`` (along with ``build``, and ``config``) supports a +``--kunitconfig`` flag. So if you have a set of tests that you want to run on a +regular basis (especially if they have other dependencies), you can create a +specific ``.kunitconfig`` for them. + +E.g. kunit has one for its tests: + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit/.kunitconfig + +Alternatively, if you're following the convention of naming your +file ``.kunitconfig``, you can just pass in the dir, e.g. + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit + +.. note:: + This is a relatively new feature (5.12+) so we don't have any + conventions yet about on what files should be checked in versus just + kept around locally. It's up to you and your maintainer to decide if a + config is useful enough to submit (and therefore have to maintain). + +.. note:: + Having ``.kunitconfig`` fragments in a parent and child directory is + iffy. There's discussion about adding an "import" statement in these + files to make it possible to have a top-level config run tests from all + child directories. But that would mean ``.kunitconfig`` files are no + longer just simple .config fragments. + + One alternative would be to have kunit tool recursively combine configs + automagically, but tests could theoretically depend on incompatible + options, so handling that would be tricky. + +Generating code coverage reports under UML +------------------------------------------ + +.. note:: + TODO(brendanhiggins(a)google.com): There are various issues with UML and + versions of gcc 7 and up. You're likely to run into missing ``.gcda`` + files or compile errors. We know one `faulty GCC commit + <https://github.com/gcc-mirror/gcc/commit/8c9434c2f9358b8b8bad2c1990edf10a21…>`_ + but not how we'd go about getting this fixed. The compile errors still + need some investigation. + +.. note:: + TODO(brendanhiggins(a)google.com): for recent versions of Linux + (5.10-5.12, maybe earlier), there's a bug with gcov counters not being + flushed in UML. This translates to very low (<1%) reported coverage. This is + related to the above issue and can be worked around by replacing the + one call to ``uml_abort()`` (it's in ``os_dump_core()``) with a plain + ``exit()``. + + +This is different from the "normal" way of getting coverage information that is +documented in Documentation/dev-tools/gcov.rst. + +Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options: + +.. code-block:: none + + CONFIG_DEBUG_KERNEL=y + CONFIG_DEBUG_INFO=y + CONFIG_GCOV=y + + +Putting it together into a copy-pastable sequence of commands: + +.. code-block:: bash + + # Append coverage options to the current config + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ ./tools/testing/kunit/kunit.py run + # Extract the coverage information from the build dir (.kunit/) + $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ + + # From here on, it's the same process as with CONFIG_GCOV_KERNEL=y + # E.g. can generate an HTML report in a tmp dir like so: + $ genhtml -o /tmp/coverage_html coverage.info + + +If your installed version of gcc doesn't work, you can tweak the steps: + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 + $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6 + + +Running tests manually +====================== + +Running tests without using ``kunit.py run`` is also an important use case. +Currently it's your only option if you want to test on architectures other than +UML. + +As running the tests under UML is fairly straightforward (configure and compile +the kernel, run the ``./linux`` binary), this section will focus on testing +non-UML architectures. + + +Running built-in tests +---------------------- + +When setting tests to ``=y``, the tests will run as part of boot and print +results to dmesg in TAP format. So you just need to add your tests to your +``.config``, build and boot your kernel as normal. + +So if we compiled our kernel with: + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=y + +Then we'd see output like this in dmesg signaling the test ran and passed: + +.. code-block:: none + + TAP version 14 + 1..1 + # Subtest: example + 1..1 + # example_simple_test: initializing + ok 1 - example_simple_test + ok 1 - example + +Running tests as modules +------------------------ + +Depending on the tests, you can build them as loadable modules. + +For example, we'd change the config options from before to + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=m + +Then after booting into our kernel, we can run the test via + +.. code-block:: none + + $ modprobe kunit-example-test + +This will then cause it to print TAP output to stdout. + +.. note:: + The ``modprobe`` will *not* have a non-zero exit code if any test + failed (as of 5.13). But ``kunit.py parse`` would, see below. + +.. note:: + You can set ``CONFIG_KUNIT=m`` as well, however, some features will not + work and thus some tests might break. Ideally tests would specify they + depend on ``KUNIT=y`` in their ``Kconfig``'s, but this is an edge case + most test authors won't think about. + As of 5.13, the only difference is that ``current->kunit_test`` will + not exist. + +Pretty-printing results +----------------------- + +You can use ``kunit.py parse`` to parse dmesg for test output and print out +results in the same familiar format that ``kunit.py run`` does. + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py parse /var/log/dmesg + + +Retrieving per suite results +---------------------------- + +Regardless of how you're running your tests, you can enable +``CONFIG_KUNIT_DEBUGFS`` to expose per-suite TAP-formatted results: + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=m + CONFIG_KUNIT_DEBUGFS=y + +The results for each suite will be exposed under +``/sys/kernel/debug/kunit/<suite>/results``. +So using our example config: + +.. code-block:: bash + + $ modprobe kunit-example-test > /dev/null + $ cat /sys/kernel/debug/kunit/example/results + ... <TAP output> ... + + # After removing the module, the corresponding files will go away + $ modprobe -r kunit-example-test + $ cat /sys/kernel/debug/kunit/example/results + /sys/kernel/debug/kunit/example/results: No such file or directory + +Generating code coverage reports +-------------------------------- + +See Documentation/dev-tools/gcov.rst for details on how to do this. + +The only vaguely KUnit-specific advice here is that you probably want to build +your tests as modules. That way you can isolate the coverage from tests from +other code executed during boot, e.g. + +.. code-block:: bash + + # Reset coverage counters before running the test. + $ echo 0 > /sys/kernel/debug/gcov/reset + $ modprobe kunit-example-test diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 0e65cabe08eb..aa56d7ca6bfb 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -236,5 +236,7 @@ Next Steps ========== * Check out the :doc:`tips` page for tips on writing idiomatic KUnit tests. +* Check out the :doc:`running_tips` page for tips on + how to make running KUnit tests easier. * Optional: see the :doc:`usage` page for a more in-depth explanation of KUnit. base-commit: de2fcb3e62013738f22bbb42cbd757d9a242574e -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

2
1
0 0

[PATCH 0/9] userfaultfd: add minor fault handling for shmem

by Axel Rasmussen

Base ==== Since the original series [1] was merged into Andrew's tree, some issues were noticed. Up to this point, we had been working on fixing what's in Andrew's tree [2], but at this point we've changed direction enough that a lot of the fix's delta is undoing what was done in the original series, thereby making it hard to review. As suggested by Hugh Dickins and Peter Xu, this series takes a step back. It can be considered a v3 of the original series [1] - it combines those patches with the fixes, reordered / broken up to allow for easier review. The idea is that it will apply cleanly to akpm's tree, *replacing* the following patches (i.e., drop these first, and then apply this series): userfaultfd-support-minor-fault-handling-for-shmem.patch userfaultfd-support-minor-fault-handling-for-shmem-fix.patch userfaultfd-support-minor-fault-handling-for-shmem-fix-2.patch userfaultfd-support-minor-fault-handling-for-shmem-fix-3.patch userfaultfd-support-minor-fault-handling-for-shmem-fix-4.patch userfaultfd-selftests-use-memfd_create-for-shmem-test-type.patch userfaultfd-selftests-create-alias-mappings-in-the-shmem-test.patch userfaultfd-selftests-reinitialize-test-context-in-each-test.patch userfaultfd-selftests-exercise-minor-fault-handling-shmem-support.patch Changelog ========= Changes since the most recent fixup patch [2]: - Squash the fixes ([2]) in with the original series ([1]). This makes reviewing easier, as we no longer have to sift through deltas undoing what we had done before. [Hugh, Peter] - Modify shmem_mcopy_atomic_pte() to use the new mcopy_atomic_install_ptes() helper, reducing code duplication. [Hugh] - Properly trigger handle_userfault() in the shmem_swapin_page() case. [Hugh] - Use shmem_getpage() instead of find_lock_page() to lookup the existing page in for continue. This properly deals with swapped-out pages. [Hugh] - Unconditionally pte_mkdirty() for anon memory (as before). [Peter] - Don't include userfaultfd_k.h in either hugetlb.h or shmem_fs.h. [Hugh] - Add comment for UFFD_FEATURE_MINOR_SHMEM (to match _HUGETLBFS). [Hugh] - Fix some small cleanup issues (parens, reworded conditionals, reduced plumbing of some parameters, simplify labels/gotos, ...). [Hugh, Peter] Overview ======== See the series which added minor faults for hugetlbfs [3] for a detailed overview of minor fault handling in general. This series adds the same support for shmem-backed areas. This series is structured as follows: - Commits 1 and 2 are cleanups. - Commits 3 and 4 implement the new feature (minor fault handling for shmem). - Commits 5, 6, 7, 8 update the userfaultfd selftest to exercise the feature. - Commit 9 is one final cleanup, modifying an existing code path to re-use a new helper we've introduced. We rely on the selftest to show that this change doesn't break anything. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra <lokeshgidra(a)google.com>) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." [1] https://lore.kernel.org/patchwork/cover/1388144/ [2] https://lore.kernel.org/patchwork/patch/1408161/ [3] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen… Axel Rasmussen (9): userfaultfd/hugetlbfs: avoid including userfaultfd_k.h in hugetlb.h userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte userfaultfd/shmem: support minor fault registration for shmem userfaultfd/shmem: support UFFDIO_CONTINUE for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support userfaultfd/shmem: modify shmem_mcopy_atomic_pte to use install_ptes fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 5 +- include/linux/shmem_fs.h | 15 +- include/linux/userfaultfd_k.h | 5 + include/uapi/linux/userfaultfd.h | 7 +- mm/hugetlb.c | 1 + mm/memory.c | 8 +- mm/shmem.c | 122 ++++------ mm/userfaultfd.c | 183 ++++++++++----- tools/testing/selftests/vm/userfaultfd.c | 280 +++++++++++++++-------- 10 files changed, 387 insertions(+), 245 deletions(-) -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

4
20
0 0

[PATCH] selftests: fix rp_filter testcase failure bug

by Qiao Ma

The rp_filter testcase is used to test whether local packets redirected from dummy1 to lo could pass the checking of rp_filter. In fact, the packets passed the checking, but the testing process cannot receive any reply packets, leading to test failure. The reason is that the device dummy1 lacks ip address, caused the incorrect routing of reply packets. This patch adds ip address for dummy1 device. Signed-off-by: Qiao Ma <mqaio(a)linux.alibaba.com> --- tools/testing/selftests/net/fib_tests.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh index 2b5707738609..9a843ca0b913 100755 --- a/tools/testing/selftests/net/fib_tests.sh +++ b/tools/testing/selftests/net/fib_tests.sh @@ -448,6 +448,7 @@ fib_rp_filter_test() $IP link set dummy0 address 52:54:00:6a:c7:5e $IP link add dummy1 type dummy $IP link set dummy1 address 52:54:00:6a:c7:5e + $IP address add 198.51.101.1/24 dev dummy1 $IP link set dev dummy1 up $NS_EXEC sysctl -qw net.ipv4.conf.all.rp_filter=1 $NS_EXEC sysctl -qw net.ipv4.conf.all.accept_local=1 -- 2.18.2

4 years, 2 months

1
0
0 0

[PATCH v4 1/3] kunit: make test->lock irq safe

by glittao＠gmail.com

From: Vlastimil Babka <vbabka(a)suse.cz> The upcoming SLUB kunit test will be calling kunit_find_named_resource() from a context with disabled interrupts. That means kunit's test->lock needs to be IRQ safe to avoid potential deadlocks and lockdep splats. This patch therefore changes the test->lock usage to spin_lock_irqsave() and spin_unlock_irqrestore(). Signed-off-by: Vlastimil Babka <vbabka(a)suse.cz> Signed-off-by: Oliver Glitta <glittao(a)gmail.com> --- include/kunit/test.h | 5 +++-- lib/kunit/test.c | 18 +++++++++++------- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 49601c4b98b8..524d4789af22 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -515,8 +515,9 @@ kunit_find_resource(struct kunit *test, void *match_data) { struct kunit_resource *res, *found = NULL; + unsigned long flags; - spin_lock(&test->lock); + spin_lock_irqsave(&test->lock, flags); list_for_each_entry_reverse(res, &test->resources, node) { if (match(test, res, (void *)match_data)) { @@ -526,7 +527,7 @@ kunit_find_resource(struct kunit *test, } } - spin_unlock(&test->lock); + spin_unlock_irqrestore(&test->lock, flags); return found; } diff --git a/lib/kunit/test.c b/lib/kunit/test.c index ec9494e914ef..2c62eeb45b82 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -442,6 +442,7 @@ int kunit_add_resource(struct kunit *test, void *data) { int ret = 0; + unsigned long flags; res->free = free; kref_init(&res->refcount); @@ -454,10 +455,10 @@ int kunit_add_resource(struct kunit *test, res->data = data; } - spin_lock(&test->lock); + spin_lock_irqsave(&test->lock, flags); list_add_tail(&res->node, &test->resources); /* refcount for list is established by kref_init() */ - spin_unlock(&test->lock); + spin_unlock_irqrestore(&test->lock, flags); return ret; } @@ -515,9 +516,11 @@ EXPORT_SYMBOL_GPL(kunit_alloc_and_get_resource); void kunit_remove_resource(struct kunit *test, struct kunit_resource *res) { - spin_lock(&test->lock); + unsigned long flags; + + spin_lock_irqsave(&test->lock, flags); list_del(&res->node); - spin_unlock(&test->lock); + spin_unlock_irqrestore(&test->lock, flags); kunit_put_resource(res); } EXPORT_SYMBOL_GPL(kunit_remove_resource); @@ -597,6 +600,7 @@ EXPORT_SYMBOL_GPL(kunit_kfree); void kunit_cleanup(struct kunit *test) { struct kunit_resource *res; + unsigned long flags; /* * test->resources is a stack - each allocation must be freed in the @@ -608,9 +612,9 @@ void kunit_cleanup(struct kunit *test) * protect against the current node being deleted, not the next. */ while (true) { - spin_lock(&test->lock); + spin_lock_irqsave(&test->lock, flags); if (list_empty(&test->resources)) { - spin_unlock(&test->lock); + spin_unlock_irqrestore(&test->lock, flags); break; } res = list_last_entry(&test->resources, @@ -621,7 +625,7 @@ void kunit_cleanup(struct kunit *test) * resource, and this can't happen if the test->lock * is held. */ - spin_unlock(&test->lock); + spin_unlock_irqrestore(&test->lock, flags); kunit_remove_resource(test, res); } #if (IS_ENABLED(CONFIG_KASAN) && IS_ENABLED(CONFIG_KUNIT)) -- 2.31.1.272.g89b43f80a5

4 years, 2 months

6
10
0 0

[PATCH] tools/testing: Remove unused variable

by zuoqilin1＠163.com

From: zuoqilin <zuoqilin(a)yulong.com> Remove unused variable "ret2". Signed-off-by: zuoqilin <zuoqilin(a)yulong.com> --- tools/testing/selftests/bpf/progs/test_tunnel_kern.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/progs/test_tunnel_kern.c b/tools/testing/selftests/bpf/progs/test_tunnel_kern.c index ba6eadf..e7b6731 100644 --- a/tools/testing/selftests/bpf/progs/test_tunnel_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tunnel_kern.c @@ -396,7 +396,7 @@ int _ip6vxlan_get_tunnel(struct __sk_buff *skb) SEC("geneve_set_tunnel") int _geneve_set_tunnel(struct __sk_buff *skb) { - int ret, ret2; + int ret; struct bpf_tunnel_key key; struct geneve_opt gopt; -- 1.9.1

4 years, 2 months

2
1
0 0

[PATCH v2] Documentation: dev-tools: Add Testing Overview

by David Gow

The kernel now has a number of testing and debugging tools, and we've seen a bit of confusion about what the differences between them are. Add a basic documentation outlining the testing tools, when to use each, and how they interact. This is a pretty quick overview rather than the idealised "kernel testing guide" that'd probably be optimal, but given the number of times questions like "When do you use KUnit and when do you use Kselftest?" are being asked, it seemed worth at least having something. Hopefully this can form the basis for more detailed documentation later. Signed-off-by: David Gow <davidgow(a)google.com> --- Thanks, everyone, for the comments on the doc. I've made a few of the suggested changes. Please let me know what you think! -- David Changes since v1: https://lore.kernel.org/linux-kselftest/20210410070529.4113432-1-davidgow@g… - Note KUnit's speed and that one should provide selftests for syscalls - Mention lockdep as a Dynamic Analysis Tool - Refer to "Dynamic Analysis Tools" instead of "Sanitizers" - A number of minor formatting tweaks and rewordings for clarity. Not changed: - I haven't included an exhaustive list of differences, advantages, etc, between KUnit and kselftest: for now, the doc continues to focus on the difference between 'in-kernel' and 'userspace' testing here. - Similarly, I'm not linking out to docs defining and describing "Unit" tests versus "End-to-end" tests. None of the existing documentation elsewhere quite matches what we do in the kernel perfectly, so it seems less confusing to focus on the 'in-kernel'/'userspace' distinction, and leave other definitions as a passing mention for those who are already familiar with the concepts. - I haven't linked to any talk videos here: a few of them are linked on (e.g.) the KUnit webpage, but I wanted to keep the Kernel documentation more self-contained for now. No objection to adding them in a follow-up patch if people feel strongly about it, though. - The link from index.rst to this doc is unchanged. I personally think that the link is prominent enough there: it's the first link, and shows up a few times. One possibility if people disagreed would be to merge this page with the index, but given not all dev-tools are going to be testing-related, it seemed a bit arrogant. :-) Documentation/dev-tools/index.rst | 3 + Documentation/dev-tools/testing-overview.rst | 117 +++++++++++++++++++ 2 files changed, 120 insertions(+) create mode 100644 Documentation/dev-tools/testing-overview.rst diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 1b1cf4f5c9d9..f590e5860794 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -7,6 +7,8 @@ be used to work on the kernel. For now, the documents have been pulled together without any significant effort to integrate them into a coherent whole; patches welcome! +A brief overview of testing-specific tools can be found in :doc:`testing-overview`. + .. class:: toc-title Table of contents @@ -14,6 +16,7 @@ whole; patches welcome! .. toctree:: :maxdepth: 2 + testing-overview coccinelle sparse kcov diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst new file mode 100644 index 000000000000..ce36a8cdf6b5 --- /dev/null +++ b/Documentation/dev-tools/testing-overview.rst @@ -0,0 +1,117 @@ +.. SPDX-License-Identifier: GPL-2.0 + +==================== +Kernel Testing Guide +==================== + + +There are a number of different tools for testing the Linux kernel, so knowing +when to use each of them can be a challenge. This document provides a rough +overview of their differences, and how they fit together. + + +Writing and Running Tests +========================= + +The bulk of kernel tests are written using either the kselftest or KUnit +frameworks. These both provide infrastructure to help make running tests and +groups of tests easier, as well as providing helpers to aid in writing new +tests. + +If you're looking to verify the behaviour of the Kernel — particularly specific +parts of the kernel — then you'll want to use KUnit or kselftest. + + +The Difference Between KUnit and kselftest +------------------------------------------ + +KUnit (Documentation/dev-tools/kunit/index.rst) is an entirely in-kernel system +for "white box" testing: because test code is part of the kernel, it can access +internal structures and functions which aren't exposed to userspace. + +KUnit tests therefore are best written against small, self-contained parts +of the kernel, which can be tested in isolation. This aligns well with the +concept of 'unit' testing. + +For example, a KUnit test might test an individual kernel function (or even a +single codepath through a function, such as an error handling case), rather +than a feature as a whole. + +This also makes KUnit tests very fast to build and run, allowing them to be +run frequently as part of the development process. + +There is a KUnit test style guide which may give further pointers in +Documentation/dev-tools/kunit/style.rst + + +kselftest (Documentation/dev-tools/kselftest.rst), on the other hand, is +largely implemented in userspace, and tests are normal userspace scripts or +programs. + +This makes it easier to write more complicated tests, or tests which need to +manipulate the overall system state more (e.g., spawning processes, etc.). +However, it's not possible to call kernel functions directly from kselftest. +This means that only kernel functionality which is exposed to userspace somhow +(e.g. by a syscall, device, filesystem, etc.) can be tested with kselftest. To +work around this, some tests include a companion kernel module which exposes +more information or functionality. If a test runs mostly or entirely within the +kernel, however, KUnit may be the more appropriate tool. + +kselftest is therefore suited well to tests of whole features, as these will +expose an interface to userspace, which can be tested, but not implementation +details. This aligns well with 'system' or 'end-to-end' testing. + +For example, all new system calls should be accompanied by kselftest tests. + +Code Coverage Tools +=================== + +The Linux Kernel supports two different code coverage measurement tools. These +can be used to verify that a test is executing particular functions or lines +of code. This is useful for determining how much of the kernel is being tested, +and for finding corner-cases which are not covered by the appropriate test. + +:doc:`gcov` is GCC's coverage testing tool, which can be used with the kernel +to get global or per-module coverage. Unlike KCOV, it does not record per-task +coverage. Coverage data can be read from debugfs, and interpreted using the +usual gcov tooling. + +:doc:`kcov` is a feature which can be built in to the kernel to allow +capturing coverage on a per-task level. It's therefore useful for fuzzing and +other situations where information about code executed during, for example, a +single syscall is useful. + + +Dynamic Analysis Tools +====================== + +The kernel also supports a number of dynamic analysis tools, which attempt to +detect classes of issues when the occur in a running kernel. These typically +look for undefined behaviour of some kind, such as invalid memory accesses, +concurrency issues such as data races, or other undefined behaviour like +integer overflows. + +Some of these tools are listed below: + +* kmemleak detects possible memory leaks. See + Documentation/dev-tools/kmemleak.rst +* KASAN detects invalid memory accesses such as out-of-bounds and + use-after-free errors. See Documentation/dev-tools/kasan.rst +* UBSAN detects behaviour that is undefined by the C standard, like integer + overflows. See Documentation/dev-tools/ubsan.rst +* KCSAN detects data races. See Documentation/dev-tools/kcsan.rst +* KFENCE is a low-overhead detector of memory issues, which is much faster than + KASAN and can be used in production. See Documentation/dev-tools/kfence.rst +* lockdep is a locking correctness validator. See + Documentation/locking/lockdep-design.rst +* There are several other pieces of debug instrumentation in the kernel, many + of which can be found in lib/Kconfig.debug + +These tools tend to test the kernel as a whole, and do not "pass" like +kselftest or KUnit tests. They can be combined with KUnit or kselftest by +running tests on a kernel with a sanitizer enabled: you can then be sure +that none of these errors are occurring during the test. + +Some of these tools integrate with KUnit or kselftest and will +automatically fail tests if an issue is detected. + -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

3
3
0 0

[PATCH v3] Documentation: kunit: add tips for running KUnit

by Daniel Latypov

This is long overdue. There are several things that aren't nailed down (in-tree .kunitconfig's), or partially broken (GCOV on UML), but having them documented, warts and all, is better than having nothing. This covers a bunch of the more recent features * kunit_filter_glob * kunit.py run --kunitconfig * slightly more detail on building tests as modules * CONFIG_KUNIT_DEBUGFS By my count, the only headline features now not mentioned are the KASAN integration and KernelCI json output support (kunit.py run --json). And then it also discusses how to get code coverage reports under UML and non-UML since this is a question people have repeatedly asked. Non-UML coverage collection is no different from normal, but we should probably explicitly call this out. As for UML, I was able to get it working again with two small hacks.* E.g. with CONFIG_KUNIT=y && CONFIG_KUNIT_ALL_TESTS=y Overall coverage rate: lines......: 15.1% (18294 of 120776 lines) functions..: 16.8% (1860 of 11050 functions) Note: this doesn't document --alltests since this is not stable yet. Hopefully being run more frequently as part of KernelCI will help... *Using gcc/gcov-6 and not using uml_abort() in os_dump_core(). I've documented these hacks in "Notes" but left TODOs for brendanhiggins(a)google.com who tracked down the runtime issue in GCC. To be clear: these are not issues specific to KUnit, but rather to UML. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- v2 -> v3: * Suggest --make_options=CC=/usr/bin/gcc-6 instead of manually editing kunit_kernel.py * update instructions on how to remove uml_abort() call v1 -> v2: Fix typos, drop --alltests, changed wordiing on config fragments. --- Documentation/dev-tools/kunit/index.rst | 1 + .../dev-tools/kunit/running_tips.rst | 258 ++++++++++++++++++ Documentation/dev-tools/kunit/start.rst | 2 + 3 files changed, 261 insertions(+) create mode 100644 Documentation/dev-tools/kunit/running_tips.rst diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index 848478838347..7f7cf8d2ab20 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -14,6 +14,7 @@ KUnit - Unit Testing for the Linux Kernel style faq tips + running_tips What is KUnit? ============== diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst new file mode 100644 index 000000000000..e2e9af711d1b --- /dev/null +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -0,0 +1,258 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================ +Tips For Running KUnit Tests +============================ + +Using ``kunit.py run`` ("kunit tool") +===================================== + +Running from any directory +-------------------------- + +It can be handy to create a bash function like: + +.. code-block:: bash + + function run_kunit() { + ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run $@ ) + } + +.. note:: + Early versions of ``kunit.py`` (before 5.6) didn't work unless run from + the kernel root, hence the use of a subshell and ``cd``. + +Running a subset of tests +------------------------- + +``kunit.py run`` accepts an optional glob argument to filter tests. Currently +this only matches against suite names, but this may change in the future. + +Say that we wanted to run the sysctl tests, we could do so via: + +.. code-block:: bash + + $ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig + $ ./tools/testing/kunit/kunit.py run 'sysctl*' + +We're paying the cost of building more tests than we need this way, but it's +easier than fiddling with ``.kunitconfig`` files or commenting out +``kunit_suite``'s. + +However, if we wanted to define a set of tests in a less ad hoc way, the next +tip is useful. + +Defining a set of tests +----------------------- + +``kunit.py run`` (along with ``build``, and ``config``) supports a +``--kunitconfig`` flag. So if you have a set of tests that you want to run on a +regular basis (especially if they have other dependencies), you can create a +specific ``.kunitconfig`` for them. + +E.g. kunit has one for its tests: + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit/.kunitconfig + +Alternatively, if you're following the convention of naming your +file ``.kunitconfig``, you can just pass in the dir, e.g. + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit + +.. note:: + This is a relatively new feature (5.12+) so we don't have any + conventions yet about on what files should be checked in versus just + kept around locally. It's up to you and your maintainer to decide if a + config is useful enough to submit (and therefore have to maintain). + +.. note:: + Having ``.kunitconfig`` fragments in a parent and child directory is + iffy. There's discussion about adding an "import" statement in these + files to make it possible to have a top-level config run tests from all + child directories. But that would mean ``.kunitconfig`` files are no + longer just simple .config fragments. + + One alternative would be to have kunit tool recursively combine configs + automagically, but tests could theoretically depend on incompatible + options, so handling that would be tricky. + +Generating code coverage reports under UML +------------------------------------------ + +.. note:: + TODO(brendanhiggins(a)google.com): There are various issues with UML and + versions of gcc 7 and up. You're likely to run into missing ``.gcda`` + files or compile errors. We know one `faulty GCC commit + <https://github.com/gcc-mirror/gcc/commit/8c9434c2f9358b8b8bad2c1990edf10a21…>`_ + but not how we'd go about getting this fixed. The compile errors still + need some investigation. + +.. note:: + TODO(brendanhiggins(a)google.com): for recent versions of Linux + (5.10-5.12, maybe earlier), there's a bug with gcov counters not being + flushed in UML. This translates to very low (<1%) reported coverage. This is + related to the above issue and can be worked around by replacing the + one call to ``uml_abort()`` with a plain ``exit()``. + + +This is different from the "normal" way of getting coverage information that is +documented in Documentation/dev-tools/gcov.rst. + +Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options: + +.. code-block:: none + + CONFIG_DEBUG_KERNEL=y + CONFIG_DEBUG_INFO=y + CONFIG_GCOV=y + + +Putting it together into a copy-pastable sequence of commands: + +.. code-block:: bash + + # Append coverage options to the current config + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ ./tools/testing/kunit/kunit.py run + # Extract the coverage information from the build dir (.kunit/) + $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ + + # From here on, it's the same process as with CONFIG_GCOV_KERNEL=y + # E.g. can generate an HTML report in a tmp dir like so: + $ genhtml -o /tmp/coverage_html coverage.info + + +If your installed version of gcc doesn't work, you can tweak the steps: + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 + $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6 + + +Running tests manually +====================== + +Running tests without using ``kunit.py run`` is also an important use case. +Currently it's your only option if you want to test on architectures other than +UML. + +As running the tests under UML is fairly straightforward (configure and compile +the kernel, run the ``./linux`` binary), this section will focus on testing +non-UML architectures. + + +Running built-in tests +---------------------- + +When setting tests to ``=y``, the tests will run as part of boot and print +results to dmesg in TAP format. So you just need to add your tests to your +``.config``, build and boot your kernel as normal. + +So if we compiled our kernel with: + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=y + +Then we'd see output like this in dmesg signaling the test ran and passed: + +.. code-block:: none + + TAP version 14 + 1..1 + # Subtest: example + 1..1 + # example_simple_test: initializing + ok 1 - example_simple_test + ok 1 - example + +Running tests as modules +------------------------ + +Depending on the tests, you can build them as loadable modules. + +For example, we'd change the config options from before to + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=m + +Then after booting into our kernel, we can run the test via + +.. code-block:: none + + $ modprobe kunit-example-test + +This will then cause it to print TAP output to stdout. + +.. note:: + The ``modprobe`` will *not* have a non-zero exit code if any test + failed (as of 5.13). But ``kunit.py parse`` would, see below. + +.. note:: + You can set ``CONFIG_KUNIT=m`` as well, however, some features will not + work and thus some tests might break. Ideally tests would specify they + depend on ``KUNIT=y`` in their ``Kconfig``'s, but this is an edge case + most test authors won't think about. + As of 5.13, the only difference is that ``current->kunit_test`` will + not exist. + +Pretty-printing results +----------------------- + +You can use ``kunit.py parse`` to parse dmesg for test output and print out +results in the same familiar format that ``kunit.py run`` does. + +.. code-block:: bash + + $ ./tools/testing/kunit/kunit.py parse /var/log/dmesg + + +Retrieving per suite results +---------------------------- + +Regardless of how you're running your tests, you can enable +``CONFIG_KUNIT_DEBUGFS`` to expose per-suite TAP-formatted results: + +.. code-block:: none + + CONFIG_KUNIT=y + CONFIG_KUNIT_EXAMPLE_TEST=m + CONFIG_KUNIT_DEBUGFS=y + +The results for each suite will be exposed under +``/sys/kernel/debug/kunit/<suite>/results``. +So using our example config: + +.. code-block:: bash + + $ modprobe kunit-example-test > /dev/null + $ cat /sys/kernel/debug/kunit/example/results + ... <TAP output> ... + + # After removing the module, the corresponding files will go away + $ modprobe -r kunit-example-test + $ cat /sys/kernel/debug/kunit/example/results + /sys/kernel/debug/kunit/example/results: No such file or directory + +Generating code coverage reports +-------------------------------- + +See Documentation/dev-tools/gcov.rst for details on how to do this. + +The only vaguely KUnit-specific advice here is that you probably want to build +your tests as modules. That way you can isolate the coverage from tests from +other code executed during boot, e.g. + +.. code-block:: bash + + # Reset coverage counters before running the test. + $ echo 0 > /sys/kernel/debug/gcov/reset + $ modprobe kunit-example-test diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 0e65cabe08eb..aa56d7ca6bfb 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -236,5 +236,7 @@ Next Steps ========== * Check out the :doc:`tips` page for tips on writing idiomatic KUnit tests. +* Check out the :doc:`running_tips` page for tips on + how to make running KUnit tests easier. * Optional: see the :doc:`usage` page for a more in-depth explanation of KUnit. base-commit: de2fcb3e62013738f22bbb42cbd757d9a242574e -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

2
2
0 0

[PATCH v2] selftests/mincore: get readahead size for check_file_mmap()

by Jeffle Xu

The readahead size used to be 2MB, thus it's reasonable to set the file size as 4MB when checking check_file_mmap(). However since commit c2e4cd57cfa1 ("block: lift setting the readahead size into the block layer"), readahead size could be as large as twice the io_opt, and thus the hardcoded file size no longer works. check_file_mmap() may report "Read-ahead pages reached the end of the file" when the readahead size actually exceeds the file size in this case. Signed-off-by: Jeffle Xu <jefflexu(a)linux.alibaba.com> --- chnages since v1: - add the test name "mincore" in the subject line - add the error message in commit message - rename @filesize to @file_size to keep a more consistent naming convention --- .../selftests/mincore/mincore_selftest.c | 57 +++++++++++++++---- 1 file changed, 47 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/mincore/mincore_selftest.c b/tools/testing/selftests/mincore/mincore_selftest.c index 5a1e85ff5d32..3ca542934080 100644 --- a/tools/testing/selftests/mincore/mincore_selftest.c +++ b/tools/testing/selftests/mincore/mincore_selftest.c @@ -15,6 +15,11 @@ #include <string.h> #include <fcntl.h> #include <string.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/ioctl.h> +#include <sys/sysmacros.h> +#include <sys/mount.h> #include "../kselftest.h" #include "../kselftest_harness.h" @@ -195,10 +200,42 @@ TEST(check_file_mmap) int fd; int i; int ra_pages = 0; + long ra_size, file_size; + struct stat stats; + dev_t devt; + unsigned int major, minor; + char devpath[32]; + + retval = stat(".", &stats); + ASSERT_EQ(0, retval) { + TH_LOG("Can't stat pwd: %s", strerror(errno)); + } + + devt = stats.st_dev; + major = major(devt); + minor = minor(devt); + snprintf(devpath, sizeof(devpath), "/dev/block/%u:%u", major, minor); + + fd = open(devpath, O_RDONLY); + ASSERT_NE(-1, fd) { + TH_LOG("Can't open underlying disk %s", strerror(errno)); + } + + retval = ioctl(fd, BLKRAGET, &ra_size); + ASSERT_EQ(0, retval) { + TH_LOG("Error ioctl with the underlying disk: %s", strerror(errno)); + } + + /* + * BLKRAGET ioctl returns the readahead size in sectors (512 bytes). + * Make file_size large enough to contain the readahead window. + */ + ra_size *= 512; + file_size = ra_size * 2; page_size = sysconf(_SC_PAGESIZE); - vec_size = FILE_SIZE / page_size; - if (FILE_SIZE % page_size) + vec_size = file_size / page_size; + if (file_size % page_size) vec_size++; vec = calloc(vec_size, sizeof(unsigned char)); @@ -213,7 +250,7 @@ TEST(check_file_mmap) strerror(errno)); } errno = 0; - retval = fallocate(fd, 0, 0, FILE_SIZE); + retval = fallocate(fd, 0, 0, file_size); ASSERT_EQ(0, retval) { TH_LOG("Error allocating space for the temporary file: %s", strerror(errno)); @@ -223,12 +260,12 @@ TEST(check_file_mmap) * Map the whole file, the pages shouldn't be fetched yet. */ errno = 0; - addr = mmap(NULL, FILE_SIZE, PROT_READ | PROT_WRITE, + addr = mmap(NULL, file_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); ASSERT_NE(MAP_FAILED, addr) { TH_LOG("mmap error: %s", strerror(errno)); } - retval = mincore(addr, FILE_SIZE, vec); + retval = mincore(addr, file_size, vec); ASSERT_EQ(0, retval); for (i = 0; i < vec_size; i++) { ASSERT_EQ(0, vec[i]) { @@ -240,14 +277,14 @@ TEST(check_file_mmap) * Touch a page in the middle of the mapping. We expect the next * few pages (the readahead window) to be populated too. */ - addr[FILE_SIZE / 2] = 1; - retval = mincore(addr, FILE_SIZE, vec); + addr[file_size / 2] = 1; + retval = mincore(addr, file_size, vec); ASSERT_EQ(0, retval); - ASSERT_EQ(1, vec[FILE_SIZE / 2 / page_size]) { + ASSERT_EQ(1, vec[file_size / 2 / page_size]) { TH_LOG("Page not found in memory after use"); } - i = FILE_SIZE / 2 / page_size + 1; + i = file_size / 2 / page_size + 1; while (i < vec_size && vec[i]) { ra_pages++; i++; @@ -271,7 +308,7 @@ TEST(check_file_mmap) } } - munmap(addr, FILE_SIZE); + munmap(addr, file_size); close(fd); free(vec); } -- 2.27.0

4 years, 2 months

3
4
0 0

[PATCH] kunit: add unit test for filtering suites by names

by Daniel Latypov

This adds unit tests for kunit_filter_subsuite() and kunit_filter_suites(). Note: what the executor means by "subsuite" is the array of suites corresponding to each test file. This patch lightly refactors executor.c to avoid the use of global variables to make it testable. It also includes a clever `kfree_at_end()` helper that makes this test easier to write than it otherwise would have been. Tested by running just the new tests using itself $ ./tools/testing/kunit/kunit.py run '*exec*' Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- lib/kunit/executor.c | 26 ++++---- lib/kunit/executor_test.c | 132 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+), 11 deletions(-) create mode 100644 lib/kunit/executor_test.c diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 15832ed44668..96a4ae983786 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -19,7 +19,7 @@ MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites run at boot-time, e.g. list*"); static struct kunit_suite * const * -kunit_filter_subsuite(struct kunit_suite * const * const subsuite) +kunit_filter_subsuite(struct kunit_suite * const * const subsuite, const char *filter_glob) { int i, n = 0; struct kunit_suite **filtered; @@ -52,19 +52,14 @@ struct suite_set { struct kunit_suite * const * const *end; }; -static struct suite_set kunit_filter_suites(void) +static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, + const char *filter_glob) { int i; struct kunit_suite * const **copy, * const *filtered_subsuite; struct suite_set filtered; - const size_t max = __kunit_suites_end - __kunit_suites_start; - - if (!filter_glob) { - filtered.start = __kunit_suites_start; - filtered.end = __kunit_suites_end; - return filtered; - } + const size_t max = suite_set->end - suite_set->start; copy = kmalloc_array(max, sizeof(*filtered.start), GFP_KERNEL); filtered.start = copy; @@ -74,7 +69,7 @@ static struct suite_set kunit_filter_suites(void) } for (i = 0; i < max; ++i) { - filtered_subsuite = kunit_filter_subsuite(__kunit_suites_start[i]); + filtered_subsuite = kunit_filter_subsuite(suite_set->start[i], filter_glob); if (filtered_subsuite) *copy++ = filtered_subsuite; } @@ -98,8 +93,13 @@ static void kunit_print_tap_header(struct suite_set *suite_set) int kunit_run_all_tests(void) { struct kunit_suite * const * const *suites; + struct suite_set suite_set = { + .start = __kunit_suites_start, + .end = __kunit_suites_end, + }; - struct suite_set suite_set = kunit_filter_suites(); + if (filter_glob) + suite_set = kunit_filter_suites(&suite_set, filter_glob); kunit_print_tap_header(&suite_set); @@ -115,4 +115,8 @@ int kunit_run_all_tests(void) return 0; } +#if IS_BUILTIN(CONFIG_KUNIT_TEST) +#include "executor_test.c" +#endif + #endif /* IS_BUILTIN(CONFIG_KUNIT) */ diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c new file mode 100644 index 000000000000..8e925395beeb --- /dev/null +++ b/lib/kunit/executor_test.c @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for the KUnit executor. + * + * Copyright (C) 2021, Google LLC. + * Author: Daniel Latypov <dlatypov(a)google.com> + */ + +#include <kunit/test.h> + +static void kfree_at_end(struct kunit *test, const void *to_free); +static struct kunit_suite *alloc_fake_suite(struct kunit *test, + const char *suite_name); + +static void filter_subsuite_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL, NULL}; + struct kunit_suite * const *filtered; + + subsuite[0] = alloc_fake_suite(test, "suite1"); + subsuite[1] = alloc_fake_suite(test, "suite2"); + + /* Want: suite1, suite2, NULL -> suite2, NULL */ + filtered = kunit_filter_subsuite(subsuite, "suite2*"); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered); + kfree_at_end(test, filtered); + + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered[0]); + KUNIT_EXPECT_STREQ(test, (const char *)filtered[0]->name, "suite2"); + + KUNIT_EXPECT_FALSE(test, filtered[1]); +} + +static void filter_subsuite_to_empty_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL, NULL}; + struct kunit_suite * const *filtered; + + subsuite[0] = alloc_fake_suite(test, "suite1"); + subsuite[1] = alloc_fake_suite(test, "suite2"); + + filtered = kunit_filter_subsuite(subsuite, "not_found"); + kfree_at_end(test, filtered); /* just in case */ + + KUNIT_EXPECT_FALSE_MSG(test, filtered, + "should be NULL to indicate no match"); +} + +static void kfree_subsuites_at_end(struct kunit *test, struct suite_set *suite_set) +{ + struct kunit_suite * const * const *suites; + + for (suites = suite_set->start; suites < suite_set->end; suites++) + kfree_at_end(test, *suites); +} + +static void filter_suites_test(struct kunit *test) +{ + /* Suites per-file are stored as a NULL terminated array */ + struct kunit_suite *subsuites[2][2] = { + {NULL, NULL}, + {NULL, NULL}, + }; + /* Match the memory layout of suite_set */ + struct kunit_suite * const * const suites[2] = { + subsuites[0], subsuites[1], + }; + + const struct suite_set suite_set = { + .start = suites, + .end = suites + 2, + }; + struct suite_set filtered = {.start = NULL, .end = NULL}; + + /* Emulate two files, each having one suite */ + subsuites[0][0] = alloc_fake_suite(test, "suite0"); + subsuites[1][0] = alloc_fake_suite(test, "suite1"); + + /* Filter out suite1 */ + filtered = kunit_filter_suites(&suite_set, "suite0"); + kfree_subsuites_at_end(test, &filtered); /* let us use ASSERTs without leaking */ + KUNIT_ASSERT_EQ(test, filtered.end - filtered.start, (ptrdiff_t) 1); + + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered.start); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, filtered.start[0]); + KUNIT_EXPECT_PTR_EQ(test, filtered.start[0][0], subsuites[0][0]); +} + +static struct kunit_case executor_test_cases[] = { + KUNIT_CASE(filter_subsuite_test), + KUNIT_CASE(filter_subsuite_to_empty_test), + KUNIT_CASE(filter_suites_test), + {} +}; + +static struct kunit_suite executor_test_suite = { + .name = "kunit_executor_test", + .test_cases = executor_test_cases, +}; + +kunit_test_suites(&executor_test_suite); + +/* Test helpers */ + +static void kfree_res_free(struct kunit_resource *res) +{ + kfree(res->data); +} + +/* Use the resource API to register a call to kfree(to_free). + * Since we never actually use the resource, it's safe to use on const data. + */ +static void kfree_at_end(struct kunit *test, const void *to_free) +{ + /* kfree() handles NULL already, but avoid allocating a no-op cleanup. */ + if (IS_ERR_OR_NULL(to_free)) + return; + kunit_alloc_and_get_resource(test, NULL, kfree_res_free, GFP_KERNEL, + (void *)to_free); +} + +static struct kunit_suite *alloc_fake_suite(struct kunit *test, + const char *suite_name) +{ + struct kunit_suite *suite; + + /* We normally never expect to allocate suites, hence the non-const cast. */ + suite = kunit_kzalloc(test, sizeof(*suite), GFP_KERNEL); + strncpy((char *)suite->name, suite_name, sizeof(suite->name)); + + return suite; +} base-commit: 1678e493d530e7977cce34e59a86bb86f3c5631e -- 2.31.1.295.g9ea45b61b8-goog

4 years, 2 months

3
3
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror