December 2024 - Linux-kselftest-mirror

[PATCH v2 0/2] unicode: kunit: refactor selftest to kunit tests

by Pedro Orlando

Hey all, We are making these changes as part of a KUnit Hackathon at LKCamp [1]. This patch sets out to refactor fs/unicode/utf8-selftest.c to KUnit tests. The main benefit of this change is that we can leverage KUnit's test suite for quickly compiling and testing the functions in utf8, instead of compiling the kernel and loading the previous utf8-selftest module, as well as adopting a pattern across all kernel tests. The first commit is the refactoring itself from self test into KUnit, which kept the original test logic intact -- maintaining the purpose of the original tests -- with the added benefit of including these tests into the KUnit test suite. The second commit applies the naming style and file path conventions defined on Documentation/dev-tools/kunit/style.rst We appreciate any feedback and suggestions. :) [1] https://lkcamp.dev/about/ Co-developed-by: Pedro Orlando <porlando(a)lkcamp.dev> Signed-off-by: Pedro Orlando <porlando(a)lkcamp.dev> Co-developed-by: Danilo Pereira <dpereira(a)lkcamp.dev> Signed-off-by: Danilo Pereira <dpereira(a)lkcamp.dev> Signed-off-by: Gabriela Bittencourt <gbittencourt(a)lkcamp.dev> Gabriela Bittencourt (2): unicode: kunit: refactor selftest to kunit tests unicode: kunit: change tests filename and path fs/unicode/Kconfig | 5 +- fs/unicode/Makefile | 2 +- fs/unicode/tests/.kunitconfig | 3 + .../{utf8-selftest.c => tests/utf8_kunit.c} | 149 ++++++++---------- 4 files changed, 76 insertions(+), 83 deletions(-) create mode 100644 fs/unicode/tests/.kunitconfig rename fs/unicode/{utf8-selftest.c => tests/utf8_kunit.c} (64%) -- 2.34.1

10 months, 2 weeks

6
11
0 0

[RFC PATCH v1 0/1] refactor mseal_test

by jeffxu＠chromium.org

From: Jeff Xu <jeffxu(a)chromium.org> This change creates the initial version of memorysealing.c. The introduction of memorysealing.c, which replaces mseal_test.c and uses the kselftest_harness, aims to initiate a discussion on using the selftest harness for memory sealing tests. Upon approval of this approach, the migration of tests from mseal_test.c to memorysealing.c can be implemented in a step-by-step manner. This tests addresses following feedbacks from previous reviews: 1> Use kselftest_harness instead of custom macro, such as EXPECT_XX, ASSERT_XX, etc. (Lorenzo Stoakes, Mark Brown, etc) [1] 2> Use MAP_FAILED to check the return of mmap (Lorenzo Stoakes). 3> Adding a check for vma size and prot bits. The discussion for this can be found in [2] [3], here is a brief summary: This is to follow up on Pedro’s in-loop change (from can_modify_mm to can_modify_vma). When mseal_test is initially created, they have a common pattern: setup memory layout, seal the memory, perform a few mm-api steps, verify return code (not zero). Because of the nature of out-of-loop, it is sufficient to just verify the error code in a few cases. With Pedro's in-loop change, the sealing check happens later in the stack, thus there are more things and scenarios to verify. And there were feedbacks to me that mseal_test should be extensive enough to discover all regressions. Hence I'm adding check for vma size and prot bits. In this change: we created two fixtures: Fixture basic: This creates a single VMA, the VMA has a PROT_NONE page at each end to prevent auto-merging. Fixture wo_vma: Two VMAs back to end, a PROT_NONE page at each end to prevent auto-merging. In addition, I add one test (mprotec) in each fixture for discussion. [1] https://lore.kernel.org/all/20240830180237.1220027-5-jeffxu@chromium.org/ [2] https://lore.kernel.org/all/CABi2SkUgDZtJtRJe+J9UNdtZn=EQzZcbMB685P=1rR7DUh… [3] https://lore.kernel.org/all/2qywbjb5ebtgwkh354w3lj3vhaothvubjokxq5fhyri5jee… Jeff Xu (1): selftest/mm: refactor mseal_test tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + tools/testing/selftests/mm/memorysealing.c | 182 +++++++++++++++++++++ tools/testing/selftests/mm/memorysealing.h | 116 +++++++++++++ tools/testing/selftests/mm/mseal_test.c | 67 +------- 5 files changed, 301 insertions(+), 66 deletions(-) create mode 100644 tools/testing/selftests/mm/memorysealing.c create mode 100644 tools/testing/selftests/mm/memorysealing.h -- 2.47.1.613.gc27f4b7a9f-goog

10 months, 3 weeks

3
5
0 0

[PATCH v1] kbuild: Allow building of samples with UML

by Mickaël Salaün

It's useful to build samples/* with UML and the only blocker is the artificial incompatibility with CONFIG_HEADERS_INSTALL. Allow the headers_install target with ARCH=um, which then allow building samples (and tests using them) with UML too: printf 'CONFIG_SAMPLES=y\nCONFIG_HEADERS_INSTALL=y\nCONFIG_SAMPLE_LANDLOCK=y\n' >.config make ARCH=um olddefconfig headers_install make ARCH=um samples/landlock/ Cc: Anton Ivanov <anton.ivanov(a)cambridgegreys.com> Cc: Johannes Berg <johannes(a)sipsolutions.net> Cc: Masahiro Yamada <masahiroy(a)kernel.org> Cc: Nathan Chancellor <nathan(a)kernel.org> Cc: Nicolas Schier <nicolas(a)fjasle.eu> Cc: Richard Weinberger <richard(a)nod.at> Fixes: 1b620d539ccc ("kbuild: disable header exports for UML in a straightforward way") Signed-off-by: Mickaël Salaün <mic(a)digikod.net> --- Makefile | 1 - lib/Kconfig.debug | 1 - 2 files changed, 2 deletions(-) diff --git a/Makefile b/Makefile index e5b8a8832c0c..6e2cce16a2a3 100644 --- a/Makefile +++ b/Makefile @@ -1355,7 +1355,6 @@ hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj PHONY += headers headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts - $(if $(filter um, $(SRCARCH)), $(error Headers not exportable for UML)) $(Q)$(MAKE) $(hdr-inst)=include/uapi $(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index f3d723705879..fac1208f48e4 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -473,7 +473,6 @@ config READABLE_ASM config HEADERS_INSTALL bool "Install uapi headers to usr/include" - depends on !UML help This option will install uapi headers (headers exported to user-space) into the usr/include directory for use during the kernel build. -- 2.47.1

10 months, 3 weeks

4
3
0 0

[PATCH RFC net-next v1 0/5] Device memory TCP TX

by Mina Almasry

The TX path had been dropped from the Device Memory TCP patch series post RFCv1 [1], to make that series slightly easier to review. This series rebases the implementation of the TX path on top of the net_iov/netmem framework agreed upon and merged. The motivation for the feature is thoroughly described in the docs & cover letter of the original proposal, so I don't repeat the lengthy descriptions here, but they are available in [1]. Sending this series as RFC as the winder closure is immenient. I plan on reposting as non-RFC once the tree re-opens, addressing any feedback I receive in the meantime. Full outline on usage of the TX path is detailed in the documentation added in the first patch. Test example is available via the kselftest included in the series as well. The series is relatively small, as the TX path for this feature largely piggybacks on the existing MSG_ZEROCOPY implementation. Patch Overview: --------------- 1. Documentation & tests to give high level overview of the feature being added. 2. Add netmem refcounting needed for the TX path. 3. Devmem TX netlink API. 4. Devmem TX net stack implementation. Testing: -------- Testing is very similar to devmem TCP RX path. The ncdevmem test used for the RX path is now augemented with client functionality to test TX path. * Test Setup: Kernel: net-next with this RFC and memory provider API cherry-picked locally. Hardware: Google Cloud A3 VMs. NIC: GVE with header split & RSS & flow steering support. Performance results are not included with this version, unfortunately. I'm having issues running the dma-buf exporter driver against the upstream kernel on my test setup. The issues are specific to that dma-buf exporter and do not affect this patch series. I plan to follow up this series with perf fixes if the tests point to issues once they're up and running. Special thanks to Stan who took a stab at rebasing the TX implementation on top of the netmem/net_iov framework merged. Parts of his proposal [2] that are reused as-is are forked off into their own patches to give full credit. [1] https://lore.kernel.org/netdev/20240909054318.1809580-1-almasrymina@google.… [2] https://lore.kernel.org/netdev/20240913150913.1280238-2-sdf@fomichev.me/T/#… Cc: sdf(a)fomichev.me Cc: asml.silence(a)gmail.com Cc: dw(a)davidwei.uk Mina Almasry (4): net: add devmem TCP TX documentation selftests: ncdevmem: Implement devmem TCP TX net: add get_netmem/put_netmem support net: devmem: Implement TX path Stanislav Fomichev (1): net: devmem TCP tx netlink api Documentation/netlink/specs/netdev.yaml | 12 + Documentation/networking/devmem.rst | 140 +++++++++- include/linux/skbuff.h | 13 +- include/linux/skbuff_ref.h | 4 +- include/net/netmem.h | 3 + include/net/sock.h | 2 + include/uapi/linux/netdev.h | 1 + include/uapi/linux/uio.h | 5 + net/core/datagram.c | 40 ++- net/core/devmem.c | 101 ++++++- net/core/devmem.h | 51 +++- net/core/netdev-genl-gen.c | 13 + net/core/netdev-genl-gen.h | 1 + net/core/netdev-genl.c | 67 ++++- net/core/skbuff.c | 38 ++- net/core/sock.c | 9 + net/ipv4/tcp.c | 36 ++- net/vmw_vsock/virtio_transport_common.c | 4 +- tools/include/uapi/linux/netdev.h | 1 + .../selftests/drivers/net/hw/ncdevmem.c | 261 +++++++++++++++++- 20 files changed, 764 insertions(+), 38 deletions(-) -- 2.47.1.613.gc27f4b7a9f-goog

10 months, 3 weeks

5
25
0 0

[PATCH 0/6] KUnit test moves / renames

by David Gow

As discussed in [1], the KUnit test naming scheme has changed to avoid name conflicts (and tab-completion woes) with the files being tested. These renames and moves have caused a nasty set of merge conflicts, so this series collates and rebases them all to be applied via mm-nonmm-unstable alongside any lib/ changes[2]. Thanks to everyone whose patches appear here, and everyone who reviewed on the original series. I hope I didn't break them too much during the rebase! Link: https://lore.kernel.org/lkml/20240720165441.it.320-kees@kernel.org/ [1] Link: https://lore.kernel.org/lkml/CABVgOSmbSzcGUi=E4piSojh3A4_0GjE0fAYbqKjtYGbE9… [2] --- Bruno Sobreira França (1): lib/math: Add int_log test suite Diego Vieira (1): lib/tests/kfifo_kunit.c: add tests for the kfifo structure Gabriela Bittencourt (2): unicode: kunit: refactor selftest to kunit tests unicode: kunit: change tests filename and path Kees Cook (1): lib: Move KUnit tests into tests/ subdirectory Luis Felipe Hernandez (1): lib: math: Move kunit tests into tests/ subdir MAINTAINERS | 19 +- arch/m68k/configs/amiga_defconfig | 2 +- arch/m68k/configs/apollo_defconfig | 2 +- arch/m68k/configs/atari_defconfig | 2 +- arch/m68k/configs/bvme6000_defconfig | 2 +- arch/m68k/configs/hp300_defconfig | 2 +- arch/m68k/configs/mac_defconfig | 2 +- arch/m68k/configs/multi_defconfig | 2 +- arch/m68k/configs/mvme147_defconfig | 2 +- arch/m68k/configs/mvme16x_defconfig | 2 +- arch/m68k/configs/q40_defconfig | 2 +- arch/m68k/configs/sun3_defconfig | 2 +- arch/m68k/configs/sun3x_defconfig | 2 +- arch/powerpc/configs/ppc64_defconfig | 2 +- fs/unicode/Kconfig | 5 +- fs/unicode/Makefile | 2 +- fs/unicode/tests/.kunitconfig | 3 + .../{utf8-selftest.c => tests/utf8_kunit.c} | 149 ++++++------ fs/unicode/utf8-norm.c | 2 +- lib/Kconfig.debug | 31 ++- lib/Makefile | 36 +-- lib/math/Makefile | 5 +- lib/math/tests/Makefile | 6 +- .../{test_div64.c => tests/div64_kunit.c} | 0 lib/math/tests/int_log_kunit.c | 75 ++++++ .../mul_u64_u64_div_u64_kunit.c} | 2 +- .../rational_kunit.c} | 0 lib/tests/Makefile | 39 +++ lib/{ => tests}/bitfield_kunit.c | 0 lib/{ => tests}/checksum_kunit.c | 0 lib/{ => tests}/cmdline_kunit.c | 0 lib/{ => tests}/cpumask_kunit.c | 0 lib/{ => tests}/fortify_kunit.c | 0 lib/{ => tests}/hashtable_test.c | 0 lib/{ => tests}/is_signed_type_kunit.c | 0 lib/tests/kfifo_kunit.c | 224 ++++++++++++++++++ lib/{ => tests}/kunit_iov_iter.c | 0 lib/{ => tests}/list-test.c | 0 lib/{ => tests}/memcpy_kunit.c | 0 lib/{ => tests}/overflow_kunit.c | 0 lib/{ => tests}/siphash_kunit.c | 0 lib/{ => tests}/slub_kunit.c | 0 lib/{ => tests}/stackinit_kunit.c | 0 lib/{ => tests}/string_helpers_kunit.c | 0 lib/{ => tests}/string_kunit.c | 0 lib/{ => tests}/test_bits.c | 0 lib/{ => tests}/test_fprobe.c | 0 lib/{ => tests}/test_hash.c | 0 lib/{ => tests}/test_kprobes.c | 0 lib/{ => tests}/test_linear_ranges.c | 0 lib/{ => tests}/test_list_sort.c | 0 lib/{ => tests}/test_sort.c | 0 lib/{ => tests}/usercopy_kunit.c | 0 53 files changed, 474 insertions(+), 150 deletions(-) create mode 100644 fs/unicode/tests/.kunitconfig rename fs/unicode/{utf8-selftest.c => tests/utf8_kunit.c} (64%) rename lib/math/{test_div64.c => tests/div64_kunit.c} (100%) create mode 100644 lib/math/tests/int_log_kunit.c rename lib/math/{test_mul_u64_u64_div_u64.c => tests/mul_u64_u64_div_u64_kunit.c} (98%) rename lib/math/{rational-test.c => tests/rational_kunit.c} (100%) create mode 100644 lib/tests/Makefile rename lib/{ => tests}/bitfield_kunit.c (100%) rename lib/{ => tests}/checksum_kunit.c (100%) rename lib/{ => tests}/cmdline_kunit.c (100%) rename lib/{ => tests}/cpumask_kunit.c (100%) rename lib/{ => tests}/fortify_kunit.c (100%) rename lib/{ => tests}/hashtable_test.c (100%) rename lib/{ => tests}/is_signed_type_kunit.c (100%) create mode 100644 lib/tests/kfifo_kunit.c rename lib/{ => tests}/kunit_iov_iter.c (100%) rename lib/{ => tests}/list-test.c (100%) rename lib/{ => tests}/memcpy_kunit.c (100%) rename lib/{ => tests}/overflow_kunit.c (100%) rename lib/{ => tests}/siphash_kunit.c (100%) rename lib/{ => tests}/slub_kunit.c (100%) rename lib/{ => tests}/stackinit_kunit.c (100%) rename lib/{ => tests}/string_helpers_kunit.c (100%) rename lib/{ => tests}/string_kunit.c (100%) rename lib/{ => tests}/test_bits.c (100%) rename lib/{ => tests}/test_fprobe.c (100%) rename lib/{ => tests}/test_hash.c (100%) rename lib/{ => tests}/test_kprobes.c (100%) rename lib/{ => tests}/test_linear_ranges.c (100%) rename lib/{ => tests}/test_list_sort.c (100%) rename lib/{ => tests}/test_sort.c (100%) rename lib/{ => tests}/usercopy_kunit.c (100%) -- 2.47.0.rc1.288.g06298d1525-goog

10 months, 3 weeks

10
24
0 0

Re: [PATCH 2/3] KVM: x86: Add support for VMware guest specific hypercalls

by Doug Covelli

On Wed, Nov 13, 2024 at 2:31 AM Paolo Bonzini <pbonzini(a)redhat.com> wrote: > > > > Il mar 12 nov 2024, 21:44 Doug Covelli <doug.covelli(a)broadcom.com> ha scritto: >> >> > Split irqchip should be the best tradeoff. Without it, moves from cr8 >> > stay in the kernel, but moves to cr8 always go to userspace with a >> > KVM_EXIT_SET_TPR exit. You also won't be able to use Intel >> > flexpriority (in-processor accelerated TPR) because KVM does not know >> > which bits are set in IRR. So it will be *really* every move to cr8 >> > that goes to userspace. >> >> Sorry to hijack this thread but is there a technical reason not to allow CR8 >> based accesses to the TPR (not MMIO accesses) when the in-kernel local APIC is >> not in use? > > > No worries, you're not hijacking :) The only reason is that it would be more code for a seldom used feature and anyway with worse performance. (To be clear, CR8 based accesses are allowed, but stores cause an exit in order to check the new TPR against IRR. That's because KVM's API does not have an equivalent of the TPR threshold as you point out below). I have not really looked at the code but it seems like it could also simplify things as CR8 would be handled more uniformly regardless of who is virtualizing the local APIC. >> Also I could not find these documented anywhere but with MSFT's APIC our monitor >> relies on extensions for trapping certain events such as INIT/SIPI plus LINT0 >> and SVR writes: >> >> UINT64 X64ApicInitSipiExitTrap : 1; // WHvRunVpExitReasonX64ApicInitSipiTrap >> UINT64 X64ApicWriteLint0ExitTrap : 1; // WHvRunVpExitReasonX64ApicWriteTrap >> UINT64 X64ApicWriteLint1ExitTrap : 1; // WHvRunVpExitReasonX64ApicWriteTrap >> UINT64 X64ApicWriteSvrExitTrap : 1; // WHvRunVpExitReasonX64ApicWriteTrap > > > There's no need for this in KVM's in-kernel APIC model. INIT and SIPI are handled in the hypervisor and you can get the current state of APs via KVM_GET_MPSTATE. LINT0 and LINT1 are injected with KVM_INTERRUPT and KVM_NMI respectively, and they obey IF/PPR and NMI blocking respectively, plus the interrupt shadow; so there's no need for userspace to know when LINT0/LINT1 themselves change. The spurious interrupt vector register is also handled completely in kernel. I realize that KVM can handle LINT0/SVR updates themselves but our interrupt subsystem relies on knowing the current values of these registers even when not virtualizing the local APIC. I suppose we could use KVM_GET_LAPIC to sync things up on demand but that seems like it might nor be great from a performance point of view. >> I did not see any similar functionality for KVM. Does anything like that exist? >> In any case we would be happy to add support for handling CR8 accesses w/o >> exiting w/o the in-kernel APIC along with some sort of a way to configure the >> TPR threshold if folks are not opposed to that. > > > As far I know everybody who's using KVM (whether proprietary or open source) has had no need for that, so I don't think it's a good idea to make the API more complex. Performance of Windows guests is going to be bad anyway with userspace APIC. From what I have seen the exit cost with KVM is significantly lower than with WHP/Hyper-V. I don't think performance of Windows guests with userspace APIC emulation would be bad if CR8 exits could be avoided (Linux guests perf isn't bad from what I have observed and the main difference is the astronomical number of CR8 exits). It seems like it would be pretty decent although I agree if you want the absolute best performance then you would want to use the in kernel APIC to speed up handling of ICR/EOI writes but those are relatively infrequent compared to CR8 accesses . Anyway I just saw Sean's response while writing this and it seems he is not in favor of avoiding CR8 exits w/o the in kernel APIC either so I suppose we will have to look into making use of the in kernel APIC. Doug > Paolo > >> Doug >> >> > > For now I think it makes sense to handle BDOOR_CMD_GET_VCPU_INFO at userlevel >> > > like we do on Windows and macOS. >> > > >> > > BDOOR_CMD_GETTIME/BDOOR_CMD_GETTIMEFULL are similar with the former being >> > > deprecated in favor of the latter. Both do essentially the same thing which is >> > > to return the host OS's time - on Linux this is obtained via gettimeofday. I >> > > believe this is mainly used by tools to fix up the VM's time when resuming from >> > > suspend. I think it is fine to continue handling these at userlevel. >> > >> > As long as the TSC is not involved it should be okay. >> > >> > Paolo >> > >> > > > >> Anyway, one question apart from this: is the API the same for the I/O >> > > > >> port and hypercall backdoors? >> > > > > >> > > > > Yeah the calls and arguments are the same. The hypercall based >> > > > > interface is an attempt to modernize the backdoor since as you pointed >> > > > > out the I/O based interface is kind of hacky as it bypasses the normal >> > > > > checks for an I/O port access at CPL3. It would be nice to get rid of >> > > > > it but unfortunately I don't think that will happen in the foreseeable >> > > > > future as there are a lot of existing VMs out there with older SW that >> > > > > still uses this interface. >> > > > >> > > > Yeah, but I think it still justifies that the KVM_ENABLE_CAP API can >> > > > enable the hypercall but not the I/O port. >> > > > >> > > > Paolo >> > >> >> -- >> This electronic communication and the information and any files transmitted >> with it, or attached to it, are confidential and are intended solely for >> the use of the individual or entity to whom it is addressed and may contain >> information that is confidential, legally privileged, protected by privacy >> laws, or otherwise restricted from disclosure to anyone else. If you are >> not the intended recipient or the person responsible for delivering the >> e-mail to the intended recipient, you are hereby notified that any use, >> copying, distributing, dissemination, forwarding, printing, or copying of >> this e-mail is strictly prohibited. If you received this e-mail in error, >> please return the e-mail to the sender, delete it from your computer, and >> destroy any printed copy of it. >> -- This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.

10 months, 4 weeks

3
12
0 0

[PATCH 0/6] selftests/nolibc: wire up riscv32

by Thomas Weißschuh

Nolibc has support for riscv32. But the testsuite did not allow to test it so far. Add a test configuration. Signed-off-by: Thomas Weißschuh <linux(a)weissschuh.net> --- Thomas Weißschuh (6): tools/nolibc: add support for waitid() selftests/nolibc: use waitid() over waitpid() selftests/nolibc: use a pipe to in vfprintf tests selftests/nolibc: skip tests for unimplemented syscalls selftests/nolibc: rename riscv to riscv64 selftests/nolibc: add configurations for riscv32 tools/include/nolibc/sys.h | 18 ++++++++++++ tools/testing/selftests/nolibc/Makefile | 11 +++++++ tools/testing/selftests/nolibc/nolibc-test.c | 44 ++++++++++++++++------------ tools/testing/selftests/nolibc/run-tests.sh | 2 +- 4 files changed, 56 insertions(+), 19 deletions(-) --- base-commit: 499551201b5f4fd3c0618a3e95e3d0d15ea18f31 change-id: 20241219-nolibc-rv32-cff8a3e22394 Best regards, -- Thomas Weißschuh <linux(a)weissschuh.net>

10 months, 4 weeks

4
12
0 0

[PATCH v11 00/14] riscv: Add support for xtheadvector

by Charlie Jenkins

xtheadvector is a custom extension that is based upon riscv vector version 0.7.1 [1]. All of the vector routines have been modified to support this alternative vector version based upon whether xtheadvector was determined to be supported at boot. vlenb is not supported on the existing xtheadvector hardware, so a devicetree property thead,vlenb is added to provide the vlenb to Linux. There is a new hwprobe key RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 that is used to request which thead vendor extensions are supported on the current platform. This allows future vendors to allocate hwprobe keys for their vendor. Support for xtheadvector is also added to the vector kselftests. Signed-off-by: Charlie Jenkins <charlie(a)rivosinc.com> [1] https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361… --- This series is a continuation of a different series that was fragmented into two other series in an attempt to get part of it merged in the 6.10 merge window. The split-off series did not get merged due to a NAK on the series that added the generic riscv,vlenb devicetree entry. This series has converted riscv,vlenb to thead,vlenb to remedy this issue. The original series is titled "riscv: Support vendor extensions and xtheadvector" [3]. I have tested this with an Allwinner Nezha board. I used SkiffOS [1] to manage building the image, but upgraded the U-Boot version to Samuel Holland's more up-to-date version [2] and changed out the device tree used by U-Boot with the device trees that are present in upstream linux and this series. Thank you Samuel for all of the work you did to make this task possible. [1] https://github.com/skiffos/SkiffOS/tree/master/configs/allwinner/nezha [2] https://github.com/smaeul/u-boot/commit/2e89b706f5c956a70c989cd31665f1429e9… [3] https://lore.kernel.org/all/20240503-dev-charlie-support_thead_vector_6_9-v… [4] https://lore.kernel.org/lkml/20240719-support_vendor_extensions-v3-4-0af758… --- Changes in v11: - Fix an issue where the mitigation was not being properly skipped when requested - Fix vstate_discard issue - Fix issue when -1 was passed into __riscv_isa_vendor_extension_available() - Remove some artifacts from being placed in the test directory - Link to v10: https://lore.kernel.org/r/20240911-xtheadvector-v10-0-8d3930091246@rivosinc… Changes in v10: - In DT probing disable vector with new function to clear vendor extension bits for xtheadvector - Add ghostwrite mitigations for c9xx CPUs. This disables xtheadvector unless mitigations=off is set as a kernel boot arg - Link to v9: https://lore.kernel.org/r/20240806-xtheadvector-v9-0-62a56d2da5d0@rivosinc.… Changes in v9: - Rebase onto palmer's for-next - Fix sparse error in arch/riscv/kernel/vendor_extensions/thead.c - Fix maybe-uninitialized warning in arch/riscv/include/asm/vendor_extensions/vendor_hwprobe.h - Wrap some long lines - Link to v8: https://lore.kernel.org/r/20240724-xtheadvector-v8-0-cf043168e137@rivosinc.… Changes in v8: - Rebase onto palmer's for-next - Link to v7: https://lore.kernel.org/r/20240724-xtheadvector-v7-0-b741910ada3e@rivosinc.… Changes in v7: - Add defs for has_xtheadvector_no_alternatives() and has_xtheadvector() when vector disabled. (Palmer) - Link to v6: https://lore.kernel.org/r/20240722-xtheadvector-v6-0-c9af0130fa00@rivosinc.… Changes in v6: - Fix return type of is_vector_supported()/is_xthead_supported() to be bool - Link to v5: https://lore.kernel.org/r/20240719-xtheadvector-v5-0-4b485fc7d55f@rivosinc.… Changes in v5: - Rebase on for-next - Link to v4: https://lore.kernel.org/r/20240702-xtheadvector-v4-0-2bad6820db11@rivosinc.… Changes in v4: - Replace inline asm with C (Samuel) - Rename VCSRs to CSRs (Samuel) - Replace .insn directives with .4byte directives - Link to v3: https://lore.kernel.org/r/20240619-xtheadvector-v3-0-bff39eb9668e@rivosinc.… Changes in v3: - Add back Heiko's signed-off-by (Conor) - Mark RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 as a bitmask - Link to v2: https://lore.kernel.org/r/20240610-xtheadvector-v2-0-97a48613ad64@rivosinc.… Changes in v2: - Removed extraneous references to "riscv,vlenb" (Jess) - Moved declaration of "thead,vlenb" into cpus.yaml and added restriction that it's only applicable to thead cores (Conor) - Check CONFIG_RISCV_ISA_XTHEADVECTOR instead of CONFIG_RISCV_ISA_V for thead,vlenb (Jess) - Fix naming of hwprobe variables (Evan) - Link to v1: https://lore.kernel.org/r/20240609-xtheadvector-v1-0-3fe591d7f109@rivosinc.… --- Charlie Jenkins (13): dt-bindings: riscv: Add xtheadvector ISA extension description dt-bindings: cpus: add a thead vlen register length property riscv: dts: allwinner: Add xtheadvector to the D1/D1s devicetree riscv: Add thead and xtheadvector as a vendor extension riscv: vector: Use vlenb from DT for thead riscv: csr: Add CSR encodings for CSR_VXRM/CSR_VXSAT riscv: Add xtheadvector instruction definitions riscv: vector: Support xtheadvector save/restore riscv: hwprobe: Add thead vendor extension probing riscv: hwprobe: Document thead vendor extensions and xtheadvector extension selftests: riscv: Fix vector tests selftests: riscv: Support xtheadvector in vector tests riscv: Add ghostwrite vulnerability Heiko Stuebner (1): RISC-V: define the elements of the VCSR vector CSR Documentation/arch/riscv/hwprobe.rst | 10 + Documentation/devicetree/bindings/riscv/cpus.yaml | 19 ++ .../devicetree/bindings/riscv/extensions.yaml | 10 + arch/riscv/Kconfig.errata | 11 + arch/riscv/Kconfig.vendor | 26 ++ arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi | 3 +- arch/riscv/errata/thead/errata.c | 28 ++ arch/riscv/include/asm/bugs.h | 22 ++ arch/riscv/include/asm/cpufeature.h | 2 + arch/riscv/include/asm/csr.h | 15 + arch/riscv/include/asm/errata_list.h | 3 +- arch/riscv/include/asm/hwprobe.h | 3 +- arch/riscv/include/asm/switch_to.h | 2 +- arch/riscv/include/asm/vector.h | 222 +++++++++++---- arch/riscv/include/asm/vendor_extensions/thead.h | 47 ++++ .../include/asm/vendor_extensions/thead_hwprobe.h | 19 ++ .../include/asm/vendor_extensions/vendor_hwprobe.h | 37 +++ arch/riscv/include/uapi/asm/hwprobe.h | 3 +- arch/riscv/include/uapi/asm/vendor/thead.h | 3 + arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/bugs.c | 60 ++++ arch/riscv/kernel/cpufeature.c | 59 +++- arch/riscv/kernel/kernel_mode_vector.c | 8 +- arch/riscv/kernel/process.c | 4 +- arch/riscv/kernel/signal.c | 6 +- arch/riscv/kernel/sys_hwprobe.c | 5 + arch/riscv/kernel/vector.c | 24 +- arch/riscv/kernel/vendor_extensions.c | 10 + arch/riscv/kernel/vendor_extensions/Makefile | 2 + arch/riscv/kernel/vendor_extensions/thead.c | 29 ++ .../riscv/kernel/vendor_extensions/thead_hwprobe.c | 19 ++ drivers/base/cpu.c | 3 + include/linux/cpu.h | 1 + tools/testing/selftests/riscv/vector/.gitignore | 3 +- tools/testing/selftests/riscv/vector/Makefile | 17 +- .../selftests/riscv/vector/v_exec_initval_nolibc.c | 94 +++++++ tools/testing/selftests/riscv/vector/v_helpers.c | 68 +++++ tools/testing/selftests/riscv/vector/v_helpers.h | 8 + tools/testing/selftests/riscv/vector/v_initval.c | 22 ++ .../selftests/riscv/vector/v_initval_nolibc.c | 68 ----- .../selftests/riscv/vector/vstate_exec_nolibc.c | 20 +- .../testing/selftests/riscv/vector/vstate_prctl.c | 305 +++++++++++++-------- 42 files changed, 1051 insertions(+), 271 deletions(-) --- base-commit: 0eb512779d642b21ced83778287a0f7a3ca8f2a1 change-id: 20240530-xtheadvector-833d3d17b423 -- - Charlie

11 months

3
22
0 0

[PATCH bpf-next/net v2 0/7] bpf: Add mptcp_subflow bpf_iter support

by Matthieu Baerts (NGI0)

Here is a series from Geliang, adding mptcp_subflow bpf_iter support. We are working on extending MPTCP with BPF, e.g. to control the path manager -- in charge of the creation, deletion, and announcements of subflows (paths) -- and the packet scheduler -- in charge of selecting which available path the next data will be sent to. These extensions need to iterate over the list of subflows attached to an MPTCP connection, and do some specific actions via some new kfunc that will be added later on. This preparation work is split in different patches: - Patch 1: extend bpf_skc_to_mptcp_sock() to be called with msk. - Patch 2: allow using skc_to_mptcp_sock() in CGroup sockopt hooks. - Patch 3: register some "basic" MPTCP kfunc. - Patch 4: add mptcp_subflow bpf_iter support. Note that previous versions of this single patch have already been shared to the BPF mailing list. The changelog has been kept with a comment, but the version number has been reset to avoid confusions. - Patch 5: add kfunc to make sure the msk is valid - Patch 6: add more MPTCP endpoints in the selftests, in order to create more than 2 subflows. - Patch 7: add a very simple test validating mptcp_subflow bpf_iter support. This test could be written without the new bpf_iter, but it is there only to make sure this specific feature works as expected. Signed-off-by: Matthieu Baerts (NGI0) <matttbe(a)kernel.org> --- Changes in v2: - Patches 1-2: new ones. - Patch 3: remove two kfunc, more restrictions. (Martin) - Patch 4: add BUILD_BUG_ON(), more restrictions. (Martin) - Patch 7: adaptations due to modifications in patches 1-4. - Link to v1: https://lore.kernel.org/r/20241108-bpf-next-net-mptcp-bpf_iter-subflows-v1-… --- Geliang Tang (7): bpf: Extend bpf_skc_to_mptcp_sock to MPTCP sock bpf: Allow use of skc_to_mptcp_sock in cg_sockopt bpf: Register mptcp common kfunc set bpf: Add mptcp_subflow bpf_iter bpf: Acquire and release mptcp socket selftests/bpf: More endpoints for endpoint_init selftests/bpf: Add mptcp_subflow bpf_iter subtest include/net/mptcp.h | 4 +- kernel/bpf/cgroup.c | 2 + net/core/filter.c | 2 +- net/mptcp/bpf.c | 113 +++++++++++++++++- tools/testing/selftests/bpf/bpf_experimental.h | 8 ++ tools/testing/selftests/bpf/prog_tests/mptcp.c | 129 ++++++++++++++++++++- tools/testing/selftests/bpf/progs/mptcp_bpf.h | 9 ++ .../testing/selftests/bpf/progs/mptcp_bpf_iters.c | 63 ++++++++++ 8 files changed, 318 insertions(+), 12 deletions(-) --- base-commit: dad704ebe38642cd405e15b9c51263356391355c change-id: 20241108-bpf-next-net-mptcp-bpf_iter-subflows-027f6d87770e Best regards, -- Matthieu Baerts (NGI0) <matttbe(a)kernel.org>

11 months

3
13
0 0

[PATCH v4 0/9] mm: workingset reporting

by Yuanchu Xie

This patch series provides workingset reporting of user pages in lruvecs, of which coldness can be tracked by accessed bits and fd references. However, the concept of workingset applies generically to all types of memory, which could be kernel slab caches, discardable userspace caches (databases), or CXL.mem. Therefore, data sources might come from slab shrinkers, device drivers, or the userspace. Another interesting idea might be hugepage workingset, so that we can measure the proportion of hugepages backing cold memory. However, with architectures like arm, there may be too many hugepage sizes leading to a combinatorial explosion when exporting stats to the userspace. Nonetheless, the kernel should provide a set of workingset interfaces that is generic enough to accommodate the various use cases, and extensible to potential future use cases. Use cases ========== Job scheduling On overcommitted hosts, workingset information improves efficiency and reliability by allowing the job scheduler to have better stats on the exact memory requirements of each job. This can manifest in efficiency by landing more jobs on the same host or NUMA node. On the other hand, the job scheduler can also ensure each node has a sufficient amount of memory and does not enter direct reclaim or the kernel OOM path. With workingset information and job priority, the userspace OOM killing or proactive reclaim policy can kick in before the system is under memory pressure. If the job shape is very different from the machine shape, knowing the workingset per-node can also help inform page allocation policies. Proactive reclaim Workingset information allows the a container manager to proactively reclaim memory while not impacting a job's performance. While PSI may provide a reactive measure of when a proactive reclaim has reclaimed too much, workingset reporting allows the policy to be more accurate and flexible. Ballooning (similar to proactive reclaim) The last patch of the series extends the virtio-balloon device to report the guest workingset. Balloon policies benefit from workingset to more precisely determine the size of the memory balloon. On end-user devices where memory is scarce and overcommitted, the balloon sizing in multiple VMs running on the same device can be orchestrated with workingset reports from each one. On the server side, workingset reporting allows the balloon controller to inflate the balloon without causing too much file cache to be reclaimed in the guest. Promotion/Demotion If different mechanisms are used for promition and demotion, workingset information can help connect the two and avoid pages being migrated back and forth. For example, given a promotion hot page threshold defined in reaccess distance of N seconds (promote pages accessed more often than every N seconds). The threshold N should be set so that ~80% (e.g.) of pages on the fast memory node passes the threshold. This calculation can be done with workingset reports. To be directly useful for promotion policies, the workingset report interfaces need to be extended to report hotness and gather hotness information from the devices[1]. [1] https://www.opencompute.org/documents/ocp-cms-hotness-tracking-requirements… Sysfs and Cgroup Interfaces ========== The interfaces are detailed in the patches that introduce them. The main idea here is we break down the workingset per-node per-memcg into time intervals (ms), e.g. 1000 anon=137368 file=24530 20000 anon=34342 file=0 30000 anon=353232 file=333608 40000 anon=407198 file=206052 9223372036854775807 anon=4925624 file=892892 Implementation ========== The reporting of user pages is based off of MGLRU, and therefore requires CONFIG_LRU_GEN=y. We would benefit from more MGLRU generations for a more fine-grained workingset report, but we can already gather a lot of data with just four generations. The workingset reporting mechanism is gated behind CONFIG_WORKINGSET_REPORT, and the aging thread is behind CONFIG_WORKINGSET_REPORT_AGING. Benchmarks ========== Ghait Ouled Amar Ben Cheikh has implemented a simple policy and ran Linux compile and redis benchmarks from openbenchmarking.org. The policy and runner is referred to as WMO (Workload Memory Optimization). The results were based on v3 of the series, but v4 doesn't change the core of the working set reporting and just adds the ballooning counterpart. The timed Linux kernel compilation benchmark shows improvements in peak memory usage with a policy of "swap out all bytes colder than 10 seconds every 40 seconds". A swapfile is configured on SSD. -------------------------------------------- peak memory usage (with WMO): 4982.61328 MiB peak memory usage (control): 9569.1367 MiB peak memory reduction: 47.9% -------------------------------------------- Benchmark | Experimental |Control | Experimental_Std_Dev | Control_Std_Dev Timed Linux Kernel Compilation - allmodconfig (sec) | 708.486 (95.91%) | 679.499 (100%) | 0.6% | 0.1% -------------------------------------------- Seconds, fewer is better The redis benchmark shows employs the same policy: -------------------------------------------- peak memory usage (with WMO): 375.9023 MiB peak memory usage (control): 509.765 MiB peak memory reduction: 26% -------------------------------------------- Benchmark | Experimental | Control | Experimental_Std_Dev | Control_Std_Dev Redis - LPOP (Reqs/sec) | 2023130 (98.22%) | 2059849 (100%) | 1.2% | 2% Redis - SADD (Reqs/sec) | 2539662 (98.63%) | 2574811 (100%) | 2.3% | 1.4% Redis - LPUSH (Reqs/sec)| 2024880 (100%) | 2000884 (98.81%) | 1.1% | 0.8% Redis - GET (Reqs/sec) | 2835764 (100%) | 2763722 (97.46%) | 2.7% | 1.6% Redis - SET (Reqs/sec) | 2340723 (100%) | 2327372 (99.43%) | 2.4% | 1.8% -------------------------------------------- Reqs/sec, more is better The detailed report and benchmarking results are in Ghait's repo: https://github.com/miloudi98/WMO Changelog ========== Changes from PATCH v3 -> v4: - Added documentation for cgroup-v2 (Waiman Long) - Fixed types in documentation (Randy Dunlap) - Added implementation for the ballooning use case - Added detailed description of benchmark results (Andrew Morton) Changes from PATCH v2 -> v3: - Fixed typos in commit messages and documentation (Lance Yang, Randy Dunlap) - Split out the force_scan patch to be reviewed separately - Added benchmarks from Ghait Ouled Amar Ben Cheikh - Fixed reported compile error without CONFIG_MEMCG Changes from PATCH v1 -> v2: - Updated selftest to use ksft_test_result_code instead of switch-case (Muhammad Usama Anjum) - Included more use cases in the cover letter (Huang, Ying) - Added documentation for sysfs and memcg interfaces - Added an aging-specific struct lru_gen_mm_walk in struct pglist_data to avoid allocating for each lruvec. [v1] https://lore.kernel.org/linux-mm/20240504073011.4000534-1-yuanchu@google.co… [v2] https://lore.kernel.org/linux-mm/20240604020549.1017540-1-yuanchu@google.co… [v3] https://lore.kernel.org/linux-mm/20240813165619.748102-1-yuanchu@google.com/ Yuanchu Xie (9): mm: aggregate workingset information into histograms mm: use refresh interval to rate-limit workingset report aggregation mm: report workingset during memory pressure driven scanning mm: extend workingset reporting to memcgs mm: add kernel aging thread for workingset reporting selftest: test system-wide workingset reporting Docs/admin-guide/mm/workingset_report: document sysfs and memcg interfaces Docs/admin-guide/cgroup-v2: document workingset reporting virtio-balloon: add workingset reporting Documentation/admin-guide/cgroup-v2.rst | 35 + Documentation/admin-guide/mm/index.rst | 1 + .../admin-guide/mm/workingset_report.rst | 105 +++ drivers/base/node.c | 6 + drivers/virtio/virtio_balloon.c | 390 ++++++++++- include/linux/balloon_compaction.h | 1 + include/linux/memcontrol.h | 21 + include/linux/mmzone.h | 13 + include/linux/workingset_report.h | 167 +++++ include/uapi/linux/virtio_balloon.h | 30 + mm/Kconfig | 15 + mm/Makefile | 2 + mm/internal.h | 19 + mm/memcontrol.c | 162 ++++- mm/mm_init.c | 2 + mm/mmzone.c | 2 + mm/vmscan.c | 56 +- mm/workingset_report.c | 653 ++++++++++++++++++ mm/workingset_report_aging.c | 127 ++++ tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 3 + tools/testing/selftests/mm/run_vmtests.sh | 5 + .../testing/selftests/mm/workingset_report.c | 306 ++++++++ .../testing/selftests/mm/workingset_report.h | 39 ++ .../selftests/mm/workingset_report_test.c | 330 +++++++++ 25 files changed, 2482 insertions(+), 9 deletions(-) create mode 100644 Documentation/admin-guide/mm/workingset_report.rst create mode 100644 include/linux/workingset_report.h create mode 100644 mm/workingset_report.c create mode 100644 mm/workingset_report_aging.c create mode 100644 tools/testing/selftests/mm/workingset_report.c create mode 100644 tools/testing/selftests/mm/workingset_report.h create mode 100644 tools/testing/selftests/mm/workingset_report_test.c -- 2.47.0.338.g60cca15819-goog

11 months

6
20
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror December 2024