Userspace applications can choose to completely ignore the kernel provided
flow key and instead regenerate a fresh key for processing in userspace.
Currently, userspace ovs-vswitchd does this in some instances (for example,
MISS upcall command). This means that kernel spends time to copy and send
the flow key into userspace without any benefit to the system.
Introduce a way for userspace to tell kernel not to send the flow key.
This lets userspace and kernel space save time and memory pressure.
This patch set is quite a bit larger because it introduces the ability to
decode a sw flow key into a compatible datapath-string. We use this as a
method of implementing a test to show that the feature is working by
decoding and dumping the flow (to make sure we capture the correct packet).
Aaron Conole (6):
openvswitch: exclude kernel flow key from upcalls
selftests: openvswitch: add interface support
selftests: openvswitch: add flow dump support
selftests: openvswitch: adjust datapath NL message
selftests: openvswitch: add upcall support
selftests: openvswitch: add exclude support for packet commands
include/uapi/linux/openvswitch.h | 6 +
net/openvswitch/datapath.c | 26 +-
net/openvswitch/datapath.h | 2 +
.../selftests/net/openvswitch/openvswitch.sh | 101 +-
.../selftests/net/openvswitch/ovs-dpctl.py | 1069 ++++++++++++++++-
5 files changed, 1183 insertions(+), 21 deletions(-)
--
2.34.3
For now, we did not support reliable R/O long-term pinning in COW mappings.
That means, if we would trigger R/O long-term pinning in MAP_PRIVATE
mapping, we could end up pinning the (R/O-mapped) shared zeropage or a
pagecache page.
The next write access would trigger a write fault and replace the pinned
page by an exclusive anonymous page in the process page table; whatever the
process would write to that private page copy would not be visible by the
owner of the previous page pin: for example, RDMA could read stale data.
The end result is essentially an unexpected and hard-to-debug memory
corruption.
Some drivers tried working around that limitation by using
"FOLL_FORCE|FOLL_WRITE|FOLL_LONGTERM" for R/O long-term pinning for now.
FOLL_WRITE would trigger a write fault, if required, and break COW before
pinning the page. FOLL_FORCE is required because the VMA might lack write
permissions, and drivers wanted to make that working as well, just like
one would expect (no write access, but still triggering a write access to
break COW).
However, that is not a practical solution, because
(1) Drivers that don't stick to that undocumented and debatable pattern
would still run into that issue. For example, VFIO only uses
FOLL_LONGTERM for R/O long-term pinning.
(2) Using FOLL_WRITE just to work around a COW mapping + page pinning
limitation is unintuitive. FOLL_WRITE would, for example, mark the
page softdirty or trigger uffd-wp, even though, there actually isn't
going to be any write access.
(3) The purpose of FOLL_FORCE is debug access, not access without lack of
VMA permissions by arbitrarty drivers.
So instead, make R/O long-term pinning work as expected, by breaking COW
in a COW mapping early, such that we can remove any FOLL_FORCE usage from
drivers and make FOLL_FORCE ptrace-specific (renaming it to FOLL_PTRACE).
More details in patch #8.
Patches #1--#3 add COW tests for non-anonymous pages.
Patches #4--#7 prepare core MM for extended FAULT_FLAG_UNSHARE support in
COW mappings.
Patch #8 implements reliable R/O long-term pinning in COW mappings
Patches #9--#19 remove any FOLL_FORCE usage from drivers.
Patch #20 renames FOLL_FORCE to FOLL_PTRACE.
I'm refraining from CCing all driver/arch maintainers on the whole patch
set, but only CC them on the cover letter and the applicable patch
(I know, I know, someone is always unhappy ... sorry).
RFC -> v1:
* Use term "ptrace" instead of "debuggers" in patch descriptions
* Added ACK/Tested-by
* "mm/frame-vector: remove FOLL_FORCE usage"
-> Adjust description
* "mm: rename FOLL_FORCE to FOLL_PTRACE"
-> Added
David Hildenbrand (20):
selftests/vm: anon_cow: prepare for non-anonymous COW tests
selftests/vm: cow: basic COW tests for non-anonymous pages
selftests/vm: cow: R/O long-term pinning reliability tests for
non-anon pages
mm: add early FAULT_FLAG_UNSHARE consistency checks
mm: add early FAULT_FLAG_WRITE consistency checks
mm: rework handling in do_wp_page() based on private vs. shared
mappings
mm: don't call vm_ops->huge_fault() in wp_huge_pmd()/wp_huge_pud() for
private mappings
mm: extend FAULT_FLAG_UNSHARE support to anything in a COW mapping
mm/gup: reliable R/O long-term pinning in COW mappings
RDMA/umem: remove FOLL_FORCE usage
RDMA/usnic: remove FOLL_FORCE usage
RDMA/siw: remove FOLL_FORCE usage
media: videobuf-dma-sg: remove FOLL_FORCE usage
drm/etnaviv: remove FOLL_FORCE usage
media: pci/ivtv: remove FOLL_FORCE usage
mm/frame-vector: remove FOLL_FORCE usage
drm/exynos: remove FOLL_FORCE usage
RDMA/hw/qib/qib_user_pages: remove FOLL_FORCE usage
habanalabs: remove FOLL_FORCE usage
mm: rename FOLL_FORCE to FOLL_PTRACE
arch/alpha/kernel/ptrace.c | 6 +-
arch/arm64/kernel/mte.c | 2 +-
arch/ia64/kernel/ptrace.c | 10 +-
arch/mips/kernel/ptrace32.c | 4 +-
arch/mips/math-emu/dsemul.c | 2 +-
arch/powerpc/kernel/ptrace/ptrace32.c | 4 +-
arch/sparc/kernel/ptrace_32.c | 4 +-
arch/sparc/kernel/ptrace_64.c | 8 +-
arch/x86/kernel/step.c | 2 +-
arch/x86/um/ptrace_32.c | 2 +-
arch/x86/um/ptrace_64.c | 2 +-
drivers/gpu/drm/etnaviv/etnaviv_gem.c | 8 +-
drivers/gpu/drm/exynos/exynos_drm_g2d.c | 2 +-
drivers/infiniband/core/umem.c | 8 +-
drivers/infiniband/hw/qib/qib_user_pages.c | 2 +-
drivers/infiniband/hw/usnic/usnic_uiom.c | 9 +-
drivers/infiniband/sw/siw/siw_mem.c | 9 +-
drivers/media/common/videobuf2/frame_vector.c | 2 +-
drivers/media/pci/ivtv/ivtv-udma.c | 2 +-
drivers/media/pci/ivtv/ivtv-yuv.c | 5 +-
drivers/media/v4l2-core/videobuf-dma-sg.c | 14 +-
drivers/misc/habanalabs/common/memory.c | 3 +-
fs/exec.c | 2 +-
fs/proc/base.c | 2 +-
include/linux/mm.h | 35 +-
include/linux/mm_types.h | 8 +-
kernel/events/uprobes.c | 4 +-
kernel/ptrace.c | 12 +-
mm/gup.c | 38 +-
mm/huge_memory.c | 13 +-
mm/hugetlb.c | 14 +-
mm/memory.c | 97 +++--
mm/util.c | 4 +-
security/tomoyo/domain.c | 2 +-
tools/testing/selftests/vm/.gitignore | 2 +-
tools/testing/selftests/vm/Makefile | 10 +-
tools/testing/selftests/vm/check_config.sh | 4 +-
.../selftests/vm/{anon_cow.c => cow.c} | 387 +++++++++++++++++-
tools/testing/selftests/vm/run_vmtests.sh | 8 +-
39 files changed, 575 insertions(+), 177 deletions(-)
rename tools/testing/selftests/vm/{anon_cow.c => cow.c} (75%)
--
2.38.1
Hello everyone,
We have prepared patches to address an issue from a previous discussion.
The previous discussion email thread is here: https://lore.kernel.org/all/CAADnVQLBt0snxv4bKwg1WKQ9wDFbaDCtZ03v1-LjOTYtsK…
This patch series adds a new field "used_entries" to struct bpf_map_info
and keeps tracking the "count" field in bpf_htab in both the preallocated
and non-preallocated cases.
bpftool is modified to report the newly added "used_entries" field in
struct bpf_map_info and to mark pre-allocated htab maps with "*".
These make it easier to view the current memory situation of a hashmap.
We have added a new interface function map_get_used_elem in bpf_map_ops
to provide an abstraction layer so that other map type implementations can
support the "used_entries" attribute in a future change.
A concurrency testing for pre-allocated and dynamically allocated
htab maps is introduced to test the correctness and performance of
htab map's used size.
Existing unit tests are integrated to test the correctness of
htab map's used size.
Thank you,
Ho-Ren (Jack) Chuang (4):
bpf: Support reporting BPF htab map's used size for monitoring
bpftool: Add tools support to show BPF htab map's used size
samples/bpf: Add concurrency testing for BPF htab map's used size
selftests/bpf: Add unit tests for BPF htab map's used size
include/linux/bpf.h | 1 +
include/uapi/linux/bpf.h | 1 +
kernel/bpf/hashtab.c | 19 +++
kernel/bpf/syscall.c | 2 +
samples/bpf/Makefile | 4 +
samples/bpf/test_map_used_kern.c | 65 ++++++++
samples/bpf/test_map_used_user.c | 204 ++++++++++++++++++++++++
tools/bpf/bpftool/map.c | 9 +-
tools/include/uapi/linux/bpf.h | 1 +
tools/testing/selftests/bpf/test_maps.c | 74 ++++++++-
10 files changed, 377 insertions(+), 3 deletions(-)
create mode 100644 samples/bpf/test_map_used_kern.c
create mode 100644 samples/bpf/test_map_used_user.c
--
Ho-Ren (Jack) Chuang
On 11/28/22 11:53, Maxime Ripard wrote:
> Hi,
>
> This series introduce Kunit tests to the vc4 KMS driver, but unlike what we
> have been doing so far in KMS, it actually tests the atomic modesetting code.
>
> In order to do so, I've had to improve a fair bit on the Kunit helpers already
> found in the tree in order to register a full blown and somewhat functional KMS
> driver.
>
> It's of course relying on a mock so that we can test it anywhere. The mocking
> approach created a number of issues, the main one being that we need to create
> a decent mock in the first place, see patch 22. The basic idea is that I
> created some structures to provide a decent approximation of the actual
> hardware, and that would support both major architectures supported by vc4.
>
> This is of course meant to evolve over time and support more tests, but I've
> focused on testing the HVS FIFO assignment code which is fairly tricky (and the
> tests have actually revealed one more bug with our current implementation). I
> used to have a userspace implementation of those tests, where I would copy and
> paste the kernel code and run the tests on a regular basis. It's was obviously
> fairly suboptimal, so it seemed like the perfect testbed for that series.
>
> It can be run using:
> ./tools/testing/kunit/kunit.py run \
> --kunitconfig=drivers/gpu/drm/vc4/tests/.kunitconfig \
> --cross_compile aarch64-linux-gnu- --arch arm64
>
> Let me know what you think,
> Maxime
Hi Maxime,
It is great to see some device mocking with KUnit! Other than the
comments that I pointed out in the series, I believe that a small entry
on the VC4 documentation would be nice to cover how to run the tests and
also what the tests are currently covering.
Best Regards,
- Maíra Canal
>
> To: David Airlie <airlied(a)gmail.com>
> To: Daniel Vetter <daniel(a)ffwll.ch>
> To: Maarten Lankhorst <maarten.lankhorst(a)linux.intel.com>
> To: Maxime Ripard <mripard(a)kernel.org>
> To: Thomas Zimmermann <tzimmermann(a)suse.de>
> Cc: Dave Stevenson <dave.stevenson(a)raspberrypi.com>
> Cc: Javier Martinez Canillas <javierm(a)redhat.com>
> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
> Cc: Maíra Canal <mairacanal(a)riseup.net>
> Cc: Brendan Higgins <brendan.higgins(a)linux.dev>
> Cc: David Gow <davidgow(a)google.com>
> Cc: linux-kselftest(a)vger.kernel.org
> Cc: kunit-dev(a)googlegroups.com
> Cc: dri-devel(a)lists.freedesktop.org
> Cc: linux-kernel(a)vger.kernel.org
> Cc: linux-media(a)vger.kernel.org
> Cc: linaro-mm-sig(a)lists.linaro.org
> Signed-off-by: Maxime Ripard <maxime(a)cerno.tech>
>
> ---
> Changes in v2:
> - Added some documentation for public functions
> - Removed the fake device probe/remove workqueue
> - Made sure the tests could be compiled as modules
> - Moved the vc4 tests in the vc4 module
> - Applied some of the preliminary patches
> - Rebased on top of current drm-misc-next branch
> - Fixed checkpatch issues
> - Introduced BCM2835 (Pi0-3) tests for muxing
> - Introduced tests to cover past bugs we had
> - Link to v1: https://lore.kernel.org/r/20221123-rpi-kunit-tests-v1-0-051a0bb60a16@cerno.…
>
> ---
> Maxime Ripard (17):
> drm/tests: helpers: Move the helper header to include/drm
> drm/tests: helpers: Document drm_kunit_device_init()
> drm/tests: helpers: Rename the device init helper
> drm/tests: helpers: Remove the name parameter
> drm/tests: helpers: Create the device in another function
> drm/tests: helpers: Switch to a platform_device
> drm/tests: helpers: Make sure the device is bound
> drm/tests: helpers: Allow for a custom device struct to be allocated
> drm/tests: helpers: Allow to pass a custom drm_driver
> drm/tests: Add a test for DRM managed actions
> drm/vc4: Move HVS state to main header
> drm/vc4: crtc: Introduce a lower-level crtc init helper
> drm/vc4: crtc: Make encoder lookup helper public
> drm/vc4: hvs: Provide a function to initialize the HVS structure
> drm/vc4: tests: Introduce a mocking infrastructure
> drm/vc4: tests: Fail the current test if we access a register
> drm/vc4: tests: Add unit test suite for the PV muxing
>
> drivers/gpu/drm/tests/Makefile | 1 +
> drivers/gpu/drm/tests/drm_client_modeset_test.c | 19 +-
> drivers/gpu/drm/tests/drm_kunit_helpers.c | 106 ++-
> drivers/gpu/drm/tests/drm_kunit_helpers.h | 11 -
> drivers/gpu/drm/tests/drm_managed_test.c | 71 ++
> drivers/gpu/drm/tests/drm_modes_test.c | 19 +-
> drivers/gpu/drm/tests/drm_probe_helper_test.c | 20 +-
> drivers/gpu/drm/vc4/Kconfig | 15 +
> drivers/gpu/drm/vc4/Makefile | 7 +
> drivers/gpu/drm/vc4/tests/.kunitconfig | 14 +
> drivers/gpu/drm/vc4/tests/vc4_mock.c | 200 +++++
> drivers/gpu/drm/vc4/tests/vc4_mock.h | 63 ++
> drivers/gpu/drm/vc4/tests/vc4_mock_crtc.c | 41 +
> drivers/gpu/drm/vc4/tests/vc4_mock_output.c | 138 +++
> drivers/gpu/drm/vc4/tests/vc4_mock_plane.c | 47 +
> drivers/gpu/drm/vc4/tests/vc4_test_pv_muxing.c | 1039 +++++++++++++++++++++++
> drivers/gpu/drm/vc4/vc4_crtc.c | 102 ++-
> drivers/gpu/drm/vc4/vc4_dpi.c | 13 +-
> drivers/gpu/drm/vc4/vc4_drv.c | 4 +-
> drivers/gpu/drm/vc4/vc4_drv.h | 91 +-
> drivers/gpu/drm/vc4/vc4_dsi.c | 9 +-
> drivers/gpu/drm/vc4/vc4_hdmi_regs.h | 4 +
> drivers/gpu/drm/vc4/vc4_hvs.c | 81 +-
> drivers/gpu/drm/vc4/vc4_kms.c | 25 +-
> drivers/gpu/drm/vc4/vc4_txp.c | 15 +-
> drivers/gpu/drm/vc4/vc4_vec.c | 13 +-
> include/drm/drm_kunit_helpers.h | 91 ++
> 27 files changed, 2087 insertions(+), 172 deletions(-)
> ---
> base-commit: 199557fab92548f8e9d5207e385097213abe0cab
> change-id: 20221123-rpi-kunit-tests-87a388492a73
>
> Best regards,