When handling page faults for many vCPUs during demand paging, KVM's MMU lock becomes highly contended. This series creates a test with a naive userfaultfd based demand paging implementation to demonstrate that contention. This test serves both as a functional test of userfaultfd and a microbenchmark of demand paging performance with a variable number of vCPUs and memory per vCPU.
The test creates N userfaultfd threads, N vCPUs, and a region of memory with M pages per vCPU. The N userfaultfd polling threads are each set up to serve faults on a region of memory corresponding to one of the vCPUs. Each of the vCPUs is then started, and touches each page of its disjoint memory region, sequentially. In response to faults, the userfaultfd threads copy a static buffer into the guest's memory. This creates a worst case for MMU lock contention as we have removed most of the contention between the userfaultfd threads and there is no time required to fetch the contents of guest memory.
This test was run successfully on Intel Haswell, Broadwell, and Cascadelake hosts with a variety of vCPU counts and memory sizes.
This test was adapted from the dirty_log_test.
The series can also be viewed in Gerrit here: https://linux-review.googlesource.com/c/virt/kvm/kvm/+/1464 (Thanks to Dmitry Vyukov dvyukov@google.com for setting up the Gerrit instance)
Ben Gardon (9): KVM: selftests: Create a demand paging test KVM: selftests: Add demand paging content to the demand paging test KVM: selftests: Add memory size parameter to the demand paging test KVM: selftests: Pass args to vCPU instead of using globals KVM: selftests: Support multiple vCPUs in demand paging test KVM: selftests: Time guest demand paging KVM: selftests: Add parameter to _vm_create for memslot 0 base paddr KVM: selftests: Support large VMs in demand paging test Add static flag
tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 4 +- .../selftests/kvm/demand_paging_test.c | 610 ++++++++++++++++++ tools/testing/selftests/kvm/dirty_log_test.c | 2 +- .../testing/selftests/kvm/include/kvm_util.h | 3 +- tools/testing/selftests/kvm/lib/kvm_util.c | 7 +- 6 files changed, 621 insertions(+), 6 deletions(-) create mode 100644 tools/testing/selftests/kvm/demand_paging_test.c