From: Fred Griffoul griffoul@casper.infradead.org
This patch series addresses performance issues in nested VMX when handling unmanaged guest memory. Unmanaged guest memory refers to memory not directly mapped by the kernel (no struct page), such as memory passed with the mem= parameter or guest_memfd for non-Confidential Computing (CoCo) VMs.
Current Problem:
During nested VMX operations, the system frequently accesses specific guest pages during L2 VM entry/exit cycles. The current workflow:
1. kvm_vcpu_map() invokes memremap() for unmanaged memory. 2. The system either directly accesses mapped memory via nested VMX or passes it to the L2 guest through vmcs02. 3. kvm_vcpu_unmap() invokes memunmap()
This repeated map/unmap cycle creates significant performance overhead due to expensive remapping operations.
Solution approach:
Our solution replaces kvm_host_map with gfn_to_pfn_cache in nested VMX. It addresses two distinct types of guest pages.
First, we handle the L1 MSR bitmap page, which requires read-only access for folding L1 and L0 MSR bitmap. We implement this conversion to gfn_to_pfn_cache in patch 1.
Second, we tackle system pages, including APIC access, virtual APIC, and posted interrupt descriptor pages. These pages are more complex as they're accessed by both nested VMX code _and_ passed to the L2 guest in vmcs02 fields. This requires to restore and complete the "guest-uses-pfn" support in pfncache through patches 2 and 3, followed by implementing kvm_host_map replacement with caches in patch 4.
Testing:
Patch 5 introduces a new selftest to verify cache invalidation and memslot update functionality.
The changes are available in a git repository at:
git://git.infradead.org/users/griffoul/linux.git tags/nvmx-gpc-v1
Suggested-by: dwmw@amazon.co.uk
Fred Griffoul (5): KVM: nVMX: Implement cache for L1 MSR bitmap KVM: pfncache: Restore guest-uses-pfn support KVM: x86: Add nested state validation for pfncache support KVM: nVMX: Implement cache for L1 APIC pages KVM: selftests: Add nested VMX APIC cache invalidation test
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/nested.c | 213 +++++++++--- arch/x86/kvm/vmx/vmx.h | 10 +- arch/x86/kvm/x86.c | 14 +- include/linux/kvm_host.h | 34 +- include/linux/kvm_types.h | 1 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/vmx_apic_update_test.c | 302 ++++++++++++++++++ virt/kvm/kvm_main.c | 3 +- virt/kvm/kvm_mm.h | 6 +- virt/kvm/pfncache.c | 43 ++- 11 files changed, 575 insertions(+), 53 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/vmx_apic_update_test.c