[PATCH 5.4 0/4] Backport fixes to avoid SEV guest with 380GB+ memory causing host cpu softhang
[ patch series targeting linux-5.4.y stable branch. ]
Creating a SEV-enabled guest with 380GB or more of memory causes a cpu soft-hang in the host running 5.4 with the following stacktrace:
kernel: watchdog: BUG: soft lockup - CPU#214 stuck for 22s! [qemu-kvm:6424] ... kernel: CPU: 214 PID: 6424 Comm: qemu-kvm Not tainted 5.4.183.stable #1 kernel: Hardware name: Oracle Corporation ORACLE SERVER E4-2c/Asm,MB Tray,2U,E4-2c, BIOS 78014000 01/05/2022 kernel: RIP: 0010:clflush_cache_range+0x35/0x40 kernel: Code: f0 0f b7 15 63 53 99 01 89 f6 48 89 d0 48 f7 d8 48 21 f8 48 01 f7 48 39 f8 73 0c 66 0f ae 38 48 01 d0 48 39 c7 77 f4 0f ae f0 <5d> c3 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 0f ae kernel: RSP: 0018:ffffacba5e98fc30 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 kernel: RAX: ffffa1193cfbc000 RBX: ffffacbba4701000 RCX: 0000000000000000 kernel: RDX: 0000000000000040 RSI: 0000000000001000 RDI: ffffa1193cfbc000 kernel: RBP: ffffacba5e98fc30 R08: ffffacba5f44aca0 R09: ffffacbac3701000 kernel: R10: 0000000000000080 R11: ffff9f8500000af0 R12: ffffa18074a22f80 kernel: R13: ffffacbaf6889dd8 R14: ffffacba5f41d960 R15: ffffacba5f44aca0 kernel: FS: 00007fbe04321f00(0000) GS:ffffa1814ed80000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 00007dfbebd7d000 CR3: 000801fb93d68002 CR4: 0000000000760ee0 kernel: PKRU: 55555554 kernel: Call Trace: kernel: sev_clflush_pages.part.56+0x50/0x70 [kvm_amd] kernel: svm_register_enc_region+0xe2/0x120 [kvm_amd] kernel: kvm_arch_vm_ioctl+0x524/0xbd0 [kvm] kernel: ? release_pages+0x212/0x430 kernel: ? __pagevec_lru_add_fn+0x192/0x2f0 kernel: kvm_vm_ioctl+0x9c/0x9d0 [kvm] kernel: ? __lru_cache_add+0x59/0x70 kernel: ? lru_cache_add_active_or_unevictable+0x39/0xb0 kernel: ? __handle_mm_fault+0xa74/0xfd0 kernel: ? __switch_to_asm+0x34/0x70 kernel: do_vfs_ioctl+0xa9/0x640 kernel: ? __audit_syscall_entry+0xdd/0x130 kernel: ksys_ioctl+0x67/0x90 kernel: __x64_sys_ioctl+0x1a/0x20 kernel: do_syscall_64+0x60/0x1d0 kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 kernel: RIP: 0033:0x7fbe0086563b kernel: Code: 0f 1e fa 48 8b 05 4d b8 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1d b8 2c 00 f7 d8 64 89 01 48 kernel: RSP: 002b:00007ffedf577418 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 kernel: RAX: ffffffffffffffda RBX: 00007dfbebe00000 RCX: 00007fbe0086563b kernel: RDX: 00007ffedf577490 RSI: ffffffff8010aebb RDI: 000000000000000d kernel: RBP: 000001c200000000 R08: ffffffffffffffff R09: ffffffffffffffff kernel: R10: ffffffffffffffff R11: 0000000000000246 R12: 000001c200000000 kernel: R13: 00007dfbebe00000 R14: 0000000000000000 R15: 0000000000000000
The problem is the time spent flushing the caches when pinning memory for SEV but it's unnecessary as it turns out - it is resolved by backporting the following commits from Linux 5.10
e1ebb2b49048 KVM: SVM: Don't flush cache if hardware enforces cache coherency across encryption domains (conflict due to the function it fixed being moved in a refactoring in 5.7).
along with 3 other commits needed as dependencies. (fbd5969d1ff2 avoids a conflict in 5866e9205b47)
fbd5969d1ff2 x86/cpufeatures: Mark two free bits in word 3 5866e9205b47 x86/cpu: Add hardware-enforced cache coherency as a CPUID feature 75d1cc0e05af x86/mm/pat: Don't flush cache if hardware enforces cache coherency across encryption domnains
Tested by creating various sized guests up to 1.8TB, with and without SEV enabled, running a few benchmarks and passing kvm-unit-tests.
Borislav Petkov (1): x86/cpufeatures: Mark two free bits in word 3
Krish Sadhukhan (3): x86/cpu: Add hardware-enforced cache coherency as a CPUID feature x86/mm/pat: Don't flush cache if hardware enforces cache coherency across encryption domnains KVM: SVM: Don't flush cache if hardware enforces cache coherency across encryption domains
arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kvm/svm.c | 3 ++- arch/x86/mm/pageattr.c | 2 +- 4 files changed, 6 insertions(+), 2 deletions(-)
From: Borislav Petkov bp@suse.de
commit fbd5969d1ff2598143d6a6fbc9491a9e40ab9b82 upstream.
... so that they get reused when needed.
No functional changes.
Signed-off-by: Borislav Petkov bp@suse.de Link: https://lkml.kernel.org/r/20200604104150.2056-1-bp@alien8.de Signed-off-by: Liam Merwick liam.merwick@oracle.com --- arch/x86/include/asm/cpufeatures.h | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index d912457f56a7..c934eb807f23 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -96,6 +96,7 @@ #define X86_FEATURE_SYSCALL32 ( 3*32+14) /* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */ +/* free ( 3*32+17) */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */ @@ -107,6 +108,7 @@ #define X86_FEATURE_EXTD_APICID ( 3*32+26) /* Extended APICID (8 bits) */ #define X86_FEATURE_AMD_DCM ( 3*32+27) /* AMD multi-node processor */ #define X86_FEATURE_APERFMPERF ( 3*32+28) /* P-State hardware coordination feedback capability (APERF/MPERF MSRs) */ +/* free ( 3*32+29) */ #define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */ #define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
From: Krish Sadhukhan krish.sadhukhan@oracle.com
commit 5866e9205b47a983a77ebc8654949f696342f2ab upstream.
In some hardware implementations, coherency between the encrypted and unencrypted mappings of the same physical page is enforced. In such a system, it is not required for software to flush the page from all CPU caches in the system prior to changing the value of the C-bit for a page. This hardware- enforced cache coherency is indicated by EAX[10] in CPUID leaf 0x8000001f.
[ bp: Use one of the free slots in word 3. ]
Suggested-by: Tom Lendacky thomas.lendacky@amd.com Signed-off-by: Krish Sadhukhan krish.sadhukhan@oracle.com Signed-off-by: Borislav Petkov bp@suse.de Link: https://lkml.kernel.org/r/20200917212038.5090-2-krish.sadhukhan@oracle.com Signed-off-by: Liam Merwick liam.merwick@oracle.com --- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index c934eb807f23..4705533eb88d 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -96,7 +96,7 @@ #define X86_FEATURE_SYSCALL32 ( 3*32+14) /* "" syscall in IA32 userspace */ #define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */ -/* free ( 3*32+17) */ +#define X86_FEATURE_SME_COHERENT ( 3*32+17) /* "" AMD hardware-enforced cache coherency */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" LFENCE synchronizes RDTSC */ #define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */ #define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */ diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c index adf9b71386ef..53004dbd55c4 100644 --- a/arch/x86/kernel/cpu/scattered.c +++ b/arch/x86/kernel/cpu/scattered.c @@ -41,6 +41,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_SME, CPUID_EAX, 0, 0x8000001f, 0 }, { X86_FEATURE_SEV, CPUID_EAX, 1, 0x8000001f, 0 }, + { X86_FEATURE_SME_COHERENT, CPUID_EAX, 10, 0x8000001f, 0 }, { 0, 0, 0, 0, 0 } };
From: Krish Sadhukhan krish.sadhukhan@oracle.com
commit 75d1cc0e05af579301ce4e49cf6399be4b4e6e76 upstream.
In some hardware implementations, coherency between the encrypted and unencrypted mappings of the same physical page is enforced. In such a system, it is not required for software to flush the page from all CPU caches in the system prior to changing the value of the C-bit for the page. So check that bit before flushing the cache.
[ bp: Massage commit message. ]
Suggested-by: Tom Lendacky thomas.lendacky@amd.com Signed-off-by: Krish Sadhukhan krish.sadhukhan@oracle.com Signed-off-by: Borislav Petkov bp@suse.de Link: https://lkml.kernel.org/r/20200917212038.5090-3-krish.sadhukhan@oracle.com Signed-off-by: Liam Merwick liam.merwick@oracle.com --- arch/x86/mm/pageattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 281e584cfe39..d61313f5c5b9 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -1967,7 +1967,7 @@ static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc) /* * Before changing the encryption attribute, we need to flush caches. */ - cpa_flush(&cpa, 1); + cpa_flush(&cpa, !this_cpu_has(X86_FEATURE_SME_COHERENT));
ret = __change_page_attr_set_clr(&cpa, 1);
From: Krish Sadhukhan krish.sadhukhan@oracle.com
commit e1ebb2b49048c4767cfa0d8466f9c701e549fa5e upstream.
In some hardware implementations, coherency between the encrypted and unencrypted mappings of the same physical page in a VM is enforced. In such a system, it is not required for software to flush the VM's page from all CPU caches in the system prior to changing the value of the C-bit for the page.
So check that bit before flushing the cache.
Signed-off-by: Krish Sadhukhan krish.sadhukhan@oracle.com Signed-off-by: Borislav Petkov bp@suse.de Acked-by: Paolo Bonzini pbonzini@redhat.com Link: https://lkml.kernel.org/r/20200917212038.5090-4-krish.sadhukhan@oracle.com Signed-off-by: Liam Merwick liam.merwick@oracle.com
Conflicts: arch/x86/kvm/svm/sev.c [ The linux-5.4.y stable branch does not have the Linux 5.7 refactoring commit eaf78265a4ab ("KVM: SVM: Move SEV code to separate file") so the change was manually applied to sev_clflush_pages() in arch/x86/kvm/svm.c. ] --- arch/x86/kvm/svm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 2f84509f2828..125970286f28 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1904,7 +1904,8 @@ static void sev_clflush_pages(struct page *pages[], unsigned long npages) uint8_t *page_virtual; unsigned long i;
- if (npages == 0 || pages == NULL) + if (this_cpu_has(X86_FEATURE_SME_COHERENT) || npages == 0 || + pages == NULL) return;
for (i = 0; i < npages; i++) {
On Fri, Mar 11, 2022 at 11:29:23AM +0000, Liam Merwick wrote:
[ patch series targeting linux-5.4.y stable branch. ]
Creating a SEV-enabled guest with 380GB or more of memory causes a cpu soft-hang in the host running 5.4 with the following stacktrace:
kernel: watchdog: BUG: soft lockup - CPU#214 stuck for 22s! [qemu-kvm:6424] ... kernel: CPU: 214 PID: 6424 Comm: qemu-kvm Not tainted 5.4.183.stable #1 kernel: Hardware name: Oracle Corporation ORACLE SERVER E4-2c/Asm,MB Tray,2U,E4-2c, BIOS 78014000 01/05/2022 kernel: RIP: 0010:clflush_cache_range+0x35/0x40 kernel: Code: f0 0f b7 15 63 53 99 01 89 f6 48 89 d0 48 f7 d8 48 21 f8 48 01 f7 48 39 f8 73 0c 66 0f ae 38 48 01 d0 48 39 c7 77 f4 0f ae f0 <5d> c3 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 0f ae kernel: RSP: 0018:ffffacba5e98fc30 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 kernel: RAX: ffffa1193cfbc000 RBX: ffffacbba4701000 RCX: 0000000000000000 kernel: RDX: 0000000000000040 RSI: 0000000000001000 RDI: ffffa1193cfbc000 kernel: RBP: ffffacba5e98fc30 R08: ffffacba5f44aca0 R09: ffffacbac3701000 kernel: R10: 0000000000000080 R11: ffff9f8500000af0 R12: ffffa18074a22f80 kernel: R13: ffffacbaf6889dd8 R14: ffffacba5f41d960 R15: ffffacba5f44aca0 kernel: FS: 00007fbe04321f00(0000) GS:ffffa1814ed80000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 00007dfbebd7d000 CR3: 000801fb93d68002 CR4: 0000000000760ee0 kernel: PKRU: 55555554 kernel: Call Trace: kernel: sev_clflush_pages.part.56+0x50/0x70 [kvm_amd] kernel: svm_register_enc_region+0xe2/0x120 [kvm_amd] kernel: kvm_arch_vm_ioctl+0x524/0xbd0 [kvm] kernel: ? release_pages+0x212/0x430 kernel: ? __pagevec_lru_add_fn+0x192/0x2f0 kernel: kvm_vm_ioctl+0x9c/0x9d0 [kvm] kernel: ? __lru_cache_add+0x59/0x70 kernel: ? lru_cache_add_active_or_unevictable+0x39/0xb0 kernel: ? __handle_mm_fault+0xa74/0xfd0 kernel: ? __switch_to_asm+0x34/0x70 kernel: do_vfs_ioctl+0xa9/0x640 kernel: ? __audit_syscall_entry+0xdd/0x130 kernel: ksys_ioctl+0x67/0x90 kernel: __x64_sys_ioctl+0x1a/0x20 kernel: do_syscall_64+0x60/0x1d0 kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 kernel: RIP: 0033:0x7fbe0086563b kernel: Code: 0f 1e fa 48 8b 05 4d b8 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1d b8 2c 00 f7 d8 64 89 01 48 kernel: RSP: 002b:00007ffedf577418 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 kernel: RAX: ffffffffffffffda RBX: 00007dfbebe00000 RCX: 00007fbe0086563b kernel: RDX: 00007ffedf577490 RSI: ffffffff8010aebb RDI: 000000000000000d kernel: RBP: 000001c200000000 R08: ffffffffffffffff R09: ffffffffffffffff kernel: R10: ffffffffffffffff R11: 0000000000000246 R12: 000001c200000000 kernel: R13: 00007dfbebe00000 R14: 0000000000000000 R15: 0000000000000000
The problem is the time spent flushing the caches when pinning memory for SEV but it's unnecessary as it turns out - it is resolved by backporting the following commits from Linux 5.10
e1ebb2b49048 KVM: SVM: Don't flush cache if hardware enforces cache coherency across encryption domains (conflict due to the function it fixed being moved in a refactoring in 5.7).
along with 3 other commits needed as dependencies. (fbd5969d1ff2 avoids a conflict in 5866e9205b47)
fbd5969d1ff2 x86/cpufeatures: Mark two free bits in word 3 5866e9205b47 x86/cpu: Add hardware-enforced cache coherency as a CPUID feature 75d1cc0e05af x86/mm/pat: Don't flush cache if hardware enforces cache coherency across encryption domnains
Tested by creating various sized guests up to 1.8TB, with and without SEV enabled, running a few benchmarks and passing kvm-unit-tests.
Borislav Petkov (1): x86/cpufeatures: Mark two free bits in word 3
Krish Sadhukhan (3): x86/cpu: Add hardware-enforced cache coherency as a CPUID feature x86/mm/pat: Don't flush cache if hardware enforces cache coherency across encryption domnains KVM: SVM: Don't flush cache if hardware enforces cache coherency across encryption domains
arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kvm/svm.c | 3 ++- arch/x86/mm/pageattr.c | 2 +- 4 files changed, 6 insertions(+), 2 deletions(-)
-- 2.27.0
All now queued up, thanks!
greg k-h
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Liam Merwick