November 2022 - Linux-stable-mirror

[PATCH] crypto: avoid unnecessary work when self-tests are disabled

by Eric Biggers

From: Eric Biggers <ebiggers(a)google.com> Currently, registering an algorithm with the crypto API always causes a notification to be posted to the "cryptomgr", which then creates a kthread to self-test the algorithm. However, if self-tests are disabled in the kconfig (as is the default option), then this kthread just notifies waiters that the algorithm has been tested, then exits. This causes a significant amount of overhead, especially in the kthread creation and destruction, which is not necessary at all. For example, in a quick test I found that booting a "minimum" x86_64 kernel with all the crypto options enabled (except for the self-tests) takes about 400ms until PID 1 can start. Of that, a full 13ms is spent just doing this pointless dance, involving a kthread being created, run, and destroyed over 200 times. That's over 3% of the entire kernel start time. Fix this by just skipping the creation of the test larval and the posting of the registration notification entirely, when self-tests are disabled. Also compile out the unnecessary code in algboss.c. While this patch is an optimization and not a "fix" per se, I've marked it as for stable, due to the large improvement it can make to boot time. Cc: stable(a)vger.kernel.org Signed-off-by: Eric Biggers <ebiggers(a)google.com> --- crypto/algapi.c | 3 ++- crypto/algboss.c | 11 +++++++---- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/crypto/algapi.c b/crypto/algapi.c index 5c69ff8e8fa5c..018935cb8417d 100644 --- a/crypto/algapi.c +++ b/crypto/algapi.c @@ -226,7 +226,8 @@ static struct crypto_larval *crypto_alloc_test_larval(struct crypto_alg *alg) { struct crypto_larval *larval; - if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER)) + if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER) || + IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS)) return NULL; larval = crypto_larval_alloc(alg->cra_name, diff --git a/crypto/algboss.c b/crypto/algboss.c index eb5fe84efb83e..e6f0443d08048 100644 --- a/crypto/algboss.c +++ b/crypto/algboss.c @@ -171,16 +171,18 @@ static int cryptomgr_schedule_probe(struct crypto_larval *larval) return NOTIFY_OK; } +#ifdef CONFIG_CRYPTO_MANAGER_DISABLE_TESTS +static int cryptomgr_schedule_test(struct crypto_alg *alg) +{ + return 0; +} +#else static int cryptomgr_test(void *data) { struct crypto_test_param *param = data; u32 type = param->type; int err = 0; -#ifdef CONFIG_CRYPTO_MANAGER_DISABLE_TESTS - goto skiptest; -#endif - if (type & CRYPTO_ALG_TESTED) goto skiptest; @@ -229,6 +231,7 @@ static int cryptomgr_schedule_test(struct crypto_alg *alg) err: return NOTIFY_OK; } +#endif /* !CONFIG_CRYPTO_MANAGER_DISABLE_TESTS */ static int cryptomgr_notify(struct notifier_block *this, unsigned long msg, void *data) base-commit: f67dd6ce0723ad013395f20a3f79d8a437d3f455 -- 2.38.1

3 years, 1 month

2
3
0 0

Greg KH on FLOSS Weekly this week

by Luna Jernberg

https://www.youtube.com/watch?v=jlqKdW-B4Ug *watching now*

3 years, 1 month

1
0
0 0

[PATCH 4.9 00/30] 4.9.333-rc1 review

by Greg Kroah-Hartman

This is the start of the stable review cycle for the 4.9.333 release. There are 30 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know. Responses should be made by Thu, 10 Nov 2022 13:33:17 +0000. Anything received after that time might be too late. The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.9.333-rc… or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-4.9.y and the diffstat can be found below. thanks, greg k-h ------------- Pseudo-Shortlog of commits: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Linux 4.9.333-rc1 Dokyung Song <dokyung.song(a)gmail.com> wifi: brcmfmac: Fix potential buffer overflow in brcmf_fweh_event_worker() Maxim Levitsky <mlevitsk(a)redhat.com> KVM: x86: emulator: update the emulation mode after CR0 write Maxim Levitsky <mlevitsk(a)redhat.com> KVM: x86: emulator: introduce emulator_recalc_and_set_mode Maxim Levitsky <mlevitsk(a)redhat.com> KVM: x86: emulator: em_sysexit should update ctxt->mode Jim Mattson <jmattson(a)google.com> KVM: x86: Mask off reserved bits in CPUID.80000008H Ye Bin <yebin10(a)huawei.com> ext4: fix warning in 'ext4_da_release_space' Helge Deller <deller(a)gmx.de> parisc: Export iosapic_serial_irq() symbol for serial port driver Helge Deller <deller(a)gmx.de> parisc: Make 8250_gsc driver dependend on CONFIG_PARISC John Veness <john-linux(a)pelago.org.uk> ALSA: usb-audio: Add quirks for MacroSilicon MS2100/MS2106 devices David Sterba <dsterba(a)suse.com> btrfs: fix type of parameter generation in btrfs_get_dentry Luiz Augusto von Dentz <luiz.von.dentz(a)intel.com> Bluetooth: L2CAP: Fix attempting to access uninitialized memory Martin Tůma <martin.tuma(a)digiteqautomotive.com> i2c: xiic: Add platform module alias Hans Verkuil <hverkuil-cisco(a)xs4all.nl> media: dvb-frontends/drxk: initialize err to 0 Hans Verkuil <hverkuil-cisco(a)xs4all.nl> media: s5p_cec: limit msg.len to CEC_MAX_MSG_SIZE Gaosheng Cui <cuigaosheng1(a)huawei.com> net: mdio: fix undefined behavior in bit shift for __mdiobus_register Zhengchao Shao <shaozhengchao(a)huawei.com> Bluetooth: L2CAP: fix use-after-free in l2cap_conn_del() Maxim Mikityanskiy <maxtram95(a)gmail.com> Bluetooth: L2CAP: Fix use-after-free caused by l2cap_reassemble_sdu Filipe Manana <fdmanana(a)suse.com> btrfs: fix ulist leaks in error paths of qgroup self tests Yang Yingliang <yangyingliang(a)huawei.com> isdn: mISDN: netjet: fix wrong check of device registration Yang Yingliang <yangyingliang(a)huawei.com> mISDN: fix possible memory leak in mISDN_register_device() Zhang Qilong <zhangqilong3(a)huawei.com> rose: Fix NULL pointer dereference in rose_send_frame() Jason A. Donenfeld <Jason(a)zx2c4.com> ipvs: use explicitly signed chars Dan Carpenter <dan.carpenter(a)oracle.com> net: sched: Fix use after free in red_enqueue() Sergey Shtylyov <s.shtylyov(a)omp.ru> ata: pata_legacy: fix pdc20230_set_piomode() Zhang Changzhong <zhangchangzhong(a)huawei.com> net: fec: fix improper use of NETDEV_TX_BUSY Shang XiaoJing <shangxiaojing(a)huawei.com> nfc: nfcmrvl: Fix potential memory leak in nfcmrvl_i2c_nci_send() Shang XiaoJing <shangxiaojing(a)huawei.com> nfc: s3fwrn5: Fix potential memory leak in s3fwrn5_nci_send() Zhang Xiaoxu <zhangxiaoxu5(a)huawei.com> nfs4: Fix kmemleak when allocate slot failed Trond Myklebust <trond.myklebust(a)hammerspace.com> NFSv4.1: We must always send RECLAIM_COMPLETE after a reboot Trond Myklebust <trond.myklebust(a)hammerspace.com> NFSv4.1: Handle RECLAIM_COMPLETE trunking errors ------------- Diffstat: Makefile | 4 +- arch/x86/kvm/cpuid.c | 1 + arch/x86/kvm/emulate.c | 102 +++++++++++++++------ drivers/ata/pata_legacy.c | 5 +- drivers/i2c/busses/i2c-xiic.c | 1 + drivers/isdn/hardware/mISDN/netjet.c | 2 +- drivers/isdn/mISDN/core.c | 5 +- drivers/media/dvb-frontends/drxk_hard.c | 2 +- drivers/net/ethernet/freescale/fec_main.c | 4 +- drivers/net/phy/mdio_bus.c | 2 +- .../wireless/broadcom/brcm80211/brcmfmac/fweh.c | 4 + drivers/nfc/nfcmrvl/i2c.c | 7 +- drivers/nfc/s3fwrn5/core.c | 8 +- drivers/parisc/iosapic.c | 1 + drivers/staging/media/s5p-cec/s5p_cec.c | 2 + drivers/tty/serial/8250/Kconfig | 2 +- fs/btrfs/export.c | 2 +- fs/btrfs/export.h | 2 +- fs/btrfs/tests/qgroup-tests.c | 20 +++- fs/ext4/migrate.c | 3 +- fs/nfs/nfs4client.c | 1 + fs/nfs/nfs4state.c | 2 + net/bluetooth/l2cap_core.c | 52 +++++++++-- net/netfilter/ipvs/ip_vs_conn.c | 4 +- net/rose/rose_link.c | 3 + net/sched/sch_red.c | 4 +- sound/usb/quirks-table.h | 58 ++++++++++++ sound/usb/quirks.c | 1 + 28 files changed, 241 insertions(+), 63 deletions(-)

3 years, 1 month

6
35
0 0

[PATCH v3] ceph: fix NULL pointer dereference for req->r_session

by xiubli＠redhat.com

From: Xiubo Li <xiubli(a)redhat.com> The request's r_session maybe changed when it was forwarded or resent. Cc: stable(a)vger.kernel.org URL: https://bugzilla.redhat.com/show_bug.cgi?id=2137955 Signed-off-by: Xiubo Li <xiubli(a)redhat.com> --- fs/ceph/caps.c | 60 ++++++++++++++++---------------------------------- 1 file changed, 19 insertions(+), 41 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 894adfb4a092..83f9e18e3169 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -2297,8 +2297,10 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) struct ceph_mds_client *mdsc = ceph_sb_to_client(inode->i_sb)->mdsc; struct ceph_inode_info *ci = ceph_inode(inode); struct ceph_mds_request *req1 = NULL, *req2 = NULL; + struct ceph_mds_session **sessions = NULL; + struct ceph_mds_session *s; unsigned int max_sessions; - int ret, err = 0; + int i, ret, err = 0; spin_lock(&ci->i_unsafe_lock); if (S_ISDIR(inode->i_mode) && !list_empty(&ci->i_unsafe_dirops)) { @@ -2315,28 +2317,19 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) } spin_unlock(&ci->i_unsafe_lock); - /* - * The mdsc->max_sessions is unlikely to be changed - * mostly, here we will retry it by reallocating the - * sessions array memory to get rid of the mdsc->mutex - * lock. - */ -retry: - max_sessions = mdsc->max_sessions; - /* * Trigger to flush the journal logs in all the relevant MDSes * manually, or in the worst case we must wait at most 5 seconds * to wait the journal logs to be flushed by the MDSes periodically. */ - if ((req1 || req2) && likely(max_sessions)) { - struct ceph_mds_session **sessions = NULL; - struct ceph_mds_session *s; + mutex_lock(&mdsc->mutex); + max_sessions = mdsc->max_sessions; + if (req1 || req2) { struct ceph_mds_request *req; - int i; sessions = kcalloc(max_sessions, sizeof(s), GFP_KERNEL); if (!sessions) { + mutex_unlock(&mdsc->mutex); err = -ENOMEM; goto out; } @@ -2346,18 +2339,8 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) list_for_each_entry(req, &ci->i_unsafe_dirops, r_unsafe_dir_item) { s = req->r_session; - if (!s) + if (!s || unlikely(s->s_mds >= max_sessions)) continue; - if (unlikely(s->s_mds >= max_sessions)) { - spin_unlock(&ci->i_unsafe_lock); - for (i = 0; i < max_sessions; i++) { - s = sessions[i]; - if (s) - ceph_put_mds_session(s); - } - kfree(sessions); - goto retry; - } if (!sessions[s->s_mds]) { s = ceph_get_mds_session(s); sessions[s->s_mds] = s; @@ -2368,18 +2351,8 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) list_for_each_entry(req, &ci->i_unsafe_iops, r_unsafe_target_item) { s = req->r_session; - if (!s) + if (!s || unlikely(s->s_mds >= max_sessions)) continue; - if (unlikely(s->s_mds >= max_sessions)) { - spin_unlock(&ci->i_unsafe_lock); - for (i = 0; i < max_sessions; i++) { - s = sessions[i]; - if (s) - ceph_put_mds_session(s); - } - kfree(sessions); - goto retry; - } if (!sessions[s->s_mds]) { s = ceph_get_mds_session(s); sessions[s->s_mds] = s; @@ -2391,13 +2364,18 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) /* the auth MDS */ spin_lock(&ci->i_ceph_lock); if (ci->i_auth_cap) { - s = ci->i_auth_cap->session; - if (!sessions[s->s_mds]) - sessions[s->s_mds] = ceph_get_mds_session(s); + s = ci->i_auth_cap->session; + if (likely(s->s_mds < max_sessions) + && !sessions[s->s_mds]) { + sessions[s->s_mds] = ceph_get_mds_session(s); + } } spin_unlock(&ci->i_ceph_lock); + } + mutex_unlock(&mdsc->mutex); - /* send flush mdlog request to MDSes */ + /* send flush mdlog request to MDSes */ + if (sessions) { for (i = 0; i < max_sessions; i++) { s = sessions[i]; if (s) { @@ -2405,7 +2383,6 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) ceph_put_mds_session(s); } } - kfree(sessions); } dout("%s %p wait on tid %llu %llu\n", __func__, @@ -2424,6 +2401,7 @@ static int flush_mdlog_and_wait_inode_unsafe_requests(struct inode *inode) } out: + kfree(sessions); if (req1) ceph_mdsc_put_request(req1); if (req2) -- 2.31.1

3 years, 1 month

3
4
0 0

[PATCH] PCI/sysfs: Fix double free in error path

by Sascha Hauer

When pci_create_attr() fails then pci_remove_resource_files() is called which will iterate over the res_attr[_wc] arrays and frees every non NULL entry. To avoid a double free here we have to set the failed entry to NULL in pci_create_attr() when freeing it. Fixes: b562ec8f74e4 ("PCI: Don't leak memory if sysfs_create_bin_file() fails") Signed-off-by: Sascha Hauer <s.hauer(a)pengutronix.de> Cc: <stable(a)vger.kernel.org> --- drivers/pci/pci-sysfs.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index fc804e08e3cb5..a07381d46ddae 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -1196,8 +1196,13 @@ static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine) res_attr->size = pci_resource_len(pdev, num); res_attr->private = (void *)(unsigned long)num; retval = sysfs_create_bin_file(&pdev->dev.kobj, res_attr); - if (retval) + if (retval) { + if (write_combine) + pdev->res_attr_wc[num] = NULL; + else + pdev->res_attr[num] = NULL; kfree(res_attr); + } return retval; } -- 2.30.2

3 years, 1 month

2
3
0 0

[PATCH 7/8] KVM: SVM: move MSR_IA32_SPEC_CTRL save/restore to assembly

by Paolo Bonzini

Restoration of the host IA32_SPEC_CTRL value is probably too late with respect to the return thunk training sequence. With respect to the user/kernel boundary, AMD says, "If software chooses to toggle STIBP (e.g., set STIBP on kernel entry, and clear it on kernel exit), software should set STIBP to 1 before executing the return thunk training sequence." I assume the same requirements apply to the guest/host boundary. The return thunk training sequence is in vmenter.S, quite close to the VM-exit. On hosts without V_SPEC_CTRL, however, the host's IA32_SPEC_CTRL value is not restored until much later. To avoid this, move the restoration of host SPEC_CTRL to assembly and, for consistency, move the restoration of the guest SPEC_CTRL as well. This is not particularly difficult, apart from some care to cover both 32- and 64-bit, and to share code between SEV-ES and normal vmentry. Cc: stable(a)vger.kernel.org Fixes: a149180fbcf3 ("x86: Add magic AMD return-thunk") Suggested-by: Jim Mattson <jmattson(a)google.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- arch/x86/kernel/asm-offsets.c | 1 + arch/x86/kernel/cpu/bugs.c | 13 ++--- arch/x86/kvm/svm/svm.c | 38 ++++++--------- arch/x86/kvm/svm/svm.h | 4 +- arch/x86/kvm/svm/vmenter.S | 92 ++++++++++++++++++++++++++++++++++- 5 files changed, 111 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index 69d1fed51086..d0bd68af0a5a 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -115,6 +115,7 @@ static void __used common(void) OFFSET(SVM_vcpu_arch_regs, vcpu_svm, vcpu.arch.regs); OFFSET(SVM_vmcb01, vcpu_svm, vmcb01); OFFSET(SVM_current_vmcb, vcpu_svm, current_vmcb); + OFFSET(SVM_spec_ctrl, vcpu_svm, spec_ctrl); OFFSET(KVM_VMCB_pa, kvm_vmcb_info, pa); } diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index da7c361f47e0..6ec0b7ce7453 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -196,22 +196,15 @@ void __init check_bugs(void) } /* - * NOTE: This function is *only* called for SVM. VMX spec_ctrl handling is - * done in vmenter.S. + * NOTE: This function is *only* called for SVM, since Intel uses + * MSR_IA32_SPEC_CTRL for SSBD. */ void x86_virt_spec_ctrl(u64 guest_spec_ctrl, u64 guest_virt_spec_ctrl, bool setguest) { - u64 msrval, guestval = guest_spec_ctrl, hostval = spec_ctrl_current(); + u64 guestval, hostval; struct thread_info *ti = current_thread_info(); - if (static_cpu_has(X86_FEATURE_MSR_SPEC_CTRL)) { - if (hostval != guestval) { - msrval = setguest ? guestval : hostval; - wrmsrl(MSR_IA32_SPEC_CTRL, msrval); - } - } - /* * If SSBD is not handled in MSR_SPEC_CTRL on AMD, update * MSR_AMD64_L2_CFG or MSR_VIRT_SPEC_CTRL if supported. diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 381c7dcffe25..31aa158a2e10 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -731,6 +731,15 @@ static bool msr_write_intercepted(struct kvm_vcpu *vcpu, u32 msr) u32 offset; u32 *msrpm; + /* + * For non-nested case: + * If the L01 MSR bitmap does not intercept the MSR, then we need to + * save it. + * + * For nested case: + * If the L02 MSR bitmap does not intercept the MSR, then we need to + * save it. + */ msrpm = is_guest_mode(vcpu) ? to_svm(vcpu)->nested.msrpm: to_svm(vcpu)->msrpm; @@ -3912,18 +3921,19 @@ static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) return EXIT_FASTPATH_NONE; } -static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) +static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, bool spec_ctrl_intercepted) { struct vcpu_svm *svm = to_svm(vcpu); guest_state_enter_irqoff(); if (sev_es_guest(vcpu->kvm)) { - __svm_sev_es_vcpu_run(svm); + __svm_sev_es_vcpu_run(svm, spec_ctrl_intercepted); } else { struct svm_cpu_data *sd = per_cpu(svm_data, vcpu->cpu); - __svm_vcpu_run(svm, __sme_page_pa(sd->save_area)); + __svm_vcpu_run(svm, __sme_page_pa(sd->save_area), + spec_ctrl_intercepted); } guest_state_exit_irqoff(); @@ -3932,6 +3942,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); + bool spec_ctrl_intercepted = msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL); trace_kvm_entry(vcpu); @@ -3990,26 +4001,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL)) x86_spec_ctrl_set_guest(svm->spec_ctrl, svm->virt_spec_ctrl); - svm_vcpu_enter_exit(vcpu); - - /* - * We do not use IBRS in the kernel. If this vCPU has used the - * SPEC_CTRL MSR it may have left it on; save the value and - * turn it off. This is much more efficient than blindly adding - * it to the atomic save/restore list. Especially as the former - * (Saving guest MSRs on vmexit) doesn't even exist in KVM. - * - * For non-nested case: - * If the L01 MSR bitmap does not intercept the MSR, then we need to - * save it. - * - * For nested case: - * If the L02 MSR bitmap does not intercept the MSR, then we need to - * save it. - */ - if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL) && - unlikely(!msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL))) - svm->spec_ctrl = native_read_msr(MSR_IA32_SPEC_CTRL); + svm_vcpu_enter_exit(vcpu, spec_ctrl_intercepted); if (!sev_es_guest(vcpu->kvm)) reload_tss(vcpu); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 99410651f2a5..9d940d8736f0 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -483,7 +483,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm); /* vmenter.S */ -void __svm_sev_es_vcpu_run(struct vcpu_svm *svm); -void __svm_vcpu_run(struct vcpu_svm *svm, unsigned long hsave_pa); +void __svm_sev_es_vcpu_run(struct vcpu_svm *svm, bool spec_ctrl_intercepted); +void __svm_vcpu_run(struct vcpu_svm *svm, unsigned long hsave_pa, bool spec_ctrl_intercepted); #endif diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index 45a4bd002494..9e381386ffdc 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -32,10 +32,64 @@ .section .noinstr.text, "ax" +.macro RESTORE_GUEST_SPEC_CTRL + /* No need to do anything if SPEC_CTRL is unset or V_SPEC_CTRL is set */ + ALTERNATIVE_2 "jmp 999f", \ + "", X86_FEATURE_MSR_SPEC_CTRL, \ + "jmp 999f", X86_FEATURE_V_SPEC_CTRL + + /* + * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the + * host's, write the MSR. + * + * IMPORTANT: To avoid RSB underflow attacks and any other nastiness, + * there must not be any returns or indirect branches between this code + * and vmentry. + */ + movl SVM_spec_ctrl(%_ASM_DI), %eax + cmp PER_CPU_VAR(x86_spec_ctrl_current), %eax + je 999f + mov $MSR_IA32_SPEC_CTRL, %ecx + xor %edx, %edx + wrmsr +999: + +.endm + +.macro RESTORE_HOST_SPEC_CTRL + /* No need to do anything if SPEC_CTRL is unset or V_SPEC_CTRL is set */ + ALTERNATIVE_2 "jmp 999f", \ + "", X86_FEATURE_MSR_SPEC_CTRL, \ + "jmp 999f", X86_FEATURE_V_SPEC_CTRL + + mov $MSR_IA32_SPEC_CTRL, %ecx + + /* + * Load the value that the guest had written into MSR_IA32_SPEC_CTRL, + * if it was not intercepted during guest execution. + */ + cmpb $0, (%_ASM_SP) + jnz 998f + rdmsr + movl %eax, SVM_spec_ctrl(%_ASM_DI) +998: + + /* Now restore the host value of the MSR if different from the guest's. */ + movl PER_CPU_VAR(x86_spec_ctrl_current), %eax + cmp SVM_spec_ctrl(%_ASM_DI), %eax + je 999f + xor %edx, %edx + wrmsr +999: + +.endm + + /** * __svm_vcpu_run - Run a vCPU via a transition to SVM guest mode * @svm: struct vcpu_svm * * @hsave_pa: unsigned long + * @spec_ctrl_intercepted: bool */ SYM_FUNC_START(__svm_vcpu_run) push %_ASM_BP @@ -50,7 +104,12 @@ SYM_FUNC_START(__svm_vcpu_run) #endif push %_ASM_BX - /* @hsave_pa is needed last after vmexit, save it first. */ + /* + * Both @spec_ctrl_intercepted and @hsave_pa are used only after vmexit. + * @spec_ctrl_intercepted is needed later and accessed directly from + * the stack in RESTORE_HOST_SPEC_CTRL, so save it first. + */ + push %_ASM_ARG3 push %_ASM_ARG2 /* Save @svm. */ @@ -61,6 +120,8 @@ SYM_FUNC_START(__svm_vcpu_run) mov %_ASM_ARG1, %_ASM_DI .endif + RESTORE_GUEST_SPEC_CTRL + /* * Use a single vmcb (vmcb01 because it's always valid) for * context switching guest state via VMLOAD/VMSAVE, that way @@ -138,6 +199,8 @@ SYM_FUNC_START(__svm_vcpu_run) FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE #endif + RESTORE_HOST_SPEC_CTRL + /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be * untrained as soon as we exit the VM and are back to the @@ -173,6 +236,9 @@ SYM_FUNC_START(__svm_vcpu_run) xor %r15d, %r15d #endif + /* "Pop" @spec_ctrl_intercepted. */ + pop %_ASM_BX + pop %_ASM_BX #ifdef CONFIG_X86_64 @@ -210,6 +276,7 @@ SYM_FUNC_END(__svm_vcpu_run) /** * __svm_sev_es_vcpu_run - Run a SEV-ES vCPU via a transition to SVM guest mode * @svm: struct vcpu_svm * + * @spec_ctrl_intercepted: bool */ SYM_FUNC_START(__svm_sev_es_vcpu_run) push %_ASM_BP @@ -224,8 +291,21 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) #endif push %_ASM_BX + /* Save @spec_ctrl_intercepted for RESTORE_HOST_SPEC_CTRL. */ + push %_ASM_ARG2 + + /* Save @svm. */ + push %_ASM_ARG1 + +.ifnc _ASM_ARG1, _ASM_DI + /* Move @svm to RDI for RESTORE_GUEST_SPEC_CTRL. */ + mov %_ASM_ARG1, %_ASM_DI +.endif + + RESTORE_GUEST_SPEC_CTRL + /* Get svm->current_vmcb->pa into RAX. */ - mov SVM_current_vmcb(%_ASM_ARG1), %_ASM_AX + mov SVM_current_vmcb(%_ASM_DI), %_ASM_AX mov KVM_VMCB_pa(%_ASM_AX), %_ASM_AX /* Enter guest mode */ @@ -235,11 +315,16 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) 2: cli + /* Pop @svm to RDI, guest registers have been saved already. */ + pop %_ASM_DI + #ifdef CONFIG_RETPOLINE /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE #endif + RESTORE_HOST_SPEC_CTRL + /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be * untrained as soon as we exit the VM and are back to the @@ -249,6 +334,9 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) */ UNTRAIN_RET + /* "Pop" @spec_ctrl_intercepted. */ + pop %_ASM_BX + pop %_ASM_BX #ifdef CONFIG_X86_64 -- 2.31.1

3 years, 1 month

2
3
0 0

stable-rc/queue/4.14 baseline: 79 runs, 10 regressions (v4.14.298-40-g486f6eb3468d)

by kernelci.org bot

stable-rc/queue/4.14 baseline: 79 runs, 10 regressions (v4.14.298-40-g486f6eb3468d) Regressions Summary ------------------- platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ meson-gxl-s905x-khadas-vim | arm64 | lab-baylibre | gcc-10 | defconfig | 1 meson-gxl-s905x-libretech-cc | arm64 | lab-broonie | gcc-10 | defconfig | 1 qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig | 1 qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1 qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig | 1 qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/job/stable-rc/branch/queue%2F4.14/kernel/v4.14.29… Test: baseline Tree: stable-rc Branch: queue/4.14 Describe: v4.14.298-40-g486f6eb3468d URL: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git SHA: 486f6eb3468dda8e59cfb63a6f63b5b3abf6b4cf Test Regressions ---------------- platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ meson-gxl-s905x-khadas-vim | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8d4cf46b2fc285e7db52 Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d4cf46b2fc285e7db53 failing since 127 days (last pass: v4.14.285-35-g61a723f50c9f, first fail: v4.14.285-46-ga87318551bac) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ meson-gxl-s905x-libretech-cc | arm64 | lab-broonie | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8dd74c677f55f7e7db7b Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8dd74c677f55f7e7db7c failing since 204 days (last pass: v4.14.275-277-gda5c0b6bebbb1, first fail: v4.14.275-284-gdf8ec5b4383b9) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 Details: https://kernelci.org/test/plan/id/636b8d128698ca05ebe7db4f Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig+arm64-chromebook Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d128698ca05ebe7db50 failing since 183 days (last pass: v4.14.277-54-gf277f09f64f4, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv2 | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8d654d6ea3f2e2e7db83 Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d654d6ea3f2e2e7db84 failing since 183 days (last pass: v4.14.277-54-gfa6de16ffc4e, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 Details: https://kernelci.org/test/plan/id/636b8d10a3f7d6f9a2e7db8d Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig+arm64-chromebook Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d10a3f7d6f9a2e7db8e failing since 183 days (last pass: v4.14.277-54-gf277f09f64f4, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv2-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8d614d6ea3f2e2e7db7a Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d614d6ea3f2e2e7db7b failing since 183 days (last pass: v4.14.277-54-gfa6de16ffc4e, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 Details: https://kernelci.org/test/plan/id/636b8d13960fb836bfe7db61 Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig+arm64-chromebook Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d13960fb836bfe7db62 failing since 183 days (last pass: v4.14.277-54-gf277f09f64f4, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv3 | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8d624d6ea3f2e2e7db7d Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d624d6ea3f2e2e7db7e failing since 183 days (last pass: v4.14.277-54-gfa6de16ffc4e, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig+arm64-chromebook | 1 Details: https://kernelci.org/test/plan/id/636b8d11960fb836bfe7db5b Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig+arm64-chromebook Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d11960fb836bfe7db5c failing since 183 days (last pass: v4.14.277-54-gf277f09f64f4, first fail: v4.14.277-75-g7a298ff98d4a) platform | arch | lab | compiler | defconfig | regressions -----------------------------+-------+--------------+----------+----------------------------+------------ qemu_arm64-virt-gicv3-uefi | arm64 | lab-baylibre | gcc-10 | defconfig | 1 Details: https://kernelci.org/test/plan/id/636b8d634d6ea3f2e2e7db80 Results: 0 PASS, 1 FAIL, 0 SKIP Full config: defconfig Compiler: gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110) Plain log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… HTML log: https://storage.kernelci.org//stable-rc/queue-4.14/v4.14.298-40-g486f6eb346… Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/2022… * baseline.login: https://kernelci.org/test/case/id/636b8d634d6ea3f2e2e7db81 failing since 183 days (last pass: v4.14.277-54-gfa6de16ffc4e, first fail: v4.14.277-75-g7a298ff98d4a)

3 years, 1 month

1
0
0 0

[PATCH 03/11] KVM: SVM: adjust register allocation for __svm_vcpu_run

by Paolo Bonzini

In preparation for moving vmload/vmsave to __svm_vcpu_run, keep the pointer to the struct vcpu_svm in %rdi. This way it is possible to load svm->vmcb01.pa in %rax without clobbering the pointer to svm itself. No functional change intended. Cc: stable(a)vger.kernel.org Fixes: a149180fbcf3 ("x86: Add magic AMD return-thunk") Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- arch/x86/kvm/svm/vmenter.S | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index f0ff41103e4c..531510ab6072 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -54,29 +54,29 @@ SYM_FUNC_START(__svm_vcpu_run) /* Save @vmcb. */ push %_ASM_ARG1 - /* Move @svm to RAX. */ - mov %_ASM_ARG2, %_ASM_AX + /* Move @svm to RDI. */ + mov %_ASM_ARG2, %_ASM_DI + + /* "POP" @vmcb to RAX. */ + pop %_ASM_AX /* Load guest registers. */ - mov VCPU_RCX(%_ASM_AX), %_ASM_CX - mov VCPU_RDX(%_ASM_AX), %_ASM_DX - mov VCPU_RBX(%_ASM_AX), %_ASM_BX - mov VCPU_RBP(%_ASM_AX), %_ASM_BP - mov VCPU_RSI(%_ASM_AX), %_ASM_SI - mov VCPU_RDI(%_ASM_AX), %_ASM_DI + mov VCPU_RCX(%_ASM_DI), %_ASM_CX + mov VCPU_RDX(%_ASM_DI), %_ASM_DX + mov VCPU_RBX(%_ASM_DI), %_ASM_BX + mov VCPU_RBP(%_ASM_DI), %_ASM_BP + mov VCPU_RSI(%_ASM_DI), %_ASM_SI #ifdef CONFIG_X86_64 - mov VCPU_R8 (%_ASM_AX), %r8 - mov VCPU_R9 (%_ASM_AX), %r9 - mov VCPU_R10(%_ASM_AX), %r10 - mov VCPU_R11(%_ASM_AX), %r11 - mov VCPU_R12(%_ASM_AX), %r12 - mov VCPU_R13(%_ASM_AX), %r13 - mov VCPU_R14(%_ASM_AX), %r14 - mov VCPU_R15(%_ASM_AX), %r15 + mov VCPU_R8 (%_ASM_DI), %r8 + mov VCPU_R9 (%_ASM_DI), %r9 + mov VCPU_R10(%_ASM_DI), %r10 + mov VCPU_R11(%_ASM_DI), %r11 + mov VCPU_R12(%_ASM_DI), %r12 + mov VCPU_R13(%_ASM_DI), %r13 + mov VCPU_R14(%_ASM_DI), %r14 + mov VCPU_R15(%_ASM_DI), %r15 #endif - - /* "POP" @vmcb to RAX. */ - pop %_ASM_AX + mov VCPU_RDI(%_ASM_DI), %_ASM_DI /* Enter guest mode */ sti -- 2.31.1

3 years, 1 month

2
2
0 0

[PATCH 10/11] KVM: SVM: move MSR_IA32_SPEC_CTRL save/restore to assembly

by Paolo Bonzini

Restoration of the host IA32_SPEC_CTRL value is probably too late with respect to the return thunk training sequence. With respect to the user/kernel boundary, AMD says, "If software chooses to toggle STIBP (e.g., set STIBP on kernel entry, and clear it on kernel exit), software should set STIBP to 1 before executing the return thunk training sequence." I assume the same requirements apply to the guest/host boundary. The return thunk training sequence is in vmenter.S, quite close to the VM-exit. On hosts without V_SPEC_CTRL, however, the host's IA32_SPEC_CTRL value is not restored until much later. To avoid this, move the restoration of host SPEC_CTRL to assembly and, for consistency, move the restoration of the guest SPEC_CTRL as well. This is not particularly difficult, apart from some care to cover both 32- and 64-bit, and to share code between SEV-ES and normal vmentry. Cc: stable(a)vger.kernel.org Fixes: a149180fbcf3 ("x86: Add magic AMD return-thunk") Suggested-by: Jim Mattson <jmattson(a)google.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- arch/x86/kernel/cpu/bugs.c | 13 +--- arch/x86/kvm/kvm-asm-offsets.c | 1 + arch/x86/kvm/svm/svm.c | 37 ++++------ arch/x86/kvm/svm/svm.h | 4 +- arch/x86/kvm/svm/vmenter.S | 119 ++++++++++++++++++++++++++++++++- 5 files changed, 136 insertions(+), 38 deletions(-) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index da7c361f47e0..6ec0b7ce7453 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -196,22 +196,15 @@ void __init check_bugs(void) } /* - * NOTE: This function is *only* called for SVM. VMX spec_ctrl handling is - * done in vmenter.S. + * NOTE: This function is *only* called for SVM, since Intel uses + * MSR_IA32_SPEC_CTRL for SSBD. */ void x86_virt_spec_ctrl(u64 guest_spec_ctrl, u64 guest_virt_spec_ctrl, bool setguest) { - u64 msrval, guestval = guest_spec_ctrl, hostval = spec_ctrl_current(); + u64 guestval, hostval; struct thread_info *ti = current_thread_info(); - if (static_cpu_has(X86_FEATURE_MSR_SPEC_CTRL)) { - if (hostval != guestval) { - msrval = setguest ? guestval : hostval; - wrmsrl(MSR_IA32_SPEC_CTRL, msrval); - } - } - /* * If SSBD is not handled in MSR_SPEC_CTRL on AMD, update * MSR_AMD64_L2_CFG or MSR_VIRT_SPEC_CTRL if supported. diff --git a/arch/x86/kvm/kvm-asm-offsets.c b/arch/x86/kvm/kvm-asm-offsets.c index 1b805cd24d66..24a710d37323 100644 --- a/arch/x86/kvm/kvm-asm-offsets.c +++ b/arch/x86/kvm/kvm-asm-offsets.c @@ -16,6 +16,7 @@ static void __used common(void) BLANK(); OFFSET(SVM_vcpu_arch_regs, vcpu_svm, vcpu.arch.regs); OFFSET(SVM_current_vmcb, vcpu_svm, current_vmcb); + OFFSET(SVM_spec_ctrl, vcpu_svm, spec_ctrl); OFFSET(SVM_vmcb01, vcpu_svm, vmcb01); OFFSET(KVM_VMCB_pa, kvm_vmcb_info, pa); OFFSET(SD_save_area_pa, svm_cpu_data, save_area_pa); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 469c1b5617af..cf1aed25f4ab 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -720,6 +720,15 @@ static bool msr_write_intercepted(struct kvm_vcpu *vcpu, u32 msr) u32 offset; u32 *msrpm; + /* + * For non-nested case: + * If the L01 MSR bitmap does not intercept the MSR, then we need to + * save it. + * + * For nested case: + * If the L02 MSR bitmap does not intercept the MSR, then we need to + * save it. + */ msrpm = is_guest_mode(vcpu) ? to_svm(vcpu)->nested.msrpm: to_svm(vcpu)->msrpm; @@ -3901,16 +3910,16 @@ static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) return EXIT_FASTPATH_NONE; } -static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) +static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, bool spec_ctrl_intercepted) { struct vcpu_svm *svm = to_svm(vcpu); guest_state_enter_irqoff(); if (sev_es_guest(vcpu->kvm)) - __svm_sev_es_vcpu_run(svm); + __svm_sev_es_vcpu_run(svm, spec_ctrl_intercepted); else - __svm_vcpu_run(svm); + __svm_vcpu_run(svm, spec_ctrl_intercepted); guest_state_exit_irqoff(); } @@ -3918,6 +3927,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); + bool spec_ctrl_intercepted = msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL); trace_kvm_entry(vcpu); @@ -3976,26 +3986,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL)) x86_spec_ctrl_set_guest(svm->spec_ctrl, svm->virt_spec_ctrl); - svm_vcpu_enter_exit(vcpu); - - /* - * We do not use IBRS in the kernel. If this vCPU has used the - * SPEC_CTRL MSR it may have left it on; save the value and - * turn it off. This is much more efficient than blindly adding - * it to the atomic save/restore list. Especially as the former - * (Saving guest MSRs on vmexit) doesn't even exist in KVM. - * - * For non-nested case: - * If the L01 MSR bitmap does not intercept the MSR, then we need to - * save it. - * - * For nested case: - * If the L02 MSR bitmap does not intercept the MSR, then we need to - * save it. - */ - if (!static_cpu_has(X86_FEATURE_V_SPEC_CTRL) && - unlikely(!msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL))) - svm->spec_ctrl = native_read_msr(MSR_IA32_SPEC_CTRL); + svm_vcpu_enter_exit(vcpu, spec_ctrl_intercepted); if (!sev_es_guest(vcpu->kvm)) reload_tss(vcpu); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 83955a4e520e..199a2ecef1ce 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -682,7 +682,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm); /* vmenter.S */ -void __svm_sev_es_vcpu_run(struct vcpu_svm *svm); -void __svm_vcpu_run(struct vcpu_svm *svm); +void __svm_sev_es_vcpu_run(struct vcpu_svm *svm, bool spec_ctrl_intercepted); +void __svm_vcpu_run(struct vcpu_svm *svm, bool spec_ctrl_intercepted); #endif diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index 57440acfc73e..34367dc203f2 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -32,9 +32,69 @@ .section .noinstr.text, "ax" +.macro RESTORE_GUEST_SPEC_CTRL + /* No need to do anything if SPEC_CTRL is unset or V_SPEC_CTRL is set */ + ALTERNATIVE_2 "", \ + "jmp 800f", X86_FEATURE_MSR_SPEC_CTRL, \ + "", X86_FEATURE_V_SPEC_CTRL +801: +.endm +.macro RESTORE_GUEST_SPEC_CTRL_BODY +800: + /* + * SPEC_CTRL handling: if the guest's SPEC_CTRL value differs from the + * host's, write the MSR. This is kept out-of-line so that the common + * case does not have to jump. + * + * IMPORTANT: To avoid RSB underflow attacks and any other nastiness, + * there must not be any returns or indirect branches between this code + * and vmentry. + */ + movl SVM_spec_ctrl(%_ASM_DI), %eax + cmp PER_CPU_VAR(x86_spec_ctrl_current), %eax + je 801b + mov $MSR_IA32_SPEC_CTRL, %ecx + xor %edx, %edx + wrmsr + jmp 801b +.endm + +.macro RESTORE_HOST_SPEC_CTRL + /* No need to do anything if SPEC_CTRL is unset or V_SPEC_CTRL is set */ + ALTERNATIVE_2 "", \ + "jmp 900f", X86_FEATURE_MSR_SPEC_CTRL, \ + "", X86_FEATURE_V_SPEC_CTRL +901: +.endm +.macro RESTORE_HOST_SPEC_CTRL_BODY +900: + /* Same for after vmexit. */ + mov $MSR_IA32_SPEC_CTRL, %ecx + + /* + * Load the value that the guest had written into MSR_IA32_SPEC_CTRL, + * if it was not intercepted during guest execution. + */ + cmpb $0, (%_ASM_SP) + jnz 998f + rdmsr + movl %eax, SVM_spec_ctrl(%_ASM_DI) +998: + + /* Now restore the host value of the MSR if different from the guest's. */ + movl PER_CPU_VAR(x86_spec_ctrl_current), %eax + cmp SVM_spec_ctrl(%_ASM_DI), %eax + je 901b + xor %edx, %edx + wrmsr + jmp 901b +.endm + + /** * __svm_vcpu_run - Run a vCPU via a transition to SVM guest mode * @svm: struct vcpu_svm * + * @spec_ctrl_intercepted: bool */ SYM_FUNC_START(__svm_vcpu_run) push %_ASM_BP @@ -54,17 +114,26 @@ SYM_FUNC_START(__svm_vcpu_run) * order compared to when they are needed. */ + /* Accessed directly from the stack in RESTORE_HOST_SPEC_CTRL. */ + push %_ASM_ARG2 + /* Needed to restore access to percpu variables. */ __ASM_SIZE(push) PER_CPU_VAR(svm_data + SD_save_area_pa) - /* Save @svm. */ + /* Finally save @svm. */ push %_ASM_ARG1 .ifnc _ASM_ARG1, _ASM_DI - /* Move @svm to RDI. */ + /* + * Stash @svm in RDI early. On 32-bit, arguments are in RAX, RCX + * and RDX which are clobbered by RESTORE_GUEST_SPEC_CTRL. + */ mov %_ASM_ARG1, %_ASM_DI .endif + /* Clobbers RAX, RCX, RDX. */ + RESTORE_GUEST_SPEC_CTRL + /* * Use a single vmcb (vmcb01 because it's always valid) for * context switching guest state via VMLOAD/VMSAVE, that way @@ -142,6 +211,9 @@ SYM_FUNC_START(__svm_vcpu_run) FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE #endif + /* Clobbers RAX, RCX, RDX. */ + RESTORE_HOST_SPEC_CTRL + /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be * untrained as soon as we exit the VM and are back to the @@ -177,6 +249,9 @@ SYM_FUNC_START(__svm_vcpu_run) xor %r15d, %r15d #endif + /* "Pop" @spec_ctrl_intercepted. */ + pop %_ASM_BX + pop %_ASM_BX #ifdef CONFIG_X86_64 @@ -191,6 +266,9 @@ SYM_FUNC_START(__svm_vcpu_run) pop %_ASM_BP RET + RESTORE_GUEST_SPEC_CTRL_BODY + RESTORE_HOST_SPEC_CTRL_BODY + 10: cmpb $0, kvm_rebooting jne 2b ud2 @@ -214,6 +292,7 @@ SYM_FUNC_END(__svm_vcpu_run) /** * __svm_sev_es_vcpu_run - Run a SEV-ES vCPU via a transition to SVM guest mode * @svm: struct vcpu_svm * + * @spec_ctrl_intercepted: bool */ SYM_FUNC_START(__svm_sev_es_vcpu_run) push %_ASM_BP @@ -228,8 +307,30 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) #endif push %_ASM_BX + /* + * Save variables needed after vmexit on the stack, in inverse + * order compared to when they are needed. + */ + + /* Accessed directly from the stack in RESTORE_HOST_SPEC_CTRL. */ + push %_ASM_ARG2 + + /* Save @svm. */ + push %_ASM_ARG1 + +.ifnc _ASM_ARG1, _ASM_DI + /* + * Stash @svm in RDI early. On 32-bit, arguments are in RAX, RCX + * and RDX which are clobbered by RESTORE_GUEST_SPEC_CTRL. + */ + mov %_ASM_ARG1, %_ASM_DI +.endif + + /* Clobbers RAX, RCX, RDX. */ + RESTORE_GUEST_SPEC_CTRL + /* Get svm->current_vmcb->pa into RAX. */ - mov SVM_current_vmcb(%_ASM_ARG1), %_ASM_AX + mov SVM_current_vmcb(%_ASM_DI), %_ASM_AX mov KVM_VMCB_pa(%_ASM_AX), %_ASM_AX /* Enter guest mode */ @@ -239,11 +340,17 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) 2: cli + /* Pop @svm to RDI, guest registers have been saved already. */ + pop %_ASM_DI + #ifdef CONFIG_RETPOLINE /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE #endif + /* Clobbers RAX, RCX, RDX. */ + RESTORE_HOST_SPEC_CTRL + /* * Mitigate RETBleed for AMD/Hygon Zen uarch. RET should be * untrained as soon as we exit the VM and are back to the @@ -253,6 +360,9 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) */ UNTRAIN_RET + /* "Pop" @spec_ctrl_intercepted. */ + pop %_ASM_BX + pop %_ASM_BX #ifdef CONFIG_X86_64 @@ -267,6 +377,9 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) pop %_ASM_BP RET + RESTORE_GUEST_SPEC_CTRL_BODY + RESTORE_HOST_SPEC_CTRL_BODY + 3: cmpb $0, kvm_rebooting jne 2b ud2 -- 2.31.1

3 years, 1 month

2
1
0 0

[PATCH 09/11] KVM: SVM: restore host save area from assembly

by Paolo Bonzini

Allow access to the percpu area via the GS segment base, which is needed in order to access the saved host spec_ctrl value. In linux-next FILL_RETURN_BUFFER also needs to access percpu data. For simplicity, the physical address of the save area is added to struct svm_cpu_data. Cc: stable(a)vger.kernel.org Fixes: a149180fbcf3 ("x86: Add magic AMD return-thunk") Reported-by: Nathan Chancellor <nathan(a)kernel.org> Analyzed-by: Andrew Cooper <andrew.cooper3(a)citrix.com> Tested-by: Nathan Chancellor <nathan(a)kernel.org> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- arch/x86/kvm/kvm-asm-offsets.c | 1 + arch/x86/kvm/svm/svm.c | 14 ++++++-------- arch/x86/kvm/svm/svm.h | 2 ++ arch/x86/kvm/svm/svm_ops.h | 5 ----- arch/x86/kvm/svm/vmenter.S | 17 +++++++++++++++++ 5 files changed, 26 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/kvm-asm-offsets.c b/arch/x86/kvm/kvm-asm-offsets.c index f83e88b85bf2..1b805cd24d66 100644 --- a/arch/x86/kvm/kvm-asm-offsets.c +++ b/arch/x86/kvm/kvm-asm-offsets.c @@ -18,6 +18,7 @@ static void __used common(void) OFFSET(SVM_current_vmcb, vcpu_svm, current_vmcb); OFFSET(SVM_vmcb01, vcpu_svm, vmcb01); OFFSET(KVM_VMCB_pa, kvm_vmcb_info, pa); + OFFSET(SD_save_area_pa, svm_cpu_data, save_area_pa); } if (IS_ENABLED(CONFIG_KVM_INTEL)) { diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 4e3a47eb5002..469c1b5617af 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -592,7 +592,7 @@ static int svm_hardware_enable(void) wrmsrl(MSR_EFER, efer | EFER_SVME); - wrmsrl(MSR_VM_HSAVE_PA, __sme_page_pa(sd->save_area)); + wrmsrl(MSR_VM_HSAVE_PA, sd->save_area_pa); if (static_cpu_has(X86_FEATURE_TSCRATEMSR)) { /* @@ -648,6 +648,7 @@ static void svm_cpu_uninit(int cpu) kfree(sd->sev_vmcbs); __free_page(sd->save_area); + sd->save_area_pa = 0; sd->save_area = NULL; } @@ -665,6 +666,7 @@ static int svm_cpu_init(int cpu) if (ret) goto free_save_area; + sd->save_area_pa = __sme_page_pa(sd->save_area); return 0; free_save_area: @@ -1450,7 +1452,7 @@ static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu) * Save additional host state that will be restored on VMEXIT (sev-es) * or subsequent vmload of host save area. */ - vmsave(__sme_page_pa(sd->save_area)); + vmsave(sd->save_area_pa); if (sev_es_guest(vcpu->kvm)) { struct sev_es_save_area *hostsa; hostsa = (struct sev_es_save_area *)(page_address(sd->save_area) + 0x400); @@ -3905,14 +3907,10 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) guest_state_enter_irqoff(); - if (sev_es_guest(vcpu->kvm)) { + if (sev_es_guest(vcpu->kvm)) __svm_sev_es_vcpu_run(svm); - } else { - struct svm_cpu_data *sd = per_cpu_ptr(&svm_data, vcpu->cpu); - + else __svm_vcpu_run(svm); - vmload(__sme_page_pa(sd->save_area)); - } guest_state_exit_irqoff(); } diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 2af6a71126c1..83955a4e520e 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -287,6 +287,8 @@ struct svm_cpu_data { struct kvm_ldttss_desc *tss_desc; struct page *save_area; + unsigned long save_area_pa; + struct vmcb *current_vmcb; /* index = sev_asid, value = vmcb pointer */ diff --git a/arch/x86/kvm/svm/svm_ops.h b/arch/x86/kvm/svm/svm_ops.h index 9430d6437c9f..36c8af87a707 100644 --- a/arch/x86/kvm/svm/svm_ops.h +++ b/arch/x86/kvm/svm/svm_ops.h @@ -61,9 +61,4 @@ static __always_inline void vmsave(unsigned long pa) svm_asm1(vmsave, "a" (pa), "memory"); } -static __always_inline void vmload(unsigned long pa) -{ - svm_asm1(vmload, "a" (pa), "memory"); -} - #endif /* __KVM_X86_SVM_OPS_H */ diff --git a/arch/x86/kvm/svm/vmenter.S b/arch/x86/kvm/svm/vmenter.S index 5bc2ed7d79c0..57440acfc73e 100644 --- a/arch/x86/kvm/svm/vmenter.S +++ b/arch/x86/kvm/svm/vmenter.S @@ -49,6 +49,14 @@ SYM_FUNC_START(__svm_vcpu_run) #endif push %_ASM_BX + /* + * Save variables needed after vmexit on the stack, in inverse + * order compared to when they are needed. + */ + + /* Needed to restore access to percpu variables. */ + __ASM_SIZE(push) PER_CPU_VAR(svm_data + SD_save_area_pa) + /* Save @svm. */ push %_ASM_ARG1 @@ -124,6 +132,11 @@ SYM_FUNC_START(__svm_vcpu_run) 5: vmsave %_ASM_AX 6: + /* Restores GSBASE among other things, allowing access to percpu data. */ + pop %_ASM_AX +7: vmload %_ASM_AX +8: + #ifdef CONFIG_RETPOLINE /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE @@ -187,10 +200,14 @@ SYM_FUNC_START(__svm_vcpu_run) 50: cmpb $0, kvm_rebooting jne 6b ud2 +70: cmpb $0, kvm_rebooting + jne 8b + ud2 _ASM_EXTABLE(1b, 10b) _ASM_EXTABLE(3b, 30b) _ASM_EXTABLE(5b, 50b) + _ASM_EXTABLE(7b, 70b) SYM_FUNC_END(__svm_vcpu_run) -- 2.31.1

3 years, 1 month

2
1
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror November 2022