Hi,
This series is a v6.12-only backport (based on v6.12.62) of fixes for system register initialization issues affecting protected KVM on arm64. This affects some contemporary and upcoming hardware which will run the v6.12.y stable kernel, or something derived from it, such as the Android common kernel.
The FEAT_E2H0 patches fix code introduced after v6.6, and so only need to be backported to v6.12.
The SCTLR_EL1 patch fixes code introduced in v5.11, but practically speaking only affects recent hardware which is unlikely to run something older than v6.12.
Note: Marc Zyngier performed the initial backport, which I have rebased and tested, hence both of our sign-offs being added to the tags from the upstream commits.
I have tested the backport and observed they solve the problems as expected.
Ahmed Genidi (1): KVM: arm64: Initialize SCTLR_EL1 in __kvm_hyp_init_cpu()
Marc Zyngier (1): arm64: Revamp HCR_EL2.E2H RES1 detection
Mark Rutland (1): KVM: arm64: Initialize HCR_EL2.E2H early
arch/arm64/include/asm/el2_setup.h | 57 +++++++++++++++++++++++++--- arch/arm64/kernel/head.S | 22 ++--------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 10 +++-- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 3 ++ 4 files changed, 65 insertions(+), 27 deletions(-)
From: Mark Rutland mark.rutland@arm.com
[ Upstream commit 7a68b55ff39b0a1638acb1694c185d49f6077a0d ]
On CPUs without FEAT_E2H0, HCR_EL2.E2H is RES1, but may reset to an UNKNOWN value out of reset and consequently may not read as 1 unless it has been explicitly initialized.
We handled this for the head.S boot code in commits:
3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative") b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented")
Unfortunately, we forgot to apply a similar fix to the KVM PSCI entry points used when relaying CPU_ON, CPU_SUSPEND, and SYSTEM SUSPEND. When KVM is entered via these entry points, the value of HCR_EL2.E2H may be consumed before it has been initialized (e.g. by the 'init_el2_state' macro).
Initialize HCR_EL2.E2H early in these paths such that it can be consumed reliably. The existing code in head.S is factored out into a new 'init_el2_hcr' macro, and this is used in the __kvm_hyp_init_cpu() function common to all the relevant PSCI entry points.
For clarity, I've tweaked the assembly used to check whether ID_AA64MMFR4_EL1.E2H0 is negative. The bitfield is extracted as a signed value, and this is checked with a signed-greater-or-equal (GE) comparison.
As the hyp code will reconfigure HCR_EL2 later in ___kvm_hyp_init(), all bits other than E2H are initialized to zero in __kvm_hyp_init_cpu().
Fixes: 3944382fa6f22b54 ("arm64: Treat HCR_EL2.E2H as RES1 when ID_AA64MMFR4_EL1.E2H0 is negative") Fixes: b3320142f3db9b3f ("arm64: Fix early handling of FEAT_E2H0 not being implemented") Signed-off-by: Mark Rutland mark.rutland@arm.com Cc: Ahmed Genidi ahmed.genidi@arm.com Cc: Ben Horgan ben.horgan@arm.com Cc: Catalin Marinas catalin.marinas@arm.com Cc: Leo Yan leo.yan@arm.com Cc: Marc Zyngier maz@kernel.org Cc: Oliver Upton oliver.upton@linux.dev Cc: Will Deacon will@kernel.org Link: https://lore.kernel.org/r/20250227180526.1204723-2-mark.rutland@arm.com [maz: fixed LT->GE thinko] Signed-off-by: Marc Zyngier maz@kernel.org Signed-off-by: Wei-Lin Chang weilin.chang@arm.com --- arch/arm64/include/asm/el2_setup.h | 26 ++++++++++++++++++++++++++ arch/arm64/kernel/head.S | 19 +------------------ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 8 +++++++- 3 files changed, 34 insertions(+), 19 deletions(-)
diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index bdbe9e08664a..00b27c8ed9a2 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -16,6 +16,32 @@ #include <asm/sysreg.h> #include <linux/irqchip/arm-gic-v3.h>
+.macro init_el2_hcr val + mov_q x0, \val + + /* + * Compliant CPUs advertise their VHE-onlyness with + * ID_AA64MMFR4_EL1.E2H0 < 0. On such CPUs HCR_EL2.E2H is RES1, but it + * can reset into an UNKNOWN state and might not read as 1 until it has + * been initialized explicitly. + * + * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but + * don't advertise it (they predate this relaxation). + * + * Initalize HCR_EL2.E2H so that later code can rely upon HCR_EL2.E2H + * indicating whether the CPU is running in E2H mode. + */ + mrs_s x1, SYS_ID_AA64MMFR4_EL1 + sbfx x1, x1, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH + cmp x1, #0 + b.ge .LnVHE_@ + + orr x0, x0, #HCR_E2H +.LnVHE_@: + msr hcr_el2, x0 + isb +.endm + .macro __init_el2_sctlr mov_q x0, INIT_SCTLR_EL2_MMU_OFF msr sctlr_el2, x0 diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index cb68adcabe07..4d28c1e56cb5 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -295,25 +295,8 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) msr sctlr_el2, x0 isb 0: - mov_q x0, HCR_HOST_NVHE_FLAGS - - /* - * Compliant CPUs advertise their VHE-onlyness with - * ID_AA64MMFR4_EL1.E2H0 < 0. HCR_EL2.E2H can be - * RES1 in that case. Publish the E2H bit early so that - * it can be picked up by the init_el2_state macro. - * - * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but - * don't advertise it (they predate this relaxation). - */ - mrs_s x1, SYS_ID_AA64MMFR4_EL1 - tbz x1, #(ID_AA64MMFR4_EL1_E2H0_SHIFT + ID_AA64MMFR4_EL1_E2H0_WIDTH - 1), 1f - - orr x0, x0, #HCR_E2H -1: - msr hcr_el2, x0 - isb
+ init_el2_hcr HCR_HOST_NVHE_FLAGS init_el2_state
/* Hypervisor stub */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index fc1866226067..3fb5504a7d7f 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -73,8 +73,12 @@ __do_hyp_init: eret SYM_CODE_END(__kvm_hyp_init)
+/* + * Initialize EL2 CPU state to sane values. + * + * HCR_EL2.E2H must have been initialized already. + */ SYM_CODE_START_LOCAL(__kvm_init_el2_state) - /* Initialize EL2 CPU state to sane values. */ init_el2_state // Clobbers x0..x2 finalise_el2_state ret @@ -206,6 +210,8 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu)
2: msr SPsel, #1 // We want to use SP_EL{1,2}
+ init_el2_hcr 0 + bl __kvm_init_el2_state
__init_el2_nvhe_prepare_eret
From: Ahmed Genidi ahmed.genidi@arm.com
[ Upstream commit 3855a7b91d42ebf3513b7ccffc44807274978b3d ]
When KVM is in protected mode, host calls to PSCI are proxied via EL2, and cold entries from CPU_ON, CPU_SUSPEND, and SYSTEM_SUSPEND bounce through __kvm_hyp_init_cpu() at EL2 before entering the host kernel's entry point at EL1. While __kvm_hyp_init_cpu() initializes SPSR_EL2 for the exception return to EL1, it does not initialize SCTLR_EL1.
Due to this, it's possible to enter EL1 with SCTLR_EL1 in an UNKNOWN state. In practice this has been seen to result in kernel crashes after CPU_ON as a result of SCTLR_EL1.M being 1 in violation of the initial core configuration specified by PSCI.
Fix this by initializing SCTLR_EL1 for cold entry to the host kernel. As it's necessary to write to SCTLR_EL12 in VHE mode, this initialization is moved into __kvm_host_psci_cpu_entry() where we can use write_sysreg_el1().
The remnants of the '__init_el2_nvhe_prepare_eret' macro are folded into its only caller, as this is clearer than having the macro.
Fixes: cdf367192766ad11 ("KVM: arm64: Intercept host's CPU_ON SMCs") Reported-by: Leo Yan leo.yan@arm.com Signed-off-by: Ahmed Genidi ahmed.genidi@arm.com [ Mark: clarify commit message, handle E2H, move to C, remove macro ] Signed-off-by: Mark Rutland mark.rutland@arm.com Cc: Ahmed Genidi ahmed.genidi@arm.com Cc: Ben Horgan ben.horgan@arm.com Cc: Catalin Marinas catalin.marinas@arm.com Cc: Leo Yan leo.yan@arm.com Cc: Marc Zyngier maz@kernel.org Cc: Oliver Upton oliver.upton@linux.dev Cc: Will Deacon will@kernel.org Reviewed-by: Leo Yan leo.yan@arm.com Link: https://lore.kernel.org/r/20250227180526.1204723-3-mark.rutland@arm.com Signed-off-by: Marc Zyngier maz@kernel.org Signed-off-by: Wei-Lin Chang weilin.chang@arm.com --- arch/arm64/include/asm/el2_setup.h | 5 ----- arch/arm64/kernel/head.S | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 2 -- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 3 +++ 4 files changed, 5 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 00b27c8ed9a2..859aa1a3b996 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -265,11 +265,6 @@ .Lskip_fgt2_@: .endm
-.macro __init_el2_nvhe_prepare_eret - mov x0, #INIT_PSTATE_EL1 - msr spsr_el2, x0 -.endm - /** * Initialize EL2 registers to sane values. This should be called early on all * cores that were booted in EL2. Note that everything gets initialised as diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 4d28c1e56cb5..25c08a0d228a 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -319,7 +319,8 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) msr sctlr_el1, x1 mov x2, xzr 3: - __init_el2_nvhe_prepare_eret + mov x0, #INIT_PSTATE_EL1 + msr spsr_el2, x0
mov w0, #BOOT_CPU_MODE_EL2 orr x0, x0, x2 diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 3fb5504a7d7f..f8af11189572 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -214,8 +214,6 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu)
bl __kvm_init_el2_state
- __init_el2_nvhe_prepare_eret - /* Enable MMU, set vectors and stack. */ mov x0, x28 bl ___kvm_hyp_init // Clobbers x0..x2 diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index dfe8fe0f7eaf..bd0f308b694c 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -218,6 +218,9 @@ asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bool is_cpu_on) if (is_cpu_on) release_boot_args(boot_args);
+ write_sysreg_el1(INIT_SCTLR_EL1_MMU_OFF, SYS_SCTLR); + write_sysreg(INIT_PSTATE_EL1, SPSR_EL2); + __host_enter(host_ctxt); }
From: Marc Zyngier maz@kernel.org
[ Upstream commit ca88ecdce5f51874a7c151809bd2c936ee0d3805 ]
We currently have two ways to identify CPUs that only implement FEAT_VHE and not FEAT_E2H0:
- either they advertise it via ID_AA64MMFR4_EL1.E2H0, - or the HCR_EL2.E2H bit is RAO/WI
However, there is a third category of "cpus" that fall between these two cases: on CPUs that do not implement FEAT_FGT, it is IMPDEF whether an access to ID_AA64MMFR4_EL1 can trap to EL2 when the register value is zero.
A consequence of this is that on systems such as Neoverse V2, a NV guest cannot reliably detect that it is in a VHE-only configuration (E2H is writable, and ID_AA64MMFR0_EL1 is 0), despite the hypervisor's best effort to repaint the id register.
Replace the RAO/WI test by a sequence that makes use of the VHE register remnapping between EL1 and EL2 to detect this situation, and work out whether we get the VHE behaviour even after having set HCR_EL2.E2H to 0.
This solves the NV problem, and provides a more reliable acid test for CPUs that do not completely follow the letter of the architecture while providing a RES1 behaviour for HCR_EL2.E2H.
Suggested-by: Mark Rutland mark.rutland@arm.com Acked-by: Mark Rutland mark.rutland@arm.com Acked-by: Catalin Marinas catalin.marinas@arm.com Reviewed-by: Oliver Upton oliver.upton@linux.dev Tested-by: Jan Kotas jank@cadence.com Signed-off-by: Marc Zyngier maz@kernel.org Link: https://lore.kernel.org/r/15A85F2B-1A0C-4FA7-9FE4-EEC2203CC09E@global.cadenc... Signed-off-by: Marc Zyngier maz@kernel.org Signed-off-by: Wei-Lin Chang weilin.chang@arm.com --- arch/arm64/include/asm/el2_setup.h | 38 +++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 859aa1a3b996..9d1c88536ee4 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -24,22 +24,48 @@ * ID_AA64MMFR4_EL1.E2H0 < 0. On such CPUs HCR_EL2.E2H is RES1, but it * can reset into an UNKNOWN state and might not read as 1 until it has * been initialized explicitly. - * - * Fruity CPUs seem to have HCR_EL2.E2H set to RAO/WI, but - * don't advertise it (they predate this relaxation). - * * Initalize HCR_EL2.E2H so that later code can rely upon HCR_EL2.E2H * indicating whether the CPU is running in E2H mode. */ mrs_s x1, SYS_ID_AA64MMFR4_EL1 sbfx x1, x1, #ID_AA64MMFR4_EL1_E2H0_SHIFT, #ID_AA64MMFR4_EL1_E2H0_WIDTH cmp x1, #0 - b.ge .LnVHE_@ + b.lt .LnE2H0_@
+ /* + * Unfortunately, HCR_EL2.E2H can be RES1 even if not advertised + * as such via ID_AA64MMFR4_EL1.E2H0: + * + * - Fruity CPUs predate the !FEAT_E2H0 relaxation, and seem to + * have HCR_EL2.E2H implemented as RAO/WI. + * + * - On CPUs that lack FEAT_FGT, a hypervisor can't trap guest + * reads of ID_AA64MMFR4_EL1 to advertise !FEAT_E2H0. NV + * guests on these hosts can write to HCR_EL2.E2H without + * trapping to the hypervisor, but these writes have no + * functional effect. + * + * Handle both cases by checking for an essential VHE property + * (system register remapping) to decide whether we're + * effectively VHE-only or not. + */ + msr hcr_el2, x0 // Setup HCR_EL2 as nVHE + isb + mov x1, #1 // Write something to FAR_EL1 + msr far_el1, x1 + isb + mov x1, #2 // Try to overwrite it via FAR_EL2 + msr far_el2, x1 + isb + mrs x1, far_el1 // If we see the latest write in FAR_EL1, + cmp x1, #2 // we can safely assume we are VHE only. + b.ne .LnVHE_@ // Otherwise, we know that nVHE works. + +.LnE2H0_@: orr x0, x0, #HCR_E2H -.LnVHE_@: msr hcr_el2, x0 isb +.LnVHE_@: .endm
.macro __init_el2_sctlr
On Fri, 19 Dec 2025 10:21:20 +0000, Wei-Lin Chang weilin.chang@arm.com wrote:
Hi,
This series is a v6.12-only backport (based on v6.12.62) of fixes for system register initialization issues affecting protected KVM on arm64. This affects some contemporary and upcoming hardware which will run the v6.12.y stable kernel, or something derived from it, such as the Android common kernel.
The FEAT_E2H0 patches fix code introduced after v6.6, and so only need to be backported to v6.12.
The SCTLR_EL1 patch fixes code introduced in v5.11, but practically speaking only affects recent hardware which is unlikely to run something older than v6.12.
Note: Marc Zyngier performed the initial backport, which I have rebased and tested, hence both of our sign-offs being added to the tags from the upstream commits.
I have tested the backport and observed they solve the problems as expected.
Ahmed Genidi (1): KVM: arm64: Initialize SCTLR_EL1 in __kvm_hyp_init_cpu()
Marc Zyngier (1): arm64: Revamp HCR_EL2.E2H RES1 detection
Mark Rutland (1): KVM: arm64: Initialize HCR_EL2.E2H early
arch/arm64/include/asm/el2_setup.h | 57 +++++++++++++++++++++++++--- arch/arm64/kernel/head.S | 22 ++--------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 10 +++-- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 3 ++ 4 files changed, 65 insertions(+), 27 deletions(-)
Great, thanks for going through this and having tested it!
Hopefully Greg will be able to take it at the next opportunity for a stable 6.12 release.
Cheers,
M.
linux-stable-mirror@lists.linaro.org