This is a combination of the RFC for nVHE here [1] and v3 of VHE version here [2]. After a few of the review comments it seemed much simpler for both versions to use the same interface and be in the same patchset.
FEAT_TRF is a Coresight feature that allows trace capture to be completely filtered at different exception levels, unlike the existing TRCVICTLR controls which may still emit target addresses of branches, even if the following trace is filtered.
Without FEAT_TRF, it was possible to start a trace session on a host and also collect trace from the guest as TRCVICTLR was never programmed to exclude guests (and it could still emit target addresses even if it was).
With FEAT_TRF, the current behavior of trace in guests exists depends on whether nVHE or VHE are being used. Both of the examples below are from the host's point of view, as Coresight isn't accessible from guests. This patchset is only relevant to when FEAT_TRF exists, otherwise there is no change.
nVHE:
Because the host and the guest are both using TRFCR_EL1, trace will be generated in guests depending on the same filter rules the host is using. For example if the host is tracing userspace only, then guest userspace trace will also be collected.
(This is further limited by whether TRBE is used because an issue with TRBE means that it's completely disabled in nVHE guests, but it's possible to have other tracing components.)
VHE:
With VHE, the host filters will be in TRFCR_EL2, but the filters in TRFCR_EL1 will be active when the guest is running. Because we don't write to TRFCR_EL1, guest trace will be completely disabled.
With this change, the guest filtering rules from the Perf session are honored for both nVHE and VHE modes. This is done by either writing to TRFCR_EL12 at the start of the Perf session and doing nothing else further, or caching the guest value and writing it at guest switch for nVHE.
The first commit moves the register to sysreg because I add the EL12 version in a later commit.
--- Changes since V1:
* Squashed all the arm64/tools/sysreg changes into the first commit * Add a new commit to move SPE and TRBE regs into the kvm sysreg array * Add a comment above the TRFCR global that it's per host CPU rather than vcpu
Changes since nVHE RFC [1]:
* Re-write just in terms of the register value to be written for the host and the guest. This removes some logic from the hyp code and a value of kvm_vcpu_arch:trfcr_el1 = 0 no longer means "don't restore". * Remove all the conditional compilation and new files. * Change the kvm_etm_update_vcpu_events macro to a function. * Re-use DEBUG_STATE_SAVE_TRFCR so iflags don't need to be expanded anymore. * Expand the cover letter.
Changes since VHE v3 [2]:
* Use the same interface as nVHE mode so TRFCR_EL12 is now written by kvm.
[1]: https://lore.kernel.org/kvmarm/20230804101317.460697-1-james.clark@arm.com/ [2]: https://lore.kernel.org/kvmarm/20230905102117.2011094-1-james.clark@arm.com/
James Clark (6): arm64/sysreg: Move TRFCR definitions to sysreg arm64: KVM: Rename DEBUG_STATE_SAVE_TRBE to DEBUG_STATE_SAVE_TRFCR arm64: KVM: Move SPE and trace registers to the sysreg array arm64: KVM: Add interface to set guest value for TRFCR register arm64: KVM: Write TRFCR value on guest switch with nVHE coresight: Pass guest TRFCR value to KVM
arch/arm64/include/asm/kvm_host.h | 13 +-- arch/arm64/include/asm/kvm_hyp.h | 6 +- arch/arm64/include/asm/sysreg.h | 12 --- arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/debug.c | 43 ++++++++- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 87 +++++++++++-------- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- arch/arm64/tools/sysreg | 41 +++++++++ .../coresight/coresight-etm4x-core.c | 42 +++++++-- drivers/hwtracing/coresight/coresight-etm4x.h | 2 +- drivers/hwtracing/coresight/coresight-priv.h | 3 + 11 files changed, 186 insertions(+), 68 deletions(-)
Add separate definitions for ELx and EL2 as TRFCR_EL1 doesn't have CX. This also mirrors the previous definition so no code change is required.
Also add TRFCR_EL12 which will start to be used in a later commit.
Reviewed-by: Mark Brown broonie@kernel.org Signed-off-by: James Clark james.clark@arm.com --- arch/arm64/include/asm/sysreg.h | 12 ---------- arch/arm64/tools/sysreg | 41 +++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 38296579a4fd..068dd6abe273 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -278,8 +278,6 @@ #define SYS_RGSR_EL1 sys_reg(3, 0, 1, 0, 5) #define SYS_GCR_EL1 sys_reg(3, 0, 1, 0, 6)
-#define SYS_TRFCR_EL1 sys_reg(3, 0, 1, 2, 1) - #define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
#define SYS_APIAKEYLO_EL1 sys_reg(3, 0, 2, 1, 0) @@ -496,7 +494,6 @@ #define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0) #define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2)
-#define SYS_TRFCR_EL2 sys_reg(3, 4, 1, 2, 1) #define SYS_HAFGRTR_EL2 sys_reg(3, 4, 3, 1, 6) #define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0) #define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1) @@ -904,15 +901,6 @@ /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */ #define SYS_MPIDR_SAFE_VAL (BIT(31))
-#define TRFCR_ELx_TS_SHIFT 5 -#define TRFCR_ELx_TS_MASK ((0x3UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_VIRTUAL ((0x1UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_GUEST_PHYSICAL ((0x2UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_PHYSICAL ((0x3UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_EL2_CX BIT(3) -#define TRFCR_ELx_ExTRE BIT(1) -#define TRFCR_ELx_E0TRE BIT(0) - /* GIC Hypervisor interface registers */ /* ICH_MISR_EL2 bit definitions */ #define ICH_MISR_EOI (1 << 0) diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 2517ef7c21cf..2104152db18e 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -2624,3 +2624,44 @@ Field 5 F Field 4 P Field 3:0 Align EndSysreg + +SysregFields TRFCR_EL2 +Res0 63:7 +UnsignedEnum 6:5 TS + 0b0000 USE_TRFCR_EL1_TS + 0b0001 VIRTUAL + 0b0010 GUEST_PHYSICAL + 0b0011 PHYSICAL +EndEnum +Res0 4 +Field 3 CX +Res0 2 +Field 1 E2TRE +Field 0 E0HTRE +EndSysregFields + +# TRFCR_EL1 doesn't have the CX bit so redefine it without CX instead of +# using a shared definition between TRFCR_EL2 and TRFCR_EL1 +SysregFields TRFCR_ELx +Res0 63:7 +UnsignedEnum 6:5 TS + 0b0001 VIRTUAL + 0b0010 GUEST_PHYSICAL + 0b0011 PHYSICAL +EndEnum +Res0 4:2 +Field 1 ExTRE +Field 0 E0TRE +EndSysregFields + +Sysreg TRFCR_EL1 3 0 1 2 1 +Fields TRFCR_ELx +EndSysreg + +Sysreg TRFCR_EL2 3 4 1 2 1 +Fields TRFCR_EL2 +EndSysreg + +Sysreg TRFCR_EL12 3 5 1 2 1 +Fields TRFCR_ELx +EndSysreg
On 05/10/2023 13:57, James Clark wrote:
Add separate definitions for ELx and EL2 as TRFCR_EL1 doesn't have CX. This also mirrors the previous definition so no code change is required.
Also add TRFCR_EL12 which will start to be used in a later commit.
Reviewed-by: Mark Brown broonie@kernel.org Signed-off-by: James Clark james.clark@arm.com
Reviewed-by: Suzuki K Poulose suzuki.poulose@arm.com
arch/arm64/include/asm/sysreg.h | 12 ---------- arch/arm64/tools/sysreg | 41 +++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 38296579a4fd..068dd6abe273 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -278,8 +278,6 @@ #define SYS_RGSR_EL1 sys_reg(3, 0, 1, 0, 5) #define SYS_GCR_EL1 sys_reg(3, 0, 1, 0, 6) -#define SYS_TRFCR_EL1 sys_reg(3, 0, 1, 2, 1)
- #define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
#define SYS_APIAKEYLO_EL1 sys_reg(3, 0, 2, 1, 0) @@ -496,7 +494,6 @@ #define SYS_VTTBR_EL2 sys_reg(3, 4, 2, 1, 0) #define SYS_VTCR_EL2 sys_reg(3, 4, 2, 1, 2) -#define SYS_TRFCR_EL2 sys_reg(3, 4, 1, 2, 1) #define SYS_HAFGRTR_EL2 sys_reg(3, 4, 3, 1, 6) #define SYS_SPSR_EL2 sys_reg(3, 4, 4, 0, 0) #define SYS_ELR_EL2 sys_reg(3, 4, 4, 0, 1) @@ -904,15 +901,6 @@ /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */ #define SYS_MPIDR_SAFE_VAL (BIT(31)) -#define TRFCR_ELx_TS_SHIFT 5 -#define TRFCR_ELx_TS_MASK ((0x3UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_VIRTUAL ((0x1UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_GUEST_PHYSICAL ((0x2UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_ELx_TS_PHYSICAL ((0x3UL) << TRFCR_ELx_TS_SHIFT) -#define TRFCR_EL2_CX BIT(3) -#define TRFCR_ELx_ExTRE BIT(1) -#define TRFCR_ELx_E0TRE BIT(0)
- /* GIC Hypervisor interface registers */ /* ICH_MISR_EL2 bit definitions */ #define ICH_MISR_EOI (1 << 0)
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 2517ef7c21cf..2104152db18e 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -2624,3 +2624,44 @@ Field 5 F Field 4 P Field 3:0 Align EndSysreg
+SysregFields TRFCR_EL2 +Res0 63:7 +UnsignedEnum 6:5 TS
- 0b0000 USE_TRFCR_EL1_TS
- 0b0001 VIRTUAL
- 0b0010 GUEST_PHYSICAL
- 0b0011 PHYSICAL
+EndEnum +Res0 4 +Field 3 CX +Res0 2 +Field 1 E2TRE +Field 0 E0HTRE +EndSysregFields
+# TRFCR_EL1 doesn't have the CX bit so redefine it without CX instead of +# using a shared definition between TRFCR_EL2 and TRFCR_EL1 +SysregFields TRFCR_ELx +Res0 63:7 +UnsignedEnum 6:5 TS
- 0b0001 VIRTUAL
- 0b0010 GUEST_PHYSICAL
- 0b0011 PHYSICAL
+EndEnum +Res0 4:2 +Field 1 ExTRE +Field 0 E0TRE +EndSysregFields
+Sysreg TRFCR_EL1 3 0 1 2 1 +Fields TRFCR_ELx +EndSysreg
+Sysreg TRFCR_EL2 3 4 1 2 1 +Fields TRFCR_EL2 +EndSysreg
+Sysreg TRFCR_EL12 3 5 1 2 1 +Fields TRFCR_ELx +EndSysreg
This flag actually causes the TRFCR register to be saved, so rename it to that effect.
Currently it only happens when TRBE is used, but in a later commit TRFCR will be saved and restored even if TRBE isn't used, so the new name will be more accurate.
Signed-off-by: James Clark james.clark@arm.com --- arch/arm64/include/asm/kvm_host.h | 4 ++-- arch/arm64/kvm/debug.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 0ca4b34f8513..e36f7e8a76ce 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -741,8 +741,8 @@ struct kvm_vcpu_arch { #define DEBUG_DIRTY __vcpu_single_flag(iflags, BIT(4)) /* Save SPE context if active */ #define DEBUG_STATE_SAVE_SPE __vcpu_single_flag(iflags, BIT(5)) -/* Save TRBE context if active */ -#define DEBUG_STATE_SAVE_TRBE __vcpu_single_flag(iflags, BIT(6)) +/* Save TRFCR and disable TRBE if necessary */ +#define DEBUG_STATE_SAVE_TRFCR __vcpu_single_flag(iflags, BIT(6)) /* vcpu running in HYP context */ #define VCPU_HYP_CONTEXT __vcpu_single_flag(iflags, BIT(7))
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 8725291cb00a..6a1bad1a921b 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -334,11 +334,11 @@ void kvm_arch_vcpu_load_debug_state_flags(struct kvm_vcpu *vcpu) /* Check if we have TRBE implemented and available at the host */ if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceBuffer_SHIFT) && !(read_sysreg_s(SYS_TRBIDR_EL1) & TRBIDR_EL1_P)) - vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRBE); + vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); }
void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) { vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_SPE); - vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_TRBE); + vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); } diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 4558c02eb352..89c208112eb7 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -85,7 +85,7 @@ void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu) if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE)) __debug_save_spe(&vcpu->arch.host_debug_state.pmscr_el1); /* Disable and flush Self-Hosted Trace generation */ - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRBE)) + if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) __debug_save_trace(&vcpu->arch.host_debug_state.trfcr_el1); }
@@ -98,7 +98,7 @@ void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu) { if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE)) __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1); - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRBE)) + if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) __debug_restore_trace(vcpu->arch.host_debug_state.trfcr_el1); }
On 05/10/2023 13:57, James Clark wrote:
This flag actually causes the TRFCR register to be saved, so rename it to that effect.
Currently it only happens when TRBE is used, but in a later commit TRFCR will be saved and restored even if TRBE isn't used, so the new name will be more accurate.
Signed-off-by: James Clark james.clark@arm.com
Reviewed-by: Suzuki K Poulose suzuki.poulose@arm.com
On 05/10/2023 17:41, Suzuki K Poulose wrote:
On 05/10/2023 13:57, James Clark wrote:
This flag actually causes the TRFCR register to be saved, so rename it to that effect.
Currently it only happens when TRBE is used, but in a later commit TRFCR will be saved and restored even if TRBE isn't used, so the new name will be more accurate.
Signed-off-by: James Clark james.clark@arm.com
Reviewed-by: Suzuki K Poulose suzuki.poulose@arm.com
Reviewing the patch 5, this change may need to be dropped and instead we need to add a new flag for TRFCR.
Suzuki
On 05/10/2023 19:04, Suzuki K Poulose wrote:
On 05/10/2023 17:41, Suzuki K Poulose wrote:
On 05/10/2023 13:57, James Clark wrote:
This flag actually causes the TRFCR register to be saved, so rename it to that effect.
Currently it only happens when TRBE is used, but in a later commit TRFCR will be saved and restored even if TRBE isn't used, so the new name will be more accurate.
Signed-off-by: James Clark james.clark@arm.com
Reviewed-by: Suzuki K Poulose suzuki.poulose@arm.com
Reviewing the patch 5, this change may need to be dropped and instead we need to add a new flag for TRFCR.
Suzuki
Yes I dropped this commit and added a new one to add a new flag.
pmscr_el1 and trfcr_el1 are currently special cased in the host_debug_state struct, but they're just registers after all so give them entries in the sysreg array and refer to them through the host context.
Signed-off-by: James Clark james.clark@arm.com --- arch/arm64/include/asm/kvm_host.h | 6 ++-- arch/arm64/include/asm/kvm_hyp.h | 4 +-- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 44 +++++++++++++++--------------- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +-- 4 files changed, 28 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e36f7e8a76ce..b5200f199692 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -439,6 +439,8 @@ enum vcpu_sysreg { CNTHP_CVAL_EL2, CNTHV_CTL_EL2, CNTHV_CVAL_EL2, + PMSCR_EL1, /* Statistical profiling extension */ + TRFCR_EL1, /* Self-hosted trace filters */
NR_SYS_REGS /* Nothing after this line! */ }; @@ -572,10 +574,6 @@ struct kvm_vcpu_arch { struct { /* {Break,watch}point registers */ struct kvm_guest_debug_arch regs; - /* Statistical profiling extension */ - u64 pmscr_el1; - /* Self-hosted trace */ - u64 trfcr_el1; } host_debug_state;
/* VGIC state */ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b7238c72a04c..37e238f526d7 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -103,8 +103,8 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu);
#ifdef __KVM_NVHE_HYPERVISOR__ -void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu); -void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu); +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); #endif
void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 89c208112eb7..128a57dddabf 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -14,12 +14,12 @@ #include <asm/kvm_hyp.h> #include <asm/kvm_mmu.h>
-static void __debug_save_spe(u64 *pmscr_el1) +static void __debug_save_spe(struct kvm_cpu_context *host_ctxt) { u64 reg;
/* Clear pmscr in case of early return */ - *pmscr_el1 = 0; + ctxt_sys_reg(host_ctxt, PMSCR_EL1) = 0;
/* * At this point, we know that this CPU implements @@ -31,7 +31,7 @@ static void __debug_save_spe(u64 *pmscr_el1) return;
/* Yes; save the control register and disable data generation */ - *pmscr_el1 = read_sysreg_s(SYS_PMSCR_EL1); + ctxt_sys_reg(host_ctxt, PMSCR_EL1) = read_sysreg_s(SYS_PMSCR_EL1); write_sysreg_s(0, SYS_PMSCR_EL1); isb();
@@ -39,21 +39,21 @@ static void __debug_save_spe(u64 *pmscr_el1) psb_csync(); }
-static void __debug_restore_spe(u64 pmscr_el1) +static void __debug_restore_spe(struct kvm_cpu_context *host_ctxt) { - if (!pmscr_el1) + if (!ctxt_sys_reg(host_ctxt, PMSCR_EL1)) return;
/* The host page table is installed, but not yet synchronised */ isb();
/* Re-enable data generation */ - write_sysreg_s(pmscr_el1, SYS_PMSCR_EL1); + write_sysreg_s(ctxt_sys_reg(host_ctxt, PMSCR_EL1), SYS_PMSCR_EL1); }
-static void __debug_save_trace(u64 *trfcr_el1) +static void __debug_save_trace(struct kvm_cpu_context *host_ctxt) { - *trfcr_el1 = 0; + ctxt_sys_reg(host_ctxt, TRFCR_EL1) = 0;
/* Check if the TRBE is enabled */ if (!(read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E)) @@ -63,30 +63,30 @@ static void __debug_save_trace(u64 *trfcr_el1) * Since access to TRFCR_EL1 is trapped, the guest can't * modify the filtering set by the host. */ - *trfcr_el1 = read_sysreg_s(SYS_TRFCR_EL1); + ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1); write_sysreg_s(0, SYS_TRFCR_EL1); isb(); /* Drain the trace buffer to memory */ tsb_csync(); }
-static void __debug_restore_trace(u64 trfcr_el1) +static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt) { - if (!trfcr_el1) + if (!ctxt_sys_reg(host_ctxt, TRFCR_EL1)) return;
/* Restore trace filter controls */ - write_sysreg_s(trfcr_el1, SYS_TRFCR_EL1); + write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); }
-void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu) +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) { /* Disable and flush SPE data generation */ - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE)) - __debug_save_spe(&vcpu->arch.host_debug_state.pmscr_el1); + if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) + __debug_save_spe(host_ctxt); /* Disable and flush Self-Hosted Trace generation */ - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_save_trace(&vcpu->arch.host_debug_state.trfcr_el1); + if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) + __debug_save_trace(host_ctxt); }
void __debug_switch_to_guest(struct kvm_vcpu *vcpu) @@ -94,12 +94,12 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu) __debug_switch_to_guest_common(vcpu); }
-void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu) +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) { - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE)) - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1); - if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_restore_trace(vcpu->arch.host_debug_state.trfcr_el1); + if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) + __debug_restore_spe(host_ctxt); + if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) + __debug_restore_trace(host_ctxt); }
void __debug_switch_to_host(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c353a06ee7e6..c8f15e4dab19 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -276,7 +276,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and * before we load guest Stage1. */ - __debug_save_host_buffers_nvhe(vcpu); + __debug_save_host_buffers_nvhe(host_ctxt);
/* * We're about to restore some new MMU state. Make sure @@ -343,7 +343,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */ - __debug_restore_host_buffers_nvhe(vcpu); + __debug_restore_host_buffers_nvhe(host_ctxt);
if (pmu_switch_needed) __pmu_switch_to_host(vcpu);
On 05/10/2023 13:57, James Clark wrote:
pmscr_el1 and trfcr_el1 are currently special cased in the host_debug_state struct, but they're just registers after all so give them entries in the sysreg array and refer to them through the host context.
Signed-off-by: James Clark james.clark@arm.com
Looks good to me.
Reviewed-by: Suzuki K Poulose suzuki.poulose@arm.com
arch/arm64/include/asm/kvm_host.h | 6 ++-- arch/arm64/include/asm/kvm_hyp.h | 4 +-- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 44 +++++++++++++++--------------- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +-- 4 files changed, 28 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e36f7e8a76ce..b5200f199692 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -439,6 +439,8 @@ enum vcpu_sysreg { CNTHP_CVAL_EL2, CNTHV_CTL_EL2, CNTHV_CVAL_EL2,
- PMSCR_EL1, /* Statistical profiling extension */
- TRFCR_EL1, /* Self-hosted trace filters */
NR_SYS_REGS /* Nothing after this line! */ }; @@ -572,10 +574,6 @@ struct kvm_vcpu_arch { struct { /* {Break,watch}point registers */ struct kvm_guest_debug_arch regs;
/* Statistical profiling extension */u64 pmscr_el1;/* Self-hosted trace */u64 trfcr_el1;/* VGIC state */ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b7238c72a04c..37e238f526d7 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -103,8 +103,8 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu); #ifdef __KVM_NVHE_HYPERVISOR__ -void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu); -void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu); +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); #endif void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 89c208112eb7..128a57dddabf 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -14,12 +14,12 @@ #include <asm/kvm_hyp.h> #include <asm/kvm_mmu.h> -static void __debug_save_spe(u64 *pmscr_el1) +static void __debug_save_spe(struct kvm_cpu_context *host_ctxt) { u64 reg; /* Clear pmscr in case of early return */
- *pmscr_el1 = 0;
- ctxt_sys_reg(host_ctxt, PMSCR_EL1) = 0;
/* * At this point, we know that this CPU implements @@ -31,7 +31,7 @@ static void __debug_save_spe(u64 *pmscr_el1) return; /* Yes; save the control register and disable data generation */
- *pmscr_el1 = read_sysreg_s(SYS_PMSCR_EL1);
- ctxt_sys_reg(host_ctxt, PMSCR_EL1) = read_sysreg_s(SYS_PMSCR_EL1); write_sysreg_s(0, SYS_PMSCR_EL1); isb();
@@ -39,21 +39,21 @@ static void __debug_save_spe(u64 *pmscr_el1) psb_csync(); } -static void __debug_restore_spe(u64 pmscr_el1) +static void __debug_restore_spe(struct kvm_cpu_context *host_ctxt) {
- if (!pmscr_el1)
- if (!ctxt_sys_reg(host_ctxt, PMSCR_EL1)) return;
/* The host page table is installed, but not yet synchronised */ isb(); /* Re-enable data generation */
- write_sysreg_s(pmscr_el1, SYS_PMSCR_EL1);
- write_sysreg_s(ctxt_sys_reg(host_ctxt, PMSCR_EL1), SYS_PMSCR_EL1); }
-static void __debug_save_trace(u64 *trfcr_el1) +static void __debug_save_trace(struct kvm_cpu_context *host_ctxt) {
- *trfcr_el1 = 0;
- ctxt_sys_reg(host_ctxt, TRFCR_EL1) = 0;
/* Check if the TRBE is enabled */ if (!(read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E)) @@ -63,30 +63,30 @@ static void __debug_save_trace(u64 *trfcr_el1) * Since access to TRFCR_EL1 is trapped, the guest can't * modify the filtering set by the host. */
- *trfcr_el1 = read_sysreg_s(SYS_TRFCR_EL1);
- ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1); write_sysreg_s(0, SYS_TRFCR_EL1); isb(); /* Drain the trace buffer to memory */ tsb_csync(); }
-static void __debug_restore_trace(u64 trfcr_el1) +static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt) {
- if (!trfcr_el1)
- if (!ctxt_sys_reg(host_ctxt, TRFCR_EL1)) return;
/* Restore trace filter controls */
- write_sysreg_s(trfcr_el1, SYS_TRFCR_EL1);
- write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); }
-void __debug_save_host_buffers_nvhe(struct kvm_vcpu *vcpu) +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) { /* Disable and flush SPE data generation */
- if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE))
__debug_save_spe(&vcpu->arch.host_debug_state.pmscr_el1);
- if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE))
__debug_save_spe(host_ctxt);
- if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR))
__debug_save_trace(&vcpu->arch.host_debug_state.trfcr_el1);
- if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR))
__debug_save_trace(host_ctxt);void __debug_switch_to_guest(struct kvm_vcpu *vcpu) @@ -94,12 +94,12 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu) __debug_switch_to_guest_common(vcpu); } -void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu) +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) {
- if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_SPE))
__debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);- if (vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR))
__debug_restore_trace(vcpu->arch.host_debug_state.trfcr_el1);
- if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE))
__debug_restore_spe(host_ctxt);- if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR))
__debug_restore_trace(host_ctxt);void __debug_switch_to_host(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c353a06ee7e6..c8f15e4dab19 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -276,7 +276,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and * before we load guest Stage1. */
- __debug_save_host_buffers_nvhe(vcpu);
- __debug_save_host_buffers_nvhe(host_ctxt);
/* * We're about to restore some new MMU state. Make sure @@ -343,7 +343,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */
- __debug_restore_host_buffers_nvhe(vcpu);
- __debug_restore_host_buffers_nvhe(host_ctxt);
if (pmu_switch_needed) __pmu_switch_to_host(vcpu);
Add an interface for the Coresight driver to use to set the value of the TRFCR register for the guest. This register controls the exclude settings for trace at different exception levels, and is used to honor the exclude_host and exclude_guest parameters from the Perf session. This will be used to later write TRFCR_EL1 on nVHE at guest switch. For VHE, TRFCR_EL1 is written immediately. Because guest writes to the register are trapped, the value will persist and can't be modified.
The settings must be copied to the vCPU before each run in the same way that PMU events are because the per-cpu struct isn't accessible in protected mode.
Now that both guest and host values are saved, rename trfcr_el1 to host_trfcr_el1 to make it clear that's the value that should be restored on return to the host.
Signed-off-by: James Clark james.clark@arm.com --- arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/debug.c | 26 ++++++++++++++++++++++++++ 3 files changed, 30 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index b5200f199692..8f2b4ec8ea61 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1123,6 +1123,8 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu); void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u32 clr); bool kvm_set_pmuserenr(u64 val); +void kvm_etm_set_guest_trfcr(u64 trfcr_guest); +void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu); #else static inline void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) {} static inline void kvm_clr_pmu_events(u32 clr) {} @@ -1130,6 +1132,7 @@ static inline bool kvm_set_pmuserenr(u64 val) { return false; } +static inline void kvm_etm_set_guest_trfcr(u64 trfcr_guest) {} #endif
void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 78b0970eb8e6..22fab356b88f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1012,6 +1012,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_vgic_flush_hwstate(vcpu);
kvm_pmu_update_vcpu_events(vcpu); + kvm_etm_update_vcpu_events(vcpu);
/* * Ensure we set mode to IN_GUEST_MODE after we disable diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 6a1bad1a921b..19e722359154 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -23,6 +23,12 @@
static DEFINE_PER_CPU(u64, mdcr_el2);
+/* + * Per CPU value for TRFCR that should be applied to any guest vcpu that may + * run on that core in the future. + */ +static DEFINE_PER_CPU(u64, guest_trfcr); + /** * save/restore_guest_debug_regs * @@ -342,3 +348,23 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_SPE); vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); } + +void kvm_etm_set_guest_trfcr(u64 trfcr_guest) +{ + if (has_vhe()) + write_sysreg_s(trfcr_guest, SYS_TRFCR_EL12); + else + *this_cpu_ptr(&guest_trfcr) = trfcr_guest; +} +EXPORT_SYMBOL_GPL(kvm_etm_set_guest_trfcr); + +/* + * Updates the vcpu's view of the etm events for this cpu. Must be + * called before every vcpu run after disabling interrupts, to ensure + * that an interrupt cannot fire and update the structure. + */ +void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu) +{ + if (!has_vhe() && vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) + ctxt_sys_reg(&vcpu->arch.ctxt, TRFCR_EL1) = *this_cpu_ptr(&guest_trfcr); +}
On 05/10/2023 13:57, James Clark wrote:
Add an interface for the Coresight driver to use to set the value of the TRFCR register for the guest. This register controls the exclude
settings for trace at different exception levels, and is used to honor the exclude_host and exclude_guest parameters from the Perf session. This will be used to later write TRFCR_EL1 on nVHE at guest switch. For VHE, TRFCR_EL1 is written immediately. Because guest writes
minor nit: May be it would be clearer to say:
For VHE, the host trace is controlled by TRFCR_EL2 and thus we can write to the TRFCR_EL1 immediately. Because, ...
to the register are trapped, the value will persist and can't be modified.
The settings must be copied to the vCPU before each run in the same way that PMU events are because the per-cpu struct isn't accessible in
super minor nit:
way that PMU events are, because ...
protected mode.
Now that both guest and host values are saved, rename trfcr_el1 to host_trfcr_el1 to make it clear that's the value that should be restored
This seems obsolete ? I couldn't find any reference to host_trfcr_el1 anywhere ?
Otherwise looks good to me.
Suzuki
on return to the host.
Signed-off-by: James Clark james.clark@arm.com
arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/debug.c | 26 ++++++++++++++++++++++++++ 3 files changed, 30 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index b5200f199692..8f2b4ec8ea61 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1123,6 +1123,8 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu); void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u32 clr); bool kvm_set_pmuserenr(u64 val); +void kvm_etm_set_guest_trfcr(u64 trfcr_guest); +void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu); #else static inline void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) {} static inline void kvm_clr_pmu_events(u32 clr) {} @@ -1130,6 +1132,7 @@ static inline bool kvm_set_pmuserenr(u64 val) { return false; } +static inline void kvm_etm_set_guest_trfcr(u64 trfcr_guest) {} #endif void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 78b0970eb8e6..22fab356b88f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1012,6 +1012,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_vgic_flush_hwstate(vcpu); kvm_pmu_update_vcpu_events(vcpu);
kvm_etm_update_vcpu_events(vcpu);/* * Ensure we set mode to IN_GUEST_MODE after we disable diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 6a1bad1a921b..19e722359154 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -23,6 +23,12 @@ static DEFINE_PER_CPU(u64, mdcr_el2); +/*
- Per CPU value for TRFCR that should be applied to any guest vcpu that may
- run on that core in the future.
- */
+static DEFINE_PER_CPU(u64, guest_trfcr);
- /**
- save/restore_guest_debug_regs
@@ -342,3 +348,23 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_SPE); vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); }
+void kvm_etm_set_guest_trfcr(u64 trfcr_guest) +{
- if (has_vhe())
write_sysreg_s(trfcr_guest, SYS_TRFCR_EL12);- else
*this_cpu_ptr(&guest_trfcr) = trfcr_guest;+} +EXPORT_SYMBOL_GPL(kvm_etm_set_guest_trfcr);
+/*
- Updates the vcpu's view of the etm events for this cpu. Must be
- called before every vcpu run after disabling interrupts, to ensure
- that an interrupt cannot fire and update the structure.
- */
+void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu) +{
- if (!has_vhe() && vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR))
ctxt_sys_reg(&vcpu->arch.ctxt, TRFCR_EL1) = *this_cpu_ptr(&guest_trfcr);+}
On 05/10/2023 17:58, Suzuki K Poulose wrote:
On 05/10/2023 13:57, James Clark wrote:
Add an interface for the Coresight driver to use to set the value of the TRFCR register for the guest. This register controls the exclude
settings for trace at different exception levels, and is used to honor the exclude_host and exclude_guest parameters from the Perf session. This will be used to later write TRFCR_EL1 on nVHE at guest switch. For VHE, TRFCR_EL1 is written immediately. Because guest writes
minor nit: May be it would be clearer to say:
For VHE, the host trace is controlled by TRFCR_EL2 and thus we can write to the TRFCR_EL1 immediately. Because, ...
to the register are trapped, the value will persist and can't be modified.
The settings must be copied to the vCPU before each run in the same way that PMU events are because the per-cpu struct isn't accessible in
super minor nit:
way that PMU events are, because ...
protected mode.
Now that both guest and host values are saved, rename trfcr_el1 to host_trfcr_el1 to make it clear that's the value that should be restored
This seems obsolete ? I couldn't find any reference to host_trfcr_el1 anywhere ?
Yep it was from an old version. I remove it and fixed all of the above too.
Otherwise looks good to me.
Suzuki
on return to the host.
Signed-off-by: James Clark james.clark@arm.com
arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/debug.c | 26 ++++++++++++++++++++++++++ 3 files changed, 30 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index b5200f199692..8f2b4ec8ea61 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1123,6 +1123,8 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu); void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u32 clr); bool kvm_set_pmuserenr(u64 val); +void kvm_etm_set_guest_trfcr(u64 trfcr_guest); +void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu); #else static inline void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) {} static inline void kvm_clr_pmu_events(u32 clr) {} @@ -1130,6 +1132,7 @@ static inline bool kvm_set_pmuserenr(u64 val) { return false; } +static inline void kvm_etm_set_guest_trfcr(u64 trfcr_guest) {} #endif void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 78b0970eb8e6..22fab356b88f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1012,6 +1012,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_vgic_flush_hwstate(vcpu); kvm_pmu_update_vcpu_events(vcpu); + kvm_etm_update_vcpu_events(vcpu); /* * Ensure we set mode to IN_GUEST_MODE after we disable diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 6a1bad1a921b..19e722359154 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -23,6 +23,12 @@ static DEFINE_PER_CPU(u64, mdcr_el2); +/*
- Per CPU value for TRFCR that should be applied to any guest vcpu
that may
- run on that core in the future.
- */
+static DEFINE_PER_CPU(u64, guest_trfcr);
/** * save/restore_guest_debug_regs * @@ -342,3 +348,23 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_SPE); vcpu_clear_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); }
+void kvm_etm_set_guest_trfcr(u64 trfcr_guest) +{ + if (has_vhe()) + write_sysreg_s(trfcr_guest, SYS_TRFCR_EL12); + else + *this_cpu_ptr(&guest_trfcr) = trfcr_guest; +} +EXPORT_SYMBOL_GPL(kvm_etm_set_guest_trfcr);
+/*
- Updates the vcpu's view of the etm events for this cpu. Must be
- called before every vcpu run after disabling interrupts, to ensure
- that an interrupt cannot fire and update the structure.
- */
+void kvm_etm_update_vcpu_events(struct kvm_vcpu *vcpu) +{ + if (!has_vhe() && vcpu_get_flag(vcpu, DEBUG_STATE_SAVE_TRFCR)) + ctxt_sys_reg(&vcpu->arch.ctxt, TRFCR_EL1) = *this_cpu_ptr(&guest_trfcr); +}
The guest value for TRFCR requested by the Coresight driver is saved in sysregs[TRFCR_EL1]. On guest switch this value needs to be written to the register. Currently TRFCR is only modified when we want to disable trace completely in guests due to an issue with TRBE. Expand the __debug_save_trace() function to always write to the register if a different value for guests is required, but also keep the existing TRBE disable behavior if that's required.
The TRFCR restore function remains functionally the same, except a value of 0 doesn't mean "don't restore" anymore. Now that we save both guest and host values the register is restored any time the guest and host values differ.
Signed-off-by: James Clark james.clark@arm.com --- arch/arm64/include/asm/kvm_hyp.h | 6 ++- arch/arm64/kvm/debug.c | 13 +++++- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 63 ++++++++++++++++++------------ arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- 4 files changed, 57 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 37e238f526d7..0383fd3d60b5 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -103,8 +103,10 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu);
#ifdef __KVM_NVHE_HYPERVISOR__ -void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); -void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt); +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt); #endif
void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 19e722359154..d949dd354464 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -337,10 +337,21 @@ void kvm_arch_vcpu_load_debug_state_flags(struct kvm_vcpu *vcpu) !(read_sysreg_s(SYS_PMBIDR_EL1) & BIT(PMBIDR_EL1_P_SHIFT))) vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_SPE);
- /* Check if we have TRBE implemented and available at the host */ + /* + * Check if we have TRBE implemented and available at the host. If it's + * in use at the time of guest switch it will need to be disabled and + * then restored. + */ if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceBuffer_SHIFT) && !(read_sysreg_s(SYS_TRBIDR_EL1) & TRBIDR_EL1_P)) vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); + /* + * Also save TRFCR on nVHE if FEAT_TRF (TraceFilt) exists. This will be + * done in cases where use of TRBE doesn't completely disable trace and + * handles the exclude_host/exclude_guest rules of the trace session. + */ + if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceFilt_SHIFT)) + vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR); }
void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 128a57dddabf..c6252029c277 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -51,42 +51,56 @@ static void __debug_restore_spe(struct kvm_cpu_context *host_ctxt) write_sysreg_s(ctxt_sys_reg(host_ctxt, PMSCR_EL1), SYS_PMSCR_EL1); }
-static void __debug_save_trace(struct kvm_cpu_context *host_ctxt) +/* + * Save TRFCR and disable trace completely if TRBE is being used, otherwise + * apply required guest TRFCR value. + */ +static void __debug_save_trace(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { - ctxt_sys_reg(host_ctxt, TRFCR_EL1) = 0; + ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1);
/* Check if the TRBE is enabled */ - if (!(read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E)) - return; - /* - * Prohibit trace generation while we are in guest. - * Since access to TRFCR_EL1 is trapped, the guest can't - * modify the filtering set by the host. - */ - ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1); - write_sysreg_s(0, SYS_TRFCR_EL1); - isb(); - /* Drain the trace buffer to memory */ - tsb_csync(); + if (read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E) { + /* + * Prohibit trace generation while we are in guest. Since access + * to TRFCR_EL1 is trapped, the guest can't modify the filtering + * set by the host. + */ + ctxt_sys_reg(guest_ctxt, TRFCR_EL1) = 0; + write_sysreg_s(0, SYS_TRFCR_EL1); + isb(); + /* Drain the trace buffer to memory */ + tsb_csync(); + } else { + /* + * Not using TRBE, so guest trace works. Apply the guest filters + * provided by the Coresight driver, if different. + */ + if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) != + ctxt_sys_reg(guest_ctxt, TRFCR_EL1)) + write_sysreg_s(ctxt_sys_reg(guest_ctxt, TRFCR_EL1), + SYS_TRFCR_EL1); + } }
-static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt) +static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { - if (!ctxt_sys_reg(host_ctxt, TRFCR_EL1)) - return; - /* Restore trace filter controls */ - write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); + if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) != ctxt_sys_reg(guest_ctxt, TRFCR_EL1)) + write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); }
-void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { /* Disable and flush SPE data generation */ if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) __debug_save_spe(host_ctxt); - /* Disable and flush Self-Hosted Trace generation */ + if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_save_trace(host_ctxt); + __debug_save_trace(host_ctxt, guest_ctxt); }
void __debug_switch_to_guest(struct kvm_vcpu *vcpu) @@ -94,12 +108,13 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu) __debug_switch_to_guest_common(vcpu); }
-void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) __debug_restore_spe(host_ctxt); if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_restore_trace(host_ctxt); + __debug_restore_trace(host_ctxt, guest_ctxt); }
void __debug_switch_to_host(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c8f15e4dab19..55207ec31bd3 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -276,7 +276,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and * before we load guest Stage1. */ - __debug_save_host_buffers_nvhe(host_ctxt); + __debug_save_host_buffers_nvhe(host_ctxt, guest_ctxt);
/* * We're about to restore some new MMU state. Make sure @@ -343,7 +343,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */ - __debug_restore_host_buffers_nvhe(host_ctxt); + __debug_restore_host_buffers_nvhe(host_ctxt, guest_ctxt);
if (pmu_switch_needed) __pmu_switch_to_host(vcpu);
On 05/10/2023 13:57, James Clark wrote:
The guest value for TRFCR requested by the Coresight driver is saved in sysregs[TRFCR_EL1]. On guest switch this value needs to be written to the register. Currently TRFCR is only modified when we want to disable trace completely in guests due to an issue with TRBE. Expand the __debug_save_trace() function to always write to the register if a different value for guests is required, but also keep the existing TRBE disable behavior if that's required.
The TRFCR restore function remains functionally the same, except a value of 0 doesn't mean "don't restore" anymore. Now that we save both guest and host values the register is restored any time the guest and host values differ.
Signed-off-by: James Clark james.clark@arm.com
arch/arm64/include/asm/kvm_hyp.h | 6 ++- arch/arm64/kvm/debug.c | 13 +++++- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 63 ++++++++++++++++++------------ arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- 4 files changed, 57 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 37e238f526d7..0383fd3d60b5 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -103,8 +103,10 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu); #ifdef __KVM_NVHE_HYPERVISOR__ -void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); -void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt);+void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt);void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 19e722359154..d949dd354464 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -337,10 +337,21 @@ void kvm_arch_vcpu_load_debug_state_flags(struct kvm_vcpu *vcpu) !(read_sysreg_s(SYS_PMBIDR_EL1) & BIT(PMBIDR_EL1_P_SHIFT))) vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_SPE);
- /* Check if we have TRBE implemented and available at the host */
- /*
* Check if we have TRBE implemented and available at the host. If it's* in use at the time of guest switch it will need to be disabled and* then restored.*/
As per A3.1 Armv9-A architecture extensions (DDI 0487J.a), FEAT_TRBE mandates FEAT_TRF. So, we could check FEAT_TRF and if we have a hit, skip the TRBE checks. But, having read the code below, it looks like we need separate flags for TRFCR and TRBE.
- /*
* Also save TRFCR on nVHE if FEAT_TRF (TraceFilt) exists. This will be* done in cases where use of TRBE doesn't completely disable trace and* handles the exclude_host/exclude_guest rules of the trace session.*/- if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceFilt_SHIFT))
vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR);
} void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 128a57dddabf..c6252029c277 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -51,42 +51,56 @@ static void __debug_restore_spe(struct kvm_cpu_context *host_ctxt) write_sysreg_s(ctxt_sys_reg(host_ctxt, PMSCR_EL1), SYS_PMSCR_EL1); } -static void __debug_save_trace(struct kvm_cpu_context *host_ctxt) +/*
- Save TRFCR and disable trace completely if TRBE is being used, otherwise
- apply required guest TRFCR value.
- */
+static void __debug_save_trace(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt)
- ctxt_sys_reg(host_ctxt, TRFCR_EL1) = 0;
- ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1);
/* Check if the TRBE is enabled */
- if (!(read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E))
return;- /*
* Prohibit trace generation while we are in guest.* Since access to TRFCR_EL1 is trapped, the guest can't* modify the filtering set by the host.*/- ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1);
- write_sysreg_s(0, SYS_TRFCR_EL1);
- isb();
- /* Drain the trace buffer to memory */
- tsb_csync();
- if (read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E) {
This is problematic. At this point, we are not sure if TRBE is available or not (e.g. we could be on a v8.4 CPU or a v9.0 with TRBE disabled by higher EL). May be we need to add a separate flag to indicate the presence of TRBE.
Suzuki
/** Prohibit trace generation while we are in guest. Since access* to TRFCR_EL1 is trapped, the guest can't modify the filtering* set by the host.*/ctxt_sys_reg(guest_ctxt, TRFCR_EL1) = 0;write_sysreg_s(0, SYS_TRFCR_EL1);isb();/* Drain the trace buffer to memory */tsb_csync();- } else {
/** Not using TRBE, so guest trace works. Apply the guest filters* provided by the Coresight driver, if different.*/if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) !=ctxt_sys_reg(guest_ctxt, TRFCR_EL1))write_sysreg_s(ctxt_sys_reg(guest_ctxt, TRFCR_EL1),SYS_TRFCR_EL1);- } }
-static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt) +static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt)
- if (!ctxt_sys_reg(host_ctxt, TRFCR_EL1))
return;- /* Restore trace filter controls */
- write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1);
- if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) != ctxt_sys_reg(guest_ctxt, TRFCR_EL1))
write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1);-void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt)
- /* Disable and flush Self-Hosted Trace generation */
- if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR))
__debug_save_trace(host_ctxt);
__debug_save_trace(host_ctxt, guest_ctxt);void __debug_switch_to_guest(struct kvm_vcpu *vcpu) @@ -94,12 +108,13 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu) __debug_switch_to_guest_common(vcpu); } -void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt,
struct kvm_cpu_context *guest_ctxt)
__debug_restore_trace(host_ctxt);
__debug_restore_trace(host_ctxt, guest_ctxt);void __debug_switch_to_host(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c8f15e4dab19..55207ec31bd3 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -276,7 +276,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and * before we load guest Stage1. */
- __debug_save_host_buffers_nvhe(host_ctxt);
- __debug_save_host_buffers_nvhe(host_ctxt, guest_ctxt);
/* * We're about to restore some new MMU state. Make sure @@ -343,7 +343,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */
- __debug_restore_host_buffers_nvhe(host_ctxt);
- __debug_restore_host_buffers_nvhe(host_ctxt, guest_ctxt);
if (pmu_switch_needed) __pmu_switch_to_host(vcpu);
On 05/10/2023 19:05, Suzuki K Poulose wrote:
On 05/10/2023 13:57, James Clark wrote:
The guest value for TRFCR requested by the Coresight driver is saved in sysregs[TRFCR_EL1]. On guest switch this value needs to be written to the register. Currently TRFCR is only modified when we want to disable trace completely in guests due to an issue with TRBE. Expand the __debug_save_trace() function to always write to the register if a different value for guests is required, but also keep the existing TRBE disable behavior if that's required.
The TRFCR restore function remains functionally the same, except a value of 0 doesn't mean "don't restore" anymore. Now that we save both guest and host values the register is restored any time the guest and host values differ.
Signed-off-by: James Clark james.clark@arm.com
arch/arm64/include/asm/kvm_hyp.h | 6 ++- arch/arm64/kvm/debug.c | 13 +++++- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 63 ++++++++++++++++++------------ arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- 4 files changed, 57 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 37e238f526d7..0383fd3d60b5 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -103,8 +103,10 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu); void __debug_switch_to_host(struct kvm_vcpu *vcpu); #ifdef __KVM_NVHE_HYPERVISOR__ -void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); -void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt); +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt); +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt); #endif void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c index 19e722359154..d949dd354464 100644 --- a/arch/arm64/kvm/debug.c +++ b/arch/arm64/kvm/debug.c @@ -337,10 +337,21 @@ void kvm_arch_vcpu_load_debug_state_flags(struct kvm_vcpu *vcpu) !(read_sysreg_s(SYS_PMBIDR_EL1) & BIT(PMBIDR_EL1_P_SHIFT))) vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_SPE); - /* Check if we have TRBE implemented and available at the host */ + /* + * Check if we have TRBE implemented and available at the host. If it's + * in use at the time of guest switch it will need to be disabled and + * then restored. + */ if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceBuffer_SHIFT) && !(read_sysreg_s(SYS_TRBIDR_EL1) & TRBIDR_EL1_P)) vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR);
As per A3.1 Armv9-A architecture extensions (DDI 0487J.a), FEAT_TRBE mandates FEAT_TRF. So, we could check FEAT_TRF and if we have a hit, skip the TRBE checks. But, having read the code below, it looks like we need separate flags for TRFCR and TRBE.
+ /* + * Also save TRFCR on nVHE if FEAT_TRF (TraceFilt) exists. This will be + * done in cases where use of TRBE doesn't completely disable trace and + * handles the exclude_host/exclude_guest rules of the trace session. + */ + if (cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_EL1_TraceFilt_SHIFT)) + vcpu_set_flag(vcpu, DEBUG_STATE_SAVE_TRFCR);
} void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/debug-sr.c b/arch/arm64/kvm/hyp/nvhe/debug-sr.c index 128a57dddabf..c6252029c277 100644 --- a/arch/arm64/kvm/hyp/nvhe/debug-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/debug-sr.c @@ -51,42 +51,56 @@ static void __debug_restore_spe(struct kvm_cpu_context *host_ctxt) write_sysreg_s(ctxt_sys_reg(host_ctxt, PMSCR_EL1), SYS_PMSCR_EL1); } -static void __debug_save_trace(struct kvm_cpu_context *host_ctxt) +/*
- Save TRFCR and disable trace completely if TRBE is being used,
otherwise
- apply required guest TRFCR value.
- */
+static void __debug_save_trace(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { - ctxt_sys_reg(host_ctxt, TRFCR_EL1) = 0; + ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1); /* Check if the TRBE is enabled */ - if (!(read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E)) - return; - /* - * Prohibit trace generation while we are in guest. - * Since access to TRFCR_EL1 is trapped, the guest can't - * modify the filtering set by the host. - */ - ctxt_sys_reg(host_ctxt, TRFCR_EL1) = read_sysreg_s(SYS_TRFCR_EL1); - write_sysreg_s(0, SYS_TRFCR_EL1); - isb(); - /* Drain the trace buffer to memory */ - tsb_csync(); + if (read_sysreg_s(SYS_TRBLIMITR_EL1) & TRBLIMITR_EL1_E) {
This is problematic. At this point, we are not sure if TRBE is available or not (e.g. we could be on a v8.4 CPU or a v9.0 with TRBE disabled by higher EL). May be we need to add a separate flag to indicate the presence of TRBE.
Suzuki
Fixed in V3
+ /* + * Prohibit trace generation while we are in guest. Since access + * to TRFCR_EL1 is trapped, the guest can't modify the filtering + * set by the host. + */ + ctxt_sys_reg(guest_ctxt, TRFCR_EL1) = 0; + write_sysreg_s(0, SYS_TRFCR_EL1); + isb(); + /* Drain the trace buffer to memory */ + tsb_csync(); + } else { + /* + * Not using TRBE, so guest trace works. Apply the guest filters + * provided by the Coresight driver, if different. + */ + if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) != + ctxt_sys_reg(guest_ctxt, TRFCR_EL1)) + write_sysreg_s(ctxt_sys_reg(guest_ctxt, TRFCR_EL1), + SYS_TRFCR_EL1); + } } -static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt) +static void __debug_restore_trace(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { - if (!ctxt_sys_reg(host_ctxt, TRFCR_EL1)) - return;
/* Restore trace filter controls */ - write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); + if (ctxt_sys_reg(host_ctxt, TRFCR_EL1) != ctxt_sys_reg(guest_ctxt, TRFCR_EL1)) + write_sysreg_s(ctxt_sys_reg(host_ctxt, TRFCR_EL1), SYS_TRFCR_EL1); } -void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_save_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { /* Disable and flush SPE data generation */ if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) __debug_save_spe(host_ctxt); - /* Disable and flush Self-Hosted Trace generation */
if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_save_trace(host_ctxt); + __debug_save_trace(host_ctxt, guest_ctxt); } void __debug_switch_to_guest(struct kvm_vcpu *vcpu) @@ -94,12 +108,13 @@ void __debug_switch_to_guest(struct kvm_vcpu *vcpu) __debug_switch_to_guest_common(vcpu); } -void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt) +void __debug_restore_host_buffers_nvhe(struct kvm_cpu_context *host_ctxt, + struct kvm_cpu_context *guest_ctxt) { if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_SPE)) __debug_restore_spe(host_ctxt); if (vcpu_get_flag(host_ctxt->__hyp_running_vcpu, DEBUG_STATE_SAVE_TRFCR)) - __debug_restore_trace(host_ctxt); + __debug_restore_trace(host_ctxt, guest_ctxt); } void __debug_switch_to_host(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c8f15e4dab19..55207ec31bd3 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -276,7 +276,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * translation regime to EL2 (via MDCR_EL2_E2PB == 0) and * before we load guest Stage1. */ - __debug_save_host_buffers_nvhe(host_ctxt); + __debug_save_host_buffers_nvhe(host_ctxt, guest_ctxt); /* * We're about to restore some new MMU state. Make sure @@ -343,7 +343,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) * This must come after restoring the host sysregs, since a non-VHE * system may enable SPE here and make use of the TTBRs. */ - __debug_restore_host_buffers_nvhe(host_ctxt); + __debug_restore_host_buffers_nvhe(host_ctxt, guest_ctxt); if (pmu_switch_needed) __pmu_switch_to_host(vcpu);
Currently the userspace and kernel filters for guests are never set, so no trace will be generated for them. Add support for tracing guests by passing the desired TRFCR value to KVM so it can be applied to the guest.
By writing either E1TRE or E0TRE, filtering on either guest kernel or guest userspace is also supported. And if both E1TRE and E0TRE are cleared when exclude_guest is set, that option is supported too. This change also brings exclude_host support which is difficult to add as a separate commit without excess churn and resulting in no trace at all.
Testing =======
The addresses were counted with the following:
$ perf report -D | grep -Eo 'EL2|EL1|EL0' | sort | uniq -c
Guest kernel only:
$ perf record -e cs_etm//Gk -a -- true 535 EL1 1 EL2
Guest user only (only 5 addresses because the guest runs slowly in the model):
$ perf record -e cs_etm//Gu -a -- true 5 EL0
Host kernel only:
$ perf record -e cs_etm//Hk -a -- true 3501 EL2
Host userspace only:
$ perf record -e cs_etm//Hu -a -- true 408 EL0 1 EL2
Signed-off-by: James Clark james.clark@arm.com --- .../coresight/coresight-etm4x-core.c | 42 ++++++++++++++++--- drivers/hwtracing/coresight/coresight-etm4x.h | 2 +- drivers/hwtracing/coresight/coresight-priv.h | 3 ++ 3 files changed, 40 insertions(+), 7 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 77b0271ce6eb..292f9da6aeaf 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -6,6 +6,7 @@ #include <linux/acpi.h> #include <linux/bitops.h> #include <linux/kernel.h> +#include <linux/kvm_host.h> #include <linux/moduleparam.h> #include <linux/init.h> #include <linux/types.h> @@ -271,9 +272,22 @@ static void etm4x_prohibit_trace(struct etmv4_drvdata *drvdata) /* If the CPU doesn't support FEAT_TRF, nothing to do */ if (!drvdata->trfcr) return; + kvm_etm_set_guest_trfcr(0); cpu_prohibit_trace(); }
+static u64 etm4x_get_kern_user_filter(struct etmv4_drvdata *drvdata) +{ + u64 trfcr = drvdata->trfcr; + + if (drvdata->config.mode & ETM_MODE_EXCL_KERN) + trfcr &= ~TRFCR_ELx_ExTRE; + if (drvdata->config.mode & ETM_MODE_EXCL_USER) + trfcr &= ~TRFCR_ELx_E0TRE; + + return trfcr; +} + /* * etm4x_allow_trace - Allow CPU tracing in the respective ELs, * as configured by the drvdata->config.mode for the current @@ -286,18 +300,28 @@ static void etm4x_prohibit_trace(struct etmv4_drvdata *drvdata) */ static void etm4x_allow_trace(struct etmv4_drvdata *drvdata) { - u64 trfcr = drvdata->trfcr; + u64 trfcr;
/* If the CPU doesn't support FEAT_TRF, nothing to do */ - if (!trfcr) + if (!drvdata->trfcr) return;
- if (drvdata->config.mode & ETM_MODE_EXCL_KERN) - trfcr &= ~TRFCR_ELx_ExTRE; - if (drvdata->config.mode & ETM_MODE_EXCL_USER) - trfcr &= ~TRFCR_ELx_E0TRE; + if (drvdata->config.mode & ETM_MODE_EXCL_HOST) + trfcr = drvdata->trfcr & ~(TRFCR_ELx_ExTRE | TRFCR_ELx_E0TRE); + else + trfcr = etm4x_get_kern_user_filter(drvdata);
write_trfcr(trfcr); + + /* Set filters for guests and pass to KVM */ + if (drvdata->config.mode & ETM_MODE_EXCL_GUEST) + trfcr = drvdata->trfcr & ~(TRFCR_ELx_ExTRE | TRFCR_ELx_E0TRE); + else + trfcr = etm4x_get_kern_user_filter(drvdata); + + /* TRFCR_EL1 doesn't have CX so mask it out. */ + trfcr &= ~TRFCR_EL2_CX; + kvm_etm_set_guest_trfcr(trfcr); }
#ifdef CONFIG_ETM4X_IMPDEF_FEATURE @@ -655,6 +679,12 @@ static int etm4_parse_event_config(struct coresight_device *csdev, if (attr->exclude_user) config->mode = ETM_MODE_EXCL_USER;
+ if (attr->exclude_host) + config->mode |= ETM_MODE_EXCL_HOST; + + if (attr->exclude_guest) + config->mode |= ETM_MODE_EXCL_GUEST; + /* Always start from the default config */ etm4_set_default_config(config);
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index 20e2e4cb7614..3f170599822f 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -841,7 +841,7 @@ enum etm_impdef_type { * @s_ex_level: Secure ELs where tracing is supported. */ struct etmv4_config { - u32 mode; + u64 mode; u32 pe_sel; u32 cfg; u32 eventctrl0; diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h index 767076e07970..727dd27ba800 100644 --- a/drivers/hwtracing/coresight/coresight-priv.h +++ b/drivers/hwtracing/coresight/coresight-priv.h @@ -39,6 +39,9 @@
#define ETM_MODE_EXCL_KERN BIT(30) #define ETM_MODE_EXCL_USER BIT(31) +#define ETM_MODE_EXCL_HOST BIT(32) +#define ETM_MODE_EXCL_GUEST BIT(33) + struct cs_pair_attribute { struct device_attribute attr; u32 lo_off;
-
James Clark -
Suzuki K Poulose