vgic_lpi_stress sends MAPTI and MAPC commands during guest GIC setup to map interrupt events to ITT entries and collection IDs to redistributors, respectively.
Theoretically, we have no guarantee that the ITS will finish handling these mapping commands before the selftest calls KVM_SIGNAL_MSI to inject LPIs to the guest. If LPIs are injected before ITS mapping completes, the ITS cannot properly pass the interrupt on to the redistributor.
In practice, KVM processes ITS commands synchronously, so SYNC calls are functionally unnecessary and ignored in vgic_its_handle_command().
However, selftests should test based on ARM specification and be blind to KVM-specific implementation optimizations. Thus, we must update the test to be architecturally compliant and logically correct.
Fix by adding a SYNC command to the selftests ITS library, then calling SYNC after ITS mapping to ensure mapping completes before signal_lpi() writes to GITS_TRANSLATER.
This patch depends on commit a24f7afce048 ("KVM: selftests: fix MAPC RDbase target formatting in vgic_lpi_stress"), which is queued in kvmarm/fixes.
Signed-off-by: Maximilian Dittgen mdittgen@amazon.de --- Validated by the following debug logging to the GITS_CMD_SYNC handler in vgic_its_handle_command():
kvm_info("ITS SYNC command: %016llx %016llx %016llx %016llx\n", its_cmd[0], its_cmd[1], its_cmd[2], its_cmd[3]);
Initialized a selftest guest with 4 vCPUs by:
./vgic_lpi_stress -v 4
Confirmed that an ITS SYNC was successfully called for all 4 vCPUs:
kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000010000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000020000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000030000 0000000000000000 --- tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c | 4 ++++ .../testing/selftests/kvm/include/arm64/gic_v3_its.h | 1 + tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c | 11 +++++++++++ 3 files changed, 16 insertions(+)
diff --git a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c index 687d04463983..e857a605f577 100644 --- a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c +++ b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c @@ -118,6 +118,10 @@ static void guest_setup_gic(void)
guest_setup_its_mappings(); guest_invalidate_all_rdists(); + + /* SYNC to ensure ITS setup is complete */ + for (cpuid = 0; cpuid < test_data.nr_cpus; cpuid++) + its_send_sync_cmd(test_data.cmdq_base_va, cpuid); }
static void guest_code(size_t nr_lpis) diff --git a/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h b/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h index 3722ed9c8f96..58feef3eb386 100644 --- a/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h +++ b/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h @@ -15,5 +15,6 @@ void its_send_mapc_cmd(void *cmdq_base, u32 vcpu_id, u32 collection_id, bool val void its_send_mapti_cmd(void *cmdq_base, u32 device_id, u32 event_id, u32 collection_id, u32 intid); void its_send_invall_cmd(void *cmdq_base, u32 collection_id); +void its_send_sync_cmd(void *cmdq_base, u32 vcpu_id);
#endif // __SELFTESTS_GIC_V3_ITS_H__ diff --git a/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c b/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c index 0e2f8ed90f30..d9ee331074ea 100644 --- a/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c +++ b/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c @@ -253,3 +253,14 @@ void its_send_invall_cmd(void *cmdq_base, u32 collection_id)
its_send_cmd(cmdq_base, &cmd); } + +void its_send_sync_cmd(void *cmdq_base, u32 vcpu_id) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_SYNC); + its_encode_target(&cmd, procnum_to_rdbase(vcpu_id)); + + its_send_cmd(cmdq_base, &cmd); +} +
On Fri, 14 Nov 2025 14:39:02 +0000, Maximilian Dittgen mdittgen@amazon.de wrote:
vgic_lpi_stress sends MAPTI and MAPC commands during guest GIC setup to map interrupt events to ITT entries and collection IDs to redistributors, respectively.
Theoretically, we have no guarantee that the ITS will finish handling these mapping commands before the selftest calls KVM_SIGNAL_MSI to inject LPIs to the guest. If LPIs are injected before ITS mapping completes, the ITS cannot properly pass the interrupt on to the redistributor.
In practice, KVM processes ITS commands synchronously, so SYNC calls are functionally unnecessary and ignored in vgic_its_handle_command().
However, selftests should test based on ARM specification and be blind to KVM-specific implementation optimizations. Thus,
That's hardly an optimisation. Quite the opposite, really. This is an implementation choice to make it simple (well, simple for an ITS emulation...) and not racy.
we must update the test to be architecturally compliant and logically correct.
Fix by adding a SYNC command to the selftests ITS library, then calling SYNC after ITS mapping to ensure mapping completes before signal_lpi() writes to GITS_TRANSLATER.
This patch depends on commit a24f7afce048 ("KVM: selftests: fix MAPC RDbase target formatting in vgic_lpi_stress"), which is queued in kvmarm/fixes.
This sentence has no place in a commit message.
Signed-off-by: Maximilian Dittgen mdittgen@amazon.de
Validated by the following debug logging to the GITS_CMD_SYNC handler in vgic_its_handle_command():
kvm_info("ITS SYNC command: %016llx %016llx %016llx %016llx\n", its_cmd[0], its_cmd[1], its_cmd[2], its_cmd[3]);Initialized a selftest guest with 4 vCPUs by:
./vgic_lpi_stress -v 4Confirmed that an ITS SYNC was successfully called for all 4 vCPUs:
kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000010000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000020000 0000000000000000 kvm [5094]: ITS SYNC command: 0000000000000005 0000000000000000 0000000000030000 0000000000000000
tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c | 4 ++++ .../testing/selftests/kvm/include/arm64/gic_v3_its.h | 1 + tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c | 11 +++++++++++ 3 files changed, 16 insertions(+)
diff --git a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c index 687d04463983..e857a605f577 100644 --- a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c +++ b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c @@ -118,6 +118,10 @@ static void guest_setup_gic(void) guest_setup_its_mappings(); guest_invalidate_all_rdists();
- /* SYNC to ensure ITS setup is complete */
- for (cpuid = 0; cpuid < test_data.nr_cpus; cpuid++)
its_send_sync_cmd(test_data.cmdq_base_va, cpuid);
You are making an implementation assumption here. There is nothing in the spec that says that the GICR_TYPER.Processor_Number associated with a given CPU is the same thing as the CPU number that the selftests infrastructure give you.
It turns out that KVM makes it so that vcpu_id and Processor_Number are the same thing. But given the blurb above about sticking to the architecture and not relying on implementation details, this is not what I'd expect.
Thanks,
M.
On Fri, Nov 14, 2025 at 03:42:30PM +0000, Marc Zyngier wrote:
On Fri, 14 Nov 2025 14:39:02 +0000, Maximilian Dittgen mdittgen@amazon.de wrote:
diff --git a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c index 687d04463983..e857a605f577 100644 --- a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c +++ b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c @@ -118,6 +118,10 @@ static void guest_setup_gic(void) guest_setup_its_mappings(); guest_invalidate_all_rdists();
- /* SYNC to ensure ITS setup is complete */
- for (cpuid = 0; cpuid < test_data.nr_cpus; cpuid++)
its_send_sync_cmd(test_data.cmdq_base_va, cpuid);You are making an implementation assumption here. There is nothing in the spec that says that the GICR_TYPER.Processor_Number associated with a given CPU is the same thing as the CPU number that the selftests infrastructure give you.
This is already rather widely assumed in the selftests GIC library as well as this test (ex. guest_invalidate_all_rdists()). I'd be happy with an assertion in gicv3_cpu_init() that ensures GICR_TYPER.Processor_Number == cpu.
Thanks, Oliver
The selftests GIC library and tests assume that the GICR_TYPER.Processor_number associated with a given CPU is the same as the CPU's selftest index.
Since this assumption is not guaranteed by specification, add an assert in gicv3_cpu_init() that validates this is true.
Signed-off-by: Maximilian Dittgen mdittgen@amazon.de --- tools/testing/selftests/kvm/lib/arm64/gic_v3.c | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/tools/testing/selftests/kvm/lib/arm64/gic_v3.c b/tools/testing/selftests/kvm/lib/arm64/gic_v3.c index 66d05506f78b..50158d08117b 100644 --- a/tools/testing/selftests/kvm/lib/arm64/gic_v3.c +++ b/tools/testing/selftests/kvm/lib/arm64/gic_v3.c @@ -304,6 +304,9 @@ static void gicv3_cpu_init(unsigned int cpu) redist_base_cpu = gicr_base_cpu(cpu); sgi_base = sgi_base_from_redist(redist_base_cpu);
+ /* Verify assumption that GICR_TYPER.Processor_number == cpu */ + GUEST_ASSERT(((readq_relaxed(redist_base_cpu + GICR_TYPER) >> 8) & 0xffff) == cpu); + gicv3_enable_redist(redist_base_cpu);
/*
vgic_lpi_stress sends MAPTI and MAPC commands during guest GIC setup to map interrupt events to ITT entries and collection IDs to redistributors, respectively.
We have no guarantee that the ITS will finish handling these mapping commands before the selftest calls KVM_SIGNAL_MSI to inject LPIs to the guest. If LPIs are injected before ITS mapping completes, the ITS cannot properly pass the interrupt on to the redistributor.
Fix by adding a SYNC command to the selftests ITS library, then calling SYNC after ITS mapping to ensure mapping completes before signal_lpi() writes to GITS_TRANSLATER.
Signed-off-by: Maximilian Dittgen mdittgen@amazon.de --- tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c | 4 ++++ .../testing/selftests/kvm/include/arm64/gic_v3_its.h | 1 + tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c | 11 +++++++++++ 3 files changed, 16 insertions(+)
diff --git a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c index 687d04463983..e857a605f577 100644 --- a/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c +++ b/tools/testing/selftests/kvm/arm64/vgic_lpi_stress.c @@ -118,6 +118,10 @@ static void guest_setup_gic(void)
guest_setup_its_mappings(); guest_invalidate_all_rdists(); + + /* SYNC to ensure ITS setup is complete */ + for (cpuid = 0; cpuid < test_data.nr_cpus; cpuid++) + its_send_sync_cmd(test_data.cmdq_base_va, cpuid); }
static void guest_code(size_t nr_lpis) diff --git a/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h b/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h index 3722ed9c8f96..58feef3eb386 100644 --- a/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h +++ b/tools/testing/selftests/kvm/include/arm64/gic_v3_its.h @@ -15,5 +15,6 @@ void its_send_mapc_cmd(void *cmdq_base, u32 vcpu_id, u32 collection_id, bool val void its_send_mapti_cmd(void *cmdq_base, u32 device_id, u32 event_id, u32 collection_id, u32 intid); void its_send_invall_cmd(void *cmdq_base, u32 collection_id); +void its_send_sync_cmd(void *cmdq_base, u32 vcpu_id);
#endif // __SELFTESTS_GIC_V3_ITS_H__ diff --git a/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c b/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c index 0e2f8ed90f30..d9ee331074ea 100644 --- a/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c +++ b/tools/testing/selftests/kvm/lib/arm64/gic_v3_its.c @@ -253,3 +253,14 @@ void its_send_invall_cmd(void *cmdq_base, u32 collection_id)
its_send_cmd(cmdq_base, &cmd); } + +void its_send_sync_cmd(void *cmdq_base, u32 vcpu_id) +{ + struct its_cmd_block cmd = {}; + + its_encode_cmd(&cmd, GITS_CMD_SYNC); + its_encode_target(&cmd, procnum_to_rdbase(vcpu_id)); + + its_send_cmd(cmdq_base, &cmd); +} +
On Wed, 19 Nov 2025 14:57:43 +0100, Maximilian Dittgen wrote:
The selftests GIC library and tests assume that the GICR_TYPER.Processor_number associated with a given CPU is the same as the CPU's selftest index.
Since this assumption is not guaranteed by specification, add an assert in gicv3_cpu_init() that validates this is true.
[...]
I did a small cleanup to patch #1 to get rid of open-coded shifts / masks.
Please new versions of a series as a separate thread, using a cover letter for anything more than a single patch. This is helpful for tracking patches on my end.
Applied to next, thanks!
[1/2] KVM: selftests: Assert GICR_TYPER.Processor_Number matches selftest CPU number https://git.kernel.org/kvmarm/kvmarm/c/31df012da496 [2/2] KVM: selftests: SYNC after guest ITS setup in vgic_lpi_stress https://git.kernel.org/kvmarm/kvmarm/c/85f329df2931
-- Best, Oliver
linux-kselftest-mirror@lists.linaro.org