[PATCH 6.10 090/121] riscv: Disable preemption while handling PR_RISCV_CTX_SW_FENCEI_OFF

6.10-stable review patch. If anyone has any objections, please let me know.

------------------

From: Charlie Jenkins charlie@rivosinc.com

[ Upstream commit 7c1e5b9690b0e14acead4ff98d8a6c40f2dff54b ]

The icache will be flushed in switch_to() if force_icache_flush is true, or in flush_icache_deferred() if icache_stale_mask is set. Between setting force_icache_flush to false and calculating the new icache_stale_mask, preemption needs to be disabled. There are two reasons for this:

1. If CPU migration happens between force_icache_flush = false, and the icache_stale_mask is set, an icache flush will not be emitted. 2. smp_processor_id() is used in set_icache_stale_mask() to mark the current CPU as not needing another flush since a flush will have happened either by userspace or by the kernel when performing the migration. smp_processor_id() is currently called twice with preemption enabled which causes a race condition. It allows icache_stale_mask to be populated with inconsistent CPU ids.

Resolve these two issues by setting the icache_stale_mask before setting force_icache_flush to false, and using get_cpu()/put_cpu() to obtain the smp_processor_id().

Signed-off-by: Charlie Jenkins charlie@rivosinc.com Fixes: 6b9391b581fd ("riscv: Include riscv_set_icache_flush_ctx prctl") Link: https://lore.kernel.org/r/20240903-fix_fencei_optimization-v2-1-8025f20171fc... Signed-off-by: Palmer Dabbelt palmer@rivosinc.com Signed-off-by: Sasha Levin sashal@kernel.org --- arch/riscv/mm/cacheflush.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index a03c994eed3b..b81672729887 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -158,6 +158,7 @@ void __init riscv_init_cbo_blocksizes(void) #ifdef CONFIG_SMP static void set_icache_stale_mask(void) { + int cpu = get_cpu(); cpumask_t *mask; bool stale_cpu;

@@ -168,10 +169,11 @@ static void set_icache_stale_mask(void) * concurrently on different harts. */ mask = &current->mm->context.icache_stale_mask; - stale_cpu = cpumask_test_cpu(smp_processor_id(), mask); + stale_cpu = cpumask_test_cpu(cpu, mask);

cpumask_setall(mask); - cpumask_assign_cpu(smp_processor_id(), mask, stale_cpu); + cpumask_assign_cpu(cpu, mask, stale_cpu); + put_cpu(); } #endif

@@ -239,14 +241,12 @@ int riscv_set_icache_flush_ctx(unsigned long ctx, unsigned long scope) case PR_RISCV_CTX_SW_FENCEI_OFF: switch (scope) { case PR_RISCV_SCOPE_PER_PROCESS: - current->mm->context.force_icache_flush = false; - set_icache_stale_mask(); + current->mm->context.force_icache_flush = false; break; case PR_RISCV_SCOPE_PER_THREAD: - current->thread.force_icache_flush = false; - set_icache_stale_mask(); + current->thread.force_icache_flush = false; break; default: return -EINVAL;