On 21/07/23 18:15, Nadav Amit wrote:
On Jul 20, 2023, at 9:30 AM, Valentin Schneider vschneid@redhat.com wrote:
vunmap()'s issued from housekeeping CPUs are a relatively common source of interference for isolated NOHZ_FULL CPUs, as they are hit by the flush_tlb_kernel_range() IPIs.
Given that CPUs executing in userspace do not access data in the vmalloc range, these IPIs could be deferred until their next kernel entry.
So I think there are a few assumptions here that it seems suitable to confirm and acknowledge the major one in the commit log (assuming they hold).
There is an assumption that VMAP page-tables are not freed. I actually never paid attention to that, but skimming the code it does seem so. To clarify the issue: if page-tables were freed and their pages were reused, there would be a problem that page-walk caches for instance would be used and “junk” entries from the reused pages would be used. See [1].
Thanks for looking into this and sharing context. This is an area I don't have much experience with, so help is much appreciated!
Indeed, accessing addresses that should be impacted by a TLB flush *before* executing the deferred flush is an issue. Deferring sync_core() for instruction patching is a similar problem - it's all in the shape of "access @addr impacted by @operation during kernel entry, before actually executing @operation".
AFAICT the only reasonable way to go about the deferral is to prove that no such access happens before the deferred @operation is done. We got to prove that for sync_core() deferral, cf. PATCH 18.
I'd like to reason about it for deferring vunmap TLB flushes:
What addresses in VMAP range, other than the stack, can early entry code access? Yes, the ranges can be checked at runtime, but is there any chance of figuring this out e.g. at build-time?
I would also assume the memory-hot-unplug of some sorts is not an issue, (i.e., you cannot have a stale TLB entry pointing to memory that was unplugged).
I also think that there might be speculative code execution using stale TLB entries that would point to memory that has been reused and perhaps controllable by the user. If somehow the CPU/OS is tricked to use the stale executable TLB entries early enough on kernel entry that might be an issue. I guess it is probably theoretical issue, but it would be helpful to confirm.
In general, deferring TLB flushes can be done safely. This patch, I think, takes it one step forward and allows the reuse of the memory before the TLB flush is actually done. This is more dangerous.
[1] https://lore.kernel.org/lkml/tip-b956575bed91ecfb136a8300742ecbbf451471ab@gi...