On Fri, Apr 14, 2023, Jeremi Piotrowski wrote:
On Fri, Apr 14, 2023 at 09:49:28AM -0700, Sean Christopherson wrote:
+Jeremi
Adding myself :)
/facepalm
This isn't some mundane detail, Michael!!!
On Fri, Apr 14, 2023, Mathias Krause wrote:
...
OTOH, the backports give nice speed-ups, ranging from ~2.2 times faster for pure EPT (legacy) MMU setups up to 18(!!!) times faster for TDP MMU on v5.10.
Anyone that's enabling the TDP MMU on v5.10 is on their own, we didn't enable the TDP MMU by default until v5.14 for very good reasons.
I backported the whole series down to v5.10 but left out the CR0.WP guest owning patch+fix for v5.4 as the code base is too different to get all the nuances right, as Sean already hinted. However, even this limited backport provides a big performance fix for our use case!
As a compromise of sorts, I propose that we disable the TDP MMU by default on v5.15, and backport these fixes to v6.1. v5.15 and earlier won't get "ludicrous speed", but I think that's perfectly acceptable since KVM has had the suboptimal behavior literally since EPT/NPT support was first added.
Disabling TDP MMU for v5.15, and backporting things to v6.1 works for me.
I'm comfortable backporting to v6.1 as that is recent enough, and there weren't substantial MMU changes between v6.1 and v6.3 in this area. I.e. I have a decent level of confidence that we aren't overlooking some subtle dependency.
For v5.15, I am less confident in the safety of a backport, and more importantly, I think we should disable the TDP MMU by default to mitigate the underlying flaw that makes the 18x speedup possible. That flaw is that KVM can end up freeing and rebuilding TDP MMU roots every time CR0.WP is toggled or a vCPU transitions to/from SMM.
The interesting thing here is that these CR0.WP fixes seem to improve things with legacy MMU as well, and legacy MMU is not affected/touched by [3].
Yep, that's totally expected. The final patch in this series allows KVM to elide VM-Exits when the guest toggles CR0.WP (but only on Intel hardware). Avoiding VM-Exit entirely is a big performance win when the guest is constantly toggling CR0.WP, e.g. each exit is roughly 1500 cycles, versus probalby something like ~50 for a native write to CR0.WP.
So I think you can consider Mathias' ask independent of disabling TDP MMU. On the one hand: there is no regression here. On the other: the gain is big and seems important to him.
Ya, that's the compromise I am proposing. Give v6.1 the full tune-up, but only do the super safe change for v5.15.