On Wed, Feb 07, 2024 at 03:47:46PM +0100, Frederic Weisbecker wrote:
Le Tue, Feb 06, 2024 at 04:15:18PM -0300, Marcelo Tosatti a écrit :
On Tue, Feb 06, 2024 at 01:56:23PM +0100, Frederic Weisbecker wrote:
Le Wed, Jan 17, 2024 at 12:15:07PM -0500, Waiman Long a écrit :
On 1/17/24 12:07, Tejun Heo wrote:
Hello,
On Wed, Jan 17, 2024 at 11:35:03AM -0500, Waiman Long wrote:
The first 2 patches are adopted from Federic with minor twists to fix merge conflicts and compilation issue. The rests are for implementing the new cpuset.cpus.isolation_full interface which is essentially a flag to globally enable or disable full CPU isolation on isolated partitions.
I think the interface is a bit premature. The cpuset partition feature is already pretty restrictive and makes it really clear that it's to isolate the CPUs. I think it'd be better to just enable all the isolation features by default. If there are valid use cases which can't be served without disabling some isolation features, we can worry about adding the interface at that point.
My current thought is to make isolated partitions act like isolcpus=domain, additional CPU isolation capabilities are optional and can be turned on using isolation_full. However, I am fine with making all these turned on by default if it is the consensus.
Right it was the consensus last time I tried. Along with the fact that mutating this isolation_full set has to be done on offline CPUs to simplify the whole picture.
So lemme try to summarize what needs to be done:
- An all-isolation feature file (that is, all the HK_TYPE_* things) on/off for
now. And if it ever proves needed, provide a way later for more finegrained tuning.
- This file must only apply to offline CPUs because it avoids migrations and
stuff.
I need to make RCU NOCB tunable only on offline CPUs, which isn't that much changes.
HK_TYPE_TIMER:
- Wrt. timers in general, not much needs to be done, the CPUs are offline. But:
- arch/x86/kvm/x86.c does something weird
- drivers/char/random.c might need some care
- watchdog needs to be (de-)activated
5) HK_TYPE_DOMAIN:
- This one I fear is not mutable, this is isolcpus...
Except for HK_TYPE_DOMAIN, i have never seen anyone use any of this flags.
HK_TYPE_DOMAIN is used by isolcpus=domain,....
HK_TYPE_MANAGED_IRQ is used by isolcpus=managed_irq,...
All the others (except HK_TYPE_SCHED) are used by nohz_full=
I mean i've never seen any use of the individual flags being set.
You either want full isolation (nohz_full and all the flags together, except for HK_TYPE_DOMAIN which is sometimes enabled/disabled), or not.
So why not group them all together ?
Do you know of any separate uses of these flags (except for HK_TYPE_DOMAIN).
Thanks.
HK_TYPE_MANAGED_IRQ:
- I prefer not to think about it :-)
HK_TYPE_TICK:
- Maybe some tiny ticks internals to revisit, I'll check that.
- There is a remote tick to take into consideration, but again the CPUs are offline so it shouldn't be too complicated.
HK_TYPE_WQ:
- Fortunately we already have all the mutable interface in place. But we must make it live nicely with the sysfs workqueue affinity files.
HK_FLAG_SCHED:
- Oops, this one is ignored by nohz_full/isolcpus, isn't it?
Should be removed?
HK_TYPE_RCU:
- That's point 3) and also some kthreads to affine, which leads us
to the following in HK_TYPE_KTHREAD:
HK_FLAG_KTHREAD:
- I'm guessing it's fine as long as isolation_full is also an isolated partition. Then unbound kthreads shouldn't run there.
HK_TYPE_MISC:
- Should be fine as ILB isn't running on offline CPUs.
Thanks.