On 19/10/17 21:43, Joel Fernandes wrote:
Hi,
I tried an experiment this weekend - basically I have RT threads bound to big CPUs running a fixed period load, with hack bench running with all CPUs allowed. The system is a Pixel2 ARM big.LITTLE 8-core (4x4).
Basically, I changed capacity_orig_of to capacity_of in capacity_spare_wake and wake_cap and I see a good performance improvement. That makes sense because wake_cap would send the task wake up to the slow-path if RT capacity was eating into the CFS capacity on prev/current CPU, and capacity_spare_wake would find a better group with spare-capacity deducted by the RT pressure capacity.
Just so I can follow. The change in capacity_spare_wake() is now part of [PATCH 1/3] sched/fair: Introduce scaled capacity awareness in find_idlest_cpu code path ?
https://patchwork.kernel.org/patch/10002411
One of the concerns for such a change to wake_cap, that I had, was that it might affect upstream cases that may still want to do a select_idle_sibling even if the capacity on the previous/waker's CPU was not enough after deducting RT pressure. In that case, the wake_cap change to use capacity_of would cause it to enter the slow-path for those cases I think.
So then you guys could switch to capacity_of(cpu) only in capacity_spare_wake(). You mentioned in the next paragraph that this change is especially worthwhile.
At the end, wake_cap() and capacity_spare_wake() have different roles and IMHO the benefit of taking rt-pressure into consideration is easier to see with the latter one.
Could you let me know your thoughts about such a change? I heard that capacity_of was attempted before and there might be some cases to consider. Anything from your previous experiences with this change that you could share? Atleast for capacity_spare_wake, the improvements seems to be worthwhile and dramatic in some cases. I also have some more changes I am thinking off to find_idlest_group but I wanted to start a discussion on the spare capacity idea first.
I'm not aware of any use cases which might suffer from using capacity_of(cpu) in both functions. It's just that it is much harder to understand the entire range of behaviours once we switch from original to dynamic capacity. The synthetic testcase you've chosen clearly shows advantage here.
This is related to Rohit's work on RT Capacity awareness, I was talking to him and we were discussing ideas on the implementation.
Unfortunately, his v5 doesn't consider the latest changes in the slow path yet but I see how your test relates to his ideas.
- Dietmar
[...]