Hi,
My first mail here, please tell me if this is not the right list for my queries. I have recently got myself a thundersoft s835 development kit and set it up for power measurement. I was looking at creating the energy model for this from scratch and measuring the impact. I stumbled across something which led me to write this mail. The eas integration document mentions no support currently exists for SMT. However, my kernel configuration for some reason has SCHED_SMT enabled. The s835 is not hyper-threaded and so I am thinking it's a mistake. My questions are:
1) Is disabling SCHED_SMT enough or are there any other configurations that need to be changed as well ?
2) I'd like to understand the technical difficulties with enabling SMT support since I have a mips device which has hyperthreading. Is it that the cpu-cluster energy model would not be enough to handle proper task placement when it comes to virtual cores ?
3) Is there any on-going work on adding SMT support or is it considered as not being worth the effort ?
Thanks for reading.
-Jason
This patch set is to optimize the energy computation on Android kernel
android-4.9-eas-dev branch [1];
Patches 0001-0012 are used to refactor the code and some minor optimization,
otherwise the task energy computation is hard to landed into current code;
Patch 0013 "sched/fair: Optimize energy computation with task oriented" is
the core patch in whole patch set, which is mainly used to implement energy
calculation for task. Patch 0014 is a sequential patch to use cached value
so we can get more benefit for performance by exchanging more memory.
Patch 0015 is a trival experiment patch to remove 'idle state estimation'.
The testing uses rt-app to generate synthetic workload, the workload duty
cycles are 1%/5%/10%/20%/30%/40%; the duration is measured interval for
func select_energy_cpu_idx(), which now is used to calculation three
candidates in single run. The result shows this patch set improve for
energy computation duration:
+----------------+-------+-------+-------+-------+-------+--------+
| workload | 1% | 5% | 10% | 20% | 30% | 40% |
+----------------+-------+-------+-------+-------+-------+--------+
| w/o patch set | 17267 | 21227 | 17019 | 13914 | 15002 | 23412 |
| w/t patch set | 10823 | 11924 | 10931 | 10785 | 11139 | 11223 |
+----------------+-------+-------+-------+-------+-------+--------+
| Opt percentage | 37% | 43% | 36% | 22% | 26% | 52% |
+----------------+-------+-------+-------+-------+-------+--------+
The detailed testing ipython notebook you could check [2][3].
[1] https://android.googlesource.com/kernel/common/+/android-4.9-eas-dev
[2] https://github.com/Leo-Yan/lisa/blob/2018_03_17_android_4.9_eas_dev_nrg_com…
[3] https://github.com/Leo-Yan/lisa/blob/2018_03_17_android_4.9_eas_dev_nrg_com…
Leo Yan (15):
sched/fair: Prepare energy env cpumask before energy calculation
sched/fair: Re-define return values for select_energy_cpu_idx()
sched/fair: Reduce indent in select_energy_cpu_brute()
sched/fair: Fix one minor typo
sched/fair: Use per cpu data to maintain energy environment
sched/fair: Use cpumask to track candidates for energy calculation
sched/fair: Lift CPU iteration out of calc_sg_energy()
sched/fair: Introduce new function compute_task_energy()
sched/fair: Decide 'eenv->sg_cap' ahead energy computation
sched/fair: Use eenv::sg_cap to select capacity index
sched/fair: Estimate capacity index ahead energy computation
sched/fair: Refactor compute_energy()
sched/fair: Optimize energy computation with task oriented
sched/fair: Optimize energy calculation with cached energy data
sched/fair: Remove idle state estimation
kernel/sched/fair.c | 542 +++++++++++++++++++++++++---------------------------
1 file changed, 256 insertions(+), 286 deletions(-)
--
1.9.1
hi Chris, Patrick, Quentin,
On android-4.9-eas-dev branch, after the commit of energy_diff
re-factor("sched/fair: re-factor energy_diff to use a single
(extensible) energy_env"), the calculation of group_idle_state for
previous cpu had changed.
Before the re-factor, for eenv_before, there is chance to continue
doing group_idle_state estimation(not "0goto end" directly);
After the re-factor, for cpu_idx=EAS_CPU_PRV, there is no chance to do
estimation("goto end" directly);
Is it deliberate for this change? If yes, do you have any test result for this?
-static int group_idle_state(struct energy_env *eenv, struct sched_group *sg)
+static int group_idle_state(struct energy_env *eenv, int cpu_idx)
{
+ struct sched_group *sg = eenv->sg;
int i, state = INT_MAX;
int src_in_grp, dst_in_grp;
long grp_util = 0;
@@ -5556,8 +5610,10 @@ static int group_idle_state(struct energy_env
*eenv, struct sched_group *sg)
/* Take non-cpuidle idling into account (active idle/arch_cpu_idle()) */
state++;
- src_in_grp = cpumask_test_cpu(eenv->src_cpu, sched_group_cpus(sg));
- dst_in_grp = cpumask_test_cpu(eenv->dst_cpu, sched_group_cpus(sg));
+ src_in_grp = cpumask_test_cpu(eenv->cpu[EAS_CPU_PRV].cpu_id,
+ sched_group_cpus(sg));
+ dst_in_grp = cpumask_test_cpu(eenv->cpu[cpu_idx].cpu_id,
+ sched_group_cpus(sg));
if (src_in_grp == dst_in_grp) {
/* both CPUs under consideration are in the same group or not in
* either group, migration should leave idle state the same.
@@ -5571,7 +5627,7 @@ static int group_idle_state(struct energy_env
*eenv, struct sched_group *sg)
*/
for_each_cpu(i, sched_group_cpus(sg)) {
grp_util += cpu_util_wake(i, eenv->p);
- if (unlikely(i == eenv->trg_cpu))
+ if (unlikely(i == eenv->cpu[cpu_idx].cpu_id))
grp_util += eenv->util_delta;
}
Thanks,
Ke Wang
Hi,
In an attempt to clarify and simplify find_best_target I've identified
four areas that I believe might benefit from
simplifications/clarifications/fixing.
Please consider the ideas below as proposals towards this purpose. Also,
consider these listed from narrow scope to broader scope.
1. Do not consider boost for !prefer_idle tasks, but always try to
minimize capacity_orig. This was more or less done already by not
having boosted and not-prefer idle tasks in typical Android
configurations, but was never enforced in find_best_target code. This
was discussed on this list as well in the "sched/fair: Prefer low
capacity idle-CPU for boosted non-prefer-idle tasks" thread.
Leo, Viresh, I would encourage you to push your patch to Gerrit and
continue the discussion there, as it is a valid point on its own, and
less controversial than the broader scope patch at 2.
2. Do not consider boost for prefer_idle tasks, but always try to
maximize capacity_orig.
This is the discussion that the patch at
https://android-review.googlesource.com/c/kernel/common/+/636583 is
trying to trigger, although in scope it touches on the point above as
well.
3. Make order of CPUs irrelevant for CPU selection in find_best_target.
This is a patch I'll try to push to Gerrit tomorrow after I have more
test results. This will not incorporate code for any of the points
above and it will try to mimic the selection of a CPU that is done now
in find_best target.
But if the points above are proven on their own, this fact will simplify
the code for this item and make the implementation of the item at 2.
unnecessary.
4. Remove the prefer_idle case from find_best_target.
https://android-review.googlesource.com/q/topic:%22strf-mainline%22+(status…
This has the broadest scope but it is more difficult to validate.
Although broader scope items would make some narrow scope items
unnecessary, they have value on their own if the boarder scope items
cannot be proven in a reasonable amount of time, as they will provide
fixes and support earlier than it takes to make a full re-factor of
code.
- 1 will provide valuable fixes now.
- 2 will simplify find_best_target decision logic to facilitate and
simplify 3.
- 3 will add support for tri-gear platforms for which the current order
of CPUs will be incorrect in some cases.
- 4 will bring us closer to the mainline behavior.
I'm adding the present of pretty pictures from what I call
"validation by storm": I've created some kernels with combinations of
the above items and tested them on a Pixel 2 device, just to make sure
there are no important regressions introduced by them. But I believe
all of these need to be validated independently to make sure we don't
miss important corner cases.
Results at https://gist.github.com/ionela-voinescu/f89815591c7f50864188094bb8c53ec4.
Given that it's difficult for all involved to keep up with discussions
on both gerrit and eas-dev, I'd suggest to discuss design and direction
ideas as part of this thread and push patches and individual test
references to android-review directly.
Let me know what you think. All comments are welcomed!
Best regards,
Ionela.
Currently energy calculation in EAS has missed to consider RT pressure,
it's quite possible to select CPU for CFS tasks which has high RT
pressure and finally accumulate total utilization; as result the other
low RT pressure CPUs lose chance to run CFS tasks and reduce contention
between CFS and RT tasks, from performance view this is not optimal;
furthermore this also harms power data due pack RT task and CFS task on
single one CPU is more easily to trigger CPU frequency increasing.
We can measure the summed CPU utilization and calculate the CPU freqency
standard deviation to get to if the tasks can be well spreading within
the same cluster for middle workload case. So below is the comparison
result for video playback on Hikey960 for before and after applied this
patch set (Using schedutil CPUFreq governor):
Without Patch Set: With Patch Set:
CPU Min(Util) Mean(Util) Mean(Util) | Min(Util) Mean(Util) Mean(Util)
0 7 67 205 | 8 52 170
1 4 53 227 | 9 47 188
2 4 57 191 | 8 38 192
3 4 35 165 | 16 47 146
s.d. 1.5 13.3 25.9 | 3.9 5.83 20.9
4 0 35 160 | 10 34 129
5 0 24 129 | 0 30 115
6 0 18 123 | 0 18 95
7 0 12 84 | 0 21 73
s.d. 0 9.8 31.2 | 5 7.5 24.4
The standard diviation for CPU utilization mean value has been decreased
after applying this patch set (Little cluster: 13.3 vs 5.83, big cluster:
9.8 vs 7.5). This also confirm from the average CPU frequency:
Without Patch Set: With Patch Set:
Average Frequency | Average Frequency
LITTLT Cluster 737MHz | 646MHz
big Cluster 916MHz | 922MHz
Leo Yan (4):
sched/fair: Select maximum spare capacity for idle candidate CPUs
sched: Introduce cpu_util_sum()/__cpu_util_sum() functions
sched/fair: Consider RT pressure for find_best_target()
sched/fair: Consider RT/DL pressure for energy calculation
kernel/sched/fair.c | 22 +++++++++++++++++++---
kernel/sched/sched.h | 29 +++++++++++++++++++++++++++++
2 files changed, 48 insertions(+), 3 deletions(-)
--
1.9.1
Hi Patrick,
I am reading the code for eas-4.9-dev branch [1], and found
schedtune_accept_deltas() has been removed from fair.c. I think this
is not purposed and might be wrongly removed by some optimization
patch and should add back?
Not sure if there have following patch to fix this. FYI.
[1] https://android.googlesource.com/kernel/common/+/android-4.9-eas-dev/kernel…
Thanks,
Leo Yan
From: Leo Yan <leo.yan(a)linaro.org>
The homescreen test-case shows unwanted disturbance on the big CPUs on
the Hikey620 platform with Android 4.9. There are multiple reasons for
that though.
By default boost and prefer_idle are enabled for both top-app and
foreground tasks and find_best_target() always (intentionally) prefers
the big CPUs if prefer_idle is enabled. And some of the foreground tasks
(like: DispSync, PowerManagerSer and PhotonicModulat) for the homescreen
test-case get placed on the big CPUs eventually because of that.
Even if prefer_idle is disabled for such foreground tasks, they don't
end up on the little CPUs. The reason being that find_best_target()
still prefers big CPUs if the task is boosted, though some of the
comments in find_best_target() routine say the exact opposite of that.
It eventually depends on the order in which CPUs are processed, which is
from big to little for boosted tasks.
This patch updates the find_best_target() routine to select low capacity
idle-CPU if the task is boosted but doesn't have prefer_idle enabled.
This would be the same for non-boosted tasks as well.
Signed-off-by: Leo Yan <leo.yan(a)linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org>
---
kernel/sched/fair.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index e45047bdd245..4534d8620989 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6998,8 +6998,11 @@ static inline int find_best_target(struct task_struct *p, int *backup_cpu,
int idle_idx = idle_get_state_idx(cpu_rq(i));
/* Select idle CPU with lower cap_orig */
- if (capacity_orig > best_idle_min_cap_orig)
+ if (capacity_orig < best_idle_min_cap_orig)
+ goto found_best_idle_cpu;
+ else if (capacity_orig > best_idle_min_cap_orig)
continue;
+
/* Favor CPUs that won't end up running at a
* high OPP.
*/
@@ -7017,6 +7020,7 @@ static inline int find_best_target(struct task_struct *p, int *backup_cpu,
best_idle_cstate <= idle_idx)
continue;
+found_best_idle_cpu:
/* Keep track of best idle CPU */
best_idle_min_cap_orig = capacity_orig;
target_idle_max_spare_cap = capacity_orig -
--
2.15.0.194.g9af6a3dea062
find_best_target() tries to find the target CPU where the task should be
placed, based on how much would be the utilization of the CPU after the
task is placed on it. This is represented by 'new_util' in the routine.
Currently it adds task_util(p) to the wake_util of the CPU to find that
out, while it should really be adding the boosted task's util to
wake_util, as that's how much the cpu utilization would be.
This is how we used to do it before commit 3bfde3b4f848 ("ANDROID:
sched/fair: Change cpu iteration order in find_best_target()"), was
merged and the commit doesn't describe the rational behind this change.
This patch reverts to the earlier formula to calculate the new_util.
Fixes: 3bfde3b4f848 ("ANDROID: sched/fair: Change cpu iteration order in find_best_target()")
Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org>
---
Just wanted to get some review done over the list before posting it to
gerrit. Not sure if this commit is doing the right thing, but I couldn't
understand why it should be done this way.
This is for the Android 4.9 EAS dev kernel.
kernel/sched/fair.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 88abd5de69ce..1c33a2ddd39c 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6860,14 +6860,13 @@ static inline int find_best_target(struct task_struct *p, int *backup_cpu,
* accounting. However, the blocked utilization may be zero.
*/
wake_util = cpu_util_wake(i, p);
- new_util = wake_util + task_util(p);
/*
* Ensure minimum capacity to grant the required boost.
* The target CPU can be already at a capacity level higher
* than the one required to boost the task.
*/
- new_util = max(min_util, new_util);
+ new_util = wake_util + min_util;
/*
* Include minimum capacity constraint:
--
2.15.0.194.g9af6a3dea062