[ +eas_dev ]
Hi Leonard,
first of all I would like to inform you that, for EAS specific
question, you should better post your requests on the eas-dev ML:
https://lists.linaro.org/mailman/listinfo/eas-dev
This is where you can reach most of the people working on EAS.
Hereafter are some comments related to your question.
On 05-Jan 06:41, Leonard Crestez wrote:
> After porting the EAS patches and I'd like to do a top-level
> comparison between EAS and non-EAS. Looking through lisa notebooks I
> didn't find anything that obviously fits; many of the tests refuse
> to even run without EAS.
Most of the tests we have on LISA_HOME/tests/eas are actually to test
that EAS is working as expected.
> If there is no good top-level synthetic evaluation for linux maybe I
> could use one for Android?
>
> I'm looking for something that can say something like "EAS consumes
> x% less power".
We actually have a workflow to run a complete set of Android workloads
on a custom target and compare power/performance results corresponding
to different kernels. That suite is named wltest and it's part of
LISA.
Here are the instructions to run wltest:
https://github.com/ARM-software/lisa/tree/master/tools/wltests
lemme know if you have any question/doubts about running that suite of
tests. Do please consider that the Google kernel team has a great
interest on checking wltests results to assess proposed scheduler
changes for the AOSP common kernel.
Here is an example of the report generated by wltest when comparing
WALT vs PELT kernels:
https://gist.github.com/derkling/3a8c3568676a29e608d6dcb15af06241
As a final remark, please do notice that wltest currently supports
out-of-the-box only hikey960 boards with an ACME energy meter.
However, it's relatively easy to integrate the support for different
targets and energy meters. Unfortunately we do not have documentation
available, but... everything needed should be just what you find
under one of platform folders:
https://github.com/ARM-software/lisa/tree/master/tools/wltests/platforms
You can just copy the content of:
https://github.com/ARM-software/lisa/tree/master/tools/wltests/platforms/hi…
and modify the contained files to match your specific target.
Internally we have an integration for Google Pixel 2 devices... but
still did not find time to push/merge it. Lemme know if you are
interested ;-)
Cheers Patrick
--
#include <best/regards.h>
Patrick Bellasi
Hi,
a new EAS integration branch (tag: 20171222_1000) is available on:
http://linux-arm.org/git?p=linux-power.git
News:
Frequency and CPU Invariance (FIE/CIE) now in base
Bugfixes from android.googlesource.com/kernel/common
experimental/android-4.14
(1) 'sched: Per-Sched-domain over utilization' now compiles w/
!CONFIG_SMP
(2) 'drivers base/arch_topology: Detect SD_SHARE_CAP_STATES flag'
now compiles w/ !CONFIG_CPU_FREQ
For further information about main features, test coverage and work
items for next integration please have a look at:
https://developer.arm.com/open-source/energy-aware-scheduling/eas-mainline-…
Best Regards,
-- Dietmar
Since the recent remote cpufreq callback work, its possible that a cpufreq
update is triggered from a remote CPU. For single policies however, the current
code uses the local CPU when trying to determine if the remote sg_cpu entered
idle or is busy. This is incorrect. To remedy this, compare with the nohz tick
idle_calls counter of the remote CPU.
Acked-by: Viresh Kumar <viresh.kumar(a)linaro.org>
Signed-off-by: Joel Fernandes <joelaf(a)google.com>
---
Just resending this which is cpufreq-related as requested by Rafael rebased
on linus/master.
The other 2 patches in my last series which can go in independent of this one are:
https://patchwork.kernel.org/patch/10115395/https://patchwork.kernel.org/patch/10115401/
I'm still waiting on scheduler maintainers to comment on those. Unfortunately,
I haven't heard back anything yet since the last repost of those.
include/linux/tick.h | 1 +
kernel/sched/cpufreq_schedutil.c | 2 +-
kernel/time/tick-sched.c | 13 +++++++++++++
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/include/linux/tick.h b/include/linux/tick.h
index f442d1a42025..7cc35921218e 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -119,6 +119,7 @@ extern void tick_nohz_idle_exit(void);
extern void tick_nohz_irq_exit(void);
extern ktime_t tick_nohz_get_sleep_length(void);
extern unsigned long tick_nohz_get_idle_calls(void);
+extern unsigned long tick_nohz_get_idle_calls_cpu(int cpu);
extern u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time);
extern u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time);
#else /* !CONFIG_NO_HZ_COMMON */
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index 2f52ec0f1539..d6717a3331a1 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -244,7 +244,7 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, unsigned long *util,
#ifdef CONFIG_NO_HZ_COMMON
static bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu)
{
- unsigned long idle_calls = tick_nohz_get_idle_calls();
+ unsigned long idle_calls = tick_nohz_get_idle_calls_cpu(sg_cpu->cpu);
bool ret = idle_calls == sg_cpu->saved_idle_calls;
sg_cpu->saved_idle_calls = idle_calls;
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 99578f06c8d4..77555faf6fbc 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -985,6 +985,19 @@ ktime_t tick_nohz_get_sleep_length(void)
return ts->sleep_length;
}
+/**
+ * tick_nohz_get_idle_calls_cpu - return the current idle calls counter value
+ * for a particular CPU.
+ *
+ * Called from the schedutil frequency scaling governor in scheduler context.
+ */
+unsigned long tick_nohz_get_idle_calls_cpu(int cpu)
+{
+ struct tick_sched *ts = tick_get_tick_sched(cpu);
+
+ return ts->idle_calls;
+}
+
/**
* tick_nohz_get_idle_calls - return the current idle calls counter value
*
--
2.15.1.504.g5279b80103-goog
capacity_spare_wake in the slow path influences choice of idlest groups,
as we search for groups with maximum spare capacity. In scenarios where
RT pressure is high, a sub optimal group can be chosen and hurt
performance of the task being woken up.
This is fixed in this patch by using capacity_of instead of
capacity_orig_of in capacity_spare_wake. Only change since v1 is change
in commit message.
Tests results from improvements with this change are below. More tests
were also done by myself and Matt Fleming to ensure no degradation in
different benchmarks.
1) Rohit ran barrier.c test (details below) with following improvements:
------------------------------------------------------------------------
This was Rohit's original use case for a patch he posted at [1] however
from his recent tests he showed my patch can replace his slow path
changes [1] and there's no need to selectively scan/skip CPUs in
find_idlest_group_cpu in the slow path to get the improvement he sees.
barrier.c (open_mp code) as a micro-benchmark. It does a number of
iterations and barrier sync at the end of each for loop.
Here barrier,c is running in along with ping on CPU 0 and 1 as:
'ping -l 10000 -q -s 10 -f hostX'
barrier.c can be found at:
http://www.spinics.net/lists/kernel/msg2506955.html
Following are the results for the iterations per second with this
micro-benchmark (higher is better), on a 44 core, 2 socket 88 Threads
Intel x86 machine:
+--------+------------------+---------------------------+
|Threads | Without patch | With patch |
| | | |
+--------+--------+---------+-----------------+---------+
| | Mean | Std Dev | Mean | Std Dev |
+--------+--------+---------+-----------------+---------+
|1 | 539.36 | 60.16 | 572.54 (+6.15%) | 40.95 |
|2 | 481.01 | 19.32 | 530.64 (+10.32%)| 56.16 |
|4 | 474.78 | 22.28 | 479.46 (+0.99%) | 18.89 |
|8 | 450.06 | 24.91 | 447.82 (-0.50%) | 12.36 |
|16 | 436.99 | 22.57 | 441.88 (+1.12%) | 7.39 |
|32 | 388.28 | 55.59 | 429.4 (+10.59%)| 31.14 |
|64 | 314.62 | 6.33 | 311.81 (-0.89%) | 11.99 |
+--------+--------+---------+-----------------+---------+
2) ping+hackbench test on bare-metal sever (by Rohit)
-----------------------------------------------------
Here hackbench is running in threaded mode along
with, running ping on CPU 0 and 1 as:
'ping -l 10000 -q -s 10 -f hostX'
This test is running on 2 socket, 20 core and 40 threads Intel x86
machine:
Number of loops is 10000 and runtime is in seconds (Lower is better).
+--------------+-----------------+--------------------------+
|Task Groups | Without patch | With patch |
| +-------+---------+----------------+---------+
|(Groups of 40)| Mean | Std Dev | Mean | Std Dev |
+--------------+-------+---------+----------------+---------+
|1 | 0.851 | 0.007 | 0.828 (+2.77%)| 0.032 |
|2 | 1.083 | 0.203 | 1.087 (-0.37%)| 0.246 |
|4 | 1.601 | 0.051 | 1.611 (-0.62%)| 0.055 |
|8 | 2.837 | 0.060 | 2.827 (+0.35%)| 0.031 |
|16 | 5.139 | 0.133 | 5.107 (+0.63%)| 0.085 |
|25 | 7.569 | 0.142 | 7.503 (+0.88%)| 0.143 |
+--------------+-------+---------+----------------+---------+
[1] https://patchwork.kernel.org/patch/9991635/
Matt Fleming also ran several different hackbench tests and cyclic test
to santiy-check that the patch doesn't harm other usecases.
Reviewed-by: Vincent Guittot <vincent.guittot(a)linaro.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann(a)arm.com>
Tested-by: Matt Fleming <matt(a)codeblueprint.co.uk>
Tested-by: Rohit Jain <rohit.k.jain(a)oracle.com>
Signed-off-by: Joel Fernandes <joelaf(a)google.com>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0989676c50e9..832f2ea069ef 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5726,7 +5726,7 @@ static int cpu_util_wake(int cpu, struct task_struct *p);
static unsigned long capacity_spare_wake(int cpu, struct task_struct *p)
{
- return capacity_orig_of(cpu) - cpu_util_wake(cpu, p);
+ return max_t(long, capacity_of(cpu) - cpu_util_wake(cpu, p), 0);
}
/*
--
2.15.1.504.g5279b80103-goog
capacity_spare_wake in the slow path influences choice of idlest groups,
as we search for groups with maximum spare capacity. In scenarios where
RT pressure is high, a sub optimal group can be chosen and hurt
performance of the task being woken up.
Several tests with results are included below to show improvements with
this change.
1) Hackbench on Pixel 2 Android device (4x4 ARM64 Octa core)
------------------------------------------------------------
Here we have RT activity running on big CPU cluster induced with rt-app,
and running hackbench in parallel. The RT tasks are bound to 4 CPUs on
the big cluster (cpu 4,5,6,7) and have 100ms periodicity with
runtime=20ms sleep=80ms.
Hackbench shows big benefit (30%) improvement when number of tasks is 8
and 32: Note: data is completion time in seconds (lower is better).
Number of loops for 8 and 16 tasks is 50000, and for 32 tasks its 20000.
+--------+-----+-------+-------------------+---------------------------+
| groups | fds | tasks | Without Patch | With Patch |
+--------+-----+-------+---------+---------+-----------------+---------+
| | | | Mean | Stdev | Mean | Stdev |
| | | +-------------------+-----------------+---------+
| 1 | 8 | 8 | 1.0534 | 0.13722 | 0.7293 (+30.7%) | 0.02653 |
| 2 | 8 | 16 | 1.6219 | 0.16631 | 1.6391 (-1%) | 0.24001 |
| 4 | 8 | 32 | 1.2538 | 0.13086 | 1.1080 (+11.6%) | 0.16201 |
+--------+-----+-------+---------+---------+-----------------+---------+
2) Rohit ran barrier.c test (details below) with following improvements:
------------------------------------------------------------------------
This was Rohit's original use case for a patch he posted at [1] however
from his recent tests he showed my patch can replace his slow path
changes [1] and there's no need to selectively scan/skip CPUs in
find_idlest_group_cpu in the slow path to get the improvement he sees.
barrier.c (open_mp code) as a micro-benchmark. It does a number of
iterations and barrier sync at the end of each for loop.
Here barrier,c is running in along with ping on CPU 0 and 1 as:
'ping -l 10000 -q -s 10 -f hostX'
barrier.c can be found at:
http://www.spinics.net/lists/kernel/msg2506955.html
Following are the results for the iterations per second with this
micro-benchmark (higher is better), on a 44 core, 2 socket 88 Threads
Intel x86 machine:
+--------+------------------+---------------------------+
|Threads | Without patch | With patch |
| | | |
+--------+--------+---------+-----------------+---------+
| | Mean | Std Dev | Mean | Std Dev |
+--------+--------+---------+-----------------+---------+
|1 | 539.36 | 60.16 | 572.54 (+6.15%) | 40.95 |
|2 | 481.01 | 19.32 | 530.64 (+10.32%)| 56.16 |
|4 | 474.78 | 22.28 | 479.46 (+0.99%) | 18.89 |
|8 | 450.06 | 24.91 | 447.82 (-0.50%) | 12.36 |
|16 | 436.99 | 22.57 | 441.88 (+1.12%) | 7.39 |
|32 | 388.28 | 55.59 | 429.4 (+10.59%)| 31.14 |
|64 | 314.62 | 6.33 | 311.81 (-0.89%) | 11.99 |
+--------+--------+---------+-----------------+---------+
3) ping+hackbench test on bare-metal sever (Rohit ran this test)
----------------------------------------------------------------
Here hackbench is running in threaded mode along
with, running ping on CPU 0 and 1 as:
'ping -l 10000 -q -s 10 -f hostX'
This test is running on 2 socket, 20 core and 40 threads Intel x86
machine:
Number of loops is 10000 and runtime is in seconds (Lower is better).
+--------------+-----------------+--------------------------+
|Task Groups | Without patch | With patch |
| +-------+---------+----------------+---------+
|(Groups of 40)| Mean | Std Dev | Mean | Std Dev |
+--------------+-------+---------+----------------+---------+
|1 | 0.851 | 0.007 | 0.828 (+2.77%)| 0.032 |
|2 | 1.083 | 0.203 | 1.087 (-0.37%)| 0.246 |
|4 | 1.601 | 0.051 | 1.611 (-0.62%)| 0.055 |
|8 | 2.837 | 0.060 | 2.827 (+0.35%)| 0.031 |
|16 | 5.139 | 0.133 | 5.107 (+0.63%)| 0.085 |
|25 | 7.569 | 0.142 | 7.503 (+0.88%)| 0.143 |
+--------------+-------+---------+----------------+---------+
[1] https://patchwork.kernel.org/patch/9991635/
Matt Fleming also ran cyclictest and several different hackbench tests
on his test machines to santiy-check that the patch doesn't harm any
of his usecases.
Cc: Dietmar Eggemann <dietmar.eggemann(a)arm.com>
Cc: Vincent Guittot <vincent.guittot(a)linaro.org>
Cc: Morten Ramussen <morten.rasmussen(a)arm.com>
Cc: Brendan Jackman <brendan.jackman(a)arm.com>
Tested-by: Rohit Jain <rohit.k.jain(a)oracle.com>
Tested-by: Matt Fleming <matt(a)codeblueprint.co.uk>
Signed-off-by: Joel Fernandes <joelaf(a)google.com>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 56f343b8e749..ba9609407cb9 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5724,7 +5724,7 @@ static int cpu_util_wake(int cpu, struct task_struct *p);
static unsigned long capacity_spare_wake(int cpu, struct task_struct *p)
{
- return capacity_orig_of(cpu) - cpu_util_wake(cpu, p);
+ return max_t(long, capacity_of(cpu) - cpu_util_wake(cpu, p), 0);
}
/*
--
2.15.0.448.gf294e3d99a-goog
Hi guys,
I've just pushed here:
git://linux-arm.org/linux-pb.git eas/v1.5/util_est/hikey960
a backport of the util_est patches [1] recently posted on LKML.
Apart from the util_est specific patches, this branch is based on top
of some patches (suggested by Linaro) to improve power/performance
testing on Hikey960.
Moreover, at the top there are some additional patches to
test with different PELT half-life values.
In attachment you can also find a "series file" which can be used with
LISA's wltest.
Unfortunately, so fare there have not been much review feedbacks on LKML.
Would be nice if someone interested can give it a go and report on the list.
Cheers Patrick
[1] https://lkml.org/lkml/2017/12/5/634
--
#include <best/regards.h>
Patrick Bellasi
Good day!
I have noticed since release that EM for the Pixel 2 doesnt cover each
frequency step. 22 steps for small cores, 31 steps for big cores.
There are 22 tuples for the small cores but only 27 tuples for big cores.
I have checked and the Pixel 2 is using all frequency steps for both small
and big cores, so why doesnt the EM account for the last 4 freq steps for
big cores?
Thanks as always for taking the time to answer my questions.
Kind Regards,
Zachariah Kennedy
Hello EAS developers,
This email is to inform you about the latest EAS integration branch that
was published last Friday. All the information on where to get the
branch from are available at:
https://developer.arm.com/open-source/energy-aware-scheduling/EAS%20Mainlin…
The integration branch was conceived to keep the latest EAS patches on
track with tip/sched/core. Hence, on top of that the integration branch
puts:
- some new scheduler features, i.e. patches that relate to scheduler but
are not main components of EAS
- EAS-core patches
- debug patches, i.e. trace events, procfs interfaces, etc.
Integration will happen every two weeks. The above website covers the
main additions to each integration and the next work items for the ones
that will follow.
Kind regards,
Michele