The number of cpuidle drivers is increasing more and more. Today we have
in total 24 drivers. A lot of the code is duplicated, at least for the
initialization. A work of consolidation has been done during this year:
* a lot of code cleanup in all the drivers
* time keeping is now part of the framework
* timer broadcast is now part of the framework
* a WFI state function for ARM is defined and used in the drivers
* an init function has been proposed to factor out the initialization across
the drivers (patchset pending)
What has been observed is a lot of code duplicationis, modification made in
the framework takes awhile before reaching the driver which uses an old API,
duplicate routine and bugs, etc ...
It appears the drivers are belonging to different trees, the cpuidle framework
is under linux-pm, the drivers are per SoC tree. The communication is made
difficult because of the different mailing lists where the cpuidle are
submitted.
After this work, it is time to prevent all these problems to occur again.
I propose to move the cpuidle drivers to the drivers/cpuidle directory, hence
having one single submission path for cpuidle in order to have the cpuidle
framework and the different drivers synced.
This series move the AT91 cpuidle driver under drivers/cpuidle. That does not
change the rule to have the patches acked-by the author of the driver.
Note the calxeda and kirkwood drivers are now in drivers/cpuidle.
Daniel Lezcano (2):
ARM: at91: cpuidle: encapsulate the standby code
ARM: at91: cpuidle: move the driver to drivers/cpuidle directory
arch/arm/mach-at91/Makefile | 1 -
arch/arm/mach-at91/cpuidle.c | 66 ------------------------------------------
arch/arm/mach-at91/pm.c | 8 ++++-
drivers/cpuidle/Makefile | 1 +
drivers/cpuidle/at91.c | 55 +++++++++++++++++++++++++++++++++++
5 files changed, 63 insertions(+), 68 deletions(-)
delete mode 100644 arch/arm/mach-at91/cpuidle.c
create mode 100644 drivers/cpuidle/at91.c
--
1.7.9.5
Hi,
This patchset takes advantage of the new per-task load tracking that is
available in the kernel for packing the tasks in as few as possible
CPU/Cluster/Core. It has got 2 packing modes:
-The 1st mode packs the small tasks when the system is not too busy. The main
goal is to reduce the power consumption in the low system load use cases by
minimizing the number of power domain that are enabled but it also keeps the
default behavior which is performance oriented.
-The 2nd mode packs all tasks in as few as possible power domains in order to
improve the power consumption of the system but at the cost of possible
performance decrease because of the increase of the rate of ressources sharing
compared to the default mode.
The packing is done in 3 steps (the last step is only applicable for the
agressive packing mode):
The 1st step looks for the best place to pack tasks in a system according to
its topology and it defines a 1st pack buddy CPU for each CPU if there is one
available. The policy for defining a buddy CPU is that we want to pack at
levels where a group of CPU can be power gated independently from others. To
describe this capability, a new flag SD_SHARE_POWERDOMAIN has been introduced,
that is used to indicate whether the groups of CPUs of a scheduling domain
share their power state. By default, this flag is set in all sched_domain in
order to keep unchanged the current behavior of the scheduler and only ARM
platform clears the SD_SHARE_POWERDOMAIN flag for MC and CPU level.
In a 2nd step, the scheduler checks the load average of a task which wakes up
as well as the load average of the buddy CPU and it can decide to migrate the
light tasks on a not busy buddy. This check is done during the wake up because
small tasks tend to wake up between periodic load balance and asynchronously
to each other which prevents the default mechanism to catch and migrate them
efficiently. A light task is defined by a runnable_avg_sum that is less than
20% of the runnable_avg_period. In fact, the former condition encloses 2 ones:
The average CPU load of the task must be less than 20% and the task must have
been runnable less than 10ms when it woke up last time in order to be
electable for the packing migration. So, a task than runs 1 ms each 5ms will
be considered as a small task but a task that runs 50 ms with a period of
500ms, will not.
Then, the business of the buddy CPU depends of the load average for the rq and
the number of running tasks. A CPU with a load average greater than 50% will
be considered as busy CPU whatever the number of running tasks is and this
threshold will be reduced by the number of running tasks in order to not
increase too much the wake up latency of a task. When the buddy CPU is busy,
the scheduler falls back to default CFS policy.
The 3rd step is only used when the agressive packing mode is enable. In this
case, the CPUs pack their tasks in their buddy until they becomes full. Unlike
the previous step, we can't keep the same buddy so we update it during load
balance. During the periodic load balance, the scheduler computes the activity
of the system thanks the runnable_avg_sum and the cpu_power of all CPUs and
then it defines the CPUs that will be used to handle the current activity. The
selected CPUs will be their own buddy and will participate to the default
load balancing mecanism in order to share the tasks in a fair way, whereas the
not selected CPUs will not, and their buddy will be the last selected CPU.
The behavior can be summarized as: The scheduler defines how many CPUs are
required to handle the current activity, keeps the tasks on these CPUS and
perform normal load balancing (or any evolution of the current load balancer
like the use of runnable load avg from Alex https://lkml.org/lkml/2013/4/1/580)
on this limited number of CPUs . Like the other steps, the CPUs are selected to
minimize the number of power domain that must stay on.
Change since V3:
- Take into account comments on previous version.
- Add an agressive packing mode and a knob to select between the various mode
Change since V2:
- Migrate only a task that wakes up
- Change the light tasks threshold to 20%
- Change the loaded CPU threshold to not pull tasks if the current number of
running tasks is null but the load average is already greater than 50%
- Fix the algorithm for selecting the buddy CPU.
Change since V1:
Patch 2/6
- Change the flag name which was not clear. The new name is
SD_SHARE_POWERDOMAIN.
- Create an architecture dependent function to tune the sched_domain flags
Patch 3/6
- Fix issues in the algorithm that looks for the best buddy CPU
- Use pr_debug instead of pr_info
- Fix for uniprocessor
Patch 4/6
- Remove the use of usage_avg_sum which has not been merged
Patch 5/6
- Change the way the coherency of runnable_avg_sum and runnable_avg_period is
ensured
Patch 6/6
- Use the arch dependent function to set/clear SD_SHARE_POWERDOMAIN for ARM
platform
Previous results for v3:
This series has been tested with hackbench on ARM platform and the results
don't show any performance regression
Hackbench 3.9-rc2 +patches
Mean Time (10 tests): 2.048 2.015
stdev : 0.047 0.068
Previous results for V2:
This series has been tested with MP3 play back on ARM platform:
TC2 HMP (dual CA-15 and 3xCA-7 cluster).
The measurements have been done on an Ubuntu image during 60 seconds of
playback and the result has been normalized to 100.
| CA15 | CA7 | total |
-------------------------------------
default | 81 | 97 | 178 |
pack | 13 | 100 | 113 |
-------------------------------------
Previous results for V1:
The patch-set has been tested on ARM platforms: quad CA-9 SMP and TC2 HMP
(dual CA-15 and 3xCA-7 cluster). For ARM platform, the results have
demonstrated that it's worth packing small tasks at all topology levels.
The performance tests have been done on both platforms with sysbench. The
results don't show any performance regressions. These results are aligned with
the policy which uses the normal behavior with heavy use cases.
test: sysbench --test=cpu --num-threads=N --max-requests=R run
Results below is the average duration of 3 tests on the quad CA-9.
default is the current scheduler behavior (pack buddy CPU is -1)
pack is the scheduler with the pack mechanism
| default | pack |
-----------------------------------
N=8; R=200 | 3.1999 | 3.1921 |
N=8; R=2000 | 31.4939 | 31.4844 |
N=12; R=200 | 3.2043 | 3.2084 |
N=12; R=2000 | 31.4897 | 31.4831 |
N=16; R=200 | 3.1774 | 3.1824 |
N=16; R=2000 | 31.4899 | 31.4897 |
-----------------------------------
The power consumption tests have been done only on TC2 platform which has got
accessible power lines and I have used cyclictest to simulate small tasks. The
tests show some power consumption improvements.
test: cyclictest -t 8 -q -e 1000000 -D 20 & cyclictest -t 8 -q -e 1000000 -D 20
The measurements have been done during 16 seconds and the result has been
normalized to 100
| CA15 | CA7 | total |
-------------------------------------
default | 100 | 40 | 140 |
pack | <1 | 45 | <46 |
-------------------------------------
The A15 cluster is less power efficient than the A7 cluster but if we assume
that the tasks is well spread on both clusters, we can guest estimate that the
power consumption on a dual cluster of CA7 would have been for a default
kernel:
| CA7 | CA7 | total |
-------------------------------------
default | 40 | 40 | 80 |
-------------------------------------
Vincent Guittot (14):
Revert "sched: Introduce temporary FAIR_GROUP_SCHED dependency for
load-tracking"
sched: add a new SD_SHARE_POWERDOMAIN flag for sched_domain
sched: pack small tasks
sched: pack the idle load balance
ARM: sched: clear SD_SHARE_POWERDOMAIN
sched: add a knob to choose the packing level
sched: agressively pack at wake/fork/exec
sched: trig ILB on an idle buddy
sched: evaluate the activity level of the system
sched: update the buddy CPU
sched: filter task pull request
sched: create a new field with available capacity
sched: update the cpu_power
sched: force migration on buddy CPU
arch/arm/kernel/topology.c | 9 +
arch/ia64/include/asm/topology.h | 1 +
arch/tile/include/asm/topology.h | 1 +
include/linux/sched.h | 11 +-
include/linux/sched/sysctl.h | 8 +
include/linux/topology.h | 4 +
kernel/sched/core.c | 14 +-
kernel/sched/fair.c | 393 +++++++++++++++++++++++++++++++++++---
kernel/sched/sched.h | 15 +-
kernel/sysctl.c | 13 ++
10 files changed, 423 insertions(+), 46 deletions(-)
--
1.7.9.5
The new context tracking subsystem unconditionally includes kvm_host.h
headers for the guest enter/exit macros. This causes a compile
failure when KVM is not enabled.
Fix by adding an IS_ENABLED(CONFIG_KVM) check to kvm_host so it can
be included/compiled even when KVM is not enabled.
Cc: Frederic Weisbecker <fweisbec(a)gmail.com>
Signed-off-by: Kevin Hilman <khilman(a)linaro.org>
---
Applies on v3.9-rc2
include/linux/kvm_host.h | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index cad77fe..a942863 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1,6 +1,8 @@
#ifndef __KVM_HOST_H
#define __KVM_HOST_H
+#if IS_ENABLED(CONFIG_KVM)
+
/*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
@@ -1055,5 +1057,8 @@ static inline bool kvm_vcpu_eligible_for_directed_yield(struct kvm_vcpu *vcpu)
}
#endif /* CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT */
+#else
+static inline void __guest_enter(void) { return; }
+static inline void __guest_exit(void) { return; }
+#endif /* IS_ENABLED(CONFIG_KVM) */
#endif
-
--
1.8.1.2
This patchset was called: "Create sched_select_cpu() and use it for workqueues"
for the first three versions.
Earlier discussions over v3, v2 and v1 can be found here:
https://lkml.org/lkml/2013/3/18/364http://lists.linaro.org/pipermail/linaro-dev/2012-November/014344.htmlhttp://www.mail-archive.com/linaro-dev@lists.linaro.org/msg13342.html
V4 is here:
https://lkml.org/lkml/2013/3/31/55
Workqueues can be performance or power oriented. For performance we may want to
keep them running on a single cpu, so that it remains cache hot. For power we
can give scheduler the liberty to choose target cpu for running work handler.
Later one (Power oriented WQ) can be achieved if the workqueue is allocated with
WQ_UNBOUND flag. Enabling CONFIG_WQ_POWER_EFFICIENT will set
'wq_power_efficient' to 'true'. Setting 'power_efficient' boot param will
override value of 'wq_power_efficient' variable. When 'wq_power_efficient' is
set to 'true', we will convert WQ_POWER_EFFICIENT flag to WQ_UNBOUND on wq
allocation. And so scheduler will have the liberty to choose where to run this
work.
Here we are migrating few users of workqueues to WQ_POWER_EFFICIENT. These
drivers are found to be very much active on idle or lightly busy system and
using WQ_POWER_EFFICIENT for these gave impressive results.
These would be used in power saving mode only if relevant configs are enabled
at compile time or in bootargs. Otherwise behavior is unchanged.
Setup:
-----
- ARM Vexpress TC2 - big.LITTLE CPU
- Core 0-1: A15, 2-4: A7
- rootfs: linaro-ubuntu-devel
This patchset has been tested on a big LITTLE system (heterogeneous) but is
useful for all other homogeneous systems as well. During these tests audio was
played in background using aplay.
Results:
-------
Cluster A15 Energy Cluster A7 Energy Total
------------------------- ----------------------- ------
Without this patchset (Energy in Joules):
---------------------------------------------------
0.151162 2.183545 2.334707
0.223730 2.687067 2.910797
0.289687 2.732702 3.022389
0.454198 2.745908 3.200106
0.495552 2.746465 3.242017
Average:
0.322866 2.619137 2.942003
With this patchset (Energy in Joules):
-----------------------------------------------
0.226421 2.283658 2.510079
0.151361 2.236656 2.388017
0.197726 2.249849 2.447575
0.221915 2.229446 2.451361
0.347098 2.257707 2.604805
Average:
0.2289042 2.2514632 2.4803674
Above tests are repeated multiple times and events are tracked using trace-cmd
and analysed using kernelshark. And it was easily noticeable that idle time for
many cpus has increased considerably, which eventually saved some power.
V4->V5:
-------
- Created new wq flag: WQ_POWER_EFFICIENT, config option:
CONFIG_WQ_POWER_EFFICIENT and kernel param workqueue.power_efficient.
- Created few system wide workqueues aligned towards power saving.
V3->V4:
-------
- Dropped changes to kernel/sched directory and hence
sched_select_non_idle_cpu().
- Dropped queue_work_on_any_cpu()
- Created system_freezable_unbound_wq
- Changed all patches accordingly.
V2->V3:
-------
- Dropped changes into core queue_work() API, rather create *_on_any_cpu()
APIs
- Dropped running timers migration patch as that was broken
- Migrated few users of workqueues to use *_on_any_cpu() APIs.
Viresh Kumar (5):
workqueues: Introduce new flag WQ_POWER_EFFICIENT for power oriented
workqueues
workqueue: Add system wide power_efficient workqueues
PHYLIB: queue work on system_power_efficient_wq
block: queue work on power efficient wq
fbcon: queue work on power efficient wq
Documentation/kernel-parameters.txt | 17 +++++++++++++++++
block/blk-core.c | 3 ++-
block/blk-ioc.c | 3 ++-
block/genhd.c | 12 ++++++++----
drivers/net/phy/phy.c | 9 +++++----
drivers/video/console/fbcon.c | 2 +-
include/linux/workqueue.h | 10 ++++++++++
kernel/power/Kconfig | 19 +++++++++++++++++++
kernel/workqueue.c | 24 +++++++++++++++++++++++-
9 files changed, 87 insertions(+), 12 deletions(-)
--
1.7.12.rc2.18.g61b472e
This patch-set implements early printk support for virtio console devices without using any hypercalls.
The current virtio early printk code in kernel expects that hypervisor will provide some mechanism generally a hypercall to support early printk. This patch-set does not break existing hypercall based early print support.
This implementation adds:
1. Early writeonly register named early_wr in virtio console's config space.
2. Host feature flags namely VIRTIO_CONSOLE_F_EARLY_WRITE for telling guest about early-write capability in console device.
Early write mechanism:
1. When a guest wants to out some character, it has to simply write the character to early_wr register in config space of virtio console device.
Pranavkumar Sawargaonkar (2):
virtio: console: Add early writeonly register to config space
arm64: earlyprintk support for virtio-mmio console
Documentation/virtual/virtio-spec.txt | 13 ++++++++++--
arch/arm64/kernel/early_printk.c | 35 +++++++++++++++++++++++++++++++++
include/uapi/linux/virtio_console.h | 3 +++
3 files changed, 49 insertions(+), 2 deletions(-)
--
1.7.9.5
Hi Rafael,
I know I am late for this merge window and you have already closed the gates :)
If you can still take them, its good, otherwise we will push this for v3.10-rc2.
These are minor fixes for ARM big LITTLE cpufreq driver. It doesn't impact any
other driver or cpufreq core and are fairly independent.
Viresh Kumar (5):
cpufreq: ARM big LITTLE: Select PM_OPP
cpufreq: ARM big LITTLE DT: Return correct transition latency
cpufreq: ARM big LITTLE DT: Return CPUFREQ_ETERNAL if clock-latency
isn't found
cpufreq: ARM big LITTLE: Move cpu_to_cluster() to arm_big_little.h
cpufreq: ARM big LITTLE: Improve print message
drivers/cpufreq/Kconfig.arm | 1 +
drivers/cpufreq/arm_big_little.c | 7 +------
drivers/cpufreq/arm_big_little.h | 5 +++++
drivers/cpufreq/arm_big_little_dt.c | 9 +++++----
4 files changed, 12 insertions(+), 10 deletions(-)
--
1.7.12.rc2.18.g61b472e
I just saw your pull request and so these should go in rc2 now.
These are fixes for cpufreq core and common governor part. They mostly impact
systems which have set have_governor_per_policy to true. i.e. big LITTLE cpufreq
driver.
Viresh Kumar (2):
cpufreq: governors: Fix CPUFREQ_GOV_POLICY_{INIT|EXIT} notifiers
cpufreq: Issue CPUFREQ_GOV_POLICY_EXIT notifier before dropping
policy refcount
drivers/cpufreq/cpufreq.c | 6 +++---
drivers/cpufreq/cpufreq_governor.c | 11 +++++++----
drivers/cpufreq/cpufreq_governor.h | 1 +
3 files changed, 11 insertions(+), 7 deletions(-)
--
1.7.12.rc2.18.g61b472e