"Power" is a very bad term in the scheduler context. There are so many meanings that can be attached to it. And with the upcoming "power aware" scheduler work confusion is sure to happen.
The definition of "power" is typically the rate at which work is performed, energy is converted or electric energy is transferred. The notion of "compute capacity" is rather at odds with "power" to the point many comments in the code have to make it explicit that "capacity" is the actual intended meaning.
So let's make it clear what we man by using "capacity" in place of "power" directly in the code. That will make the introduction of actual "power consumption" concepts much clearer later on.
This is based on the latest tip tree where scheduler changes are already queued.
Note: The diffstat is not completely symetric wrt added/removed lines as some comments were reflowed.
arch/arm/kernel/topology.c | 54 +++---- include/linux/sched.h | 8 +- kernel/sched/core.c | 89 ++++++----- kernel/sched/fair.c | 322 ++++++++++++++++++++------------------- kernel/sched/sched.h | 18 +-- 5 files changed, 246 insertions(+), 245 deletions(-)
Nicolas
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
To make things explicit and not create more confusion with the existing "capacity" member, let's rename things as follows:
power -> compute_capacity capacity -> task_capacity
Note: none of those fields are actually used outside update_numa_stats().
Signed-off-by: Nicolas Pitre nico@linaro.org --- kernel/sched/fair.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 28ccf502c6..e375dcc3f2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1026,10 +1026,10 @@ struct numa_stats { unsigned long load;
/* Total compute capacity of CPUs on a node */ - unsigned long power; + unsigned long compute_capacity;
/* Approximate capacity in terms of runnable tasks on a node */ - unsigned long capacity; + unsigned long task_capacity; int has_capacity; };
@@ -1046,7 +1046,7 @@ static void update_numa_stats(struct numa_stats *ns, int nid)
ns->nr_running += rq->nr_running; ns->load += weighted_cpuload(cpu); - ns->power += power_of(cpu); + ns->compute_capacity += power_of(cpu);
cpus++; } @@ -1062,9 +1062,10 @@ static void update_numa_stats(struct numa_stats *ns, int nid) if (!cpus) return;
- ns->load = (ns->load * SCHED_POWER_SCALE) / ns->power; - ns->capacity = DIV_ROUND_CLOSEST(ns->power, SCHED_POWER_SCALE); - ns->has_capacity = (ns->nr_running < ns->capacity); + ns->load = (ns->load * SCHED_POWER_SCALE) / ns->compute_capacity; + ns->task_capacity = + DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_POWER_SCALE); + ns->has_capacity = (ns->nr_running < ns->task_capacity); }
struct task_numa_env {
The capacity of a CPU/group should be some intrinsic value that doesn't change with task placement. It is like a container which capacity is stable regardless of the amount of liquid in it... unless the container itself is crushed that is, but that's another story.
Therefore let's rename "has_capacity" to "has_free_capacity" in order to better convey the wanted meaning.
Signed-off-by: Nicolas Pitre nico@linaro.org --- kernel/sched/fair.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e375dcc3f2..0eda4c527e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1030,7 +1030,7 @@ struct numa_stats {
/* Approximate capacity in terms of runnable tasks on a node */ unsigned long task_capacity; - int has_capacity; + int has_free_capacity; };
/* @@ -1056,8 +1056,8 @@ static void update_numa_stats(struct numa_stats *ns, int nid) * the @ns structure is NULL'ed and task_numa_compare() will * not find this node attractive. * - * We'll either bail at !has_capacity, or we'll detect a huge imbalance - * and bail there. + * We'll either bail at !has_free_capacity, or we'll detect a huge + * imbalance and bail there. */ if (!cpus) return; @@ -1065,7 +1065,7 @@ static void update_numa_stats(struct numa_stats *ns, int nid) ns->load = (ns->load * SCHED_POWER_SCALE) / ns->compute_capacity; ns->task_capacity = DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_POWER_SCALE); - ns->has_capacity = (ns->nr_running < ns->task_capacity); + ns->has_free_capacity = (ns->nr_running < ns->task_capacity); }
struct task_numa_env { @@ -1167,8 +1167,8 @@ static void task_numa_compare(struct task_numa_env *env,
if (!cur) { /* Is there capacity at our destination? */ - if (env->src_stats.has_capacity && - !env->dst_stats.has_capacity) + if (env->src_stats.has_free_capacity && + !env->dst_stats.has_free_capacity) goto unlock;
goto balance; @@ -1276,8 +1276,8 @@ static int task_numa_migrate(struct task_struct *p) groupimp = group_weight(p, env.dst_nid) - groupweight; update_numa_stats(&env.dst_stats, env.dst_nid);
- /* If the preferred nid has capacity, try to use it. */ - if (env.dst_stats.has_capacity) + /* If the preferred nid has free capacity, try to use it. */ + if (env.dst_stats.has_free_capacity) task_numa_find_cpu(&env, taskimp, groupimp);
/* No space available on the preferred nid. Look elsewhere. */ @@ -5491,7 +5491,7 @@ struct sg_lb_stats { unsigned int idle_cpus; unsigned int group_weight; int group_imb; /* Is there an imbalance in the group ? */ - int group_has_capacity; /* Is there extra capacity in the group? */ + int group_has_free_capacity; #ifdef CONFIG_NUMA_BALANCING unsigned int nr_numa_running; unsigned int nr_preferred_running; @@ -5858,7 +5858,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, sgs->group_capacity = sg_capacity(env, group);
if (sgs->group_capacity > sgs->sum_nr_running) - sgs->group_has_capacity = 1; + sgs->group_has_free_capacity = 1; }
/** @@ -5982,7 +5982,7 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd * with a large weight task outweighs the tasks on the system). */ if (prefer_sibling && sds->local && - sds->local_stat.group_has_capacity) + sds->local_stat.group_has_free_capacity) sgs->group_capacity = min(sgs->group_capacity, 1U);
if (update_sd_pick_busiest(env, sds, sg, sgs)) { @@ -6242,8 +6242,8 @@ static struct sched_group *find_busiest_group(struct lb_env *env) goto force_balance;
/* SD_BALANCE_NEWIDLE trumps SMP nice when underutilized */ - if (env->idle == CPU_NEWLY_IDLE && local->group_has_capacity && - !busiest->group_has_capacity) + if (env->idle == CPU_NEWLY_IDLE && local->group_has_free_capacity && + !busiest->group_has_free_capacity) goto force_balance;
/*
On Wed, May 14, 2014 at 04:57:06PM -0400, Nicolas Pitre wrote:
The capacity of a CPU/group should be some intrinsic value that doesn't change with task placement. It is like a container which capacity is stable regardless of the amount of liquid in it... unless the container itself is crushed that is, but that's another story.
Therefore let's rename "has_capacity" to "has_free_capacity" in order to better convey the wanted meaning.
Yeah, not really, the whole capacity thing is bollocks, but sure we can rename it for as long as it still lives ;-)
On 15 May 2014 09:21, Peter Zijlstra peterz@infradead.org wrote:
On Wed, May 14, 2014 at 04:57:06PM -0400, Nicolas Pitre wrote:
The capacity of a CPU/group should be some intrinsic value that doesn't change with task placement. It is like a container which capacity is stable regardless of the amount of liquid in it... unless the container itself is crushed that is, but that's another story.
Therefore let's rename "has_capacity" to "has_free_capacity" in order to better convey the wanted meaning.
Yeah, not really, the whole capacity thing is bollocks, but sure we can rename it for as long as it still lives ;-)
Hi,
Yes, I'm preparing an update of my rework around cpu_power and this field will disappear
Vincent
We have "power" (which should actually become "capacity") and "capacity" which is a scaled down "capacity factor" in terms of possible tasks. Let's use "capa_factor" to make room for proper usage of "capacity" later.
Signed-off-by: Nicolas Pitre nico@linaro.org --- kernel/sched/fair.c | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0eda4c527e..2633c42692 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5487,7 +5487,7 @@ struct sg_lb_stats { unsigned long load_per_task; unsigned long group_power; unsigned int sum_nr_running; /* Nr tasks running in the group */ - unsigned int group_capacity; + unsigned int group_capa_factor; unsigned int idle_cpus; unsigned int group_weight; int group_imb; /* Is there an imbalance in the group ? */ @@ -5782,15 +5782,15 @@ static inline int sg_imbalanced(struct sched_group *group) }
/* - * Compute the group capacity. + * Compute the group capacity factor. * * Avoid the issue where N*frac(smt_power) >= 1 creates 'phantom' cores by * first dividing out the smt factor and computing the actual number of cores * and limit power unit capacity with that. */ -static inline int sg_capacity(struct lb_env *env, struct sched_group *group) +static inline int sg_capa_factor(struct lb_env *env, struct sched_group *group) { - unsigned int capacity, smt, cpus; + unsigned int capa_factor, smt, cpus; unsigned int power, power_orig;
power = group->sgp->power; @@ -5799,13 +5799,13 @@ static inline int sg_capacity(struct lb_env *env, struct sched_group *group)
/* smt := ceil(cpus / power), assumes: 1 < smt_power < 2 */ smt = DIV_ROUND_UP(SCHED_POWER_SCALE * cpus, power_orig); - capacity = cpus / smt; /* cores */ + capa_factor = cpus / smt; /* cores */
- capacity = min_t(unsigned, capacity, DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE)); - if (!capacity) - capacity = fix_small_capacity(env->sd, group); + capa_factor = min_t(unsigned, capa_factor, DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE)); + if (!capa_factor) + capa_factor = fix_small_capacity(env->sd, group);
- return capacity; + return capa_factor; }
/** @@ -5855,9 +5855,9 @@ static inline void update_sg_lb_stats(struct lb_env *env, sgs->group_weight = group->group_weight;
sgs->group_imb = sg_imbalanced(group); - sgs->group_capacity = sg_capacity(env, group); + sgs->group_capa_factor = sg_capa_factor(env, group);
- if (sgs->group_capacity > sgs->sum_nr_running) + if (sgs->group_capa_factor > sgs->sum_nr_running) sgs->group_has_free_capacity = 1; }
@@ -5882,7 +5882,7 @@ static bool update_sd_pick_busiest(struct lb_env *env, if (sgs->avg_load <= sds->busiest_stat.avg_load) return false;
- if (sgs->sum_nr_running > sgs->group_capacity) + if (sgs->sum_nr_running > sgs->group_capa_factor) return true;
if (sgs->group_imb) @@ -5973,17 +5973,17 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd
/* * In case the child domain prefers tasks go to siblings - * first, lower the sg capacity to one so that we'll try + * first, lower the sg capacity factor to one so that we'll try * and move all the excess tasks away. We lower the capacity * of a group only if the local group has the capacity to fit - * these excess tasks, i.e. nr_running < group_capacity. The + * these excess tasks, i.e. nr_running < group_capa_factor. The * extra check prevents the case where you always pull from the * heaviest group when it is already under-utilized (possible * with a large weight task outweighs the tasks on the system). */ if (prefer_sibling && sds->local && sds->local_stat.group_has_free_capacity) - sgs->group_capacity = min(sgs->group_capacity, 1U); + sgs->group_capa_factor = min(sgs->group_capa_factor, 1U);
if (update_sd_pick_busiest(env, sds, sg, sgs)) { sds->busiest = sg; @@ -6157,7 +6157,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s * have to drop below capacity to reach cpu-load equilibrium. */ load_above_capacity = - (busiest->sum_nr_running - busiest->group_capacity); + (busiest->sum_nr_running - busiest->group_capa_factor);
load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_POWER_SCALE); load_above_capacity /= busiest->group_power; @@ -6301,7 +6301,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, int i;
for_each_cpu_and(i, sched_group_cpus(group), env->cpus) { - unsigned long power, capacity, wl; + unsigned long power, capa_factor, wl; enum fbq_type rt;
rq = cpu_rq(i); @@ -6330,9 +6330,9 @@ static struct rq *find_busiest_queue(struct lb_env *env, continue;
power = power_of(i); - capacity = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE); - if (!capacity) - capacity = fix_small_capacity(env->sd, group); + capa_factor = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE); + if (!capa_factor) + capa_factor = fix_small_capacity(env->sd, group);
wl = weighted_cpuload(i);
@@ -6340,7 +6340,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, * When comparing with imbalance, use weighted_cpuload() * which is not scaled with the cpu power. */ - if (capacity && rq->nr_running == 1 && wl > env->imbalance) + if (capa_factor && rq->nr_running == 1 && wl > env->imbalance) continue;
/*
On 14 May 2014 22:57, Nicolas Pitre nicolas.pitre@linaro.org wrote:
We have "power" (which should actually become "capacity") and "capacity" which is a scaled down "capacity factor" in terms of possible tasks. Let's use "capa_factor" to make room for proper usage of "capacity" later.
Signed-off-by: Nicolas Pitre nico@linaro.org
kernel/sched/fair.c | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0eda4c527e..2633c42692 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5487,7 +5487,7 @@ struct sg_lb_stats { unsigned long load_per_task; unsigned long group_power; unsigned int sum_nr_running; /* Nr tasks running in the group */
unsigned int group_capacity;
unsigned int group_capa_factor;
As it is mainly compared to sum_nr_running, you might rename it to group_nr_capacity instead of group_capa_factor
unsigned int idle_cpus; unsigned int group_weight; int group_imb; /* Is there an imbalance in the group ? */@@ -5782,15 +5782,15 @@ static inline int sg_imbalanced(struct sched_group *group) }
/*
- Compute the group capacity.
*/
- Compute the group capacity factor.
- Avoid the issue where N*frac(smt_power) >= 1 creates 'phantom' cores by
- first dividing out the smt factor and computing the actual number of cores
- and limit power unit capacity with that.
-static inline int sg_capacity(struct lb_env *env, struct sched_group *group) +static inline int sg_capa_factor(struct lb_env *env, struct sched_group *group) {
unsigned int capacity, smt, cpus;
unsigned int capa_factor, smt, cpus; unsigned int power, power_orig; power = group->sgp->power;@@ -5799,13 +5799,13 @@ static inline int sg_capacity(struct lb_env *env, struct sched_group *group)
/* smt := ceil(cpus / power), assumes: 1 < smt_power < 2 */ smt = DIV_ROUND_UP(SCHED_POWER_SCALE * cpus, power_orig);
capacity = cpus / smt; /* cores */
capa_factor = cpus / smt; /* cores */
capacity = min_t(unsigned, capacity, DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE));if (!capacity)capacity = fix_small_capacity(env->sd, group);
capa_factor = min_t(unsigned, capa_factor, DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE));if (!capa_factor)capa_factor = fix_small_capacity(env->sd, group);
return capacity;
return capa_factor;}
/** @@ -5855,9 +5855,9 @@ static inline void update_sg_lb_stats(struct lb_env *env, sgs->group_weight = group->group_weight;
sgs->group_imb = sg_imbalanced(group);
sgs->group_capacity = sg_capacity(env, group);
sgs->group_capa_factor = sg_capa_factor(env, group);
if (sgs->group_capacity > sgs->sum_nr_running)
if (sgs->group_capa_factor > sgs->sum_nr_running) sgs->group_has_free_capacity = 1;}
@@ -5882,7 +5882,7 @@ static bool update_sd_pick_busiest(struct lb_env *env, if (sgs->avg_load <= sds->busiest_stat.avg_load) return false;
if (sgs->sum_nr_running > sgs->group_capacity)
if (sgs->sum_nr_running > sgs->group_capa_factor) return true; if (sgs->group_imb)@@ -5973,17 +5973,17 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd
/* * In case the child domain prefers tasks go to siblings
* first, lower the sg capacity to one so that we'll try
* first, lower the sg capacity factor to one so that we'll try * and move all the excess tasks away. We lower the capacity * of a group only if the local group has the capacity to fit
* these excess tasks, i.e. nr_running < group_capacity. The
* these excess tasks, i.e. nr_running < group_capa_factor. The * extra check prevents the case where you always pull from the * heaviest group when it is already under-utilized (possible * with a large weight task outweighs the tasks on the system). */ if (prefer_sibling && sds->local && sds->local_stat.group_has_free_capacity)
sgs->group_capacity = min(sgs->group_capacity, 1U);
sgs->group_capa_factor = min(sgs->group_capa_factor, 1U); if (update_sd_pick_busiest(env, sds, sg, sgs)) { sds->busiest = sg;@@ -6157,7 +6157,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s * have to drop below capacity to reach cpu-load equilibrium. */ load_above_capacity =
(busiest->sum_nr_running - busiest->group_capacity);
(busiest->sum_nr_running - busiest->group_capa_factor); load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_POWER_SCALE); load_above_capacity /= busiest->group_power;@@ -6301,7 +6301,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, int i;
for_each_cpu_and(i, sched_group_cpus(group), env->cpus) {
unsigned long power, capacity, wl;
unsigned long power, capa_factor, wl; enum fbq_type rt; rq = cpu_rq(i);@@ -6330,9 +6330,9 @@ static struct rq *find_busiest_queue(struct lb_env *env, continue;
power = power_of(i);
capacity = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE);if (!capacity)capacity = fix_small_capacity(env->sd, group);
capa_factor = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE);if (!capa_factor)capa_factor = fix_small_capacity(env->sd, group); wl = weighted_cpuload(i);@@ -6340,7 +6340,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, * When comparing with imbalance, use weighted_cpuload() * which is not scaled with the cpu power. */
if (capacity && rq->nr_running == 1 && wl > env->imbalance)
if (capa_factor && rq->nr_running == 1 && wl > env->imbalance) continue; /*-- 1.8.4.108.g55ea5f6
On Thu, 15 May 2014, Vincent Guittot wrote:
On 14 May 2014 22:57, Nicolas Pitre nicolas.pitre@linaro.org wrote:
We have "power" (which should actually become "capacity") and "capacity" which is a scaled down "capacity factor" in terms of possible tasks. Let's use "capa_factor" to make room for proper usage of "capacity" later.
Signed-off-by: Nicolas Pitre nico@linaro.org
kernel/sched/fair.c | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0eda4c527e..2633c42692 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5487,7 +5487,7 @@ struct sg_lb_stats { unsigned long load_per_task; unsigned long group_power; unsigned int sum_nr_running; /* Nr tasks running in the group */
unsigned int group_capacity;
unsigned int group_capa_factor;As it is mainly compared to sum_nr_running, you might rename it to group_nr_capacity instead of group_capa_factor
But what actual meaning would "group_nr_capacity" convey? This could be interpreted as the total number of groups possible for example.
Nicolas
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
Since struct sched_group_power is really about compute capacity of sched groups, let's rename it to struct sched_group_capacity. Similarly sgp becomes sgc. Related variables and functions dealing with groups are also adjusted accordingly.
Signed-off-by: Nicolas Pitre nico@linaro.org --- include/linux/sched.h | 2 +- kernel/sched/core.c | 83 ++++++++++++++++---------------- kernel/sched/fair.c | 131 +++++++++++++++++++++++++------------------------- kernel/sched/sched.h | 16 +++--- 4 files changed, 116 insertions(+), 116 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h index 59002c5dd9..31ca7b60e8 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1013,7 +1013,7 @@ typedef const int (*sched_domain_flags_f)(void); struct sd_data { struct sched_domain **__percpu sd; struct sched_group **__percpu sg; - struct sched_group_power **__percpu sgp; + struct sched_group_capacity **__percpu sgc; };
struct sched_domain_topology_level { diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 4ea7b3f1a0..d8ba3ab1c0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5221,14 +5221,13 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level, }
/* - * Even though we initialize ->power to something semi-sane, - * we leave power_orig unset. This allows us to detect if + * Even though we initialize ->capacity to something semi-sane, + * we leave capacity_orig unset. This allows us to detect if * domain iteration is still funny without causing /0 traps. */ - if (!group->sgp->power_orig) { + if (!group->sgc->capacity_orig) { printk(KERN_CONT "\n"); - printk(KERN_ERR "ERROR: domain->cpu_power not " - "set\n"); + printk(KERN_ERR "ERROR: domain->cpu_capacity not set\n"); break; }
@@ -5250,9 +5249,9 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level, cpulist_scnprintf(str, sizeof(str), sched_group_cpus(group));
printk(KERN_CONT " %s", str); - if (group->sgp->power != SCHED_POWER_SCALE) { - printk(KERN_CONT " (cpu_power = %d)", - group->sgp->power); + if (group->sgc->capacity != SCHED_POWER_SCALE) { + printk(KERN_CONT " (cpu_capacity = %d)", + group->sgc->capacity); }
group = group->next; @@ -5466,7 +5465,7 @@ static struct root_domain *alloc_rootdomain(void) return rd; }
-static void free_sched_groups(struct sched_group *sg, int free_sgp) +static void free_sched_groups(struct sched_group *sg, int free_sgc) { struct sched_group *tmp, *first;
@@ -5477,8 +5476,8 @@ static void free_sched_groups(struct sched_group *sg, int free_sgp) do { tmp = sg->next;
- if (free_sgp && atomic_dec_and_test(&sg->sgp->ref)) - kfree(sg->sgp); + if (free_sgc && atomic_dec_and_test(&sg->sgc->ref)) + kfree(sg->sgc);
kfree(sg); sg = tmp; @@ -5496,7 +5495,7 @@ static void free_sched_domain(struct rcu_head *rcu) if (sd->flags & SD_OVERLAP) { free_sched_groups(sd->groups, 1); } else if (atomic_dec_and_test(&sd->groups->ref)) { - kfree(sd->groups->sgp); + kfree(sd->groups->sgc); kfree(sd->groups); } kfree(sd); @@ -5707,17 +5706,17 @@ build_overlap_sched_groups(struct sched_domain *sd, int cpu)
cpumask_or(covered, covered, sg_span);
- sg->sgp = *per_cpu_ptr(sdd->sgp, i); - if (atomic_inc_return(&sg->sgp->ref) == 1) + sg->sgc = *per_cpu_ptr(sdd->sgc, i); + if (atomic_inc_return(&sg->sgc->ref) == 1) build_group_mask(sd, sg);
/* - * Initialize sgp->power such that even if we mess up the + * Initialize sgc->capacity such that even if we mess up the * domains and no possible iteration will get us here, we won't * die on a /0 trap. */ - sg->sgp->power = SCHED_POWER_SCALE * cpumask_weight(sg_span); - sg->sgp->power_orig = sg->sgp->power; + sg->sgc->capacity = SCHED_POWER_SCALE * cpumask_weight(sg_span); + sg->sgc->capacity_orig = sg->sgc->capacity;
/* * Make sure the first group of this domain contains the @@ -5755,8 +5754,8 @@ static int get_group(int cpu, struct sd_data *sdd, struct sched_group **sg)
if (sg) { *sg = *per_cpu_ptr(sdd->sg, cpu); - (*sg)->sgp = *per_cpu_ptr(sdd->sgp, cpu); - atomic_set(&(*sg)->sgp->ref, 1); /* for claim_allocations */ + (*sg)->sgc = *per_cpu_ptr(sdd->sgc, cpu); + atomic_set(&(*sg)->sgc->ref, 1); /* for claim_allocations */ }
return cpu; @@ -5798,7 +5797,7 @@ build_sched_groups(struct sched_domain *sd, int cpu)
group = get_group(i, sdd, &sg); cpumask_clear(sched_group_cpus(sg)); - sg->sgp->power = 0; + sg->sgc->capacity = 0; cpumask_setall(sched_group_mask(sg));
for_each_cpu(j, span) { @@ -5821,16 +5820,16 @@ build_sched_groups(struct sched_domain *sd, int cpu) }
/* - * Initialize sched groups cpu_power. + * Initialize sched groups cpu_capacity. * - * cpu_power indicates the capacity of sched group, which is used while + * cpu_capacity indicates the capacity of sched group, which is used while * distributing the load between different sched groups in a sched domain. - * Typically cpu_power for all the groups in a sched domain will be same unless - * there are asymmetries in the topology. If there are asymmetries, group - * having more cpu_power will pickup more load compared to the group having - * less cpu_power. + * Typically cpu_capacity for all the groups in a sched domain will be same + * unless there are asymmetries in the topology. If there are asymmetries, + * group having more cpu_capacity will pickup more load compared to the + * group having less cpu_capacity. */ -static void init_sched_groups_power(int cpu, struct sched_domain *sd) +static void init_sched_groups_capacity(int cpu, struct sched_domain *sd) { struct sched_group *sg = sd->groups;
@@ -5844,8 +5843,8 @@ static void init_sched_groups_power(int cpu, struct sched_domain *sd) if (cpu != group_balance_cpu(sg)) return;
- update_group_power(sd, cpu); - atomic_set(&sg->sgp->nr_busy_cpus, sg->group_weight); + update_group_capacity(sd, cpu); + atomic_set(&sg->sgc->nr_busy_cpus, sg->group_weight); }
/* @@ -5936,8 +5935,8 @@ static void claim_allocations(int cpu, struct sched_domain *sd) if (atomic_read(&(*per_cpu_ptr(sdd->sg, cpu))->ref)) *per_cpu_ptr(sdd->sg, cpu) = NULL;
- if (atomic_read(&(*per_cpu_ptr(sdd->sgp, cpu))->ref)) - *per_cpu_ptr(sdd->sgp, cpu) = NULL; + if (atomic_read(&(*per_cpu_ptr(sdd->sgc, cpu))->ref)) + *per_cpu_ptr(sdd->sgc, cpu) = NULL; }
#ifdef CONFIG_NUMA @@ -6339,14 +6338,14 @@ static int __sdt_alloc(const struct cpumask *cpu_map) if (!sdd->sg) return -ENOMEM;
- sdd->sgp = alloc_percpu(struct sched_group_power *); - if (!sdd->sgp) + sdd->sgc = alloc_percpu(struct sched_group_capacity *); + if (!sdd->sgc) return -ENOMEM;
for_each_cpu(j, cpu_map) { struct sched_domain *sd; struct sched_group *sg; - struct sched_group_power *sgp; + struct sched_group_capacity *sgc;
sd = kzalloc_node(sizeof(struct sched_domain) + cpumask_size(), GFP_KERNEL, cpu_to_node(j)); @@ -6364,12 +6363,12 @@ static int __sdt_alloc(const struct cpumask *cpu_map)
*per_cpu_ptr(sdd->sg, j) = sg;
- sgp = kzalloc_node(sizeof(struct sched_group_power) + cpumask_size(), + sgc = kzalloc_node(sizeof(struct sched_group_capacity) + cpumask_size(), GFP_KERNEL, cpu_to_node(j)); - if (!sgp) + if (!sgc) return -ENOMEM;
- *per_cpu_ptr(sdd->sgp, j) = sgp; + *per_cpu_ptr(sdd->sgc, j) = sgc; } }
@@ -6396,15 +6395,15 @@ static void __sdt_free(const struct cpumask *cpu_map)
if (sdd->sg) kfree(*per_cpu_ptr(sdd->sg, j)); - if (sdd->sgp) - kfree(*per_cpu_ptr(sdd->sgp, j)); + if (sdd->sgc) + kfree(*per_cpu_ptr(sdd->sgc, j)); } free_percpu(sdd->sd); sdd->sd = NULL; free_percpu(sdd->sg); sdd->sg = NULL; - free_percpu(sdd->sgp); - sdd->sgp = NULL; + free_percpu(sdd->sgc); + sdd->sgc = NULL; } }
@@ -6481,7 +6480,7 @@ static int build_sched_domains(const struct cpumask *cpu_map,
for (sd = *per_cpu_ptr(d.sd, i); sd; sd = sd->parent) { claim_allocations(i, sd); - init_sched_groups_power(i, sd); + init_sched_groups_capacity(i, sd); } }
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 2633c42692..6bc1e54d30 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4341,8 +4341,8 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p, avg_load += load; }
- /* Adjust by relative CPU power of the group */ - avg_load = (avg_load * SCHED_POWER_SCALE) / group->sgp->power; + /* Adjust by relative CPU capacity of the group */ + avg_load = (avg_load * SCHED_POWER_SCALE) / group->sgc->capacity;
if (local_group) { this_load = avg_load; @@ -5485,7 +5485,7 @@ struct sg_lb_stats { unsigned long group_load; /* Total load over the CPUs of the group */ unsigned long sum_weighted_load; /* Weighted load of group's tasks */ unsigned long load_per_task; - unsigned long group_power; + unsigned long group_capacity; unsigned int sum_nr_running; /* Nr tasks running in the group */ unsigned int group_capa_factor; unsigned int idle_cpus; @@ -5506,7 +5506,7 @@ struct sd_lb_stats { struct sched_group *busiest; /* Busiest group in this sd */ struct sched_group *local; /* Local group in this sd */ unsigned long total_load; /* Total load of all groups in sd */ - unsigned long total_pwr; /* Total power of all groups in sd */ + unsigned long total_capa; /* Total capacity of all groups in sd */ unsigned long avg_load; /* Average load across all groups in sd */
struct sg_lb_stats busiest_stat;/* Statistics of the busiest group */ @@ -5525,7 +5525,7 @@ static inline void init_sd_lb_stats(struct sd_lb_stats *sds) .busiest = NULL, .local = NULL, .total_load = 0UL, - .total_pwr = 0UL, + .total_capa = 0UL, .busiest_stat = { .avg_load = 0UL, }, @@ -5634,7 +5634,7 @@ static void update_cpu_power(struct sched_domain *sd, int cpu) power >>= SCHED_POWER_SHIFT; }
- sdg->sgp->power_orig = power; + sdg->sgc->capacity_orig = power;
if (sched_feat(ARCH_POWER)) power *= arch_scale_freq_power(sd, cpu); @@ -5650,26 +5650,26 @@ static void update_cpu_power(struct sched_domain *sd, int cpu) power = 1;
cpu_rq(cpu)->cpu_power = power; - sdg->sgp->power = power; + sdg->sgc->capacity = power; }
-void update_group_power(struct sched_domain *sd, int cpu) +void update_group_capacity(struct sched_domain *sd, int cpu) { struct sched_domain *child = sd->child; struct sched_group *group, *sdg = sd->groups; - unsigned long power, power_orig; + unsigned long capacity, capacity_orig; unsigned long interval;
interval = msecs_to_jiffies(sd->balance_interval); interval = clamp(interval, 1UL, max_load_balance_interval); - sdg->sgp->next_update = jiffies + interval; + sdg->sgc->next_update = jiffies + interval;
if (!child) { update_cpu_power(sd, cpu); return; }
- power_orig = power = 0; + capacity_orig = capacity = 0;
if (child->flags & SD_OVERLAP) { /* @@ -5678,31 +5678,31 @@ void update_group_power(struct sched_domain *sd, int cpu) */
for_each_cpu(cpu, sched_group_cpus(sdg)) { - struct sched_group_power *sgp; + struct sched_group_capacity *sgc; struct rq *rq = cpu_rq(cpu);
/* - * build_sched_domains() -> init_sched_groups_power() + * build_sched_domains() -> init_sched_groups_capacity() * gets here before we've attached the domains to the * runqueues. * * Use power_of(), which is set irrespective of domains * in update_cpu_power(). * - * This avoids power/power_orig from being 0 and + * This avoids capacity/capacity_orig from being 0 and * causing divide-by-zero issues on boot. * - * Runtime updates will correct power_orig. + * Runtime updates will correct capacity_orig. */ if (unlikely(!rq->sd)) { - power_orig += power_of(cpu); - power += power_of(cpu); + capacity_orig += power_of(cpu); + capacity += power_of(cpu); continue; }
- sgp = rq->sd->groups->sgp; - power_orig += sgp->power_orig; - power += sgp->power; + sgc = rq->sd->groups->sgc; + capacity_orig += sgc->capacity_orig; + capacity += sgc->capacity; } } else { /* @@ -5712,14 +5712,14 @@ void update_group_power(struct sched_domain *sd, int cpu)
group = child->groups; do { - power_orig += group->sgp->power_orig; - power += group->sgp->power; + capacity_orig += group->sgc->capacity_orig; + capacity += group->sgc->capacity; group = group->next; } while (group != child->groups); }
- sdg->sgp->power_orig = power_orig; - sdg->sgp->power = power; + sdg->sgc->capacity_orig = capacity_orig; + sdg->sgc->capacity = capacity; }
/* @@ -5739,9 +5739,9 @@ fix_small_capacity(struct sched_domain *sd, struct sched_group *group) return 0;
/* - * If ~90% of the cpu_power is still there, we're good. + * If ~90% of the cpu_capacity is still there, we're good. */ - if (group->sgp->power * 32 > group->sgp->power_orig * 29) + if (group->sgc->capacity * 32 > group->sgc->capacity_orig * 29) return 1;
return 0; @@ -5778,7 +5778,7 @@ fix_small_capacity(struct sched_domain *sd, struct sched_group *group)
static inline int sg_imbalanced(struct sched_group *group) { - return group->sgp->imbalance; + return group->sgc->imbalance; }
/* @@ -5786,22 +5786,23 @@ static inline int sg_imbalanced(struct sched_group *group) * * Avoid the issue where N*frac(smt_power) >= 1 creates 'phantom' cores by * first dividing out the smt factor and computing the actual number of cores - * and limit power unit capacity with that. + * and limit unit capacity with that. */ static inline int sg_capa_factor(struct lb_env *env, struct sched_group *group) { unsigned int capa_factor, smt, cpus; - unsigned int power, power_orig; + unsigned int capacity, capacity_orig;
- power = group->sgp->power; - power_orig = group->sgp->power_orig; + capacity = group->sgc->capacity; + capacity_orig = group->sgc->capacity_orig; cpus = group->group_weight;
- /* smt := ceil(cpus / power), assumes: 1 < smt_power < 2 */ - smt = DIV_ROUND_UP(SCHED_POWER_SCALE * cpus, power_orig); + /* smt := ceil(cpus / capacity), assumes: 1 < smt_capacity < 2 */ + smt = DIV_ROUND_UP(SCHED_POWER_SCALE * cpus, capacity_orig); capa_factor = cpus / smt; /* cores */
- capa_factor = min_t(unsigned, capa_factor, DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE)); + capa_factor = min_t(unsigned, + capa_factor, DIV_ROUND_CLOSEST(capacity, SCHED_POWER_SCALE)); if (!capa_factor) capa_factor = fix_small_capacity(env->sd, group);
@@ -5845,9 +5846,9 @@ static inline void update_sg_lb_stats(struct lb_env *env, sgs->idle_cpus++; }
- /* Adjust by relative CPU power of the group */ - sgs->group_power = group->sgp->power; - sgs->avg_load = (sgs->group_load*SCHED_POWER_SCALE) / sgs->group_power; + /* Adjust by relative CPU capacity of the group */ + sgs->group_capacity = group->sgc->capacity; + sgs->avg_load = (sgs->group_load*SCHED_POWER_SCALE) / sgs->group_capacity;
if (sgs->sum_nr_running) sgs->load_per_task = sgs->sum_weighted_load / sgs->sum_nr_running; @@ -5962,8 +5963,8 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd sgs = &sds->local_stat;
if (env->idle != CPU_NEWLY_IDLE || - time_after_eq(jiffies, sg->sgp->next_update)) - update_group_power(env->sd, env->dst_cpu); + time_after_eq(jiffies, sg->sgc->next_update)) + update_group_capacity(env->sd, env->dst_cpu); }
update_sg_lb_stats(env, sg, load_idx, local_group, sgs); @@ -5993,7 +5994,7 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd next_group: /* Now, start updating sd_lb_stats */ sds->total_load += sgs->group_load; - sds->total_pwr += sgs->group_power; + sds->total_capa += sgs->group_capacity;
sg = sg->next; } while (sg != env->sd->groups); @@ -6040,7 +6041,7 @@ static int check_asym_packing(struct lb_env *env, struct sd_lb_stats *sds) return 0;
env->imbalance = DIV_ROUND_CLOSEST( - sds->busiest_stat.avg_load * sds->busiest_stat.group_power, + sds->busiest_stat.avg_load * sds->busiest_stat.group_capacity, SCHED_POWER_SCALE);
return 1; @@ -6056,7 +6057,7 @@ static int check_asym_packing(struct lb_env *env, struct sd_lb_stats *sds) static inline void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds) { - unsigned long tmp, pwr_now = 0, pwr_move = 0; + unsigned long tmp, capa_now = 0, capa_move = 0; unsigned int imbn = 2; unsigned long scaled_busy_load_per_task; struct sg_lb_stats *local, *busiest; @@ -6071,7 +6072,7 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds)
scaled_busy_load_per_task = (busiest->load_per_task * SCHED_POWER_SCALE) / - busiest->group_power; + busiest->group_capacity;
if (busiest->avg_load + scaled_busy_load_per_task >= local->avg_load + (scaled_busy_load_per_task * imbn)) { @@ -6085,34 +6086,34 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds) * moving them. */
- pwr_now += busiest->group_power * + capa_now += busiest->group_capacity * min(busiest->load_per_task, busiest->avg_load); - pwr_now += local->group_power * + capa_now += local->group_capacity * min(local->load_per_task, local->avg_load); - pwr_now /= SCHED_POWER_SCALE; + capa_now /= SCHED_POWER_SCALE;
/* Amount of load we'd subtract */ if (busiest->avg_load > scaled_busy_load_per_task) { - pwr_move += busiest->group_power * + capa_move += busiest->group_capacity * min(busiest->load_per_task, busiest->avg_load - scaled_busy_load_per_task); }
/* Amount of load we'd add */ - if (busiest->avg_load * busiest->group_power < + if (busiest->avg_load * busiest->group_capacity < busiest->load_per_task * SCHED_POWER_SCALE) { - tmp = (busiest->avg_load * busiest->group_power) / - local->group_power; + tmp = (busiest->avg_load * busiest->group_capacity) / + local->group_capacity; } else { tmp = (busiest->load_per_task * SCHED_POWER_SCALE) / - local->group_power; + local->group_capacity; } - pwr_move += local->group_power * + capa_move += local->group_capacity * min(local->load_per_task, local->avg_load + tmp); - pwr_move /= SCHED_POWER_SCALE; + capa_move /= SCHED_POWER_SCALE;
/* Move if we gain throughput */ - if (pwr_move > pwr_now) + if (capa_move > capa_now) env->imbalance = busiest->load_per_task; }
@@ -6160,7 +6161,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s (busiest->sum_nr_running - busiest->group_capa_factor);
load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_POWER_SCALE); - load_above_capacity /= busiest->group_power; + load_above_capacity /= busiest->group_capacity; }
/* @@ -6175,8 +6176,8 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s
/* How much load to actually move to equalise the imbalance */ env->imbalance = min( - max_pull * busiest->group_power, - (sds->avg_load - local->avg_load) * local->group_power + max_pull * busiest->group_capacity, + (sds->avg_load - local->avg_load) * local->group_capacity ) / SCHED_POWER_SCALE;
/* @@ -6231,7 +6232,7 @@ static struct sched_group *find_busiest_group(struct lb_env *env) if (!sds.busiest || busiest->sum_nr_running == 0) goto out_balanced;
- sds.avg_load = (SCHED_POWER_SCALE * sds.total_load) / sds.total_pwr; + sds.avg_load = (SCHED_POWER_SCALE * sds.total_load) / sds.total_capa;
/* * If the busiest group is imbalanced the below checks don't @@ -6564,7 +6565,7 @@ more_balance: * We failed to reach balance because of affinity. */ if (sd_parent) { - int *group_imbalance = &sd_parent->groups->sgp->imbalance; + int *group_imbalance = &sd_parent->groups->sgc->imbalance;
if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) { *group_imbalance = 1; @@ -6922,7 +6923,7 @@ static inline void set_cpu_sd_state_busy(void) goto unlock; sd->nohz_idle = 0;
- atomic_inc(&sd->groups->sgp->nr_busy_cpus); + atomic_inc(&sd->groups->sgc->nr_busy_cpus); unlock: rcu_read_unlock(); } @@ -6939,7 +6940,7 @@ void set_cpu_sd_state_idle(void) goto unlock; sd->nohz_idle = 1;
- atomic_dec(&sd->groups->sgp->nr_busy_cpus); + atomic_dec(&sd->groups->sgc->nr_busy_cpus); unlock: rcu_read_unlock(); } @@ -7144,7 +7145,7 @@ end: * of an idle cpu is the system. * - This rq has more than one task. * - At any scheduler domain level, this cpu's scheduler group has multiple - * busy cpu's exceeding the group's power. + * busy cpu's exceeding the group's capacity. * - For SD_ASYM_PACKING, if the lower numbered cpu's in the scheduler * domain span are idle. */ @@ -7152,7 +7153,7 @@ static inline int nohz_kick_needed(struct rq *rq) { unsigned long now = jiffies; struct sched_domain *sd; - struct sched_group_power *sgp; + struct sched_group_capacity *sgc; int nr_busy, cpu = rq->cpu;
if (unlikely(rq->idle_balance)) @@ -7182,8 +7183,8 @@ static inline int nohz_kick_needed(struct rq *rq) sd = rcu_dereference(per_cpu(sd_busy, cpu));
if (sd) { - sgp = sd->groups->sgp; - nr_busy = atomic_read(&sgp->nr_busy_cpus); + sgc = sd->groups->sgc; + nr_busy = atomic_read(&sgc->nr_busy_cpus);
if (nr_busy > 1) goto need_kick_unlock; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b2cbe81308..b384ccd07b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -728,15 +728,15 @@ DECLARE_PER_CPU(struct sched_domain *, sd_numa); DECLARE_PER_CPU(struct sched_domain *, sd_busy); DECLARE_PER_CPU(struct sched_domain *, sd_asym);
-struct sched_group_power { +struct sched_group_capacity { atomic_t ref; /* - * CPU power of this group, SCHED_LOAD_SCALE being max power for a - * single CPU. + * CPU capacity of this group, SCHED_LOAD_SCALE being max capacity + * for a single CPU. */ - unsigned int power, power_orig; + unsigned int capacity, capacity_orig; unsigned long next_update; - int imbalance; /* XXX unrelated to power but shared group state */ + int imbalance; /* XXX unrelated to capacity but shared group state */ /* * Number of busy cpus in this group. */ @@ -750,7 +750,7 @@ struct sched_group { atomic_t ref;
unsigned int group_weight; - struct sched_group_power *sgp; + struct sched_group_capacity *sgc;
/* * The CPUs this group covers. @@ -773,7 +773,7 @@ static inline struct cpumask *sched_group_cpus(struct sched_group *sg) */ static inline struct cpumask *sched_group_mask(struct sched_group *sg) { - return to_cpumask(sg->sgp->cpumask); + return to_cpumask(sg->sgc->cpumask); }
/** @@ -1167,7 +1167,7 @@ extern const struct sched_class idle_sched_class;
#ifdef CONFIG_SMP
-extern void update_group_power(struct sched_domain *sd, int cpu); +extern void update_group_capacity(struct sched_domain *sd, int cpu);
extern void trigger_load_balance(struct rq *rq);
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This is the remaining "power" -> "capacity" rename. Those symbols visible to the rest of the kernel are not included yet.
Signed-off-by: Nicolas Pitre nico@linaro.org --- kernel/sched/core.c | 6 +-- kernel/sched/fair.c | 102 +++++++++++++++++++++++++-------------------------- kernel/sched/sched.h | 2 +- 3 files changed, 55 insertions(+), 55 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d8ba3ab1c0..00d04e9e7b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5764,7 +5764,7 @@ static int get_group(int cpu, struct sd_data *sdd, struct sched_group **sg) /* * build_sched_groups will build a circular linked list of the groups * covered by the given span, and will set each group's ->cpumask correctly, - * and ->cpu_power to 0. + * and ->cpu_capacity to 0. * * Assumes the sched_domain tree is fully constructed */ @@ -6473,7 +6473,7 @@ static int build_sched_domains(const struct cpumask *cpu_map, } }
- /* Calculate CPU power for physical packages and nodes */ + /* Calculate CPU capacity for physical packages and nodes */ for (i = nr_cpumask_bits-1; i >= 0; i--) { if (!cpumask_test_cpu(i, cpu_map)) continue; @@ -6923,7 +6923,7 @@ void __init sched_init(void) #ifdef CONFIG_SMP rq->sd = NULL; rq->rd = NULL; - rq->cpu_power = SCHED_POWER_SCALE; + rq->cpu_capacity = SCHED_POWER_SCALE; rq->post_schedule = 0; rq->active_balance = 0; rq->next_balance = jiffies; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6bc1e54d30..06619b52dd 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1017,7 +1017,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct page * page, static unsigned long weighted_cpuload(const int cpu); static unsigned long source_load(int cpu, int type); static unsigned long target_load(int cpu, int type); -static unsigned long power_of(int cpu); +static unsigned long capacity_of(int cpu); static long effective_load(struct task_group *tg, int cpu, long wl, long wg);
/* Cached statistics for all CPUs within a node */ @@ -1046,7 +1046,7 @@ static void update_numa_stats(struct numa_stats *ns, int nid)
ns->nr_running += rq->nr_running; ns->load += weighted_cpuload(cpu); - ns->compute_capacity += power_of(cpu); + ns->compute_capacity += capacity_of(cpu);
cpus++; } @@ -1185,7 +1185,7 @@ balance: dst_load = env->dst_stats.load; src_load = env->src_stats.load;
- /* XXX missing power terms */ + /* XXX missing capacity terms */ load = task_h_load(env->p); dst_load += load; src_load -= load; @@ -4015,9 +4015,9 @@ static unsigned long target_load(int cpu, int type) return max(rq->cpu_load[type-1], total); }
-static unsigned long power_of(int cpu) +static unsigned long capacity_of(int cpu) { - return cpu_rq(cpu)->cpu_power; + return cpu_rq(cpu)->cpu_capacity; }
static unsigned long cpu_avg_load_per_task(int cpu) @@ -4260,12 +4260,12 @@ static int wake_affine(struct sched_domain *sd, struct task_struct *p, int sync) s64 this_eff_load, prev_eff_load;
this_eff_load = 100; - this_eff_load *= power_of(prev_cpu); + this_eff_load *= capacity_of(prev_cpu); this_eff_load *= this_load + effective_load(tg, this_cpu, weight, weight);
prev_eff_load = 100 + (sd->imbalance_pct - 100) / 2; - prev_eff_load *= power_of(this_cpu); + prev_eff_load *= capacity_of(this_cpu); prev_eff_load *= load + effective_load(tg, prev_cpu, 0, weight);
balanced = this_eff_load <= prev_eff_load; @@ -4919,14 +4919,14 @@ static bool yield_to_task_fair(struct rq *rq, struct task_struct *p, bool preemp * * W'_i,n = (2^n - 1) / 2^n * W_i,n + 1 / 2^n * W_i,0 (3) * - * P_i is the cpu power (or compute capacity) of cpu i, typically it is the + * C_i is the compute capacity of cpu i, typically it is the * fraction of 'recent' time available for SCHED_OTHER task execution. But it * can also include other factors [XXX]. * * To achieve this balance we define a measure of imbalance which follows * directly from (1): * - * imb_i,j = max{ avg(W/P), W_i/P_i } - min{ avg(W/P), W_j/P_j } (4) + * imb_i,j = max{ avg(W/C), W_i/C_i } - min{ avg(W/C), W_j/C_j } (4) * * We them move tasks around to minimize the imbalance. In the continuous * function space it is obvious this converges, in the discrete case we get @@ -5560,17 +5560,17 @@ static inline int get_sd_load_idx(struct sched_domain *sd, return load_idx; }
-static unsigned long default_scale_freq_power(struct sched_domain *sd, int cpu) +static unsigned long default_scale_capacity(struct sched_domain *sd, int cpu) { return SCHED_POWER_SCALE; }
unsigned long __weak arch_scale_freq_power(struct sched_domain *sd, int cpu) { - return default_scale_freq_power(sd, cpu); + return default_scale_capacity(sd, cpu); }
-static unsigned long default_scale_smt_power(struct sched_domain *sd, int cpu) +static unsigned long default_scale_smt_capacity(struct sched_domain *sd, int cpu) { unsigned long weight = sd->span_weight; unsigned long smt_gain = sd->smt_gain; @@ -5582,10 +5582,10 @@ static unsigned long default_scale_smt_power(struct sched_domain *sd, int cpu)
unsigned long __weak arch_scale_smt_power(struct sched_domain *sd, int cpu) { - return default_scale_smt_power(sd, cpu); + return default_scale_smt_capacity(sd, cpu); }
-static unsigned long scale_rt_power(int cpu) +static unsigned long scale_rt_capacity(int cpu) { struct rq *rq = cpu_rq(cpu); u64 total, available, age_stamp, avg; @@ -5605,7 +5605,7 @@ static unsigned long scale_rt_power(int cpu) total = sched_avg_period() + delta;
if (unlikely(total < avg)) { - /* Ensures that power won't end up being negative */ + /* Ensures that capacity won't end up being negative */ available = 0; } else { available = total - avg; @@ -5619,38 +5619,38 @@ static unsigned long scale_rt_power(int cpu) return div_u64(available, total); }
-static void update_cpu_power(struct sched_domain *sd, int cpu) +static void update_cpu_capacity(struct sched_domain *sd, int cpu) { unsigned long weight = sd->span_weight; - unsigned long power = SCHED_POWER_SCALE; + unsigned long capacity = SCHED_POWER_SCALE; struct sched_group *sdg = sd->groups;
if ((sd->flags & SD_SHARE_CPUPOWER) && weight > 1) { if (sched_feat(ARCH_POWER)) - power *= arch_scale_smt_power(sd, cpu); + capacity *= arch_scale_smt_power(sd, cpu); else - power *= default_scale_smt_power(sd, cpu); + capacity *= default_scale_smt_capacity(sd, cpu);
- power >>= SCHED_POWER_SHIFT; + capacity >>= SCHED_POWER_SHIFT; }
- sdg->sgc->capacity_orig = power; + sdg->sgc->capacity_orig = capacity;
if (sched_feat(ARCH_POWER)) - power *= arch_scale_freq_power(sd, cpu); + capacity *= arch_scale_freq_power(sd, cpu); else - power *= default_scale_freq_power(sd, cpu); + capacity *= default_scale_capacity(sd, cpu);
- power >>= SCHED_POWER_SHIFT; + capacity >>= SCHED_POWER_SHIFT;
- power *= scale_rt_power(cpu); - power >>= SCHED_POWER_SHIFT; + capacity *= scale_rt_capacity(cpu); + capacity >>= SCHED_POWER_SHIFT;
- if (!power) - power = 1; + if (!capacity) + capacity = 1;
- cpu_rq(cpu)->cpu_power = power; - sdg->sgc->capacity = power; + cpu_rq(cpu)->cpu_capacity = capacity; + sdg->sgc->capacity = capacity; }
void update_group_capacity(struct sched_domain *sd, int cpu) @@ -5665,7 +5665,7 @@ void update_group_capacity(struct sched_domain *sd, int cpu) sdg->sgc->next_update = jiffies + interval;
if (!child) { - update_cpu_power(sd, cpu); + update_cpu_capacity(sd, cpu); return; }
@@ -5686,8 +5686,8 @@ void update_group_capacity(struct sched_domain *sd, int cpu) * gets here before we've attached the domains to the * runqueues. * - * Use power_of(), which is set irrespective of domains - * in update_cpu_power(). + * Use capacity_of(), which is set irrespective of domains + * in update_cpu_capacity(). * * This avoids capacity/capacity_orig from being 0 and * causing divide-by-zero issues on boot. @@ -5695,8 +5695,8 @@ void update_group_capacity(struct sched_domain *sd, int cpu) * Runtime updates will correct capacity_orig. */ if (unlikely(!rq->sd)) { - capacity_orig += power_of(cpu); - capacity += power_of(cpu); + capacity_orig += capacity_of(cpu); + capacity += capacity_of(cpu); continue; }
@@ -5784,7 +5784,7 @@ static inline int sg_imbalanced(struct sched_group *group) /* * Compute the group capacity factor. * - * Avoid the issue where N*frac(smt_power) >= 1 creates 'phantom' cores by + * Avoid the issue where N*frac(smt_capacity) >= 1 creates 'phantom' cores by * first dividing out the smt factor and computing the actual number of cores * and limit unit capacity with that. */ @@ -6082,7 +6082,7 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds)
/* * OK, we don't have enough imbalance to justify moving tasks, - * however we may be able to increase total CPU power used by + * however we may be able to increase total CPU capacity used by * moving them. */
@@ -6143,7 +6143,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s /* * In the presence of smp nice balancing, certain scenarios can have * max load less than avg load(as we skip the groups at or below - * its cpu_power, while calculating max_load..) + * its cpu_capacity, while calculating max_load..) */ if (busiest->avg_load <= sds->avg_load || local->avg_load >= sds->avg_load) { @@ -6298,11 +6298,11 @@ static struct rq *find_busiest_queue(struct lb_env *env, struct sched_group *group) { struct rq *busiest = NULL, *rq; - unsigned long busiest_load = 0, busiest_power = 1; + unsigned long busiest_load = 0, busiest_capacity = 1; int i;
for_each_cpu_and(i, sched_group_cpus(group), env->cpus) { - unsigned long power, capa_factor, wl; + unsigned long capacity, capa_factor, wl; enum fbq_type rt;
rq = cpu_rq(i); @@ -6330,8 +6330,8 @@ static struct rq *find_busiest_queue(struct lb_env *env, if (rt > env->fbq_type) continue;
- power = power_of(i); - capa_factor = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE); + capacity = capacity_of(i); + capa_factor = DIV_ROUND_CLOSEST(capacity, SCHED_POWER_SCALE); if (!capa_factor) capa_factor = fix_small_capacity(env->sd, group);
@@ -6339,25 +6339,25 @@ static struct rq *find_busiest_queue(struct lb_env *env,
/* * When comparing with imbalance, use weighted_cpuload() - * which is not scaled with the cpu power. + * which is not scaled with the cpu capacity. */ if (capa_factor && rq->nr_running == 1 && wl > env->imbalance) continue;
/* * For the load comparisons with the other cpu's, consider - * the weighted_cpuload() scaled with the cpu power, so that - * the load can be moved away from the cpu that is potentially - * running at a lower capacity. + * the weighted_cpuload() scaled with the cpu capacity, so + * that the load can be moved away from the cpu that is + * potentially running at a lower capacity. * - * Thus we're looking for max(wl_i / power_i), crosswise + * Thus we're looking for max(wl_i / capacity_i), crosswise * multiplication to rid ourselves of the division works out - * to: wl_i * power_j > wl_j * power_i; where j is our - * previous maximum. + * to: wl_i * capacity_j > wl_j * capacity_i; where j is + * our previous maximum. */ - if (wl * busiest_power > busiest_load * power) { + if (wl * busiest_capacity > busiest_load * capacity) { busiest_load = wl; - busiest_power = power; + busiest_capacity = capacity; busiest = rq; } } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b384ccd07b..fe89387931 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -567,7 +567,7 @@ struct rq { struct root_domain *rd; struct sched_domain *sd;
- unsigned long cpu_power; + unsigned long cpu_capacity;
unsigned char idle_balance; /* For active balancing */
Hi Nicolas, You might want to change the subject.
s/sched: remove remaining power to the CPU/ sched: remove remaining usage of cpu *power* .
The subject has to explicitly specify in some way that it is a change made to the terminology.
Regards Preeti U Murthy
On Thu, May 15, 2014 at 2:27 AM, Nicolas Pitre nicolas.pitre@linaro.org wrote:
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This is the remaining "power" -> "capacity" rename. Those symbols visible to the rest of the kernel are not included yet.
Signed-off-by: Nicolas Pitre nico@linaro.org
kernel/sched/core.c | 6 +-- kernel/sched/fair.c | 102 +++++++++++++++++++++++++-------------------------- kernel/sched/sched.h | 2 +- 3 files changed, 55 insertions(+), 55 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d8ba3ab1c0..00d04e9e7b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5764,7 +5764,7 @@ static int get_group(int cpu, struct sd_data *sdd, struct sched_group **sg) /*
- build_sched_groups will build a circular linked list of the groups
- covered by the given span, and will set each group's ->cpumask correctly,
- and ->cpu_power to 0.
*/
- and ->cpu_capacity to 0.
- Assumes the sched_domain tree is fully constructed
@@ -6473,7 +6473,7 @@ static int build_sched_domains(const struct cpumask *cpu_map, } }
/* Calculate CPU power for physical packages and nodes */
/* Calculate CPU capacity for physical packages and nodes */ for (i = nr_cpumask_bits-1; i >= 0; i--) { if (!cpumask_test_cpu(i, cpu_map)) continue;@@ -6923,7 +6923,7 @@ void __init sched_init(void) #ifdef CONFIG_SMP rq->sd = NULL; rq->rd = NULL;
rq->cpu_power = SCHED_POWER_SCALE;
rq->cpu_capacity = SCHED_POWER_SCALE; rq->post_schedule = 0; rq->active_balance = 0; rq->next_balance = jiffies;diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6bc1e54d30..06619b52dd 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1017,7 +1017,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct page * page, static unsigned long weighted_cpuload(const int cpu); static unsigned long source_load(int cpu, int type); static unsigned long target_load(int cpu, int type); -static unsigned long power_of(int cpu); +static unsigned long capacity_of(int cpu); static long effective_load(struct task_group *tg, int cpu, long wl, long wg);
/* Cached statistics for all CPUs within a node */ @@ -1046,7 +1046,7 @@ static void update_numa_stats(struct numa_stats *ns, int nid)
ns->nr_running += rq->nr_running; ns->load += weighted_cpuload(cpu);
ns->compute_capacity += power_of(cpu);
ns->compute_capacity += capacity_of(cpu); cpus++; }@@ -1185,7 +1185,7 @@ balance: dst_load = env->dst_stats.load; src_load = env->src_stats.load;
/* XXX missing power terms */
/* XXX missing capacity terms */ load = task_h_load(env->p); dst_load += load; src_load -= load;@@ -4015,9 +4015,9 @@ static unsigned long target_load(int cpu, int type) return max(rq->cpu_load[type-1], total); }
-static unsigned long power_of(int cpu) +static unsigned long capacity_of(int cpu) {
return cpu_rq(cpu)->cpu_power;
return cpu_rq(cpu)->cpu_capacity;}
static unsigned long cpu_avg_load_per_task(int cpu) @@ -4260,12 +4260,12 @@ static int wake_affine(struct sched_domain *sd, struct task_struct *p, int sync) s64 this_eff_load, prev_eff_load;
this_eff_load = 100;
this_eff_load *= power_of(prev_cpu);
this_eff_load *= capacity_of(prev_cpu); this_eff_load *= this_load + effective_load(tg, this_cpu, weight, weight); prev_eff_load = 100 + (sd->imbalance_pct - 100) / 2;
prev_eff_load *= power_of(this_cpu);
prev_eff_load *= capacity_of(this_cpu); prev_eff_load *= load + effective_load(tg, prev_cpu, 0, weight); balanced = this_eff_load <= prev_eff_load;@@ -4919,14 +4919,14 @@ static bool yield_to_task_fair(struct rq *rq, struct task_struct *p, bool preemp
- W'_i,n = (2^n - 1) / 2^n * W_i,n + 1 / 2^n * W_i,0 (3)
- P_i is the cpu power (or compute capacity) of cpu i, typically it is the
- C_i is the compute capacity of cpu i, typically it is the
- fraction of 'recent' time available for SCHED_OTHER task execution. But it
- can also include other factors [XXX].
- To achieve this balance we define a measure of imbalance which follows
- directly from (1):
- imb_i,j = max{ avg(W/P), W_i/P_i } - min{ avg(W/P), W_j/P_j } (4)
- imb_i,j = max{ avg(W/C), W_i/C_i } - min{ avg(W/C), W_j/C_j } (4)
- We them move tasks around to minimize the imbalance. In the continuous
- function space it is obvious this converges, in the discrete case we get
@@ -5560,17 +5560,17 @@ static inline int get_sd_load_idx(struct sched_domain *sd, return load_idx; }
-static unsigned long default_scale_freq_power(struct sched_domain *sd, int cpu) +static unsigned long default_scale_capacity(struct sched_domain *sd, int cpu) { return SCHED_POWER_SCALE; }
unsigned long __weak arch_scale_freq_power(struct sched_domain *sd, int cpu) {
return default_scale_freq_power(sd, cpu);
return default_scale_capacity(sd, cpu);}
-static unsigned long default_scale_smt_power(struct sched_domain *sd, int cpu) +static unsigned long default_scale_smt_capacity(struct sched_domain *sd, int cpu) { unsigned long weight = sd->span_weight; unsigned long smt_gain = sd->smt_gain; @@ -5582,10 +5582,10 @@ static unsigned long default_scale_smt_power(struct sched_domain *sd, int cpu)
unsigned long __weak arch_scale_smt_power(struct sched_domain *sd, int cpu) {
return default_scale_smt_power(sd, cpu);
return default_scale_smt_capacity(sd, cpu);}
-static unsigned long scale_rt_power(int cpu) +static unsigned long scale_rt_capacity(int cpu) { struct rq *rq = cpu_rq(cpu); u64 total, available, age_stamp, avg; @@ -5605,7 +5605,7 @@ static unsigned long scale_rt_power(int cpu) total = sched_avg_period() + delta;
if (unlikely(total < avg)) {
/* Ensures that power won't end up being negative */
/* Ensures that capacity won't end up being negative */ available = 0; } else { available = total - avg;@@ -5619,38 +5619,38 @@ static unsigned long scale_rt_power(int cpu) return div_u64(available, total); }
-static void update_cpu_power(struct sched_domain *sd, int cpu) +static void update_cpu_capacity(struct sched_domain *sd, int cpu) { unsigned long weight = sd->span_weight;
unsigned long power = SCHED_POWER_SCALE;
unsigned long capacity = SCHED_POWER_SCALE; struct sched_group *sdg = sd->groups; if ((sd->flags & SD_SHARE_CPUPOWER) && weight > 1) { if (sched_feat(ARCH_POWER))
power *= arch_scale_smt_power(sd, cpu);
capacity *= arch_scale_smt_power(sd, cpu); else
power *= default_scale_smt_power(sd, cpu);
capacity *= default_scale_smt_capacity(sd, cpu);
power >>= SCHED_POWER_SHIFT;
capacity >>= SCHED_POWER_SHIFT; }
sdg->sgc->capacity_orig = power;
sdg->sgc->capacity_orig = capacity; if (sched_feat(ARCH_POWER))
power *= arch_scale_freq_power(sd, cpu);
capacity *= arch_scale_freq_power(sd, cpu); else
power *= default_scale_freq_power(sd, cpu);
capacity *= default_scale_capacity(sd, cpu);
power >>= SCHED_POWER_SHIFT;
capacity >>= SCHED_POWER_SHIFT;
power *= scale_rt_power(cpu);power >>= SCHED_POWER_SHIFT;
capacity *= scale_rt_capacity(cpu);capacity >>= SCHED_POWER_SHIFT;
if (!power)power = 1;
if (!capacity)capacity = 1;
cpu_rq(cpu)->cpu_power = power;sdg->sgc->capacity = power;
cpu_rq(cpu)->cpu_capacity = capacity;sdg->sgc->capacity = capacity;}
void update_group_capacity(struct sched_domain *sd, int cpu) @@ -5665,7 +5665,7 @@ void update_group_capacity(struct sched_domain *sd, int cpu) sdg->sgc->next_update = jiffies + interval;
if (!child) {
update_cpu_power(sd, cpu);
update_cpu_capacity(sd, cpu); return; }@@ -5686,8 +5686,8 @@ void update_group_capacity(struct sched_domain *sd, int cpu) * gets here before we've attached the domains to the * runqueues. *
* Use power_of(), which is set irrespective of domains* in update_cpu_power().
* Use capacity_of(), which is set irrespective of domains* in update_cpu_capacity(). * * This avoids capacity/capacity_orig from being 0 and * causing divide-by-zero issues on boot.@@ -5695,8 +5695,8 @@ void update_group_capacity(struct sched_domain *sd, int cpu) * Runtime updates will correct capacity_orig. */ if (unlikely(!rq->sd)) {
capacity_orig += power_of(cpu);capacity += power_of(cpu);
capacity_orig += capacity_of(cpu);capacity += capacity_of(cpu); continue; }@@ -5784,7 +5784,7 @@ static inline int sg_imbalanced(struct sched_group *group) /*
- Compute the group capacity factor.
- Avoid the issue where N*frac(smt_power) >= 1 creates 'phantom' cores by
*/
- Avoid the issue where N*frac(smt_capacity) >= 1 creates 'phantom' cores by
- first dividing out the smt factor and computing the actual number of cores
- and limit unit capacity with that.
@@ -6082,7 +6082,7 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds)
/* * OK, we don't have enough imbalance to justify moving tasks,
* however we may be able to increase total CPU power used by
* however we may be able to increase total CPU capacity used by * moving them. */@@ -6143,7 +6143,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s /* * In the presence of smp nice balancing, certain scenarios can have * max load less than avg load(as we skip the groups at or below
* its cpu_power, while calculating max_load..)
* its cpu_capacity, while calculating max_load..) */ if (busiest->avg_load <= sds->avg_load || local->avg_load >= sds->avg_load) {@@ -6298,11 +6298,11 @@ static struct rq *find_busiest_queue(struct lb_env *env, struct sched_group *group) { struct rq *busiest = NULL, *rq;
unsigned long busiest_load = 0, busiest_power = 1;
unsigned long busiest_load = 0, busiest_capacity = 1; int i; for_each_cpu_and(i, sched_group_cpus(group), env->cpus) {
unsigned long power, capa_factor, wl;
unsigned long capacity, capa_factor, wl; enum fbq_type rt; rq = cpu_rq(i);@@ -6330,8 +6330,8 @@ static struct rq *find_busiest_queue(struct lb_env *env, if (rt > env->fbq_type) continue;
power = power_of(i);capa_factor = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE);
capacity = capacity_of(i);capa_factor = DIV_ROUND_CLOSEST(capacity, SCHED_POWER_SCALE); if (!capa_factor) capa_factor = fix_small_capacity(env->sd, group);@@ -6339,25 +6339,25 @@ static struct rq *find_busiest_queue(struct lb_env *env,
/* * When comparing with imbalance, use weighted_cpuload()
* which is not scaled with the cpu power.
* which is not scaled with the cpu capacity. */ if (capa_factor && rq->nr_running == 1 && wl > env->imbalance) continue; /* * For the load comparisons with the other cpu's, consider
* the weighted_cpuload() scaled with the cpu power, so that* the load can be moved away from the cpu that is potentially* running at a lower capacity.
* the weighted_cpuload() scaled with the cpu capacity, so* that the load can be moved away from the cpu that is* potentially running at a lower capacity. *
* Thus we're looking for max(wl_i / power_i), crosswise
* Thus we're looking for max(wl_i / capacity_i), crosswise * multiplication to rid ourselves of the division works out
* to: wl_i * power_j > wl_j * power_i; where j is our* previous maximum.
* to: wl_i * capacity_j > wl_j * capacity_i; where j is* our previous maximum. */
if (wl * busiest_power > busiest_load * power) {
if (wl * busiest_capacity > busiest_load * capacity) { busiest_load = wl;
busiest_power = power;
busiest_capacity = capacity; busiest = rq; } }diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index b384ccd07b..fe89387931 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -567,7 +567,7 @@ struct rq { struct root_domain *rd; struct sched_domain *sd;
unsigned long cpu_power;
unsigned long cpu_capacity; unsigned char idle_balance; /* For active balancing */-- 1.8.4.108.g55ea5f6
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
On Fri, 16 May 2014, Preeti Murthy wrote:
Hi Nicolas, You might want to change the subject.
That was a tongue-in-cheek subject. I was wondering if anyone would react to it. ;-)
s/sched: remove remaining power to the CPU/ sched: remove remaining usage of cpu *power* .
The subject has to explicitly specify in some way that it is a change made to the terminology.
Sure.
Nicolas
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This contains the architecture visible changes. Incidentally, only ARM takes advantage of the available pow^H^H^Hcapacity scaling hooks and therefore those changes outside kernel/sched/ are confined to one ARM specific file. The default arch_scale_smt_power() hook is not overridden by anyone.
Replacements are as follows:
arch_scale_freq_power --> arch_scale_freq_capacity arch_scale_smt_power --> arch_scale_smt_capacity SCHED_POWER_SCALE --> SCHED_CAPA_SCALE SCHED_POWER_SHIFT --> SCHED_POWER_SHIFT
The local usage of "power" in arch/arm/kernel/topology.c is also changed to "capacity" as appropriate.
Signed-off-by: Nicolas Pitre nico@linaro.org --- arch/arm/kernel/topology.c | 54 +++++++++++++++++++++--------------------- include/linux/sched.h | 6 ++--- kernel/sched/core.c | 6 ++--- kernel/sched/fair.c | 58 +++++++++++++++++++++++----------------------- 4 files changed, 62 insertions(+), 62 deletions(-)
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c index 71e1fec6d3..7dee34f786 100644 --- a/arch/arm/kernel/topology.c +++ b/arch/arm/kernel/topology.c @@ -26,30 +26,30 @@ #include <asm/topology.h>
/* - * cpu power scale management + * cpu capacity scale management */
/* - * cpu power table + * cpu capacity table * This per cpu data structure describes the relative capacity of each core. * On a heteregenous system, cores don't have the same computation capacity - * and we reflect that difference in the cpu_power field so the scheduler can - * take this difference into account during load balance. A per cpu structure - * is preferred because each CPU updates its own cpu_power field during the - * load balance except for idle cores. One idle core is selected to run the - * rebalance_domains for all idle cores and the cpu_power can be updated - * during this sequence. + * and we reflect that difference in the cpu_capacity field so the scheduler + * can take this difference into account during load balance. A per cpu + * structure is preferred because each CPU updates its own cpu_capacity field + * during the load balance except for idle cores. One idle core is selected + * to run the rebalance_domains for all idle cores and the cpu_capacity can be + * updated during this sequence. */ static DEFINE_PER_CPU(unsigned long, cpu_scale);
-unsigned long arch_scale_freq_power(struct sched_domain *sd, int cpu) +unsigned long arch_scale_freq_capacity(struct sched_domain *sd, int cpu) { return per_cpu(cpu_scale, cpu); }
-static void set_power_scale(unsigned int cpu, unsigned long power) +static void set_capacity_scale(unsigned int cpu, unsigned long capacity) { - per_cpu(cpu_scale, cpu) = power; + per_cpu(cpu_scale, cpu) = capacity; }
#ifdef CONFIG_OF @@ -62,11 +62,11 @@ struct cpu_efficiency { * Table of relative efficiency of each processors * The efficiency value must fit in 20bit and the final * cpu_scale value must be in the range - * 0 < cpu_scale < 3*SCHED_POWER_SCALE/2 + * 0 < cpu_scale < 3*SCHED_CAPA_SCALE/2 * in order to return at most 1 when DIV_ROUND_CLOSEST * is used to compute the capacity of a CPU. * Processors that are not defined in the table, - * use the default SCHED_POWER_SCALE value for cpu_scale. + * use the default SCHED_CAPA_SCALE value for cpu_scale. */ static const struct cpu_efficiency table_efficiency[] = { {"arm,cortex-a15", 3891}, @@ -83,9 +83,9 @@ static unsigned long middle_capacity = 1; * Iterate all CPUs' descriptor in DT and compute the efficiency * (as per table_efficiency). Also calculate a middle efficiency * as close as possible to (max{eff_i} - min{eff_i}) / 2 - * This is later used to scale the cpu_power field such that an - * 'average' CPU is of middle power. Also see the comments near - * table_efficiency[] and update_cpu_power(). + * This is later used to scale the cpu_capacity field such that an + * 'average' CPU is of middle capacity. Also see the comments near + * table_efficiency[] and update_cpu_capacity(). */ static void __init parse_dt_topology(void) { @@ -141,15 +141,15 @@ static void __init parse_dt_topology(void) * cpu_scale because all CPUs have the same capacity. Otherwise, we * compute a middle_capacity factor that will ensure that the capacity * of an 'average' CPU of the system will be as close as possible to - * SCHED_POWER_SCALE, which is the default value, but with the + * SCHED_CAPA_SCALE, which is the default value, but with the * constraint explained near table_efficiency[]. */ if (4*max_capacity < (3*(max_capacity + min_capacity))) middle_capacity = (min_capacity + max_capacity) - >> (SCHED_POWER_SHIFT+1); + >> (SCHED_CAPA_SHIFT+1); else middle_capacity = ((max_capacity / 3) - >> (SCHED_POWER_SHIFT-1)) + 1; + >> (SCHED_CAPA_SHIFT-1)) + 1;
}
@@ -158,20 +158,20 @@ static void __init parse_dt_topology(void) * boot. The update of all CPUs is in O(n^2) for heteregeneous system but the * function returns directly for SMP system. */ -static void update_cpu_power(unsigned int cpu) +static void update_cpu_capacity(unsigned int cpu) { if (!cpu_capacity(cpu)) return;
- set_power_scale(cpu, cpu_capacity(cpu) / middle_capacity); + set_capacity_scale(cpu, cpu_capacity(cpu) / middle_capacity);
- printk(KERN_INFO "CPU%u: update cpu_power %lu\n", - cpu, arch_scale_freq_power(NULL, cpu)); + printk(KERN_INFO "CPU%u: update cpu_capacity %lu\n", + cpu, arch_scale_freq_capacity(NULL, cpu)); }
#else static inline void parse_dt_topology(void) {} -static inline void update_cpu_power(unsigned int cpuid) {} +static inline void update_cpu_capacity(unsigned int cpuid) {} #endif
/* @@ -267,7 +267,7 @@ void store_cpu_topology(unsigned int cpuid)
update_siblings_masks(cpuid);
- update_cpu_power(cpuid); + update_cpu_capacity(cpuid);
printk(KERN_INFO "CPU%u: thread %d, cpu %d, socket %d, mpidr %x\n", cpuid, cpu_topology[cpuid].thread_id, @@ -297,7 +297,7 @@ void __init init_cpu_topology(void) { unsigned int cpu;
- /* init core mask and power*/ + /* init core mask and capacity */ for_each_possible_cpu(cpu) { struct cputopo_arm *cpu_topo = &(cpu_topology[cpu]);
@@ -307,7 +307,7 @@ void __init init_cpu_topology(void) cpumask_clear(&cpu_topo->core_sibling); cpumask_clear(&cpu_topo->thread_sibling);
- set_power_scale(cpu, SCHED_POWER_SCALE); + set_capacity_scale(cpu, SCHED_CAPA_SCALE); } smp_wmb();
diff --git a/include/linux/sched.h b/include/linux/sched.h index 31ca7b60e8..64651c0254 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -854,10 +854,10 @@ enum cpu_idle_type { };
/* - * Increase resolution of cpu_power calculations + * Increase resolution of cpu_capacity calculations */ -#define SCHED_POWER_SHIFT 10 -#define SCHED_POWER_SCALE (1L << SCHED_POWER_SHIFT) +#define SCHED_CAPA_SHIFT 10 +#define SCHED_CAPA_SCALE (1L << SCHED_CAPA_SHIFT)
/* * sched-domains (multiprocessor balancing) declarations: diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 00d04e9e7b..6fffb84a82 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5249,7 +5249,7 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level, cpulist_scnprintf(str, sizeof(str), sched_group_cpus(group));
printk(KERN_CONT " %s", str); - if (group->sgc->capacity != SCHED_POWER_SCALE) { + if (group->sgc->capacity != SCHED_CAPA_SCALE) { printk(KERN_CONT " (cpu_capacity = %d)", group->sgc->capacity); } @@ -5715,7 +5715,7 @@ build_overlap_sched_groups(struct sched_domain *sd, int cpu) * domains and no possible iteration will get us here, we won't * die on a /0 trap. */ - sg->sgc->capacity = SCHED_POWER_SCALE * cpumask_weight(sg_span); + sg->sgc->capacity = SCHED_CAPA_SCALE * cpumask_weight(sg_span); sg->sgc->capacity_orig = sg->sgc->capacity;
/* @@ -6923,7 +6923,7 @@ void __init sched_init(void) #ifdef CONFIG_SMP rq->sd = NULL; rq->rd = NULL; - rq->cpu_capacity = SCHED_POWER_SCALE; + rq->cpu_capacity = SCHED_CAPA_SCALE; rq->post_schedule = 0; rq->active_balance = 0; rq->next_balance = jiffies; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 06619b52dd..c8a748111b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1062,9 +1062,9 @@ static void update_numa_stats(struct numa_stats *ns, int nid) if (!cpus) return;
- ns->load = (ns->load * SCHED_POWER_SCALE) / ns->compute_capacity; + ns->load = (ns->load * SCHED_CAPA_SCALE) / ns->compute_capacity; ns->task_capacity = - DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_POWER_SCALE); + DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPA_SCALE); ns->has_free_capacity = (ns->nr_running < ns->task_capacity); }
@@ -4342,7 +4342,7 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p, }
/* Adjust by relative CPU capacity of the group */ - avg_load = (avg_load * SCHED_POWER_SCALE) / group->sgc->capacity; + avg_load = (avg_load * SCHED_CAPA_SCALE) / group->sgc->capacity;
if (local_group) { this_load = avg_load; @@ -5562,10 +5562,10 @@ static inline int get_sd_load_idx(struct sched_domain *sd,
static unsigned long default_scale_capacity(struct sched_domain *sd, int cpu) { - return SCHED_POWER_SCALE; + return SCHED_CAPA_SCALE; }
-unsigned long __weak arch_scale_freq_power(struct sched_domain *sd, int cpu) +unsigned long __weak arch_scale_freq_capacity(struct sched_domain *sd, int cpu) { return default_scale_capacity(sd, cpu); } @@ -5580,7 +5580,7 @@ static unsigned long default_scale_smt_capacity(struct sched_domain *sd, int cpu return smt_gain; }
-unsigned long __weak arch_scale_smt_power(struct sched_domain *sd, int cpu) +unsigned long __weak arch_scale_smt_capacity(struct sched_domain *sd, int cpu) { return default_scale_smt_capacity(sd, cpu); } @@ -5611,10 +5611,10 @@ static unsigned long scale_rt_capacity(int cpu) available = total - avg; }
- if (unlikely((s64)total < SCHED_POWER_SCALE)) - total = SCHED_POWER_SCALE; + if (unlikely((s64)total < SCHED_CAPA_SCALE)) + total = SCHED_CAPA_SCALE;
- total >>= SCHED_POWER_SHIFT; + total >>= SCHED_CAPA_SHIFT;
return div_u64(available, total); } @@ -5622,29 +5622,29 @@ static unsigned long scale_rt_capacity(int cpu) static void update_cpu_capacity(struct sched_domain *sd, int cpu) { unsigned long weight = sd->span_weight; - unsigned long capacity = SCHED_POWER_SCALE; + unsigned long capacity = SCHED_CAPA_SCALE; struct sched_group *sdg = sd->groups;
if ((sd->flags & SD_SHARE_CPUPOWER) && weight > 1) { if (sched_feat(ARCH_POWER)) - capacity *= arch_scale_smt_power(sd, cpu); + capacity *= arch_scale_smt_capacity(sd, cpu); else capacity *= default_scale_smt_capacity(sd, cpu);
- capacity >>= SCHED_POWER_SHIFT; + capacity >>= SCHED_CAPA_SHIFT; }
sdg->sgc->capacity_orig = capacity;
if (sched_feat(ARCH_POWER)) - capacity *= arch_scale_freq_power(sd, cpu); + capacity *= arch_scale_freq_capacity(sd, cpu); else capacity *= default_scale_capacity(sd, cpu);
- capacity >>= SCHED_POWER_SHIFT; + capacity >>= SCHED_CAPA_SHIFT;
capacity *= scale_rt_capacity(cpu); - capacity >>= SCHED_POWER_SHIFT; + capacity >>= SCHED_CAPA_SHIFT;
if (!capacity) capacity = 1; @@ -5733,7 +5733,7 @@ static inline int fix_small_capacity(struct sched_domain *sd, struct sched_group *group) { /* - * Only siblings can have significantly less than SCHED_POWER_SCALE + * Only siblings can have significantly less than SCHED_CAPA_SCALE */ if (!(sd->flags & SD_SHARE_CPUPOWER)) return 0; @@ -5798,11 +5798,11 @@ static inline int sg_capa_factor(struct lb_env *env, struct sched_group *group) cpus = group->group_weight;
/* smt := ceil(cpus / capacity), assumes: 1 < smt_capacity < 2 */ - smt = DIV_ROUND_UP(SCHED_POWER_SCALE * cpus, capacity_orig); + smt = DIV_ROUND_UP(SCHED_CAPA_SCALE * cpus, capacity_orig); capa_factor = cpus / smt; /* cores */
capa_factor = min_t(unsigned, - capa_factor, DIV_ROUND_CLOSEST(capacity, SCHED_POWER_SCALE)); + capa_factor, DIV_ROUND_CLOSEST(capacity, SCHED_CAPA_SCALE)); if (!capa_factor) capa_factor = fix_small_capacity(env->sd, group);
@@ -5848,7 +5848,7 @@ static inline void update_sg_lb_stats(struct lb_env *env,
/* Adjust by relative CPU capacity of the group */ sgs->group_capacity = group->sgc->capacity; - sgs->avg_load = (sgs->group_load*SCHED_POWER_SCALE) / sgs->group_capacity; + sgs->avg_load = (sgs->group_load*SCHED_CAPA_SCALE) / sgs->group_capacity;
if (sgs->sum_nr_running) sgs->load_per_task = sgs->sum_weighted_load / sgs->sum_nr_running; @@ -6042,7 +6042,7 @@ static int check_asym_packing(struct lb_env *env, struct sd_lb_stats *sds)
env->imbalance = DIV_ROUND_CLOSEST( sds->busiest_stat.avg_load * sds->busiest_stat.group_capacity, - SCHED_POWER_SCALE); + SCHED_CAPA_SCALE);
return 1; } @@ -6071,7 +6071,7 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds) imbn = 1;
scaled_busy_load_per_task = - (busiest->load_per_task * SCHED_POWER_SCALE) / + (busiest->load_per_task * SCHED_CAPA_SCALE) / busiest->group_capacity;
if (busiest->avg_load + scaled_busy_load_per_task >= @@ -6090,7 +6090,7 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds) min(busiest->load_per_task, busiest->avg_load); capa_now += local->group_capacity * min(local->load_per_task, local->avg_load); - capa_now /= SCHED_POWER_SCALE; + capa_now /= SCHED_CAPA_SCALE;
/* Amount of load we'd subtract */ if (busiest->avg_load > scaled_busy_load_per_task) { @@ -6101,16 +6101,16 @@ void fix_small_imbalance(struct lb_env *env, struct sd_lb_stats *sds)
/* Amount of load we'd add */ if (busiest->avg_load * busiest->group_capacity < - busiest->load_per_task * SCHED_POWER_SCALE) { + busiest->load_per_task * SCHED_CAPA_SCALE) { tmp = (busiest->avg_load * busiest->group_capacity) / local->group_capacity; } else { - tmp = (busiest->load_per_task * SCHED_POWER_SCALE) / + tmp = (busiest->load_per_task * SCHED_CAPA_SCALE) / local->group_capacity; } capa_move += local->group_capacity * min(local->load_per_task, local->avg_load + tmp); - capa_move /= SCHED_POWER_SCALE; + capa_move /= SCHED_CAPA_SCALE;
/* Move if we gain throughput */ if (capa_move > capa_now) @@ -6160,7 +6160,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s load_above_capacity = (busiest->sum_nr_running - busiest->group_capa_factor);
- load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_POWER_SCALE); + load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_CAPA_SCALE); load_above_capacity /= busiest->group_capacity; }
@@ -6178,7 +6178,7 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s env->imbalance = min( max_pull * busiest->group_capacity, (sds->avg_load - local->avg_load) * local->group_capacity - ) / SCHED_POWER_SCALE; + ) / SCHED_CAPA_SCALE;
/* * if *imbalance is less than the average load per runnable task @@ -6232,7 +6232,7 @@ static struct sched_group *find_busiest_group(struct lb_env *env) if (!sds.busiest || busiest->sum_nr_running == 0) goto out_balanced;
- sds.avg_load = (SCHED_POWER_SCALE * sds.total_load) / sds.total_capa; + sds.avg_load = (SCHED_CAPA_SCALE * sds.total_load) / sds.total_capa;
/* * If the busiest group is imbalanced the below checks don't @@ -6331,7 +6331,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, continue;
capacity = capacity_of(i); - capa_factor = DIV_ROUND_CLOSEST(capacity, SCHED_POWER_SCALE); + capa_factor = DIV_ROUND_CLOSEST(capacity, SCHED_CAPA_SCALE); if (!capa_factor) capa_factor = fix_small_capacity(env->sd, group);
On Wed, May 14, 2014 at 04:57:10PM -0400, Nicolas Pitre wrote:
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This contains the architecture visible changes. Incidentally, only ARM takes advantage of the available pow^H^H^Hcapacity scaling hooks and therefore those changes outside kernel/sched/ are confined to one ARM specific file. The default arch_scale_smt_power() hook is not overridden by anyone.
Replacements are as follows:
arch_scale_freq_power --> arch_scale_freq_capacity arch_scale_smt_power --> arch_scale_smt_capacity SCHED_POWER_SCALE --> SCHED_CAPA_SCALE SCHED_POWER_SHIFT --> SCHED_POWER_SHIFT
The patch seems to actually make that CAPA_SHIFT
The local usage of "power" in arch/arm/kernel/topology.c is also changed to "capacity" as appropriate.
For some reason every time I read: 'capa' I think of some south American monster -- http://en.wikipedia.org/wiki/Chupacabra, I'm not at all sure why my brain links them.
But yes, once we kill the capacity stuff we have now with some utilization bound, capacity becomes uniquely the compute capacity.
On Thu, 15 May 2014, Peter Zijlstra wrote:
On Wed, May 14, 2014 at 04:57:10PM -0400, Nicolas Pitre wrote:
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This contains the architecture visible changes. Incidentally, only ARM takes advantage of the available pow^H^H^Hcapacity scaling hooks and therefore those changes outside kernel/sched/ are confined to one ARM specific file. The default arch_scale_smt_power() hook is not overridden by anyone.
Replacements are as follows:
arch_scale_freq_power --> arch_scale_freq_capacity arch_scale_smt_power --> arch_scale_smt_capacity SCHED_POWER_SCALE --> SCHED_CAPA_SCALE SCHED_POWER_SHIFT --> SCHED_POWER_SHIFT
The patch seems to actually make that CAPA_SHIFT
Huh... right, of course.
The local usage of "power" in arch/arm/kernel/topology.c is also changed to "capacity" as appropriate.
For some reason every time I read: 'capa' I think of some south American monster -- http://en.wikipedia.org/wiki/Chupacabra, I'm not at all sure why my brain links them.
:-)
capa != paca
I chose that not to make this much longer than "POWER", and since there are already "LOAD" related constants, I thought there was some symetry to another 4-letter identifier. Do you have other suggestions?
Nicolas
On Thu, 15 May 2014, Nicolas Pitre wrote:
On Thu, 15 May 2014, Peter Zijlstra wrote:
On Wed, May 14, 2014 at 04:57:10PM -0400, Nicolas Pitre wrote:
It is better not to think about compute capacity as being equivalent to "CPU power". The upcoming "power aware" scheduler may create confusion with the notion of energy consumption if "power" is used too liberally.
This contains the architecture visible changes. Incidentally, only ARM takes advantage of the available pow^H^H^Hcapacity scaling hooks and therefore those changes outside kernel/sched/ are confined to one ARM specific file. The default arch_scale_smt_power() hook is not overridden by anyone.
Replacements are as follows:
arch_scale_freq_power --> arch_scale_freq_capacity arch_scale_smt_power --> arch_scale_smt_capacity SCHED_POWER_SCALE --> SCHED_CAPA_SCALE SCHED_POWER_SHIFT --> SCHED_POWER_SHIFT
The patch seems to actually make that CAPA_SHIFT
Huh... right, of course.
The local usage of "power" in arch/arm/kernel/topology.c is also changed to "capacity" as appropriate.
For some reason every time I read: 'capa' I think of some south American monster -- http://en.wikipedia.org/wiki/Chupacabra, I'm not at all sure why my brain links them.
:-)
capa != paca
I chose that not to make this much longer than "POWER", and since there are already "LOAD" related constants, I thought there was some symetry to another 4-letter identifier. Do you have other suggestions?
I understand that Vincent is inclined to rebase his future work on top of this renaming.
Should I repost or you're happy to fix the commit log manually? Any other concerns I should address?
Nicolas
Hi Nico,
Thanks for doing this renaming. I remember that you asked me to do this while working on cpu_power but my work has not evolved as fast as expected and as it already implies some renaming other than s/power/capacity/ i have postponed it to not make review to complex.
Nevertheless, i can manage the conflicts afterward and rebase my patches, depending of the review status
Vincent
On 14 May 2014 22:57, Nicolas Pitre nicolas.pitre@linaro.org wrote:
"Power" is a very bad term in the scheduler context. There are so many meanings that can be attached to it. And with the upcoming "power aware" scheduler work confusion is sure to happen.
The definition of "power" is typically the rate at which work is performed, energy is converted or electric energy is transferred. The notion of "compute capacity" is rather at odds with "power" to the point many comments in the code have to make it explicit that "capacity" is the actual intended meaning.
So let's make it clear what we man by using "capacity" in place of "power" directly in the code. That will make the introduction of actual "power consumption" concepts much clearer later on.
This is based on the latest tip tree where scheduler changes are already queued.
Note: The diffstat is not completely symetric wrt added/removed lines as some comments were reflowed.
arch/arm/kernel/topology.c | 54 +++---- include/linux/sched.h | 8 +- kernel/sched/core.c | 89 ++++++----- kernel/sched/fair.c | 322 ++++++++++++++++++++------------------- kernel/sched/sched.h | 18 +-- 5 files changed, 246 insertions(+), 245 deletions(-)
Nicolas
On Thu, 15 May 2014, Vincent Guittot wrote:
Hi Nico,
Thanks for doing this renaming. I remember that you asked me to do this while working on cpu_power but my work has not evolved as fast as expected and as it already implies some renaming other than s/power/capacity/ i have postponed it to not make review to complex.
Nevertheless, i can manage the conflicts afterward and rebase my patches, depending of the review status
OK. Could you provide review status of your own then?
Nicolas
linaro-kernel@lists.linaro.org
-
Nicolas Pitre -
Peter Zijlstra -
Preeti Murthy -
Vincent Guittot