eas-dev

eas-dev@lists.linaro.org

423 discussions

cpu_util() after use cumulative_runnable_avg always hit 0

by Ke Wang

Hi Joonwoo, Recently, I backport EAS1.3 related patches (the latest commit is ec888d46d8993b2bf205ed375e538a3819c23659) on google android-4.4 branch to SPREADTRUM platform(kernel3.18, 4 A53 LITTLE + 4A53 big), and enabled WALT signal, tracing util_avg_pelt(avg.util_avg), util_avg_walt(cumulative_runnable_avg), util_avg_freq(prev_runnable_sum) at the same time. The event ftrace (a game scenario: Subway Surf) is as below: <idle>-0 [002] dn.3 53.899765: sched_load_avg_cpu: cpu=2 load_avg=306 util_avg=27 util_avg_pelt=57 util_avg_walt=27 util_avg_freq=69 <idle>-0 [000] d.s5 53.899766: sched_load_avg_cpu: cpu=0 load_avg=385 util_avg=0 util_avg_pelt=115 util_avg_walt=0 util_avg_freq=121 UnityMain-4608 [006] d..3 53.899773: sched_load_avg_cpu: cpu=6 load_avg=964 util_avg=0 util_avg_pelt=923 util_avg_walt=0 util_avg_freq=678 UnityMain-4608 [006] d..3 53.899774: sched_load_avg_cpu: cpu=6 load_avg=964 util_avg=0 util_avg_pelt=923 util_avg_walt=0 util_avg_freq=678 <idle>-0 [000] dn.3 53.899813: sched_load_avg_cpu: cpu=0 load_avg=385 util_avg=9 util_avg_pelt=115 util_avg_walt=9 util_avg_freq=121 kworker/u17:2-4204 [001] d..3 53.899830: sched_load_avg_cpu: cpu=1 load_avg=5445 util_avg=223 util_avg_pelt=139 util_avg_walt=223 util_avg_freq=175 kworker/u17:2-4204 [001] d..3 53.899836: sched_load_avg_cpu: cpu=1 load_avg=5445 util_avg=144 util_avg_pelt=139 util_avg_walt=144 util_avg_freq=175 kworker/u17:1-2763 [001] d..3 53.899853: sched_load_avg_cpu: cpu=1 load_avg=5445 util_avg=144 util_avg_pelt=139 util_avg_walt=144 util_avg_freq=175 kworker/u17:1-2763 [001] d..3 53.899858: sched_load_avg_cpu: cpu=1 load_avg=5445 util_avg=100 util_avg_pelt=139 util_avg_walt=100 util_avg_freq=175 adbd-2915 [000] d..3 53.899900: sched_load_avg_cpu: cpu=0 load_avg=385 util_avg=9 util_avg_pelt=115 util_avg_walt=9 util_avg_freq=121 adbd-2915 [000] d..3 53.899907: sched_load_avg_cpu: cpu=0 load_avg=385 util_avg=0 util_avg_pelt=115 util_avg_walt=0 util_avg_freq=121 adbd-2915 [000] d..3 53.899909: sched_load_avg_cpu: cpu=0 load_avg=385 util_avg=0 util_avg_pelt=115 util_avg_walt=0 util_avg_freq=121 mali-event-hnd-2919 [001] d..4 53.899910: sched_load_avg_cpu: cpu=3 load_avg=934 util_avg=0 util_avg_pelt=190 util_avg_walt=0 util_avg_freq=155 >From the ftrace, we found that util_avg_walt always hit 0 while util_pelt&util_avg_freq stay on a relative big value. Could you give some suggestion for this? Thanks in advance.

7 years, 8 months

[PATCH v5 0/3] sched/fair: Introduce scaled capacity awareness in enqueue

by Rohit Jain

Changelog: --------------------------------------------------------------------------- v1->v2: * Changed the dynamic threshold calculation as the having global state can be avoided. v2->v3: * Split up the patch for find_idlest_cpu and select_idle_sibling code paths. v3->v4: * Rebased it to peterz's tree (apologies for wrong tree for v3) v4->v5: * Changed the threshold to 768 from 819 for easier shifts * Changed the find_idlest_cpu code path to be simpler * Changed the select_idle_core code path to search for idlest+full_capacity core * Added scaled capacity awareness to wake_affine_idle code path --------------------------------------------------------------------------- During OLTP workload runs, threads can end up on CPUs with a lot of softIRQ activity, thus delaying progress. For more reliable and faster runs, if the system can spare it, these threads should be scheduled on CPUs with lower IRQ/RT activity. Currently, the scheduler takes into account the original capacity of CPUs when providing 'hints' for select_idle_sibling code path to return an idle CPU. However, the rest of the select_idle_* code paths remain capacity agnostic. Further, these code paths are only aware of the original capacity and not the capacity stolen by IRQ/RT activity. This patch introduces capacity awarness in scheduler (CAS) which avoids CPUs which might have their capacities reduced (due to IRQ/RT activity) when trying to schedule threads (on the push side) in the system. This awareness has been added into the fair scheduling class. It does so by, using the following algorithm: 1) As in rt_avg the scaled capacities are already calculated. 2) Any CPU which is running below 80% capacity is considered running low on capacity. 3) During idle CPU search if a CPU is found running low on capacity, it is skipped if better CPUs are available. 4) If none of the CPUs are better in terms of idleness and capacity, then the low-capacity CPU is considered to be the best available CPU. The performance numbers: --------------------------------------------------------------------------- CAS shows upto 1.5% improvement on x86 when running 'SELECT' database workload. For microbenchmark results, I used hackbench running with process along with, running ping on CPU 0,1 and 2 as: 'ping -l 10000 -q -s 10 -f hostX' The results below should be read as: * 'Baseline without ping' is how the workload would've behaved if there was no IRQ activity. * Compare 'Baseline with ping' and 'Baseline without ping' to see the effect of ping * Compare 'Baseline with ping' and 'CAS with ping' to see the improvement CAS can give over baseline Following are the runtime(s) with hackbench and ping activity as described above (lower is better), on a 44 core 2 socket x86 machine: +---------------+------+--------+--------+ |Num. |CAS |Baseline|Baseline| |Tasks |with |with |without | |(groups of 40) |ping |ping |ping | +---------------+------+--------+--------+ | |Mean |Mean |Mean | +---------------+------+--------+--------+ |1 | 0.55 | 0.59 | 0.53 | |2 | 0.66 | 0.81 | 0.51 | |4 | 0.99 | 1.16 | 0.95 | |8 | 1.92 | 1.93 | 1.88 | |16 | 3.24 | 3.26 | 3.15 | |32 | 5.93 | 5.98 | 5.68 | |64 | 11.55| 11.94 | 10.89 | +---------------+------+--------+--------+ Rohit Jain (3): sched/fair: Introduce scaled capacity awareness in find_idlest_cpu code path sched/fair: Introduce scaled capacity awareness in select_idle_sibling code path sched/fair: Introduce scaled capacity awareness in wake_affine_idle code path kernel/sched/fair.c | 66 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 53 insertions(+), 13 deletions(-) -- 2.7.4

7 years, 8 months

Re: [Eas-dev] [PATCH 1/3] sched/fair: Introduce scaled capacity awareness in find_idlest_cpu code path

by Atish Patra

Minor nit: Patch version missing in the subject line. Other than that: Reviewed-by: Atish Patra <atish.patra(a)oracle.com> Regards, Atish ----- Original Message ----- From: rohit.k.jain(a)oracle.com To: linux-kernel(a)vger.kernel.org, eas-dev(a)lists.linaro.org Cc: peterz(a)infradead.org, mingo(a)redhat.com, joelaf(a)google.com, atish.patra(a)oracle.com, vincent.guittot(a)linaro.org, dietmar.eggemann(a)arm.com, morten.rasmussen(a)arm.com Sent: Saturday, October 7, 2017 6:44:47 PM GMT -06:00 US/Canada Central Subject: [PATCH 1/3] sched/fair: Introduce scaled capacity awareness in find_idlest_cpu code path While looking for idle CPUs for a waking task, we should also account for the delays caused due to the bandwidth reduction by RT/IRQ tasks. This patch does that by trying to find a higher capacity CPU with minimum wake up latency. Signed-off-by: Rohit Jain <rohit.k.jain(a)oracle.com> --- kernel/sched/fair.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0107280..eaede50 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5579,6 +5579,11 @@ static unsigned long capacity_orig_of(int cpu) return cpu_rq(cpu)->cpu_capacity_orig; } +static inline bool full_capacity(int cpu) +{ + return (capacity_of(cpu) >= (capacity_orig_of(cpu)*768 >> 10)); +} + static unsigned long cpu_avg_load_per_task(int cpu) { struct rq *rq = cpu_rq(cpu); @@ -5865,8 +5870,10 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu) unsigned long load, min_load = ULONG_MAX; unsigned int min_exit_latency = UINT_MAX; u64 latest_idle_timestamp = 0; + unsigned int backup_cap = 0; int least_loaded_cpu = this_cpu; int shallowest_idle_cpu = -1; + int shallowest_idle_cpu_backup = -1; int i; /* Check if we have any choice: */ @@ -5876,6 +5883,7 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu) /* Traverse only the allowed CPUs */ for_each_cpu_and(i, sched_group_span(group), &p->cpus_allowed) { if (idle_cpu(i)) { + int idle_candidate = -1; struct rq *rq = cpu_rq(i); struct cpuidle_state *idle = idle_get_state(rq); if (idle && idle->exit_latency < min_exit_latency) { @@ -5886,7 +5894,7 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu) */ min_exit_latency = idle->exit_latency; latest_idle_timestamp = rq->idle_stamp; - shallowest_idle_cpu = i; + idle_candidate = i; } else if ((!idle || idle->exit_latency == min_exit_latency) && rq->idle_stamp > latest_idle_timestamp) { /* @@ -5895,7 +5903,16 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu) * a warmer cache. */ latest_idle_timestamp = rq->idle_stamp; - shallowest_idle_cpu = i; + idle_candidate = i; + } + + if (idle_candidate != -1) { + if (full_capacity(idle_candidate)) { + shallowest_idle_cpu = idle_candidate; + } else if (capacity_of(idle_candidate) > backup_cap) { + shallowest_idle_cpu_backup = idle_candidate; + backup_cap = capacity_of(idle_candidate); + } } } else if (shallowest_idle_cpu == -1) { load = weighted_cpuload(cpu_rq(i)); @@ -5906,7 +5923,11 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu) } } - return shallowest_idle_cpu != -1 ? shallowest_idle_cpu : least_loaded_cpu; + if (shallowest_idle_cpu != -1) + return shallowest_idle_cpu; + + return (shallowest_idle_cpu_backup != -1 ? + shallowest_idle_cpu_backup : least_loaded_cpu); } #ifdef CONFIG_SCHED_SMT -- 2.7.4

7 years, 8 months

[PATCH V3] Per Sched domain over utilization

by Thara Gopinath

The current implementation of overutilization, aborts energy aware scheduling if any cpu in the system is over-utilized. This patch introduces over utilization flag per sched domain level instead of a single flag system wide. Load balancing is done at the sched domain where any of the cpu is over utilized. If energy aware scheduling is enabled and no cpu in a sched domain is overuttilized, load balancing is skipped for that sched domain and energy aware scheduling continues at that level. The implementation takes advantage of the shared sched_domain structure that is common across all the sched domains at a level. The new flag introduced is placed in this structure so that all the sched domains the same level share the flag. In case of an overutilized cpu, the flag gets set at level1 sched_domain. The flag at the parent sched_domain level gets set in either of the two following scenarios. 1. There is a misfit task in one of the cpu's in this sched_domain. 2. The total utilization of the domain is greater than the domain capacity The flag is cleared if no cpu in a sched domain is overutilized. This implementation still can have corner scenarios with respect to misfit tasks. For example consider a sched group with n cpus and n+1 70%utilized tasks. Ideally this is a case for load balance to happen in a parent sched domain. But neither the total group utilization is high enough for the load balance to be triggered in the parent domain nor there is a cpu with a single overutilized task so that aload balance is triggered in a parent domain. But again this could be a purely academic sceanrio, as during task wake up these tasks will be placed more appropriately. Signed-off-by: Thara Gopinath <thara.gopinath(a)linaro.org> --- V2->V3: - Rebased on latest kernel. - The previous check for misfit task is replaced with the newely introduced rq->misfit_task flag. V1->V2: - Removed overutilized flag from sched_group structure. - In case of misfit task, it is ensured that a load balance is triggered in a parent sched domain with assymetric cpu capacities. include/linux/sched/topology.h | 1 + kernel/sched/fair.c | 137 +++++++++++++++++++++++++++++++++-------- kernel/sched/sched.h | 3 - kernel/sched/topology.c | 8 +-- 4 files changed, 117 insertions(+), 32 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 3137750..ae44044 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -88,6 +88,7 @@ struct sched_domain_shared { atomic_t ref; atomic_t nr_busy_cpus; int has_idle_cores; + bool overutilized; }; struct sched_domain { diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a9ac67c..34bdfeb 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4791,6 +4791,29 @@ static inline void hrtick_update(struct rq *rq) static bool cpu_overutilized(int cpu); +static bool +is_sd_overutilized(struct sched_domain *sd) +{ + if (sd) + return sd->shared->overutilized; + else + return false; +} + +static void +set_sd_overutilized(struct sched_domain *sd) +{ + if (sd) + sd->shared->overutilized = true; +} + +static void +clear_sd_overutilized(struct sched_domain *sd) +{ + if (sd) + sd->shared->overutilized = false; +} + /* * The enqueue_task method is called before nr_running is * increased. Here we update the fair scheduling stats and @@ -4800,6 +4823,7 @@ static void enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) { struct cfs_rq *cfs_rq; + struct sched_domain *sd; struct sched_entity *se = &p->se; int task_new = !(flags & ENQUEUE_WAKEUP); @@ -4843,9 +4867,12 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) if (!se) { add_nr_running(rq, 1); - if (!task_new && !rq->rd->overutilized && - cpu_overutilized(rq->cpu)) - rq->rd->overutilized = true; + rcu_read_lock(); + sd = rcu_dereference(rq->sd); + if (!task_new && !is_sd_overutilized(sd) && + cpu_overutilized(rq->cpu)) + set_sd_overutilized(sd); + rcu_read_unlock(); } hrtick_update(rq); } @@ -6276,8 +6303,7 @@ static int select_energy_cpu_brute(struct task_struct *p, int prev_cpu) unsigned long max_spare = 0; struct sched_domain *sd; - rcu_read_lock(); - + /* The rcu lock is/should be held in the caller function */ sd = rcu_dereference(per_cpu(sd_ea, prev_cpu)); if (!sd) @@ -6315,8 +6341,6 @@ static int select_energy_cpu_brute(struct task_struct *p, int prev_cpu) } unlock: - rcu_read_unlock(); - if (energy_cpu == prev_cpu && !cpu_overutilized(prev_cpu)) return prev_cpu; @@ -6350,10 +6374,16 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f && cpumask_test_cpu(cpu, &p->cpus_allowed); } - if (energy_aware() && !(cpu_rq(prev_cpu)->rd->overutilized)) - return select_energy_cpu_brute(p, prev_cpu); - rcu_read_lock(); + sd = rcu_dereference(cpu_rq(prev_cpu)->sd); + if (energy_aware() && + !is_sd_overutilized(sd)) { + new_cpu = select_energy_cpu_brute(p, prev_cpu); + goto unlock; + } + + sd = NULL; + for_each_domain(cpu, tmp) { if (!(tmp->flags & SD_LOAD_BALANCE)) break; @@ -6418,6 +6448,8 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f } /* while loop will break here if sd == NULL */ } + +unlock: rcu_read_unlock(); return new_cpu; @@ -7478,6 +7510,7 @@ struct sd_lb_stats { struct sched_group *local; /* Local group in this sd */ unsigned long total_load; /* Total load of all groups in sd */ unsigned long total_capacity; /* Total capacity of all groups in sd */ + unsigned long total_util; /* Total util of all groups in sd */ unsigned long avg_load; /* Average load across all groups in sd */ struct sg_lb_stats busiest_stat;/* Statistics of the busiest group */ @@ -7497,6 +7530,7 @@ static inline void init_sd_lb_stats(struct sd_lb_stats *sds) .local = NULL, .total_load = 0UL, .total_capacity = 0UL, + .total_util = 0UL, .busiest_stat = { .avg_load = 0UL, .sum_nr_running = 0, @@ -7792,7 +7826,7 @@ group_type group_classify(struct sched_group *group, static inline void update_sg_lb_stats(struct lb_env *env, struct sched_group *group, int load_idx, int local_group, struct sg_lb_stats *sgs, - bool *overload, bool *overutilized) + bool *overload, bool *overutilized, bool *misfit_task) { unsigned long load; int i, nr_running; @@ -7831,8 +7865,16 @@ static inline void update_sg_lb_stats(struct lb_env *env, !sgs->group_misfit_task && rq->misfit_task) sgs->group_misfit_task = capacity_of(i); - if (cpu_overutilized(i)) + if (cpu_overutilized(i)) { *overutilized = true; + /* + * If the cpu is overutilized and if there is only one + * current task in cfs runqueue, it is potentially a misfit + * task. + */ + if (rq->misfit_task) + *misfit_task = true; + } } /* Adjust by relative CPU capacity of the group */ @@ -7974,12 +8016,12 @@ static inline enum fbq_type fbq_classify_rq(struct rq *rq) */ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sds) { - struct sched_domain *child = env->sd->child; + struct sched_domain *child = env->sd->child, *sd; struct sched_group *sg = env->sd->groups; struct sg_lb_stats *local = &sds->local_stat; struct sg_lb_stats tmp_sgs; int load_idx, prefer_sibling = 0; - bool overload = false, overutilized = false; + bool overload = false, overutilized = false, misfit_task = false; if (child && child->flags & SD_PREFER_SIBLING) prefer_sibling = 1; @@ -8001,7 +8043,8 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd } update_sg_lb_stats(env, sg, load_idx, local_group, sgs, - &overload, &overutilized); + &overload, &overutilized, + &misfit_task); if (local_group) goto next_group; @@ -8032,6 +8075,7 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd /* Now, start updating sd_lb_stats */ sds->total_load += sgs->group_load; sds->total_capacity += sgs->group_capacity; + sds->total_util += sgs->group_util; sg = sg->next; } while (sg != env->sd->groups); @@ -8045,14 +8089,45 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd /* update overload indicator if we are at root domain */ if (env->dst_rq->rd->overload != overload) env->dst_rq->rd->overload = overload; + } - /* Update over-utilization (tipping point, U >= 0) indicator */ - if (env->dst_rq->rd->overutilized != overutilized) - env->dst_rq->rd->overutilized = overutilized; - } else { - if (!env->dst_rq->rd->overutilized && overutilized) - env->dst_rq->rd->overutilized = true; + if (overutilized) + set_sd_overutilized(env->sd); + else + clear_sd_overutilized(env->sd); + + /* + * If there is a misfit task in one cpu in this sched_domain + * it is likely that the imbalance cannot be sorted out among + * the cpu's in this sched_domain. In this case set the + * overutilized flag at the parent sched_domain. + */ + if (misfit_task) { + + sd = env->sd->parent; + + /* + * In case of a misfit task, load balance at the parent + * sched domain level will make sense only if the the cpus + * have a different capacity. If cpus at a domain level have + * the same capacity, the misfit task cannot be well + * accomodated in any of the cpus and there in no point in + * trying a load balance at this level + */ + while (sd) { + if (sd->flags & SD_ASYM_CPUCAPACITY) { + set_sd_overutilized(sd); + break; + } + sd = sd->parent; + } } + + /* If the domain util is greater that domain capacity, load balancing + * needs to be done at the next sched domain level as well + */ + if (sds->total_capacity * 1024 < sds->total_util * capacity_margin) + set_sd_overutilized(env->sd->parent); } /** @@ -8279,8 +8354,10 @@ static struct sched_group *find_busiest_group(struct lb_env *env) */ update_sd_lb_stats(env, &sds); - if (energy_aware() && !env->dst_rq->rd->overutilized) - goto out_balanced; + if (energy_aware()) { + if (!is_sd_overutilized(env->sd)) + goto out_balanced; + } local = &sds.local_stat; busiest = &sds.busiest_stat; @@ -9164,6 +9241,11 @@ static void rebalance_domains(struct rq *rq, enum cpu_idle_type idle) rcu_read_lock(); for_each_domain(cpu, sd) { + if (energy_aware()) { + if (!is_sd_overutilized(sd)) + continue; + } + /* * Decay the newidle max times here because this is a regular * visit to all the domains. Decay ~1% per second. @@ -9466,6 +9548,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) { struct cfs_rq *cfs_rq; struct sched_entity *se = &curr->se; + struct sched_domain *sd; for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); @@ -9477,8 +9560,12 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued) rq->misfit_task = !task_fits_capacity(curr, capacity_of(rq->cpu)); - if (!rq->rd->overutilized && cpu_overutilized(task_cpu(curr))) - rq->rd->overutilized = true; + rcu_read_lock(); + sd = rcu_dereference(rq->sd); + if (!is_sd_overutilized(sd) && + cpu_overutilized(task_cpu(curr))) + set_sd_overutilized(sd); + rcu_read_unlock(); } /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8d27d5b..1604ef2 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -585,9 +585,6 @@ struct root_domain { /* Indicate more than one runnable task for any CPU */ bool overload; - /* Indicate one or more cpus over-utilized (tipping point) */ - bool overutilized; - /* * The bit corresponding to a CPU gets set here if such CPU has more * than one runnable -deadline task (as it is below for RT tasks). diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 263e549..e5ba6fc 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1040,11 +1040,11 @@ sd_init(struct sched_domain_topology_level *tl, * For all levels sharing cache; connect a sched_domain_shared * instance. */ - if (sd->flags & SD_SHARE_PKG_RESOURCES) { - sd->shared = *per_cpu_ptr(sdd->sds, sd_id); - atomic_inc(&sd->shared->ref); + sd->shared = *per_cpu_ptr(sdd->sds, sd_id); + atomic_inc(&sd->shared->ref); + + if (sd->flags & SD_SHARE_PKG_RESOURCES) atomic_set(&sd->shared->nr_busy_cpus, sd_weight); - } sd->private = sdd; -- 2.1.4

7 years, 8 months

[PATCH v4 0/3] sched/fair: Introduce scaled capacity awareness in enqueue

by Rohit Jain

During OLTP workload runs, threads can end up on CPUs with a lot of softIRQ activity, thus delaying progress. For more reliable and faster runs, if the system can spare it, these threads should be scheduled on CPUs with lower IRQ/RT activity. Currently, the scheduler takes into account the original capacity of CPUs when providing 'hints' for select_idle_sibling code path to return an idle CPU. However, the rest of the select_idle_* code paths remain capacity agnostic. Further, these code paths are only aware of the original capacity and not the capacity stolen by IRQ/RT activity. This patch introduces capacity awarness in scheduler (CAS) which avoids CPUs which might have their capacities reduced (due to IRQ/RT activity) when trying to schedule threads (on the push side) in the system. This awareness has been added into the fair scheduling class. It does so by, using the following algorithm: 1) As in rt_avg the scaled capacities are already calculated. 2) Any CPU which is running below 80% capacity is considered running low on capacity. 3) During idle CPU search if a CPU is found running low on capacity, it is skipped if better CPUs are available. 4) If none of the CPUs are better in terms of idleness and capacity, then the low-capacity CPU is considered to be the best available CPU. The performance numbers: --------------------------------------------------------------------------- CAS shows upto 1.5% improvement on x86 when running 'SELECT' database workload. I also used barrier.c (open_mp code) as a micro-benchmark. It does a number of iterations and barrier sync at the end of each for loop. I was also running ping on CPU 0 as: 'ping -l 10000 -q -s 10 -f host2' The results below should be read as: * 'Baseline without ping' is how the workload would've behaved if there was no IRQ activity. * Compare 'Baseline with ping' and 'Baseline without ping' to see the effect of ping * Compare 'Baseline with ping' and 'CAS with ping' to see the improvement CAS can give over baseline The program (barrier.c) can be found at: http://www.spinics.net/lists/kernel/msg2506955.html Following are the results for the iterations per second with this micro-benchmark (higher is better), on a 20 core x86 machine: +-------+----------------+----------------+------------------+ |Num. |CAS |Baseline |Baseline without | |Threads|with ping |with ping |ping | +-------+-------+--------+-------+--------+-------+----------+ | |Mean |Std. Dev|Mean |Std. Dev|Mean |Std. Dev | +-------+-------+--------+-------+--------+-------+----------+ |1 | 511.7 | 6.9 | 508.3 | 17.3 | 514.6 | 4.7 | |2 | 486.8 | 16.3 | 463.9 | 17.4 | 510.8 | 3.9 | |4 | 466.1 | 11.7 | 451.4 | 12.5 | 489.3 | 4.1 | |8 | 433.6 | 3.7 | 427.5 | 2.2 | 447.6 | 5.0 | |16 | 391.9 | 7.9 | 385.5 | 16.4 | 396.2 | 0.3 | |32 | 269.3 | 5.3 | 266.0 | 6.6 | 276.8 | 0.2 | +-------+-------+--------+-------+--------+-------+----------+ Following are the runtime(s) with hackbench and ping activity as described above (lower is better), on a 20 core x86 machine: +---------------+------+--------+--------+ |Num. |CAS |Baseline|Baseline| |Tasks |with |with |without | |(groups of 40) |ping |ping |ping | +---------------+------+--------+--------+ | |Mean |Mean |Mean | +---------------+------+--------+--------+ |1 | 0.97 | 0.97 | 0.68 | |2 | 1.36 | 1.36 | 1.30 | |4 | 2.57 | 2.57 | 1.84 | |8 | 3.31 | 3.34 | 2.86 | |16 | 5.63 | 5.71 | 4.61 | |25 | 7.99 | 8.23 | 6.78 | +---------------+------+--------+--------+ Changelog: --------------------------------------------------------------------------- v1->v2: * Changed the dynamic threshold calculation as the having global state can be avoided. v2->v3: * Split up the patch for find_idlest_cpu and select_idle_sibling code paths. v3->v4: * Rebased it to peterz's tree (apologies for wrong tree for v3) Previous discussion can be found at: --------------------------------------------------------------------------- https://patchwork.kernel.org/patch/9741351/ https://lists.linaro.org/pipermail/eas-dev/2017-August/000933.html Rohit Jain (3): sched/fair: Introduce scaled capacity awareness in find_idlest_cpu code path sched/fair: Introduce scaled capacity awareness in select_idle_sibling code path ignore_this_patch: Fixing compilation error on Peter's tree kernel/sched/fair.c | 81 +++++++++++++++++++++++++++++++++++++++--------- kernel/time/tick-sched.c | 1 + 2 files changed, 68 insertions(+), 14 deletions(-) -- 2.7.4

7 years, 9 months

WALT panic on Hikey960

by Leo Yan

Hi Vikram, Joonwoo, [ + EAS mailing list ] On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance. void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); } static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p); rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p)); } --- Panic Log --- [ 1108.441865] init: Untracked pid 15425 exited with status 0 [ 1108.657107] ------------[ cut here ]------------ [ 1108.661746] kernel BUG at kernel/sched/walt.c:109! [ 1108.666538] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 1108.672026] CPU: 1 PID: 1248 Comm: kschedfreq:0 Not tainted 4.4.78-07635-g0255026 #45 [ 1108.679851] Hardware name: HiKey960 (DT) [ 1108.683770] task: ffffffc0b166c080 ti: ffffffc0b0e64000 task.ti: ffffffc0b0e64000 [ 1108.691261] PC is at walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1108.697179] LR is at dequeue_task_rt+0x40/0x8c [ 1108.701617] pc : [<ffffff8008112428>] lr : [<ffffff800810c82c>] pstate: 60000185 [ 1108.709007] sp : ffffffc0b0e67b90 [ 1108.712315] x29: ffffffc0b0e67b90 x28: 0000000000000001 [ 1108.717633] x27: ffffff8008bc4fc4 x26: ffffffc0bff13400 [ 1108.722948] x25: ffffffc0b166c6c8 x24: 0000000000000000 [ 1108.728263] x23: ffffff8009095000 x22: ffffffc0b166c080 [ 1108.733579] x21: ffffffc0bff13be8 x20: ffffffc0b166c080 [ 1108.738895] x19: ffffffc0bff13400 x18: 0000000000000000 [ 1108.744209] x17: 0000000000000000 x16: 0000000000000000 [ 1108.749524] x15: 0000000000000000 x14: 0000000000000000 [ 1108.754839] x13: 0000000000000000 x12: 0000000034d5d91d [ 1108.760156] x11: ffffff8008be13cc x10: 00000000000009d0 [ 1108.765471] x9 : ffffffc0b0e64000 x8 : ffffffc0b0e67ce0 [ 1108.770786] x7 : ffffffc0ae6cfe30 x6 : ffffff8009095000 [ 1108.776101] x5 : 0000000000000001 x4 : 00000040b6ea8000 [ 1108.781415] x3 : 0000000000000002 x2 : 0000000000000000 [ 1108.786730] x1 : fffffffffffedce2 x0 : 00000000000bc75e [ 1108.792047] [ 1108.792047] SP: 0xffffffc0b0e67b10: [ 1108.797006] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1108.805223] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1108.813440] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1108.821656] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1108.829872] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1108.838090] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1108.846306] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1108.854522] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1108.862741] [ 1108.862741] X1: 0xfffffffffffedc62: [ 1108.867700] dc60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.875924] dc80 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.884140] dca0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.892358] dcc0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.900576] dce0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.908793] dd00 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.917009] dd20 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.925227] dd40 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.933446] dd60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.941666] [ 1108.941666] X7: 0xffffffc0ae6cfdb0: [ 1108.946625] fdb0 0000c350 00000000 00000001 00000000 00000000 00000000 ae6cfeb0 ffffffc0 [ 1108.954840] fdd0 00000000 00000000 00000001 00000000 00000000 00000000 0000c350 00000001 [ 1108.963055] fdf0 ae6cfe90 ffffffc0 0813dd98 ffffff80 b55c0418 0000007f 00000000 00000000 [ 1108.971271] fe10 ffffffff ffffffff b76b299c 0000007f ae6cfe60 ffffffc0 080efae0 00000001 [ 1108.979489] fe30 b0e67ce0 ffffffc0 00000000 00000000 00000000 00000000 0c0b8de9 00000102 [ 1108.987707] fe50 0c0aca99 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 00000825 [ 1108.995923] fe70 08bc80e8 ffffff80 696c616d 6d656d2d 7275702d 00006567 b11f3100 ffffffc0 [ 1109.004140] fe90 00000000 00000000 08085f30 ffffff80 00000000 00000000 b6ee1020 0000007f [ 1109.012359] [ 1109.012359] X8: 0xffffffc0b0e67c60: [ 1109.017318] 7c60 b1092800 ffffffc0 091a8000 ffffff80 00000000 00000000 00009f4c 00000000 [ 1109.025536] 7c80 b0e67ca0 ffffffc0 08bc82a8 ffffff80 b0e67d98 ffffffc0 00000100 00000000 [ 1109.033752] 7ca0 b0e67d40 ffffffc0 08bc8348 ffffff80 b0e67d98 ffffffc0 00000064 00000000 [ 1109.041970] 7cc0 b11bb580 ffffffc0 b0e67d30 ffffffc0 0808e7e4 ffffff80 b166c080 00000001 [ 1109.050186] 7ce0 bff0f2d1 ffffffc0 ae6cfe30 ffffffc0 bff0f170 ffffffc0 09828528 00000102 [ 1109.058405] 7d00 0980fe88 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 000004e0 [ 1109.066621] 7d20 08bc829c ffffff80 6863736b 72666465 303a7165 00000000 b166c080 ffffffc0 [ 1109.074839] 7d40 b0e67d70 ffffffc0 08bc8068 ffffff80 026e40e0 00000000 001a13c8 00000000 [ 1109.083058] [ 1109.083058] X9: 0xffffffc0b0e63f80: [ 1109.088015] 3f80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.096232] 3fa0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.104448] 3fc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.112665] 3fe0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.120883] 4000 00000000 00000000 ffffffff ffffffff b166c080 ffffffc0 00000003 00000001 [ 1109.129100] 4020 57ac6e9d 00000000 32273028 0d0f1c33 10233816 201d111a 3b013e3c 532f2f26 [ 1109.137317] 4040 ae7a3648 ffffffc0 ae354db8 ffffffc0 ae7a36c0 ffffffc0 ae7a36c0 ffffffc0 [ 1109.145535] 4060 00000001 00000000 aca585e0 ffffffc0 aca584e0 ffffffc0 07fb7c71 00000000 [ 1109.153753] [ 1109.153753] X19: 0xffffffc0bff13380: [ 1109.158799] 3380 00000000[ 1109.160774] mali e82c0000.mali: Reset interrupt didn't reach CPU. Check interrupt assignments. [ 1109.169934] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.177103] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.185319] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.193537] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.201755] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.209971] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.218187] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.226402] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.234619] [ 1109.234619] X20: 0xffffffc0b166c000: [ 1109.239664] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.247881] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.256097] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.264313] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.272528] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.280745] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.288960] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.297178] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.305395] [ 1109.305395] X21: 0xffffffc0bff13b68: [ 1109.310440] 3b68 bff13b60 ffffffc0 bff13b70 ffffffc0 bff13b70 ffffffc0 bff13b80 ffffffc0 [ 1109.318658] 3b88 bff13b80 ffffffc0 bff13b90 ffffffc0 bff13b90 ffffffc0 bff13ba0 ffffffc0 [ 1109.326875] 3ba8 bff13ba0 ffffffc0 bff13bb0 ffffffc0 bff13bb0 ffffffc0 00000000 00000064 [ 1109.335092] 3bc8 00000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.343309] 3be8 bff13be8 ffffffc0 bff13be8 ffffffc0 00000000 00000000 00000000 00000000 [ 1109.351526] 3c08 00000000 00000000 0810cb88 ffffff80 00020002 00000000 00000000 00000000 [ 1109.359742] 3c28 006303e4 00000000 389fd980 00000000 f80df80d 00000000 00000000 00000000 [ 1109.367959] 3c48 bff13400 ffffffc0 091c42a0 ffffff80 00000000 00000000 00000000 00000000 [ 1109.376176] [ 1109.376176] X22: 0xffffffc0b166c000: [ 1109.381221] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.389439] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.397656] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.405873] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.414090] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.422307] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.430524] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.438741] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.446959] [ 1109.446959] X25: 0xffffffc0b166c648: [ 1109.452004] c648 b166c648 ffffffc0 b166c648 ffffffc0 b10c8910 ffffffc0 b10c8910 ffffffc0 [ 1109.460220] c668 b0e67ea0 ffffffc0 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.468436] c688 00000073 00000000 00000000 00000000 00000073 00000000 00000000 00000000 [ 1109.476653] c6a8 00000000 00000000 00000000 00000000 00000000 00000000 00009814 00000000 [ 1109.484869] c6c8 00000004 00000000 09ec2856 00000001 09ec2856 00000001 00000000 00000000 [ 1109.493087] c6e8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.501305] c708 b166c708 ffffffc0 b166c708 ffffffc0 b166c718 ffffffc0 b166c718 ffffffc0 [ 1109.509521] c728 b166c728 ffffffc0 b166c728 ffffffc0 00000000 00000000 b0e0b880 ffffffc0 [ 1109.517740] [ 1109.517740] X26: 0xffffffc0bff13380: [ 1109.522785] 3380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.531001] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.539217] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.547434] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.555651] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.563868] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.572085] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.580301] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.588521] [ 1109.588521] X29: 0xffffffc0b0e67b10: [ 1109.593567] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1109.601784] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1109.610001] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1109.618218] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1109.626436] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1109.634653] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1109.642869] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1109.651086] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1109.659303] [ 1109.660788] Process kschedfreq:0 (pid: 1248, stack limit = 0xffffffc0b0e64020) [ 1109.668006] Stack: (0xffffffc0b0e67b90 to 0xffffffc0b0e68000) [ 1109.673748] 7b80: ffffffc0b0e67bc0 ffffff800810c82c [ 1109.681574] 7ba0: ffffffc0bff13400 ffffff800810c820 ffffffc0bff13400 ffffffc0b166c080 [ 1109.689399] 7bc0: ffffffc0b0e67bf0 ffffff80080eead8 ffffffc0bff13400 ffffff800906b000 [ 1109.697225] 7be0: ffffffc0bff13400 ffffff800906b000 ffffffc0b0e67c20 ffffff8008bc4b80 [ 1109.705052] 7c00: ffffffc0bff13400 ffffff8008bc47e4 ffffffc000000001 ffffffc0b166c080 [ 1109.712877] 7c20: ffffffc0b0e67c80 ffffff8008bc4fc4 ffffffc0b0e64000 0000000000000001 [ 1109.720703] 7c40: 00000000000186a0 ffffffc0b0e64000 ffffff8008be0000 00000000001a13c8 [ 1109.728529] 7c60: ffffffc0b1092800 ffffff80091a8000 0000000000000000 0000000000009f4c [ 1109.736355] 7c80: ffffffc0b0e67ca0 ffffff8008bc82a8 ffffffc0b0e67d98 0000000000000100 [ 1109.744181] 7ca0: ffffffc0b0e67d40 ffffff8008bc8348 ffffffc0b0e67d98 0000000000000064 [ 1109.752006] 7cc0: ffffffc0b11bb580 ffffffc0b0e67d30 ffffff800808e7e4 00000001b166c080 [ 1109.759832] 7ce0: ffffffc0bff0f2d1 ffffffc0ae6cfe30 ffffffc0bff0f170 0000010209828528 [ 1109.767658] 7d00: 000001020980fe88 ffffff800813c598 ffffffc0bff0ed40 000004e000000001 [ 1109.775483] 7d20: ffffff8008bc829c 726664656863736b 00000000303a7165 ffffffc0b166c080 [ 1109.783309] 7d40: ffffffc0b0e67d70 ffffff8008bc8068 00000000026e40e0 00000000001a13c8 [ 1109.791135] 7d60: 00000000000186a0 0000000108142a54 ffffffc0b0e67da0 ffffff800811a770 [ 1109.798961] 7d80: 000001020980f324 ffffffc0b0e64000 ffffffc0b11bb580 00000000026e40e0 [ 1109.806787] 7da0: ffffffc0b0e67e20 ffffff80080e1a14 ffffffc0b11bb400 ffffffc0b0e64000 [ 1109.814613] 7dc0: ffffff80091c41c8 ffffffc0b1092800 ffffff800811a6b0 0000000000000000 [ 1109.822438] 7de0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.830263] 7e00: ffffffc0b11bb400 ffffffc0b0e64000 ffffff80091c41c8 ffffffc000000032 [ 1109.838089] 7e20: 0000000000000000 ffffff8008085ed0 ffffff80080e192c ffffffc0b11bb400 [ 1109.845914] 7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080efe18 [ 1109.853740] 7e60: 0000000000000000 0000000000000000 0000000000000000 ffffffc0b1092800 [ 1109.861565] 7e80: ffffffc000000000 ffffff8000000000 ffffffc0b0e67e90 ffffffc0b0e67e90 [ 1109.869392] 7ea0: 0000000000000000 ffffff8000000000 ffffffc0b0e67eb0 ffffffc0b0e67eb0 [ 1109.877217] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.885042] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.892867] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.900692] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.908518] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.916344] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.924169] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.931994] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.939820] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000 [ 1109.947646] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.955470] Call trace: [ 1109.957911] Exception stack(0xffffffc0b0e679c0 to 0xffffffc0b0e67af0) [ 1109.964348] 79c0: ffffffc0bff13400 0000008000000000 ffffffc0b0e67b90 ffffff8008112428 [ 1109.972173] 79e0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.979999] 7a00: 0000000000000000 0000000000000000 0000000000000000 0000000000000009 [ 1109.987824] 7a20: 0000000000000010 0000000000000010 0000000000000000 000000000000068a [ 1109.995650] 7a40: ffffffc0b0e67a90 ffffff8008bc8bac 0000000000000180 ffffff800928c688 [ 1110.003476] 7a60: 00000000000bc75e fffffffffffedce2 0000000000000000 0000000000000002 [ 1110.011301] 7a80: 00000040b6ea8000 0000000000000001 ffffff8009095000 ffffffc0ae6cfe30 [ 1110.019127] 7aa0: ffffffc0b0e67ce0 ffffffc0b0e64000 00000000000009d0 ffffff8008be13cc [ 1110.026952] 7ac0: 0000000034d5d91d 0000000000000000 0000000000000000 0000000000000000 [ 1110.034777] 7ae0: 0000000000000000 0000000000000000 [ 1110.039651] [<ffffff8008112428>] walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1110.046609] [<ffffff800810c82c>] dequeue_task_rt+0x40/0x8c [ 1110.052093] [<ffffff80080eead8>] deactivate_task+0x98/0xbc [ 1110.057580] [<ffffff8008bc4b80>] __schedule+0x44c/0x7c0 [ 1110.062800] [<ffffff8008bc4fc4>] schedule+0x40/0xa0 [ 1110.067674] [<ffffff8008bc82a8>] schedule_hrtimeout_range_clock+0x94/0x100 [ 1110.074544] [<ffffff8008bc8348>] schedule_hrtimeout_range+0x34/0x40 [ 1110.080806] [<ffffff8008bc8068>] usleep_range+0x4c/0x58 [ 1110.086028] [<ffffff800811a770>] cpufreq_sched_thread+0xc0/0x1e4 [ 1110.092032] [<ffffff80080e1a14>] kthread+0xe8/0xfc [ 1110.096821] [<ffffff8008085ed0>] ret_from_fork+0x10/0x40 [ 1110.102129] Code: b7f80081 f9400bf3 a8c37bfd d65f03c0 (d4210000)

7 years, 9 months

[RFC PATCH 0/2] sched: Introduce CPU soft affinity for processes

by Rohit Jain

For multi-tenancy currently there are mechanisms to share the system CPUs by time-sharing (e.g: CFS) and by dividing up the system in 'rigid' containers by using system calls like sched_setaffinity. There is no existing way in the linux kernel today, for flexible workloads where there is a need to give the whole system while still maintaining a notion of preference to CPUs. This patch introduces a new CPU mask, 'cpus_preferred' within the task_struct structure and allows applications a way to specify a set of CPUs which the application would like to run on. The scheduler will try to honor the applications' request the best it can, however if the scheduler finds that there are no idle CPUs within the preferred list, it shall run the application anywhere within the system. This can be used to design soft containers which allows a tenant to use more capacity than he is entitled to when others aren't fully using theirs. The advantage of space sharing the system as opposed to time sharing is that you maintain more cache locality when the soft containers are being utilized. Since this behavior is observed on every scheduling decision, the application gets to run on its preferred CPUs as long as the application does not overuse its specified resources. The design of soft containers still needs more user-space code however, this is what is needed from the kernel. FAQs: Q) What if I set "hard" affinity after I set a preference by using soft affinity? A: Hard affinity will over-ride any previous soft affinity. Q) What if my application had already specified a "hard" affinity? Can I still provide a set of CPUs for soft affinity? A: Yes, it will work as long as the new soft affinity is a subset of the "hard" affinity. Q) Can I have mutually exclusive hard and soft affinities? A: No, soft affinity is always a subset of hard affinity. Note: Ignore the kernel/sched/tick-sched.c change. It is just fixing a build error on Peter's tree. Rohit Jain (2): sched: Introduce new flags to sched_setaffinity to support soft affinity. sched: Actual changes after adding SCHED_SOFT_AFFINITY to make it work with the scheduler arch/x86/entry/syscalls/syscall_64.tbl | 1 + include/linux/init_task.h | 1 + include/linux/sched.h | 4 +- include/linux/syscalls.h | 3 + include/uapi/asm-generic/unistd.h | 4 +- include/uapi/linux/sched.h | 3 + kernel/compat.c | 2 +- kernel/sched/core.c | 167 ++++++++++++++++++++++++++++----- kernel/sched/cpudeadline.c | 4 +- kernel/sched/cpupri.c | 4 +- kernel/sched/fair.c | 116 +++++++++++++++++------ kernel/time/tick-sched.c | 1 + 12 files changed, 250 insertions(+), 60 deletions(-) -- 2.7.4

7 years, 9 months

Energy Model Question

by Zachariah Kennedy

Good day! Been really enjoying watching development of eas on Android Gerrit and in other places so first let me say thanks again for all the great work. With that, I had a couple questions about idle states in the energy model. I am mainly curious about the number of tuples used for the idle states. On the Pixel they used "2 2 0" for the CPU Idle States, I would think that would be for: wfi fpc-def fpc But for Cluster Idle states they used "0 0" but I would think there should be 3 tuples since the Cluster Idle States are: l2-wfi l2-gdhs l2-fpc So why only the two "0 0" for the Pixel EM? Now this leads up to my ultimate question and that is about the SD835 (from the OnePlus5) For CPU Idle States we have: wfi ret pc But we disable idle_enable for "ret". So would this mean that in my own EM for the OP5 I should only have 2 tuples? And for Cluster Idle States we have: l2-wfi l2-dynret l2-ret l2-pc But on the OP5 we disable l2-dynret and l2-ret. So once again, should I only have 2 tuples for the number of idle states used or a tuple for each physical idle state possible? Kind Regards, Zachariah Kennedy

7 years, 9 months

[RFC PATCH v3 0/2] sched: Introduce scaled capacity awareness in enqueue

by Rohit Jain

During OLTP workload runs, threads can end up on CPUs with a lot of softIRQ activity, thus delaying progress. For more reliable and faster runs, if the system can spare it, these threads should be scheduled on CPUs with lower IRQ/RT activity. Currently, the scheduler takes into account the original capacity of CPUs when providing 'hints' for select_idle_sibling code path to return an idle CPU. However, the rest of the select_idle_* code paths remain capacity agnostic. Further, these code paths are only aware of the original capacity and not the capacity stolen by IRQ/RT activity. This patch introduces capacity awarness in scheduler (CAS) which avoids CPUs which might have their capacities reduced (due to IRQ/RT activity) when trying to schedule threads (on the push side) in the system. This awareness has been added into the fair scheduling class. It does so by, using the following algorithm: 1) As in rt_avg the scaled capacities are already calculated. 2) Any CPU which is running below 80% capacity is considered running low on capacity. 3) During idle CPU search if a CPU is found running low on capacity, it is skipped if better CPUs are available. 4) If none of the CPUs are better in terms of idleness and capacity, then the low-capacity CPU is considered to be the best available CPU. The performance numbers*: --------------------------------------------------------------------------- CAS shows upto 1.5% improvement on x86 when running 'SELECT' database workload. I also used barrier.c (open_mp code) as a micro-benchmark. It does a number of iterations and barrier sync at the end of each for loop. I was also running ping on CPU 0 as: 'ping -l 10000 -q -s 10 -f host2' The results below should be read as: * 'Baseline without ping' is how the workload would've behaved if there was no IRQ activity. * Compare 'Baseline with ping' and 'Baseline without ping' to see the effect of ping * Compare 'Baseline with ping' and 'CAS with ping' to see the improvement CAS can give over baseline The program (barrier.c) can be found at: http://www.spinics.net/lists/kernel/msg2506955.html Following are the results for the iterations per second with this micro-benchmark (higher is better), on a 20 core x86 machine: +-------+----------------+----------------+------------------+ |Num. |CAS |Baseline |Baseline without | |Threads|with ping |with ping |ping | +-------+-------+--------+-------+--------+-------+----------+ | |Mean |Std. Dev|Mean |Std. Dev|Mean |Std. Dev | +-------+-------+--------+-------+--------+-------+----------+ |1 | 511.7 | 6.9 | 508.3 | 17.3 | 514.6 | 4.7 | |2 | 486.8 | 16.3 | 463.9 | 17.4 | 510.8 | 3.9 | |4 | 466.1 | 11.7 | 451.4 | 12.5 | 489.3 | 4.1 | |8 | 433.6 | 3.7 | 427.5 | 2.2 | 447.6 | 5.0 | |16 | 391.9 | 7.9 | 385.5 | 16.4 | 396.2 | 0.3 | |32 | 269.3 | 5.3 | 266.0 | 6.6 | 276.8 | 0.2 | +-------+-------+--------+-------+--------+-------+----------+ Following are the runtime(s) with hackbench and ping activity as described above (lower is better), on a 20 core x86 machine: +---------------+------+--------+--------+ |Num. |CAS |Baseline|Baseline| |Tasks |with |with |without | |(groups of 40) |ping |ping |ping | +---------------+------+--------+--------+ | |Mean |Mean |Mean | +---------------+------+--------+--------+ |1 | 0.97 | 0.97 | 0.68 | |2 | 1.36 | 1.36 | 1.30 | |4 | 2.57 | 2.57 | 1.84 | |8 | 3.31 | 3.34 | 2.86 | |16 | 5.63 | 5.71 | 4.61 | |25 | 7.99 | 8.23 | 6.78 | +---------------+------+--------+--------+ *Performance numbers for ARM: --------------------------------------------------------------------------- I was asked to show the efficacy on ARM in v2 review, however I am having some difficulty gathering an ARM machine. Would it be possible for someone to give try this out on ARM? Changelog: --------------------------------------------------------------------------- v1->v2: * Changed the dynamic threshold calculation as the having global state can be avoided. v2->v3: * Split up the patch for find_idlest_cpu and select_idle_sibling code paths. Previous discussion can be found at: --------------------------------------------------------------------------- https://patchwork.kernel.org/patch/9741351/ Rohit Jain (2): sched: Introduce scaled capacity awareness in find_idlest_cpu code path sched: Introduce scaled capacity awareness in select_idle_sibling code path kernel/sched/fair.c | 80 +++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 66 insertions(+), 14 deletions(-) -- 2.7.4

7 years, 10 months

Linux Users List

by alyssa.healy＠forematica.com

<div dir="ltr">Hi, We would like to learn your interest in acquiring our recently updated Linux Users List which helps you to improve your business campaign. We have a verified list of MSPs with complete contact information like Company name, Website, Contact name (First, Middle, Last), Title, Direct email address, Phone number, Postal address, Industry, Employee size, Revenue size, Fax etc. We have other Innovation information also like: Ubuntu, CentOS, Fedora, macOS Sierra, Chromium OS, Oracle Linux, Tizen, and many more. Specialties: Ubuntu, CentOS, Fedora, macOS Sierra, Chromium OS, Oracle Linux, Tizen. Please let me know if this is something of interest to you? I would love to share further details for your review. Best Regards, Alyssa Healy Database Consultant- Global IT Growth If you don’t wish to receive further emails, please reply with Remove. </div>  <a style='display: block; margin: 32px 0 40px 0; padding: 10px; font-size: 1em; text-align: center; border: 0; border-top: 1px solid gray; ' href='https://goo.gl/2ksdRv'>powered by GSM. Free mail merge and email marketing software for Gmail.</a>

7 years, 10 months

← Newer
1
...
26
27
28
29
30
31
32
...
43
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

eas-dev