We already have ONESHOT_STOPPED mode available now and can use that to disable spurious interrupts hurting our tickless CPU.
Thomas suggested not to do this 'behind the scene' and make sure all of the core code is aware of it.
So here are the scenarios I could figure out (relevant for both NO_HZ_IDLE and NO_HZ_FULL cases):
1: Mode: NOHZ_MODE_LOWRES
We don't reprogram clock event device from tick_nohz_stop_sched_tick() when 'expires == KTIME_MAX'. But tick_nohz_handler() have already reprogrammed it for next tick (and will continue doing that for all spurious ticks). So, we must stop clock-event device from here.
Once tickless, we restart tick as soon as request for any timer comes. We need not stop clockevent device again when that timer expires as tick_nohz_stop_sched_tick() will be called again.
What if hrtimer is added on a tickless system? It looks we aren't handling that right now and should be fixed separately.
2. Mode: NOHZ_MODE_HIGHRES
We cancel tick-sched hrtimer from tick_nohz_stop_sched_tick() when 'expires == KTIME_MAX'. But tick_sched_timer() has already reprogrammed clockevent device for next tick and so must be stopped. hrtimer_cancel() would finally call hrtimer_force_reprogram() and we reevaluate need to reprogram clockevent device there. Currently if 'expires == KTIME_MAX', we skip reprogramming clockevent device. At this point, we are guaranteed that no timers/hrtimers are pending and so stopping clockevent device from here shouldn't hurt.
Also if a hrtimer is added and then removed on a tickless CPU, we are guaranteed to stop clockevent device from here.
Now left the case where a hrtimer is added on a tickless CPU and it expires.
From tick_nohz_irq_exit() we will call tick_nohz_stop_sched_tick() which must
reevaluate pending timers to check if tick must be disabled again. We will call hrtimer_cancel() when we decide to go tickless again, BUT the tick-sched hrtimer was never re-queued on a tickless CPU and so we wouldn't reach hrtimer_force_reprogram().
To get this fixed, some special handling is also added to hrtimer_interrupt().
Signed-off-by: Viresh Kumar viresh.kumar@linaro.org --- kernel/hrtimer.c | 47 +++++++++++++++++++++++++++++++++++++++++++---- kernel/time/tick-sched.c | 2 ++ 2 files changed, 45 insertions(+), 4 deletions(-)
diff --git a/kernel/hrtimer.c b/kernel/hrtimer.c index 3c0bff2..519c229 100644 --- a/kernel/hrtimer.c +++ b/kernel/hrtimer.c @@ -591,8 +591,22 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal) if (cpu_base->hang_detected) return;
- if (cpu_base->expires_next.tv64 != KTIME_MAX) + if (cpu_base->expires_next.tv64 != KTIME_MAX) { tick_program_event(cpu_base->expires_next, 1); + } else { + /* + * Don't need clockevent device anymore, stop it. + * + * We reach here only for NOHZ_MODE_HIGHRES mode and we are + * guaranteed that no timers/hrtimers are enqueued on this cpu. + * + * We will reach here: + * - when tick-sched hrtimer is cancelled to enter into tickless + * or CPU idle state + * - or a hrtimer is cancelled on a tickless cpu + */ + tick_stop_event(); + } }
/* @@ -1371,9 +1385,34 @@ retry: cpu_base->expires_next = expires_next; raw_spin_unlock(&cpu_base->lock);
- /* Reprogramming necessary ? */ - if (expires_next.tv64 == KTIME_MAX || - !tick_program_event(expires_next, 0)) { + if (expires_next.tv64 == KTIME_MAX) { + cpu_base->hang_detected = 0; + + /* + * Don't need clockevent device anymore, stop it. + * + * We reach here only for NOHZ_MODE_HIGHRES and we are + * guaranteed to not have any pending timers/hrtimers on this + * cpu. Most of the scenarios will be covered by similar code + * present in hrtimer_force_reprogram(), as we always try to + * evaluate tick requirement on idle/irq exit and cancel + * tick-sched hrtimer when tick isn't required anymore. + * + * It is required here as well for the extreme corner case. + * + * Suppose last hrtimer fires on a tickless CPU, we reach + * tick_nohz_irq_exit(). While reevaluating we will get expires + * == KTIME_MAX and try to cancel tick-sched hrtimer. BUT, the + * tick-sched hrtimer was never queued on a tickless CPU and so + * we wouldn't reach hrtimer_force_reprogram(). + * + * So, disable Clockevent device here as well. + */ + tick_stop_event(); + return; + } + + if (!tick_program_event(expires_next, 0)) { cpu_base->hang_detected = 0; return; } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 2c9a145..93f6f2e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -652,6 +652,8 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts, if (unlikely(expires.tv64 == KTIME_MAX)) { if (ts->nohz_mode == NOHZ_MODE_HIGHRES) hrtimer_cancel(&ts->sched_timer); + else + tick_stop_event(); goto out; }