In lowres mode, hrtimers are serviced by the tick. When a hrtimer is enqueued on a target, that CPU must re-evaluate the next tick to service that hrtimer.
Now while we correctly call wake_up_nohz_cpu() for periodic timers, hrtimers aren't supported that well on dynticks targets.
get_nohz_timer_target() doesn't return a remote idle target but there is a small race window here. The elected target has all the time to turn dynticks idle between the call to get_nohz_timer_target() and the locking of its base. Hence a risk that we enqueue a hrtimer on a dynticks idle destination without kicking it. As a result, the hrtimer might be serviced too late in the future.
Also a target elected by get_nohz_timer_target() can be in full dynticks mode and thus require to be kicked as well. And unlike idle dynticks, this concern both local and remote targets.
To fix this, lets centralize dynticks kick to __hrtimer_start_range_ns(), so that it is well handled for all sort of hrtimer enqueue APIs. It will be performed by an IPI kick on the target, exactly the way it is performed for periodic timers currently.
Signed-off-by: Viresh Kumar viresh.kumar@linaro.org --- kernel/hrtimer.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)
diff --git a/kernel/hrtimer.c b/kernel/hrtimer.c index dd6fb1d..8f0c79c 100644 --- a/kernel/hrtimer.c +++ b/kernel/hrtimer.c @@ -1013,14 +1013,25 @@ int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
leftmost = enqueue_hrtimer(timer, new_base);
- /* - * Only allow reprogramming if the new base is on this CPU. - * (it might still be on another CPU if the timer was pending) - * - * XXX send_remote_softirq() ? - */ - if (leftmost && new_base->cpu_base == &__get_cpu_var(hrtimer_bases) - && hrtimer_enqueue_reprogram(timer, new_base)) { + if (!leftmost) { + unlock_hrtimer_base(timer, &flags); + return 0; + } + + if (!hrtimer_is_hres_active(timer)) { + /* + * Kicked to reevaluate the timer wheel for both idle and full + * dynticks target. + */ + wake_up_nohz_cpu(base->cpu_base->cpu); + } else if (new_base->cpu_base == &__get_cpu_var(hrtimer_bases) && + hrtimer_enqueue_reprogram(timer, new_base)) { + /* + * Only allow reprogramming if the new base is on this CPU. + * (it might still be on another CPU if the timer was pending) + * + * XXX send_remote_softirq() ? + */ if (wakeup) { /* * We need to drop cpu_base->lock to avoid a