CC to kernel mailing list.
What's you expected result for twd?
On 11/14/2013 08:36 AM, Shaojie Sun wrote:
Hi all When I enble NO_HZ_FULL and NO_HZ_FULL_ALL option in K3V2 board. I find that the twd interrupt will be trigged in a large time pre second. In our landing team branch, we used lsk branch.
Do you find this bugs on your board? And how to resolve it?cat /proc/interrupts
CPU0 CPU129: 293 395 GIC twd
On 11/14/2013 10:41 AM, Alex Shi wrote:
CC to kernel mailing list.
What's you expected result for twd?
BTW, the upstream kernel has same results for twd.
On 11/14/2013 08:36 AM, Shaojie Sun wrote:
Hi all When I enble NO_HZ_FULL and NO_HZ_FULL_ALL option in K3V2 board. I find that the twd interrupt will be trigged in a large time pre second. In our landing team branch, we used lsk branch.
Do you find this bugs on your board? And how to resolve it?cat /proc/interrupts
CPU0 CPU129: 293 395 GIC twd
Hi Shaojie,
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
CC'ing Frederic
Regards, Vincent
On 14 November 2013 08:46, Alex Shi alex.shi@linaro.org wrote:
On 11/14/2013 10:41 AM, Alex Shi wrote:
CC to kernel mailing list.
What's you expected result for twd?
BTW, the upstream kernel has same results for twd.
On 11/14/2013 08:36 AM, Shaojie Sun wrote:
Hi all When I enble NO_HZ_FULL and NO_HZ_FULL_ALL option in K3V2 board. I find that the twd interrupt will be trigged in a large time pre second. In our landing team branch, we used lsk branch.
Do you find this bugs on your board? And how to resolve it?cat /proc/interrupts
CPU0 CPU129: 293 395 GIC twd
-- Thanks Alex
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTickless
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTickless
On Thu, Nov 14, 2013 at 05:54:10PM +0800, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
If you select:
NO_HZ_FULL=y NO_HZ_FULL_ALL=n
then you need to pass a nohz_full= cpu range in the boot parameter, otherwise it's simply going to behave like NO_HZ_FULL=n
OTOH, if you select NO_HZ_FULL_ALL=y, the "nohz_full=" parameter is not needed and all CPUs will be full dynticks except CPU 0 where you should see more tick than usual because it's handling the timekeeping for every other CPUs. Don't forget to select CONFIG_NO_HZ_FULL_SYSIDLE=y or CPU 0 will never shutdown its tick even if the entire system is idle.
Do you mean I must select CONFIG_NO_HZ_FULL_SYSIDLE option and merge the patch of "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"?
-----邮件原件----- 发件人: Frederic Weisbecker [mailto:fweisbec@gmail.com] 发送时间: 2013年11月14日 19:50 收件人: Shaojie Sun 抄送: viresh kumar; Vincent Guittot; Alex Shi; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Kevin Hilman; Sunshaojie 主题: Re: a bug on NO_HZ_FULL_ALL
On Thu, Nov 14, 2013 at 05:54:10PM +0800, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
If you select:
NO_HZ_FULL=y NO_HZ_FULL_ALL=n
then you need to pass a nohz_full= cpu range in the boot parameter, otherwise it's simply going to behave like NO_HZ_FULL=n
OTOH, if you select NO_HZ_FULL_ALL=y, the "nohz_full=" parameter is not needed and all CPUs will be full dynticks except CPU 0 where you should see more tick than usual because it's handling the timekeeping for every other CPUs. Don't forget to select CONFIG_NO_HZ_FULL_SYSIDLE=y or CPU 0 will never shutdown its tick even if the entire system is idle.
On Thu, Nov 14, 2013 at 12:39:24PM +0000, Sunshaojie wrote:
Do you mean I must select CONFIG_NO_HZ_FULL_SYSIDLE option and merge the patch of "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"?
Yes you need to select CONFIG_NO_HZ_FULL_SYSIDLE=y. But latest linus tree should be fine as it contains that patch.
Thanks.
-----邮件原件----- 发件人: Frederic Weisbecker [mailto:fweisbec@gmail.com] 发送时间: 2013年11月14日 19:50 收件人: Shaojie Sun 抄送: viresh kumar; Vincent Guittot; Alex Shi; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Kevin Hilman; Sunshaojie 主题: Re: a bug on NO_HZ_FULL_ALL
On Thu, Nov 14, 2013 at 05:54:10PM +0800, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
If you select:
NO_HZ_FULL=y NO_HZ_FULL_ALL=n
then you need to pass a nohz_full= cpu range in the boot parameter, otherwise it's simply going to behave like NO_HZ_FULL=n
OTOH, if you select NO_HZ_FULL_ALL=y, the "nohz_full=" parameter is not needed and all CPUs will be full dynticks except CPU 0 where you should see more tick than usual because it's handling the timekeeping for every other CPUs. Don't forget to select CONFIG_NO_HZ_FULL_SYSIDLE=y or CPU 0 will never shutdown its tick even if the entire system is idle.
BTW, support for ARM's full dynticks is uncomplete without "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"
...which I'm not sure is applied upstream, or even any ARM tree yet.
On Thu, Nov 14, 2013 at 05:54:10PM +0800, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTickless
On Thu, Nov 14, 2013 at 12:53:15PM +0100, Frederic Weisbecker wrote:
BTW, support for ARM's full dynticks is uncomplete without "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"
...which I'm not sure is applied upstream, or even any ARM tree yet.
It's in mainline as of last night, along with a fix to the above patch.
On Thu, Nov 14, 2013 at 12:08:46PM +0000, Russell King - ARM Linux wrote:
On Thu, Nov 14, 2013 at 12:53:15PM +0100, Frederic Weisbecker wrote:
BTW, support for ARM's full dynticks is uncomplete without "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"
...which I'm not sure is applied upstream, or even any ARM tree yet.
It's in mainline as of last night, along with a fix to the above patch.
Great, thanks!
On 11/14/2013 08:08 PM, Russell King - ARM Linux wrote:
On Thu, Nov 14, 2013 at 12:53:15PM +0100, Frederic Weisbecker wrote:
BTW, support for ARM's full dynticks is uncomplete without "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"
...which I'm not sure is applied upstream, or even any ARM tree yet.
It's in mainline as of last night, along with a fix to the above patch.
I saw this in linus tree and in tip/master. According to content, it seems no effort on this issue. The following testing base on latest code.
btw, the full_all do cause more interrupts.
# # Timers subsystem # CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ_COMMON=y # CONFIG_HZ_PERIODIC is not set # CONFIG_NO_HZ_IDLE is not set CONFIG_NO_HZ_FULL=y # CONFIG_NO_HZ_FULL_ALL is not set CONFIG_NO_HZ_FULL_SYSIDLE=y CONFIG_NO_HZ_FULL_SYSIDLE_SMALL=2 CONFIG_NO_HZ=y CONFIG_HIGH_RES_TIMERS=y
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 12567 9783 GIC 29 twd CPU0 CPU1 29: 12814 9942 GIC 29 twd
Then enabled CONFIG_NO_HZ_FULL_ALL. more than 200/second interrupt increased.
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 34969 6384 GIC 29 twd CPU0 CPU1 29: 37172 6646 GIC 29 twd
On Thu, Nov 14, 2013 at 09:08:29PM +0800, Alex Shi wrote:
On 11/14/2013 08:08 PM, Russell King - ARM Linux wrote:
On Thu, Nov 14, 2013 at 12:53:15PM +0100, Frederic Weisbecker wrote:
BTW, support for ARM's full dynticks is uncomplete without "[PATCH] ARM: Support arch_irq_work_raise() via self IPIs"
...which I'm not sure is applied upstream, or even any ARM tree yet.
It's in mainline as of last night, along with a fix to the above patch.
I saw this in linus tree and in tip/master. According to content, it seems no effort on this issue. The following testing base on latest code.
btw, the full_all do cause more interrupts.
# # Timers subsystem # CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ_COMMON=y # CONFIG_HZ_PERIODIC is not set # CONFIG_NO_HZ_IDLE is not set CONFIG_NO_HZ_FULL=y # CONFIG_NO_HZ_FULL_ALL is not set CONFIG_NO_HZ_FULL_SYSIDLE=y CONFIG_NO_HZ_FULL_SYSIDLE_SMALL=2 CONFIG_NO_HZ=y CONFIG_HIGH_RES_TIMERS=y
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 12567 9783 GIC 29 twd CPU0 CPU1 29: 12814 9942 GIC 29 twd
Then enabled CONFIG_NO_HZ_FULL_ALL. more than 200/second interrupt increased.
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 34969 6384 GIC 29 twd CPU0 CPU1 29: 37172 6646 GIC 29 twd
There are various context requirements to let a CPU shut down its tick most of the time: having at most 1 task running, no posix cpu timers, not use perf events based on frequency (which is the default behaviour of perf record), etc...
Try this selftest if you want to see how it behaves it your system:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 12567 9783 GIC 29 twd CPU0 CPU1 29: 12814 9942 GIC 29 twd
Then enabled CONFIG_NO_HZ_FULL_ALL. more than 200/second interrupt increased.
alexs@alex-panda:~$ head -2 /proc/interrupts; sleep 10 ; head -2 /proc/interrupts CPU0 CPU1 29: 34969 6384 GIC 29 twd CPU0 CPU1 29: 37172 6646 GIC 29 twd
There are various context requirements to let a CPU shut down its tick most of the time: having at most 1 task running, no posix cpu timers, not use perf events based on frequency (which is the default behaviour of perf record), etc...
Try this selftest if you want to see how it behaves it your system:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Oh, I know. This patch could get better power consumption when there is a task running.
But how could it do better in idle state instead of preventing CPU0 in deep idle.
On 11/14/2013 05:54 PM, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTickless
Shaojie, which kernel version has this bug? I just find only the v3.13 kernel has opened the NO_HZ_FULL for 32 bit kernel. and my testing base it.
but for lsk, it is 3.10 kernel, so unless you are using 64bit kernel to do testing, is it right?
Hi Alex
I tested it on kernel V3.10 and on 32bits ARM system. Where did you see this option opened only for 64bits?
-----邮件原件----- 发件人: Alex Shi [mailto:alex.shi@linaro.org] 发送时间: 2013年11月19日 11:03 收件人: Shaojie Sun; viresh kumar 抄送: Vincent Guittot; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Frédéric Weisbecker; Kevin Hilman; Sunshaojie 主题: Re: a bug on NO_HZ_FULL_ALL
On 11/14/2013 05:54 PM, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTic kless
Shaojie, which kernel version has this bug? I just find only the v3.13 kernel has opened the NO_HZ_FULL for 32 bit kernel. and my testing base it.
but for lsk, it is 3.10 kernel, so unless you are using 64bit kernel to do testing, is it right?
-- Thanks Alex
On 11/19/2013 11:10 AM, Sunshaojie wrote:
Hi Alex
I tested it on kernel V3.10 and on 32bits ARM system. Where did you see this option opened only for 64bits?
kernel/time/Kconfig:
config NO_HZ_FULL bool "Full dynticks system (tickless)" # NO_HZ_COMMON dependency depends on !ARCH_USES_GETTIMEOFFSET && GENERIC_CLOCKEVENTS # We need at least one periodic CPU for timekeeping depends on SMP # RCU_USER_QS dependency depends on HAVE_CONTEXT_TRACKING # VIRT_CPU_ACCOUNTING_GEN dependency depends on 64BIT <======= select NO_HZ_COMMON select RCU_USER_QS select RCU_NOCB_CPU select VIRT_CPU_ACCOUNTING_GEN select IRQ_WORK
-----邮件原件----- 发件人: Alex Shi [mailto:alex.shi@linaro.org] 发送时间: 2013年11月19日 11:03 收件人: Shaojie Sun; viresh kumar 抄送: Vincent Guittot; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Frédéric Weisbecker; Kevin Hilman; Sunshaojie 主题: Re: a bug on NO_HZ_FULL_ALL
On 11/14/2013 05:54 PM, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTic kless
Shaojie, which kernel version has this bug? I just find only the v3.13 kernel has opened the NO_HZ_FULL for 32 bit kernel. and my testing base it.
but for lsk, it is 3.10 kernel, so unless you are using 64bit kernel to do testing, is it right?
-- Thanks Alex
Hi Alex I found a patch, It could let NO_HZ_FULL running on 32bits. Do you means when I merged this patch. It could work well on kernel v3.10?
-----邮件原件----- 发件人: Alex Shi [mailto:alex.shi@linaro.org] 发送时间: 2013年11月19日 11:15 收件人: Sunshaojie; Shaojie Sun; viresh kumar 抄送: Vincent Guittot; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Frédéric Weisbecker; Kevin Hilman 主题: Re: 答复: a bug on NO_HZ_FULL_ALL
On 11/19/2013 11:10 AM, Sunshaojie wrote:
Hi Alex
I tested it on kernel V3.10 and on 32bits ARM system. Where did you see this option opened only for 64bits?
kernel/time/Kconfig:
config NO_HZ_FULL bool "Full dynticks system (tickless)" # NO_HZ_COMMON dependency depends on !ARCH_USES_GETTIMEOFFSET && GENERIC_CLOCKEVENTS # We need at least one periodic CPU for timekeeping depends on SMP # RCU_USER_QS dependency depends on HAVE_CONTEXT_TRACKING # VIRT_CPU_ACCOUNTING_GEN dependency depends on 64BIT <======= select NO_HZ_COMMON select RCU_USER_QS select RCU_NOCB_CPU select VIRT_CPU_ACCOUNTING_GEN select IRQ_WORK
-----邮件原件----- 发件人: Alex Shi [mailto:alex.shi@linaro.org] 发送时间: 2013年11月19日 11:03 收件人: Shaojie Sun; viresh kumar 抄送: Vincent Guittot; Fathi Boudra; Guodong Xu; Amit Kucheria; Chris Redpath; Linaro Kernel; LAK; Frédéric Weisbecker; Kevin Hilman; Sunshaojie 主题: Re: a bug on NO_HZ_FULL_ALL
On 11/14/2013 05:54 PM, Shaojie Sun wrote:
No, I think it is a bug.
Because I tested the option with NO_HZ_FULL and without NO_HZ_FULL_ALL. It had only little interruptes on CPU0 twd. With same code, I added NO_HZ_FULL_ALL option. It had too many interruptes on CPU0 twd.
So the sumbitter just didn't test twd interrupts, when he expanded NO_HZ_FULL option to all cpu.
On Thu, Nov 14, 2013 at 4:09 PM, viresh kumar viresh.kumar@linaro.org wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTi c kless
Shaojie, which kernel version has this bug? I just find only the v3.13 kernel has opened the NO_HZ_FULL for 32 bit kernel. and my testing base it.
but for lsk, it is 3.10 kernel, so unless you are using 64bit kernel to do testing, is it right?
-- Thanks Alex
-- Thanks Alex
On Thu, Nov 14, 2013 at 01:39:43PM +0530, viresh kumar wrote:
On Thursday 14 November 2013 01:35 PM, Vincent Guittot wrote:
AFAICT, It's a none issue. In full nohz, a timer fires periodically (around 4sec period on ARM IIRC) on one cpu (cpu0).
Timer should always be running on CPU0, its out of nohz-full domain. Its cpu 1, where it will fire after long delays..
https://wiki.linaro.org/WorkingGroups/PowerManagement/Doc/AdaptiveTickless
Right. CPU 0 stays pure periodic unless you select CONFIG_NO_HZ_FULL_SYSIDLE=y then it can be also nohz when the full system is idle. I strongly recommend for power consumption.
linaro-kernel@lists.linaro.org
-
Alex Shi -
Frederic Weisbecker -
Russell King - ARM Linux -
Shaojie Sun -
Sunshaojie -
Vincent Guittot -
viresh kumar