Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
I also have a plugin for new cpu_hotplug trace points but i'm waiting for trace approval before sending it
Vincent
Subject: [PATCH 1/1] cpuidle: new plugin new tracepoints are available for cpuidle
Signed-off-by: Vincent Guittot vincent.guittot@linaro.org --- timechart/plugins/cpuidle.py | 35 +++++++++++++++++++++++++++++++++++ 1 files changed, 35 insertions(+), 0 deletions(-) create mode 100644 timechart/plugins/cpuidle.py
diff --git a/timechart/plugins/cpuidle.py b/timechart/plugins/cpuidle.py new file mode 100644 index 0000000..32dce91 --- /dev/null +++ b/timechart/plugins/cpuidle.py @@ -0,0 +1,35 @@ +from timechart.plugin import * +from timechart import colors +from timechart.model import tcProcess, c_state_table + +class cpu_idle(plugin): + additional_colors = """ +""" + additional_ftrace_parsers = [ + ('cpu_idle', 'state=%d cpu_id=%d', 'state', 'cpuid'), + ] + + additional_process_types = { + } + + @staticmethod + def do_event_cpu_idle(self,event): + self.ensure_cpu_allocated(event.cpuid) + tc = self.tmp_c_states[event.cpuid] + if event.state != 4294967295 : + if len(tc['start_ts'])>len(tc['end_ts']): + tc['end_ts'].append(event.timestamp) + self.missed_power_end +=1 + if self.missed_power_end < 10: + print "warning: missed cpu_idle end" + if self.missed_power_end == 10: + print "warning: missed cpu_idle end: wont warn anymore!" + tc['start_ts'].append(event.timestamp) + tc['types'].append(colors.get_color_id(c_state_table[int(event.state)])) + tc['linenumbers'].append(event.linenumber) + else : + if len(tc['start_ts'])>len(tc['end_ts']): + tc['end_ts'].append(event.timestamp) + +plugin_register(cpu_idle) +
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Pierre
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Pierre
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle. <idle>-0 [000] 270.645935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.036560: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.073395: cpu_idle: state=1 cpu_id=0
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
Regards, Pierre
Hi Pierre,
On Mon, Feb 28, 2011 at 4:09 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle.
That comes from a known problem about the timer on OMAP. The 32K timer is feeding the trace timestamps and so the granularity is not high enough (30us).
<idle>-0 [000] 270.645935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.036560: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.073395: cpu_idle: state=1 cpu_id=0
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt.
Here is an excerpt of it: " The value of '-1' or '4294967295' for state means an exit from the current state, i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id()) means that the system exits the previous idle state.
The event which has 'state=4294967295' in the trace is very important to the user space tools which are using it to detect the end of the current state, and so to correctly draw the states diagrams and to calculate accurate statistics etc. "
I would be glad to help on pytimechart, please let me know!
Regards, Jean
Regards, Pierre
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On 28 February 2011 16:30, Jean Pihet jean.pihet@newoldbits.com wrote:
Hi Pierre,
On Mon, Feb 28, 2011 at 4:09 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle.
That comes from a known problem about the timer on OMAP. The 32K timer is feeding the trace timestamps and so the granularity is not high enough (30us).
<idle>-0 [000] 270.645935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.020935: cpu_idle: state=2 cpu_id=0 <idle>-0 [000] 271.036560: cpu_idle: state=4294967295 cpu_id=0 <idle>-0 [000] 271.073395: cpu_idle: state=1 cpu_id=0
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt.
Here is an excerpt of it: " The value of '-1' or '4294967295' for state means an exit from the current state, i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id()) means that the system exits the previous idle state.
The event which has 'state=4294967295' in the trace is very important to the user space tools which are using it to detect the end of the current state, and so to correctly draw the states diagrams and to calculate accurate statistics etc. "
I would be glad to help on pytimechart, please let me know!
We currently have no consistent state name description across arm platform but I hope we will have soon. May be displaying them as C0, C1 and so on should be a solution until we have a common description of idle state across arm platform.
Regards, Vincent
Regards, Jean
Regards, Pierre
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Mon, Feb 28, 2011 at 4:30 PM, Jean Pihet jean.pihet@newoldbits.com wrote:
Hi Pierre,
On Mon, Feb 28, 2011 at 4:09 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle.
That comes from a known problem about the timer on OMAP. The 32K timer is feeding the trace timestamps and so the granularity is not high enough (30us).
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt. i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id())
Yes, I know this.
This state is depending on the cpuidle state table. on Intel arch, some CStates are reserved and unimplemented. thus the same of the Cstate does not correspond to the id of the cpuidle state. e.g. 0 -> C0 (not an idle state, in IA) 1 -> C1 2 -> C2 3 -> C4 4 -> C6 5 -> S0i1 ( a platform idle state) 6 -> S0i3 ( another deeper platform idle state) This is reflected in the table (I just moved this table in cpuidle.py): c_state_table = ["C0","C1","C2","C4","C6","S0i1","S0i3"]
So.. beware! cpuidle id 3 will currently be displayed as "C4" in pytimechart, which is what *intel_mid* guys want, but probably not what OMAP guys want.
The cpuidle state names are usually available on sysfs. perf and trace-cmd are dumping this in their binary format, but ftrace text interface is not. So this has to be hardcoded or configured.
Regards, Pierre
On Mon, Feb 28, 2011 at 4:49 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 4:30 PM, Jean Pihet jean.pihet@newoldbits.com wrote:
Hi Pierre,
On Mon, Feb 28, 2011 at 4:09 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote:
Hi,
I have started to use the new cpuidle tracepoint and created a plugin for pytimechart. I'm not sure if it's the right way to add this cpuidle trace format update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle.
That comes from a known problem about the timer on OMAP. The 32K timer is feeding the trace timestamps and so the granularity is not high enough (30us).
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt. i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id())
Yes, I know this.
This state is depending on the cpuidle state table. on Intel arch, some CStates are reserved and unimplemented. thus the same of the Cstate does not correspond to the id of the cpuidle state. e.g. 0 -> C0 (not an idle state, in IA) 1 -> C1 2 -> C2 3 -> C4 4 -> C6 5 -> S0i1 ( a platform idle state) 6 -> S0i3 ( another deeper platform idle state) This is reflected in the table (I just moved this table in cpuidle.py): c_state_table = ["C0","C1","C2","C4","C6","S0i1","S0i3"]
So.. beware! cpuidle id 3 will currently be displayed as "C4" in pytimechart, which is what *intel_mid* guys want, but probably not what OMAP guys want.
Ok sorry I misunterstood the point.
The cpuidle state names are usually available on sysfs. perf and trace-cmd are dumping this in their binary format, but ftrace text interface is not. So this has to be hardcoded or configured.
We could add the state name (or abbr) in the ftrace format but that requires some generic kernel changes (again).
Thomas (added in the To: list) attempted to introduce some abbreviated states names but I do not know if this went through to the mainline and if it is applicable to pytimechart. Cf. [1] for the patch.
Thomas, can you help?
[1] http://marc.info/?l=linux-omap&m=129439625501174&w=2
Jean
Regards, Pierre
On Mon, Feb 28, 2011 at 9:36 PM, Jean Pihet jean.pihet@newoldbits.com wrote:
On Mon, Feb 28, 2011 at 4:49 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 4:30 PM, Jean Pihet jean.pihet@newoldbits.com wrote:
Hi Pierre,
On Mon, Feb 28, 2011 at 4:09 PM, Pierre Tardy tardyp@gmail.com wrote:
On Mon, Feb 28, 2011 at 2:43 PM, Vincent Guittot vincent.guittot@linaro.org wrote:
On 27 February 2011 17:31, Pierre Tardy tardyp@gmail.com wrote:
On Fri, Feb 18, 2011 at 11:03 AM, Vincent Guittot vincent.guittot@linaro.org wrote: > Hi, > > I have started to use the new cpuidle tracepoint and created a plugin > for pytimechart. > I'm not sure if it's the right way to add this cpuidle trace format > update but it's the less intrusive one.
Yes, it is okay. I actually plan to also put the whole cpuidle trace handling into such plugin. Would you care to send me one of your trace file, so that I can actually test it.
Yes, I have attached the trace file which is Vishwa's one in fact.
Thanks, I applied and pushed the patch.
Please note that your trace has some suspicious tracepoints with same timestamps for end of cpuidle and start of next cpuidle.
That comes from a known problem about the timer on OMAP. The 32K timer is feeding the trace timestamps and so the granularity is not high enough (30us).
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt. i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id())
Yes, I know this.
This state is depending on the cpuidle state table. on Intel arch, some CStates are reserved and unimplemented. thus the same of the Cstate does not correspond to the id of the cpuidle state. e.g. 0 -> C0 (not an idle state, in IA) 1 -> C1 2 -> C2 3 -> C4 4 -> C6 5 -> S0i1 ( a platform idle state) 6 -> S0i3 ( another deeper platform idle state) This is reflected in the table (I just moved this table in cpuidle.py): c_state_table = ["C0","C1","C2","C4","C6","S0i1","S0i3"]
So.. beware! cpuidle id 3 will currently be displayed as "C4" in pytimechart, which is what *intel_mid* guys want, but probably not what OMAP guys want.
Ok sorry I misunterstood the point.
The cpuidle state names are usually available on sysfs. perf and trace-cmd are dumping this in their binary format, but ftrace text interface is not. So this has to be hardcoded or configured.
We could add the state name (or abbr) in the ftrace format but that requires some generic kernel changes (again).
Thomas (added in the To: list) attempted to introduce some abbreviated states names but I do not know if this went through to the mainline and if it is applicable to pytimechart. Cf. [1] for the patch.
There is another patch for omap from santosh that adds description for each of the C states. https://patchwork.kernel.org/patch/549681/
Vishwa
Thomas, can you help?
[1] http://marc.info/?l=linux-omap&m=129439625501174&w=2
Jean
Regards, Pierre
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Thomas (added in the To: list) attempted to introduce some abbreviated states names but I do not know if this went through to the mainline and if it is applicable to pytimechart. Cf. [1] for the patch.
There is another patch for omap from santosh that adds description for each of the C states. https://patchwork.kernel.org/patch/549681/
+ omap3_power_states[OMAP3_STATE_C1].desc = "MPU ON + CORE ON"; Thanks for the link. Look like this is not what we are looking for. Those description are to be displayed inside the graph. They IMHO have to be less that 5 chars in order to be useable inside pytimechart.
+ .abbr = "C6", Looks more like what we need. Dont know if we are changing the stable tracepoints to display that instead of cpuidle index. ;-)
Pierre
Hi,
sorry for the late reply...
On Monday 28 February 2011 17:52:23 Vishwanath Sripathy wrote:
On Mon, Feb 28, 2011 at 9:36 PM, Jean Pihet jean.pihet@newoldbits.com wrote:
...
Also, cpuidle states name in pytimechart are very selfishly hardcoded with intel's convention. Can you tell what is your convention, so that we can think of a best way to handle display of state's name for ftrace text output?
The power trace points API is described in Documentation/trace/events-power.txt. i.e. trace_cpu_idle(4, smp_processor_id()) means that the system enters the idle state 4, while trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id())
Yes, I know this.
This state is depending on the cpuidle state table. on Intel arch, some CStates are reserved and unimplemented. thus the same of the Cstate does not correspond to the id of the cpuidle state. e.g. 0 -> C0 (not an idle state, in IA) 1 -> C1 2 -> C2 3 -> C4 4 -> C6 5 -> S0i1 ( a platform idle state) 6 -> S0i3 ( another deeper platform idle state) This is reflected in the table (I just moved this table in cpuidle.py): c_state_table = ["C0","C1","C2","C4","C6","S0i1","S0i3"]
So.. beware! cpuidle id 3 will currently be displayed as "C4" in pytimechart, which is what *intel_mid* guys want, but probably not what OMAP guys want.
Ok sorry I misunterstood the point.
The cpuidle state names are usually available on sysfs. perf and trace-cmd are dumping this in their binary format, but ftrace text interface is not. So this has to be hardcoded or configured.
We could add the state name (or abbr) in the ftrace format but that requires some generic kernel changes (again).
Thomas (added in the To: list) attempted to introduce some abbreviated states names but I do not know if this went through to the mainline and if it is applicable to pytimechart. Cf. [1] for the patch.
My patch was not accepted: http://www.spinics.net/lists/linux-omap/msg43800.html http://www.spinics.net/lists/linux-omap/msg43824.html
Instead I sent another one correcting the "not that fortunate" idle state naming in the intel_idle driver: https://patchwork.kernel.org/patch/593701
I also had preferred the .abbr solution. I have another tool reading out c-state names and there I correct the NHM-Cx into Cx-NHM and then I cut off the rest (utils/idle_monitor/cpuidle_sysfs.c in fix_up_intel_idle_driver_name())
And now we are at the reason for my late reply :) This tools should be arch independent and also provide some worthy info on arm. It would be great if someone can give it a quick test. I test compiled it on ppc64, but there the cpuidle framework patches did not make it into the kernel yet and I do not have an arm system.
It's hosted here: git clone git://gitorious.org/cpupowerutils/cpupowerutils.git
git branch --track cpupowerutils origin/cpupowerutils git checkout cpupowerutils make
For some reason I need another make call, so that translation succeeds, but you can ignore it, if you should see a xgettext message.
cpupower frequency-set/frequency-info are the same as cpufreq-set/cpufreq-info which you might know.
New is and there you should see some info on a cpuidle supporting system: cpupower idle-info
cpupower monitor -l cpupower monitor
I hope you come that far...
The tool also implements reading out some registers on X86 to retrieve the actual residency time in C-states from HW. This can easily get added for ARM or other archs as well, but I've heard such registers do not exist on ARM?
Thomas
-
Jean Pihet -
Pierre Tardy -
Thomas Renninger -
Vincent Guittot -
Vishwanath Sripathy