Sched-tools

sched-tools@lists.linaro.org

94 discussions

by Vincent Guittot

Add a tutorial.txt file that describes the grammar used in json file to emulated a use case Signed-off-by: Vincent Guittot <vincent.guittot(a)linaro.org> --- doc/tutorial.txt | 374 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 374 insertions(+) create mode 100644 doc/tutorial.txt diff --git a/doc/tutorial.txt b/doc/tutorial.txt new file mode 100644 index 0000000..3b2fa03 --- /dev/null +++ b/doc/tutorial.txt @@ -0,0 +1,374 @@ +HOW-TO use rt-app and workgen to emulate a workload ? + +**** rt-app and workgen **** +rt-app/workgen is a tool that can be used to emulate a use case. Not only the +sleep and run pattern can be emulated but also the dependency between tasks +like accessing same critical resources, creating sequencial wake up or syncing +the wake up of threads. The use case is described in a json like file which is +parsed by workgen then rt-app. + +workgen is a python script that will parse, check and update the json like file +in order to strictly match the json grammar before calling rt-app that will +execute the sequence described in it. + +Even if the json protocol is used in rt-app, workgen enables some freedom +compared to json grammar: +- You don't have to ensure uniqueness of key in json object, workgen will check +the file and ensure that each key is unique by appending an index if necessary. +The feature is quite useful when you describe a sequence made of same kind of +events like a sequence of multiple sleep and run events. +- Some values can be omitted and workgen will add them when it parses the file +before starting the use case. + +**** json file skeleton **** + +The json file that describes a workload is made on 3 main objects: tasks, +resources and global objects. Only task object is mandatory; default value will +be used if global object is not defined and resources object is kept for +backward compatibility with old version of rt-app but it doesn't give any +benefit to declare it as the resources are now dynamiccally created by rt-app +when it parses the file. + +**** global object **** + +The global object defines parameters for the whole use case: + +* duration : Integer. Duration of the use case in second. All the threads will +be killed once the duration has elapsed. if -1 has been set, the use case will +run indefinitly until all threads kill themselve (as an example if a finite +number of loop has been defined in their running pattern) or if a signal +is received to stop the use case. + +* calibration : Text or Integer: A text defines the CPU that will be used to +calibrate the ns per loop value. "CPU0" is the default value (see run event +section for details about ns per loop value). A integer skips the calibration +step and uses the integer value as ns per loop. + +* default_policy : String. Default scheduling policy of threads. Default +value is SCHED_OTHER. + +* pi_enabled: Boolean. Enable the prority inheritance of mutex. Default value +is False. + +* lock_pages : Boolean. Lock the mem page in RAM. The lock of the page is +only possible for non CFS tasks. Default value is False. + +* logdir : Text. Path to store the various log files. The default path is +the current directory (./). + +* log_basename : Text. Prefix used for all log files of the use case. +"rt-app-" is used by default. + +* ftrace: Boolean. If enable, rt-app logs in ftrace the main events of the use +case. Default value is False. + +* gnuplot : Boolean. if True, it will create a gnu plot compatible file for +each threads (see gnuplot section for more details). Default value is False. + +*** default global object: + "global" : { + "duration" : -1, + "calibration" : "CPU0", + "default_policy" : "SCHED_OTHER", + "pi_enabled" : false, + "lock_pages" : false, + "logdir" : "./", + "log_basename" : "rt-app", + "ftrace" : false, + "gnuplot" : false, + } + +**** tasks object **** + + The tasks object is made of sub-objects that describe threads. No other kind + of object than thread object is allowed. The key value of each object will be + used as thread's name. + +"tasks" : { + "thread_name1" : { + ... + }, + "thread_name2" : { + ... + }, + "thread_name3" : { + ... + }, + ... +} + +*** thread object *** + +Thread object describes the behavior of one kind of thread which means that +several threads can be created with the same object (see instance parameter +below). + +* instance : Integer. Define the number of threads that will be created with +the properties of this thread object. Default value is 1. + +*** policy : String: Define the scheduling policy of the thread. default_policy +is used if not defined + +* priority : Integrer. Define the scheduling priority of the thread. The value +must be aligned with the range allowed by the policy. The default priority is +0 for sched_other class and 10 for other classes + +* runtime : Integer: Define the runtime budget for deadline scheduling class. +Default value is 0. + +* period : Integer. Define the period duration for deadline scheduling class. +Default value is runtime. + +* deadline : Integer. Define the deadline parameter for deadline scheduling +class. Default value is period. + +* cpus: Array of Integer. Define the CPU affinity of the thread. Default +value is all CPUs of the system. An example : "cpus" : [0, 2, 3] + +* phases: Object. The phases object described TBF + +If there is only 1 phase, the sequence of events can be directly put in the +thread object instead of creating a phases object. As an example, the 2 objects +below are equals: + +"task" : { + "thread1" : { + "phases" : { + "phase1" : { + "run" : 10, + "sleep" : 10 + } + } + } +} + +"task" : { + "thread1" : { + "run" : 10, + "sleep" : 10 + } +} + +* loop: Integer. Define the number of times the parent object must be run. +The parent object can be a phase or a thread. For phase object, the loop +defines the number of time the phase will be executed before starting the next +phase. For thread object, the loop defines the number of times that the +complete "phases" object will be executed. The default value is -1. + +*** events *** + +events are simple action that will be performed by the thread or on the +thread. They have to be listed by execution order. + +* run : Integer. Emulate the execution of a load. The duration is defined in +usec but the run event will effectively run a number of time a loop that waste +cpu cycles. Whe the run event is executed, the duration is transformed in a +number of loop thanks to the ns per loop value (see calibration). This way of +working enables to emulate a load with a duration that will vary with the +frequency or the compute capacity of the CPU. + +* sleep : Integer. Emulate the sleep of a task. The duration is defined in +usec. + +* timer : Object. Emulate the wake up of the thread by a timer. Timer differs +from sleep event by the start time of the timer duration. Sleep duration starts +at the beg of the sleep event whereas timer duration starts at the end of the +last use of the timer. So Timer event are immunized against preemption, +frequency scaling and computing capacity of a CPU. The initial starting time of +the timer is set during the 1st use of the latter. + + |<------19---------->|<------19---------->|<------19---------->| +task A ...|run 5|timer 19 |------run 5|timer 19|run 5|timer 19 | +task B |run 10 | + + +As an example to point out the difference betwwen sleep and timer, let consider +a task A that run 5 and then sleep 10. The period of task A should be 15. Let +add a task B that runs 5 and use a timer to wakes up every 19. + + |<------15------>|<------15------>|<------19---------->|<------15------>|<------16------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->| +taskA ...|run 5|sleep 10 |run 5|sleep 10 |----run 5|sleep 10 |run 5|sleep 10 |-run 5|sleep 10 |----run 5|sleep 10 |----run 5|sleep 10 |----run 5|sleep 10 |----run 5|sleep 10 |----run 5|sleep 10 | +taskB ...|------run 5|timer 19|--run 5|timer 19 |run 5|timer 19 |run 5|timer 19 |run 5|timer 19 |run 5|timer 19 |run 5|timer 19 |run 5|timer 19 |run 5|timer 19 | + |<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->| + +We can see that task B period stays to 19 even if the run event is delayed +because of scheduling preemption whereas the period of task A starts at 15 but +increases to 19 because of the scheduling delay. + +"unique" timer device: When a thread that use a timer is instanciated, the +timer will be shared across the thread instance which disturbe the original +sequence. In order to have 1 timer per instance, you can use the "unique" +value so a new timer will be created for each instance. + +* lock : String. Lock the mutex defined by the string value. + +* unlock : String. Unlock the mutex defined by the string value. + +* wait : Object {"ref" : String, "mutex" : String }. Block the calling thread +until another thread sends a wake up signal to the resource defined by "ref". +The mutex defined by "mutex" is used during the block sequence: See +pthread_cond_wait() for more details about the sequence (especially regarding +the use of the mutex). + +* signal : String. Send a wake up signal to the resource defined by the +string. The 1st thread in the wait list will be wokn up. See +pthread_cond_signal() for more details. + +* broad : String. Send a wake up signal to the resource defined by the +string. All threads that are blocked on the resource wil wake up. See +pthread_cond_broadcast() for more details. + +* sync : Object {"ref" : String, "mutex" : String }. Atomically wakes +up a blocked thread and then blocks the calling thread on the condition. +taskA ...|run 5|wait|------------|run 5| wait |---------------- +taskB ...-------------------|sync|-------- + +The sync event "sync" : {"ref" : "CondA", "mutex" : "mutexA" } generates the +following sequence: +{ + "lock" : "mutexA", + "signal" : "CondA", + "wait" : { "ref" : "condA", "mutex" : "mutexA" }, + "unlock" : "mutexA" +} + +* suspend : String. Block the calling thread until another thread wakes it up +with resume. The String can be let empty as it will be filled by workgen with +the right thread's name before starting the use case. + +* resume : String. Wake up the thread defined by the string. + +taskA ...|run 5|suspend |----------------|run 5|suspend |---------------- +taskB ...-------------------|resume taskA|run 10 |-------------------- + + +**** Trace and Log **** + +Some traces and log hooks have been added to ease the debug and monitor various +metrics of the use cases + +*** Trace *** + +A trace can be inserted into ftrace buffer for each event in order to sync +kernel events like sched_switch with the use case's sequence. + +A simple example of ftrace log: + + thread0-23345 [003] 115591.560884: tracing_mark_write: [0] starts + thread0-23345 [003] 115591.560890: tracing_mark_write: [0] begins loop 0 phase 0 step 0 + thread0-23345 [003] 115591.560894: tracing_mark_write: [0] executing 0 + thread0-23345 [003] 115591.580212: tracing_mark_write: [0] executing 1 + thread0-23345 [003] 115591.580217: sched_switch: prev_comm=thread0 prev_pid=23345 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120 + <idle>-0 [003] 115591.670198: sched_switch: prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=thread0 next_pid=23345 next_prio=120 + thread0-23345 [003] 115591.670243: tracing_mark_write: [0] end loop 0 phase 0 step 0 + thread0-23345 [003] 115591.670251: tracing_mark_write: [0] begins loop 0 phase 0 step 1 + thread0-23345 [003] 115591.670254: tracing_mark_write: [0] executing 0 + thread0-23345 [003] 115591.688081: tracing_mark_write: [0] executing 1 + thread0-23345 [003] 115591.688085: sched_switch: prev_comm=thread0 prev_pid=23345 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120 + <idle>-0 [003] 115591.778063: sched_switch: prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=thread0 next_pid=23345 next_prio=120 + thread0-23345 [003] 115591.778108: tracing_mark_write: [0] end loop 0 phase 0 step 1 + thread0-23345 [003] 115591.778116: tracing_mark_write: [0] begins loop 0 phase 0 step 2 + thread0-23345 [003] 115591.778119: tracing_mark_write: [0] executing 0 + thread0-23345 [003] 115591.794619: tracing_mark_write: [0] executing 1 + thread0-23345 [003] 115591.794623: sched_switch: prev_comm=thread0 prev_pid=23345 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120 + <idle>-0 [003] 115591.884607: sched_switch: prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=thread0 next_pid=23345 next_prio=120 + thread0-23345 [003] 115591.884652: tracing_mark_write: [0] end loop 0 phase 0 step 2 + ... + thread0-23345 [003] 115593.394022: tracing_mark_write: [0] begins loop 0 phase 0 step 17 + thread0-23345 [003] 115593.394025: tracing_mark_write: [0] executing 0 + thread0-23345 [003] 115593.410583: tracing_mark_write: [0] executing 1 + thread0-23345 [003] 115593.410594: sched_switch: prev_comm=thread0 prev_pid=23345 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120 + <idle>-0 [003] 115593.500567: sched_switch: prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=thread0 next_pid=23345 next_prio=120 + thread0-23345 [003] 115593.500612: tracing_mark_write: [0] end loop 0 phase 0 step 17 + thread0-23345 [003] 115593.500620: tracing_mark_write: [0] begins loop 0 phase 0 step 18 + thread0-23345 [003] 115593.500623: tracing_mark_write: [0] executing 0 + thread0-23345 [003] 115593.520198: tracing_mark_write: [0] executing 1 + thread0-23345 [003] 115593.520202: sched_switch: prev_comm=thread0 prev_pid=23345 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120 + <idle>-0 [003] 115593.610185: sched_switch: prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=thread0 next_pid=23345 next_prio=120 + thread0-23345 [003] 115593.610230: tracing_mark_write: [0] end loop 0 phase 0 step 18 + thread0-23345 [003] 115593.610258: tracing_mark_write: [0] exiting + +The main events are : + +* "tracing_mark_write: [0] starts" indicates the start of the thread with index 0 + +* "tracing_mark_write: [0] begins loop A phase B loop C" indicates that the +thread with index 0 starts to execute the loop C of the phase B of the main +loop A + +* "tracing_mark_write: [0] executing X": indicates that the thread with index 0 +execute the event X of the current phase +-"tracing_mark_write: [0] end loop A phase B step C" indicates that the +thread with index 0 ends to execute the event C of the phase B of the loop A + +*"tracing_mark_write: [0] exiting": indiscates that the trhead with index 0 has +finished to execute its events. + +*** Log and gnuplot *** + +You can log some metrics for each phase that is executed by a thread. These +metrics are: +- perf: number of loop that has been executed by run events +- run: time spent by the thread to execute the run events +- period: duration to execute the complte phase +- start/end : absolute start and end time of a phase. Same time base is used in +ftrace +- rel_st: start time of a phase relatively to the beg of the use case + +Below is an extract of a log: + +# Policy : SCHED_OTHER priority : 0 +#idx perf run period start end rel_st +0 500000 34382 114492 175604833899 175604948391 785 +0 500000 29786 109863 175604948437 175605058300 115323 +0 500000 26977 107077 175605058336 175605165413 225223 +0 500000 28667 108771 175605165458 175605274229 332344 +0 500000 26644 106745 175605274274 175605381019 441160 +0 500000 26745 106867 175605381061 175605487928 547947 +0 500000 29513 109620 175605487974 175605597595 654860 + +Some gnuplot files are also created to generate charts based on the log files +for each thread and for each kind of metrics. The format of the chart that +will be generated by gnuplot is eps (text art has been used for the chart +below) + + Measured thread0 Loop stats + Load [nb loop] + 120000 ++--+----+---+----+---+---+----+---+----+--++ 506000 load ****** + + + + + + + + + + + + run ###### + |$$ : : : : $$$ : : : : | period $$$$$$ + 110000 ++.$$$........$$$$...$...$..........$$$.$$$$+ + | : $$$$$$$$ $$ $: $ $$$$$$$$ $: $ + | : : : : $ : :$ : : : ++ 504000 + | : : : : : : : : : | + 100000 ++.........................................++ + | : : : : : : : : : | + | : : : : : : : : : | + 90000 ++.........................................++ + | : : : : : : : : : ++ 502000 + | : : : : : : : : : | + 80000 ++.........................................++ + | : : : : : : : : : | + | : : : : : : : : : | + 70000 +******************************************** 500000 + | : : : : : : : : : | + | : : : : : : : : : | + | : : : : : : : : : | + 60000 ++.........................................++ + | : : : : : : : : : | + | : : : : : : : : : ++ 498000 + 50000 ++.........................................++ + | : : : : : : : : : | + | : : : : : : : : : | + 40000 ++.........................................++ + | : : : : : : : : : ++ 496000 + |# : : : : : : : : : | + | # : : : : ### : : : : | + 30000 ++.###.###....####...#...#..........###.####+ + | : # :#### ### : # ######## #: # + + + + + + + +# + + + + + 20000 ++--+----+---+----+---+---+----+---+----+--++ 494000 + 17560556057560556057560656065606756065606756075607 + Loop start time [msec] + + -- 1.9.1

9 years, 9 months

[PATCH] rt-app: fix ftrace log

by Vincent Guittot

replace "locking" by "executing" as we don't have only mutex Signed-off-by: Vincent Guittot <vincent.guittot(a)linaro.org> --- src/rt-app.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rt-app.c b/src/rt-app.c index a867214..97aee0f 100644 --- a/src/rt-app.c +++ b/src/rt-app.c @@ -222,8 +222,8 @@ int run(int ind, event_data_t *events, log_debug("[%d] runs events %d type %d ", ind, i, events[i].type); if (opts.ftrace) log_ftrace(ft_data.marker_fd, - "[%d] locking %d", - ind, events[i].type); + "[%d] executing %d", + ind, i); lock += run_event(&events[i], !continue_running, &perf, duration, resources); } -- 1.9.1

9 years, 9 months

[PATCH] rt-app: fix resume with multi instance

by Vincent Guittot

Use pthread_cond_broadcast instead of pthread_cond_signal in order to wake up all instances of a thread Signed-off-by: Vincent Guittot <vincent.guittot(a)linaro.org> --- src/rt-app.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rt-app.c b/src/rt-app.c index d2945e1..4376313 100644 --- a/src/rt-app.c +++ b/src/rt-app.c @@ -198,7 +198,7 @@ static int run_event(event_data_t *event, int dry_run, { log_debug("resume %s ", rdata->name); pthread_mutex_lock(&(ddata->res.mtx.obj)); - pthread_cond_signal(&(rdata->res.cond.obj)); + pthread_cond_broadcast(&(rdata->res.cond.obj)); pthread_mutex_unlock(&(ddata->res.mtx.obj)); break; } -- 1.9.1

9 years, 9 months

idlestat: topology.c:90: add_topo_info: Assertion `info->physical_id >= 0' failed.

by Zhizhou Zhang (张治洲)

Dear all, I'm using idlestat on a quad-core machine. There is only one cluster. I didn't implement cpu-map node in cpus node of device-tree file. My cluster id is read as -1. It incurred idlestat assert failed. Need we judge this condition in idlestat? Or must we write cpu-map node in our dts file? Thank you. -- Zhizhou

9 years, 9 months

how is the going about workload auto-generator

by Zhizhou Zhang (张治洲)

Hi everyone, For tuning schedule on big-LITTLE architecture, the workload replay helps. I have tried to use perf sched record/replay. But it seems not so exactly. I have heard linaro group is going to implement the workload replay with rt-app project. Could anybody tell me what's the schedule about this feature? Thank you. -- Zhizhou

9 years, 9 months

Comparison mode broken?

by Amit Kucheria

Tuukka, While testing out the various outstanding patches, I found that the comparison mode was broken. Specifically, the following command failed with an assertion in merged_pstates() ./idlestat --import -f changedstate -b mytrace -r comparison -p Data files can be obtained from pastebin: mytrace = http://paste.ubuntu.com/10582474/ changedstate = http://paste.ubuntu.com/10582483/ During a quick investigation, it seemed like merged_pstates() isn't doing what it says in the comments. The following patch seems to fix it and make comparison mode work again. Can you confirm this is the intended behaviour? Minimal testing done. Regards, Amit

9 years, 9 months

[PATCH] rt-app: fix files' name for multi instance thread

by Vincent Guittot

Add the thread's index in the log and plot filename in order to have one set of files per instance. Signed-off-by: Vincent Guittot <vincent.guittot(a)linaro.org> --- src/rt-app.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/rt-app.c b/src/rt-app.c index 68c278d..cc96955 100644 --- a/src/rt-app.c +++ b/src/rt-app.c @@ -557,10 +557,11 @@ int main(int argc, char* argv[]) tdata->lock_pages = opts.lock_pages; if (opts.logdir) { - snprintf(tmp, PATH_LENGTH, "%s/%s-%s.log", + snprintf(tmp, PATH_LENGTH, "%s/%s-%s-%d.log", opts.logdir, opts.logbasename, - tdata->name); + tdata->name, + tdata->ind); tdata->log_handler = fopen(tmp, "w"); if (!tdata->log_handler) { log_error("Cannot open logfile %s", tmp); @@ -641,8 +642,8 @@ int main(int argc, char* argv[]) /* gnuplot of each task */ for (i=0; i<nthreads; i++) { - snprintf(tmp, PATH_LENGTH, "%s/%s-%s.plot", - opts.logdir, opts.logbasename, opts.threads_data[i].name); + snprintf(tmp, PATH_LENGTH, "%s/%s-%s-%d.plot", + opts.logdir, opts.logbasename, opts.threads_data[i].name, opts.threads_data[i].ind ); gnuplot_script = fopen(tmp, "w+"); snprintf(tmp, PATH_LENGTH, "%s-%s.eps", opts.logbasename, opts.threads_data[i].name); -- 1.9.1

9 years, 9 months

[PATCH] rt-app: Add a script to tune parameters in json file.

by pi-cheng.chen

This scripts strips all comments in the input JSON file, modifies the parameters according to commnad line arguments, and write the content to a new JSON file. Signed-off-by: pi-cheng.chen <pi-cheng.chen(a)linaro.org> --- doc/tune_json.py | 130 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100755 doc/tune_json.py diff --git a/doc/tune_json.py b/doc/tune_json.py new file mode 100755 index 0000000..8b633f3 --- /dev/null +++ b/doc/tune_json.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python + +import argparse +import collections +import json +import os +import re +import sys +import tempfile + + +def find_dict_by_key(doc, key): + if key in doc and type(doc[key]) is collections.OrderedDict: + return doc[key] + + for k in doc: + if type(doc[k]) is collections.OrderedDict: + return find_dict_by_key(doc[k], key) + + +def dict_find_and_replace_value(dic, key, val): + for k in dic: + if type(dic[k]) is collections.OrderedDict: + dict_find_and_replace_value(dic[k], key, val) + if k == key: + dic[k] = val + + +def dict_of_loading(dic): + if not 'run' in dic: + return False, None + + for k in dic: + if 'timer' in k and 'period' in dic[k]: + return True, k + else: + return False, None + + +def calculate_and_update_loading(dic, loading): + of_loading, timer_id = dict_of_loading(dic) + + if of_loading: + period = dic[timer_id]['period'] + run = period * loading / 100 + dic['run'] = run + + for k in dic: + if type(dic[k]) is collections.OrderedDict: + calculate_and_update_loading(dic[k], loading) + + +# strip comments in json file and load the file as a dict +def load_json_file(filename): + try: + f = open(filename, 'r') + except: + print 'ERROR: Unable to open %s' %filename + sys.exit(2) + + comment_re = re.compile( + '(^)?[^\S\n]*/(?:\*(.*?)\*/[^\S\n]*|/[^\n]*)($)?', + re.DOTALL | re.MULTILINE) + + content = ''.join(f.readlines()) + f.close() + + match = comment_re.search(content) + while match: + content = content[:match.start()] + content[match.end():] + match = comment_re.search(content) + + return json.JSONDecoder(object_pairs_hook=collections.OrderedDict).decode(content) + + +def dump_json_file(doc, outfile): + tmp = tempfile.NamedTemporaryFile(delete=False) + json.dump(doc, tmp, indent=4, sort_keys=False) + tmp.close() + + os.rename(tmp.name, args.outfile) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + + parser.add_argument('-f', '--file', dest='infile', default='', help='input json filename') + parser.add_argument('-o', '--out', dest='outfile', default='workload.json', help='output json filename'); + parser.add_argument('--instance', default=0, type=int, help='number of thread instance') + parser.add_argument('--period', default=0, type=int, help='period of each thread/phase (ms)') + parser.add_argument('--run', default=0, type=int, help='run time of each thread/phase (ms)') + parser.add_argument('--sleep', default=0, type=int, help='sleep time of each thread/phase (ms)') + parser.add_argument('--loop', default=0,type=int, help='loop count of each thread/phase (-1 as infinite loop)') + parser.add_argument('--loading', default=0, type=int, help='loading of each thread (%%)') + parser.add_argument('--key', type=str, help='the key id of thread/phase in which the parameters will be changed') + + args = parser.parse_args() + + if not os.path.isfile(args.infile): + print 'ERROR: input file %s does not exist\n' %args.infile + parser.print_help() + sys.exit(2) + + doc = target = load_json_file(args.infile) + + if args.key: + target = find_dict_by_key(doc, args.key) + if not target: + print 'ERROR: key id %s is not found' %args.key + sys.exit(2) + + if args.instance > 0: + dict_find_and_replace_value(target, 'instance', args.instance) + + if args.period > 0: + dict_find_and_replace_value(target, 'period', args.period * 1000) + + if args.run > 0: + dict_find_and_replace_value(target, 'run', args.run * 1000) + + if args.sleep > 0: + dict_find_and_replace_value(target, 'sleep', args.sleep * 1000) + + if args.loop > 0 or args.loop == -1: + dict_find_and_replace_value(target, 'loop', args.loop) + + if args.loading > 0: + calculate_and_update_loading(target, args.loading); + + dump_json_file(doc, args.outfile) -- 1.9.1

9 years, 9 months

Re: [Sched-tools] [PATCHv3] add initial manual page

by Amit Kucheria

On Tue, Mar 10, 2015 at 12:58 PM, Koan-Sin Tan <freedom.tan(a)linaro.org> wrote: > There was no manual page. To make users who do not compile > idlestat from source code, it's more convenient to have > man page for idlestat. > > Add install: in Makefile for installing idlestat and idlestat to > /usr/local/bin and /usr/local/man/man1 respectively. > > Signed-off-by: Koan-Sin Tan <freedom.tan(a)linaro.org> > --- > idlestat.1 | 276 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Missed the changes to the makefile when generating the patch? > 1 file changed, 276 insertions(+) > create mode 100644 idlestat.1 > > diff --git a/idlestat.1 b/idlestat.1 > new file mode 100644 > index 0000000..25d2f47 > --- /dev/null > +++ b/idlestat.1 > @@ -0,0 +1,276 @@ > +.TH idlestat 1 "2015-01-10" "0.1" "idlestat utils" > +.SH NAME > +idlestat \- A CPU power-state analysis tool. > +.SH SYNOPSIS > +.\" The general command line > +Trace mode: > +.IP > +.B idlestat > +--trace -f|--trace-file \fIfilename\fR -t|--duration \fIseconds\fR [\fIOPTION\fR] [command] > +.P > +Reporting mode: > +.IP > +.B idlestat > +--import -f|--trace-file \fIfilename\fR [\fIOPTION\fR] > +.SH DESCRIPTION > +\fBIdlestat\fR comes with two modes: in \fBtrace mode\fR, it measures how long the CPUs have been in the different idle and operating states, analyzes captured events, logs them, and generates a report; in \fBreporting mode\fR, it reads the trace file, analyzes logged events in the trace file, and generates a report. A report by idlestat shows statistics of power related states. Currently, it handles P-states, C-states, and IRQ states. > + > +For trace mode, \fBidlestat\fR relies on the kernel's FTRACE function to monitor and capture C-state and P-state transitions of CPUs over a time interval. That is, for trace mode, idlestat needs a kernel with FTRACE related configurations enabled. And since it uses FTRACE, root privilege is needed when running in trace mode. Idlestat extracts the following information from trace file: > +.IP "" 2 > +Times when CPUs entered and exited a certain C-state > +.IP "" 2 > +Times when CPUs entered and exited a certain P-state > +.IP "" 2 > +Raised IRQs > +.LP > +Following a successful run, idlestat calculates and reports the following > +information: > +.IP - 2 > +Total, average, minimum, and maximum time spent in each C-state, per-CPU. > +.IP - 2 > +Total, average, minimum, and maximum time spent in each P-state, per-CPU. > +.IP - 2 > +Total, average, minimum, and maximum time during which all CPUs in a cluster > +were in a "shallowest" (closest to running) state of all the constituent > +CPUs. > +.IP - 2 > +Number of times a certain IRQ caused a CPU to exit idle state, per-CPU and per-IRQ > + > +.SH OPTIONS > +.TP > +\fB\-\-trace\fR > +Run idlestat in trace mode, as noted above, root privilege is needed. Used with \fB\-t\fR and \fB\-f\fR to specify running duration and trace output filename, respectively. > + > +.TP > +\fB\-\-import\fR > +Run idlestat in reporting mode. Used with \fB\-f\fR to specify the trace file to import. > + > +.TP > +\fB\-f\fR, \fB\-\-trace-file\fR \fIfilename\fR > +Specify the trace filename to generate (for \fB\-\-trace\fR) or read (for \fB\-\-import\fR). > + > +.TP > +\fB\-t\fR, \fB\-\-duration\fR \fIseconds\fR > +Time to capture in seconds > + > +.TP > +\fB[command]\fR > +If the command argument is given, idlestat will fork a child to run the command so that you can capture related states when the command is running. Note that if the running command does not terminate before the time specified by \fB\-t\fR, idlestat will send a SIGTERM to the child process to ask it to terminate. If the command to run lasts less than the time specified by \fB\-t\fR, idlestat will terminate after the command finished. > + > +When there is no command, idlestat does trace capture and other works dutifully. > + > +.TP > +\fB\-b\fR, \fB\-\-baseline_trace\fR \fIbaseline_filename\fR > +Specify baseline filename for trace comparison. See \fBCOMPARISON\fR and \fBEXAMPLES\fR for more information. > + > +.TP > +\fB\-c\fR, \fB\-\-idle\fR > +Show idle, C-state in ACPI term, statistics. This is the default mode if neither \fB\-w\fR nor \fB\-p\fR is set. Note that \fB\-w\fR, \fB\-c\fR, and \fB\-p\fR are not exclusive, that is, you can choose any combination of the three. > + > +.TP > +\fB\-e\fR, \fB\-\-energy-model-file\fR \fIenergy_model\fR > +Specify energy model file. See \fBENERGY MODEL\fR > +for the description of the energy file format. > + > +.TP > +\fB\-h\fR, \fB\-\-help\fR, \fB\-?\fR > +Print simple help messages. > + > +.TP > +\fB\-o\fR, \fB\-\-output-file\fR \fIfilename\fR > +Specify the file name to output statistics. If not specified, statistics > +are printed to stdout. > + > +.TP > +\fB\-p\fR, \fB\-\-frequency\fR > +Show cpufreq (or say P-state) statistics. > + > +.TP > +\fB\-r\fR, \fB\-\-report-format\fR > +Specify the report format. Valid types are: default, boxless, csv, and > +comparison. > + > +.TP > +\fB\-v\fR, \fB\-\-verbose\fR > +Specify the verbosity level. This option could be specified multiple times. Each copy of the option increases verbosity level by one, currently the highest verbosity level used for any message is 2. > + > +.TP > +\fB\-w\fR, \fB\-\-wakeup\fR > +Show wakeup statistics. > + > +.TP > +\fB\-B\fR, \fB\-\-boxless\fR > +Set the report format to boxless > + > +.TP > +\fB\-C\fR, \fB\-\-csv\fR > +Set the report format to comma separated values (CSV) > + > +.TP > +\fB\-I, \fB\-\-poll\-interval\fR > +Set kernel polling interval, which is used to determine if it’s time move data from kernel FTRACE buffer to other places. > + > +.TP > +\fB\-S, \fB\-\-buffer\-size\fR > +Set the kernel FTRACE buffer size to use. > + > +.TP > +\fB\-V\fR, \fB\-\-version\fR > +Show idlestat version information and exit. > + > +.SH COMPARISON > +The comparison report is used to compare changes between the active trace (specified by \fB\-f\fR,\fB\-\-trace\-file\fR) and the baseline trace. It becomes active by providing a baseline trace (\fB-b\fR,\fB\-\-baseline_trace\fR) and specifying the comparison report style with "\fB\-r\fR comparison". > + > +.SH ENERGY MODEL > +The following describes the format of idlestat energy model files > +(using the included energy_model file as an example): > + > +Lines starting with # or which are blank are ignored. > + > +First, specify how many clusters there are. > + > +.nf > +clusters 1 > +.fi > + > +For each cluster (named clusterA, clusterB, etc.) specify > +how many cap states and C states there are. Idlestat will check > +that the correct number of clusters are provided. > + > +.nf > +# > +clusterA: 16 cap states 5 C states > +.fi > + > +Then specify the name, cluster power and core power for each P-state > +(idlestat will check if there are the correct number of P-states provided) > +in the cluster. > +.nf > + > +P-states: > +# speed, cluster power, core power > +2901 6200 3200 > +2900 6190 3190 > +2800 6180 3180 > +2700 6170 3170 > +2500 6160 3160 > +2400 6150 3150 > +2300 6140 3140 > +2200 6130 3130 > +2000 6120 3120 > +1900 6110 3110 > +1800 6100 3100 > +1700 6090 3090 > +1600 6080 3080 > +1400 6070 3070 > +1300 6060 3060 > +1200 6050 3050 > + > +.fi > +Then specify the name, cluster power and core power for each C-state > +(idlestat will check if there are the correct number of C-states provided) > +in the cluster. > + > +.nf > + > +C-states: > +C1-IVB 25 0 > +C1E-IVB 30 0 > +C3-IVB 35 0 > +C6-IVB 40 0 > +C7-IVB 35 0 > + > +.fi > + > +Finally, specify the wakeup power. > + > +wakeup 210 6 > + > +Repeat for each cluster. > + > +.SH TRACE FILE FORMAT > + > +Idlestat has its own trace file format, which is based on ftrace's format (see Documentation/trace/ftrace.txt in kernel source). Besides standard FTRACE entries, idlestat adds CPU topology, C-state information, and some artificial P-State entries. Idlestat can also import standard FTRACE format and "trace-cmd report" format. Note that since there is no CPU topology and C-state information in FTRACE or trace-cmd trace files, they should be used on the machines those traces are captured. > + > +.SH REPORT FORMATS > +Currently, idlestat supports four report formats: default, boxless, csv, and comparison. > +.IP 1. 4 > +default: show C-state, P-State, and IRQ state statistics tables if corresponding -c, -p, and -w command line options are set. Tables come with - and | as boundaries. > +.IP 2. 4 > +boxless: it's the default without - and | as table boundaries. This is more Braille terminal > +friendly than the default one > +.IP 3. 4 > +csv: comma separated values. This is easier to process with scripting languages, such as > +awk and perl. Also it's easier to be processed with spreadsheet software. > +.IP 4. 4 > +comparison: This is similar to default report style, except that it displays changes > +in active trace compared to the baseline trace. It becomes active by providing a > +baseline trace (--baseline) and specifying the comparison report style with > +"-r comparison". > +.SH EXAMPLES > +Assuming the idlestat binary is in your PATH. > +.IP 1. 4 > +Run a trace, post-process the results (default is to show only C-state statistics): > +.RS 8 > +sudo idlestat --trace -f /tmp/mytrace -t 10 > +.RE > +.IP 2. 4 > +Run a trace, post-process the results and print all statistics: > +.RS 8 > +sudo idlestat --trace -f /tmp/mytrace -t 10 -p -c -w > +.RE > +.IP 3. 4 > +Run a trace with an external workload, post-process the results: > +.RS 8 > +sudo idlestat --trace -f /tmp/mytrace -t 10 -p -c -w -- rt-app /tmp/mp3.json > +.RE > +.IP 4. 4 > +Post-process a trace captured earlier: > +.RS 8 > +idlestat --import -f /tmp/mytrace > +.RE > +.IP 5. 4 > +Run a trace, post-process the results and print all statistics into a file: > +.RS 8 > +sudo idlestat --trace -f /tmp/mytrace -t 10 -p -c -w -o /tmp/myreport > +.RE > +.IP 6. 4 > +Run a comparison trace, say, before and after making changes to system behavior > +.RS 8 > +sudo idlestat --trace -f /tmp/baseline -t 10 > +.br > +sudo idlestat --trace -f /tmp/changedstate -t 10 > +.br > +idlestat --import -f /tmp/changedstate -b /tmp/baseline -r comparison > +.RE > +.SH LIMITATIONS > +During the acquisition, idlestat tries to stay quiescent to prevent disturbing the traces. For this reason the traces are buffered in a fixed buffer size. If the duration of the acquisition produces more traces than what the buffer is capable to store, that will result in a truncated result. > +.SH AUTHOR > +Started by > +.MT daniel.lezcano(a)linaro.org > +Daniel Lezcano > +.ME > +with contributions from others. > +See the git log of the source code (https://git.linaro.org/power/idlestat.git) for all the contributors. > +.SH REPORTING BUGS > +Send mail to > +.MT sched-tools(a)linaro.org > +.ME . > +Or, submit bugs at > +.UR https://bugs.linaro.org/enter_bug.cgi > +.UE > +under Product "power management", Component "idlestat". > + > +.SH COPYRIGHT > +Copyright \(co 2015, Linaro Limited. > +License GPLv2: GNU GPL version 2 > +.UR http://gnu.org/licenses/gpl.html > +.UE . > + > +This is free software: you are free to change and redistribute it. > +There is NO WARRANTY, to the extent permitted by law. > +Or, say, there is NO warranty; not even for MERCHANTABILITY > +or FITNESS FOR A PARTICULAR PURPOSE. > + > +.SH SEE ALSO > +trace-cmd(1), trace-cmd-format(1), trace-cmd.dat(5) > -- > 1.9.1 >

9 years, 9 months

Re: [Sched-tools] [PATCH 1/4] Topology: Match cpu names only if they begin with "cpu"

by Amit Kucheria

On Wed, Jan 28, 2015 at 7:58 PM, Tuukka Tikkanen <idlestat(a)tic0.net> wrote: > Previously any directory entry in /sys/devices/system/cpu/ with > the substring "cpu" within the name would be considered to be > a subdirectory for a cpu. The substring should begin the name, > not just appear within the name. OK. Will apply. But what directories do you see other than cpuX, cpuidle and cpufreq under /sys/devices/system/cpu/ ? cpuidle and cpufreq are already filtered in the previous line. I'm curious what else are you guarding against? > Signed-off-by: Tuukka Tikkanen <idlestat(a)tic0.net> > --- > topology.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/topology.c b/topology.c > index ff0b6fb..c7528af 100644 > --- a/topology.c > +++ b/topology.c > @@ -333,7 +333,7 @@ static struct cpu_topology *topo_folder_scan(char *path, folder_filter_t filter) > if (filter && filter(direntp->d_name)) > continue; > > - if (!strstr(direntp->d_name, "cpu")) > + if (strncmp(direntp->d_name, "cpu", 3)) > continue; > > ret = asprintf(&newpath, "%s/%s/%s", basedir, > -- > 1.7.9.5 >

9 years, 9 months

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Sched-tools