The cpufreq (dvfs) test script attempts to update the number of loops to run the workload by searching for the pattern ^loops (searching just for loops won't work as that appears in multiple places). But the dvfs.json file currently has the desired instance of "loops" indented. Remove the indentations so the dvfs.sh script works as expected.

Signed-off-by: Steve Muckle smuckle@linaro.org --- doc/examples/cpufreq_governor_efficiency/dvfs.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/examples/cpufreq_governor_efficiency/dvfs.json b/doc/examples/cpufreq_governor_efficiency/dvfs.json index c1eefe4b978e..b7a6dcd8d2b8 100755 --- a/doc/examples/cpufreq_governor_efficiency/dvfs.json +++ b/doc/examples/cpufreq_governor_efficiency/dvfs.json @@ -4,7 +4,7 @@ "instance" : 1, "policy" : "SCHED_FIFO", "cpus" : [1], - "loop" : 10, +"loop" : 10, "phases" : { "sleeping" : { "loop" : 1,

Apply various clarifications and fixups to the README for the cpufreq governor efficiency test.

Signed-off-by: Steve Muckle smuckle@linaro.org --- doc/examples/cpufreq_governor_efficiency/README | 48 ++++++++++++------------- 1 file changed, 23 insertions(+), 25 deletions(-)

diff --git a/doc/examples/cpufreq_governor_efficiency/README b/doc/examples/cpufreq_governor_efficiency/README index 10482b85f529..0d9123daa1cd 100755 --- a/doc/examples/cpufreq_governor_efficiency/README +++ b/doc/examples/cpufreq_governor_efficiency/README @@ -1,23 +1,22 @@ Measure the efficiency of cpufreq governors using rt-app

BACKGROUND: - DVFS adds a latency in the execution of task because of the time to - decide to move at max freq. We need to measure this latency and check - that the governor stays in an acceptable range. + DVFS adds a latency in the execution of a task because some or all of the + task may not be executed at maximum frequency. We need to measure this + latency and check that the governor stays in an acceptable range.

- When rt-app runs a json file, a log file is created for each thread. - This log file records the number of loop that has been executed and + When rt-app runs a json file a log file is created for each thread. + This log file records the number of loops that have been executed and the duration for executing these loops (per phase). We can use these - figures to evaluate to latency that is added by a cpufreq governor + figures to evaluate the latency that is added by a cpufreq governor and its "performance efficiency".

- We use the run+sleep pattern to do the measurement, for the run time per - loop, the performance governor should run the expected duration as the - CPU stays a max freq. At the opposite, the powersave governor will give - use the longest duration (as it stays at lowest OPP). Other governor will - be somewhere between the 2 previous duration as they will use several OPP - and will go back to max frequency after a defined duration which depends - on its monitoring period. + We use the run+sleep pattern to do the measurement. For the run time per + loop the performance governor should run the expected duration as the CPU + stays at max freq. At the opposite, the powersave governor will give us the + longest duration (as it stays at lowest OPP). Other governors will be + somewhere between the 2 previous durations as they may use any number of + OPPs for various portions of the workload.

The formula:

@@ -34,20 +33,22 @@ Usage: ./calibration.sh <cpu> cpu: cpu number on which you want to run the test

- ./test.sh <governor> <cpu> <runtime> <sleeptime> [<loops>] + ./test.sh <governor> <cpu> <runtime> <period> [<loops>] governor: target CPUFreq governor you want to test - cpu: cpu number on which you want to run the test. Be the same - as the one passing to "calibration.sh". - runtime: running time in ms per loop of the workload pattern - sleeptime: sleeping time in ms per loop of the workload pattern + cpu: cpu number on which you want to run the test. It should + be the same as the one passed to "calibration.sh". + runtime: running time in ms per loop of the workload pattern, + at fmax + period: period in ms of the loop of the workload pattern loops: repeat times of the workload pattern. default: 10

Example: - "./calibration.sh 0" means to calculate the computing capacity of CPU0 which - will be used in the following test. + "./calibration.sh 0" means to calculate the computing capacity of CPU0 + which will be used in the following test.

- "./test.sh ondemand 0 100 100 20" means to - test "ondemand" on CPU0 with workload pattern "run 100ms + sleep 100ms"(20 loops). + "./test.sh ondemand 0 100 200 20" means to test "ondemand" on CPU0 with a + periodic workload where the busy portion is 100ms of work (if it were to be + run at fmax), the period is 200ms, and the workload runs for 20 cycles.

NOTE: - Make sure there are "sed", "cut", "grep", "rt-app", etc tools on @@ -55,6 +56,3 @@ NOTE:

- Run the test while the system is idle.

- - You can change the target governor's parameters after running "calibration.sh", - but before "test.sh". -

Add support for testing multiple governors and workloads. Display output in a table format, suitable for email etc., as tests are completed. Add support for frequency residency capture and output this as well.

Signed-off-by: Steve Muckle smuckle@linaro.org --- doc/examples/cpufreq_governor_efficiency/README | 37 ++++ doc/examples/cpufreq_governor_efficiency/dvfs.sh | 8 + doc/examples/cpufreq_governor_efficiency/test.sh | 246 +++++++++++++++-------- 3 files changed, 209 insertions(+), 82 deletions(-)

diff --git a/doc/examples/cpufreq_governor_efficiency/README b/doc/examples/cpufreq_governor_efficiency/README index 0d9123daa1cd..19059b7282eb 100755 --- a/doc/examples/cpufreq_governor_efficiency/README +++ b/doc/examples/cpufreq_governor_efficiency/README @@ -27,12 +27,23 @@ BACKGROUND: will give the efficiency of the governor. 100% means as efficient as the perf governor and 0% means as efficient as the powersave governor.

+ In addition to measuring this efficiency figure it is also useful to know + the number of times the busy part of the loop exceeded the intended loop + period. These are termed "overruns." + This test offers json files and shell scripts to do the measurement.

Usage: + Before running the tests it is necessary to calibrate the test for your + particular target. This is done with the calibration.sh script: + ./calibration.sh <cpu> cpu: cpu number on which you want to run the test

+ The test script has two modes of operation. Test parameters can be passed + on the command line in order to test a single governor with a single + workload specification: + ./test.sh <governor> <cpu> <runtime> <period> [<loops>] governor: target CPUFreq governor you want to test cpu: cpu number on which you want to run the test. It should @@ -42,6 +53,19 @@ Usage: period: period in ms of the loop of the workload pattern loops: repeat times of the workload pattern. default: 10

+ Alternately, a test file can be supplied to test.sh: + + ./test.sh -f <test_file> + test_file: text file indicating the governors and workloads to + test + + The test file format is as follows: + first line: number of CPU to run the tests on, similar to the command + line "cpu" parameter above + second line: a list of governors to test separated by spaces + other lines: workloads to run in the format + <busy ms @fmax> <period length in ms> <num loops> + Example: "./calibration.sh 0" means to calculate the computing capacity of CPU0 which will be used in the following test. @@ -50,6 +74,19 @@ Example: periodic workload where the busy portion is 100ms of work (if it were to be run at fmax), the period is 200ms, and the workload runs for 20 cycles.

+ "./test.sh -f mytests.txt" would run the tests listed in mytests.txt. If + mytests.txt contained the following: + + 1 + ondemand interactive + 1 100 100 + 10 1000 10 + + The tests would be run on CPU 1, the ondemand and interactive governors + would each be tested (in addition to performance and powersave), and the + two workloads would be 1ms busy with 100ms loop period for 100 loops, + and 10ms busy with 1000ms loop period for 10 loops. + NOTE: - Make sure there are "sed", "cut", "grep", "rt-app", etc tools on your test machine, and run the scripts under root privilege. diff --git a/doc/examples/cpufreq_governor_efficiency/dvfs.sh b/doc/examples/cpufreq_governor_efficiency/dvfs.sh index a9173a41f602..1deded1c0e3f 100755 --- a/doc/examples/cpufreq_governor_efficiency/dvfs.sh +++ b/doc/examples/cpufreq_governor_efficiency/dvfs.sh @@ -24,8 +24,16 @@ sync

sleep 1

+if [ -e /sys/devices/system/cpu/cpu$2/cpufreq/stats/time_in_state ]; then + cat /sys/devices/system/cpu/cpu$2/cpufreq/stats/time_in_state > "$1-cpu$2-$3-$4-$5-pre-residency.txt" +fi + rt-app dvfs.json 2> /dev/null

+if [ -e /sys/devices/system/cpu/cpu$2/cpufreq/stats/time_in_state ]; then + cat /sys/devices/system/cpu/cpu$2/cpufreq/stats/time_in_state > "$1-cpu$2-$3-$4-$5-post-residency.txt" +fi + echo $savedgov > /sys/devices/system/cpu/cpu$2/cpufreq/scaling_governor

if [ $1 ] ; then diff --git a/doc/examples/cpufreq_governor_efficiency/test.sh b/doc/examples/cpufreq_governor_efficiency/test.sh index 4d8cb9c21855..7fdb90650db6 100755 --- a/doc/examples/cpufreq_governor_efficiency/test.sh +++ b/doc/examples/cpufreq_governor_efficiency/test.sh @@ -1,104 +1,186 @@ #!/bin/sh

-set -e +parse_testfile=0 +cmdline_test_parms=0

-test_efficiency() { - - FILENAME="results_$RANDOM$$.txt" - - if [ -e /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors ]; then - for i in $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors); do - if [ $i = $1 ] ; then - gov_target=$i - fi - export gov_$i=$(echo $i) - done - else - echo "cpufreq sysfs is not available!" - exit +# Read final residency numbers and calculate delta. +calculate_residencies() { + if [ ! -e "$test_gov-cpu$cpu-$runtime_us-$period_us-$loops-pre-residency.txt" ]; then + return fi + if [ ! -e "$test_gov-cpu$cpu-$runtime_us-$period_us-$loops-post-residency.txt" ]; then + return + fi + echo -ne "$runtime\t$period\t$loops" >> "$test_gov-residency.txt" + i=0 + while read -r speed time; do + residency[$i]=$time + i=$((i + 1)) + done < "$test_gov-cpu$cpu-$runtime_us-$period_us-$loops-pre-residency.txt" + i=0 + total=0 + while read -r speed time; do + residency[$i]=$((time - ${residency[$i]})) + total=$((total + ${residency[$i]})) + i=$((i+1)) + done < "$test_gov-cpu$cpu-$runtime_us-$period_us-$loops-post-residency.txt" + for i in "${residency[@]}"; do + i=$(expr $i * 100) + i=$(expr $i / $total) + echo -ne "\t$i%" >> "$test_gov-residency.txt" + done + echo "" >> "$test_gov-residency.txt" +}

- if [ ! $gov_target ] ; then - echo " Can't find $1 governor!" - exit - fi +# Uses global vars: +# $governor: a single line list of governors to run this workload on +# $cpu: the CPU to run this workload on +# $runtime: busy time in the duty cycle +# $period: sleep time in the duty cycle +# $loops: number of iterations of the duty cycle +test_efficiency() { + tmpfile=$(mktemp)

- if [ ! $gov_performance ] ; then - echo "Can't find performance governor!" - exit - fi + echo -ne "$runtime\t$period\t$loops\t"

- if [ ! $gov_powersave ] ; then - echo "Can't find powersave governor!" - exit - fi + # Convert duty cycle parameteres into usec from msec. + runtime_us=$((runtime * 1000)) + period_us=$((period * 1000))

- if [ $gov_target = $gov_performance ] || [ $gov_target = $gov_powersave ] ; then - echo "Please input a governor other than "performance" or "powersave"" - exit - fi + # Get performance governor data for this workload. + ./dvfs.sh performance $cpu $runtime_us $period_us $loops > $tmpfile + performance=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 1) + over_perf=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 2) + echo -ne "100%\t$over_perf\t" + + # Get powersave governor data for this workload. + ./dvfs.sh powersave $cpu $runtime_us $period_us $loops > $tmpfile + powersave=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 1) + over_save=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 2) + echo -ne "0%\t$over_save"

- # Get target gov data first - dvfs.sh $1 $2 $3 $4 $5 > $FILENAME - target=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 1) - over_target=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 2) - # Get performance data - dvfs.sh performance $2 $3 $4 $5 > $FILENAME - performance=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 1) - over_perf=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 2) - # Get powersave data - dvfs.sh powersave $2 $3 $4 $5 > $FILENAME - powersave=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 1) - over_save=$(cat $FILENAME |sed -n '$p' | cut -d " " -f 2) if [ $performance -ge $powersave ] ; then echo "powersave: $powersave" echo "performance: $performance" echo "Error! performance spent more time than powersave!" - exit + rm -f $tmpfile + exit 1 fi

- echo ""powersave" efficiency: 0% overrun: "$over_save - echo ""performance" efficiency: 100% overrun: "$over_perf - - denominator=$(expr $powersave - $performance) - - if [ $powersave -le $target ]; then - target=0 - else - numerator=$(expr $powersave - $target) - numerator=$(expr $numerator * 100) - target=$(expr $numerator / $denominator) - if [ $target -gt 100 ]; then - target=100 + for test_gov in $governor; do + echo -ne "\t" + ./dvfs.sh $test_gov $cpu $runtime_us $period_us $loops > $tmpfile + calculate_residencies + test_gov_perf=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 1) + test_gov_over=$(cat $tmpfile |sed -n '$p' | cut -d " " -f 2) + denominator=$(expr $powersave - $performance) + if [ $powersave -le $test_gov_perf ]; then + test_gov_perf=0 + else + numerator=$(expr $powersave - $test_gov_perf) + numerator=$(expr $numerator * 100) + test_gov_perf=$(expr $numerator / $denominator) + if [ $test_gov_perf -gt 100 ]; then + test_gov_perf=100 + fi fi - fi - - echo ""$gov_target" efficiency: $target% overrun: "$over_target - - rm -f $FILENAME + echo -ne "$test_gov_perf%\t$test_gov_over" + done + echo "" + rm -f $tmpfile }

-if [ $# -lt 4 ]; then - echo "Usage: ./test.sh <governor> <cpu> <runtime> <sleeptime> [<loops>]" - echo "governor: target CPUFreq governor you want to test" - echo "cpu: cpu number on which you want to run the test" - echo "runtime: running time in ms per loop of the workload pattern" - echo "sleeptime: sleeping time in ms per loop of the workload pattern" - echo "loops: repeat times of the workload pattern. default: 10" - echo "\nExample:\n"./test.sh ondemand 0 100 100 20" means\nTest "ondemand" on CPU0 with workload pattern "run 100ms + sleep 100ms"(20 loops).\n" - exit +# Parse command line arguments. +for var in "$@" +do + if [ "$parse_testfile" -eq 1 ]; then + testfile=$var + parse_testfile=0 + continue + fi + if [ "$var" == "-f" ]; then + parse_testfile=1 + continue + fi + case "$cmdline_test_parms" in + 0) + if [ "$var" == "performance" ] || [ "$var" == "powersave" ]; then + echo "A governor other than performance or powersave must be specified." + exit 1 + fi + governor=$var + ;; + 1) + cpu=$var + ;; + 2) + runtime=$var + ;; + 3) + period=$var + ;; + 4) + loops=$var + ;; + *) + echo "Too many arguments provided." + exit 1; + esac + cmdline_test_parms=$((cmdline_test_parms+1)) +done + +if [ -z "$testfile" ]; then + if [ -z "$governor" ] || [ -z "$cpu" ] || [ -z "$runtime" ] || \ + [ -z "$period" ] || [ -z "$loops" ]; then + echo "You must provide either a test filename or a full set of test parameters." + exit 1; + fi fi

-if [ $# = 4 ]; then - loops=10 +# $1 is the governor +open_residency_file() { + if [ -e "$1-residency.txt" ]; then + echo -e "\n\nError: $1-residency.txt already exists, will not overwrite." + exit 1 + fi + echo -ne "run\tperiod\tloops\t" > "$1-residency.txt" + for speed in $(cat /sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_available_frequencies); do + echo -ne "$speed\t" >> "$1-residency.txt" + done + echo "" >> "$1-residency.txt" +} + +if [ -z "$testfile" ]; then + ./calibration.sh $cpu + open_residency_file $governor + echo -e " test\t\tperformance\tpowersave\t$governor" + echo -e "run\tperiod\tloops\tPERF\tOVR\tPERF\tOVR\tPERF\tOVR" + test_efficiency else - loops=$5 + { + read -r cpu + read -r governor + ./calibration.sh $cpu + echo -en " test\t\tperformance\tpowersave" + for printgov in $governor; do + open_residency_file $printgov + echo -en "\t$printgov" + done + echo "" + echo -en "run\tperiod\tloops\tPERF\tOVR\tPERF\tOVR" + for printgov in $governor; do + echo -en "\tPERF\tOVR" + done + echo "" + while read -r runtime period loops; do + if [ -z "$runtime" ]; then + continue; + fi + test_efficiency + done + for gov in $governor; do + echo "$gov frequency residency:" + cat "$gov-residency.txt" + done + } < "$testfile" fi - -echo "Test "$1" on CPU$2 with workload pattern "run $3ms + sleep $4ms"($loops loops)." - -sleep 1 -PATH=$PATH:. - -test_efficiency $1 $2 $(expr $3 * 1000) $(expr $4 * 1000) $loops -