Hi all:
Update the patches base on the latest kselftest/next.
Add speedometer.sh trigger the speedometer testing and monitor the cpu information.
Modify rst document to introduce test steps and results etc.
Thanks, Jasmine
Meng Li (2): selftests: amd-pstate: Trigger speedometer benchmark and test cpus Documentation: amd-pstate: Add speedometer test introduction
Documentation/admin-guide/pm/amd-pstate.rst | 75 +++- tools/testing/selftests/amd-pstate/Makefile | 2 +- tools/testing/selftests/amd-pstate/run.sh | 32 +- .../selftests/amd-pstate/selenium_server.sh | 15 + .../selftests/amd-pstate/speedometer.sh | 349 ++++++++++++++++++ .../selftests/amd-pstate/speedometer_web.py | 106 ++++++ 6 files changed, 571 insertions(+), 8 deletions(-) create mode 100755 tools/testing/selftests/amd-pstate/selenium_server.sh create mode 100755 tools/testing/selftests/amd-pstate/speedometer.sh create mode 100755 tools/testing/selftests/amd-pstate/speedometer_web.py
Add speedometer.sh trigger the speedometer testing and monitor the cpu desire performance, frequency, load, power consumption and throughput etc.
Signed-off-by: Meng Li li.meng@amd.com --- tools/testing/selftests/amd-pstate/Makefile | 2 +- tools/testing/selftests/amd-pstate/run.sh | 32 +- .../selftests/amd-pstate/selenium_server.sh | 15 + .../selftests/amd-pstate/speedometer.sh | 349 ++++++++++++++++++ .../selftests/amd-pstate/speedometer_web.py | 106 ++++++ 5 files changed, 497 insertions(+), 7 deletions(-) create mode 100755 tools/testing/selftests/amd-pstate/selenium_server.sh create mode 100755 tools/testing/selftests/amd-pstate/speedometer.sh create mode 100755 tools/testing/selftests/amd-pstate/speedometer_web.py
diff --git a/tools/testing/selftests/amd-pstate/Makefile b/tools/testing/selftests/amd-pstate/Makefile index 5f195ee756d6..d6171775fb61 100644 --- a/tools/testing/selftests/amd-pstate/Makefile +++ b/tools/testing/selftests/amd-pstate/Makefile @@ -13,6 +13,6 @@ TEST_GEN_FILES += ../../../power/x86/intel_pstate_tracer/intel_pstate_tracer.py endif
TEST_PROGS := run.sh -TEST_FILES := basic.sh tbench.sh gitsource.sh +TEST_FILES := basic.sh tbench.sh gitsource.sh speedometer.sh speedometer_web.py selenium_server.sh
include ../lib.mk diff --git a/tools/testing/selftests/amd-pstate/run.sh b/tools/testing/selftests/amd-pstate/run.sh index 57cad57e59c0..8283832b3858 100755 --- a/tools/testing/selftests/amd-pstate/run.sh +++ b/tools/testing/selftests/amd-pstate/run.sh @@ -11,6 +11,7 @@ fi source basic.sh source tbench.sh source gitsource.sh +source speedometer.sh
# amd-pstate-ut only run on x86/x86_64 AMD systems. ARCH=$(uname -m 2>/dev/null | sed -e 's/i.86/x86/' -e 's/x86_64/x86/') @@ -21,6 +22,7 @@ FUNC=all OUTFILE=selftest OUTFILE_TBENCH="$OUTFILE.tbench" OUTFILE_GIT="$OUTFILE.gitsource" +OUTFILE_SPEEDOMETER="$OUTFILE.speedometer"
SYSFS= CPUROOT= @@ -136,6 +138,9 @@ amd_pstate_all()
# gitsource amd_pstate_gitsource + + # speedometer + amd_pstate_speedometer }
help() @@ -146,10 +151,11 @@ help() [-c <all: All testing, basic: Basic testing, tbench: Tbench testing, - gitsource: Gitsource testing.>] + gitsource: Gitsource testing, + speedometer: Speedometer testing.>] [-t <tbench time limit>] [-p <tbench process number>] - [-l <loop times for tbench>] + [-l <loop times>] [-i <amd tracer interval>] [-m <comparative test: acpi-cpufreq>] \n" @@ -165,7 +171,7 @@ parse_arguments() help ;;
- c) # --func_type (Function to perform: basic, tbench, gitsource (default: all)) + c) # --func_type (Function to perform: basic, tbench, gitsource, speedometer (default: all)) FUNC=$OPTARG ;;
@@ -181,7 +187,7 @@ parse_arguments() PROCESS_NUM=$OPTARG ;;
- l) # --tbench/gitsource-loop-times + l) # --tbench/gitsource/speedometer-loop-times LOOP_TIMES=$OPTARG ;;
@@ -249,7 +255,7 @@ prerequisite() fi else case "$FUNC" in - "tbench" | "gitsource") + "tbench" | "gitsource" | "speedometer") if [ "$scaling_driver" != "$COMPARATIVE_TEST" ]; then echo "$0 # Skipped: Comparison test can only run on $COMPARISON_TEST driver." echo "$0 # Current cpufreq scaling drvier is $scaling_driver." @@ -258,7 +264,7 @@ prerequisite() ;;
*) - echo "$0 # Skipped: Comparison test are only for tbench or gitsource." + echo "$0 # Skipped: Comparison test are only for tbench, gitsource or speedometer." echo "$0 # Current comparative test is for $FUNC." exit $ksft_skip ;; @@ -284,6 +290,10 @@ prerequisite() "gitsource") command_perf ;; + + "speedometer") + command_perf + ;; esac
SYSFS=`mount -t sysfs | head -1 | awk '{ print $3 }'` @@ -335,6 +345,10 @@ do_test() amd_pstate_gitsource ;;
+ "speedometer") + amd_pstate_speedometer + ;; + *) echo "Invalid [-f] function type" help @@ -364,6 +378,12 @@ pre_clear_dumps() rm -rf gitsource_*.png ;;
+ "speedometer") + rm -rf $OUTFILE.log + rm -rf $OUTFILE.backup_governor.log + rm -rf speedometer_*.png + ;; + *) ;; esac diff --git a/tools/testing/selftests/amd-pstate/selenium_server.sh b/tools/testing/selftests/amd-pstate/selenium_server.sh new file mode 100755 index 000000000000..d2cb5aeaea3f --- /dev/null +++ b/tools/testing/selftests/amd-pstate/selenium_server.sh @@ -0,0 +1,15 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 + +# Before test speedometer benchmark, this script trigger selenium server. + +selenium_server_name="selenium-server.jar" +selenium_server_url="https://github.com/SeleniumHQ/selenium/releases/download/selenium-4.3.0/sele..." + +if [ ! -f $selenium_server_name ]; then + printf "Download selenium server, please wait a moment ...\n\n" + wget -O $selenium_server_name $selenium_server_url +fi + +printf "Run selenium server standalone ...\n\n" +java -jar $selenium_server_name standalone --port 9515 diff --git a/tools/testing/selftests/amd-pstate/speedometer.sh b/tools/testing/selftests/amd-pstate/speedometer.sh new file mode 100755 index 000000000000..04539876a094 --- /dev/null +++ b/tools/testing/selftests/amd-pstate/speedometer.sh @@ -0,0 +1,349 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 + +# Testing and monitor the cpu frequency and performance when +# this script trigger speedometer test. + +# protect against multiple inclusion +if [ $FILE_SPEEDOMETER ]; then + return 0 +else + FILE_SPEEDOMETER=DONE +fi + +speedometer_governors=("ondemand" "schedutil") + +# $1: governor, $2: round, $3: des-perf, $4: freq, $5: load, $6: goal, $7: time $8: energy, $9: PPW +store_csv_speedometer() +{ + echo "$1, $2, $3, $4, $5, $6, $7, $8, $9" | tee -a $OUTFILE_SPEEDOMETER.csv > /dev/null 2>&1 +} + +# clear some special lines +clear_csv_speedometer() +{ + if [ -f $OUTFILE_SPEEDOMETER.csv ]; then + sed -i '/Comprison(%)/d' $OUTFILE_SPEEDOMETER.csv + sed -i "/$(scaling_name)/d" $OUTFILE_SPEEDOMETER.csv + fi +} + +# find string $1 in file csv and get the number of lines +get_lines_csv_speedometer() +{ + if [ -f $OUTFILE_SPEEDOMETER.csv ]; then + return `grep -c "$1" $OUTFILE_SPEEDOMETER.csv` + else + return 0 + fi +} + +pre_clear_speedometer() +{ + post_clear_speedometer + rm -rf speedometer_*.png + clear_csv_speedometer +} + +post_clear_speedometer() +{ + rm -rf results/tracer-speedometer* + rm -rf $OUTFILE_SPEEDOMETER*.log + rm -rf $OUTFILE_SPEEDOMETER*.result +} + +# $1: governor, $2: loop +run_speedometer() +{ + local_host="`hostname --fqdn`" + local_ip=`host $local_host 2>/dev/null | awk '{print $NF}'` + + echo "Launching amd pstate tracer for $1 #$2 tracer_interval: $TRACER_INTERVAL" + ./amd_pstate_trace.py -n tracer-speedometer-$1-$2 -i $TRACER_INTERVAL > /dev/null 2>&1 & + + echo "Run speedometer for $1 #$2" + perf stat -a --per-socket -I 1000 -e power/energy-pkg/ > $OUTFILE_SPEEDOMETER-perf-$1-$2.log 2>&1 & pid_perf=$! + ./speedometer_web.py -i $local_ip -n $OUTFILE_SPEEDOMETER-web-$1-$2.log + + kill $pid_perf + for job in `jobs -p` + do + echo "Waiting for job id $job" + wait $job + done + +} + +# $1: governor, $2: loop +parse_speedometer() +{ + awk '{print $5}' results/tracer-speedometer-$1-$2/cpu.csv | sed -e '1d' | sed s/,// > $OUTFILE_SPEEDOMETER-des-perf-$1-$2.log + avg_des_perf=$(awk 'BEGIN {i=0; sum=0};{i++; sum += $1};END {print sum/i}' $OUTFILE_SPEEDOMETER-des-perf-$1-$2.log) + printf "Speedometer-$1-#$2 avg des perf: $avg_des_perf\n" | tee -a $OUTFILE_SPEEDOMETER.result + + awk '{print $7}' results/tracer-speedometer-$1-$2/cpu.csv | sed -e '1d' | sed s/,// > $OUTFILE_SPEEDOMETER-freq-$1-$2.log + avg_freq=$(awk 'BEGIN {i=0; sum=0};{i++; sum += $1};END {print sum/i}' $OUTFILE_SPEEDOMETER-freq-$1-$2.log) + printf "Speedometer-$1-#$2 avg freq: $avg_freq\n" | tee -a $OUTFILE_SPEEDOMETER.result + + awk '{print $11}' results/tracer-speedometer-$1-$2/cpu.csv | sed -e '1d' | sed s/,// > $OUTFILE_SPEEDOMETER-load-$1-$2.log + avg_load=$(awk 'BEGIN {i=0; sum=0};{i++; sum += $1};END {print sum/i}' $OUTFILE_SPEEDOMETER-load-$1-$2.log) + printf "Speedometer-$1-#$2 avg load: $avg_load\n" | tee -a $OUTFILE_SPEEDOMETER.result + + awk '{print $1}' $OUTFILE_SPEEDOMETER-web-$1-$2.log > $OUTFILE_SPEEDOMETER-time-$1-$2.log + time_sum=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum}' $OUTFILE_SPEEDOMETER-time-$1-$2.log) + printf "Speedometer-$1-#$2 user time(s): $time_sum\n" | tee -a $OUTFILE_SPEEDOMETER.result + + awk '{print $2}' $OUTFILE_SPEEDOMETER-web-$1-$2.log > $OUTFILE_SPEEDOMETER-goal-$1-$2.log + goal_sum=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum}' $OUTFILE_SPEEDOMETER-goal-$1-$2.log) + printf "Speedometer-$1-#$2 goal: $goal_sum\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep Joules $OUTFILE_SPEEDOMETER-perf-$1-$2.log | awk '{print $4}' > $OUTFILE_SPEEDOMETER-energy-$1-$2.log + en_sum=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum}' $OUTFILE_SPEEDOMETER-energy-$1-$2.log) + printf "Speedometer-$1-#$2 power consumption(J): $en_sum\n" | tee -a $OUTFILE_SPEEDOMETER.result + + # Permance is runs per second, denoted G/60, where G is runs per minute. + # It is well known that P=E/t, where P is power measured in watts(W), E is energy measured in joules(J), + # and t is time measured in seconds(s). This means that performance per watt becomes + # G/60 G/60 Gt + # ----- = ----- = ----- + # P E/t 60*E + # with unit given by runs per wattt. + ppw=`echo "scale=4;$goal_sum*$time_sum/($en_sum*60)" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 performance per watt(goal/w): $ppw\n" | tee -a $OUTFILE_SPEEDOMETER.result + printf "\n" | tee -a $OUTFILE_SPEEDOMETER.result + + driver_name=`echo $(scaling_name)` + store_csv_speedometer "$driver_name-$1" $2 $avg_des_perf $avg_freq $avg_load $goal_sum $time_sum $en_sum $ppw +} + +# $1: governor +loop_speedometer() +{ + printf "\nSpeedometer total test times is $LOOP_TIMES for $1\n\n" + for i in `seq 1 $LOOP_TIMES` + do + run_speedometer $1 $i + parse_speedometer $1 $i + done +} + +# $1: governor +gather_speedometer() +{ + printf "Speedometer test result for $1 (loops:$LOOP_TIMES)" | tee -a $OUTFILE_SPEEDOMETER.result + printf "\n--------------------------------------------------\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "avg des perf:" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-des-perf-$1.log + avg_des_perf=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-des-perf-$1.log) + printf "Speedometer-$1 avg des perf: $avg_des_perf\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "avg freq:" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-freq-$1.log + avg_freq=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-freq-$1.log) + printf "Speedometer-$1 avg freq: $avg_freq\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "avg load:" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-load-$1.log + avg_load=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-load-$1.log) + printf "Speedometer-$1 avg load: $avg_load\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "goal:" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-goal-$1.log + avg_goal=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-goal-$1.log) + printf "Speedometer-$1 avg goal: $avg_goal\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "user time(s):" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-time-$1.log + time_sum=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum}' $OUTFILE_SPEEDOMETER-time-$1.log) + printf "Speedometer-$1 total user time(s): $time_sum\n" | tee -a $OUTFILE_SPEEDOMETER.result + + avg_time=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-time-$1.log) + printf "Speedometer-$1 avg user times(s): $avg_time\n" | tee -a $OUTFILE_SPEEDOMETER.result + + grep "Speedometer-$1-#" $OUTFILE_SPEEDOMETER.result | grep "power consumption(J):" | awk '{print $NF}' > $OUTFILE_SPEEDOMETER-energy-$1.log + en_sum=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum}' $OUTFILE_SPEEDOMETER-energy-$1.log) + printf "Speedometer-$1 total power consumption(J): $en_sum\n" | tee -a $OUTFILE_SPEEDOMETER.result + + avg_en=$(awk 'BEGIN {sum=0};{sum += $1};END {print sum/'$LOOP_TIMES'}' $OUTFILE_SPEEDOMETER-energy-$1.log) + printf "Speedometer-$1 avg power consumption(J): $avg_en\n" | tee -a $OUTFILE_SPEEDOMETER.result + + # Permance is runs per second, denoted G/60, where G is runs per minute. + # It is well known that P=E/t, where P is power measured in watts(W), E is energy measured in joules(J), + # and t is time measured in seconds(s). This means that performance per watt becomes + # G/60 G/60 Gt + # ----- = ----- = ----- + # P E/t 60*E + # with unit given by runs per wattt. + ppw=`echo "scale=4;$avg_goal*$avg_time/($avg_en*60)" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 performance per watt(goal/w): $ppw\n" | tee -a $OUTFILE_SPEEDOMETER.result + printf "\n" | tee -a $OUTFILE_SPEEDOMETER.result + + driver_name=`echo $(scaling_name)` + store_csv_speedometer "$driver_name-$1" "Average" $avg_des_perf $avg_freq $avg_load $avg_goal $avg_time $avg_en $ppw +} + +# $1: base scaling_driver $2: base governor $3: comparison scaling_driver $4: comparison governor +__calc_comp_speedometer() +{ + base=`grep "$1-$2" $OUTFILE_SPEEDOMETER.csv | grep "Average"` + comp=`grep "$3-$4" $OUTFILE_SPEEDOMETER.csv | grep "Average"` + + if [ -n "$base" -a -n "$comp" ]; then + printf "\n==================================================\n" | tee -a $OUTFILE_SPEEDOMETER.result + printf "Speedometer comparison $1-$2 VS $3-$4" | tee -a $OUTFILE_SPEEDOMETER.result + printf "\n==================================================\n" | tee -a $OUTFILE_SPEEDOMETER.result + + # get the base values + des_perf_base=`echo "$base" | awk '{print $3}' | sed s/,//` + freq_base=`echo "$base" | awk '{print $4}' | sed s/,//` + load_base=`echo "$base" | awk '{print $5}' | sed s/,//` + goal_base=`echo "$base" | awk '{print $6}' | sed s/,//` + time_base=`echo "$base" | awk '{print $7}' | sed s/,//` + energy_base=`echo "$base" | awk '{print $8}' | sed s/,//` + ppw_base=`echo "$base" | awk '{print $9}' | sed s/,//` + + # get the comparison values + des_perf_comp=`echo "$comp" | awk '{print $3}' | sed s/,//` + freq_comp=`echo "$comp" | awk '{print $4}' | sed s/,//` + load_comp=`echo "$comp" | awk '{print $5}' | sed s/,//` + goal_comp=`echo "$comp" | awk '{print $6}' | sed s/,//` + time_comp=`echo "$comp" | awk '{print $7}' | sed s/,//` + energy_comp=`echo "$comp" | awk '{print $8}' | sed s/,//` + ppw_comp=`echo "$comp" | awk '{print $9}' | sed s/,//` + + # compare the base and comp values + des_perf_drop=`echo "scale=4;($des_perf_comp-$des_perf_base)*100/$des_perf_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 des perf base: $des_perf_base comprison: $des_perf_comp percent: $des_perf_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + freq_drop=`echo "scale=4;($freq_comp-$freq_base)*100/$freq_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 freq base: $freq_base comprison: $freq_comp percent: $freq_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + load_drop=`echo "scale=4;($load_comp-$load_base)*100/$load_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 load base: $load_base comprison: $load_comp percent: $load_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + time_drop=`echo "scale=4;($time_comp-$time_base)*100/$time_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 time base: $time_base comprison: $time_comp percent: $time_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + goal_drop=`echo "scale=4;($goal_comp-$goal_base)*100/$goal_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 goal base: $goal_base comprison: $goal_comp percent: $goal_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + energy_drop=`echo "scale=4;($energy_comp-$energy_base)*100/$energy_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 energy base: $energy_base comprison: $energy_comp percent: $energy_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + + ppw_drop=`echo "scale=4;($ppw_comp-$ppw_base)*100/$ppw_base" | bc | awk '{printf "%.4f", $0}'` + printf "Speedometer-$1 performance per watt base: $ppw_base comprison: $ppw_comp percent: $ppw_drop\n" | tee -a $OUTFILE_SPEEDOMETER.result + printf "\n" | tee -a $OUTFILE_SPEEDOMETER.result + + store_csv_speedometer "$1-$2 VS $3-$4" "Comprison(%)" "$des_perf_drop" "$freq_drop" "$load_drop" "$goal_drop" "$time_drop" "$energy_drop" "$ppw_drop" + fi +} + +# calculate the comparison(%) +calc_comp_speedometer() +{ + # acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil + __calc_comp_speedometer ${all_scaling_names[0]} ${speedometer_governors[0]} ${all_scaling_names[0]} ${speedometer_governors[1]} + + # amd-pstate-ondemand VS amd-pstate-schedutil + __calc_comp_speedometer ${all_scaling_names[1]} ${speedometer_governors[0]} ${all_scaling_names[1]} ${speedometer_governors[1]} + + # acpi-cpufreq-ondemand VS amd-pstate-ondemand + __calc_comp_speedometer ${all_scaling_names[0]} ${speedometer_governors[0]} ${all_scaling_names[1]} ${speedometer_governors[0]} + + # acpi-cpufreq-schedutil VS amd-pstate-schedutil + __calc_comp_speedometer ${all_scaling_names[0]} ${speedometer_governors[1]} ${all_scaling_names[1]} ${speedometer_governors[1]} +} + +# $1: file_name, $2: title, $3: ylable, $4: column +plot_png_speedometer() +{ + # all_scaling_names[1] all_scaling_names[0] flag + # amd-pstate acpi-cpufreq + # N N 0 + # N Y 1 + # Y N 2 + # Y Y 3 + ret=`grep -c "${all_scaling_names[1]}" $OUTFILE_SPEEDOMETER.csv` + if [ $ret -eq 0 ]; then + ret=`grep -c "${all_scaling_names[0]}" $OUTFILE_SPEEDOMETER.csv` + if [ $ret -eq 0 ]; then + flag=0 + else + flag=1 + fi + else + ret=`grep -c "${all_scaling_names[0]}" $OUTFILE_SPEEDOMETER.csv` + if [ $ret -eq 0 ]; then + flag=2 + else + flag=3 + fi + fi + + gnuplot << EOF + set term png + set output "$1" + + set title "$2" + set xlabel "Test Cycles (round)" + set ylabel "$3" + + set grid + set style data histogram + set style fill solid 0.5 border + set boxwidth 0.8 + + if ($flag == 1) { + plot \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[0]}-${speedometer_governors[0]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[0]}-${speedometer_governors[0]}", \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[0]}-${speedometer_governors[1]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[0]}-${speedometer_governors[1]}" + } else { + if ($flag == 2) { + plot \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[1]}-${speedometer_governors[0]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[1]}-${speedometer_governors[0]}", \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[1]}-${speedometer_governors[1]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[1]}-${speedometer_governors[1]}" + } else { + if ($flag == 3 ) { + plot \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[0]}-${speedometer_governors[0]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[0]}-${speedometer_governors[0]}", \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[0]}-${speedometer_governors[1]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[0]}-${speedometer_governors[1]}", \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[1]}-${speedometer_governors[0]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[1]}-${speedometer_governors[0]}", \ + "<(sed -n -e 's/,//g' -e '/${all_scaling_names[1]}-${speedometer_governors[1]}/p' $OUTFILE_SPEEDOMETER.csv)" using $4:xtic(2) title "${all_scaling_names[1]}-${speedometer_governors[1]}" + } + } + } + quit +EOF +} + +amd_pstate_speedometer() +{ + echo "!!!*** Please make sure to run selenium_server.sh on the server before you start testing speedometer. ***!!!" + + printf "\n---------------------------------------------\n" + printf "*** Running speedometer ***" + printf "\n---------------------------------------------\n" + + pre_clear_speedometer + + get_lines_csv_speedometer "Governor" + if [ $? -eq 0 ]; then + # add titles and unit for csv file + store_csv_speedometer "Governor" "Round" "Des-perf" "Freq" "Load" "Goal" "Time" "Energy" "Performance Per Watt" + store_csv_speedometer "Unit" "" "" "GHz" "" "Runs/Minute" "s" "J" "Runs/w" + fi + + backup_governor + for governor in ${speedometer_governors[*]} ; do + printf "\nSpecified governor is $governor\n\n" + switch_governor $governor + loop_speedometer $governor + gather_speedometer $governor + done + restore_governor + + plot_png_speedometer "speedometer_goal.png" "Speedometer Goal" "Goal (Runs/Minute)" 6 + plot_png_speedometer "speedometer_time.png" "Speedometer Time" "Time (s)" 7 + plot_png_speedometer "speedometer_energy.png" "Speedometer Energy" "Energy (J)" 8 + plot_png_speedometer "speedometer_ppw.png" "Speedometer Performance Per Watt" "Performance Per Watt (Runs/W)" 9 + + calc_comp_speedometer + + post_clear_speedometer +} diff --git a/tools/testing/selftests/amd-pstate/speedometer_web.py b/tools/testing/selftests/amd-pstate/speedometer_web.py new file mode 100755 index 000000000000..2f969e261ed6 --- /dev/null +++ b/tools/testing/selftests/amd-pstate/speedometer_web.py @@ -0,0 +1,106 @@ +#!/usr/bin/python3 +# SPDX-License-Identifier: GPL-2.0-only +# -*- coding: utf-8 -*- +# +""" This utility can be used to run speedometer. + +Prerequisites: + Python version 3.0.x or higher + chromium-browser + chromium-chromedriver + selenium + selenium-server + + see print_help(): for Usage and Output details + +""" + +from selenium import webdriver +from selenium.webdriver.chrome.options import Options +from selenium.webdriver.common.by import By +from selenium.webdriver.remote import webelement +from time import sleep +import os +import sys +import getopt + +test_result = False +ip_addr = "" +file_name = "" + +def print_help(): + print('speedometer_web.py:') + print(' Usage:') + print(' To generate speedometer test result file, parse and plot, use:') + print(' ./speedometer_web.py -i <ip_addr> -n <test_name>') + +def store_data_file(time, result_text): + """ Store speedometer test results """ + + try: + f_handle = open(file_name, 'a') + str_buffer = "%u %s" % (time, result_text) + f_handle.write(str_buffer) + f_handle.close() + except: + print('IO error', file_name) + return + +def remove_data_file(): + """ Remove data files """ + + if os.path.exists(file_name): + os.remove(file_name) + +if __name__ == "__main__": + + valid1 = False + valid2 = False + + try: + opts, args = getopt.getopt(sys.argv[1:],"hn:i:",["help","name="]) + except getopt.GetoptError: + print_help() + sys.exit(2) + for opt, arg in opts: + if opt == '-h': + print_help() + sys.exit() + elif opt in ("-i", "--ip"): + valid1 = True + ip_addr = arg + elif opt in ("-n", "--name"): + valid2 = True + file_name = arg + + if (valid1 and valid2) : + chrome_options = Options() + chrome_options.add_argument('--no-sandbox') + chrome_options.add_argument('--incognito') + + driver = webdriver.Remote(command_executor='http://%7B%7D:9515/wd/hub%27.format(ip_addr), options=chrome_options) + + driver.get('https://browserbench.org/Speedometer2.0/') + sleep(1) + + driver.find_element(By.XPATH,"//div[@class='buttons']/button").click() + for i in range(0, 200, +1): + print("\rRun speedometer: \033[0;31m{:2}\033[0ms".format(i), end="", flush=True) + sleep(1) + if i > 50: + result = driver.find_element(By.XPATH,"//div[@id='result-number']") + if result.text: + test_result = True + print("\nTest result: " + result.text) + remove_data_file() + store_data_file(i, result.text) + break + + if test_result == False: + print("\nTest fail, please test again!") + + driver.quit() + else: + print_help() + sys.exit() +
Introduce speedometer test cases design and implementation. Monitor cpus changes about performance and power consumption etc.
Signed-off-by: Meng Li li.meng@amd.com --- Documentation/admin-guide/pm/amd-pstate.rst | 75 ++++++++++++++++++++- 1 file changed, 74 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/pm/amd-pstate.rst b/Documentation/admin-guide/pm/amd-pstate.rst index 29c50e96eb95..0163d1231aa7 100644 --- a/Documentation/admin-guide/pm/amd-pstate.rst +++ b/Documentation/admin-guide/pm/amd-pstate.rst @@ -451,6 +451,23 @@ Unit Tests for amd-pstate The specified governor is ondemand or schedutil. Gitsource can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
+ 4). Speedometer test + + Test and monitor the cpu changes when running speedometer benchmark under the specified governor. + These changes include desire performance, frequency, load, time, energy etc. + The specified governor is ondemand or schedutil. + Speedometer can also be tested on the ``acpi-cpufreq`` kernel driver for comparison. + +#. Preparations before tests + + 1). Speedometer + + + Python version 3.0.x or higher + + Install chromium-browser + + Install chromium-chromedriver + + Install selenium on the client + + Install selenium-server on the server + #. How to execute the tests
We use test module in the kselftest frameworks to implement it. @@ -487,6 +504,8 @@ Unit Tests for amd-pstate $ sudo ./run.sh -t tbench -m acpi-cpufreq $ sudo ./run.sh -t gitsource $ sudo ./run.sh -t gitsource -m acpi-cpufreq + $ sudo ./run.sh -t speedometer + $ sudo ./run.sh -t speedometer -m acpi-cpufreq $ ./run.sh --help ./run.sh: illegal option -- - Usage: ./run.sh [OPTION...] @@ -495,7 +514,8 @@ Unit Tests for amd-pstate [-c <all: All testing, basic: Basic testing, tbench: Tbench testing, - gitsource: Gitsource testing.>] + gitsource: Gitsource testing, + speedometer: Speedometer testing.>] [-t <tbench time limit>] [-p <tbench process number>] [-l <loop times for tbench>] @@ -621,6 +641,59 @@ Unit Tests for amd-pstate + acpi-cpufreq-schedutil VS amd-pstate-schedutil | Comprison(%) | | | | 2.1115 | 4.2873 | -4.1110 | +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
+ + speedometer + + When you finish test, you will get selftest.speedometer.csv and png images. + The selftest.speedometer.csv file contains the raw data and the drop of the comparative test. + The png images shows the goal, time, energy and performan per watt of each test. + Open selftest.speedometer.csv : + + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + Governor | Round | Des-perf | Freq | Load | Goal | Time | Energy | Performance Per Watt | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + Unit | | | GHz | | Runs/Minute | s | J | Runs/w | + +=================================================+==============+==========+==========+==========+=============+=============+=========+======================+ + + acpi-cpufreq-ondemand | 1 | | | | 209 | 51 | 731.84 | 0.2427 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-ondemand | 2 | | | | 205 | 51 | 759.03 | 0.2295 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-ondemand | 3 | | | | 206 | 51 | 755.15 | 0.2318 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-ondemand | Average | | | | 206.667 | 51 | 748.673 | 0.2346 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-schedutil | 1 | | | | 206 | 56 | 775.08 | 0.248 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-schedutil | 2 | | | | 204 | 51 | 762.06 | 0.2275 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-schedutil | 3 | | | | 207 | 56 | 776.35 | 0.2488 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-schedutil | Average | | | | 205.667 | 54.3333 | 771.163 | 0.2415 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-ondemand | 1 | 24.7974 | 1.73142 | 7.71728 | 195 | 64 | 756.6 | 0.2749 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-ondemand | 2 | 26.1653 | 1.91492 | 8.5525 | 195 | 51 | 705.67 | 0.2348 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-ondemand | 3 | 24.1789 | 1.69516 | 7.41152 | 196 | 65 | 758.98 | 0.2797 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-ondemand | Average | 25.0472 | 1.7805 | 7.89377 | 195.333 | 60 | 740.417 | 0.2638 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-schedutil | 1 | 67.0214 | 2.76691 | 17.1314 | 197 | 51 | 737.52 | 0.227 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-schedutil | 2 | 64.3032 | 2.75981 | 16.1196 | 198 | 55 | 763.57 | 0.2376 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-schedutil | 3 | 65.5175 | 2.59423 | 17.0067 | 201 | 51 | 735.96 | 0.2321 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-schedutil | Average | 65.614 | 2.70698 | 16.7526 | 198.667 | 52.3333 | 745.683 | 0.2323 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil | Comprison(%) | | | | -0.4838 | 6.5358 | 3.0039 | 2.9411 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + amd-pstate-ondemand VS amd-pstate-schedutil | Comprison(%) | 161.9614 | 52.0348 | 112.2255 | 1.7068 | -12.7778 | 0.7112 | -11.9408 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-ondemand VS amd-pstate-ondemand | Comprison(%) | | | | -5.4841 | 17.647 | -1.1027 | 12.4467 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + + acpi-cpufreq-schedutil VS amd-pstate-schedutil | Comprison(%) | | | | -3.4035 | -3.6809 | -3.3041 | -3.8095 | + +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+ + Reference ===========
On Mon, Nov 07, 2022 at 09:11:27AM +0800, Meng, Li (Jassmine) wrote:
Introduce speedometer test cases design and implementation. Monitor cpus changes about performance and power consumption etc.
Signed-off-by: Meng Li li.meng@amd.com
Documentation/admin-guide/pm/amd-pstate.rst | 75 ++++++++++++++++++++- 1 file changed, 74 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/pm/amd-pstate.rst b/Documentation/admin-guide/pm/amd-pstate.rst index 29c50e96eb95..0163d1231aa7 100644 --- a/Documentation/admin-guide/pm/amd-pstate.rst +++ b/Documentation/admin-guide/pm/amd-pstate.rst @@ -451,6 +451,23 @@ Unit Tests for amd-pstate The specified governor is ondemand or schedutil. Gitsource can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
- 4). Speedometer test
Test and monitor the cpu changes when running speedometer benchmark under the specified governor.
These changes include desire performance, frequency, load, time, energy etc.
The specified governor is ondemand or schedutil.
Speedometer can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
+#. Preparations before tests
- 1). Speedometer
+ Python version 3.0.x or higher
+ Install chromium-browser
+ Install chromium-chromedriver
+ Install selenium on the client
+ Install selenium-server on the server
#. How to execute the tests We use test module in the kselftest frameworks to implement it. @@ -487,6 +504,8 @@ Unit Tests for amd-pstate $ sudo ./run.sh -t tbench -m acpi-cpufreq $ sudo ./run.sh -t gitsource $ sudo ./run.sh -t gitsource -m acpi-cpufreq
$ sudo ./run.sh -t speedometer
$ sudo ./run.sh -t speedometer -m acpi-cpufreq
While I run the above scripts: sudo ./run.sh -t speedometer
I found Tbench test result will be reported as well, but it reported as a runtime error.
Tbench test result for ondemand (loops:3) -------------------------------------------------- Tbench-ondemand avg des perf: -nan Tbench-ondemand avg freq: -nan Tbench-ondemand avg load: -nan Tbench-ondemand total throughput(MB/s): 0 Tbench-ondemand avg throughput(MB/s): 0 Tbench-ondemand total power consumption(J): 0 Tbench-ondemand avg power consumption(J): 0 Runtime error (func=(main), adr=14): Divide by zero Tbench-ondemand performance per watt(MB/J):
I think we may have more detailed steps to tell every one how to execute the tests easily.
Thanks, Ray
$ ./run.sh --help ./run.sh: illegal option -- - Usage: ./run.sh [OPTION...]
@@ -495,7 +514,8 @@ Unit Tests for amd-pstate [-c <all: All testing, basic: Basic testing, tbench: Tbench testing,
gitsource: Gitsource testing.>]
gitsource: Gitsource testing,
speedometer: Speedometer testing.>] [-t <tbench time limit>] [-p <tbench process number>] [-l <loop times for tbench>]
@@ -621,6 +641,59 @@ Unit Tests for amd-pstate + acpi-cpufreq-schedutil VS amd-pstate-schedutil | Comprison(%) | | | | 2.1115 | 4.2873 | -4.1110 | +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
+ speedometer
When you finish test, you will get selftest.speedometer.csv and png images.
The selftest.speedometer.csv file contains the raw data and the drop of the comparative test.
The png images shows the goal, time, energy and performan per watt of each test.
Open selftest.speedometer.csv :
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ Governor | Round | Des-perf | Freq | Load | Goal | Time | Energy | Performance Per Watt |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ Unit | | | GHz | | Runs/Minute | s | J | Runs/w |
+=================================================+==============+==========+==========+==========+=============+=============+=========+======================+
+ acpi-cpufreq-ondemand | 1 | | | | 209 | 51 | 731.84 | 0.2427 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-ondemand | 2 | | | | 205 | 51 | 759.03 | 0.2295 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-ondemand | 3 | | | | 206 | 51 | 755.15 | 0.2318 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-ondemand | Average | | | | 206.667 | 51 | 748.673 | 0.2346 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-schedutil | 1 | | | | 206 | 56 | 775.08 | 0.248 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-schedutil | 2 | | | | 204 | 51 | 762.06 | 0.2275 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-schedutil | 3 | | | | 207 | 56 | 776.35 | 0.2488 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-schedutil | Average | | | | 205.667 | 54.3333 | 771.163 | 0.2415 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-ondemand | 1 | 24.7974 | 1.73142 | 7.71728 | 195 | 64 | 756.6 | 0.2749 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-ondemand | 2 | 26.1653 | 1.91492 | 8.5525 | 195 | 51 | 705.67 | 0.2348 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-ondemand | 3 | 24.1789 | 1.69516 | 7.41152 | 196 | 65 | 758.98 | 0.2797 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-ondemand | Average | 25.0472 | 1.7805 | 7.89377 | 195.333 | 60 | 740.417 | 0.2638 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-schedutil | 1 | 67.0214 | 2.76691 | 17.1314 | 197 | 51 | 737.52 | 0.227 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-schedutil | 2 | 64.3032 | 2.75981 | 16.1196 | 198 | 55 | 763.57 | 0.2376 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-schedutil | 3 | 65.5175 | 2.59423 | 17.0067 | 201 | 51 | 735.96 | 0.2321 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-schedutil | Average | 65.614 | 2.70698 | 16.7526 | 198.667 | 52.3333 | 745.683 | 0.2323 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil | Comprison(%) | | | | -0.4838 | 6.5358 | 3.0039 | 2.9411 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ amd-pstate-ondemand VS amd-pstate-schedutil | Comprison(%) | 161.9614 | 52.0348 | 112.2255 | 1.7068 | -12.7778 | 0.7112 | -11.9408 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-ondemand VS amd-pstate-ondemand | Comprison(%) | | | | -5.4841 | 17.647 | -1.1027 | 12.4467 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
+ acpi-cpufreq-schedutil VS amd-pstate-schedutil | Comprison(%) | | | | -3.4035 | -3.6809 | -3.3041 | -3.8095 |
+-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
Reference
2.34.1
On Mon, Nov 07, 2022 at 09:11:25AM +0800, Meng, Li (Jassmine) wrote:
Hi all:
Update the patches base on the latest kselftest/next.
Add speedometer.sh trigger the speedometer testing and monitor the cpu information.
Modify rst document to introduce test steps and results etc.
Jassmine, thanks for the patches. Let me verify the tests in my side and get it back to you.
Thanks, Ray
linux-kselftest-mirror@lists.linaro.org