When running the ftrace selftests, 2 failures and 6 unresolved cases were observed. The failures can be avoided by setting a sysctl prior to test execution (fixed in patch 1) and by having unresolved cases not return 0 from ftracetest by default since they indicate an absence of testing modules/programs rather than ftrace issues (patch 2).
The latter are classified as "unresolved" tests, which operate differently from "unsupported" tests. For unsupported tests, we note the unsupported count but do not consider the tests as having failed, whereas with unresolved the test run is considered to have failed so returns "not ok" when run via kselftest ("make -C tools/testing/selftest/ftrace run_tests").
Patch 2 aligns the unresolved behaviour with the unsupported; by default, unresolved outcomes do not trigger overall failure, but they can if --fail-unresolved is specified.
Changes since v1:
- updated patch 1 to use /proc path instead of sysctl (Masami) - updated patch 2 to modify unresolved handling in ftracetest rather than change individual unresolved -> unsupported (Masami)
Alan Maguire (2): ftrace/selftests: workaround cgroup RT scheduling issues ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set
tools/testing/selftests/ftrace/ftracetest | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-)
wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory fail due to the chrt command returning:
chrt: failed to set pid 0's policy: Operation not permitted.
To work around this, temporarily disable grout RT scheduling during ftracetest execution. Restore original value on test run completion. With these changes in place, both tests consistently pass.
Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase") Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com --- tools/testing/selftests/ftrace/ftracetest | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 063ecb2..144308a 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -29,8 +29,25 @@ err_ret=1 # kselftest skip code is 4 err_skip=4
+# cgroup RT scheduling prevents chrt commands from succeeding, which +# induces failures in test wakeup tests. Disable for the duration of +# the tests. + +readonly sched_rt_runtime=/proc/sys/kernel/sched_rt_runtime_us + +sched_rt_runtime_orig=$(cat $sched_rt_runtime) + +setup() { + echo -1 > $sched_rt_runtime +} + +cleanup() { + echo $sched_rt_runtime_orig > $sched_rt_runtime +} + errexit() { # message echo "Error: $1" 1>&2 + cleanup exit $err_ret }
@@ -39,6 +56,8 @@ if [ `id -u` -ne 0 ]; then errexit "this must be run by root user" fi
+setup + # Utilities absdir() { # file_path (cd `dirname $1`; pwd) @@ -235,6 +254,7 @@ TOTAL_RESULT=0
INSTANCE= CASENO=0 + testcase() { # testfile CASENO=$((CASENO+1)) desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:` @@ -406,5 +426,7 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w`
+cleanup + # if no error, return 0 exit $TOTAL_RESULT
Currently, ftracetest will return 1 (failure) if any unresolved cases are encountered. The unresolved status results from modules and programs not being available, and as such does not indicate any issues with ftrace itself. As such, change the behaviour of ftracetest in line with unsupported cases; if unsupported cases happen, ftracetest still returns 0 unless --fail-unsupported. Here --fail-unresolved is added and the default is to return 0 if unresolved results occur.
Signed-off-by: Alan Maguire alan.maguire@oracle.com --- tools/testing/selftests/ftrace/ftracetest | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 144308a..19e9236 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -17,6 +17,7 @@ echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" echo " --fail-unsupported Treat UNSUPPORTED as a failure" +echo " --fail-unresolved Treat UNRESOLVED as a failure" echo " -d|--debug Debug mode (trace all shell commands)" echo " -l|--logdir <dir> Save logs on the <dir>" echo " If <dir> is -, all logs output in console only" @@ -112,6 +113,10 @@ parse_opts() { # opts UNSUPPORTED_RESULT=1 shift 1 ;; + --fail-unresolved) + UNRESOLVED_RESULT=1 + shift 1 + ;; --logdir|-l) LOG_DIR=$2 shift 2 @@ -176,6 +181,7 @@ KEEP_LOG=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 +UNRESOLVED_RESULT=0 STOP_FAILURE=0 # Parse command-line options parse_opts $* @@ -280,7 +286,7 @@ eval_result() { # sigval $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO" - return 1 # this is a kind of bug.. something happened. + return $UNRESOLVED_RESULT # depends on use case ;; $UNTESTED) prlog " [${color_blue}UNTESTED${color_reset}]"
On Wed, 19 Feb 2020 09:33:30 +0000 Alan Maguire alan.maguire@oracle.com wrote:
Currently, ftracetest will return 1 (failure) if any unresolved cases are encountered. The unresolved status results from modules and programs not being available, and as such does not indicate any issues with ftrace itself. As such, change the behaviour of ftracetest in line with unsupported cases; if unsupported cases happen, ftracetest still returns 0 unless --fail-unsupported. Here --fail-unresolved is added and the default is to return 0 if unresolved results occur.
OK, this looks good to me. One note, with this change, ftracetest doesn't fail even if your test environment is not well prepared anymore.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Thank you,
Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/ftracetest | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 144308a..19e9236 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -17,6 +17,7 @@ echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" echo " --fail-unsupported Treat UNSUPPORTED as a failure" +echo " --fail-unresolved Treat UNRESOLVED as a failure" echo " -d|--debug Debug mode (trace all shell commands)" echo " -l|--logdir <dir> Save logs on the <dir>" echo " If <dir> is -, all logs output in console only" @@ -112,6 +113,10 @@ parse_opts() { # opts UNSUPPORTED_RESULT=1 shift 1 ;;
- --fail-unresolved)
UNRESOLVED_RESULT=1shift 1- ;; --logdir|-l) LOG_DIR=$2 shift 2
@@ -176,6 +181,7 @@ KEEP_LOG=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 +UNRESOLVED_RESULT=0 STOP_FAILURE=0 # Parse command-line options parse_opts $* @@ -280,7 +286,7 @@ eval_result() { # sigval $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
return 1 # this is a kind of bug.. something happened.
return $UNRESOLVED_RESULT # depends on use case-- 1.8.3.1
Shuah,
Can you take these two patches through your tree?
Acked-by: Steven Rostedt (VMware) rostedt@goodmis.org
-- Steve
On Wed, 19 Feb 2020 09:33:28 +0000 Alan Maguire alan.maguire@oracle.com wrote:
When running the ftrace selftests, 2 failures and 6 unresolved cases were observed. The failures can be avoided by setting a sysctl prior to test execution (fixed in patch 1) and by having unresolved cases not return 0 from ftracetest by default since they indicate an absence of testing modules/programs rather than ftrace issues (patch 2).
The latter are classified as "unresolved" tests, which operate differently from "unsupported" tests. For unsupported tests, we note the unsupported count but do not consider the tests as having failed, whereas with unresolved the test run is considered to have failed so returns "not ok" when run via kselftest ("make -C tools/testing/selftest/ftrace run_tests").
Patch 2 aligns the unresolved behaviour with the unsupported; by default, unresolved outcomes do not trigger overall failure, but they can if --fail-unresolved is specified.
Changes since v1:
- updated patch 1 to use /proc path instead of sysctl (Masami)
- updated patch 2 to modify unresolved handling in ftracetest rather than change individual unresolved -> unsupported (Masami)
Alan Maguire (2): ftrace/selftests: workaround cgroup RT scheduling issues ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set
tools/testing/selftests/ftrace/ftracetest | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-)
On Wed, 19 Feb 2020 20:39:41 +0900 Masami Hiramatsu mhiramat@kernel.org wrote:
On Wed, 19 Feb 2020 09:33:30 +0000 Alan Maguire alan.maguire@oracle.com wrote:
Currently, ftracetest will return 1 (failure) if any unresolved cases are encountered. The unresolved status results from modules and programs not being available, and as such does not indicate any issues with ftrace itself. As such, change the behaviour of ftracetest in line with unsupported cases; if unsupported cases happen, ftracetest still returns 0 unless --fail-unsupported. Here --fail-unresolved is added and the default is to return 0 if unresolved results occur.
OK, this looks good to me. One note, with this change, ftracetest doesn't fail even if your test environment is not well prepared anymore.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Hi Shuah, Could you pick this up?
Po-Hsu Lin seemed to face same problem recently. If this applied, it will be solved.
Thank you,
Thank you,
Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/ftracetest | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 144308a..19e9236 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -17,6 +17,7 @@ echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" echo " --fail-unsupported Treat UNSUPPORTED as a failure" +echo " --fail-unresolved Treat UNRESOLVED as a failure" echo " -d|--debug Debug mode (trace all shell commands)" echo " -l|--logdir <dir> Save logs on the <dir>" echo " If <dir> is -, all logs output in console only" @@ -112,6 +113,10 @@ parse_opts() { # opts UNSUPPORTED_RESULT=1 shift 1 ;;
- --fail-unresolved)
UNRESOLVED_RESULT=1shift 1- ;; --logdir|-l) LOG_DIR=$2 shift 2
@@ -176,6 +181,7 @@ KEEP_LOG=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 +UNRESOLVED_RESULT=0 STOP_FAILURE=0 # Parse command-line options parse_opts $* @@ -280,7 +286,7 @@ eval_result() { # sigval $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
return 1 # this is a kind of bug.. something happened.
return $UNRESOLVED_RESULT # depends on use case-- 1.8.3.1
-- Masami Hiramatsu mhiramat@kernel.org
On 5/1/20 3:42 AM, Masami Hiramatsu wrote:
On Wed, 19 Feb 2020 20:39:41 +0900 Masami Hiramatsu mhiramat@kernel.org wrote:
On Wed, 19 Feb 2020 09:33:30 +0000 Alan Maguire alan.maguire@oracle.com wrote:
Currently, ftracetest will return 1 (failure) if any unresolved cases are encountered. The unresolved status results from modules and programs not being available, and as such does not indicate any issues with ftrace itself. As such, change the behaviour of ftracetest in line with unsupported cases; if unsupported cases happen, ftracetest still returns 0 unless --fail-unsupported. Here --fail-unresolved is added and the default is to return 0 if unresolved results occur.
OK, this looks good to me. One note, with this change, ftracetest doesn't fail even if your test environment is not well prepared anymore.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Hi Shuah, Could you pick this up?
Po-Hsu Lin seemed to face same problem recently. If this applied, it will be solved.
Sorry about this. I will get these in
thanks, -- Shuah
On Fri, 1 May 2020 08:25:50 -0600 shuah shuah@kernel.org wrote:
On 5/1/20 3:42 AM, Masami Hiramatsu wrote:
On Wed, 19 Feb 2020 20:39:41 +0900 Masami Hiramatsu mhiramat@kernel.org wrote:
On Wed, 19 Feb 2020 09:33:30 +0000 Alan Maguire alan.maguire@oracle.com wrote:
Currently, ftracetest will return 1 (failure) if any unresolved cases are encountered. The unresolved status results from modules and programs not being available, and as such does not indicate any issues with ftrace itself. As such, change the behaviour of ftracetest in line with unsupported cases; if unsupported cases happen, ftracetest still returns 0 unless --fail-unsupported. Here --fail-unresolved is added and the default is to return 0 if unresolved results occur.
OK, this looks good to me. One note, with this change, ftracetest doesn't fail even if your test environment is not well prepared anymore.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Hi Shuah, Could you pick this up?
Po-Hsu Lin seemed to face same problem recently. If this applied, it will be solved.
Sorry about this. I will get these in
Thanks Shuah!
thanks, -- Shuah
linux-kselftest-mirror@lists.linaro.org
-
Alan Maguire -
Masami Hiramatsu -
shuah -
Steven Rostedt