On 4/15/23 12:38 AM, Luis Chamberlain wrote:

The default timeout for selftests tests is 45 seconds. Although we already have 13 settings for tests of about 96 sefltests which use a timeout greater than this, we want to try to avoid encouraging more tests to forcing a higher test timeout as selftests strives to run all tests quickly. Selftests also uses the timeout as a non-fatal error. Only tests runners which have control over a system would know if to treat a timeout as fatal or not.

To help with all this:

o Enhance documentation to avoid future increases of insane timeouts o Add the option to allow overriding the default timeout with test runners with a command line option

Suggested-by: Shuah Khan skhan@linuxfoundation.org Signed-off-by: Luis Chamberlain mcgrof@kernel.org

Reviewed-by: Muhammad Usama Anjum usama.anjum@collabora.com Tested-by:Muhammad Usama Anjum usama.anjum@collabora.com

---

Documentation/dev-tools/kselftest.rst | 22 +++++++++++++++++++++ tools/testing/selftests/kselftest/runner.sh | 11 ++++++++++- tools/testing/selftests/run_kselftest.sh | 5 +++++ 3 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst index 12b575b76b20..dd214af7b7ff 100644 --- a/Documentation/dev-tools/kselftest.rst +++ b/Documentation/dev-tools/kselftest.rst @@ -168,6 +168,28 @@ the `-t` option for specific single tests. Either can be used multiple times:: For other features see the script usage output, seen with the `-h` option. +Timeout for selftests +=====================

+Selftests are designed to be quick and so a default timeout is used of 45 +seconds for each test. Tests can override the default timeout by adding +a settings file in their directory and set a timeout variable there to the +configured a desired upper timeout for the test. Only a few tests override +the timeout with a value higher than 45 seconds, selftests strives to keep +it that way. Timeouts in selftests are not considered fatal because the +system under which a test runs may change and this can also modify the +expected time it takes to run a test. If you have control over the systems +which will run the tests you can configure a test runner on those systems to +use a greater or lower timeout on the command line as with the `-o` or +the `--override-timeout` argument. For example to use 165 seconds instead +one would use:

• $ ./run_kselftest.sh --override-timeout 165

+You can look at the TAP output to see if you ran into the timeout. Test +runners which know a test must run under a specific time can then optionally +treat these timeouts then as fatal.

Packaging selftests

diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index 294619ade49f..1c952d1401d4 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -8,7 +8,8 @@ export logfile=/dev/stdout export per_test_logging= # Defaults for "settings" file fields: -# "timeout" how many seconds to let each test run before failing. +# "timeout" how many seconds to let each test run before running +# over our soft timeout limit. export kselftest_default_timeout=45 # There isn't a shell-agnostic way to find the path of a sourced file, @@ -90,6 +91,14 @@ run_one() done < "$settings" fi

• # Command line timeout overrides the settings file
• if [ -n "$kselftest_override_timeout" ]; then

• kselftest_timeout="$kselftest_override_timeout"
  

• echo "# overriding timeout to $kselftest_timeout" >> "$logfile"
  

• else

• echo "# timeout set to $kselftest_timeout" >> "$logfile"
  

• fi
• TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST" echo "# $TEST_HDR_MSG" if [ ! -e "$TEST" ]; then

diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh index 97165a83df63..9a981b36bd7f 100755 --- a/tools/testing/selftests/run_kselftest.sh +++ b/tools/testing/selftests/run_kselftest.sh @@ -26,6 +26,7 @@ Usage: $0 [OPTIONS] -l | --list List the available collection:test entries -d | --dry-run Don't actually run any tests -h | --help Show this usage info

• -o | --override-timeout Number of seconds after which we timeout

EOF exit $1 } @@ -33,6 +34,7 @@ EOF COLLECTIONS="" TESTS="" dryrun="" +kselftest_override_timeout="" while true; do case "$1" in -s | --summary) @@ -51,6 +53,9 @@ while true; do -d | --dry-run) dryrun="echo" shift ;;

• -o | --override-timeout)
  

• 	kselftest_override_timeout="$2"
  

• 	shift 2 ;;
  

  -h | --help) usage 0 ;; "")