Re: [PATCHv5 kselftest next] selftests/run_kselftest.sh: make each test individually selectable
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
steps do run: # run_kselftest.sh file generated by kselftest Makefile and included in tarball ./run_kselftest.sh 2>&1 | tee "${LOGFILE}"
ref: https://github.com/nareshkamboju/test-definitions/blob/master/automated/linu...
full test run log: https://lkft.validation.linaro.org/scheduler/job/1786826#L1391
Before the patch:
$ cat run_kselftest.sh #!/bin/sh BASE_DIR=$(realpath $(dirname $0)) cd $BASE_DIR . ./kselftest/runner.sh ROOT=$PWD if [ "$1" = "--summary" ]; then logfile=$BASE_DIR/output.log cat /dev/null > $logfile fi [ -w /dev/kmsg ] && echo "kselftest: Running tests in android" >> /dev/kmsg cd android run_many \ "run.sh" cd $ROOT ...<snip>... [ -w /dev/kmsg ] && echo "kselftest: Running tests in zram" >> /dev/kmsg cd zram run_many \ "zram.sh" cd $ROOT
After the patch:
$ cat run_kselftest.sh #!/bin/sh BASE_DIR=$(realpath $(dirname $0)) . ./kselftest/runner.sh TESTS="android ...<snip>... filesystems/binderfs ...<snip>... zram"
run_android() { [ -w /dev/kmsg ] && echo "kselftest: Running tests in android" >> /dev/kmsg cd android run_many \ "run.sh" cd $ROOT }
...<snip>...
run_filesystems_binderfs() { [ -w /dev/kmsg ] && echo "kselftest: Running tests in filesystems/binderfs" >> /dev/kmsg cd filesystems/binderfs run_many \ "binderfs_test" cd $ROOT }
...<snip>...
run_zram() { [ -w /dev/kmsg ] && echo "kselftest: Running tests in zram" >> /dev/kmsg cd zram run_many \ "zram.sh" cd $ROOT }
usage() { cat <<EOF usage: ${0##*/} OPTS -s | --summary Only print summary info and put detailed log in output.log -t | --tests Test name you want to run specifically -h | --help Show this usage info EOF }
while true; do case "$1" in -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;; -t | --tests ) TESTS=$2; shift 2 ;; -l | --list ) echo $TESTS; exit 0 ;; -h | --help ) usage; exit 0 ;; "" ) break;; * ) usage; exit 1;; esac done
cd $BASE_DIR ROOT=$PWD for folder in $TESTS; do folder=$(echo $folder | tr -s '/-' '_') run_$folder done
Signed-off-by: Hangbin Liu liuhangbin@gmail.com
Reported-by: Naresh Kamboju naresh.kamboju@linaro.org
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
BASE_DIR=$(realpath $(dirname $0)) . ./kselftest/runner.sh TESTS="seccomp"
run_seccomp() { -e [ -w /dev/kmsg ] && echo "kselftest: Running tests in seccomp" >> /dev/kmsg -e cd seccomp -en run_many \ -ne "seccomp_bpf" \ -ne "seccomp_benchmark"
-e cd $ROOT }
On 9/25/20 3:16 PM, Kees Cook wrote:
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
BASE_DIR=$(realpath $(dirname $0)) . ./kselftest/runner.sh TESTS="seccomp"
run_seccomp() { -e [ -w /dev/kmsg ] && echo "kselftest: Running tests in seccomp" >> /dev/kmsg -e cd seccomp -en run_many \ -ne "seccomp_bpf" \ -ne "seccomp_benchmark"
-e cd $ROOT }
Will do.
thanks, -- Shuah
On 9/25/20 3:16 PM, Kees Cook wrote:
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
Thank you both. Now reverted.
thanks, -- Shuah
On Fri, Sep 25, 2020 at 05:06:02PM -0600, Shuah Khan wrote:
On 9/25/20 3:16 PM, Kees Cook wrote:
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
Thank you both. Now reverted.
I've sent an alternative that I think should do nicely. It will work well with LAVA as well.
On Fri, Sep 25, 2020 at 02:16:14PM -0700, Kees Cook wrote:
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
BASE_DIR=$(realpath $(dirname $0)) . ./kselftest/runner.sh TESTS="seccomp"
run_seccomp() { -e [ -w /dev/kmsg ] && echo "kselftest: Running tests in seccomp" >> /dev/kmsg -e cd seccomp -en run_many \ -ne "seccomp_bpf" \ -ne "seccomp_benchmark"
-e cd $ROOT }
I'm really sorry to make this trouble. And I'm OK to revert the patch. I just a little wondering how do you generate this script. I tested with 'make -C tools/testing/selftests gen_tar FORMAT=.xz' before post the patch and all looks good to me.
```
run_seccomp() { [ -w /dev/kmsg ] && echo "kselftest: Running tests in seccomp" >> /dev/kmsg cd seccomp run_many \ "seccomp_bpf" \ "seccomp_benchmark" cd $ROOT }
```
Thanks Hangbin
On Sun, Sep 27, 2020 at 09:53:34AM +0800, Hangbin Liu wrote:
On Fri, Sep 25, 2020 at 02:16:14PM -0700, Kees Cook wrote:
On Fri, Sep 25, 2020 at 01:51:53PM +0530, Naresh Kamboju wrote:
On Mon, 14 Sep 2020 at 07:53, Hangbin Liu liuhangbin@gmail.com wrote:
Currently, after generating run_kselftest.sh, there is no way to choose which test we could run. All the tests are listed together and we have to run all every time. This patch enhanced the run_kselftest.sh to make the tests individually selectable. e.g.
$ ./run_kselftest.sh -t "bpf size timers"
My test run break on linux next
./run_kselftest.sh: line 1331: syntax error near unexpected token `)' ./run_kselftest.sh: line 1331: `-e -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'
Yes, please revert this patch. The resulting script is completely trashed:
BASE_DIR=$(realpath $(dirname $0)) . ./kselftest/runner.sh TESTS="seccomp"
run_seccomp() { -e [ -w /dev/kmsg ] && echo "kselftest: Running tests in seccomp" >> /dev/kmsg -e cd seccomp -en run_many \ -ne "seccomp_bpf" \ -ne "seccomp_benchmark"
-e cd $ROOT }
I'm really sorry to make this trouble. And I'm OK to revert the patch. I just a little wondering how do you generate this script.
This issue is with which shell is used. I suspect your /bin/sh is full /bin/bash, where as Naresh's, the CI's, and mine are /bin/dash (which lacks "-e" support for the built-in "echo").
On Mon, Sep 28, 2020 at 01:06:15PM -0700, Kees Cook wrote:
I'm really sorry to make this trouble. And I'm OK to revert the patch. I just a little wondering how do you generate this script.
This issue is with which shell is used. I suspect your /bin/sh is full /bin/bash, where as Naresh's, the CI's, and mine are /bin/dash (which lacks "-e" support for the built-in "echo").
Ah, got it. Thanks for your explanation.
Regards Hangbin
-
Hangbin Liu -
Kees Cook -
Naresh Kamboju -
Shuah Khan