When kvm test is skipped because of unmet dependencies and/or unsupported
configuration, it exits with error which is treated as a fail by the
Kselftest framework. This leads to false negative result even when the test
could not be run.
Change it to return kselftest skip code when a test gets skipped to clearly
report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped. In addition,
refine test_assert() message to include strerror() string and add explicit
check for EACCES to cleary identify when test doesn't run when access is
denied to resources required e.g: open /dev/kvm failed, rc: -1 errno: 13
Signed-off-by: Shuah Khan (Samsung OSG) <shuah(a)kernel.org>
---
Changes since v1:
- Don't do root check at the top of assert.
- Instead do check for EACCES.
tools/testing/selftests/kvm/lib/assert.c | 9 +++++++--
tools/testing/selftests/kvm/vmx_tsc_adjust_test.c | 4 +++-
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/kvm/lib/assert.c b/tools/testing/selftests/kvm/lib/assert.c
index c9f5b7d4ce38..cd01144d27c8 100644
--- a/tools/testing/selftests/kvm/lib/assert.c
+++ b/tools/testing/selftests/kvm/lib/assert.c
@@ -13,6 +13,8 @@
#include <execinfo.h>
#include <sys/syscall.h>
+#include "../../kselftest.h"
+
/* Dumps the current stack trace to stderr. */
static void __attribute__((noinline)) test_dump_stack(void);
static void test_dump_stack(void)
@@ -70,8 +72,9 @@ test_assert(bool exp, const char *exp_str,
fprintf(stderr, "==== Test Assertion Failure ====\n"
" %s:%u: %s\n"
- " pid=%d tid=%d\n",
- file, line, exp_str, getpid(), gettid());
+ " pid=%d tid=%d - %s\n",
+ file, line, exp_str, getpid(), gettid(),
+ strerror(errno));
test_dump_stack();
if (fmt) {
fputs(" ", stderr);
@@ -80,6 +83,8 @@ test_assert(bool exp, const char *exp_str,
}
va_end(ap);
+ if (errno == EACCES)
+ ksft_exit_skip("Access denied - Exiting.\n");
exit(254);
}
diff --git a/tools/testing/selftests/kvm/vmx_tsc_adjust_test.c b/tools/testing/selftests/kvm/vmx_tsc_adjust_test.c
index 8f7f62093add..62fb73699eb6 100644
--- a/tools/testing/selftests/kvm/vmx_tsc_adjust_test.c
+++ b/tools/testing/selftests/kvm/vmx_tsc_adjust_test.c
@@ -28,6 +28,8 @@
#include <string.h>
#include <sys/ioctl.h>
+#include "../kselftest.h"
+
#ifndef MSR_IA32_TSC_ADJUST
#define MSR_IA32_TSC_ADJUST 0x3b
#endif
@@ -190,7 +192,7 @@ int main(int argc, char *argv[])
if (!(entry->ecx & CPUID_VMX)) {
printf("nested VMX not enabled, skipping test");
- return 0;
+ return KSFT_SKIP;
}
vm = vm_create_default_vmx(VCPU_ID, (void *) l1_guest_code);
--
2.14.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
hello,
the following is the error found...
-----------------------------------------------------------------------------------
protection_keys.c:421:5: error: conflicting types for ‘pkey_set’
int pkey_set(int pkey, unsigned long rights, unsigned long flags)
^~~~~~~~
------------------------------------------------------------------------------------
to reproduce this error...
make -C tools/testing/selftests
Details about software:
Linux debian 4.17.0-rc1+ #1 SMP Thu Apr 19 18:59:45 IST 2018 x86_64 GNU/Linux
GNU Make 4.2.1
Binutils 2.30
Util-linux 2.31.1
Mount 2.31.1
Linux C Library 2.27
Dynamic linker (ldd) 2.27
readlink: missing operand
Try 'readlink --help' for more information.
Procps 3.3.14
Kbd 2.0.4
Console-tools 2.0.4
Sh-utils 8.28
Udev 238
--
software engineer
rajagiri school of engineering and technology
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
When a script file that isn't generated uses the variable
TEST_GEN_PROGS_EXTENDED and a 'make -C tools/testing/selftests clean' is
performed the script file gets removed and git shows the file as
deleted. For script files that isn't generated TEST_PROGS_EXTENDED
should be used.
Fixes: 9faedd643fd9 ("selftests: net: add in_netns.sh TEST_GEN_PROGS_EXTENDED")
Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org>
---
tools/testing/selftests/net/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 44895de1a0c4..eee47a4bc675 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -7,7 +7,7 @@ CFLAGS += -I../../../../usr/include/
TEST_PROGS := run_netsocktests run_afpackettests test_bpf.sh netdevice.sh rtnetlink.sh
TEST_PROGS += fib_tests.sh fib-onlink-tests.sh pmtu.sh udpgso.sh
TEST_PROGS += udpgso_bench.sh
-TEST_GEN_PROGS_EXTENDED := in_netns.sh
+TEST_PROGS_EXTENDED := in_netns.sh
TEST_GEN_FILES = socket
TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy
TEST_GEN_FILES += tcp_mmap tcp_inq
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
The generated files udpgso* shouldn't be part of TEST_PROGS, they are
used by udpgso.sh and udpgsp_bench.sh. They should be added to the
TEST_GEN_FILES to get installed without being added to the main
run_kselftest.sh script.
Fixes: 3a687bef148d ("selftests: udp gso benchmark")
Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org>
---
tools/testing/selftests/net/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 44895de1a0c4..f0363387ef2f 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -11,9 +11,9 @@ TEST_GEN_PROGS_EXTENDED := in_netns.sh
TEST_GEN_FILES = socket
TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy
TEST_GEN_FILES += tcp_mmap tcp_inq
+TEST_GEN_FILES += udpgso udpgso_bench_tx udpgso_bench_rx
TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa
TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict
-TEST_GEN_PROGS += udpgso udpgso_bench_tx udpgso_bench_rx
include ../lib.mk
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
When memfd test is skipped because of unmet dependencies and/or unsupported
configuration, it returns non-zero value which is treated as a fail by the
Kselftest framework. This leads to false negative result even when the test
could not be run.
Change it to return kselftest skip code when a test gets skipped to clearly
report that the test could not be run.
Added an explicit check for root user at the start of memfd hugetlbfs test
and return skip code if a non-root user attempts to run it.
In addition, return skip code when not enough huge pages are available to
run the test.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah(a)kernel.org>
---
Changes since v1:
-- addressed review comments on v1
-- Fixed a regression in v1 that changed the behavior to require root
for all tests not just the hugetlbfs test.
tools/testing/selftests/memfd/run_tests.sh | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/memfd/run_tests.sh b/tools/testing/selftests/memfd/run_tests.sh
index c2d41ed81b24..2013f195e623 100755
--- a/tools/testing/selftests/memfd/run_tests.sh
+++ b/tools/testing/selftests/memfd/run_tests.sh
@@ -1,6 +1,9 @@
#!/bin/bash
# please run as root
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
#
# Normal tests requiring no special resources
#
@@ -29,12 +32,13 @@ if [ -n "$freepgs" ] && [ $freepgs -lt $hpages_test ]; then
nr_hugepgs=`cat /proc/sys/vm/nr_hugepages`
hpages_needed=`expr $hpages_test - $freepgs`
+ if [ $UID != 0 ]; then
+ echo "Please run memfd with hugetlbfs test as root"
+ exit $ksft_skip
+ fi
+
echo 3 > /proc/sys/vm/drop_caches
echo $(( $hpages_needed + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages
- if [ $? -ne 0 ]; then
- echo "Please run this test as root"
- exit 1
- fi
while read name size unit; do
if [ "$name" = "HugePages_Free:" ]; then
freepgs=$size
@@ -53,7 +57,7 @@ if [ $freepgs -lt $hpages_test ]; then
fi
printf "Not enough huge pages available (%d < %d)\n" \
$freepgs $needpgs
- exit 1
+ exit $ksft_skip
fi
#
--
2.14.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Linus,
Please pull the following Kselftest update for 4.17-rc4
This Kselftest update for 4.17-rc4 consists of a fix for a syntax error
in the script that runs selftests. Mathieu Desnoyers found this bug in
the script on systems running GNU Make 3.8 or older.
diff is attached.
thanks,
-- Shuah
-----------------------------------------------------------------------------------
The following changes since commit 8bf24e8319613bbe950d4188682b3a0d9441b76b:
selftests/filesystems: Don't run dnotify_test by default (2018-04-17 17:01:16 -0600)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-4.17-rc4
for you to fetch changes up to a33554401e4746cc33307910a1baad63ce3fd650:
selftests: Fix lib.mk run_tests target shell script (2018-04-27 16:06:36 -0600)
----------------------------------------------------------------
linux-kselftest-4.17-rc4
This Kselftest update for 4.17-rc4 consists of a fix for a syntax error
in the script that runs selftests. Mathieu Desnoyers found this bug in
the script on systems running GNU Make 3.8 or older.
----------------------------------------------------------------
Mathieu Desnoyers (1):
selftests: Fix lib.mk run_tests target shell script
tools/testing/selftests/lib.mk | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
-----------------------------------------------------------------------------------
Some selftests need to adjust sysctl settings. In order to be neutral to
the system that the test is run on, it is a good practice to change back
to the original setting after the test ends. That involves some
boilerplate that can be abstracted away.
In patch #1, introduce two functions, sysctl_set() and sysctl_restore().
The former stores the current value of a given setting, and sets a new
value. The latter restores the setting to the previously-stored value.
In patch #2, use these wrappers in a number of tests.
Additionally in patch #3, fix a problem in mirror_gre_nh.sh, which
neglected to set a sysctl that's crucial for the test to work.
Petr Machata (3):
selftests: forwarding: lib: Add sysctl_set(), sysctl_restore()
selftests: forwarding: Use sysctl_set(), sysctl_restore()
selftests: forwarding: mirror_gre_nh: Unset RP filter
tools/testing/selftests/net/forwarding/lib.sh | 28 ++++++++++++++++------
.../selftests/net/forwarding/mirror_gre_changes.sh | 7 ++----
.../selftests/net/forwarding/mirror_gre_nh.sh | 6 +++++
.../selftests/net/forwarding/router_multipath.sh | 12 ++++------
4 files changed, 33 insertions(+), 20 deletions(-)
--
2.4.11
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Kselftest framework currently treats all non-zero return codes from tests
as failures. When tests are skipped with non-zero return code, because of
unmet dependencies and/or unsupported configuration, it reports them as
failed. This will lead to too many false negatives even on the tests that
couldn't be run.
This patch series fixes the problem by adding SKIP handling:
The common RUN_TESTS and EMIT_TESTS functions are changed to test for
SKIP=4 return from tests. This change enables the framework and tests
that return SKIP=4 will report SKIP, other continue to report FAIL/PASS.
KSFT_SKIP has been changed to report Skip code of 4 instead of KSFT_PASS
to clearly differentiate the skipped tests.
I am working on Test changes to report SKIP instead FAIL/PASS when they get
skipped. I will be sending them out later this week.
In addition, I made changes to improve the reporting and cleaned up both
RUN_TESTS and EMIT_TESTS define to remove duplicate test output strings.
Shuah Khan (Samsung OSG) (7):
selftests: lib.mk: cleanup RUN_TESTS define and make it readable
selftests: lib.mk: add SKIP handling to RUN_TESTS define
selftests: lib.mk: move running and printing result to a new function
selftests: lib.mk: Include test suite name in the RUN_TESTS output
selftests: lib.mk: add SKIP handling and test suite name to EMIT_TESTS
selftests: lib.mk: add test execute bit check to EMIT_TESTS
selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS
tools/testing/selftests/Makefile | 3 +-
tools/testing/selftests/kselftest.h | 2 +-
tools/testing/selftests/lib.mk | 55 +++++++++++++++++++++++++++----------
3 files changed, 44 insertions(+), 16 deletions(-)
--
2.14.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html