Add kselftest-all target to build tests from the top level
Makefile. This is to simplify kselftest use-cases for CI and
distributions where build and test systems are different.
Current kselftest target builds and runs tests on a development
system which is a developer use-case.
Add kselftest-install target to install tests from the top level
Makefile. This is to simplify kselftest use-cases for CI and
distributions where build and test systems are different.
This change addresses requests from developers and testers to add
support for installing kselftest from the main Makefile.
In addition, make the install directory the same when install is
run using "make kselftest-install" or by running kselftest_install.sh.
Also fix the INSTALL_PATH variable conflict between main Makefile and
selftests Makefile.
Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org>
---
Changes since v1:
- Collpased two patches that added separate targets to
build and install into one patch using pattern rule to
invoke all, install, and clean targets from main Makefile.
Makefile | 5 ++---
tools/testing/selftests/Makefile | 8 ++++++--
tools/testing/selftests/kselftest_install.sh | 4 ++--
3 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/Makefile b/Makefile
index d456746da347..ec296c60c1af 100644
--- a/Makefile
+++ b/Makefile
@@ -1237,9 +1237,8 @@ PHONY += kselftest
kselftest:
$(Q)$(MAKE) -C $(srctree)/tools/testing/selftests run_tests
-PHONY += kselftest-clean
-kselftest-clean:
- $(Q)$(MAKE) -C $(srctree)/tools/testing/selftests clean
+kselftest-%: FORCE
+ $(Q)$(MAKE) -C $(srctree)/tools/testing/selftests $*
PHONY += kselftest-merge
kselftest-merge:
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index c3feccb99ff5..bad18145ed1a 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -171,9 +171,12 @@ run_pstore_crash:
# 1. output_dir=kernel_src
# 2. a separate output directory is specified using O= KBUILD_OUTPUT
# 3. a separate output directory is specified using KBUILD_OUTPUT
+# Avoid conflict with INSTALL_PATH set by the main Makefile
#
-INSTALL_PATH ?= $(BUILD)/install
-INSTALL_PATH := $(abspath $(INSTALL_PATH))
+KSFT_INSTALL_PATH ?= $(BUILD)/kselftest_install
+KSFT_INSTALL_PATH := $(abspath $(KSFT_INSTALL_PATH))
+# Avoid changing the rest of the logic here and lib.mk.
+INSTALL_PATH := $(KSFT_INSTALL_PATH)
ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh
install: all
@@ -203,6 +206,7 @@ ifdef INSTALL_PATH
echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \
echo "cd $$TARGET" >> $(ALL_SCRIPT); \
echo -n "run_many" >> $(ALL_SCRIPT); \
+ echo -n "Emit Tests for $$TARGET\n"; \
$(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \
echo "" >> $(ALL_SCRIPT); \
echo "cd \$$ROOT" >> $(ALL_SCRIPT); \
diff --git a/tools/testing/selftests/kselftest_install.sh b/tools/testing/selftests/kselftest_install.sh
index ec304463883c..e2e1911d62d5 100755
--- a/tools/testing/selftests/kselftest_install.sh
+++ b/tools/testing/selftests/kselftest_install.sh
@@ -24,12 +24,12 @@ main()
echo "$0: Installing in specified location - $install_loc ..."
fi
- install_dir=$install_loc/kselftest
+ install_dir=$install_loc/kselftest_install
# Create install directory
mkdir -p $install_dir
# Build tests
- INSTALL_PATH=$install_dir make install
+ KSFT_INSTALL_PATH=$install_dir make install
}
main "$@"
--
2.20.1
Livepatch uses ftrace for redirection to new patched functions. It means
that if ftrace is disabled, all live patched functions are disabled as
well. Toggling global 'ftrace_enabled' sysctl thus affect it directly.
It is not a problem per se, because only administrator can set sysctl
values, but it still may be surprising.
Introduce PERMANENT ftrace_ops flag to amend this. If the
FTRACE_OPS_FL_PERMANENT is set on any ftrace ops, the tracing cannot be
disabled by disabling ftrace_enabled. Equally, a callback with the flag
set cannot be registered if ftrace_enabled is disabled.
v2->v3:
- ftrace_enabled explicitly set to true
- selftest from Joe Lawrence (I just split it to two patches)
- typo fix
v1->v2:
- different logic, proposed by Joe Lawrence
Joe Lawrence (2):
selftests/livepatch: Make dynamic debug setup and restore generic
selftests/livepatch: Test interaction with ftrace_enabled
Miroslav Benes (1):
ftrace: Introduce PERMANENT ftrace_ops flag
Documentation/trace/ftrace-uses.rst | 8 +++
Documentation/trace/ftrace.rst | 4 +-
include/linux/ftrace.h | 3 +
kernel/livepatch/patch.c | 3 +-
kernel/trace/ftrace.c | 23 ++++++-
tools/testing/selftests/livepatch/Makefile | 3 +-
.../testing/selftests/livepatch/functions.sh | 34 +++++++---
.../selftests/livepatch/test-callbacks.sh | 2 +-
.../selftests/livepatch/test-ftrace.sh | 65 +++++++++++++++++++
.../selftests/livepatch/test-livepatch.sh | 2 +-
.../selftests/livepatch/test-shadow-vars.sh | 2 +-
11 files changed, 132 insertions(+), 17 deletions(-)
create mode 100755 tools/testing/selftests/livepatch/test-ftrace.sh
--
2.23.0
On 2019-10-07 17:21:50, brendanhiggins(a)google.com wrote:
> On Fri, Oct 4, 2019 at 11:55 AM Konstantin Ryabitsev via RT
> <kernel-helpdesk(a)rt.linuxfoundation.org> wrote:
> >
> > On 2019-10-02 17:53:58, brendanhiggins(a)google.com wrote:
> > > Hi,
> > >
> > > I am thinking about requesting a Bugzilla component for my kernel
> > > project KUnit. I am not sure if this is the right place for it.
> > > Some
> > > background on KUnit: We are working on adding unit testing for the
> > > Linux
> > > kernel[1][2]. We have our initial patchset that introduces the
> > > subsystem
> > > in the process of being merged (Linus sent our PR back to us for a
> > > minor
> > > fix[3], so it should be in either 5.4-rc2 or 5.5, but is
> > > nevertheless
> > > in
> > > linux-next). However, we also have a staging repo that people are
> > > using
> > > and some supporting code that lives outside of the kernel.
> > >
> > > So I am trying to figure out:
> > >
> > > 1. Is it appropriate to request a Bugzilla component before our
> > > subsystem has been merged into torvalds/master? I would just
> > > wait,
> > > but I have some users looking to file issues, so I would prefer
> > > to
> > > provide them something sooner rather than later.
> > >
> > > 2. Is it appropriate to use the kernel's Bugzilla to track issues
> > > outside of the Linux kernel? As I mention above, we have code
> > > that
> > > lives outside of the kernel; is it appropriate to use
> > > kernel.org's
> > > Bugzilla for this?
> > >
> > > 3. Does Bugzilla match my planned usage model? It doesn't look like
> > > Bugzilla get's much usage aside from reporting bugs. I want to
> > > use
> > > it for tracking feature progress and things like that. Is that
> > > okay?
> >
> > Yes, we can certainly host this on bugzilla.kernel.org. Would you be
> > okay with Tools/KUnit as product/category?
>
> Cool, well as long as none of my above points are an issue for you.
> Then yes, can you create me a component? I am fine with Tools/KUnit as
> the product/category.
Apologies for the delay, mostly due to Thanksgiving in Canada. You should be able to start using Tools/KUnit now. It uses a default virtual assignee tools_kunit(a)kernel-bugs.kernel.org, so to start receiving bugmail for it, please follow instructions on this page:
https://korg.wiki.kernel.org/userdoc/bugzilla#to_start_getting_bug_reports_…
Best regards,
--
Konstantin Ryabitsev
Director, LF Projects IT
The Linux Foundation
"
On Wed, Oct 16, 2019 at 3:59 PM Prabhakar Kushwaha
<prabhakar.pkin(a)gmail.com> wrote:
>
> On Tue, Sep 24, 2019 at 12:56:53PM -0600, Shuah Khan wrote:
> > On 9/24/19 12:49 PM, Daniel Borkmann wrote:
> > > On Tue, Sep 24, 2019 at 09:48:35AM -0600, Shuah Khan wrote:
> > > > On 9/24/19 9:43 AM, Yonghong Song wrote:
> > > > > On 9/24/19 8:26 AM, Shuah Khan wrote:
> > > > > > Hi Alexei and Daniel,
> > > > > >
> > > > > > bpf test doesn't build on Linux 5.4 mainline. Do you know what's
> > > > > > happening here.
> > > > > >
> > > > > > make -C tools/testing/selftests/bpf/
> > > > > >
> > > > > > -c progs/test_core_reloc_ptr_as_arr.c -o - || echo "clang failed") | \
> > > > > > llc -march=bpf -mcpu=generic -filetype=obj -o
> > > > > > /mnt/data/lkml/linux_5.4/tools/testing/selftests/bpf/test_core_reloc_ptr_as_arr.o
> > > > > >
> > > > > > progs/test_core_reloc_ptr_as_arr.c:25:6: error: use of unknown builtin
> > > > > > '__builtin_preserve_access_index'
> > > > > > [-Wimplicit-function-declaration]
> > > > > > if (BPF_CORE_READ(&out->a, &in[2].a))
> > > > > > ^
> > > > > > ./bpf_helpers.h:533:10: note: expanded from macro 'BPF_CORE_READ'
> > > > > > __builtin_preserve_access_index(src))
> > > > > > ^
> > > > > > progs/test_core_reloc_ptr_as_arr.c:25:6: warning: incompatible
> > > > > > integer to
> > > > > > pointer conversion passing 'int' to parameter of type 'const
> > > > > > void *'
> > > > > > [-Wint-conversion]
> > > > > > if (BPF_CORE_READ(&out->a, &in[2].a))
> > > > > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > ./bpf_helpers.h:533:10: note: expanded from macro 'BPF_CORE_READ'
> > > > > > __builtin_preserve_access_index(src))
> > > > > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > 1 warning and 1 error generated.
> > > > > > llc: error: llc: <stdin>:1:1: error: expected top-level entity
> > > > > > clang failed
> > > > > >
> > > > > > Also
> > > > > >
> > > > > > make TARGETS=bpf kselftest fails as well. Dependency between
> > > > > > tools/lib/bpf and the test. How can we avoid this type of
> > > > > > dependency or resolve it in a way it doesn't result in build
> > > > > > failures?
> > > > >
> > > > > Thanks, Shuah.
> > > > >
> > > > > The clang __builtin_preserve_access_index() intrinsic is
> > > > > introduced in LLVM9 (which just released last week) and
> > > > > the builtin and other CO-RE features are only supported
> > > > > in LLVM10 (current development branch) with more bug fixes
> > > > > and added features.
> > > > >
> > > > > I think we should do a feature test for llvm version and only
> > > > > enable these tests when llvm version >= 10.
> > > >
> > > > Yes. If new tests depend on a particular llvm revision, the failing
> > > > the build is a regression. I would like to see older tests that don't
> > > > have dependency build and run.
> > >
> > > So far we haven't made it a requirement as majority of BPF contributors
> > > that would run/add tests in here are also on bleeding edge LLVM anyway
> > > and other CIs like 0-day bot have simply upgraded their LLVM version
> > > from git whenever there was a failure similar to the one here so its
> > > ensured that really /all/ test cases are running and nothing would be
> > > skipped. There is worry to some degree that CIs just keep sticking to
> > > an old compiler since tests "just" pass and regressions wouldn't be
> > > caught on new releases for those that are skipped. >
> >
> > Sure. Bleeding edge is developer mode. We still have to be concerned
> > about users that might not upgrade quickly.
> >
> > > That said, for the C based tests, it should actually be straight forward
> > > to categorize them based on built-in macros like ...
> > >
> > > $ echo | clang -dM -E -
> > > [...]
> > > #define __clang_major__ 10
> > > #define __clang_minor__ 0
> > > [...]
> >
> > What would nice running the tests you can run and then say some tests
> > aren't going to run. Is this something you can support?
>
> Once there is such infra in place, should be possible.
>
> > > ... given there is now also bpf-gcc, the test matrix gets bigger anyway,
> > > so it might be worth rethinking to run the suite multiple times with
> > > different major llvm{,gcc} versions at some point to make sure their
> > > generated BPF bytecode keeps passing the verifier, and yell loudly if
> > > newer features had to be skipped due to lack of recent compiler version.
> > > This would be a super set of /just/ skipping tests and improve coverage
> > > at the same time.
> >
> > Probably. Reality is most users will just quit and add bpf to "hard to
> > run category" of tests.
>
> I don't really worry too much about such users at this point, more important
> is that we have a way to test bpf-gcc and llvm behavior side by side to
> make sure behavior is consistent and to have some sort of automated CI
> integration that runs BPF kselftests before we even stare at a patch for
> review. These are right now the two highest prio items from BPF testing
> side where we need to get to.
>
I am also facing same issue with Linux-5.4 for ARM64 platforms.
I am using ubuntu-18.04 which has LLVM version 6.0.0. Looks like
there is no LLVM of version 9.0 or 10.0 available for ARM64.
https://apt.llvm.org/
I also tried "sudo ./llvm.sh 9". But it does not install llvm version.
root@ubuntu$ sudo ./llvm.sh 9
+ LLVM_VERSION=9
+ '[' 1 -eq 1 ']'
+ LLVM_VERSION=9
++ lsb_release -is
+ DISTRO=Ubuntu
++ lsb_release -sr
+ VERSION=18.04
+ DIST_VERSION=Ubuntu_18.04
+ [[ 0 -ne 0 ]]
+ declare -A LLVM_VERSION_PATTERNS
+ LLVM_VERSION_PATTERNS[8]=-8
+ LLVM_VERSION_PATTERNS[9]=-9
+ LLVM_VERSION_PATTERNS[10]=
+ '[' '!' _ ']'
+ LLVM_VERSION_STRING=-9
+ case "$DIST_VERSION" in
+ REPO_NAME='deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main'
+ wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key
+ apt-key add -
--2019-10-16 03:33:34-- https://apt.llvm.org/llvm-snapshot.gpg.key
Resolving apt.llvm.org (apt.llvm.org)... 151.101.190.49, 2a04:4e42:a::561
Connecting to apt.llvm.org (apt.llvm.org)|151.101.190.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3145 (3.1K) [application/octet-stream]
Saving to: ?STDOUT?
-
100%[====================================================================================================>]
3.07K --.-KB/s in 0s
2019-10-16 03:33:35 (74.8 MB/s) - written to stdout [3145/3145]
OK
+ add-apt-repository 'deb http://apt.llvm.org/bionic/
llvm-toolchain-bionic-9 main'
Hit:1 http://apt.llvm.org/bionic llvm-toolchain-bionic InRelease
Hit:2 http://apt.llvm.org/bionic llvm-toolchain-bionic-9 InRelease
Hit:3 http://apt.llvm.org/bionic llvm-toolchain-bionic-8 InRelease
Hit:4 http://us.ports.ubuntu.com/ubuntu-ports bionic InRelease
Hit:5 http://us.ports.ubuntu.com/ubuntu-ports bionic-updates InRelease
Hit:6 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease
Hit:7 http://us.ports.ubuntu.com/ubuntu-ports bionic-backports InRelease
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic
InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic-9
InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic-8
InRelease' doesn't support architecture 'arm64'
+ apt-get update
Hit:1 http://apt.llvm.org/bionic llvm-toolchain-bionic InRelease
Hit:2 http://apt.llvm.org/bionic llvm-toolchain-bionic-9 InRelease
Hit:3 http://apt.llvm.org/bionic llvm-toolchain-bionic-8 InRelease
Hit:4 http://us.ports.ubuntu.com/ubuntu-ports bionic InRelease
Hit:5 http://us.ports.ubuntu.com/ubuntu-ports bionic-updates InRelease
Hit:6 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease
Hit:7 http://us.ports.ubuntu.com/ubuntu-ports bionic-backports InRelease
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic
InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic-9
InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as
repository 'http://apt.llvm.org/bionic llvm-toolchain-bionic-8
InRelease' doesn't support architecture 'arm64'
+ apt-get install -y clang-9 lldb-9 lld-9 clangd-9
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package clang-9
E: Unable to locate package lldb-9
E: Unable to locate package lld-9
E: Unable to locate package clangd-9
--prabhakar(pk)