Hello.
This patch set updates clone3 selftest in several aspects:
- adding checks for exit_signal invalid values handling;
- adding clone3 to selftests targets;
- enabling clone3 tests on all architectures;
- minor cleanups of the clone3 test.
Applied on top of brauer/linux.git/for-next.
Changes since v1[1]:
- exit_signal check extended to cover more cases of invalid
exit_signal value.
[1] https://lkml.org/lkml/2019/9/10/416
Eugene Syromiatnikov (6):
selftests/clone3: convert test modes into an enum
selftests/clone3: add a check for invalid exit_signal
selftests/clone3: use uint64_t for flags parameter
selftests/clone3: fix up format strings
selftests/clone3: enable clone3 self-tests on all architectures
selftests: add clone3 to TARGETS
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/clone3/Makefile | 4 +--
tools/testing/selftests/clone3/clone3.c | 64 ++++++++++++++++++++++++++++-----
3 files changed, 57 insertions(+), 12 deletions(-)
--
2.1.4
Hi,friend,
This is Julian Smith and i am purchasing manager from E-cloth Co.,LTD in the UK.
We are glad to know about your company from the web and we are interested in your products.
Could you kindly send us your Latest catalog and price list for our trial order.
Thanks and Best Regards,
Ms Julian Smith
Purchasing Manager
E-cloth Co.,LTD
Packet capture is useful from a general debugging standpoint, and is
useful in particular in debugging BPF programs that do packet processing.
For general debugging, being able to initiate arbitrary packet capture
from kprobes and tracepoints is highly valuable; e.g. what do the packets
that reach kfree_skb() - representing error codepaths - look like?
Arbitrary packet capture is distinct from the traditional concept of
pre-defined hooks, and gives much more flexibility in probing system
behaviour. For packet-processing BPF programs, packet capture can be useful
for doing things such as debugging checksum errors.
The intent of this RFC patchset is to initiate discussion on if and how to
work packet capture-specific capabilities into BPF. It is possible -
and indeed projects like xdpcap [1] have demonstrated how - to carry out
packet capture in BPF today via perf events, but the aim here is to
simplify both the in-BPF capture and the userspace collection.
The suggested approach is to add a new bpf helper - bpf_pcap() - to
simplify packet capture within BPF programs, and to enhance bpftool
to add a "pcap" subcommand to aid in retrieving packets. The helper
is for the most part a wrapper around perf event sending, using
data relevant for packet capture as metadata.
The end result is being able to capture packet data in the following
manner. For example if we add an iptables drop rule, we can observe
TCP SYN segments being freed at kfree_skb:
$ iptables -A INPUT -p tcp --dport 6666 -j DROP
$ bpftool pcap trace kprobe:kfree_skb proto ip data_out /tmp/cap &
$ nc 127.0.0.1 6666
Ncat: Connection timed out.
$ fg
^C
$ tshark -r /tmp/cap
Running as user "root" and group "root". This could be dangerous.
...
3 7 127.0.0.1 -> 127.0.0.1 TCP 60 54732 > ircu [SYN] Seq=0 Win=65495 Len=0 MSS=65495 SACK_PERM=1 TSval=696475539 TSecr=0 WS=128
...
Tracepoints are also supported, and by default data is sent to
stdout, so we can pipe to tcpdump:
$ bpftool pcap trace tracepoint:net_dev_xmit:arg1 proto eth | tcpdump -r -
reading from file -, link-type EN10MB (Ethernet)
00:16:49.150880 IP 10.11.12.13 > 10.11.12.14: ICMP echo reply, id 10519, seq 1, length 64
...
Patch 1 adds support for bpf_pcap() in skb and XDP programs. In those cases,
the argument is the relevant context (struct __sk_buff or xdp metadata)
from which we capture.
Patch 2 extends the helper to allow it to work for tracing programs, and in
that case the data argument is a pointer to an skb, derived from raw
tracepoint or kprobe arguments.
Patch 3 syncs uapi and tools headers for the new helper, flags and associated
pcap header type.
Patch 4 adds a feature test for libpcap which will be used in the next patch.
Patch 5 adds a "pcap" subcommand to bpftool to collect packet data from
BPF-driven perf event maps in existing programs. Also supplied are simple
tracepoint and kprobe programs which can be used to attach to a kprobe
or raw tracepoint to retrieve arguments and capture the associated skb.
Patch 6 adds documentation for the new pcap subcommand.
Patch 7 tests the pcap subcommand for tracing, skb and xdp programs.
Alan Maguire (7):
bpf: add bpf_pcap() helper to simplify packet capture
bpf: extend bpf_pcap support to tracing programs
bpf: sync tools/include/uapi/linux/bpf.h for pcap support
bpf: add libpcap feature test
bpf: add pcap support to bpftool
bpf: add documentation for bpftool pcap subcommand
bpf: add tests for bpftool packet capture
include/linux/bpf.h | 20 +
include/uapi/linux/bpf.h | 92 +++-
kernel/bpf/verifier.c | 4 +-
kernel/trace/bpf_trace.c | 214 +++++++++
net/core/filter.c | 67 +++
tools/bpf/bpftool/Documentation/bpftool-btf.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 1 +
.../bpf/bpftool/Documentation/bpftool-feature.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-map.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-net.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-pcap.rst | 119 +++++
tools/bpf/bpftool/Documentation/bpftool-perf.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-prog.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool.rst | 1 +
tools/bpf/bpftool/Makefile | 39 +-
tools/bpf/bpftool/main.c | 3 +-
tools/bpf/bpftool/main.h | 1 +
tools/bpf/bpftool/pcap.c | 496 +++++++++++++++++++++
tools/bpf/bpftool/progs/bpftool_pcap_kprobe.c | 80 ++++
tools/bpf/bpftool/progs/bpftool_pcap_tracepoint.c | 68 +++
tools/build/Makefile.feature | 2 +
tools/build/feature/Makefile | 4 +
tools/build/feature/test-libpcap.c | 26 ++
tools/include/uapi/linux/bpf.h | 92 +++-
tools/testing/selftests/bpf/Makefile | 3 +-
tools/testing/selftests/bpf/bpf_helpers.h | 11 +
.../testing/selftests/bpf/progs/bpftool_pcap_tc.c | 41 ++
.../testing/selftests/bpf/progs/bpftool_pcap_xdp.c | 39 ++
tools/testing/selftests/bpf/test_bpftool_pcap.sh | 132 ++++++
29 files changed, 1549 insertions(+), 12 deletions(-)
create mode 100644 tools/bpf/bpftool/Documentation/bpftool-pcap.rst
create mode 100644 tools/bpf/bpftool/pcap.c
create mode 100644 tools/bpf/bpftool/progs/bpftool_pcap_kprobe.c
create mode 100644 tools/bpf/bpftool/progs/bpftool_pcap_tracepoint.c
create mode 100644 tools/build/feature/test-libpcap.c
create mode 100644 tools/testing/selftests/bpf/progs/bpftool_pcap_tc.c
create mode 100644 tools/testing/selftests/bpf/progs/bpftool_pcap_xdp.c
create mode 100755 tools/testing/selftests/bpf/test_bpftool_pcap.sh
--
1.8.3.1
KTF has already been available for a while as a separate git repository with
means to facilitate use with any kernel version.
KTF can be used both for "pure" unit testing and for more pragmatic
approaches to component testing. Apart from some useful features that
KTF uses from the kernel toolbox (such as kallsyms, kprobes),
KTF does not depend on any special environment such as UML or on a
large set of mocked up APIs to be useful. KTF basically allows test
code to be inserted and managed as separate kernel modules, while
providing the tests convenient access to almost the full range of kernel
APIs, both exposed and private. And once a KTF test set exists, it
should be fairly easy to compile and run it against older versions of
the kernel as well.
This series proposes a non-intrusive integration of KTF into the kernel
hopefully presented in digestable pieces.
For convenience, the patch set is also available on top of v5.2
at https://github.com/knuto/linux/pull/new/ktf_v1 .
The high level structure of the KTF code is as follows:
External dependencies for the user land side:
* libnl3 for netlink communication
* googletest for test runner, reporting and test selection support.
Kernel components:
* Simple core test and test suite related abstractions:
core data structures ktf_case, ktf_test, an assertion macro "infrastructure"
with ASSERT_* and EXPECT_* macros and helper functions
* Bookkeeping data structures:
- ktf_map - a (key, value) mapping container used to implement management
of instances of the higher level abstraction needs, such as ktf_handle and ktf_context.
- ktf_handle: A global environment that a test runs within.
- ktf_context: a test suite specific abstraction to allow a test to execute within
one or more contexts. An example use of context can be a device
instance: A test can be instantiated to run on all available such
devices, according to test suite defined criteria.
* A generic netlink protocol (ktf_nl, ktf_unlproto) which defines operations to
allow a user space part of a test to query the kernel for test information,
invoke tests and get feedback reports about results.
* An alternative debugfs interface to allow examining and executing kernel-only tests
without a user level program.
* Support for overriding and modifying behaviour of kernel calls.
User mode components:
* A test executor based on and integrated with Googletest.
Googletest is one of several mature user land unit test suites for
C/C++. The choice allowed us to focus on kernel specific
functionality rather than having to reinvent too many wheels.
* Tools to aid in creating new test modules (suites):
To facilitate a developer friendly way of testing internals of a module or the
kernel itself, one of the important features of KTF, we often need to access
symbols deliberately not exposed from a module.
KTF contains a script used to create definitions based on kallsyms
lookup for easy access to symbols not exposed by a module or the kernel.
The user just provides a simple text file with a list of the symbols by
module.
This series is an attempt to address feedback from several people
that having the functionality readily available within the kernel repository
is desired.
An in-tree KTF allows test suites to be provided with the kernel, which makes
it easy to use KTF based suites as part of the kernel selftests and driver
test suites. Having the ability to still build and run the latest versions of
test suites against older kernels should be of great value to stable maintainers,
distributions and driver maintainers, who would want to have an easy path,
with minimal backporting efforts to make sure that criterias implemented by
new test logic is also valid for these kernels.
Our definite goal moving forward is to try to satisfy both needs in a
transparent way. The plan is to let the standalone KTF repository follow the
in-kernel one, and to allow test suites to be maintained similarly,
and to support maintenance by proper tooling.
Mode of integration into the kernel
===================================
One feature of KTF is that it allows tests to work, look and feel similar
whether they execute entirely in user mode, entirely in kernel mode,
or half and half (hybrid tests). KTF consist of both user space
and kernel code. Unlike e.g. kselftest, KTF in the Github version
does not attempt to address the test runner aspects of testing.
Due to the need for building modules, KTF requires access to kernel module
build facilities (obj-m). But KTF also has nontrivial needs for user
land building, and we think it is good to keep the build structure in a way that
allows KTF to be built both in-tree and out-of-tree without
necessarily having to reconfigure the kernel.
This first version of kernel integration of KTF solves this challenge
by co-locating everything associated with KTF under ktf/ as in the
github version, but use the little used hostprogs-y and hostlibs-y
features to build the user space side. The first patch in the series is
fixes to make it work in a natural way to suit our needs.
Positioning for natural building within the kernel tree
=======================================================
Currently we find significant amount of C level tests within the following paths:
tools/testing/selftests/ (kselftests, almost entirely user space so far)
lib/ (various kernel level mostly unit tests)
and in the making::
kunit/ (kernel only (UML))
So all kernel code are currently located directly within the kernel
build paths, accessed from the top level Makefile, to allow everything
to be controlled by config and from the main build targets for the
kernel. But this also poses challenges, in that .config has to be
modified to build tests. And once a .config is changed, we no longer
in principle logically operate on the same kernel.
A better approach seems to be to follow the principle
taken by kselftest: To have all the logic associated with the test
inside the test tree, and make it available for building separately
from the kernel itself. This require us to have a means to build
kernel modules from within the test tree, separately from the main
kernel paths. Currently this seems to only by supported via the M=
option used to build out-of-tree modules. This was also easy to get to work
for the kernel parts, based on the Github version of KTF, where we
already do this. With the additional need to compile user land code,
using the corresponding hostprogs-y and hostNNlibs-y seemed natural,
but this has been challenging: The build macros does not really
support hostprogs-y etc as "first class citizens" so some amount of
hacking is in there in this first draft version.
Using hostprogs-y etc is also a different approach that what is used
for C code in kselftest today, but we imagine that there's room for
unification here to get the best of both worlds, with the help of
the wider Kbuild community.
As an initial proposal, we have positioned ktf as an additional
kselftest target, under tools/testing/selftests/ktf, and the recommended:
make TARGETS="ktf" kselftest
way of building and running should work, even from normal user accounts,
if the user running it has sudo rights (for the kernel module insertion
and removal). This will run the selftests for KTF itself, and should
be a good starting point for adding more test cases. We also have had
activities going to take some of the existing test suites under lib/
and convert them into KTF based test suites, and will get back to this
later.
A trimmed down output from the above make target would look like this:
...
CC [M] tools/testing/selftests/ktf/kernel/ktf_override.o
LD [M] tools/testing/selftests/ktf/kernel/ktf.o
HOSTCC -fPIC tools/testing/selftests/ktf/lib/ktf_unlproto.o
HOSTCXX -fPIC tools/testing/selftests/ktf/lib/ktf_int.o
KTFSYMS tools/testing/selftests/ktf/selftest/ktf_syms.h
CC [M] tools/testing/selftests/ktf/selftest/self.o
LD [M] tools/testing/selftests/ktf/selftest/selftest.o
HOSTCXX tools/testing/selftests/ktf/user/ktftest.o
HOSTCXX tools/testing/selftests/ktf/user/hybrid.o
HOSTLD tools/testing/selftests/ktf/user/ktftest
Building modules, stage 2.
MODPOST 7 modules
LD [M] tools/testing/selftests/ktf/kernel/ktf.ko
LD [M] tools/testing/selftests/ktf/selftest/selftest.ko
running tests
make BUILD=/net/abi/local/abi/build/kernel/ktf/tools/testing/selftests -f scripts/runtests.mk run_tests
TAP version 13
1..1
...
ok 1 selftests: ktf: runtests.sh
We're looking forward to feedback on this, and also to more discussion
around unit testing at the testing & fuzzing workshop at LPC!
Alan Maguire (3):
ktf: Implementation of ktf support for overriding function entry and return.
ktf: A simple debugfs interface to test results
ktf: Simple coverage support
Knut Omang (16):
kbuild: Fixes to rules for host-cshlib and host-cxxshlib
ktf: Introduce the main part of the kernel side of ktf
ktf: Introduce a generic netlink protocol for test result communication
ktf: An implementation of a generic associative array container
ktf: Configurable context support for network info setup
ktf: resolve: A helper utility to aid in exposing private kernel symbols to KTF tests.
ktf: Add documentation for Kernel Test Framework (KTF)
ktf: Add a small test suite with a few tests to test KTF itself
ktf: Main part of user land library for executing tests
ktf: Integration logic for running ktf tests from googletest
ktf: Internal debugging facilities
ktf: Some simple examples
ktf: Some user applications to run tests
ktf: Toplevel ktf Makefile/makefile includes and scripts to run from kselftest
kselftests: Enable building ktf
Documentation/dev-tools: Add index entry for KTF documentation
Documentation/dev-tools/index.rst | 1 +-
Documentation/dev-tools/ktf/concepts.rst | 242 +++-
Documentation/dev-tools/ktf/debugging.rst | 248 +++-
Documentation/dev-tools/ktf/examples.rst | 26 +-
Documentation/dev-tools/ktf/features.rst | 307 ++++-
Documentation/dev-tools/ktf/implementation.rst | 70 +-
Documentation/dev-tools/ktf/index.rst | 14 +-
Documentation/dev-tools/ktf/installation.rst | 73 +-
Documentation/dev-tools/ktf/introduction.rst | 134 ++-
Documentation/dev-tools/ktf/progref.rst | 144 ++-
scripts/Makefile.host | 17 +-
tools/testing/selftests/Makefile | 1 +-
tools/testing/selftests/ktf/Makefile | 21 +-
tools/testing/selftests/ktf/examples/Makefile | 17 +-
tools/testing/selftests/ktf/examples/h2.c | 45 +-
tools/testing/selftests/ktf/examples/h3.c | 84 +-
tools/testing/selftests/ktf/examples/h4.c | 62 +-
tools/testing/selftests/ktf/examples/hello.c | 38 +-
tools/testing/selftests/ktf/examples/kgdemo.c | 61 +-
tools/testing/selftests/ktf/kernel/Makefile | 15 +-
tools/testing/selftests/ktf/kernel/ktf.h | 604 +++++++-
tools/testing/selftests/ktf/kernel/ktf_context.c | 409 +++++-
tools/testing/selftests/ktf/kernel/ktf_cov.c | 690 ++++++++-
tools/testing/selftests/ktf/kernel/ktf_cov.h | 94 +-
tools/testing/selftests/ktf/kernel/ktf_debugfs.c | 356 ++++-
tools/testing/selftests/ktf/kernel/ktf_debugfs.h | 34 +-
tools/testing/selftests/ktf/kernel/ktf_map.c | 261 +++-
tools/testing/selftests/ktf/kernel/ktf_map.h | 154 ++-
tools/testing/selftests/ktf/kernel/ktf_netctx.c | 132 ++-
tools/testing/selftests/ktf/kernel/ktf_netctx.h | 64 +-
tools/testing/selftests/ktf/kernel/ktf_nl.c | 516 ++++++-
tools/testing/selftests/ktf/kernel/ktf_nl.h | 15 +-
tools/testing/selftests/ktf/kernel/ktf_override.c | 45 +-
tools/testing/selftests/ktf/kernel/ktf_override.h | 15 +-
tools/testing/selftests/ktf/kernel/ktf_test.c | 397 +++++-
tools/testing/selftests/ktf/kernel/ktf_test.h | 381 ++++-
tools/testing/selftests/ktf/kernel/ktf_unlproto.h | 105 +-
tools/testing/selftests/ktf/lib/Makefile | 21 +-
tools/testing/selftests/ktf/lib/ktf.h | 114 +-
tools/testing/selftests/ktf/lib/ktf_debug.cc | 20 +-
tools/testing/selftests/ktf/lib/ktf_debug.h | 59 +-
tools/testing/selftests/ktf/lib/ktf_int.cc | 1031 ++++++++++++-
tools/testing/selftests/ktf/lib/ktf_int.h | 84 +-
tools/testing/selftests/ktf/lib/ktf_run.cc | 177 ++-
tools/testing/selftests/ktf/lib/ktf_unlproto.c | 21 +-
tools/testing/selftests/ktf/scripts/ktf_syms.mk | 16 +-
tools/testing/selftests/ktf/scripts/resolve | 188 ++-
tools/testing/selftests/ktf/scripts/runtests.mk | 3 +-
tools/testing/selftests/ktf/scripts/runtests.sh | 100 +-
tools/testing/selftests/ktf/scripts/top_make.mk | 14 +-
tools/testing/selftests/ktf/selftest/Makefile | 17 +-
tools/testing/selftests/ktf/selftest/context.c | 149 ++-
tools/testing/selftests/ktf/selftest/context.h | 15 +-
tools/testing/selftests/ktf/selftest/context_self.h | 34 +-
tools/testing/selftests/ktf/selftest/hybrid.c | 35 +-
tools/testing/selftests/ktf/selftest/hybrid.h | 24 +-
tools/testing/selftests/ktf/selftest/hybrid_self.h | 27 +-
tools/testing/selftests/ktf/selftest/ktf_syms.txt | 17 +-
tools/testing/selftests/ktf/selftest/self.c | 661 ++++++++-
tools/testing/selftests/ktf/user/Makefile | 26 +-
tools/testing/selftests/ktf/user/hybrid.cc | 39 +-
tools/testing/selftests/ktf/user/ktfcov.cc | 68 +-
tools/testing/selftests/ktf/user/ktfrun.cc | 20 +-
tools/testing/selftests/ktf/user/ktftest.cc | 46 +-
64 files changed, 8909 insertions(+), 9 deletions(-)
create mode 100644 Documentation/dev-tools/ktf/concepts.rst
create mode 100644 Documentation/dev-tools/ktf/debugging.rst
create mode 100644 Documentation/dev-tools/ktf/examples.rst
create mode 100644 Documentation/dev-tools/ktf/features.rst
create mode 100644 Documentation/dev-tools/ktf/implementation.rst
create mode 100644 Documentation/dev-tools/ktf/index.rst
create mode 100644 Documentation/dev-tools/ktf/installation.rst
create mode 100644 Documentation/dev-tools/ktf/introduction.rst
create mode 100644 Documentation/dev-tools/ktf/progref.rst
create mode 100644 tools/testing/selftests/ktf/Makefile
create mode 100644 tools/testing/selftests/ktf/examples/Makefile
create mode 100644 tools/testing/selftests/ktf/examples/h2.c
create mode 100644 tools/testing/selftests/ktf/examples/h3.c
create mode 100644 tools/testing/selftests/ktf/examples/h4.c
create mode 100644 tools/testing/selftests/ktf/examples/hello.c
create mode 100644 tools/testing/selftests/ktf/examples/kgdemo.c
create mode 100644 tools/testing/selftests/ktf/kernel/Makefile
create mode 100644 tools/testing/selftests/ktf/kernel/ktf.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_context.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_cov.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_cov.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_debugfs.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_debugfs.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_map.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_map.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_netctx.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_netctx.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_nl.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_nl.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_override.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_override.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_test.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_test.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_unlproto.h
create mode 100644 tools/testing/selftests/ktf/lib/Makefile
create mode 100644 tools/testing/selftests/ktf/lib/ktf.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_debug.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_debug.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_int.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_int.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_run.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_unlproto.c
create mode 100644 tools/testing/selftests/ktf/scripts/ktf_syms.mk
create mode 100755 tools/testing/selftests/ktf/scripts/resolve
create mode 100644 tools/testing/selftests/ktf/scripts/runtests.mk
create mode 100755 tools/testing/selftests/ktf/scripts/runtests.sh
create mode 100644 tools/testing/selftests/ktf/scripts/top_make.mk
create mode 100644 tools/testing/selftests/ktf/selftest/Makefile
create mode 100644 tools/testing/selftests/ktf/selftest/context.c
create mode 100644 tools/testing/selftests/ktf/selftest/context.h
create mode 100644 tools/testing/selftests/ktf/selftest/context_self.h
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid.c
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid.h
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid_self.h
create mode 100644 tools/testing/selftests/ktf/selftest/ktf_syms.txt
create mode 100644 tools/testing/selftests/ktf/selftest/self.c
create mode 100644 tools/testing/selftests/ktf/user/Makefile
create mode 100644 tools/testing/selftests/ktf/user/hybrid.cc
create mode 100644 tools/testing/selftests/ktf/user/ktfcov.cc
create mode 100644 tools/testing/selftests/ktf/user/ktfrun.cc
create mode 100644 tools/testing/selftests/ktf/user/ktftest.cc
base-commit: 0ecfebd2b52404ae0c54a878c872bb93363ada36
--
git-series 0.9.1
The vprintk_emit() function is not available when CONFIG_PRINTK
is disabled:
kunit/test.c:22:9: error: implicit declaration of function 'vprintk_emit' [-Werror,-Wimplicit-function-declaration]
I suppose without printk(), there is not much use in kunit
either, so add a Kconfig depenedency here.
Signed-off-by: Arnd Bergmann <arnd(a)arndb.de>
---
kunit/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/kunit/Kconfig b/kunit/Kconfig
index 8541ef95b65a..e80d8af00454 100644
--- a/kunit/Kconfig
+++ b/kunit/Kconfig
@@ -6,6 +6,7 @@ menu "KUnit support"
config KUNIT
bool "Enable support for unit tests (KUnit)"
+ depends on PRINTK
help
Enables support for kernel unit tests (KUnit), a lightweight unit
testing and mocking framework for the Linux kernel. These tests are
--
2.20.0
The current logic prepends $(OUTPUT) only to the first member of
$(TEST_PROGS). Use $(foreach) loop to prepend it to each member.
Fixes: 1a940687e424 ("selftests: lib.mk: copy test scripts and test files for make O=dir run")
Signed-off-by: Ilya Leoshkevich <iii(a)linux.ibm.com>
---
tools/testing/selftests/lib.mk | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 1c8a1963d03f..857916ebbb9b 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -75,7 +75,8 @@ ifdef building_out_of_srctree
@rsync -aq $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT)
fi
@if [ "X$(TEST_PROGS)" != "X" ]; then
- $(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(OUTPUT)/$(TEST_PROGS))
+ $(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) \
+ $(foreach p,$(TEST_PROGS),$(OUTPUT)$(p)))
else
$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS))
fi
--
2.21.0
tags_test.c relies on PR_SET_TAGGED_ADDR_CTRL/PR_TAGGED_ADDR_ENABLE being
present in system headers. When this is not the case the build of this
test fails with undeclared identifier errors.
Fix by providing the path to the KSFT installed kernel headers in CFLAGS.
Reported-by: Cristian Marussi <cristian.marussi(a)arm.com>
Suggested-by: Cristian Marussi <cristian.marussi(a)arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl(a)google.com>
---
tools/testing/selftests/arm64/Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index a61b2e743e99..f9f79fb272f0 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -4,6 +4,7 @@
ARCH ?= $(shell uname -m 2>/dev/null || echo not)
ifneq (,$(filter $(ARCH),aarch64 arm64))
+CFLAGS += -I../../../../usr/include/
TEST_GEN_PROGS := tags_test
TEST_PROGS := run_tags_test.sh
endif
--
2.23.0.187.g17f5b7556c-goog
Problem:
Currently tasks attempting to allocate more hugetlb memory than is available get
a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1].
However, if a task attempts to allocate hugetlb memory only more than its
hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call,
but will SIGBUS the task when it attempts to fault the memory in.
We have developers interested in using hugetlb_cgroups, and they have expressed
dissatisfaction regarding this behavior. We'd like to improve this
behavior such that tasks violating the hugetlb_cgroup limits get an error on
mmap/shmget time, rather than getting SIGBUS'd when they try to fault
the excess memory in.
The underlying problem is that today's hugetlb_cgroup accounting happens
at hugetlb memory *fault* time, rather than at *reservation* time.
Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and
the offending task gets SIGBUS'd.
Proposed Solution:
A new page counter named hugetlb.xMB.reservation_[limit|usage]_in_bytes. This
counter has slightly different semantics than
hugetlb.xMB.[limit|usage]_in_bytes:
- While usage_in_bytes tracks all *faulted* hugetlb memory,
reservation_usage_in_bytes tracks all *reserved* hugetlb memory.
- If a task attempts to reserve more memory than limit_in_bytes allows,
the kernel will allow it to do so. But if a task attempts to reserve
more memory than reservation_limit_in_bytes, the kernel will fail this
reservation.
This proposal is implemented in this patch, with tests to verify
functionality and show the usage.
Alternatives considered:
1. A new cgroup, instead of only a new page_counter attached to
the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code
duplication with hugetlb_cgroup. Keeping hugetlb related page counters under
hugetlb_cgroup seemed cleaner as well.
2. Instead of adding a new counter, we considered adding a sysctl that modifies
the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at
reservation time rather than fault time. Adding a new page_counter seems
better as userspace could, if it wants, choose to enforce different cgroups
differently: one via limit_in_bytes, and another via
reservation_limit_in_bytes. This could be very useful if you're
transitioning how hugetlb memory is partitioned on your system one
cgroup at a time, for example. Also, someone may find usage for both
limit_in_bytes and reservation_limit_in_bytes concurrently, and this
approach gives them the option to do so.
Caveats:
1. This support is implemented for cgroups-v1. I have not tried
hugetlb_cgroups with cgroups v2, and AFAICT it's not supported yet.
This is largely because we use cgroups-v1 for now. If required, I
can add hugetlb_cgroup support to cgroups v2 in this patch or
a follow up.
2. Most complicated bit of this patch I believe is: where to store the
pointer to the hugetlb_cgroup to uncharge at unreservation time?
Normally the cgroup pointers hang off the struct page. But, with
hugetlb_cgroup reservations, one task can reserve a specific page and another
task may fault it in (I believe), so storing the pointer in struct
page is not appropriate. Proposed approach here is to store the pointer in
the resv_map. See patch for details.
Signed-off-by: Mina Almasry <almasrymina(a)google.com>
[1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html
Changes in v3:
- Addressed comments of Hillf Danton:
- Added docs.
- cgroup_files now uses enum.
- Various readability improvements.
- Addressed comments of Mike Kravetz.
- region_* functions no longer coalesce file_region entries in the resv_map.
- region_add() and region_chg() refactored to make them much easier to
understand and remove duplicated code so this patch doesn't add too much
complexity.
- Refactored common functionality into helpers.
Changes in v2:
- Split the patch into a 5 patch series.
- Fixed patch subject.
Mina Almasry (6):
hugetlb_cgroup: Add hugetlb_cgroup reservation counter
hugetlb_cgroup: add interface for charge/uncharge hugetlb reservations
hugetlb_cgroup: add reservation accounting for private mappings
hugetlb_cgroup: add accounting for shared mappings
hugetlb_cgroup: Add hugetlb_cgroup reservation tests
hugetlb_cgroup: Add hugetlb_cgroup reservation docs
.../admin-guide/cgroup-v1/hugetlb.rst | 84 ++-
include/linux/hugetlb.h | 24 +-
include/linux/hugetlb_cgroup.h | 19 +-
mm/hugetlb.c | 493 ++++++++++++------
mm/hugetlb_cgroup.c | 187 +++++--
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 4 +
.../selftests/vm/charge_reserved_hugetlb.sh | 438 ++++++++++++++++
.../selftests/vm/write_hugetlb_memory.sh | 22 +
.../testing/selftests/vm/write_to_hugetlbfs.c | 252 +++++++++
10 files changed, 1300 insertions(+), 224 deletions(-)
create mode 100755 tools/testing/selftests/vm/charge_reserved_hugetlb.sh
create mode 100644 tools/testing/selftests/vm/write_hugetlb_memory.sh
create mode 100644 tools/testing/selftests/vm/write_to_hugetlbfs.c
--
2.23.0.187.g17f5b7556c-goog