TL;DR: We expect KUnit to be accepted upstream into Linus' branch in
the next week or two, and we now need to figure out what we are going
to do with our non-upstream 'kunit/alpha/master' branch.
Hello everyone,
We've put together a rough proposal of what we should do with our
not-upstream branch, known to most people using it as
'kunit/alpha/master'[1], now that KUnit's acceptance into mainline
appears to be imminent (the KUnit MVP patchset is now in linux-next,
and the merge window just opened).
==========
Background
==========
KUnit development is currently split between two versions: the
'kunit/alpha/master'[1] git branch, and the version being submitted to
the upstream Linux kernel. While there are some good reasons to
continue to have two separate versions of KUnit, at present there is
some uncertainty around the difference between these versions, and in
which circumstances each version is useful.
At present, the 'kunit/alpha/master' branch serves a few different
purposes. It is a place for code not-yet-ready for upstream -- such as
the mocking framework -- while being developed, while also acting as a
stable version for customers who do not wish to follow along with the
changes made during the upstreaming process. Adding to the confusion,
the name 'kunit/alpha/master' refers to an early (alpha) version of
KUnit, and the version of KUnit being upstreamed has now diverged
significantly from this version, requiring significant differences in
documentation, and requiring a number of changes to tests when porting
from one version to the other. Finally, it is not clear how the
'kunit/alpha/master' version should evolve as features it contains are
upstreamed.
On the other hand, the version being upstreamed has its own
complications. It contains significantly fewer features (as features
such as the mocking frameworks will be upstreamed individually), and
so is less useful for the average customer. Until each feature is
upstreamed, it is iterated on rapidly to address comments from the
kernel community, so in-progress features are not stable enough to
reasonably build on. Finally, it exists only as a set of patches on
mailing lists, rather than as a maintained git repository (due to the
fact that the patches themselves are changing rapidly), making it
difficult for early adopters to incorporate into their own trees.
Whilst we believe there to be enough (at times conflicting) goals
above to justify having multiple versions of KUnit, we want to ensure
that they are meeting their goals, and that we have a process to
ensure that code finds its way into the correct version, that we can
deprecate and remove failed experiments or superseded versions, and
that we can keep pace with upstream kernel releases.
============
The Proposal
============
We propose having two tracks of development: the upstream kernel
(comprising both code that has been upstreamed, and code which is in
the process of being upstreamed -- i.e. is being reviewed on the
mailing lists), and an 'experimental' branch, which contains features
which are yet to be submitted upstream.
Periodically, the 'experimental' branch will be replaced with a new
'experimental' branch based on the latest upstream version. This will
allow us to focus on code based on upstream versions by deprecating
earlier, experimental versions of features. The older experimental
branch will stay around, but receive no new features or support:
customers will need to eventually migrate to either the upstream
kernel (if all of the features they require have been upstreamed), or
to the latest 'experimental' branch (if they still require
non-upstreamed features).
Where practical, 'experimental' branches will try to provide a stable
interface (and/or compatibility layers/shims) such that users can
write tests against an experimental version and expect our changes to
break them infrequently. We don't intend to make any solid guarantee,
however: so users will need to be able to deal with some breakage.
We expect most changes to KUnit to be submitted directly upstream, and
reserve the 'experimental' branch for features where:
a) the design is yet to be proven by early adopters, and is likely to change;
b) the feature is large enough that having a solid staging environment
in which several developers can work is necessary; and/or
c) it depends on existing features only available in the 'experimental' branch.
An example of features suited to the 'experimental' branch are the
mocking systems[2].
=====================
Lifetime of a feature
=====================
A new feature (that for one of the above reasons may require some
experimentation before being sent upstream) would typically be
developed as follows:
1. An initial version of the feature is developed against the latest
'experimental' branch.
2. The feature is tested by users of the 'experimental' branch.
3. It is revised as needed within the 'experimental' branch. If
breaking changes are required, and supporting some form of
compatibility layer is practical, such a layer is implemented to avoid
breaking existing 'experimental' customers.
4. Once the feature is deemed ready, it is submitted to the mailing
lists for the upstream kernel. Any compatibility layers for earlier
experimental version are removed -- this is a 'clean' version of the
change.
5. Development of the change continues on the kernel mailing lists:
any particularly important changes or bugfixes are ported to the
'experimental' branch if feasible, but complete convergence is not
expected at this stage.
6. When the feature is accepted upstream, a new 'experimental' branch
is created based on the new upstream kernel. Development on the old
'experimental' branch ceases.
7. The upstream version of the feature has now superseded the
experimental version. All future development is done based on the
upstream version, either in the new 'experimental' branch, or directly
against the upstream kernel.
8. Any customers still using the experimental version of the feature
will need to update their code to work with the upstream version in
order to update to the new (supported) 'experimental' branch, or to
the upstream kernel.
===========================
What does this mean for me?
===========================
...as a user?
-------------
If you're using the upstream version of KUnit, nothing! Features will
continue to be pushed upstream when they're ready.
If you're using the 'kunit/alpha/master' branch, then you'll need to
switch to the new 'experimental-<version>' branches. Every so often,
we'll create a new 'experimental-<version>' branch, and you'll need to
rebase your code on top of that to keep receiving updated features and
bugfixes. While we intend to make this as painless as possible, there
may be breaking changes which require you to rework some of your
tests.
Once a feature exists in the upstream kernel, the version of that
feature in the 'experimental' branches should stabilise somewhat.
...as a KUnit developer?
------------------------
For most changes/bugfixes which apply cleanly to the upstream version,
you should directly submit these upstream as before. If it is feasible
and useful to backport them to the latest 'experimental' branch (e.g.,
they are bugfixes to code that the experimental branch already has),
please do so.
For more major features (or anything which depends on
yet-to-be-upstreamed code), you should develop them in the latest
'experimental' branch. For the most part, you can treat this the same
way as the old 'kunit/alpha/master' branch, but note that changes will
need to be rebased on top of a newer experimental branch periodically.
It may make sense to use this as an opportunity to tidy the code up
somewhat: by removing compatibility code for deprecated features, or
to merge 'fixup' commits. Once the code has been upstreamed, the next
rebase of the 'experimental' branch should only use the upstream
version: there should not be a separate 'experimental' version of
upstream code save for the development of a new 'experimental' feature
or bugfix.
For more details, see the 'Lifetime of a feature' section above.
=============
In Conclusion
=============
We'll continue to be working with upstream as before, but we intend to
rename and more explicitly specify the purpose of the
'kunit/alpha/master' branch as above. This will require a bit more
work from people using and developing against this 'experimental'
branch, but allows us to continue to use it for features such as the
mocking framework while limiting its divergence from upstream.
Do let us know if this sounds reasonable to you, if it will affect
your use case, or if you have any questions.
Cheers,
-- David & Brendan
[1] https://kunit.googlesource.com/linux/+/refs/heads/kunit/alpha/master
[2] https://kunit.googlesource.com/linux/+/kunit/alpha/master/Documentation/tes…
## TL;DR
This revision addresses comments from Linus[1] and Randy[2], by moving
top level `kunit/` directory to `lib/kunit/` and likewise moves top
level Kconfig entry under lib/Kconfig.debug, so the KUnit submenu now
shows up under the "Kernel Hacking" menu.
As a consequence of this, I rewrote patch 06/18 (kbuild: enable building
KUnit), and now needs to be re-acked/reviewed.
## Background
This patch set proposes KUnit, a lightweight unit testing and mocking
framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework;
it does not require installing the kernel on a test machine or in a VM
(however, KUnit still allows you to run tests on test machines or in VMs
if you want[3]) and does not require tests to be written in userspace
running on a host kernel. Additionally, KUnit is fast: From invocation
to completion KUnit can run several dozen tests in about a second.
Currently, the entire KUnit test suite for KUnit runs in under a second
from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining
unit test cases, grouping related test cases into test suites, providing
common infrastructure for running tests, mocking, spying, and much more.
### What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation,
hence the name. There should be no dependencies outside the control of
the test; this means no external dependencies, which makes tests orders
of magnitudes faster. Likewise, since there are no external dependencies,
there are no hoops to jump through to run the tests. Additionally, this
makes unit tests deterministic: a failing unit test always indicates a
problem. Finally, because unit tests necessarily have finer granularity,
they are able to test all code paths easily solving the classic problem
of difficulty in exercising error handling code.
### Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which
have their place. A well tested system has lots of unit tests, a
reasonable number of integration tests, and some end-to-end tests. KUnit
is just trying to address the unit test space which is currently not
being addressed.
### More information on KUnit
There is a bunch of documentation near the end of this patch set that
describes how to use KUnit and best practices for writing unit tests.
For convenience I am hosting the compiled docs here[4].
Additionally for convenience, I have applied these patches to a
branch[5]. The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/initial/v5.3/v17 branch.
## History since v15
### v17
- Addressed comments on 06/19 (lib: enable building KUnit in lib/) from
Stephen Boyd by moving KUnit submenu ahead of Runtime Testing
submenu.
### v16
- Addressed comments from Linus Torvalds by moving all kunit/ paths to
lib/kunit/.
- Addressed comments by Randy Dunlap by moving KUnit Kconfig under
lib/Kconfig.debug so the KUnit submenu shows up under the "Kernel
Hacking" menu.
[1] https://www.lkml.org/lkml/2019/9/20/696
[2] https://www.lkml.org/lkml/2019/9/20/738
[3] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kuni…
[4] https://google.github.io/kunit-docs/third_party/kernel/docs/
[5] https://kunit.googlesource.com/linux/+/kunit/initial/v5.3/v17
---
Avinash Kondareddy (1):
kunit: test: add tests for KUnit managed resources
Brendan Higgins (16):
kunit: test: add KUnit test runner core
kunit: test: add test resource management API
kunit: test: add string_stream a std::stream like string builder
kunit: test: add assertion printing library
kunit: test: add the concept of expectations
lib: enable building KUnit in lib/
kunit: test: add initial tests
objtool: add kunit_try_catch_throw to the noreturn list
kunit: test: add support for test abort
kunit: test: add tests for kunit test abort
kunit: test: add the concept of assertions
kunit: defconfig: add defconfigs for building KUnit tests
Documentation: kunit: add documentation for KUnit
MAINTAINERS: add entry for KUnit the unit testing framework
MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
kunit: fix failure to build without printk
Felix Guo (1):
kunit: tool: add Python wrappers for running KUnit tests
Iurii Zaikin (1):
kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()
Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kunit/api/index.rst | 16 +
Documentation/dev-tools/kunit/api/test.rst | 11 +
Documentation/dev-tools/kunit/faq.rst | 62 +
Documentation/dev-tools/kunit/index.rst | 79 +
Documentation/dev-tools/kunit/start.rst | 180 ++
Documentation/dev-tools/kunit/usage.rst | 576 +++++++
MAINTAINERS | 13 +
arch/um/configs/kunit_defconfig | 3 +
include/kunit/assert.h | 356 ++++
include/kunit/string-stream.h | 51 +
include/kunit/test.h | 1490 +++++++++++++++++
include/kunit/try-catch.h | 75 +
kernel/Makefile | 2 +
kernel/sysctl-test.c | 392 +++++
lib/Kconfig.debug | 13 +
lib/Makefile | 2 +
lib/kunit/Kconfig | 38 +
lib/kunit/Makefile | 9 +
lib/kunit/assert.c | 141 ++
lib/kunit/example-test.c | 88 +
lib/kunit/string-stream-test.c | 52 +
lib/kunit/string-stream.c | 217 +++
lib/kunit/test-test.c | 331 ++++
lib/kunit/test.c | 478 ++++++
lib/kunit/try-catch.c | 118 ++
tools/objtool/check.c | 1 +
tools/testing/kunit/.gitignore | 3 +
tools/testing/kunit/configs/all_tests.config | 3 +
tools/testing/kunit/kunit.py | 136 ++
tools/testing/kunit/kunit_config.py | 66 +
tools/testing/kunit/kunit_kernel.py | 149 ++
tools/testing/kunit/kunit_parser.py | 310 ++++
tools/testing/kunit/kunit_tool_test.py | 206 +++
.../test_is_test_passed-all_passed.log | 32 +
.../test_data/test_is_test_passed-crash.log | 69 +
.../test_data/test_is_test_passed-failure.log | 36 +
.../test_is_test_passed-no_tests_run.log | 75 +
.../test_output_isolated_correctly.log | 106 ++
.../test_data/test_read_from_file.kconfig | 17 +
40 files changed, 6003 insertions(+)
create mode 100644 Documentation/dev-tools/kunit/api/index.rst
create mode 100644 Documentation/dev-tools/kunit/api/test.rst
create mode 100644 Documentation/dev-tools/kunit/faq.rst
create mode 100644 Documentation/dev-tools/kunit/index.rst
create mode 100644 Documentation/dev-tools/kunit/start.rst
create mode 100644 Documentation/dev-tools/kunit/usage.rst
create mode 100644 arch/um/configs/kunit_defconfig
create mode 100644 include/kunit/assert.h
create mode 100644 include/kunit/string-stream.h
create mode 100644 include/kunit/test.h
create mode 100644 include/kunit/try-catch.h
create mode 100644 kernel/sysctl-test.c
create mode 100644 lib/kunit/Kconfig
create mode 100644 lib/kunit/Makefile
create mode 100644 lib/kunit/assert.c
create mode 100644 lib/kunit/example-test.c
create mode 100644 lib/kunit/string-stream-test.c
create mode 100644 lib/kunit/string-stream.c
create mode 100644 lib/kunit/test-test.c
create mode 100644 lib/kunit/test.c
create mode 100644 lib/kunit/try-catch.c
create mode 100644 tools/testing/kunit/.gitignore
create mode 100644 tools/testing/kunit/configs/all_tests.config
create mode 100755 tools/testing/kunit/kunit.py
create mode 100644 tools/testing/kunit/kunit_config.py
create mode 100644 tools/testing/kunit/kunit_kernel.py
create mode 100644 tools/testing/kunit/kunit_parser.py
create mode 100755 tools/testing/kunit/kunit_tool_test.py
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log
create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log
create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
--
2.23.0.351.gc4317032e6-goog
This patch set proposes KUnit, a lightweight unit testing and mocking
framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework;
it does not require installing the kernel on a test machine or in a VM
and does not require tests to be written in userspace running on a host
kernel. Additionally, KUnit is fast: From invocation to completion KUnit
can run several dozen tests in under a second. Currently, the entire
KUnit test suite for KUnit runs in under a second from the initial
invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining
unit test cases, grouping related test cases into test suites, providing
common infrastructure for running tests, mocking, spying, and much more.
## What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation,
hence the name. There should be no dependencies outside the control of
the test; this means no external dependencies, which makes tests orders
of magnitudes faster. Likewise, since there are no external dependencies,
there are no hoops to jump through to run the tests. Additionally, this
makes unit tests deterministic: a failing unit test always indicates a
problem. Finally, because unit tests necessarily have finer granularity,
they are able to test all code paths easily solving the classic problem
of difficulty in exercising error handling code.
## Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which
have their place. A well tested system has lots of unit tests, a
reasonable number of integration tests, and some end-to-end tests. KUnit
is just trying to address the unit test space which is currently not
being addressed.
## More information on KUnit
There is a bunch of documentation near the end of this patch set that
describes how to use KUnit and best practices for writing unit tests.
For convenience I am hosting the compiled docs here:
https://google.github.io/kunit-docs/third_party/kernel/docs/
Additionally for convenience, I have applied these patches to a branch:
https://kunit.googlesource.com/linux/+/kunit/rfc/4.19/v3
The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/rfc/4.19/v3 branch.
## Changes Since Last Version
- Changed namespace prefix from `test_*` to `kunit_*` as requested by
Shuah.
- Started converting/cleaning up the device tree unittest to use KUnit.
- Started adding KUnit expectations with custom messages.
--
2.20.0.rc0.387.gc7a69e6b6c-goog
## TL;DR
This revision addresses comments from Linus[1] and Randy[2], by moving
top level `kunit/` directory to `lib/kunit/` and likewise moves top
level Kconfig entry under lib/Kconfig.debug, so the KUnit submenu now
shows up under the "Kernel Hacking" menu.
As a consequence of this, I rewrote patch 06/18 (kbuild: enable building
KUnit), and now needs to be re-acked/reviewed.
## Background
This patch set proposes KUnit, a lightweight unit testing and mocking
framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework;
it does not require installing the kernel on a test machine or in a VM
(however, KUnit still allows you to run tests on test machines or in VMs
if you want[3]) and does not require tests to be written in userspace
running on a host kernel. Additionally, KUnit is fast: From invocation
to completion KUnit can run several dozen tests in about a second.
Currently, the entire KUnit test suite for KUnit runs in under a second
from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining
unit test cases, grouping related test cases into test suites, providing
common infrastructure for running tests, mocking, spying, and much more.
### What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation,
hence the name. There should be no dependencies outside the control of
the test; this means no external dependencies, which makes tests orders
of magnitudes faster. Likewise, since there are no external dependencies,
there are no hoops to jump through to run the tests. Additionally, this
makes unit tests deterministic: a failing unit test always indicates a
problem. Finally, because unit tests necessarily have finer granularity,
they are able to test all code paths easily solving the classic problem
of difficulty in exercising error handling code.
### Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which
have their place. A well tested system has lots of unit tests, a
reasonable number of integration tests, and some end-to-end tests. KUnit
is just trying to address the unit test space which is currently not
being addressed.
### More information on KUnit
There is a bunch of documentation near the end of this patch set that
describes how to use KUnit and best practices for writing unit tests.
For convenience I am hosting the compiled docs here[4].
Additionally for convenience, I have applied these patches to a
branch[5]. The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/initial/v5.3/v16 branch.
[1] https://www.lkml.org/lkml/2019/9/20/696
[2] https://www.lkml.org/lkml/2019/9/20/738
[3] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kuni…
[4] https://google.github.io/kunit-docs/third_party/kernel/docs/
[5] https://kunit.googlesource.com/linux/+/kunit/initial/v5.3/v16
---
Avinash Kondareddy (1):
kunit: test: add tests for KUnit managed resources
Brendan Higgins (16):
kunit: test: add KUnit test runner core
kunit: test: add test resource management API
kunit: test: add string_stream a std::stream like string builder
kunit: test: add assertion printing library
kunit: test: add the concept of expectations
lib: enable building KUnit in lib/
kunit: test: add initial tests
objtool: add kunit_try_catch_throw to the noreturn list
kunit: test: add support for test abort
kunit: test: add tests for kunit test abort
kunit: test: add the concept of assertions
kunit: defconfig: add defconfigs for building KUnit tests
Documentation: kunit: add documentation for KUnit
MAINTAINERS: add entry for KUnit the unit testing framework
MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
kunit: fix failure to build without printk
Felix Guo (1):
kunit: tool: add Python wrappers for running KUnit tests
Iurii Zaikin (1):
kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()
Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kunit/api/index.rst | 16 +
Documentation/dev-tools/kunit/api/test.rst | 11 +
Documentation/dev-tools/kunit/faq.rst | 62 +
Documentation/dev-tools/kunit/index.rst | 79 +
Documentation/dev-tools/kunit/start.rst | 180 ++
Documentation/dev-tools/kunit/usage.rst | 576 +++++++
MAINTAINERS | 13 +
arch/um/configs/kunit_defconfig | 3 +
include/kunit/assert.h | 356 ++++
include/kunit/string-stream.h | 51 +
include/kunit/test.h | 1490 +++++++++++++++++
include/kunit/try-catch.h | 75 +
kernel/Makefile | 2 +
kernel/sysctl-test.c | 392 +++++
lib/Kconfig.debug | 13 +
lib/Makefile | 2 +
lib/kunit/Kconfig | 38 +
lib/kunit/Makefile | 9 +
lib/kunit/assert.c | 141 ++
lib/kunit/example-test.c | 88 +
lib/kunit/string-stream-test.c | 52 +
lib/kunit/string-stream.c | 217 +++
lib/kunit/test-test.c | 331 ++++
lib/kunit/test.c | 478 ++++++
lib/kunit/try-catch.c | 118 ++
tools/objtool/check.c | 1 +
tools/testing/kunit/.gitignore | 3 +
tools/testing/kunit/configs/all_tests.config | 3 +
tools/testing/kunit/kunit.py | 136 ++
tools/testing/kunit/kunit_config.py | 66 +
tools/testing/kunit/kunit_kernel.py | 149 ++
tools/testing/kunit/kunit_parser.py | 310 ++++
tools/testing/kunit/kunit_tool_test.py | 206 +++
.../test_is_test_passed-all_passed.log | 32 +
.../test_data/test_is_test_passed-crash.log | 69 +
.../test_data/test_is_test_passed-failure.log | 36 +
.../test_is_test_passed-no_tests_run.log | 75 +
.../test_output_isolated_correctly.log | 106 ++
.../test_data/test_read_from_file.kconfig | 17 +
40 files changed, 6003 insertions(+)
create mode 100644 Documentation/dev-tools/kunit/api/index.rst
create mode 100644 Documentation/dev-tools/kunit/api/test.rst
create mode 100644 Documentation/dev-tools/kunit/faq.rst
create mode 100644 Documentation/dev-tools/kunit/index.rst
create mode 100644 Documentation/dev-tools/kunit/start.rst
create mode 100644 Documentation/dev-tools/kunit/usage.rst
create mode 100644 arch/um/configs/kunit_defconfig
create mode 100644 include/kunit/assert.h
create mode 100644 include/kunit/string-stream.h
create mode 100644 include/kunit/test.h
create mode 100644 include/kunit/try-catch.h
create mode 100644 kernel/sysctl-test.c
create mode 100644 lib/kunit/Kconfig
create mode 100644 lib/kunit/Makefile
create mode 100644 lib/kunit/assert.c
create mode 100644 lib/kunit/example-test.c
create mode 100644 lib/kunit/string-stream-test.c
create mode 100644 lib/kunit/string-stream.c
create mode 100644 lib/kunit/test-test.c
create mode 100644 lib/kunit/test.c
create mode 100644 lib/kunit/try-catch.c
create mode 100644 tools/testing/kunit/.gitignore
create mode 100644 tools/testing/kunit/configs/all_tests.config
create mode 100755 tools/testing/kunit/kunit.py
create mode 100644 tools/testing/kunit/kunit_config.py
create mode 100644 tools/testing/kunit/kunit_kernel.py
create mode 100644 tools/testing/kunit/kunit_parser.py
create mode 100755 tools/testing/kunit/kunit_tool_test.py
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log
create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log
create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
--
2.23.0.351.gc4317032e6-goog
Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per
test") solves the problem of kselftest_harness.h-using binary tests
possibly hanging forever. However, scripts and other binaries can still
hang forever. This adds a global timeout to each test script run.
To make this configurable (e.g. as needed in the "rtc" test case),
include a new per-test-directory "settings" file (similar to "config")
that can contain kselftest-specific settings. The first recognized field
is "timeout".
Additionally, this splits the reporting for timeouts into a specific
"TIMEOUT" not-ok (and adds exit code reporting in the remaining case).
Signed-off-by: Kees Cook <keescook(a)chromium.org>
---
tools/testing/selftests/kselftest/runner.sh | 36 +++++++++++++++++++--
tools/testing/selftests/rtc/settings | 1 +
2 files changed, 34 insertions(+), 3 deletions(-)
create mode 100644 tools/testing/selftests/rtc/settings
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 00c9020bdda8..84de7bc74f2c 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -3,9 +3,14 @@
#
# Runs a set of tests in a given subdirectory.
export skip_rc=4
+export timeout_rc=124
export logfile=/dev/stdout
export per_test_logging=
+# Defaults for "settings" file fields:
+# "timeout" how many seconds to let each test run before failing.
+export kselftest_default_timeout=45
+
# There isn't a shell-agnostic way to find the path of a sourced file,
# so we must rely on BASE_DIR being set to find other tools.
if [ -z "$BASE_DIR" ]; then
@@ -24,6 +29,16 @@ tap_prefix()
fi
}
+tap_timeout()
+{
+ # Make sure tests will time out if utility is available.
+ if [ -x /usr/bin/timeout ] ; then
+ /usr/bin/timeout "$kselftest_timeout" "$1"
+ else
+ "$1"
+ fi
+}
+
run_one()
{
DIR="$1"
@@ -32,6 +47,18 @@ run_one()
BASENAME_TEST=$(basename $TEST)
+ # Reset any "settings"-file variables.
+ export kselftest_timeout="$kselftest_default_timeout"
+ # Load per-test-directory kselftest "settings" file.
+ settings="$BASE_DIR/$DIR/settings"
+ if [ -r "$settings" ] ; then
+ while read line ; do
+ field=$(echo "$line" | cut -d= -f1)
+ value=$(echo "$line" | cut -d= -f2-)
+ eval "kselftest_$field"="$value"
+ done < "$settings"
+ fi
+
TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST"
echo "# $TEST_HDR_MSG"
if [ ! -x "$TEST" ]; then
@@ -44,14 +71,17 @@ run_one()
echo "not ok $test_num $TEST_HDR_MSG"
else
cd `dirname $TEST` > /dev/null
- (((((./$BASENAME_TEST 2>&1; echo $? >&3) |
+ ((((( tap_timeout ./$BASENAME_TEST 2>&1; echo $? >&3) |
tap_prefix >&4) 3>&1) |
(read xs; exit $xs)) 4>>"$logfile" &&
echo "ok $test_num $TEST_HDR_MSG") ||
- (if [ $? -eq $skip_rc ]; then \
+ (rc=$?; \
+ if [ $rc -eq $skip_rc ]; then \
echo "not ok $test_num $TEST_HDR_MSG # SKIP"
+ elif [ $rc -eq $timeout_rc ]; then \
+ echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"
else
- echo "not ok $test_num $TEST_HDR_MSG"
+ echo "not ok $test_num $TEST_HDR_MSG # exit=$rc"
fi)
cd - >/dev/null
fi
diff --git a/tools/testing/selftests/rtc/settings b/tools/testing/selftests/rtc/settings
new file mode 100644
index 000000000000..ba4d85f74cd6
--- /dev/null
+++ b/tools/testing/selftests/rtc/settings
@@ -0,0 +1 @@
+timeout=90
--
2.17.1
--
Kees Cook
On Tue, Sep 17, 2019 at 12:26 PM Shuah Khan <skhan(a)linuxfoundation.org> wrote:
>
> This Kselftest update for Linux 5.4-rc1 consists of several fixes to
> existing tests and adds KUnit, a lightweight unit testing and mocking
> framework for the Linux kernel from Brendan Higgins.
So I pulled this, but then I almost immediately unpulled it.
My reason for doing that may be odd, but it's because of the top-level
'kunit' directory. This shouldn't be on the top level.
The reason I react so strongly is that it actually breaks my finger
memory. I don't type out filenames - I auto-compete them. So "kernel/"
is "k<tab>", "drivers/" is "d<tab>" etc.
It already doesn't work for everything ("mm/" is actually "mm<tab>"
not because we have files in the git tree, but because the build
creates various "module" files), but this breaks a common pattern for
me.
> In the future KUnit will be linked to Kselftest framework to provide
> a way to trigger KUnit tests from user-space.
Can the kernel parts please move to lib/kunit/ or something like that?
Linus
Hey everyone,
This is the patchset coming out of the KSummit session Kees and I gave
in Lisbon last week (cf. [3] which also contains slides with more
details on related things such as deep argument inspection).
The simple idea is to extend the seccomp notifier to allow for the
continuation of a syscall. The rationale for this can be found in the
commit message to [1]. For the curious there is more detail in [2].
This patchset would unblock supervising an extended set of syscalls such
as mount() where a privileged process is supervising the syscalls of a
lesser privileged process and emulates the syscall for the latter in
userspace.
For more comments on security see [1].
Kees, if you prefer a pr the series can be pulled from:
git@gitolite.kernel.org:pub/scm/linux/kernel/git/brauner/linux tags/seccomp-notify-syscall-continue-v5.5
For anyone who wants to play with this it's sitting in:
https://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git/log/?h=se…
/* v1 */
- Kees Cook <keescook(a)chromium.org>:
- dropped patch because it is already present in linux-next
[PATCH 2/4] seccomp: add two missing ptrace ifdefines
Link: https://lore.kernel.org/r/20190918084833.9369-3-christian.brauner@ubuntu.com
/* v0 */
Link: https://lore.kernel.org/r/20190918084833.9369-1-christian.brauner@ubuntu.com
Thanks!
Christian
/* References */
[1]: [PATCH 1/3] seccomp: add SECCOMP_USER_NOTIF_FLAG_CONTINUE
[2]: https://lore.kernel.org/r/20190719093538.dhyopljyr5ns33qx@brauner.io
[3]: https://linuxplumbersconf.org/event/4/contributions/560
Christian Brauner (3):
seccomp: add SECCOMP_USER_NOTIF_FLAG_CONTINUE
seccomp: avoid overflow in implicit constant conversion
seccomp: test SECCOMP_USER_NOTIF_FLAG_CONTINUE
include/uapi/linux/seccomp.h | 20 ++++
kernel/seccomp.c | 28 ++++-
tools/testing/selftests/seccomp/seccomp_bpf.c | 105 +++++++++++++++++-
3 files changed, 146 insertions(+), 7 deletions(-)
--
2.23.0
Hey everyone,
This is the patchset coming out of the KSummit session Kees and I gave
in Lisbon last week (cf. [3] which also contains slides with more
details on related things such as deep argument inspection).
The simple idea is to extend the seccomp notifier to allow for the
continuation of a syscall. The rationale for this can be found in the
commit message to [1]. For the curious there is more detail in [2].
This patchset would unblock supervising an extended set of syscalls such
as mount() where a privileged process is supervising the syscalls of a
lesser privileged process and emulates the syscall for the latter in
userspace.
For more comments on security see [1].
Thanks!
Christian
/* References */
[1]: [PATCH 1/4] seccomp: add SECCOMP_RET_USER_NOTIF_ALLOW
[2]: https://lore.kernel.org/r/20190719093538.dhyopljyr5ns33qx@brauner.io
[3]: https://linuxplumbersconf.org/event/4/contributions/560
Christian Brauner (4):
seccomp: add SECCOMP_RET_USER_NOTIF_ALLOW
seccomp: add two missing ptrace ifdefines
seccomp: avoid overflow in implicit constant conversion
seccomp: test SECCOMP_RET_USER_NOTIF_ALLOW
include/uapi/linux/seccomp.h | 2 +
kernel/seccomp.c | 24 +++-
tools/testing/selftests/seccomp/seccomp_bpf.c | 110 +++++++++++++++++-
3 files changed, 131 insertions(+), 5 deletions(-)
--
2.23.0
From: "George G. Davis" <george_davis(a)mentor.com>
The newly added optional file argument does not validate if the
file is indeed a watchdog, e.g.:
./watchdog-test -f /dev/zero
Watchdog Ticking Away!
Fix it by confirming that the WDIOC_GETSUPPORT ioctl succeeds.
Fixes: c3f2490d6e9257 ("selftests: watchdog: Add optional file argument")
Reported-by: Eugeniu Rosca <erosca(a)de.adit-jv.com>
Signed-off-by: George G. Davis <george_davis(a)mentor.com>
Signed-off-by: Eugeniu Rosca <erosca(a)de.adit-jv.com>
---
v3:
- Used v1 as starting point and simplified commit description
- Added Fixes tag (WARNING: commit id is from linux-next!)
- No change in the contents
- Applied cleanly to the same base as used in [v1]
v2:
- https://patchwork.kernel.org/patch/11147663/
v1:
- https://patchwork.kernel.org/patch/11136283/
- Applied/tested on commit ce54eab71e210f ("kunit: fix failure to build without printk") of
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/l…
---
tools/testing/selftests/watchdog/watchdog-test.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/tools/testing/selftests/watchdog/watchdog-test.c b/tools/testing/selftests/watchdog/watchdog-test.c
index afff120c7be6..6ed822dc2222 100644
--- a/tools/testing/selftests/watchdog/watchdog-test.c
+++ b/tools/testing/selftests/watchdog/watchdog-test.c
@@ -97,6 +97,7 @@ int main(int argc, char *argv[])
int c;
int oneshot = 0;
char *file = "/dev/watchdog";
+ struct watchdog_info info;
setbuf(stdout, NULL);
@@ -118,6 +119,16 @@ int main(int argc, char *argv[])
exit(-1);
}
+ /*
+ * Validate that `file` is a watchdog device
+ */
+ ret = ioctl(fd, WDIOC_GETSUPPORT, &info);
+ if (ret) {
+ printf("WDIOC_GETSUPPORT error '%s'\n", strerror(errno));
+ close(fd);
+ exit(ret);
+ }
+
optind = 0;
while ((c = getopt_long(argc, argv, sopts, lopts, NULL)) != -1) {
--
2.23.0
Problem:
Currently tasks attempting to allocate more hugetlb memory than is available get
a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1].
However, if a task attempts to allocate hugetlb memory only more than its
hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call,
but will SIGBUS the task when it attempts to fault the memory in.
We have developers interested in using hugetlb_cgroups, and they have expressed
dissatisfaction regarding this behavior. We'd like to improve this
behavior such that tasks violating the hugetlb_cgroup limits get an error on
mmap/shmget time, rather than getting SIGBUS'd when they try to fault
the excess memory in.
The underlying problem is that today's hugetlb_cgroup accounting happens
at hugetlb memory *fault* time, rather than at *reservation* time.
Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and
the offending task gets SIGBUS'd.
Proposed Solution:
A new page counter named hugetlb.xMB.reservation_[limit|usage]_in_bytes. This
counter has slightly different semantics than
hugetlb.xMB.[limit|usage]_in_bytes:
- While usage_in_bytes tracks all *faulted* hugetlb memory,
reservation_usage_in_bytes tracks all *reserved* hugetlb memory.
- If a task attempts to reserve more memory than limit_in_bytes allows,
the kernel will allow it to do so. But if a task attempts to reserve
more memory than reservation_limit_in_bytes, the kernel will fail this
reservation.
This proposal is implemented in this patch, with tests to verify
functionality and show the usage.
Alternatives considered:
1. A new cgroup, instead of only a new page_counter attached to
the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code
duplication with hugetlb_cgroup. Keeping hugetlb related page counters under
hugetlb_cgroup seemed cleaner as well.
2. Instead of adding a new counter, we considered adding a sysctl that modifies
the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at
reservation time rather than fault time. Adding a new page_counter seems
better as userspace could, if it wants, choose to enforce different cgroups
differently: one via limit_in_bytes, and another via
reservation_limit_in_bytes. This could be very useful if you're
transitioning how hugetlb memory is partitioned on your system one
cgroup at a time, for example. Also, someone may find usage for both
limit_in_bytes and reservation_limit_in_bytes concurrently, and this
approach gives them the option to do so.
Caveats:
1. This support is implemented for cgroups-v1. I have not tried
hugetlb_cgroups with cgroups v2, and AFAICT it's not supported yet.
This is largely because we use cgroups-v1 for now. If required, I
can add hugetlb_cgroup support to cgroups v2 in this patch or
a follow up.
2. Most complicated bit of this patch I believe is: where to store the
pointer to the hugetlb_cgroup to uncharge at unreservation time?
Normally the cgroup pointers hang off the struct page. But, with
hugetlb_cgroup reservations, one task can reserve a specific page and another
task may fault it in (I believe), so storing the pointer in struct
page is not appropriate. Proposed approach here is to store the pointer in
the resv_map. See patch for details.
Testing:
- Added tests passing.
- libhugetlbfs tests mostly passing, but some tests have trouble with and
without this patch series. Seems environment issue rather than code:
- Overall results:
********** TEST SUMMARY
* 2M
* 32-bit 64-bit
* Total testcases: 84 0
* Skipped: 0 0
* PASS: 66 0
* FAIL: 14 0
* Killed by signal: 0 0
* Bad configuration: 4 0
* Expected FAIL: 0 0
* Unexpected PASS: 0 0
* Test not present: 0 0
* Strange test result: 0 0
**********
- Failing tests:
- elflink_rw_and_share_test("linkhuge_rw") segfaults with and without this
patch series.
- LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc (2M: 32):
FAIL Address is not hugepage
- LD_PRELOAD=libhugetlbfs.so HUGETLB_RESTRICT_EXE=unknown:malloc
HUGETLB_MORECORE=yes malloc (2M: 32):
FAIL Address is not hugepage
- LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc_manysmall (2M: 32):
FAIL Address is not hugepage
- GLIBC_TUNABLES=glibc.malloc.tcache_count=0 LD_PRELOAD=libhugetlbfs.so
HUGETLB_MORECORE=yes heapshrink (2M: 32):
FAIL Heap not on hugepages
- GLIBC_TUNABLES=glibc.malloc.tcache_count=0 LD_PRELOAD=libhugetlbfs.so
libheapshrink.so HUGETLB_MORECORE=yes heapshrink (2M: 32):
FAIL Heap not on hugepages
- HUGETLB_ELFMAP=RW linkhuge_rw (2M: 32): FAIL small_data is not hugepage
- HUGETLB_ELFMAP=RW HUGETLB_MINIMAL_COPY=no linkhuge_rw (2M: 32):
FAIL small_data is not hugepage
- alloc-instantiate-race shared (2M: 32):
Bad configuration: sched_setaffinity(cpu1): Invalid argument -
FAIL Child 1 killed by signal Killed
- shmoverride_linked (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
- HUGETLB_SHM=yes shmoverride_linked (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
- shmoverride_linked_static (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
- HUGETLB_SHM=yes shmoverride_linked_static (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
- LD_PRELOAD=libhugetlbfs.so shmoverride_unlinked (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
- LD_PRELOAD=libhugetlbfs.so HUGETLB_SHM=yes shmoverride_unlinked (2M: 32):
FAIL shmget failed size 2097152 from line 176: Invalid argument
Signed-off-by: Mina Almasry <almasrymina(a)google.com>
[1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html
Changes in v4:
- Split up 'hugetlb_cgroup: add accounting for shared mappings' into 4 patches
for better isolation and context on the individual changes:
- hugetlb_cgroup: add accounting for shared mappings
- hugetlb: disable region_add file_region coalescing
- hugetlb: remove duplicated code
- hugetlb: region_chg provides only cache entry
- Fixed resv->adds_in_progress accounting.
- Retained behavior that region_add never fails, in earlier patchsets region_add
could return failure.
- Fixed libhugetlbfs failure.
- Minor fix to the added tests that was preventing them from running on some
environments.
Changes in v3:
- Addressed comments of Hillf Danton:
- Added docs.
- cgroup_files now uses enum.
- Various readability improvements.
- Addressed comments of Mike Kravetz.
- region_* functions no longer coalesce file_region entries in the resv_map.
- region_add() and region_chg() refactored to make them much easier to
understand and remove duplicated code so this patch doesn't add too much
complexity.
- Refactored common functionality into helpers.
Changes in v2:
- Split the patch into a 5 patch series.
- Fixed patch subject.
Mina Almasry (9):
hugetlb_cgroup: Add hugetlb_cgroup reservation counter
hugetlb_cgroup: add interface for charge/uncharge hugetlb reservations
hugetlb_cgroup: add reservation accounting for private mappings
hugetlb: region_chg provides only cache entry
hugetlb: remove duplicated code
hugetlb: disable region_add file_region coalescing
hugetlb_cgroup: add accounting for shared mappings
hugetlb_cgroup: Add hugetlb_cgroup reservation tests
hugetlb_cgroup: Add hugetlb_cgroup reservation docs
.../admin-guide/cgroup-v1/hugetlb.rst | 84 ++-
include/linux/hugetlb.h | 24 +-
include/linux/hugetlb_cgroup.h | 24 +-
mm/hugetlb.c | 516 +++++++++++-------
mm/hugetlb_cgroup.c | 189 +++++--
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 4 +
.../selftests/vm/charge_reserved_hugetlb.sh | 440 +++++++++++++++
.../selftests/vm/write_hugetlb_memory.sh | 22 +
.../testing/selftests/vm/write_to_hugetlbfs.c | 252 +++++++++
10 files changed, 1304 insertions(+), 252 deletions(-)
create mode 100755 tools/testing/selftests/vm/charge_reserved_hugetlb.sh
create mode 100644 tools/testing/selftests/vm/write_hugetlb_memory.sh
create mode 100644 tools/testing/selftests/vm/write_to_hugetlbfs.c
--
2.23.0.162.g0b9fbb3734-goog
This patchset is being developed here:
<https://github.com/cyphar/linux/tree/resolveat/master>
Patch changelog:
v12:
* Remove @how->reserved field from openat2(2), and instead use the
(struct, size) design for syscall extensions.
* Implement copy_struct_{to,from}_user() to unify (struct, size)
syscall extension designs (as well as make them slightly more
efficient by using memchr_inv() as well as using buffers and
avoiding repeated access_ok() checks for trailing byte operations).
* Port sched_setattr(), perf_event_attr(), and clone3() to use the
new helpers.
v11: <https://lore.kernel.org/lkml/20190820033406.29796-1-cyphar@cyphar.com/>
<https://lore.kernel.org/lkml/20190728010207.9781-1-cyphar@cyphar.com/>
v10: <https://lore.kernel.org/lkml/20190719164225.27083-1-cyphar@cyphar.com/>
v09: <https://lore.kernel.org/lkml/20190706145737.5299-1-cyphar@cyphar.com/>
v08: <https://lore.kernel.org/lkml/20190520133305.11925-1-cyphar@cyphar.com/>
v07: <https://lore.kernel.org/lkml/20190507164317.13562-1-cyphar@cyphar.com/>
v06: <https://lore.kernel.org/lkml/20190506165439.9155-1-cyphar@cyphar.com/>
v05: <https://lore.kernel.org/lkml/20190320143717.2523-1-cyphar@cyphar.com/>
v04: <https://lore.kernel.org/lkml/20181112142654.341-1-cyphar@cyphar.com/>
v03: <https://lore.kernel.org/lkml/20181009070230.12884-1-cyphar@cyphar.com/>
v02: <https://lore.kernel.org/lkml/20181009065300.11053-1-cyphar@cyphar.com/>
v01: <https://lore.kernel.org/lkml/20180929103453.12025-1-cyphar@cyphar.com/>
The need for some sort of control over VFS's path resolution (to avoid
malicious paths resulting in inadvertent breakouts) has been a very
long-standing desire of many userspace applications. This patchset is a
revival of Al Viro's old AT_NO_JUMPS[1,2] patchset (which was a variant
of David Drysdale's O_BENEATH patchset[3] which was a spin-off of the
Capsicum project[4]) with a few additions and changes made based on the
previous discussion within [5] as well as others I felt were useful.
In line with the conclusions of the original discussion of AT_NO_JUMPS,
the flag has been split up into separate flags. However, instead of
being an openat(2) flag it is provided through a new syscall openat2(2)
which provides several other improvements to the openat(2) interface (see the
patch description for more details). The following new LOOKUP_* flags are
added:
* LOOKUP_NO_XDEV blocks all mountpoint crossings (upwards, downwards,
or through absolute links). Absolute pathnames alone in openat(2) do
not trigger this.
* LOOKUP_NO_MAGICLINKS blocks resolution through /proc/$pid/fd-style
links. This is done by blocking the usage of nd_jump_link() during
resolution in a filesystem. The term "magic-links" is used to match
with the only reference to these links in Documentation/, but I'm
happy to change the name.
It should be noted that this is different to the scope of
~LOOKUP_FOLLOW in that it applies to all path components. However,
you can do openat2(NO_FOLLOW|NO_MAGICLINKS) on a magic-link and it
will *not* fail (assuming that no parent component was a
magic-link), and you will have an fd for the magic-link.
* LOOKUP_BENEATH disallows escapes to outside the starting dirfd's
tree, using techniques such as ".." or absolute links. Absolute
paths in openat(2) are also disallowed. Conceptually this flag is to
ensure you "stay below" a certain point in the filesystem tree --
but this requires some additional to protect against various races
that would allow escape using "..".
Currently LOOKUP_BENEATH implies LOOKUP_NO_MAGICLINKS, because it
can trivially beam you around the filesystem (breaking the
protection). In future, there might be similar safety checks done as
in LOOKUP_IN_ROOT, but that requires more discussion.
In addition, two new flags are added that expand on the above ideas:
* LOOKUP_NO_SYMLINKS does what it says on the tin. No symlink
resolution is allowed at all, including magic-links. Just as with
LOOKUP_NO_MAGICLINKS this can still be used with NOFOLLOW to open an
fd for the symlink as long as no parent path had a symlink
component.
* LOOKUP_IN_ROOT is an extension of LOOKUP_BENEATH that, rather than
blocking attempts to move past the root, forces all such movements
to be scoped to the starting point. This provides chroot(2)-like
protection but without the cost of a chroot(2) for each filesystem
operation, as well as being safe against race attacks that chroot(2)
is not.
If a race is detected (as with LOOKUP_BENEATH) then an error is
generated, and similar to LOOKUP_BENEATH it is not permitted to cross
magic-links with LOOKUP_IN_ROOT.
The primary need for this is from container runtimes, which
currently need to do symlink scoping in userspace[6] when opening
paths in a potentially malicious container. There is a long list of
CVEs that could have bene mitigated by having RESOLVE_THIS_ROOT
(such as CVE-2017-1002101, CVE-2017-1002102, CVE-2018-15664, and
CVE-2019-5736, just to name a few).
And further, several semantics of file descriptor "re-opening" are now
changed to prevent attacks like CVE-2019-5736 by restricting how
magic-links can be resolved (based on their mode). This required some
other changes to the semantics of the modes of O_PATH file descriptor's
associated /proc/self/fd magic-links. openat2(2) has the ability to
further restrict re-opening of its own O_PATH fds, so that users can
make even better use of this feature.
Finally, O_EMPTYPATH was added so that users can do /proc/self/fd-style
re-opening without depending on procfs. The new restricted semantics for
magic-links are applied here too.
In order to make all of the above more usable, I'm working on
libpathrs[7] which is a C-friendly library for safe path resolution. It
features a userspace-emulated backend if the kernel doesn't support
openat2(2). Hopefully we can get userspace to switch to using it, and
thus get openat2(2) support for free once it's ready.
Cc: Al Viro <viro(a)zeniv.linux.org.uk>
Cc: Eric Biederman <ebiederm(a)xmission.com>
Cc: Andy Lutomirski <luto(a)kernel.org>
Cc: David Howells <dhowells(a)redhat.com>
Cc: Jann Horn <jannh(a)google.com>
Cc: Christian Brauner <christian(a)brauner.io>
Cc: David Drysdale <drysdale(a)google.com>
Cc: Tycho Andersen <tycho(a)tycho.ws>
Cc: Kees Cook <keescook(a)chromium.org>
Cc: Linus Torvalds <torvalds(a)linux-foundation.org>
[1]: https://lwn.net/Articles/721443/
[2]: https://lore.kernel.org/patchwork/patch/784221/
[3]: https://lwn.net/Articles/619151/
[4]: https://lwn.net/Articles/603929/
[5]: https://lwn.net/Articles/723057/
[6]: https://github.com/cyphar/filepath-securejoin
[7]: https://github.com/openSUSE/libpathrs
Aleksa Sarai (12):
lib: introduce copy_struct_{to,from}_user helpers
clone3: switch to copy_struct_from_user()
sched_setattr: switch to copy_struct_{to,from}_user()
perf_event_open: switch to copy_struct_from_user()
namei: obey trailing magic-link DAC permissions
procfs: switch magic-link modes to be more sane
open: O_EMPTYPATH: procfs-less file descriptor re-opening
namei: O_BENEATH-style path resolution flags
namei: LOOKUP_IN_ROOT: chroot-like path resolution
namei: aggressively check for nd->root escape on ".." resolution
open: openat2(2) syscall
selftests: add openat2(2) selftests
Documentation/filesystems/path-lookup.rst | 12 +-
arch/alpha/include/uapi/asm/fcntl.h | 1 +
arch/alpha/kernel/syscalls/syscall.tbl | 1 +
arch/arm/tools/syscall.tbl | 1 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/ia64/kernel/syscalls/syscall.tbl | 1 +
arch/m68k/kernel/syscalls/syscall.tbl | 1 +
arch/microblaze/kernel/syscalls/syscall.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n32.tbl | 1 +
arch/mips/kernel/syscalls/syscall_n64.tbl | 1 +
arch/mips/kernel/syscalls/syscall_o32.tbl | 1 +
arch/parisc/include/uapi/asm/fcntl.h | 39 +-
arch/parisc/kernel/syscalls/syscall.tbl | 1 +
arch/powerpc/kernel/syscalls/syscall.tbl | 1 +
arch/s390/kernel/syscalls/syscall.tbl | 1 +
arch/sh/kernel/syscalls/syscall.tbl | 1 +
arch/sparc/include/uapi/asm/fcntl.h | 1 +
arch/sparc/kernel/syscalls/syscall.tbl | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/xtensa/kernel/syscalls/syscall.tbl | 1 +
fs/fcntl.c | 2 +-
fs/internal.h | 1 +
fs/namei.c | 270 ++++++++++--
fs/open.c | 100 ++++-
fs/proc/base.c | 20 +-
fs/proc/fd.c | 23 +-
fs/proc/namespaces.c | 2 +-
include/linux/fcntl.h | 21 +-
include/linux/fs.h | 8 +-
include/linux/namei.h | 9 +
include/linux/syscalls.h | 14 +-
include/linux/uaccess.h | 5 +
include/uapi/asm-generic/fcntl.h | 4 +
include/uapi/asm-generic/unistd.h | 5 +-
include/uapi/linux/fcntl.h | 42 ++
include/uapi/linux/sched.h | 2 +
kernel/events/core.c | 45 +-
kernel/fork.c | 34 +-
kernel/sched/core.c | 85 +---
lib/Makefile | 2 +-
lib/struct_user.c | 182 ++++++++
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/memfd/memfd_test.c | 7 +-
tools/testing/selftests/openat2/.gitignore | 1 +
tools/testing/selftests/openat2/Makefile | 8 +
tools/testing/selftests/openat2/helpers.c | 167 ++++++++
tools/testing/selftests/openat2/helpers.h | 118 +++++
.../testing/selftests/openat2/linkmode_test.c | 333 +++++++++++++++
.../testing/selftests/openat2/openat2_test.c | 106 +++++
.../selftests/openat2/rename_attack_test.c | 127 ++++++
.../testing/selftests/openat2/resolve_test.c | 402 ++++++++++++++++++
53 files changed, 1971 insertions(+), 248 deletions(-)
create mode 100644 lib/struct_user.c
create mode 100644 tools/testing/selftests/openat2/.gitignore
create mode 100644 tools/testing/selftests/openat2/Makefile
create mode 100644 tools/testing/selftests/openat2/helpers.c
create mode 100644 tools/testing/selftests/openat2/helpers.h
create mode 100644 tools/testing/selftests/openat2/linkmode_test.c
create mode 100644 tools/testing/selftests/openat2/openat2_test.c
create mode 100644 tools/testing/selftests/openat2/rename_attack_test.c
create mode 100644 tools/testing/selftests/openat2/resolve_test.c
--
2.23.0
Hello.
This patch set updates clone3 selftest in several aspects:
- adding a check for kernel not ignoring highest 32 bits of exit_signal
field of clone3 syscall arguments structure;
- adding clone3 to selftests targets;
- enabling clone3 tests on all architectures;
- minor cleanups of the clone3 test.
Applied on top of brauer/linux.git/for-next.
Eugene Syromiatnikov (6):
selftests/clone3: convert test modes into an enum
selftests/clone3: add a check for invalid exit_signal
selftests/clone3: use uint64_t for flags parameter
selftests/clone3: fix up format strings
selftests/clone3: enable clone3 self-tests on all architectures
selftests: add clone3 to TARGETS
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/clone3/Makefile | 4 +---
tools/testing/selftests/clone3/clone3.c | 37 +++++++++++++++++++++++++--------
3 files changed, 30 insertions(+), 12 deletions(-)
--
2.1.4
Hi
this patchset aims to add the initial arch-specific arm64 support to
kselftest starting with signals-related test-cases.
A common internal test-case layout is proposed which then it is anyway
wired-up to the toplevel kselftest Makefile, so that it should be possible
at the end to run it on an arm64 target in the usual way with KSFT.
~/linux# make TARGETS=arm64 kselftest
New KSFT arm64 testcases live inside tools/testing/selftests/arm64 grouped by
family inside subdirectories: arm64/signal is the first family proposed with
this series.
This series converts also to this subdirectory scheme the pre-existing
(already queued on arm64/for-next/core) KSFT arm64 tags tests, moving them
into arm64/tags.
Thanks
Cristian
Notes:
-----
- further details in the included READMEs
- more tests still to be written (current strategy is going through the related
Kernel signal-handling code and write a test for each possible and sensible code-path)
A few ideas for more TODO testcases:
- fake_sigreturn_unmapped_sp: SP into unmapped addrs
- fake_sigreturn_kernelspace_sp: SP into kernel addrs
- fake_sigreturn_sve_bad_extra_context: SVE extra context badly formed
- mangle_sve_invalid_extra_context: SVE extra_context invalid
- SVE signal testcases and special handling will be part of an additional patch
still to be released
- KSFT arm64 tags test patch
https://lore.kernel.org/linux-arm-kernel/c1e6aad230658bc175b42d92daeff2e300…
is relocated into its own directory under tools/testing/selftests/arm64/tags
Changes:
--------
v4-->v5:
- rebased on arm64/for-next-core merging 01/11 with KSFT tags tests commmit
9ce1263033cd ("selftests, arm64: add a selftest for passing tagged pointers to kernel")
- moved .gitignore up on elevel
- moved kernel header search mechanism into KSFT arm64 toplevel Makefile
so that it can be used easily also by each arm64 KSFT subsystem inside
subdirs of arm64
v3-->v4:
- rebased on v5.3-rc6
- added test descriptions
- fixed commit messages (imperative mood)
- added missing includes and removed unneeded ones
- added/used new get_starting_head() helper
- fixed/simplified signal.S::fakke_sigreturn()
- added set_regval() macro and .init initialization func
- better synchonization in get_current_context()
- macroization of mangle_pstate_invalid_mode_el
- splitted mangle_pstate_invalid_mode_el h/t
- removed standalone mode
- simplified CPU features checks
- fixed/refactored get_header() and validation routines
- simplfied docs
v2-->v3:
- rebased on v5.3-rc2
- better test result characterization looking for
SEGV_ACCERR in si_code on SIGSEGV
- using KSFT Framework macros for retvalues
- removed SAFE_WRITE()/dump_uc: buggy, un-needed and unused
- reviewed generation process of test_arm64_signals.sh runner script
- re-added a fixed fake_sigreturn_misaligned_sp testcase and a properly
extended fake_sigreturn() helper
- added tests' TODO notes
v1-->v2:
- rebased on 5.2-rc7
- various makefile's cleanups
- mixed READMEs fixes
- fixed test_arm64_signals.sh runner script
- cleaned up assembly code in signal.S
- improved get_current_context() logic
- fixed SAFE_WRITE()
- common support code splitted into more chunks, each one introduced when
needed by some new testcases
- fixed some headers validation routines in testcases.c
- removed some still broken/immature tests:
+ fake_sigreturn_misaligned
+ fake_sigreturn_overflow_reserved
+ mangle_pc_invalid
+ mangle_sp_misaligned
- fixed some other testcases:
+ mangle_pstate_ssbs_regs: better checks of SSBS bit when feature unsupported
+ mangle_pstate_invalid_compat_toggle: name fix
+ mangle_pstate_invalid_mode_el[1-3]: precautionary zeroing PSTATE.MODE
+ fake_sigreturn_bad_magic, fake_sigreturn_bad_size,
fake_sigreturn_bad_size_for_magic0:
- accounting for available space...dropping extra when needed
- keeping alignent
- new testcases on FPSMID context:
+ fake_sigreturn_missing_fpsimd
+ fake_sigreturn_duplicated_fpsimd
Cristian Marussi (11):
kselftest: arm64: add skeleton Makefile
kselftest: arm64: add common utils and one testcase
kselftest: arm64: mangle_pstate_invalid_daif_bits
kselftest: arm64: mangle_pstate_invalid_mode_el[123][ht]
kselftest: arm64: mangle_pstate_ssbs_regs
kselftest: arm64: fake_sigreturn_bad_magic
kselftest: arm64: fake_sigreturn_bad_size_for_magic0
kselftest: arm64: fake_sigreturn_missing_fpsimd
kselftest: arm64: fake_sigreturn_duplicated_fpsimd
kselftest: arm64: fake_sigreturn_bad_size
kselftest: arm64: fake_sigreturn_misaligned_sp
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/arm64/Makefile | 70 +++-
tools/testing/selftests/arm64/README | 20 +
.../testing/selftests/arm64/signal/.gitignore | 3 +
tools/testing/selftests/arm64/signal/Makefile | 32 ++
tools/testing/selftests/arm64/signal/README | 59 +++
.../testing/selftests/arm64/signal/signals.S | 62 +++
.../selftests/arm64/signal/test_signals.c | 29 ++
.../selftests/arm64/signal/test_signals.h | 123 ++++++
.../arm64/signal/test_signals_utils.c | 367 ++++++++++++++++++
.../arm64/signal/test_signals_utils.h | 16 +
.../testcases/fake_sigreturn_bad_magic.c | 54 +++
.../testcases/fake_sigreturn_bad_size.c | 77 ++++
.../fake_sigreturn_bad_size_for_magic0.c | 49 +++
.../fake_sigreturn_duplicated_fpsimd.c | 52 +++
.../testcases/fake_sigreturn_misaligned_sp.c | 37 ++
.../testcases/fake_sigreturn_missing_fpsimd.c | 50 +++
.../mangle_pstate_invalid_compat_toggle.c | 31 ++
.../mangle_pstate_invalid_daif_bits.c | 35 ++
.../mangle_pstate_invalid_mode_el1h.c | 15 +
.../mangle_pstate_invalid_mode_el1t.c | 15 +
.../mangle_pstate_invalid_mode_el2h.c | 15 +
.../mangle_pstate_invalid_mode_el2t.c | 15 +
.../mangle_pstate_invalid_mode_el3h.c | 15 +
.../mangle_pstate_invalid_mode_el3t.c | 15 +
.../mangle_pstate_invalid_mode_template.h | 28 ++
.../testcases/mangle_pstate_ssbs_regs.c | 69 ++++
.../arm64/signal/testcases/testcases.c | 179 +++++++++
.../arm64/signal/testcases/testcases.h | 90 +++++
tools/testing/selftests/arm64/tags/Makefile | 10 +
.../arm64/{ => tags}/run_tags_test.sh | 0
.../selftests/arm64/{ => tags}/tags_test.c | 0
32 files changed, 1627 insertions(+), 6 deletions(-)
create mode 100644 tools/testing/selftests/arm64/README
create mode 100644 tools/testing/selftests/arm64/signal/.gitignore
create mode 100644 tools/testing/selftests/arm64/signal/Makefile
create mode 100644 tools/testing/selftests/arm64/signal/README
create mode 100644 tools/testing/selftests/arm64/signal/signals.S
create mode 100644 tools/testing/selftests/arm64/signal/test_signals.c
create mode 100644 tools/testing/selftests/arm64/signal/test_signals.h
create mode 100644 tools/testing/selftests/arm64/signal/test_signals_utils.c
create mode 100644 tools/testing/selftests/arm64/signal/test_signals_utils.h
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_magic.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_size.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_size_for_magic0.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_duplicated_fpsimd.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_misaligned_sp.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_missing_fpsimd.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_compat_toggle.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_daif_bits.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el1h.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el1t.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el2h.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el2t.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el3h.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el3t.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_template.h
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_ssbs_regs.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/testcases.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/testcases.h
create mode 100644 tools/testing/selftests/arm64/tags/Makefile
rename tools/testing/selftests/arm64/{ => tags}/run_tags_test.sh (100%)
rename tools/testing/selftests/arm64/{ => tags}/tags_test.c (100%)
--
2.17.1
Hello.
This patch set updates clone3 selftest in several aspects:
- adding checks for exit_signal invalid values handling;
- adding clone3 to selftests targets;
- enabling clone3 tests on all architectures;
- minor cleanups of the clone3 test.
This respin alignes additional clone3 self-tests with v3 of the
exit_signal checking patch[1].
Applied on top of brauer/linux.git/for-next.
Changes since v2[2]:
- CLONE3_ARGS_INVAL_EXIT_SIGNAL_NSIG check is now expected to fail.
Changes since v1[3]:
- exit_signal check extended to cover more cases of invalid
exit_signal value.
[1] https://lkml.org/lkml/2019/9/11/677
[2] https://lkml.org/lkml/2019/9/10/768
[3] https://lkml.org/lkml/2019/9/10/416
Eugene Syromiatnikov (6):
selftests/clone3: convert test modes into an enum
selftests/clone3: add a check for invalid exit_signal
selftests/clone3: use uint64_t for flags parameter
selftests/clone3: fix up format strings
selftests/clone3: enable clone3 self-tests on all architectures
selftests: add clone3 to TARGETS
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/clone3/Makefile | 4 +--
tools/testing/selftests/clone3/clone3.c | 64 ++++++++++++++++++++++++++++-----
3 files changed, 57 insertions(+), 12 deletions(-)
--
2.1.4
When running test_vmalloc.sh smoke the following print out states that
the fragment is missing.
# ./test_vmalloc.sh: You must have the following enabled in your kernel:
# CONFIG_TEST_VMALLOC=m
Rework to add the fragment 'CONFIG_TEST_VMALLOC=m' to the config file.
Fixes: a05ef00c9790 ("selftests/vm: add script helper for CONFIG_TEST_VMALLOC_MODULE")
Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org>
---
tools/testing/selftests/vm/config | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/vm/config b/tools/testing/selftests/vm/config
index 1c0d76cb5adf..93b90a9b1eeb 100644
--- a/tools/testing/selftests/vm/config
+++ b/tools/testing/selftests/vm/config
@@ -1,2 +1,3 @@
CONFIG_SYSVIPC=y
CONFIG_USERFAULTFD=y
+CONFIG_TEST_VMALLOC=m
--
2.20.1
Hello.
This patch set updates clone3 selftest in several aspects:
- adding checks for exit_signal invalid values handling;
- adding clone3 to selftests targets;
- enabling clone3 tests on all architectures;
- minor cleanups of the clone3 test.
Applied on top of brauer/linux.git/for-next.
Changes since v1[1]:
- exit_signal check extended to cover more cases of invalid
exit_signal value.
[1] https://lkml.org/lkml/2019/9/10/416
Eugene Syromiatnikov (6):
selftests/clone3: convert test modes into an enum
selftests/clone3: add a check for invalid exit_signal
selftests/clone3: use uint64_t for flags parameter
selftests/clone3: fix up format strings
selftests/clone3: enable clone3 self-tests on all architectures
selftests: add clone3 to TARGETS
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/clone3/Makefile | 4 +--
tools/testing/selftests/clone3/clone3.c | 64 ++++++++++++++++++++++++++++-----
3 files changed, 57 insertions(+), 12 deletions(-)
--
2.1.4
Hi,friend,
This is Julian Smith and i am purchasing manager from E-cloth Co.,LTD in the UK.
We are glad to know about your company from the web and we are interested in your products.
Could you kindly send us your Latest catalog and price list for our trial order.
Thanks and Best Regards,
Ms Julian Smith
Purchasing Manager
E-cloth Co.,LTD
Packet capture is useful from a general debugging standpoint, and is
useful in particular in debugging BPF programs that do packet processing.
For general debugging, being able to initiate arbitrary packet capture
from kprobes and tracepoints is highly valuable; e.g. what do the packets
that reach kfree_skb() - representing error codepaths - look like?
Arbitrary packet capture is distinct from the traditional concept of
pre-defined hooks, and gives much more flexibility in probing system
behaviour. For packet-processing BPF programs, packet capture can be useful
for doing things such as debugging checksum errors.
The intent of this RFC patchset is to initiate discussion on if and how to
work packet capture-specific capabilities into BPF. It is possible -
and indeed projects like xdpcap [1] have demonstrated how - to carry out
packet capture in BPF today via perf events, but the aim here is to
simplify both the in-BPF capture and the userspace collection.
The suggested approach is to add a new bpf helper - bpf_pcap() - to
simplify packet capture within BPF programs, and to enhance bpftool
to add a "pcap" subcommand to aid in retrieving packets. The helper
is for the most part a wrapper around perf event sending, using
data relevant for packet capture as metadata.
The end result is being able to capture packet data in the following
manner. For example if we add an iptables drop rule, we can observe
TCP SYN segments being freed at kfree_skb:
$ iptables -A INPUT -p tcp --dport 6666 -j DROP
$ bpftool pcap trace kprobe:kfree_skb proto ip data_out /tmp/cap &
$ nc 127.0.0.1 6666
Ncat: Connection timed out.
$ fg
^C
$ tshark -r /tmp/cap
Running as user "root" and group "root". This could be dangerous.
...
3 7 127.0.0.1 -> 127.0.0.1 TCP 60 54732 > ircu [SYN] Seq=0 Win=65495 Len=0 MSS=65495 SACK_PERM=1 TSval=696475539 TSecr=0 WS=128
...
Tracepoints are also supported, and by default data is sent to
stdout, so we can pipe to tcpdump:
$ bpftool pcap trace tracepoint:net_dev_xmit:arg1 proto eth | tcpdump -r -
reading from file -, link-type EN10MB (Ethernet)
00:16:49.150880 IP 10.11.12.13 > 10.11.12.14: ICMP echo reply, id 10519, seq 1, length 64
...
Patch 1 adds support for bpf_pcap() in skb and XDP programs. In those cases,
the argument is the relevant context (struct __sk_buff or xdp metadata)
from which we capture.
Patch 2 extends the helper to allow it to work for tracing programs, and in
that case the data argument is a pointer to an skb, derived from raw
tracepoint or kprobe arguments.
Patch 3 syncs uapi and tools headers for the new helper, flags and associated
pcap header type.
Patch 4 adds a feature test for libpcap which will be used in the next patch.
Patch 5 adds a "pcap" subcommand to bpftool to collect packet data from
BPF-driven perf event maps in existing programs. Also supplied are simple
tracepoint and kprobe programs which can be used to attach to a kprobe
or raw tracepoint to retrieve arguments and capture the associated skb.
Patch 6 adds documentation for the new pcap subcommand.
Patch 7 tests the pcap subcommand for tracing, skb and xdp programs.
Alan Maguire (7):
bpf: add bpf_pcap() helper to simplify packet capture
bpf: extend bpf_pcap support to tracing programs
bpf: sync tools/include/uapi/linux/bpf.h for pcap support
bpf: add libpcap feature test
bpf: add pcap support to bpftool
bpf: add documentation for bpftool pcap subcommand
bpf: add tests for bpftool packet capture
include/linux/bpf.h | 20 +
include/uapi/linux/bpf.h | 92 +++-
kernel/bpf/verifier.c | 4 +-
kernel/trace/bpf_trace.c | 214 +++++++++
net/core/filter.c | 67 +++
tools/bpf/bpftool/Documentation/bpftool-btf.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 1 +
.../bpf/bpftool/Documentation/bpftool-feature.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-map.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-net.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-pcap.rst | 119 +++++
tools/bpf/bpftool/Documentation/bpftool-perf.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool-prog.rst | 1 +
tools/bpf/bpftool/Documentation/bpftool.rst | 1 +
tools/bpf/bpftool/Makefile | 39 +-
tools/bpf/bpftool/main.c | 3 +-
tools/bpf/bpftool/main.h | 1 +
tools/bpf/bpftool/pcap.c | 496 +++++++++++++++++++++
tools/bpf/bpftool/progs/bpftool_pcap_kprobe.c | 80 ++++
tools/bpf/bpftool/progs/bpftool_pcap_tracepoint.c | 68 +++
tools/build/Makefile.feature | 2 +
tools/build/feature/Makefile | 4 +
tools/build/feature/test-libpcap.c | 26 ++
tools/include/uapi/linux/bpf.h | 92 +++-
tools/testing/selftests/bpf/Makefile | 3 +-
tools/testing/selftests/bpf/bpf_helpers.h | 11 +
.../testing/selftests/bpf/progs/bpftool_pcap_tc.c | 41 ++
.../testing/selftests/bpf/progs/bpftool_pcap_xdp.c | 39 ++
tools/testing/selftests/bpf/test_bpftool_pcap.sh | 132 ++++++
29 files changed, 1549 insertions(+), 12 deletions(-)
create mode 100644 tools/bpf/bpftool/Documentation/bpftool-pcap.rst
create mode 100644 tools/bpf/bpftool/pcap.c
create mode 100644 tools/bpf/bpftool/progs/bpftool_pcap_kprobe.c
create mode 100644 tools/bpf/bpftool/progs/bpftool_pcap_tracepoint.c
create mode 100644 tools/build/feature/test-libpcap.c
create mode 100644 tools/testing/selftests/bpf/progs/bpftool_pcap_tc.c
create mode 100644 tools/testing/selftests/bpf/progs/bpftool_pcap_xdp.c
create mode 100755 tools/testing/selftests/bpf/test_bpftool_pcap.sh
--
1.8.3.1
KTF has already been available for a while as a separate git repository with
means to facilitate use with any kernel version.
KTF can be used both for "pure" unit testing and for more pragmatic
approaches to component testing. Apart from some useful features that
KTF uses from the kernel toolbox (such as kallsyms, kprobes),
KTF does not depend on any special environment such as UML or on a
large set of mocked up APIs to be useful. KTF basically allows test
code to be inserted and managed as separate kernel modules, while
providing the tests convenient access to almost the full range of kernel
APIs, both exposed and private. And once a KTF test set exists, it
should be fairly easy to compile and run it against older versions of
the kernel as well.
This series proposes a non-intrusive integration of KTF into the kernel
hopefully presented in digestable pieces.
For convenience, the patch set is also available on top of v5.2
at https://github.com/knuto/linux/pull/new/ktf_v1 .
The high level structure of the KTF code is as follows:
External dependencies for the user land side:
* libnl3 for netlink communication
* googletest for test runner, reporting and test selection support.
Kernel components:
* Simple core test and test suite related abstractions:
core data structures ktf_case, ktf_test, an assertion macro "infrastructure"
with ASSERT_* and EXPECT_* macros and helper functions
* Bookkeeping data structures:
- ktf_map - a (key, value) mapping container used to implement management
of instances of the higher level abstraction needs, such as ktf_handle and ktf_context.
- ktf_handle: A global environment that a test runs within.
- ktf_context: a test suite specific abstraction to allow a test to execute within
one or more contexts. An example use of context can be a device
instance: A test can be instantiated to run on all available such
devices, according to test suite defined criteria.
* A generic netlink protocol (ktf_nl, ktf_unlproto) which defines operations to
allow a user space part of a test to query the kernel for test information,
invoke tests and get feedback reports about results.
* An alternative debugfs interface to allow examining and executing kernel-only tests
without a user level program.
* Support for overriding and modifying behaviour of kernel calls.
User mode components:
* A test executor based on and integrated with Googletest.
Googletest is one of several mature user land unit test suites for
C/C++. The choice allowed us to focus on kernel specific
functionality rather than having to reinvent too many wheels.
* Tools to aid in creating new test modules (suites):
To facilitate a developer friendly way of testing internals of a module or the
kernel itself, one of the important features of KTF, we often need to access
symbols deliberately not exposed from a module.
KTF contains a script used to create definitions based on kallsyms
lookup for easy access to symbols not exposed by a module or the kernel.
The user just provides a simple text file with a list of the symbols by
module.
This series is an attempt to address feedback from several people
that having the functionality readily available within the kernel repository
is desired.
An in-tree KTF allows test suites to be provided with the kernel, which makes
it easy to use KTF based suites as part of the kernel selftests and driver
test suites. Having the ability to still build and run the latest versions of
test suites against older kernels should be of great value to stable maintainers,
distributions and driver maintainers, who would want to have an easy path,
with minimal backporting efforts to make sure that criterias implemented by
new test logic is also valid for these kernels.
Our definite goal moving forward is to try to satisfy both needs in a
transparent way. The plan is to let the standalone KTF repository follow the
in-kernel one, and to allow test suites to be maintained similarly,
and to support maintenance by proper tooling.
Mode of integration into the kernel
===================================
One feature of KTF is that it allows tests to work, look and feel similar
whether they execute entirely in user mode, entirely in kernel mode,
or half and half (hybrid tests). KTF consist of both user space
and kernel code. Unlike e.g. kselftest, KTF in the Github version
does not attempt to address the test runner aspects of testing.
Due to the need for building modules, KTF requires access to kernel module
build facilities (obj-m). But KTF also has nontrivial needs for user
land building, and we think it is good to keep the build structure in a way that
allows KTF to be built both in-tree and out-of-tree without
necessarily having to reconfigure the kernel.
This first version of kernel integration of KTF solves this challenge
by co-locating everything associated with KTF under ktf/ as in the
github version, but use the little used hostprogs-y and hostlibs-y
features to build the user space side. The first patch in the series is
fixes to make it work in a natural way to suit our needs.
Positioning for natural building within the kernel tree
=======================================================
Currently we find significant amount of C level tests within the following paths:
tools/testing/selftests/ (kselftests, almost entirely user space so far)
lib/ (various kernel level mostly unit tests)
and in the making::
kunit/ (kernel only (UML))
So all kernel code are currently located directly within the kernel
build paths, accessed from the top level Makefile, to allow everything
to be controlled by config and from the main build targets for the
kernel. But this also poses challenges, in that .config has to be
modified to build tests. And once a .config is changed, we no longer
in principle logically operate on the same kernel.
A better approach seems to be to follow the principle
taken by kselftest: To have all the logic associated with the test
inside the test tree, and make it available for building separately
from the kernel itself. This require us to have a means to build
kernel modules from within the test tree, separately from the main
kernel paths. Currently this seems to only by supported via the M=
option used to build out-of-tree modules. This was also easy to get to work
for the kernel parts, based on the Github version of KTF, where we
already do this. With the additional need to compile user land code,
using the corresponding hostprogs-y and hostNNlibs-y seemed natural,
but this has been challenging: The build macros does not really
support hostprogs-y etc as "first class citizens" so some amount of
hacking is in there in this first draft version.
Using hostprogs-y etc is also a different approach that what is used
for C code in kselftest today, but we imagine that there's room for
unification here to get the best of both worlds, with the help of
the wider Kbuild community.
As an initial proposal, we have positioned ktf as an additional
kselftest target, under tools/testing/selftests/ktf, and the recommended:
make TARGETS="ktf" kselftest
way of building and running should work, even from normal user accounts,
if the user running it has sudo rights (for the kernel module insertion
and removal). This will run the selftests for KTF itself, and should
be a good starting point for adding more test cases. We also have had
activities going to take some of the existing test suites under lib/
and convert them into KTF based test suites, and will get back to this
later.
A trimmed down output from the above make target would look like this:
...
CC [M] tools/testing/selftests/ktf/kernel/ktf_override.o
LD [M] tools/testing/selftests/ktf/kernel/ktf.o
HOSTCC -fPIC tools/testing/selftests/ktf/lib/ktf_unlproto.o
HOSTCXX -fPIC tools/testing/selftests/ktf/lib/ktf_int.o
KTFSYMS tools/testing/selftests/ktf/selftest/ktf_syms.h
CC [M] tools/testing/selftests/ktf/selftest/self.o
LD [M] tools/testing/selftests/ktf/selftest/selftest.o
HOSTCXX tools/testing/selftests/ktf/user/ktftest.o
HOSTCXX tools/testing/selftests/ktf/user/hybrid.o
HOSTLD tools/testing/selftests/ktf/user/ktftest
Building modules, stage 2.
MODPOST 7 modules
LD [M] tools/testing/selftests/ktf/kernel/ktf.ko
LD [M] tools/testing/selftests/ktf/selftest/selftest.ko
running tests
make BUILD=/net/abi/local/abi/build/kernel/ktf/tools/testing/selftests -f scripts/runtests.mk run_tests
TAP version 13
1..1
...
ok 1 selftests: ktf: runtests.sh
We're looking forward to feedback on this, and also to more discussion
around unit testing at the testing & fuzzing workshop at LPC!
Alan Maguire (3):
ktf: Implementation of ktf support for overriding function entry and return.
ktf: A simple debugfs interface to test results
ktf: Simple coverage support
Knut Omang (16):
kbuild: Fixes to rules for host-cshlib and host-cxxshlib
ktf: Introduce the main part of the kernel side of ktf
ktf: Introduce a generic netlink protocol for test result communication
ktf: An implementation of a generic associative array container
ktf: Configurable context support for network info setup
ktf: resolve: A helper utility to aid in exposing private kernel symbols to KTF tests.
ktf: Add documentation for Kernel Test Framework (KTF)
ktf: Add a small test suite with a few tests to test KTF itself
ktf: Main part of user land library for executing tests
ktf: Integration logic for running ktf tests from googletest
ktf: Internal debugging facilities
ktf: Some simple examples
ktf: Some user applications to run tests
ktf: Toplevel ktf Makefile/makefile includes and scripts to run from kselftest
kselftests: Enable building ktf
Documentation/dev-tools: Add index entry for KTF documentation
Documentation/dev-tools/index.rst | 1 +-
Documentation/dev-tools/ktf/concepts.rst | 242 +++-
Documentation/dev-tools/ktf/debugging.rst | 248 +++-
Documentation/dev-tools/ktf/examples.rst | 26 +-
Documentation/dev-tools/ktf/features.rst | 307 ++++-
Documentation/dev-tools/ktf/implementation.rst | 70 +-
Documentation/dev-tools/ktf/index.rst | 14 +-
Documentation/dev-tools/ktf/installation.rst | 73 +-
Documentation/dev-tools/ktf/introduction.rst | 134 ++-
Documentation/dev-tools/ktf/progref.rst | 144 ++-
scripts/Makefile.host | 17 +-
tools/testing/selftests/Makefile | 1 +-
tools/testing/selftests/ktf/Makefile | 21 +-
tools/testing/selftests/ktf/examples/Makefile | 17 +-
tools/testing/selftests/ktf/examples/h2.c | 45 +-
tools/testing/selftests/ktf/examples/h3.c | 84 +-
tools/testing/selftests/ktf/examples/h4.c | 62 +-
tools/testing/selftests/ktf/examples/hello.c | 38 +-
tools/testing/selftests/ktf/examples/kgdemo.c | 61 +-
tools/testing/selftests/ktf/kernel/Makefile | 15 +-
tools/testing/selftests/ktf/kernel/ktf.h | 604 +++++++-
tools/testing/selftests/ktf/kernel/ktf_context.c | 409 +++++-
tools/testing/selftests/ktf/kernel/ktf_cov.c | 690 ++++++++-
tools/testing/selftests/ktf/kernel/ktf_cov.h | 94 +-
tools/testing/selftests/ktf/kernel/ktf_debugfs.c | 356 ++++-
tools/testing/selftests/ktf/kernel/ktf_debugfs.h | 34 +-
tools/testing/selftests/ktf/kernel/ktf_map.c | 261 +++-
tools/testing/selftests/ktf/kernel/ktf_map.h | 154 ++-
tools/testing/selftests/ktf/kernel/ktf_netctx.c | 132 ++-
tools/testing/selftests/ktf/kernel/ktf_netctx.h | 64 +-
tools/testing/selftests/ktf/kernel/ktf_nl.c | 516 ++++++-
tools/testing/selftests/ktf/kernel/ktf_nl.h | 15 +-
tools/testing/selftests/ktf/kernel/ktf_override.c | 45 +-
tools/testing/selftests/ktf/kernel/ktf_override.h | 15 +-
tools/testing/selftests/ktf/kernel/ktf_test.c | 397 +++++-
tools/testing/selftests/ktf/kernel/ktf_test.h | 381 ++++-
tools/testing/selftests/ktf/kernel/ktf_unlproto.h | 105 +-
tools/testing/selftests/ktf/lib/Makefile | 21 +-
tools/testing/selftests/ktf/lib/ktf.h | 114 +-
tools/testing/selftests/ktf/lib/ktf_debug.cc | 20 +-
tools/testing/selftests/ktf/lib/ktf_debug.h | 59 +-
tools/testing/selftests/ktf/lib/ktf_int.cc | 1031 ++++++++++++-
tools/testing/selftests/ktf/lib/ktf_int.h | 84 +-
tools/testing/selftests/ktf/lib/ktf_run.cc | 177 ++-
tools/testing/selftests/ktf/lib/ktf_unlproto.c | 21 +-
tools/testing/selftests/ktf/scripts/ktf_syms.mk | 16 +-
tools/testing/selftests/ktf/scripts/resolve | 188 ++-
tools/testing/selftests/ktf/scripts/runtests.mk | 3 +-
tools/testing/selftests/ktf/scripts/runtests.sh | 100 +-
tools/testing/selftests/ktf/scripts/top_make.mk | 14 +-
tools/testing/selftests/ktf/selftest/Makefile | 17 +-
tools/testing/selftests/ktf/selftest/context.c | 149 ++-
tools/testing/selftests/ktf/selftest/context.h | 15 +-
tools/testing/selftests/ktf/selftest/context_self.h | 34 +-
tools/testing/selftests/ktf/selftest/hybrid.c | 35 +-
tools/testing/selftests/ktf/selftest/hybrid.h | 24 +-
tools/testing/selftests/ktf/selftest/hybrid_self.h | 27 +-
tools/testing/selftests/ktf/selftest/ktf_syms.txt | 17 +-
tools/testing/selftests/ktf/selftest/self.c | 661 ++++++++-
tools/testing/selftests/ktf/user/Makefile | 26 +-
tools/testing/selftests/ktf/user/hybrid.cc | 39 +-
tools/testing/selftests/ktf/user/ktfcov.cc | 68 +-
tools/testing/selftests/ktf/user/ktfrun.cc | 20 +-
tools/testing/selftests/ktf/user/ktftest.cc | 46 +-
64 files changed, 8909 insertions(+), 9 deletions(-)
create mode 100644 Documentation/dev-tools/ktf/concepts.rst
create mode 100644 Documentation/dev-tools/ktf/debugging.rst
create mode 100644 Documentation/dev-tools/ktf/examples.rst
create mode 100644 Documentation/dev-tools/ktf/features.rst
create mode 100644 Documentation/dev-tools/ktf/implementation.rst
create mode 100644 Documentation/dev-tools/ktf/index.rst
create mode 100644 Documentation/dev-tools/ktf/installation.rst
create mode 100644 Documentation/dev-tools/ktf/introduction.rst
create mode 100644 Documentation/dev-tools/ktf/progref.rst
create mode 100644 tools/testing/selftests/ktf/Makefile
create mode 100644 tools/testing/selftests/ktf/examples/Makefile
create mode 100644 tools/testing/selftests/ktf/examples/h2.c
create mode 100644 tools/testing/selftests/ktf/examples/h3.c
create mode 100644 tools/testing/selftests/ktf/examples/h4.c
create mode 100644 tools/testing/selftests/ktf/examples/hello.c
create mode 100644 tools/testing/selftests/ktf/examples/kgdemo.c
create mode 100644 tools/testing/selftests/ktf/kernel/Makefile
create mode 100644 tools/testing/selftests/ktf/kernel/ktf.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_context.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_cov.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_cov.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_debugfs.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_debugfs.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_map.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_map.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_netctx.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_netctx.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_nl.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_nl.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_override.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_override.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_test.c
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_test.h
create mode 100644 tools/testing/selftests/ktf/kernel/ktf_unlproto.h
create mode 100644 tools/testing/selftests/ktf/lib/Makefile
create mode 100644 tools/testing/selftests/ktf/lib/ktf.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_debug.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_debug.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_int.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_int.h
create mode 100644 tools/testing/selftests/ktf/lib/ktf_run.cc
create mode 100644 tools/testing/selftests/ktf/lib/ktf_unlproto.c
create mode 100644 tools/testing/selftests/ktf/scripts/ktf_syms.mk
create mode 100755 tools/testing/selftests/ktf/scripts/resolve
create mode 100644 tools/testing/selftests/ktf/scripts/runtests.mk
create mode 100755 tools/testing/selftests/ktf/scripts/runtests.sh
create mode 100644 tools/testing/selftests/ktf/scripts/top_make.mk
create mode 100644 tools/testing/selftests/ktf/selftest/Makefile
create mode 100644 tools/testing/selftests/ktf/selftest/context.c
create mode 100644 tools/testing/selftests/ktf/selftest/context.h
create mode 100644 tools/testing/selftests/ktf/selftest/context_self.h
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid.c
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid.h
create mode 100644 tools/testing/selftests/ktf/selftest/hybrid_self.h
create mode 100644 tools/testing/selftests/ktf/selftest/ktf_syms.txt
create mode 100644 tools/testing/selftests/ktf/selftest/self.c
create mode 100644 tools/testing/selftests/ktf/user/Makefile
create mode 100644 tools/testing/selftests/ktf/user/hybrid.cc
create mode 100644 tools/testing/selftests/ktf/user/ktfcov.cc
create mode 100644 tools/testing/selftests/ktf/user/ktfrun.cc
create mode 100644 tools/testing/selftests/ktf/user/ktftest.cc
base-commit: 0ecfebd2b52404ae0c54a878c872bb93363ada36
--
git-series 0.9.1
The vprintk_emit() function is not available when CONFIG_PRINTK
is disabled:
kunit/test.c:22:9: error: implicit declaration of function 'vprintk_emit' [-Werror,-Wimplicit-function-declaration]
I suppose without printk(), there is not much use in kunit
either, so add a Kconfig depenedency here.
Signed-off-by: Arnd Bergmann <arnd(a)arndb.de>
---
kunit/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/kunit/Kconfig b/kunit/Kconfig
index 8541ef95b65a..e80d8af00454 100644
--- a/kunit/Kconfig
+++ b/kunit/Kconfig
@@ -6,6 +6,7 @@ menu "KUnit support"
config KUNIT
bool "Enable support for unit tests (KUnit)"
+ depends on PRINTK
help
Enables support for kernel unit tests (KUnit), a lightweight unit
testing and mocking framework for the Linux kernel. These tests are
--
2.20.0
The current logic prepends $(OUTPUT) only to the first member of
$(TEST_PROGS). Use $(foreach) loop to prepend it to each member.
Fixes: 1a940687e424 ("selftests: lib.mk: copy test scripts and test files for make O=dir run")
Signed-off-by: Ilya Leoshkevich <iii(a)linux.ibm.com>
---
tools/testing/selftests/lib.mk | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 1c8a1963d03f..857916ebbb9b 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -75,7 +75,8 @@ ifdef building_out_of_srctree
@rsync -aq $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT)
fi
@if [ "X$(TEST_PROGS)" != "X" ]; then
- $(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(OUTPUT)/$(TEST_PROGS))
+ $(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) \
+ $(foreach p,$(TEST_PROGS),$(OUTPUT)$(p)))
else
$(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS))
fi
--
2.21.0
tags_test.c relies on PR_SET_TAGGED_ADDR_CTRL/PR_TAGGED_ADDR_ENABLE being
present in system headers. When this is not the case the build of this
test fails with undeclared identifier errors.
Fix by providing the path to the KSFT installed kernel headers in CFLAGS.
Reported-by: Cristian Marussi <cristian.marussi(a)arm.com>
Suggested-by: Cristian Marussi <cristian.marussi(a)arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl(a)google.com>
---
tools/testing/selftests/arm64/Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index a61b2e743e99..f9f79fb272f0 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -4,6 +4,7 @@
ARCH ?= $(shell uname -m 2>/dev/null || echo not)
ifneq (,$(filter $(ARCH),aarch64 arm64))
+CFLAGS += -I../../../../usr/include/
TEST_GEN_PROGS := tags_test
TEST_PROGS := run_tags_test.sh
endif
--
2.23.0.187.g17f5b7556c-goog
Problem:
Currently tasks attempting to allocate more hugetlb memory than is available get
a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1].
However, if a task attempts to allocate hugetlb memory only more than its
hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call,
but will SIGBUS the task when it attempts to fault the memory in.
We have developers interested in using hugetlb_cgroups, and they have expressed
dissatisfaction regarding this behavior. We'd like to improve this
behavior such that tasks violating the hugetlb_cgroup limits get an error on
mmap/shmget time, rather than getting SIGBUS'd when they try to fault
the excess memory in.
The underlying problem is that today's hugetlb_cgroup accounting happens
at hugetlb memory *fault* time, rather than at *reservation* time.
Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and
the offending task gets SIGBUS'd.
Proposed Solution:
A new page counter named hugetlb.xMB.reservation_[limit|usage]_in_bytes. This
counter has slightly different semantics than
hugetlb.xMB.[limit|usage]_in_bytes:
- While usage_in_bytes tracks all *faulted* hugetlb memory,
reservation_usage_in_bytes tracks all *reserved* hugetlb memory.
- If a task attempts to reserve more memory than limit_in_bytes allows,
the kernel will allow it to do so. But if a task attempts to reserve
more memory than reservation_limit_in_bytes, the kernel will fail this
reservation.
This proposal is implemented in this patch, with tests to verify
functionality and show the usage.
Alternatives considered:
1. A new cgroup, instead of only a new page_counter attached to
the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code
duplication with hugetlb_cgroup. Keeping hugetlb related page counters under
hugetlb_cgroup seemed cleaner as well.
2. Instead of adding a new counter, we considered adding a sysctl that modifies
the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at
reservation time rather than fault time. Adding a new page_counter seems
better as userspace could, if it wants, choose to enforce different cgroups
differently: one via limit_in_bytes, and another via
reservation_limit_in_bytes. This could be very useful if you're
transitioning how hugetlb memory is partitioned on your system one
cgroup at a time, for example. Also, someone may find usage for both
limit_in_bytes and reservation_limit_in_bytes concurrently, and this
approach gives them the option to do so.
Caveats:
1. This support is implemented for cgroups-v1. I have not tried
hugetlb_cgroups with cgroups v2, and AFAICT it's not supported yet.
This is largely because we use cgroups-v1 for now. If required, I
can add hugetlb_cgroup support to cgroups v2 in this patch or
a follow up.
2. Most complicated bit of this patch I believe is: where to store the
pointer to the hugetlb_cgroup to uncharge at unreservation time?
Normally the cgroup pointers hang off the struct page. But, with
hugetlb_cgroup reservations, one task can reserve a specific page and another
task may fault it in (I believe), so storing the pointer in struct
page is not appropriate. Proposed approach here is to store the pointer in
the resv_map. See patch for details.
Signed-off-by: Mina Almasry <almasrymina(a)google.com>
[1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html
Changes in v3:
- Addressed comments of Hillf Danton:
- Added docs.
- cgroup_files now uses enum.
- Various readability improvements.
- Addressed comments of Mike Kravetz.
- region_* functions no longer coalesce file_region entries in the resv_map.
- region_add() and region_chg() refactored to make them much easier to
understand and remove duplicated code so this patch doesn't add too much
complexity.
- Refactored common functionality into helpers.
Changes in v2:
- Split the patch into a 5 patch series.
- Fixed patch subject.
Mina Almasry (6):
hugetlb_cgroup: Add hugetlb_cgroup reservation counter
hugetlb_cgroup: add interface for charge/uncharge hugetlb reservations
hugetlb_cgroup: add reservation accounting for private mappings
hugetlb_cgroup: add accounting for shared mappings
hugetlb_cgroup: Add hugetlb_cgroup reservation tests
hugetlb_cgroup: Add hugetlb_cgroup reservation docs
.../admin-guide/cgroup-v1/hugetlb.rst | 84 ++-
include/linux/hugetlb.h | 24 +-
include/linux/hugetlb_cgroup.h | 19 +-
mm/hugetlb.c | 493 ++++++++++++------
mm/hugetlb_cgroup.c | 187 +++++--
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 4 +
.../selftests/vm/charge_reserved_hugetlb.sh | 438 ++++++++++++++++
.../selftests/vm/write_hugetlb_memory.sh | 22 +
.../testing/selftests/vm/write_to_hugetlbfs.c | 252 +++++++++
10 files changed, 1300 insertions(+), 224 deletions(-)
create mode 100755 tools/testing/selftests/vm/charge_reserved_hugetlb.sh
create mode 100644 tools/testing/selftests/vm/write_hugetlb_memory.sh
create mode 100644 tools/testing/selftests/vm/write_to_hugetlbfs.c
--
2.23.0.187.g17f5b7556c-goog
=== Overview
arm64 has a feature called Top Byte Ignore, which allows to embed pointer
tags into the top byte of each pointer. Userspace programs (such as
HWASan, a memory debugging tool [1]) might use this feature and pass
tagged user pointers to the kernel through syscalls or other interfaces.
Right now the kernel is already able to handle user faults with tagged
pointers, due to these patches:
1. 81cddd65 ("arm64: traps: fix userspace cache maintenance emulation on a
tagged pointer")
2. 7dcd9dd8 ("arm64: hw_breakpoint: fix watchpoint matching for tagged
pointers")
3. 276e9327 ("arm64: entry: improve data abort handling of tagged
pointers")
This patchset extends tagged pointer support to syscall arguments.
As per the proposed ABI change [3], tagged pointers are only allowed to be
passed to syscalls when they point to memory ranges obtained by anonymous
mmap() or sbrk() (see the patchset [3] for more details).
For non-memory syscalls this is done by untaging user pointers when the
kernel performs pointer checking to find out whether the pointer comes
from userspace (most notably in access_ok). The untagging is done only
when the pointer is being checked, the tag is preserved as the pointer
makes its way through the kernel and stays tagged when the kernel
dereferences the pointer when perfoming user memory accesses.
The mmap and mremap (only new_addr) syscalls do not currently accept
tagged addresses. Architectures may interpret the tag as a background
colour for the corresponding vma.
Other memory syscalls (mprotect, etc.) don't do user memory accesses but
rather deal with memory ranges, and untagged pointers are better suited to
describe memory ranges internally. Thus for memory syscalls we untag
pointers completely when they enter the kernel.
=== Other approaches
One of the alternative approaches to untagging that was considered is to
completely strip the pointer tag as the pointer enters the kernel with
some kind of a syscall wrapper, but that won't work with the countless
number of different ioctl calls. With this approach we would need a custom
wrapper for each ioctl variation, which doesn't seem practical.
An alternative approach to untagging pointers in memory syscalls prologues
is to inspead allow tagged pointers to be passed to find_vma() (and other
vma related functions) and untag them there. Unfortunately, a lot of
find_vma() callers then compare or subtract the returned vma start and end
fields against the pointer that was being searched. Thus this approach
would still require changing all find_vma() callers.
=== Testing
The following testing approaches has been taken to find potential issues
with user pointer untagging:
1. Static testing (with sparse [2] and separately with a custom static
analyzer based on Clang) to track casts of __user pointers to integer
types to find places where untagging needs to be done.
2. Static testing with grep to find parts of the kernel that call
find_vma() (and other similar functions) or directly compare against
vm_start/vm_end fields of vma.
3. Static testing with grep to find parts of the kernel that compare
user pointers with TASK_SIZE or other similar consts and macros.
4. Dynamic testing: adding BUG_ON(has_tag(addr)) to find_vma() and running
a modified syzkaller version that passes tagged pointers to the kernel.
Based on the results of the testing the requried patches have been added
to the patchset.
=== Notes
This patchset is meant to be merged together with "arm64 relaxed ABI" [3].
This patchset is a prerequisite for ARM's memory tagging hardware feature
support [4].
This patchset has been merged into the Pixel 2 & 3 kernel trees and is
now being used to enable testing of Pixel phones with HWASan.
Thanks!
[1] http://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
[2] https://github.com/lucvoo/sparse-dev/commit/5f960cb10f56ec2017c128ef9d16060…
[3] https://lkml.org/lkml/2019/6/12/745
[4] https://community.arm.com/processors/b/blog/posts/arm-a-profile-architectur…
=== History
Changes in v18:
- Reverted the selftest back to not using the LD_PRELOAD approach.
- Added prctl(PR_SET_TAGGED_ADDR_CTRL) call to the selftest.
- Reworded the patch descriptions to make them less oriented on arm64
only.
- Catalin's patch: "I added a Kconfig option and dropped the prctl args
zero check. There is some minor clean-up as well".
Changes in v17:
- The "uaccess: add noop untagged_addr definition" patch is dropped, as it
was merged into upstream named as "uaccess: add noop untagged_addr
definition".
- Merged "mm, arm64: untag user pointers in do_pages_move" into
"mm, arm64: untag user pointers passed to memory syscalls".
- Added "arm64: Introduce prctl() options to control the tagged user
addresses ABI" patch from Catalin.
- Add tags_lib.so to tools/testing/selftests/arm64/.gitignore.
- Added a comment clarifying untagged in mremap.
- Moved untagging back into mlx4_get_umem_mr() for the IB patch.
Changes in v16:
- Moved untagging for memory syscalls from arm64 wrappers back to generic
code.
- Dropped untagging for the following memory syscalls: brk, mmap, munmap;
mremap (only dropped for new_address); mmap_pgoff (not used on arm64);
remap_file_pages (deprecated); shmat, shmdt (work on shared memory).
- Changed kselftest to LD_PRELOAD a shared library that overrides malloc
to return tagged pointers.
- Rebased onto 5.2-rc3.
Changes in v15:
- Removed unnecessary untagging from radeon_ttm_tt_set_userptr().
- Removed unnecessary untagging from amdgpu_ttm_tt_set_userptr().
- Moved untagging to validate_range() in userfaultfd code.
- Moved untagging to ib_uverbs_(re)reg_mr() from mlx4_get_umem_mr().
- Rebased onto 5.1.
Changes in v14:
- Moved untagging for most memory syscalls to an arm64 specific
implementation, instead of doing that in the common code.
- Dropped "net, arm64: untag user pointers in tcp_zerocopy_receive", since
the provided user pointers don't come from an anonymous map and thus are
not covered by this ABI relaxation.
- Dropped "kernel, arm64: untag user pointers in prctl_set_mm*".
- Moved untagging from __check_mem_type() to tee_shm_register().
- Updated untagging for the amdgpu and radeon drivers to cover the MMU
notifier, as suggested by Felix.
- Since this ABI relaxation doesn't actually allow tagged instruction
pointers, dropped the following patches:
- Dropped "tracing, arm64: untag user pointers in seq_print_user_ip".
- Dropped "uprobes, arm64: untag user pointers in find_active_uprobe".
- Dropped "bpf, arm64: untag user pointers in stack_map_get_build_id_offset".
- Rebased onto 5.1-rc7 (37624b58).
Changes in v13:
- Simplified untagging in tcp_zerocopy_receive().
- Looked at find_vma() callers in drivers/, which allowed to identify a
few other places where untagging is needed.
- Added patch "mm, arm64: untag user pointers in get_vaddr_frames".
- Added patch "drm/amdgpu, arm64: untag user pointers in
amdgpu_ttm_tt_get_user_pages".
- Added patch "drm/radeon, arm64: untag user pointers in
radeon_ttm_tt_pin_userptr".
- Added patch "IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr".
- Added patch "media/v4l2-core, arm64: untag user pointers in
videobuf_dma_contig_user_get".
- Added patch "tee/optee, arm64: untag user pointers in check_mem_type".
- Added patch "vfio/type1, arm64: untag user pointers".
Changes in v12:
- Changed untagging in tcp_zerocopy_receive() to also untag zc->address.
- Fixed untagging in prctl_set_mm* to only untag pointers for vma lookups
and validity checks, but leave them as is for actual user space accesses.
- Updated the link to the v2 of the "arm64 relaxed ABI" patchset [3].
- Dropped the documentation patch, as the "arm64 relaxed ABI" patchset [3]
handles that.
Changes in v11:
- Added "uprobes, arm64: untag user pointers in find_active_uprobe" patch.
- Added "bpf, arm64: untag user pointers in stack_map_get_build_id_offset"
patch.
- Fixed "tracing, arm64: untag user pointers in seq_print_user_ip" to
correctly perform subtration with a tagged addr.
- Moved untagged_addr() from SYSCALL_DEFINE3(mprotect) and
SYSCALL_DEFINE4(pkey_mprotect) to do_mprotect_pkey().
- Moved untagged_addr() definition for other arches from
include/linux/memory.h to include/linux/mm.h.
- Changed untagging in strn*_user() to perform userspace accesses through
tagged pointers.
- Updated the documentation to mention that passing tagged pointers to
memory syscalls is allowed.
- Updated the test to use malloc'ed memory instead of stack memory.
Changes in v10:
- Added "mm, arm64: untag user pointers passed to memory syscalls" back.
- New patch "fs, arm64: untag user pointers in fs/userfaultfd.c".
- New patch "net, arm64: untag user pointers in tcp_zerocopy_receive".
- New patch "kernel, arm64: untag user pointers in prctl_set_mm*".
- New patch "tracing, arm64: untag user pointers in seq_print_user_ip".
Changes in v9:
- Rebased onto 4.20-rc6.
- Used u64 instead of __u64 in type casts in the untagged_addr macro for
arm64.
- Added braces around (addr) in the untagged_addr macro for other arches.
Changes in v8:
- Rebased onto 65102238 (4.20-rc1).
- Added a note to the cover letter on why syscall wrappers/shims that untag
user pointers won't work.
- Added a note to the cover letter that this patchset has been merged into
the Pixel 2 kernel tree.
- Documentation fixes, in particular added a list of syscalls that don't
support tagged user pointers.
Changes in v7:
- Rebased onto 17b57b18 (4.19-rc6).
- Dropped the "arm64: untag user address in __do_user_fault" patch, since
the existing patches already handle user faults properly.
- Dropped the "usb, arm64: untag user addresses in devio" patch, since the
passed pointer must come from a vma and therefore be untagged.
- Dropped the "arm64: annotate user pointers casts detected by sparse"
patch (see the discussion to the replies of the v6 of this patchset).
- Added more context to the cover letter.
- Updated Documentation/arm64/tagged-pointers.txt.
Changes in v6:
- Added annotations for user pointer casts found by sparse.
- Rebased onto 050cdc6c (4.19-rc1+).
Changes in v5:
- Added 3 new patches that add untagging to places found with static
analysis.
- Rebased onto 44c929e1 (4.18-rc8).
Changes in v4:
- Added a selftest for checking that passing tagged pointers to the
kernel succeeds.
- Rebased onto 81e97f013 (4.18-rc1+).
Changes in v3:
- Rebased onto e5c51f30 (4.17-rc6+).
- Added linux-arch@ to the list of recipients.
Changes in v2:
- Rebased onto 2d618bdf (4.17-rc3+).
- Removed excessive untagging in gup.c.
- Removed untagging pointers returned from __uaccess_mask_ptr.
Changes in v1:
- Rebased onto 4.17-rc1.
Changes in RFC v2:
- Added "#ifndef untagged_addr..." fallback in linux/uaccess.h instead of
defining it for each arch individually.
- Updated Documentation/arm64/tagged-pointers.txt.
- Dropped "mm, arm64: untag user addresses in memory syscalls".
- Rebased onto 3eb2ce82 (4.16-rc7).
Signed-off-by: Andrey Konovalov <andreyknvl(a)google.com>
Andrey Konovalov (14):
arm64: untag user pointers in access_ok and __uaccess_mask_ptr
lib: untag user pointers in strn*_user
mm: untag user pointers passed to memory syscalls
mm: untag user pointers in mm/gup.c
mm: untag user pointers in get_vaddr_frames
fs/namespace: untag user pointers in copy_mount_options
userfaultfd: untag user pointers
drm/amdgpu: untag user pointers
drm/radeon: untag user pointers in radeon_gem_userptr_ioctl
IB/mlx4: untag user pointers in mlx4_get_umem_mr
media/v4l2-core: untag user pointers in videobuf_dma_contig_user_get
tee/shm: untag user pointers in tee_shm_register
vfio/type1: untag user pointers in vaddr_get_pfn
selftests, arm64: add a selftest for passing tagged pointers to kernel
Catalin Marinas (1):
arm64: Introduce prctl() options to control the tagged user addresses
ABI
arch/arm64/Kconfig | 9 +++
arch/arm64/include/asm/processor.h | 8 +++
arch/arm64/include/asm/thread_info.h | 1 +
arch/arm64/include/asm/uaccess.h | 12 +++-
arch/arm64/kernel/process.c | 72 +++++++++++++++++++
.../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 2 +
drivers/gpu/drm/radeon/radeon_gem.c | 2 +
drivers/infiniband/hw/mlx4/mr.c | 7 +-
drivers/media/v4l2-core/videobuf-dma-contig.c | 9 +--
drivers/tee/tee_shm.c | 1 +
drivers/vfio/vfio_iommu_type1.c | 2 +
fs/namespace.c | 2 +-
fs/userfaultfd.c | 22 +++---
include/uapi/linux/prctl.h | 5 ++
kernel/sys.c | 12 ++++
lib/strncpy_from_user.c | 3 +-
lib/strnlen_user.c | 3 +-
mm/frame_vector.c | 2 +
mm/gup.c | 4 ++
mm/madvise.c | 2 +
mm/mempolicy.c | 3 +
mm/migrate.c | 2 +-
mm/mincore.c | 2 +
mm/mlock.c | 4 ++
mm/mprotect.c | 2 +
mm/mremap.c | 7 ++
mm/msync.c | 2 +
tools/testing/selftests/arm64/.gitignore | 1 +
tools/testing/selftests/arm64/Makefile | 11 +++
.../testing/selftests/arm64/run_tags_test.sh | 12 ++++
tools/testing/selftests/arm64/tags_test.c | 29 ++++++++
32 files changed, 232 insertions(+), 25 deletions(-)
create mode 100644 tools/testing/selftests/arm64/.gitignore
create mode 100644 tools/testing/selftests/arm64/Makefile
create mode 100755 tools/testing/selftests/arm64/run_tags_test.sh
create mode 100644 tools/testing/selftests/arm64/tags_test.c
--
2.22.0.410.gd8fdbe21b5-goog
From: Anders Roxell <anders.roxell(a)linaro.org>
[ Upstream commit 0604409df9e04cdec7b08d471c8c1c0c10b5554d ]
When running test_kmod.sh the following shows up
# sysctl cannot stat /proc/sys/net/core/bpf_jit_enable No such file or directory
cannot: stat_/proc/sys/net/core/bpf_jit_enable #
# sysctl cannot stat /proc/sys/net/core/bpf_jit_harden No such file or directory
cannot: stat_/proc/sys/net/core/bpf_jit_harden #
Rework to enable CONFIG_BPF_JIT to solve "No such file or directory"
Signed-off-by: Anders Roxell <anders.roxell(a)linaro.org>
Signed-off-by: Daniel Borkmann <daniel(a)iogearbox.net>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/bpf/config | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index f7a0744db31e1..5dc109f4c0970 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -34,3 +34,4 @@ CONFIG_NET_MPLS_GSO=m
CONFIG_MPLS_ROUTING=m
CONFIG_MPLS_IPTUNNEL=m
CONFIG_IPV6_SIT=m
+CONFIG_BPF_JIT=y
--
2.20.1
From: Ilya Leoshkevich <iii(a)linux.ibm.com>
[ Upstream commit 806ce6e2117a42528e7bb979e04e28229b34a612 ]
test_cgroup_storage fails on s390 with an assertion failure: packets are
dropped when they shouldn't. The problem is that BPF_DW packet count is
accessed as BPF_W with an offset of 0, which is not correct on
big-endian machines.
Since the point of this test is not to verify narrow loads/stores,
simply use BPF_DW when working with packet counts.
Fixes: 68cfa3ac6b8d ("selftests/bpf: add a cgroup storage test")
Fixes: 919646d2a3a9 ("selftests/bpf: extend the storage test to test per-cpu cgroup storage")
Signed-off-by: Ilya Leoshkevich <iii(a)linux.ibm.com>
Signed-off-by: Daniel Borkmann <daniel(a)iogearbox.net>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/bpf/test_cgroup_storage.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/bpf/test_cgroup_storage.c b/tools/testing/selftests/bpf/test_cgroup_storage.c
index 2fc4625c1a150..6557290043911 100644
--- a/tools/testing/selftests/bpf/test_cgroup_storage.c
+++ b/tools/testing/selftests/bpf/test_cgroup_storage.c
@@ -20,9 +20,9 @@ int main(int argc, char **argv)
BPF_MOV64_IMM(BPF_REG_2, 0), /* flags, not used */
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
BPF_FUNC_get_local_storage),
- BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_0, 0),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_3, 0x1),
- BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_3, 0),
+ BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_3, 0),
BPF_LD_MAP_FD(BPF_REG_1, 0), /* map fd */
BPF_MOV64_IMM(BPF_REG_2, 0), /* flags, not used */
@@ -30,7 +30,7 @@ int main(int argc, char **argv)
BPF_FUNC_get_local_storage),
BPF_MOV64_IMM(BPF_REG_1, 1),
BPF_STX_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0),
- BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0),
BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 0x1),
BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
--
2.20.1
Hi
this patchset aims to add the initial arch-specific arm64 support to
kselftest starting with signals-related test-cases.
A common internal test-case layout is proposed which then it is anyway
wired-up to the toplevel kselftest Makefile, so that it should be possible
at the end to run it on an arm64 target in the usual way with KSFT.
~/linux# make TARGETS=arm64 kselftest
New KSFT arm64 testcases live inside tools/testing/selftests/arm64 grouped by
family inside subdirectories: arm64/signal is the first family proposed with
this series. arm64/signal tests can be run via KSFT or standalone.
Thanks
Cristian
Notes:
-----
- further details in the included READMEs
- more tests still to be written (current strategy is going through the related
Kernel signal-handling code and write a test for each possible and sensible code-path)
A few ideas in testcases/TODO.readme
- a bit of overlap around KSFT arm64/ Makefiles is expected with this:
https://lore.kernel.org/linux-arm-kernel/c1e6aad230658bc175b42d92daeff2e300…
Changes:
--------
v2-->v3:
- rebased on v5.3-rc2
- better test result characterization looking for
SEGV_ACCERR in si_code on SIGSEGV
- using KSFT Framework macros for retvalues
- removed SAFE_WRITE()/dump_uc: buggy, un-needed and unused
- reviewed generation process of test_arm64_signals.sh runner script
- re-added a fixed fake_sigreturn_misaligned_sp testcase and a properly
extended fake_sigreturn() helper
- added tests' TODO notes
v1-->v2:
- rebased on 5.2-rc7
- various makefile's cleanups
- mixed READMEs fixes
- fixed test_arm64_signals.sh runner script
- cleaned up assembly code in signal.S
- improved get_current_context() logic
- fixed SAFE_WRITE()
- common support code splitted into more chunks, each one introduced when
needed by some new testcases
- fixed some headers validation routines in testcases.c
- removed some still broken/immature tests:
+ fake_sigreturn_misaligned
+ fake_sigreturn_overflow_reserved
+ mangle_pc_invalid
+ mangle_sp_misaligned
- fixed some other testcases:
+ mangle_pstate_ssbs_regs: better checks of SSBS bit when feature unsupported
+ mangle_pstate_invalid_compat_toggle: name fix
+ mangle_pstate_invalid_mode_el[1-3]: precautionary zeroing PSTATE.MODE
+ fake_sigreturn_bad_magic, fake_sigreturn_bad_size,
fake_sigreturn_bad_size_for_magic0:
- accounting for available space...dropping extra when needed
- keeping alignent
- new testcases on FPSMID context:
+ fake_sigreturn_missing_fpsimd
+ fake_sigreturn_duplicated_fpsimd
Cristian Marussi (11):
kselftest: arm64: introduce new boilerplate code
kselftest: arm64: adds first test and common utils
kselftest: arm64: mangle_pstate_invalid_daif_bits
kselftest: arm64: mangle_pstate_invalid_mode_el
kselftest: arm64: mangle_pstate_ssbs_regs
kselftest: arm64: fake_sigreturn_bad_magic
kselftest: arm64: fake_sigreturn_bad_size_for_magic0
kselftest: arm64: fake_sigreturn_missing_fpsimd
kselftest: arm64: fake_sigreturn_duplicated_fpsimd
kselftest: arm64: fake_sigreturn_bad_size
kselftest: arm64: fake_sigreturn_misaligned_sp
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/arm64/Makefile | 51 +++
tools/testing/selftests/arm64/README | 43 +++
.../testing/selftests/arm64/signal/.gitignore | 6 +
tools/testing/selftests/arm64/signal/Makefile | 88 +++++
tools/testing/selftests/arm64/signal/README | 59 +++
.../testing/selftests/arm64/signal/signals.S | 73 ++++
.../arm64/signal/test_arm64_signals.src_shell | 55 +++
.../selftests/arm64/signal/test_signals.c | 26 ++
.../selftests/arm64/signal/test_signals.h | 141 +++++++
.../arm64/signal/test_signals_utils.c | 354 ++++++++++++++++++
.../arm64/signal/test_signals_utils.h | 16 +
.../arm64/signal/testcases/.gitignore | 11 +
.../arm64/signal/testcases/TODO.readme | 7 +
.../testcases/fake_sigreturn_bad_magic.c | 63 ++++
.../testcases/fake_sigreturn_bad_size.c | 85 +++++
.../fake_sigreturn_bad_size_for_magic0.c | 57 +++
.../fake_sigreturn_duplicated_fpsimd.c | 62 +++
.../testcases/fake_sigreturn_misaligned_sp.c | 30 ++
.../testcases/fake_sigreturn_missing_fpsimd.c | 44 +++
.../mangle_pstate_invalid_compat_toggle.c | 25 ++
.../mangle_pstate_invalid_daif_bits.c | 28 ++
.../mangle_pstate_invalid_mode_el1.c | 29 ++
.../mangle_pstate_invalid_mode_el2.c | 29 ++
.../mangle_pstate_invalid_mode_el3.c | 29 ++
.../testcases/mangle_pstate_ssbs_regs.c | 56 +++
.../arm64/signal/testcases/testcases.c | 150 ++++++++
.../arm64/signal/testcases/testcases.h | 83 ++++
28 files changed, 1701 insertions(+)
create mode 100644 tools/testing/selftests/arm64/Makefile
create mode 100644 tools/testing/selftests/arm64/README
create mode 100644 tools/testing/selftests/arm64/signal/.gitignore
create mode 100644 tools/testing/selftests/arm64/signal/Makefile
create mode 100644 tools/testing/selftests/arm64/signal/README
create mode 100644 tools/testing/selftests/arm64/signal/signals.S
create mode 100755 tools/testing/selftests/arm64/signal/test_arm64_signals.src_shell
create mode 100644 tools/testing/selftests/arm64/signal/test_signals.c
create mode 100644 tools/testing/selftests/arm64/signal/test_signals.h
create mode 100644 tools/testing/selftests/arm64/signal/test_signals_utils.c
create mode 100644 tools/testing/selftests/arm64/signal/test_signals_utils.h
create mode 100644 tools/testing/selftests/arm64/signal/testcases/.gitignore
create mode 100644 tools/testing/selftests/arm64/signal/testcases/TODO.readme
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_magic.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_size.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_bad_size_for_magic0.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_duplicated_fpsimd.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_misaligned_sp.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/fake_sigreturn_missing_fpsimd.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_compat_toggle.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_daif_bits.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el1.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el2.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_invalid_mode_el3.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/mangle_pstate_ssbs_regs.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/testcases.c
create mode 100644 tools/testing/selftests/arm64/signal/testcases/testcases.h
--
2.17.1
Previously KUnit assumed that printk would always be present, which is
not a valid assumption to make. Fix that by removing call to
vprintk_emit, and calling printk directly.
This fixes a build error[1] reported by Randy.
For context this change comes after much discussion. My first stab[2] at
this was just to make the KUnit logging code compile out; however, it
was agreed that if we were going to use vprintk_emit, then vprintk_emit
should provide a no-op stub, which lead to my second attempt[3]. In
response to me trying to stub out vprintk_emit, Sergey Senozhatsky
suggested a way for me to remove our usage of vprintk_emit, which led to
my third attempt at solving this[4].
In my third version of this patch[4], I completely removed vprintk_emit,
as suggested by Sergey; however, there was a bit of debate over whether
Sergey's solution was the best. The debate arose due to Sergey's version
resulting in a checkpatch warning, which resulted in a debate over
correct printk usage. Joe Perches offered an alternative fix which was
somewhat less far reaching than what Sergey had suggested and
importantly relied on continuing to use %pV. Much of the debated
centered around whether %pV should be widely used, and whether Sergey's
version would result in object size bloat. Ultimately, we decided to go
with Sergey's version.
Reported-by: Randy Dunlap <rdunlap(a)infradead.org>
Link[1]: https://lore.kernel.org/linux-kselftest/c7229254-0d90-d90e-f3df-5b6d6fc0b51…
Link[2]: https://lore.kernel.org/linux-kselftest/20190827174932.44177-1-brendanhiggi…
Link[3]: https://lore.kernel.org/linux-kselftest/20190827234835.234473-1-brendanhigg…
Link[4]: https://lore.kernel.org/linux-kselftest/20190828093143.163302-1-brendanhigg…
Cc: Stephen Rothwell <sfr(a)canb.auug.org.au>
Cc: Sergey Senozhatsky <sergey.senozhatsky.work(a)gmail.com>
Cc: Joe Perches <joe(a)perches.com>
Cc: Tim.Bird(a)sony.com
Signed-off-by: Brendan Higgins <brendanhiggins(a)google.com>
Acked-by: Randy Dunlap <rdunlap(a)infradead.org> # build-tested
Reviewed-by: Petr Mladek <pmladek(a)suse.com>
---
Sorry for the long commit message, but given the long discussion (and
some of the confusion that occurred in the discussion), it seemed
appropriate to summarize the discussion around this patch up to this
point (especially since one of the proposed patches was under a separate
patch subject).
Changes Since v3:
Renamed kunit_print_level to kunit_printk, and changed the KERN_LEVEL
macro parameter (in kunit_printk) to lvl, as suggested by Joe.
---
include/kunit/test.h | 5 ++--
kunit/test.c | 57 +++++---------------------------------------
2 files changed, 8 insertions(+), 54 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 8b7eb03d4971..dba48304b3bd 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -339,9 +339,8 @@ static inline void *kunit_kzalloc(struct kunit *test, size_t size, gfp_t gfp)
void kunit_cleanup(struct kunit *test);
-void __printf(3, 4) kunit_printk(const char *level,
- const struct kunit *test,
- const char *fmt, ...);
+#define kunit_printk(lvl, test, fmt, ...) \
+ printk(lvl "\t# %s: " fmt, (test)->name, ##__VA_ARGS__)
/**
* kunit_info() - Prints an INFO level message associated with @test.
diff --git a/kunit/test.c b/kunit/test.c
index b2ca9b94c353..c83c0fa59cbd 100644
--- a/kunit/test.c
+++ b/kunit/test.c
@@ -16,36 +16,12 @@ static void kunit_set_failure(struct kunit *test)
WRITE_ONCE(test->success, false);
}
-static int kunit_vprintk_emit(int level, const char *fmt, va_list args)
-{
- return vprintk_emit(0, level, NULL, 0, fmt, args);
-}
-
-static int kunit_printk_emit(int level, const char *fmt, ...)
-{
- va_list args;
- int ret;
-
- va_start(args, fmt);
- ret = kunit_vprintk_emit(level, fmt, args);
- va_end(args);
-
- return ret;
-}
-
-static void kunit_vprintk(const struct kunit *test,
- const char *level,
- struct va_format *vaf)
-{
- kunit_printk_emit(level[1] - '0', "\t# %s: %pV", test->name, vaf);
-}
-
static void kunit_print_tap_version(void)
{
static bool kunit_has_printed_tap_version;
if (!kunit_has_printed_tap_version) {
- kunit_printk_emit(LOGLEVEL_INFO, "TAP version 14\n");
+ pr_info("TAP version 14\n");
kunit_has_printed_tap_version = true;
}
}
@@ -64,10 +40,8 @@ static size_t kunit_test_cases_len(struct kunit_case *test_cases)
static void kunit_print_subtest_start(struct kunit_suite *suite)
{
kunit_print_tap_version();
- kunit_printk_emit(LOGLEVEL_INFO, "\t# Subtest: %s\n", suite->name);
- kunit_printk_emit(LOGLEVEL_INFO,
- "\t1..%zd\n",
- kunit_test_cases_len(suite->test_cases));
+ pr_info("\t# Subtest: %s\n", suite->name);
+ pr_info("\t1..%zd\n", kunit_test_cases_len(suite->test_cases));
}
static void kunit_print_ok_not_ok(bool should_indent,
@@ -87,9 +61,7 @@ static void kunit_print_ok_not_ok(bool should_indent,
else
ok_not_ok = "not ok";
- kunit_printk_emit(LOGLEVEL_INFO,
- "%s%s %zd - %s\n",
- indent, ok_not_ok, test_number, description);
+ pr_info("%s%s %zd - %s\n", indent, ok_not_ok, test_number, description);
}
static bool kunit_suite_has_succeeded(struct kunit_suite *suite)
@@ -133,11 +105,11 @@ static void kunit_print_string_stream(struct kunit *test,
kunit_err(test,
"Could not allocate buffer, dumping stream:\n");
list_for_each_entry(fragment, &stream->fragments, node) {
- kunit_err(test, fragment->fragment);
+ kunit_err(test, "%s", fragment->fragment);
}
kunit_err(test, "\n");
} else {
- kunit_err(test, buf);
+ kunit_err(test, "%s", buf);
kunit_kfree(test, buf);
}
}
@@ -504,20 +476,3 @@ void kunit_cleanup(struct kunit *test)
kunit_resource_free(test, resource);
}
}
-
-void kunit_printk(const char *level,
- const struct kunit *test,
- const char *fmt, ...)
-{
- struct va_format vaf;
- va_list args;
-
- va_start(args, fmt);
-
- vaf.fmt = fmt;
- vaf.va = &args;
-
- kunit_vprintk(test, level, &vaf);
-
- va_end(args);
-}
--
2.23.0.187.g17f5b7556c-goog
Previously KUnit assumed that printk would always be present, which is
not a valid assumption to make. Fix that by removing call to
vprintk_emit, and calling printk directly.
This fixes a build error[1] reported by Randy.
For context this change comes after much discussion. My first stab[2] at
this was just to make the KUnit logging code compile out; however, it
was agreed that if we were going to use vprintk_emit, then vprintk_emit
should provide a no-op stub, which lead to my second attempt[3]. In
response to me trying to stub out vprintk_emit, Sergey Senozhatsky
suggested a way for me to remove our usage of vprintk_emit, which led to
my third attempt at solving this[4].
In my previous version of this patch[4], I completely removed
vprintk_emit, as suggested by Sergey; however, there was a bit of debate
over whether Sergey's solution was the best. The debate arose due to
Sergey's version resulting in a checkpatch warning, which resulted in a
debate over correct printk usage. Joe Perches offered an alternative fix
which was somewhat less far reaching than what Sergey had suggested and
importantly relied on continuing to use %pV. Much of the debated
centered around whether %pV should be widely used, and whether Sergey's
version would result in object size bloat. Ultimately, we decided to go
with Sergey's version.
Reported-by: Randy Dunlap <rdunlap(a)infradead.org>
Link[1]: https://lore.kernel.org/linux-kselftest/c7229254-0d90-d90e-f3df-5b6d6fc0b51…
Link[2]: https://lore.kernel.org/linux-kselftest/20190827174932.44177-1-brendanhiggi…
Link[3]: https://lore.kernel.org/linux-kselftest/20190827234835.234473-1-brendanhigg…
Link[4]: https://lore.kernel.org/linux-kselftest/20190828093143.163302-1-brendanhigg…
Cc: Stephen Rothwell <sfr(a)canb.auug.org.au>
Cc: Sergey Senozhatsky <sergey.senozhatsky.work(a)gmail.com>
Cc: Joe Perches <joe(a)perches.com>
Cc: Tim.Bird(a)sony.com
Signed-off-by: Brendan Higgins <brendanhiggins(a)google.com>
Acked-by: Randy Dunlap <rdunlap(a)infradead.org> # build-tested
Reviewed-by: Petr Mladek <pmladek(a)suse.com>
---
Sorry for the long commit message, but given the long discussion (and
some of the confusion that occurred in the discussion), it seemed
appropriate to summarize the discussion around this patch up to this
point (especially since one of the proposed patches was under a separate
patch subject).
No changes have been made to this patch since v2, other than the commit
log.
---
include/kunit/test.h | 11 ++++-----
kunit/test.c | 57 +++++---------------------------------------
2 files changed, 11 insertions(+), 57 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 8b7eb03d4971..efad2eacd6ba 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -339,9 +339,8 @@ static inline void *kunit_kzalloc(struct kunit *test, size_t size, gfp_t gfp)
void kunit_cleanup(struct kunit *test);
-void __printf(3, 4) kunit_printk(const char *level,
- const struct kunit *test,
- const char *fmt, ...);
+#define kunit_print_level(KERN_LEVEL, test, fmt, ...) \
+ printk(KERN_LEVEL "\t# %s: " fmt, (test)->name, ##__VA_ARGS__)
/**
* kunit_info() - Prints an INFO level message associated with @test.
@@ -353,7 +352,7 @@ void __printf(3, 4) kunit_printk(const char *level,
* Takes a variable number of format parameters just like printk().
*/
#define kunit_info(test, fmt, ...) \
- kunit_printk(KERN_INFO, test, fmt, ##__VA_ARGS__)
+ kunit_print_level(KERN_INFO, test, fmt, ##__VA_ARGS__)
/**
* kunit_warn() - Prints a WARN level message associated with @test.
@@ -364,7 +363,7 @@ void __printf(3, 4) kunit_printk(const char *level,
* Prints a warning level message.
*/
#define kunit_warn(test, fmt, ...) \
- kunit_printk(KERN_WARNING, test, fmt, ##__VA_ARGS__)
+ kunit_print_level(KERN_WARNING, test, fmt, ##__VA_ARGS__)
/**
* kunit_err() - Prints an ERROR level message associated with @test.
@@ -375,7 +374,7 @@ void __printf(3, 4) kunit_printk(const char *level,
* Prints an error level message.
*/
#define kunit_err(test, fmt, ...) \
- kunit_printk(KERN_ERR, test, fmt, ##__VA_ARGS__)
+ kunit_print_level(KERN_ERR, test, fmt, ##__VA_ARGS__)
/**
* KUNIT_SUCCEED() - A no-op expectation. Only exists for code clarity.
diff --git a/kunit/test.c b/kunit/test.c
index b2ca9b94c353..c83c0fa59cbd 100644
--- a/kunit/test.c
+++ b/kunit/test.c
@@ -16,36 +16,12 @@ static void kunit_set_failure(struct kunit *test)
WRITE_ONCE(test->success, false);
}
-static int kunit_vprintk_emit(int level, const char *fmt, va_list args)
-{
- return vprintk_emit(0, level, NULL, 0, fmt, args);
-}
-
-static int kunit_printk_emit(int level, const char *fmt, ...)
-{
- va_list args;
- int ret;
-
- va_start(args, fmt);
- ret = kunit_vprintk_emit(level, fmt, args);
- va_end(args);
-
- return ret;
-}
-
-static void kunit_vprintk(const struct kunit *test,
- const char *level,
- struct va_format *vaf)
-{
- kunit_printk_emit(level[1] - '0', "\t# %s: %pV", test->name, vaf);
-}
-
static void kunit_print_tap_version(void)
{
static bool kunit_has_printed_tap_version;
if (!kunit_has_printed_tap_version) {
- kunit_printk_emit(LOGLEVEL_INFO, "TAP version 14\n");
+ pr_info("TAP version 14\n");
kunit_has_printed_tap_version = true;
}
}
@@ -64,10 +40,8 @@ static size_t kunit_test_cases_len(struct kunit_case *test_cases)
static void kunit_print_subtest_start(struct kunit_suite *suite)
{
kunit_print_tap_version();
- kunit_printk_emit(LOGLEVEL_INFO, "\t# Subtest: %s\n", suite->name);
- kunit_printk_emit(LOGLEVEL_INFO,
- "\t1..%zd\n",
- kunit_test_cases_len(suite->test_cases));
+ pr_info("\t# Subtest: %s\n", suite->name);
+ pr_info("\t1..%zd\n", kunit_test_cases_len(suite->test_cases));
}
static void kunit_print_ok_not_ok(bool should_indent,
@@ -87,9 +61,7 @@ static void kunit_print_ok_not_ok(bool should_indent,
else
ok_not_ok = "not ok";
- kunit_printk_emit(LOGLEVEL_INFO,
- "%s%s %zd - %s\n",
- indent, ok_not_ok, test_number, description);
+ pr_info("%s%s %zd - %s\n", indent, ok_not_ok, test_number, description);
}
static bool kunit_suite_has_succeeded(struct kunit_suite *suite)
@@ -133,11 +105,11 @@ static void kunit_print_string_stream(struct kunit *test,
kunit_err(test,
"Could not allocate buffer, dumping stream:\n");
list_for_each_entry(fragment, &stream->fragments, node) {
- kunit_err(test, fragment->fragment);
+ kunit_err(test, "%s", fragment->fragment);
}
kunit_err(test, "\n");
} else {
- kunit_err(test, buf);
+ kunit_err(test, "%s", buf);
kunit_kfree(test, buf);
}
}
@@ -504,20 +476,3 @@ void kunit_cleanup(struct kunit *test)
kunit_resource_free(test, resource);
}
}
-
-void kunit_printk(const char *level,
- const struct kunit *test,
- const char *fmt, ...)
-{
- struct va_format vaf;
- va_list args;
-
- va_start(args, fmt);
-
- vaf.fmt = fmt;
- vaf.va = &args;
-
- kunit_vprintk(test, level, &vaf);
-
- va_end(args);
-}
--
2.23.0.187.g17f5b7556c-goog
Hello everyone,
We are planning to put together a Automated Testing devroom at FOSDEM
2020 [1] and would like to see if there is any interest in attending
such an event.
If we're not able to put together a set of talks that fills up a (half) day we
shouldn't apply for a devroom.
We have a few people prepared to give talks if the devroom happens.
We would like to know your interest in attending (or, better yet, make a
proposal to speak or for a discussion session) before 16 September.
Cheers,
Anders
[1] https://fosdem.org/2020/
Convert hard spaces to tabs in usage options.
Suggested-by: Shuah Khan <shuah(a)kernel.org>
Signed-off-by: George G. Davis <george_davis(a)mentor.com>
---
tools/testing/selftests/watchdog/watchdog-test.c | 25 ++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/tools/testing/selftests/watchdog/watchdog-test.c b/tools/testing/selftests/watchdog/watchdog-test.c
index c35989ffbc6b..3fff1ee20a7f 100644
--- a/tools/testing/selftests/watchdog/watchdog-test.c
+++ b/tools/testing/selftests/watchdog/watchdog-test.c
@@ -70,18 +70,19 @@ static void term(int sig)
static void usage(char *progname)
{
printf("Usage: %s [options]\n", progname);
- printf(" -f, --file Open watchdog device file\n");
- printf(" Default is /dev/watchdog\n");
- printf(" -b, --bootstatus Get last boot status (Watchdog/POR)\n");
- printf(" -d, --disable Turn off the watchdog timer\n");
- printf(" -e, --enable Turn on the watchdog timer\n");
- printf(" -h, --help Print the help message\n");
- printf(" -p, --pingrate=P Set ping rate to P seconds (default %d)\n", DEFAULT_PING_RATE);
- printf(" -t, --timeout=T Set timeout to T seconds\n");
- printf(" -T, --gettimeout Get the timeout\n");
- printf(" -n, --pretimeout=T Set the pretimeout to T seconds\n");
- printf(" -N, --getpretimeout Get the pretimeout\n");
- printf(" -L, --gettimeleft Get the time left until timer expires\n");
+ printf(" -f, --file\t\tOpen watchdog device file\n");
+ printf("\t\t\tDefault is /dev/watchdog\n");
+ printf(" -b, --bootstatus\tGet last boot status (Watchdog/POR)\n");
+ printf(" -d, --disable\t\tTurn off the watchdog timer\n");
+ printf(" -e, --enable\t\tTurn on the watchdog timer\n");
+ printf(" -h, --help\t\tPrint the help message\n");
+ printf(" -p, --pingrate=P\tSet ping rate to P seconds (default %d)\n",
+ DEFAULT_PING_RATE);
+ printf(" -t, --timeout=T\tSet timeout to T seconds\n");
+ printf(" -T, --gettimeout\tGet the timeout\n");
+ printf(" -n, --pretimeout=T\tSet the pretimeout to T seconds\n");
+ printf(" -N, --getpretimeout\tGet the pretimeout\n");
+ printf(" -L, --gettimeleft\tGet the time left until timer expires\n");
printf("\n");
printf("Parameters are parsed left-to-right in real-time.\n");
printf("Example: %s -d -t 10 -p 5 -e\n", progname);
--
2.7.4