If a signed number field starts with a '-' the field width must be > 1,
or unlimited, to allow at least one digit after the '-'.
This patch adds a check for this. If a signed field starts with '-'
and field_width == 1 the scanf will quit.
It is ok for a signed number field to have a field width of 1 if it
starts with a digit. In that case the single digit can be converted.
Signed-off-by: Richard Fitzgerald <rf(a)opensource.cirrus.com>
Reviewed-by: Petr Mladek <pmladek(a)suse.com>
---
lib/vsprintf.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 3b53c73580c5..28bb26cd1f67 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -3434,8 +3434,12 @@ int vsscanf(const char *buf, const char *fmt, va_list args)
str = skip_spaces(str);
digit = *str;
- if (is_sign && digit == '-')
+ if (is_sign && digit == '-') {
+ if (field_width == 1)
+ break;
+
digit = *(str + 1);
+ }
if (!digit
|| (base == 16 && !isxdigit(digit))
--
2.20.1
When using `kunit.py run` to run tests, users must populate a
`kunitconfig` file to select the options the tests are hidden behind and
all their dependencies.
The patch [1] to allow specifying a path to kunitconfig promises to make
this nicer as we can have checked in files corresponding to different
sets of tests.
But it's still annoying
1) when trying to run a subet of tests
2) when you want to run tests that don't have such a pre-existing
kunitconfig and selecting all the necessary options is tricky.
This patch series aims to alleviate both:
1) `kunit.py run 'my-suite-*'`
I.e. use my current kunitconfig, but just run suites that match this glob
2) `kunit.py run --alltests 'my-suite-*'`
I.e. use allyesconfig so I don't have to worry about writing a
kunitconfig at all (this is a bit overkill, but it works!)
See the first commit message for more details and discussion about
future work.
This patch series also includes a bugfix for a latent bug that can't be
triggered right now but has worse consequences as a result of the
changes needed to plumb in this suite name glob.
[1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@g…
Daniel Latypov (3):
kunit: add kunit.filter_glob cmdline option to filter suites
kunit: tool: add support for filtering suites by glob
kunit: tool: fix unintentional statefulness in run_kernel()
lib/kunit/Kconfig | 1 +
lib/kunit/executor.c | 85 ++++++++++++++++++++++++++---
tools/testing/kunit/kunit.py | 21 +++++--
tools/testing/kunit/kunit_kernel.py | 6 +-
4 files changed, 99 insertions(+), 14 deletions(-)
base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
--
2.30.0.365.g02bc693789-goog
Sequence Number api provides interfaces for unsigned atomic up counters.
There are a number of atomic_t usages in the kernel where atomic_t api
is used for counting sequence numbers and other statistical counters.
Several of these usages, convert atomic_read() and atomic_inc_return()
return values to unsigned. Introducing sequence number ops supports
these use-cases with a standard core-api.
Sequence Number ops provide interfaces to initialize, increment and get
the sequence number. These ops also check for overflow and log message to
indicate when overflow occurs. This check is intended to help catch cases
where overflow could lead to problems.
Since v2:
- Uses atomic_inc_return() for incrementing the sequence number.
- No longer uses atomic_read()
Shuah Khan (7):
seqnum_ops: Introduce Sequence Number Ops
selftests: lib:test_seqnum_ops: add new test for seqnum_ops
drivers/acpi: convert seqno to use seqnum_ops
drivers/acpi/apei: convert seqno to seqnum_ops
drivers/staging/rtl8723bs: convert event_seq to use seqnum_ops
drivers/staging/rtl8188eu: convert event_seq to use seqnum_ops
kobject: convert uevent_seqnum to seqnum_ops
Documentation/core-api/index.rst | 1 +
Documentation/core-api/seqnum_ops.rst | 62 ++++++++
MAINTAINERS | 8 ++
drivers/acpi/acpi_extlog.c | 8 +-
drivers/acpi/apei/ghes.c | 8 +-
drivers/staging/rtl8188eu/core/rtw_mlme_ext.c | 23 ++-
.../staging/rtl8188eu/include/rtw_mlme_ext.h | 3 +-
drivers/staging/rtl8723bs/core/rtw_cmd.c | 3 +-
drivers/staging/rtl8723bs/core/rtw_mlme_ext.c | 33 +++--
drivers/staging/rtl8723bs/include/rtw_cmd.h | 3 +-
.../staging/rtl8723bs/include/rtw_mlme_ext.h | 3 +-
include/linux/kobject.h | 3 +-
include/linux/seqnum_ops.h | 131 +++++++++++++++++
kernel/ksysfs.c | 3 +-
lib/Kconfig | 9 ++
lib/Makefile | 1 +
lib/kobject_uevent.c | 9 +-
lib/test_seqnum_ops.c | 133 ++++++++++++++++++
tools/testing/selftests/lib/Makefile | 1 +
tools/testing/selftests/lib/config | 1 +
.../testing/selftests/lib/test_seqnum_ops.sh | 10 ++
21 files changed, 423 insertions(+), 33 deletions(-)
create mode 100644 Documentation/core-api/seqnum_ops.rst
create mode 100644 include/linux/seqnum_ops.h
create mode 100644 lib/test_seqnum_ops.c
create mode 100755 tools/testing/selftests/lib/test_seqnum_ops.sh
--
2.27.0
When using `kunit.py run` to run tests, users must populate a
`kunitconfig` file to select the options the tests are hidden behind and
all their dependencies.
The patch [1] to allow specifying a path to kunitconfig promises to make
this nicer as we can have checked in files corresponding to different
sets of tests.
But it's still annoying
1) when trying to run a subet of tests
2) when you want to run tests that don't have such a pre-existing
kunitconfig and selecting all the necessary options is tricky.
This patch series aims to alleviate both:
1) `kunit.py run 'my-suite-*'`
I.e. use my current kunitconfig, but just run suites that match this glob
2) `kunit.py run --alltests 'my-suite-*'`
I.e. use allyesconfig so I don't have to worry about writing a
kunitconfig at all.
See the first commit message for more details and discussion about
future work.
This patch series also includes a bugfix for a latent bug that can't be
triggered right now but has worse consequences as a result of the
changes needed to plumb in this suite name glob.
[1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@g…
---
v1 -> v2:
Fix free of `suites` subarray in suite_set.
Found by Dan Carpenter and kernel test robot.
v2 -> v3:
Add MODULE_PARM_DESC() for kunit.filter_glob.
Daniel Latypov (3):
kunit: add kunit.filter_glob cmdline option to filter suites
kunit: tool: add support for filtering suites by glob
kunit: tool: fix unintentional statefulness in run_kernel()
lib/kunit/Kconfig | 1 +
lib/kunit/executor.c | 93 ++++++++++++++++++++++++++---
tools/testing/kunit/kunit.py | 21 +++++--
tools/testing/kunit/kunit_kernel.py | 6 +-
4 files changed, 106 insertions(+), 15 deletions(-)
base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
--
2.30.0.478.g8a0d178c01-goog
Currently running tests via KUnit tool means tweaking a .kunitconfig
file, which you'd keep around locally and never commit.
This changes makes it so users can pass in a path to a kunitconfig.
One of the imagined use cases is having kunitconfig fragments in-tree
to formalize interesting sets of tests for features/subsystems, e.g.
$ ./tools/testing/kunit/kunit.py run --kunticonfig=fs/ext4/kunitconfig
For now, this hypothetical fs/ext4/kunitconfig would contain
CONFIG_KUNIT=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_KUNIT_TESTS=y
At the moment, it's not hard to manually whip up this file, but as more
and more tests get added, this will get tedious.
It also opens the door to documenting how to run all the tests relevant
to a specific subsystem or feature as a simple one-liner.
This can be seen as an analogue to tools/testing/selftests/*/config
But in the case of KUnit, the tests live in the same directory as the
code-under-test, so it feels more natural to allow the kunitconfig
fragments to live anywhere. (Though, people could create a separate
directory if wanted; this patch imposes no restrictions on the path).
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Changes since v1: change from a positional arg to a flag --kunitconfig.
Ensure that it gets added for `kunit.py config` and all other commands.
---
tools/testing/kunit/kunit.py | 9 +++++---
tools/testing/kunit/kunit_kernel.py | 12 ++++++----
tools/testing/kunit/kunit_tool_test.py | 32 ++++++++++++++++++++++++++
3 files changed, 46 insertions(+), 7 deletions(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index e808a47c839b..02871a363f76 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -182,6 +182,9 @@ def add_common_opts(parser) -> None:
parser.add_argument('--alltests',
help='Run all KUnit tests through allyesconfig',
action='store_true')
+ parser.add_argument('--kunitconfig',
+ help='Path to Kconfig fragment that enables KUnit tests',
+ metavar='kunitconfig')
def add_build_opts(parser) -> None:
parser.add_argument('--jobs',
@@ -256,7 +259,7 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
@@ -274,7 +277,7 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
@@ -286,7 +289,7 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 2076a5a2d060..0b461663e7d9 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -123,7 +123,7 @@ def get_outfile_path(build_dir) -> str:
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, defconfig=DEFAULT_KUNITCONFIG_PATH) -> None:
+ def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
signal.signal(signal.SIGINT, self.signal_handler)
self._ops = LinuxSourceTreeOperations()
@@ -131,9 +131,13 @@ class LinuxSourceTree(object):
if not load_config:
return
- kunitconfig_path = get_kunitconfig_path(build_dir)
- if not os.path.exists(kunitconfig_path):
- shutil.copyfile(defconfig, kunitconfig_path)
+ if kunitconfig_path:
+ if not os.path.exists(kunitconfig_path):
+ raise ConfigError(f'Specified kunitconfig ({kunitconfig_path}) does not exist')
+ else:
+ kunitconfig_path = get_kunitconfig_path(build_dir)
+ if not os.path.exists(kunitconfig_path):
+ shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, kunitconfig_path)
self._kconfig = kunit_config.Kconfig()
self._kconfig.read_from_file(kunitconfig_path)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index b593f4448e83..22f50b931138 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -12,6 +12,7 @@ from unittest import mock
import tempfile, shutil # Handling test_tmpdir
import json
+import signal
import os
import kunit_config
@@ -250,6 +251,23 @@ class KUnitParserTest(unittest.TestCase):
result.status)
self.assertEqual('kunit-resource-test', result.suites[0].name)
+class LinuxSourceTreeTest(unittest.TestCase):
+
+ def setUp(self):
+ mock.patch.object(signal, 'signal').start()
+ self.addCleanup(mock.patch.stopall)
+
+ def test_invalid_kunitconfig(self):
+ with self.assertRaisesRegex(kunit_kernel.ConfigError, 'nonexistent.* does not exist'):
+ kunit_kernel.LinuxSourceTree('', kunitconfig_path='/nonexistent_file')
+
+ def test_valid_kunitconfig(self):
+ with tempfile.NamedTemporaryFile('wt') as kunitconfig:
+ tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=kunitconfig.name)
+
+ # TODO: add more test cases.
+
+
class KUnitJsonTest(unittest.TestCase):
def _json_for(self, log_file):
@@ -399,5 +417,19 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.'))
+ @mock.patch.object(kunit_kernel, 'LinuxSourceTree')
+ def test_run_kunitconfig(self, mock_linux_init):
+ mock_linux_init.return_value = self.linux_source_mock
+ kunit.main(['run', '--kunitconfig=mykunitconfig'])
+ # Just verify that we parsed and initialized it correctly here.
+ mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+
+ @mock.patch.object(kunit_kernel, 'LinuxSourceTree')
+ def test_config_kunitconfig(self, mock_linux_init):
+ mock_linux_init.return_value = self.linux_source_mock
+ kunit.main(['config', '--kunitconfig=mykunitconfig'])
+ # Just verify that we parsed and initialized it correctly here.
+ mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+
if __name__ == '__main__':
unittest.main()
base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
--
2.30.0.365.g02bc693789-goog
./usage.rst contains fairly long examples and explanations of things
like how to fake a class and how to use parameterized tests (and how you
could do table-driven tests yourself).
It's not exactly necessary information, so we add a new page with more
digestible tips like "use kunit_kzalloc() instead of kzalloc() so you
don't have to worry about calling kfree() yourself" and the like.
Change start.rst to point users to this new page first and let them know
that usage.rst is more of optional further reading.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Documentation/dev-tools/kunit/index.rst | 2 +
Documentation/dev-tools/kunit/start.rst | 4 +-
Documentation/dev-tools/kunit/tips.rst | 115 ++++++++++++++++++++++++
3 files changed, 120 insertions(+), 1 deletion(-)
create mode 100644 Documentation/dev-tools/kunit/tips.rst
diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst
index c234a3ab3c34..848478838347 100644
--- a/Documentation/dev-tools/kunit/index.rst
+++ b/Documentation/dev-tools/kunit/index.rst
@@ -13,6 +13,7 @@ KUnit - Unit Testing for the Linux Kernel
api/index
style
faq
+ tips
What is KUnit?
==============
@@ -88,6 +89,7 @@ How do I use it?
================
* :doc:`start` - for new users of KUnit
+* :doc:`tips` - for short examples of best practices
* :doc:`usage` - for a more detailed explanation of KUnit features
* :doc:`api/index` - for the list of KUnit APIs used for testing
* :doc:`kunit-tool` - for more information on the kunit_tool helper script
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 454f307813ea..c09e2747c958 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -233,5 +233,7 @@ Congrats! You just wrote your first KUnit test!
Next Steps
==========
-* Check out the :doc:`usage` page for a more
+* Check out the :doc:`tips` page for tips on
+ writing idiomatic KUnit tests.
+* Optional: see the :doc:`usage` page for a more
in-depth explanation of KUnit.
diff --git a/Documentation/dev-tools/kunit/tips.rst b/Documentation/dev-tools/kunit/tips.rst
new file mode 100644
index 000000000000..a6ca0af14098
--- /dev/null
+++ b/Documentation/dev-tools/kunit/tips.rst
@@ -0,0 +1,115 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============================
+Tips For Writing KUnit Tests
+============================
+
+Exiting early on failed expectations
+------------------------------------
+
+``KUNIT_EXPECT_EQ`` and friends will mark the test as failed and continue
+execution. In some cases, it's unsafe to continue and you can use the
+``KUNIT_ASSERT`` variant to exit on failure.
+
+.. code-block:: c
+
+ void example_test_user_alloc_function(struct kunit *test)
+ {
+ void *object = alloc_some_object_for_me();
+
+ /* Make sure we got a valid pointer back. */
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);
+ do_something_with_object(object);
+ }
+
+Allocating memory
+-----------------
+
+Where you would use ``kzalloc``, you should prefer ``kunit_kzalloc`` instead.
+KUnit will ensure the memory is freed once the test completes.
+
+This is particularly useful since it lets you use the ``KUNIT_ASSERT_EQ``
+macros to exit early from a test without having to worry about remembering to
+call ``kfree``.
+
+Example:
+
+.. code-block:: c
+
+ void example_test_allocation(struct kunit *test)
+ {
+ char *buffer = kunit_kzalloc(test, 16, GFP_KERNEL);
+ /* Ensure allocation succeeded. */
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer);
+
+ KUNIT_ASSERT_STREQ(test, buffer, "");
+ }
+
+
+Testing static functions
+------------------------
+
+If you don't want to expose functions or variables just for testing, one option
+is to conditionally ``#include`` the test file at the end of your .c file, e.g.
+
+.. code-block:: c
+
+ /* In my_file.c */
+
+ static int do_interesting_thing();
+
+ #ifdef CONFIG_MY_KUNIT_TEST
+ #include "my_kunit_test.c"
+ #endif
+
+Injecting test-only code
+------------------------
+
+Similarly to the above, it can be useful to add test-specific logic.
+
+.. code-block:: c
+
+ /* In my_file.h */
+
+ #ifdef CONFIG_MY_KUNIT_TEST
+ /* Defined in my_kunit_test.c */
+ void test_only_hook(void);
+ #else
+ void test_only_hook(void) { }
+ #endif
+
+TODO(dlatypov(a)google.com): add an example of using ``current->kunit_test`` in
+such a hook when it's not only updated for ``CONFIG_KASAN=y``.
+
+Customizing error messages
+--------------------------
+
+Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG`` variant.
+These take a format string and arguments to provide additional context to the automatically generated error messages.
+
+.. code-block:: c
+
+ char some_str[41];
+ generate_sha1_hex_string(some_str);
+
+ /* Before. Not easy to tell why the test failed. */
+ KUNIT_EXPECT_EQ(test, strlen(some_str), 40);
+
+ /* After. Now we see the offending string. */
+ KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);
+
+Alternatively, one can take full control over the error message by using ``KUNIT_FAIL()``, e.g.
+
+.. code-block:: c
+
+ /* Before */
+ KUNIT_EXPECT_EQ(test, some_setup_function(), 0);
+
+ /* After: full control over the failure message. */
+ if (some_setup_function())
+ KUNIT_FAIL(test, "Failed to setup thing for testing");
+
+Next Steps
+==========
+* Optional: see the :doc:`usage` page for a more
+ in-depth explanation of KUnit.
base-commit: 6ee1d745b7c9fd573fba142a2efdad76a9f1cb04
--
2.30.0.280.ga3ce27912f-goog
The primary change is that we want to encourage people to respect
KUNIT_ALL_TESTS to make it easy to run all the relevant tests for a
given config.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Documentation/dev-tools/kunit/start.rst | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 454f307813ea..560f27af4619 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -196,8 +196,9 @@ Now add the following to ``drivers/misc/Kconfig``:
.. code-block:: kconfig
config MISC_EXAMPLE_TEST
- bool "Test for my example"
+ tristate "Test for my example" if !KUNIT_ALL_TESTS
depends on MISC_EXAMPLE && KUNIT=y
+ default KUNIT_ALL_TESTS
and the following to ``drivers/misc/Makefile``:
base-commit: 146620506274bd24d52fb1c589110a30eed8240b
--
2.30.0.296.g2bfb1c46d8-goog
When a number of tests fail, it can be useful to get higher-level
statistics of how many tests are failing (or how many parameters are
failing in parameterised tests), and in what cases or suites. This is
already done by some non-KUnit tests, so add support for automatically
generating these for KUnit tests.
This change adds a 'kunit_stats_enabled' switch which has three values:
- 0: No stats are printed (current behaviour)
- 1: Stats are printed only for tests/suites with more than one
subtests, and at least one failure (new default)
- 2: Always print test statistics
For parameterised tests, the summary line looks as follows:
" # inode_test_xtimestamp_decoding: 0 / 16 test parameters failed"
For test suites, it looks like this:
"# ext4_inode_test: (0 / 1) tests failed (0 / 16 test parameters)"
kunit_tool is also updated to correctly ignore diagnostic lines, so that
these statistics do not prevent the result from parsing.
Signed-off-by: David Gow <davidgow(a)google.com>
---
This is largely a follow-up to the discussion here:
https://lore.kernel.org/linux-kselftest/CABVgOSmy4n_LGwDS7yWfoLftcQzxv6S+iX…
Does this seem like a sensible addition?
Cheers,
-- David
lib/kunit/test.c | 71 +++++++++++++++++++++++++++++
tools/testing/kunit/kunit_parser.py | 2 +-
2 files changed, 72 insertions(+), 1 deletion(-)
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index ec9494e914ef..711e269366a7 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -9,6 +9,7 @@
#include <kunit/test.h>
#include <linux/kernel.h>
#include <linux/kref.h>
+#include <linux/moduleparam.h>
#include <linux/sched/debug.h>
#include <linux/sched.h>
@@ -16,6 +17,40 @@
#include "string-stream.h"
#include "try-catch-impl.h"
+/*
+ * KUnit statistic mode:
+ * 0 - disabled
+ * 1 - only when there is at least one failure, and more than one subtest
+ * 2 - enabled
+ */
+static int kunit_stats_enabled = 1;
+core_param(kunit_stats_enabled, kunit_stats_enabled, int, 0644);
+
+static bool kunit_should_print_stats(int num_failures, int num_subtests)
+{
+ if (kunit_stats_enabled == 0)
+ return false;
+
+ if (kunit_stats_enabled == 2)
+ return true;
+
+ return (num_failures > 0 && num_subtests > 1);
+}
+
+static void kunit_print_test_stats(struct kunit *test,
+ size_t num_failures, size_t num_subtests)
+{
+ if (!kunit_should_print_stats(num_failures, num_subtests))
+ return;
+
+ kunit_log(KERN_INFO, test,
+ KUNIT_SUBTEST_INDENT
+ "# %s: %lu / %lu test parameters failed",
+ test->name,
+ num_failures,
+ num_subtests);
+}
+
/*
* Append formatted message to log, size of which is limited to
* KUNIT_LOG_SIZE bytes (including null terminating byte).
@@ -346,15 +381,37 @@ static void kunit_run_case_catch_errors(struct kunit_suite *suite,
test_case->success = test->success;
}
+static void kunit_print_suite_stats(struct kunit_suite *suite,
+ size_t num_failures,
+ size_t total_param_failures,
+ size_t total_params)
+{
+ size_t num_cases = kunit_suite_num_test_cases(suite);
+
+ if (!kunit_should_print_stats(num_failures, num_cases))
+ return;
+
+ kunit_log(KERN_INFO, suite,
+ "# %s: (%lu / %lu) tests failed (%lu / %lu test parameters)",
+ suite->name,
+ num_failures,
+ num_cases,
+ total_param_failures,
+ total_params);
+}
+
int kunit_run_tests(struct kunit_suite *suite)
{
char param_desc[KUNIT_PARAM_DESC_SIZE];
struct kunit_case *test_case;
+ size_t num_suite_failures = 0;
+ size_t total_param_failures = 0, total_params = 0;
kunit_print_subtest_start(suite);
kunit_suite_for_each_test_case(suite, test_case) {
struct kunit test = { .param_value = NULL, .param_index = 0 };
+ size_t num_params = 0, num_failures = 0;
bool test_success = true;
if (test_case->generate_params) {
@@ -385,13 +442,27 @@ int kunit_run_tests(struct kunit_suite *suite)
test.param_value = test_case->generate_params(test.param_value, param_desc);
test.param_index++;
}
+
+ if (!test.success)
+ num_failures++;
+ num_params++;
+
} while (test.param_value);
+ kunit_print_test_stats(&test, num_failures, num_params);
+
kunit_print_ok_not_ok(&test, true, test_success,
kunit_test_case_num(suite, test_case),
test_case->name);
+
+ if (!test_success)
+ num_suite_failures++;
+ total_params += num_params;
+ total_param_failures += num_failures;
}
+ kunit_print_suite_stats(suite, num_suite_failures,
+ total_param_failures, total_params);
kunit_print_subtest_end(suite);
return 0;
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 6614ec4d0898..88ee2b2668ad 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -95,7 +95,7 @@ def print_log(log):
for m in log:
print_with_timestamp(m)
-TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*#).*$')
+TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*# Subtest:).*$')
def consume_non_diagnositic(lines: List[str]) -> None:
while lines and not TAP_ENTRIES.match(lines[0]):
base-commit: 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a
--
2.29.2.576.ga3fc446d84-goog
Don't use an O(nm) algorithm* and make it more readable by using a dict.
*Most obviously, it does a nested for-loop over the entire other config.
A bit more subtle, it calls .entries(), which constructs a set from the
list for _every_ outer iteration.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit_config.py | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py
index 02ffc3a3e5dc..f1101075d458 100644
--- a/tools/testing/kunit/kunit_config.py
+++ b/tools/testing/kunit/kunit_config.py
@@ -40,15 +40,14 @@ class Kconfig(object):
self._entries.append(entry)
def is_subset_of(self, other: 'Kconfig') -> bool:
+ other_dict = {e.name: e.value for e in other.entries()}
for a in self.entries():
- found = False
- for b in other.entries():
- if a.name != b.name:
+ b = other_dict.get(a.name)
+ if b is None:
+ if a.value == 'n':
continue
- if a.value != b.value:
- return False
- found = True
- if a.value != 'n' and found == False:
+ return False
+ elif a.value != b:
return False
return True
base-commit: c6f7e1510b872c281ff603a3108c084b6548d35c
--
2.29.2.576.ga3fc446d84-goog
Currently, given something (fairly dystopian) like
> KUNIT_EXPECT_EQ(test, 2 + 2, 5)
KUnit will prints a failure message like this.
> Expected 2 + 2 == 5, but
> 2 + 2 == 4
> 5 == 5
With this patch, the output just becomes
> Expected 2 + 2 == 5, but
> 2 + 2 == 4
This patch is slightly hacky, but it's quite common* to compare an
expression to a literal integer value, so this can make KUnit less
chatty in many cases. (This patch also fixes variants like
KUNIT_EXPECT_GT, LE, et al.).
It also allocates an additional string briefly, but given this only
happens on test failures, it doesn't seem too bad a tradeoff.
Also, in most cases it'll realize the lengths are unequal and bail out
before the allocation.
We could save the result of the formatted string to avoid wasting this
extra work, but it felt cleaner to leave it as-is.
Edge case: for something silly and unrealistic like
> KUNIT_EXPECT_EQ(test, 4, 5);
It'll generate this message with a trailing "but"
> Expected 4 == 5, but
> <next line of normal output>
It didn't feel worth adding a check up-front to see if both sides are
literals to handle this better.
*A quick grep suggests 100+ comparisons to an integer literal as the
right hand side.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
Tested-by: David Gow <davidgow(a)google.com>
Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com>
---
lib/kunit/assert.c | 39 +++++++++++++++++++++++++++++++++------
1 file changed, 33 insertions(+), 6 deletions(-)
diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c
index 33acdaa28a7d..e0ec7d6fed6f 100644
--- a/lib/kunit/assert.c
+++ b/lib/kunit/assert.c
@@ -85,6 +85,29 @@ void kunit_ptr_not_err_assert_format(const struct kunit_assert *assert,
}
EXPORT_SYMBOL_GPL(kunit_ptr_not_err_assert_format);
+/* Checks if `text` is a literal representing `value`, e.g. "5" and 5 */
+static bool is_literal(struct kunit *test, const char *text, long long value,
+ gfp_t gfp)
+{
+ char *buffer;
+ int len;
+ bool ret;
+
+ len = snprintf(NULL, 0, "%lld", value);
+ if (strlen(text) != len)
+ return false;
+
+ buffer = kunit_kmalloc(test, len+1, gfp);
+ if (!buffer)
+ return false;
+
+ snprintf(buffer, len+1, "%lld", value);
+ ret = strncmp(buffer, text, len) == 0;
+
+ kunit_kfree(test, buffer);
+ return ret;
+}
+
void kunit_binary_assert_format(const struct kunit_assert *assert,
struct string_stream *stream)
{
@@ -97,12 +120,16 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
binary_assert->left_text,
binary_assert->operation,
binary_assert->right_text);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
- binary_assert->left_text,
- binary_assert->left_value);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
- binary_assert->right_text,
- binary_assert->right_value);
+ if (!is_literal(stream->test, binary_assert->left_text,
+ binary_assert->left_value, stream->gfp))
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
+ binary_assert->left_text,
+ binary_assert->left_value);
+ if (!is_literal(stream->test, binary_assert->right_text,
+ binary_assert->right_value, stream->gfp))
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
+ binary_assert->right_text,
+ binary_assert->right_value);
kunit_assert_print_msg(assert, stream);
}
EXPORT_SYMBOL_GPL(kunit_binary_assert_format);
base-commit: 1e0d27fce010b0a4a9e595506b6ede75934c31be
--
2.30.0.478.g8a0d178c01-goog
Commit c2aa8afc36fa has renamed run_vmtests in Makefile,
but the file still uses the old name.
The kernel test robot reported the following issue:
# selftests: vm: run_vmtests.sh
# Warning: file run_vmtests.sh is missing!
not ok 1 selftests: vm: run_vmtests.sh
Reported-by: kernel test robot <lkp(a)intel.com>
Fixes: c2aa8afc36fa (selftests/vm: rename run_vmtests --> run_vmtests.sh)
Signed-off-by: Rong Chen <rong.a.chen(a)intel.com>
---
tools/testing/selftests/vm/{run_vmtests => run_vmtests.sh} | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename tools/testing/selftests/vm/{run_vmtests => run_vmtests.sh} (100%)
diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests.sh
similarity index 100%
rename from tools/testing/selftests/vm/run_vmtests
rename to tools/testing/selftests/vm/run_vmtests.sh
--
2.20.1
When using `kunit.py run` to run tests, users must populate a
`kunitconfig` file to select the options the tests are hidden behind and
all their dependencies.
The patch [1] to allow specifying a path to kunitconfig promises to make
this nicer as we can have checked in files corresponding to different
sets of tests.
But it's still annoying
1) when trying to run a subet of tests
2) when you want to run tests that don't have such a pre-existing
kunitconfig and selecting all the necessary options is tricky.
This patch series aims to alleviate both:
1) `kunit.py run 'my-suite-*'`
I.e. use my current kunitconfig, but just run suites that match this glob
2) `kunit.py run --alltests 'my-suite-*'`
I.e. use allyesconfig so I don't have to worry about writing a
kunitconfig at all.
See the first commit message for more details and discussion about
future work.
This patch series also includes a bugfix for a latent bug that can't be
triggered right now but has worse consequences as a result of the
changes needed to plumb in this suite name glob.
[1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@g…
---
v1 -> v2:
Fix free of `suites` subarray in suite_set.
Found by Dan Carpenter and kernel test robot.
Daniel Latypov (3):
kunit: add kunit.filter_glob cmdline option to filter suites
kunit: tool: add support for filtering suites by glob
kunit: tool: fix unintentional statefulness in run_kernel()
lib/kunit/Kconfig | 1 +
lib/kunit/executor.c | 91 ++++++++++++++++++++++++++---
tools/testing/kunit/kunit.py | 21 +++++--
tools/testing/kunit/kunit_kernel.py | 6 +-
4 files changed, 104 insertions(+), 15 deletions(-)
base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
--
2.30.0.365.g02bc693789-goog
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
@Andrew, this is based on v5.11-rc4-mmots-2021-01-19-13-54 with secretmem
patches dropped from there, I can rebase whatever way you prefer.
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v16:
* Fix memory leak intorduced in v15
* Clean the data left from previous page user before handing the page to
the userspace
v15: https://lore.kernel.org/lkml/20210120180612.1058-1-rppt@kernel.org
* Add riscv/Kconfig update to disable set_memory operations for nommu
builds (patch 3)
* Update the code around add_to_page_cache() per Matthew's comments
(patches 6,7)
* Add fixups for build/checkpatch errors discovered by CI systems
v14: https://lore.kernel.org/lkml/20201203062949.5484-1-rppt@kernel.org
* Finally s/mod_node_page_state/mod_lruvec_page_state/
v13: https://lore.kernel.org/lkml/20201201074559.27742-1-rppt@kernel.org
* Added Reviewed-by, thanks Catalin and David
* s/mod_node_page_state/mod_lruvec_page_state/ as Shakeel suggested
v12: https://lore.kernel.org/lkml/20201125092208.12544-1-rppt@kernel.org
* Add detection of whether set_direct_map has actual effect on arm64 and bail
out of CMA allocation for secretmem and the memfd_secret() syscall if pages
would not be removed from the direct map
Older history:
v11: https://lore.kernel.org/lkml/20201124092556.12009-1-rppt@kernel.org
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (11):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
riscv/Kconfig: make direct map manipulation options depend on MMU
set_memory: allow set_direct_map_*_noflush() for multiple pages
set_memory: allow querying whether set_direct_map_*() is actually enabled
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call where relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/cacheflush.h | 6 -
arch/arm64/include/asm/set_memory.h | 17 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/kernel/machine_kexec.c | 1 +
arch/arm64/mm/mmu.c | 6 +-
arch/arm64/mm/pageattr.c | 23 +-
arch/riscv/Kconfig | 4 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 16 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 451 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 296 ++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 917 insertions(+), 52 deletions(-)
create mode 100644 arch/arm64/include/asm/set_memory.h
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
--
2.28.0
According to the error message, the first argument of ptrace() should be
PTRACE_SINGLESTEP instead of PTRACE_CONT when ptrace single step.
Fixes: f43365ee17f8 ("selftests: arm64: add test for unaligned/inexact watchpoint handling")
Signed-off-by: Tiezhu Yang <yangtiezhu(a)loongson.cn>
---
tools/testing/selftests/breakpoints/breakpoint_test_arm64.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/breakpoints/breakpoint_test_arm64.c b/tools/testing/selftests/breakpoints/breakpoint_test_arm64.c
index ad41ea6..2f4d4d6 100644
--- a/tools/testing/selftests/breakpoints/breakpoint_test_arm64.c
+++ b/tools/testing/selftests/breakpoints/breakpoint_test_arm64.c
@@ -143,7 +143,7 @@ static bool run_test(int wr_size, int wp_size, int wr, int wp)
if (!set_watchpoint(pid, wp_size, wp))
return false;
- if (ptrace(PTRACE_CONT, pid, NULL, NULL) < 0) {
+ if (ptrace(PTRACE_SINGLESTEP, pid, NULL, NULL) < 0) {
ksft_print_msg(
"ptrace(PTRACE_SINGLESTEP) failed: %s\n",
strerror(errno));
--
2.1.0
Changelog
---------
v8
- Added reviewed by's from John Hubbard
- Fixed subjects for selftests patches
- Moved zero page check inside is_pinnable_page() as requested by Jason
Gunthorpe.
v7
- Added reviewed-by's
- Fixed a compile bug on non-mmu builds reported by robot
v6
Small update, but I wanted to send it out quicker, as it removes a
controversial patch and replaces it with something sane.
- Removed forcing FOLL_WRITE for longterm gup, instead added a patch to
skip zero pages during migration.
- Added reviewed-by's and minor log changes.
v5
- Added the following patches to the beginning of series, which are fixes
to the other existing problems with CMA migration code:
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors also at the beginning of series
mm/gup: do not allow zero page for pinned pages
- remove .gfp_mask/.reclaim_idx changes from mm/vmscan.c
- update movable zone header comment in patch 8 instead of patch 3, fix
the comment
- Added acked, sign-offs
- Updated commit logs based on feedback
- Addressed issues reported by Michal and Jason.
- Remove:
#define PINNABLE_MIGRATE_MAX 10
#define PINNABLE_ISOLATE_MAX 100
Instead: fail on the first migration failure, and retry isolation
forever as their failures are transient.
- In self-set addressed some of the comments from John Hubbard, updated
commit logs, and added comments. Renamed gup->flags with gup->test_flags.
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
v4
https://lore.kernel.org/lkml/20201217185243.3288048-1-pasha.tatashin@soleen…
v5
https://lore.kernel.org/lkml/20210119043920.155044-1-pasha.tatashin@soleen.…
v6
https://lore.kernel.org/lkml/20210120014333.222547-1-pasha.tatashin@soleen.…
v7
https://lore.kernel.org/lkml/20210122033748.924330-1-pasha.tatashin@soleen.…
Pavel Tatashin (14):
mm/gup: don't pin migrated cma pages in movable zone
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: do not migrate zero page
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: longterm pin migration cleanup
selftests/vm: gup_test: fix test flag
selftests/vm: gup_test: test faulting in kernel, and verify pinnable
pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 12 ++
include/linux/mmzone.h | 13 +-
include/linux/pgtable.h | 3 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 174 ++++++++----------
mm/gup_test.c | 29 +--
mm/gup_test.h | 3 +-
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++--
tools/testing/selftests/vm/gup_test.c | 36 +++-
14 files changed, 190 insertions(+), 159 deletions(-)
--
2.25.1
Changelog
---------
v9
- Renamed gpf_to_alloc_flags() to gfp_to_alloc_flags_cma(); thanks Lecopzer
Chen for noticing.
- Fixed warning reported scripts/checkpatch.pl:
"Logical continuations should be on the previous line"
v8
- Added reviewed by's from John Hubbard
- Fixed subjects for selftests patches
- Moved zero page check inside is_pinnable_page() as requested by Jason
Gunthorpe.
v7
- Added reviewed-by's
- Fixed a compile bug on non-mmu builds reported by robot
v6
Small update, but I wanted to send it out quicker, as it removes a
controversial patch and replaces it with something sane.
- Removed forcing FOLL_WRITE for longterm gup, instead added a patch to
skip zero pages during migration.
- Added reviewed-by's and minor log changes.
v5
- Added the following patches to the beginning of series, which are fixes
to the other existing problems with CMA migration code:
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors also at the beginning of series
mm/gup: do not allow zero page for pinned pages
- remove .gfp_mask/.reclaim_idx changes from mm/vmscan.c
- update movable zone header comment in patch 8 instead of patch 3, fix
the comment
- Added acked, sign-offs
- Updated commit logs based on feedback
- Addressed issues reported by Michal and Jason.
- Remove:
#define PINNABLE_MIGRATE_MAX 10
#define PINNABLE_ISOLATE_MAX 100
Instead: fail on the first migration failure, and retry isolation
forever as their failures are transient.
- In self-set addressed some of the comments from John Hubbard, updated
commit logs, and added comments. Renamed gup->flags with gup->test_flags.
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
v4
https://lore.kernel.org/lkml/20201217185243.3288048-1-pasha.tatashin@soleen…
v5
https://lore.kernel.org/lkml/20210119043920.155044-1-pasha.tatashin@soleen.…
v6
https://lore.kernel.org/lkml/20210120014333.222547-1-pasha.tatashin@soleen.…
v7
https://lore.kernel.org/lkml/20210122033748.924330-1-pasha.tatashin@soleen.…
v8
https://lore.kernel.org/lkml/20210125194751.1275316-1-pasha.tatashin@soleen…
Pavel Tatashin (14):
mm/gup: don't pin migrated cma pages in movable zone
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: do not migrate zero page
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: longterm pin migration cleanup
selftests/vm: gup_test: fix test flag
selftests/vm: gup_test: test faulting in kernel, and verify pinnable
pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 12 ++
include/linux/mmzone.h | 13 +-
include/linux/pgtable.h | 3 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 174 ++++++++----------
mm/gup_test.c | 29 +--
mm/gup_test.h | 3 +-
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++--
tools/testing/selftests/vm/gup_test.c | 36 +++-
14 files changed, 190 insertions(+), 159 deletions(-)
--
2.25.1
What the short summary is saying now, is that this commit would make the
existing code to use vDSO base address. It's already doing that.
You could instead just "Use getauxval() to simplify the code".
Also, I'd prefer to properly use upper and lower case letter, e.g. vDSO
instead of vdso.
Reply-To:
In-Reply-To: <20210124062907.88229-2-tianjia.zhang(a)linux.alibaba.com>
On Sun, Jan 24, 2021 at 02:29:03PM +0800, Tianjia Zhang wrote:
> This patch uses the library function `getauxval(AT_SYSINFO_EHDR)`
> instead of the custom function `vdso_get_base_addr` to obtain the
Use either double or single quotation mark instead of hyphen.
> base address of vDSO, which will simplify the code implementation.
>
> Signed-off-by: Tianjia Zhang <tianjia.zhang(a)linux.alibaba.com>
This needs to be imperative form, e.g. "Simplify the code implemntation
by using getauxval() instead of a custom function."
> ---
> tools/testing/selftests/sgx/main.c | 24 ++++--------------------
> 1 file changed, 4 insertions(+), 20 deletions(-)
>
> diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c
> index 724cec700926..365d01dea67b 100644
> --- a/tools/testing/selftests/sgx/main.c
> +++ b/tools/testing/selftests/sgx/main.c
> @@ -15,6 +15,7 @@
> #include <sys/stat.h>
> #include <sys/time.h>
> #include <sys/types.h>
> +#include <sys/auxv.h>
> #include "defines.h"
> #include "main.h"
> #include "../kselftest.h"
> @@ -28,24 +29,6 @@ struct vdso_symtab {
> Elf64_Word *elf_hashtab;
> };
>
> -static void *vdso_get_base_addr(char *envp[])
> -{
> - Elf64_auxv_t *auxv;
> - int i;
> -
> - for (i = 0; envp[i]; i++)
> - ;
> -
> - auxv = (Elf64_auxv_t *)&envp[i + 1];
> -
> - for (i = 0; auxv[i].a_type != AT_NULL; i++) {
> - if (auxv[i].a_type == AT_SYSINFO_EHDR)
> - return (void *)auxv[i].a_un.a_val;
> - }
> -
> - return NULL;
> -}
> -
> static Elf64_Dyn *vdso_get_dyntab(void *addr)
> {
> Elf64_Ehdr *ehdr = addr;
> @@ -162,7 +145,7 @@ static int user_handler(long rdi, long rsi, long rdx, long ursp, long r8, long r
> return 0;
> }
>
> -int main(int argc, char *argv[], char *envp[])
> +int main(int argc, char *argv[])
> {
> struct sgx_enclave_run run;
> struct vdso_symtab symtab;
> @@ -203,7 +186,8 @@ int main(int argc, char *argv[], char *envp[])
> memset(&run, 0, sizeof(run));
> run.tcs = encl.encl_base;
>
> - addr = vdso_get_base_addr(envp);
> + /* Get vDSO base address */
> + addr = (void *)(uintptr_t)getauxval(AT_SYSINFO_EHDR);
You could just case the result the result directly to void *.
> if (!addr)
> goto err;
>
> --
> 2.19.1.3.ge56e4f7
>
>
/Jarkko
From: Arnd Bergmann <arnd(a)arndb.de>
I ran into a couple of problems with kunit tests taking too much stack
space, sometimes dangerously so. These the the three instances that
cause an increase over the warning limit of some architectures:
lib/bitfield_kunit.c:93:1: error: the frame size of 7440 bytes is larger than 2048 bytes [-Werror=frame-larger-than=]
drivers/base/test/property-entry-test.c:481:1: error: the frame size of 2640 bytes is larger than 2048 bytes [-Werror=frame-larger-than=]
drivers/thunderbolt/test.c:1529:1: error: the frame size of 1176 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
Ideally there should be a way to rewrite the kunit infrastructure
that avoids the explosion of stack data when the structleak plugin
is used.
A rather drastic measure would be to use Kconfig logic to make
the two options mutually exclusive. This would clearly work, but
is probably not needed.
As a simpler workaround, this disables the plugin for the three
files in which the excessive stack usage was observed.
Arnd
Arnd Bergmann (3):
bitfield: build kunit tests without structleak plugin
drivers/base: build kunit tests without structleak plugin
thunderbolt: build kunit tests without structleak plugin
drivers/base/test/Makefile | 1 +
drivers/thunderbolt/Makefile | 1 +
lib/Makefile | 1 +
3 files changed, 3 insertions(+)
Cc: Kees Cook <keescook(a)chromium.org>
Cc: Brendan Higgins <brendanhiggins(a)google.com>
Cc: Shuah Khan <skhan(a)linuxfoundation.org>
Cc: Geert Uytterhoeven <geert+renesas(a)glider.be>
Cc: Alan Maguire <alan.maguire(a)oracle.com>
Cc: Dmitry Torokhov <dmitry.torokhov(a)gmail.com>
Cc: Mika Westerberg <mika.westerberg(a)linux.intel.com>
Cc: Vitor Massaru Iha <vitor(a)massaru.org>
Cc: linux-hardening(a)vger.kernel.org
Cc: linux-kselftest(a)vger.kernel.org
Cc: kunit-dev(a)googlegroups.com
Cc: linux-kernel(a)vger.kernel.org
--
2.29.2
Currently, given something (fairly dystopian) like
> KUNIT_EXPECT_EQ(test, 2 + 2, 5)
KUnit will prints a failure message like this.
> Expected 2 + 2 == 5, but
> 2 + 2 == 4
> 5 == 5
With this patch, the output just becomes
> Expected 2 + 2 == 5, but
> 2 + 2 == 4
This patch is slightly hacky, but it's quite common* to compare an
expression to a literal integer value, so this can make KUnit less
chatty in many cases. (This patch also fixes variants like
KUNIT_EXPECT_GT, LE, et al.).
It also allocates an additional string briefly, but given this only
happens on test failures, it doesn't seem too bad a tradeoff.
Also, in most cases it'll realize the lengths are unequal and bail out
before the allocation.
We could save the result of the formatted string to avoid wasting this
extra work, but it felt cleaner to leave it as-is.
Edge case: for something silly and unrealistic like
> KUNIT_EXPECT_EQ(test, 4, 5);
It'll generate this message with a trailing "but"
> Expected 2 + 2 == 5, but
> <next line of normal output>
It didn't feel worth adding a check up-front to see if both sides are
literals to handle this better.
*A quick grep suggests 100+ comparisons to an integer literal as the
right hand side.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
lib/kunit/assert.c | 39 +++++++++++++++++++++++++++++++++------
1 file changed, 33 insertions(+), 6 deletions(-)
diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c
index 33acdaa28a7d..e0ec7d6fed6f 100644
--- a/lib/kunit/assert.c
+++ b/lib/kunit/assert.c
@@ -85,6 +85,29 @@ void kunit_ptr_not_err_assert_format(const struct kunit_assert *assert,
}
EXPORT_SYMBOL_GPL(kunit_ptr_not_err_assert_format);
+/* Checks if `text` is a literal representing `value`, e.g. "5" and 5 */
+static bool is_literal(struct kunit *test, const char *text, long long value,
+ gfp_t gfp)
+{
+ char *buffer;
+ int len;
+ bool ret;
+
+ len = snprintf(NULL, 0, "%lld", value);
+ if (strlen(text) != len)
+ return false;
+
+ buffer = kunit_kmalloc(test, len+1, gfp);
+ if (!buffer)
+ return false;
+
+ snprintf(buffer, len+1, "%lld", value);
+ ret = strncmp(buffer, text, len) == 0;
+
+ kunit_kfree(test, buffer);
+ return ret;
+}
+
void kunit_binary_assert_format(const struct kunit_assert *assert,
struct string_stream *stream)
{
@@ -97,12 +120,16 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
binary_assert->left_text,
binary_assert->operation,
binary_assert->right_text);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
- binary_assert->left_text,
- binary_assert->left_value);
- string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
- binary_assert->right_text,
- binary_assert->right_value);
+ if (!is_literal(stream->test, binary_assert->left_text,
+ binary_assert->left_value, stream->gfp))
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n",
+ binary_assert->left_text,
+ binary_assert->left_value);
+ if (!is_literal(stream->test, binary_assert->right_text,
+ binary_assert->right_value, stream->gfp))
+ string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld",
+ binary_assert->right_text,
+ binary_assert->right_value);
kunit_assert_print_msg(assert, stream);
}
EXPORT_SYMBOL_GPL(kunit_binary_assert_format);
base-commit: e5ff2cb9cf67a542f2ec7fb87e24934c88b32678
--
2.30.0.365.g02bc693789-goog
Currently running tests via KUnit tool means tweaking a .kunitconfig
file, which you'd keep around locally and never commit.
This changes makes it so users can pass in a path to a kunitconfig.
One of the imagined use cases is having kunitconfig fragments in-tree
to formalize interesting sets of tests for features/subsystems, e.g.
$ ./tools/testing/kunit/kunit.py run fs/ext4/kunitconfig
For now, this hypothetical fs/ext4/kunitconfig would contain
CONFIG_KUNIT=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_KUNIT_TESTS=y
At the moment, it's not hard to manually whip up this file, but as more
and more tests get added, this will get tedious.
It also opens the door to documenting how to run all the tests relevant
to a specific subsystem or feature as a simple one-liner.
This can be seen as an analogue to tools/testing/selftests/*/config
But in the case of KUnit, the tests live in the same directory as the
code-under-test, so it feels more natural to allow the kunitconfig
fragments to live anywhere. (Though, people could create a separate
directory if wanted; this patch imposes no restrictions on the path).
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit.py | 9 ++++++---
tools/testing/kunit/kunit_kernel.py | 12 ++++++++----
tools/testing/kunit/kunit_tool_test.py | 25 +++++++++++++++++++++++++
3 files changed, 39 insertions(+), 7 deletions(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index e808a47c839b..3204a23bd16e 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -188,6 +188,9 @@ def add_build_opts(parser) -> None:
help='As in the make command, "Specifies the number of '
'jobs (commands) to run simultaneously."',
type=int, default=8, metavar='jobs')
+ parser.add_argument('kunitconfig',
+ help='Path to Kconfig fragment that enables KUnit tests',
+ type=str, nargs='?', metavar='kunitconfig')
def add_exec_opts(parser) -> None:
parser.add_argument('--timeout',
@@ -256,7 +259,7 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
@@ -274,7 +277,7 @@ def main(argv, linux=None):
os.mkdir(cli_args.build_dir)
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
@@ -286,7 +289,7 @@ def main(argv, linux=None):
sys.exit(1)
elif cli_args.subcommand == 'build':
if not linux:
- linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir)
+ linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig)
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 2076a5a2d060..0b461663e7d9 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -123,7 +123,7 @@ def get_outfile_path(build_dir) -> str:
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
- def __init__(self, build_dir: str, load_config=True, defconfig=DEFAULT_KUNITCONFIG_PATH) -> None:
+ def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None:
signal.signal(signal.SIGINT, self.signal_handler)
self._ops = LinuxSourceTreeOperations()
@@ -131,9 +131,13 @@ class LinuxSourceTree(object):
if not load_config:
return
- kunitconfig_path = get_kunitconfig_path(build_dir)
- if not os.path.exists(kunitconfig_path):
- shutil.copyfile(defconfig, kunitconfig_path)
+ if kunitconfig_path:
+ if not os.path.exists(kunitconfig_path):
+ raise ConfigError(f'Specified kunitconfig ({kunitconfig_path}) does not exist')
+ else:
+ kunitconfig_path = get_kunitconfig_path(build_dir)
+ if not os.path.exists(kunitconfig_path):
+ shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, kunitconfig_path)
self._kconfig = kunit_config.Kconfig()
self._kconfig.read_from_file(kunitconfig_path)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index b593f4448e83..533fe41b5123 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -12,6 +12,7 @@ from unittest import mock
import tempfile, shutil # Handling test_tmpdir
import json
+import signal
import os
import kunit_config
@@ -250,6 +251,23 @@ class KUnitParserTest(unittest.TestCase):
result.status)
self.assertEqual('kunit-resource-test', result.suites[0].name)
+class LinuxSourceTreeTest(unittest.TestCase):
+
+ def setUp(self):
+ mock.patch.object(signal, 'signal').start()
+ self.addCleanup(mock.patch.stopall)
+
+ def test_invalid_kunitconfig(self):
+ with self.assertRaisesRegex(kunit_kernel.ConfigError, 'nonexistent.* does not exist'):
+ kunit_kernel.LinuxSourceTree('', kunitconfig_path='/nonexistent_file')
+
+ def test_valid_kunitconfig(self):
+ with tempfile.NamedTemporaryFile('wt') as kunitconfig:
+ tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=kunitconfig.name)
+
+ # TODO: add more test cases.
+
+
class KUnitJsonTest(unittest.TestCase):
def _json_for(self, log_file):
@@ -399,5 +417,12 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.'))
+ @mock.patch.object(kunit_kernel, 'LinuxSourceTree')
+ def test_run_kunitconfig(self, mock_linux_init):
+ mock_linux_init.return_value = self.linux_source_mock
+ kunit.main(['run', 'mykunitconfig'])
+ # Just verify that we parsed and initialized it correctly here.
+ mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig')
+
if __name__ == '__main__':
unittest.main()
base-commit: 2b8fdbbf1c616300312f71fe5b21fe8f03129950
--
2.30.0.280.ga3ce27912f-goog
Hi,
This patch series adjusts the semantic of file hierarchy access-control
per layer to get a more pragmatic and compatible approach. I updated
the documentation to explain how layers, bind mounts and overlayfs are
handled by Landlock. A syscall is also renamed to make it less
ambiguous for future evolution. Last but not least, the test file
layout cleanups are more resilient, and a lot of tests are added to
cover bind mounts and overlayfs, which are fully supported.
The SLOC count is 1292 for security/landlock/ and 2425 for
tools/testing/selftest/landlock/ . Test coverage for security/landlock/
is 94.7% of lines. The code not covered only deals with internal kernel
errors (e.g. memory allocation) and race conditions. This series is
being fuzzed by syzkaller, and patches are on their way:
https://github.com/google/syzkaller/pull/2380
The compiled documentation is available here:
https://landlock.io/linux-doc/landlock-v27/userspace-api/landlock.html
This series can be applied on top of v5.11-rc4 . This can be tested
with CONFIG_SECURITY_LANDLOCK, CONFIG_SAMPLE_LANDLOCK and by prepending
"landlock," to CONFIG_LSM. This patch series can be found in a Git
repository here:
https://github.com/landlock-lsm/linux/commits/landlock-v27
This patch series seems ready for upstream and I would really appreciate
final reviews.
# Landlock LSM
The goal of Landlock is to enable to restrict ambient rights (e.g.
global filesystem access) for a set of processes. Because Landlock is a
stackable LSM [1], it makes possible to create safe security sandboxes
as new security layers in addition to the existing system-wide
access-controls. This kind of sandbox is expected to help mitigate the
security impact of bugs or unexpected/malicious behaviors in user-space
applications. Landlock empowers any process, including unprivileged
ones, to securely restrict themselves.
Landlock is inspired by seccomp-bpf but instead of filtering syscalls
and their raw arguments, a Landlock rule can restrict the use of kernel
objects like file hierarchies, according to the kernel semantic.
Landlock also takes inspiration from other OS sandbox mechanisms: XNU
Sandbox, FreeBSD Capsicum or OpenBSD Pledge/Unveil.
In this current form, Landlock misses some access-control features.
This enables to minimize this patch series and ease review. This series
still addresses multiple use cases, especially with the combined use of
seccomp-bpf: applications with built-in sandboxing, init systems,
security sandbox tools and security-oriented APIs [2].
Previous version:
https://lore.kernel.org/lkml/20201209192839.1396820-1-mic@digikod.net/
[1] https://lore.kernel.org/lkml/50db058a-7dde-441b-a7f9-f6837fe8b69f@schaufler…
[2] https://lore.kernel.org/lkml/f646e1c7-33cf-333f-070c-0a40ad0468cd@digikod.n…
Casey Schaufler (1):
LSM: Infrastructure management of the superblock
Mickaël Salaün (11):
landlock: Add object management
landlock: Add ruleset and domain management
landlock: Set up the security framework and manage credentials
landlock: Add ptrace restrictions
fs,security: Add sb_delete hook
landlock: Support filesystem access-control
landlock: Add syscall implementations
arch: Wire up Landlock syscalls
selftests/landlock: Add user space tests
samples/landlock: Add a sandbox manager example
landlock: Add user and kernel documentation
Documentation/security/index.rst | 1 +
Documentation/security/landlock.rst | 79 +
Documentation/userspace-api/index.rst | 1 +
Documentation/userspace-api/landlock.rst | 306 ++
MAINTAINERS | 13 +
arch/Kconfig | 7 +
arch/alpha/kernel/syscalls/syscall.tbl | 3 +
arch/arm/tools/syscall.tbl | 3 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 6 +
arch/ia64/kernel/syscalls/syscall.tbl | 3 +
arch/m68k/kernel/syscalls/syscall.tbl | 3 +
arch/microblaze/kernel/syscalls/syscall.tbl | 3 +
arch/mips/kernel/syscalls/syscall_n32.tbl | 3 +
arch/mips/kernel/syscalls/syscall_n64.tbl | 3 +
arch/mips/kernel/syscalls/syscall_o32.tbl | 3 +
arch/parisc/kernel/syscalls/syscall.tbl | 3 +
arch/powerpc/kernel/syscalls/syscall.tbl | 3 +
arch/s390/kernel/syscalls/syscall.tbl | 3 +
arch/sh/kernel/syscalls/syscall.tbl | 3 +
arch/sparc/kernel/syscalls/syscall.tbl | 3 +
arch/um/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 3 +
arch/x86/entry/syscalls/syscall_64.tbl | 3 +
arch/xtensa/kernel/syscalls/syscall.tbl | 3 +
fs/super.c | 1 +
include/linux/lsm_hook_defs.h | 1 +
include/linux/lsm_hooks.h | 3 +
include/linux/security.h | 4 +
include/linux/syscalls.h | 7 +
include/uapi/asm-generic/unistd.h | 8 +-
include/uapi/linux/landlock.h | 128 +
kernel/sys_ni.c | 5 +
samples/Kconfig | 7 +
samples/Makefile | 1 +
samples/landlock/.gitignore | 1 +
samples/landlock/Makefile | 13 +
samples/landlock/sandboxer.c | 239 ++
security/Kconfig | 11 +-
security/Makefile | 2 +
security/landlock/Kconfig | 21 +
security/landlock/Makefile | 4 +
security/landlock/common.h | 20 +
security/landlock/cred.c | 46 +
security/landlock/cred.h | 58 +
security/landlock/fs.c | 621 ++++
security/landlock/fs.h | 56 +
security/landlock/limits.h | 21 +
security/landlock/object.c | 67 +
security/landlock/object.h | 91 +
security/landlock/ptrace.c | 120 +
security/landlock/ptrace.h | 14 +
security/landlock/ruleset.c | 466 +++
security/landlock/ruleset.h | 161 +
security/landlock/setup.c | 40 +
security/landlock/setup.h | 18 +
security/landlock/syscalls.c | 429 +++
security/security.c | 51 +-
security/selinux/hooks.c | 58 +-
security/selinux/include/objsec.h | 6 +
security/selinux/ss/services.c | 3 +-
security/smack/smack.h | 6 +
security/smack/smack_lsm.c | 35 +-
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/landlock/.gitignore | 2 +
tools/testing/selftests/landlock/Makefile | 24 +
tools/testing/selftests/landlock/base_test.c | 219 ++
tools/testing/selftests/landlock/common.h | 166 ++
tools/testing/selftests/landlock/config | 6 +
tools/testing/selftests/landlock/fs_test.c | 2585 +++++++++++++++++
.../testing/selftests/landlock/ptrace_test.c | 314 ++
tools/testing/selftests/landlock/true.c | 5 +
72 files changed, 6552 insertions(+), 77 deletions(-)
create mode 100644 Documentation/security/landlock.rst
create mode 100644 Documentation/userspace-api/landlock.rst
create mode 100644 include/uapi/linux/landlock.h
create mode 100644 samples/landlock/.gitignore
create mode 100644 samples/landlock/Makefile
create mode 100644 samples/landlock/sandboxer.c
create mode 100644 security/landlock/Kconfig
create mode 100644 security/landlock/Makefile
create mode 100644 security/landlock/common.h
create mode 100644 security/landlock/cred.c
create mode 100644 security/landlock/cred.h
create mode 100644 security/landlock/fs.c
create mode 100644 security/landlock/fs.h
create mode 100644 security/landlock/limits.h
create mode 100644 security/landlock/object.c
create mode 100644 security/landlock/object.h
create mode 100644 security/landlock/ptrace.c
create mode 100644 security/landlock/ptrace.h
create mode 100644 security/landlock/ruleset.c
create mode 100644 security/landlock/ruleset.h
create mode 100644 security/landlock/setup.c
create mode 100644 security/landlock/setup.h
create mode 100644 security/landlock/syscalls.c
create mode 100644 tools/testing/selftests/landlock/.gitignore
create mode 100644 tools/testing/selftests/landlock/Makefile
create mode 100644 tools/testing/selftests/landlock/base_test.c
create mode 100644 tools/testing/selftests/landlock/common.h
create mode 100644 tools/testing/selftests/landlock/config
create mode 100644 tools/testing/selftests/landlock/fs_test.c
create mode 100644 tools/testing/selftests/landlock/ptrace_test.c
create mode 100644 tools/testing/selftests/landlock/true.c
base-commit: 19c329f6808995b142b3966301f217c831e7cf31
--
2.30.0
From: Bongsu Jeon <bongsu.jeon(a)samsung.com>
A NCI virtual device can be made to simulate a NCI device in user space.
Using the virtual NCI device, The NCI module and application can be
validated. This driver supports to communicate between the virtual NCI
device and NCI module. To test the basic features of NCI module, selftest
for nci is added. Test cases consist of making the virtual NCI device
on/off and controlling the device's polling for NCI1.0 and NCI2.0 version.
1/2 is the Virtual NCI device driver.
2/2 is the NCI selftest suite
v3:
1/2
- change the Kconfig help comment.
- remove the mutex init code.
- remove the unnecessary mutex(nci_send_mutex).
- remove the full_txbuff.
- add the code to release skb at error case.
- refactor some code.
v2:
1/2
- change the permission of the Virtual NCI device.
- add the ioctl to find the nci device index.
2/2
- add the NCI selftest suite.
MAINTAINERS | 8 +
drivers/nfc/Kconfig | 11 +
drivers/nfc/Makefile | 1 +
drivers/nfc/virtual_ncidev.c | 227 ++++++++++
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/nci/Makefile | 6 +
tools/testing/selftests/nci/config | 3 +
tools/testing/selftests/nci/nci_dev.c | 599 ++++++++++++++++++++++++++
8 files changed, 856 insertions(+)
create mode 100644 drivers/nfc/virtual_ncidev.c
create mode 100644 tools/testing/selftests/nci/Makefile
create mode 100644 tools/testing/selftests/nci/config
create mode 100644 tools/testing/selftests/nci/nci_dev.c
--
2.25.1
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
@Andrew, this is based on v5.11-rc4-mmots-2021-01-19-13-54 with secretmem
patches dropped from there, I can rebase whatever way you prefer.
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v15:
* Add riscv/Kconfig update to disable set_memory operations for nommu
builds (patch 3)
* Update the code around add_to_page_cache() per Matthew's comments
(patches 6,7)
* Add fixups for build/checkpatch errors discovered by CI systems
v14: https://lore.kernel.org/lkml/20201203062949.5484-1-rppt@kernel.org
* Finally s/mod_node_page_state/mod_lruvec_page_state/
v13: https://lore.kernel.org/lkml/20201201074559.27742-1-rppt@kernel.org
* Added Reviewed-by, thanks Catalin and David
* s/mod_node_page_state/mod_lruvec_page_state/ as Shakeel suggested
v12: https://lore.kernel.org/lkml/20201125092208.12544-1-rppt@kernel.org
* Add detection of whether set_direct_map has actual effect on arm64 and bail
out of CMA allocation for secretmem and the memfd_secret() syscall if pages
would not be removed from the direct map
v11: https://lore.kernel.org/lkml/20201124092556.12009-1-rppt@kernel.org
* Drop support for uncached mappings
Older history:
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (11):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
riscv/Kconfig: make direct map manipulation options depend on MMU
set_memory: allow set_direct_map_*_noflush() for multiple pages
set_memory: allow querying whether set_direct_map_*() is actually enabled
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call where relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/cacheflush.h | 6 -
arch/arm64/include/asm/set_memory.h | 17 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/kernel/machine_kexec.c | 1 +
arch/arm64/mm/mmu.c | 6 +-
arch/arm64/mm/pageattr.c | 23 +-
arch/riscv/Kconfig | 4 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 16 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 444 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 296 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 910 insertions(+), 52 deletions(-)
create mode 100644 arch/arm64/include/asm/set_memory.h
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
--
2.28.0
Changelog
---------
v6
Small update, but I wanted to send it out quicker, as it removes a
controversial patch and replaces it with something sane.
- Removed forcing FOLL_WRITE for longterm gup, instead added a patch to
skip zero pages during migration.
- Added reviewed-by's and minor log changes.
v5
- Added the following patches to the beginning of series, which are fixes
to the other existing problems with CMA migration code:
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors also at the beginning of series
mm/gup: do not allow zero page for pinned pages
- remove .gfp_mask/.reclaim_idx changes from mm/vmscan.c
- update movable zone header comment in patch 8 instead of patch 3, fix
the comment
- Added acked, sign-offs
- Updated commit logs based on feedback
- Addressed issues reported by Michal and Jason.
- Remove:
#define PINNABLE_MIGRATE_MAX 10
#define PINNABLE_ISOLATE_MAX 100
Instead: fail on the first migration failure, and retry isolation
forever as their failures are transient.
- In self-set addressed some of the comments from John Hubbard, updated
commit logs, and added comments. Renamed gup->flags with gup->test_flags.
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
v4
https://lore.kernel.org/lkml/20201217185243.3288048-1-pasha.tatashin@soleen…
v5
https://lore.kernel.org/lkml/20210119043920.155044-1-pasha.tatashin@soleen.…
Pavel Tatashin (14):
mm/gup: don't pin migrated cma pages in movable zone
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: do not migrate zero page
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: longterm pin migration cleaup
selftests/vm: test flag is broken
selftests/vm: test faulting in kernel, and verify pinnable pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 11 ++
include/linux/mmzone.h | 13 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 175 ++++++++----------
mm/gup_test.c | 29 +--
mm/gup_test.h | 3 +-
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++--
tools/testing/selftests/vm/gup_test.c | 36 +++-
13 files changed, 185 insertions(+), 161 deletions(-)
--
2.25.1
Changelog
---------
v7
- Added reviewed-by's
- Fixed a compile bug on non-mmu builds reported by robot
v6
Small update, but I wanted to send it out quicker, as it removes a
controversial patch and replaces it with something sane.
- Removed forcing FOLL_WRITE for longterm gup, instead added a patch to
skip zero pages during migration.
- Added reviewed-by's and minor log changes.
v5
- Added the following patches to the beginning of series, which are fixes
to the other existing problems with CMA migration code:
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors also at the beginning of series
mm/gup: do not allow zero page for pinned pages
- remove .gfp_mask/.reclaim_idx changes from mm/vmscan.c
- update movable zone header comment in patch 8 instead of patch 3, fix
the comment
- Added acked, sign-offs
- Updated commit logs based on feedback
- Addressed issues reported by Michal and Jason.
- Remove:
#define PINNABLE_MIGRATE_MAX 10
#define PINNABLE_ISOLATE_MAX 100
Instead: fail on the first migration failure, and retry isolation
forever as their failures are transient.
- In self-set addressed some of the comments from John Hubbard, updated
commit logs, and added comments. Renamed gup->flags with gup->test_flags.
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
v4
https://lore.kernel.org/lkml/20201217185243.3288048-1-pasha.tatashin@soleen…
v5
https://lore.kernel.org/lkml/20210119043920.155044-1-pasha.tatashin@soleen.…
v6
https://lore.kernel.org/lkml/20210120014333.222547-1-pasha.tatashin@soleen.…
Pavel Tatashin (14):
mm/gup: don't pin migrated cma pages in movable zone
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: do not migrate zero page
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: longterm pin migration cleanup
selftests/vm: test flag is broken
selftests/vm: test faulting in kernel, and verify pinnable pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 11 ++
include/linux/mmzone.h | 13 +-
include/linux/pgtable.h | 3 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 176 ++++++++----------
mm/gup_test.c | 29 +--
mm/gup_test.h | 3 +-
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++--
tools/testing/selftests/vm/gup_test.c | 36 +++-
14 files changed, 191 insertions(+), 159 deletions(-)
--
2.25.1
In function sgx_encl_create(), the logic of directly assigning
value to attributes_mask determines that the call to
SGX_IOC_ENCLAVE_PROVISION must be after the command of
SGX_IOC_ENCLAVE_CREATE. If change this assignment statement to
or operation, the PROVISION command can be executed earlier and
more flexibly.
Reported-by: Jia Zhang <zhang.jia(a)linux.alibaba.com>
Signed-off-by: Tianjia Zhang <tianjia.zhang(a)linux.alibaba.com>
---
arch/x86/kernel/cpu/sgx/ioctl.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c
index f45957c05f69..0ca3fc238bc2 100644
--- a/arch/x86/kernel/cpu/sgx/ioctl.c
+++ b/arch/x86/kernel/cpu/sgx/ioctl.c
@@ -108,7 +108,7 @@ static int sgx_encl_create(struct sgx_encl *encl, struct sgx_secs *secs)
encl->base = secs->base;
encl->size = secs->size;
encl->attributes = secs->attributes;
- encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS;
+ encl->attributes_mask |= SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS;
/* Set only after completion, as encl->lock has not been taken. */
set_bit(SGX_ENCL_CREATED, &encl->flags);
--
2.19.1.3.ge56e4f7
Hi Linus,
Please pull the following KUnit fixes update for Linux 5.11-rc5.
This KUnit update for Linux 5.11-rc5 consists of 5 fixes to kunit tool
and documentation from Daniel Latypov and David Gow.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 65a4e5299739abe0888cda0938d21f8ea3b5c606:
kunit: tool: Force the use of the 'tty' console for UML (2021-01-04
09:18:38 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-fixes-5.11-rc5
for you to fetch changes up to 2b8fdbbf1c616300312f71fe5b21fe8f03129950:
kunit: tool: move kunitconfig parsing into __init__, make it optional
(2021-01-15 17:52:12 -0700)
----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.11-rc5
This KUnit update for Linux 5.11-rc5 consist of 5 fixes to kunit tool
and documentation from Daniel Latypov and David Gow.
----------------------------------------------------------------
Daniel Latypov (4):
Documentation: kunit: include example of a parameterized test
kunit: tool: surface and address more typing issues
kunit: tool: fix minor typing issue with None status
kunit: tool: move kunitconfig parsing into __init__, make it optional
David Gow (1):
kunit: tool: Fix spelling of "diagnostic" in kunit_parser
Documentation/dev-tools/kunit/usage.rst | 57 +++++++++++++++++++++++
tools/testing/kunit/kunit.py | 34 +++++---------
tools/testing/kunit/kunit_config.py | 7 +--
tools/testing/kunit/kunit_json.py | 2 +-
tools/testing/kunit/kunit_kernel.py | 54 +++++++++++-----------
tools/testing/kunit/kunit_parser.py | 81
++++++++++++++++-----------------
6 files changed, 141 insertions(+), 94 deletions(-)
----------------------------------------------------------------
From: Bongsu Jeon <bongsu.jeon(a)samsung.com>
A NCI virtual device can be made to simulate a NCI device in user space.
Using the virtual NCI device, The NCI module and application can be
validated. This driver supports to communicate between the virtual NCI
device and NCI module. To test the basic features of NCI module, selftest
for NCI is added. Test cases consist of making the virtual NCI device
on/off and controlling the device's polling for NCI1.0 and NCI2.0 version.
1/2 is the Virtual NCI device driver.
2/2 is the NCI selftest suite
v2:
1/2
- change the permission of the Virtual NCI device.
- add the ioctl to find the nci device index.
2/2
- add the NCI selftest suite.
Bongsu Jeon (2):
nfc: Add a virtual nci device driver
selftests: Add nci suite
MAINTAINERS | 8 +
drivers/nfc/Kconfig | 11 +
drivers/nfc/Makefile | 1 +
drivers/nfc/virtual_ncidev.c | 235 ++++++++++
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/nci/Makefile | 6 +
tools/testing/selftests/nci/config | 3 +
tools/testing/selftests/nci/nci_dev.c | 599 ++++++++++++++++++++++++++
8 files changed, 864 insertions(+)
create mode 100644 drivers/nfc/virtual_ncidev.c
create mode 100644 tools/testing/selftests/nci/Makefile
create mode 100644 tools/testing/selftests/nci/config
create mode 100644 tools/testing/selftests/nci/nci_dev.c
--
2.25.1
Add a libbpf dumper function that supports dumping a representation
of data passed in using the BTF id associated with the data in a
manner similar to the bpf_snprintf_btf helper.
Default output format is identical to that dumped by bpf_snprintf_btf(),
for example a "struct sk_buff" representation would look like this:
struct sk_buff){
(union){
(struct){
.next = (struct sk_buff *)0xffffffffffffffff,
.prev = (struct sk_buff *)0xffffffffffffffff,
(union){
.dev = (struct net_device *)0xffffffffffffffff,
.dev_scratch = (long unsigned int)18446744073709551615,
},
},
...
Patches 1 and 2 make functions available that are needed during
dump operations.
Patch 3 implements the dump functionality in a manner similar
to that in kernel/bpf/btf.c, but with a view to fitting into
libbpf more naturally. For example, rather than using flags,
boolean dump options are used to control output.
Patch 4 is a selftest that utilizes a dump printf function
to snprintf the dump output to a string for comparison with
expected output. Tests deliberately mirror those in
snprintf_btf helper test to keep output consistent.
Changes since RFC [1]
- The initial approach explored was to share the kernel code
with libbpf using #defines to paper over the different needs;
however it makes more sense to try and fit in with libbpf
code style for maintenance. A comment in the code points at
the implementation in kernel/bpf/btf.c and notes that any
issues found in it should be fixed there or vice versa;
mirroring the tests should help with this also
(Andrii)
[1] https://lore.kernel.org/bpf/1610386373-24162-1-git-send-email-alan.maguire@…
Alan Maguire (4):
libbpf: add btf_has_size() and btf_int() inlines
libbpf: make skip_mods_and_typedefs available internally in libbpf
libbpf: BTF dumper support for typed data
selftests/bpf: add dump type data tests to btf dump tests
tools/lib/bpf/btf.h | 36 +
tools/lib/bpf/btf_dump.c | 974 ++++++++++++++++++++++
tools/lib/bpf/libbpf.c | 4 +-
tools/lib/bpf/libbpf.map | 5 +
tools/lib/bpf/libbpf_internal.h | 2 +
tools/testing/selftests/bpf/prog_tests/btf_dump.c | 233 ++++++
6 files changed, 1251 insertions(+), 3 deletions(-)
--
1.8.3.1
Obviously, the error variable detection of the if statement is
for the mprotect callback function, so it is also put into the
scope of calling callbck.
Reported-by: Jia Zhang <zhang.jia(a)linux.alibaba.com>
Signed-off-by: Tianjia Zhang <tianjia.zhang(a)linux.alibaba.com>
---
mm/mprotect.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/mm/mprotect.c b/mm/mprotect.c
index ab709023e9aa..94188df1ee55 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -617,10 +617,11 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
if (tmp > end)
tmp = end;
- if (vma->vm_ops && vma->vm_ops->mprotect)
+ if (vma->vm_ops && vma->vm_ops->mprotect) {
error = vma->vm_ops->mprotect(vma, nstart, tmp, newflags);
- if (error)
- goto out;
+ if (error)
+ goto out;
+ }
error = mprotect_fixup(vma, &prev, nstart, tmp, newflags);
if (error)
--
2.19.1.3.ge56e4f7
Hi Jon,
This series have three parts:
1) 10 remaining fixup patches from the series I sent back on Dec, 1st:
parport: fix a kernel-doc markup
rapidio: fix kernel-doc a markup
fs: fix kernel-doc markups
pstore/zone: fix a kernel-doc markup
firmware: stratix10-svc: fix kernel-doc markups
connector: fix a kernel-doc markup
lib/crc7: fix a kernel-doc markup
memblock: fix kernel-doc markups
w1: fix a kernel-doc markup
selftests: kselftest_harness.h: partially fix kernel-doc markups
2) The patch adding the new check to ensure that the kernel-doc
markup will be used for the right declaration;
3) 5 additional patches, produced against next-20210114 with new
problems detected after the original series:
net: tip: fix a couple kernel-doc markups
net: cfg80211: fix a kerneldoc markup
reset: core: fix a kernel-doc markup
drm: drm_crc: fix a kernel-doc markup
platform/surface: aggregator: fix a kernel-doc markup
It probably makes sense to merge at least the first 11 patches
via the doc tree, as they should apply cleanly there, and
having the last 5 patches merged via each maintainer's tree.
-
Kernel-doc has always be limited to a probably bad documented
rule:
The kernel-doc markups should appear *imediatelly before* the
function or data structure that it documents.
On other words, if a C file would contain something like this:
/**
* foo - function foo
* @args: foo args
*/
static inline void bar(int args);
/**
* bar - function bar
* @args: foo args
*/
static inline void foo(void *args);
The output (in ReST format) will be:
.. c:function:: void bar (int args)
function foo
**Parameters**
``int args``
foo args
.. c:function:: void foo (void *args)
function bar
**Parameters**
``void *args``
foo args
Which is clearly a wrong result. Before this changeset,
not even a warning is produced on such cases.
As placing such markups just before the documented
data is a common practice, on most cases this is fine.
However, as patches touch things, identifiers may be
renamed, and people may forget to update the kernel-doc
markups to follow such changes.
This has been happening for quite a while, as there are
lots of files with kernel-doc problems.
This series address those issues and add a file at the
end that will enforce that the identifier will match the
kernel-doc markup, avoiding this problem from
keep happening as time goes by.
This series is based on current upstream tree.
@maintainers: feel free to pick the patches and
apply them directly on your trees, as all patches on
this series are independent from the other ones.
--
v6:
- rebased on the top of next-20210114 and added a few extra fixes
v5:
- The completion.h patch was replaced by another one which drops
an obsolete macro;
- Some typos got fixed and review tags got added;
- Dropped patches that were already merged at linux-next.
v4:
- Patches got rebased and got some acks.
Mauro Carvalho Chehab (16):
parport: fix a kernel-doc markup
rapidio: fix kernel-doc a markup
fs: fix kernel-doc markups
pstore/zone: fix a kernel-doc markup
firmware: stratix10-svc: fix kernel-doc markups
connector: fix a kernel-doc markup
lib/crc7: fix a kernel-doc markup
memblock: fix kernel-doc markups
w1: fix a kernel-doc markup
selftests: kselftest_harness.h: partially fix kernel-doc markups
scripts: kernel-doc: validate kernel-doc markup with the actual names
net: tip: fix a couple kernel-doc markups
net: cfg80211: fix a kerneldoc markup
reset: core: fix a kernel-doc markup
drm: drm_crc: fix a kernel-doc markup
platform/surface: aggregator: fix a kernel-doc markup
drivers/parport/share.c | 2 +-
.../surface/aggregator/ssh_request_layer.c | 2 +-
drivers/rapidio/rio.c | 2 +-
drivers/reset/core.c | 4 +-
fs/dcache.c | 73 ++++++++++---------
fs/inode.c | 4 +-
fs/pstore/zone.c | 2 +-
fs/seq_file.c | 5 +-
fs/super.c | 12 +--
include/drm/drm_crtc.h | 2 +-
include/linux/connector.h | 2 +-
.../firmware/intel/stratix10-svc-client.h | 10 +--
include/linux/memblock.h | 4 +-
include/linux/parport.h | 31 ++++++++
include/linux/w1.h | 2 +-
include/net/cfg80211.h | 2 +-
lib/crc7.c | 2 +-
net/tipc/link.c | 2 +-
net/tipc/node.c | 2 +-
scripts/kernel-doc | 62 ++++++++++++----
tools/testing/selftests/kselftest_harness.h | 26 ++++---
21 files changed, 160 insertions(+), 93 deletions(-)
--
2.29.2
Initially I just wanted to port the selftests to the latest GPIO uAPI,
but on finding that, due to dependency issues, the selftests are not built
for the buildroot environments that I do most of my GPIO testing in, I
decided to take a closer look.
The first patch is essentially a rewrite of the exising test suite.
It uses a simplified abstraction of the uAPI interfaces to allow a common
test suite to test the gpio-mockup using either of the uAPI interfaces.
The simplified cdev interface is implemented in gpio-mockup.sh, with the
actual driving of the uAPI implemented in gpio-mockup-cdev.c.
The simplified sysfs interface replaces gpio-mockup-sysfs.sh and is
loaded over the cdev implementation when selected.
The new tests should also be simpler to extend to cover new mockup
interfaces, such as the one Bart has been working on.
I have dropped support for testing modules other than gpio-mockup from
the command line options, as the tests are very gpio-mockup specific so
I didn't see any calling for it.
I have also tried to emphasise in the test output that the tests are
covering the gpio-mockup itself. They do perform some implicit testing
of gpiolib and the uAPI interfaces, and so can be useful as smoke tests
for those, but their primary focus is the gpio-mockup.
Patches 2 through 5 do some cleaning up that is now possible with the
new implementation, including enabling building in buildroot environments.
Patch 4 doesn't strictly clean up all the old gpio references that it
could - the gpio was the only Level 1 test, so the Level 1 tests could
potentially be removed, but I was unsure if there may be other
implications to removing a whole test level, or that it may be useful
as a placeholder in case other static LDLIBS tests are added in
the future??
Patch 6 finally gets around to porting the tests to the latest GPIO uAPI.
And Patch 7 updates the config to set the CONFIG_GPIO_CDEV option that
was added in v5.10.
Cheers,
Kent.
Changes v2 -> v3:
- remove 'commit' from Fixes tag in patch 1.
- rebase on Bart's gpio/for-next
Changes v1 -> v2 (all in patch 1 and gpio-mockup.sh unless stated
otherwise):
- reorder includes in gpio-mockup-cdev.c
- a multitude of improvements to gpio-mockup.sh and gpio-mockup-sysfs.sh
based on Andy's review comments
- improved cleanup to ensure all child processes are killed on exit
- added race condition prevention or mitigation including the wait in
release_line, the retries in assert_mock, the assert_mock in set_mock,
and the sleep in set_line
Kent Gibson (7):
selftests: gpio: rework and simplify test implementation
selftests: gpio: remove obsolete gpio-mockup-chardev.c
selftests: remove obsolete build restriction for gpio
selftests: remove obsolete gpio references from kselftest_deps.sh
tools: gpio: remove uAPI v1 code no longer used by selftests
selftests: gpio: port to GPIO uAPI v2
selftests: gpio: add CONFIG_GPIO_CDEV to config
tools/gpio/gpio-utils.c | 89 ----
tools/gpio/gpio-utils.h | 6 -
tools/testing/selftests/Makefile | 9 -
tools/testing/selftests/gpio/Makefile | 26 +-
tools/testing/selftests/gpio/config | 1 +
.../testing/selftests/gpio/gpio-mockup-cdev.c | 198 +++++++
.../selftests/gpio/gpio-mockup-chardev.c | 323 ------------
.../selftests/gpio/gpio-mockup-sysfs.sh | 168 ++----
tools/testing/selftests/gpio/gpio-mockup.sh | 497 ++++++++++++------
tools/testing/selftests/kselftest_deps.sh | 4 +-
10 files changed, 603 insertions(+), 718 deletions(-)
create mode 100644 tools/testing/selftests/gpio/gpio-mockup-cdev.c
delete mode 100644 tools/testing/selftests/gpio/gpio-mockup-chardev.c
base-commit: 64e6066e16b8c562983dd9d33e604c0001ae0fc7
--
2.30.0
If you are not using the test-definitions project to run kselftest,
please ignore this email.
A new run script for kselftest, run_kselftest.sh [1], was created during the
Linux v5.10 release.
This script allows someone to run both individual test cases and sets of
test cases. Accordingly, the test-definitions kselftest script [2] was also
improved to support these upstream changes [1]. Currently this change is in
the test-definitions repository in a separate branch "kselftest". This has been
running in LKFT's CI since November 2020 [3].
The test-definitions kselftest script will stop supporting older versions of
the kselftest run script starting 1st-Feb-2021 from master branch.
OTOH, One have to use test-definitions Tag 2021.01 (will be created) for older
kselftest versions.
We request that any users of test-definitions project start updating your
kselftest sources to version v5.10 and above.
Upstream patch,
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/patch/to…
[2] https://github.com/Linaro/test-definitions/tree/kselftest/automated/linux/k…
[3] https://github.com/Linaro/test-definitions/tree/kselftest
---
>From 5da1918446a1d50d57f2f6062f7fdede0b052473 Mon Sep 17 00:00:00 2001
From: Kees Cook <keescook(a)chromium.org>
Date: Mon, 28 Sep 2020 13:26:49 -0700
Subject: selftests/run_kselftest.sh: Make each test individually selectable
Currently with run_kselftest.sh there is no way to choose which test
we could run. All the tests listed in kselftest-list.txt are all run
every time. This patch enhanced the run_kselftest.sh to make the test
collections (or tests) individually selectable. e.g.:
$ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
Additionally adds a way to list all known tests with "-l", usage
with "-h", and perform a dry run without running tests with "-n".
Co-developed-by: Hangbin Liu <liuhangbin(a)gmail.com>
Signed-off-by: Hangbin Liu <liuhangbin(a)gmail.com>
Signed-off-by: Kees Cook <keescook(a)chromium.org>
Tested-by: Naresh Kamboju <naresh.kamboju(a)linaro.org>
Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org>
--
- Naresh Kamboju
On Wed, Jan 20, 2021 at 6:22 AM angkery <angkery(a)163.com> wrote:
>
> From: Junlin Yang <yangjunlin(a)yulong.com>
>
> Change 'exeeds' to 'exceeds'.
>
> Signed-off-by: Junlin Yang <yangjunlin(a)yulong.com>
The patch didn't reach patchwork.
Please reduce cc list and resubmit to bpf@vger only.
Also pls mention [PATCH bpf-next] in the subject.
> ---
> tools/testing/selftests/bpf/prog_tests/btf.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
> index 8ae97e2..ea008d0 100644
> --- a/tools/testing/selftests/bpf/prog_tests/btf.c
> +++ b/tools/testing/selftests/bpf/prog_tests/btf.c
> @@ -914,7 +914,7 @@ struct btf_raw_test {
> .err_str = "Member exceeds struct_size",
> },
>
> -/* Test member exeeds the size of struct
> +/* Test member exceeds the size of struct
> *
> * struct A {
> * int m;
> @@ -948,7 +948,7 @@ struct btf_raw_test {
> .err_str = "Member exceeds struct_size",
> },
>
> -/* Test member exeeds the size of struct
> +/* Test member exceeds the size of struct
> *
> * struct A {
> * int m;
> --
> 1.9.1
>
>
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
@Andrew, this is based on v5.10-rc2-mmotm-2020-11-07-21-40, I can rebase on
current mmotm if you prefer.
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v14:
* Finally s/mod_node_page_state/mod_lruvec_page_state/
v13: https://lore.kernel.org/lkml/20201201074559.27742-1-rppt@kernel.org
* Added Reviewed-by, thanks Catalin and David
* s/mod_node_page_state/mod_lruvec_page_state/ as Shakeel suggested
v12: https://lore.kernel.org/lkml/20201125092208.12544-1-rppt@kernel.org
* Add detection of whether set_direct_map has actual effect on arm64 and bail
out of CMA allocation for secretmem and the memfd_secret() syscall if pages
would not be removed from the direct map
v11: https://lore.kernel.org/lkml/20201124092556.12009-1-rppt@kernel.org
* Drop support for uncached mappings
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
* Drop changes to arm64 compatibility layer
* Add Roman's Ack for memcg accounting
Older history:
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (10):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
set_memory: allow querying whether set_direct_map_*() is actually enabled
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/cacheflush.h | 6 -
arch/arm64/include/asm/set_memory.h | 17 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/kernel/machine_kexec.c | 1 +
arch/arm64/mm/mmu.c | 6 +-
arch/arm64/mm/pageattr.c | 23 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 2 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 16 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 439 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 906 insertions(+), 51 deletions(-)
create mode 100644 arch/arm64/include/asm/set_memory.h
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
In this scenario, there is no case where va_page is NULL, and
the error has been checked. The if condition statement here is
redundant, so remove the condition detection.
Reported-by: Jia Zhang <zhang.jia(a)linux.alibaba.com>
Signed-off-by: Tianjia Zhang <tianjia.zhang(a)linux.alibaba.com>
---
arch/x86/kernel/cpu/sgx/ioctl.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c
index 90a5caf76939..f45957c05f69 100644
--- a/arch/x86/kernel/cpu/sgx/ioctl.c
+++ b/arch/x86/kernel/cpu/sgx/ioctl.c
@@ -66,9 +66,8 @@ static int sgx_encl_create(struct sgx_encl *encl, struct sgx_secs *secs)
va_page = sgx_encl_grow(encl);
if (IS_ERR(va_page))
return PTR_ERR(va_page);
- else if (va_page)
- list_add(&va_page->list, &encl->va_pages);
- /* else the tail page of the VA page list had free slots. */
+
+ list_add(&va_page->list, &encl->va_pages);
/* The extra page goes to SECS. */
encl_size = secs->size + PAGE_SIZE;
--
2.19.1.3.ge56e4f7
Increase `section->free_cnt` in sgx_sanitize_section() is
more reasonable, which is called in ksgxd kernel thread,
instead of assigning it to epc section pages number at
initialization. Although this is unlikely to fail, these
pages cannot be allocated after initialization, and which
need to be reset by ksgxd.
Reported-by: Jia Zhang <zhang.jia(a)linux.alibaba.com>
Signed-off-by: Tianjia Zhang <tianjia.zhang(a)linux.alibaba.com>
---
arch/x86/kernel/cpu/sgx/main.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index c519fc5f6948..9e9a3cf7c00b 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -48,9 +48,10 @@ static void sgx_sanitize_section(struct sgx_epc_section *section)
struct sgx_epc_page, list);
ret = __eremove(sgx_get_epc_virt_addr(page));
- if (!ret)
+ if (!ret) {
list_move(&page->list, §ion->page_list);
- else
+ section->free_cnt += 1;
+ } else
list_move_tail(&page->list, &dirty);
spin_unlock(§ion->lock);
@@ -646,7 +647,6 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
list_add_tail(§ion->pages[i].list, §ion->init_laundry_list);
}
- section->free_cnt = nr_pages;
return true;
}
--
2.19.1.3.ge56e4f7
From: Colin Ian King <colin.king(a)canonical.com>
There are two spelling mistakes in check_fail messages. Fix them.
Signed-off-by: Colin Ian King <colin.king(a)canonical.com>
---
tools/testing/selftests/net/forwarding/tc_chains.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/net/forwarding/tc_chains.sh b/tools/testing/selftests/net/forwarding/tc_chains.sh
index 2934fb5ed2a2..b95de0463ebd 100755
--- a/tools/testing/selftests/net/forwarding/tc_chains.sh
+++ b/tools/testing/selftests/net/forwarding/tc_chains.sh
@@ -136,7 +136,7 @@ template_filter_fits()
tc filter add dev $h2 ingress protocol ip pref 1 handle 1102 \
flower src_mac $h2mac action drop &> /dev/null
- check_fail $? "Incorrectly succeded to insert filter which does not template"
+ check_fail $? "Incorrectly succeeded to insert filter which does not template"
tc filter add dev $h2 ingress chain 1 protocol ip pref 1 handle 1101 \
flower src_mac $h2mac action drop
@@ -144,7 +144,7 @@ template_filter_fits()
tc filter add dev $h2 ingress chain 1 protocol ip pref 1 handle 1102 \
flower dst_mac $h2mac action drop &> /dev/null
- check_fail $? "Incorrectly succeded to insert filter which does not template"
+ check_fail $? "Incorrectly succeeded to insert filter which does not template"
tc filter del dev $h2 ingress chain 1 protocol ip pref 1 handle 1102 \
flower &> /dev/null
--
2.29.2
Changelog
---------
v5
- Added the following patches to the beginning of series, which are fixes
to the other existing problems with CMA migration code:
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors also at the beginning of series
mm/gup: do not allow zero page for pinned pages
- remove .gfp_mask/.reclaim_idx changes from mm/vmscan.c
- update movable zone header comment in patch 8 instead of patch 3, fix the comment
- Added acked, sign-offs
- Updated commit logs based on feedback
- Addressed issues reported by Michal and Jason.
- Remove:
#define PINNABLE_MIGRATE_MAX 10
#define PINNABLE_ISOLATE_MAX 100
Instead: fail on the first migration failure, and retry isolation forever as their
failures are transient.
- In self-set addressed some of the comments from John Hubbard, updated commit logs,
and added comments. Renamed gup->flags with gup->test_flags.
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
v4
https://lore.kernel.org/lkml/20201217185243.3288048-1-pasha.tatashin@soleen…
Pavel Tatashin (14):
mm/gup: don't pin migrated cma pages in movable zone
mm/gup: check every subpage of a compound page during isolation
mm/gup: return an error on migration failure
mm/gup: check for isolation errors
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: do not allow zero page for pinned pages
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: longterm pin migration cleaup
selftests/vm: test flag is broken
selftests/vm: test faulting in kernel, and verify pinnable pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 11 ++
include/linux/mmzone.h | 9 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 178 ++++++++----------
mm/gup_test.c | 29 +--
mm/gup_test.h | 3 +-
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++--
tools/testing/selftests/vm/gup_test.c | 36 +++-
13 files changed, 185 insertions(+), 160 deletions(-)
--
2.25.1
Initially I just wanted to port the selftests to the latest GPIO uAPI,
but on finding that, due to dependency issues, the selftests are not built
for the buildroot environments that I do most of my GPIO testing in, I
decided to take a closer look.
The first patch is essentially a rewrite of the exising test suite.
It uses a simplified abstraction of the uAPI interfaces to allow a common
test suite to test the gpio-mockup using either of the uAPI interfaces.
The simplified cdev interface is implemented in gpio-mockup.sh, with the
actual driving of the uAPI implemented in gpio-mockup-cdev.c.
The simplified sysfs interface replaces gpio-mockup-sysfs.sh and is
loaded over the cdev implementation when selected.
The new tests should also be simpler to extend to cover new mockup
interfaces, such as the one Bart has been working on.
I have dropped support for testing modules other than gpio-mockup from
the command line options, as the tests are very gpio-mockup specific so
I didn't see any calling for it.
I have also tried to emphasise in the test output that the tests are
covering the gpio-mockup itself. They do perform some implicit testing
of gpiolib and the uAPI interfaces, and so can be useful as smoke tests
for those, but their primary focus is the gpio-mockup.
Patches 2 through 5 do some cleaning up that is now possible with the
new implementation, including enabling building in buildroot environments.
Patch 4 doesn't strictly clean up all the old gpio references that it
could - the gpio was the only Level 1 test, so the Level 1 tests could
potentially be removed, but I was unsure if there may be other
implications to removing a whole test level, or that it may be useful
as a placeholder in case other static LDLIBS tests are added in
the future??
Patch 6 finally gets around to porting the tests to the latest GPIO uAPI.
And Patch 7 updates the config to set the CONFIG_GPIO_CDEV option that
was added in v5.10.
Cheers,
Kent.
Changes v1 -> v2 (all in patch 1 and gpio-mockup.sh unless stated
otherwise):
- reorder includes in gpio-mockup-cdev.c
- a multitude of improvements to gpio-mockup.sh and gpio-mockup-sysfs.sh
based on Andy's review comments
- improved cleanup to ensure all child processes are killed on exit
- added race condition prevention or mitigation including the wait in
release_line, the retries in assert_mock, the assert_mock in set_mock,
and the sleep in set_line
Kent Gibson (7):
selftests: gpio: rework and simplify test implementation
selftests: gpio: remove obsolete gpio-mockup-chardev.c
selftests: remove obsolete build restriction for gpio
selftests: remove obsolete gpio references from kselftest_deps.sh
tools: gpio: remove uAPI v1 code no longer used by selftests
selftests: gpio: port to GPIO uAPI v2
selftests: gpio: add CONFIG_GPIO_CDEV to config
tools/gpio/gpio-utils.c | 89 ----
tools/gpio/gpio-utils.h | 6 -
tools/testing/selftests/Makefile | 9 -
tools/testing/selftests/gpio/Makefile | 26 +-
tools/testing/selftests/gpio/config | 1 +
.../testing/selftests/gpio/gpio-mockup-cdev.c | 198 +++++++
.../selftests/gpio/gpio-mockup-chardev.c | 323 ------------
.../selftests/gpio/gpio-mockup-sysfs.sh | 168 ++----
tools/testing/selftests/gpio/gpio-mockup.sh | 497 ++++++++++++------
tools/testing/selftests/kselftest_deps.sh | 4 +-
10 files changed, 603 insertions(+), 718 deletions(-)
create mode 100644 tools/testing/selftests/gpio/gpio-mockup-cdev.c
delete mode 100644 tools/testing/selftests/gpio/gpio-mockup-chardev.c
--
2.30.0
This series contains a few cleanups that didn't make it into previous
series, including some cosmetic changes and small bug fixes. The series
also lays the groundwork for a memslot modification test which stresses
the memslot update and page fault code paths in an attempt to expose races.
Tested: dirty_log_perf_test, memslot_modification_stress_test, and
demand_paging_test were run, with all the patches in this series
applied, on an Intel Skylake machine.
echo Y > /sys/module/kvm/parameters/tdp_mmu; \
./memslot_modification_stress_test -i 1000 -v 64 -b 1G; \
./memslot_modification_stress_test -i 1000 -v 64 -b 64M -o; \
./dirty_log_perf_test -v 64 -b 1G; \
./dirty_log_perf_test -v 64 -b 64M -o; \
./demand_paging_test -v 64 -b 1G; \
./demand_paging_test -v 64 -b 64M -o; \
echo N > /sys/module/kvm/parameters/tdp_mmu; \
./memslot_modification_stress_test -i 1000 -v 64 -b 1G; \
./memslot_modification_stress_test -i 1000 -v 64 -b 64M -o; \
./dirty_log_perf_test -v 64 -b 1G; \
./dirty_log_perf_test -v 64 -b 64M -o; \
./demand_paging_test -v 64 -b 1G; \
./demand_paging_test -v 64 -b 64M -o
The tests behaved as expected, and fixed the problem of the
population stage being skipped in dirty_log_perf_test. This can be
seen in the output, with the population stage taking about the time
dirty pass 1 took and dirty pass 1 falling closer to the times for
the other passes.
Note that when running these tests, the -o option causes the test to take
much longer as the work each vCPU must do increases proportional to the
number of vCPUs.
You can view this series in Gerrit at:
https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/7…
Ben Gardon (6):
KVM: selftests: Rename timespec_diff_now to timespec_elapsed
KVM: selftests: Avoid flooding debug log while populating memory
KVM: selftests: Convert iterations to int in dirty_log_perf_test
KVM: selftests: Fix population stage in dirty_log_perf_test
KVM: selftests: Add option to overlap vCPU memory access
KVM: selftests: Add memslot modification stress test
tools/testing/selftests/kvm/.gitignore | 1 +
tools/testing/selftests/kvm/Makefile | 1 +
.../selftests/kvm/demand_paging_test.c | 40 +++-
.../selftests/kvm/dirty_log_perf_test.c | 72 +++---
.../selftests/kvm/include/perf_test_util.h | 4 +-
.../testing/selftests/kvm/include/test_util.h | 2 +-
.../selftests/kvm/lib/perf_test_util.c | 25 ++-
tools/testing/selftests/kvm/lib/test_util.c | 2 +-
.../kvm/memslot_modification_stress_test.c | 211 ++++++++++++++++++
9 files changed, 307 insertions(+), 51 deletions(-)
create mode 100644 tools/testing/selftests/kvm/memslot_modification_stress_test.c
--
2.30.0.284.gd98b1dd5eaa7-goog
Changelog
---------
v4
- Address page migration comments. New patch:
mm/gup: limit number of gup migration failures, honor failures
Implements the limiting number of retries for migration failures, and
also check for isolation failures.
Added a test case into gup_test to verify that pages never long-term
pinned in a movable zone, and also added tests to fault both in kernel
and in userland.
v3
- Merged with linux-next, which contains clean-up patch from Jason,
therefore this series is reduced by two patches which did the same
thing.
v2
- Addressed all review comments
- Added Reviewed-by's.
- Renamed PF_MEMALLOC_NOMOVABLE to PF_MEMALLOC_PIN
- Added is_pinnable_page() to check if page can be longterm pinned
- Fixed gup fast path by checking is_in_pinnable_zone()
- rename cma_page_list to movable_page_list
- add a admin-guide note about handling pinned pages in ZONE_MOVABLE,
updated caveat about pinned pages from linux/mmzone.h
- Move current_gfp_context() to fast-path
---------
When page is pinned it cannot be moved and its physical address stays
the same until pages is unpinned.
This is useful functionality to allows userland to implementation DMA
access. For example, it is used by vfio in vfio_pin_pages().
However, this functionality breaks memory hotplug/hotremove assumptions
that pages in ZONE_MOVABLE can always be migrated.
This patch series fixes this issue by forcing new allocations during
page pinning to omit ZONE_MOVABLE, and also to migrate any existing
pages from ZONE_MOVABLE during pinning.
It uses the same scheme logic that is currently used by CMA, and extends
the functionality for all allocations.
For more information read the discussion [1] about this problem.
[1] https://lore.kernel.org/lkml/CA+CK2bBffHBxjmb9jmSKacm0fJMinyt3Nhk8Nx6iudcQS…
Previous versions:
v1
https://lore.kernel.org/lkml/20201202052330.474592-1-pasha.tatashin@soleen.…
v2
https://lore.kernel.org/lkml/20201210004335.64634-1-pasha.tatashin@soleen.c…
v3
https://lore.kernel.org/lkml/20201211202140.396852-1-pasha.tatashin@soleen.…
Pavel Tatashin (10):
mm/gup: don't pin migrated cma pages in movable zone
mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN
mm: apply per-task gfp constraints in fast path
mm: honor PF_MEMALLOC_PIN for all movable pages
mm/gup: migrate pinned pages out of movable zone
memory-hotplug.rst: add a note about ZONE_MOVABLE and page pinning
mm/gup: change index type to long as it counts pages
mm/gup: limit number of gup migration failures, honor failures
selftests/vm: test flag is broken
selftests/vm: test faulting in kernel, and verify pinnable pages
.../admin-guide/mm/memory-hotplug.rst | 9 +
include/linux/migrate.h | 1 +
include/linux/mm.h | 11 +
include/linux/mmzone.h | 11 +-
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 27 +--
include/trace/events/migrate.h | 3 +-
mm/gup.c | 217 ++++++++++--------
mm/gup_test.c | 15 +-
mm/gup_test.h | 1 +
mm/hugetlb.c | 4 +-
mm/page_alloc.c | 33 ++-
mm/vmscan.c | 10 +-
tools/testing/selftests/vm/gup_test.c | 26 ++-
14 files changed, 221 insertions(+), 149 deletions(-)
--
2.25.1
On Mon, Jan 11, 2021 at 9:34 AM Alan Maguire <alan.maguire(a)oracle.com> wrote:
>
> libbpf already supports a "dumper" API for dumping type information,
> but there is currently no support for dumping typed _data_ via libbpf.
> However this functionality does exist in the kernel, in part to
> facilitate the bpf_snprintf_btf() helper which dumps a string
> representation of the pointer passed in utilizing the BTF type id
> of the data pointed to. For example, the pair of a pointer to
> a "struct sk_buff" and the BTF type id of "struct sk_buff" can be
> used.
>
> Here the kernel code is generalized into btf_show_common.c. For the
> most part, code is identical for userspace and kernel, beyond a few API
> differences and missing functions. The only significant differences are
>
> - the "safe copy" logic used by the kernel to ensure we do not induce a
> crash during BPF operation; and
> - the BTF seq file support that is kernel-only.
>
> The mechanics are to maintain identical btf_show_common.c files in
> kernel/bpf and tools/lib/bpf , and a common header btf_common.h in
> include/linux/ and tools/lib/bpf/. This file duplication seems to
> be the common practice with duplication between kernel and tools/
> so it's the approach taken here.
>
> The common code approach could likely be explored further, but here
> the minimum common code required to support BTF show functionality is
> used.
>
I don't think this approach will work. libbpf and kernel have
considerably different restrictions and styles, I don't think it's
appropriate to take kernel code and try to fit it into libbpf almost
as is, with a bunch of #defines. It would be much cleaner, simpler,
and more maintainable to just re-implement core logic for libbpf, IMO.
> Currently the only "show" function for userspace is to write the
> representation of the typed data to a string via
>
> LIBBPF_API int
> btf__snprintf(struct btf *btf, char *buf, int len, __u32 id, void *obj,
> __u64 flags);
>
> ...but other approaches could be pursued including printf()-based
> show, or even a callback mechanism could be supported to allow
> user-defined show functions.
>
It's strange that you saw btf_dump APIs, and yet decided to go with
this API instead. snprintf() is not a natural "method" of struct btf.
Using char buffer as an output is overly restrictive and inconvenient.
It's appropriate for kernel and BPF program due to their restrictions,
but there is no need to cripple libbpf APIs for that. I think it
should follow btf_dump APIs with custom callback so that it's easy to
just printf() everything, but also user can create whatever elaborate
mechanism they need and that fits their use case.
Code reuse is not the ultimate goal, it should facilitate
maintainability, not harm it. There are times where sharing code
introduces unnecessary coupling and maintainability issues. And I
think this one is a very obvious case of that.
See below a few comments as well. But overall it's really hard to
review such a humongous patch, of course. So I so far just skimmed
through it.
> Here's an example usage, storing a string representation of
> struct sk_buff *skb in buf:
>
> struct btf *btf = libbpf_find_kernel_btf();
> char buf[8192];
> __s32 skb_id;
>
> skb_id = btf__find_by_name_kind(btf, "sk_buff", BTF_KIND_STRUCT);
> if (skb_id < 0)
> fprintf(stderr, "no skbuff, err %d\n", skb_id);
> else
> btf__snprintf(btf, buf, sizeof(buf), skb_id, skb, 0);
>
> Suggested-by: Alexei Starovoitov <ast(a)kernel.org>
> Signed-off-by: Alan Maguire <alan.maguire(a)oracle.com>
> ---
> include/linux/btf.h | 121 +---
> include/linux/btf_common.h | 286 +++++++++
> kernel/bpf/Makefile | 2 +-
> kernel/bpf/arraymap.c | 1 +
> kernel/bpf/bpf_struct_ops.c | 1 +
> kernel/bpf/btf.c | 1215 +-------------------------------------
> kernel/bpf/btf_show_common.c | 1218 +++++++++++++++++++++++++++++++++++++++
> kernel/bpf/core.c | 1 +
> kernel/bpf/hashtab.c | 1 +
> kernel/bpf/local_storage.c | 1 +
> kernel/bpf/verifier.c | 1 +
> kernel/trace/bpf_trace.c | 1 +
> tools/lib/bpf/Build | 2 +-
> tools/lib/bpf/btf.h | 7 +
> tools/lib/bpf/btf_common.h | 286 +++++++++
> tools/lib/bpf/btf_show_common.c | 1218 +++++++++++++++++++++++++++++++++++++++
> tools/lib/bpf/libbpf.map | 1 +
> 17 files changed, 3044 insertions(+), 1319 deletions(-)
> create mode 100644 include/linux/btf_common.h
> create mode 100644 kernel/bpf/btf_show_common.c
> create mode 100644 tools/lib/bpf/btf_common.h
> create mode 100644 tools/lib/bpf/btf_show_common.c
>
[...]
> +/* For kernel u64 is long long unsigned int... */
> +#define FMT64 "ll"
> +
> +#else
> +/* ...while for userspace it is long unsigned int. These definitions avoid
> + * format specifier warnings.
> + */
that's not true, it depends on the architecture
> +#define FMT64 "l"
> +
> +/* libbpf names differ slightly to in-kernel function names. */
> +#define btf_type_by_id btf__type_by_id
> +#define btf_name_by_offset btf__name_by_offset
> +#define btf_str_by_offset btf__str_by_offset
> +#define btf_resolve_size btf__resolve_size
ugh... good luck navigating the code in libbpf....
> +
> +#endif /* __KERNEL__ */
> +/*
> + * Options to control show behaviour.
> + * - BTF_SHOW_COMPACT: no formatting around type information
> + * - BTF_SHOW_NONAME: no struct/union member names/types
> + * - BTF_SHOW_PTR_RAW: show raw (unobfuscated) pointer values;
> + * equivalent to %px.
> + * - BTF_SHOW_ZERO: show zero-valued struct/union members; they
> + * are not displayed by default
> + * - BTF_SHOW_UNSAFE: skip use of bpf_probe_read() to safely read
> + * data before displaying it.
> + */
> +#define BTF_SHOW_COMPACT BTF_F_COMPACT
> +#define BTF_SHOW_NONAME BTF_F_NONAME
> +#define BTF_SHOW_PTR_RAW BTF_F_PTR_RAW
> +#define BTF_SHOW_ZERO BTF_F_ZERO
> +#define BTF_SHOW_UNSAFE (1ULL << 4)
this (or some subset of them) should be done as opts struct's bool
fields for libbpf
> +
> +/*
> + * Copy len bytes of string representation of obj of BTF type_id into buf.
> + *
> + * @btf: struct btf object
> + * @type_id: type id of type obj points to
> + * @obj: pointer to typed data
> + * @buf: buffer to write to
> + * @len: maximum length to write to buf
> + * @flags: show options (see above)
> + *
> + * Return: length that would have been/was copied as per snprintf, or
> + * negative error.
> + */
> +int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
> + char *buf, int len, u64 flags);
> +
> +#define for_each_member(i, struct_type, member) \
> + for (i = 0, member = btf_type_member(struct_type); \
> + i < btf_type_vlen(struct_type); \
> + i++, member++)
> +
> +#define for_each_vsi(i, datasec_type, member) \
> + for (i = 0, member = btf_type_var_secinfo(datasec_type); \
> + i < btf_type_vlen(datasec_type); \
> + i++, member++)
> +
> +static inline bool btf_type_is_ptr(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_PTR;
> +}
> +
> +static inline bool btf_type_is_int(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_INT;
> +}
> +
> +static inline bool btf_type_is_small_int(const struct btf_type *t)
> +{
> + return btf_type_is_int(t) && t->size <= sizeof(u64);
> +}
> +
> +static inline bool btf_type_is_enum(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM;
> +}
> +
> +static inline bool btf_type_is_typedef(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_TYPEDEF;
> +}
> +
> +static inline bool btf_type_is_func(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_FUNC;
> +}
> +
> +static inline bool btf_type_is_func_proto(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_FUNC_PROTO;
> +}
> +
> +static inline bool btf_type_is_var(const struct btf_type *t)
> +{
> + return BTF_INFO_KIND(t->info) == BTF_KIND_VAR;
> +}
> +
> +/* union is only a special case of struct:
> + * all its offsetof(member) == 0
> + */
> +static inline bool btf_type_is_struct(const struct btf_type *t)
> +{
> + u8 kind = BTF_INFO_KIND(t->info);
> +
> + return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION;
> +}
> +
> +static inline bool btf_type_is_modifier(const struct btf_type *t)
> +{
> + /* Some of them is not strictly a C modifier
> + * but they are grouped into the same bucket
> + * for BTF concern:
> + * A type (t) that refers to another
> + * type through t->type AND its size cannot
> + * be determined without following the t->type.
> + *
> + * ptr does not fall into this bucket
> + * because its size is always sizeof(void *).
> + */
> + switch (BTF_INFO_KIND(t->info)) {
> + case BTF_KIND_TYPEDEF:
> + case BTF_KIND_VOLATILE:
> + case BTF_KIND_CONST:
> + case BTF_KIND_RESTRICT:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> +static inline
> +const struct btf_type *btf_type_skip_modifiers(const struct btf *btf,
> + u32 id, u32 *res_id)
> +{
> + const struct btf_type *t = btf_type_by_id(btf, id);
> +
> + while (btf_type_is_modifier(t)) {
> + id = t->type;
> + t = btf_type_by_id(btf, t->type);
> + }
> +
> + if (res_id)
> + *res_id = id;
> +
> + return t;
> +}
> +
> +static inline u32 btf_type_int(const struct btf_type *t)
> +{
> + return *(u32 *)(t + 1);
> +}
> +
> +static inline const struct btf_array *btf_type_array(const struct btf_type *t)
> +{
> + return (const struct btf_array *)(t + 1);
> +}
> +
> +static inline const struct btf_enum *btf_type_enum(const struct btf_type *t)
> +{
> + return (const struct btf_enum *)(t + 1);
> +}
> +
> +static inline const struct btf_var *btf_type_var(const struct btf_type *t)
> +{
> + return (const struct btf_var *)(t + 1);
> +}
> +
> +static inline u16 btf_type_vlen(const struct btf_type *t)
> +{
> + return BTF_INFO_VLEN(t->info);
> +}
> +
> +static inline u16 btf_func_linkage(const struct btf_type *t)
> +{
> + return BTF_INFO_VLEN(t->info);
> +}
> +
> +/* size can be used */
> +static inline bool btf_type_has_size(const struct btf_type *t)
> +{
> + switch (BTF_INFO_KIND(t->info)) {
> + case BTF_KIND_INT:
> + case BTF_KIND_STRUCT:
> + case BTF_KIND_UNION:
> + case BTF_KIND_ENUM:
> + case BTF_KIND_DATASEC:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> +static inline const struct btf_member *btf_type_member(const struct btf_type *t)
> +{
> + return (const struct btf_member *)(t + 1);
> +}
> +
> +static inline const struct btf_var_secinfo *btf_type_var_secinfo(
> + const struct btf_type *t)
> +{
> + return (const struct btf_var_secinfo *)(t + 1);
> +}
> +
> +static inline const char *__btf_name_by_offset(const struct btf *btf,
> + u32 offset)
> +{
> + const char *name;
> +
> + if (!offset)
> + return "(anon)";
> +
> + name = btf_str_by_offset(btf, offset);
> + return name ?: "(invalid-name-offset)";
> +}
> +
(almost?) all of the above helpers are already defined in libbpf's
btf.h, no need to add all this duplication
> +/* functions shared between btf.c and btf_show_common.c */
> +void btf_type_ops_show(const struct btf *btf, const struct btf_type *t,
> + __u32 type_id, void *obj, u8 bits_offset,
> + struct btf_show *show);
[...]
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index 1c0fd2d..35bd9dc 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -346,6 +346,7 @@ LIBBPF_0.3.0 {
> btf__parse_split;
> btf__new_empty_split;
> btf__new_split;
> + btf__snprintf;
It's LIBBPF_0.4.0 already, I or someone else should send a patch
adding a new section in .map file.
> ring_buffer__epoll_fd;
> xsk_setup_xdp_prog;
> xsk_socket__update_xskmap;
> --
> 1.8.3.1
>
Commit fadb08e7c750 ("kunit: Support for Parameterized Testing")
introduced support but lacks documentation for how to use it.
This patch builds on commit 1f0e943df68a ("Documentation: kunit: provide
guidance for testing many inputs") to show a minimal example of the new
feature.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Documentation/dev-tools/kunit/usage.rst | 57 +++++++++++++++++++++++++
1 file changed, 57 insertions(+)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index d9fdc14f0677..650f99590df5 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -522,6 +522,63 @@ There's more boilerplate involved, but it can:
* E.g. if we wanted to also test ``sha256sum``, we could add a ``sha256``
field and reuse ``cases``.
+* be converted to a "parameterized test", see below.
+
+Parameterized Testing
+~~~~~~~~~~~~~~~~~~~~~
+
+The table-driven testing pattern is common enough that KUnit has special
+support for it.
+
+Reusing the same ``cases`` array from above, we can write the test as a
+"parameterized test" with the following.
+
+.. code-block:: c
+
+ // This is copy-pasted from above.
+ struct sha1_test_case {
+ const char *str;
+ const char *sha1;
+ };
+ struct sha1_test_case cases[] = {
+ {
+ .str = "hello world",
+ .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
+ },
+ {
+ .str = "hello world!",
+ .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169",
+ },
+ };
+
+ // Need a helper function to generate a name for each test case.
+ static void case_to_desc(const struct sha1_test_case *t, char *desc)
+ {
+ strcpy(desc, t->str);
+ }
+ // Creates `sha1_gen_params()` to iterate over `cases`.
+ KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc);
+
+ // Looks no different from a normal test.
+ static void sha1_test(struct kunit *test)
+ {
+ // This function can just contain the body of the for-loop.
+ // The former `cases[i]` is accessible under test->param_value.
+ char out[40];
+ struct sha1_test_case *test_param = (struct sha1_test_case *)(test->param_value);
+
+ sha1sum(test_param->str, out);
+ KUNIT_EXPECT_STREQ_MSG(test, (char *)out, test_param->sha1,
+ "sha1sum(%s)", test_param->str);
+ }
+
+ // Instead of KUNIT_CASE, we use KUNIT_CASE_PARAM and pass in the
+ // function declared by KUNIT_ARRAY_PARAM.
+ static struct kunit_case sha1_test_cases[] = {
+ KUNIT_CASE_PARAM(sha1_test, sha1_gen_params),
+ {}
+ };
+
.. _kunit-on-non-uml:
KUnit on non-UML architectures
base-commit: 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a
--
2.29.2.684.gfbc64c5ab5-goog
Hi Linus,
Please pull the following Kselftest fixes update for Linux 5.11-rc4
This Kselftest fixes update for Linux 5.11-rc4 consists of one single
fix to skip BPF selftests by default. BPF selftests have a hard
dependency on cutting edge versions of tools in the BPF ecosystem
including LLVM.
Skipping BPF allows by default will make it easier for users interested
in running kselftest as a whole. Users can include BPF in Kselftest
build by via SKIP_TARGETS variable.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62:
Linux 5.11-rc2 (2021-01-03 15:55:30 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-fixes-5.11-rc4
for you to fetch changes up to 7a6eb7c34a78498742b5f82543b7a68c1c443329:
selftests: Skip BPF seftests by default (2021-01-04 09:27:20 -0700)
----------------------------------------------------------------
linux-kselftest-fixes-5.11-rc4
This Kselftest fixes update for Linux 5.11-rc4 consists of one single
fix to skip BPF selftests by default. BPF selftests have a hard
dependency on cutting edge versions of tools in the BPF ecosystem
including LLVM.
Skipping BPF allows by default will make it easier for users interested
in running kselftest as a whole. Users can include BPF in Kselftest
build by via SKIP_TARGETS variable.
----------------------------------------------------------------
Mark Brown (1):
selftests: Skip BPF seftests by default
tools/testing/selftests/Makefile | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------
Add support for pointer to mem register spilling, to allow the verifier
to track pointers to valid memory addresses. Such pointers are returned
for example by a successful call of the bpf_ringbuf_reserve helper.
The patch was partially contributed by CyberArk Software, Inc.
Fixes: 457f44363a88 ("bpf: Implement BPF ring buffer and verifier support for it")
Suggested-by: Yonghong Song <yhs(a)fb.com>
Signed-off-by: Gilad Reti <gilad.reti(a)gmail.com>
---
kernel/bpf/verifier.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 17270b8404f1..36af69fac591 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2217,6 +2217,8 @@ static bool is_spillable_regtype(enum bpf_reg_type type)
case PTR_TO_RDWR_BUF:
case PTR_TO_RDWR_BUF_OR_NULL:
case PTR_TO_PERCPU_BTF_ID:
+ case PTR_TO_MEM:
+ case PTR_TO_MEM_OR_NULL:
return true;
default:
return false;
--
2.27.0
A few fixes for cross-building the sefltests out of tree. This will
enable wider automated testing on various Arm hardware.
Changes since v1 [1]:
* Use wildcard in patch 5
* Move the MAKE_DIRS declaration in patch 1
[1] https://lore.kernel.org/bpf/20210112135959.649075-1-jean-philippe@linaro.or…
Jean-Philippe Brucker (5):
selftests/bpf: Enable cross-building
selftests/bpf: Fix out-of-tree build
selftests/bpf: Move generated test files to $(TEST_GEN_FILES)
selftests/bpf: Fix installation of urandom_read
selftests/bpf: Install btf_dump test cases
tools/testing/selftests/bpf/Makefile | 58 ++++++++++++++++++++--------
1 file changed, 41 insertions(+), 17 deletions(-)
--
2.30.0
Following the build fixes for bpftool [1], here are a few fixes for
cross-building the sefltests out of tree. This will enable wider
automated testing on various Arm hardware.
[1] https://lore.kernel.org/bpf/20201110164310.2600671-1-jean-philippe@linaro.o…
Jean-Philippe Brucker (5):
selftests/bpf: Enable cross-building
selftests/bpf: Fix out-of-tree build
selftests/bpf: Move generated test files to $(TEST_GEN_FILES)
selftests/bpf: Fix installation of urandom_read
selftests/bpf: Install btf_dump test cases
tools/testing/selftests/bpf/Makefile | 61 +++++++++++++++++++++-------
1 file changed, 46 insertions(+), 15 deletions(-)
--
2.30.0
The BPF Type Format (BTF) can be used in conjunction with the helper
bpf_snprintf_btf() to display kernel data with type information.
This series generalizes that support and shares it with libbpf so
that libbpf can display typed data. BTF display functionality is
factored out of kernel/bpf/btf.c into kernel/bpf/btf_show_common.c,
and that file is duplicated in tools/lib/bpf. Similarly, common
definitions and inline functions needed for this support are
extracted into include/linux/btf_common.h and this header is again
duplicated in tools/lib/bpf.
Patch 1 carries out the refactoring, for which no kernel changes
are intended, and introduces btf__snprintf() a libbpf function
that supports dumping a string representation of typed data using
the struct btf * and id associated with that type.
Patch 2 tests btf__snprintf() with built-in and kernel types to
ensure data is of expected format. The test closely mirrors
the BPF program associated with the snprintf_btf.c; in this case
however the string representations are verified in userspace rather
than in BPF program context.
Alan Maguire (2):
bpf: share BTF "show" implementation between kernel and libbpf
selftests/bpf: test libbpf-based type display
include/linux/btf.h | 121 +-
include/linux/btf_common.h | 286 +++++
kernel/bpf/Makefile | 2 +-
kernel/bpf/arraymap.c | 1 +
kernel/bpf/bpf_struct_ops.c | 1 +
kernel/bpf/btf.c | 1215 +------------------
kernel/bpf/btf_show_common.c | 1218 ++++++++++++++++++++
kernel/bpf/core.c | 1 +
kernel/bpf/hashtab.c | 1 +
kernel/bpf/local_storage.c | 1 +
kernel/bpf/verifier.c | 1 +
kernel/trace/bpf_trace.c | 1 +
tools/lib/bpf/Build | 2 +-
tools/lib/bpf/btf.h | 7 +
tools/lib/bpf/btf_common.h | 286 +++++
tools/lib/bpf/btf_show_common.c | 1218 ++++++++++++++++++++
tools/lib/bpf/libbpf.map | 1 +
.../selftests/bpf/prog_tests/snprintf_btf_user.c | 192 +++
18 files changed, 3236 insertions(+), 1319 deletions(-)
create mode 100644 include/linux/btf_common.h
create mode 100644 kernel/bpf/btf_show_common.c
create mode 100644 tools/lib/bpf/btf_common.h
create mode 100644 tools/lib/bpf/btf_show_common.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/snprintf_btf_user.c
--
1.8.3.1
If a signed number field starts with a '-' the field width must be > 1,
or unlimited, to allow at least one digit after the '-'.
This patch adds a check for this. If a signed field starts with '-'
and field_width == 1 the scanf will quit.
It is ok for a signed number field to have a field width of 1 if it
starts with a digit. In that case the single digit can be converted.
Signed-off-by: Richard Fitzgerald <rf(a)opensource.cirrus.com>
---
lib/vsprintf.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 14c9a6af1b23..8954ff94a53c 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -3433,8 +3433,12 @@ int vsscanf(const char *buf, const char *fmt, va_list args)
str = skip_spaces(str);
digit = *str;
- if (is_sign && digit == '-')
+ if (is_sign && digit == '-') {
+ if (field_width == 1)
+ break;
+
digit = *(str + 1);
+ }
if (!digit
|| (base == 16 && !isxdigit(digit))
--
2.20.1
Hi Linus,
Please pull the following fixes update for Linux 5.11-rc3.
This fixes update for 5.11-rc3 consists of two minor fixes to vDSO test
changes in 5.11-rc1 update.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62:
Linux 5.11-rc2 (2021-01-03 15:55:30 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-next-5.11-rc3
for you to fetch changes up to df00d02989024d193a6efd1a85513a5658c6a10f:
selftests/vDSO: fix -Wformat warning in vdso_test_correctness
(2021-01-04 09:25:45 -0700)
----------------------------------------------------------------
linux-kselftest-next-5.11-rc3
This fixes update for 5.11-rc3 consists of two minor fixes to vDSO test
changes in 5.11-rc1 update.
----------------------------------------------------------------
Tobias Klauser (2):
selftests/vDSO: add additional binaries to .gitignore
selftests/vDSO: fix -Wformat warning in vdso_test_correctness
tools/testing/selftests/vDSO/.gitignore | 3 +++
tools/testing/selftests/vDSO/vdso_test_correctness.c | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------
Hi Linus,
Please pull the following KUnit fixes update for Linux 5.11-rc3.
This kunit update for Linux 5.11-rc3 consists one fix to force the use
of the 'tty' console for UML. Given that kunit tool requires the console
output, explicitly stating the dependency makes sense than relying on
it being the default.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62:
Linux 5.11-rc2 (2021-01-03 15:55:30 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-fixes-5.11-rc3
for you to fetch changes up to 65a4e5299739abe0888cda0938d21f8ea3b5c606:
kunit: tool: Force the use of the 'tty' console for UML (2021-01-04
09:18:38 -0700)
----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.11-rc3
This kunit update for Linux 5.11-rc3 consists one fix to force the use
of the 'tty' console for UML. Given that kunit tool requires the console
output, explicitly stating the dependency makes sense than relying on
it being the default.
----------------------------------------------------------------
David Gow (1):
kunit: tool: Force the use of the 'tty' console for UML
tools/testing/kunit/kunit_kernel.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------
kunit_tool relies on the UML console outputting printk() output to the
tty in order to get results. Since the default console driver could
change, pass 'console=tty' to the kernel.
This is triggered by a change[1] to use ttynull as a fallback console
driver which -- by chance or by design -- seems to have changed the
default console output on UML, breaking kunit_tool. While this may be
fixed, we should be less fragile to such changes in the default.
[1]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?…
Signed-off-by: David Gow <davidgow(a)google.com>
Fixes: 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console")
---
tools/testing/kunit/kunit_kernel.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index 57c1724b7e5d..698358c9c0d6 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -198,7 +198,7 @@ class LinuxSourceTree(object):
return self.validate_config(build_dir)
def run_kernel(self, args=[], build_dir='', timeout=None):
- args.extend(['mem=1G'])
+ args.extend(['mem=1G', 'console=tty'])
self._ops.linux_bin(args, timeout, build_dir)
outfile = get_outfile_path(build_dir)
subprocess.call(['stty', 'sane'])
--
2.29.2.729.g45daf8777d-goog
Hi!
For last few weeks KUnit stopped working. Any insight?
P.S. I guess no need to tell that my kernel on which I run tests has not been
changed as well as command line for wrapper:
tools/testing/kunit/kunit.py run --build_dir ~/$OUT_DIR
--
With Best Regards,
Andy Shevchenko
Initially I just wanted to port the selftests to the latest GPIO uAPI,
but on finding that, due to dependency issues, the selftests are not built
for the buildroot environments that I do most of my GPIO testing in, I
decided to take a closer look.
The first patch is essentially a rewrite of the exising test suite.
It uses a simplified abstraction of the uAPI interfaces to allow a common
test suite to test the gpio-mockup using either of the uAPI interfaces.
The simplified cdev interface is implemented in gpio-mockup.sh, with the
actual driving of the uAPI implemented in gpio-mockup-cdev.c.
The simplified sysfs interface replaces gpio-mockup-sysfs.sh and is
loaded over the cdev implementation when selected.
The new tests should also be simpler to extend to cover new mockup
interfaces, such as the one Bart has been working on.
I have dropped support for testing modules other than gpio-mockup from
the command line options, as the tests are very gpio-mockup specific so
I didn't see any calling for it.
I have also tried to emphasise in the test output that the tests are
covering the gpio-mockup itself. They do perform some implicit testing
of gpiolib and the uAPI interfaces, and so can be useful as smoke tests
for those, but their primary focus is the gpio-mockup.
Patches 2 through 5 do some cleaning up that is now possible with the
new implementation, including enabling building in buildroot environments.
Patch 4 doesn't strictly clean up all the old gpio references that it
could - the gpio was the only Level 1 test, so the Level 1 tests could
potentially be removed, but I was unsure if there may be other
implications to removing a whole test level, or that it may be useful
as a placeholder in case other static LDLIBS tests are added in
the future??
Patch 6 finally gets around to porting the tests to the latest GPIO uAPI.
And Patch 7 updates the config to set the CONFIG_GPIO_CDEV option that
was added in v5.10.
Cheers,
Kent.
Kent Gibson (7):
selftests: gpio: rework and simplify test implementation
selftests: gpio: remove obsolete gpio-mockup-chardev.c
selftests: remove obsolete build restriction for gpio
selftests: remove obsolete gpio references from kselftest_deps.sh
tools: gpio: remove uAPI v1 code no longer used by selftests
selftests: gpio: port to GPIO uAPI v2
selftests: gpio: add CONFIG_GPIO_CDEV to config
tools/gpio/gpio-utils.c | 89 ----
tools/gpio/gpio-utils.h | 6 -
tools/testing/selftests/Makefile | 9 -
tools/testing/selftests/gpio/Makefile | 26 +-
tools/testing/selftests/gpio/config | 1 +
.../testing/selftests/gpio/gpio-mockup-cdev.c | 198 ++++++++
.../selftests/gpio/gpio-mockup-chardev.c | 323 ------------
.../selftests/gpio/gpio-mockup-sysfs.sh | 168 ++-----
tools/testing/selftests/gpio/gpio-mockup.sh | 469 ++++++++++++------
tools/testing/selftests/kselftest_deps.sh | 4 +-
10 files changed, 573 insertions(+), 720 deletions(-)
create mode 100644 tools/testing/selftests/gpio/gpio-mockup-cdev.c
delete mode 100644 tools/testing/selftests/gpio/gpio-mockup-chardev.c
--
2.30.0
When running this xfrm_policy.sh test script, even with some cases
marked as FAIL, the overall test result will still be PASS:
$ sudo ./xfrm_policy.sh
PASS: policy before exception matches
FAIL: expected ping to .254 to fail (exceptions)
PASS: direct policy matches (exceptions)
PASS: policy matches (exceptions)
FAIL: expected ping to .254 to fail (exceptions and block policies)
PASS: direct policy matches (exceptions and block policies)
PASS: policy matches (exceptions and block policies)
FAIL: expected ping to .254 to fail (exceptions and block policies after hresh changes)
PASS: direct policy matches (exceptions and block policies after hresh changes)
PASS: policy matches (exceptions and block policies after hresh changes)
FAIL: expected ping to .254 to fail (exceptions and block policies after hthresh change in ns3)
PASS: direct policy matches (exceptions and block policies after hthresh change in ns3)
PASS: policy matches (exceptions and block policies after hthresh change in ns3)
FAIL: expected ping to .254 to fail (exceptions and block policies after htresh change to normal)
PASS: direct policy matches (exceptions and block policies after htresh change to normal)
PASS: policy matches (exceptions and block policies after htresh change to normal)
PASS: policies with repeated htresh change
$ echo $?
0
This is because the $lret in check_xfrm() is not a local variable.
Therefore when a test failed in check_exceptions(), the non-zero $lret
will later get reset to 0 when the next test calls check_xfrm().
With this fix, the final return value will be 1. Make it easier for
testers to spot this failure.
Fixes: 39aa6928d462d0 ("xfrm: policy: fix netlink/pf_key policy lookups")
Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com>
---
tools/testing/selftests/net/xfrm_policy.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/xfrm_policy.sh b/tools/testing/selftests/net/xfrm_policy.sh
index 7a1bf94..5922941 100755
--- a/tools/testing/selftests/net/xfrm_policy.sh
+++ b/tools/testing/selftests/net/xfrm_policy.sh
@@ -202,7 +202,7 @@ check_xfrm() {
# 1: iptables -m policy rule count != 0
rval=$1
ip=$2
- lret=0
+ local lret=0
ip netns exec ns1 ping -q -c 1 10.0.2.$ip > /dev/null
--
2.7.4
On Sat, Jan 02, 2021 at 03:52:32PM +0200, Andy Shevchenko wrote:
> On Saturday, January 2, 2021, Kent Gibson <warthog618(a)gmail.com> wrote:
>
> > The GPIO mockup selftests are overly complicated with separate
> > implementations of the tests for sysfs and cdev uAPI, and with the cdev
> > implementation being dependent on tools/gpio and libmount.
> >
> > Rework the test implementation to provide a common test suite with a
> > simplified pluggable uAPI interface. The cdev implementation utilises
> > the GPIO uAPI directly to remove the dependence on tools/gpio.
> > The simplified uAPI interface removes the need for any file system mount
> > checks in C, and so removes the dependence on libmount.
> >
> > The rework also fixes the sysfs test implementation which has been broken
> > since the device created in the multiple gpiochip case was split into
> > separate devices.
> >
> >
>
> I briefly looked at code in shell below... there are places to improve
> (useless use of: cat, test, negation, etc).
>
My shell is clearly pretty poor, so I would really appreciate a pointer
to an example of each, and I'll then hunt down the rest.
Thanks,
Kent.
From: Ira Weiny <ira.weiny(a)intel.com>
Changes from V2 [4]
Rebased on tip-tree/core/entry
From Thomas Gleixner
Address bisectability
Drop Patch:
x86/entry: Move nmi entry/exit into common code
From Greg KH
Remove WARN_ON's
From Dan Williams
Add __must_check to pks_key_alloc()
New patch: x86/pks: Add PKS defines and config options
Split from Enable patch to build on through the series
Fix compile errors
Changes from V1
Rebase to TIP master; resolve conflicts and test
Clean up some kernel docs updates missed in V1
Add irqentry_state_t kernel doc for PKRS field
Removed redundant irq_state->pkrs
This is only needed when we add the global state and somehow
ended up in this patch series. That will come back when we add
the global functionality in.
From Thomas Gleixner
Update commit messages
Add kernel doc for struct irqentry_state_t
From Dave Hansen add flags to pks_key_alloc()
Changes from RFC V3[3]
Rebase to TIP master
Update test error output
Standardize on 'irq_state' for state variables
From Dave Hansen
Update commit messages
Add/clean up comments
Add X86_FEATURE_PKS to disabled-features.h and remove some
explicit CONFIG checks
Move saved_pkrs member of thread_struct
Remove superfluous preempt_disable()
s/irq_save_pks/irq_save_set_pks/
Ensure PKRS is not seen in faults if not configured or not
supported
s/pks_mknoaccess/pks_mk_noaccess/
s/pks_mkread/pks_mk_readonly/
s/pks_mkrdwr/pks_mk_readwrite/
Change pks_key_alloc return to -EOPNOTSUPP when not supported
From Peter Zijlstra
Clean up Attribution
Remove superfluous preempt_disable()
Add union to differentiate exit_rcu/lockdep use in
irqentry_state_t
From Thomas Gleixner
Add preliminary clean up patch and adjust series as needed
Introduce a new page protection mechanism for supervisor pages, Protection Key
Supervisor (PKS).
2 use cases for PKS are being developed, trusted keys and PMEM. Trusted keys
is a newer use case which is still being explored. PMEM was submitted as part
of the RFC (v2) series[1]. However, since then it was found that some callers
of kmap() require a global implementation of PKS. Specifically some users of
kmap() expect mappings to be available to all kernel threads. While global use
of PKS is rare it needs to be included for correctness. Unfortunately the
kmap() updates required a large patch series to make the needed changes at the
various kmap() call sites so that patch set has been split out. Because the
global PKS feature is only required for that use case it will be deferred to
that set as well.[2] This patch set is being submitted as a precursor to both
of the use cases.
For an overview of the entire PKS ecosystem, a git tree including this series
and 2 proposed use cases can be found here:
https://lore.kernel.org/lkml/20201009195033.3208459-1-ira.weiny@intel.com/https://lore.kernel.org/lkml/20201009201410.3209180-1-ira.weiny@intel.com/
PKS enables protections on 'domains' of supervisor pages to limit supervisor
mode access to those pages beyond the normal paging protections. PKS works in
a similar fashion to user space pkeys, PKU. As with PKU, supervisor pkeys are
checked in addition to normal paging protections and Access or Writes can be
disabled via a MSR update without TLB flushes when permissions change. Also
like PKU, a page mapping is assigned to a domain by setting pkey bits in the
page table entry for that mapping.
Access is controlled through a PKRS register which is updated via WRMSR/RDMSR.
XSAVE is not supported for the PKRS MSR. Therefore the implementation
saves/restores the MSR across context switches and during exceptions. Nested
exceptions are supported by each exception getting a new PKS state.
For consistent behavior with current paging protections, pkey 0 is reserved and
configured to allow full access via the pkey mechanism, thus preserving the
default paging protections on mappings with the default pkey value of 0.
Other keys, (1-15) are allocated by an allocator which prepares us for key
contention from day one. Kernel users should be prepared for the allocator to
fail either because of key exhaustion or due to PKS not being supported on the
arch and/or CPU instance.
The following are key attributes of PKS.
1) Fast switching of permissions
1a) Prevents access without page table manipulations
1b) No TLB flushes required
2) Works on a per thread basis
PKS is available with 4 and 5 level paging. Like PKRU it consumes 4 bits from
the PTE to store the pkey within the entry.
[1] https://lore.kernel.org/lkml/20200717072056.73134-1-ira.weiny@intel.com/
[2] https://lore.kernel.org/lkml/20201009195033.3208459-2-ira.weiny@intel.com/
[3] https://lore.kernel.org/lkml/20201009194258.3207172-1-ira.weiny@intel.com/
[4] https://lore.kernel.org/lkml/20201102205320.1458656-1-ira.weiny@intel.com/
Fenghua Yu (2):
x86/pks: Add PKS kernel API
x86/pks: Enable Protection Keys Supervisor (PKS)
Ira Weiny (8):
x86/pkeys: Create pkeys_common.h
x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
x86/pks: Add PKS defines and Kconfig options
x86/pks: Preserve the PKRS MSR on context switch
x86/entry: Pass irqentry_state_t by reference
x86/entry: Preserve PKRS MSR across exceptions
x86/fault: Report the PKRS state on fault
x86/pks: Add PKS test code
Documentation/core-api/protection-keys.rst | 103 ++-
arch/x86/Kconfig | 1 +
arch/x86/entry/common.c | 46 +-
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/include/asm/disabled-features.h | 8 +-
arch/x86/include/asm/idtentry.h | 25 +-
arch/x86/include/asm/msr-index.h | 1 +
arch/x86/include/asm/pgtable.h | 13 +-
arch/x86/include/asm/pgtable_types.h | 12 +
arch/x86/include/asm/pkeys.h | 15 +
arch/x86/include/asm/pkeys_common.h | 40 ++
arch/x86/include/asm/processor.h | 18 +-
arch/x86/include/uapi/asm/processor-flags.h | 2 +
arch/x86/kernel/cpu/common.c | 15 +
arch/x86/kernel/cpu/mce/core.c | 4 +-
arch/x86/kernel/fpu/xstate.c | 22 +-
arch/x86/kernel/kvm.c | 6 +-
arch/x86/kernel/nmi.c | 4 +-
arch/x86/kernel/process.c | 26 +
arch/x86/kernel/traps.c | 21 +-
arch/x86/mm/fault.c | 87 ++-
arch/x86/mm/pkeys.c | 196 +++++-
include/linux/entry-common.h | 31 +-
include/linux/pgtable.h | 4 +
include/linux/pkeys.h | 24 +
kernel/entry/common.c | 44 +-
lib/Kconfig.debug | 12 +
lib/Makefile | 3 +
lib/pks/Makefile | 3 +
lib/pks/pks_test.c | 692 ++++++++++++++++++++
mm/Kconfig | 2 +
tools/testing/selftests/x86/Makefile | 3 +-
tools/testing/selftests/x86/test_pks.c | 66 ++
33 files changed, 1410 insertions(+), 140 deletions(-)
create mode 100644 arch/x86/include/asm/pkeys_common.h
create mode 100644 lib/pks/Makefile
create mode 100644 lib/pks/pks_test.c
create mode 100644 tools/testing/selftests/x86/test_pks.c
--
2.28.0.rc0.12.gb6a658bd00c9
This patch set contains dm-user, a device mapper target that proxies incoming
BIOs to userspace via a misc device. Essentially it's FUSE, but for block
devices. There's more information in the documentation patch and as a handful
of commends, so I'm just going to avoid duplicating that here. I don't really
think there's any fundamental functionality that dm-user enables, as one could
use something along the lines of nbd/iscsi, but dm-user does result in
extremely simple userspace daemons -- so simple that when I tried to write a
helper userspace library for dm-user I just ended up with nothing.
I talked about this a bit at Plumbers and was hoping to send patches a bit
earlier on in the process, but got tied up with a few things. As a result this
is actually quite far along: it's at the point where we're starting to run this
on real devices as part of an updated Android OTA update flow, where we're
using this to provide an Android-specific compressed backing store for
dm-snap-persistent. The bulk of that project is scattered throughout the
various Android trees, so there are kselftests and a (somewhat bare for now)
Documentation entry with the intent of making this a self-contained
contribution. There's a lot to the Android userspace daemon, but it doesn't
interact with dm-user in a very complex manner.
This is still in a somewhat early stage, but it's at the point where things
largely function. I'm certainly not ready to commit to the user ABI
implemented here and there are a bunch of FIXMEs scattered throughout the code,
but I do think that it's far along enough to begin a more concrete discussion
of where folks would like to go with something like this. While I'd intending
on sorting that stuff out, I'd like to at least get a feel for whether this is
a path worth pursuing before spending a bunch more time on it.
I haven't done much in the way of performance analysis for dm-user. Earlier on
I did some simple throughput tests and found that dm-user/ext4 was faster than
half the speed of tmpfs, which is way out of the realm of being an issue for
our use case (decompressing blocks out of a phone's storage). The design of
dm-user does preclude an extremely high performance implementation, where I
assume one would want an explicit ring buffer and zero copy, but I feel like
users who want that degree of performance are probably better served writing a
proper kernel driver. I wouldn't be opposed to pushing on performance (ideally
without a major design change), but for now I feel like time is better spent
fortifying the user ABI and fixing the various issues with the implementation.
The patches follow as usual, but in case it's easier I've published a tree as
well:
git://git.kernel.org/pub/scm/linux/kernel/git/palmer/dm-user.git -b dm-user-v1
From: Andy Lutomirski <luto(a)kernel.org>
[ Upstream commit 716572b0003ef67a4889bd7d85baf5099c5a0248 ]
Setting GS to 1, 2, or 3 causes a nonsensical part of the IRET microcode
to change GS back to zero on a return from kernel mode to user mode. The
result is that these tests fail randomly depending on when interrupts
happen. Detect when this happens and let the test pass.
Signed-off-by: Andy Lutomirski <luto(a)kernel.org>
Signed-off-by: Borislav Petkov <bp(a)suse.de>
Link: https://lkml.kernel.org/r/7567fd44a1d60a9424f25b19a998f12149993b0d.16043465…
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/x86/fsgsbase.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/x86/fsgsbase.c b/tools/testing/selftests/x86/fsgsbase.c
index f249e042b3b51..026cd644360f6 100644
--- a/tools/testing/selftests/x86/fsgsbase.c
+++ b/tools/testing/selftests/x86/fsgsbase.c
@@ -318,8 +318,8 @@ static void set_gs_and_switch_to(unsigned long local,
local = read_base(GS);
/*
- * Signal delivery seems to mess up weird selectors. Put it
- * back.
+ * Signal delivery is quite likely to change a selector
+ * of 1, 2, or 3 back to 0 due to IRET being defective.
*/
asm volatile ("mov %0, %%gs" : : "rm" (force_sel));
} else {
@@ -337,6 +337,14 @@ static void set_gs_and_switch_to(unsigned long local,
if (base == local && sel_pre_sched == sel_post_sched) {
printf("[OK]\tGS/BASE remained 0x%hx/0x%lx\n",
sel_pre_sched, local);
+ } else if (base == local && sel_pre_sched >= 1 && sel_pre_sched <= 3 &&
+ sel_post_sched == 0) {
+ /*
+ * IRET is misdesigned and will squash selectors 1, 2, or 3
+ * to zero. Don't fail the test just because this happened.
+ */
+ printf("[OK]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx because IRET is defective\n",
+ sel_pre_sched, local, sel_post_sched, base);
} else {
nerrs++;
printf("[FAIL]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx\n",
--
2.27.0
From: Andy Lutomirski <luto(a)kernel.org>
[ Upstream commit 716572b0003ef67a4889bd7d85baf5099c5a0248 ]
Setting GS to 1, 2, or 3 causes a nonsensical part of the IRET microcode
to change GS back to zero on a return from kernel mode to user mode. The
result is that these tests fail randomly depending on when interrupts
happen. Detect when this happens and let the test pass.
Signed-off-by: Andy Lutomirski <luto(a)kernel.org>
Signed-off-by: Borislav Petkov <bp(a)suse.de>
Link: https://lkml.kernel.org/r/7567fd44a1d60a9424f25b19a998f12149993b0d.16043465…
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/x86/fsgsbase.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/x86/fsgsbase.c b/tools/testing/selftests/x86/fsgsbase.c
index f249e042b3b51..026cd644360f6 100644
--- a/tools/testing/selftests/x86/fsgsbase.c
+++ b/tools/testing/selftests/x86/fsgsbase.c
@@ -318,8 +318,8 @@ static void set_gs_and_switch_to(unsigned long local,
local = read_base(GS);
/*
- * Signal delivery seems to mess up weird selectors. Put it
- * back.
+ * Signal delivery is quite likely to change a selector
+ * of 1, 2, or 3 back to 0 due to IRET being defective.
*/
asm volatile ("mov %0, %%gs" : : "rm" (force_sel));
} else {
@@ -337,6 +337,14 @@ static void set_gs_and_switch_to(unsigned long local,
if (base == local && sel_pre_sched == sel_post_sched) {
printf("[OK]\tGS/BASE remained 0x%hx/0x%lx\n",
sel_pre_sched, local);
+ } else if (base == local && sel_pre_sched >= 1 && sel_pre_sched <= 3 &&
+ sel_post_sched == 0) {
+ /*
+ * IRET is misdesigned and will squash selectors 1, 2, or 3
+ * to zero. Don't fail the test just because this happened.
+ */
+ printf("[OK]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx because IRET is defective\n",
+ sel_pre_sched, local, sel_post_sched, base);
} else {
nerrs++;
printf("[FAIL]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx\n",
--
2.27.0
From: Andy Lutomirski <luto(a)kernel.org>
[ Upstream commit 716572b0003ef67a4889bd7d85baf5099c5a0248 ]
Setting GS to 1, 2, or 3 causes a nonsensical part of the IRET microcode
to change GS back to zero on a return from kernel mode to user mode. The
result is that these tests fail randomly depending on when interrupts
happen. Detect when this happens and let the test pass.
Signed-off-by: Andy Lutomirski <luto(a)kernel.org>
Signed-off-by: Borislav Petkov <bp(a)suse.de>
Link: https://lkml.kernel.org/r/7567fd44a1d60a9424f25b19a998f12149993b0d.16043465…
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/x86/fsgsbase.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/x86/fsgsbase.c b/tools/testing/selftests/x86/fsgsbase.c
index f249e042b3b51..026cd644360f6 100644
--- a/tools/testing/selftests/x86/fsgsbase.c
+++ b/tools/testing/selftests/x86/fsgsbase.c
@@ -318,8 +318,8 @@ static void set_gs_and_switch_to(unsigned long local,
local = read_base(GS);
/*
- * Signal delivery seems to mess up weird selectors. Put it
- * back.
+ * Signal delivery is quite likely to change a selector
+ * of 1, 2, or 3 back to 0 due to IRET being defective.
*/
asm volatile ("mov %0, %%gs" : : "rm" (force_sel));
} else {
@@ -337,6 +337,14 @@ static void set_gs_and_switch_to(unsigned long local,
if (base == local && sel_pre_sched == sel_post_sched) {
printf("[OK]\tGS/BASE remained 0x%hx/0x%lx\n",
sel_pre_sched, local);
+ } else if (base == local && sel_pre_sched >= 1 && sel_pre_sched <= 3 &&
+ sel_post_sched == 0) {
+ /*
+ * IRET is misdesigned and will squash selectors 1, 2, or 3
+ * to zero. Don't fail the test just because this happened.
+ */
+ printf("[OK]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx because IRET is defective\n",
+ sel_pre_sched, local, sel_post_sched, base);
} else {
nerrs++;
printf("[FAIL]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx\n",
--
2.27.0
From: Andy Lutomirski <luto(a)kernel.org>
[ Upstream commit 716572b0003ef67a4889bd7d85baf5099c5a0248 ]
Setting GS to 1, 2, or 3 causes a nonsensical part of the IRET microcode
to change GS back to zero on a return from kernel mode to user mode. The
result is that these tests fail randomly depending on when interrupts
happen. Detect when this happens and let the test pass.
Signed-off-by: Andy Lutomirski <luto(a)kernel.org>
Signed-off-by: Borislav Petkov <bp(a)suse.de>
Link: https://lkml.kernel.org/r/7567fd44a1d60a9424f25b19a998f12149993b0d.16043465…
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/x86/fsgsbase.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/x86/fsgsbase.c b/tools/testing/selftests/x86/fsgsbase.c
index 757bdb218a661..f2916838a7eb5 100644
--- a/tools/testing/selftests/x86/fsgsbase.c
+++ b/tools/testing/selftests/x86/fsgsbase.c
@@ -391,8 +391,8 @@ static void set_gs_and_switch_to(unsigned long local,
local = read_base(GS);
/*
- * Signal delivery seems to mess up weird selectors. Put it
- * back.
+ * Signal delivery is quite likely to change a selector
+ * of 1, 2, or 3 back to 0 due to IRET being defective.
*/
asm volatile ("mov %0, %%gs" : : "rm" (force_sel));
} else {
@@ -410,6 +410,14 @@ static void set_gs_and_switch_to(unsigned long local,
if (base == local && sel_pre_sched == sel_post_sched) {
printf("[OK]\tGS/BASE remained 0x%hx/0x%lx\n",
sel_pre_sched, local);
+ } else if (base == local && sel_pre_sched >= 1 && sel_pre_sched <= 3 &&
+ sel_post_sched == 0) {
+ /*
+ * IRET is misdesigned and will squash selectors 1, 2, or 3
+ * to zero. Don't fail the test just because this happened.
+ */
+ printf("[OK]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx because IRET is defective\n",
+ sel_pre_sched, local, sel_post_sched, base);
} else {
nerrs++;
printf("[FAIL]\tGS/BASE changed from 0x%hx/0x%lx to 0x%hx/0x%lx\n",
--
2.27.0
From: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
[ Upstream commit 77ce220c0549dcc3db8226c61c60e83fc59dfafc ]
The test fails because of a recent fix to the verifier, even though this
program is valid. In details what happens is:
7: (61) r1 = *(u32 *)(r0 +0)
Load a 32-bit value, with signed bounds [S32_MIN, S32_MAX]. The bounds
of the 64-bit value are [0, U32_MAX]...
8: (65) if r1 s> 0xffffffff goto pc+1
... therefore this is always true (the operand is sign-extended).
10: (b4) w2 = 11
11: (6d) if r2 s> r1 goto pc+1
When true, the 64-bit bounds become [0, 10]. The 32-bit bounds are still
[S32_MIN, 10].
13: (64) w1 <<= 2
Because this is a 32-bit operation, the verifier propagates the new
32-bit bounds to the 64-bit ones, and the knowledge gained from insn 11
is lost.
14: (0f) r0 += r1
15: (7a) *(u64 *)(r0 +0) = 4
Then the verifier considers r0 unbounded here, rejecting the test. To
make the test work, change insn 8 to check the sign of the 32-bit value.
Signed-off-by: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
Acked-by: John Fastabend <john.fastabend(a)gmail.com>
Signed-off-by: Alexei Starovoitov <ast(a)kernel.org>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/bpf/verifier/array_access.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/verifier/array_access.c b/tools/testing/selftests/bpf/verifier/array_access.c
index f3c33e128709b..a80d806ead15f 100644
--- a/tools/testing/selftests/bpf/verifier/array_access.c
+++ b/tools/testing/selftests/bpf/verifier/array_access.c
@@ -68,7 +68,7 @@
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 9),
BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
- BPF_JMP_IMM(BPF_JSGT, BPF_REG_1, 0xffffffff, 1),
+ BPF_JMP32_IMM(BPF_JSGT, BPF_REG_1, 0xffffffff, 1),
BPF_MOV32_IMM(BPF_REG_1, 0),
BPF_MOV32_IMM(BPF_REG_2, MAX_ENTRIES),
BPF_JMP_REG(BPF_JSGT, BPF_REG_2, BPF_REG_1, 1),
--
2.27.0
On Sat, Dec 12, 2020 at 5:18 PM Guenter Roeck <linux(a)roeck-us.net> wrote:
>
> This patch results in:
>
> arch/sh/kernel/cpu/sh4a/smp-shx3.c: In function 'shx3_prepare_cpus':
> arch/sh/kernel/cpu/sh4a/smp-shx3.c:76:3: error: ignoring return value of 'request_irq' declared with attribute 'warn_unused_result'
>
> when building sh:defconfig. Checking for calls to request_irq()
> suggests that there will be other similar errors in various builds.
> Reverting the patch fixes the problem.
Which ones? From a quick grep and some filtering I could only find one
file with wrong usage apart from this one:
drivers/net/ethernet/lantiq_etop.c:
request_irq(irq, ltq_etop_dma_irq, 0, "etop_tx", priv);
drivers/net/ethernet/lantiq_etop.c:
request_irq(irq, ltq_etop_dma_irq, 0, "etop_rx", priv);
Of course, this does not cover other functions, but it means there
aren't many issues and/or people building the code if nobody complains
within a few weeks. So I think we can fix them as they come.
Cheers,
Miguel
From: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
[ Upstream commit 3615bdf6d9b19db12b1589861609b4f1c6a8d303 ]
The verifier trace changed following a bugfix. After checking the 64-bit
sign, only the upper bit mask is known, not bit 31. Update the test
accordingly.
Signed-off-by: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
Acked-by: John Fastabend <john.fastabend(a)gmail.com>
Signed-off-by: Alexei Starovoitov <ast(a)kernel.org>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/bpf/prog_tests/align.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/align.c b/tools/testing/selftests/bpf/prog_tests/align.c
index c548aded65859..2a15aa3d49c74 100644
--- a/tools/testing/selftests/bpf/prog_tests/align.c
+++ b/tools/testing/selftests/bpf/prog_tests/align.c
@@ -456,10 +456,10 @@ static struct bpf_align_test tests[] = {
*/
{7, "R5_w=inv(id=0,smin_value=-9223372036854775806,smax_value=9223372036854775806,umin_value=2,umax_value=18446744073709551614,var_off=(0x2; 0xfffffffffffffffc)"},
/* Checked s>=0 */
- {9, "R5=inv(id=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2; 0x7fffffff7ffffffc)"},
+ {9, "R5=inv(id=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
/* packet pointer + nonnegative (4n+2) */
- {11, "R6_w=pkt(id=1,off=0,r=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2; 0x7fffffff7ffffffc)"},
- {13, "R4_w=pkt(id=1,off=4,r=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2; 0x7fffffff7ffffffc)"},
+ {11, "R6_w=pkt(id=1,off=0,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
+ {13, "R4_w=pkt(id=1,off=4,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
/* NET_IP_ALIGN + (4n+2) == (4n), alignment is fine.
* We checked the bounds, but it might have been able
* to overflow if the packet pointer started in the
@@ -467,7 +467,7 @@ static struct bpf_align_test tests[] = {
* So we did not get a 'range' on R6, and the access
* attempt will fail.
*/
- {15, "R6_w=pkt(id=1,off=0,r=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2; 0x7fffffff7ffffffc)"},
+ {15, "R6_w=pkt(id=1,off=0,r=0,umin_value=2,umax_value=9223372036854775806,var_off=(0x2; 0x7ffffffffffffffc)"},
}
},
{
--
2.27.0
From: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
[ Upstream commit 77ce220c0549dcc3db8226c61c60e83fc59dfafc ]
The test fails because of a recent fix to the verifier, even though this
program is valid. In details what happens is:
7: (61) r1 = *(u32 *)(r0 +0)
Load a 32-bit value, with signed bounds [S32_MIN, S32_MAX]. The bounds
of the 64-bit value are [0, U32_MAX]...
8: (65) if r1 s> 0xffffffff goto pc+1
... therefore this is always true (the operand is sign-extended).
10: (b4) w2 = 11
11: (6d) if r2 s> r1 goto pc+1
When true, the 64-bit bounds become [0, 10]. The 32-bit bounds are still
[S32_MIN, 10].
13: (64) w1 <<= 2
Because this is a 32-bit operation, the verifier propagates the new
32-bit bounds to the 64-bit ones, and the knowledge gained from insn 11
is lost.
14: (0f) r0 += r1
15: (7a) *(u64 *)(r0 +0) = 4
Then the verifier considers r0 unbounded here, rejecting the test. To
make the test work, change insn 8 to check the sign of the 32-bit value.
Signed-off-by: Jean-Philippe Brucker <jean-philippe(a)linaro.org>
Acked-by: John Fastabend <john.fastabend(a)gmail.com>
Signed-off-by: Alexei Starovoitov <ast(a)kernel.org>
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
tools/testing/selftests/bpf/verifier/array_access.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/verifier/array_access.c b/tools/testing/selftests/bpf/verifier/array_access.c
index 1c4b1939f5a8d..bed53b561e044 100644
--- a/tools/testing/selftests/bpf/verifier/array_access.c
+++ b/tools/testing/selftests/bpf/verifier/array_access.c
@@ -68,7 +68,7 @@
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 9),
BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
- BPF_JMP_IMM(BPF_JSGT, BPF_REG_1, 0xffffffff, 1),
+ BPF_JMP32_IMM(BPF_JSGT, BPF_REG_1, 0xffffffff, 1),
BPF_MOV32_IMM(BPF_REG_1, 0),
BPF_MOV32_IMM(BPF_REG_2, MAX_ENTRIES),
BPF_JMP_REG(BPF_JSGT, BPF_REG_2, BPF_REG_1, 1),
--
2.27.0
The BPF selftests have build time dependencies on cutting edge versions
of tools in the BPF ecosystem including LLVM which are more involved
to satisfy than more typical requirements like installing a package from
your distribution. This causes issues for users looking at kselftest in
as a whole who find that a default build of kselftest fails and that
resolving this is time consuming and adds administrative overhead. The
fast pace of BPF development and the need for a full BPF stack to do
substantial development or validation work on the code mean that people
working directly on it don't see a reasonable way to keep supporting
older environments without causing problems with the usability of the
BPF tests in BPF development so these requirements are unlikely to be
relaxed in the immediate future.
There is already support for skipping targets so in order to reduce the
barrier to entry for people interested in kselftest as a whole let's use
that to skip the BPF tests by default when people work with the top
level kselftest build system. Users can still build the BPF selftests
as part of the wider kselftest build by specifying SKIP_TARGETS,
including setting an empty SKIP_TARGETS to build everything. They can
also continue to build the BPF selftests individually in cases where
they are specifically focused on BPF.
This isn't ideal since it means people will need to take special steps
to build the BPF tests but the dependencies mean that realistically this
is already the case to some extent and it makes it easier for people to
pick up and work with the other selftests which is hopefully a net win.
Signed-off-by: Mark Brown <broonie(a)kernel.org>
---
tools/testing/selftests/Makefile | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index afbab4aeef3c..8a917cb4426a 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -77,8 +77,10 @@ TARGETS += zram
TARGETS_HOTPLUG = cpu-hotplug
TARGETS_HOTPLUG += memory-hotplug
-# User can optionally provide a TARGETS skiplist.
-SKIP_TARGETS ?=
+# User can optionally provide a TARGETS skiplist. By default we skip
+# BPF since it has cutting edge build time dependencies which require
+# more effort to install.
+SKIP_TARGETS ?= bpf
ifneq ($(SKIP_TARGETS),)
TMP := $(filter-out $(SKIP_TARGETS), $(TARGETS))
override TARGETS := $(TMP)
--
2.20.1
Fix the following -Wformat warnings in vdso_test_correctness.c:
vdso_test_correctness.c: In function ‘test_one_clock_gettime64’:
vdso_test_correctness.c:352:21: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘long long int’ [-Wformat=]
352 | printf("\t%llu.%09ld %llu.%09ld %llu.%09ld\n",
| ~~~~^
| |
| long int
| %09lld
353 | (unsigned long long)start.tv_sec, start.tv_nsec,
| ~~~~~~~~~~~~~
| |
| long long int
vdso_test_correctness.c:352:32: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 5 has type ‘long long int’ [-Wformat=]
352 | printf("\t%llu.%09ld %llu.%09ld %llu.%09ld\n",
| ~~~~^
| |
| long int
| %09lld
353 | (unsigned long long)start.tv_sec, start.tv_nsec,
354 | (unsigned long long)vdso.tv_sec, vdso.tv_nsec,
| ~~~~~~~~~~~~
| |
| long long int
vdso_test_correctness.c:352:43: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 7 has type ‘long long int’ [-Wformat=]
The tv_sec member of __kernel_timespec is long long, both in
uapi/linux/time_types.h and locally in vdso_test_correctness.c.
Signed-off-by: Tobias Klauser <tklauser(a)distanz.ch>
---
tools/testing/selftests/vDSO/vdso_test_correctness.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/vDSO/vdso_test_correctness.c b/tools/testing/selftests/vDSO/vdso_test_correctness.c
index 5029ef9b228c..c4aea794725a 100644
--- a/tools/testing/selftests/vDSO/vdso_test_correctness.c
+++ b/tools/testing/selftests/vDSO/vdso_test_correctness.c
@@ -349,7 +349,7 @@ static void test_one_clock_gettime64(int clock, const char *name)
return;
}
- printf("\t%llu.%09ld %llu.%09ld %llu.%09ld\n",
+ printf("\t%llu.%09lld %llu.%09lld %llu.%09lld\n",
(unsigned long long)start.tv_sec, start.tv_nsec,
(unsigned long long)vdso.tv_sec, vdso.tv_nsec,
(unsigned long long)end.tv_sec, end.tv_nsec);
--
2.29.0
This is a repost of the mremap speed up patches, adding Kirill's
Acked-by's (from a separate discussion). The previous versions are
posted at:
v1 - https://lore.kernel.org/r/20200930222130.4175584-1-kaleshsingh@google.com
v2 - https://lore.kernel.org/r/20201002162101.665549-1-kaleshsingh@google.com
v3 - http://lore.kernel.org/r/20201005154017.474722-1-kaleshsingh@google.com
mremap time can be optimized by moving entries at the PMD/PUD level if
the source and destination addresses are PMD/PUD-aligned and
PMD/PUD-sized. Enable moving at the PMD and PUD levels on arm64 and
x86. Other architectures where this type of move is supported and known to
be safe can also opt-in to these optimizations by enabling HAVE_MOVE_PMD
and HAVE_MOVE_PUD.
Observed Performance Improvements for remapping a PUD-aligned 1GB-sized
region on x86 and arm64:
- HAVE_MOVE_PMD is already enabled on x86 : N/A
- Enabling HAVE_MOVE_PUD on x86 : ~13x speed up
- Enabling HAVE_MOVE_PMD on arm64 : ~ 8x speed up
- Enabling HAVE_MOVE_PUD on arm64 : ~19x speed up
Altogether, HAVE_MOVE_PMD and HAVE_MOVE_PUD
give a total of ~150x speed up on arm64.
Changes in v2:
- Reduce mremap_test time by only validating a configurable
threshold of the remapped region, as per John.
- Use a random pattern for mremap validation. Provide pattern
seed in test output, as per John.
- Moved set_pud_at() to separate patch, per Kirill.
- Use switch() instead of ifs in move_pgt_entry(), per Kirill.
- Update commit message with description of Android
garbage collector use case for HAVE_MOVE_PUD, as per Joel.
- Fix build test error reported by kernel test robot in [1].
Changes in v3:
- Make lines 80 cols or less where they don’t need to be longer,
per John.
- Removed unused PATTERN_SIZE in mremap_test
- Added Reviewed-by tag for patch 1/5 (mremap kselftest patch).
- Use switch() instead of ifs in get_extent(), per Kirill
- Add BUILD_BUG() is get_extent() default case.
- Move get_old_pud() and alloc_new_pud() out of
#ifdef CONFIG_HAVE_MOVE_PUD, per Kirill.
- Have get_old_pmd() and alloc_new_pmd() use get_old_pud() and
alloc_old_pud(), per Kirill.
- Replace #ifdef CONFIG_HAVE_MOVE_PMD / PUD in move_page_tables()
with IS_ENABLED(CONFIG_HAVE_MOVE_PMD / PUD), per Kirill.
- Fold Add set_pud_at() patch into patch 4/5, per Kirill.
[1] https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org/thread/CKPGL4F…
Kalesh Singh (5):
kselftests: vm: Add mremap tests
arm64: mremap speedup - Enable HAVE_MOVE_PMD
mm: Speedup mremap on 1GB or larger regions
arm64: mremap speedup - Enable HAVE_MOVE_PUD
x86: mremap speedup - Enable HAVE_MOVE_PUD
arch/Kconfig | 7 +
arch/arm64/Kconfig | 2 +
arch/arm64/include/asm/pgtable.h | 1 +
arch/x86/Kconfig | 1 +
mm/mremap.c | 230 ++++++++++++---
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 1 +
tools/testing/selftests/vm/mremap_test.c | 344 +++++++++++++++++++++++
tools/testing/selftests/vm/run_vmtests | 11 +
9 files changed, 558 insertions(+), 40 deletions(-)
create mode 100644 tools/testing/selftests/vm/mremap_test.c
--
2.28.0.1011.ga647a8990f-goog
From: Lai Jiangshan <laijs(a)linux.alibaba.com>
In my box, all CPUs are allowed to be offline. The test tries to offline
all offline-able CPUs and causes fail on the last one. We should just
skip offlining the last CPU
Signed-off-by: Lai Jiangshan <laijs(a)linux.alibaba.com>
---
tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh b/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
index 0d26b5e3f966..5cdef96326a7 100755
--- a/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
+++ b/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
@@ -126,6 +126,11 @@ offline_cpu_expect_success()
{
local cpu=$1
+ # don't offline the last CPU if all CPUs are offline-able
+ if [[ a$cpu = a`cat $SYSFS/devices/system/cpu/online` ]]; then
+ return
+ fi
+
if ! offline_cpu $cpu; then
echo $FUNCNAME $cpu: unexpected fail >&2
exit 1
--
2.19.1.6.gb485710b
On Tue, 15 Dec 2020 22:17:06 PST (-0800), ruby.wktk(a)gmail.com wrote:
> Hi my name is Akira Hayakawa. I am maintaining an out-of-tree DM target
> named dm-writeboost.
>
> Sorry to step in. But this is a very interesting topic at least to me.
>
> I have been looking for something like dm-user because I believe we should
> be able to implement virtual block devices in Rust language.
>
> I know proxying IO requests to userland always causes some overhead but for
> some type of device that performance doesn't matter or some research
> prototyping or pseudo device for testing, this way should be developed. Of
> course, implementation in Rust will give us opportunities to develop more
> complicated software in high quality.
>
> I noticed this thread few days ago then I started to prototype this library
> https://github.com/akiradeveloper/userland-io
>
> It is what I want but the transport is still NBD which I don't like so
> much. If dm-user is available, I will implement a transport using dm-user.
Great, I'm glad to hear that. Obviously this is still in the early days and
we're talking about high-level ABI design here, so things are almost certainly
going to change, but it's always good to have people pushing on stuff.
Just be warned: we've only had two people write userspaces for this (one of
which was me, and all that is test code) so I'd be shocked if you manage to
avoid running into bugs.
>
> - Akira
>
> On Tue, Dec 15, 2020 at 7:00 PM Palmer Dabbelt <palmer(a)dabbelt.com> wrote:
>
>> On Thu, 10 Dec 2020 09:03:21 PST (-0800), josef(a)toxicpanda.com wrote:
>> > On 12/9/20 10:38 PM, Bart Van Assche wrote:
>> >> On 12/7/20 10:55 AM, Palmer Dabbelt wrote:
>> >>> All in all, I've found it a bit hard to figure out what sort of
>> interest
>> >>> people
>> >>> have in dm-user: when I bring this up I seem to run into people who've
>> done
>> >>> similar things before and are vaguely interested, but certainly nobody
>> is
>> >>> chomping at the bit. I'm sending it out in this early state to try and
>> >>> figure
>> >>> out if it's interesting enough to keep going.
>> >>
>> >> Cc-ing Josef and Mike since their nbd contributions make me wonder
>> >> whether this new driver could be useful to their use cases?
>> >>
>> >
>> > Sorry gmail+imap sucks and I can't get my email client to get at the
>> original
>> > thread. However here is my take.
>>
>> and I guess I then have to apoligize for missing your email ;). Hopefully
>> that
>> was the problem, but who knows.
>>
>> > 1) The advantages of using dm-user of NBD that you listed aren't actually
>> > problems for NBD. We have NBD working in production where you can hand
>> off the
>> > sockets for the server without ending in timeouts, it was actually the
>> main
>> > reason we wrote our own server so we could use the FD transfer stuff to
>> restart
>> > the server without impacting any clients that had the device in use.
>>
>> OK. So you just send the FD around using one of the standard mechanisms to
>> orchestrate the handoff? I guess that might work for our use case,
>> assuming
>> whatever the security side of things was doing was OK with the old FD.
>> TBH I'm
>> not sure how all that works and while we thought about doing that sort of
>> transfer scheme we decided to just open it again -- not sure how far we
>> were
>> down the dm-user rabbit hole at that point, though, as this sort of arose
>> out
>> of some other ideas.
>>
>> > 2) The extra copy is a big deal, in fact we already have too many copies
>> in our
>> > existing NBD setup and are actively looking for ways to avoid those.
>> >
>> > Don't take this as I don't think dm-user is a good idea, but I think at
>> the very
>> > least it should start with the very best we have to offer, starting with
>> as few
>> > copies as possible.
>>
>> I was really experting someone to say that. It does seem kind of silly to
>> build
>> out the new interface, but not go all the way to a ring buffer. We just
>> didn't
>> really have any way to justify the extra complexity as our use cases aren't
>> that high performance. I kind of like to have benchmarks for this sort of
>> thing, though, and I didn't have anyone who had bothered avoiding the last
>> copy
>> to compare against.
>>
>> > If you are using it currently in production then cool, there's clearly a
>> usecase
>> > for it. Personally as I get older and grouchier I want less things in
>> the
>> > kernel, so if this enables us to eventually do everything NBD related in
>> > userspace with no performance drop then I'd be down. I don't think you
>> need to
>> > make that your primary goal, but at least polishing this up so it could
>> > potentially be abused in the future would make it more compelling for
>> merging.
>> > Thanks,
>>
>> Ya, it's in Android already and we'll be shipping it as part of the new OTA
>> flow for the next release. The rules on deprecation are a bit different
>> over
>> there, though, so it's not like we're wed to it. The whole point of
>> bringing
>> this up here was to try and get something usable by everyone, and while I'd
>> eventually like to get whatever's in Android into the kernel proper we'd
>> really
>> planned on supporting an extra Android-only ABI for a cycle at least.
>>
>> I'm kind of inclined to take a crack at the extra copy, to at least see if
>> building something that eliminates it is viable. I'm not really sure if
>> it is
>> (or at least, if it'll net us a meaningful amount of performance), but
>> it'd at
>> least be interesting to try.
>>
>> It'd be nice to have some benchmark target, though, as otherwise this stuff
>> hangs on forever. My workloads are in selftests later on in the patch
>> set, but
>> I'm essentially using tmpfs as a baseline to compare against ext4+dm-user
>> with
>> some FIO examples as workloads. Our early benchmark numbers indicated
>> this was
>> way faster than we needed, so I didn't even bother putting together a
>> proper
>> system to run on so I don't really have any meaningful numbers there. Is
>> there
>> an NBD server that's fast that I should be comparing against?
>>
>> I haven't gotten a whole lot of feedback, so I'm inclined to at least have
>> some
>> reasonable performance numbers before bothering with a v2.
>>
>> --
>> dm-devel mailing list
>> dm-devel(a)redhat.com
>> https://www.redhat.com/mailman/listinfo/dm-devel
If Makefile cannot find any of the vmlinux's in its VMLINUX_BTF_PATHS list,
it tries to run btftool incorrectly, with VMLINUX_BTF unset:
bpftool btf dump file $(VMLINUX_BTF) format c
Such that the keyword 'format' is misinterpreted as the path to vmlinux.
The resulting build error message is fairly cryptic:
GEN vmlinux.h
Error: failed to load BTF from format: No such file or directory
This patch makes the failure reason clearer by yielding this instead:
Makefile:...: *** cannot find a vmlinux for VMLINUX_BTF at any of
"{paths}". Stop.
Fixes: acbd06206bbb ("selftests/bpf: Add vmlinux.h selftest exercising tracing of syscalls")
Cc: stable(a)vger.kernel.org # 5.7+
Signed-off-by: Kamal Mostafa <kamal(a)canonical.com>
---
tools/testing/selftests/bpf/Makefile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 542768f5195b..93ed34ef6e3f 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -196,6 +196,9 @@ $(BUILD_DIR)/libbpf $(BUILD_DIR)/bpftool $(BUILD_DIR)/resolve_btfids $(INCLUDE_D
$(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) | $(BPFTOOL) $(INCLUDE_DIR)
ifeq ($(VMLINUX_H),)
$(call msg,GEN,,$@)
+ifeq ($(VMLINUX_BTF),)
+$(error cannot find a vmlinux for VMLINUX_BTF at any of "$(VMLINUX_BTF_PATHS)")
+endif
$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@
else
$(call msg,CP,,$@)
--
2.17.1
The existing code attempted to handle numbers by doing a strto[u]l(),
ignoring the field width, and then bitshifting the field out of the
converted value. If the string contains a run of valid digits longer
than will fit in a long or long long, this would overflow and no amount
of bitshifting can recover the correct value.
This patch fixes vsscanf to obey number field widths when parsing
the number.
A new _parse_integer_limit() is added that takes a limit for the number
of characters to parse. The number field conversion in vsscanf is changed
to use this new function.
The cases of a base prefix or leading '-' that is >= the maximum field
width is handled such that the result of a sccanf is consistent with the
observed behaviour of userland sscanf.
Signed-off-by: Richard Fitzgerald <rf(a)opensource.cirrus.com>
---
lib/kstrtox.c | 13 +++++--
lib/kstrtox.h | 2 ++
lib/vsprintf.c | 93 ++++++++++++++++++++++++++++++--------------------
3 files changed, 68 insertions(+), 40 deletions(-)
diff --git a/lib/kstrtox.c b/lib/kstrtox.c
index a14ccf905055..0ac2ee8bd9d0 100644
--- a/lib/kstrtox.c
+++ b/lib/kstrtox.c
@@ -39,20 +39,22 @@ const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
/*
* Convert non-negative integer string representation in explicitly given radix
- * to an integer.
+ * to an integer. A maximum of max_chars characters will be converted.
+ *
* Return number of characters consumed maybe or-ed with overflow bit.
* If overflow occurs, result integer (incorrect) is still returned.
*
* Don't you dare use this function.
*/
-unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *p)
+unsigned int _parse_integer_limit(const char *s, unsigned int base,
+ unsigned long long *p, size_t max_chars)
{
unsigned long long res;
unsigned int rv;
res = 0;
rv = 0;
- while (1) {
+ for (; max_chars > 0; max_chars--) {
unsigned int c = *s;
unsigned int lc = c | 0x20; /* don't tolower() this line */
unsigned int val;
@@ -82,6 +84,11 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
return rv;
}
+unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *p)
+{
+ return _parse_integer_limit(s, base, p, INT_MAX);
+}
+
static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
{
unsigned long long _res;
diff --git a/lib/kstrtox.h b/lib/kstrtox.h
index 3b4637bcd254..4c6536f85cac 100644
--- a/lib/kstrtox.h
+++ b/lib/kstrtox.h
@@ -4,6 +4,8 @@
#define KSTRTOX_OVERFLOW (1U << 31)
const char *_parse_integer_fixup_radix(const char *s, unsigned int *base);
+unsigned int _parse_integer_limit(const char *s, unsigned int base,
+ unsigned long long *res, size_t max_chars);
unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *res);
#endif
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 14c9a6af1b23..21145da468e0 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -53,29 +53,47 @@
#include <linux/string_helpers.h>
#include "kstrtox.h"
-/**
- * simple_strtoull - convert a string to an unsigned long long
- * @cp: The start of the string
- * @endp: A pointer to the end of the parsed string will be placed here
- * @base: The number base to use
- *
- * This function has caveats. Please use kstrtoull instead.
- */
-unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base)
+static unsigned long long simple_strntoull(const char *startp, size_t max_chars,
+ char **endp, unsigned int base)
{
- unsigned long long result;
+ const char *cp;
+ unsigned long long result = 0ULL;
unsigned int rv;
- cp = _parse_integer_fixup_radix(cp, &base);
- rv = _parse_integer(cp, base, &result);
+ if (max_chars == 0) {
+ cp = startp;
+ goto out;
+ }
+
+ cp = _parse_integer_fixup_radix(startp, &base);
+ if ((cp - startp) >= max_chars) {
+ cp = startp + max_chars;
+ goto out;
+ }
+
+ max_chars -= (cp - startp);
+ rv = _parse_integer_limit(cp, base, &result, max_chars);
/* FIXME */
cp += (rv & ~KSTRTOX_OVERFLOW);
-
+out:
if (endp)
*endp = (char *)cp;
return result;
}
+
+/**
+ * simple_strtoull - convert a string to an unsigned long long
+ * @cp: The start of the string
+ * @endp: A pointer to the end of the parsed string will be placed here
+ * @base: The number base to use
+ *
+ * This function has caveats. Please use kstrtoull instead.
+ */
+unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base)
+{
+ return simple_strntoull(cp, UINT_MAX, endp, base);
+}
EXPORT_SYMBOL(simple_strtoull);
/**
@@ -88,7 +106,7 @@ EXPORT_SYMBOL(simple_strtoull);
*/
unsigned long simple_strtoul(const char *cp, char **endp, unsigned int base)
{
- return simple_strtoull(cp, endp, base);
+ return simple_strntoull(cp, UINT_MAX, endp, base);
}
EXPORT_SYMBOL(simple_strtoul);
@@ -109,6 +127,19 @@ long simple_strtol(const char *cp, char **endp, unsigned int base)
}
EXPORT_SYMBOL(simple_strtol);
+static long long simple_strntoll(const char *cp, size_t max_chars, char **endp,
+ unsigned int base)
+{
+ /*
+ * simple_strntoull safely handles receiving max_chars==0 in the
+ * case we start with max_chars==1 and find a '-' prefix.
+ */
+ if (*cp == '-' && max_chars > 0)
+ return -simple_strntoull(cp + 1, max_chars - 1, endp, base);
+
+ return simple_strntoull(cp, max_chars, endp, base);
+}
+
/**
* simple_strtoll - convert a string to a signed long long
* @cp: The start of the string
@@ -119,10 +150,7 @@ EXPORT_SYMBOL(simple_strtol);
*/
long long simple_strtoll(const char *cp, char **endp, unsigned int base)
{
- if (*cp == '-')
- return -simple_strtoull(cp + 1, endp, base);
-
- return simple_strtoull(cp, endp, base);
+ return simple_strntoll(cp, UINT_MAX, endp, base);
}
EXPORT_SYMBOL(simple_strtoll);
@@ -3433,8 +3461,11 @@ int vsscanf(const char *buf, const char *fmt, va_list args)
str = skip_spaces(str);
digit = *str;
- if (is_sign && digit == '-')
+ if (is_sign && digit == '-') {
+ if (field_width == 1)
+ break;
digit = *(str + 1);
+ }
if (!digit
|| (base == 16 && !isxdigit(digit))
@@ -3444,25 +3475,13 @@ int vsscanf(const char *buf, const char *fmt, va_list args)
break;
if (is_sign)
- val.s = qualifier != 'L' ?
- simple_strtol(str, &next, base) :
- simple_strtoll(str, &next, base);
+ val.s = simple_strntoll(str,
+ field_width > 0 ? field_width : UINT_MAX,
+ &next, base);
else
- val.u = qualifier != 'L' ?
- simple_strtoul(str, &next, base) :
- simple_strtoull(str, &next, base);
-
- if (field_width > 0 && next - str > field_width) {
- if (base == 0)
- _parse_integer_fixup_radix(str, &base);
- while (next - str > field_width) {
- if (is_sign)
- val.s = div_s64(val.s, base);
- else
- val.u = div_u64(val.u, base);
- --next;
- }
- }
+ val.u = simple_strntoull(str,
+ field_width > 0 ? field_width : UINT_MAX,
+ &next, base);
switch (qualifier) {
case 'H': /* that's 'hh' in format */
--
2.20.1
Hi Linus,
Please pull the following KUnit update for Linux 5.11-rc1.
This kunit update for Linux 5.11-rc1 consists of:
-- documentation update and fix to kunit_tool to parse diagnostic
messages correctly from David Gow
-- Support for Parameterized Testing and fs/ext4 test updates to use
KUnit parameterized testing feature from Arpitha Raghunandan
-- Helper to derive file names depending on --build_dir argument
from Andy Shevchenko
Please note that fs/ext4 test change is included in this update
along with the KUnit framework support it depends on.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit b65054597872ce3aefbc6a666385eabdf9e288da:
Linux 5.10-rc6 (2020-11-29 15:50:50 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-5.11-rc1
for you to fetch changes up to 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a:
fs: ext4: Modify inode-test.c to use KUnit parameterized testing
feature (2020-12-02 16:07:25 -0700)
----------------------------------------------------------------
linux-kselftest-kunit-5.11-rc1
This kunit update for Linux 5.11-rc1 consists of:
-- documentation update and fix to kunit_tool to parse diagnostic
messages correctly from David Gow
-- Support for Parameterized Testing and fs/ext4 test updates to use
KUnit parameterized testing feature from Arpitha Raghunandan
-- Helper to derive file names depending on --build_dir argument
from Andy Shevchenko
----------------------------------------------------------------
Andy Shevchenko (1):
kunit: Introduce get_file_path() helper
Arpitha Raghunandan (2):
kunit: Support for Parameterized Testing
fs: ext4: Modify inode-test.c to use KUnit parameterized testing
feature
Daniel Latypov (1):
Documentation: kunit: provide guidance for testing many inputs
David Gow (1):
kunit: kunit_tool: Correctly parse diagnostic messages
Documentation/dev-tools/kunit/usage.rst | 83 ++++++++-
fs/ext4/inode-test.c | 320
++++++++++++++++----------------
include/kunit/test.h | 51 +++++
lib/kunit/test.c | 59 ++++--
tools/testing/kunit/kunit_kernel.py | 24 +--
tools/testing/kunit/kunit_parser.py | 7 +-
6 files changed, 351 insertions(+), 193 deletions(-)
----------------------------------------------------------------
Hi Linus,
Please pull the following Kselftest update for Linux 5.11-rc1.
This kselftest update for Linux 5.11-rc1 consists of:
- Much needed gpio test Makefile cleanup to various problems with
test dependencies and build errors from Michael Ellerman
- Enabling vDSO test on non x86 platforms from Vincenzo Frascino
- Fix intel_pstate to replace deprecated ftime() usages with
clock_gettime() from Tommi Rantala
- cgroup test build fix on older releases from Sachin Sant
- A couple of spelling mistake fixes
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 3650b228f83adda7e5ee532e2b90429c03f7b9ec:
Linux 5.10-rc1 (2020-10-25 15:14:11 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-next-5.11-rc1
for you to fetch changes up to c2e46f6b3e3551558d44c4dc518b9667cb0d5f8b:
selftests/cgroup: Fix build on older distros (2020-11-10 15:13:25 -0700)
----------------------------------------------------------------
linux-kselftest-next-5.11-rc1
This kselftest update for Linux 5.11-rc1 consists of:
- Much needed gpio test Makefile cleanup to various problems with
test dependencies and build errors from Michael Ellerman
- Enabling vDSO test on non x86 platforms from Vincenzo Frascino
- Fix intel_pstate to replace deprecated ftime() usages with
clock_gettime() from Tommi Rantala
- cgroup test build fix on older releases from Sachin Sant
- A couple of spelling mistake fixes
----------------------------------------------------------------
Hangbin Liu (1):
selftests/run_kselftest.sh: fix dry-run typo
Michael Ellerman (5):
selftests/gpio: Use TEST_GEN_PROGS_EXTENDED
selftests/gpio: Move include of lib.mk up
selftests/gpio: Fix build when source tree is read only
selftests/gpio: Add to CLEAN rule rather than overriding
selftests/memfd: Fix implicit declaration warnings
Sachin Sant (1):
selftests/cgroup: Fix build on older distros
Tommi Rantala (1):
selftests: intel_pstate: ftime() is deprecated
Vincenzo Frascino (5):
kselftest: Enable vDSO test on non x86 platforms
kselftest: Extend vDSO selftest
kselftest: Extend vDSO selftest to clock_getres
kselftest: Move test_vdso to the vDSO test suite
kselftest: Extend vdso correctness test to clock_gettime64
Wang Qing (1):
tool: selftests: fix spelling typo of 'writting'
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/cgroup/cgroup_util.c | 4 +-
tools/testing/selftests/gpio/Makefile | 25 +--
tools/testing/selftests/intel_pstate/aperf.c | 22 +-
tools/testing/selftests/memfd/fuse_test.c | 2 +-
tools/testing/selftests/memfd/memfd_test.c | 2 +-
tools/testing/selftests/run_kselftest.sh | 2 +-
tools/testing/selftests/vDSO/Makefile | 16 +-
tools/testing/selftests/vDSO/vdso_config.h | 92 ++++++++
tools/testing/selftests/vDSO/vdso_test_abi.c | 244
+++++++++++++++++++++
.../selftests/vDSO/vdso_test_clock_getres.c | 124 +++++++++++
.../test_vdso.c => vDSO/vdso_test_correctness.c} | 115 +++++++++-
tools/testing/selftests/vm/userfaultfd.c | 4 +-
tools/testing/selftests/x86/Makefile | 2 +-
14 files changed, 621 insertions(+), 34 deletions(-)
create mode 100644 tools/testing/selftests/vDSO/vdso_config.h
create mode 100644 tools/testing/selftests/vDSO/vdso_test_abi.c
create mode 100644 tools/testing/selftests/vDSO/vdso_test_clock_getres.c
rename tools/testing/selftests/{x86/test_vdso.c =>
vDSO/vdso_test_correctness.c} (73%)
----------------------------------------------------------------
Hi Linus,
Please pull this Kselftest fixes update for Linux 5.11-rc1.
This Kselftest fixes update for Linux 5.11-rc1 consists of build error
fixes for clone3 and rseq tests.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 0477e92881850d44910a7e94fc2c46f96faa131f:
Linux 5.10-rc7 (2020-12-06 14:25:12 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-fixes-5.11-rc1
for you to fetch changes up to 88f4ede44c585b24674dd99841040b2a1a856a76:
selftests/clone3: Fix build error (2020-12-07 14:34:55 -0700)
----------------------------------------------------------------
linux-kselftest-fixes-5.11-rc1
This Kselftest fixes update for Linux 5.11-rc1 consists of build error
fixes for clone3 and rseq tests.
----------------------------------------------------------------
Xingxing Su (2):
rseq/selftests: Fix MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ build
error under other arch.
selftests/clone3: Fix build error
tools/testing/selftests/clone3/Makefile | 2 +-
tools/testing/selftests/rseq/param_test.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------
hello,
i have worked on to fix depreciated api issue from
tools/testing/selftests/intel_pstate/aerf.c
i met with the following error related...
--------------x------------------x----------------->
$pwd
/home/jeffrin/UP/linux-kselftest/tools/testing/selftests/intel_pstate
$make
gcc -Wall -D_GNU_SOURCE aperf.c /home/jeffrin/UP/linux-
kselftest/tools/testing/selftests/kselftest_harness.h
/home/jeffrin/UP/linux-kselftest/tools/testing/selftests/kselftest.h -
lm -o /home/jeffrin/UP/linux-
kselftest/tools/testing/selftests/intel_pstate/aperf
aperf.c: In function ‘main’:
aperf.c:58:2: warning: ‘ftime’ is deprecated [-Wdeprecated-
declarations]
58 | ftime(&before);
| ^~~~~
In file included from aperf.c:9:
/usr/include/x86_64-linux-gnu/sys/timeb.h:39:12: note: declared here
39 | extern int ftime (struct timeb *__timebuf)
| ^~~~~
aperf.c:67:2: warning: ‘ftime’ is deprecated [-Wdeprecated-
declarations]
67 | ftime(&after);
| ^~~~~
In file included from aperf.c:9:
/usr/include/x86_64-linux-gnu/sys/timeb.h:39:12: note: declared here
39 | extern int ftime (struct timeb *__timebuf)
| ^~~~~
$
----------------x---------------x---------------------->
from ftime manual i found that it is depreciated...
This function is deprecated, and will be removed in a future version
of the GNU C library. Use clock_gettime(2) instead.
now clock_gettime gives new data structure.
struct timespec {
time_t tv_sec; /* seconds */
long tv_nsec; /* nanoseconds */
};
i worked on with the new data structure and some errors that came
along.
typical final output looks good but values of runtime and typical
frequency
does not look normal during "sudo bash run.sh".
output of "git diff" and a portion of output of "sudo bash run.sh".
is attached.
--
software engineer
rajagiri school of engineering and technology - autonomous
Hi!
This is the second version of the work to make TSC migration more accurate,
as was defined by Paulo at:
https://www.spinics.net/lists/kvm/msg225525.html
I omitted most of the semi-offtopic points I raised related to TSC
in the previous RFC where we can continue the discussion.
I do want to raise another thing that I almost forgot.
On AMD systems, the Linux kernel will mark the guest tsc as
unstable unless invtsc is set which is set on recent AMD
hardware.
Take a look at 'unsynchronized_tsc()' to verify this.
This is another thing that IMHO should be fixed at least when
running under KVM.
Note that I forgot to mention that
X86_FEATURE_TSC_RELIABLE also short-circuits this code,
thus giving another reason to enable it under KVM.
Changes from V1:
- added KVM_TSC_STATE_TIMESTAMP_VALID instead of testing ns == 0
- allow diff < 0, because it is still better that capping it to 0
- updated tsc_msr_test unit test to cover this feature
- refactoring
Patches to enable this feature in qemu are in the process of
being sent to qemu-devel mailing list.
Best regards,
Maxim Levitsky
Maxim Levitsky (3):
KVM: x86: implement KVM_{GET|SET}_TSC_STATE
KVM: x86: introduce KVM_X86_QUIRK_TSC_HOST_ACCESS
kvm/selftests: update tsc_msrs_test to cover
KVM_X86_QUIRK_TSC_HOST_ACCESS
Documentation/virt/kvm/api.rst | 65 +++++++++++++
arch/x86/include/uapi/asm/kvm.h | 1 +
arch/x86/kvm/x86.c | 92 ++++++++++++++++++-
include/uapi/linux/kvm.h | 15 +++
.../selftests/kvm/x86_64/tsc_msrs_test.c | 79 ++++++++++++++--
5 files changed, 237 insertions(+), 15 deletions(-)
--
2.26.2
The cleanup function in this script that tries to delete hv-1 / hv-2
vm-1 / vm-2 netns will generate some uncessary error messages:
Cannot remove namespace file "/run/netns/hv-2": No such file or directory
Cannot remove namespace file "/run/netns/vm-1": No such file or directory
Cannot remove namespace file "/run/netns/vm-2": No such file or directory
Redirect it to /dev/null like other commands in the cleanup function
to reduce confusion.
Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com>
---
tools/testing/selftests/net/test_vxlan_under_vrf.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/test_vxlan_under_vrf.sh b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
index 09f9ed9..534c8b7 100755
--- a/tools/testing/selftests/net/test_vxlan_under_vrf.sh
+++ b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
@@ -50,7 +50,7 @@ cleanup() {
ip link del veth-tap 2>/dev/null || true
for ns in hv-1 hv-2 vm-1 vm-2; do
- ip netns del $ns || true
+ ip netns del $ns 2>/dev/null || true
done
}
--
2.7.4
lib.mk defaults to gcc when CC is not set. When building selftests
as part of a kernel compilation, MAKEFLAGS is cleared to allow implicit
build rules to be used. This has the side-effect of clearing the CC
variable, which will cause selftests to be built with gcc regardless of
if we are using gcc or clang. To remedy this, propagate the CC variable
when clearing makeflags to ensure the correct compiler is used.
Signed-off-by: Andrew Delgadillo <adelg(a)google.com>
---
tools/testing/selftests/Makefile | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index d9c283503159..a4dd6d7e8276 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -90,10 +90,12 @@ FORCE_TARGETS ?=
# Clear LDFLAGS and MAKEFLAGS when implicit rules are missing. This provides
# implicit rules to sub-test Makefiles which avoids build failures in test
-# Makefile that don't have explicit build rules.
+# Makefile that don't have explicit build rules. Since lib.mk defaults to
+# using gcc for compilation when the CC variable is not set, we propagate the
+# CC variable so if clang is being used, selftests will build with clang.
ifeq (,$(LINK.c))
override LDFLAGS =
-override MAKEFLAGS =
+override MAKEFLAGS = CC=$(CC)
endif
# Append kselftest to KBUILD_OUTPUT and O to avoid cluttering
--
2.29.2.576.ga3fc446d84-goog
The cleanup function in this script that tries to delete hv-1 / hv-2
vm-1 / vm-2 netns will generate some uncessary error messages:
Cannot remove namespace file "/run/netns/hv-2": No such file or directory
Cannot remove namespace file "/run/netns/vm-1": No such file or directory
Cannot remove namespace file "/run/netns/vm-2": No such file or directory
Redirect it to /dev/null like other commands in the cleanup function
to reduce confusion.
Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com>
---
tools/testing/selftests/net/test_vxlan_under_vrf.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/test_vxlan_under_vrf.sh b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
index 09f9ed9..534c8b7 100755
--- a/tools/testing/selftests/net/test_vxlan_under_vrf.sh
+++ b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
@@ -50,7 +50,7 @@ cleanup() {
ip link del veth-tap 2>/dev/null || true
for ns in hv-1 hv-2 vm-1 vm-2; do
- ip netns del $ns || true
+ ip netns del $ns 2>/dev/null || true
done
}
--
2.7.4
Building the BPF selftests with clang 11, I'm getting the following
error:
CLNG-LLC [test_maps] profiler1.o
In file included from progs/profiler1.c:6:
progs/profiler.inc.h:260:17: error: use of unknown builtin '__builtin_preserve_enum_value' [-Wimplicit-function-declaration]
int cgrp_id = bpf_core_enum_value(enum cgroup_subsys_id___local,
^
/home/ubuntu/unstable/tools/testing/selftests/bpf/tools/include/bpf/bpf_core_read.h:179:2: note: expanded from macro 'bpf_core_enum_value'
__builtin_preserve_enum_value(*(typeof(enum_type) *)enum_value, BPF_ENUMVAL_VALUE)
^
1 error generated.
llc: error: llc: <stdin>:1:1: error: expected top-level entity
BPF obj compilation failed
I see that test_core_reloc_enumval.c takes precautions around the use of
__builtin_preserve_enum_value as it is currently only available in clang
12 nightlies. Is it possible to do something similar here? Though I see
that the use of the builtin is not nearly so neatly localized as it is
in test_core_reloc_enumval.c.
Thanks,
Seth
This patch set adds AF_XDP selftests based on veth to selftests/bpf.
# Topology:
# ---------
# -----------
# _ | Process | _
# / ----------- \
# / | \
# / | \
# ----------- | -----------
# | Thread1 | | | Thread2 |
# ----------- | -----------
# | | |
# ----------- | -----------
# | xskX | | | xskY |
# ----------- | -----------
# | | |
# ----------- | ----------
# | vethX | --------- | vethY |
# ----------- peer ----------
# | | |
# namespaceX | namespaceY
These selftests test AF_XDP SKB and Native/DRV modes using veth Virtual
Ethernet interfaces.
The test program contains two threads, each thread is single socket with
a unique UMEM. It validates in-order packet delivery and packet content
by sending packets to each other.
Prerequisites setup by script test_xsk.sh:
Set up veth interfaces as per the topology shown ^^:
* setup two veth interfaces and one namespace
** veth<xxxx> in root namespace
** veth<yyyy> in af_xdp<xxxx> namespace
** namespace af_xdp<xxxx>
* create a spec file veth.spec that includes this run-time configuration
*** xxxx and yyyy are randomly generated 4 digit numbers used to avoid
conflict with any existing interface
Adds xsk framework test to validate veth xdp DRV and SKB modes.
The following tests are provided:
1. AF_XDP SKB mode
Generic mode XDP is driver independent, used when the driver does
not have support for XDP. Works on any netdevice using sockets and
generic XDP path. XDP hook from netif_receive_skb().
a. nopoll - soft-irq processing
b. poll - using poll() syscall
c. Socket Teardown
Create a Tx and a Rx socket, Tx from one socket, Rx on another.
Destroy both sockets, then repeat multiple times. Only nopoll mode
is used
d. Bi-directional Sockets
Configure sockets as bi-directional tx/rx sockets, sets up fill
and completion rings on each socket, tx/rx in both directions.
Only nopoll mode is used
2. AF_XDP DRV/Native mode
Works on any netdevice with XDP_REDIRECT support, driver dependent.
Processes packets before SKB allocation. Provides better performance
than SKB. Driver hook available just after DMA of buffer descriptor.
a. nopoll
b. poll
c. Socket Teardown
d. Bi-directional Sockets
* Only copy mode is supported because veth does not currently support
zero-copy mode
Total tests: 8
Flow:
* Single process spawns two threads: Tx and Rx
* Each of these two threads attach to a veth interface within their
assigned namespaces
* Each thread creates one AF_XDP socket connected to a unique umem
for each veth interface
* Tx thread transmits 10k packets from veth<xxxx> to veth<yyyy>
* Rx thread verifies if all 10k packets were received and delivered
in-order, and have the right content
v2 changes:
* Move selftests/xsk to selftests/bpf
* Remove Makefiles under selftests/xsk, and utilize selftests/bpf/Makefile
v3 changes:
* merge all test scripts test_xsk_*.sh into test_xsk.sh
v4 changes:
* merge xsk_env.sh into xsk_prereqs.sh
* test_xsk.sh add cliarg -c for color-coded output
* test_xsk.sh PREREQUISITES disables IPv6 on veth interfaces
* test_xsk.sh PREREQUISITES adds xsk framework test
* test_xsk.sh is independently executable
* xdpxceiver.c Tx/Rx validates only IPv4 packets with TOS 0x9, ignores
others
Structure of the patch set:
Patch 1: Adds XSK selftests framework and test under selftests/bpf
Patch 2: Adds tests: SKB poll and nopoll mode, and mac-ip-udp debug
Patch 3: Adds tests: DRV poll and nopoll mode
Patch 4: Adds tests: SKB and DRV Socket Teardown
Patch 5: Adds tests: SKB and DRV Bi-directional Sockets
Thanks: Weqaar
Weqaar Janjua (5):
selftests/bpf: xsk selftests framework
selftests/bpf: xsk selftests - SKB POLL, NOPOLL
selftests/bpf: xsk selftests - DRV POLL, NOPOLL
selftests/bpf: xsk selftests - Socket Teardown - SKB, DRV
selftests/bpf: xsk selftests - Bi-directional Sockets - SKB, DRV
tools/testing/selftests/bpf/Makefile | 7 +-
tools/testing/selftests/bpf/test_xsk.sh | 259 +++++
tools/testing/selftests/bpf/xdpxceiver.c | 1074 ++++++++++++++++++++
tools/testing/selftests/bpf/xdpxceiver.h | 160 +++
tools/testing/selftests/bpf/xsk_prereqs.sh | 135 +++
5 files changed, 1633 insertions(+), 2 deletions(-)
create mode 100755 tools/testing/selftests/bpf/test_xsk.sh
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.c
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.h
create mode 100755 tools/testing/selftests/bpf/xsk_prereqs.sh
--
2.20.1
The test program udpgso_bench_rx always invokes the poll()
syscall with a timeout of 10ms. If a larger timeout is specified
via the command line, udpgso_bench_rx is supposed to do multiple
poll() calls till the timeout is expired or an event is received.
Currently the poll() loop errors out after the first invocation with
no events, and may causes self-tests failure alike:
failed
GRO with custom segment size ./udpgso_bench_rx: poll: 0x0 expected 0x1
This change addresses the issue allowing the poll() loop to consume
all the configured timeout.
Fixes: ada641ff6ed3 ("selftests: fixes for UDP GRO")
Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
---
tools/testing/selftests/net/udpgso_bench_rx.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/testing/selftests/net/udpgso_bench_rx.c b/tools/testing/selftests/net/udpgso_bench_rx.c
index db3d4a8b5a4c..76a24052f4b4 100644
--- a/tools/testing/selftests/net/udpgso_bench_rx.c
+++ b/tools/testing/selftests/net/udpgso_bench_rx.c
@@ -113,6 +113,9 @@ static void do_poll(int fd, int timeout_ms)
interrupted = true;
break;
}
+
+ /* no events and more time to wait, do poll again */
+ continue;
}
if (pfd.revents != POLLIN)
error(1, errno, "poll: 0x%x expected 0x%x\n",
--
2.26.2
The DIAGNOSE 0x0318 instruction, unique to s390x, is a privileged call
that must be intercepted via SIE, handled in userspace, and the
information set by the instruction is communicated back to KVM.
To test the instruction interception, an ad-hoc handler is defined which
simply has a VM execute the instruction and then userspace will extract
the necessary info. The handler is defined such that the instruction
invocation occurs only once. It is up to the caller to determine how the
info returned by this handler should be used.
The diag318 info is communicated from userspace to KVM via a sync_regs
call. This is tested During a sync_regs test, where the diag318 info is
requested via the handler, then the info is stored in the appropriate
register in KVM via a sync registers call.
If KVM does not support diag318, then the tests will print a message
stating that diag318 was skipped, and the asserts will simply test
against a value of 0.
Signed-off-by: Collin Walling <walling(a)linux.ibm.com>
---
tools/testing/selftests/kvm/Makefile | 2 +-
.../kvm/include/s390x/diag318_test_handler.h | 13 +++
.../kvm/lib/s390x/diag318_test_handler.c | 82 +++++++++++++++++++
.../selftests/kvm/s390x/sync_regs_test.c | 16 +++-
4 files changed, 111 insertions(+), 2 deletions(-)
create mode 100644 tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
create mode 100644 tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 3d14ef77755e..426c78449044 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -36,7 +36,7 @@ endif
LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c
LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
-LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
+LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
TEST_GEN_PROGS_x86_64 += x86_64/evmcs_test
diff --git a/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h b/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
new file mode 100644
index 000000000000..b0ed71302722
--- /dev/null
+++ b/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Test handler for the s390x DIAGNOSE 0x0318 instruction.
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#ifndef SELFTEST_KVM_DIAG318_TEST_HANDLER
+#define SELFTEST_KVM_DIAG318_TEST_HANDLER
+
+uint64_t get_diag318_info(void);
+
+#endif
diff --git a/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c b/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
new file mode 100644
index 000000000000..86b9e611ad87
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Test handler for the s390x DIAGNOSE 0x0318 instruction.
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+
+#define VCPU_ID 6
+
+#define ICPT_INSTRUCTION 0x04
+#define IPA0_DIAG 0x8300
+
+static void guest_code(void)
+{
+ uint64_t diag318_info = 0x12345678;
+
+ asm volatile ("diag %0,0,0x318\n" : : "d" (diag318_info));
+}
+
+/*
+ * The DIAGNOSE 0x0318 instruction call must be handled via userspace. As such,
+ * we create an ad-hoc VM here to handle the instruction then extract the
+ * necessary data. It is up to the caller to decide what to do with that data.
+ */
+static uint64_t diag318_handler(void)
+{
+ struct kvm_vm *vm;
+ struct kvm_run *run;
+ uint64_t reg;
+ uint64_t diag318_info;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_code);
+ vcpu_run(vm, VCPU_ID);
+ run = vcpu_state(vm, VCPU_ID);
+
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
+ "DIAGNOSE 0x0318 instruction was not intercepted");
+ TEST_ASSERT(run->s390_sieic.icptcode == ICPT_INSTRUCTION,
+ "Unexpected intercept code: 0x%x", run->s390_sieic.icptcode);
+ TEST_ASSERT((run->s390_sieic.ipa & 0xff00) == IPA0_DIAG,
+ "Unexpected IPA0 code: 0x%x", (run->s390_sieic.ipa & 0xff00));
+
+ reg = (run->s390_sieic.ipa & 0x00f0) >> 4;
+ diag318_info = run->s.regs.gprs[reg];
+
+ TEST_ASSERT(diag318_info != 0, "DIAGNOSE 0x0318 info not set");
+
+ kvm_vm_free(vm);
+
+ return diag318_info;
+}
+
+uint64_t get_diag318_info(void)
+{
+ static uint64_t diag318_info;
+ static bool printed_skip;
+
+ /*
+ * If KVM does not support diag318, then return 0 to
+ * ensure tests do not break.
+ */
+ if (!kvm_check_cap(KVM_CAP_S390_DIAG318)) {
+ if (!printed_skip) {
+ fprintf(stdout, "KVM_CAP_S390_DIAG318 not supported. "
+ "Skipping diag318 test.\n");
+ printed_skip = true;
+ }
+ return 0;
+ }
+
+ /*
+ * If a test has previously requested the diag318 info,
+ * then don't bother spinning up a temporary VM again.
+ */
+ if (!diag318_info)
+ diag318_info = diag318_handler();
+
+ return diag318_info;
+}
diff --git a/tools/testing/selftests/kvm/s390x/sync_regs_test.c b/tools/testing/selftests/kvm/s390x/sync_regs_test.c
index 5731ccf34917..caf7b8859a94 100644
--- a/tools/testing/selftests/kvm/s390x/sync_regs_test.c
+++ b/tools/testing/selftests/kvm/s390x/sync_regs_test.c
@@ -20,6 +20,7 @@
#include "test_util.h"
#include "kvm_util.h"
+#include "diag318_test_handler.h"
#define VCPU_ID 5
@@ -70,7 +71,7 @@ static void compare_sregs(struct kvm_sregs *left, struct kvm_sync_regs *right)
#undef REG_COMPARE
-#define TEST_SYNC_FIELDS (KVM_SYNC_GPRS|KVM_SYNC_ACRS|KVM_SYNC_CRS)
+#define TEST_SYNC_FIELDS (KVM_SYNC_GPRS|KVM_SYNC_ACRS|KVM_SYNC_CRS|KVM_SYNC_DIAG318)
#define INVALID_SYNC_FIELD 0x80000000
int main(int argc, char *argv[])
@@ -152,6 +153,12 @@ int main(int argc, char *argv[])
run->kvm_valid_regs = TEST_SYNC_FIELDS;
run->kvm_dirty_regs = KVM_SYNC_GPRS | KVM_SYNC_ACRS;
+
+ if (get_diag318_info() > 0) {
+ run->s.regs.diag318 = get_diag318_info();
+ run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
+ }
+
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(rv == 0, "vcpu_run failed: %d\n", rv);
TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
@@ -164,6 +171,9 @@ int main(int argc, char *argv[])
TEST_ASSERT(run->s.regs.acrs[0] == 1 << 11,
"acr0 sync regs value incorrect 0x%x.",
run->s.regs.acrs[0]);
+ TEST_ASSERT(run->s.regs.diag318 == get_diag318_info(),
+ "diag318 sync regs value incorrect 0x%llx.",
+ run->s.regs.diag318);
vcpu_regs_get(vm, VCPU_ID, ®s);
compare_regs(®s, &run->s.regs);
@@ -177,6 +187,7 @@ int main(int argc, char *argv[])
run->kvm_valid_regs = TEST_SYNC_FIELDS;
run->kvm_dirty_regs = 0;
run->s.regs.gprs[11] = 0xDEADBEEF;
+ run->s.regs.diag318 = 0x4B1D;
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(rv == 0, "vcpu_run failed: %d\n", rv);
TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
@@ -186,6 +197,9 @@ int main(int argc, char *argv[])
TEST_ASSERT(run->s.regs.gprs[11] != 0xDEADBEEF,
"r11 sync regs value incorrect 0x%llx.",
run->s.regs.gprs[11]);
+ TEST_ASSERT(run->s.regs.diag318 != 0x4B1D,
+ "diag318 sync regs value incorrect 0x%llx.",
+ run->s.regs.diag318);
kvm_vm_free(vm);
--
2.26.2
On Fri, 27 Nov 2020 at 04:19, Yonghong Song <yhs(a)fb.com> wrote:
>
>
>
> On 11/26/20 1:22 PM, Weqaar Janjua wrote:
> > On Thu, 26 Nov 2020 at 09:01, Björn Töpel <bjorn.topel(a)intel.com> wrote:
> >>
> >> On 2020-11-26 07:44, Yonghong Song wrote:
> >>>
> >> [...]
> >>>
> >>> What other configures I am missing?
> >>>
> >>> BTW, I cherry-picked the following pick from bpf tree in this experiment.
> >>> commit e7f4a5919bf66e530e08ff352d9b78ed89574e6b (HEAD -> xsk)
> >>> Author: Björn Töpel <bjorn.topel(a)intel.com>
> >>> Date: Mon Nov 23 18:56:00 2020 +0100
> >>>
> >>> net, xsk: Avoid taking multiple skbuff references
> >>>
> >>
> >> Hmm, I'm getting an oops, unless I cherry-pick:
> >>
> >> 36ccdf85829a ("net, xsk: Avoid taking multiple skbuff references")
> >>
> >> *AND*
> >>
> >> 537cf4e3cc2f ("xsk: Fix umem cleanup bug at socket destruct")
> >>
> >> from bpf/master.
> >>
> >
> > Same as Bjorn's findings ^^^, additionally applying the second patch
> > 537cf4e3cc2f [PASS] all tests for me
> >
> > PREREQUISITES: [ PASS ]
> > SKB NOPOLL: [ PASS ]
> > SKB POLL: [ PASS ]
> > DRV NOPOLL: [ PASS ]
> > DRV POLL: [ PASS ]
> > SKB SOCKET TEARDOWN: [ PASS ]
> > DRV SOCKET TEARDOWN: [ PASS ]
> > SKB BIDIRECTIONAL SOCKETS: [ PASS ]
> > DRV BIDIRECTIONAL SOCKETS: [ PASS ]
> >
> > With the first patch alone, as soon as we enter DRV/Native NOPOLL mode
> > kernel panics, whereas in your case NOPOLL tests were falling with
> > packets being *lost* as per seqnum mismatch.
> >
> > Can you please test this out with both patches and let us know?
>
> I applied both the above patches in bpf-next as well as this patch set,
> I still see failures. I am attaching my config file. Maybe you can take
> a look at what is the issue.
>
Thanks for the config, can you please confirm the compiler version,
and resource limits i.e. stack size, memory, etc.?
Only NOPOLL tests are failing for you as I see it, do the same tests
fail every time?
I will need to spend some time debugging this to have a fix.
Thanks,
/Weqaar
> >
> >> Can I just run test_xsk.sh at tools/testing/selftests/bpf/ directory?
> >> This will be easier than the above for bpf developers. If it does not
> >> work, I would like to recommend to make it work.
> >>
> > yes test_xsk.shis self contained, will update the instructions in there with v4.
>
> That will be great. Thanks!
>
> >
> > Thanks,
> > /Weqaar
> >>
> >> Björn
This series aims to add support to bpf_snprintf_btf() and
bpf_seq_printf_btf() allowing them to store string representations
of module-specific types, as well as the kernel-specific ones
they currently support.
Patch 1 removes the btf_module_mutex, as since we will need to
look up module BTF during BPF program execution, we don't want
to risk sleeping in the various contexts in which BPF can run.
The access patterns to the btf module list seem to conform to
classic list RCU usage so with a few minor tweaks this seems
workable.
Patch 2 replaces the unused flags field in struct btf_ptr with
an obj_id field, allowing the specification of the id of a
BTF module. If the value is 0, the core kernel vmlinux is
assumed to contain the type's BTF information. Otherwise the
module with that id is used to identify the type. If the
object-id based lookup fails, we again fall back to vmlinux
BTF.
Patch 3 is a selftest that uses veth (when built as a
module) and a kprobe to display both a module-specific
and kernel-specific type; both are arguments to veth_stats_rx().
Currently it looks up the module-specific type and object ids
using libbpf; in future, these lookups will likely be supported
directly in the BPF program via __builtin_btf_type_id(); but
I need to determine a good test to determine if that builtin
supports object ids.
Changes since RFC
- add patch to remove module mutex
- modify to use obj_id instead of module name as identifier
in "struct btf_ptr" (Andrii)
Alan Maguire (3):
bpf: eliminate btf_module_mutex as RCU synchronization can be used
bpf: add module support to btf display helpers
selftests/bpf: verify module-specific types can be shown via
bpf_snprintf_btf
include/linux/btf.h | 12 ++
include/uapi/linux/bpf.h | 13 ++-
kernel/bpf/btf.c | 49 +++++---
kernel/trace/bpf_trace.c | 44 ++++++--
tools/include/uapi/linux/bpf.h | 13 ++-
.../selftests/bpf/prog_tests/snprintf_btf_mod.c | 124 +++++++++++++++++++++
tools/testing/selftests/bpf/progs/bpf_iter.h | 2 +-
tools/testing/selftests/bpf/progs/btf_ptr.h | 2 +-
tools/testing/selftests/bpf/progs/veth_stats_rx.c | 72 ++++++++++++
9 files changed, 292 insertions(+), 39 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/snprintf_btf_mod.c
create mode 100644 tools/testing/selftests/bpf/progs/veth_stats_rx.c
--
1.8.3.1
When compiling the selftests with the -std=gnu99 option the build can
fail with.
Following build error:
test_core.c: In function ‘test_cgcore_destroy’:
test_core.c:87:2: error: ‘for’ loop initial declarations are only
allowed in C99 mode
for (int i = 0; i < 10; i++) {
^
test_core.c:87:2: note: use option -std=c99 or -std=gnu99 to compile
Add -std=gnu99 to the clone3 selftest Makefile to fix this.
Signed-off-by: Xingxing Su <suxingxing(a)loongson.cn>
Acked-by: Christian Brauner <christian.brauner(a)ubuntu.com>
---
tools/testing/selftests/clone3/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/clone3/Makefile b/tools/testing/selftests/clone3/Makefile
index ef7564c..88354a8 100644
--- a/tools/testing/selftests/clone3/Makefile
+++ b/tools/testing/selftests/clone3/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-CFLAGS += -g -I../../../../usr/include/
+CFLAGS += -g -std=gnu99 -I../../../../usr/include/
LDLIBS += -lcap
TEST_GEN_PROGS := clone3 clone3_clear_sighand clone3_set_tid \
--
1.8.3.1
Except arch x86, the function rseq_offset_deref_addv is not defined.
The function test_membarrier_manager_thread call rseq_offset_deref_addv
produces a build error.
The RSEQ_ARCH_HAS_OFFSET_DEREF_ADD should contain all the code
for the MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ.
If the other Arch implements this feature,
defined RSEQ_ARCH_HAS_OFFSET_DEREF_ADD in the header file
to ensure that this feature is available.
Following build errors:
param_test.c: In function ‘test_membarrier_worker_thread’:
param_test.c:1164:10: warning: implicit declaration of function ‘rseq_offset_deref_addv’
ret = rseq_offset_deref_addv(&args->percpu_list_ptr,
^~~~~~~~~~~~~~~~~~~~~~
/tmp/ccMj9yHJ.o: In function `test_membarrier_worker_thread':
param_test.c:1164: undefined reference to `rseq_offset_deref_addv'
param_test.c:1164: undefined reference to `rseq_offset_deref_addv'
collect2: error: ld returned 1 exit status
make: *** [/selftests/rseq/param_test_benchmark] Error 1
Signed-off-by: Xingxing Su <suxingxing(a)loongson.cn>
---
tools/testing/selftests/rseq/param_test.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rseq/param_test.c b/tools/testing/selftests/rseq/param_test.c
index 3845890..699ad5f 100644
--- a/tools/testing/selftests/rseq/param_test.c
+++ b/tools/testing/selftests/rseq/param_test.c
@@ -1133,6 +1133,8 @@ static int set_signal_handler(void)
return ret;
}
+/* Test MEMBARRIER_CMD_PRIVATE_RESTART_RSEQ_ON_CPU membarrier command. */
+#ifdef RSEQ_ARCH_HAS_OFFSET_DEREF_ADDV
struct test_membarrier_thread_args {
int stop;
intptr_t percpu_list_ptr;
@@ -1286,8 +1288,6 @@ void *test_membarrier_manager_thread(void *arg)
return NULL;
}
-/* Test MEMBARRIER_CMD_PRIVATE_RESTART_RSEQ_ON_CPU membarrier command. */
-#ifdef RSEQ_ARCH_HAS_OFFSET_DEREF_ADDV
void test_membarrier(void)
{
const int num_threads = opt_threads;
--
1.8.3.1
The DIAGNOSE 0x0318 instruction, unique to s390x, is a privileged call
that must be intercepted via SIE, handled in userspace, and the
information set by the instruction is communicated back to KVM.
To test the instruction interception, an ad-hoc handler is defined which
simply has a VM execute the instruction and then userspace will extract
the necessary info. The handler is defined such that the instruction
invocation occurs only once. It is up to the caller to determine how the
info returned by this handler should be used.
The diag318 info is communicated from userspace to KVM via a sync_regs
call. This is tested during a sync_regs test, where the diag318 info is
requested via the handler, then the info is stored in the appropriate
register in KVM via a sync registers call.
If KVM does not support diag318, then the tests will print a message
stating that diag318 was skipped, and the asserts will simply test
against a value of 0.
Signed-off-by: Collin Walling <walling(a)linux.ibm.com>
---
v3 changes: no longer testing the reset code, as it is handled
entirely via userspace. The respective reset tests have been removed
---
tools/testing/selftests/kvm/Makefile | 2 +-
.../kvm/include/s390x/diag318_test_handler.h | 13 +++
.../kvm/lib/s390x/diag318_test_handler.c | 82 +++++++++++++++++++
tools/testing/selftests/kvm/s390x/resets.c | 1 +
.../selftests/kvm/s390x/sync_regs_test.c | 16 +++-
5 files changed, 112 insertions(+), 2 deletions(-)
create mode 100644 tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
create mode 100644 tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 3d14ef77755e..426c78449044 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -36,7 +36,7 @@ endif
LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c
LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
-LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
+LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
TEST_GEN_PROGS_x86_64 += x86_64/evmcs_test
diff --git a/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h b/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
new file mode 100644
index 000000000000..b0ed71302722
--- /dev/null
+++ b/tools/testing/selftests/kvm/include/s390x/diag318_test_handler.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Test handler for the s390x DIAGNOSE 0x0318 instruction.
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#ifndef SELFTEST_KVM_DIAG318_TEST_HANDLER
+#define SELFTEST_KVM_DIAG318_TEST_HANDLER
+
+uint64_t get_diag318_info(void);
+
+#endif
diff --git a/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c b/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
new file mode 100644
index 000000000000..1e0b766efeb7
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/s390x/diag318_test_handler.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Test handler for the s390x DIAGNOSE 0x0318 instruction.
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+
+#define VCPU_ID 5
+
+#define ICPT_INSTRUCTION 0x04
+#define IPA0_DIAG 0x8300
+
+static void guest_code(void)
+{
+ uint64_t diag318_info = 0x12345678;
+
+ asm volatile ("diag %0,0,0x318\n" : : "d" (diag318_info));
+}
+
+/*
+ * The DIAGNOSE 0x0318 instruction call must be handled via userspace. As such,
+ * we create an ad-hoc VM here to handle the instruction then extract the
+ * necessary data. It is up to the caller to decide what to do with that data.
+ */
+static uint64_t diag318_handler(void)
+{
+ struct kvm_vm *vm;
+ struct kvm_run *run;
+ uint64_t reg;
+ uint64_t diag318_info;
+
+ vm = vm_create_default(VCPU_ID, 0, guest_code);
+ vcpu_run(vm, VCPU_ID);
+ run = vcpu_state(vm, VCPU_ID);
+
+ TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
+ "DIAGNOSE 0x0318 instruction was not intercepted");
+ TEST_ASSERT(run->s390_sieic.icptcode == ICPT_INSTRUCTION,
+ "Unexpected intercept code: 0x%x", run->s390_sieic.icptcode);
+ TEST_ASSERT((run->s390_sieic.ipa & 0xff00) == IPA0_DIAG,
+ "Unexpected IPA0 code: 0x%x", (run->s390_sieic.ipa & 0xff00));
+
+ reg = (run->s390_sieic.ipa & 0x00f0) >> 4;
+ diag318_info = run->s.regs.gprs[reg];
+
+ TEST_ASSERT(diag318_info != 0, "DIAGNOSE 0x0318 info not set");
+
+ kvm_vm_free(vm);
+
+ return diag318_info;
+}
+
+uint64_t get_diag318_info(void)
+{
+ static uint64_t diag318_info;
+ static bool printed_skip;
+
+ /*
+ * If KVM does not support diag318, then return 0 to
+ * ensure tests do not break.
+ */
+ if (!kvm_check_cap(KVM_CAP_S390_DIAG318)) {
+ if (!printed_skip) {
+ fprintf(stdout, "KVM_CAP_S390_DIAG318 not supported. "
+ "Skipping diag318 test.\n");
+ printed_skip = true;
+ }
+ return 0;
+ }
+
+ /*
+ * If a test has previously requested the diag318 info,
+ * then don't bother spinning up a temporary VM again.
+ */
+ if (!diag318_info)
+ diag318_info = diag318_handler();
+
+ return diag318_info;
+}
diff --git a/tools/testing/selftests/kvm/s390x/resets.c b/tools/testing/selftests/kvm/s390x/resets.c
index b143db6d8693..b3d7d4ac2d54 100644
--- a/tools/testing/selftests/kvm/s390x/resets.c
+++ b/tools/testing/selftests/kvm/s390x/resets.c
@@ -110,6 +110,7 @@ static void assert_clear(void)
TEST_ASSERT(!memcmp(sync_regs->vrs, regs_null, sizeof(sync_regs->vrs)),
"vrs0-15 == 0 (sync_regs)");
+
}
static void assert_initial_noclear(void)
diff --git a/tools/testing/selftests/kvm/s390x/sync_regs_test.c b/tools/testing/selftests/kvm/s390x/sync_regs_test.c
index 5731ccf34917..caf7b8859a94 100644
--- a/tools/testing/selftests/kvm/s390x/sync_regs_test.c
+++ b/tools/testing/selftests/kvm/s390x/sync_regs_test.c
@@ -20,6 +20,7 @@
#include "test_util.h"
#include "kvm_util.h"
+#include "diag318_test_handler.h"
#define VCPU_ID 5
@@ -70,7 +71,7 @@ static void compare_sregs(struct kvm_sregs *left, struct kvm_sync_regs *right)
#undef REG_COMPARE
-#define TEST_SYNC_FIELDS (KVM_SYNC_GPRS|KVM_SYNC_ACRS|KVM_SYNC_CRS)
+#define TEST_SYNC_FIELDS (KVM_SYNC_GPRS|KVM_SYNC_ACRS|KVM_SYNC_CRS|KVM_SYNC_DIAG318)
#define INVALID_SYNC_FIELD 0x80000000
int main(int argc, char *argv[])
@@ -152,6 +153,12 @@ int main(int argc, char *argv[])
run->kvm_valid_regs = TEST_SYNC_FIELDS;
run->kvm_dirty_regs = KVM_SYNC_GPRS | KVM_SYNC_ACRS;
+
+ if (get_diag318_info() > 0) {
+ run->s.regs.diag318 = get_diag318_info();
+ run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
+ }
+
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(rv == 0, "vcpu_run failed: %d\n", rv);
TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
@@ -164,6 +171,9 @@ int main(int argc, char *argv[])
TEST_ASSERT(run->s.regs.acrs[0] == 1 << 11,
"acr0 sync regs value incorrect 0x%x.",
run->s.regs.acrs[0]);
+ TEST_ASSERT(run->s.regs.diag318 == get_diag318_info(),
+ "diag318 sync regs value incorrect 0x%llx.",
+ run->s.regs.diag318);
vcpu_regs_get(vm, VCPU_ID, ®s);
compare_regs(®s, &run->s.regs);
@@ -177,6 +187,7 @@ int main(int argc, char *argv[])
run->kvm_valid_regs = TEST_SYNC_FIELDS;
run->kvm_dirty_regs = 0;
run->s.regs.gprs[11] = 0xDEADBEEF;
+ run->s.regs.diag318 = 0x4B1D;
rv = _vcpu_run(vm, VCPU_ID);
TEST_ASSERT(rv == 0, "vcpu_run failed: %d\n", rv);
TEST_ASSERT(run->exit_reason == KVM_EXIT_S390_SIEIC,
@@ -186,6 +197,9 @@ int main(int argc, char *argv[])
TEST_ASSERT(run->s.regs.gprs[11] != 0xDEADBEEF,
"r11 sync regs value incorrect 0x%llx.",
run->s.regs.gprs[11]);
+ TEST_ASSERT(run->s.regs.diag318 != 0x4B1D,
+ "diag318 sync regs value incorrect 0x%llx.",
+ run->s.regs.diag318);
kvm_vm_free(vm);
--
2.26.2
This series adds support for transparent huge page migration to
migrate_vma_*() and adds nouveau SVM and HMM selftests as consumers.
Earlier versions were posted previously [1] and [2].
The patches apply cleanly to the linux-mm 5.10.0-rc2 tree. There are a
lot of other THP patches being posted. I don't think there are any
semantic conflicts but there may be some merge conflicts depending on
the order Andrew applies these.
Changes in v3:
Sent the patch ("mm/thp: fix __split_huge_pmd_locked() for migration PMD")
as a separate patch from this series.
Rebased to linux-mm 5.10.0-rc2.
Changes in v2:
Added splitting a THP midway in the migration process:
i.e., in migrate_vma_pages().
[1] https://lore.kernel.org/linux-mm/20200619215649.32297-1-rcampbell@nvidia.com
[2] https://lore.kernel.org/linux-mm/20200902165830.5367-1-rcampbell@nvidia.com
Ralph Campbell (6):
mm/thp: add prep_transhuge_device_private_page()
mm/migrate: move migrate_vma_collect_skip()
mm: support THP migration to device private memory
mm/thp: add THP allocation helper
mm/hmm/test: add self tests for THP migration
nouveau: support THP migration to private memory
drivers/gpu/drm/nouveau/nouveau_dmem.c | 289 +++++++++++-----
drivers/gpu/drm/nouveau/nouveau_svm.c | 11 +-
drivers/gpu/drm/nouveau/nouveau_svm.h | 3 +-
include/linux/gfp.h | 10 +
include/linux/huge_mm.h | 12 +
include/linux/memremap.h | 9 +
include/linux/migrate.h | 2 +
lib/test_hmm.c | 437 +++++++++++++++++++++----
lib/test_hmm_uapi.h | 3 +
mm/huge_memory.c | 147 +++++++--
mm/memcontrol.c | 25 +-
mm/memory.c | 10 +-
mm/memremap.c | 4 +-
mm/migrate.c | 429 +++++++++++++++++++-----
mm/rmap.c | 2 +-
tools/testing/selftests/vm/hmm-tests.c | 404 +++++++++++++++++++++++
16 files changed, 1522 insertions(+), 275 deletions(-)
--
2.20.1
This patchset provides support for the SRv6 End.DT4 and End.DT6 (VRF mode)
behaviors.
The SRv6 End.DT4 behavior is used to implement multi-tenant IPv4 L3 VPNs. It
decapsulates the received packets and performs IPv4 routing lookup in the
routing table of the tenant. The SRv6 End.DT4 Linux implementation leverages a
VRF device in order to force the routing lookup into the associated routing
table.
The SRv6 End.DT4 behavior is defined in the SRv6 Network Programming [1].
The Linux kernel already offers an implementation of the SRv6 End.DT6 behavior
which allows us to set up IPv6 L3 VPNs over SRv6 networks. This new
implementation of DT6 is based on the same VRF infrastructure already exploited
for implementing the SRv6 End.DT4 behavior. The aim of the new SRv6 End.DT6 in
VRF mode consists in simplifying the construction of IPv6 L3 VPN services in
the multi-tenant environment.
Currently, the two SRv6 End.DT6 implementations (legacy and VRF mode)
coexist seamlessly and can be chosen according to the context and the user
preferences.
- Patch 1 is needed to solve a pre-existing issue with tunneled packets
when a sniffer is attached;
- Patch 2 improves the management of the seg6local attributes used by the
SRv6 behaviors;
- Patch 3 adds support for optional attributes in SRv6 behaviors;
- Patch 4 introduces two callbacks used for customizing the
creation/destruction of a SRv6 behavior;
- Patch 5 is the core patch that adds support for the SRv6 End.DT4
behavior;
- Patch 6 introduces the VRF support for SRv6 End.DT6 behavior;
- Patch 7 adds the selftest for SRv6 End.DT4 behavior;
- Patch 8 adds the selftest for SRv6 End.DT6 (VRF mode) behavior.
Regarding iproute2, the support for the new "vrftable" attribute, required by
both SRv6 End.DT4 and End.DT6 (VRF mode) behaviors, is provided in a different
patchset that will follow shortly.
I would like to thank David Ahern for his support during the development of
this patchset.
Comments, suggestions and improvements are very welcome!
Thanks,
Andrea Mayer
v4
seg6: add support for the SRv6 End.DT4 behavior
- remove IS_ERR() checks in cmp_nla_vrftable(), thanks to Jakub Kicinski.
remove patch for iproute2:
- mixing the iproute2 patch with this patchset confused patchwork.
v3
notes about the build bot:
- apparently the ',' (comma) in the subject prefix confused the build bot.
Removed the ',' in favor of ' ' (space).
Thanks to David Ahern and Konstantin Ryabitsev for shedding light on this
fact.
Thanks also to Nathan Chancellor for trying to build the patchset v2 by
simulating the bot issue.
add new patch for iproute2:
- [9/9] seg6: add support for vrftable attribute in End.DT4/DT6 behaviors
add new patch:
- [8/9] selftests: add selftest for the SRv6 End.DT6 (VRF) behavior
add new patch:
- [6/9] seg6: add VRF support for SRv6 End.DT6 behavior
add new patch:
- [3/9] seg6: add support for optional attributes in SRv6 behaviors
selftests: add selftest for the SRv6 End.DT4 behavior
- keep David Ahern's review tag since the code wasn't changed. Thanks to David
Ahern for his review.
seg6: add support for the SRv6 End.DT4 behavior
- remove useless error in seg6_end_dt4_build();
- remove #ifdef/#endif stubs for DT4 when CONFIG_NET_L3_MASTER_DEV is not
defined;
- fix coding style.
Thanks to Jakub Kicinski for his review and for all his suggestions.
seg6: add callbacks for customizing the creation/destruction of a behavior
- remove typedef(s) slwt_{build/destroy}_state_t;
- fix coding style: remove empty lines, trivial comments and rename labels in
the seg6_local_build_state() function.
Thanks to Jakub Kicinski for his review and for all his suggestions.
seg6: improve management of behavior attributes
- remove defensive programming approach in destroy_attr_srh(),
destroy_attr_bpf() and destroy_attrs();
- change the __destroy_attrs() function signature, renaming the 'end' argument
'parsed_max'. Now, the __destroy_attrs() keeps only the 'parsed_max' and
'slwt' arguments.
Thanks to Jakub Kicinski for his review and for all his suggestions.
vrf: add mac header for tunneled packets when sniffer is attached
- keep David Ahern's review tag since the code wasn't changed.
Thanks to Jakub Kicinski for pointing it out and David Ahern for his review.
v2
no changes made: resubmitted after false build report.
v1
improve comments;
add new patch 2/5 titled: seg6: improve management of behavior attributes
seg6: add support for the SRv6 End.DT4 behavior
- remove the inline keyword in the definition of fib6_config_get_net().
selftests: add selftest for the SRv6 End.DT4 behavior
- add check for the vrf sysctl
[1] https://tools.ietf.org/html/draft-ietf-spring-srv6-network-programming
Andrea Mayer (8):
vrf: add mac header for tunneled packets when sniffer is attached
seg6: improve management of behavior attributes
seg6: add support for optional attributes in SRv6 behaviors
seg6: add callbacks for customizing the creation/destruction of a
behavior
seg6: add support for the SRv6 End.DT4 behavior
seg6: add VRF support for SRv6 End.DT6 behavior
selftests: add selftest for the SRv6 End.DT4 behavior
selftests: add selftest for the SRv6 End.DT6 (VRF) behavior
drivers/net/vrf.c | 78 ++-
include/uapi/linux/seg6_local.h | 1 +
net/ipv6/seg6_local.c | 590 +++++++++++++++++-
.../selftests/net/srv6_end_dt4_l3vpn_test.sh | 494 +++++++++++++++
.../selftests/net/srv6_end_dt6_l3vpn_test.sh | 502 +++++++++++++++
5 files changed, 1646 insertions(+), 19 deletions(-)
create mode 100755 tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh
create mode 100755 tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh
--
2.20.1
The error handling in hugetlb_allocate_area() was incorrect for the
hugetlb_shared test case.
Previously the behavior was:
- mmap a hugetlb area
- If this fails, set the pointer to NULL, and carry on
- mmap an alias of the same hugetlb fd
- If this fails, munmap the original area
If the original mmap failed, it's likely the second one did too. If
both failed, we'd blindly try to munmap a NULL pointer, causing a
SIGSEGV. Instead, "goto fail" so we return before trying to mmap the
alias.
This issue can be hit "in real life" by forgetting to set
/proc/sys/vm/nr_hugepages (leaving it at 0), and then trying to run the
hugetlb_shared test.
Another small improvement is, when the original mmap fails, don't just
print "it failed": perror(), so we can see *why*. :)
Signed-off-by: Axel Rasmussen <axelrasmussen(a)google.com>
---
tools/testing/selftests/vm/userfaultfd.c | 25 +++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 9b0912a01777..c4425597769a 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -206,19 +206,19 @@ static int hugetlb_release_pages(char *rel_area)
return ret;
}
-
static void hugetlb_allocate_area(void **alloc_area)
{
void *area_alias = NULL;
char **alloc_area_alias;
+
*alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
MAP_HUGETLB,
huge_fd, *alloc_area == area_src ? 0 :
nr_pages * page_size);
if (*alloc_area == MAP_FAILED) {
- fprintf(stderr, "mmap of hugetlbfs file failed\n");
- *alloc_area = NULL;
+ perror("mmap of hugetlbfs file failed");
+ goto fail;
}
if (map_shared) {
@@ -227,14 +227,11 @@ static void hugetlb_allocate_area(void **alloc_area)
huge_fd, *alloc_area == area_src ? 0 :
nr_pages * page_size);
if (area_alias == MAP_FAILED) {
- if (munmap(*alloc_area, nr_pages * page_size) < 0) {
- perror("hugetlb munmap");
- exit(1);
- }
- *alloc_area = NULL;
- return;
+ perror("mmap of hugetlb file alias failed");
+ goto fail_munmap;
}
}
+
if (*alloc_area == area_src) {
huge_fd_off0 = *alloc_area;
alloc_area_alias = &area_src_alias;
@@ -243,6 +240,16 @@ static void hugetlb_allocate_area(void **alloc_area)
}
if (area_alias)
*alloc_area_alias = area_alias;
+
+ return;
+
+fail_munmap:
+ if (munmap(*alloc_area, nr_pages * page_size) < 0) {
+ perror("hugetlb munmap");
+ exit(1);
+ }
+fail:
+ *alloc_area = NULL;
}
static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
--
2.29.2.576.ga3fc446d84-goog
From: "Maciej S. Szmigiero" <maciej.szmigiero(a)oracle.com>
The current memory region move test correctly handles the situation that
the second (realigning) memslot move operation would temporarily trigger
MMIO until it completes, however it does not handle the case in which the
first (misaligning) move operation does this, too.
This results in false test assertions in case it does so.
Fix this by handling temporary MMIO from the first memslot move operation
in the test guest code, too.
Fixes: 8a0639fe9201 ("KVM: sefltests: Add explicit synchronization to move mem region test")
Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero(a)oracle.com>
---
The race is pretty hard to trigger on the current KVM memslot code,
to trigger it reliably an extra delay in memslot move op is needed:
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1173,7 +1173,7 @@ static struct kvm_memslots *kvm_dup_memslots(struct kvm_memslots *old,
return slots;
}
-
+#include <linux/delay.h>
static int kvm_set_memslot(struct kvm *kvm,
const struct kvm_userspace_memory_region *mem,
struct kvm_memory_slot *old,
@@ -1212,6 +1212,8 @@ static int kvm_set_memslot(struct kvm *kvm,
* - kvm_is_visible_gfn (mmu_check_root)
*/
kvm_arch_flush_shadow_memslot(kvm, slot);
+
+ if (change == KVM_MR_MOVE) mdelay(100);
}
r = kvm_arch_prepare_memory_region(kvm, new, mem, change);
.../selftests/kvm/set_memory_region_test.c | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/kvm/set_memory_region_test.c b/tools/testing/selftests/kvm/set_memory_region_test.c
index b3ece55a2da6..6f441dd9f33c 100644
--- a/tools/testing/selftests/kvm/set_memory_region_test.c
+++ b/tools/testing/selftests/kvm/set_memory_region_test.c
@@ -156,14 +156,23 @@ static void guest_code_move_memory_region(void)
GUEST_SYNC(0);
/*
- * Spin until the memory region is moved to a misaligned address. This
- * may or may not trigger MMIO, as the window where the memslot is
- * invalid is quite small.
+ * Spin until the memory region starts getting moved to a
+ * misaligned address.
+ * Every region move may or may not trigger MMIO, as the
+ * window where the memslot is invalid is usually quite small.
*/
val = guest_spin_on_val(0);
GUEST_ASSERT_1(val == 1 || val == MMIO_VAL, val);
- /* Spin until the memory region is realigned. */
+ /* Spin until the misaligning memory region move completes. */
+ val = guest_spin_on_val(MMIO_VAL);
+ GUEST_ASSERT_1(val == 1 || val == 0, val);
+
+ /* Spin until the memory region starts to get re-aligned. */
+ val = guest_spin_on_val(0);
+ GUEST_ASSERT_1(val == 1 || val == MMIO_VAL, val);
+
+ /* Spin until the re-aligning memory region move completes. */
val = guest_spin_on_val(MMIO_VAL);
GUEST_ASSERT_1(val == 1, val);
Implementation of support for parameterized testing in KUnit. This
approach requires the creation of a test case using the
KUNIT_CASE_PARAM() macro that accepts a generator function as input.
This generator function should return the next parameter given the
previous parameter in parameterized tests. It also provides a macro to
generate common-case generators based on arrays. Generators may also
optionally provide a human-readable description of parameters, which is
displayed where available.
Note, currently the result of each parameter run is displayed in
diagnostic lines, and only the overall test case output summarizes
TAP-compliant success or failure of all parameter runs. In future, when
supported by kunit-tool, these can be turned into subsubtest outputs.
Signed-off-by: Arpitha Raghunandan <98.arpi(a)gmail.com>
Co-developed-by: Marco Elver <elver(a)google.com>
Signed-off-by: Marco Elver <elver(a)google.com>
---
Changes v8->v9:
- No change to this patch of the patch series
Changes v7->v8:
- Increase KUNIT_PARAM_DESC_SIZE to 128
- Format pointer style appropriately
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
- Before execution of parameterized test case, count number of
parameters and display number of parameters. Currently also as a
diagnostic line, but this may be used in future to generate a subsubtest
plan. A requirement of this change is that generators must generate a
deterministic number of parameters.
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
include/kunit/test.h | 51 ++++++++++++++++++++++++++++++++++++++
lib/kunit/test.c | 59 ++++++++++++++++++++++++++++++++++----------
2 files changed, 97 insertions(+), 13 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index db1b0ae666c4..27b42a008c7a 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -94,6 +94,9 @@ struct kunit;
/* Size of log associated with test. */
#define KUNIT_LOG_SIZE 512
+/* Maximum size of parameter description string. */
+#define KUNIT_PARAM_DESC_SIZE 128
+
/*
* TAP specifies subtest stream indentation of 4 spaces, 8 spaces for a
* sub-subtest. See the "Subtests" section in
@@ -107,6 +110,7 @@ struct kunit;
*
* @run_case: the function representing the actual test case.
* @name: the name of the test case.
+ * @generate_params: the generator function for parameterized tests.
*
* A test case is a function with the signature,
* ``void (*)(struct kunit *)``
@@ -141,6 +145,7 @@ struct kunit;
struct kunit_case {
void (*run_case)(struct kunit *test);
const char *name;
+ const void* (*generate_params)(const void *prev, char *desc);
/* private: internal use only. */
bool success;
@@ -163,6 +168,27 @@ static inline char *kunit_status_to_string(bool status)
*/
#define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name }
+/**
+ * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case
+ *
+ * @test_name: a reference to a test case function.
+ * @gen_params: a reference to a parameter generator function.
+ *
+ * The generator function::
+ *
+ * const void* gen_params(const void *prev, char *desc)
+ *
+ * is used to lazily generate a series of arbitrarily typed values that fit into
+ * a void*. The argument @prev is the previously returned value, which should be
+ * used to derive the next value; @prev is set to NULL on the initial generator
+ * call. When no more values are available, the generator must return NULL.
+ * Optionally write a string into @desc (size of KUNIT_PARAM_DESC_SIZE)
+ * describing the parameter.
+ */
+#define KUNIT_CASE_PARAM(test_name, gen_params) \
+ { .run_case = test_name, .name = #test_name, \
+ .generate_params = gen_params }
+
/**
* struct kunit_suite - describes a related collection of &struct kunit_case
*
@@ -208,6 +234,10 @@ struct kunit {
const char *name; /* Read only after initialization! */
char *log; /* Points at case log after initialization */
struct kunit_try_catch try_catch;
+ /* param_value is the current parameter value for a test case. */
+ const void *param_value;
+ /* param_index stores the index of the parameter in parameterized tests. */
+ int param_index;
/*
* success starts as true, and may only be set to false during a
* test case; thus, it is safe to update this across multiple
@@ -1742,4 +1772,25 @@ do { \
fmt, \
##__VA_ARGS__)
+/**
+ * KUNIT_ARRAY_PARAM() - Define test parameter generator from an array.
+ * @name: prefix for the test parameter generator function.
+ * @array: array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses @array to generate parameters.
+ */
+#define KUNIT_ARRAY_PARAM(name, array, get_desc) \
+ static const void *name##_gen_params(const void *prev, char *desc) \
+ { \
+ typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
+ if (__next - (array) < ARRAY_SIZE((array))) { \
+ void (*__get_desc)(typeof(__next), char *) = get_desc; \
+ if (__get_desc) \
+ __get_desc(__next, desc); \
+ return __next; \
+ } \
+ return NULL; \
+ }
+
#endif /* _KUNIT_TEST_H */
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 750704abe89a..ec9494e914ef 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -325,39 +325,72 @@ static void kunit_catch_run_case(void *data)
* occur in a test case and reports them as failures.
*/
static void kunit_run_case_catch_errors(struct kunit_suite *suite,
- struct kunit_case *test_case)
+ struct kunit_case *test_case,
+ struct kunit *test)
{
struct kunit_try_catch_context context;
struct kunit_try_catch *try_catch;
- struct kunit test;
- kunit_init_test(&test, test_case->name, test_case->log);
- try_catch = &test.try_catch;
+ kunit_init_test(test, test_case->name, test_case->log);
+ try_catch = &test->try_catch;
kunit_try_catch_init(try_catch,
- &test,
+ test,
kunit_try_run_case,
kunit_catch_run_case);
- context.test = &test;
+ context.test = test;
context.suite = suite;
context.test_case = test_case;
kunit_try_catch_run(try_catch, &context);
- test_case->success = test.success;
-
- kunit_print_ok_not_ok(&test, true, test_case->success,
- kunit_test_case_num(suite, test_case),
- test_case->name);
+ test_case->success = test->success;
}
int kunit_run_tests(struct kunit_suite *suite)
{
+ char param_desc[KUNIT_PARAM_DESC_SIZE];
struct kunit_case *test_case;
kunit_print_subtest_start(suite);
- kunit_suite_for_each_test_case(suite, test_case)
- kunit_run_case_catch_errors(suite, test_case);
+ kunit_suite_for_each_test_case(suite, test_case) {
+ struct kunit test = { .param_value = NULL, .param_index = 0 };
+ bool test_success = true;
+
+ if (test_case->generate_params) {
+ /* Get initial param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(NULL, param_desc);
+ }
+
+ do {
+ kunit_run_case_catch_errors(suite, test_case, &test);
+ test_success &= test_case->success;
+
+ if (test_case->generate_params) {
+ if (param_desc[0] == '\0') {
+ snprintf(param_desc, sizeof(param_desc),
+ "param-%d", test.param_index);
+ }
+
+ kunit_log(KERN_INFO, &test,
+ KUNIT_SUBTEST_INDENT
+ "# %s: %s %d - %s",
+ test_case->name,
+ kunit_status_to_string(test.success),
+ test.param_index + 1, param_desc);
+
+ /* Get next param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(test.param_value, param_desc);
+ test.param_index++;
+ }
+ } while (test.param_value);
+
+ kunit_print_ok_not_ok(&test, true, test_success,
+ kunit_test_case_num(suite, test_case),
+ test_case->name);
+ }
kunit_print_subtest_end(suite);
--
2.25.1
On Wed, Oct 28, 2020 at 10:50:42AM +1100, Aleksa Sarai wrote:
> This was an oversight in the original implementation, as it makes no
> sense to specify both scoping flags to the same openat2(2) invocation
> (before this patch, the result of such an invocation was equivalent to
> RESOLVE_IN_ROOT being ignored).
>
> This is a userspace-visible ABI change, but the only user of openat2(2)
> at the moment is LXC which doesn't specify both flags and so no
> userspace programs will break as a result.
>
> Changelog:
> v2: Split patch so as to separate selftest changes. [Shuah Khan]
> v1: <https://lore.kernel.org/lkml/20201007103608.17349-1-cyphar@cyphar.com/>
>
> Aleksa Sarai (2):
> openat2: reject RESOLVE_BENEATH|RESOLVE_IN_ROOT
> selftests: openat2: add RESOLVE_ conflict test
>
> fs/open.c | 4 ++++
> tools/testing/selftests/openat2/openat2_test.c | 8 +++++++-
> 2 files changed, 11 insertions(+), 1 deletion(-)
I've applied this patchset now. There's no need to have this sit around
another merge window. I'm happy to drop it again in case you're picking
it up later, Al.
Thanks!
Christian
Revert commit cebc04ba9aeb ("add CONFIG_ENABLE_MUST_CHECK").
A lot of warn_unused_result warnings existed in 2006, but until now
they have been fixed thanks to people doing allmodconfig tests.
Our goal is to always enable __must_check where appropriate, so this
CONFIG option is no longer needed.
I see a lot of defconfig (arch/*/configs/*_defconfig) files having:
# CONFIG_ENABLE_MUST_CHECK is not set
I did not touch them for now since it would be a big churn. If arch
maintainers want to clean them up, please go ahead.
While I was here, I also moved __must_check to compiler_attributes.h
from compiler_types.h
Signed-off-by: Masahiro Yamada <masahiroy(a)kernel.org>
Acked-by: Jason A. Donenfeld <Jason(a)zx2c4.com>
---
Changes in v3:
- Fix a typo
Changes in v2:
- Move __must_check to compiler_attributes.h
include/linux/compiler_attributes.h | 7 +++++++
include/linux/compiler_types.h | 6 ------
lib/Kconfig.debug | 8 --------
tools/testing/selftests/wireguard/qemu/debug.config | 1 -
4 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/include/linux/compiler_attributes.h b/include/linux/compiler_attributes.h
index b2a3f4f641a7..5f3b7edad1a7 100644
--- a/include/linux/compiler_attributes.h
+++ b/include/linux/compiler_attributes.h
@@ -171,6 +171,13 @@
*/
#define __mode(x) __attribute__((__mode__(x)))
+/*
+ * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-wa…
+ * clang: https://clang.llvm.org/docs/AttributeReference.html#nodiscard-warn-unused-r…
+ *
+ */
+#define __must_check __attribute__((__warn_unused_result__))
+
/*
* Optional: only supported since gcc >= 7
*
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index ac3fa37a84f9..7ef20d1a6c28 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -110,12 +110,6 @@ struct ftrace_likely_data {
unsigned long constant;
};
-#ifdef CONFIG_ENABLE_MUST_CHECK
-#define __must_check __attribute__((__warn_unused_result__))
-#else
-#define __must_check
-#endif
-
#if defined(CC_USING_HOTPATCH)
#define notrace __attribute__((hotpatch(0, 0)))
#elif defined(CC_USING_PATCHABLE_FUNCTION_ENTRY)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c789b39ed527..cb8ef4fd0d02 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -286,14 +286,6 @@ config GDB_SCRIPTS
endif # DEBUG_INFO
-config ENABLE_MUST_CHECK
- bool "Enable __must_check logic"
- default y
- help
- Enable the __must_check logic in the kernel build. Disable this to
- suppress the "warning: ignoring return value of 'foo', declared with
- attribute warn_unused_result" messages.
-
config FRAME_WARN
int "Warn for stack frames larger than"
range 0 8192
diff --git a/tools/testing/selftests/wireguard/qemu/debug.config b/tools/testing/selftests/wireguard/qemu/debug.config
index b50c2085c1ac..fe07d97df9fa 100644
--- a/tools/testing/selftests/wireguard/qemu/debug.config
+++ b/tools/testing/selftests/wireguard/qemu/debug.config
@@ -1,5 +1,4 @@
CONFIG_LOCALVERSION="-debug"
-CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_POINTER=y
CONFIG_STACK_VALIDATION=y
CONFIG_DEBUG_KERNEL=y
--
2.27.0
Kernel-doc has always be limited to a probably bad documented
rule:
The kernel-doc markups should appear *imediatelly before* the
function or data structure that it documents.
On other words, if a C file would contain something like this:
/**
* foo - function foo
* @args: foo args
*/
static inline void bar(int args);
/**
* bar - function bar
* @args: foo args
*/
static inline void foo(void *args);
The output (in ReST format) will be:
.. c:function:: void bar (int args)
function foo
**Parameters**
``int args``
foo args
.. c:function:: void foo (void *args)
function bar
**Parameters**
``void *args``
foo args
Which is clearly a wrong result. Before this changeset,
not even a warning is produced on such cases.
As placing such markups just before the documented
data is a common practice, on most cases this is fine.
However, as patches touch things, identifiers may be
renamed, and people may forget to update the kernel-doc
markups to follow such changes.
This has been happening for quite a while, as there are
lots of files with kernel-doc problems.
This series address those issues and add a file at the
end that will enforce that the identifier will match the
kernel-doc markup, avoiding this problem from
keep happening as time goes by.
This series is based on current upstream tree.
@maintainers: feel free to pick the patches and
apply them directly on your trees, as all patches on
this series are independent from the other ones.
--
v5:
- The completion.h patch was replaced by another one which drops
an obsolete macro;
- Some typos got fixed and review tags got added;
- Dropped patches that were already merged at linux-next.
v4:
- Patches got rebased and got some acks.
Mauro Carvalho Chehab (16):
HSI: fix a kernel-doc markup
IB: fix kernel-doc markups
parport: fix a kernel-doc markup
rapidio: fix kernel-doc a markup
fs: fix kernel-doc markups
pstore/zone: fix a kernel-doc markup
completion: drop init_completion define
firmware: stratix10-svc: fix kernel-doc markups
connector: fix a kernel-doc markup
lib/crc7: fix a kernel-doc markup
memblock: fix kernel-doc markups
w1: fix a kernel-doc markup
sched: fix kernel-doc markup
selftests: kselftest_harness.h: partially fix kernel-doc markups
refcount.h: fix a kernel-doc markup
scripts: kernel-doc: validate kernel-doc markup with the actual names
drivers/hsi/hsi_core.c | 2 +-
drivers/infiniband/core/cm.c | 5 +-
drivers/infiniband/core/cq.c | 4 +-
drivers/infiniband/core/iwpm_util.h | 2 +-
drivers/infiniband/core/sa_query.c | 3 +-
drivers/infiniband/core/verbs.c | 4 +-
drivers/infiniband/sw/rdmavt/ah.c | 2 +-
drivers/infiniband/sw/rdmavt/mcast.c | 12 ++--
drivers/infiniband/sw/rdmavt/qp.c | 8 +--
drivers/infiniband/ulp/iser/iscsi_iser.c | 2 +-
.../infiniband/ulp/opa_vnic/opa_vnic_encap.h | 2 +-
.../ulp/opa_vnic/opa_vnic_vema_iface.c | 2 +-
drivers/infiniband/ulp/srpt/ib_srpt.h | 2 +-
drivers/parport/share.c | 2 +-
drivers/rapidio/rio.c | 2 +-
fs/dcache.c | 72 +++++++++----------
fs/inode.c | 4 +-
fs/pstore/zone.c | 2 +-
fs/seq_file.c | 5 +-
fs/super.c | 12 ++--
include/linux/completion.h | 5 +-
include/linux/connector.h | 2 +-
.../firmware/intel/stratix10-svc-client.h | 10 +--
include/linux/memblock.h | 4 +-
include/linux/parport.h | 31 ++++++++
include/linux/refcount.h | 2 +-
include/linux/w1.h | 2 +-
include/rdma/ib_verbs.h | 11 +++
kernel/sched/core.c | 16 ++---
kernel/sched/fair.c | 2 +-
lib/crc7.c | 2 +-
scripts/kernel-doc | 62 +++++++++++-----
tools/testing/selftests/kselftest_harness.h | 22 +++---
33 files changed, 197 insertions(+), 123 deletions(-)
--
2.28.0
Hi,
This patch series mainly extend Landlock rules to store the whole access
rights stack. This enables to tie access rights with their respective
layers to be able to have a sane semantic regardless of the previous
enforced rulesets. This also enables to get back the union of access
rights when building a ruleset. See layout1.interleaved_masked_accesses
tests from tools/testing/selftests/landlock/fs_test.c for corner cases.
Cf.
https://lore.kernel.org/lkml/CAG48ez2cmsrZbUEmQmzPQugJikkvfs_MWmMizxmoyspCe…
The SLOC count is 1260 for security/landlock/ and 1711 for
tools/testing/selftest/landlock/ . Test coverage for security/landlock/
is 94% of lines. The code not covered only deals with internal kernel
errors (e.g. memory allocation) and race conditions.
The compiled documentation is available here:
https://landlock.io/linux-doc/landlock-v25/userspace-api/landlock.html
This series can be applied on top of v5.10-rc6 . This can be tested
with CONFIG_SECURITY_LANDLOCK, CONFIG_SAMPLE_LANDLOCK and by prepending
"landlock," to CONFIG_LSM. This patch series can be found in a Git
repository here:
https://github.com/landlock-lsm/linux/commits/landlock-v25
I would really appreciate constructive comments on this patch series.
# Landlock LSM
The goal of Landlock is to enable to restrict ambient rights (e.g.
global filesystem access) for a set of processes. Because Landlock is a
stackable LSM [1], it makes possible to create safe security sandboxes
as new security layers in addition to the existing system-wide
access-controls. This kind of sandbox is expected to help mitigate the
security impact of bugs or unexpected/malicious behaviors in user-space
applications. Landlock empowers any process, including unprivileged
ones, to securely restrict themselves.
Landlock is inspired by seccomp-bpf but instead of filtering syscalls
and their raw arguments, a Landlock rule can restrict the use of kernel
objects like file hierarchies, according to the kernel semantic.
Landlock also takes inspiration from other OS sandbox mechanisms: XNU
Sandbox, FreeBSD Capsicum or OpenBSD Pledge/Unveil.
In this current form, Landlock misses some access-control features.
This enables to minimize this patch series and ease review. This series
still addresses multiple use cases, especially with the combined use of
seccomp-bpf: applications with built-in sandboxing, init systems,
security sandbox tools and security-oriented APIs [2].
Previous version:
https://lore.kernel.org/lkml/20201112205141.775752-1-mic@digikod.net/
[1] https://lore.kernel.org/lkml/50db058a-7dde-441b-a7f9-f6837fe8b69f@schaufler…
[2] https://lore.kernel.org/lkml/f646e1c7-33cf-333f-070c-0a40ad0468cd@digikod.n…
Casey Schaufler (1):
LSM: Infrastructure management of the superblock
Mickaël Salaün (11):
landlock: Add object management
landlock: Add ruleset and domain management
landlock: Set up the security framework and manage credentials
landlock: Add ptrace restrictions
fs,security: Add sb_delete hook
landlock: Support filesystem access-control
landlock: Add syscall implementations
arch: Wire up Landlock syscalls
selftests/landlock: Add user space tests
samples/landlock: Add a sandbox manager example
landlock: Add user and kernel documentation
Documentation/security/index.rst | 1 +
Documentation/security/landlock.rst | 79 +
Documentation/userspace-api/index.rst | 1 +
Documentation/userspace-api/landlock.rst | 280 +++
MAINTAINERS | 13 +
arch/Kconfig | 7 +
arch/alpha/kernel/syscalls/syscall.tbl | 3 +
arch/arm/tools/syscall.tbl | 3 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 6 +
arch/ia64/kernel/syscalls/syscall.tbl | 3 +
arch/m68k/kernel/syscalls/syscall.tbl | 3 +
arch/microblaze/kernel/syscalls/syscall.tbl | 3 +
arch/mips/kernel/syscalls/syscall_n32.tbl | 3 +
arch/mips/kernel/syscalls/syscall_n64.tbl | 3 +
arch/mips/kernel/syscalls/syscall_o32.tbl | 3 +
arch/parisc/kernel/syscalls/syscall.tbl | 3 +
arch/powerpc/kernel/syscalls/syscall.tbl | 3 +
arch/s390/kernel/syscalls/syscall.tbl | 3 +
arch/sh/kernel/syscalls/syscall.tbl | 3 +
arch/sparc/kernel/syscalls/syscall.tbl | 3 +
arch/um/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 3 +
arch/x86/entry/syscalls/syscall_64.tbl | 3 +
arch/xtensa/kernel/syscalls/syscall.tbl | 3 +
fs/super.c | 1 +
include/linux/lsm_hook_defs.h | 1 +
include/linux/lsm_hooks.h | 3 +
include/linux/security.h | 4 +
include/linux/syscalls.h | 7 +
include/uapi/asm-generic/unistd.h | 8 +-
include/uapi/linux/landlock.h | 128 ++
kernel/sys_ni.c | 5 +
samples/Kconfig | 7 +
samples/Makefile | 1 +
samples/landlock/.gitignore | 1 +
samples/landlock/Makefile | 15 +
samples/landlock/sandboxer.c | 236 +++
security/Kconfig | 11 +-
security/Makefile | 2 +
security/landlock/Kconfig | 21 +
security/landlock/Makefile | 4 +
security/landlock/common.h | 20 +
security/landlock/cred.c | 46 +
security/landlock/cred.h | 58 +
security/landlock/fs.c | 635 ++++++
security/landlock/fs.h | 60 +
security/landlock/object.c | 67 +
security/landlock/object.h | 91 +
security/landlock/ptrace.c | 120 ++
security/landlock/ptrace.h | 14 +
security/landlock/ruleset.c | 427 ++++
security/landlock/ruleset.h | 163 ++
security/landlock/setup.c | 40 +
security/landlock/setup.h | 18 +
security/landlock/syscall.c | 426 ++++
security/security.c | 51 +-
security/selinux/hooks.c | 58 +-
security/selinux/include/objsec.h | 6 +
security/selinux/ss/services.c | 3 +-
security/smack/smack.h | 6 +
security/smack/smack_lsm.c | 35 +-
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/landlock/.gitignore | 2 +
tools/testing/selftests/landlock/Makefile | 24 +
tools/testing/selftests/landlock/base_test.c | 117 ++
tools/testing/selftests/landlock/common.h | 113 ++
tools/testing/selftests/landlock/config | 5 +
tools/testing/selftests/landlock/fs_test.c | 1798 +++++++++++++++++
.../testing/selftests/landlock/ptrace_test.c | 307 +++
tools/testing/selftests/landlock/true.c | 5 +
71 files changed, 5532 insertions(+), 77 deletions(-)
create mode 100644 Documentation/security/landlock.rst
create mode 100644 Documentation/userspace-api/landlock.rst
create mode 100644 include/uapi/linux/landlock.h
create mode 100644 samples/landlock/.gitignore
create mode 100644 samples/landlock/Makefile
create mode 100644 samples/landlock/sandboxer.c
create mode 100644 security/landlock/Kconfig
create mode 100644 security/landlock/Makefile
create mode 100644 security/landlock/common.h
create mode 100644 security/landlock/cred.c
create mode 100644 security/landlock/cred.h
create mode 100644 security/landlock/fs.c
create mode 100644 security/landlock/fs.h
create mode 100644 security/landlock/object.c
create mode 100644 security/landlock/object.h
create mode 100644 security/landlock/ptrace.c
create mode 100644 security/landlock/ptrace.h
create mode 100644 security/landlock/ruleset.c
create mode 100644 security/landlock/ruleset.h
create mode 100644 security/landlock/setup.c
create mode 100644 security/landlock/setup.h
create mode 100644 security/landlock/syscall.c
create mode 100644 tools/testing/selftests/landlock/.gitignore
create mode 100644 tools/testing/selftests/landlock/Makefile
create mode 100644 tools/testing/selftests/landlock/base_test.c
create mode 100644 tools/testing/selftests/landlock/common.h
create mode 100644 tools/testing/selftests/landlock/config
create mode 100644 tools/testing/selftests/landlock/fs_test.c
create mode 100644 tools/testing/selftests/landlock/ptrace_test.c
create mode 100644 tools/testing/selftests/landlock/true.c
base-commit: b65054597872ce3aefbc6a666385eabdf9e288da
--
2.29.2
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
@Andrew, this is based on v5.10-rc2-mmotm-2020-11-07-21-40, I can rebase on
current mmotm if you prefer.
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v13:
* Added Reviewed-by, thanks Catalin and David
* s/mod_node_page_state/mod_lruvec_page_state/ as Shakeel suggested
v12: https://lore.kernel.org/lkml/20201125092208.12544-1-rppt@kernel.org
* Add detection of whether set_direct_map has actual effect on arm64 and bail
out of CMA allocation for secretmem and the memfd_secret() syscall if pages
would not be removed from the direct map
v11: https://lore.kernel.org/lkml/20201124092556.12009-1-rppt@kernel.org
* Drop support for uncached mappings
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
* Drop changes to arm64 compatibility layer
* Add Roman's Ack for memcg accounting
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
* Fix build with and without CONFIG_MEMCG
* Update memcg accounting to avoid copying memcg_data, per Roman comments
* Fix issues in secretmem_fault(), thanks Matthew
* Do not wire up syscall in arm64 compatibility layer
Older history:
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (10):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
set_memory: allow querying whether set_direct_map_*() is actually enabled
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/cacheflush.h | 6 -
arch/arm64/include/asm/set_memory.h | 17 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/kernel/machine_kexec.c | 1 +
arch/arm64/mm/mmu.c | 6 +-
arch/arm64/mm/pageattr.c | 23 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 2 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 16 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 439 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 906 insertions(+), 51 deletions(-)
create mode 100644 arch/arm64/include/asm/set_memory.h
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
Implementation of support for parameterized testing in KUnit. This
approach requires the creation of a test case using the
KUNIT_CASE_PARAM() macro that accepts a generator function as input.
This generator function should return the next parameter given the
previous parameter in parameterized tests. It also provides a macro to
generate common-case generators based on arrays. Generators may also
optionally provide a human-readable description of parameters, which is
displayed where available.
Note, currently the result of each parameter run is displayed in
diagnostic lines, and only the overall test case output summarizes
TAP-compliant success or failure of all parameter runs. In future, when
supported by kunit-tool, these can be turned into subsubtest outputs.
Signed-off-by: Arpitha Raghunandan <98.arpi(a)gmail.com>
Co-developed-by: Marco Elver <elver(a)google.com>
Signed-off-by: Marco Elver <elver(a)google.com>
---
Changes v7->v8:
- Increase KUNIT_PARAM_DESC_SIZE to 128
- Format pointer style appropriately
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
- Before execution of parameterized test case, count number of
parameters and display number of parameters. Currently also as a
diagnostic line, but this may be used in future to generate a subsubtest
plan. A requirement of this change is that generators must generate a
deterministic number of parameters.
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
include/kunit/test.h | 51 ++++++++++++++++++++++++++++++++++++++
lib/kunit/test.c | 59 ++++++++++++++++++++++++++++++++++----------
2 files changed, 97 insertions(+), 13 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index db1b0ae666c4..27b42a008c7a 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -94,6 +94,9 @@ struct kunit;
/* Size of log associated with test. */
#define KUNIT_LOG_SIZE 512
+/* Maximum size of parameter description string. */
+#define KUNIT_PARAM_DESC_SIZE 128
+
/*
* TAP specifies subtest stream indentation of 4 spaces, 8 spaces for a
* sub-subtest. See the "Subtests" section in
@@ -107,6 +110,7 @@ struct kunit;
*
* @run_case: the function representing the actual test case.
* @name: the name of the test case.
+ * @generate_params: the generator function for parameterized tests.
*
* A test case is a function with the signature,
* ``void (*)(struct kunit *)``
@@ -141,6 +145,7 @@ struct kunit;
struct kunit_case {
void (*run_case)(struct kunit *test);
const char *name;
+ const void* (*generate_params)(const void *prev, char *desc);
/* private: internal use only. */
bool success;
@@ -163,6 +168,27 @@ static inline char *kunit_status_to_string(bool status)
*/
#define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name }
+/**
+ * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case
+ *
+ * @test_name: a reference to a test case function.
+ * @gen_params: a reference to a parameter generator function.
+ *
+ * The generator function::
+ *
+ * const void* gen_params(const void *prev, char *desc)
+ *
+ * is used to lazily generate a series of arbitrarily typed values that fit into
+ * a void*. The argument @prev is the previously returned value, which should be
+ * used to derive the next value; @prev is set to NULL on the initial generator
+ * call. When no more values are available, the generator must return NULL.
+ * Optionally write a string into @desc (size of KUNIT_PARAM_DESC_SIZE)
+ * describing the parameter.
+ */
+#define KUNIT_CASE_PARAM(test_name, gen_params) \
+ { .run_case = test_name, .name = #test_name, \
+ .generate_params = gen_params }
+
/**
* struct kunit_suite - describes a related collection of &struct kunit_case
*
@@ -208,6 +234,10 @@ struct kunit {
const char *name; /* Read only after initialization! */
char *log; /* Points at case log after initialization */
struct kunit_try_catch try_catch;
+ /* param_value is the current parameter value for a test case. */
+ const void *param_value;
+ /* param_index stores the index of the parameter in parameterized tests. */
+ int param_index;
/*
* success starts as true, and may only be set to false during a
* test case; thus, it is safe to update this across multiple
@@ -1742,4 +1772,25 @@ do { \
fmt, \
##__VA_ARGS__)
+/**
+ * KUNIT_ARRAY_PARAM() - Define test parameter generator from an array.
+ * @name: prefix for the test parameter generator function.
+ * @array: array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses @array to generate parameters.
+ */
+#define KUNIT_ARRAY_PARAM(name, array, get_desc) \
+ static const void *name##_gen_params(const void *prev, char *desc) \
+ { \
+ typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array); \
+ if (__next - (array) < ARRAY_SIZE((array))) { \
+ void (*__get_desc)(typeof(__next), char *) = get_desc; \
+ if (__get_desc) \
+ __get_desc(__next, desc); \
+ return __next; \
+ } \
+ return NULL; \
+ }
+
#endif /* _KUNIT_TEST_H */
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 750704abe89a..ec9494e914ef 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -325,39 +325,72 @@ static void kunit_catch_run_case(void *data)
* occur in a test case and reports them as failures.
*/
static void kunit_run_case_catch_errors(struct kunit_suite *suite,
- struct kunit_case *test_case)
+ struct kunit_case *test_case,
+ struct kunit *test)
{
struct kunit_try_catch_context context;
struct kunit_try_catch *try_catch;
- struct kunit test;
- kunit_init_test(&test, test_case->name, test_case->log);
- try_catch = &test.try_catch;
+ kunit_init_test(test, test_case->name, test_case->log);
+ try_catch = &test->try_catch;
kunit_try_catch_init(try_catch,
- &test,
+ test,
kunit_try_run_case,
kunit_catch_run_case);
- context.test = &test;
+ context.test = test;
context.suite = suite;
context.test_case = test_case;
kunit_try_catch_run(try_catch, &context);
- test_case->success = test.success;
-
- kunit_print_ok_not_ok(&test, true, test_case->success,
- kunit_test_case_num(suite, test_case),
- test_case->name);
+ test_case->success = test->success;
}
int kunit_run_tests(struct kunit_suite *suite)
{
+ char param_desc[KUNIT_PARAM_DESC_SIZE];
struct kunit_case *test_case;
kunit_print_subtest_start(suite);
- kunit_suite_for_each_test_case(suite, test_case)
- kunit_run_case_catch_errors(suite, test_case);
+ kunit_suite_for_each_test_case(suite, test_case) {
+ struct kunit test = { .param_value = NULL, .param_index = 0 };
+ bool test_success = true;
+
+ if (test_case->generate_params) {
+ /* Get initial param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(NULL, param_desc);
+ }
+
+ do {
+ kunit_run_case_catch_errors(suite, test_case, &test);
+ test_success &= test_case->success;
+
+ if (test_case->generate_params) {
+ if (param_desc[0] == '\0') {
+ snprintf(param_desc, sizeof(param_desc),
+ "param-%d", test.param_index);
+ }
+
+ kunit_log(KERN_INFO, &test,
+ KUNIT_SUBTEST_INDENT
+ "# %s: %s %d - %s",
+ test_case->name,
+ kunit_status_to_string(test.success),
+ test.param_index + 1, param_desc);
+
+ /* Get next param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(test.param_value, param_desc);
+ test.param_index++;
+ }
+ } while (test.param_value);
+
+ kunit_print_ok_not_ok(&test, true, test_success,
+ kunit_test_case_num(suite, test_case),
+ test_case->name);
+ }
kunit_print_subtest_end(suite);
--
2.25.1
usage.rst goes into a detailed section about faking out classes, but
currently lacks wording about how one might idiomatically test a range
of inputs.
Add a new chapter for "Common Patterns" and group "Isolating behvaior"
and this new section under there.
Give an example of how one might test a hash function via macros/helper
funcs and a table-driven test and very briefly discuss pros and cons.
Also highlight the KUNIT_EXPECT_*_MSG() variants (that aren't mentioned
elsewhere [1]) which are particularly useful in these situations.
It is also criminally underused at the moment, only appearing in 2
tests (both written by people involved in KUnit).
[1] not even on
https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Documentation/dev-tools/kunit/usage.rst | 83 +++++++++++++++++++++++--
1 file changed, 77 insertions(+), 6 deletions(-)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 9c28c518e6a3..d9fdc14f0677 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -15,10 +15,10 @@ project, see :doc:`start`.
Organization of this document
=============================
-This document is organized into two main sections: Testing and Isolating
-Behavior. The first covers what unit tests are and how to use KUnit to write
-them. The second covers how to use KUnit to isolate code and make it possible
-to unit test code that was otherwise un-unit-testable.
+This document is organized into two main sections: Testing and Common Patterns.
+The first covers what unit tests are and how to use KUnit to write them. The
+second covers common testing patterns, e.g. how to isolate code and make it
+possible to unit test code that was otherwise un-unit-testable.
Testing
=======
@@ -218,8 +218,11 @@ test was built in or not).
For more information on these types of things see the :doc:`api/test`.
+Common Patterns
+===============
+
Isolating Behavior
-==================
+------------------
The most important aspect of unit testing that other forms of testing do not
provide is the ability to limit the amount of code under test to a single unit.
@@ -233,7 +236,7 @@ implementer, and architecture-specific functions which have definitions selected
at compile time.
Classes
--------
+~~~~~~~
Classes are not a construct that is built into the C programming language;
however, it is an easily derived concept. Accordingly, pretty much every project
@@ -451,6 +454,74 @@ We can now use it to test ``struct eeprom_buffer``:
destroy_eeprom_buffer(ctx->eeprom_buffer);
}
+Testing against multiple inputs
+-------------------------------
+
+Testing just a few inputs might not be enough to have confidence that the code
+works correctly, e.g. for a hash function.
+
+In such cases, it can be helpful to have a helper macro or function, e.g. this
+fictitious example for ``sha1sum(1)``
+
+.. code-block:: c
+
+ /* Note: the cast is to satisfy overly strict type-checking. */
+ #define TEST_SHA1(in, want) \
+ sha1sum(in, out); \
+ KUNIT_EXPECT_STREQ_MSG(test, (char *)out, want, "sha1sum(%s)", in);
+
+ char out[40];
+ TEST_SHA1("hello world", "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
+ TEST_SHA1("hello world!", "430ce34d020724ed75a196dfc2ad67c77772d169");
+
+
+Note the use of ``KUNIT_EXPECT_STREQ_MSG`` to give more context when it fails
+and make it easier to track down. (Yes, in this example, ``want`` is likely
+going to be unique enough on its own).
+
+The ``_MSG`` variants are even more useful when the same expectation is called
+multiple times (in a loop or helper function) and thus the line number isn't
+enough to identify what failed, like below.
+
+In some cases, it can be helpful to write a *table-driven test* instead, e.g.
+
+.. code-block:: c
+
+ int i;
+ char out[40];
+
+ struct sha1_test_case {
+ const char *str;
+ const char *sha1;
+ };
+
+ struct sha1_test_case cases[] = {
+ {
+ .str = "hello world",
+ .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
+ },
+ {
+ .str = "hello world!",
+ .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169",
+ },
+ };
+ for (i = 0; i < ARRAY_SIZE(cases); ++i) {
+ sha1sum(cases[i].str, out);
+ KUNIT_EXPECT_STREQ_MSG(test, (char *)out, cases[i].sha1,
+ "sha1sum(%s)", cases[i].str);
+ }
+
+
+There's more boilerplate involved, but it can:
+
+* be more readable when there are multiple inputs/outputs thanks to field names,
+
+ * E.g. see ``fs/ext4/inode-test.c`` for an example of both.
+* reduce duplication if test cases can be shared across multiple tests.
+
+ * E.g. if we wanted to also test ``sha256sum``, we could add a ``sha256``
+ field and reuse ``cases``.
+
.. _kunit-on-non-uml:
KUnit on non-UML architectures
base-commit: 418baf2c28f3473039f2f7377760bd8f6897ae18
--
2.29.2.454.gaff20da3a2-goog
Currently, kunit_tool expects all diagnostic lines in test results to
contain ": " somewhere, as both the subtest header and the crash report
do. Fix this to accept any line starting with (minus indent) "# " as
being a valid diagnostic line.
This matches what the TAP spec[1] and the draft KTAP spec[2] are
expecting.
[1]: http://testanything.org/tap-specification.html
[2]: https://lore.kernel.org/linux-kselftest/CY4PR13MB1175B804E31E502221BC8163FD…
Signed-off-by: David Gow <davidgow(a)google.com>
---
tools/testing/kunit/kunit_parser.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 84a1af2581f5..dab4cfa05b74 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -134,8 +134,8 @@ def parse_ok_not_ok_test_case(lines: List[str], test_case: TestCase) -> bool:
else:
return False
-SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# .*?: (.*)$')
-DIAGNOSTIC_CRASH_MESSAGE = 'kunit test case crashed!'
+SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# (.*)$')
+DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^[\s]+# .*?: kunit test case crashed!$')
def parse_diagnostic(lines: List[str], test_case: TestCase) -> bool:
save_non_diagnositic(lines, test_case)
@@ -145,7 +145,8 @@ def parse_diagnostic(lines: List[str], test_case: TestCase) -> bool:
match = SUBTEST_DIAGNOSTIC.match(line)
if match:
test_case.log.append(lines.pop(0))
- if match.group(1) == DIAGNOSTIC_CRASH_MESSAGE:
+ crash_match = DIAGNOSTIC_CRASH_MESSAGE.match(line)
+ if crash_match:
test_case.status = TestStatus.TEST_CRASHED
return True
else:
--
2.29.2.222.g5d2a92d10f8-goog
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v12:
* Add detection of whether set_direct_map has actual effect on arm64 and bail
out of CMA allocation for secretmem and the memfd_secret() syscall if pages
would not be removed from the direct map
v11: https://lore.kernel.org/lkml/20201124092556.12009-1-rppt@kernel.org
* Drop support for uncached mappings
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
* Drop changes to arm64 compatibility layer
* Add Roman's Ack for memcg accounting
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
* Fix build with and without CONFIG_MEMCG
* Update memcg accounting to avoid copying memcg_data, per Roman comments
* Fix issues in secretmem_fault(), thanks Matthew
* Do not wire up syscall in arm64 compatibility layer
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
* Use CMA for all secretmem allocations as David suggested
* Update memcg accounting after transtion to CMA
* Prevent hibernation when there are active secretmem users
* Add zeroing of the memory before releasing it back to cma/page allocator
* Rebase on v5.10-rc2-mmotm-2020-11-07-21-40
Older history:
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (10):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
set_memory: allow querying whether set_direct_map_*() is actually enabled
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/cacheflush.h | 6 -
arch/arm64/include/asm/set_memory.h | 17 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/kernel/machine_kexec.c | 1 +
arch/arm64/mm/mmu.c | 6 +-
arch/arm64/mm/pageattr.c | 23 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 2 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 16 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 439 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 906 insertions(+), 51 deletions(-)
create mode 100644 arch/arm64/include/asm/set_memory.h
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
Revert commit cebc04ba9aeb ("add CONFIG_ENABLE_MUST_CHECK").
A lot of warn_unused_result wearings existed in 2006, but until now
they have been fixed thanks to people doing allmodconfig tests.
Our goal is to always enable __must_check where appropriate, so this
CONFIG option is no longer needed.
I see a lot of defconfig (arch/*/configs/*_defconfig) files having:
# CONFIG_ENABLE_MUST_CHECK is not set
I did not touch them for now since it would be a big churn. If arch
maintainers want to clean them up, please go ahead.
While I was here, I also moved __must_check to compiler_attributes.h
from compiler_types.h
Signed-off-by: Masahiro Yamada <masahiroy(a)kernel.org>
---
Changes in v2:
- Move __must_check to compiler_attributes.h
include/linux/compiler_attributes.h | 7 +++++++
include/linux/compiler_types.h | 6 ------
lib/Kconfig.debug | 8 --------
tools/testing/selftests/wireguard/qemu/debug.config | 1 -
4 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/include/linux/compiler_attributes.h b/include/linux/compiler_attributes.h
index b2a3f4f641a7..5f3b7edad1a7 100644
--- a/include/linux/compiler_attributes.h
+++ b/include/linux/compiler_attributes.h
@@ -171,6 +171,13 @@
*/
#define __mode(x) __attribute__((__mode__(x)))
+/*
+ * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-wa…
+ * clang: https://clang.llvm.org/docs/AttributeReference.html#nodiscard-warn-unused-r…
+ *
+ */
+#define __must_check __attribute__((__warn_unused_result__))
+
/*
* Optional: only supported since gcc >= 7
*
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index ac3fa37a84f9..7ef20d1a6c28 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -110,12 +110,6 @@ struct ftrace_likely_data {
unsigned long constant;
};
-#ifdef CONFIG_ENABLE_MUST_CHECK
-#define __must_check __attribute__((__warn_unused_result__))
-#else
-#define __must_check
-#endif
-
#if defined(CC_USING_HOTPATCH)
#define notrace __attribute__((hotpatch(0, 0)))
#elif defined(CC_USING_PATCHABLE_FUNCTION_ENTRY)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c789b39ed527..cb8ef4fd0d02 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -286,14 +286,6 @@ config GDB_SCRIPTS
endif # DEBUG_INFO
-config ENABLE_MUST_CHECK
- bool "Enable __must_check logic"
- default y
- help
- Enable the __must_check logic in the kernel build. Disable this to
- suppress the "warning: ignoring return value of 'foo', declared with
- attribute warn_unused_result" messages.
-
config FRAME_WARN
int "Warn for stack frames larger than"
range 0 8192
diff --git a/tools/testing/selftests/wireguard/qemu/debug.config b/tools/testing/selftests/wireguard/qemu/debug.config
index b50c2085c1ac..fe07d97df9fa 100644
--- a/tools/testing/selftests/wireguard/qemu/debug.config
+++ b/tools/testing/selftests/wireguard/qemu/debug.config
@@ -1,5 +1,4 @@
CONFIG_LOCALVERSION="-debug"
-CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_POINTER=y
CONFIG_STACK_VALIDATION=y
CONFIG_DEBUG_KERNEL=y
--
2.27.0
Revert commit cebc04ba9aeb ("add CONFIG_ENABLE_MUST_CHECK").
A lot of warn_unused_result warnings existed in 2006, but until now
they have been fixed thanks to people doing allmodconfig tests.
Our goal is to always enable __must_check where appreciate, so this
CONFIG option is no longer needed.
I see a lot of defconfig (arch/*/configs/*_defconfig) files having:
# CONFIG_ENABLE_MUST_CHECK is not set
I did not touch them for now since it would be a big churn. If arch
maintainers want to clean them up, please go ahead.
Signed-off-by: Masahiro Yamada <masahiroy(a)kernel.org>
---
include/linux/compiler_types.h | 4 ----
lib/Kconfig.debug | 8 --------
tools/testing/selftests/wireguard/qemu/debug.config | 1 -
3 files changed, 13 deletions(-)
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index ac3fa37a84f9..02f6d3fbad9d 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -110,11 +110,7 @@ struct ftrace_likely_data {
unsigned long constant;
};
-#ifdef CONFIG_ENABLE_MUST_CHECK
#define __must_check __attribute__((__warn_unused_result__))
-#else
-#define __must_check
-#endif
#if defined(CC_USING_HOTPATCH)
#define notrace __attribute__((hotpatch(0, 0)))
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c789b39ed527..cb8ef4fd0d02 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -286,14 +286,6 @@ config GDB_SCRIPTS
endif # DEBUG_INFO
-config ENABLE_MUST_CHECK
- bool "Enable __must_check logic"
- default y
- help
- Enable the __must_check logic in the kernel build. Disable this to
- suppress the "warning: ignoring return value of 'foo', declared with
- attribute warn_unused_result" messages.
-
config FRAME_WARN
int "Warn for stack frames larger than"
range 0 8192
diff --git a/tools/testing/selftests/wireguard/qemu/debug.config b/tools/testing/selftests/wireguard/qemu/debug.config
index b50c2085c1ac..fe07d97df9fa 100644
--- a/tools/testing/selftests/wireguard/qemu/debug.config
+++ b/tools/testing/selftests/wireguard/qemu/debug.config
@@ -1,5 +1,4 @@
CONFIG_LOCALVERSION="-debug"
-CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_POINTER=y
CONFIG_STACK_VALIDATION=y
CONFIG_DEBUG_KERNEL=y
--
2.25.1
This patch set adds AF_XDP selftests based on veth to selftests/bpf.
# Topology:
# ---------
# -----------
# _ | Process | _
# / ----------- \
# / | \
# / | \
# ----------- | -----------
# | Thread1 | | | Thread2 |
# ----------- | -----------
# | | |
# ----------- | -----------
# | xskX | | | xskY |
# ----------- | -----------
# | | |
# ----------- | ----------
# | vethX | --------- | vethY |
# ----------- peer ----------
# | | |
# namespaceX | namespaceY
These selftests test AF_XDP SKB and Native/DRV modes using veth Virtual
Ethernet interfaces.
The test program contains two threads, each thread is single socket with
a unique UMEM. It validates in-order packet delivery and packet content
by sending packets to each other.
Prerequisites setup by script test_xsk.sh:
Set up veth interfaces as per the topology shown ^^:
* setup two veth interfaces and one namespace
** veth<xxxx> in root namespace
** veth<yyyy> in af_xdp<xxxx> namespace
** namespace af_xdp<xxxx>
* create a spec file veth.spec that includes this run-time configuration
*** xxxx and yyyy are randomly generated 4 digit numbers used to avoid
conflict with any existing interface
The following tests are provided:
1. AF_XDP SKB mode
Generic mode XDP is driver independent, used when the driver does
not have support for XDP. Works on any netdevice using sockets and
generic XDP path. XDP hook from netif_receive_skb().
a. nopoll - soft-irq processing
b. poll - using poll() syscall
c. Socket Teardown
Create a Tx and a Rx socket, Tx from one socket, Rx on another.
Destroy both sockets, then repeat multiple times. Only nopoll mode
is used
d. Bi-directional Sockets
Configure sockets as bi-directional tx/rx sockets, sets up fill
and completion rings on each socket, tx/rx in both directions.
Only nopoll mode is used
2. AF_XDP DRV/Native mode
Works on any netdevice with XDP_REDIRECT support, driver dependent.
Processes packets before SKB allocation. Provides better performance
than SKB. Driver hook available just after DMA of buffer descriptor.
a. nopoll
b. poll
c. Socket Teardown
d. Bi-directional Sockets
* Only copy mode is supported because veth does not currently support
zero-copy mode
Total tests: 8
Flow:
* Single process spawns two threads: Tx and Rx
* Each of these two threads attach to a veth interface within their
assigned namespaces
* Each thread creates one AF_XDP socket connected to a unique umem
for each veth interface
* Tx thread transmits 10k packets from veth<xxxx> to veth<yyyy>
* Rx thread verifies if all 10k packets were received and delivered
in-order, and have the right content
v2 changes:
* Move selftests/xsk to selftests/bpf
* Remove Makefiles under selftests/xsk, and utilize selftests/bpf/Makefile
v3 changes:
* merge all test scripts test_xsk_*.sh into test_xsk.sh
This patch set requires applying patch from bpf stable tree:
commit 36ccdf85829a by Björn Töpel <bjorn.topel(a)intel.com>
[PATCH bpf v2] net, xsk: Avoid taking multiple skbuff references
Structure of the patch set:
Patch 1: This patch adds XSK Selftests framework under selftests/bpf
Patch 2: Adds tests: SKB poll and nopoll mode, and mac-ip-udp debug
Patch 3: Adds tests: DRV poll and nopoll mode
Patch 4: Adds tests: SKB and DRV Socket Teardown
Patch 5: Adds tests: SKB and DRV Bi-directional Sockets
Thanks: Weqaar
Weqaar Janjua (5):
selftests/bpf: xsk selftests framework
selftests/bpf: xsk selftests - SKB POLL, NOPOLL
selftests/bpf: xsk selftests - DRV POLL, NOPOLL
selftests/bpf: xsk selftests - Socket Teardown - SKB, DRV
selftests/bpf: xsk selftests - Bi-directional Sockets - SKB, DRV
tools/testing/selftests/bpf/Makefile | 7 +-
tools/testing/selftests/bpf/test_xsk.sh | 238 +++++
tools/testing/selftests/bpf/xdpxceiver.c | 1056 ++++++++++++++++++++
tools/testing/selftests/bpf/xdpxceiver.h | 158 +++
tools/testing/selftests/bpf/xsk_env.sh | 28 +
tools/testing/selftests/bpf/xsk_prereqs.sh | 119 +++
6 files changed, 1604 insertions(+), 2 deletions(-)
create mode 100755 tools/testing/selftests/bpf/test_xsk.sh
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.c
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.h
create mode 100755 tools/testing/selftests/bpf/xsk_env.sh
create mode 100755 tools/testing/selftests/bpf/xsk_prereqs.sh
--
2.20.1
This patchset provides support for the SRv6 End.DT4 and SRv6 End.DT6 (VRF mode)
behavior.
The SRv6 End.DT4 is used to implement multi-tenant IPv4 L3 VPN. It decapsulates
the received packets and performs IPv4 routing lookup in the routing table of
the tenant. The SRv6 End.DT4 Linux implementation leverages a VRF device. The
SRv6 End.DT4 is defined in the SRv6 Network Programming [1].
The Linux kernel already offers an implementation of the SRv6 End.DT6
behavior which permits IPv6 L3 VPNs over SRv6 networks. This new
implementation of DT6 is based on the same VRF infrastructure already
exploited for implementing the SRv6 End.DT4 behavior. The aim of the new
SRv6 End.DT6 in VRF mode consists in simplifying the construction of IPv6
L3 VPN services in the multi-tenant environment.
Currently, the two SRv6 End.DT6 implementations (legacy and VRF mode)
coexist seamlessly and can be chosen according to the context and the user
preferences.
- Patch 1 is needed to solve a pre-existing issue with tunneled packets
when a sniffer is attached;
- Patch 2 improves the management of the seg6local attributes used by the
SRv6 behaviors;
- Patch 3 adds support for optional attributes in SRv6 behaviors;
- Patch 4 introduces two callbacks used for customizing the
creation/destruction of a SRv6 behavior;
- Patch 5 is the core patch that adds support for the SRv6 End.DT4
behavior;
- Patch 6 introduces the VRF support for SRv6 End.DT6 behavior;
- Patch 7 adds the selftest for SRv6 End.DT4 behavior;
- Patch 8 adds the selftest for SRv6 End.DT6 (VRF mode) behavior;
- Patch 9 adds the vrftable attribute for End.DT4/DT6 behaviors in iproute2.
I would like to thank David Ahern for his support during the development of
this patchset.
Comments, suggestions and improvements are very welcome!
Thanks,
Andrea Mayer
v3
notes about the build bot:
- apparently the ',' (comma) in the subject prefix confused the build bot.
Removed the ',' in favor of ' ' (space).
Thanks to David Ahern and Konstantin Ryabitsev for shedding light on this
fact.
Thanks also to Nathan Chancellor for trying to build the patchset v2 by
simulating the bot issue.
add new patch for iproute2:
- [9/9] seg6: add support for vrftable attribute in End.DT4/DT6 behaviors
add new patch:
- [8/9] selftests: add selftest for the SRv6 End.DT6 (VRF) behavior
add new patch:
- [6/9] seg6: add VRF support for SRv6 End.DT6 behavior
add new patch:
- [3/9] seg6: add support for optional attributes in SRv6 behaviors
selftests: add selftest for the SRv6 End.DT4 behavior
- keep David Ahern's review tag since the code wasn't changed. Thanks to David
Ahern for his review.
seg6: add support for the SRv6 End.DT4 behavior
- remove useless error in seg6_end_dt4_build();
- remove #ifdef/#endif stubs for DT4 when CONFIG_NET_L3_MASTER_DEV is not
defined;
- fix coding style.
Thanks to Jakub Kicinski for his review and for all his suggestions.
seg6: add callbacks for customizing the creation/destruction of a behavior
- remove typedef(s) slwt_{build/destroy}_state_t;
- fix coding style: remove empty lines, trivial comments and rename labels in
the seg6_local_build_state() function.
Thanks to Jakub Kicinski for his review and for all his suggestions.
seg6: improve management of behavior attributes
- remove defensive programming approach in destroy_attr_srh(),
destroy_attr_bpf() and destroy_attrs();
- change the __destroy_attrs() function signature, renaming the 'end' argument
'parsed_max'. Now, the __destroy_attrs() keeps only the 'parsed_max' and
'slwt' arguments.
Thanks to Jakub Kicinski for his review and for all his suggestions.
vrf: add mac header for tunneled packets when sniffer is attached
- keep David Ahern's review tag since the code wasn't changed.
Thanks to Jakub Kicinski for pointing it out and David Ahern for his review.
v2
no changes made: resubmitted after false build report.
v1
improve comments;
add new patch 2/5 titled: seg6: improve management of behavior attributes
seg6: add support for the SRv6 End.DT4 behavior
- remove the inline keyword in the definition of fib6_config_get_net().
selftests: add selftest for the SRv6 End.DT4 behavior
- add check for the vrf sysctl
[1] https://tools.ietf.org/html/draft-ietf-spring-srv6-network-programming
Andrea Mayer (8):
vrf: add mac header for tunneled packets when sniffer is attached
seg6: improve management of behavior attributes
seg6: add support for optional attributes in SRv6 behaviors
seg6: add callbacks for customizing the creation/destruction of a
behavior
seg6: add support for the SRv6 End.DT4 behavior
seg6: add VRF support for SRv6 End.DT6 behavior
selftests: add selftest for the SRv6 End.DT4 behavior
selftests: add selftest for the SRv6 End.DT6 (VRF) behavior
drivers/net/vrf.c | 78 ++-
include/uapi/linux/seg6_local.h | 1 +
net/ipv6/seg6_local.c | 593 +++++++++++++++++-
.../selftests/net/srv6_end_dt4_l3vpn_test.sh | 494 +++++++++++++++
.../selftests/net/srv6_end_dt6_l3vpn_test.sh | 502 +++++++++++++++
5 files changed, 1649 insertions(+), 19 deletions(-)
create mode 100755 tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh
create mode 100755 tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh
Paolo Lungaroni (1):
seg6: add support for vrftable attribute in End.DT4/DT6 behaviors
include/uapi/linux/seg6_local.h | 1 +
ip/iproute_lwtunnel.c | 19 ++++++++++++++++---
2 files changed, 17 insertions(+), 3 deletions(-)
--
2.20.1
The eeh-basic test got its own 60 seconds timeout (defined in commit
414f50434aa2 "selftests/eeh: Bump EEH wait time to 60s") per breakable
device.
And we have discovered that the number of breakable devices varies
on different hardware. The device recovery time ranges from 0 to 35
seconds. In our test pool it will take about 30 seconds to run on a
Power8 system that with 5 breakable devices, 60 seconds to run on a
Power9 system that with 4 breakable devices.
Extend the timeout setting in the kselftest framework to 5 minutes
to give it a chance to finish.
Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com>
---
tools/testing/selftests/powerpc/eeh/Makefile | 2 +-
tools/testing/selftests/powerpc/eeh/settings | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/powerpc/eeh/settings
diff --git a/tools/testing/selftests/powerpc/eeh/Makefile b/tools/testing/selftests/powerpc/eeh/Makefile
index b397bab..ae963eb 100644
--- a/tools/testing/selftests/powerpc/eeh/Makefile
+++ b/tools/testing/selftests/powerpc/eeh/Makefile
@@ -3,7 +3,7 @@ noarg:
$(MAKE) -C ../
TEST_PROGS := eeh-basic.sh
-TEST_FILES := eeh-functions.sh
+TEST_FILES := eeh-functions.sh settings
top_srcdir = ../../../../..
include ../../lib.mk
diff --git a/tools/testing/selftests/powerpc/eeh/settings b/tools/testing/selftests/powerpc/eeh/settings
new file mode 100644
index 0000000..694d707
--- /dev/null
+++ b/tools/testing/selftests/powerpc/eeh/settings
@@ -0,0 +1 @@
+timeout=300
--
2.7.4
This patch set adds AF_XDP selftests based on veth to selftests/bpf.
# Topology:
# ---------
# -----------
# _ | Process | _
# / ----------- \
# / | \
# / | \
# ----------- | -----------
# | Thread1 | | | Thread2 |
# ----------- | -----------
# | | |
# ----------- | -----------
# | xskX | | | xskY |
# ----------- | -----------
# | | |
# ----------- | ----------
# | vethX | --------- | vethY |
# ----------- peer ----------
# | | |
# namespaceX | namespaceY
These selftests test AF_XDP SKB and Native/DRV modes using veth Virtual
Ethernet interfaces.
The test program contains two threads, each thread is single socket with
a unique UMEM. It validates in-order packet delivery and packet content
by sending packets to each other.
Prerequisites setup by script test_xsk_prerequisites.sh:
Set up veth interfaces as per the topology shown ^^:
* setup two veth interfaces and one namespace
** veth<xxxx> in root namespace
** veth<yyyy> in af_xdp<xxxx> namespace
** namespace af_xdp<xxxx>
* create a spec file veth.spec that includes this run-time configuration
that is read by test scripts - filenames prefixed with test_xsk_
*** xxxx and yyyy are randomly generated 4 digit numbers used to avoid
conflict with any existing interface
The following tests are provided:
1. AF_XDP SKB mode
Generic mode XDP is driver independent, used when the driver does
not have support for XDP. Works on any netdevice using sockets and
generic XDP path. XDP hook from netif_receive_skb().
a. nopoll - soft-irq processing
b. poll - using poll() syscall
c. Socket Teardown
Create a Tx and a Rx socket, Tx from one socket, Rx on another.
Destroy both sockets, then repeat multiple times. Only nopoll mode
is used
d. Bi-directional Sockets
Configure sockets as bi-directional tx/rx sockets, sets up fill
and completion rings on each socket, tx/rx in both directions.
Only nopoll mode is used
2. AF_XDP DRV/Native mode
Works on any netdevice with XDP_REDIRECT support, driver dependent.
Processes packets before SKB allocation. Provides better performance
than SKB. Driver hook available just after DMA of buffer descriptor.
a. nopoll
b. poll
c. Socket Teardown
d. Bi-directional Sockets
* Only copy mode is supported because veth does not currently support
zero-copy mode
Total tests: 8
Flow:
* Single process spawns two threads: Tx and Rx
* Each of these two threads attach to a veth interface within their
assigned namespaces
* Each thread creates one AF_XDP socket connected to a unique umem
for each veth interface
* Tx thread transmits 10k packets from veth<xxxx> to veth<yyyy>
* Rx thread verifies if all 10k packets were received and delivered
in-order, and have the right content
v2 changes:
* Move selftests/xsk to selftests/bpf
* Remove Makefiles under selftests/xsk, and utilize selftests/bpf/Makefile
Structure of the patch set:
Patch 1: This patch adds XSK Selftests framework under selftests/bpf
Patch 2: Adds tests: SKB poll and nopoll mode, and mac-ip-udp debug
Patch 3: Adds tests: DRV poll and nopoll mode
Patch 4: Adds tests: SKB and DRV Socket Teardown
Patch 5: Adds tests: SKB and DRV Bi-directional Sockets
Thanks: Weqaar
Weqaar Janjua (5):
selftests/bpf: xsk selftests framework
selftests/bpf: xsk selftests - SKB POLL, NOPOLL
selftests/bpf: xsk selftests - DRV POLL, NOPOLL
selftests/bpf: xsk selftests - Socket Teardown - SKB, DRV
selftests/bpf: xsk selftests - Bi-directional Sockets - SKB, DRV
tools/testing/selftests/bpf/Makefile | 15 +-
.../bpf/test_xsk_drv_bidirectional.sh | 23 +
.../selftests/bpf/test_xsk_drv_nopoll.sh | 20 +
.../selftests/bpf/test_xsk_drv_poll.sh | 20 +
.../selftests/bpf/test_xsk_drv_teardown.sh | 20 +
.../selftests/bpf/test_xsk_prerequisites.sh | 127 ++
.../bpf/test_xsk_skb_bidirectional.sh | 20 +
.../selftests/bpf/test_xsk_skb_nopoll.sh | 20 +
.../selftests/bpf/test_xsk_skb_poll.sh | 20 +
.../selftests/bpf/test_xsk_skb_teardown.sh | 20 +
tools/testing/selftests/bpf/xdpxceiver.c | 1056 +++++++++++++++++
tools/testing/selftests/bpf/xdpxceiver.h | 158 +++
tools/testing/selftests/bpf/xsk_env.sh | 28 +
tools/testing/selftests/bpf/xsk_prereqs.sh | 119 ++
14 files changed, 1664 insertions(+), 2 deletions(-)
create mode 100755 tools/testing/selftests/bpf/test_xsk_drv_bidirectional.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_drv_nopoll.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_drv_poll.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_drv_teardown.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_prerequisites.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_skb_bidirectional.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_skb_nopoll.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_skb_poll.sh
create mode 100755 tools/testing/selftests/bpf/test_xsk_skb_teardown.sh
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.c
create mode 100644 tools/testing/selftests/bpf/xdpxceiver.h
create mode 100755 tools/testing/selftests/bpf/xsk_env.sh
create mode 100755 tools/testing/selftests/bpf/xsk_prereqs.sh
--
2.20.1
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will be present only in the page table of the owning mm.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v11:
* Drop support for uncached mappings
v10: https://lore.kernel.org/lkml/20201123095432.5860-1-rppt@kernel.org
* Drop changes to arm64 compatibility layer
* Add Roman's Ack for memcg accounting
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
* Fix build with and without CONFIG_MEMCG
* Update memcg accounting to avoid copying memcg_data, per Roman comments
* Fix issues in secretmem_fault(), thanks Matthew
* Do not wire up syscall in arm64 compatibility layer
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
* Use CMA for all secretmem allocations as David suggested
* Update memcg accounting after transtion to CMA
* Prevent hibernation when there are active secretmem users
* Add zeroing of the memory before releasing it back to cma/page allocator
* Rebase on v5.10-rc2-mmotm-2020-11-07-21-40
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
* Use set_direct_map() instead of __kernel_map_pages() to ensure error
handling in case the direct map update fails
* Add accounting of large pages used to reduce the direct map fragmentation
* Teach get_user_pages() and frieds to refuse get/pin secretmem pages
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
* Silence the warning about missing syscall, thanks to Qian Cai
* Replace spaces with tabs in Kconfig additions, per Randy
* Add a selftest.
Older history:
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (9):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/arm64/include/asm/cacheflush.h | 4 +-
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/mm/pageattr.c | 10 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 2 +-
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 4 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 436 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
34 files changed, 863 insertions(+), 39 deletions(-)
create mode 100644 include/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
Nowadays, there are increasing requirements to benchmark the performance
of dma_map and dma_unmap particually while the device is attached to an
IOMMU.
This patchset provides the benchmark infrastruture for streaming DMA
mapping. The architecture of the code is pretty much similar with GUP
benchmark:
* mm/gup_benchmark.c provides kernel interface;
* tools/testing/selftests/vm/gup_benchmark.c provides user program to
call the interface provided by mm/gup_benchmark.c.
In our case, kernel/dma/map_benchmark.c is like mm/gup_benchmark.c;
tools/testing/selftests/dma/dma_map_benchmark.c is like tools/testing/
selftests/vm/gup_benchmark.c
A major difference with GUP benchmark is DMA_MAP benchmark needs to run
on a device. Considering one board with below devices and IOMMUs
device A ------- IOMMU 1
device B ------- IOMMU 2
device C ------- non-IOMMU
Different devices might attach to different IOMMU or non-IOMMU. To make
benchmark run, we can either
* create a virtual device and hack the kernel code to attach the virtual
device to IOMMU1, IOMMU2 or non-IOMMU.
* use the existing driver_override mechinism, unbind device A,B, OR c from
their original driver and bind A to dma_map_benchmark platform driver or
pci driver for benchmarking.
In this patchset, I prefer to use the driver_override and avoid the ugly
hack in kernel. We can dynamically switch device behind different IOMMUs
to get the performance of IOMMU or non-IOMMU.
-v4:
* add dma direction support according to Christoph Hellwig's comment;
* add dma mask bit set according to Christoph Hellwig's comment;
* make the benchmark depend on DEBUG_FS according to John Garry's comment;
* strictly check parameters in ioctl
-v3:
* fix build issues reported by 0day kernel test robot
-v2:
* add PCI support; v1 supported platform devices only
* replace ssleep by msleep_interruptible() to permit users to exit
benchmark before it is completed
* many changes according to Robin's suggestions, thanks! Robin
- add standard deviation output to reflect the worst case
- check users' parameters strictly like the number of threads
- make cache dirty before dma_map
- fix unpaired dma_map_page and dma_unmap_single;
- remove redundant "long long" before ktime_to_ns();
- use devm_add_action()
Barry Song (2):
dma-mapping: add benchmark support for streaming DMA APIs
selftests/dma: add test application for DMA_MAP_BENCHMARK
MAINTAINERS | 6 +
kernel/dma/Kconfig | 9 +
kernel/dma/Makefile | 1 +
kernel/dma/map_benchmark.c | 361 ++++++++++++++++++
tools/testing/selftests/dma/Makefile | 6 +
tools/testing/selftests/dma/config | 1 +
.../testing/selftests/dma/dma_map_benchmark.c | 123 ++++++
7 files changed, 507 insertions(+)
create mode 100644 kernel/dma/map_benchmark.c
create mode 100644 tools/testing/selftests/dma/Makefile
create mode 100644 tools/testing/selftests/dma/config
create mode 100644 tools/testing/selftests/dma/dma_map_benchmark.c
--
2.25.1
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will have desired protection bits set in the user page
table. For instance, current implementation allows uncached mappings.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v10:
* Drop changes to arm64 compatibility layer
* Add Roman's Ack for memcg accounting
v9: https://lore.kernel.org/lkml/20201117162932.13649-1-rppt@kernel.org
* Fix build with and without CONFIG_MEMCG
* Update memcg accounting to avoid copying memcg_data, per Roman comments
* Fix issues in secretmem_fault(), thanks Matthew
* Do not wire up syscall in arm64 compatibility layer
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
* Use CMA for all secretmem allocations as David suggested
* Update memcg accounting after transtion to CMA
* Prevent hibernation when there are active secretmem users
* Add zeroing of the memory before releasing it back to cma/page allocator
* Rebase on v5.10-rc2-mmotm-2020-11-07-21-40
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
* Use set_direct_map() instead of __kernel_map_pages() to ensure error
handling in case the direct map update fails
* Add accounting of large pages used to reduce the direct map fragmentation
* Teach get_user_pages() and frieds to refuse get/pin secretmem pages
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
* Silence the warning about missing syscall, thanks to Qian Cai
* Replace spaces with tabs in Kconfig additions, per Randy
* Add a selftest.
Older history:
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (9):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/Kconfig | 7 +
arch/arm64/include/asm/cacheflush.h | 4 +-
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/mm/pageattr.c | 10 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 4 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
include/uapi/linux/secretmem.h | 8 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 446 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
36 files changed, 888 insertions(+), 38 deletions(-)
create mode 100644 include/linux/secretmem.h
create mode 100644 include/uapi/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
usage.rst goes into a detailed about faking out classes, but currently
lacks wording about how one might idiomatically test a range of inputs.
Give an example of how one might test a hash function via macros/helper
funcs and a table-driven test and very briefly discuss pros and cons.
Also highlight the KUNIT_EXPECT_*_MSG() variants (that aren't mentioned
elsewhere [1]) which are particularly useful in these situations.
It is also criminally underused at the moment, only appearing in 2
tests (both written by people involved in KUnit).
[1] not even on
https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
Documentation/dev-tools/kunit/usage.rst | 66 +++++++++++++++++++++++++
1 file changed, 66 insertions(+)
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 62142a47488c..317390df2b96 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -451,6 +451,72 @@ We can now use it to test ``struct eeprom_buffer``:
destroy_eeprom_buffer(ctx->eeprom_buffer);
}
+Testing various inputs
+----------------------
+
+Testing just a few inputs might not be enough to have confidence that the code
+works correctly, e.g. for a hash function.
+
+In such cases, it can be helpful to have a helper macro or function, e.g. this
+fictitious example for ``md5sum(1)``
+
+.. code-block:: c
+
+ /* Note: the cast is to satisfy overly strict type-checking. */
+ #define TEST_MD5(in, want) \
+ md5sum(in, out); \
+ KUNIT_EXPECT_STREQ_MSG(test, (char *)out, want, "md5sum(%s)", in);
+
+ char out[16];
+ TEST_MD5("hello world", "5eb63bbbe01eeed093cb22bb8f5acdc3");
+ TEST_MD5("hello world!", "fc3ff98e8c6a0d3087d515c0473f8677");
+
+Note the use of ``KUNIT_EXPECT_STREQ_MSG`` to give more context when it fails
+and make it easier to track down. (Yes, in this example, ``want`` is likely
+going to be unique enough on its own).
+
+The ``_MSG`` variants are even more useful when the same expectation is called
+multiple times (in a loop or helper function) and thus the line number isn't
+enough to identify what failed, like below.
+
+In some cases, it can be helpful to write a *table-driven test* instead, e.g.
+
+.. code-block:: c
+
+ int i;
+ char out[16];
+
+ struct md5_test_case {
+ const char *str;
+ const char *md5;
+ };
+
+ struct md5_test_case cases[] = {
+ {
+ .str = "hello world",
+ .md5 = "5eb63bbbe01eeed093cb22bb8f5acdc3",
+ },
+ {
+ .str = "hello world!",
+ .md5 = "fc3ff98e8c6a0d3087d515c0473f8677",
+ },
+ };
+ for (i = 0; i < ARRAY_SIZE(cases); ++i) {
+ md5sum(cases[i].str, out);
+ KUNIT_EXPECT_STREQ_MSG(test, (char *)out, cases[i].md5,
+ "md5sum(%s)", cases[i].str);
+ }
+
+
+There's more boilerplate involved, but it can:
+
+* be more readable when there are multiple inputs/outputs thanks to field names,
+
+ * E.g. see ``fs/ext4/inode-test.c`` for an example of both.
+* reduce duplication if test cases can be shared across multiple tests.
+
+ * E.g. if we had a magical ``undo_md5sum`` function, we could reuse ``cases``.
+
.. _kunit-on-non-uml:
KUnit on non-UML architectures
base-commit: 77c8473edf7f7664137f555cfcdc8c460bbd947d
--
2.29.1.341.ge80a0c044ae-goog
Hi,
This is the v7 of syscall user dispatch. This version is a bit
different from v6 on the following points, after the modifications
requested on that submission.
* The interface no longer receives <start>,<end> end parameters, but
<start>,<length> as suggested by PeterZ.
* Syscall User Dispatch is now done before ptrace, and this means there
is some SYSCALL_WORK_EXIT work that needs to be done. No challenges
there, but I'd like to draw attention to that region of the code that
is new in this submission.
* The previous TIF_SYSCALL_USER_DISPATCH is now handled through
SYSCALL_WORK flags.
* Introduced a new test as patch 6, which benchmarks the fast submission
path and test the return in blocked selector state.
* Nothing is architecture dependent anymore. No config switches. It
only depends on CONFIG_GENERIC_ENTRY.
Other smaller changes are documented one each commit.
This was tested using the kselftests tests in patch 5 and 6 and compiled
tested with !CONFIG_GENERIC_ENTRY.
This patchset is based on the core/entry branch of the TIP tree.
A working tree with this patchset is available at:
https://gitlab.collabora.com/krisman/linux -b syscall-user-dispatch-v7
Previous submissions are archived at:
RFC/v1: https://lkml.org/lkml/2020/7/8/96
v2: https://lkml.org/lkml/2020/7/9/17
v3: https://lkml.org/lkml/2020/7/12/4
v4: https://www.spinics.net/lists/linux-kselftest/msg16377.html
v5: https://lkml.org/lkml/2020/8/10/1320
v6: https://lkml.org/lkml/2020/9/4/1122
Gabriel Krisman Bertazi (7):
x86: vdso: Expose sigreturn address on vdso to the kernel
signal: Expose SYS_USER_DISPATCH si_code type
kernel: Implement selective syscall userspace redirection
entry: Support Syscall User Dispatch on common syscall entry
selftests: Add kselftest for syscall user dispatch
selftests: Add benchmark for syscall user dispatch
docs: Document Syscall User Dispatch
.../admin-guide/syscall-user-dispatch.rst | 87 +++++
arch/x86/entry/vdso/vdso2c.c | 2 +
arch/x86/entry/vdso/vdso32/sigreturn.S | 2 +
arch/x86/entry/vdso/vma.c | 15 +
arch/x86/include/asm/elf.h | 2 +
arch/x86/include/asm/vdso.h | 2 +
arch/x86/kernel/signal_compat.c | 2 +-
fs/exec.c | 3 +
include/linux/entry-common.h | 2 +
include/linux/sched.h | 2 +
include/linux/syscall_user_dispatch.h | 40 +++
include/linux/thread_info.h | 2 +
include/uapi/asm-generic/siginfo.h | 3 +-
include/uapi/linux/prctl.h | 5 +
kernel/entry/Makefile | 2 +-
kernel/entry/common.c | 17 +
kernel/entry/common.h | 16 +
kernel/entry/syscall_user_dispatch.c | 102 ++++++
kernel/fork.c | 1 +
kernel/sys.c | 5 +
tools/testing/selftests/Makefile | 1 +
.../syscall_user_dispatch/.gitignore | 3 +
.../selftests/syscall_user_dispatch/Makefile | 9 +
.../selftests/syscall_user_dispatch/config | 1 +
.../syscall_user_dispatch/sud_benchmark.c | 200 +++++++++++
.../syscall_user_dispatch/sud_test.c | 310 ++++++++++++++++++
26 files changed, 833 insertions(+), 3 deletions(-)
create mode 100644 Documentation/admin-guide/syscall-user-dispatch.rst
create mode 100644 include/linux/syscall_user_dispatch.h
create mode 100644 kernel/entry/common.h
create mode 100644 kernel/entry/syscall_user_dispatch.c
create mode 100644 tools/testing/selftests/syscall_user_dispatch/.gitignore
create mode 100644 tools/testing/selftests/syscall_user_dispatch/Makefile
create mode 100644 tools/testing/selftests/syscall_user_dispatch/config
create mode 100644 tools/testing/selftests/syscall_user_dispatch/sud_benchmark.c
create mode 100644 tools/testing/selftests/syscall_user_dispatch/sud_test.c
--
2.29.2
It looks like the seccomp selftests were never actually built for sh.
This fixes it, though I don't have an environment to do a runtime test
of it yet.
Fixes: 0bb605c2c7f2b4b3 ("sh: Add SECCOMP_FILTER")
Signed-off-by: Kees Cook <keescook(a)chromium.org>
---
tools/testing/selftests/seccomp/seccomp_bpf.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c
index 7f7ecfcd66db..26c72f2b61b1 100644
--- a/tools/testing/selftests/seccomp/seccomp_bpf.c
+++ b/tools/testing/selftests/seccomp/seccomp_bpf.c
@@ -1804,8 +1804,8 @@ TEST_F(TRACE_poke, getpid_runs_normally)
#define SYSCALL_RET(_regs) (_regs).a[(_regs).windowbase * 4 + 2]
#elif defined(__sh__)
# define ARCH_REGS struct pt_regs
-# define SYSCALL_NUM(_regs) (_regs).gpr[3]
-# define SYSCALL_RET(_regs) (_regs).gpr[0]
+# define SYSCALL_NUM(_regs) (_regs).regs[3]
+# define SYSCALL_RET(_regs) (_regs).regs[0]
#else
# error "Do not know how to find your architecture's registers and syscalls"
#endif
--
2.25.1
Hi,
This patch series fixes some issues and makes the Landlock filesystem
access-control more consistent and deterministic when stacking multiple
rulesets. This is checked by current and new tests. I also extended
documentation and example to help users.
This series can be applied on top of
https://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git/…
Regards,
Mickaël Salaün (9):
landlock: Fix memory allocation error handling
landlock: Cosmetic fixes for filesystem management
landlock: Enforce deterministic interleaved path rules
landlock: Always intersect access rights
landlock: Add extra checks when inserting a rule
selftests/landlock: Extend layout1.inherit_superset
landlock: Clean up get_ruleset_from_fd()
landlock: Add help to enable Landlock as a stacked LSM
landlock: Extend documentation about limitations
Documentation/userspace-api/landlock.rst | 17 +++
samples/landlock/sandboxer.c | 21 +++-
security/landlock/Kconfig | 4 +-
security/landlock/fs.c | 67 +++++-----
security/landlock/object.c | 5 +-
security/landlock/ruleset.c | 34 ++---
security/landlock/syscall.c | 24 ++--
tools/testing/selftests/landlock/fs_test.c | 140 +++++++++++++++++++--
8 files changed, 239 insertions(+), 73 deletions(-)
base-commit: 96b3198c4025c11347651700b77e45a686d78553
--
2.29.2
From: Zi Yan <ziy(a)nvidia.com>
Hi all,
With Matthew's THP in pagecache patches[1], we will be able to handle any size
pagecache THPs, but currently split_huge_page can only split a THP to order-0
pages. This can easily erase the benefit of having pagecache THPs, when
operations like truncate might want to keep pages larger than order-0. In
response, here is the patches to add support for splitting a THP to any lower
order pages. In addition, this patchset prepares for my PUD THP patchset[2],
since splitting a PUD THP to multiple PMD THPs can be handled by
split_huge_page_to_list_to_order function added by this patchset, which reduces
a lot of redundant code without just replicating split_huge_page for PUD THP.
To help the tests of splitting huge pages, I added a new debugfs interface
at <debugfs>/split_huge_pages_in_range_pid, so developers can split THPs in a
given range from a process with the given pid by writing
"<pid>,<vaddr_start>,<vaddr_end>,<to_order>" to the interface. I also added a
new test program to test 1) splitting PMD THPs, 2) splitting PTE-mapped THPs,
3) splitting pagecache THPs to any lower order, 4) truncating a pagecache
THP to a page with a lower order, and 5) punching holes in a pagecache THP to
cause splitting THPs to lower order THPs.
The patchset is on top of Matthew's pagecache/next tree[3].
* Patch 1 is cherry-picked from Matthew's recent xarray fix [4] just to make sure
Patch 3 to 7 can run without problem. I let Matthew decide how it should get
picked up.
* Patch 2 is self-contained and can be merged if it looks OK.
Comments and/or suggestions are welcome.
ChangeLog
===
>From RFC:
1. Fixed debugfs to handle splitting PTE-mapped THPs properly and added stats
for split THPs.
2. Added a new test case for splitting PTE-mapped THPs. Each of the four PTEs
points to a different subpage from four THPs and used kpageflags to check
whether a PTE points to a THP or not (AnonHugePages from smap does not show
PTE-mapped THPs).
3. mem_cgroup_split_huge_fixup() takes order instead of nr.
4. split_page_owner takes old_order and new_order instead of nr and new_order.
5. Corrected __split_page_owner declaration and fixed its implementation when
splitting a THP to a new order.
6. Renamed left to remaining in truncate_inode_partial_page().
7. Use VM_BUG_ON instead of WARN_ONCE when splitting a THP to the unsupported
order-0 and splitting anonymous THPs to non-zero orders.
8. Added punching holes in a file as a new pagecache THP split test case, which
uncovered an xarray bug.
[1] https://lore.kernel.org/linux-mm/20201029193405.29125-1-willy@infradead.org/
[2] https://lore.kernel.org/linux-mm/20200928175428.4110504-1-zi.yan@sent.com/
[3] https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/next
[4] https://git.infradead.org/users/willy/xarray.git
Matthew Wilcox (Oracle) (1):
XArray: Fix splitting to non-zero orders
Zi Yan (6):
mm: huge_memory: add new debugfs interface to trigger split huge page
on any page range.
mm: memcg: make memcg huge page split support any order split.
mm: page_owner: add support for splitting to any order in split
page_owner.
mm: thp: split huge page to any lower order pages.
mm: truncate: split thp to a non-zero order if possible.
mm: huge_memory: enable debugfs to split huge pages to any order.
include/linux/huge_mm.h | 8 +
include/linux/memcontrol.h | 5 +-
include/linux/page_owner.h | 10 +-
lib/test_xarray.c | 26 +-
lib/xarray.c | 4 +-
mm/huge_memory.c | 219 ++++++--
mm/internal.h | 1 +
mm/memcontrol.c | 6 +-
mm/migrate.c | 2 +-
mm/page_alloc.c | 2 +-
mm/page_owner.c | 13 +-
mm/swap.c | 1 -
mm/truncate.c | 29 +-
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 1 +
.../selftests/vm/split_huge_page_test.c | 479 ++++++++++++++++++
16 files changed, 742 insertions(+), 65 deletions(-)
create mode 100644 tools/testing/selftests/vm/split_huge_page_test.c
--
2.28.0
On Tue, 17 Nov 2020 at 19:02, Greg Kroah-Hartman
<gregkh(a)linuxfoundation.org> wrote:
>
> This is the start of the stable review cycle for the 5.9.9 release.
> There are 255 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Thu, 19 Nov 2020 12:20:51 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.9.9-rc1.…
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.9.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
Results from Linaro’s test farm.
No regressions on arm64, arm, x86_64, and i386.
Tested-by: Linux Kernel Functional Testing <lkft(a)linaro.org>
NOTE:
1)
BUG: Invalid wait context on arm64 db410c device while booting.
This issue has not reproduced after several testing loops.
https://lore.kernel.org/stable/CA+G9fYsk54r9Re4E9BWpqsoxLjpCvxRKFWRgdiKVcPo…
2)
kselftest test suite version upgrade to v5.9
3)
While running kselftest netfilter on x86, i386, arm64 and arm devices
the following kernel warning was noticed.
WARNING: at net/netfilter/nf_tables_api.c:622
lockdep_nfnl_nft_mutex_not_held+0x19/0x20 [nf_tables]
https://lore.kernel.org/linux-kselftest/CA+G9fYvFUpODs+NkSYcnwKnXm62tmP=ksL…
4)
>From this release we have started building kernels with clang-10 toolchain
and testing LTP testsuite on qemu_arm64, qemu_arm, qemu_x86_64 and qemu_i386.
Summary
------------------------------------------------------------------------
kernel: 5.9.9-rc1
git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
git branch: linux-5.9.y
git commit: fb1622495321923cbb1ae2c6cf2da1e9ca286800
git describe: v5.9.8-256-gfb1622495321
Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-5.9.y/build/v5.9.8…
No regressions (compared to build v5.9.8)
No fixes (compared to build v5.9.8)
Ran 52946 total tests in the following environments and test suites.
Environments
--------------
- dragonboard-410c
- hi6220-hikey
- i386
- juno-r2
- juno-r2-compat
- juno-r2-kasan
- nxp-ls2088
- qemu-arm-clang
- qemu-arm64-clang
- qemu-arm64-kasan
- qemu-i386-clang
- qemu-x86_64-clang
- qemu-x86_64-kasan
- qemu_arm
- qemu_arm64
- qemu_arm64-compat
- qemu_i386
- qemu_x86_64
- qemu_x86_64-compat
- x15
- x86
- x86-kasan
Test Suites
-----------
* build
* install-android-platform-tools-r2600
* libhugetlbfs
* linux-log-parser
* ltp-cap_bounds-tests
* ltp-commands-tests
* ltp-containers-tests
* ltp-cpuhotplug-tests
* ltp-crypto-tests
* ltp-dio-tests
* ltp-fcntl-locktests-tests
* ltp-filecaps-tests
* ltp-fs-tests
* ltp-fs_bind-tests
* ltp-fs_perms_simple-tests
* ltp-fsx-tests
* ltp-hugetlb-tests
* ltp-io-tests
* ltp-ipc-tests
* ltp-math-tests
* ltp-mm-tests
* ltp-nptl-tests
* ltp-pty-tests
* ltp-sched-tests
* ltp-securebits-tests
* ltp-syscalls-tests
* ltp-tracing-tests
* perf
* v4l2-compliance
* ltp-controllers-tests
* ltp-cve-tests
* network-basic-tests
* kselftest
* ltp-open-posix-tests
* kvm-unit-tests
* kunit
* kselftest-vsyscall-mode-native
* kselftest-vsyscall-mode-none
--
Linaro LKFT
https://lkft.linaro.org
On Wed, 18 Nov 2020 16:43:33 -0500 Willem de Bruijn wrote:
> On Tue, Nov 17, 2020 at 12:19 PM Jakub Kicinski <kuba(a)kernel.org> wrote:
> > Sorry I misremembered it's 4. We can leave that as is.
>
> Instead of having to remember, maybe we should have a file in
> tools/testing/selftest to define constants?
>
> I defined them one-off in tools/testing/selftests/net/udpgso_bench.sh
>
> readonly KSFT_PASS=0
> readonly KSFT_FAIL=1
> readonly KSFT_SKIP=4
>
> along with some other kselftest shell support infra. But having each
> test figure this out independently is duplicative and error prone.
Sounds like a good idea, I was surprised it wasn't already defined in
any lib.
CCing the selftest ML.
Hi Linus,
Please pull the following Kunit fixes update for Linux 5.10-rc5.
This Kunit update for Linux 5.10-rc5 consists of several fixes Kunit
documentation, tool, compile time fixes not pollute source directory,
and fix to remove tools/testing/kunit/.gitattributes file.
diff is attached.
Brendan fixed the weirdness with tools/testing/kunit/.gitattributes
file. Thanks for noticing it.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 0d0d245104a42e593adcf11396017a6420c08ba8:
kunit: tools: fix kunit_tool tests for parsing test plans (2020-10-26
13:25:40 -0600)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-fixes-5.10-rc5
for you to fetch changes up to 3084db0e0d5076cd48408274ab0911cd3ccdae88:
kunit: fix display of failed expectations for strings (2020-11-10
13:45:15 -0700)
----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.10-rc5
This Kunit update for Linux 5.10-rc5 consists of several fixes Kunit
documentation, tool, compile time fixes not pollute source directory,
and fix to remove tools/testing/kunit/.gitattributes file.
----------------------------------------------------------------
Andy Shevchenko (2):
kunit: Do not pollute source directory with generated files
(.kunitconfig)
kunit: Do not pollute source directory with generated files
(test.log)
Brendan Higgins (1):
kunit: tool: unmark test_data as binary blobs
Daniel Latypov (4):
kunit: tool: fix pre-existing python type annotation errors
kunit: tool: print out stderr from make (like build warnings)
kunit: tool: fix extra trailing \n in raw + parsed test output
kunit: fix display of failed expectations for strings
David Gow (1):
kunit: Fix kunit.py parse subcommand (use null build_dir)
Randy Dunlap (3):
KUnit: Docs: fix a wording typo
KUnit: Docs: style: fix some Kconfig example issues
KUnit: Docs: usage: wording fixes
Documentation/dev-tools/kunit/faq.rst | 2 +-
Documentation/dev-tools/kunit/style.rst | 18 +++++------
Documentation/dev-tools/kunit/usage.rst | 10 +++----
include/kunit/test.h | 2 +-
tools/testing/kunit/.gitattributes | 1 -
tools/testing/kunit/kunit.py | 27 ++++++++---------
tools/testing/kunit/kunit_kernel.py | 53
+++++++++++++++++++++++++--------
tools/testing/kunit/kunit_parser.py | 17 ++++++-----
tools/testing/kunit/kunit_tool_test.py | 4 +--
9 files changed, 80 insertions(+), 54 deletions(-)
delete mode 100644 tools/testing/kunit/.gitattributes
----------------------------------------------------------------
Hello linux-kselftest(a)vger.kernel.org
We are Base Investment Company offering Corporate and Personal Loan at 3% Interest Rate for a duration of 10Years.
We also pay 1% commission to brokers, who introduce project owners for finance or other opportunities.
Please get back to me if you are interested for more
details.
Yours faithfully,
Hashim Murrah
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will have desired protection bits set in the user page
table. For instance, current implementation allows uncached mappings.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v9:
* Fix build with and without CONFIG_MEMCG
* Update memcg accounting to avoid copying memcg_data, per Roman comments
* Fix issues in secretmem_fault(), thanks Matthew
* Do not wire up syscall in arm64 compatibility layer
v8: https://lore.kernel.org/lkml/20201110151444.20662-1-rppt@kernel.org
* Use CMA for all secretmem allocations as David suggested
* Update memcg accounting after transtion to CMA
* Prevent hibernation when there are active secretmem users
* Add zeroing of the memory before releasing it back to cma/page allocator
* Rebase on v5.10-rc2-mmotm-2020-11-07-21-40
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
* Use set_direct_map() instead of __kernel_map_pages() to ensure error
handling in case the direct map update fails
* Add accounting of large pages used to reduce the direct map fragmentation
* Teach get_user_pages() and frieds to refuse get/pin secretmem pages
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
* Silence the warning about missing syscall, thanks to Qian Cai
* Replace spaces with tabs in Kconfig additions, per Randy
* Add a selftest.
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
* rebase on v5.9-rc5
* drop boot time memory reservation patch
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
* rebase on v5.9-rc1
* Do not redefine PMD_PAGE_ORDER in fs/dax.c, thanks Kirill
* Make secret mappings exclusive by default and only require flags to
memfd_secret() system call for uncached mappings, thanks again Kirill :)
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
* Squash kernel-parameters.txt update into the commit that added the
command line option.
* Make uncached mode explicitly selectable by architectures. For now enable
it only on x86.
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
* Follow Michael's suggestion and name the new system call 'memfd_secret'
* Add kernel-parameters documentation about the boot option
* Fix i386-tinyconfig regression reported by the kbuild bot.
CONFIG_SECRETMEM now depends on !EMBEDDED to disable it on small systems
from one side and still make it available unconditionally on
architectures that support SET_DIRECT_MAP.
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (9):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/Kconfig | 7 +
arch/arm64/include/asm/cacheflush.h | 4 +-
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/mm/pageattr.c | 10 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 4 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
include/uapi/linux/secretmem.h | 8 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 3 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 446 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 +++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 891 insertions(+), 39 deletions(-)
create mode 100644 include/linux/secretmem.h
create mode 100644 include/uapi/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0
From: Zi Yan <ziy(a)nvidia.com>
Hi all,
With Matthew's THP in pagecache patches[1], we will be able to handle any size
pagecache THPs, but currently split_huge_page can only split a THP to order-0
pages. This can easily erase the benefit of having pagecache THPs, when
operations like truncate might want to keep pages larger than order-0. In
response, here is the patches to add support for splitting a THP to any lower
order pages. In addition, this patchset prepares for my PUD THP patchset[2],
since splitting a PUD THP to multiple PMD THPs can be handled by
split_huge_page_to_list_to_order function added by this patchset, which reduces
a lot of redundant code without just replicating split_huge_page for PUD THP.
The patchset is on top of Matthew's pagecache/next tree[3].
To ease the tests of split_huge_page functions, I added a new debugfs interface
at <debugfs>/split_huge_pages_in_range_pid, so developers can split THPs in a
given range from a process with the given pid by writing
"<pid>,<vaddr_start>,<vaddr_end>,<to_order>" to the interface. I also added a
new test program to test 1) split PMD THPs, 2) split pagecache THPs to any lower
order, and 3) truncating a pagecache THP to a page with a lower order.
Suggestions and comments are welcome. Thanks.
[1] https://lore.kernel.org/linux-mm/20201029193405.29125-1-willy@infradead.org/
[2] https://lore.kernel.org/linux-mm/20200928175428.4110504-1-zi.yan@sent.com/
[3] https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/next
Zi Yan (6):
mm: huge_memory: add new debugfs interface to trigger split huge page
on any page range.
mm: memcg: make memcg huge page split support any order split.
mm: page_owner: add support for splitting to any order in split
page_owner.
mm: thp: add support for split huge page to any lower order pages.
mm: truncate: split thp to a non-zero order if possible.
mm: huge_memory: enable debugfs to split huge pages to any order.
include/linux/huge_mm.h | 8 +
include/linux/memcontrol.h | 5 +-
include/linux/page_owner.h | 7 +-
mm/huge_memory.c | 177 ++++++++++--
mm/internal.h | 1 +
mm/memcontrol.c | 4 +-
mm/migrate.c | 2 +-
mm/page_alloc.c | 2 +-
mm/page_owner.c | 6 +-
mm/swap.c | 1 -
mm/truncate.c | 22 +-
tools/testing/selftests/vm/Makefile | 1 +
.../selftests/vm/split_huge_page_test.c | 255 ++++++++++++++++++
13 files changed, 453 insertions(+), 38 deletions(-)
create mode 100644 tools/testing/selftests/vm/split_huge_page_test.c
--
2.28.0
From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will have desired protection bits set in the user page
table. For instance, current implementation allows uncached mappings.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, in the future the secret mappings may be used as a mean to
protect guest memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/secret-memory-preloade…
that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
To limit fragmentation of the direct map to splitting only PUD-size pages,
I've added an amortizing cache of PMD-size pages to each file descriptor
that is used as an allocation pool for the secret memory areas.
As the memory allocated by secretmem becomes unmovable, we use CMA to back
large page caches so that page allocator won't be surprised by failing attempt
to migrate these pages.
v8:
* Use CMA for all secretmem allocations as David suggested
* Update memcg accounting after transtion to CMA
* Prevent hibernation when there are active secretmem users
* Add zeroing of the memory before releasing it back to cma/page allocator
* Rebase on v5.10-rc2-mmotm-2020-11-07-21-40
v7: https://lore.kernel.org/lkml/20201026083752.13267-1-rppt@kernel.org
* Use set_direct_map() instead of __kernel_map_pages() to ensure error
handling in case the direct map update fails
* Add accounting of large pages used to reduce the direct map fragmentation
* Teach get_user_pages() and frieds to refuse get/pin secretmem pages
v6: https://lore.kernel.org/lkml/20200924132904.1391-1-rppt@kernel.org
* Silence the warning about missing syscall, thanks to Qian Cai
* Replace spaces with tabs in Kconfig additions, per Randy
* Add a selftest.
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
* rebase on v5.9-rc5
* drop boot time memory reservation patch
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
* rebase on v5.9-rc1
* Do not redefine PMD_PAGE_ORDER in fs/dax.c, thanks Kirill
* Make secret mappings exclusive by default and only require flags to
memfd_secret() system call for uncached mappings, thanks again Kirill :)
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
* Squash kernel-parameters.txt update into the commit that added the
command line option.
* Make uncached mode explicitly selectable by architectures. For now enable
it only on x86.
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
* Follow Michael's suggestion and name the new system call 'memfd_secret'
* Add kernel-parameters documentation about the boot option
* Fix i386-tinyconfig regression reported by the kbuild bot.
CONFIG_SECRETMEM now depends on !EMBEDDED to disable it on small systems
from one side and still make it available unconditionally on
architectures that support SET_DIRECT_MAP.
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (9):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
set_memory: allow set_direct_map_*_noflush() for multiple pages
mm: introduce memfd_secret system call to create "secret" memory areas
secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: add memcg accounting
PM: hibernate: disable when there are active secretmem users
arch, mm: wire up memfd_secret system call were relevant
secretmem: test: add basic selftest for memfd_secret(2)
arch/Kconfig | 7 +
arch/arm64/include/asm/cacheflush.h | 4 +-
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/arm64/mm/pageattr.c | 10 +-
arch/riscv/include/asm/set_memory.h | 4 +-
arch/riscv/include/asm/unistd.h | 1 +
arch/riscv/mm/pageattr.c | 8 +-
arch/x86/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
arch/x86/include/asm/set_memory.h | 4 +-
arch/x86/mm/pat/set_memory.c | 8 +-
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/secretmem.h | 30 ++
include/linux/set_memory.h | 4 +-
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 6 +-
include/uapi/linux/magic.h | 1 +
include/uapi/linux/secretmem.h | 8 +
kernel/power/hibernate.c | 5 +-
kernel/power/snapshot.c | 4 +-
kernel/sys_ni.c | 2 +
mm/Kconfig | 5 +
mm/Makefile | 1 +
mm/filemap.c | 2 +-
mm/gup.c | 10 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 451 ++++++++++++++++++++++
mm/vmalloc.c | 5 +-
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 298 ++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 +
38 files changed, 895 insertions(+), 39 deletions(-)
create mode 100644 include/linux/secretmem.h
create mode 100644 include/uapi/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
base-commit: 9f8ce377d420db12b19d6a4f636fecbd88a725a5
--
2.28.0