From: Mike Rapoport <rppt(a)linux.ibm.com>
Hi,
This is an implementation of "secret" mappings backed by a file descriptor.
I've dropped the boot time reservation patch for now as it is not strictly
required for the basic usage and can be easily added later either with or
without CMA.
v6 changes:
* Silence the warning about missing syscall, thanks to Qian Cai
* Replace spaces with tabs in Kconfig additions, per Randy
* Add a selftest.
v5 changes:
* rebase on v5.9-rc5
* drop boot time memory reservation patch
v4 changes:
* rebase on v5.9-rc1
* Do not redefine PMD_PAGE_ORDER in fs/dax.c, thanks Kirill
* Make secret mappings exclusive by default and only require flags to
memfd_secret() system call for uncached mappings, thanks again Kirill :)
v3 changes:
* Squash kernel-parameters.txt update into the commit that added the
command line option.
* Make uncached mode explicitly selectable by architectures. For now enable
it only on x86.
v2 changes:
* Follow Michael's suggestion and name the new system call 'memfd_secret'
* Add kernel-parameters documentation about the boot option
* Fix i386-tinyconfig regression reported by the kbuild bot.
CONFIG_SECRETMEM now depends on !EMBEDDED to disable it on small systems
from one side and still make it available unconditionally on
architectures that support SET_DIRECT_MAP.
The file descriptor backing secret memory mappings is created using a
dedicated memfd_secret system call The desired protection mode for the
memory is configured using flags parameter of the system call. The mmap()
of the file descriptor created with memfd_secret() will create a "secret"
memory mapping. The pages in that mapping will be marked as not present in
the direct map and will have desired protection bits set in the user page
table. For instance, current implementation allows uncached mappings.
Although normally Linux userspace mappings are protected from other users,
such secret mappings are useful for environments where a hostile tenant is
trying to trick the kernel into giving them access to other tenants
mappings.
Additionally, the secret mappings may be used as a mean to protect guest
memory in a virtual machine host.
For demonstration of secret memory usage we've created a userspace library
[1] that does two things: the first is act as a preloader for openssl to
redirect all the OPENSSL_malloc calls to secret memory meaning any secret
keys get automatically protected this way and the other thing it does is
expose the API to the user who needs it. We anticipate that a lot of the
use cases would be like the openssl one: many toolkits that deal with
secret keys already have special handling for the memory to try to give
them greater protection, so this would simply be pluggable into the
toolkits without any need for user application modification.
I've hesitated whether to continue to use new flags to memfd_create() or to
add a new system call and I've decided to use a new system call after I've
started to look into man pages update. There would have been two completely
independent descriptions and I think it would have been very confusing.
Hiding secret memory mappings behind an anonymous file allows (ab)use of
the page cache for tracking pages allocated for the "secret" mappings as
well as using address_space_operations for e.g. page migration callbacks.
The anonymous file may be also used implicitly, like hugetlb files, to
implement mmap(MAP_SECRET) and use the secret memory areas with "native" mm
ABIs in the future.
As the fragmentation of the direct map was one of the major concerns raised
during the previous postings, I've added an amortizing cache of PMD-size
pages to each file descriptor that is used as an allocation pool for the
secret memory areas.
v5: https://lore.kernel.org/lkml/20200916073539.3552-1-rppt@kernel.org
v4: https://lore.kernel.org/lkml/20200818141554.13945-1-rppt@kernel.org
v3: https://lore.kernel.org/lkml/20200804095035.18778-1-rppt@kernel.org
v2: https://lore.kernel.org/lkml/20200727162935.31714-1-rppt@kernel.org
v1: https://lore.kernel.org/lkml/20200720092435.17469-1-rppt@kernel.org
Mike Rapoport (6):
mm: add definition of PMD_PAGE_ORDER
mmap: make mlock_future_check() global
mm: introduce memfd_secret system call to create "secret" memory areas
arch, mm: wire up memfd_secret system call were relevant
mm: secretmem: use PMD-size pages to amortize direct map fragmentation
secretmem: test: add basic selftest for memfd_secret(2)
arch/Kconfig | 7 +
arch/arm64/include/asm/unistd.h | 2 +-
arch/arm64/include/asm/unistd32.h | 2 +
arch/arm64/include/uapi/asm/unistd.h | 1 +
arch/riscv/include/asm/unistd.h | 1 +
arch/x86/Kconfig | 1 +
arch/x86/entry/syscalls/syscall_32.tbl | 1 +
arch/x86/entry/syscalls/syscall_64.tbl | 1 +
fs/dax.c | 11 +-
include/linux/pgtable.h | 3 +
include/linux/syscalls.h | 1 +
include/uapi/asm-generic/unistd.h | 7 +-
include/uapi/linux/magic.h | 1 +
include/uapi/linux/secretmem.h | 8 +
kernel/sys_ni.c | 2 +
mm/Kconfig | 4 +
mm/Makefile | 1 +
mm/internal.h | 3 +
mm/mmap.c | 5 +-
mm/secretmem.c | 333 ++++++++++++++++++++++
scripts/checksyscalls.sh | 4 +
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +-
tools/testing/selftests/vm/memfd_secret.c | 296 +++++++++++++++++++
tools/testing/selftests/vm/run_vmtests | 17 ++
25 files changed, 703 insertions(+), 13 deletions(-)
create mode 100644 include/uapi/linux/secretmem.h
create mode 100644 mm/secretmem.c
create mode 100644 tools/testing/selftests/vm/memfd_secret.c
--
2.28.0
hello,
i wrote a small program to check for the existence of "config" files
for testing projects under kselftest framework.
chmod 755 test_config.py
This file should be located in "tools/testing/selftests"
This can be run as "./test_config.py"
--
software engineer
rajagiri school of engineering and technology - autonomous
Nowadays, there are increasing requirements to benchmark the performance
of dma_map and dma_unmap particually while the device is attached to an
IOMMU.
This patchset provides the benchmark infrastruture for streaming DMA
mapping. The architecture of the code is pretty much similar with GUP
benchmark:
* mm/gup_benchmark.c provides kernel interface;
* tools/testing/selftests/vm/gup_benchmark.c provides user program to
call the interface provided by mm/gup_benchmark.c.
In our case, kernel/dma/map_benchmark.c is like mm/gup_benchmark.c;
tools/testing/selftests/dma/dma_map_benchmark.c is like tools/testing/
selftests/vm/gup_benchmark.c
A major difference with GUP benchmark is DMA_MAP benchmark needs to run
on a device. Considering one board with below devices and IOMMUs
device A ------- IOMMU 1
device B ------- IOMMU 2
device C ------- non-IOMMU
Different devices might attach to different IOMMU or non-IOMMU. To make
benchmark run, we can either
* create a virtual device and hack the kernel code to attach the virtual
device to IOMMU1, IOMMU2 or non-IOMMU.
* use the existing driver_override mechinism, unbind device A,B, OR c from
their original driver and bind A to dma_map_benchmark platform driver or
pci driver for benchmarking.
In this patchset, I prefer to use the driver_override and avoid the ugly
hack in kernel. We can dynamically switch device behind different IOMMUs
to get the performance of IOMMU or non-IOMMU.
-v3:
* fix build issues reported by 0day kernel test robot
-v2:
* add PCI support; v1 supported platform devices only
* replace ssleep by msleep_interruptible() to permit users to exit
benchmark before it is completed
* many changes according to Robin's suggestions, thanks! Robin
- add standard deviation output to reflect the worst case
- check users' parameters strictly like the number of threads
- make cache dirty before dma_map
- fix unpaired dma_map_page and dma_unmap_single;
- remove redundant "long long" before ktime_to_ns();
- use devm_add_action()
Barry Song (2):
dma-mapping: add benchmark support for streaming DMA APIs
selftests/dma: add test application for DMA_MAP_BENCHMARK
MAINTAINERS | 6 +
kernel/dma/Kconfig | 8 +
kernel/dma/Makefile | 1 +
kernel/dma/map_benchmark.c | 296 ++++++++++++++++++
tools/testing/selftests/dma/Makefile | 6 +
tools/testing/selftests/dma/config | 1 +
.../testing/selftests/dma/dma_map_benchmark.c | 87 +++++
7 files changed, 405 insertions(+)
create mode 100644 kernel/dma/map_benchmark.c
create mode 100644 tools/testing/selftests/dma/Makefile
create mode 100644 tools/testing/selftests/dma/config
create mode 100644 tools/testing/selftests/dma/dma_map_benchmark.c
--
2.25.1
Implementation of support for parameterized testing in KUnit. This
approach requires the creation of a test case using the
KUNIT_CASE_PARAM() macro that accepts a generator function as input.
This generator function should return the next parameter given the
previous parameter in parameterized tests. It also provides a macro to
generate common-case generators based on arrays. Generators may also
optionally provide a human-readable description of parameters, which is
displayed where available.
Note, currently the result of each parameter run is displayed in
diagnostic lines, and only the overall test case output summarizes
TAP-compliant success or failure of all parameter runs. In future, when
supported by kunit-tool, these can be turned into subsubtest outputs.
Signed-off-by: Arpitha Raghunandan <98.arpi(a)gmail.com>
Co-developed-by: Marco Elver <elver(a)google.com>
Signed-off-by: Marco Elver <elver(a)google.com>
---
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters;
if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
- Before execution of parameterized test case, count number of
parameters and display number of parameters. Currently also as a
diagnostic line, but this may be used in future to generate a subsubtest
plan. A requirement of this change is that generators must generate a
deterministic number of parameters.
Changes v5->v6:
- Fix alignment to maintain consistency
Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed
as if it was its own test case, with its own test initialization and cleanup
(init and exit are called, etc.). However, we cannot add new test cases per TAP
protocol once we have already started execution. Instead, log the result of
each parameter run as a diagnostic comment.
Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index
Changes v2->v3:
- Modifictaion of generator macro and method
Changes v1->v2:
- Use of a generator method to access test case parameters
include/kunit/test.h | 51 ++++++++++++++++++++++++++++++++++++++
lib/kunit/test.c | 59 ++++++++++++++++++++++++++++++++++----------
2 files changed, 97 insertions(+), 13 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h
index db1b0ae666c4..cf5f33b1c890 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -94,6 +94,9 @@ struct kunit;
/* Size of log associated with test. */
#define KUNIT_LOG_SIZE 512
+/* Maximum size of parameter description string. */
+#define KUNIT_PARAM_DESC_SIZE 64
+
/*
* TAP specifies subtest stream indentation of 4 spaces, 8 spaces for a
* sub-subtest. See the "Subtests" section in
@@ -107,6 +110,7 @@ struct kunit;
*
* @run_case: the function representing the actual test case.
* @name: the name of the test case.
+ * @generate_params: the generator function for parameterized tests.
*
* A test case is a function with the signature,
* ``void (*)(struct kunit *)``
@@ -141,6 +145,7 @@ struct kunit;
struct kunit_case {
void (*run_case)(struct kunit *test);
const char *name;
+ const void* (*generate_params)(const void *prev, char *desc);
/* private: internal use only. */
bool success;
@@ -163,6 +168,27 @@ static inline char *kunit_status_to_string(bool status)
*/
#define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name }
+/**
+ * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case
+ *
+ * @test_name: a reference to a test case function.
+ * @gen_params: a reference to a parameter generator function.
+ *
+ * The generator function::
+ *
+ * const void* gen_params(const void *prev, char *desc)
+ *
+ * is used to lazily generate a series of arbitrarily typed values that fit into
+ * a void*. The argument @prev is the previously returned value, which should be
+ * used to derive the next value; @prev is set to NULL on the initial generator
+ * call. When no more values are available, the generator must return NULL.
+ * Optionally write a string into @desc (size of KUNIT_PARAM_DESC_SIZE)
+ * describing the parameter.
+ */
+#define KUNIT_CASE_PARAM(test_name, gen_params) \
+ { .run_case = test_name, .name = #test_name, \
+ .generate_params = gen_params }
+
/**
* struct kunit_suite - describes a related collection of &struct kunit_case
*
@@ -208,6 +234,10 @@ struct kunit {
const char *name; /* Read only after initialization! */
char *log; /* Points at case log after initialization */
struct kunit_try_catch try_catch;
+ /* param_value is the current parameter value for a test case. */
+ const void *param_value;
+ /* param_index stores the index of the parameter in parameterized tests. */
+ int param_index;
/*
* success starts as true, and may only be set to false during a
* test case; thus, it is safe to update this across multiple
@@ -1742,4 +1772,25 @@ do { \
fmt, \
##__VA_ARGS__)
+/**
+ * KUNIT_ARRAY_PARAM() - Define test parameter generator from an array.
+ * @name: prefix for the test parameter generator function.
+ * @array: array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses @array to generate parameters.
+ */
+#define KUNIT_ARRAY_PARAM(name, array, get_desc) \
+ static const void *name##_gen_params(const void *prev, char *desc) \
+ { \
+ typeof((array)[0]) * __next = prev ? ((typeof(__next)) prev) + 1 : (array); \
+ if (__next - (array) < ARRAY_SIZE((array))) { \
+ void (*__get_desc)(typeof(__next), char *) = get_desc; \
+ if (__get_desc) \
+ __get_desc(__next, desc); \
+ return __next; \
+ } \
+ return NULL; \
+ }
+
#endif /* _KUNIT_TEST_H */
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 750704abe89a..ec9494e914ef 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -325,39 +325,72 @@ static void kunit_catch_run_case(void *data)
* occur in a test case and reports them as failures.
*/
static void kunit_run_case_catch_errors(struct kunit_suite *suite,
- struct kunit_case *test_case)
+ struct kunit_case *test_case,
+ struct kunit *test)
{
struct kunit_try_catch_context context;
struct kunit_try_catch *try_catch;
- struct kunit test;
- kunit_init_test(&test, test_case->name, test_case->log);
- try_catch = &test.try_catch;
+ kunit_init_test(test, test_case->name, test_case->log);
+ try_catch = &test->try_catch;
kunit_try_catch_init(try_catch,
- &test,
+ test,
kunit_try_run_case,
kunit_catch_run_case);
- context.test = &test;
+ context.test = test;
context.suite = suite;
context.test_case = test_case;
kunit_try_catch_run(try_catch, &context);
- test_case->success = test.success;
-
- kunit_print_ok_not_ok(&test, true, test_case->success,
- kunit_test_case_num(suite, test_case),
- test_case->name);
+ test_case->success = test->success;
}
int kunit_run_tests(struct kunit_suite *suite)
{
+ char param_desc[KUNIT_PARAM_DESC_SIZE];
struct kunit_case *test_case;
kunit_print_subtest_start(suite);
- kunit_suite_for_each_test_case(suite, test_case)
- kunit_run_case_catch_errors(suite, test_case);
+ kunit_suite_for_each_test_case(suite, test_case) {
+ struct kunit test = { .param_value = NULL, .param_index = 0 };
+ bool test_success = true;
+
+ if (test_case->generate_params) {
+ /* Get initial param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(NULL, param_desc);
+ }
+
+ do {
+ kunit_run_case_catch_errors(suite, test_case, &test);
+ test_success &= test_case->success;
+
+ if (test_case->generate_params) {
+ if (param_desc[0] == '\0') {
+ snprintf(param_desc, sizeof(param_desc),
+ "param-%d", test.param_index);
+ }
+
+ kunit_log(KERN_INFO, &test,
+ KUNIT_SUBTEST_INDENT
+ "# %s: %s %d - %s",
+ test_case->name,
+ kunit_status_to_string(test.success),
+ test.param_index + 1, param_desc);
+
+ /* Get next param. */
+ param_desc[0] = '\0';
+ test.param_value = test_case->generate_params(test.param_value, param_desc);
+ test.param_index++;
+ }
+ } while (test.param_value);
+
+ kunit_print_ok_not_ok(&test, true, test_success,
+ kunit_test_case_num(suite, test_case),
+ test_case->name);
+ }
kunit_print_subtest_end(suite);
--
2.25.1