February 2020 - Linux-kselftest-mirror

[PATCH v3 0/7] binfmt_elf: Update READ_IMPLIES_EXEC logic for modern CPUs

by Kees Cook

Hi, This is a refresh of my earlier attempt to fix READ_IMPLIES_EXEC. I think it incorporates the feedback from v2, and I've now added a selftest. This series is for x86, arm, and arm64; I'd like it to go via -tip, though, just to keep this change together with the selftest. To that end, I'd like to collect Acks from the respective architecture maintainers. (Note that most other architectures don't suffer from this problem. e.g. powerpc's behavior appears to already be correct. MIPS may need adjusting but the history of CPU features and toolchain behavior is very unclear to me.) Repeating the commit log from later in the series: The READ_IMPLIES_EXEC work-around was designed for old toolchains that lacked the ELF PT_GNU_STACK marking under the assumption that toolchains that couldn't specify executable permission flags for the stack may not know how to do it correctly for any memory region. This logic is sensible for having ancient binaries coexist in a system with possibly NX memory, but was implemented in a way that equated having a PT_GNU_STACK marked executable as being as "broken" as lacking the PT_GNU_STACK marking entirely. Things like unmarked assembly and stack trampolines may cause PT_GNU_STACK to need an executable bit, but they do not imply all mappings must be executable. This confusion has led to situations where modern programs with explicitly marked executable stack are forced into the READ_IMPLIES_EXEC state when no such thing is needed. (And leads to unexpected failures when mmap()ing regions of device driver memory that wish to disallow VM_EXEC[1].) In looking for other reasons for the READ_IMPLIES_EXEC behavior, Jann Horn noted that glibc thread stacks have always been marked RWX (until 2003 when they started tracking the PT_GNU_STACK flag instead[2]). And musl doesn't support executable stacks at all[3]. As such, no breakage for multithreaded applications is expected from this change. [1] https://lkml.kernel.org/r/20190418055759.GA3155@mellanox.com [2] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=54ee14b3882 [3] https://lkml.kernel.org/r/20190423192534.GN23599@brightrain.aerifal.cx -Kees v3: - split steps in to distinct patches - include arm32 and arm64/compat - add selftests to validate behavior v2: https://lore.kernel.org/lkml/20190424203408.GA11386@beast/ v1: https://lore.kernel.org/lkml/20190423181210.GA2443@beast/ Kees Cook (7): x86/elf: Add table to document READ_IMPLIES_EXEC x86/elf: Split READ_IMPLIES_EXEC from executable GNU_STACK x86/elf: Disable automatic READ_IMPLIES_EXEC for 64-bit address spaces arm32/64, elf: Add tables to document READ_IMPLIES_EXEC arm32/64, elf: Split READ_IMPLIES_EXEC from executable GNU_STACK arm64, elf: Disable automatic READ_IMPLIES_EXEC for 64-bit address spaces selftests/exec: Add READ_IMPLIES_EXEC tests arch/arm/kernel/elf.c | 27 +++- arch/arm64/include/asm/elf.h | 23 +++- arch/x86/include/asm/elf.h | 22 +++- fs/compat_binfmt_elf.c | 5 + tools/testing/selftests/exec/Makefile | 42 +++++- .../selftests/exec/read_implies_exec.c | 121 ++++++++++++++++++ .../selftests/exec/strip-gnu-stack-bits.c | 34 +++++ .../testing/selftests/exec/strip-gnu-stack.c | 69 ++++++++++ 8 files changed, 336 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/exec/read_implies_exec.c create mode 100644 tools/testing/selftests/exec/strip-gnu-stack-bits.c create mode 100644 tools/testing/selftests/exec/strip-gnu-stack.c -- 2.20.1

4 years, 4 months

4
16
0 0

[RFC v1] xarray: start converting test_xarray to KUnit

by Brendan Higgins

xarray predates KUnit; however, it would be good to get all unit tests in the kernel implemented using the same framework. This initial conversion is a bit hacky as it doesn't switch everything over to the KUnit semantics; however, it does break out the test cases as KUnit test cases, and calls KUNIT_FAIL() when XA_BUG_ON() fails. So the test does now take advantage of now reporting using the TAP format. The reason I didn't convert everything over the KUnit semantics is that I didn't want to get too far into this before getting some feedback from Willy. I don't think there is anything I have made worse with the changes I have made so far; this also at least serves as a starting point. TODO: - Convert tests to use KUnit semantics. - Figured out what to do with tools/testing/radix-tree/ xarray stuff. Signed-off-by: Brendan Higgins <brendanhiggins(a)google.com> --- include/linux/xarray.h | 2 +- lib/Kconfig.debug | 1 + lib/test_xarray.c | 1228 +++++++++++++++++++++------------------- lib/xarray.c | 4 +- 4 files changed, 658 insertions(+), 577 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index f73e1775ded01..85cded5fd7917 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1140,7 +1140,7 @@ struct xa_node { void xa_dump(const struct xarray *); void xa_dump_node(const struct xa_node *); -#ifdef XA_DEBUG +#ifdef CONFIG_TEST_XARRAY #define XA_BUG_ON(xa, x) do { \ if (x) { \ xa_dump(xa); \ diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 69def4a9df009..aa51269fb34f3 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1893,6 +1893,7 @@ config TEST_UUID config TEST_XARRAY tristate "Test the XArray code at runtime" + depends on KUNIT config TEST_OVERFLOW tristate "Test check_*_overflow() functions at runtime" diff --git a/lib/test_xarray.c b/lib/test_xarray.c index 55c14e8c88591..6c4c545f7858d 100644 --- a/lib/test_xarray.c +++ b/lib/test_xarray.c @@ -8,26 +8,16 @@ #include <linux/xarray.h> #include <linux/module.h> +#include <kunit/test.h> -static unsigned int tests_run; -static unsigned int tests_passed; - -#ifndef XA_DEBUG -# ifdef __KERNEL__ -void xa_dump(const struct xarray *xa) { } -# endif #undef XA_BUG_ON -#define XA_BUG_ON(xa, x) do { \ - tests_run++; \ +#define XA_BUG_ON(test, xa, x) do { \ if (x) { \ - printk("BUG at %s:%d\n", __func__, __LINE__); \ + KUNIT_FAIL(test, ""); \ xa_dump(xa); \ dump_stack(); \ - } else { \ - tests_passed++; \ } \ } while (0) -#endif static void *xa_mk_index(unsigned long index) { @@ -39,25 +29,32 @@ static void *xa_store_index(struct xarray *xa, unsigned long index, gfp_t gfp) return xa_store(xa, index, xa_mk_index(index), gfp); } -static void xa_insert_index(struct xarray *xa, unsigned long index) +static void xa_insert_index(struct kunit *test, + struct xarray *xa, + unsigned long index) { - XA_BUG_ON(xa, xa_insert(xa, index, xa_mk_index(index), + XA_BUG_ON(test, xa, xa_insert(xa, index, xa_mk_index(index), GFP_KERNEL) != 0); } -static void xa_alloc_index(struct xarray *xa, unsigned long index, gfp_t gfp) +static void xa_alloc_index(struct kunit *test, + struct xarray *xa, + unsigned long index, + gfp_t gfp) { u32 id; - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(index), xa_limit_32b, + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(index), xa_limit_32b, gfp) != 0); - XA_BUG_ON(xa, id != index); + XA_BUG_ON(test, xa, id != index); } -static void xa_erase_index(struct xarray *xa, unsigned long index) +static void xa_erase_index(struct kunit *test, + struct xarray *xa, + unsigned long index) { - XA_BUG_ON(xa, xa_erase(xa, index) != xa_mk_index(index)); - XA_BUG_ON(xa, xa_load(xa, index) != NULL); + XA_BUG_ON(test, xa, xa_erase(xa, index) != xa_mk_index(index)); + XA_BUG_ON(test, xa, xa_load(xa, index) != NULL); } /* @@ -80,24 +77,31 @@ static void *xa_store_order(struct xarray *xa, unsigned long index, return curr; } -static noinline void check_xa_err(struct xarray *xa) +static noinline void check_xa_err(struct kunit *test) { - XA_BUG_ON(xa, xa_err(xa_store_index(xa, 0, GFP_NOWAIT)) != 0); - XA_BUG_ON(xa, xa_err(xa_erase(xa, 0)) != 0); + struct xarray *xa = test->priv; + + XA_BUG_ON(test, xa, xa_err(xa_store_index(xa, 0, GFP_NOWAIT)) != 0); + XA_BUG_ON(test, xa, xa_err(xa_erase(xa, 0)) != 0); #ifndef __KERNEL__ /* The kernel does not fail GFP_NOWAIT allocations */ - XA_BUG_ON(xa, xa_err(xa_store_index(xa, 1, GFP_NOWAIT)) != -ENOMEM); - XA_BUG_ON(xa, xa_err(xa_store_index(xa, 1, GFP_NOWAIT)) != -ENOMEM); + XA_BUG_ON(test, xa, + xa_err(xa_store_index(xa, 1, GFP_NOWAIT)) != -ENOMEM); + XA_BUG_ON(test, + xa, xa_err(xa_store_index(xa, 1, GFP_NOWAIT)) != -ENOMEM); #endif - XA_BUG_ON(xa, xa_err(xa_store_index(xa, 1, GFP_KERNEL)) != 0); - XA_BUG_ON(xa, xa_err(xa_store(xa, 1, xa_mk_value(0), GFP_KERNEL)) != 0); - XA_BUG_ON(xa, xa_err(xa_erase(xa, 1)) != 0); + XA_BUG_ON(test, xa, xa_err(xa_store_index(xa, 1, GFP_KERNEL)) != 0); + XA_BUG_ON(test, xa, + xa_err(xa_store(xa, 1, xa_mk_value(0), GFP_KERNEL)) != 0); + XA_BUG_ON(test, xa, xa_err(xa_erase(xa, 1)) != 0); // kills the test-suite :-( -// XA_BUG_ON(xa, xa_err(xa_store(xa, 0, xa_mk_internal(0), 0)) != -EINVAL); +// XA_BUG_ON(test, xa, +// xa_err(xa_store(xa, 0, xa_mk_internal(0), 0)) != -EINVAL); } -static noinline void check_xas_retry(struct xarray *xa) +static noinline void check_xas_retry(struct kunit *test) { + struct xarray *xa = test->priv; XA_STATE(xas, xa, 0); void *entry; @@ -105,23 +109,23 @@ static noinline void check_xas_retry(struct xarray *xa) xa_store_index(xa, 1, GFP_KERNEL); rcu_read_lock(); - XA_BUG_ON(xa, xas_find(&xas, ULONG_MAX) != xa_mk_value(0)); - xa_erase_index(xa, 1); - XA_BUG_ON(xa, !xa_is_retry(xas_reload(&xas))); - XA_BUG_ON(xa, xas_retry(&xas, NULL)); - XA_BUG_ON(xa, xas_retry(&xas, xa_mk_value(0))); + XA_BUG_ON(test, xa, xas_find(&xas, ULONG_MAX) != xa_mk_value(0)); + xa_erase_index(test, xa, 1); + XA_BUG_ON(test, xa, !xa_is_retry(xas_reload(&xas))); + XA_BUG_ON(test, xa, xas_retry(&xas, NULL)); + XA_BUG_ON(test, xa, xas_retry(&xas, xa_mk_value(0))); xas_reset(&xas); - XA_BUG_ON(xa, xas.xa_node != XAS_RESTART); - XA_BUG_ON(xa, xas_next_entry(&xas, ULONG_MAX) != xa_mk_value(0)); - XA_BUG_ON(xa, xas.xa_node != NULL); + XA_BUG_ON(test, xa, xas.xa_node != XAS_RESTART); + XA_BUG_ON(test, xa, xas_next_entry(&xas, ULONG_MAX) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xas.xa_node != NULL); rcu_read_unlock(); - XA_BUG_ON(xa, xa_store_index(xa, 1, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, 1, GFP_KERNEL) != NULL); rcu_read_lock(); - XA_BUG_ON(xa, !xa_is_internal(xas_reload(&xas))); + XA_BUG_ON(test, xa, !xa_is_internal(xas_reload(&xas))); xas.xa_node = XAS_RESTART; - XA_BUG_ON(xa, xas_next_entry(&xas, ULONG_MAX) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xas_next_entry(&xas, ULONG_MAX) != xa_mk_value(0)); rcu_read_unlock(); /* Make sure we can iterate through retry entries */ @@ -137,64 +141,67 @@ static noinline void check_xas_retry(struct xarray *xa) } xas_unlock(&xas); - xa_erase_index(xa, 0); - xa_erase_index(xa, 1); + xa_erase_index(test, xa, 0); + xa_erase_index(test, xa, 1); } -static noinline void check_xa_load(struct xarray *xa) +static noinline void check_xa_load(struct kunit *test) { + struct xarray *xa = test->priv; unsigned long i, j; for (i = 0; i < 1024; i++) { for (j = 0; j < 1024; j++) { void *entry = xa_load(xa, j); if (j < i) - XA_BUG_ON(xa, xa_to_value(entry) != j); + XA_BUG_ON(test, xa, xa_to_value(entry) != j); else - XA_BUG_ON(xa, entry); + XA_BUG_ON(test, xa, entry); } - XA_BUG_ON(xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); } for (i = 0; i < 1024; i++) { for (j = 0; j < 1024; j++) { void *entry = xa_load(xa, j); if (j >= i) - XA_BUG_ON(xa, xa_to_value(entry) != j); + XA_BUG_ON(test, xa, xa_to_value(entry) != j); else - XA_BUG_ON(xa, entry); + XA_BUG_ON(test, xa, entry); } - xa_erase_index(xa, i); + xa_erase_index(test, xa, i); } - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_xa_mark_1(struct xarray *xa, unsigned long index) +static noinline void check_xa_mark_1(struct kunit *test, + struct xarray *xa, + unsigned long index) { unsigned int order; unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 8 : 1; /* NULL elements have no marks set */ - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_0)); xa_set_mark(xa, index, XA_MARK_0); - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_0)); /* Storing a pointer will not make a mark appear */ - XA_BUG_ON(xa, xa_store_index(xa, index, GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_store_index(xa, index, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_0)); xa_set_mark(xa, index, XA_MARK_0); - XA_BUG_ON(xa, !xa_get_mark(xa, index, XA_MARK_0)); + XA_BUG_ON(test, xa, !xa_get_mark(xa, index, XA_MARK_0)); /* Setting one mark will not set another mark */ - XA_BUG_ON(xa, xa_get_mark(xa, index + 1, XA_MARK_0)); - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_1)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index + 1, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_1)); /* Storing NULL clears marks, and they can't be set again */ - xa_erase_index(xa, index); - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_0)); + xa_erase_index(test, xa, index); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_0)); xa_set_mark(xa, index, XA_MARK_0); - XA_BUG_ON(xa, xa_get_mark(xa, index, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, index, XA_MARK_0)); /* * Storing a multi-index entry over entries with marks gives the @@ -206,11 +213,11 @@ static noinline void check_xa_mark_1(struct xarray *xa, unsigned long index) unsigned long next = base + (1UL << order); unsigned long i; - XA_BUG_ON(xa, xa_store_index(xa, index + 1, GFP_KERNEL)); + XA_BUG_ON(test, xa, xa_store_index(xa, index + 1, GFP_KERNEL)); xa_set_mark(xa, index + 1, XA_MARK_0); - XA_BUG_ON(xa, xa_store_index(xa, index + 2, GFP_KERNEL)); + XA_BUG_ON(test, xa, xa_store_index(xa, index + 2, GFP_KERNEL)); xa_set_mark(xa, index + 2, XA_MARK_2); - XA_BUG_ON(xa, xa_store_index(xa, next, GFP_KERNEL)); + XA_BUG_ON(test, xa, xa_store_index(xa, next, GFP_KERNEL)); xa_store_order(xa, index, order, xa_mk_index(index), GFP_KERNEL); for (i = base; i < next; i++) { @@ -218,16 +225,16 @@ static noinline void check_xa_mark_1(struct xarray *xa, unsigned long index) unsigned int seen = 0; void *entry; - XA_BUG_ON(xa, !xa_get_mark(xa, i, XA_MARK_0)); - XA_BUG_ON(xa, xa_get_mark(xa, i, XA_MARK_1)); - XA_BUG_ON(xa, !xa_get_mark(xa, i, XA_MARK_2)); + XA_BUG_ON(test, xa, !xa_get_mark(xa, i, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, i, XA_MARK_1)); + XA_BUG_ON(test, xa, !xa_get_mark(xa, i, XA_MARK_2)); /* We should see two elements in the array */ rcu_read_lock(); xas_for_each(&xas, entry, ULONG_MAX) seen++; rcu_read_unlock(); - XA_BUG_ON(xa, seen != 2); + XA_BUG_ON(test, xa, seen != 2); /* One of which is marked */ xas_set(&xas, 0); @@ -236,19 +243,19 @@ static noinline void check_xa_mark_1(struct xarray *xa, unsigned long index) xas_for_each_marked(&xas, entry, ULONG_MAX, XA_MARK_0) seen++; rcu_read_unlock(); - XA_BUG_ON(xa, seen != 1); + XA_BUG_ON(test, xa, seen != 1); } - XA_BUG_ON(xa, xa_get_mark(xa, next, XA_MARK_0)); - XA_BUG_ON(xa, xa_get_mark(xa, next, XA_MARK_1)); - XA_BUG_ON(xa, xa_get_mark(xa, next, XA_MARK_2)); - xa_erase_index(xa, index); - xa_erase_index(xa, next); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_get_mark(xa, next, XA_MARK_0)); + XA_BUG_ON(test, xa, xa_get_mark(xa, next, XA_MARK_1)); + XA_BUG_ON(test, xa, xa_get_mark(xa, next, XA_MARK_2)); + xa_erase_index(test, xa, index); + xa_erase_index(test, xa, next); + XA_BUG_ON(test, xa, !xa_empty(xa)); } - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_xa_mark_2(struct xarray *xa) +static noinline void check_xa_mark_2(struct kunit *test, struct xarray *xa) { XA_STATE(xas, xa, 0); unsigned long index; @@ -261,7 +268,7 @@ static noinline void check_xa_mark_2(struct xarray *xa) xas_load(&xas); xas_init_marks(&xas); xas_unlock(&xas); - XA_BUG_ON(xa, !xa_get_mark(xa, 0, XA_MARK_0) == 0); + XA_BUG_ON(test, xa, !xa_get_mark(xa, 0, XA_MARK_0) == 0); for (index = 3500; index < 4500; index++) { xa_store_index(xa, index, GFP_KERNEL); @@ -273,196 +280,213 @@ static noinline void check_xa_mark_2(struct xarray *xa) xas_for_each_marked(&xas, entry, ULONG_MAX, XA_MARK_0) count++; rcu_read_unlock(); - XA_BUG_ON(xa, count != 1000); + XA_BUG_ON(test, xa, count != 1000); xas_lock(&xas); xas_for_each(&xas, entry, ULONG_MAX) { xas_init_marks(&xas); - XA_BUG_ON(xa, !xa_get_mark(xa, xas.xa_index, XA_MARK_0)); - XA_BUG_ON(xa, !xas_get_mark(&xas, XA_MARK_0)); + XA_BUG_ON(test, xa, !xa_get_mark(xa, xas.xa_index, XA_MARK_0)); + XA_BUG_ON(test, xa, !xas_get_mark(&xas, XA_MARK_0)); } xas_unlock(&xas); xa_destroy(xa); } -static noinline void check_xa_mark(struct xarray *xa) +static noinline void check_xa_mark(struct kunit *test) { + struct xarray *xa = test->priv; unsigned long index; for (index = 0; index < 16384; index += 4) - check_xa_mark_1(xa, index); + check_xa_mark_1(test, xa, index); - check_xa_mark_2(xa); + check_xa_mark_2(test, xa); } -static noinline void check_xa_shrink(struct xarray *xa) +static noinline void check_xa_shrink(struct kunit *test) { + struct xarray *xa = test->priv; XA_STATE(xas, xa, 1); struct xa_node *node; unsigned int order; unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 15 : 1; - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_store_index(xa, 0, GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_store_index(xa, 1, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_store_index(xa, 0, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, 1, GFP_KERNEL) != NULL); /* * Check that erasing the entry at 1 shrinks the tree and properly * marks the node as being deleted. */ xas_lock(&xas); - XA_BUG_ON(xa, xas_load(&xas) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xas_load(&xas) != xa_mk_value(1)); node = xas.xa_node; - XA_BUG_ON(xa, xa_entry_locked(xa, node, 0) != xa_mk_value(0)); - XA_BUG_ON(xa, xas_store(&xas, NULL) != xa_mk_value(1)); - XA_BUG_ON(xa, xa_load(xa, 1) != NULL); - XA_BUG_ON(xa, xas.xa_node != XAS_BOUNDS); - XA_BUG_ON(xa, xa_entry_locked(xa, node, 0) != XA_RETRY_ENTRY); - XA_BUG_ON(xa, xas_load(&xas) != NULL); + XA_BUG_ON(test, xa, xa_entry_locked(xa, node, 0) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xas_store(&xas, NULL) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xa_load(xa, 1) != NULL); + XA_BUG_ON(test, xa, xas.xa_node != XAS_BOUNDS); + XA_BUG_ON(test, xa, xa_entry_locked(xa, node, 0) != XA_RETRY_ENTRY); + XA_BUG_ON(test, xa, xas_load(&xas) != NULL); xas_unlock(&xas); - XA_BUG_ON(xa, xa_load(xa, 0) != xa_mk_value(0)); - xa_erase_index(xa, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_load(xa, 0) != xa_mk_value(0)); + xa_erase_index(test, xa, 0); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (order = 0; order < max_order; order++) { unsigned long max = (1UL << order) - 1; xa_store_order(xa, 0, order, xa_mk_value(0), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, max) != xa_mk_value(0)); - XA_BUG_ON(xa, xa_load(xa, max + 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, max) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xa_load(xa, max + 1) != NULL); rcu_read_lock(); node = xa_head(xa); rcu_read_unlock(); - XA_BUG_ON(xa, xa_store_index(xa, ULONG_MAX, GFP_KERNEL) != + XA_BUG_ON(test, xa, xa_store_index(xa, ULONG_MAX, GFP_KERNEL) != NULL); rcu_read_lock(); - XA_BUG_ON(xa, xa_head(xa) == node); + XA_BUG_ON(test, xa, xa_head(xa) == node); rcu_read_unlock(); - XA_BUG_ON(xa, xa_load(xa, max + 1) != NULL); - xa_erase_index(xa, ULONG_MAX); - XA_BUG_ON(xa, xa->xa_head != node); - xa_erase_index(xa, 0); + XA_BUG_ON(test, xa, xa_load(xa, max + 1) != NULL); + xa_erase_index(test, xa, ULONG_MAX); + XA_BUG_ON(test, xa, xa->xa_head != node); + xa_erase_index(test, xa, 0); } } -static noinline void check_insert(struct xarray *xa) +static noinline void check_insert(struct kunit *test) { + struct xarray *xa = test->priv; unsigned long i; for (i = 0; i < 1024; i++) { - xa_insert_index(xa, i); - XA_BUG_ON(xa, xa_load(xa, i - 1) != NULL); - XA_BUG_ON(xa, xa_load(xa, i + 1) != NULL); - xa_erase_index(xa, i); + xa_insert_index(test, xa, i); + XA_BUG_ON(test, xa, xa_load(xa, i - 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, i + 1) != NULL); + xa_erase_index(test, xa, i); } for (i = 10; i < BITS_PER_LONG; i++) { - xa_insert_index(xa, 1UL << i); - XA_BUG_ON(xa, xa_load(xa, (1UL << i) - 1) != NULL); - XA_BUG_ON(xa, xa_load(xa, (1UL << i) + 1) != NULL); - xa_erase_index(xa, 1UL << i); - - xa_insert_index(xa, (1UL << i) - 1); - XA_BUG_ON(xa, xa_load(xa, (1UL << i) - 2) != NULL); - XA_BUG_ON(xa, xa_load(xa, 1UL << i) != NULL); - xa_erase_index(xa, (1UL << i) - 1); + xa_insert_index(test, xa, 1UL << i); + XA_BUG_ON(test, xa, xa_load(xa, (1UL << i) - 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, (1UL << i) + 1) != NULL); + xa_erase_index(test, xa, 1UL << i); + + xa_insert_index(test, xa, (1UL << i) - 1); + XA_BUG_ON(test, xa, xa_load(xa, (1UL << i) - 2) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, 1UL << i) != NULL); + xa_erase_index(test, xa, (1UL << i) - 1); } - xa_insert_index(xa, ~0UL); - XA_BUG_ON(xa, xa_load(xa, 0UL) != NULL); - XA_BUG_ON(xa, xa_load(xa, ~1UL) != NULL); - xa_erase_index(xa, ~0UL); + xa_insert_index(test, xa, ~0UL); + XA_BUG_ON(test, xa, xa_load(xa, 0UL) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, ~1UL) != NULL); + xa_erase_index(test, xa, ~0UL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_cmpxchg(struct xarray *xa) +static noinline void check_cmpxchg(struct kunit *test) { + struct xarray *xa = test->priv; void *FIVE = xa_mk_value(5); void *SIX = xa_mk_value(6); void *LOTS = xa_mk_value(12345678); - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_store_index(xa, 12345678, GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_insert(xa, 12345678, xa, GFP_KERNEL) != -EBUSY); - XA_BUG_ON(xa, xa_cmpxchg(xa, 12345678, SIX, FIVE, GFP_KERNEL) != LOTS); - XA_BUG_ON(xa, xa_cmpxchg(xa, 12345678, LOTS, FIVE, GFP_KERNEL) != LOTS); - XA_BUG_ON(xa, xa_cmpxchg(xa, 12345678, FIVE, LOTS, GFP_KERNEL) != FIVE); - XA_BUG_ON(xa, xa_cmpxchg(xa, 5, FIVE, NULL, GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_cmpxchg(xa, 5, NULL, FIVE, GFP_KERNEL) != NULL); - xa_erase_index(xa, 12345678); - xa_erase_index(xa, 5); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_store_index(xa, 12345678, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_insert(xa, 12345678, xa, GFP_KERNEL) != -EBUSY); + XA_BUG_ON(test, xa, + xa_cmpxchg(xa, 12345678, SIX, FIVE, GFP_KERNEL) != LOTS); + XA_BUG_ON(test, xa, + xa_cmpxchg(xa, 12345678, LOTS, FIVE, GFP_KERNEL) != LOTS); + XA_BUG_ON(test, xa, + xa_cmpxchg(xa, 12345678, FIVE, LOTS, GFP_KERNEL) != FIVE); + XA_BUG_ON(test, xa, xa_cmpxchg(xa, 5, FIVE, NULL, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_cmpxchg(xa, 5, NULL, FIVE, GFP_KERNEL) != NULL); + xa_erase_index(test, xa, 12345678); + xa_erase_index(test, xa, 5); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_reserve(struct xarray *xa) +static noinline void check_reserve(struct kunit *test, struct xarray *xa) { void *entry; unsigned long index; int count; /* An array with a reserved entry is not empty */ - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_empty(xa)); - XA_BUG_ON(xa, xa_load(xa, 12345678)); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_load(xa, 12345678)); xa_release(xa, 12345678); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Releasing a used entry does nothing */ - XA_BUG_ON(xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_store_index(xa, 12345678, GFP_NOWAIT) != NULL); + XA_BUG_ON(test, xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_store_index(xa, 12345678, GFP_NOWAIT) != NULL); xa_release(xa, 12345678); - xa_erase_index(xa, 12345678); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, 12345678); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* cmpxchg sees a reserved entry as ZERO */ - XA_BUG_ON(xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_cmpxchg(xa, 12345678, XA_ZERO_ENTRY, + XA_BUG_ON(test, xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_cmpxchg(xa, 12345678, XA_ZERO_ENTRY, xa_mk_value(12345678), GFP_NOWAIT) != NULL); xa_release(xa, 12345678); - xa_erase_index(xa, 12345678); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, 12345678); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* xa_insert treats it as busy */ - XA_BUG_ON(xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_insert(xa, 12345678, xa_mk_value(12345678), 0) != + XA_BUG_ON(test, xa, xa_reserve(xa, 12345678, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_insert(xa, 12345678, xa_mk_value(12345678), 0) != -EBUSY); - XA_BUG_ON(xa, xa_empty(xa)); - XA_BUG_ON(xa, xa_erase(xa, 12345678) != NULL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_erase(xa, 12345678) != NULL); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Can iterate through a reserved entry */ xa_store_index(xa, 5, GFP_KERNEL); - XA_BUG_ON(xa, xa_reserve(xa, 6, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_reserve(xa, 6, GFP_KERNEL) != 0); xa_store_index(xa, 7, GFP_KERNEL); count = 0; xa_for_each(xa, index, entry) { - XA_BUG_ON(xa, index != 5 && index != 7); + XA_BUG_ON(test, xa, index != 5 && index != 7); count++; } - XA_BUG_ON(xa, count != 2); + XA_BUG_ON(test, xa, count != 2); /* If we free a reserved entry, we should be able to allocate it */ if (xa->xa_flags & XA_FLAGS_ALLOC) { u32 id; - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_value(8), + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_value(8), XA_LIMIT(5, 10), GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 8); + XA_BUG_ON(test, xa, id != 8); xa_release(xa, 6); - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_value(6), + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_value(6), XA_LIMIT(5, 10), GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 6); + XA_BUG_ON(test, xa, id != 6); } xa_destroy(xa); } -static noinline void check_xas_erase(struct xarray *xa) +static void check_reserve_all(struct kunit *test) +{ + struct xarray *xa = test->priv; + DEFINE_XARRAY_ALLOC(xa0); + + check_reserve(test, xa); + check_reserve(test, &xa0); +} + +static noinline void check_xas_erase(struct kunit *test) { + struct xarray *xa = test->priv; XA_STATE(xas, xa, 0); void *entry; unsigned long i, j; @@ -490,12 +514,12 @@ static noinline void check_xas_erase(struct xarray *xa) xas_set(&xas, 0); j = i; xas_for_each(&xas, entry, ULONG_MAX) { - XA_BUG_ON(xa, entry != xa_mk_index(j)); + XA_BUG_ON(test, xa, entry != xa_mk_index(j)); xas_store(&xas, NULL); j++; } xas_unlock(&xas); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } } @@ -508,21 +532,22 @@ static noinline void check_multi_store_1(struct xarray *xa, unsigned long index, unsigned long max = min + (1UL << order); xa_store_order(xa, index, order, xa_mk_index(index), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, min) != xa_mk_index(index)); - XA_BUG_ON(xa, xa_load(xa, max - 1) != xa_mk_index(index)); - XA_BUG_ON(xa, xa_load(xa, max) != NULL); - XA_BUG_ON(xa, xa_load(xa, min - 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, min) != xa_mk_index(index)); + XA_BUG_ON(test, xa, xa_load(xa, max - 1) != xa_mk_index(index)); + XA_BUG_ON(test, xa, xa_load(xa, max) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, min - 1) != NULL); xas_lock(&xas); - XA_BUG_ON(xa, xas_store(&xas, xa_mk_index(min)) != xa_mk_index(index)); + XA_BUG_ON(test, xa, + xas_store(&xas, xa_mk_index(min)) != xa_mk_index(index)); xas_unlock(&xas); - XA_BUG_ON(xa, xa_load(xa, min) != xa_mk_index(min)); - XA_BUG_ON(xa, xa_load(xa, max - 1) != xa_mk_index(min)); - XA_BUG_ON(xa, xa_load(xa, max) != NULL); - XA_BUG_ON(xa, xa_load(xa, min - 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, min) != xa_mk_index(min)); + XA_BUG_ON(test, xa, xa_load(xa, max - 1) != xa_mk_index(min)); + XA_BUG_ON(test, xa, xa_load(xa, max) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, min - 1) != NULL); - xa_erase_index(xa, min); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, min); + XA_BUG_ON(test, xa, !xa_empty(xa)); } static noinline void check_multi_store_2(struct xarray *xa, unsigned long index, @@ -532,11 +557,11 @@ static noinline void check_multi_store_2(struct xarray *xa, unsigned long index, xa_store_order(xa, index, order, xa_mk_value(0), GFP_KERNEL); xas_lock(&xas); - XA_BUG_ON(xa, xas_store(&xas, xa_mk_value(1)) != xa_mk_value(0)); - XA_BUG_ON(xa, xas.xa_index != index); - XA_BUG_ON(xa, xas_store(&xas, NULL) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xas_store(&xas, xa_mk_value(1)) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xas.xa_index != index); + XA_BUG_ON(test, xa, xas_store(&xas, NULL) != xa_mk_value(1)); xas_unlock(&xas); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } static noinline void check_multi_store_3(struct xarray *xa, unsigned long index, @@ -550,69 +575,70 @@ static noinline void check_multi_store_3(struct xarray *xa, unsigned long index, xas_lock(&xas); xas_for_each(&xas, entry, ULONG_MAX) { - XA_BUG_ON(xa, entry != xa_mk_index(index)); + XA_BUG_ON(test, xa, entry != xa_mk_index(index)); n++; } - XA_BUG_ON(xa, n != 1); + XA_BUG_ON(test, xa, n != 1); xas_set(&xas, index + 1); xas_for_each(&xas, entry, ULONG_MAX) { - XA_BUG_ON(xa, entry != xa_mk_index(index)); + XA_BUG_ON(test, xa, entry != xa_mk_index(index)); n++; } - XA_BUG_ON(xa, n != 2); + XA_BUG_ON(test, xa, n != 2); xas_unlock(&xas); xa_destroy(xa); } #endif -static noinline void check_multi_store(struct xarray *xa) +static noinline void check_multi_store(struct kunit *test) { #ifdef CONFIG_XARRAY_MULTI + struct xarray *xa = test->priv; unsigned long i, j, k; unsigned int max_order = (sizeof(long) == 4) ? 30 : 60; /* Loading from any position returns the same value */ xa_store_order(xa, 0, 1, xa_mk_value(0), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, 0) != xa_mk_value(0)); - XA_BUG_ON(xa, xa_load(xa, 1) != xa_mk_value(0)); - XA_BUG_ON(xa, xa_load(xa, 2) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, 0) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xa_load(xa, 1) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xa_load(xa, 2) != NULL); rcu_read_lock(); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->count != 2); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->nr_values != 2); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->count != 2); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->nr_values != 2); rcu_read_unlock(); /* Storing adjacent to the value does not alter the value */ xa_store(xa, 3, xa, GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, 0) != xa_mk_value(0)); - XA_BUG_ON(xa, xa_load(xa, 1) != xa_mk_value(0)); - XA_BUG_ON(xa, xa_load(xa, 2) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, 0) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xa_load(xa, 1) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xa_load(xa, 2) != NULL); rcu_read_lock(); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->count != 3); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->nr_values != 2); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->count != 3); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->nr_values != 2); rcu_read_unlock(); /* Overwriting multiple indexes works */ xa_store_order(xa, 0, 2, xa_mk_value(1), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, 0) != xa_mk_value(1)); - XA_BUG_ON(xa, xa_load(xa, 1) != xa_mk_value(1)); - XA_BUG_ON(xa, xa_load(xa, 2) != xa_mk_value(1)); - XA_BUG_ON(xa, xa_load(xa, 3) != xa_mk_value(1)); - XA_BUG_ON(xa, xa_load(xa, 4) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, 0) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xa_load(xa, 1) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xa_load(xa, 2) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xa_load(xa, 3) != xa_mk_value(1)); + XA_BUG_ON(test, xa, xa_load(xa, 4) != NULL); rcu_read_lock(); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->count != 4); - XA_BUG_ON(xa, xa_to_node(xa_head(xa))->nr_values != 4); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->count != 4); + XA_BUG_ON(test, xa, xa_to_node(xa_head(xa))->nr_values != 4); rcu_read_unlock(); /* We can erase multiple values with a single store */ xa_store_order(xa, 0, BITS_PER_LONG - 1, NULL, GFP_KERNEL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Even when the first slot is empty but the others aren't */ xa_store_index(xa, 1, GFP_KERNEL); xa_store_index(xa, 2, GFP_KERNEL); xa_store_order(xa, 0, 2, NULL, GFP_KERNEL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (i = 0; i < max_order; i++) { for (j = 0; j < max_order; j++) { @@ -622,13 +648,14 @@ static noinline void check_multi_store(struct xarray *xa) for (k = 0; k < max_order; k++) { void *entry = xa_load(xa, (1UL << k) - 1); if ((i < k) && (j < k)) - XA_BUG_ON(xa, entry != NULL); + XA_BUG_ON(test, xa, entry != NULL); else - XA_BUG_ON(xa, entry != xa_mk_index(j)); + XA_BUG_ON(test, + xa, entry != xa_mk_index(j)); } xa_erase(xa, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } } @@ -646,133 +673,142 @@ static noinline void check_multi_store(struct xarray *xa) #endif } -static noinline void check_xa_alloc_1(struct xarray *xa, unsigned int base) +static noinline void check_xa_alloc_1(struct kunit *test, struct xarray *xa, + unsigned int base) { int i; u32 id; - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* An empty array should assign %base to the first alloc */ - xa_alloc_index(xa, base, GFP_KERNEL); + xa_alloc_index(test, xa, base, GFP_KERNEL); /* Erasing it should make the array empty again */ - xa_erase_index(xa, base); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, base); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* And it should assign %base again */ - xa_alloc_index(xa, base, GFP_KERNEL); + xa_alloc_index(test, xa, base, GFP_KERNEL); /* Allocating and then erasing a lot should not lose base */ for (i = base + 1; i < 2 * XA_CHUNK_SIZE; i++) - xa_alloc_index(xa, i, GFP_KERNEL); + xa_alloc_index(test, xa, i, GFP_KERNEL); for (i = base; i < 2 * XA_CHUNK_SIZE; i++) - xa_erase_index(xa, i); - xa_alloc_index(xa, base, GFP_KERNEL); + xa_erase_index(test, xa, i); + xa_alloc_index(test, xa, base, GFP_KERNEL); /* Destroying the array should do the same as erasing */ xa_destroy(xa); /* And it should assign %base again */ - xa_alloc_index(xa, base, GFP_KERNEL); + xa_alloc_index(test, xa, base, GFP_KERNEL); /* The next assigned ID should be base+1 */ - xa_alloc_index(xa, base + 1, GFP_KERNEL); - xa_erase_index(xa, base + 1); + xa_alloc_index(test, xa, base + 1, GFP_KERNEL); + xa_erase_index(test, xa, base + 1); /* Storing a value should mark it used */ xa_store_index(xa, base + 1, GFP_KERNEL); - xa_alloc_index(xa, base + 2, GFP_KERNEL); + xa_alloc_index(test, xa, base + 2, GFP_KERNEL); /* If we then erase base, it should be free */ - xa_erase_index(xa, base); - xa_alloc_index(xa, base, GFP_KERNEL); + xa_erase_index(test, xa, base); + xa_alloc_index(test, xa, base, GFP_KERNEL); - xa_erase_index(xa, base + 1); - xa_erase_index(xa, base + 2); + xa_erase_index(test, xa, base + 1); + xa_erase_index(test, xa, base + 2); for (i = 1; i < 5000; i++) { - xa_alloc_index(xa, base + i, GFP_KERNEL); + xa_alloc_index(test, xa, base + i, GFP_KERNEL); } xa_destroy(xa); /* Check that we fail properly at the limit of allocation */ - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(UINT_MAX - 1), + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(UINT_MAX - 1), XA_LIMIT(UINT_MAX - 1, UINT_MAX), GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 0xfffffffeU); - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(UINT_MAX), + XA_BUG_ON(test, xa, id != 0xfffffffeU); + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(UINT_MAX), XA_LIMIT(UINT_MAX - 1, UINT_MAX), GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 0xffffffffU); + XA_BUG_ON(test, xa, id != 0xffffffffU); id = 3; - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(0), + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(0), XA_LIMIT(UINT_MAX - 1, UINT_MAX), GFP_KERNEL) != -EBUSY); - XA_BUG_ON(xa, id != 3); + XA_BUG_ON(test, xa, id != 3); xa_destroy(xa); - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(10), XA_LIMIT(10, 5), + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(10), XA_LIMIT(10, 5), GFP_KERNEL) != -EBUSY); - XA_BUG_ON(xa, xa_store_index(xa, 3, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_alloc(xa, &id, xa_mk_index(10), XA_LIMIT(10, 5), + XA_BUG_ON(test, xa, xa_store_index(xa, 3, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_alloc(xa, &id, xa_mk_index(10), XA_LIMIT(10, 5), GFP_KERNEL) != -EBUSY); - xa_erase_index(xa, 3); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, 3); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_xa_alloc_2(struct xarray *xa, unsigned int base) +static noinline void check_xa_alloc_2(struct kunit *test, struct xarray *xa, + unsigned int base) { unsigned int i, id; unsigned long index; void *entry; /* Allocate and free a NULL and check xa_empty() behaves */ - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != base); - XA_BUG_ON(xa, xa_empty(xa)); - XA_BUG_ON(xa, xa_erase(xa, id) != NULL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, + xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, id != base); + XA_BUG_ON(test, xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_erase(xa, id) != NULL); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Ditto, but check destroy instead of erase */ - XA_BUG_ON(xa, !xa_empty(xa)); - XA_BUG_ON(xa, xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != base); - XA_BUG_ON(xa, xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, + xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, id != base); + XA_BUG_ON(test, xa, xa_empty(xa)); xa_destroy(xa); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (i = base; i < base + 10; i++) { - XA_BUG_ON(xa, xa_alloc(xa, &id, NULL, xa_limit_32b, + XA_BUG_ON(test, xa, xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != i); + XA_BUG_ON(test, xa, id != i); } - XA_BUG_ON(xa, xa_store(xa, 3, xa_mk_index(3), GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_store(xa, 4, xa_mk_index(4), GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_store(xa, 4, NULL, GFP_KERNEL) != xa_mk_index(4)); - XA_BUG_ON(xa, xa_erase(xa, 5) != NULL); - XA_BUG_ON(xa, xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 5); + XA_BUG_ON(test, xa, + xa_store(xa, 3, xa_mk_index(3), GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, + xa_store(xa, 4, xa_mk_index(4), GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, + xa_store(xa, 4, NULL, GFP_KERNEL) != xa_mk_index(4)); + XA_BUG_ON(test, xa, xa_erase(xa, 5) != NULL); + XA_BUG_ON(test, xa, + xa_alloc(xa, &id, NULL, xa_limit_32b, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, id != 5); xa_for_each(xa, index, entry) { - xa_erase_index(xa, index); + xa_erase_index(test, xa, index); } for (i = base; i < base + 9; i++) { - XA_BUG_ON(xa, xa_erase(xa, i) != NULL); - XA_BUG_ON(xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_erase(xa, i) != NULL); + XA_BUG_ON(test, xa, xa_empty(xa)); } - XA_BUG_ON(xa, xa_erase(xa, 8) != NULL); - XA_BUG_ON(xa, xa_empty(xa)); - XA_BUG_ON(xa, xa_erase(xa, base + 9) != NULL); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_erase(xa, 8) != NULL); + XA_BUG_ON(test, xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_erase(xa, base + 9) != NULL); + XA_BUG_ON(test, xa, !xa_empty(xa)); xa_destroy(xa); } -static noinline void check_xa_alloc_3(struct xarray *xa, unsigned int base) +static noinline void check_xa_alloc_3(struct kunit *test, struct xarray *xa, + unsigned int base) { struct xa_limit limit = XA_LIMIT(1, 0x3fff); u32 next = 0; @@ -780,64 +816,67 @@ static noinline void check_xa_alloc_3(struct xarray *xa, unsigned int base) unsigned long index; void *entry; - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, xa_mk_index(1), limit, + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, xa_mk_index(1), limit, &next, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 1); + XA_BUG_ON(test, xa, id != 1); next = 0x3ffd; - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, xa_mk_index(0x3ffd), limit, + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, xa_mk_index(0x3ffd), limit, &next, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != 0x3ffd); - xa_erase_index(xa, 0x3ffd); - xa_erase_index(xa, 1); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, id != 0x3ffd); + xa_erase_index(test, xa, 0x3ffd); + xa_erase_index(test, xa, 1); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (i = 0x3ffe; i < 0x4003; i++) { if (i < 0x4000) entry = xa_mk_index(i); else entry = xa_mk_index(i - 0x3fff); - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, entry, limit, + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, entry, limit, &next, GFP_KERNEL) != (id == 1)); - XA_BUG_ON(xa, xa_mk_index(id) != entry); + XA_BUG_ON(test, xa, xa_mk_index(id) != entry); } /* Check wrap-around is handled correctly */ if (base != 0) - xa_erase_index(xa, base); - xa_erase_index(xa, base + 1); + xa_erase_index(test, xa, base); + xa_erase_index(test, xa, base + 1); next = UINT_MAX; - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, xa_mk_index(UINT_MAX), + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, xa_mk_index(UINT_MAX), xa_limit_32b, &next, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != UINT_MAX); - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, xa_mk_index(base), + XA_BUG_ON(test, xa, id != UINT_MAX); + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, xa_mk_index(base), xa_limit_32b, &next, GFP_KERNEL) != 1); - XA_BUG_ON(xa, id != base); - XA_BUG_ON(xa, xa_alloc_cyclic(xa, &id, xa_mk_index(base + 1), + XA_BUG_ON(test, xa, id != base); + XA_BUG_ON(test, xa, xa_alloc_cyclic(xa, &id, xa_mk_index(base + 1), xa_limit_32b, &next, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != base + 1); + XA_BUG_ON(test, xa, id != base + 1); xa_for_each(xa, index, entry) - xa_erase_index(xa, index); + xa_erase_index(test, xa, index); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static DEFINE_XARRAY_ALLOC(xa0); -static DEFINE_XARRAY_ALLOC1(xa1); - -static noinline void check_xa_alloc(void) +static noinline void check_xa_alloc(struct kunit *test) { - check_xa_alloc_1(&xa0, 0); - check_xa_alloc_1(&xa1, 1); - check_xa_alloc_2(&xa0, 0); - check_xa_alloc_2(&xa1, 1); - check_xa_alloc_3(&xa0, 0); - check_xa_alloc_3(&xa1, 1); + DEFINE_XARRAY_ALLOC(xa0); + DEFINE_XARRAY_ALLOC1(xa1); + + check_xa_alloc_1(test, &xa0, 0); + check_xa_alloc_1(test, &xa1, 1); + check_xa_alloc_2(test, &xa0, 0); + check_xa_alloc_2(test, &xa1, 1); + check_xa_alloc_3(test, &xa0, 0); + check_xa_alloc_3(test, &xa1, 1); } -static noinline void __check_store_iter(struct xarray *xa, unsigned long start, - unsigned int order, unsigned int present) +static noinline void __check_store_iter(struct kunit *test, + struct xarray *xa, + unsigned long start, + unsigned int order, + unsigned int present) { XA_STATE_ORDER(xas, xa, start, order); void *entry; @@ -846,9 +885,10 @@ static noinline void __check_store_iter(struct xarray *xa, unsigned long start, retry: xas_lock(&xas); xas_for_each_conflict(&xas, entry) { - XA_BUG_ON(xa, !xa_is_value(entry)); - XA_BUG_ON(xa, entry < xa_mk_index(start)); - XA_BUG_ON(xa, entry > xa_mk_index(start + (1UL << order) - 1)); + XA_BUG_ON(test, xa, !xa_is_value(entry)); + XA_BUG_ON(test, xa, entry < xa_mk_index(start)); + XA_BUG_ON(test, + xa, entry > xa_mk_index(start + (1UL << order) - 1)); count++; } xas_store(&xas, xa_mk_index(start)); @@ -857,53 +897,55 @@ static noinline void __check_store_iter(struct xarray *xa, unsigned long start, count = 0; goto retry; } - XA_BUG_ON(xa, xas_error(&xas)); - XA_BUG_ON(xa, count != present); - XA_BUG_ON(xa, xa_load(xa, start) != xa_mk_index(start)); - XA_BUG_ON(xa, xa_load(xa, start + (1UL << order) - 1) != + XA_BUG_ON(test, xa, xas_error(&xas)); + XA_BUG_ON(test, xa, count != present); + XA_BUG_ON(test, xa, xa_load(xa, start) != xa_mk_index(start)); + XA_BUG_ON(test, xa, xa_load(xa, start + (1UL << order) - 1) != xa_mk_index(start)); - xa_erase_index(xa, start); + xa_erase_index(test, xa, start); } -static noinline void check_store_iter(struct xarray *xa) +static noinline void check_store_iter(struct kunit *test) { + struct xarray *xa = test->priv; unsigned int i, j; unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 20 : 1; for (i = 0; i < max_order; i++) { unsigned int min = 1 << i; unsigned int max = (2 << i) - 1; - __check_store_iter(xa, 0, i, 0); - XA_BUG_ON(xa, !xa_empty(xa)); - __check_store_iter(xa, min, i, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + __check_store_iter(test, xa, 0, i, 0); + XA_BUG_ON(test, xa, !xa_empty(xa)); + __check_store_iter(test, xa, min, i, 0); + XA_BUG_ON(test, xa, !xa_empty(xa)); xa_store_index(xa, min, GFP_KERNEL); - __check_store_iter(xa, min, i, 1); - XA_BUG_ON(xa, !xa_empty(xa)); + __check_store_iter(test, xa, min, i, 1); + XA_BUG_ON(test, xa, !xa_empty(xa)); xa_store_index(xa, max, GFP_KERNEL); - __check_store_iter(xa, min, i, 1); - XA_BUG_ON(xa, !xa_empty(xa)); + __check_store_iter(test, xa, min, i, 1); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (j = 0; j < min; j++) xa_store_index(xa, j, GFP_KERNEL); - __check_store_iter(xa, 0, i, min); - XA_BUG_ON(xa, !xa_empty(xa)); + __check_store_iter(test, xa, 0, i, min); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (j = 0; j < min; j++) xa_store_index(xa, min + j, GFP_KERNEL); - __check_store_iter(xa, min, i, min); - XA_BUG_ON(xa, !xa_empty(xa)); + __check_store_iter(test, xa, min, i, min); + XA_BUG_ON(test, xa, !xa_empty(xa)); } #ifdef CONFIG_XARRAY_MULTI xa_store_index(xa, 63, GFP_KERNEL); xa_store_index(xa, 65, GFP_KERNEL); __check_store_iter(xa, 64, 2, 1); - xa_erase_index(xa, 63); + xa_erase_index(test, xa, 63); #endif - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_multi_find_1(struct xarray *xa, unsigned order) +static noinline void check_multi_find_1(struct kunit *test, struct xarray *xa, + unsigned int order) { #ifdef CONFIG_XARRAY_MULTI unsigned long multi = 3 << order; @@ -911,31 +953,32 @@ static noinline void check_multi_find_1(struct xarray *xa, unsigned order) unsigned long index; xa_store_order(xa, multi, order, xa_mk_value(multi), GFP_KERNEL); - XA_BUG_ON(xa, xa_store_index(xa, next, GFP_KERNEL) != NULL); - XA_BUG_ON(xa, xa_store_index(xa, next + 1, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, next, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, next + 1, GFP_KERNEL) != NULL); index = 0; - XA_BUG_ON(xa, xa_find(xa, &index, ULONG_MAX, XA_PRESENT) != + XA_BUG_ON(test, xa, xa_find(xa, &index, ULONG_MAX, XA_PRESENT) != xa_mk_value(multi)); - XA_BUG_ON(xa, index != multi); + XA_BUG_ON(test, xa, index != multi); index = multi + 1; - XA_BUG_ON(xa, xa_find(xa, &index, ULONG_MAX, XA_PRESENT) != + XA_BUG_ON(test, xa, xa_find(xa, &index, ULONG_MAX, XA_PRESENT) != xa_mk_value(multi)); - XA_BUG_ON(xa, (index < multi) || (index >= next)); - XA_BUG_ON(xa, xa_find_after(xa, &index, ULONG_MAX, XA_PRESENT) != + XA_BUG_ON(test, xa, (index < multi) || (index >= next)); + XA_BUG_ON(test, xa, xa_find_after(xa, &index, ULONG_MAX, XA_PRESENT) != xa_mk_value(next)); - XA_BUG_ON(xa, index != next); - XA_BUG_ON(xa, xa_find_after(xa, &index, next, XA_PRESENT) != NULL); - XA_BUG_ON(xa, index != next); - - xa_erase_index(xa, multi); - xa_erase_index(xa, next); - xa_erase_index(xa, next + 1); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, index != next); + XA_BUG_ON(test, xa, + xa_find_after(xa, &index, next, XA_PRESENT) != NULL); + XA_BUG_ON(test, xa, index != next); + + xa_erase_index(test, xa, multi); + xa_erase_index(test, xa, next); + xa_erase_index(test, xa, next + 1); + XA_BUG_ON(test, xa, !xa_empty(xa)); #endif } -static noinline void check_multi_find_2(struct xarray *xa) +static noinline void check_multi_find_2(struct kunit *test, struct xarray *xa) { unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 10 : 1; unsigned int i, j; @@ -950,30 +993,30 @@ static noinline void check_multi_find_2(struct xarray *xa) GFP_KERNEL); rcu_read_lock(); xas_for_each(&xas, entry, ULONG_MAX) { - xa_erase_index(xa, index); + xa_erase_index(test, xa, index); } rcu_read_unlock(); - xa_erase_index(xa, index - 1); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, index - 1); + XA_BUG_ON(test, xa, !xa_empty(xa)); } } } -static noinline void check_find_1(struct xarray *xa) +static noinline void check_find_1(struct kunit *test, struct xarray *xa) { unsigned long i, j, k; - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* * Check xa_find with all pairs between 0 and 99 inclusive, * starting at every index between 0 and 99 */ for (i = 0; i < 100; i++) { - XA_BUG_ON(xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); xa_set_mark(xa, i, XA_MARK_0); for (j = 0; j < i; j++) { - XA_BUG_ON(xa, xa_store_index(xa, j, GFP_KERNEL) != + XA_BUG_ON(test, xa, xa_store_index(xa, j, GFP_KERNEL) != NULL); xa_set_mark(xa, j, XA_MARK_0); for (k = 0; k < 100; k++) { @@ -981,54 +1024,54 @@ static noinline void check_find_1(struct xarray *xa) void *entry = xa_find(xa, &index, ULONG_MAX, XA_PRESENT); if (k <= j) - XA_BUG_ON(xa, index != j); + XA_BUG_ON(test, xa, index != j); else if (k <= i) - XA_BUG_ON(xa, index != i); + XA_BUG_ON(test, xa, index != i); else - XA_BUG_ON(xa, entry != NULL); + XA_BUG_ON(test, xa, entry != NULL); index = k; entry = xa_find(xa, &index, ULONG_MAX, XA_MARK_0); if (k <= j) - XA_BUG_ON(xa, index != j); + XA_BUG_ON(test, xa, index != j); else if (k <= i) - XA_BUG_ON(xa, index != i); + XA_BUG_ON(test, xa, index != i); else - XA_BUG_ON(xa, entry != NULL); + XA_BUG_ON(test, xa, entry != NULL); } - xa_erase_index(xa, j); - XA_BUG_ON(xa, xa_get_mark(xa, j, XA_MARK_0)); - XA_BUG_ON(xa, !xa_get_mark(xa, i, XA_MARK_0)); + xa_erase_index(test, xa, j); + XA_BUG_ON(test, xa, xa_get_mark(xa, j, XA_MARK_0)); + XA_BUG_ON(test, xa, !xa_get_mark(xa, i, XA_MARK_0)); } - xa_erase_index(xa, i); - XA_BUG_ON(xa, xa_get_mark(xa, i, XA_MARK_0)); + xa_erase_index(test, xa, i); + XA_BUG_ON(test, xa, xa_get_mark(xa, i, XA_MARK_0)); } - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_find_2(struct xarray *xa) +static noinline void check_find_2(struct kunit *test, struct xarray *xa) { void *entry; unsigned long i, j, index; xa_for_each(xa, index, entry) { - XA_BUG_ON(xa, true); + XA_BUG_ON(test, xa, true); } for (i = 0; i < 1024; i++) { xa_store_index(xa, index, GFP_KERNEL); j = 0; xa_for_each(xa, index, entry) { - XA_BUG_ON(xa, xa_mk_index(index) != entry); - XA_BUG_ON(xa, index != j++); + XA_BUG_ON(test, xa, xa_mk_index(index) != entry); + XA_BUG_ON(test, xa, index != j++); } } xa_destroy(xa); } -static noinline void check_find_3(struct xarray *xa) +static noinline void check_find_3(struct kunit *test, struct xarray *xa) { XA_STATE(xas, xa, 0); unsigned long i, j, k; @@ -1042,7 +1085,7 @@ static noinline void check_find_3(struct xarray *xa) xas_for_each_marked(&xas, entry, k, XA_MARK_0) ; if (j > k) - XA_BUG_ON(xa, + XA_BUG_ON(test, xa, xas.xa_node != XAS_RESTART); } rcu_read_unlock(); @@ -1053,7 +1096,7 @@ static noinline void check_find_3(struct xarray *xa) xa_destroy(xa); } -static noinline void check_find_4(struct xarray *xa) +static noinline void check_find_4(struct kunit *test, struct xarray *xa) { unsigned long index = 0; void *entry; @@ -1061,26 +1104,27 @@ static noinline void check_find_4(struct xarray *xa) xa_store_index(xa, ULONG_MAX, GFP_KERNEL); entry = xa_find_after(xa, &index, ULONG_MAX, XA_PRESENT); - XA_BUG_ON(xa, entry != xa_mk_index(ULONG_MAX)); + XA_BUG_ON(test, xa, entry != xa_mk_index(ULONG_MAX)); entry = xa_find_after(xa, &index, ULONG_MAX, XA_PRESENT); - XA_BUG_ON(xa, entry); + XA_BUG_ON(test, xa, entry); - xa_erase_index(xa, ULONG_MAX); + xa_erase_index(test, xa, ULONG_MAX); } -static noinline void check_find(struct xarray *xa) +static noinline void check_find(struct kunit *test) { + struct xarray *xa = test->priv; unsigned i; - check_find_1(xa); - check_find_2(xa); - check_find_3(xa); - check_find_4(xa); + check_find_1(test, xa); + check_find_2(test, xa); + check_find_3(test, xa); + check_find_4(test, xa); for (i = 2; i < 10; i++) - check_multi_find_1(xa, i); - check_multi_find_2(xa); + check_multi_find_1(test, xa, i); + check_multi_find_2(test, xa); } /* See find_swap_entry() in mm/shmem.c */ @@ -1106,8 +1150,9 @@ static noinline unsigned long xa_find_entry(struct xarray *xa, void *item) return entry ? xas.xa_index : -1; } -static noinline void check_find_entry(struct xarray *xa) +static noinline void check_find_entry(struct kunit *test) { + struct xarray *xa = test->priv; #ifdef CONFIG_XARRAY_MULTI unsigned int order; unsigned long offset, index; @@ -1119,69 +1164,72 @@ static noinline void check_find_entry(struct xarray *xa) index += (1UL << order)) { xa_store_order(xa, index, order, xa_mk_index(index), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, index) != + XA_BUG_ON(test, xa, xa_load(xa, index) != xa_mk_index(index)); - XA_BUG_ON(xa, xa_find_entry(xa, + XA_BUG_ON(test, xa, xa_find_entry(xa, xa_mk_index(index)) != index); } - XA_BUG_ON(xa, xa_find_entry(xa, xa) != -1); + XA_BUG_ON(test, xa, xa_find_entry(xa, xa) != -1); xa_destroy(xa); } } #endif - XA_BUG_ON(xa, xa_find_entry(xa, xa) != -1); + XA_BUG_ON(test, xa, xa_find_entry(xa, xa) != -1); xa_store_index(xa, ULONG_MAX, GFP_KERNEL); - XA_BUG_ON(xa, xa_find_entry(xa, xa) != -1); - XA_BUG_ON(xa, xa_find_entry(xa, xa_mk_index(ULONG_MAX)) != -1); - xa_erase_index(xa, ULONG_MAX); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, xa_find_entry(xa, xa) != -1); + XA_BUG_ON(test, xa, xa_find_entry(xa, xa_mk_index(ULONG_MAX)) != -1); + xa_erase_index(test, xa, ULONG_MAX); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_move_tiny(struct xarray *xa) +static noinline void check_move_tiny(struct kunit *test, struct xarray *xa) { XA_STATE(xas, xa, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); rcu_read_lock(); - XA_BUG_ON(xa, xas_next(&xas) != NULL); - XA_BUG_ON(xa, xas_next(&xas) != NULL); + XA_BUG_ON(test, xa, xas_next(&xas) != NULL); + XA_BUG_ON(test, xa, xas_next(&xas) != NULL); rcu_read_unlock(); xa_store_index(xa, 0, GFP_KERNEL); rcu_read_lock(); xas_set(&xas, 0); - XA_BUG_ON(xa, xas_next(&xas) != xa_mk_index(0)); - XA_BUG_ON(xa, xas_next(&xas) != NULL); + XA_BUG_ON(test, xa, xas_next(&xas) != xa_mk_index(0)); + XA_BUG_ON(test, xa, xas_next(&xas) != NULL); xas_set(&xas, 0); - XA_BUG_ON(xa, xas_prev(&xas) != xa_mk_index(0)); - XA_BUG_ON(xa, xas_prev(&xas) != NULL); + XA_BUG_ON(test, xa, xas_prev(&xas) != xa_mk_index(0)); + XA_BUG_ON(test, xa, xas_prev(&xas) != NULL); rcu_read_unlock(); - xa_erase_index(xa, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, 0); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_move_max(struct xarray *xa) +static noinline void check_move_max(struct kunit *test, struct xarray *xa) { XA_STATE(xas, xa, 0); xa_store_index(xa, ULONG_MAX, GFP_KERNEL); rcu_read_lock(); - XA_BUG_ON(xa, xas_find(&xas, ULONG_MAX) != xa_mk_index(ULONG_MAX)); - XA_BUG_ON(xa, xas_find(&xas, ULONG_MAX) != NULL); + XA_BUG_ON(test, xa, + xas_find(&xas, ULONG_MAX) != xa_mk_index(ULONG_MAX)); + XA_BUG_ON(test, xa, xas_find(&xas, ULONG_MAX) != NULL); rcu_read_unlock(); xas_set(&xas, 0); rcu_read_lock(); - XA_BUG_ON(xa, xas_find(&xas, ULONG_MAX) != xa_mk_index(ULONG_MAX)); + XA_BUG_ON(test, xa, + xas_find(&xas, ULONG_MAX) != xa_mk_index(ULONG_MAX)); xas_pause(&xas); - XA_BUG_ON(xa, xas_find(&xas, ULONG_MAX) != NULL); + XA_BUG_ON(test, xa, xas_find(&xas, ULONG_MAX) != NULL); rcu_read_unlock(); - xa_erase_index(xa, ULONG_MAX); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, ULONG_MAX); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_move_small(struct xarray *xa, unsigned long idx) +static noinline void check_move_small(struct kunit *test, struct xarray *xa, + unsigned long idx) { XA_STATE(xas, xa, 0); unsigned long i; @@ -1193,71 +1241,72 @@ static noinline void check_move_small(struct xarray *xa, unsigned long idx) for (i = 0; i < idx * 4; i++) { void *entry = xas_next(&xas); if (i <= idx) - XA_BUG_ON(xa, xas.xa_node == XAS_RESTART); - XA_BUG_ON(xa, xas.xa_index != i); + XA_BUG_ON(test, xa, xas.xa_node == XAS_RESTART); + XA_BUG_ON(test, xa, xas.xa_index != i); if (i == 0 || i == idx) - XA_BUG_ON(xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); else - XA_BUG_ON(xa, entry != NULL); + XA_BUG_ON(test, xa, entry != NULL); } xas_next(&xas); - XA_BUG_ON(xa, xas.xa_index != i); + XA_BUG_ON(test, xa, xas.xa_index != i); do { void *entry = xas_prev(&xas); i--; if (i <= idx) - XA_BUG_ON(xa, xas.xa_node == XAS_RESTART); - XA_BUG_ON(xa, xas.xa_index != i); + XA_BUG_ON(test, xa, xas.xa_node == XAS_RESTART); + XA_BUG_ON(test, xa, xas.xa_index != i); if (i == 0 || i == idx) - XA_BUG_ON(xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); else - XA_BUG_ON(xa, entry != NULL); + XA_BUG_ON(test, xa, entry != NULL); } while (i > 0); xas_set(&xas, ULONG_MAX); - XA_BUG_ON(xa, xas_next(&xas) != NULL); - XA_BUG_ON(xa, xas.xa_index != ULONG_MAX); - XA_BUG_ON(xa, xas_next(&xas) != xa_mk_value(0)); - XA_BUG_ON(xa, xas.xa_index != 0); - XA_BUG_ON(xa, xas_prev(&xas) != NULL); - XA_BUG_ON(xa, xas.xa_index != ULONG_MAX); + XA_BUG_ON(test, xa, xas_next(&xas) != NULL); + XA_BUG_ON(test, xa, xas.xa_index != ULONG_MAX); + XA_BUG_ON(test, xa, xas_next(&xas) != xa_mk_value(0)); + XA_BUG_ON(test, xa, xas.xa_index != 0); + XA_BUG_ON(test, xa, xas_prev(&xas) != NULL); + XA_BUG_ON(test, xa, xas.xa_index != ULONG_MAX); rcu_read_unlock(); - xa_erase_index(xa, 0); - xa_erase_index(xa, idx); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, 0); + xa_erase_index(test, xa, idx); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_move(struct xarray *xa) +static noinline void check_move(struct kunit *test) { + struct xarray *xa = test->priv; XA_STATE(xas, xa, (1 << 16) - 1); unsigned long i; for (i = 0; i < (1 << 16); i++) - XA_BUG_ON(xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, xa_store_index(xa, i, GFP_KERNEL) != NULL); rcu_read_lock(); do { void *entry = xas_prev(&xas); i--; - XA_BUG_ON(xa, entry != xa_mk_index(i)); - XA_BUG_ON(xa, i != xas.xa_index); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, i != xas.xa_index); } while (i != 0); - XA_BUG_ON(xa, xas_prev(&xas) != NULL); - XA_BUG_ON(xa, xas.xa_index != ULONG_MAX); + XA_BUG_ON(test, xa, xas_prev(&xas) != NULL); + XA_BUG_ON(test, xa, xas.xa_index != ULONG_MAX); do { void *entry = xas_next(&xas); - XA_BUG_ON(xa, entry != xa_mk_index(i)); - XA_BUG_ON(xa, i != xas.xa_index); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, i != xas.xa_index); i++; } while (i < (1 << 16)); rcu_read_unlock(); for (i = (1 << 8); i < (1 << 15); i++) - xa_erase_index(xa, i); + xa_erase_index(test, xa, i); i = xas.xa_index; @@ -1266,39 +1315,39 @@ static noinline void check_move(struct xarray *xa) void *entry = xas_prev(&xas); i--; if ((i < (1 << 8)) || (i >= (1 << 15))) - XA_BUG_ON(xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); else - XA_BUG_ON(xa, entry != NULL); - XA_BUG_ON(xa, i != xas.xa_index); + XA_BUG_ON(test, xa, entry != NULL); + XA_BUG_ON(test, xa, i != xas.xa_index); } while (i != 0); - XA_BUG_ON(xa, xas_prev(&xas) != NULL); - XA_BUG_ON(xa, xas.xa_index != ULONG_MAX); + XA_BUG_ON(test, xa, xas_prev(&xas) != NULL); + XA_BUG_ON(test, xa, xas.xa_index != ULONG_MAX); do { void *entry = xas_next(&xas); if ((i < (1 << 8)) || (i >= (1 << 15))) - XA_BUG_ON(xa, entry != xa_mk_index(i)); + XA_BUG_ON(test, xa, entry != xa_mk_index(i)); else - XA_BUG_ON(xa, entry != NULL); - XA_BUG_ON(xa, i != xas.xa_index); + XA_BUG_ON(test, xa, entry != NULL); + XA_BUG_ON(test, xa, i != xas.xa_index); i++; } while (i < (1 << 16)); rcu_read_unlock(); xa_destroy(xa); - check_move_tiny(xa); - check_move_max(xa); + check_move_tiny(test, xa); + check_move_max(test, xa); for (i = 0; i < 16; i++) - check_move_small(xa, 1UL << i); + check_move_small(test, xa, 1UL << i); for (i = 2; i < 16; i++) - check_move_small(xa, (1UL << i) - 1); + check_move_small(test, xa, (1UL << i) - 1); } -static noinline void xa_store_many_order(struct xarray *xa, +static noinline void xa_store_many_order(struct kunit *test, struct xarray *xa, unsigned long index, unsigned order) { XA_STATE_ORDER(xas, xa, index, order); @@ -1306,53 +1355,55 @@ static noinline void xa_store_many_order(struct xarray *xa, do { xas_lock(&xas); - XA_BUG_ON(xa, xas_find_conflict(&xas)); + XA_BUG_ON(test, xa, xas_find_conflict(&xas)); xas_create_range(&xas); if (xas_error(&xas)) goto unlock; for (i = 0; i < (1U << order); i++) { - XA_BUG_ON(xa, xas_store(&xas, xa_mk_index(index + i))); + XA_BUG_ON(test, xa, + xas_store(&xas, xa_mk_index(index + i))); xas_next(&xas); } unlock: xas_unlock(&xas); } while (xas_nomem(&xas, GFP_KERNEL)); - XA_BUG_ON(xa, xas_error(&xas)); + XA_BUG_ON(test, xa, xas_error(&xas)); } -static noinline void check_create_range_1(struct xarray *xa, +static noinline void check_create_range_1(struct kunit *test, struct xarray *xa, unsigned long index, unsigned order) { unsigned long i; - xa_store_many_order(xa, index, order); + xa_store_many_order(test, xa, index, order); for (i = index; i < index + (1UL << order); i++) - xa_erase_index(xa, i); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, i); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_create_range_2(struct xarray *xa, unsigned order) +static noinline void check_create_range_2(struct kunit *test, struct xarray *xa, + unsigned int order) { unsigned long i; unsigned long nr = 1UL << order; for (i = 0; i < nr * nr; i += nr) - xa_store_many_order(xa, i, order); + xa_store_many_order(test, xa, i, order); for (i = 0; i < nr * nr; i++) - xa_erase_index(xa, i); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, i); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_create_range_3(void) +static noinline void check_create_range_3(struct kunit *test) { XA_STATE(xas, NULL, 0); xas_set_err(&xas, -EEXIST); xas_create_range(&xas); - XA_BUG_ON(NULL, xas_error(&xas) != -EEXIST); + XA_BUG_ON(test, NULL, xas_error(&xas) != -EEXIST); } -static noinline void check_create_range_4(struct xarray *xa, +static noinline void check_create_range_4(struct kunit *test, struct xarray *xa, unsigned long index, unsigned order) { XA_STATE_ORDER(xas, xa, index, order); @@ -1368,88 +1419,94 @@ static noinline void check_create_range_4(struct xarray *xa, for (i = 0; i < (1UL << order); i++) { void *old = xas_store(&xas, xa_mk_index(base + i)); if (xas.xa_index == index) - XA_BUG_ON(xa, old != xa_mk_index(base + i)); + XA_BUG_ON(test, xa, + old != xa_mk_index(base + i)); else - XA_BUG_ON(xa, old != NULL); + XA_BUG_ON(test, xa, old != NULL); xas_next(&xas); } unlock: xas_unlock(&xas); } while (xas_nomem(&xas, GFP_KERNEL)); - XA_BUG_ON(xa, xas_error(&xas)); + XA_BUG_ON(test, xa, xas_error(&xas)); for (i = base; i < base + (1UL << order); i++) - xa_erase_index(xa, i); - XA_BUG_ON(xa, !xa_empty(xa)); + xa_erase_index(test, xa, i); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_create_range(struct xarray *xa) +static noinline void check_create_range(struct kunit *test) { + struct xarray *xa = test->priv; unsigned int order; unsigned int max_order = IS_ENABLED(CONFIG_XARRAY_MULTI) ? 12 : 1; for (order = 0; order < max_order; order++) { - check_create_range_1(xa, 0, order); - check_create_range_1(xa, 1U << order, order); - check_create_range_1(xa, 2U << order, order); - check_create_range_1(xa, 3U << order, order); - check_create_range_1(xa, 1U << 24, order); + check_create_range_1(test, xa, 0, order); + check_create_range_1(test, xa, 1U << order, order); + check_create_range_1(test, xa, 2U << order, order); + check_create_range_1(test, xa, 3U << order, order); + check_create_range_1(test, xa, 1U << 24, order); if (order < 10) - check_create_range_2(xa, order); - - check_create_range_4(xa, 0, order); - check_create_range_4(xa, 1U << order, order); - check_create_range_4(xa, 2U << order, order); - check_create_range_4(xa, 3U << order, order); - check_create_range_4(xa, 1U << 24, order); - - check_create_range_4(xa, 1, order); - check_create_range_4(xa, (1U << order) + 1, order); - check_create_range_4(xa, (2U << order) + 1, order); - check_create_range_4(xa, (2U << order) - 1, order); - check_create_range_4(xa, (3U << order) + 1, order); - check_create_range_4(xa, (3U << order) - 1, order); - check_create_range_4(xa, (1U << 24) + 1, order); + check_create_range_2(test, xa, order); + + check_create_range_4(test, xa, 0, order); + check_create_range_4(test, xa, 1U << order, order); + check_create_range_4(test, xa, 2U << order, order); + check_create_range_4(test, xa, 3U << order, order); + check_create_range_4(test, xa, 1U << 24, order); + + check_create_range_4(test, xa, 1, order); + check_create_range_4(test, xa, (1U << order) + 1, order); + check_create_range_4(test, xa, (2U << order) + 1, order); + check_create_range_4(test, xa, (2U << order) - 1, order); + check_create_range_4(test, xa, (3U << order) + 1, order); + check_create_range_4(test, xa, (3U << order) - 1, order); + check_create_range_4(test, xa, (1U << 24) + 1, order); } - check_create_range_3(); + check_create_range_3(test); } -static noinline void __check_store_range(struct xarray *xa, unsigned long first, - unsigned long last) +static noinline void __check_store_range(struct kunit *test, + struct xarray *xa, + unsigned long first, + unsigned long last) { #ifdef CONFIG_XARRAY_MULTI xa_store_range(xa, first, last, xa_mk_index(first), GFP_KERNEL); - XA_BUG_ON(xa, xa_load(xa, first) != xa_mk_index(first)); - XA_BUG_ON(xa, xa_load(xa, last) != xa_mk_index(first)); - XA_BUG_ON(xa, xa_load(xa, first - 1) != NULL); - XA_BUG_ON(xa, xa_load(xa, last + 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, first) != xa_mk_index(first)); + XA_BUG_ON(test, xa, xa_load(xa, last) != xa_mk_index(first)); + XA_BUG_ON(test, xa, xa_load(xa, first - 1) != NULL); + XA_BUG_ON(test, xa, xa_load(xa, last + 1) != NULL); xa_store_range(xa, first, last, NULL, GFP_KERNEL); #endif - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_store_range(struct xarray *xa) +static noinline void check_store_range(struct kunit *test) { + struct xarray *xa = test->priv; unsigned long i, j; for (i = 0; i < 128; i++) { for (j = i; j < 128; j++) { - __check_store_range(xa, i, j); - __check_store_range(xa, 128 + i, 128 + j); - __check_store_range(xa, 4095 + i, 4095 + j); - __check_store_range(xa, 4096 + i, 4096 + j); - __check_store_range(xa, 123456 + i, 123456 + j); - __check_store_range(xa, (1 << 24) + i, (1 << 24) + j); + __check_store_range(test, xa, i, j); + __check_store_range(test, xa, 128 + i, 128 + j); + __check_store_range(test, xa, 4095 + i, 4095 + j); + __check_store_range(test, xa, 4096 + i, 4096 + j); + __check_store_range(test, xa, 123456 + i, 123456 + j); + __check_store_range(test, xa, + (1 << 24) + i, (1 << 24) + j); } } } -static void check_align_1(struct xarray *xa, char *name) +static void check_align_1(struct kunit *test, struct xarray *xa, char *name) { int i; unsigned int id; @@ -1457,12 +1514,12 @@ static void check_align_1(struct xarray *xa, char *name) void *entry; for (i = 0; i < 8; i++) { - XA_BUG_ON(xa, xa_alloc(xa, &id, name + i, xa_limit_32b, + XA_BUG_ON(test, xa, xa_alloc(xa, &id, name + i, xa_limit_32b, GFP_KERNEL) != 0); - XA_BUG_ON(xa, id != i); + XA_BUG_ON(test, xa, id != i); } xa_for_each(xa, index, entry) - XA_BUG_ON(xa, xa_is_err(entry)); + XA_BUG_ON(test, xa, xa_is_err(entry)); xa_destroy(xa); } @@ -1470,35 +1527,37 @@ static void check_align_1(struct xarray *xa, char *name) * We should always be able to store without allocating memory after * reserving a slot. */ -static void check_align_2(struct xarray *xa, char *name) +static void check_align_2(struct kunit *test, struct xarray *xa, char *name) { int i; - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); for (i = 0; i < 8; i++) { - XA_BUG_ON(xa, xa_store(xa, 0, name + i, GFP_KERNEL) != NULL); + XA_BUG_ON(test, xa, + xa_store(xa, 0, name + i, GFP_KERNEL) != NULL); xa_erase(xa, 0); } for (i = 0; i < 8; i++) { - XA_BUG_ON(xa, xa_reserve(xa, 0, GFP_KERNEL) != 0); - XA_BUG_ON(xa, xa_store(xa, 0, name + i, 0) != NULL); + XA_BUG_ON(test, xa, xa_reserve(xa, 0, GFP_KERNEL) != 0); + XA_BUG_ON(test, xa, xa_store(xa, 0, name + i, 0) != NULL); xa_erase(xa, 0); } - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } -static noinline void check_align(struct xarray *xa) +static noinline void check_align(struct kunit *test) { char name[] = "Motorola 68000"; + DEFINE_XARRAY_ALLOC(xa); - check_align_1(xa, name); - check_align_1(xa, name + 1); - check_align_1(xa, name + 2); - check_align_1(xa, name + 3); - check_align_2(xa, name); + check_align_1(test, &xa, name); + check_align_1(test, &xa, name + 1); + check_align_1(test, &xa, name + 2); + check_align_1(test, &xa, name + 3); + check_align_2(test, &xa, name); } static LIST_HEAD(shadow_nodes); @@ -1514,7 +1573,7 @@ static void test_update_node(struct xa_node *node) } } -static noinline void shadow_remove(struct xarray *xa) +static noinline void shadow_remove(struct kunit *test, struct xarray *xa) { struct xa_node *node; @@ -1522,7 +1581,7 @@ static noinline void shadow_remove(struct xarray *xa) while ((node = list_first_entry_or_null(&shadow_nodes, struct xa_node, private_list))) { XA_STATE(xas, node->array, 0); - XA_BUG_ON(xa, node->array != xa); + XA_BUG_ON(test, xa, node->array != xa); list_del_init(&node->private_list); xas.xa_node = xa_parent_locked(node->array, node); xas.xa_offset = node->offset; @@ -1533,7 +1592,9 @@ static noinline void shadow_remove(struct xarray *xa) xa_unlock(xa); } -static noinline void check_workingset(struct xarray *xa, unsigned long index) +static noinline void check_workingset(struct kunit *test, + struct xarray *xa, + unsigned long index) { XA_STATE(xas, xa, index); xas_set_update(&xas, test_update_node); @@ -1546,29 +1607,39 @@ static noinline void check_workingset(struct xarray *xa, unsigned long index) xas_unlock(&xas); } while (xas_nomem(&xas, GFP_KERNEL)); - XA_BUG_ON(xa, list_empty(&shadow_nodes)); + XA_BUG_ON(test, xa, list_empty(&shadow_nodes)); xas_lock(&xas); xas_next(&xas); xas_store(&xas, &xas); - XA_BUG_ON(xa, !list_empty(&shadow_nodes)); + XA_BUG_ON(test, xa, !list_empty(&shadow_nodes)); xas_store(&xas, xa_mk_value(2)); xas_unlock(&xas); - XA_BUG_ON(xa, list_empty(&shadow_nodes)); + XA_BUG_ON(test, xa, list_empty(&shadow_nodes)); - shadow_remove(xa); - XA_BUG_ON(xa, !list_empty(&shadow_nodes)); - XA_BUG_ON(xa, !xa_empty(xa)); + shadow_remove(test, xa); + XA_BUG_ON(test, xa, !list_empty(&shadow_nodes)); + XA_BUG_ON(test, xa, !xa_empty(xa)); +} + +static void check_workingset_all(struct kunit *test) +{ + struct xarray *xa = test->priv; + + check_workingset(test, xa, 0); + check_workingset(test, xa, 64); + check_workingset(test, xa, 4096); } /* * Check that the pointer / value / sibling entries are accounted the * way we expect them to be. */ -static noinline void check_account(struct xarray *xa) +static noinline void check_account(struct kunit *test) { #ifdef CONFIG_XARRAY_MULTI + struct xarray *xa = test->priv; unsigned int order; for (order = 1; order < 12; order++) { @@ -1577,96 +1648,105 @@ static noinline void check_account(struct xarray *xa) xa_store_order(xa, 0, order, xa, GFP_KERNEL); rcu_read_lock(); xas_load(&xas); - XA_BUG_ON(xa, xas.xa_node->count == 0); - XA_BUG_ON(xa, xas.xa_node->count > (1 << order)); - XA_BUG_ON(xa, xas.xa_node->nr_values != 0); + XA_BUG_ON(test, xa, xas.xa_node->count == 0); + XA_BUG_ON(test, xa, xas.xa_node->count > (1 << order)); + XA_BUG_ON(test, xa, xas.xa_node->nr_values != 0); rcu_read_unlock(); xa_store_order(xa, 1 << order, order, xa_mk_index(1UL << order), GFP_KERNEL); - XA_BUG_ON(xa, xas.xa_node->count != xas.xa_node->nr_values * 2); + XA_BUG_ON(test, xa, + xas.xa_node->count != xas.xa_node->nr_values * 2); xa_erase(xa, 1 << order); - XA_BUG_ON(xa, xas.xa_node->nr_values != 0); + XA_BUG_ON(test, xa, xas.xa_node->nr_values != 0); xa_erase(xa, 0); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } #endif } -static noinline void check_destroy(struct xarray *xa) +static noinline void check_destroy(struct kunit *test) { + struct xarray *xa = test->priv; unsigned long index; - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Destroying an empty array is a no-op */ xa_destroy(xa); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); /* Destroying an array with a single entry */ for (index = 0; index < 1000; index++) { xa_store_index(xa, index, GFP_KERNEL); - XA_BUG_ON(xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_empty(xa)); xa_destroy(xa); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); } /* Destroying an array with a single entry at ULONG_MAX */ xa_store(xa, ULONG_MAX, xa, GFP_KERNEL); - XA_BUG_ON(xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_empty(xa)); xa_destroy(xa); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); #ifdef CONFIG_XARRAY_MULTI /* Destroying an array with a multi-index entry */ xa_store_order(xa, 1 << 11, 11, xa, GFP_KERNEL); - XA_BUG_ON(xa, xa_empty(xa)); + XA_BUG_ON(test, xa, xa_empty(xa)); xa_destroy(xa); - XA_BUG_ON(xa, !xa_empty(xa)); + XA_BUG_ON(test, xa, !xa_empty(xa)); #endif } -static DEFINE_XARRAY(array); - -static int xarray_checks(void) +static int xarray_test_init(struct kunit *test) { - check_xa_err(&array); - check_xas_retry(&array); - check_xa_load(&array); - check_xa_mark(&array); - check_xa_shrink(&array); - check_xas_erase(&array); - check_insert(&array); - check_cmpxchg(&array); - check_reserve(&array); - check_reserve(&xa0); - check_multi_store(&array); - check_xa_alloc(); - check_find(&array); - check_find_entry(&array); - check_account(&array); - check_destroy(&array); - check_move(&array); - check_create_range(&array); - check_store_range(&array); - check_store_iter(&array); - check_align(&xa0); - - check_workingset(&array, 0); - check_workingset(&array, 64); - check_workingset(&array, 4096); - - printk("XArray: %u of %u tests passed\n", tests_passed, tests_run); - return (tests_run == tests_passed) ? 0 : -EINVAL; -} + struct xarray *array; -static void xarray_exit(void) -{ + array = kunit_kzalloc(test, sizeof(*array), GFP_KERNEL); + KUNIT_ASSERT_TRUE(test, array); + + xa_init(array); + + test->priv = array; + + return 0; } -module_init(xarray_checks); -module_exit(xarray_exit); +static struct kunit_case xarray_test_cases[] = { + KUNIT_CASE(check_xa_err), + KUNIT_CASE(check_xas_retry), + KUNIT_CASE(check_xa_load), + KUNIT_CASE(check_xa_mark), + KUNIT_CASE(check_xa_shrink), + KUNIT_CASE(check_xas_erase), + KUNIT_CASE(check_insert), + KUNIT_CASE(check_cmpxchg), + KUNIT_CASE(check_multi_store), + KUNIT_CASE(check_find), + KUNIT_CASE(check_find_entry), + KUNIT_CASE(check_account), + KUNIT_CASE(check_destroy), + KUNIT_CASE(check_move), + KUNIT_CASE(check_create_range), + KUNIT_CASE(check_store_range), + KUNIT_CASE(check_store_iter), + KUNIT_CASE(check_xa_alloc), + KUNIT_CASE(check_workingset_all), + KUNIT_CASE(check_reserve_all), + KUNIT_CASE(check_align), + {} +}; + +static struct kunit_suite xarray_test_suite = { + .name = "xarray", + .init = xarray_test_init, + .test_cases = xarray_test_cases, +}; + +kunit_test_suites(&xarray_test_suite); + MODULE_AUTHOR("Matthew Wilcox <willy(a)infradead.org>"); MODULE_LICENSE("GPL"); diff --git a/lib/xarray.c b/lib/xarray.c index 1d9fab7db8dad..ed221e3936606 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -2000,7 +2000,7 @@ void xa_destroy(struct xarray *xa) } EXPORT_SYMBOL(xa_destroy); -#ifdef XA_DEBUG +#ifdef CONFIG_TEST_XARRAY void xa_dump_node(const struct xa_node *node) { unsigned i, j; @@ -2078,4 +2078,4 @@ void xa_dump(const struct xarray *xa) shift = xa_to_node(entry)->shift + XA_CHUNK_SHIFT; xa_dump_entry(entry, 0, shift); } -#endif +#endif // CONFIG_TEST_XARRAY -- 2.25.0.341.g760bfbb309-goog

4 years, 4 months

2
1
0 0

[PATCH v11 1/9] hugetlb_cgroup: Add hugetlb_cgroup reservation counter

by Mina Almasry

These counters will track hugetlb reservations rather than hugetlb memory faulted in. This patch only adds the counter, following patches add the charging and uncharging of the counter. This is patch 1 of an 9 patch series. Problem: Currently tasks attempting to reserve more hugetlb memory than is available get a failure at mmap/shmget time. This is thanks to Hugetlbfs Reservations [1]. However, if a task attempts to reserve more hugetlb memory than its hugetlb_cgroup limit allows, the kernel will allow the mmap/shmget call, but will SIGBUS the task when it attempts to fault in the excess memory. We have users hitting their hugetlb_cgroup limits and thus we've been looking at this failure mode. We'd like to improve this behavior such that users violating the hugetlb_cgroup limits get an error on mmap/shmget time, rather than getting SIGBUS'd when they try to fault the excess memory in. This gives the user an opportunity to fallback more gracefully to non-hugetlbfs memory for example. The underlying problem is that today's hugetlb_cgroup accounting happens at hugetlb memory *fault* time, rather than at *reservation* time. Thus, enforcing the hugetlb_cgroup limit only happens at fault time, and the offending task gets SIGBUS'd. Proposed Solution: A new page counter named 'hugetlb.xMB.rsvd.[limit|usage|max_usage]_in_bytes'. This counter has slightly different semantics than 'hugetlb.xMB.[limit|usage|max_usage]_in_bytes': - While usage_in_bytes tracks all *faulted* hugetlb memory, rsvd.usage_in_bytes tracks all *reserved* hugetlb memory and hugetlb memory faulted in without a prior reservation. - If a task attempts to reserve more memory than limit_in_bytes allows, the kernel will allow it to do so. But if a task attempts to reserve more memory than rsvd.limit_in_bytes, the kernel will fail this reservation. This proposal is implemented in this patch series, with tests to verify functionality and show the usage. Alternatives considered: 1. A new cgroup, instead of only a new page_counter attached to the existing hugetlb_cgroup. Adding a new cgroup seemed like a lot of code duplication with hugetlb_cgroup. Keeping hugetlb related page counters under hugetlb_cgroup seemed cleaner as well. 2. Instead of adding a new counter, we considered adding a sysctl that modifies the behavior of hugetlb.xMB.[limit|usage]_in_bytes, to do accounting at reservation time rather than fault time. Adding a new page_counter seems better as userspace could, if it wants, choose to enforce different cgroups differently: one via limit_in_bytes, and another via rsvd.limit_in_bytes. This could be very useful if you're transitioning how hugetlb memory is partitioned on your system one cgroup at a time, for example. Also, someone may find usage for both limit_in_bytes and rsvd.limit_in_bytes concurrently, and this approach gives them the option to do so. Testing: - Added tests passing. - Used libhugetlbfs for regression testing. [1]: https://www.kernel.org/doc/html/latest/vm/hugetlbfs_reserv.html Signed-off-by: Mina Almasry <almasrymina(a)google.com> Acked-by: David Rientjes <rientjes(a)google.com> --- Changes in v11: - Renamed resv.* or 'reservation' or 'reserved' to rsvd.* - Renamed hugetlb_cgroup_get_counter() to hugetlb_cgroup_counter_from_cgroup(). Changes in v10: - Renamed reservation_* to resv.* --- include/linux/hugetlb.h | 4 +- mm/hugetlb_cgroup.c | 116 +++++++++++++++++++++++++++++++++++----- 2 files changed, 105 insertions(+), 15 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1e897e4168ac1..dea6143aa0685 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -432,8 +432,8 @@ struct hstate { unsigned int surplus_huge_pages_node[MAX_NUMNODES]; #ifdef CONFIG_CGROUP_HUGETLB /* cgroup control files */ - struct cftype cgroup_files_dfl[5]; - struct cftype cgroup_files_legacy[5]; + struct cftype cgroup_files_dfl[7]; + struct cftype cgroup_files_legacy[9]; #endif char name[HSTATE_NAME_LEN]; }; diff --git a/mm/hugetlb_cgroup.c b/mm/hugetlb_cgroup.c index e434b05416c68..cd1f44775259c 100644 --- a/mm/hugetlb_cgroup.c +++ b/mm/hugetlb_cgroup.c @@ -36,6 +36,11 @@ struct hugetlb_cgroup { */ struct page_counter hugepage[HUGE_MAX_HSTATE]; + /* + * the counter to account for hugepage reservations from hugetlb. + */ + struct page_counter rsvd_hugepage[HUGE_MAX_HSTATE]; + atomic_long_t events[HUGE_MAX_HSTATE][HUGETLB_NR_MEMORY_EVENTS]; atomic_long_t events_local[HUGE_MAX_HSTATE][HUGETLB_NR_MEMORY_EVENTS]; @@ -55,6 +60,15 @@ struct hugetlb_cgroup { static struct hugetlb_cgroup *root_h_cgroup __read_mostly; +static inline struct page_counter * +hugetlb_cgroup_counter_from_cgroup(struct hugetlb_cgroup *h_cg, int idx, + bool rsvd) +{ + if (rsvd) + return &h_cg->rsvd_hugepage[idx]; + return &h_cg->hugepage[idx]; +} + static inline struct hugetlb_cgroup *hugetlb_cgroup_from_css(struct cgroup_subsys_state *s) { @@ -295,28 +309,42 @@ void hugetlb_cgroup_uncharge_cgroup(int idx, unsigned long nr_pages, enum { RES_USAGE, + RES_RSVD_USAGE, RES_LIMIT, + RES_RSVD_LIMIT, RES_MAX_USAGE, + RES_RSVD_MAX_USAGE, RES_FAILCNT, + RES_RSVD_FAILCNT, }; static u64 hugetlb_cgroup_read_u64(struct cgroup_subsys_state *css, struct cftype *cft) { struct page_counter *counter; + struct page_counter *rsvd_counter; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(css); counter = &h_cg->hugepage[MEMFILE_IDX(cft->private)]; + rsvd_counter = &h_cg->rsvd_hugepage[MEMFILE_IDX(cft->private)]; switch (MEMFILE_ATTR(cft->private)) { case RES_USAGE: return (u64)page_counter_read(counter) * PAGE_SIZE; + case RES_RSVD_USAGE: + return (u64)page_counter_read(rsvd_counter) * PAGE_SIZE; case RES_LIMIT: return (u64)counter->max * PAGE_SIZE; + case RES_RSVD_LIMIT: + return (u64)rsvd_counter->max * PAGE_SIZE; case RES_MAX_USAGE: return (u64)counter->watermark * PAGE_SIZE; + case RES_RSVD_MAX_USAGE: + return (u64)rsvd_counter->watermark * PAGE_SIZE; case RES_FAILCNT: return counter->failcnt; + case RES_RSVD_FAILCNT: + return rsvd_counter->failcnt; default: BUG(); } @@ -338,10 +366,16 @@ static int hugetlb_cgroup_read_u64_max(struct seq_file *seq, void *v) 1 << huge_page_order(&hstates[idx])); switch (MEMFILE_ATTR(cft->private)) { + case RES_RSVD_USAGE: + counter = &h_cg->rsvd_hugepage[idx]; + /* Fall through. */ case RES_USAGE: val = (u64)page_counter_read(counter); seq_printf(seq, "%llu\n", val * PAGE_SIZE); break; + case RES_RSVD_LIMIT: + counter = &h_cg->rsvd_hugepage[idx]; + /* Fall through. */ case RES_LIMIT: val = (u64)counter->max; if (val == limit) @@ -365,6 +399,7 @@ static ssize_t hugetlb_cgroup_write(struct kernfs_open_file *of, int ret, idx; unsigned long nr_pages; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of)); + bool rsvd = false; if (hugetlb_cgroup_is_root(h_cg)) /* Can't set limit on root */ return -EINVAL; @@ -378,9 +413,14 @@ static ssize_t hugetlb_cgroup_write(struct kernfs_open_file *of, nr_pages = round_down(nr_pages, 1 << huge_page_order(&hstates[idx])); switch (MEMFILE_ATTR(of_cft(of)->private)) { + case RES_RSVD_LIMIT: + rsvd = true; + /* Fall through. */ case RES_LIMIT: mutex_lock(&hugetlb_limit_mutex); - ret = page_counter_set_max(&h_cg->hugepage[idx], nr_pages); + ret = page_counter_set_max( + hugetlb_cgroup_counter_from_cgroup(h_cg, idx, rsvd), + nr_pages); mutex_unlock(&hugetlb_limit_mutex); break; default: @@ -406,18 +446,26 @@ static ssize_t hugetlb_cgroup_reset(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { int ret = 0; - struct page_counter *counter; + struct page_counter *counter, *rsvd_counter; struct hugetlb_cgroup *h_cg = hugetlb_cgroup_from_css(of_css(of)); counter = &h_cg->hugepage[MEMFILE_IDX(of_cft(of)->private)]; + rsvd_counter = + &h_cg->rsvd_hugepage[MEMFILE_IDX(of_cft(of)->private)]; switch (MEMFILE_ATTR(of_cft(of)->private)) { case RES_MAX_USAGE: page_counter_reset_watermark(counter); break; + case RES_RSVD_MAX_USAGE: + page_counter_reset_watermark(rsvd_counter); + break; case RES_FAILCNT: counter->failcnt = 0; break; + case RES_RSVD_FAILCNT: + rsvd_counter->failcnt = 0; + break; default: ret = -EINVAL; break; @@ -472,7 +520,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) struct hstate *h = &hstates[idx]; /* format the size */ - mem_fmt(buf, 32, huge_page_size(h)); + mem_fmt(buf, sizeof(buf), huge_page_size(h)); /* Add the limit file */ cft = &h->cgroup_files_dfl[0]; @@ -482,15 +530,30 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->write = hugetlb_cgroup_write_dfl; cft->flags = CFTYPE_NOT_ON_ROOT; - /* Add the current usage file */ + /* Add the reservation limit file */ cft = &h->cgroup_files_dfl[1]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.max", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_LIMIT); + cft->seq_show = hugetlb_cgroup_read_u64_max; + cft->write = hugetlb_cgroup_write_dfl; + cft->flags = CFTYPE_NOT_ON_ROOT; + + /* Add the current usage file */ + cft = &h->cgroup_files_dfl[2]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.current", buf); cft->private = MEMFILE_PRIVATE(idx, RES_USAGE); cft->seq_show = hugetlb_cgroup_read_u64_max; cft->flags = CFTYPE_NOT_ON_ROOT; + /* Add the current reservation usage file */ + cft = &h->cgroup_files_dfl[3]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.current", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_USAGE); + cft->seq_show = hugetlb_cgroup_read_u64_max; + cft->flags = CFTYPE_NOT_ON_ROOT; + /* Add the events file */ - cft = &h->cgroup_files_dfl[2]; + cft = &h->cgroup_files_dfl[4]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.events", buf); cft->private = MEMFILE_PRIVATE(idx, 0); cft->seq_show = hugetlb_events_show; @@ -498,7 +561,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->flags = CFTYPE_NOT_ON_ROOT; /* Add the events.local file */ - cft = &h->cgroup_files_dfl[3]; + cft = &h->cgroup_files_dfl[5]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.events.local", buf); cft->private = MEMFILE_PRIVATE(idx, 0); cft->seq_show = hugetlb_events_local_show; @@ -507,7 +570,7 @@ static void __init __hugetlb_cgroup_file_dfl_init(int idx) cft->flags = CFTYPE_NOT_ON_ROOT; /* NULL terminate the last cft */ - cft = &h->cgroup_files_dfl[4]; + cft = &h->cgroup_files_dfl[6]; memset(cft, 0, sizeof(*cft)); WARN_ON(cgroup_add_dfl_cftypes(&hugetlb_cgrp_subsys, @@ -521,7 +584,7 @@ static void __init __hugetlb_cgroup_file_legacy_init(int idx) struct hstate *h = &hstates[idx]; /* format the size */ - mem_fmt(buf, 32, huge_page_size(h)); + mem_fmt(buf, sizeof(buf), huge_page_size(h)); /* Add the limit file */ cft = &h->cgroup_files_legacy[0]; @@ -530,28 +593,55 @@ static void __init __hugetlb_cgroup_file_legacy_init(int idx) cft->read_u64 = hugetlb_cgroup_read_u64; cft->write = hugetlb_cgroup_write_legacy; - /* Add the usage file */ + /* Add the reservation limit file */ cft = &h->cgroup_files_legacy[1]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.limit_in_bytes", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_LIMIT); + cft->read_u64 = hugetlb_cgroup_read_u64; + cft->write = hugetlb_cgroup_write_legacy; + + /* Add the usage file */ + cft = &h->cgroup_files_legacy[2]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.usage_in_bytes", buf); cft->private = MEMFILE_PRIVATE(idx, RES_USAGE); cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the reservation usage file */ + cft = &h->cgroup_files_legacy[3]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.usage_in_bytes", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_USAGE); + cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the MAX usage file */ - cft = &h->cgroup_files_legacy[2]; + cft = &h->cgroup_files_legacy[4]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.max_usage_in_bytes", buf); cft->private = MEMFILE_PRIVATE(idx, RES_MAX_USAGE); cft->write = hugetlb_cgroup_reset; cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the MAX reservation usage file */ + cft = &h->cgroup_files_legacy[5]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.max_usage_in_bytes", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_MAX_USAGE); + cft->write = hugetlb_cgroup_reset; + cft->read_u64 = hugetlb_cgroup_read_u64; + /* Add the failcntfile */ - cft = &h->cgroup_files_legacy[3]; + cft = &h->cgroup_files_legacy[6]; snprintf(cft->name, MAX_CFTYPE_NAME, "%s.failcnt", buf); - cft->private = MEMFILE_PRIVATE(idx, RES_FAILCNT); + cft->private = MEMFILE_PRIVATE(idx, RES_FAILCNT); + cft->write = hugetlb_cgroup_reset; + cft->read_u64 = hugetlb_cgroup_read_u64; + + /* Add the reservation failcntfile */ + cft = &h->cgroup_files_legacy[7]; + snprintf(cft->name, MAX_CFTYPE_NAME, "%s.rsvd.failcnt", buf); + cft->private = MEMFILE_PRIVATE(idx, RES_RSVD_FAILCNT); cft->write = hugetlb_cgroup_reset; cft->read_u64 = hugetlb_cgroup_read_u64; /* NULL terminate the last cft */ - cft = &h->cgroup_files_legacy[4]; + cft = &h->cgroup_files_legacy[8]; memset(cft, 0, sizeof(*cft)); WARN_ON(cgroup_add_legacy_cftypes(&hugetlb_cgrp_subsys, -- 2.25.0.341.g760bfbb309-goog

4 years, 4 months

3
28
0 0

[PATCH] Documentation: kunit: Make the KUnit documentation less UML-specific

by David Gow

Remove some of the outmoded "Why KUnit" rationale -- which focuses significantly on UML -- and update the Getting Started guide to mention running tests without the kunit_tool wrapper. Signed-off-by: David Gow <davidgow(a)google.com> --- This is an attempt at resolving some of the issues with the KUnit documentation pointed out here: https://lore.kernel.org/linux-kselftest/CABVgOSkiLi0UNijH1xTSvmsJEE5+ocCZ7n… There's definitely room for further work on the KUnit documentation (e.g., adding more information around the environment tests run in), but this hopefully is better than nothing as a starting point. Documentation/dev-tools/kunit/index.rst | 60 ++++--------------- Documentation/dev-tools/kunit/start.rst | 80 +++++++++++++++++++++---- 2 files changed, 78 insertions(+), 62 deletions(-) diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index d16a4d2c3a41..6064cd14dfad 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -17,63 +17,23 @@ What is KUnit? ============== KUnit is a lightweight unit testing and mocking framework for the Linux kernel. -These tests are able to be run locally on a developer's workstation without a VM -or special hardware. KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, and much more. -Get started now: :doc:`start` - -Why KUnit? -========== - -A unit test is supposed to test a single unit of code in isolation, hence the -name. A unit test should be the finest granularity of testing and as such should -allow all possible code paths to be tested in the code under test; this is only -possible if the code under test is very small and does not have any external -dependencies outside of the test's control like hardware. - -Outside of KUnit, there are no testing frameworks currently -available for the kernel that do not require installing the kernel on a test -machine or in a VM and all require tests to be written in userspace running on -the kernel; this is true for Autotest, and kselftest, disqualifying -any of them from being considered unit testing frameworks. +KUnit consists of a kernel component, which provides a set of macros for easily +writing unit tests. Tests written against KUnit will run on kernel boot if +built-in, or when loaded if built as a module. These tests write out results to +the kernel log in `TAP <https://testanything.org/>`_ format. -KUnit addresses the problem of being able to run tests without needing a virtual -machine or actual hardware with User Mode Linux. User Mode Linux is a Linux -architecture, like ARM or x86; however, unlike other architectures it compiles -to a standalone program that can be run like any other program directly inside -of a host operating system; to be clear, it does not require any virtualization -support; it is just a regular program. +To make running these tests (and reading the results) easier, KUnit offsers +:doc:`kunit_tool <kunit-tool>`, which builds a `User Mode Linux +<http://user-mode-linux.sourceforge.net>`_ kernel, runs it, and parses the test +results. This provides a quick way of running KUnit tests during development. -Alternatively, kunit and kunit tests can be built as modules and tests will -run when the test module is loaded. - -KUnit is fast. Excluding build time, from invocation to completion KUnit can run -several dozen tests in only 10 to 20 seconds; this might not sound like a big -deal to some people, but having such fast and easy to run tests fundamentally -changes the way you go about testing and even writing code in the first place. -Linus himself said in his `git talk at Google -<https://gist.github.com/lorn/1272686/revisions#diff-53c65572127855f1b003db4…>`_: - - "... a lot of people seem to think that performance is about doing the - same thing, just doing it faster, and that is not true. That is not what - performance is all about. If you can do something really fast, really - well, people will start using it differently." - -In this context Linus was talking about branching and merging, -but this point also applies to testing. If your tests are slow, unreliable, are -difficult to write, and require a special setup or special hardware to run, -then you wait a lot longer to write tests, and you wait a lot longer to run -tests; this means that tests are likely to break, unlikely to test a lot of -things, and are unlikely to be rerun once they pass. If your tests are really -fast, you run them all the time, every time you make a change, and every time -someone sends you some code. Why trust that someone ran all their tests -correctly on every change when you can just run them yourself in less time than -it takes to read their test log? +Get started now: :doc:`start` How do I use it? ================ @@ -81,3 +41,5 @@ How do I use it? * :doc:`start` - for new users of KUnit * :doc:`usage` - for a more detailed explanation of KUnit features * :doc:`api/index` - for the list of KUnit APIs used for testing +* :doc:`kunit-tool` - for more information on the kunit_tool helper script +* :doc:`faq` - for answers to some common questions about KUnit diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 4e1d24db6b13..e1c5ce80ce12 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -9,11 +9,10 @@ Installing dependencies KUnit has the same dependencies as the Linux kernel. As long as you can build the kernel, you can run KUnit. -KUnit Wrapper -============= -Included with KUnit is a simple Python wrapper that helps format the output to -easily use and read KUnit output. It handles building and running the kernel, as -well as formatting the output. +Running tests with the KUnit Wrapper +==================================== +Included with KUnit is a simple Python wrapper which runs tests under User Mode +Linux, and formats the test results. The wrapper can be run with: @@ -21,22 +20,42 @@ The wrapper can be run with: ./tools/testing/kunit/kunit.py run --defconfig -For more information on this wrapper (also called kunit_tool) checkout the +For more information on this wrapper (also called kunit_tool) check out the :doc:`kunit-tool` page. Creating a .kunitconfig -======================= -The Python script is a thin wrapper around Kbuild. As such, it needs to be -configured with a ``.kunitconfig`` file. This file essentially contains the -regular Kernel config, with the specific test targets as well. - +----------------------- +If you want to run a specific set of tests (rather than those listed in the +KUnit defconfig), you can provide Kconfig options in the ``.kunitconfig`` file. +This file essentially contains the regular Kernel config, with the specific +test targets as well. The ``.kunitconfig`` should also contain any other config +options required by the tests. + +A good starting point for a ``.kunitconfig`` is the KUnit defconfig: .. code-block:: bash cd $PATH_TO_LINUX_REPO cp arch/um/configs/kunit_defconfig .kunitconfig -Verifying KUnit Works ---------------------- +You can then add any other Kconfig options you wish, e.g.: +.. code-block:: none + + CONFIG_LIST_KUNIT_TEST=y + +:doc:`kunit_tool <kunit-tool>` will ensure that all config options set in +``.kunitconfig`` are set in the kernel ``.config`` before running the tests. +It'll warn you if you haven't included the dependencies of the options you're +using. + +.. note:: + Note that removing something from the ``.kunitconfig`` will not trigger a + rebuild of the ``.config`` file: the configuration is only updated if the + ``.kunitconfig`` is not a subset of ``.config``. This means that you can use + other tools (such as make menuconfig) to adjust other config options. + + +Running the tests +----------------- To make sure that everything is set up correctly, simply invoke the Python wrapper from your kernel repo: @@ -62,6 +81,41 @@ followed by a list of tests that are run. All of them should be passing. Because it is building a lot of sources for the first time, the ``Building KUnit kernel`` step may take a while. +Running tests without the KUnit Wrapper +======================================= + +If you'd rather not use the KUnit Wrapper (if, for example, you need to +integrate with other systems, or use an architecture other than UML), KUnit can +be included in any kernel, and the results read out and parsed manually. + +.. note:: + KUnit is not designed for use in a production system, and it's possible that + tests may reduce the stability or security of the system. + + + +Configuring the kernel +---------------------- + +In order to enable KUnit itself, you simply need to enable the ``CONFIG_KUNIT`` +Kconfig option (it's under Kernel Hacking/Kernel Testing and Coverage in +menuconfig). From there, you can enable any KUnit tests you want: they usually +have config options ending in ``_KUNIT_TEST``. + +KUnit and KUnit tests can be compiled as modules: in this case the tests in a +module will be run when the module is loaded. + +Running the tests +----------------- + +Build and run your kernel as usual. Test output will be written to the kernel +log in `TAP <https://testanything.org/>`_ format. + +.. note:: + It's possible that there will be other lines and/or data interspersed in the + TAP output. + + Writing your first test ======================= -- 2.25.0.341.g760bfbb309-goog

4 years, 4 months

3
2
0 0

[PATCH] Removing a duplicate condition.

by Isaac Young

Signed-off-by: Isaac Young <isaac.young5(a)gmail.com> --- tools/testing/selftests/bpf/prog_tests/select_reuseport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c index 098bcae5f827..0954c7a8aa08 100644 --- a/tools/testing/selftests/bpf/prog_tests/select_reuseport.c +++ b/tools/testing/selftests/bpf/prog_tests/select_reuseport.c @@ -823,7 +823,7 @@ void test_select_reuseport(void) saved_tcp_fo = read_int_sysctl(TCP_FO_SYSCTL); saved_tcp_syncookie = read_int_sysctl(TCP_SYNCOOKIE_SYSCTL); - if (saved_tcp_syncookie < 0 || saved_tcp_syncookie < 0) + if (saved_tcp_syncookie < 0) goto out; if (enable_fastopen()) -- 2.24.1

4 years, 4 months

3
2
0 0

next-20200211 kselftest results

by ci_notify＠linaro.org

Summary ------------------------------------------------------------------------ kernel: 5.6.0-rc1 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: ac431e2d7b1be81bfe58163b9f81ba79bc987dc3 git describe: next-20200211 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20200211 Regressions (compared to build next-20200117) ------------------------------------------------------------------------ No regressions Fixes (compared to build next-20200117) ------------------------------------------------------------------------ No fixes In total: ------------------------------------------------------------------------ Ran 0 total tests in the following environments and test suites. pass 0 fail 0 xfail 0 skip 0 Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - x15 - arm - x86_64 Test Suites ----------- Failures ------------------------------------------------------------------------ juno-r2: x15: x86: i386: hi6220-hikey: dragonboard-410c: Skips ------------------------------------------------------------------------ No skips -- Linaro LKFT https://lkft.linaro.org

4 years, 4 months

1
0
0 0

[PATCH] Documentation: kunit: fix Sphinx directive warning

by Randy Dunlap

From: Randy Dunlap <rdunlap(a)infradead.org> Fix Documentation warning due to missing a blank line after a directive: linux/Documentation/dev-tools/kunit/usage.rst:553: WARNING: Error in "code-block" directive: maximum 1 argument(s) allowed, 3 supplied. .. code-block:: bash modprobe example-test Fixes: 6ae2bfd3df06 ("kunit: update documentation to describe module-based build") Signed-off-by: Randy Dunlap <rdunlap(a)infradead.org> Cc: Alan Maguire <alan.maguire(a)oracle.com> Cc: Knut Omang <knut.omang(a)oracle.com> Cc: Brendan Higgins <brendanhiggins(a)google.com> Cc: linux-kselftest(a)vger.kernel.org Cc: kunit-dev(a)googlegroups.com Cc: Shuah Khan <skhan(a)linuxfoundation.org> --- Documentation/dev-tools/kunit/usage.rst | 1 + 1 file changed, 1 insertion(+) --- lnx-56-rc1.orig/Documentation/dev-tools/kunit/usage.rst +++ lnx-56-rc1/Documentation/dev-tools/kunit/usage.rst @@ -551,6 +551,7 @@ options to your ``.config``: Once the kernel is built and installed, a simple .. code-block:: bash + modprobe example-test ...will run the tests.

4 years, 4 months

3
2
0 0

[RESEND PATCH v9 00/13] selftests/resctrl: Add resctrl selftest

by Fenghua Yu

[Resend the v9 patch set to Shuah Khan and linux-kselftest mailing list. No code and commit message change.] With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected. We introduce resctrl selftest to cover resctrl features on X86, AMD and ARM architectures. It first implements MBM (Memory Bandwidth Monitoring), MBA (Memory Bandwidth Allocation), L3 CAT (Cache Allocation Technology), and CQM (Cache QoS Monitoring) tests. We will enhance the selftest tool to include more functionality tests in the future. The tool has been tested on Intel RDT, AMD QoS and ARM MPAM and is in tools/testing/selftests/resctrl in order to have generic test code base for all architectures. The selftest tool we are introducing here provides a convenient tool which does automatic resctrl testing, is easily available in kernel tree, and covers Intel RDT, AMD QoS and ARM MPAM. There is an existing resctrl test suite 'intel_cmt_cat'. But its major purpose is to test Intel RDT hardware via writing and reading MSR registers. It does access resctrl file system; but the functionalities are very limited. And it doesn't support automatic test and a lot of manual verifications are involved. Changelog: v9: - Per Boris suggestion, add Co-developed-by in each patch to make it clear who contributed to the patch set. v8: Update code per comments from Andre Przywara from ARM: - Change Makefile and remove inline assembly code to build and test the tool on ARM - Change the output to TAP format because the format is both readable by human and other test tools. - Detect resctrl feature from /proc/cpuinfo instead of dmesg to support generic detection on all architectures. - Fix a few coding issues. v7: - Fix a few warnings when compiling patches separately, pointed by Babu v6: - Fix a benchmark reading optimized out issue in newer GCC. - Fix a few coding style issues. - Re-arrange code among patches to make cleaner code. No change in patches structure. v5: - Based the v4 patches submitted by Fenghua Yu and added changes to support AMD. - Changed the function name get_sock_num to get_resource_id. Intel uses socket number for schemata and AMD uses l3 index id. To generalize, changed the function name to get_resource_id. - Added the code to detect vendor. - Disabled the few tests for AMD where the test results are not clear. Also AMD does not have IMC. - Fixed few compile issues. - Some cleanup to make each patch independent. - Tested the patches on AMD system. Fenghua, Need your help to test on Intel box. Please feel free to change and resubmit if something broken. v4: - address comments from Balu and Randy - Add CAT and CQM tests v3: - Change code based on comments from Babu Moger - Remove some unnessary code and use pipe to communicate b/w processes v2: - Change code based on comments from Babu Moger - Clean up other places. Babu Moger (3): selftests/resctrl: Add vendor detection mechanism selftests/resctrl: Use cache index3 id for AMD schemata masks selftests/resctrl: Disable MBA and MBM tests for AMD Fenghua Yu (6): selftests/resctrl: Add README for resctrl tests selftests/resctrl: Add MBM test selftests/resctrl: Add MBA test selftests/resctrl: Add Cache QoS Monitoring (CQM) selftest selftests/resctrl: Add Cache Allocation Technology (CAT) selftest selftests/resctrl: Add the test in MAINTAINERS Sai Praneeth Prakhya (4): selftests/resctrl: Add basic resctrl file system operations and data selftests/resctrl: Read memory bandwidth from perf IMC counter and from resctrl file system selftests/resctrl: Add callback to start a benchmark selftests/resctrl: Add built in benchmark MAINTAINERS | 1 + tools/testing/selftests/resctrl/Makefile | 17 + tools/testing/selftests/resctrl/README | 53 ++ tools/testing/selftests/resctrl/cache.c | 272 +++++++ tools/testing/selftests/resctrl/cat_test.c | 250 ++++++ tools/testing/selftests/resctrl/cqm_test.c | 176 +++++ tools/testing/selftests/resctrl/fill_buf.c | 213 +++++ tools/testing/selftests/resctrl/mba_test.c | 171 ++++ tools/testing/selftests/resctrl/mbm_test.c | 145 ++++ tools/testing/selftests/resctrl/resctrl.h | 107 +++ .../testing/selftests/resctrl/resctrl_tests.c | 202 +++++ tools/testing/selftests/resctrl/resctrl_val.c | 744 ++++++++++++++++++ tools/testing/selftests/resctrl/resctrlfs.c | 722 +++++++++++++++++ 13 files changed, 3073 insertions(+) create mode 100644 tools/testing/selftests/resctrl/Makefile create mode 100644 tools/testing/selftests/resctrl/README create mode 100644 tools/testing/selftests/resctrl/cache.c create mode 100644 tools/testing/selftests/resctrl/cat_test.c create mode 100644 tools/testing/selftests/resctrl/cqm_test.c create mode 100644 tools/testing/selftests/resctrl/fill_buf.c create mode 100644 tools/testing/selftests/resctrl/mba_test.c create mode 100644 tools/testing/selftests/resctrl/mbm_test.c create mode 100644 tools/testing/selftests/resctrl/resctrl.h create mode 100644 tools/testing/selftests/resctrl/resctrl_tests.c create mode 100644 tools/testing/selftests/resctrl/resctrl_val.c create mode 100644 tools/testing/selftests/resctrl/resctrlfs.c -- 2.19.1

4 years, 4 months

2
15
0 0

[PATCH v5 00/12] mm/gup: track FOLL_PIN pages

by John Hubbard

Hi, Here's another update after another round of reviews from Kirill and Jan. There is a git repo and branch, for convenience in reviewing: git@github.com:johnhubbard/linux.git track_user_pages_v5 ============================================================ Changes since v4: * Added documentation about the huge page behavior of the new /proc/vmstat items. * Added a missing mode_node_page_state() call to put_compound_head(). * Fixed a tracepoint call in page_ref_sub_return(). * Added a trailing underscore to a URL in pin_user_pages.rst, to fix a broken generated link. * Added ACKs and reviewed-by's from Jan Kara and Kirill Shutemov. * Rebased onto today's linux.git, and * I am experimenting here with "git format-patch --base=<commit>". This generated the "base-commit:" tag you'll see at the end of this cover letter. I was inspired to do so after trying out a new get-lore-mbox.py tool (it's very nice), mentioned in a recent LWN article (https://lwn.net/Articles/811528/ ). That tool relies on the base-commit tag for some things. ============================================================ Changes since v3: * Rebased onto latest linux.git * Added ACKs and reviewed-by's from Kirill Shutemov and Jan Kara. * /proc/vmstat: * Renamed items, after realizing that I hate the previous names: nr_foll_pin_requested --> nr_foll_pin_acquired nr_foll_pin_returned --> nr_foll_pin_released * Removed the CONFIG_DEBUG_VM guard, and collapsed away a wrapper routine: now just calls mod_node_page_state() directly. * Tweaked the WARN_ON_ONCE() statements in mm/hugetlb.c to be more informative, and added comments above them as well. * Fixed gup_benchmark: signed int --> unsigned long. * One or two minor formatting changes. ============================================================ Changes since v2: * Rebased onto linux.git, because the akpm tree for 5.6 has been merged. * Split the tracking patch into even more patches, as requested. * Merged Matthew Wilcox's dump_page() changes into mine, as part of the first patch. * Renamed: page_dma_pinned() --> page_maybe_dma_pinned(), in response to Kirill Shutemov's review. * Moved a WARN to the top of a routine, and fixed a typo in the commit description of patch #7, also as suggested by Kirill. ============================================================ Changes since v1: * Split the tracking patch into 6 smaller patches * Rebased onto today's linux-next/akpm (there weren't any conflicts). * Fixed an "unsigned int" vs. "int" problem in gup_benchmark, reported by Nathan Chancellor. (I don't see it in my local builds, probably because they use gcc, but an LLVM test found the mismatch.) * Fixed a huge page pincount problem (add/subtract vs. increment/decrement), spotted by Jan Kara. ============================================================ There is a reasonable case to be made for merging two of the patches (patches 7 and 8), given that patch 7 provides tracking that has upper limits on the number of pins that can be done with huge pages. Let me know if anyone wants those merged, but unless there is some weird chance of someone grabbing patch 7 and not patch 8, I don't really see the need. Meanwhile, it's easier to review in this form. Also, patch 3 has been revived. Earlier reviewers asked for it to be merged into the tracking patch (one cannot please everyone, heh), but now it's back out on it's own. This activates tracking of FOLL_PIN pages. This is in support of fixing the get_user_pages()+DMA problem described in [1]-[4]. FOLL_PIN support is now in the main linux tree. However, the patch to use FOLL_PIN to track pages was *not* submitted, because Leon saw an RDMA test suite failure that involved (I think) page refcount overflows when huge pages were used. This patch definitively solves that kind of overflow problem, by adding an exact pincount, for compound pages (of order > 1), in the 3rd struct page of a compound page. If available, that form of pincounting is used, instead of the GUP_PIN_COUNTING_BIAS approach. Thanks again to Jan Kara for that idea. Other interesting changes: * dump_page(): added one, or two new things to report for compound pages: head refcount (for all compound pages), and map_pincount (for compound pages of order > 1). * Documentation/core-api/pin_user_pages.rst: removed the "TODO" for the huge page refcount upper limit problems, and added notes about how it works now. Also added a note about the dump_page() enhancements. * Added some comments in gup.c and mm.h, to explain that there are two ways to count pinned pages: exact (for compound pages of order > 1) and fuzzy (GUP_PIN_COUNTING_BIAS: for all other pages). ============================================================ General notes about the tracking patch: This is a prerequisite to solving the problem of proper interactions between file-backed pages, and [R]DMA activities, as discussed in [1], [2], [3], [4] and in a remarkable number of email threads since about 2017. :) In contrast to earlier approaches, the page tracking can be incrementally applied to the kernel call sites that, until now, have been simply calling get_user_pages() ("gup"). In other words, opt-in by changing from this: get_user_pages() (sets FOLL_GET) put_page() to this: pin_user_pages() (sets FOLL_PIN) unpin_user_page() ============================================================ Next steps: * Convert more subsystems from get_user_pages() to pin_user_pages(). * Work with Ira and others to connect this all up with file system leases. [1] Some slow progress on get_user_pages() (Apr 2, 2019): https://lwn.net/Articles/784574/ [2] DMA and get_user_pages() (LPC: Dec 12, 2018): https://lwn.net/Articles/774411/ [3] The trouble with get_user_pages() (Apr 30, 2018): https://lwn.net/Articles/753027/ [4] LWN kernel index: get_user_pages() https://lwn.net/Kernel/Index/#Memory_management-get_user_pages John Hubbard (12): mm: dump_page(): better diagnostics for compound pages mm/gup: split get_user_pages_remote() into two routines mm/gup: pass a flags arg to __gup_device_* functions mm: introduce page_ref_sub_return() mm/gup: pass gup flags to two more routines mm/gup: require FOLL_GET for get_user_pages_fast() mm/gup: track FOLL_PIN pages mm/gup: page->hpage_pinned_refcount: exact pin counts for huge pages mm: dump_page(): better diagnostics for huge pinned pages mm/gup: /proc/vmstat: pin_user_pages (FOLL_PIN) reporting mm/gup_benchmark: support pin_user_pages() and related calls selftests/vm: run_vmtests: invoke gup_benchmark with basic FOLL_PIN coverage Documentation/core-api/pin_user_pages.rst | 86 ++-- include/linux/mm.h | 108 ++++- include/linux/mm_types.h | 7 +- include/linux/mmzone.h | 2 + include/linux/page_ref.h | 9 + mm/debug.c | 61 ++- mm/gup.c | 452 ++++++++++++++++----- mm/gup_benchmark.c | 71 +++- mm/huge_memory.c | 29 +- mm/hugetlb.c | 60 ++- mm/page_alloc.c | 2 + mm/rmap.c | 6 + mm/vmstat.c | 2 + tools/testing/selftests/vm/gup_benchmark.c | 15 +- tools/testing/selftests/vm/run_vmtests | 22 + 15 files changed, 752 insertions(+), 180 deletions(-) base-commit: 90568ecf561540fa330511e21fcd823b0c3829c6 -- 2.25.0

4 years, 4 months

4
19
0 0

Re: linux-5.6-rc1/tools/testing/selftests/bpf/prog_tests/select_reuseport.cc :826: possible cut-n-paste error ?

by Martin KaFai Lau

On Mon, Feb 10, 2020 at 01:42:45PM +0000, David Binderman wrote: > Hello there, > > saved_tcp_fo = read_int_sysctl(TCP_FO_SYSCTL); > saved_tcp_syncookie = read_int_sysctl(TCP_SYNCOOKIE_SYSCTL); > if (saved_tcp_syncookie < 0 || saved_tcp_syncookie < 0) > goto out; > > Maybe better code > > saved_tcp_fo = read_int_sysctl(TCP_FO_SYSCTL); > saved_tcp_syncookie = read_int_sysctl(TCP_SYNCOOKIE_SYSCTL); > if (saved_tcp_fo < 0 || saved_tcp_syncookie < 0) Thanks for the report. I will post a fix.

4 years, 4 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror February 2020