This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in under a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more.
## What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code.
## Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed.
## More information on KUnit
There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here: https://google.github.io/kunit-docs/third_party/kernel/docs/
A number of test features need to do pretty complicated string printing where it may not be possible to rely on a single preallocated string with parameters.
This provides a library for constructing the string as you go similar to C++'s std::string.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/string-stream.h | 44 ++++++++++ kunit/Kconfig | 16 ++++ kunit/Makefile | 1 + kunit/string-stream.c | 149 ++++++++++++++++++++++++++++++++++ 4 files changed, 210 insertions(+) create mode 100644 include/kunit/string-stream.h create mode 100644 kunit/Kconfig create mode 100644 kunit/Makefile create mode 100644 kunit/string-stream.c
diff --git a/include/kunit/string-stream.h b/include/kunit/string-stream.h new file mode 100644 index 0000000000000..933ed5740cf07 --- /dev/null +++ b/include/kunit/string-stream.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * C++ stream style string builder used in KUnit for building messages. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _KUNIT_STRING_STREAM_H +#define _KUNIT_STRING_STREAM_H + +#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/kref.h> +#include <stdarg.h> + +struct string_stream_fragment { + struct list_head node; + char *fragment; +}; + +struct string_stream { + size_t length; + struct list_head fragments; + + /* length and fragments are protected by this lock */ + spinlock_t lock; + struct kref refcount; + int (*add)(struct string_stream *this, const char *fmt, ...); + int (*vadd)(struct string_stream *this, const char *fmt, va_list args); + char *(*get_string)(struct string_stream *this); + void (*clear)(struct string_stream *this); + bool (*is_empty)(struct string_stream *this); +}; + +struct string_stream *new_string_stream(void); + +void destroy_string_stream(struct string_stream *stream); + +void string_stream_get(struct string_stream *stream); + +int string_stream_put(struct string_stream *stream); + +#endif /* _KUNIT_STRING_STREAM_H */ diff --git a/kunit/Kconfig b/kunit/Kconfig new file mode 100644 index 0000000000000..64480092b2c24 --- /dev/null +++ b/kunit/Kconfig @@ -0,0 +1,16 @@ +# +# KUnit base configuration +# + +menu "KUnit support" + +config KUNIT + bool "Enable support for unit tests (KUnit)" + help + Enables support for kernel unit tests (KUnit), a lightweight unit + testing and mocking framework for the Linux kernel. These tests are + able to be run locally on a developer's workstation without a VM or + special hardware. For more information, please see + Documentation/kunit/ + +endmenu diff --git a/kunit/Makefile b/kunit/Makefile new file mode 100644 index 0000000000000..de16cce9d6a27 --- /dev/null +++ b/kunit/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_KUNIT) += string-stream.o diff --git a/kunit/string-stream.c b/kunit/string-stream.c new file mode 100644 index 0000000000000..1e7efa630cc35 --- /dev/null +++ b/kunit/string-stream.c @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * C++ stream style string builder used in KUnit for building messages. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/list.h> +#include <linux/slab.h> +#include <kunit/string-stream.h> + +static int string_stream_vadd(struct string_stream *this, + const char *fmt, + va_list args) +{ + struct string_stream_fragment *fragment; + int len; + va_list args_for_counting; + unsigned long flags; + + /* Make a copy because `vsnprintf` could change it */ + va_copy(args_for_counting, args); + + /* Need space for null byte. */ + len = vsnprintf(NULL, 0, fmt, args_for_counting) + 1; + + va_end(args_for_counting); + + fragment = kmalloc(sizeof(*fragment), GFP_KERNEL); + if (!fragment) + return -ENOMEM; + + fragment->fragment = kmalloc(len, GFP_KERNEL); + if (!fragment->fragment) { + kfree(fragment); + return -ENOMEM; + } + + len = vsnprintf(fragment->fragment, len, fmt, args); + spin_lock_irqsave(&this->lock, flags); + this->length += len; + list_add_tail(&fragment->node, &this->fragments); + spin_unlock_irqrestore(&this->lock, flags); + return 0; +} + +static int string_stream_add(struct string_stream *this, const char *fmt, ...) +{ + va_list args; + int result; + + va_start(args, fmt); + result = string_stream_vadd(this, fmt, args); + va_end(args); + return result; +} + +static void string_stream_clear(struct string_stream *this) +{ + struct string_stream_fragment *fragment, *fragment_safe; + unsigned long flags; + + spin_lock_irqsave(&this->lock, flags); + list_for_each_entry_safe(fragment, + fragment_safe, + &this->fragments, + node) { + list_del(&fragment->node); + kfree(fragment->fragment); + kfree(fragment); + } + this->length = 0; + spin_unlock_irqrestore(&this->lock, flags); +} + +static char *string_stream_get_string(struct string_stream *this) +{ + struct string_stream_fragment *fragment; + size_t buf_len = this->length + 1; /* +1 for null byte. */ + char *buf; + unsigned long flags; + + buf = kzalloc(buf_len, GFP_KERNEL); + if (!buf) + return NULL; + + spin_lock_irqsave(&this->lock, flags); + list_for_each_entry(fragment, &this->fragments, node) + strlcat(buf, fragment->fragment, buf_len); + spin_unlock_irqrestore(&this->lock, flags); + + return buf; +} + +static bool string_stream_is_empty(struct string_stream *this) +{ + bool is_empty; + unsigned long flags; + + spin_lock_irqsave(&this->lock, flags); + is_empty = list_empty(&this->fragments); + spin_unlock_irqrestore(&this->lock, flags); + + return is_empty; +} + +void destroy_string_stream(struct string_stream *stream) +{ + stream->clear(stream); + kfree(stream); +} + +static void string_stream_destroy(struct kref *kref) +{ + struct string_stream *stream = container_of(kref, + struct string_stream, + refcount); + destroy_string_stream(stream); +} + +struct string_stream *new_string_stream(void) +{ + struct string_stream *stream = kzalloc(sizeof(*stream), GFP_KERNEL); + + if (!stream) + return NULL; + + INIT_LIST_HEAD(&stream->fragments); + spin_lock_init(&stream->lock); + kref_init(&stream->refcount); + stream->add = string_stream_add; + stream->vadd = string_stream_vadd; + stream->get_string = string_stream_get_string; + stream->clear = string_stream_clear; + stream->is_empty = string_stream_is_empty; + return stream; +} + +void string_stream_get(struct string_stream *stream) +{ + kref_get(&stream->refcount); +} + +int string_stream_put(struct string_stream *stream) +{ + return kref_put(&stream->refcount, &string_stream_destroy); +} +
Adds the KUnit core, which allows test cases to be defined and associated with common initialization and cleanup logic.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 165 ++++++++++++++++++++++++++++++++++++++++++ kunit/Makefile | 2 +- kunit/test.c | 168 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 334 insertions(+), 1 deletion(-) create mode 100644 include/kunit/test.h create mode 100644 kunit/test.c
diff --git a/include/kunit/test.h b/include/kunit/test.h new file mode 100644 index 0000000000000..e0b14b227ac44 --- /dev/null +++ b/include/kunit/test.h @@ -0,0 +1,165 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Base unit test (KUnit) API. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _KUNIT_TEST_H +#define _KUNIT_TEST_H + +#include <linux/types.h> +#include <linux/slab.h> + +struct test; + +/** + * struct test_case - represents an individual test case. + * @run_case: the function representing the actual test case. + * @name: the name of the test case. + * + * A test case is a function with the signature, ``void (*)(struct test *)`` + * that makes expectations (see TEST_EXPECT_TRUE()) about code under test. Each + * test case is associated with a &struct test_module and will be run after the + * module's init function and followed by the module's exit function. + * + * A test case should be static and should only be created with the TEST_CASE() + * macro; additionally, every array of test cases should be terminated with an + * empty test case. + * + * Example: + * + * .. code-block:: c + * + * void add_test_basic(struct test *test) + * { + * TEST_EXPECT_EQ(test, 1, add(1, 0)); + * TEST_EXPECT_EQ(test, 2, add(1, 1)); + * TEST_EXPECT_EQ(test, 0, add(-1, 1)); + * TEST_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX)); + * TEST_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN)); + * } + * + * static struct test_case example_test_cases[] = { + * TEST_CASE(add_test_basic), + * {}, + * }; + * + */ +struct test_case { + void (*run_case)(struct test *test); + const char name[256]; + + /* private: internal use only. */ + bool success; +}; + +/** + * TEST_CASE - A helper for creating a &struct test_case + * @test_name: a reference to a test case function. + * + * Takes a symbol for a function representing a test case and creates a &struct + * test_case object from it. See the documentation for &struct test_case for an + * example on how to use it. + */ +#define TEST_CASE(test_name) { .run_case = test_name, .name = #test_name } + +/** + * struct test_module - describes a related collection of &struct test_case s. + * @name: the name of the test. Purely informational. + * @init: called before every test case. + * @exit: called after every test case. + * @test_cases: a null terminated array of test cases. + * + * A test_module is a collection of related &struct test_case s, such that + * @init is called before every test case and @exit is called after every test + * case, similar to the notion of a *test fixture* or a *test class* in other + * unit testing frameworks like JUnit or Googletest. + * + * Every &struct test_case must be associated with a test_module for KUnit to + * run it. + */ +struct test_module { + const char name[256]; + int (*init)(struct test *test); + void (*exit)(struct test *test); + struct test_case *test_cases; +}; + +/** + * struct test - represents a running instance of a test. + * @priv: for user to store arbitrary data. Commonly used to pass data created + * in the init function (see &struct test_module). + * + * Used to store information about the current context under which the test is + * running. Most of this data is private and should only be accessed indirectly + * via public functions; the one exception is @priv which can be used by the + * test writer to store arbitrary data. + */ +struct test { + void *priv; + + /* private: internal use only. */ + const char *name; /* Read only after initialization! */ + spinlock_t lock; /* Gaurds all mutable test state. */ + bool success; /* Protected by lock. */ + void (*vprintk)(const struct test *test, + const char *level, + struct va_format *vaf); +}; + +int test_init_test(struct test *test, const char *name); + +int test_run_tests(struct test_module *module); + +/** + * module_test() - used to register a &struct test_module with KUnit. + * @module: a statically allocated &struct test_module. + * + * Registers @module with the test framework. See &struct test_module for more + * information. + */ +#define module_test(module) \ + static int module_test_init##module(void) \ + { \ + return test_run_tests(&module); \ + } \ + late_initcall(module_test_init##module) + +void __printf(3, 4) test_printk(const char *level, + const struct test *test, + const char *fmt, ...); + +/** + * test_info() - Prints an INFO level message associated with the current test. + * @test: The test context object. + * @fmt: A printk() style format string. + * + * Prints an info level message associated with the test module being run. Takes + * a variable number of format parameters just like printk(). + */ +#define test_info(test, fmt, ...) \ + test_printk(KERN_INFO, test, fmt, ##__VA_ARGS__) + +/** + * test_warn() - Prints a WARN level message associated with the current test. + * @test: The test context object. + * @fmt: A printk() style format string. + * + * See test_info(). + */ +#define test_warn(test, fmt, ...) \ + test_printk(KERN_WARNING, test, fmt, ##__VA_ARGS__) + +/** + * test_err() - Prints an ERROR level message associated with the current test. + * @test: The test context object. + * @fmt: A printk() style format string. + * + * See test_info(). + */ +#define test_err(test, fmt, ...) \ + test_printk(KERN_ERR, test, fmt, ##__VA_ARGS__) + +#endif /* _KUNIT_TEST_H */ diff --git a/kunit/Makefile b/kunit/Makefile index de16cce9d6a27..7fc613a9b383b 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1 +1 @@ -obj-$(CONFIG_KUNIT) += string-stream.o +obj-$(CONFIG_KUNIT) += test.o string-stream.o diff --git a/kunit/test.c b/kunit/test.c new file mode 100644 index 0000000000000..4732e5f0d7575 --- /dev/null +++ b/kunit/test.c @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Base unit test (KUnit) API. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/sched.h> +#include <linux/sched/debug.h> +#include <os.h> +#include <kunit/test.h> + +static bool test_get_success(struct test *test) +{ + unsigned long flags; + bool success; + + spin_lock_irqsave(&test->lock, flags); + success = test->success; + spin_unlock_irqrestore(&test->lock, flags); + + return success; +} + +static void test_set_success(struct test *test, bool success) +{ + unsigned long flags; + + spin_lock_irqsave(&test->lock, flags); + test->success = success; + spin_unlock_irqrestore(&test->lock, flags); +} + +static int test_vprintk_emit(const struct test *test, + int level, + const char *fmt, + va_list args) +{ + return vprintk_emit(0, level, NULL, 0, fmt, args); +} + +static int test_printk_emit(const struct test *test, + int level, + const char *fmt, ...) +{ + va_list args; + int ret; + + va_start(args, fmt); + ret = test_vprintk_emit(test, level, fmt, args); + va_end(args); + + return ret; +} + +static void test_vprintk(const struct test *test, + const char *level, + struct va_format *vaf) +{ + test_printk_emit(test, + level[1] - '0', + "kunit %s: %pV", test->name, vaf); +} + +int test_init_test(struct test *test, const char *name) +{ + spin_lock_init(&test->lock); + test->name = name; + test->vprintk = test_vprintk; + + return 0; +} + +/* + * Initializes and runs test case. Does not clean up or do post validations. + */ +static void test_run_case_internal(struct test *test, + struct test_module *module, + struct test_case *test_case) +{ + int ret; + + if (module->init) { + ret = module->init(test); + if (ret) { + test_err(test, "failed to initialize: %d", ret); + test_set_success(test, false); + return; + } + } + + test_case->run_case(test); +} + +/* + * Performs post validations and cleanup after a test case was run. + * XXX: Should ONLY BE CALLED AFTER test_run_case_internal! + */ +static void test_run_case_cleanup(struct test *test, + struct test_module *module, + struct test_case *test_case) +{ + if (module->exit) + module->exit(test); +} + +/* + * Performs all logic to run a test case. + */ +static bool test_run_case(struct test *test, + struct test_module *module, + struct test_case *test_case) +{ + test_set_success(test, true); + + test_run_case_internal(test, module, test_case); + test_run_case_cleanup(test, module, test_case); + + return test_get_success(test); +} + +int test_run_tests(struct test_module *module) +{ + bool all_passed = true, success; + struct test_case *test_case; + struct test test; + int ret; + + ret = test_init_test(&test, module->name); + if (ret) + return ret; + + for (test_case = module->test_cases; test_case->run_case; test_case++) { + success = test_run_case(&test, module, test_case); + if (!success) + all_passed = false; + + test_info(&test, + "%s %s", + test_case->name, + success ? "passed" : "failed"); + } + + if (all_passed) + test_info(&test, "all tests passed"); + else + test_info(&test, "one or more tests failed"); + + return 0; +} + +void test_printk(const char *level, + const struct test *test, + const char *fmt, ...) +{ + struct va_format vaf; + va_list args; + + va_start(args, fmt); + + vaf.fmt = fmt; + vaf.va = &args; + + test->vprintk(test, level, &vaf); + + va_end(args); +}
This creates a common API for test managed resources like memory and test objects.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 109 +++++++++++++++++++++++++++++++++++++++++++ kunit/test.c | 95 +++++++++++++++++++++++++++++++++++++ 2 files changed, 204 insertions(+)
diff --git a/include/kunit/test.h b/include/kunit/test.h index e0b14b227ac44..1c116a20063da 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -12,6 +12,69 @@ #include <linux/types.h> #include <linux/slab.h>
+struct test_resource; + +typedef int (*test_resource_init_t)(struct test_resource *, void *); +typedef void (*test_resource_free_t)(struct test_resource *); + +/** + * struct test_resource - represents a *test managed resource* + * @allocation: for the user to store arbitrary data. + * @free: a user supplied function to free the resource. Populated by + * test_alloc_resource(). + * + * Represents a *test managed resource*, a resource which will automatically be + * cleaned up at the end of a test case. + * + * Example: + * + * .. code-block:: c + * + * struct test_kmalloc_params { + * size_t size; + * gfp_t gfp; + * }; + * + * static int test_kmalloc_init(struct test_resource *res, void *context) + * { + * struct test_kmalloc_params *params = context; + * res->allocation = kmalloc(params->size, params->gfp); + * + * if (!res->allocation) + * return -ENOMEM; + * + * return 0; + * } + * + * static void test_kmalloc_free(struct test_resource *res) + * { + * kfree(res->allocation); + * } + * + * void *test_kmalloc(struct test *test, size_t size, gfp_t gfp) + * { + * struct test_kmalloc_params params; + * struct test_resource *res; + * + * params.size = size; + * params.gfp = gfp; + * + * res = test_alloc_resource(test, test_kmalloc_init, + * test_kmalloc_free, ¶ms); + * if (res) + * return res->allocation; + * else + * return NULL; + * } + */ +struct test_resource { + void *allocation; + test_resource_free_t free; + + /* private: internal use only. */ + struct list_head node; +}; + struct test;
/** @@ -104,6 +167,7 @@ struct test { const char *name; /* Read only after initialization! */ spinlock_t lock; /* Gaurds all mutable test state. */ bool success; /* Protected by lock. */ + struct list_head resources; /* Protected by lock. */ void (*vprintk)(const struct test *test, const char *level, struct va_format *vaf); @@ -127,6 +191,51 @@ int test_run_tests(struct test_module *module); } \ late_initcall(module_test_init##module)
+/** + * test_alloc_resource() - Allocates a *test managed resource*. + * @test: The test context object. + * @init: a user supplied function to initialize the resource. + * @free: a user supplied function to free the resource. + * @context: for the user to pass in arbitrary data. + * + * Allocates a *test managed resource*, a resource which will automatically be + * cleaned up at the end of a test case. See &struct test_resource for an + * example. + */ +struct test_resource *test_alloc_resource(struct test *test, + test_resource_init_t init, + test_resource_free_t free, + void *context); + +void test_free_resource(struct test *test, struct test_resource *res); + +/** + * test_kmalloc() - Just like kmalloc() except the allocation is *test managed*. + * @test: The test context object. + * @size: The size in bytes of the desired memory. + * @gfp: flags passed to underlying kmalloc(). + * + * Just like `kmalloc(...)`, except the allocation is managed by the test case + * and is automatically cleaned up after the test case concludes. See &struct + * test_resource for more information. + */ +void *test_kmalloc(struct test *test, size_t size, gfp_t gfp); + +/** + * test_kzalloc() - Just like test_kmalloc(), but zeroes the allocation. + * @test: The test context object. + * @size: The size in bytes of the desired memory. + * @gfp: flags passed to underlying kmalloc(). + * + * See kzalloc() and test_kmalloc() for more information. + */ +static inline void *test_kzalloc(struct test *test, size_t size, gfp_t gfp) +{ + return test_kmalloc(test, size, gfp | __GFP_ZERO); +} + +void test_cleanup(struct test *test); + void __printf(3, 4) test_printk(const char *level, const struct test *test, const char *fmt, ...); diff --git a/kunit/test.c b/kunit/test.c index 4732e5f0d7575..fd0a51245215e 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -66,6 +66,7 @@ static void test_vprintk(const struct test *test, int test_init_test(struct test *test, const char *name) { spin_lock_init(&test->lock); + INIT_LIST_HEAD(&test->resources); test->name = name; test->vprintk = test_vprintk;
@@ -93,6 +94,11 @@ static void test_run_case_internal(struct test *test, test_case->run_case(test); }
+static void test_case_internal_cleanup(struct test *test) +{ + test_cleanup(test); +} + /* * Performs post validations and cleanup after a test case was run. * XXX: Should ONLY BE CALLED AFTER test_run_case_internal! @@ -103,6 +109,8 @@ static void test_run_case_cleanup(struct test *test, { if (module->exit) module->exit(test); + + test_case_internal_cleanup(test); }
/* @@ -150,6 +158,93 @@ int test_run_tests(struct test_module *module) return 0; }
+struct test_resource *test_alloc_resource(struct test *test, + test_resource_init_t init, + test_resource_free_t free, + void *context) +{ + struct test_resource *res; + unsigned long flags; + int ret; + + res = kzalloc(sizeof(*res), GFP_KERNEL); + if (!res) + return NULL; + + ret = init(res, context); + if (ret) + return NULL; + + res->free = free; + spin_lock_irqsave(&test->lock, flags); + list_add_tail(&res->node, &test->resources); + spin_unlock_irqrestore(&test->lock, flags); + + return res; +} + +void test_free_resource(struct test *test, struct test_resource *res) +{ + res->free(res); + list_del(&res->node); + kfree(res); +} + +struct test_kmalloc_params { + size_t size; + gfp_t gfp; +}; + +static int test_kmalloc_init(struct test_resource *res, void *context) +{ + struct test_kmalloc_params *params = context; + + res->allocation = kmalloc(params->size, params->gfp); + if (!res->allocation) + return -ENOMEM; + + return 0; +} + +static void test_kmalloc_free(struct test_resource *res) +{ + kfree(res->allocation); +} + +void *test_kmalloc(struct test *test, size_t size, gfp_t gfp) +{ + struct test_kmalloc_params params; + struct test_resource *res; + + params.size = size; + params.gfp = gfp; + + res = test_alloc_resource(test, + test_kmalloc_init, + test_kmalloc_free, + ¶ms); + + if (res) + return res->allocation; + else + return NULL; +} + +void test_cleanup(struct test *test) +{ + struct test_resource *resource, *resource_safe; + unsigned long flags; + + spin_lock_irqsave(&test->lock, flags); + list_for_each_entry_safe(resource, + resource_safe, + &test->resources, + node) { + test_free_resource(test, resource); + } + spin_unlock_irqrestore(&test->lock, flags); +} + void test_printk(const char *level, const struct test *test, const char *fmt, ...)
Adds a C++ style log library for for logging test results.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test-stream.h | 49 ++++++++++++ include/kunit/test.h | 2 + kunit/Makefile | 2 +- kunit/test-stream.c | 153 ++++++++++++++++++++++++++++++++++++ kunit/test.c | 8 ++ 5 files changed, 213 insertions(+), 1 deletion(-) create mode 100644 include/kunit/test-stream.h create mode 100644 kunit/test-stream.c
diff --git a/include/kunit/test-stream.h b/include/kunit/test-stream.h new file mode 100644 index 0000000000000..c5dfd95ef21bb --- /dev/null +++ b/include/kunit/test-stream.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * C++ stream style string formatter and printer used in KUnit for outputting + * KUnit messages. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _KUNIT_TEST_STREAM_H +#define _KUNIT_TEST_STREAM_H + +#include <linux/types.h> +#include <kunit/string-stream.h> + +struct test; + +/** + * struct test_stream - a std::stream style string builder. + * @set_level: sets the level that this string should be printed at. + * @add: adds the formatted input to the internal buffer. + * @commit: prints out the internal buffer to the user. + * @clear: clears the internal buffer. + * + * A std::stream style string builder. Allows messages to be built up and + * printed all at once. + */ +struct test_stream { + void (*set_level)(struct test_stream *this, const char *level); + void (*add)(struct test_stream *this, const char *fmt, ...); + void (*append)(struct test_stream *this, struct test_stream *other); + void (*commit)(struct test_stream *this); + void (*clear)(struct test_stream *this); + /* private: internal use only. */ + struct test *test; + spinlock_t lock; /* Guards level. */ + const char *level; + struct string_stream *internal_stream; +}; + +/** + * test_new_stream() - constructs a new &struct test_stream. + * @test: The test context object. + * + * Constructs a new test managed &struct test_stream. + */ +struct test_stream *test_new_stream(struct test *test); + +#endif /* _KUNIT_TEST_STREAM_H */ diff --git a/include/kunit/test.h b/include/kunit/test.h index 1c116a20063da..68320fa2452de 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -11,6 +11,7 @@
#include <linux/types.h> #include <linux/slab.h> +#include <kunit/test-stream.h>
struct test_resource;
@@ -171,6 +172,7 @@ struct test { void (*vprintk)(const struct test *test, const char *level, struct va_format *vaf); + void (*fail)(struct test *test, struct test_stream *stream); };
int test_init_test(struct test *test, const char *name); diff --git a/kunit/Makefile b/kunit/Makefile index 7fc613a9b383b..5b4562ea7f322 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1 +1 @@ -obj-$(CONFIG_KUNIT) += test.o string-stream.o +obj-$(CONFIG_KUNIT) += test.o string-stream.o test-stream.o diff --git a/kunit/test-stream.c b/kunit/test-stream.c new file mode 100644 index 0000000000000..392966864a708 --- /dev/null +++ b/kunit/test-stream.c @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * C++ stream style string formatter and printer used in KUnit for outputting + * KUnit messages. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> +#include <kunit/test-stream.h> +#include <kunit/string-stream.h> + +static const char *test_stream_get_level(struct test_stream *this) +{ + unsigned long flags; + const char *level; + + spin_lock_irqsave(&this->lock, flags); + level = this->level; + spin_unlock_irqrestore(&this->lock, flags); + + return level; +} + +static void test_stream_set_level(struct test_stream *this, const char *level) +{ + unsigned long flags; + + spin_lock_irqsave(&this->lock, flags); + this->level = level; + spin_unlock_irqrestore(&this->lock, flags); +} + +static void test_stream_add(struct test_stream *this, const char *fmt, ...) +{ + va_list args; + struct string_stream *stream = this->internal_stream; + + va_start(args, fmt); + if (stream->vadd(stream, fmt, args) < 0) + test_err(this->test, "Failed to allocate fragment: %s", fmt); + + va_end(args); +} + +static void test_stream_append(struct test_stream *this, + struct test_stream *other) +{ + struct string_stream *other_stream = other->internal_stream; + const char *other_content; + + other_content = other_stream->get_string(other_stream); + + if (!other_content) { + test_err(this->test, + "Failed to get string from second argument for appending."); + return; + } + + this->add(this, other_content); +} + +static void test_stream_clear(struct test_stream *this) +{ + this->internal_stream->clear(this->internal_stream); +} + +static void test_stream_commit(struct test_stream *this) +{ + struct string_stream *stream = this->internal_stream; + struct string_stream_fragment *fragment; + const char *level; + char *buf; + + level = test_stream_get_level(this); + if (!level) { + test_err(this->test, + "Stream was committed without a specified log level."); + level = KERN_ERR; + this->set_level(this, level); + } + + buf = stream->get_string(stream); + if (!buf) { + test_err(this->test, + "Could not allocate buffer, dumping stream:"); + list_for_each_entry(fragment, &stream->fragments, node) { + test_err(this->test, fragment->fragment); + } + goto cleanup; + } + + test_printk(level, this->test, buf); + kfree(buf); + +cleanup: + this->clear(this); +} + +static int test_stream_init(struct test_resource *res, void *context) +{ + struct test *test = context; + struct test_stream *stream; + + stream = kzalloc(sizeof(*stream), GFP_KERNEL); + if (!stream) + return -ENOMEM; + res->allocation = stream; + stream->test = test; + spin_lock_init(&stream->lock); + stream->internal_stream = new_string_stream(); + + if (!stream->internal_stream) + return -ENOMEM; + + stream->set_level = test_stream_set_level; + stream->add = test_stream_add; + stream->append = test_stream_append; + stream->commit = test_stream_commit; + stream->clear = test_stream_clear; + + return 0; +} + +static void test_stream_free(struct test_resource *res) +{ + struct test_stream *stream = res->allocation; + + if (!stream->internal_stream->is_empty(stream->internal_stream)) { + test_err(stream->test, + "End of test case reached with uncommitted stream entries."); + stream->commit(stream); + } + + destroy_string_stream(stream->internal_stream); + kfree(stream); +} + +struct test_stream *test_new_stream(struct test *test) +{ + struct test_resource *res; + + res = test_alloc_resource(test, + test_stream_init, + test_stream_free, + test); + + if (res) + return res->allocation; + else + return NULL; +} diff --git a/kunit/test.c b/kunit/test.c index fd0a51245215e..f798183533c8d 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -63,12 +63,20 @@ static void test_vprintk(const struct test *test, "kunit %s: %pV", test->name, vaf); }
+static void test_fail(struct test *test, struct test_stream *stream) +{ + test_set_success(test, false); + stream->set_level(stream, KERN_ERR); + stream->commit(stream); +} + int test_init_test(struct test *test, const char *name) { spin_lock_init(&test->lock); INIT_LIST_HEAD(&test->resources); test->name = name; test->vprintk = test_vprintk; + test->fail = test_fail;
return 0; }
Added support for expectations, which allow properties to be specified and then verified in tests.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 259 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 259 insertions(+)
diff --git a/include/kunit/test.h b/include/kunit/test.h index 68320fa2452de..d652825d7296f 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -273,4 +273,263 @@ void __printf(3, 4) test_printk(const char *level, #define test_err(test, fmt, ...) \ test_printk(KERN_ERR, test, fmt, ##__VA_ARGS__)
+static inline struct test_stream *test_expect_start(struct test *test, + const char *file, + const char *line) +{ + struct test_stream *stream = test_new_stream(test); + + stream->add(stream, "EXPECTATION FAILED at %s:%s\n\t", file, line); + + return stream; +} + +static inline void test_expect_end(struct test *test, + bool success, + struct test_stream *stream) +{ + if (!success) + test->fail(test, stream); + else + stream->clear(stream); +} + +#define TEST_EXPECT_START(test) \ + test_expect_start(test, __FILE__, __stringify(__LINE__)) + +#define TEST_EXPECT_END(test, success, stream) \ + test_expect_end(test, success, stream) + +#define TEST_EXPECT(test, success, message) do { \ + struct test_stream *__stream = TEST_EXPECT_START(test); \ + \ + __stream->add(__stream, message); \ + TEST_EXPECT_END(test, success, __stream); \ +} while (0) + +/** + * TEST_SUCCEED() - A no-op expectation. Only exists for code clarity. + * @test: The test context object. + * + * The opposite of TEST_FAIL(), it is an expectation that cannot fail. In other + * words, it does nothing and only exists for code clarity. See + * TEST_EXPECT_TRUE() for more information. + */ +#define TEST_SUCCEED(test) do {} while (0) + +/** + * TEST_FAIL() - Always causes a test to fail when evaluated. + * @test: The test context object. + * @message: an informational message to be printed when the assertion is made. + * + * The opposite of TEST_SUCCEED(), it is an expectation that always fails. In + * other words, it always results in a failed expectation, and consequently + * always causes the test case to fail when evaluated. See TEST_EXPECT_TRUE() + * for more information. + */ +#define TEST_FAIL(test, message) TEST_EXPECT(test, false, message) + +/** + * TEST_EXPECT_TRUE() - Causes a test failure when the expression is not true. + * @test: The test context object. + * @condition: an arbitrary boolean expression. The test fails when this does + * not evaluate to true. + * + * This and expectations of the form `TEST_EXPECT_*` will cause the test case to + * fail when the specified condition is not met; however, it will not prevent + * the test case from continuing to run; this is otherwise known as an + * *expectation failure*. + */ +#define TEST_EXPECT_TRUE(test, condition) \ + TEST_EXPECT(test, (condition), \ + "Expected " #condition " is true, but is false.") + +/** + * TEST_EXPECT_FALSE() - Causes a test failure when the expression is not false. + * @test: The test context object. + * @condition: an arbitrary boolean expression. The test fails when this does + * not evaluate to false. + * + * Sets an expectation that @condition evaluates to false. See + * TEST_EXPECT_TRUE() for more information. + */ +#define TEST_EXPECT_FALSE(test, condition) \ + TEST_EXPECT(test, !(condition), \ + "Expected " #condition " is false, but is true.") + +static inline void test_expect_binary(struct test *test, + long long left, const char *left_name, + long long right, const char *right_name, + bool compare_result, + const char *compare_name, + const char *file, + const char *line) +{ + struct test_stream *stream = test_expect_start(test, file, line); + + stream->add(stream, + "Expected %s %s %s, but\n", + left_name, compare_name, right_name); + stream->add(stream, "\t\t%s == %lld\n", left_name, left); + stream->add(stream, "\t\t%s == %lld", right_name, right); + + test_expect_end(test, compare_result, stream); +} + +/* + * A factory macro for defining the expectations for the basic comparisons + * defined for the built in types. + * + * Unfortunately, there is no common type that all types can be promoted to for + * which all the binary operators behave the same way as for the actual types + * (for example, there is no type that long long and unsigned long long can + * both be cast to where the comparison result is preserved for all values). So + * the best we can do is do the comparison in the original types and then coerce + * everything to long long for printing; this way, the comparison behaves + * correctly and the printed out value usually makes sense without + * interpretation, but can always be interpretted to figure out the actual + * value. + */ +#define TEST_EXPECT_BINARY(test, left, condition, right) do { \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + test_expect_binary(test, \ + (long long) __left, #left, \ + (long long) __right, #right, \ + __left condition __right, #condition, \ + __FILE__, __stringify(__LINE__)); \ +} while (0) + +/** + * TEST_EXPECT_EQ() - Sets an expectation that @left and @right are equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the values that @left and @right evaluate to are + * equal. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, (@left) == (@right)). See TEST_EXPECT_TRUE() for more + * information. + */ +#define TEST_EXPECT_EQ(test, left, right) \ + TEST_EXPECT_BINARY(test, left, ==, right) + +/** + * TEST_EXPECT_NE() - An expectation that @left and @right are not equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the values that @left and @right evaluate to are not + * equal. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, (@left) != (@right)). See TEST_EXPECT_TRUE() for more + * information. + */ +#define TEST_EXPECT_NE(test, left, right) \ + TEST_EXPECT_BINARY(test, left, !=, right) + +/** + * TEST_EXPECT_LT() - An expectation that @left is less than @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the value that @left evaluates to is less than the + * value that @right evaluates to. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, (@left) < (@right)). See TEST_EXPECT_TRUE() for more + * information. + */ +#define TEST_EXPECT_LT(test, left, right) \ + TEST_EXPECT_BINARY(test, left, <, right) + +/** + * TEST_EXPECT_LE() - An expectation that @left is less than or equal to @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the value that @left evaluates to is less than or + * equal to the value that @right evaluates to. Semantically this is equivalent + * to TEST_EXPECT_TRUE(@test, (@left) <= (@right)). See TEST_EXPECT_TRUE() for + * more information. + */ +#define TEST_EXPECT_LE(test, left, right) \ + TEST_EXPECT_BINARY(test, left, <=, right) + +/** + * TEST_EXPECT_GT() - An expectation that @left is greater than @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the value that @left evaluates to is greater than + * the value that @right evaluates to. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, (@left) > (@right)). See TEST_EXPECT_TRUE() for more + * information. + */ +#define TEST_EXPECT_GT(test, left, right) \ + TEST_EXPECT_BINARY(test, left, >, right) + +/** + * TEST_EXPECT_GE() - Expects that @left is greater than or equal to @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an expectation that the value that @left evaluates to is greater than + * the value that @right evaluates to. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, (@left) >= (@right)). See TEST_EXPECT_TRUE() for more + * information. + */ +#define TEST_EXPECT_GE(test, left, right) \ + TEST_EXPECT_BINARY(test, left, >=, right) + +/** + * TEST_EXPECT_STREQ() - An expectation that strings @left and @right are equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a null terminated string. + * @right: an arbitrary expression that evaluates to a null terminated string. + * + * Sets an expectation that the values that @left and @right evaluate to are + * equal. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, !strcmp((@left), (@right))). See TEST_EXPECT_TRUE() + * for more information. + */ +#define TEST_EXPECT_STREQ(test, left, right) do { \ + struct test_stream *__stream = TEST_EXPECT_START(test); \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + \ + __stream->add(__stream, "Expected " #left " == " #right ", but\n"); \ + __stream->add(__stream, "\t\t%s == %s\n", #left, __left); \ + __stream->add(__stream, "\t\t%s == %s\n", #right, __right); \ + \ + TEST_EXPECT_END(test, !strcmp(left, right), __stream); \ +} while (0) + +/** + * TEST_EXPECT_NOT_ERR_OR_NULL() - Expects that @ptr is not null and not err. + * @test: The test context object. + * @ptr: an arbitrary pointer. + * + * Sets an expectation that the value that @ptr evaluates to is not null and not + * an errno stored in a pointer. This is semantically equivalent to + * TEST_EXPECT_TRUE(@test, !IS_ERR_OR_NULL(@ptr)). See TEST_EXPECT_TRUE() for + * more information. + */ +#define TEST_EXPECT_NOT_ERR_OR_NULL(test, ptr) do { \ + struct test_stream *__stream = TEST_EXPECT_START(test); \ + typeof(ptr) __ptr = (ptr); \ + \ + if (!__ptr) \ + __stream->add(__stream, \ + "Expected " #ptr " is not null, but is."); \ + if (IS_ERR(__ptr)) \ + __stream->add(__stream, \ + "Expected " #ptr " is not error, but is: %ld", \ + PTR_ERR(__ptr)); \ + \ + TEST_EXPECT_END(test, !IS_ERR_OR_NULL(__ptr), __stream); \ +} while (0) + #endif /* _KUNIT_TEST_H */
Makes minimum number of changes outside of the KUnit directories for KUnit to build and run using UML.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- Makefile | 2 +- arch/um/Kconfig.rest | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile index 863f58503beed..c8e659468ed49 100644 --- a/Makefile +++ b/Makefile @@ -944,7 +944,7 @@ endif
ifeq ($(KBUILD_EXTMOD),) -core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ +core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ kunit/
vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \ diff --git a/arch/um/Kconfig.rest b/arch/um/Kconfig.rest index 08327b9c0cbea..484c4cfcad1ef 100644 --- a/arch/um/Kconfig.rest +++ b/arch/um/Kconfig.rest @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 + +source "kunit/Kconfig" + source "init/Kconfig"
source "kernel/Kconfig.freezer"
Hi Brendan,
I very excitedly jumped on these patches to try them out, as this is essentially something I was trying to do a few weeks back.
On 17/10/18 00:50, Brendan Higgins wrote:
Makes minimum number of changes outside of the KUnit directories for KUnit to build and run using UML.
Signed-off-by: Brendan Higgins brendanhiggins@google.com
Makefile | 2 +- arch/um/Kconfig.rest | 3 +++
But this file isn't present on v4.19-rc8
It looks like the file is removed at f163977d21a2 ("um: cleanup Kconfig files")
What version have you currently based these patches on? Do you expect to keep a branch somewhere that's easy to pull in?
Please add me to the CC list as an interested party on later versions :-)
It might be of interest to the automated testing mailing list too ? (Tim?)
-- regards
Kieran
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile index 863f58503beed..c8e659468ed49 100644 --- a/Makefile +++ b/Makefile @@ -944,7 +944,7 @@ endif ifeq ($(KBUILD_EXTMOD),) -core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ +core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ kunit/ vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \ diff --git a/arch/um/Kconfig.rest b/arch/um/Kconfig.rest index 08327b9c0cbea..484c4cfcad1ef 100644 --- a/arch/um/Kconfig.rest +++ b/arch/um/Kconfig.rest @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0
+source "kunit/Kconfig"
source "init/Kconfig" source "kernel/Kconfig.freezer"
On Wed, Oct 17, 2018 at 8:29 AM Kieran Bingham kieran.bingham@ideasonboard.com wrote:
Hi Brendan,
I very excitedly jumped on these patches to try them out, as this is essentially something I was trying to do a few weeks back.
On 17/10/18 00:50, Brendan Higgins wrote:
Makes minimum number of changes outside of the KUnit directories for KUnit to build and run using UML.
Signed-off-by: Brendan Higgins brendanhiggins@google.com
Makefile | 2 +- arch/um/Kconfig.rest | 3 +++
But this file isn't present on v4.19-rc8
It looks like the file is removed at f163977d21a2 ("um: cleanup Kconfig files")
Whoops.
What version have you currently based these patches on?
This patch set is based on 4.18, but I have versions for 4.4, 4.9, 4.14, 4.16, and 4.17.
I will rebase this patch set to 4.19-rc8 and send it out shortly.
Do you expect to keep a branch somewhere that's easy to pull in?
I have one here: https://kunit-review.googlesource.com/c/linux/+/1030 (you can check it out with git fetch https://kunit.googlesource.com/linux refs/changes/30/1030/4 && git checkout FETCH_HEAD)
Please add me to the CC list as an interested party on later versions :-)
Will do.
-----Original Message----- From: Kieran Bingham
Hi Brendan,
I very excitedly jumped on these patches to try them out, as this is essentially something I was trying to do a few weeks back.
On 17/10/18 00:50, Brendan Higgins wrote:
Makes minimum number of changes outside of the KUnit directories for KUnit to build and run using UML.
Signed-off-by: Brendan Higgins brendanhiggins@google.com
Makefile | 2 +- arch/um/Kconfig.rest | 3 +++
But this file isn't present on v4.19-rc8
It looks like the file is removed at f163977d21a2 ("um: cleanup Kconfig files")
What version have you currently based these patches on? Do you expect to keep a branch somewhere that's easy to pull in?
Please add me to the CC list as an interested party on later versions :-)
It might be of interest to the automated testing mailing list too ? (Tim?)
I think this is interesting to groups doing automated testing of the kernel (including myself) as another set of tests to run. Right now I don't see it as having any special attributes related to automation. But I could be wrong. -- Tim
On Wed, Oct 17, 2018 at 10:52 AM Tim.Bird@sony.com wrote:
It might be of interest to the automated testing mailing list too ? (Tim?)
I think this is interesting to groups doing automated testing of the kernel (including myself) as another set of tests to run. Right now I don't see it as having any special attributes related to automation. But I could be wrong.
Pardon my ignorance, but by automated testing you mean a CI server with presubmits, nightlys, and things of the sort?
If that's the case, KUnit could be helpful because of the low resource cost in running them and the speed at which they run. There are some other features we would like to add which would help with that goal as well like test isolation. We actually have a presubmit server internally for running KUnit tests that can usually respond to patches with test results within a couple minutes. Would something like that be interesting?
-----Original Message----- From: Brendan Higgins
On Wed, Oct 17, 2018 at 10:52 AM Tim.Bird@sony.com wrote:
It might be of interest to the automated testing mailing list too ? (Tim?)
I think this is interesting to groups doing automated testing of the kernel (including myself) as another set of tests to run. Right now I don't see it as having any special attributes related to automation. But I could be
wrong.
Pardon my ignorance, but by automated testing you mean a CI server with presubmits, nightlys, and things of the sort?
Yes.
If that's the case, KUnit could be helpful because of the low resource cost in running them and the speed at which they run.
True.
There are some other features we would like to add which would help with that goal as well like test isolation. We actually have a presubmit server internally for running KUnit tests that can usually respond to patches with test results within a couple minutes. Would something like that be interesting?
I think the code and architecture of the software that handles presubmit, mail-list scanning, notifications, etc. would be of interest. But KUnit features themselves (macro definitions, mocking vs. faking, etc.) would not. I only say that in the context of CC-ing the automated testing list on the patch set. Of course the KUnit features are interesting by themselves for testers doing unit testing. -- Tim
On Wed, Oct 17, 2018 at 2:18 PM Tim.Bird@sony.com wrote: <snip>
There are some other features we would like to add which would help with that goal as well like test isolation. We actually have a presubmit server internally for running KUnit tests that can usually respond to patches with test results within a couple minutes. Would something like that be interesting?
I think the code and architecture of the software that handles presubmit, mail-list scanning, notifications, etc. would be of interest. But KUnit features themselves (macro definitions, mocking vs. faking, etc.) would not. I only say that in the context of CC-ing the automated testing list on the patch set. Of course the KUnit features are interesting by themselves for testers doing unit testing.
Fair enough. Well, we will share what we have in that regard, but that belongs on a separate thread.
BTW, what mailing list is that? I am also guessing your code isn't in the kernel repo; where is it?
Added a test for string stream along with a more simple example.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- kunit/Kconfig | 12 ++++++ kunit/Makefile | 2 + kunit/example-test.c | 88 ++++++++++++++++++++++++++++++++++++++ kunit/string-stream-test.c | 61 ++++++++++++++++++++++++++ 4 files changed, 163 insertions(+) create mode 100644 kunit/example-test.c create mode 100644 kunit/string-stream-test.c
diff --git a/kunit/Kconfig b/kunit/Kconfig index 64480092b2c24..5cb500355c873 100644 --- a/kunit/Kconfig +++ b/kunit/Kconfig @@ -13,4 +13,16 @@ config KUNIT special hardware. For more information, please see Documentation/kunit/
+config KUNIT_TEST + bool "KUnit test for KUnit" + depends on KUNIT + help + Enables KUnit test to test KUnit. + +config KUNIT_EXAMPLE_TEST + bool "Example test for KUnit" + depends on KUNIT + help + Enables example KUnit test to demo features of KUnit. + endmenu diff --git a/kunit/Makefile b/kunit/Makefile index 5b4562ea7f322..319eb9dc8be0e 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1 +1,3 @@ obj-$(CONFIG_KUNIT) += test.o string-stream.o test-stream.o +obj-$(CONFIG_KUNIT_TEST) += string-stream-test.o +obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/example-test.c b/kunit/example-test.c new file mode 100644 index 0000000000000..e9bd2b41c5fd2 --- /dev/null +++ b/kunit/example-test.c @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Example KUnit test to show how to use KUnit. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> + +/* + * This is the most fundamental element of KUnit, the test case. A test case + * makes a set EXPECTATIONs and ASSERTIONs about the behavior of some code; if + * any expectations or assertions are not met, the test fails; otherwise, the + * test passes. + * + * In KUnit, a test case is just a function with the signature + * `void (*)(struct test *)`. `struct test` is a context object that stores + * information about the current test. + */ +static void example_simple_test(struct test *test) +{ + /* + * This is an EXPECTATION; it is how KUnit tests things. When you want + * to test a piece of code, you set some expectations about what the + * code should do. KUnit then runs the test and verifies that the code's + * behavior matched what was expected. + */ + TEST_EXPECT_EQ(test, 1 + 1, 2); +} + +/* + * This is run once before each test case, see the comment on + * example_test_module for more information. + */ +static int example_test_init(struct test *test) +{ + test_info(test, "initializing"); + + return 0; +} + +/* + * Here we make a list of all the test cases we want to add to the test module + * below. + */ +static struct test_case example_test_cases[] = { + /* + * This is a helper to create a test case object from a test case + * function; its exact function is not important to understand how to + * use KUnit, just know that this is how you associate test cases with a + * test module. + */ + TEST_CASE(example_simple_test), + {}, +}; + +/* + * This defines a suite or grouping of tests. + * + * Test cases are defined as belonging to the suite by adding them to + * `test_cases`. + * + * Often it is desirable to run some function which will set up things which + * will be used by every test; this is accomplished with an `init` function + * which runs before each test case is invoked. Similarly, an `exit` function + * may be specified which runs after every test case and can be used to for + * cleanup. For clarity, running tests in a test module would behave as follows: + * + * module.init(test); + * module.test_case[0](test); + * module.exit(test); + * module.init(test); + * module.test_case[1](test); + * module.exit(test); + * ...; + */ +static struct test_module example_test_module = { + .name = "example", + .init = example_test_init, + .test_cases = example_test_cases, +}; + +/* + * This registers the above test module telling KUnit that this is a suite of + * tests that need to be run. + */ +module_test(example_test_module); diff --git a/kunit/string-stream-test.c b/kunit/string-stream-test.c new file mode 100644 index 0000000000000..07c626cbfffbf --- /dev/null +++ b/kunit/string-stream-test.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for struct string_stream. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/slab.h> +#include <kunit/test.h> +#include <kunit/string-stream.h> + +static void string_stream_test_get_string(struct test *test) +{ + struct string_stream *stream = new_string_stream(); + char *output; + + stream->add(stream, "Foo"); + stream->add(stream, " %s", "bar"); + + output = stream->get_string(stream); + TEST_EXPECT_STREQ(test, output, "Foo bar"); + kfree(output); + destroy_string_stream(stream); +} + +static void string_stream_test_add_and_clear(struct test *test) +{ + struct string_stream *stream = new_string_stream(); + char *output; + int i; + + for (i = 0; i < 10; i++) + stream->add(stream, "A"); + + output = stream->get_string(stream); + TEST_EXPECT_STREQ(test, output, "AAAAAAAAAA"); + TEST_EXPECT_EQ(test, stream->length, 10); + TEST_EXPECT_FALSE(test, stream->is_empty(stream)); + kfree(output); + + stream->clear(stream); + + output = stream->get_string(stream); + TEST_EXPECT_STREQ(test, output, ""); + TEST_EXPECT_TRUE(test, stream->is_empty(stream)); + destroy_string_stream(stream); +} + +static struct test_case string_stream_test_cases[] = { + TEST_CASE(string_stream_test_get_string), + TEST_CASE(string_stream_test_add_and_clear), + {} +}; + +static struct test_module string_stream_test_module = { + .name = "string-stream-test", + .test_cases = string_stream_test_cases +}; +module_test(string_stream_test_module); +
Added context to current thread that allows a test to specify that it wants to skip the normal checks to run an installed fault catcher.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- arch/um/include/asm/processor-generic.h | 4 +++- arch/um/kernel/trap.c | 15 +++++++++++---- 2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/arch/um/include/asm/processor-generic.h b/arch/um/include/asm/processor-generic.h index b58b746d3f2ca..d566cd416ff02 100644 --- a/arch/um/include/asm/processor-generic.h +++ b/arch/um/include/asm/processor-generic.h @@ -27,6 +27,7 @@ struct thread_struct { struct task_struct *prev_sched; struct arch_thread arch; jmp_buf switch_buf; + bool is_running_test; struct { int op; union { @@ -51,7 +52,8 @@ struct thread_struct { .fault_addr = NULL, \ .prev_sched = NULL, \ .arch = INIT_ARCH_THREAD, \ - .request = { 0 } \ + .request = { 0 }, \ + .is_running_test = false, \ }
static inline void release_thread(struct task_struct *task) diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c index ec9a42c14c565..9b97712daf14f 100644 --- a/arch/um/kernel/trap.c +++ b/arch/um/kernel/trap.c @@ -201,6 +201,12 @@ void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs) segv(*fi, UPT_IP(regs), UPT_IS_USER(regs), regs); }
+static void segv_run_catcher(jmp_buf *catcher, void *fault_addr) +{ + current->thread.fault_addr = fault_addr; + UML_LONGJMP(catcher, 1); +} + /* * We give a *copy* of the faultinfo in the regs to segv. * This must be done, since nesting SEGVs could overwrite @@ -219,7 +225,10 @@ unsigned long segv(struct faultinfo fi, unsigned long ip, int is_user, if (!is_user && regs) current->thread.segv_regs = container_of(regs, struct pt_regs, regs);
- if (!is_user && (address >= start_vm) && (address < end_vm)) { + catcher = current->thread.fault_catcher; + if (catcher && current->thread.is_running_test) + segv_run_catcher(catcher, (void *) address); + else if (!is_user && (address >= start_vm) && (address < end_vm)) { flush_tlb_kernel_vm(); goto out; } @@ -246,12 +255,10 @@ unsigned long segv(struct faultinfo fi, unsigned long ip, int is_user, address = 0; }
- catcher = current->thread.fault_catcher; if (!err) goto out; else if (catcher != NULL) { - current->thread.fault_addr = (void *) address; - UML_LONGJMP(catcher, 1); + segv_run_catcher(catcher, (void *) address); } else if (current->thread.fault_addr != NULL) panic("fault_addr set but no fault catcher");
Added support for assertions which are like expectations except the test terminates if the assertion is not satisfied.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 272 ++++++++++++++++++++++++++++++++++++- kunit/Makefile | 2 +- kunit/string-stream-test.c | 12 +- kunit/test-test.c | 37 +++++ kunit/test.c | 131 ++++++++++++++++-- 5 files changed, 436 insertions(+), 18 deletions(-) create mode 100644 kunit/test-test.c
diff --git a/include/kunit/test.h b/include/kunit/test.h index d652825d7296f..49a9d6e43992c 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -84,9 +84,10 @@ struct test; * @name: the name of the test case. * * A test case is a function with the signature, ``void (*)(struct test *)`` - * that makes expectations (see TEST_EXPECT_TRUE()) about code under test. Each - * test case is associated with a &struct test_module and will be run after the - * module's init function and followed by the module's exit function. + * that makes expectations and assertions (see TEST_EXPECT_TRUE() and + * TEST_ASSERT_TRUE()) about code under test. Each test case is associated with + * a &struct test_module and will be run after the module's init function and + * followed by the module's exit function. * * A test case should be static and should only be created with the TEST_CASE() * macro; additionally, every array of test cases should be terminated with an @@ -168,11 +169,14 @@ struct test { const char *name; /* Read only after initialization! */ spinlock_t lock; /* Gaurds all mutable test state. */ bool success; /* Protected by lock. */ + bool death_test; /* Protected by lock. */ struct list_head resources; /* Protected by lock. */ + void (*set_death_test)(struct test *test, bool death_test); void (*vprintk)(const struct test *test, const char *level, struct va_format *vaf); void (*fail)(struct test *test, struct test_stream *stream); + void (*abort)(struct test *test); };
int test_init_test(struct test *test, const char *name); @@ -532,4 +536,266 @@ static inline void test_expect_binary(struct test *test, TEST_EXPECT_END(test, !IS_ERR_OR_NULL(__ptr), __stream); \ } while (0)
+static inline struct test_stream *test_assert_start(struct test *test, + const char *file, + const char *line) +{ + struct test_stream *stream = test_new_stream(test); + + stream->add(stream, "ASSERTION FAILED at %s:%s\n\t", file, line); + + return stream; +} + +static inline void test_assert_end(struct test *test, + bool success, + struct test_stream *stream) +{ + if (!success) { + test->fail(test, stream); + test->abort(test); + } else { + stream->clear(stream); + } +} + +#define TEST_ASSERT_START(test) \ + test_assert_start(test, __FILE__, __stringify(__LINE__)) + +#define TEST_ASSERT_END(test, success, stream) \ + test_assert_end(test, success, stream) + +#define TEST_ASSERT(test, success, message) do { \ + struct test_stream *__stream = TEST_ASSERT_START(test); \ + \ + __stream->add(__stream, message); \ + TEST_ASSERT_END(test, success, __stream); \ +} while (0) + +#define TEST_ASSERT_FAILURE(test, message) TEST_ASSERT(test, false, message) + +/** + * TEST_ASSERT_TRUE() - Causes an assertion failure when expression is not true. + * @test: The test context object. + * @condition: an arbitrary boolean expression. The test fails and aborts when + * this does not evaluate to true. + * + * This and assertions of the form `TEST_ASSERT_*` will cause the test case to + * fail *and immediately abort* when the specified condition is not met. Unlike + * an expectation failure, it will prevent the test case from continuing to run; + * this is otherwise known as an *assertion failure*. + */ +#define TEST_ASSERT_TRUE(test, condition) \ + TEST_ASSERT(test, (condition), \ + "Asserted " #condition " is true, but is false.") + +/** + * TEST_ASSERT_FALSE() - Sets an assertion that @condition is false. + * @test: The test context object. + * @condition: an arbitrary boolean expression. + * + * Sets an assertion that the value that @condition evaluates to is false. This + * is the same as TEST_EXPECT_FALSE(), except it causes an assertion failure + * (see TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_FALSE(test, condition) \ + TEST_ASSERT(test, !(condition), \ + "Asserted " #condition " is false, but is true.") + +static inline void test_assert_binary(struct test *test, + long long left, const char *left_name, + long long right, const char *right_name, + bool compare_result, + const char *compare_name, + const char *file, + const char *line) +{ + struct test_stream *stream = test_assert_start(test, file, line); + + stream->add(stream, + "Asserted %s %s %s, but\n", + left_name, compare_name, right_name); + stream->add(stream, "\t\t%s == %lld\n", left_name, left); + stream->add(stream, "\t\t%s == %lld", right_name, right); + + test_assert_end(test, compare_result, stream); +} + +/* + * A factory macro for defining the expectations for the basic comparisons + * defined for the built in types. + * + * Unfortunately, there is no common type that all types can be promoted to for + * which all the binary operators behave the same way as for the actual types + * (for example, there is no type that long long and unsigned long long can + * both be cast to where the comparison result is preserved for all values). So + * the best we can do is do the comparison in the original types and then coerce + * everything to long long for printing; this way, the comparison behaves + * correctly and the printed out value usually makes sense without + * interpretation, but can always be interpretted to figure out the actual + * value. + */ +#define TEST_ASSERT_BINARY(test, left, condition, right) do { \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + test_assert_binary(test, \ + (long long) __left, #left, \ + (long long) __right, #right, \ + __left condition __right, #condition, \ + __FILE__, __stringify(__LINE__)); \ +} while (0) + +/** + * TEST_ASSERT_EQ() - Sets an assertion that @left and @right are equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the values that @left and @right evaluate to are + * equal. This is the same as TEST_EXPECT_EQ(), except it causes an assertion + * failure (see TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_EQ(test, left, right) \ + TEST_ASSERT_BINARY(test, left, ==, right) + +/** + * TEST_ASSERT_NE() - An assertion that @left and @right are not equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the values that @left and @right evaluate to are not + * equal. This is the same as TEST_EXPECT_NE(), except it causes an assertion + * failure (see TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_NE(test, left, right) \ + TEST_ASSERT_BINARY(test, left, !=, right) + +/** + * TEST_ASSERT_LT() - An assertion that @left is less than @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the value that @left evaluates to is less than the + * value that @right evaluates to. This is the same as TEST_EXPECT_LT(), except + * it causes an assertion failure (see TEST_ASSERT_TRUE()) when the assertion is + * not met. + */ +#define TEST_ASSERT_LT(test, left, right) \ + TEST_ASSERT_BINARY(test, left, <, right) + +/** + * TEST_ASSERT_LE() - An assertion that @left is less than or equal to @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the value that @left evaluates to is less than or + * equal to the value that @right evaluates to. This is the same as + * TEST_EXPECT_LE(), except it causes an assertion failure (see + * TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_LE(test, left, right) \ + TEST_ASSERT_BINARY(test, left, <=, right) + +/** + * TEST_ASSERT_GT() - An assertion that @left is greater than @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the value that @left evaluates to is greater than the + * value that @right evaluates to. This is the same as TEST_EXPECT_GT(), except + * it causes an assertion failure (see TEST_ASSERT_TRUE()) when the assertion is + * not met. + */ +#define TEST_ASSERT_GT(test, left, right) \ + TEST_ASSERT_BINARY(test, left, >, right) + +/** + * TEST_ASSERT_GE() - Assertion that @left is greater than or equal to @right. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a primitive C type. + * @right: an arbitrary expression that evaluates to a primitive C type. + * + * Sets an assertion that the value that @left evaluates to is greater than the + * value that @right evaluates to. This is the same as TEST_EXPECT_GE(), except + * it causes an assertion failure (see TEST_ASSERT_TRUE()) when the assertion is + * not met. + */ +#define TEST_ASSERT_GE(test, left, right) \ + TEST_ASSERT_BINARY(test, left, >=, right) + +/** + * TEST_ASSERT_STREQ() - An assertion that strings @left and @right are equal. + * @test: The test context object. + * @left: an arbitrary expression that evaluates to a null terminated string. + * @right: an arbitrary expression that evaluates to a null terminated string. + * + * Sets an assertion that the values that @left and @right evaluate to are + * equal. This is the same as TEST_EXPECT_STREQ(), except it causes an + * assertion failure (see TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_STREQ(test, left, right) do { \ + struct test_stream *__stream = TEST_ASSERT_START(test); \ + typeof(left) __left = (left); \ + typeof(right) __right = (right); \ + \ + __stream->add(__stream, "Asserted " #left " == " #right ", but\n"); \ + __stream->add(__stream, "\t\t%s == %s\n", #left, __left); \ + __stream->add(__stream, "\t\t%s == %s\n", #right, __right); \ + \ + TEST_ASSERT_END(test, !strcmp(left, right), __stream); \ +} while (0) + +/** + * TEST_ASSERT_NOT_ERR_OR_NULL() - Assertion that @ptr is not null and not err. + * @test: The test context object. + * @ptr: an arbitrary pointer. + * + * Sets an assertion that the value that @ptr evaluates to is not null and not + * an errno stored in a pointer. This is the same as + * TEST_EXPECT_NOT_ERR_OR_NULL(), except it causes an assertion failure (see + * TEST_ASSERT_TRUE()) when the assertion is not met. + */ +#define TEST_ASSERT_NOT_ERR_OR_NULL(test, ptr) do { \ + struct test_stream *__stream = TEST_ASSERT_START(test); \ + typeof(ptr) __ptr = (ptr); \ + \ + if (!__ptr) \ + __stream->add(__stream, \ + "Asserted " #ptr " is not null, but is."); \ + if (IS_ERR(__ptr)) \ + __stream->add(__stream, \ + "Asserted " #ptr " is not error, but is: %ld", \ + PTR_ERR(__ptr)); \ + \ + TEST_ASSERT_END(test, !IS_ERR_OR_NULL(__ptr), __stream); \ +} while (0) + +/** + * TEST_ASSERT_SIGSEGV() - An assertion that @expr will cause a segfault. + * @test: The test context object. + * @expr: an arbitrary block of code. + * + * Sets an assertion that @expr, when evaluated, will cause a segfault. + * Currently this assertion is only really useful for testing the KUnit + * framework, as a segmentation fault in normal kernel code is always incorrect. + * However, the plan is to replace this assertion with an arbitrary death + * assertion similar to + * https://github.com/google/googletest/blob/master/googletest/docs/advanced.md... + * which will probably be massaged to make sense in the context of the kernel + * (maybe assert that a panic occurred, or that BUG() was called). + * + * NOTE: no code after this assertion will ever be executed. + */ +#define TEST_ASSERT_SIGSEGV(test, expr) do { \ + test->set_death_test(test, true); \ + expr; \ + test->set_death_test(test, false); \ + TEST_ASSERT_FAILURE(test, \ + "Asserted that " #expr " would cause death, but did not.");\ +} while (0) + #endif /* _KUNIT_TEST_H */ diff --git a/kunit/Makefile b/kunit/Makefile index 319eb9dc8be0e..2f1c069e165cb 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,3 +1,3 @@ obj-$(CONFIG_KUNIT) += test.o string-stream.o test-stream.o -obj-$(CONFIG_KUNIT_TEST) += string-stream-test.o +obj-$(CONFIG_KUNIT_TEST) += test-test.o string-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/string-stream-test.c b/kunit/string-stream-test.c index 07c626cbfffbf..5947fada67d96 100644 --- a/kunit/string-stream-test.c +++ b/kunit/string-stream-test.c @@ -19,7 +19,7 @@ static void string_stream_test_get_string(struct test *test) stream->add(stream, " %s", "bar");
output = stream->get_string(stream); - TEST_EXPECT_STREQ(test, output, "Foo bar"); + TEST_ASSERT_STREQ(test, output, "Foo bar"); kfree(output); destroy_string_stream(stream); } @@ -34,16 +34,16 @@ static void string_stream_test_add_and_clear(struct test *test) stream->add(stream, "A");
output = stream->get_string(stream); - TEST_EXPECT_STREQ(test, output, "AAAAAAAAAA"); - TEST_EXPECT_EQ(test, stream->length, 10); - TEST_EXPECT_FALSE(test, stream->is_empty(stream)); + TEST_ASSERT_STREQ(test, output, "AAAAAAAAAA"); + TEST_ASSERT_EQ(test, stream->length, 10); + TEST_ASSERT_FALSE(test, stream->is_empty(stream)); kfree(output);
stream->clear(stream);
output = stream->get_string(stream); - TEST_EXPECT_STREQ(test, output, ""); - TEST_EXPECT_TRUE(test, stream->is_empty(stream)); + TEST_ASSERT_STREQ(test, output, ""); + TEST_ASSERT_TRUE(test, stream->is_empty(stream)); destroy_string_stream(stream); }
diff --git a/kunit/test-test.c b/kunit/test-test.c new file mode 100644 index 0000000000000..fd4b90208f0c3 --- /dev/null +++ b/kunit/test-test.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for core test infrastructure. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ +#include <kunit/test.h> + +static void test_test_catches_segfault(struct test *test) +{ + void (*invalid_func)(void) = (void (*)(void)) SIZE_MAX; + + TEST_ASSERT_SIGSEGV(test, invalid_func()); +} + +static int test_test_init(struct test *test) +{ + return 0; +} + +static void test_test_exit(struct test *test) +{ +} + +static struct test_case test_test_cases[] = { + TEST_CASE(test_test_catches_segfault), + {}, +}; + +static struct test_module test_test_module = { + .name = "test-test", + .init = test_test_init, + .exit = test_test_exit, + .test_cases = test_test_cases, +}; +module_test(test_test_module); diff --git a/kunit/test.c b/kunit/test.c index f798183533c8d..f89cfaaf5eb79 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -32,6 +32,27 @@ static void test_set_success(struct test *test, bool success) spin_unlock_irqrestore(&test->lock, flags); }
+static bool test_get_death_test(struct test *test) +{ + unsigned long flags; + bool death_test; + + spin_lock_irqsave(&test->lock, flags); + death_test = test->death_test; + spin_unlock_irqrestore(&test->lock, flags); + + return death_test; +} + +static void test_set_death_test(struct test *test, bool death_test) +{ + unsigned long flags; + + spin_lock_irqsave(&test->lock, flags); + test->death_test = death_test; + spin_unlock_irqrestore(&test->lock, flags); +} + static int test_vprintk_emit(const struct test *test, int level, const char *fmt, @@ -70,13 +91,34 @@ static void test_fail(struct test *test, struct test_stream *stream) stream->commit(stream); }
+static void __noreturn test_abort(struct test *test) +{ + test_set_death_test(test, true); + if (current->thread.fault_catcher && current->thread.is_running_test) + UML_LONGJMP(current->thread.fault_catcher, 1); + + /* + * Attempted to abort from a not properly initialized test context. + */ + test_err(test, + "Attempted to abort from a not properly initialized test context!"); + if (!current->thread.fault_catcher) + test_err(test, "No fault_catcher present!"); + if (!current->thread.is_running_test) + test_err(test, "is_running_test not set!"); + show_stack(NULL, NULL); + BUG(); +} + int test_init_test(struct test *test, const char *name) { spin_lock_init(&test->lock); INIT_LIST_HEAD(&test->resources); test->name = name; + test->set_death_test = test_set_death_test; test->vprintk = test_vprintk; test->fail = test_fail; + test->abort = test_abort;
return 0; } @@ -122,16 +164,89 @@ static void test_run_case_cleanup(struct test *test, }
/* - * Performs all logic to run a test case. + * Handles an unexpected crash in a test case. */ -static bool test_run_case(struct test *test, - struct test_module *module, - struct test_case *test_case) +static void test_handle_test_crash(struct test *test, + struct test_module *module, + struct test_case *test_case) { - test_set_success(test, true); + test_err(test, "%s crashed", test_case->name); + /* + * TODO(brendanhiggins@google.com): This prints the stack trace up + * through this frame, not up to the frame that caused the crash. + */ + show_stack(NULL, NULL); + + test_case_internal_cleanup(test); +}
- test_run_case_internal(test, module, test_case); - test_run_case_cleanup(test, module, test_case); +/* + * Performs all logic to run a test case. It also catches most errors that + * occurs in a test case and reports them as failures. + * + * XXX: THIS DOES NOT FOLLOW NORMAL CONTROL FLOW. READ CAREFULLY!!! + */ +static bool test_run_case_catch_errors(struct test *test, + struct test_module *module, + struct test_case *test_case) +{ + jmp_buf fault_catcher; + int faulted; + + test_set_success(test, true); + test_set_death_test(test, false); + + /* + * Tell the trap subsystem that we want to catch any segfaults that + * occur. + */ + current->thread.is_running_test = true; + current->thread.fault_catcher = &fault_catcher; + + /* + * ENTER HANDLER: If a failure occurs, we enter here. + */ + faulted = UML_SETJMP(&fault_catcher); + if (faulted == 0) { + /* + * NORMAL CASE: we have not run test_run_case_internal yet. + * + * test_run_case_internal may encounter a fatal error; if it + * does, we will jump to ENTER_HANDLER above instead of + * continuing normal control flow. + */ + test_run_case_internal(test, module, test_case); + /* + * This line may never be reached. + */ + test_run_case_cleanup(test, module, test_case); + } else if (test_get_death_test(test)) { + /* + * EXPECTED DEATH: test_run_case_internal encountered + * anticipated fatal error. Everything should be in a safe + * state. + */ + test_run_case_cleanup(test, module, test_case); + } else { + /* + * UNEXPECTED DEATH: test_run_case_internal encountered an + * unanticipated fatal error. We have no idea what the state of + * the test case is in. + */ + test_handle_test_crash(test, module, test_case); + test_set_success(test, false); + } + /* + * EXIT HANDLER: test case has been run and all possible errors have + * been handled. + */ + + /* + * Tell the trap subsystem that we no longer want to catch any + * segfaults. + */ + current->thread.fault_catcher = NULL; + current->thread.is_running_test = false;
return test_get_success(test); } @@ -148,7 +263,7 @@ int test_run_tests(struct test_module *module) return ret;
for (test_case = module->test_cases; test_case->run_case; test_case++) { - success = test_run_case(&test, module, test_case); + success = test_run_case_catch_errors(&test, module, test_case); if (!success) all_passed = false;
Added a way to add plugins that require a test module to be loaded during initialization.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 19 +++++++++++++++++++ kunit/test.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+)
diff --git a/include/kunit/test.h b/include/kunit/test.h index 49a9d6e43992c..58dbe2aee423f 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -152,6 +152,12 @@ struct test_module { struct test_case *test_cases; };
+struct test_initcall { + struct list_head node; + int (*init)(struct test_initcall *this, struct test *test); + void (*exit)(struct test_initcall *this); +}; + /** * struct test - represents a running instance of a test. * @priv: for user to store arbitrary data. Commonly used to pass data created @@ -183,6 +189,19 @@ int test_init_test(struct test *test, const char *name);
int test_run_tests(struct test_module *module);
+void test_install_initcall(struct test_initcall *initcall); + +#define test_pure_initcall(fn) postcore_initcall(fn) + +#define test_register_initcall(initcall) \ + static int register_test_initcall_##initcall(void) \ + { \ + test_install_initcall(&initcall); \ + \ + return 0; \ + } \ + test_pure_initcall(register_test_initcall_##initcall) + /** * module_test() - used to register a &struct test_module with KUnit. * @module: a statically allocated &struct test_module. diff --git a/kunit/test.c b/kunit/test.c index f89cfaaf5eb79..9737465fb0568 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -53,6 +53,19 @@ static void test_set_death_test(struct test *test, bool death_test) spin_unlock_irqrestore(&test->lock, flags); }
+struct test_global_context { + struct list_head initcalls; +}; + +static struct test_global_context test_global_context = { + .initcalls = LIST_HEAD_INIT(test_global_context.initcalls), +}; + +void test_install_initcall(struct test_initcall *initcall) +{ + list_add_tail(&initcall->node, &test_global_context.initcalls); +} + static int test_vprintk_emit(const struct test *test, int level, const char *fmt, @@ -130,8 +143,18 @@ static void test_run_case_internal(struct test *test, struct test_module *module, struct test_case *test_case) { + struct test_initcall *initcall; int ret;
+ list_for_each_entry(initcall, &test_global_context.initcalls, node) { + ret = initcall->init(initcall, test); + if (ret) { + test_err(test, "failed to initialize: %d", ret); + test->success = false; + return; + } + } + if (module->init) { ret = module->init(test); if (ret) { @@ -146,6 +169,12 @@ static void test_run_case_internal(struct test *test,
static void test_case_internal_cleanup(struct test *test) { + struct test_initcall *initcall; + + list_for_each_entry(initcall, &test_global_context.initcalls, node) { + initcall->exit(initcall); + } + test_cleanup(test); }
Adds a way to specify that certain conditions must be met at the end of a test case.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/test.h | 6 ++++++ kunit/test.c | 13 ++++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h index 58dbe2aee423f..be2b11117d1de 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -158,6 +158,11 @@ struct test_initcall { void (*exit)(struct test_initcall *this); };
+struct test_post_condition { + struct list_head node; + void (*validate)(struct test_post_condition *condition); +}; + /** * struct test - represents a running instance of a test. * @priv: for user to store arbitrary data. Commonly used to pass data created @@ -177,6 +182,7 @@ struct test { bool success; /* Protected by lock. */ bool death_test; /* Protected by lock. */ struct list_head resources; /* Protected by lock. */ + struct list_head post_conditions; void (*set_death_test)(struct test *test, bool death_test); void (*vprintk)(const struct test *test, const char *level, diff --git a/kunit/test.c b/kunit/test.c index 9737465fb0568..6ea60059b4918 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -125,8 +125,9 @@ static void __noreturn test_abort(struct test *test)
int test_init_test(struct test *test, const char *name) { - spin_lock_init(&test->lock); INIT_LIST_HEAD(&test->resources); + INIT_LIST_HEAD(&test->post_conditions); + spin_lock_init(&test->lock); test->name = name; test->set_death_test = test_set_death_test; test->vprintk = test_vprintk; @@ -186,6 +187,16 @@ static void test_run_case_cleanup(struct test *test, struct test_module *module, struct test_case *test_case) { + struct test_post_condition *condition, *condition_safe; + + list_for_each_entry_safe(condition, + condition_safe, + &test->post_conditions, + node) { + condition->validate(condition); + list_del(&condition->node); + } + if (module->exit) module->exit(test);
This adds struct MOCK(foo) as a NonptrType so that it is recognized correctly in declarations.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- scripts/checkpatch.pl | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 447857ffaf6be..9806f190796de 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -745,6 +745,10 @@ sub build_types { (?: (?:typeof|__typeof__)\s*([^)]*)| (?:$typeTypedefs\b)| + # Matching a \b breaks struct MOCK(foo) syntax, + # so we need to have it not lumped in with the + # types in @typeList. + (?:struct\s+MOCK($Ident))| (?:${all}\b) ) (?:\s+$Modifier|\s+const)*
On Tue, 2018-10-16 at 16:51 -0700, Brendan Higgins wrote:
This adds struct MOCK(foo) as a NonptrType so that it is recognized correctly in declarations.
I think this commit message is lacking context.
Signed-off-by: Brendan Higgins brendanhiggins@google.com
scripts/checkpatch.pl | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 447857ffaf6be..9806f190796de 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -745,6 +745,10 @@ sub build_types { (?: (?:typeof|__typeof__)\s*([^)]*)| (?:$typeTypedefs\b)|
# Matching a \b breaks struct MOCK(foo) syntax,# so we need to have it not lumped in with the# types in @typeList.(?:struct\s+MOCK\($Ident\))| (?:${all}\b) ) (?:\s+$Modifier|\s+const)*
On Tue, Oct 16, 2018 at 4:59 PM Joe Perches joe@perches.com wrote:
On Tue, 2018-10-16 at 16:51 -0700, Brendan Higgins wrote:
This adds struct MOCK(foo) as a NonptrType so that it is recognized correctly in declarations.
I think this commit message is lacking context.
Oh sorry, you are right. We added macros for generating mocks from types, so we decided to name the generated mock types struct MOCK(foo) (where the base type is struct foo); this commit makes checkpatch not complain at this new syntax.
Adds macros for parsing and manipulating parameter lists needed for generating mocks.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/params.h | 305 ++++++++++++++++++++++++++++++++++++++++ kunit/Makefile | 2 +- kunit/mock-macro-test.c | 149 ++++++++++++++++++++ 3 files changed, 455 insertions(+), 1 deletion(-) create mode 100644 include/kunit/params.h create mode 100644 kunit/mock-macro-test.c
diff --git a/include/kunit/params.h b/include/kunit/params.h new file mode 100644 index 0000000000000..f9692d3cd703f --- /dev/null +++ b/include/kunit/params.h @@ -0,0 +1,305 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Macros for parsing and manipulating parameter lists needed for generating + * mocks. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _KUNIT_PARAMS_H +#define _KUNIT_PARAMS_H + +#define NUM_VA_ARGS_IMPL(__dummy, \ + __1, \ + __2, \ + __3, \ + __4, \ + __5, \ + __6, \ + __7, \ + __8, \ + __9, \ + __10, \ + __11, \ + __12, \ + __13, \ + __14, \ + __15, \ + __16, \ + __nargs, args...) __nargs + +#define NUM_VA_ARGS(args...) NUM_VA_ARGS_IMPL(__dummy, ##args, \ + 16, \ + 15, \ + 14, \ + 13, \ + 12, \ + 11, \ + 10, \ + 9, \ + 8, \ + 7, \ + 6, \ + 5, \ + 4, \ + 3, \ + 2, \ + 1, \ + 0) + +#define CONCAT_INTERNAL(left, right) left##right +#define CONCAT(left, right) CONCAT_INTERNAL(left, right) + +#define EMPTY() + +/* + * Takes the name of a function style macro such as FOO() and prevents it from + * being evaluated on the current pass. + * + * This is useful when you need to write a "recursive" macro since a macro name + * becomes painted after it is pasted. If a different macro is pasted, this + * different macro won't be pasted; if we then defer the evaluation of the this + * "indirection macro", we can prevent the original definition from getting + * painted. + * + * Example: + * #define EXAMPLE EXPAND(FOO()) // FOO() is evaluated on 1st pass. + * #define EXAMPLE EXPAND(DEFER(FOO)()) // FOO() is evaluated on the second + * // pass. + */ +#define DEFER(macro_id) macro_id EMPTY() + +/* + * Takes the name of a function style macro such as FOO() and prevents it from + * being evaluated on the current or following pass. + * + * This is useful when you need to DEFER inside an operation which causes an + * extra pass, like IF. + * + * Example: + * #define EXAMPLE EXPAND(FOO()) // FOO() is evaluated on 1st pass. + * #define EXAMPLE EXPAND(DEFER(FOO)()) // FOO() is evaluated on the second + * // pass. + * #define EXAMPLE EXPAND(OBSTRUCT(FOO)()) // FOO() is evaluated on the third + * // pass. + */ +#define OBSTRUCT(macro_id) macro_id DEFER(EMPTY)() + +#define EXPAND_1(args...) args +#define EXPAND_2(args...) EXPAND_1(EXPAND_1(args)) +#define EXPAND_4(args...) EXPAND_2(EXPAND_2(args)) +#define EXPAND_8(args...) EXPAND_4(EXPAND_4(args)) +#define EXPAND_16(args...) EXPAND_8(EXPAND_8(args)) + +/* + * Causes multiple evaluation passes of a macro. + * + * CPP is implemented as a push down automaton. It consumes a stream of tokens + * and as it comes across macros, it either evaluates them and pastes the + * result, or if the macro is a function macro, it pushes the macro to a stack, + * it evaluates the input to the function macro, pops the state from the stack + * and continues. + * + * This macro serves the function of making the cursor return to the beginging + * of a macro that requires mulitple passes to evaluate. It is most useful when + * used with DEFER(...) and OBSTRUCT(...). + */ +#define EXPAND(args...) EXPAND_16(args) + +#define INC(id) INC_##id +#define INC_0 1 +#define INC_1 2 +#define INC_2 3 +#define INC_3 4 +#define INC_4 5 +#define INC_5 6 +#define INC_6 7 +#define INC_7 8 +#define INC_8 9 +#define INC_9 10 +#define INC_10 11 +#define INC_11 12 +#define INC_12 13 +#define INC_13 14 +#define INC_14 15 +#define INC_15 16 +#define INC_16 17 + +#define DEC(id) DEC_##id +#define DEC_1 0 +#define DEC_2 1 +#define DEC_3 2 +#define DEC_4 3 +#define DEC_5 4 +#define DEC_6 5 +#define DEC_7 6 +#define DEC_8 7 +#define DEC_9 8 +#define DEC_10 9 +#define DEC_11 10 +#define DEC_12 11 +#define DEC_13 12 +#define DEC_14 13 +#define DEC_15 14 +#define DEC_16 15 + +#define DROP_FIRST_ARG_INTERNAL(dropped, x, args...) x +#define DROP_FIRST_ARG(args...) DROP_FIRST_ARG_INTERNAL(args) + +#define EQUAL(left, right) EQUAL_##left##_##right +#define EQUAL_0_0 dropped, 1 +#define EQUAL_1_1 dropped, 1 +#define EQUAL_2_2 dropped, 1 +#define EQUAL_3_3 dropped, 1 +#define EQUAL_4_4 dropped, 1 +#define EQUAL_5_5 dropped, 1 +#define EQUAL_6_6 dropped, 1 +#define EQUAL_7_7 dropped, 1 +#define EQUAL_8_8 dropped, 1 +#define EQUAL_9_9 dropped, 1 +#define EQUAL_10_10 dropped, 1 +#define EQUAL_11_11 dropped, 1 +#define EQUAL_12_12 dropped, 1 +#define EQUAL_13_13 dropped, 1 +#define EQUAL_14_14 dropped, 1 +#define EQUAL_15_15 dropped, 1 +#define EQUAL_16_16 dropped, 1 + +#define IS_EQUAL(left, right) DROP_FIRST_ARG(EQUAL(left, right), 0) + +#define NOT_INTERNAL(condition) NOT_##condition +#define NOT(condition) NOT_INTERNAL(condition) +#define NOT_0 1 +#define NOT_1 0 + +#define IS_NOT_EQUAL(left, right) NOT(IS_EQUAL(left, right)) + +#define EMPTY_IMPL(tokens) CONCAT(EMPTY_, tokens) +#define IS_EMPTY(tokens) + +#define OR_INTERNAL(left, right) OR_##left##_##right +#define OR(left, right) OR_INTERNAL(left, right) +#define OR_0_0 0 +#define OR_0_1 1 +#define OR_1_0 1 +#define OR_1_1 1 + +#define IF(condition) CONCAT(IF_, condition) +#define IF_0(body) +#define IF_1(body) body + +#define COMMA() , + +#define APPLY_TOKENS_INTERNAL(tokens, yield_token, seen_token) \ + IF(yield_token)(IF(seen_token)(COMMA()) tokens) +#define APPLY_TOKENS(tokens, yield_token, seen_token) \ + APPLY_TOKENS_INTERNAL(tokens, yield_token, seen_token) + +/* + * Provides the indirection to keep the PARAM_LIST_RECURSE_INTERNAL from getting + * pasted, only useful if used with DEFER(...) or OBSTRUCT(...). + */ +#define PARAM_LIST_RECURSE_INDIRECT() PARAM_LIST_RECURSE_INTERNAL + +/* + * Given a starting index, a number of args, a MACRO to apply, and a list of + * types (with at least one element) this will call MACRO with the first type in + * the list and index; it will then call itself again on all remaining types, if + * any, while incrementing index, and decrementing nargs. + * + * Assumes nargs is the number of types in the list. + */ +#define PARAM_LIST_RECURSE_INTERNAL(index, \ + nargs, \ + MACRO, \ + FILTER, \ + context, \ + seen_token, \ + type, \ + args...) \ + APPLY_TOKENS(MACRO(context, type, index), \ + FILTER(context, type, index), \ + seen_token) \ + IF(IS_NOT_EQUAL(nargs, 1)) \ + (OBSTRUCT(PARAM_LIST_RECURSE_INDIRECT)() \ + (INC(index), DEC(nargs), \ + MACRO, FILTER, context, \ + OR(seen_token, FILTER(context, type, index)), \ + args)) + +#define PARAM_LIST_RECURSE(index, nargs, MACRO, FILTER, context, args...) \ + IF(IS_NOT_EQUAL(nargs, 0)) \ + (OBSTRUCT(PARAM_LIST_RECURSE_INTERNAL)(index, \ + nargs, \ + MACRO, \ + FILTER, \ + context, \ + 0, \ + args)) + +#define FILTER_NONE(context, type, index) 1 + +#define FILTER_INDEX_INTERNAL(index_to_drop, type, index) \ + IS_NOT_EQUAL(index, index_to_drop) +#define FILTER_INDEX(index_to_drop, type, index) \ + FILTER_INDEX_INTERNAL(index_to_drop, type, index) + +/* + * Applies a MACRO which takes a type and the index of the type and outputs a + * sequence of tokens to a list of types. + */ +#define FOR_EACH_PARAM(MACRO, FILTER, context, args...) \ + EXPAND(PARAM_LIST_RECURSE(0,\ + NUM_VA_ARGS(args),\ + MACRO,\ + FILTER,\ + context,\ + args)) + +#define PRODUCE_TYPE_AND_ARG(context, type, index) type arg##index +#define PARAM_LIST_FROM_TYPES(args...) \ + FOR_EACH_PARAM(PRODUCE_TYPE_AND_ARG, \ + FILTER_NONE, \ + not_used, \ + args) + +#define PRODUCE_TYPE_NAME(context, type, index) #type +#define TYPE_NAMES_FROM_TYPES(handle_index, args...) \ + FOR_EACH_PARAM(PRODUCE_TYPE_NAME, \ + FILTER_INDEX, \ + handle_index, \ + args) + +#define PRODUCE_PTR_TO_ARG(context, type, index) &arg##index +#define PTR_TO_ARG_FROM_TYPES(handle_index, args...) \ + FOR_EACH_PARAM(PRODUCE_PTR_TO_ARG, \ + FILTER_INDEX, \ + handle_index, \ + args) + +#define PRODUCE_MATCHER_AND_ARG(ctrl_index, type, index) \ + IF(IS_EQUAL(index, ctrl_index))(struct mock *arg##ctrl_index) \ + IF(IS_NOT_EQUAL(index, ctrl_index))( \ + struct mock_param_matcher *arg##index) +#define MATCHER_PARAM_LIST_FROM_TYPES(ctrl_index, args...) \ + FOR_EACH_PARAM(PRODUCE_MATCHER_AND_ARG, \ + FILTER_NONE, \ + ctrl_index, \ + args) + +#define PRODUCE_ARG(context, type, index) arg##index +#define ARG_NAMES_FROM_TYPES(ctrl_index, args...) \ + FOR_EACH_PARAM(PRODUCE_ARG, \ + FILTER_INDEX, \ + ctrl_index, \ + args) + +#define PRODUCE_ARRAY_ACCESSOR(context, type, index) *((type *) params[index]) +#define ARRAY_ACCESSORS_FROM_TYPES(args...) \ + FOR_EACH_PARAM(PRODUCE_ARRAY_ACCESSOR, \ + FILTER_NONE, \ + not_used, \ + args) + +#endif /* _KUNIT_PARAMS_H */ diff --git a/kunit/Makefile b/kunit/Makefile index 2f1c069e165cb..f72a02cb9f23d 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,3 +1,3 @@ obj-$(CONFIG_KUNIT) += test.o string-stream.o test-stream.o -obj-$(CONFIG_KUNIT_TEST) += test-test.o string-stream-test.o +obj-$(CONFIG_KUNIT_TEST) += test-test.o mock-macro-test.o string-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/mock-macro-test.c b/kunit/mock-macro-test.c new file mode 100644 index 0000000000000..c30b859ff2b14 --- /dev/null +++ b/kunit/mock-macro-test.c @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for parameter list parsing macros. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> +#include <kunit/params.h> + +#define TO_STR_INTERNAL(...) #__VA_ARGS__ +#define TO_STR(...) TO_STR_INTERNAL(__VA_ARGS__) + +static void mock_macro_is_equal(struct test *test) +{ + TEST_EXPECT_STREQ(test, "dropped, 1", TO_STR(EQUAL(1, 1))); + TEST_EXPECT_EQ(test, 1, DROP_FIRST_ARG(dropped, 1, 0)); + TEST_EXPECT_EQ(test, 0, DROP_FIRST_ARG(1, 0)); + TEST_EXPECT_EQ(test, 0, DROP_FIRST_ARG(EQUAL(1, 0), 0)); + TEST_EXPECT_EQ(test, 1, IS_EQUAL(1, 1)); + TEST_EXPECT_EQ(test, 0, IS_EQUAL(1, 0)); +} + +static void mock_macro_if(struct test *test) +{ + TEST_EXPECT_STREQ(test, "body", ""IF(1)("body")); + TEST_EXPECT_STREQ(test, "", ""IF(0)("body")); + TEST_EXPECT_STREQ(test, "body", ""IF(IS_EQUAL(1, 1))("body")); + TEST_EXPECT_STREQ(test, "", ""IF(IS_EQUAL(0, 1))("body")); +} + +static void mock_macro_apply_tokens(struct test *test) +{ + TEST_EXPECT_STREQ(test, "type", TO_STR(APPLY_TOKENS(type, 1, 0))); + TEST_EXPECT_STREQ(test, ", type", TO_STR(APPLY_TOKENS(type, 1, 1))); + TEST_EXPECT_STREQ(test, "", TO_STR(APPLY_TOKENS(type, 0, 1))); +} + +#define IDENTITY(context, type, index) type + +static void mock_macro_param_list_recurse(struct test *test) +{ + TEST_EXPECT_STREQ(test, "", TO_STR(PARAM_LIST_RECURSE(0, + 0, + IDENTITY, + FILTER_NONE, + not_used))); + TEST_EXPECT_STREQ(test, + "type", + TO_STR(EXPAND(PARAM_LIST_RECURSE(0, + 1, + IDENTITY, + FILTER_NONE, + not_used, + type)))); + TEST_EXPECT_STREQ(test, + "type0 , type1 , type2 , type3 , type4 , type5 , " + "type6 , type7 , type8 , type9 , type10 , type11 , " + "type12 , type13 , type14 , type15", + TO_STR(EXPAND(PARAM_LIST_RECURSE(0, 16, + IDENTITY, + FILTER_NONE, + not_used, + type0, type1, type2, + type3, type4, type5, + type6, type7, type8, + type9, type10, + type11, type12, + type13, type14, + type15)))); +} + +static void mock_macro_for_each_param(struct test *test) +{ + TEST_EXPECT_STREQ(test, + "type0 , type1", + TO_STR(FOR_EACH_PARAM(IDENTITY, + FILTER_NONE, + not_used, + type0, + type1))); + TEST_EXPECT_STREQ(test, + "type1", + TO_STR(FOR_EACH_PARAM(IDENTITY, + FILTER_INDEX, + 0, + type0, + type1))); +} + +static void mock_macro_param_list_from_types_basic(struct test *test) +{ + TEST_EXPECT_STREQ(test, "", TO_STR(PARAM_LIST_FROM_TYPES())); + TEST_EXPECT_STREQ(test, "int arg0", TO_STR(PARAM_LIST_FROM_TYPES(int))); + TEST_EXPECT_STREQ(test, "struct test_struct * arg0 , int arg1", + TO_STR(PARAM_LIST_FROM_TYPES(struct test_struct *, + int))); + TEST_EXPECT_STREQ(test, + "type0 arg0 , type1 arg1 , type2 arg2 , type3 arg3 , " + "type4 arg4 , type5 arg5 , type6 arg6 , type7 arg7 , " + "type8 arg8 , type9 arg9 , type10 arg10 , " + "type11 arg11 , type12 arg12 , type13 arg13 , " + "type14 arg14 , type15 arg15", + TO_STR(PARAM_LIST_FROM_TYPES(type0, type1, type2, + type3, type4, type5, + type6, type7, type8, + type9, type10, type11, + type12, type13, type14, + type15))); +} + +static void mock_macro_arg_names_from_types(struct test *test) +{ + TEST_EXPECT_STREQ(test, "", TO_STR(ARG_NAMES_FROM_TYPES(0))); + TEST_EXPECT_STREQ(test, "", TO_STR(ARG_NAMES_FROM_TYPES(0, int))); + TEST_EXPECT_STREQ(test, + "arg1", + TO_STR(ARG_NAMES_FROM_TYPES(0, + struct test_struct *, + int))); + TEST_EXPECT_STREQ(test, + "arg0 , arg1 , arg3 , arg4 , arg5 , arg6 , arg7 , " + "arg8 , arg9 , arg10 , arg11 , arg12 , arg13 , " + "arg14 , arg15", + TO_STR(ARG_NAMES_FROM_TYPES(2, type0, type1, type2, + type3, type4, type5, + type6, type7, type8, + type9, type10, type11, + type12, type13, type14, + type15))); +} + +static struct test_case mock_macro_test_cases[] = { + TEST_CASE(mock_macro_is_equal), + TEST_CASE(mock_macro_if), + TEST_CASE(mock_macro_apply_tokens), + TEST_CASE(mock_macro_param_list_recurse), + TEST_CASE(mock_macro_for_each_param), + TEST_CASE(mock_macro_param_list_from_types_basic), + TEST_CASE(mock_macro_arg_names_from_types), + {}, +}; + +static struct test_module mock_macro_test_module = { + .name = "mock-macro-test", + .test_cases = mock_macro_test_cases, +}; +module_test(mock_macro_test_module);
Adds the core internal mechanisms that mocks are implemented with; in particular, this adds the mechanisms by which expectation on mocks are validated and by which actions may be supplied and then executed when mocks are called.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 125 +++++++++++++++ kunit/Makefile | 5 +- kunit/mock.c | 359 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 487 insertions(+), 2 deletions(-) create mode 100644 include/kunit/mock.h create mode 100644 kunit/mock.c
diff --git a/include/kunit/mock.h b/include/kunit/mock.h new file mode 100644 index 0000000000000..1a35c5702cb15 --- /dev/null +++ b/include/kunit/mock.h @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Mocking API for KUnit. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _KUNIT_MOCK_H +#define _KUNIT_MOCK_H + +#include <linux/types.h> +#include <linux/tracepoint.h> /* For PARAMS(...) */ +#include <kunit/test.h> +#include <kunit/test-stream.h> +#include <kunit/params.h> + +/** + * struct mock_param_matcher - represents a matcher used in a *call expectation* + * @match: the function that performs the matching + * + * The matching function takes a couple of parameters: + * + * - ``this``: refers to the parent struct + * - ``stream``: a &test_stream to which a detailed message should be added as + * to why the parameter matches or not + * - ``param``: a pointer to the parameter to check for a match + * + * The matching function should return whether or not the passed parameter + * matches. + */ +struct mock_param_matcher { + bool (*match)(struct mock_param_matcher *this, + struct test_stream *stream, + const void *param); +}; + +#define MOCK_MAX_PARAMS 255 + +struct mock_matcher { + struct mock_param_matcher *matchers[MOCK_MAX_PARAMS]; + int num; +}; + +/** + * struct mock_action - Represents an action that a mock performs when + * expectation is matched + * @do_action: the action to perform + * + * The action function is given some parameters: + * + * - ``this``: refers to the parent struct + * - ``params``: an array of pointers to the params passed into the mocked + * method or function. **The class argument is excluded for a mocked class + * method.** + * - ``len``: size of ``params`` + * + * The action function returns a pointer to the value that the mocked method + * or function should be returning. + */ +struct mock_action { + void *(*do_action)(struct mock_action *this, + const void **params, + int len); +}; + +/** + * struct mock_expectation - represents a *call expectation* on a function. + * @action: A &struct mock_action to perform when the function is called. + * @max_calls_expected: maximum number of times an expectation may be called. + * @min_calls_expected: minimum number of times an expectation may be called. + * @retire_on_saturation: no longer match once ``max_calls_expected`` is + * reached. + * + * Represents a *call expectation* on a function created with EXPECT_CALL(). + */ +struct mock_expectation { + struct mock_action *action; + int max_calls_expected; + int min_calls_expected; + bool retire_on_saturation; + /* private: internal use only. */ + const char *expectation_name; + struct list_head node; + struct mock_matcher *matcher; + int times_called; +}; + +struct mock_method { + struct list_head node; + const char *method_name; + const void *method_ptr; + struct mock_action *default_action; + struct list_head expectations; +}; + +struct mock { + struct test_post_condition parent; + struct test *test; + struct list_head methods; + /* TODO(brendanhiggins@google.com): add locking to do_expect. */ + const void *(*do_expect)(struct mock *mock, + const char *method_name, + const void *method_ptr, + const char * const *param_types, + const void **params, + int len); +}; + +void mock_init_ctrl(struct test *test, struct mock *mock); + +void mock_validate_expectations(struct mock *mock); + +int mock_set_default_action(struct mock *mock, + const char *method_name, + const void *method_ptr, + struct mock_action *action); + +struct mock_expectation *mock_add_matcher(struct mock *mock, + const char *method_name, + const void *method_ptr, + struct mock_param_matcher *matchers[], + int len); + +#endif /* _KUNIT_MOCK_H */ diff --git a/kunit/Makefile b/kunit/Makefile index f72a02cb9f23d..ad58110de695c 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,3 +1,4 @@ -obj-$(CONFIG_KUNIT) += test.o string-stream.o test-stream.o -obj-$(CONFIG_KUNIT_TEST) += test-test.o mock-macro-test.o string-stream-test.o +obj-$(CONFIG_KUNIT) += test.o mock.o string-stream.o test-stream.o +obj-$(CONFIG_KUNIT_TEST) += \ + test-test.o mock-macro-test.o string-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/mock.c b/kunit/mock.c new file mode 100644 index 0000000000000..424c612de553b --- /dev/null +++ b/kunit/mock.c @@ -0,0 +1,359 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Mocking API for KUnit. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/mock.h> + +static bool mock_match_params(struct mock_matcher *matcher, + struct test_stream *stream, + const void **params, + int len) +{ + struct mock_param_matcher *param_matcher; + bool ret = true, tmp; + int i; + + BUG_ON(matcher->num != len); + + for (i = 0; i < matcher->num; i++) { + param_matcher = matcher->matchers[i]; + stream->add(stream, "\t"); + tmp = param_matcher->match(param_matcher, stream, params[i]); + ret = ret && tmp; + stream->add(stream, "\n"); + } + + return ret; +} + +static const void *mock_do_expect(struct mock *mock, + const char *method_name, + const void *method_ptr, + const char * const *type_names, + const void **params, + int len); + +void mock_validate_expectations(struct mock *mock) +{ + struct mock_expectation *expectation, *expectation_safe; + struct mock_method *method; + struct test_stream *stream; + int times_called; + + stream = test_new_stream(mock->test); + list_for_each_entry(method, &mock->methods, node) { + list_for_each_entry_safe(expectation, expectation_safe, + &method->expectations, node) { + times_called = expectation->times_called; + if (!(expectation->min_calls_expected <= times_called && + times_called <= expectation->max_calls_expected) + ) { + stream->add(stream, + "Expectation was not called the specified number of times:\n\t"); + stream->add(stream, + "Function: %s, min calls: %d, max calls: %d, actual calls: %d", + method->method_name, + expectation->min_calls_expected, + expectation->max_calls_expected, + times_called); + mock->test->fail(mock->test, stream); + } + list_del(&expectation->node); + } + } +} + +static void mock_validate_wrapper(struct test_post_condition *condition) +{ + struct mock *mock = container_of(condition, struct mock, parent); + + mock_validate_expectations(mock); +} + +void mock_init_ctrl(struct test *test, struct mock *mock) +{ + mock->test = test; + INIT_LIST_HEAD(&mock->methods); + mock->do_expect = mock_do_expect; + mock->parent.validate = mock_validate_wrapper; + list_add_tail(&mock->parent.node, &test->post_conditions); +} + +static struct mock_method *mock_lookup_method(struct mock *mock, + const void *method_ptr) +{ + struct mock_method *ret; + + list_for_each_entry(ret, &mock->methods, node) { + if (ret->method_ptr == method_ptr) + return ret; + } + + return NULL; +} + +static struct mock_method *mock_add_method(struct mock *mock, + const char *method_name, + const void *method_ptr) +{ + struct mock_method *method; + + method = test_kzalloc(mock->test, sizeof(*method), GFP_KERNEL); + if (!method) + return NULL; + + INIT_LIST_HEAD(&method->expectations); + method->method_name = method_name; + method->method_ptr = method_ptr; + list_add_tail(&method->node, &mock->methods); + + return method; +} + +static int mock_add_expectation(struct mock *mock, + const char *method_name, + const void *method_ptr, + struct mock_expectation *expectation) +{ + struct mock_method *method; + + method = mock_lookup_method(mock, method_ptr); + if (!method) { + method = mock_add_method(mock, method_name, method_ptr); + if (!method) + return -ENOMEM; + } + + list_add_tail(&expectation->node, &method->expectations); + + return 0; +} + +struct mock_expectation *mock_add_matcher(struct mock *mock, + const char *method_name, + const void *method_ptr, + struct mock_param_matcher *matchers[], + int len) +{ + struct mock_expectation *expectation; + struct mock_matcher *matcher; + int ret; + + expectation = test_kzalloc(mock->test, + sizeof(*expectation), + GFP_KERNEL); + if (!expectation) + return NULL; + + matcher = test_kmalloc(mock->test, sizeof(*matcher), GFP_KERNEL); + if (!matcher) + return NULL; + + memcpy(&matcher->matchers, matchers, sizeof(*matchers) * len); + matcher->num = len; + + expectation->matcher = matcher; + expectation->max_calls_expected = 1; + expectation->min_calls_expected = 1; + + ret = mock_add_expectation(mock, method_name, method_ptr, expectation); + if (ret < 0) + return NULL; + + return expectation; +} + +int mock_set_default_action(struct mock *mock, + const char *method_name, + const void *method_ptr, + struct mock_action *action) +{ + struct mock_method *method; + + method = mock_lookup_method(mock, method_ptr); + if (!method) { + method = mock_add_method(mock, method_name, method_ptr); + if (!method) + return -ENOMEM; + } + + method->default_action = action; + + return 0; +} + +static void mock_format_param(struct test_stream *stream, + const char *type_name, + const void *param) +{ + /* + * Cannot find formatter, so just print the pointer of the + * symbol. + */ + stream->add(stream, "<%pS>", param); +} + +static void mock_add_method_declaration_to_stream( + struct test_stream *stream, + const char *function_name, + const char * const *type_names, + const void **params, + int len) +{ + int i; + + stream->add(stream, "%s(", function_name); + for (i = 0; i < len; i++) { + mock_format_param(stream, type_names[i], params[i]); + if (i < len - 1) + stream->add(stream, ", "); + } + stream->add(stream, ")\n"); +} + +static struct test_stream *mock_initialize_failure_message( + struct test *test, + const char *function_name, + const char * const *type_names, + const void **params, + int len) +{ + struct test_stream *stream; + + stream = test_new_stream(test); + if (!stream) + return NULL; + + stream->add(stream, "EXPECTATION FAILED: no expectation for call: "); + mock_add_method_declaration_to_stream(stream, + function_name, + type_names, + params, + len); + return stream; +} + +static bool mock_is_expectation_retired(struct mock_expectation *expectation) +{ + return expectation->retire_on_saturation && + expectation->times_called == + expectation->max_calls_expected; +} + +static void mock_add_method_expectation_error(struct test *test, + struct test_stream *stream, + char *message, + struct mock *mock, + struct mock_method *method, + const char * const *type_names, + const void **params, + int len) +{ + stream->clear(stream); + stream->set_level(stream, KERN_WARNING); + stream->add(stream, message); + mock_add_method_declaration_to_stream(stream, + method->method_name, type_names, params, len); +} + +static struct mock_expectation *mock_apply_expectations( + struct mock *mock, + struct mock_method *method, + const char * const *type_names, + const void **params, + int len) +{ + struct test *test = mock->test; + struct mock_expectation *ret; + struct test_stream *attempted_matching_stream; + bool expectations_all_saturated = true; + + struct test_stream *stream = test_new_stream(test); + + if (list_empty(&method->expectations)) { + mock_add_method_expectation_error(test, stream, + "Method was called with no expectations declared: ", + mock, method, type_names, params, len); + stream->commit(stream); + return NULL; + } + + attempted_matching_stream = mock_initialize_failure_message( + test, + method->method_name, + type_names, + params, + len); + + list_for_each_entry(ret, &method->expectations, node) { + if (mock_is_expectation_retired(ret)) + continue; + expectations_all_saturated = false; + + attempted_matching_stream->add(attempted_matching_stream, + "Tried expectation: %s, but\n", ret->expectation_name); + if (mock_match_params(ret->matcher, + attempted_matching_stream, params, len)) { + /* + * Matcher was found; we won't print, so clean up the + * log. + */ + attempted_matching_stream->clear( + attempted_matching_stream); + return ret; + } + } + + if (expectations_all_saturated) { + mock_add_method_expectation_error(test, stream, + "Method was called with fully saturated expectations: ", + mock, method, type_names, params, len); + } else { + mock_add_method_expectation_error(test, stream, + "Method called that did not match any expectations: ", + mock, method, type_names, params, len); + stream->append(stream, attempted_matching_stream); + } + test->fail(test, stream); + attempted_matching_stream->clear(attempted_matching_stream); + return NULL; +} + +static const void *mock_do_expect(struct mock *mock, + const char *method_name, + const void *method_ptr, + const char * const *param_types, + const void **params, + int len) +{ + struct mock_expectation *expectation; + struct mock_method *method; + struct mock_action *action; + + method = mock_lookup_method(mock, method_ptr); + if (!method) + return NULL; + + expectation = mock_apply_expectations(mock, + method, + param_types, + params, + len); + if (!expectation) { + action = method->default_action; + } else { + expectation->times_called++; + if (expectation->action) + action = expectation->action; + else + action = method->default_action; + } + if (!action) + return NULL; + + return action->do_action(action, params, len); +}
Added basic matchers and actions needed for any kind of mocking to be useful; these matchers and actions are how expectations for mocks are described: what calls the mocks are expected to receive, and what the mock should do under those circumstances.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 222 +++++++++++++++++++++++++++++++++++ kunit/Makefile | 3 +- kunit/common-mocks.c | 272 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 496 insertions(+), 1 deletion(-) create mode 100644 kunit/common-mocks.c
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 1a35c5702cb15..62e8afcaeab55 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -122,4 +122,226 @@ struct mock_expectation *mock_add_matcher(struct mock *mock, struct mock_param_matcher *matchers[], int len);
+#define CONVERT_TO_ACTUAL_TYPE(type, ptr) (*((type *) ptr)) + +/** + * DOC: Built In Matchers + * + * These are the matchers that can be used when matching arguments in + * :c:func:`EXPECT_CALL` (more can be defined manually). + * + * For example, there's a matcher that matches any arguments: + * + * .. code-block:: c + * + * struct mock_param_matcher *any(struct test *test); + * + * There are matchers for integers based on the binary condition: + * + * * eq: equals to + * * ne: not equal to + * * lt: less than + * * le: less than or equal to + * * gt: greater than + * * ge: greater than or equal to + * + * .. code-block:: c + * + * struct mock_param_matcher *test_int_eq(struct test *test, int expected); + * struct mock_param_matcher *test_int_ne(struct test *test, int expected); + * struct mock_param_matcher *test_int_lt(struct test *test, int expected); + * struct mock_param_matcher *test_int_le(struct test *test, int expected); + * struct mock_param_matcher *test_int_gt(struct test *test, int expected); + * struct mock_param_matcher *test_int_ge(struct test *test, int expected); + * + * For a detailed list, please see + * ``include/linux/mock.h``. + */ + +/* Matches any argument */ +struct mock_param_matcher *test_any(struct test *test); + +/* + * Matches different types of integers, the argument is compared to the + * `expected` field, based on the comparison defined. + */ +struct mock_param_matcher *test_u8_eq(struct test *test, u8 expected); +struct mock_param_matcher *test_u8_ne(struct test *test, u8 expected); +struct mock_param_matcher *test_u8_le(struct test *test, u8 expected); +struct mock_param_matcher *test_u8_lt(struct test *test, u8 expected); +struct mock_param_matcher *test_u8_ge(struct test *test, u8 expected); +struct mock_param_matcher *test_u8_gt(struct test *test, u8 expected); + +struct mock_param_matcher *test_u16_eq(struct test *test, u16 expected); +struct mock_param_matcher *test_u16_ne(struct test *test, u16 expected); +struct mock_param_matcher *test_u16_le(struct test *test, u16 expected); +struct mock_param_matcher *test_u16_lt(struct test *test, u16 expected); +struct mock_param_matcher *test_u16_ge(struct test *test, u16 expected); +struct mock_param_matcher *test_u16_gt(struct test *test, u16 expected); + +struct mock_param_matcher *test_u32_eq(struct test *test, u32 expected); +struct mock_param_matcher *test_u32_ne(struct test *test, u32 expected); +struct mock_param_matcher *test_u32_le(struct test *test, u32 expected); +struct mock_param_matcher *test_u32_lt(struct test *test, u32 expected); +struct mock_param_matcher *test_u32_ge(struct test *test, u32 expected); +struct mock_param_matcher *test_u32_gt(struct test *test, u32 expected); + +struct mock_param_matcher *test_u64_eq(struct test *test, u64 expected); +struct mock_param_matcher *test_u64_ne(struct test *test, u64 expected); +struct mock_param_matcher *test_u64_le(struct test *test, u64 expected); +struct mock_param_matcher *test_u64_lt(struct test *test, u64 expected); +struct mock_param_matcher *test_u64_ge(struct test *test, u64 expected); +struct mock_param_matcher *test_u64_gt(struct test *test, u64 expected); + +struct mock_param_matcher *test_char_eq(struct test *test, char expected); +struct mock_param_matcher *test_char_ne(struct test *test, char expected); +struct mock_param_matcher *test_char_le(struct test *test, char expected); +struct mock_param_matcher *test_char_lt(struct test *test, char expected); +struct mock_param_matcher *test_char_ge(struct test *test, char expected); +struct mock_param_matcher *test_char_gt(struct test *test, char expected); + +struct mock_param_matcher *test_uchar_eq(struct test *test, + unsigned char expected); +struct mock_param_matcher *test_uchar_ne(struct test *test, + unsigned char expected); +struct mock_param_matcher *test_uchar_le(struct test *test, + unsigned char expected); +struct mock_param_matcher *test_uchar_lt(struct test *test, + unsigned char expected); +struct mock_param_matcher *test_uchar_ge(struct test *test, + unsigned char expected); +struct mock_param_matcher *test_uchar_gt(struct test *test, + unsigned char expected); + +struct mock_param_matcher *test_schar_eq(struct test *test, + signed char expected); +struct mock_param_matcher *test_schar_ne(struct test *test, + signed char expected); +struct mock_param_matcher *test_schar_le(struct test *test, + signed char expected); +struct mock_param_matcher *test_schar_lt(struct test *test, + signed char expected); +struct mock_param_matcher *test_schar_ge(struct test *test, + signed char expected); +struct mock_param_matcher *test_schar_gt(struct test *test, + signed char expected); + +struct mock_param_matcher *test_short_eq(struct test *test, short expected); +struct mock_param_matcher *test_short_ne(struct test *test, short expected); +struct mock_param_matcher *test_short_le(struct test *test, short expected); +struct mock_param_matcher *test_short_lt(struct test *test, short expected); +struct mock_param_matcher *test_short_ge(struct test *test, short expected); +struct mock_param_matcher *test_short_gt(struct test *test, short expected); + +struct mock_param_matcher *test_ushort_eq(struct test *test, + unsigned short expected); +struct mock_param_matcher *test_ushort_ne(struct test *test, + unsigned short expected); +struct mock_param_matcher *test_ushort_le(struct test *test, + unsigned short expected); +struct mock_param_matcher *test_ushort_lt(struct test *test, + unsigned short expected); +struct mock_param_matcher *test_ushort_ge(struct test *test, + unsigned short expected); +struct mock_param_matcher *test_ushort_gt(struct test *test, + unsigned short expected); + +struct mock_param_matcher *test_int_eq(struct test *test, int expected); +struct mock_param_matcher *test_int_ne(struct test *test, int expected); +struct mock_param_matcher *test_int_lt(struct test *test, int expected); +struct mock_param_matcher *test_int_le(struct test *test, int expected); +struct mock_param_matcher *test_int_gt(struct test *test, int expected); +struct mock_param_matcher *test_int_ge(struct test *test, int expected); + +struct mock_param_matcher *test_uint_eq(struct test *test, + unsigned int expected); +struct mock_param_matcher *test_uint_ne(struct test *test, + unsigned int expected); +struct mock_param_matcher *test_uint_lt(struct test *test, + unsigned int expected); +struct mock_param_matcher *test_uint_le(struct test *test, + unsigned int expected); +struct mock_param_matcher *test_uint_gt(struct test *test, + unsigned int expected); +struct mock_param_matcher *test_uint_ge(struct test *test, + unsigned int expected); + +struct mock_param_matcher *test_long_eq(struct test *test, long expected); +struct mock_param_matcher *test_long_ne(struct test *test, long expected); +struct mock_param_matcher *test_long_le(struct test *test, long expected); +struct mock_param_matcher *test_long_lt(struct test *test, long expected); +struct mock_param_matcher *test_long_ge(struct test *test, long expected); +struct mock_param_matcher *test_long_gt(struct test *test, long expected); + +struct mock_param_matcher *test_ulong_eq(struct test *test, + unsigned long expected); +struct mock_param_matcher *test_ulong_ne(struct test *test, + unsigned long expected); +struct mock_param_matcher *test_ulong_le(struct test *test, + unsigned long expected); +struct mock_param_matcher *test_ulong_lt(struct test *test, + unsigned long expected); +struct mock_param_matcher *test_ulong_ge(struct test *test, + unsigned long expected); +struct mock_param_matcher *test_ulong_gt(struct test *test, + unsigned long expected); + +struct mock_param_matcher *test_longlong_eq(struct test *test, + long long expected); +struct mock_param_matcher *test_longlong_ne(struct test *test, + long long expected); +struct mock_param_matcher *test_longlong_le(struct test *test, + long long expected); +struct mock_param_matcher *test_longlong_lt(struct test *test, + long long expected); +struct mock_param_matcher *test_longlong_ge(struct test *test, + long long expected); +struct mock_param_matcher *test_longlong_gt(struct test *test, + long long expected); + +struct mock_param_matcher *test_ulonglong_eq(struct test *test, + unsigned long long expected); +struct mock_param_matcher *test_ulonglong_ne(struct test *test, + unsigned long long expected); +struct mock_param_matcher *test_ulonglong_le(struct test *test, + unsigned long long expected); +struct mock_param_matcher *test_ulonglong_lt(struct test *test, + unsigned long long expected); +struct mock_param_matcher *test_ulonglong_ge(struct test *test, + unsigned long long expected); +struct mock_param_matcher *test_ulonglong_gt(struct test *test, + unsigned long long expected); + +/* Matches pointers. */ +struct mock_param_matcher *test_ptr_eq(struct test *test, void *expected); +struct mock_param_matcher *test_ptr_ne(struct test *test, void *expected); +struct mock_param_matcher *test_ptr_lt(struct test *test, void *expected); +struct mock_param_matcher *test_ptr_le(struct test *test, void *expected); +struct mock_param_matcher *test_ptr_gt(struct test *test, void *expected); +struct mock_param_matcher *test_ptr_ge(struct test *test, void *expected); + +/* Matches memory sections and strings. */ +struct mock_param_matcher *test_memeq(struct test *test, + const void *buf, + size_t size); +struct mock_param_matcher *test_streq(struct test *test, const char *str); + +struct mock_action *test_u8_return(struct test *test, u8 ret); +struct mock_action *test_u16_return(struct test *test, u16 ret); +struct mock_action *test_u32_return(struct test *test, u32 ret); +struct mock_action *test_u64_return(struct test *test, u64 ret); +struct mock_action *test_char_return(struct test *test, char ret); +struct mock_action *test_uchar_return(struct test *test, unsigned char ret); +struct mock_action *test_schar_return(struct test *test, signed char ret); +struct mock_action *test_short_return(struct test *test, short ret); +struct mock_action *test_ushort_return(struct test *test, unsigned short ret); +struct mock_action *test_int_return(struct test *test, int ret); +struct mock_action *test_uint_return(struct test *test, unsigned int ret); +struct mock_action *test_long_return(struct test *test, long ret); +struct mock_action *test_ulong_return(struct test *test, unsigned long ret); +struct mock_action *test_longlong_return(struct test *test, long long ret); +struct mock_action *test_ulonglong_return(struct test *test, + unsigned long long ret); +struct mock_action *test_ptr_return(struct test *test, void *ret); + #endif /* _KUNIT_MOCK_H */ diff --git a/kunit/Makefile b/kunit/Makefile index ad58110de695c..52a1da46cbd21 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,4 +1,5 @@ -obj-$(CONFIG_KUNIT) += test.o mock.o string-stream.o test-stream.o +obj-$(CONFIG_KUNIT) += test.o mock.o common-mocks.o string-stream.o \ + test-stream.o obj-$(CONFIG_KUNIT_TEST) += \ test-test.o mock-macro-test.o string-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/common-mocks.c b/kunit/common-mocks.c new file mode 100644 index 0000000000000..ecac9c1c29c0e --- /dev/null +++ b/kunit/common-mocks.c @@ -0,0 +1,272 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Common KUnit mock call arg matchers and formatters. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/kernel.h> +#include <kunit/mock.h> + +static bool match_any(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *actual) +{ + stream->add(stream, "don't care"); + return true; +} + +static struct mock_param_matcher any_matcher = { + .match = match_any, +}; + +struct mock_param_matcher *test_any(struct test *test) +{ + return &any_matcher; +} + +#define DEFINE_MATCHER_STRUCT(type_name, type) \ + struct mock_##type_name##_matcher { \ + struct mock_param_matcher matcher; \ + type expected; \ + }; + +#define DEFINE_TO_MATCHER_STRUCT(type_name) \ + struct mock_##type_name##_matcher * \ + to_mock_##type_name##_matcher( \ + struct mock_param_matcher *matcher) \ + { \ + return container_of(matcher, \ + struct mock_##type_name##_matcher, \ + matcher); \ + } + +#define DEFINE_MATCH_FUNC(type_name, type, op_name, op) \ + bool match_##type_name##_##op_name( \ + struct mock_param_matcher *pmatcher, \ + struct test_stream *stream, \ + const void *pactual) \ + { \ + struct mock_##type_name##_matcher *matcher = \ + to_mock_##type_name##_matcher(pmatcher); \ + type actual = *((type *) pactual); \ + bool matches = actual op matcher->expected; \ + \ + if (matches) \ + stream->add(stream, \ + "%d "#op" %d", \ + actual, \ + matcher->expected); \ + else \ + stream->add(stream, \ + "%d not "#op" %d", \ + actual, \ + matcher->expected); \ + \ + return matches; \ + } + +#define DEFINE_MATCH_FACTORY(type_name, type, op_name) \ + struct mock_param_matcher *test_##type_name##_##op_name( \ + struct test *test, type expected) \ + { \ + struct mock_##type_name##_matcher *matcher; \ + \ + matcher = test_kmalloc(test, \ + sizeof(*matcher), \ + GFP_KERNEL); \ + if (!matcher) \ + return NULL; \ + \ + matcher->matcher.match = match_##type_name##_##op_name;\ + matcher->expected = expected; \ + return &matcher->matcher; \ + } + +#define DEFINE_MATCHER_WITH_TYPENAME(type_name, type) \ + DEFINE_MATCHER_STRUCT(type_name, type) \ + DEFINE_TO_MATCHER_STRUCT(type_name) \ + DEFINE_MATCH_FUNC(type_name, type, eq, ==) \ + DEFINE_MATCH_FACTORY(type_name, type, eq) \ + DEFINE_MATCH_FUNC(type_name, type, ne, !=) \ + DEFINE_MATCH_FACTORY(type_name, type, ne) \ + DEFINE_MATCH_FUNC(type_name, type, le, <=) \ + DEFINE_MATCH_FACTORY(type_name, type, le) \ + DEFINE_MATCH_FUNC(type_name, type, lt, <) \ + DEFINE_MATCH_FACTORY(type_name, type, lt) \ + DEFINE_MATCH_FUNC(type_name, type, ge, >=) \ + DEFINE_MATCH_FACTORY(type_name, type, ge) \ + DEFINE_MATCH_FUNC(type_name, type, gt, >) \ + DEFINE_MATCH_FACTORY(type_name, type, gt) + +#define DEFINE_MATCHER(type) DEFINE_MATCHER_WITH_TYPENAME(type, type) + +DEFINE_MATCHER(u8); +DEFINE_MATCHER(u16); +DEFINE_MATCHER(u32); +DEFINE_MATCHER(u64); +DEFINE_MATCHER(char); +DEFINE_MATCHER_WITH_TYPENAME(uchar, unsigned char); +DEFINE_MATCHER_WITH_TYPENAME(schar, signed char); +DEFINE_MATCHER(short); +DEFINE_MATCHER_WITH_TYPENAME(ushort, unsigned short); +DEFINE_MATCHER(int); +DEFINE_MATCHER_WITH_TYPENAME(uint, unsigned int); +DEFINE_MATCHER(long); +DEFINE_MATCHER_WITH_TYPENAME(ulong, unsigned long); +DEFINE_MATCHER_WITH_TYPENAME(longlong, long long); +DEFINE_MATCHER_WITH_TYPENAME(ulonglong, unsigned long long); + +DEFINE_MATCHER_WITH_TYPENAME(ptr, void *); + +struct mock_memeq_matcher { + struct mock_param_matcher matcher; + const void *expected; + size_t size; +}; + +static bool match_memeq(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *pactual) +{ + struct mock_memeq_matcher *matcher = + container_of(pmatcher, + struct mock_memeq_matcher, + matcher); + const void *actual = CONVERT_TO_ACTUAL_TYPE(const void *, pactual); + bool matches = !memcmp(actual, matcher->expected, matcher->size); + int i; + + for (i = 0; i < matcher->size; i++) + stream->add(stream, "%02x, ", ((const char *) actual)[i]); + if (matches) + stream->add(stream, "== "); + else + stream->add(stream, "!= "); + for (i = 0; i < matcher->size; i++) + stream->add(stream, + "%02x, ", + ((const char *) matcher->expected)[i]); + + return matches; +} + +struct mock_param_matcher *test_memeq(struct test *test, + const void *buf, + size_t size) +{ + struct mock_memeq_matcher *matcher; + + matcher = test_kzalloc(test, sizeof(*matcher), GFP_KERNEL); + if (!matcher) + return NULL; + + matcher->matcher.match = match_memeq; + matcher->expected = buf; + matcher->size = size; + + return &matcher->matcher; +} + +struct mock_streq_matcher { + struct mock_param_matcher matcher; + const char *expected; +}; + +static bool match_streq(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *pactual) +{ + struct mock_streq_matcher *matcher = + container_of(pmatcher, + struct mock_streq_matcher, + matcher); + const char *actual = CONVERT_TO_ACTUAL_TYPE(const char *, pactual); + bool matches = !strcmp(actual, matcher->expected); + + if (matches) + stream->add(stream, "%s == %s", actual, matcher->expected); + else + stream->add(stream, "%s != %s", actual, matcher->expected); + + return matches; +} + +struct mock_param_matcher *test_streq(struct test *test, const char *str) +{ + struct mock_streq_matcher *matcher; + + matcher = test_kzalloc(test, sizeof(*matcher), GFP_KERNEL); + if (!matcher) + return NULL; + + matcher->matcher.match = match_streq; + matcher->expected = str; + + return &matcher->matcher; +} + +#define DEFINE_RETURN_ACTION_STRUCT(type_name, type) \ + struct mock_##type_name##_action { \ + struct mock_action action; \ + type ret; \ + }; + +#define DEFINE_RETURN_ACTION_FUNC(type_name, type) \ + void *do_##type_name##_return(struct mock_action *paction, \ + const void **params, \ + int len) \ + { \ + struct mock_##type_name##_action *action = \ + container_of(paction, \ + struct mock_##type_name##_action,\ + action); \ + \ + return (void *) &action->ret; \ + } + +#define DEFINE_RETURN_ACTION_FACTORY(type_name, type) \ + struct mock_action *test_##type_name##_return( \ + struct test *test, \ + type ret) \ + { \ + struct mock_##type_name##_action *action; \ + \ + action = test_kmalloc(test, \ + sizeof(*action), \ + GFP_KERNEL); \ + if (!action) \ + return NULL; \ + \ + action->action.do_action = do_##type_name##_return; \ + action->ret = ret; \ + \ + return &action->action; \ + } + +#define DEFINE_RETURN_ACTION_WITH_TYPENAME(type_name, type) \ + DEFINE_RETURN_ACTION_STRUCT(type_name, type); \ + DEFINE_RETURN_ACTION_FUNC(type_name, type); \ + DEFINE_RETURN_ACTION_FACTORY(type_name, type); + +#define DEFINE_RETURN_ACTION(type) \ + DEFINE_RETURN_ACTION_WITH_TYPENAME(type, type) + +DEFINE_RETURN_ACTION(u8); +DEFINE_RETURN_ACTION(u16); +DEFINE_RETURN_ACTION(u32); +DEFINE_RETURN_ACTION(u64); +DEFINE_RETURN_ACTION(char); +DEFINE_RETURN_ACTION_WITH_TYPENAME(uchar, unsigned char); +DEFINE_RETURN_ACTION_WITH_TYPENAME(schar, signed char); +DEFINE_RETURN_ACTION(short); +DEFINE_RETURN_ACTION_WITH_TYPENAME(ushort, unsigned short); +DEFINE_RETURN_ACTION(int); +DEFINE_RETURN_ACTION_WITH_TYPENAME(uint, unsigned int); +DEFINE_RETURN_ACTION(long); +DEFINE_RETURN_ACTION_WITH_TYPENAME(ulong, unsigned long); +DEFINE_RETURN_ACTION_WITH_TYPENAME(longlong, long long); +DEFINE_RETURN_ACTION_WITH_TYPENAME(ulonglong, unsigned long long); +DEFINE_RETURN_ACTION_WITH_TYPENAME(ptr, void *); +
Introduces basic class mocking, the ability to automatically generate a Linux C-style class implementation whose behavior is controlled by test cases, which can also set expectations on when and how mocks are called.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 438 +++++++++++++++++++++++++++++++++++++++- kunit/Makefile | 2 +- kunit/example-test.c | 90 +++++++++ kunit/mock-macro-test.c | 82 ++++++++ kunit/mock-test.c | 276 +++++++++++++++++++++++++ kunit/test-mock.c | 39 ++++ kunit/test-mock.h | 23 +++ 7 files changed, 947 insertions(+), 3 deletions(-) create mode 100644 kunit/mock-test.c create mode 100644 kunit/test-mock.c create mode 100644 kunit/test-mock.h
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 62e8afcaeab55..1b7485e2cedb8 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -72,7 +72,8 @@ struct mock_action { * @retire_on_saturation: no longer match once ``max_calls_expected`` is * reached. * - * Represents a *call expectation* on a function created with EXPECT_CALL(). + * Represents a *call expectation* on a function created with + * TEST_EXPECT_CALL(). */ struct mock_expectation { struct mock_action *action; @@ -122,13 +123,446 @@ struct mock_expectation *mock_add_matcher(struct mock *mock, struct mock_param_matcher *matchers[], int len);
+#define MOCK(name) name##_mock + +/** + * TEST_EXPECT_CALL() - Declares a *call expectation* on a mock function. + * @expectation_call: a mocked method or function with parameters replaced with + * matchers. + * + * Example: + * + * .. code-block:: c + * + * // Class to mock. + * struct example { + * int (*foo)(struct example *, int); + * }; + * + * // Define the mock. + * DECLARE_STRUCT_CLASS_MOCK_PREREQS(example); + * + * DEFINE_STRUCT_CLASS_MOCK(METHOD(foo), CLASS(example), + * RETURNS(int), + * PARAMS(struct example *, int)); + * + * static int example_init(struct MOCK(example) *mock_example) + * { + * struct example *example = mock_get_trgt(mock_example); + * + * example->foo = foo; + * return 0; + * } + * + * DEFINE_STRUCT_CLASS_MOCK_INIT(example, example_init); + * + * static void foo_example_test_success(struct test *test) + * { + * struct MOCK(example) *mock_example; + * struct example *example = mock_get_trgt(mock_example); + * struct mock_expectation *handle; + * + * mock_example = CONSTRUCT_MOCK(example, test); + * + * handle = TEST_EXPECT_CALL(foo(mock_get_ctrl(mock_example), + * test_int_eq(test, 5))); + * handle->action = int_return(test, 2); + * + * EXPECT_EQ(test, 2, example_bar(example, 5)); + * } + * + * Return: + * A &struct mock_expectation representing the call expectation. + * allowing additional conditions and actions to be specified. + */ +#define TEST_EXPECT_CALL(expectation_call) mock_master_##expectation_call + +#define mock_get_ctrl_internal(mock_object) (&(mock_object)->ctrl) +#define mock_get_ctrl(mock_object) mock_get_ctrl_internal(mock_object) + +#define mock_get_trgt_internal(mock_object) (&(mock_object)->trgt) +#define mock_get_trgt(mock_object) mock_get_trgt_internal(mock_object) + +#define mock_get_test(mock_object) (mock_get_ctrl(mock_object)->test) + +#define CLASS(struct_name) struct_name +#define HANDLE_INDEX(index) index +#define METHOD(method_name) method_name +#define RETURNS(return_type) return_type +/* #define PARAMS(...) __VA_ARGS__ included by linux/tracepoint.h */ + +#define MOCK_INIT_ID(struct_name) struct_name##mock_init +#define REAL_ID(func_name) __real__##func_name +#define INVOKE_ID(func_name) __invoke__##func_name + +#define DECLARE_MOCK_CLIENT(name, return_type, param_types...) \ + return_type name(PARAM_LIST_FROM_TYPES(param_types)) + +#define DECLARE_MOCK_MASTER(name, ctrl_index, param_types...) \ + struct mock_expectation *mock_master_##name( \ + MATCHER_PARAM_LIST_FROM_TYPES(ctrl_index, \ + param_types)); + +#define DECLARE_MOCK_COMMON(name, handle_index, return_type, param_types...) \ + DECLARE_MOCK_CLIENT(name, return_type, param_types); \ + DECLARE_MOCK_MASTER(name, handle_index, param_types) + +#define DECLARE_STRUCT_CLASS_MOCK_STRUCT(struct_name) \ + struct MOCK(struct_name) { \ + struct mock ctrl; \ + struct struct_name trgt; \ + } + +#define DECLARE_STRUCT_CLASS_MOCK_CONVERTER(struct_name) \ + static inline struct mock *from_##struct_name##_to_mock( \ + const struct struct_name *trgt) \ + { \ + return mock_get_ctrl( \ + container_of(trgt, \ + struct MOCK(struct_name), \ + trgt)); \ + } + +/** + * DECLARE_STRUCT_CLASS_MOCK_PREREQS() - Create a mock child class + * @struct_name: name of the class/struct to be mocked + * + * Creates a mock child class of ``struct_name`` named + * ``struct MOCK(struct_name)`` along with supporting internally used methods. + * + * See TEST_EXPECT_CALL() for example usages. + */ +#define DECLARE_STRUCT_CLASS_MOCK_PREREQS(struct_name) \ + DECLARE_STRUCT_CLASS_MOCK_STRUCT(struct_name); \ + DECLARE_STRUCT_CLASS_MOCK_CONVERTER(struct_name) + +#define DECLARE_STRUCT_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types...) \ + DECLARE_MOCK_COMMON(name, \ + handle_index, \ + return_type, \ + param_types) + +#define DECLARE_STRUCT_CLASS_MOCK_HANDLE_INDEX(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types...) \ + DECLARE_STRUCT_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types) + +/** + * DECLARE_STRUCT_CLASS_MOCK() + * @name: method name + * @struct_name: name of the class/struct + * @return_type: return type of the method + * @param_types: parameters of the method + * + * Same as DEFINE_STRUCT_CLASS_MOCK(), but only makes header compatible + * declarations. + */ +#define DECLARE_STRUCT_CLASS_MOCK(name, \ + struct_name, \ + return_type, \ + param_types...) \ + DECLARE_STRUCT_CLASS_MOCK_HANDLE_INDEX(name, \ + struct_name, \ + 0, \ + return_type, \ + param_types) + +/** + * DECLARE_STRUCT_CLASS_MOCK_VOID_RETURN() + * @name: method name + * @struct_name: name of the class/struct + * @param_types: parameters of the method + * + * Same as DEFINE_STRUCT_CLASS_MOCK_VOID_RETURN(), but only makes header + * compatible declarations. + */ +#define DECLARE_STRUCT_CLASS_MOCK_VOID_RETURN(name, \ + struct_name, \ + param_types...) \ + DECLARE_STRUCT_CLASS_MOCK_HANDLE_INDEX(name, \ + struct_name, \ + 0, \ + void, \ + param_types) + +/** + * DECLARE_STRUCT_CLASS_MOCK_INIT() + * @struct_name: name of the class/struct + * + * Same as DEFINE_STRUCT_CLASS_MOCK_INIT(), but only makes header compatible + * declarations. + */ +#define DECLARE_STRUCT_CLASS_MOCK_INIT(struct_name) \ + struct MOCK(struct_name) *MOCK_INIT_ID(struct_name)( \ + struct test *test) + +/** + * CONSTRUCT_MOCK() + * @struct_name: name of the class + * @test: associated test + * + * Constructs and allocates a test managed ``struct MOCK(struct_name)`` given + * the name of the class for which the mock is defined and a test object. + * + * See TEST_EXPECT_CALL() for example usage. + */ +#define CONSTRUCT_MOCK(struct_name, test) MOCK_INIT_ID(struct_name)(test) + +#define DEFINE_MOCK_CLIENT_COMMON(name, \ + handle_index, \ + MOCK_SOURCE, \ + mock_source_ctx, \ + return_type, \ + RETURN, \ + param_types...) \ + return_type name(PARAM_LIST_FROM_TYPES(param_types)) \ + { \ + struct mock *mock = MOCK_SOURCE(mock_source_ctx, \ + handle_index); \ + static const char * const param_type_names[] = { \ + TYPE_NAMES_FROM_TYPES(handle_index, \ + param_types) \ + }; \ + const void *params[] = { \ + PTR_TO_ARG_FROM_TYPES(handle_index, \ + param_types) \ + }; \ + const void *retval; \ + \ + retval = mock->do_expect(mock, \ + #name, \ + name, \ + param_type_names, \ + params, \ + ARRAY_SIZE(params)); \ + TEST_ASSERT_NOT_ERR_OR_NULL(mock->test, retval); \ + if (!retval) { \ + test_info(mock->test, \ + "no action installed for "#name); \ + BUG(); \ + } \ + RETURN(return_type, retval); \ + } + +#define CLASS_MOCK_CLIENT_SOURCE(ctx, handle_index) ctx(arg##handle_index) +#define DEFINE_MOCK_METHOD_CLIENT_COMMON(name, \ + handle_index, \ + mock_converter, \ + return_type, \ + RETURN, \ + param_types...) \ + DEFINE_MOCK_CLIENT_COMMON(name, \ + handle_index, \ + CLASS_MOCK_CLIENT_SOURCE, \ + mock_converter, \ + return_type, \ + RETURN, \ + param_types) + +#define CAST_AND_RETURN(return_type, retval) return *((return_type *) retval) +#define NO_RETURN(return_type, retval) + +#define DEFINE_MOCK_METHOD_CLIENT(name, \ + handle_index, \ + mock_converter, \ + return_type, \ + param_types...) \ + DEFINE_MOCK_METHOD_CLIENT_COMMON(name, \ + handle_index, \ + mock_converter, \ + return_type, \ + CAST_AND_RETURN, \ + param_types) + +#define DEFINE_MOCK_METHOD_CLIENT_VOID_RETURN(name, \ + handle_index, \ + mock_converter, \ + param_types...) \ + DEFINE_MOCK_METHOD_CLIENT_COMMON(name, \ + handle_index, \ + mock_converter, \ + void, \ + NO_RETURN, \ + param_types) + +#define DEFINE_MOCK_MASTER_COMMON_INTERNAL(name, \ + ctrl_index, \ + MOCK_SOURCE, \ + param_types...) \ + struct mock_expectation *mock_master_##name( \ + MATCHER_PARAM_LIST_FROM_TYPES(ctrl_index, \ + param_types)) \ + { \ + struct mock_param_matcher *matchers[] = { \ + ARG_NAMES_FROM_TYPES(ctrl_index, param_types) \ + }; \ + \ + return mock_add_matcher(MOCK_SOURCE(ctrl_index), \ + #name, \ + (const void *) name, \ + matchers, \ + ARRAY_SIZE(matchers)); \ + } +#define DEFINE_MOCK_MASTER_COMMON(name, \ + ctrl_index, \ + MOCK_SOURCE, \ + param_types...) \ + DEFINE_MOCK_MASTER_COMMON_INTERNAL(name, \ + ctrl_index, \ + MOCK_SOURCE, \ + param_types) + +#define CLASS_MOCK_MASTER_SOURCE(ctrl_index) arg##ctrl_index +#define DEFINE_MOCK_METHOD_MASTER(name, ctrl_index, param_types...) \ + DEFINE_MOCK_MASTER_COMMON(name, \ + ctrl_index, \ + CLASS_MOCK_MASTER_SOURCE, \ + param_types) + +#define DEFINE_MOCK_COMMON(name, \ + handle_index, \ + mock_converter, \ + return_type, \ + param_types...) \ + DEFINE_MOCK_METHOD_CLIENT(name, \ + handle_index, \ + mock_converter, \ + return_type, \ + param_types); \ + DEFINE_MOCK_METHOD_MASTER(name, handle_index, param_types) + +#define DEFINE_MOCK_COMMON_VOID_RETURN(name, \ + handle_index, \ + mock_converter, \ + param_types...) \ + DEFINE_MOCK_METHOD_CLIENT_VOID_RETURN(name, \ + handle_index, \ + mock_converter, \ + param_types); \ + DEFINE_MOCK_METHOD_MASTER(name, handle_index, param_types) + +#define DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types...) \ + DEFINE_MOCK_COMMON(name, \ + handle_index, \ + from_##struct_name##_to_mock, \ + return_type, param_types) +#define DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types...) \ + DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + struct_name, \ + handle_index, \ + return_type, \ + param_types) + +#define DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_VOID_RETURN_INTERNAL( \ + name, \ + struct_name, \ + handle_index, \ + param_types...) \ + DEFINE_MOCK_COMMON_VOID_RETURN(name, \ + handle_index, \ + from_##struct_name##_to_mock, \ + param_types) +#define DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_VOID_RETURN(name, \ + struct_name, \ + handle_index, \ + param_types...) \ + DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_VOID_RETURN_INTERNAL( \ + name, \ + struct_name, \ + handle_index, \ + param_types) + +/** + * DEFINE_STRUCT_CLASS_MOCK() + * @name: name of the method + * @struct_name: name of the class of which the method belongs + * @return_type: return type of the method to be created. **Must not be void.** + * @param_types: parameters to method to be created. + * + * See TEST_EXPECT_CALL() for example usage. + */ +#define DEFINE_STRUCT_CLASS_MOCK(name, \ + struct_name, \ + return_type, \ + param_types...) \ + DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX(name, \ + struct_name, \ + 0, \ + return_type, \ + param_types) + +/** + * DEFINE_STRUCT_CLASS_MOCK_VOID_RETURN() + * @name: name of the method + * @struct_name: name of the class of which the method belongs + * @param_types: parameters to method to be created. + * + * Same as DEFINE_STRUCT_CLASS_MOCK() except the method has a ``void`` return + * type. + */ +#define DEFINE_STRUCT_CLASS_MOCK_VOID_RETURN(name, struct_name, param_types...)\ + DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX_VOID_RETURN(name, \ + struct_name, \ + 0, \ + param_types) + +/** + * DEFINE_STRUCT_CLASS_MOCK_INIT() + * @struct_name: name of the class + * @init_func: a function of type ``int (*)(struct MOCK(struct_name) *)``. This + * function is passed a pointer to an allocated, *but not + * initialized*, ``struct MOCK(struct_name)``. The job of this user + * provided function is to perform remaining initialization. Usually + * this entails assigning mock methods to the function pointers in + * the parent struct. + * + * See TEST_EXPECT_CALL() for example usage. + */ +#define DEFINE_STRUCT_CLASS_MOCK_INIT(struct_name, init_func) \ + struct MOCK(struct_name) *MOCK_INIT_ID(struct_name)( \ + struct test *test) \ + { \ + struct MOCK(struct_name) *mock_obj; \ + \ + mock_obj = test_kzalloc(test, \ + sizeof(*mock_obj), \ + GFP_KERNEL); \ + if (!mock_obj) \ + return NULL; \ + \ + mock_init_ctrl(test, mock_get_ctrl(mock_obj)); \ + \ + if (init_func(mock_obj)) \ + return NULL; \ + \ + return mock_obj; \ + } + #define CONVERT_TO_ACTUAL_TYPE(type, ptr) (*((type *) ptr))
/** * DOC: Built In Matchers * * These are the matchers that can be used when matching arguments in - * :c:func:`EXPECT_CALL` (more can be defined manually). + * :c:func:`TEST_EXPECT_CALL` (more can be defined manually). * * For example, there's a matcher that matches any arguments: * diff --git a/kunit/Makefile b/kunit/Makefile index 52a1da46cbd21..6fccfcdbc6f84 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,5 +1,5 @@ obj-$(CONFIG_KUNIT) += test.o mock.o common-mocks.o string-stream.o \ test-stream.o obj-$(CONFIG_KUNIT_TEST) += \ - test-test.o mock-macro-test.o string-stream-test.o + test-test.o test-mock.o mock-macro-test.o mock-test.o string-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/example-test.c b/kunit/example-test.c index e9bd2b41c5fd2..dd3b75a61d5b4 100644 --- a/kunit/example-test.c +++ b/kunit/example-test.c @@ -7,6 +7,7 @@ */
#include <kunit/test.h> +#include <kunit/mock.h>
/* * This is the most fundamental element of KUnit, the test case. A test case @@ -29,6 +30,84 @@ static void example_simple_test(struct test *test) TEST_EXPECT_EQ(test, 1 + 1, 2); }
+/* + * A lot of times, you have a C-style class like this, which acts an abstraction + * over hardware, a file system implementation, or some other subsystem that you + * want to reason about in a generic way. + */ +struct example { + int (*foo)(struct example *example, int num); +}; + +static int example_bar(struct example *example, int num) +{ + return example->foo(example, num); +} + +/* + * KUnit allows such a class to be "mocked out" with the following: + */ + +/* + * This macro creates a mock subclass of the specified class. + */ +DECLARE_STRUCT_CLASS_MOCK_PREREQS(example); + +/* + * This macro creates a mock implementation of the specified method of the + * specified class. + */ +DEFINE_STRUCT_CLASS_MOCK(METHOD(foo), CLASS(example), + RETURNS(int), + PARAMS(struct example *, int)); + +/* + * This tells KUnit how to initialize the parts of the mock that come from the + * parent. In this example, all we have to do is populate the member functions + * of the parent class with the mock versions we defined. + */ +static int example_init(struct MOCK(example) *mock_example) +{ + /* This is how you get a pointer to the parent class of a mock. */ + struct example *example = mock_get_trgt(mock_example); + + /* + * Here we populate the member function (method) with our mock method. + */ + example->foo = foo; + return 0; +} + +/* + * This registers our parent init function above, allowing KUnit to create a + * constructor for the mock. + */ +DEFINE_STRUCT_CLASS_MOCK_INIT(example, example_init); + +/* + * This is a test case where we use our mock. + */ +static void example_mock_test(struct test *test) +{ + struct MOCK(example) *mock_example = test->priv; + struct example *example = mock_get_trgt(mock_example); + struct mock_expectation *handle; + + /* + * Here we make an expectation that our mock method will be called with + * a parameter equal to 5 passed in. + */ + handle = TEST_EXPECT_CALL(foo(mock_get_ctrl(mock_example), + test_int_eq(test, 5))); + /* + * We specify that when our mock is called in this way, we want it to + * return 2. + */ + handle->action = test_int_return(test, 2); + + TEST_EXPECT_EQ(test, 2, example_bar(example, 5)); +} + /* * This is run once before each test case, see the comment on * example_test_module for more information. @@ -37,6 +116,16 @@ static int example_test_init(struct test *test) { test_info(test, "initializing");
+ /* + * Here we construct the mock and store it in test's `priv` field; this + * field is for KUnit users. You can put whatever you want here, but + * most often it is a place that the init function can put stuff to be + * used by test cases. + */ + test->priv = CONSTRUCT_MOCK(example, test); + if (!test->priv) + return -EINVAL; + return 0; }
@@ -52,6 +141,7 @@ static struct test_case example_test_cases[] = { * test module. */ TEST_CASE(example_simple_test), + TEST_CASE(example_mock_test), {}, };
diff --git a/kunit/mock-macro-test.c b/kunit/mock-macro-test.c index c30b859ff2b14..84d9d3f484366 100644 --- a/kunit/mock-macro-test.c +++ b/kunit/mock-macro-test.c @@ -8,6 +8,49 @@
#include <kunit/test.h> #include <kunit/params.h> +#include <kunit/mock.h> + +struct test_struct { + int (*one_param)(struct test_struct *test_struct); + int (*two_param)(struct test_struct *test_struct, int num); + int (*non_first_slot_param)(int num, struct test_struct *test_struct); + void *(*void_ptr_return)(struct test_struct *test_struct); +}; + +DECLARE_STRUCT_CLASS_MOCK_PREREQS(test_struct); + +DEFINE_STRUCT_CLASS_MOCK(METHOD(one_param), CLASS(test_struct), + RETURNS(int), + PARAMS(struct test_struct *)); + +DEFINE_STRUCT_CLASS_MOCK(METHOD(two_param), CLASS(test_struct), + RETURNS(int), + PARAMS(struct test_struct *, int)); + +DEFINE_STRUCT_CLASS_MOCK_HANDLE_INDEX(METHOD(non_first_slot_param), + CLASS(test_struct), HANDLE_INDEX(1), + RETURNS(int), + PARAMS(int, struct test_struct *)); + +DEFINE_STRUCT_CLASS_MOCK(METHOD(void_ptr_return), CLASS(test_struct), + RETURNS(void *), + PARAMS(struct test_struct *)); + +static int test_struct_init(struct MOCK(test_struct) *mock_test_struct) +{ + struct test_struct *test_struct = mock_get_trgt(mock_test_struct); + + test_struct->one_param = one_param; + test_struct->two_param = two_param; + test_struct->non_first_slot_param = non_first_slot_param; + return 0; +} + +DEFINE_STRUCT_CLASS_MOCK_INIT(test_struct, test_struct_init); + +struct mock_macro_context { + struct MOCK(test_struct) *mock_test_struct; +};
#define TO_STR_INTERNAL(...) #__VA_ARGS__ #define TO_STR(...) TO_STR_INTERNAL(__VA_ARGS__) @@ -131,6 +174,43 @@ static void mock_macro_arg_names_from_types(struct test *test) type15))); }
+static void mock_macro_test_generated_method_code_works(struct test *test) +{ + struct mock_macro_context *ctx = test->priv; + struct MOCK(test_struct) *mock_test_struct = ctx->mock_test_struct; + struct test_struct *test_struct = mock_get_trgt(mock_test_struct); + struct mock_expectation *handle; + + handle = TEST_EXPECT_CALL(one_param(mock_get_ctrl(mock_test_struct))); + handle->action = test_int_return(test, 0); + handle = TEST_EXPECT_CALL(two_param(mock_get_ctrl(mock_test_struct), + test_int_eq(test, 5))); + handle->action = test_int_return(test, 1); + handle = TEST_EXPECT_CALL(non_first_slot_param( + test_int_eq(test, 5), mock_get_ctrl(mock_test_struct))); + handle->action = test_int_return(test, 1); + + test_struct->one_param(test_struct); + test_struct->two_param(test_struct, 5); + test_struct->non_first_slot_param(5, test_struct); +} + +static int mock_macro_test_init(struct test *test) +{ + struct mock_macro_context *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + test->priv = ctx; + + ctx->mock_test_struct = CONSTRUCT_MOCK(test_struct, test); + if (!ctx->mock_test_struct) + return -EINVAL; + + return 0; +} + static struct test_case mock_macro_test_cases[] = { TEST_CASE(mock_macro_is_equal), TEST_CASE(mock_macro_if), @@ -139,11 +219,13 @@ static struct test_case mock_macro_test_cases[] = { TEST_CASE(mock_macro_for_each_param), TEST_CASE(mock_macro_param_list_from_types_basic), TEST_CASE(mock_macro_arg_names_from_types), + TEST_CASE(mock_macro_test_generated_method_code_works), {}, };
static struct test_module mock_macro_test_module = { .name = "mock-macro-test", + .init = mock_macro_test_init, .test_cases = mock_macro_test_cases, }; module_test(mock_macro_test_module); diff --git a/kunit/mock-test.c b/kunit/mock-test.c new file mode 100644 index 0000000000000..523ddee8f24e2 --- /dev/null +++ b/kunit/mock-test.c @@ -0,0 +1,276 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for mock.h. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> +#include <kunit/mock.h> + +#include "test-mock.h" + +struct mock_test_context { + struct MOCK(test) *mock_test; + struct mock *mock; +}; + +static void mock_test_do_expect_basic(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -4; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_param_matcher *matchers_any_two[] = { + test_any(trgt), test_any(trgt) + }; + struct mock_expectation *expectation; + const void *ret; + + expectation = mock_add_matcher(mock, + "", + NULL, + matchers_any_two, + ARRAY_SIZE(matchers_any_two)); + expectation->action = test_int_return(trgt, 5); + TEST_EXPECT_EQ(test, 0, expectation->times_called); + + ret = mock->do_expect(mock, + "", + NULL, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + TEST_ASSERT_NOT_ERR_OR_NULL(test, ret); + TEST_EXPECT_EQ(test, 5, *((int *) ret)); + TEST_EXPECT_EQ(test, 1, expectation->times_called); +} + +static void mock_test_ptr_eq(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + void *param0 = ctx, *param1 = trgt; + static const char * const two_param_types[] = {"void *", "void *"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_param_matcher *matchers_two_ptrs[] = { + test_ptr_eq(trgt, param0), test_ptr_eq(trgt, param1) + }; + struct mock_expectation *expectation; + const void *ret; + + expectation = mock_add_matcher(mock, + "", + NULL, + matchers_two_ptrs, + ARRAY_SIZE(matchers_two_ptrs)); + expectation->action = test_int_return(trgt, 0); + TEST_EXPECT_EQ(test, 0, expectation->times_called); + + ret = mock->do_expect(mock, + "", + NULL, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + TEST_ASSERT_NOT_ERR_OR_NULL(test, ret); + TEST_EXPECT_EQ(test, 1, expectation->times_called); +} + +static void mock_test_ptr_eq_not_equal(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + void *param0 = ctx, *param1 = trgt; + static const char * const two_param_types[] = {"void *", "void *"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_param_matcher *matchers_two_ptrs[] = { + test_ptr_eq(trgt, param0), test_ptr_eq(trgt, param1 - 1) + }; + struct mock_expectation *expectation; + const void *ret; + + expectation = mock_add_matcher(mock, + "", + NULL, + matchers_two_ptrs, + ARRAY_SIZE(matchers_two_ptrs)); + expectation->action = test_int_return(trgt, 0); + TEST_EXPECT_EQ(test, 0, expectation->times_called); + + ret = mock->do_expect(mock, + "", + NULL, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + TEST_EXPECT_FALSE(test, ret); + TEST_EXPECT_EQ(test, 0, expectation->times_called); +} + +/* + * In order for us to be able to rely on TEST_EXPECT_CALL to validate other + * behavior, we need to test that unsatisfied TEST_EXPECT_CALL causes a test + * failure. + */ +static void mock_test_failed_expect_call_fails_test(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct mock *mock = ctx->mock; + + /* mock is a pretend mock belonging to our mocked_test */ + + /* Put an expectation on mocked mock */ + TEST_EXPECT_CALL(fail(mock, test_any(mock_get_trgt(mock_test)))); + + /* + * Expect that mock_test will fail because the above won't be satisfied + */ + TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), test_any(test))); + + /* + * Validate expectations of mocked mock, which should fail mocked test + */ + mock_validate_expectations(mock); + + /* Validate mock_test's expectations, that is, it should have failed */ + mock_validate_expectations(mock_get_ctrl(mock_test)); + TEST_EXPECT_FALSE(test, mock_get_trgt(mock_test)->success); +} + +static void mock_test_do_expect_default_return(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_param_matcher *matchers[] = { + test_int_eq(trgt, 5), + test_int_eq(trgt, -4) + }; + struct mock_expectation *expectation; + const void *ret; + + expectation = mock_add_matcher(mock, + "test_printk", + test_printk, + matchers, + ARRAY_SIZE(matchers)); + expectation->action = test_int_return(trgt, 5); + TEST_EXPECT_EQ(test, 0, expectation->times_called); + + TEST_EXPECT_FALSE(test, mock_set_default_action(mock, + "test_printk", + test_printk, + test_int_return(trgt, -4))); + + ret = mock->do_expect(mock, + "test_printk", + test_printk, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + TEST_ASSERT_NOT_ERR_OR_NULL(test, ret); + TEST_EXPECT_EQ(test, -4, *((int *) ret)); + TEST_EXPECT_EQ(test, 0, expectation->times_called); +} + +static void mock_test_mock_validate_expectations(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + struct mock_param_matcher *matchers[] = { + test_int_eq(trgt, 5), + test_int_eq(trgt, -4) + }; + struct mock_expectation *expectation; + + TEST_EXPECT_EQ(test, mock_get_trgt(mock_test), mock->test); + + expectation = mock_add_matcher(mock, + "test_printk", + test_printk, + matchers, + ARRAY_SIZE(matchers)); + expectation->times_called = 0; + expectation->min_calls_expected = 1; + expectation->max_calls_expected = 1; + + TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), test_any(test))); + + mock_validate_expectations(mock); +} + +void *do_mocked_fail(struct mock_action *this, const void **params, int len) +{ + static const int ret; + struct test_stream * const *stream_ptr = params[0]; + struct test_stream *stream = *stream_ptr; + + stream->set_level(stream, KERN_ERR); + stream->commit(stream); + return (void *) &ret; +} + +static struct mock_action mocked_fail = { + .do_action = do_mocked_fail +}; + +static int mock_test_init(struct test *test) +{ + struct mock_test_context *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + test->priv = ctx; + + ctx->mock_test = CONSTRUCT_MOCK(test, test); + if (!ctx->mock_test) + return -EINVAL; + + ctx->mock = test_kzalloc(test, sizeof(*ctx->mock), GFP_KERNEL); + if (!ctx->mock) + return -ENOMEM; + mock_init_ctrl(mock_get_trgt(ctx->mock_test), ctx->mock); + + /* This test suite tests the behaviour of the error messages printed + * when mocks fail, which requires the mocked fail to commit the + * stream. + */ + mock_set_default_action(mock_get_ctrl(ctx->mock_test), + "fail", fail, &mocked_fail); + return 0; +} + +static struct test_case mock_test_cases[] = { + TEST_CASE(mock_test_do_expect_basic), + TEST_CASE(mock_test_ptr_eq), + TEST_CASE(mock_test_ptr_eq_not_equal), + TEST_CASE(mock_test_failed_expect_call_fails_test), + TEST_CASE(mock_test_do_expect_default_return), + TEST_CASE(mock_test_mock_validate_expectations), + {}, +}; + +static struct test_module mock_test_module = { + .name = "mock-test", + .init = mock_test_init, + .test_cases = mock_test_cases, +}; + +module_test(mock_test_module); diff --git a/kunit/test-mock.c b/kunit/test-mock.c new file mode 100644 index 0000000000000..a0858d6b3f9c1 --- /dev/null +++ b/kunit/test-mock.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit mock for struct test. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include "test-mock.h" + +DEFINE_STRUCT_CLASS_MOCK_VOID_RETURN(METHOD(fail), CLASS(test), + PARAMS(struct test *, + struct test_stream *)); + +DEFINE_STRUCT_CLASS_MOCK_VOID_RETURN(METHOD(mock_vprintk), CLASS(test), + PARAMS(const struct test *, + const char *, + struct va_format *)); + +static int test_init(struct MOCK(test) *mock_test) +{ + struct test *trgt = mock_get_trgt(mock_test); + int ret; + + ret = test_init_test(trgt, "MOCK(test)"); + trgt->fail = fail; + mock_set_default_action(mock_get_ctrl(mock_test), + "fail", + fail, + test_int_return(mock_get_test(mock_test), 0)); + trgt->vprintk = mock_vprintk; + mock_set_default_action(mock_get_ctrl(mock_test), + "mock_vprintk", + mock_vprintk, + test_int_return(mock_get_test(mock_test), 0)); + return ret; +} + +DEFINE_STRUCT_CLASS_MOCK_INIT(test, test_init); diff --git a/kunit/test-mock.h b/kunit/test-mock.h new file mode 100644 index 0000000000000..c57e9384b1d8a --- /dev/null +++ b/kunit/test-mock.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KUnit mock for struct test. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> +#include <kunit/mock.h> + +DECLARE_STRUCT_CLASS_MOCK_PREREQS(test); + +DECLARE_STRUCT_CLASS_MOCK_VOID_RETURN(METHOD(fail), CLASS(test), + PARAMS(struct test *, + struct test_stream *)); + +DECLARE_STRUCT_CLASS_MOCK_VOID_RETURN(METHOD(mock_vprintk), CLASS(test), + PARAMS(const struct test *, + const char *, + struct va_format *)); + +DECLARE_STRUCT_CLASS_MOCK_INIT(test);
Added parameter matcher builder for matching struct values.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 56 +++++++++++++++ kunit/Makefile | 5 +- kunit/common-mocks.c | 116 ++++++++++++++++++++++++++++++ kunit/mock-test.c | 43 +++++++++++ kunit/test-stream-test.c | 150 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 368 insertions(+), 2 deletions(-) create mode 100644 kunit/test-stream-test.c
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 1b7485e2cedb8..daf965cf954e6 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -760,6 +760,14 @@ struct mock_param_matcher *test_memeq(struct test *test, size_t size); struct mock_param_matcher *test_streq(struct test *test, const char *str);
+struct mock_param_matcher *test_str_contains(struct test *test, + const char *needle); + +/* Matches var-arg arguments. */ +struct mock_param_matcher *test_va_format_cmp(struct test *test, + struct mock_param_matcher *fmt_matcher, + struct mock_param_matcher *va_matcher); + struct mock_action *test_u8_return(struct test *test, u8 ret); struct mock_action *test_u16_return(struct test *test, u16 ret); struct mock_action *test_u32_return(struct test *test, u32 ret); @@ -778,4 +786,52 @@ struct mock_action *test_ulonglong_return(struct test *test, unsigned long long ret); struct mock_action *test_ptr_return(struct test *test, void *ret);
+/** + * struct mock_struct_matcher_entry - composed with other &struct + * mock_struct_matcher_entry to make a + * &struct struct_matcher + * @member_offset: offset of this member + * @matcher: matcher for this particular member + * + * This is used for struct_cmp() matchers. + */ +struct mock_struct_matcher_entry { + size_t member_offset; + struct mock_param_matcher *matcher; +}; + +static inline void init_mock_struct_matcher_entry_internal( + struct mock_struct_matcher_entry *entry, + size_t offset, + struct mock_param_matcher *matcher) +{ + entry->member_offset = offset; + entry->matcher = matcher; +} + +/** + * INIT_MOCK_STRUCT_MATCHER_ENTRY() + * @entry: the &struct mock_struct_matcher_entry to initialize + * @type: the struct being matched + * @member: the member of the struct being matched, used to calculate the offset + * @matcher: matcher to match that member + * + * Initializes ``entry`` to match ``type->member`` with ``matcher``. + */ +#define INIT_MOCK_STRUCT_MATCHER_ENTRY(entry, type, member, matcher) \ + init_mock_struct_matcher_entry_internal(entry, \ + offsetof(type, member),\ + matcher) + +static inline void INIT_MOCK_STRUCT_MATCHER_ENTRY_LAST( + struct mock_struct_matcher_entry *entry) +{ + entry->matcher = NULL; +} + +struct mock_param_matcher *test_struct_cmp( + struct test *test, + const char *struct_name, + struct mock_struct_matcher_entry *entries); + #endif /* _KUNIT_MOCK_H */ diff --git a/kunit/Makefile b/kunit/Makefile index 6fccfcdbc6f84..e05fbcae8bfb0 100644 --- a/kunit/Makefile +++ b/kunit/Makefile @@ -1,5 +1,6 @@ obj-$(CONFIG_KUNIT) += test.o mock.o common-mocks.o string-stream.o \ test-stream.o -obj-$(CONFIG_KUNIT_TEST) += \ - test-test.o test-mock.o mock-macro-test.o mock-test.o string-stream-test.o +obj-$(CONFIG_KUNIT_TEST) += \ + test-test.o test-mock.o mock-macro-test.o mock-test.o string-stream-test.o \ + test-stream-test.o obj-$(CONFIG_EXAMPLE_TEST) += example-test.o diff --git a/kunit/common-mocks.c b/kunit/common-mocks.c index ecac9c1c29c0e..ef88f8b8acda3 100644 --- a/kunit/common-mocks.c +++ b/kunit/common-mocks.c @@ -207,6 +207,122 @@ struct mock_param_matcher *test_streq(struct test *test, const char *str) return &matcher->matcher; }
+struct mock_str_contains_matcher { + struct mock_param_matcher matcher; + const char *needle; +}; + +static bool match_str_contains(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *phaystack) +{ + struct mock_str_contains_matcher *matcher = + container_of(pmatcher, + struct mock_str_contains_matcher, + matcher); + const char *haystack = CONVERT_TO_ACTUAL_TYPE(const char *, phaystack); + bool matches = strstr(haystack, matcher->needle); + + if (matches) + stream->add(stream, + "'%s' found in '%s'", + matcher->needle, + haystack); + else + stream->add(stream, + "'%s' not found in '%s'", + matcher->needle, + haystack); + return matches; +} + +struct mock_param_matcher *test_str_contains(struct test *test, const char *str) +{ + struct mock_str_contains_matcher *matcher; + + matcher = test_kzalloc(test, sizeof(*matcher), GFP_KERNEL); + if (!matcher) + return NULL; + + matcher->matcher.match = match_str_contains; + matcher->needle = str; + + return &matcher->matcher; +} + +struct mock_param_matcher *test_va_format_cmp( + struct test *test, + struct mock_param_matcher *fmt_matcher, + struct mock_param_matcher *va_matcher) +{ + struct mock_struct_matcher_entry *entries; + + entries = test_kzalloc(test, sizeof(*entries) * 3, GFP_KERNEL); + if (!entries) + return NULL; + + INIT_MOCK_STRUCT_MATCHER_ENTRY(&entries[0], + struct va_format, + fmt, + fmt_matcher); + INIT_MOCK_STRUCT_MATCHER_ENTRY(&entries[1], + struct va_format, + va, + va_matcher); + INIT_MOCK_STRUCT_MATCHER_ENTRY_LAST(&entries[2]); + + return test_struct_cmp(test, "va_format", entries); +} + +struct mock_struct_matcher { + struct mock_param_matcher matcher; + const char *struct_name; + struct mock_struct_matcher_entry *entries; +}; + +static bool match_struct(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *pactual) +{ + struct mock_struct_matcher *matcher = + container_of(pmatcher, + struct mock_struct_matcher, + matcher); + struct mock_struct_matcher_entry *entry; + const char *actual = CONVERT_TO_ACTUAL_TYPE(const char *, pactual); + const char *member_ptr; + bool matches = true, tmp; + + stream->add(stream, "struct %s {", matcher->struct_name); + for (entry = matcher->entries; entry->matcher; entry++) { + member_ptr = actual + entry->member_offset; + tmp = entry->matcher->match(entry->matcher, stream, member_ptr); + matches = matches && tmp; + stream->add(stream, ", "); + } + stream->add(stream, "}"); + + return matches; +} + +struct mock_param_matcher *test_struct_cmp( + struct test *test, + const char *struct_name, + struct mock_struct_matcher_entry *entries) +{ + struct mock_struct_matcher *matcher; + + matcher = test_kzalloc(test, sizeof(*matcher), GFP_KERNEL); + if (!matcher) + return NULL; + + matcher->matcher.match = match_struct; + matcher->struct_name = struct_name; + matcher->entries = entries; + + return &matcher->matcher; +} + #define DEFINE_RETURN_ACTION_STRUCT(type_name, type) \ struct mock_##type_name##_action { \ struct mock_action action; \ diff --git a/kunit/mock-test.c b/kunit/mock-test.c index 523ddee8f24e2..77b16ad754424 100644 --- a/kunit/mock-test.c +++ b/kunit/mock-test.c @@ -187,6 +187,48 @@ static void mock_test_do_expect_default_return(struct test *test) TEST_EXPECT_EQ(test, 0, expectation->times_called); }
+/* + * Method called on naggy mock with no expectations will not fail, but will show + * a warning message + */ +static void mock_test_naggy_no_expectations_no_fail(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_expectation *expectation; + + mock_set_default_action(mock, + "test_printk", + test_printk, + test_int_return(trgt, -4)); + + expectation = TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), + test_any(test))); + expectation->min_calls_expected = 0; + expectation->max_calls_expected = 0; + + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_str_contains(test, + "Method was called with no expectations declared"), + test_any(test)))); + + mock->do_expect(mock, + "test_printk", + test_printk, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + mock_validate_expectations(mock); +} + static void mock_test_mock_validate_expectations(struct test *test) { struct mock_test_context *ctx = test->priv; @@ -264,6 +306,7 @@ static struct test_case mock_test_cases[] = { TEST_CASE(mock_test_failed_expect_call_fails_test), TEST_CASE(mock_test_do_expect_default_return), TEST_CASE(mock_test_mock_validate_expectations), + TEST_CASE(mock_test_naggy_no_expectations_no_fail), {}, };
diff --git a/kunit/test-stream-test.c b/kunit/test-stream-test.c new file mode 100644 index 0000000000000..875b0db15878d --- /dev/null +++ b/kunit/test-stream-test.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test for struct test_stream. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <kunit/test.h> +#include <kunit/mock.h> +#include <kunit/test-stream.h> + +#include "test-mock.h" + +struct test_stream_test_context { + struct MOCK(test) *mock_test; + struct test_stream *stream; +}; + +static void test_stream_test_add(struct test *test) +{ + struct test_stream_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test_stream *stream = ctx->stream; + + stream->add(stream, "Foo"); + stream->add(stream, " %s", "bar"); + stream->set_level(stream, KERN_INFO); + + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, "Foo bar"), + test_any(test)))); + + stream->commit(stream); +} + +static void test_stream_test_append(struct test *test) +{ + struct test_stream_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test_stream *stream = ctx->stream; + struct test_stream *other_stream; + + stream->add(stream, "Foo"); + stream->set_level(stream, KERN_INFO); + other_stream = test_new_stream(mock_get_trgt(mock_test)); + other_stream->add(other_stream, " %s", "bar"); + + stream->append(stream, other_stream); + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, "Foo bar"), + test_any(test)))); + + stream->commit(stream); +} + +static void test_stream_error_message_when_no_level_set(struct test *test) +{ + struct test_stream_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test_stream *stream = ctx->stream; + struct test_stream *other_stream; + + stream->add(stream, "Foo bar"); + other_stream = test_new_stream(mock_get_trgt(mock_test)); + + stream->append(stream, other_stream); + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, + "Stream was committed without a specified log level."), + test_any(test)))); + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, "Foo bar"), + test_any(test)))); + stream->commit(stream); +} + +static void test_stream_test_commits_any_uncommitted_when_cleanup( + struct test *test) +{ + struct test_stream_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test_stream *stream = ctx->stream; + + stream->add(stream, "Hello World"); + stream->set_level(stream, KERN_WARNING); + + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, + "End of test case reached with uncommitted stream entries."), + test_any(test)))); + TEST_EXPECT_CALL(mock_vprintk( + mock_get_ctrl(mock_test), + test_any(test), + test_va_format_cmp(test, + test_streq(test, + "Hello World"), + test_any(test)))); + test_cleanup(mock_get_trgt(mock_test)); +} + +static int test_stream_test_init(struct test *test) +{ + struct test_stream_test_context *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + test->priv = ctx; + + ctx->mock_test = CONSTRUCT_MOCK(test, test); + if (!ctx->mock_test) + return -EINVAL; + + ctx->stream = test_new_stream(mock_get_trgt(ctx->mock_test)); + if (!ctx->stream) + return -ENOMEM; + + return 0; +} + +static struct test_case test_stream_test_cases[] = { + TEST_CASE(test_stream_test_add), + TEST_CASE(test_stream_test_append), + TEST_CASE(test_stream_test_commits_any_uncommitted_when_cleanup), + TEST_CASE(test_stream_error_message_when_no_level_set), + {}, +}; + +static struct test_module test_stream_test_module = { + .name = "test-stream-test", + .init = test_stream_test_init, + .test_cases = test_stream_test_cases, +}; +module_test(test_stream_test_module);
Added parameter formatters which provide string formatting for parameters that have no matchers to be matched against.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 49 +++++++++++++++ kunit/common-mocks.c | 132 +++++++++++++++++++++++++++++++++++++++ kunit/mock.c | 48 ++++++++++++-- kunit/test-stream-test.c | 28 +++++++++ 4 files changed, 252 insertions(+), 5 deletions(-)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index daf965cf954e6..4f85b39d628d0 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -123,6 +123,18 @@ struct mock_expectation *mock_add_matcher(struct mock *mock, struct mock_param_matcher *matchers[], int len);
+struct mock_param_formatter { + struct list_head node; + const char *type_name; + void (*format)(struct mock_param_formatter *formatter, + struct test_stream *stream, + const void *param); +}; + +void mock_register_formatter(struct mock_param_formatter *formatter); + +void mock_unregister_formatter(struct mock_param_formatter *formatter); + #define MOCK(name) name##_mock
/** @@ -834,4 +846,41 @@ struct mock_param_matcher *test_struct_cmp( const char *struct_name, struct mock_struct_matcher_entry *entries);
+struct mock_struct_formatter_entry { + size_t member_offset; + struct mock_param_formatter *formatter; +}; + +static inline void init_mock_struct_formatter_entry_internal( + struct mock_struct_formatter_entry *entry, + size_t offset, + struct mock_param_formatter *formatter) +{ + entry->member_offset = offset; + entry->formatter = formatter; +} + +#define INIT_MOCK_STRUCT_FORMATTER_ENTRY(entry, type, member, formatter) \ + init_mock_struct_formatter_entry_internal(entry, \ + offsetof(type, \ + member), \ + formatter) + +static inline void INIT_MOCK_STRUCT_FORMATTER_ENTRY_LAST( + struct mock_struct_formatter_entry *entry) +{ + entry->formatter = NULL; +} + +struct mock_param_formatter *mock_struct_formatter( + struct test *test, + const char *struct_name, + struct mock_struct_formatter_entry *entries); + +struct mock_param_formatter *mock_find_formatter(const char *type_name); + +#define FORMATTER_FROM_TYPE(type) mock_find_formatter(#type) + +extern struct mock_param_formatter unknown_formatter[]; + #endif /* _KUNIT_MOCK_H */ diff --git a/kunit/common-mocks.c b/kunit/common-mocks.c index ef88f8b8acda3..1c52522808cab 100644 --- a/kunit/common-mocks.c +++ b/kunit/common-mocks.c @@ -386,3 +386,135 @@ DEFINE_RETURN_ACTION_WITH_TYPENAME(longlong, long long); DEFINE_RETURN_ACTION_WITH_TYPENAME(ulonglong, unsigned long long); DEFINE_RETURN_ACTION_WITH_TYPENAME(ptr, void *);
+struct mock_param_integer_formatter { + struct mock_param_formatter formatter; + const char *fmt_str; +}; + +static void mock_format_integer(struct mock_param_formatter *pformatter, + struct test_stream *stream, + const void *pparam) +{ + struct mock_param_integer_formatter *formatter = + container_of(pformatter, + struct mock_param_integer_formatter, + formatter); + long long param = CONVERT_TO_ACTUAL_TYPE(long long, pparam); + + stream->add(stream, formatter->fmt_str, param); +} + +#define INTEGER_FORMATTER_INIT(type, fmt) { \ + .formatter = { \ + .type_name = #type, \ + .format = mock_format_integer, \ + }, \ + .fmt_str = fmt, \ +} + +static struct mock_param_integer_formatter integer_formatters[] = { + INTEGER_FORMATTER_INIT(u8, "%PRIu8"), + INTEGER_FORMATTER_INIT(u16, "%PRIu16"), + INTEGER_FORMATTER_INIT(u32, "%PRIu32"), + INTEGER_FORMATTER_INIT(u64, "%PRIu64"), + INTEGER_FORMATTER_INIT(char, "%c"), + INTEGER_FORMATTER_INIT(unsigned char, "%hhu"), + INTEGER_FORMATTER_INIT(signed char, "%hhd"), + INTEGER_FORMATTER_INIT(short, "%hd"), + INTEGER_FORMATTER_INIT(unsigned short, "%hu"), + INTEGER_FORMATTER_INIT(int, "%d"), + INTEGER_FORMATTER_INIT(unsigned int, "%u"), + INTEGER_FORMATTER_INIT(long, "%ld"), + INTEGER_FORMATTER_INIT(unsigned long, "%lu"), + INTEGER_FORMATTER_INIT(long long, "%lld"), + INTEGER_FORMATTER_INIT(unsigned long long, "%llu"), + INTEGER_FORMATTER_INIT(void *, "%px"), +}; + +static void mock_format_string(struct mock_param_formatter *formatter, + struct test_stream *stream, + const void *pparam) +{ + const char *param = CONVERT_TO_ACTUAL_TYPE(const char *, pparam); + + stream->add(stream, "%s", param); +} + +static struct mock_param_formatter string_formatter = { + .type_name = "const char *", + .format = mock_format_string, +}; + +static void mock_format_unknown(struct mock_param_formatter *formatter, + struct test_stream *stream, + const void *param) +{ + stream->add(stream, "%pS", param); +} + +struct mock_param_formatter unknown_formatter[] = { + { + .type_name = "<unknown>", + .format = mock_format_unknown, + }, + {}, +}; + +static int mock_register_all_formatters(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(integer_formatters); i++) + mock_register_formatter(&integer_formatters[i].formatter); + + mock_register_formatter(&string_formatter); + + return 0; +} +test_pure_initcall(mock_register_all_formatters); + +struct mock_struct_formatter { + struct mock_param_formatter formatter; + const char *type_name; + struct mock_struct_formatter_entry *entries; +}; + +static void mock_format_struct(struct mock_param_formatter *pformatter, + struct test_stream *stream, + const void *pparam) +{ + struct mock_struct_formatter *formatter = + container_of(pformatter, + struct mock_struct_formatter, + formatter); + struct mock_struct_formatter_entry *entry; + const char *param = CONVERT_TO_ACTUAL_TYPE(const char *, pparam); + const char *member_ptr; + + stream->add(stream, "%s {", formatter->type_name); + for (entry = formatter->entries; entry->formatter; entry++) { + member_ptr = param + entry->member_offset; + entry->formatter->format(entry->formatter, stream, member_ptr); + stream->add(stream, ", "); + } + stream->add(stream, "}"); +} + +struct mock_param_formatter *mock_struct_formatter( + struct test *test, + const char *type_name, + struct mock_struct_formatter_entry *entries) +{ + struct mock_struct_formatter *formatter; + + formatter = test_kzalloc(test, sizeof(*formatter), GFP_KERNEL); + if (!formatter) + return NULL; + + formatter->formatter.type_name = type_name; + formatter->formatter.format = mock_format_struct; + formatter->type_name = type_name; + formatter->entries = entries; + + return &formatter->formatter; +} diff --git a/kunit/mock.c b/kunit/mock.c index 424c612de553b..9be6b2d3621c4 100644 --- a/kunit/mock.c +++ b/kunit/mock.c @@ -186,15 +186,53 @@ int mock_set_default_action(struct mock *mock, return 0; }
+struct mock_param_formatter_repo { + struct list_head formatters; +}; + +static struct mock_param_formatter_repo mock_param_formatter_repo = { + .formatters = LIST_HEAD_INIT(mock_param_formatter_repo.formatters), +}; + +void mock_register_formatter(struct mock_param_formatter *formatter) +{ + list_add_tail(&formatter->node, &mock_param_formatter_repo.formatters); +} + +void mock_unregister_formatter(struct mock_param_formatter *formatter) +{ + list_del(&formatter->node); +} + +struct mock_param_formatter *mock_find_formatter(const char *type_name) +{ + struct mock_param_formatter *formatter; + + list_for_each_entry(formatter, + &mock_param_formatter_repo.formatters, + node) { + if (!strcmp(type_name, formatter->type_name)) + return formatter; + } + + return NULL; +} + static void mock_format_param(struct test_stream *stream, const char *type_name, const void *param) { - /* - * Cannot find formatter, so just print the pointer of the - * symbol. - */ - stream->add(stream, "<%pS>", param); + struct mock_param_formatter *formatter; + + formatter = mock_find_formatter(type_name); + if (formatter) + formatter->format(formatter, stream, param); + else + /* + * Cannot find formatter, so just print the pointer of the + * symbol. + */ + stream->add(stream, "<%pS>", param); }
static void mock_add_method_declaration_to_stream( diff --git a/kunit/test-stream-test.c b/kunit/test-stream-test.c index 875b0db15878d..b335e09805a0f 100644 --- a/kunit/test-stream-test.c +++ b/kunit/test-stream-test.c @@ -116,6 +116,7 @@ static void test_stream_test_commits_any_uncommitted_when_cleanup(
static int test_stream_test_init(struct test *test) { + struct mock_struct_formatter_entry *entries; struct test_stream_test_context *ctx;
ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); @@ -131,9 +132,35 @@ static int test_stream_test_init(struct test *test) if (!ctx->stream) return -ENOMEM;
+ entries = test_kzalloc(test, sizeof(*entries) * 3, GFP_KERNEL); + if (!entries) { + test_warn(test, + "Could not allocate arg formatter for struct va_format"); + return 0; + } + + INIT_MOCK_STRUCT_FORMATTER_ENTRY(&entries[0], + struct va_format, + fmt, + FORMATTER_FROM_TYPE(const char *)); + INIT_MOCK_STRUCT_FORMATTER_ENTRY(&entries[1], + struct va_format, + va, + unknown_formatter); + INIT_MOCK_STRUCT_FORMATTER_ENTRY_LAST(&entries[2]); + + mock_register_formatter(mock_struct_formatter(test, + "struct va_format *", + entries)); + return 0; }
+static void test_stream_test_exit(struct test *test) +{ + mock_unregister_formatter(mock_find_formatter("struct va_format *")); +} + static struct test_case test_stream_test_cases[] = { TEST_CASE(test_stream_test_add), TEST_CASE(test_stream_test_append), @@ -145,6 +172,7 @@ static struct test_case test_stream_test_cases[] = { static struct test_module test_stream_test_module = { .name = "test-stream-test", .init = test_stream_test_init, + .exit = test_stream_test_exit, .test_cases = test_stream_test_cases, }; module_test(test_stream_test_module);
Nice mocks only fail when there is an expectation on a method, but none match a given call. Strict mocks only pass when there is a matching expectation for every call. Naggy mocks have the same pass/fail behavior as nice, but report a warning in any case a strict mock would fail.
Signed-off-by: Felix Guo felixguoxiuping@gmail.com Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 63 +++++++++++++ kunit/mock-test.c | 192 ++++++++++++++++++++++++++++++++++++++- kunit/mock.c | 10 +- kunit/test-stream-test.c | 6 +- 4 files changed, 265 insertions(+), 6 deletions(-)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 4f85b39d628d0..8d155b27a257a 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -95,10 +95,17 @@ struct mock_method { struct list_head expectations; };
+enum mock_type { + MOCK_TYPE_NICE, + MOCK_TYPE_NAGGY, + MOCK_TYPE_STRICT +}; + struct mock { struct test_post_condition parent; struct test *test; struct list_head methods; + enum mock_type type; /* TODO(brendanhiggins@google.com): add locking to do_expect. */ const void *(*do_expect)(struct mock *mock, const char *method_name, @@ -108,6 +115,8 @@ struct mock { int len); };
+#define DEFAULT_MOCK_TYPE MOCK_TYPE_NAGGY + void mock_init_ctrl(struct test *test, struct mock *mock);
void mock_validate_expectations(struct mock *mock); @@ -137,6 +146,60 @@ void mock_unregister_formatter(struct mock_param_formatter *formatter);
#define MOCK(name) name##_mock
+/** + * STRICT_MOCK() - sets the mock to be strict and returns the mock + * @mock: the mock + * + * For an example, see ``The Nice, the Strict, and the Naggy`` under + * ``Using KUnit``. + */ +#define STRICT_MOCK(mock) \ +({ \ + mock_get_ctrl(mock)->type = MOCK_TYPE_STRICT; \ + mock; \ +}) + +static inline bool is_strict_mock(struct mock *mock) +{ + return mock->type == MOCK_TYPE_STRICT; +} + +/** + * NICE_MOCK() - sets the mock to be nice and returns the mock + * @mock: the mock + * + * For an example, see ``The Nice, the Strict, and the Naggy`` under + * ``Using KUnit``. + */ +#define NICE_MOCK(mock) \ +({ \ + mock_get_ctrl(mock)->type = MOCK_TYPE_NICE; \ + mock; \ +}) + +static inline bool is_nice_mock(struct mock *mock) +{ + return mock->type == MOCK_TYPE_NICE; +} + +/** + * NAGGY_MOCK() - sets the mock to be naggy and returns the mock + * @mock: the mock + * + * For an example, see ``The Nice, the Strict, and the Naggy`` under + * ``Using KUnit``. + */ +#define NAGGY_MOCK(mock) \ +({ \ + mock_get_ctrl(mock)->type = MOCK_TYPE_NAGGY; \ + mock; \ +}) + +static inline bool is_naggy_mock(struct mock *mock) +{ + return mock->type == MOCK_TYPE_NAGGY; +} + /** * TEST_EXPECT_CALL() - Declares a *call expectation* on a mock function. * @expectation_call: a mocked method or function with parameters replaced with diff --git a/kunit/mock-test.c b/kunit/mock-test.c index 77b16ad754424..675387743ada4 100644 --- a/kunit/mock-test.c +++ b/kunit/mock-test.c @@ -150,7 +150,7 @@ static void mock_test_failed_expect_call_fails_test(struct test *test) static void mock_test_do_expect_default_return(struct test *test) { struct mock_test_context *ctx = test->priv; - struct MOCK(test) *mock_test = ctx->mock_test; + struct MOCK(test) *mock_test = NICE_MOCK(ctx->mock_test); struct test *trgt = mock_get_trgt(mock_test); struct mock *mock = ctx->mock; int param0 = 5, param1 = -5; @@ -187,6 +187,49 @@ static void mock_test_do_expect_default_return(struct test *test) TEST_EXPECT_EQ(test, 0, expectation->times_called); }
+/** + * DOC: Testing the failure condition of different mock types. + * + * The following tests will test the behaviour of expectations under different + * conditions. For example, what happens when an expectation: + * - is not satisfied at the end of the test + * - is fulfilled but the expected function is called again + * - a function is called without expectations set on it + * + * For each of these conditions, there may be variations between the different + * types of mocks: nice mocks, naggy mocks (the default) and strict mocks. + * + * More information about these mocks can be found in the kernel documentation + * under Documentation/test/api/class-and-function-mocking + */ + +/* Method called on strict mock with no expectations will fail */ +static void mock_test_strict_no_expectations_will_fail(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_expectation *expectation; + + mock->type = MOCK_TYPE_STRICT; + + mock_set_default_action(mock, + "test_printk", + test_printk, + test_int_return(trgt, -4)); + + expectation = TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), + test_any(test))); + + mock->do_expect(mock, "test_printk", test_printk, two_param_types, + two_params, ARRAY_SIZE(two_params)); + mock_validate_expectations(mock); +} + /* * Method called on naggy mock with no expectations will not fail, but will show * a warning message @@ -202,6 +245,8 @@ static void mock_test_naggy_no_expectations_no_fail(struct test *test) const void *two_params[] = {¶m0, ¶m1}; struct mock_expectation *expectation;
+ mock->type = MOCK_TYPE_NAGGY; + mock_set_default_action(mock, "test_printk", test_printk, @@ -229,6 +274,93 @@ static void mock_test_naggy_no_expectations_no_fail(struct test *test) mock_validate_expectations(mock); }
+/* Method called on nice mock with no expectations will do nothing. */ +static void mock_test_nice_no_expectations_do_nothing(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_expectation *expectation; + + mock->type = MOCK_TYPE_NICE; + + mock_set_default_action(mock, + "test_printk", + test_printk, + test_int_return(trgt, -4)); + + expectation = TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), + test_any(test))); + expectation->min_calls_expected = 0; + expectation->max_calls_expected = 0; + + expectation = TEST_EXPECT_CALL(mock_vprintk(mock_get_ctrl(mock_test), + test_any(test), + test_any(test))); + expectation->min_calls_expected = 0; + expectation->max_calls_expected = 0; + + mock->do_expect(mock, + "test_printk", + test_printk, + two_param_types, + two_params, + ARRAY_SIZE(two_params)); + mock_validate_expectations(mock); +} + +/* Test that method called on a mock (of any type) with no matching expectations + * will fail test and print all the tried expectations. + */ +static void +run_method_called_but_no_matching_expectation_test(struct test *test, + enum mock_type mock_type) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + struct mock_expectation *handle; + struct mock_param_matcher *two_matchers[] = { + test_int_eq(trgt, 100), + test_int_eq(trgt, 100) + }; + mock_add_matcher(mock, "test_printk", test_printk, two_matchers, + ARRAY_SIZE(two_matchers)); + handle = TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), + test_any(test))); + + mock->type = mock_type; + + mock->do_expect(mock, "test_printk", test_printk, two_param_types, + two_params, ARRAY_SIZE(two_params)); +} + +static void mock_test_naggy_no_matching_expectations_fail(struct test *test) +{ + run_method_called_but_no_matching_expectation_test(test, + MOCK_TYPE_NAGGY); +} + +static void mock_test_strict_no_matching_expectations_fail(struct test *test) +{ + run_method_called_but_no_matching_expectation_test(test, + MOCK_TYPE_STRICT); +} + +static void mock_test_nice_no_matching_expectations_fail(struct test *test) +{ + run_method_called_but_no_matching_expectation_test(test, + MOCK_TYPE_NICE); +} + static void mock_test_mock_validate_expectations(struct test *test) { struct mock_test_context *ctx = test->priv; @@ -257,6 +389,58 @@ static void mock_test_mock_validate_expectations(struct test *test) mock_validate_expectations(mock); }
+static void mock_test_validate_clears_expectations(struct test *test) +{ + struct mock_test_context *ctx = test->priv; + struct MOCK(test) *mock_test = ctx->mock_test; + struct test *trgt = mock_get_trgt(mock_test); + struct mock *mock = ctx->mock; + struct mock_param_matcher *matchers[] = { + test_int_eq(trgt, 5), + test_int_eq(trgt, -4) + }; + int param0 = 5, param1 = -4; + static const char * const two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + + struct mock_expectation *expectation; + + mock->type = MOCK_TYPE_STRICT; + + /* If all goes well, the mock_test should not fail. */ + expectation = TEST_EXPECT_CALL(fail(mock_get_ctrl(mock_test), + test_any(test))); + expectation->min_calls_expected = 0; + expectation->max_calls_expected = 0; + + /* Add an arbitrary matcher for 0 calls */ + expectation = mock_add_matcher(mock, "test_printk", test_printk, + matchers, ARRAY_SIZE(matchers)); + expectation->times_called = 0; + expectation->min_calls_expected = 0; + expectation->max_calls_expected = 0; + + /* Should have 0 calls and should clear the previous expectation */ + mock_validate_expectations(mock); + + /* Add a new matcher for 1 call */ + expectation = mock_add_matcher(mock, "test_printk", test_printk, + matchers, ARRAY_SIZE(matchers)); + expectation->times_called = 0; + expectation->min_calls_expected = 1; + expectation->max_calls_expected = 1; + + /* Satisfy previous matcher */ + mock->do_expect(mock, "test_printk", test_printk, two_param_types, + two_params, ARRAY_SIZE(two_params)); + + /* + * Validate previous satisfy; if we didn't clear the previous + * expectation, it would fail the mock_test. + */ + mock_validate_expectations(mock); +} + void *do_mocked_fail(struct mock_action *this, const void **params, int len) { static const int ret; @@ -306,7 +490,13 @@ static struct test_case mock_test_cases[] = { TEST_CASE(mock_test_failed_expect_call_fails_test), TEST_CASE(mock_test_do_expect_default_return), TEST_CASE(mock_test_mock_validate_expectations), + TEST_CASE(mock_test_strict_no_expectations_will_fail), TEST_CASE(mock_test_naggy_no_expectations_no_fail), + TEST_CASE(mock_test_nice_no_expectations_do_nothing), + TEST_CASE(mock_test_strict_no_matching_expectations_fail), + TEST_CASE(mock_test_naggy_no_matching_expectations_fail), + TEST_CASE(mock_test_nice_no_matching_expectations_fail), + TEST_CASE(mock_test_validate_clears_expectations), {}, };
diff --git a/kunit/mock.c b/kunit/mock.c index 9be6b2d3621c4..314cebb54e236 100644 --- a/kunit/mock.c +++ b/kunit/mock.c @@ -79,6 +79,7 @@ void mock_init_ctrl(struct test *test, struct mock *mock) mock->test = test; INIT_LIST_HEAD(&mock->methods); mock->do_expect = mock_do_expect; + mock->type = DEFAULT_MOCK_TYPE; mock->parent.validate = mock_validate_wrapper; list_add_tail(&mock->parent.node, &test->post_conditions); } @@ -316,7 +317,12 @@ static struct mock_expectation *mock_apply_expectations( mock_add_method_expectation_error(test, stream, "Method was called with no expectations declared: ", mock, method, type_names, params, len); - stream->commit(stream); + if (is_strict_mock(mock)) + test->fail(test, stream); + else if (is_naggy_mock(mock)) + stream->commit(stream); + else + stream->clear(stream); return NULL; }
@@ -346,7 +352,7 @@ static struct mock_expectation *mock_apply_expectations( } }
- if (expectations_all_saturated) { + if (expectations_all_saturated && !is_nice_mock(mock)) { mock_add_method_expectation_error(test, stream, "Method was called with fully saturated expectations: ", mock, method, type_names, params, len); diff --git a/kunit/test-stream-test.c b/kunit/test-stream-test.c index b335e09805a0f..738a2692f7ba4 100644 --- a/kunit/test-stream-test.c +++ b/kunit/test-stream-test.c @@ -20,7 +20,7 @@ struct test_stream_test_context { static void test_stream_test_add(struct test *test) { struct test_stream_test_context *ctx = test->priv; - struct MOCK(test) *mock_test = ctx->mock_test; + struct MOCK(test) *mock_test = NICE_MOCK(ctx->mock_test); struct test_stream *stream = ctx->stream;
stream->add(stream, "Foo"); @@ -40,7 +40,7 @@ static void test_stream_test_add(struct test *test) static void test_stream_test_append(struct test *test) { struct test_stream_test_context *ctx = test->priv; - struct MOCK(test) *mock_test = ctx->mock_test; + struct MOCK(test) *mock_test = NICE_MOCK(ctx->mock_test); struct test_stream *stream = ctx->stream; struct test_stream *other_stream;
@@ -63,7 +63,7 @@ static void test_stream_test_append(struct test *test) static void test_stream_error_message_when_no_level_set(struct test *test) { struct test_stream_test_context *ctx = test->priv; - struct MOCK(test) *mock_test = ctx->mock_test; + struct MOCK(test) *mock_test = NICE_MOCK(ctx->mock_test); struct test_stream *stream = ctx->stream; struct test_stream *other_stream;
Adds ability to mock functions with a void* context object.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 53 +++++++++++++++++++++++++++++++++++++++++ kunit/mock-macro-test.c | 30 +++++++++++++++++++++++ kunit/mock.c | 9 +++++++ 3 files changed, 92 insertions(+)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 8d155b27a257a..89e95b3fcf09e 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -381,6 +381,24 @@ static inline bool is_naggy_mock(struct mock *mock) struct MOCK(struct_name) *MOCK_INIT_ID(struct_name)( \ struct test *test)
+#define DECLARE_VOID_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + handle_index, \ + return_type, \ + param_types...) \ + DECLARE_MOCK_COMMON(name, \ + handle_index, \ + return_type, \ + param_types) + +#define DECLARE_VOID_CLASS_MOCK_HANDLE_INDEX(name, \ + handle_index, \ + return_type, \ + param_types...) \ + DECLARE_VOID_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + handle_index, \ + return_type, \ + param_types) + /** * CONSTRUCT_MOCK() * @struct_name: name of the class @@ -631,6 +649,41 @@ static inline bool is_naggy_mock(struct mock *mock) return mock_obj; \ }
+struct MOCK(void) { + struct mock ctrl; + void *trgt; +}; + +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdiscarded-qualifiers" +static inline struct mock *from_void_ptr_to_mock(const void *ptr) +{ + struct MOCK(void) *mock_void_ptr = ptr; + + return mock_get_ctrl(mock_void_ptr); +} +#pragma GCC diagnostic pop + +#define DEFINE_VOID_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + handle_index, \ + return_type, \ + param_types...) \ + DEFINE_MOCK_COMMON(name, \ + handle_index, \ + from_void_ptr_to_mock, \ + return_type, \ + param_types) +#define DEFINE_VOID_CLASS_MOCK_HANDLE_INDEX(name, \ + handle_index, \ + return_type, \ + param_types...) \ + DEFINE_VOID_CLASS_MOCK_HANDLE_INDEX_INTERNAL(name, \ + handle_index, \ + return_type, \ + param_types) + +DECLARE_STRUCT_CLASS_MOCK_INIT(void); + #define CONVERT_TO_ACTUAL_TYPE(type, ptr) (*((type *) ptr))
/** diff --git a/kunit/mock-macro-test.c b/kunit/mock-macro-test.c index 84d9d3f484366..0f95105ec032a 100644 --- a/kunit/mock-macro-test.c +++ b/kunit/mock-macro-test.c @@ -48,8 +48,19 @@ static int test_struct_init(struct MOCK(test_struct) *mock_test_struct)
DEFINE_STRUCT_CLASS_MOCK_INIT(test_struct, test_struct_init);
+DECLARE_VOID_CLASS_MOCK_HANDLE_INDEX(METHOD(test_void_ptr_func), + HANDLE_INDEX(0), + RETURNS(int), + PARAMS(void*, int)); + +DEFINE_VOID_CLASS_MOCK_HANDLE_INDEX(METHOD(test_void_ptr_func), + HANDLE_INDEX(0), + RETURNS(int), + PARAMS(void*, int)); + struct mock_macro_context { struct MOCK(test_struct) *mock_test_struct; + struct MOCK(void) *mock_void_ptr; };
#define TO_STR_INTERNAL(...) #__VA_ARGS__ @@ -195,6 +206,20 @@ static void mock_macro_test_generated_method_code_works(struct test *test) test_struct->non_first_slot_param(5, test_struct); }
+static void mock_macro_test_generated_method_void_code_works(struct test *test) +{ + struct mock_macro_context *ctx = test->priv; + struct MOCK(void) *mock_void_ptr = ctx->mock_void_ptr; + struct mock_expectation *handle; + + handle = TEST_EXPECT_CALL(test_void_ptr_func( + mock_get_ctrl(mock_void_ptr), + test_int_eq(test, 3))); + handle->action = test_int_return(test, 0); + + test_void_ptr_func(mock_void_ptr, 3); +} + static int mock_macro_test_init(struct test *test) { struct mock_macro_context *ctx; @@ -208,6 +233,10 @@ static int mock_macro_test_init(struct test *test) if (!ctx->mock_test_struct) return -EINVAL;
+ ctx->mock_void_ptr = CONSTRUCT_MOCK(void, test); + if (!ctx->mock_void_ptr) + return -EINVAL; + return 0; }
@@ -220,6 +249,7 @@ static struct test_case mock_macro_test_cases[] = { TEST_CASE(mock_macro_param_list_from_types_basic), TEST_CASE(mock_macro_arg_names_from_types), TEST_CASE(mock_macro_test_generated_method_code_works), + TEST_CASE(mock_macro_test_generated_method_void_code_works), {}, };
diff --git a/kunit/mock.c b/kunit/mock.c index 314cebb54e236..7a9fcf6ae4a55 100644 --- a/kunit/mock.c +++ b/kunit/mock.c @@ -8,6 +8,15 @@
#include <kunit/mock.h>
+static int mock_void_ptr_init(struct MOCK(void) *mock_void_ptr) +{ + mock_void_ptr->trgt = mock_void_ptr; + + return 0; +} + +DEFINE_STRUCT_CLASS_MOCK_INIT(void, mock_void_ptr_init); + static bool mock_match_params(struct mock_matcher *matcher, struct test_stream *stream, const void **params,
Up to this point KUnit only supported method style function mocking where there was some type of class or context object and the function was only accessed via a pointer.
This adds support for mocking any function via the __mockable attribute.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 107 ++++++++++++++++++++++++++++++++++++++++ kunit/mock-macro-test.c | 14 ++++++ kunit/mock.c | 41 +++++++++++++++ 3 files changed, 162 insertions(+)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index 89e95b3fcf09e..b58e30ba02ce2 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -144,6 +144,8 @@ void mock_register_formatter(struct mock_param_formatter *formatter);
void mock_unregister_formatter(struct mock_param_formatter *formatter);
+struct mock *mock_get_global_mock(void); + #define MOCK(name) name##_mock
/** @@ -282,6 +284,12 @@ static inline bool is_naggy_mock(struct mock *mock) DECLARE_MOCK_CLIENT(name, return_type, param_types); \ DECLARE_MOCK_MASTER(name, handle_index, param_types)
+#define DECLARE_MOCK_FUNC_CLIENT(name, return_type, param_types...) \ + DECLARE_MOCK_CLIENT(name, return_type, param_types) + +#define DECLARE_MOCK_FUNC_MASTER(name, param_types...) \ + DECLARE_MOCK_MASTER(name, MOCK_MAX_PARAMS, param_types) + #define DECLARE_STRUCT_CLASS_MOCK_STRUCT(struct_name) \ struct MOCK(struct_name) { \ struct mock ctrl; \ @@ -411,6 +419,16 @@ static inline bool is_naggy_mock(struct mock *mock) */ #define CONSTRUCT_MOCK(struct_name, test) MOCK_INIT_ID(struct_name)(test)
+#define DECLARE_FUNCTION_MOCK_INTERNAL(name, return_type, param_types...) \ + DECLARE_MOCK_FUNC_CLIENT(name, return_type, param_types); \ + DECLARE_MOCK_FUNC_MASTER(name, param_types); + +#define DECLARE_FUNCTION_MOCK(name, return_type, param_types...) \ + DECLARE_FUNCTION_MOCK_INTERNAL(name, return_type, param_types) + +#define DECLARE_FUNCTION_MOCK_VOID_RETURN(name, param_types...) \ + DECLARE_FUNCTION_MOCK(name, void, param_types) + #define DEFINE_MOCK_CLIENT_COMMON(name, \ handle_index, \ MOCK_SOURCE, \ @@ -488,6 +506,31 @@ static inline bool is_naggy_mock(struct mock *mock) NO_RETURN, \ param_types)
+#define FUNC_MOCK_SOURCE(ctx, handle_index) mock_get_global_mock() +#define DEFINE_MOCK_FUNC_CLIENT_COMMON(name, \ + return_type, \ + RETURN, \ + param_types...) \ + DEFINE_MOCK_CLIENT_COMMON(name, \ + MOCK_MAX_PARAMS, \ + FUNC_MOCK_SOURCE, \ + name, \ + return_type, \ + RETURN, \ + param_types) + +#define DEFINE_MOCK_FUNC_CLIENT(name, return_type, param_types...) \ + DEFINE_MOCK_FUNC_CLIENT_COMMON(name, \ + return_type, \ + CAST_AND_RETURN, \ + param_types) + +#define DEFINE_MOCK_FUNC_CLIENT_VOID_RETURN(name, param_types...) \ + DEFINE_MOCK_FUNC_CLIENT_COMMON(name, \ + void, \ + NO_RETURN, \ + param_types) + #define DEFINE_MOCK_MASTER_COMMON_INTERNAL(name, \ ctrl_index, \ MOCK_SOURCE, \ @@ -522,6 +565,13 @@ static inline bool is_naggy_mock(struct mock *mock) CLASS_MOCK_MASTER_SOURCE, \ param_types)
+#define FUNC_MOCK_CLIENT_SOURCE(ctrl_index) mock_get_global_mock() +#define DEFINE_MOCK_FUNC_MASTER(name, param_types...) \ + DEFINE_MOCK_MASTER_COMMON(name, \ + MOCK_MAX_PARAMS, \ + FUNC_MOCK_CLIENT_SOURCE, \ + param_types) + #define DEFINE_MOCK_COMMON(name, \ handle_index, \ mock_converter, \ @@ -684,6 +734,63 @@ static inline struct mock *from_void_ptr_to_mock(const void *ptr)
DECLARE_STRUCT_CLASS_MOCK_INIT(void);
+#define DEFINE_FUNCTION_MOCK_INTERNAL(name, return_type, param_types...) \ + DEFINE_MOCK_FUNC_CLIENT(name, return_type, param_types); \ + DEFINE_MOCK_FUNC_MASTER(name, param_types) + +/** + * DEFINE_FUNCTION_MOCK() + * @name: name of the function + * @return_type: return type of the function + * @...: parameter types of the function + * + * Same as DEFINE_STRUCT_CLASS_MOCK() except can be used to mock any function + * declared %__mockable or DEFINE_REDIRECT_MOCKABLE() + */ +#define DEFINE_FUNCTION_MOCK(name, return_type, param_types...) \ + DEFINE_FUNCTION_MOCK_INTERNAL(name, return_type, param_types) + +#define DEFINE_FUNCTION_MOCK_VOID_RETURN_INTERNAL(name, param_types...) \ + DEFINE_MOCK_FUNC_CLIENT_VOID_RETURN(name, param_types); \ + DEFINE_MOCK_FUNC_MASTER(name, param_types) + +/** + * DEFINE_FUNCTION_MOCK_VOID_RETURN() + * @name: name of the function + * @...: parameter types of the function + * + * Same as DEFINE_FUNCTION_MOCK() except the method has a ``void`` return + * type. + */ +#define DEFINE_FUNCTION_MOCK_VOID_RETURN(name, param_types...) \ + DEFINE_FUNCTION_MOCK_VOID_RETURN_INTERNAL(name, param_types) + +#if IS_ENABLED(CONFIG_KUNIT) + +/** + * __mockable - A function decorator that allows the function to be mocked. + * + * Example: + * + * .. code-block:: c + * + * int __mockable example(int arg) { ... } + */ +#define __mockable __weak + +/** + * __visible_for_testing - Makes a static function visible when testing. + * + * A macro that replaces the `static` specifier on functions and global + * variables that is static when compiled normally and visible when compiled for + * tests. + */ +#define __visible_for_testing +#else +#define __mockable +#define __visible_for_testing static +#endif + #define CONVERT_TO_ACTUAL_TYPE(type, ptr) (*((type *) ptr))
/** diff --git a/kunit/mock-macro-test.c b/kunit/mock-macro-test.c index 0f95105ec032a..a2628a70bc4e4 100644 --- a/kunit/mock-macro-test.c +++ b/kunit/mock-macro-test.c @@ -58,6 +58,8 @@ DEFINE_VOID_CLASS_MOCK_HANDLE_INDEX(METHOD(test_void_ptr_func), RETURNS(int), PARAMS(void*, int));
+DEFINE_FUNCTION_MOCK(add, RETURNS(int), PARAMS(int, int)); + struct mock_macro_context { struct MOCK(test_struct) *mock_test_struct; struct MOCK(void) *mock_void_ptr; @@ -220,6 +222,17 @@ static void mock_macro_test_generated_method_void_code_works(struct test *test) test_void_ptr_func(mock_void_ptr, 3); }
+static void mock_macro_test_generated_function_code_works(struct test *test) +{ + struct mock_expectation *handle; + + handle = TEST_EXPECT_CALL(add(test_int_eq(test, 4), + test_int_eq(test, 3))); + handle->action = test_int_return(test, 7); + + TEST_EXPECT_EQ(test, 7, add(4, 3)); +} + static int mock_macro_test_init(struct test *test) { struct mock_macro_context *ctx; @@ -250,6 +263,7 @@ static struct test_case mock_macro_test_cases[] = { TEST_CASE(mock_macro_arg_names_from_types), TEST_CASE(mock_macro_test_generated_method_code_works), TEST_CASE(mock_macro_test_generated_method_void_code_works), + TEST_CASE(mock_macro_test_generated_function_code_works), {}, };
diff --git a/kunit/mock.c b/kunit/mock.c index 7a9fcf6ae4a55..2b91ea08b6064 100644 --- a/kunit/mock.c +++ b/kunit/mock.c @@ -93,6 +93,47 @@ void mock_init_ctrl(struct test *test, struct mock *mock) list_add_tail(&mock->parent.node, &test->post_conditions); }
+struct global_mock { + struct mock ctrl; + bool is_initialized; +}; + +static struct global_mock global_mock = { + .is_initialized = false, +}; + +static int mock_init_global_mock(struct test_initcall *initcall, + struct test *test) +{ + BUG_ON(global_mock.is_initialized); + + mock_init_ctrl(test, &global_mock.ctrl); + global_mock.is_initialized = true; + + return 0; +} + +static void mock_exit_global_mock(struct test_initcall *initcall) +{ + BUG_ON(!global_mock.is_initialized); + + global_mock.ctrl.test = NULL; + global_mock.is_initialized = false; +} + +static struct test_initcall global_mock_initcall = { + .init = mock_init_global_mock, + .exit = mock_exit_global_mock, +}; +test_register_initcall(global_mock_initcall); + +struct mock *mock_get_global_mock(void) +{ + BUG_ON(!global_mock.is_initialized); + + return &global_mock.ctrl; +} + static struct mock_method *mock_lookup_method(struct mock *mock, const void *method_ptr) {
Adds the concept of spying like in Mockito (http://static.javadoc.io/org.mockito/mockito-core/2.20.0/org/mockito/Mockito...). This allows a function declaration to be labled as spyable which allows the function to be mocked *and* to allow the mock to invoke the original function definition.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 123 ++++++++++++++++++++++++++++++++++++++++++- kunit/common-mocks.c | 36 +++++++++++++ 2 files changed, 158 insertions(+), 1 deletion(-)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index b58e30ba02ce2..c3615e80d96ee 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -284,6 +284,13 @@ static inline bool is_naggy_mock(struct mock *mock) DECLARE_MOCK_CLIENT(name, return_type, param_types); \ DECLARE_MOCK_MASTER(name, handle_index, param_types)
+#define DECLARE_SPYABLE(name, return_type, param_types...) \ + return_type REAL_ID(name)(param_types); \ + return_type name(param_types); \ + void *INVOKE_ID(name)(struct test *test, \ + const void *params[], \ + int len) + #define DECLARE_MOCK_FUNC_CLIENT(name, return_type, param_types...) \ DECLARE_MOCK_CLIENT(name, return_type, param_types)
@@ -465,6 +472,100 @@ static inline bool is_naggy_mock(struct mock *mock) RETURN(return_type, retval); \ }
+#if IS_ENABLED(CONFIG_KUNIT) +#define DEFINE_INVOKABLE(name, return_type, RETURN_ASSIGN, param_types...) \ + void *INVOKE_ID(name)(struct test *test, \ + const void *params[], \ + int len) { \ + return_type *retval; \ + \ + TEST_ASSERT_EQ(test, NUM_VA_ARGS(param_types), len); \ + retval = test_kzalloc(test, \ + sizeof(*retval), \ + GFP_KERNEL); \ + TEST_ASSERT_NOT_ERR_OR_NULL(test, retval); \ + RETURN_ASSIGN() REAL_ID(name)( \ + ARRAY_ACCESSORS_FROM_TYPES( \ + param_types)); \ + return retval; \ + } +#else +#define DEFINE_INVOKABLE(name, return_type, RETURN_ASSIGN, param_types...) +#endif + +#define DEFINE_SPYABLE_COMMON(name, \ + return_type, \ + RETURN_ASSIGN, \ + param_types...) \ + return_type REAL_ID(name)(param_types); \ + return_type name(param_types) __mockable_alias(REAL_ID(name)); \ + DEFINE_INVOKABLE(name, return_type, RETURN_ASSIGN, param_types); + +#define ASSIGN() *retval = + +/** + * DEFINE_SPYABLE() + * @name: name of the function + * @return_type: return type of the function + * @param_types: parameter types of the function + * + * Used to define a function which is *redirect-mockable*, which allows the + * function to be mocked and refer to the original definition via + * INVOKE_REAL(). + * + * Example: + * + * .. code-block:: c + * + * DEFINE_SPYABLE(i2c_add_adapter, + * RETURNS(int), PARAMS(struct i2c_adapter *)); + * int REAL_ID(i2c_add_adapter)(struct i2c_adapter *adapter) + * { + * ... + * } + * + * static int aspeed_i2c_test_init(struct test *test) + * { + * struct mock_param_capturer *adap_capturer; + * struct mock_expectation *handle; + * struct aspeed_i2c_test *ctx; + * int ret; + * + * ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + * if (!ctx) + * return -ENOMEM; + * test->priv = ctx; + * + * handle = TEST_EXPECT_CALL( + * i2c_add_adapter(capturer_to_matcher( + * adap_capturer))); + * handle->action = INVOKE_REAL(test, i2c_add_adapter); + * ret = of_fake_probe_platform_by_name(test, + * "aspeed-i2c-bus", + * "test-i2c-bus"); + * if (ret < 0) + * return ret; + * + * TEST_ASSERT_PARAM_CAPTURED(test, adap_capturer); + * ctx->adap = mock_capturer_get(adap_capturer, + * struct i2c_adapter *); + * + * return 0; + * } + */ +#define DEFINE_SPYABLE(name, return_type, param_types...) \ + DEFINE_SPYABLE_COMMON(name, \ + return_type, \ + ASSIGN, \ + param_types) + +#define NO_ASSIGN() +#define DEFINE_SPYABLE_VOID_RETURN(name, param_types) \ + DEFINE_SPYABLE_COMMON(name, \ + void, \ + NO_ASSIGN, \ + param_types) + #define CLASS_MOCK_CLIENT_SOURCE(ctx, handle_index) ctx(arg##handle_index) #define DEFINE_MOCK_METHOD_CLIENT_COMMON(name, \ handle_index, \ @@ -745,7 +846,7 @@ DECLARE_STRUCT_CLASS_MOCK_INIT(void); * @...: parameter types of the function * * Same as DEFINE_STRUCT_CLASS_MOCK() except can be used to mock any function - * declared %__mockable or DEFINE_REDIRECT_MOCKABLE() + * declared %__mockable or DEFINE_SPYABLE() */ #define DEFINE_FUNCTION_MOCK(name, return_type, param_types...) \ DEFINE_FUNCTION_MOCK_INTERNAL(name, return_type, param_types) @@ -777,6 +878,7 @@ DECLARE_STRUCT_CLASS_MOCK_INIT(void); * int __mockable example(int arg) { ... } */ #define __mockable __weak +#define __mockable_alias(id) __weak __alias(id)
/** * __visible_for_testing - Makes a static function visible when testing. @@ -788,6 +890,7 @@ DECLARE_STRUCT_CLASS_MOCK_INIT(void); #define __visible_for_testing #else #define __mockable +#define __mockable_alias(id) __alias(id) #define __visible_for_testing static #endif
@@ -1069,6 +1172,24 @@ struct mock_param_matcher *test_struct_cmp( const char *struct_name, struct mock_struct_matcher_entry *entries);
+struct mock_action *invoke(struct test *test, + void *(*invokable)(struct test *, + const void *params[], + int len)); + +/** + * INVOKE_REAL() + * @test: associated test + * @func_name: name of the function + * + * See DEFINE_SPYABLE() for an example. + * + * Return: &struct mock_action that makes the associated mock method or function + * call the original function definition of a redirect-mockable + * function. + */ +#define INVOKE_REAL(test, func_name) invoke(test, INVOKE_ID(func_name)) + struct mock_struct_formatter_entry { size_t member_offset; struct mock_param_formatter *formatter; diff --git a/kunit/common-mocks.c b/kunit/common-mocks.c index 1c52522808cab..ce0159923814d 100644 --- a/kunit/common-mocks.c +++ b/kunit/common-mocks.c @@ -386,6 +386,42 @@ DEFINE_RETURN_ACTION_WITH_TYPENAME(longlong, long long); DEFINE_RETURN_ACTION_WITH_TYPENAME(ulonglong, unsigned long long); DEFINE_RETURN_ACTION_WITH_TYPENAME(ptr, void *);
+struct mock_invoke_action { + struct mock_action action; + struct test *test; + void *(*invokable)(struct test *test, const void *params[], int len); +}; + +static void *do_invoke(struct mock_action *paction, + const void *params[], + int len) +{ + struct mock_invoke_action *action = + container_of(paction, + struct mock_invoke_action, + action); + + return action->invokable(action->test, params, len); +} + +struct mock_action *invoke(struct test *test, + void *(*invokable)(struct test *, + const void *params[], + int len)) +{ + struct mock_invoke_action *action; + + action = test_kmalloc(test, sizeof(*action), GFP_KERNEL); + if (!action) + return NULL; + + action->action.do_action = do_invoke; + action->test = test; + action->invokable = invokable; + + return &action->action; +} + struct mock_param_integer_formatter { struct mock_param_formatter formatter; const char *fmt_str;
On Tue, Oct 16, 2018 at 6:54 PM Brendan Higgins brendanhiggins@google.com wrote:
Adds the concept of spying like in Mockito (http://static.javadoc.io/org.mockito/mockito-core/2.20.0/org/mockito/Mockito...). This allows a function declaration to be labled as spyable which allows the function to be mocked *and* to allow the mock to invoke the original function definition.
We can already hook into arbitrary functions ftrace. Wouldn't utilizing that simplify features like this and avoid having to touch existing code for testing? Not sure what it would take to enable ftrace on UML. It is at least partially compiler dependent.
Rob
On Wed, Oct 17, 2018 at 3:47 PM Rob Herring rob.herring@linaro.org wrote:
On Tue, Oct 16, 2018 at 6:54 PM Brendan Higgins brendanhiggins@google.com wrote:
Adds the concept of spying like in Mockito (http://static.javadoc.io/org.mockito/mockito-core/2.20.0/org/mockito/Mockito...). This allows a function declaration to be labled as spyable which allows the function to be mocked *and* to allow the mock to invoke the original function definition.
We can already hook into arbitrary functions ftrace. Wouldn't utilizing that simplify features like this and avoid having to touch existing code for testing? Not sure what it would take to enable ftrace on UML. It is at least partially compiler dependent.
Regardless of what we end up doing, this definitely needs more work.
After looking at include/linux/ftrace.h, it does look feasible to use ftrace for spying. I totally agree that we don't want to reinvent this wheel if we can avoid it, but I have no idea what getting it working on UML would look like. Dependence on compiler features could be an issue: pretty much anyone who can build the kernel should be able to run our tests.
Adds the concept of an argument capturer which, when used with a matcher in an EXPECT_CALL(...), will capture the value of the matching argument.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- include/kunit/mock.h | 83 ++++++++++++++++++++++++++++++++++++++++++++ kunit/common-mocks.c | 78 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 161 insertions(+)
diff --git a/include/kunit/mock.h b/include/kunit/mock.h index c3615e80d96ee..0e1aa568709a1 100644 --- a/include/kunit/mock.h +++ b/include/kunit/mock.h @@ -1172,6 +1172,89 @@ struct mock_param_matcher *test_struct_cmp( const char *struct_name, struct mock_struct_matcher_entry *entries);
+/** + * struct mock_param_capturer - used to capture parameter when matching + * + * Use the associated helper macros to access relevant fields. + * Example: + * + * .. code-block::c + * + * static int some_test(struct test *test) + * { + * // imagine a mocked function: int add(int a, int b) + * struct mock_param_capturer *capturer = + * mock_int_capturer_create(test, any(test)); + * TEST_EXPECT_CALL(add(any(test), capturer_to_matcher(capturer))); + * TEST_ASSERT_PARAM_CAPTURED(test, capturer); + * + * int captured_value = mock_capturer_get(capturer, int); + * } + */ +struct mock_param_capturer { + /* private: internal use only. */ + struct mock_param_matcher matcher; + struct mock_param_matcher *child_matcher; + void *(*capture_param)(struct test *test, const void *param); + void *captured_param; +}; + +struct mock_param_capturer *mock_param_capturer_create( + struct test *test, + struct mock_param_matcher *child_matcher, + void *(*capture_param)(struct test *, const void *)); + +/** + * mock_int_capturer_create() - creates a int parameter capturer + * @test: associated test + * @child_matcher: matcher used to match the integer + * + * The capturer will capture the value if the matcher is satisfied. + */ +struct mock_param_capturer *mock_int_capturer_create( + struct test *test, struct mock_param_matcher *child_matcher); + +/** + * mock_int_capturer_create() - creates a generic pointer parameter capturer + * @test: associated test + * @child_matcher: matcher used to match the pointer + * + * The capturer will capture the value if the matcher is satisfied + */ +struct mock_param_capturer *mock_ptr_capturer_create( + struct test *test, struct mock_param_matcher *child_matcher); + +/** + * capturer_to_matcher() + * @capturer: the param capturer + * + * Use this function when passing a capturer into an EXPECT_CALL() where a + * matcher would be expected. See the example for &struct mock_param_capturer. + */ +#define capturer_to_matcher(capturer) (&(capturer)->matcher) + +/** + * TEST_ASSERT_PARAM_CAPTURED(): Asserts that a parameter has been captured. + * @test: the associated test + * @capturer: the param capturer + * + * See &struct mock_param_capturer for an example. + */ +#define TEST_ASSERT_PARAM_CAPTURED(test, capturer) \ + TEST_ASSERT(test, \ + !IS_ERR_OR_NULL((capturer)->captured_param), \ + "Asserted " #capturer " captured param, but did not.") + +/** + * mock_capturer_get(): Returns the value captured by ``capturer`` + * @capturer: the param capturer + * @type: the type of the value + * + * See &struct mock_param_capturer for an example. + */ +#define mock_capturer_get(capturer, type) \ + CONVERT_TO_ACTUAL_TYPE(type, (capturer)->captured_param) + struct mock_action *invoke(struct test *test, void *(*invokable)(struct test *, const void *params[], diff --git a/kunit/common-mocks.c b/kunit/common-mocks.c index ce0159923814d..62528b7df83c6 100644 --- a/kunit/common-mocks.c +++ b/kunit/common-mocks.c @@ -323,6 +323,84 @@ struct mock_param_matcher *test_struct_cmp( return &matcher->matcher; }
+static bool match_and_capture_param(struct mock_param_matcher *pmatcher, + struct test_stream *stream, + const void *param) +{ + struct mock_param_capturer *capturer = + container_of(pmatcher, + struct mock_param_capturer, + matcher); + struct mock_param_matcher *child_matcher = capturer->child_matcher; + bool matches; + + matches = child_matcher->match(child_matcher, stream, param); + if (matches) + capturer->captured_param = capturer->capture_param(stream->test, + param); + + return matches; +} + +struct mock_param_capturer *mock_param_capturer_create( + struct test *test, + struct mock_param_matcher *child_matcher, + void *(*capture_param)(struct test *, const void *)) +{ + struct mock_param_capturer *capturer; + + capturer = test_kzalloc(test, sizeof(*capturer), GFP_KERNEL); + if (!capturer) + return NULL; + + capturer->matcher.match = match_and_capture_param; + capturer->child_matcher = child_matcher; + capturer->capture_param = capture_param; + capturer->captured_param = NULL; + + return capturer; +} + +static void *mock_capture_int(struct test *test, const void *param) +{ + int value = CONVERT_TO_ACTUAL_TYPE(int, param); + int *pvalue; + + pvalue = test_kzalloc(test, sizeof(*pvalue), GFP_KERNEL); + if (!pvalue) + return NULL; + *pvalue = value; + + return pvalue; +} + +struct mock_param_capturer *mock_int_capturer_create( + struct test *test, struct mock_param_matcher *child_matcher) +{ + return mock_param_capturer_create(test, + child_matcher, + mock_capture_int); +} + +static void *mock_capture_ptr(struct test *test, const void *param) +{ + void *ptr = CONVERT_TO_ACTUAL_TYPE(void *, param); + void **pptr; + + pptr = test_kzalloc(test, sizeof(*pptr), GFP_KERNEL); + *pptr = ptr; + + return pptr; +} + +struct mock_param_capturer *mock_ptr_capturer_create( + struct test *test, struct mock_param_matcher *child_matcher) +{ + return mock_param_capturer_create(test, + child_matcher, + mock_capture_ptr); +} + #define DEFINE_RETURN_ACTION_STRUCT(type_name, type) \ struct mock_##type_name##_action { \ struct mock_action action; \
Sacrificed the control of printing stack trace within the crash handler in the test runner for getting a better stack trace; this is still not ideal, but much better than before.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- arch/um/kernel/trap.c | 11 ++++++++++- kunit/test.c | 13 +++++++++---- 2 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c index 9b97712daf14f..c3ff8346800c4 100644 --- a/arch/um/kernel/trap.c +++ b/arch/um/kernel/trap.c @@ -226,8 +226,17 @@ unsigned long segv(struct faultinfo fi, unsigned long ip, int is_user, current->thread.segv_regs = container_of(regs, struct pt_regs, regs);
catcher = current->thread.fault_catcher; - if (catcher && current->thread.is_running_test) + if (catcher && current->thread.is_running_test) { + /* + * TODO(b/77223210): Right now we don't have a way to store a + * copy of the stack, or a copy of information from the stack, + * so we need to print it now; otherwise, the stack will be + * destroyed by segv_run_catcher which works by popping off + * stack frames. + */ + show_stack(NULL, NULL); segv_run_catcher(catcher, (void *) address); + } else if (!is_user && (address >= start_vm) && (address < end_vm)) { flush_tlb_kernel_vm(); goto out; diff --git a/kunit/test.c b/kunit/test.c index 6ea60059b4918..5d78f76b421af 100644 --- a/kunit/test.c +++ b/kunit/test.c @@ -210,12 +210,17 @@ static void test_handle_test_crash(struct test *test, struct test_module *module, struct test_case *test_case) { - test_err(test, "%s crashed", test_case->name); /* - * TODO(brendanhiggins@google.com): This prints the stack trace up - * through this frame, not up to the frame that caused the crash. + * TODO(brendanhiggins@google.com): Right now we don't have a way to + * store a copy of the stack, or a copy of information from the stack, + * so we need to print it in the "trap" handler; otherwise, the stack + * will be destroyed when it returns to us by popping off the + * appropriate stack frames (see longjmp). + * + * Ideally we would print the stack trace here, but we do not have the + * ability to do so with meaningful information at this time. */ - show_stack(NULL, NULL); + test_err(test, "%s crashed", test_case->name);
test_case_internal_cleanup(test); }
Platform mocking is the mocking of all platform specific functions that interact directly with hardware. In effect, this provides the ability to mock any hardware behavior.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- drivers/base/Makefile | 1 + drivers/base/platform-mock.c | 65 ++++++++++++++++++++++++++++ include/linux/platform_device_mock.h | 64 +++++++++++++++++++++++++++ 3 files changed, 130 insertions(+) create mode 100644 drivers/base/platform-mock.c create mode 100644 include/linux/platform_device_mock.h
diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 704f442958103..77cc599daa020 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -23,6 +23,7 @@ obj-$(CONFIG_PINCTRL) += pinctrl.o obj-$(CONFIG_DEV_COREDUMP) += devcoredump.o obj-$(CONFIG_GENERIC_MSI_IRQ_DOMAIN) += platform-msi.o obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o +obj-$(CONFIG_PLATFORM_MOCK) += platform-mock.o
obj-y += test/
diff --git a/drivers/base/platform-mock.c b/drivers/base/platform-mock.c new file mode 100644 index 0000000000000..3df9f1b0bb50f --- /dev/null +++ b/drivers/base/platform-mock.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Fake platform device API for unit testing platform drivers. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/platform_device_mock.h> +#include <linux/of_platform.h> + +struct device_node *of_fake_node(struct test *test, const char *name) +{ + struct device_node *node; + + node = test_kzalloc(test, sizeof(*node), GFP_KERNEL); + if (!node) + return NULL; + + of_node_init(node); + + return node; +} + +struct platform_device * +of_fake_probe_platform(struct test *test, + const struct platform_driver *driver, + const char *node_name) +{ + struct platform_device *pdev; + struct device_node *of_node; + int ret; + + of_node = of_fake_node(test, node_name); + if (!of_node) + return ERR_PTR(-ENOMEM); + + test_info(test, "Creating device"); + pdev = of_platform_device_create(of_node, node_name, NULL); + if (!pdev) + return ERR_PTR(-ENODEV); + + test_info(test, "Probing"); + ret = driver->probe(pdev); + if (ret) + return ERR_PTR(ret); + + return pdev; +} + +struct platform_device *of_fake_probe_platform_by_name(struct test *test, + const char *driver_name, + const char *node_name) +{ + const struct device_driver *driver; + + test_info(test, "Locating driver by name"); + driver = driver_find(driver_name, &platform_bus_type); + if (!driver) + return ERR_PTR(-ENODEV); + + return of_fake_probe_platform(test, + to_platform_driver(driver), + node_name); +} diff --git a/include/linux/platform_device_mock.h b/include/linux/platform_device_mock.h new file mode 100644 index 0000000000000..898539d166f66 --- /dev/null +++ b/include/linux/platform_device_mock.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Fake platform device API for unit testing platform drivers. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/platform_device.h> +#include <kunit/mock.h> + +static inline struct platform_driver *platform_driver_find(const char *name) +{ + struct device_driver *driver; + + driver = driver_find(name, &platform_bus_type); + if (!driver) + return NULL; + + return to_platform_driver(driver); +} + +/** + * of_fake_node() + * @test: the test to associate node with + * @name: name of the node + * + * The &struct device_node returned is allocated as a root node with the given + * name and otherwise behaves as a real &struct device_node. + * + * Returns: the faked &struct device_node + */ +struct device_node *of_fake_node(struct test *test, const char *name); + +/** + * of_fake_probe_platform() + * @test: the test to associate the fake platform device with + * @driver: driver to probe + * @node_name: name of the device node created + * + * Creates a &struct platform_device and an associated &struct device_node, + * probes the provided &struct platform_driver with the &struct platform_device. + * + * Returns: the &struct platform_device that was created + */ +struct platform_device * +of_fake_probe_platform(struct test *test, + const struct platform_driver *driver, + const char *node_name); + +/** + * of_fake_probe_platform_by_name() + * @test: the test to associate the fake platform device with + * @driver_name: name of the driver to probe + * @node_name: name of the device node created + * + * Same as of_fake_probe_platform() but looks up the &struct platform_driver by + * the provided name. + * + * Returns: the &struct platform_device that was created + */ +struct platform_device *of_fake_probe_platform_by_name(struct test *test, + const char *driver_name, + const char *node_name);
This mocks out some iomem functions (functions like readl and writel), for mocking hardware interfaces.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- arch/um/Kconfig.common | 8 +++++- arch/um/Kconfig.um | 5 ++++ arch/um/include/asm/Kbuild | 1 - arch/um/include/asm/io-mock-shared.h | 33 +++++++++++++++++++++ arch/um/include/asm/io-mock.h | 43 ++++++++++++++++++++++++++++ arch/um/include/asm/io.h | 8 ++++++ arch/um/kernel/Makefile | 1 + arch/um/kernel/io-mock.c | 40 ++++++++++++++++++++++++++ kunit/Kconfig | 1 + 9 files changed, 138 insertions(+), 2 deletions(-) create mode 100644 arch/um/include/asm/io-mock-shared.h create mode 100644 arch/um/include/asm/io-mock.h create mode 100644 arch/um/kernel/io-mock.c
diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common index 07f84c842cc31..72e7efb74f7fd 100644 --- a/arch/um/Kconfig.common +++ b/arch/um/Kconfig.common @@ -19,7 +19,13 @@ config MMU default y
config NO_IOMEM - def_bool y + bool + default y if !KUNIT + +config HAS_IOMEM + bool "Turns on fake IOMEM support for KUnit" + depends on KUNIT + select MOCK_IOMEM
config ISA bool diff --git a/arch/um/Kconfig.um b/arch/um/Kconfig.um index 20da5a8ca9490..8d35e0e2c23d1 100644 --- a/arch/um/Kconfig.um +++ b/arch/um/Kconfig.um @@ -122,3 +122,8 @@ config SECCOMP defined by each seccomp mode.
If unsure, say Y. + +config PLATFORM_MOCK + bool "Enable a mock architecture used for unit testing." + depends on KUNIT && OF + default n diff --git a/arch/um/include/asm/Kbuild b/arch/um/include/asm/Kbuild index b10dde6cb793b..9fd2827ab76d1 100644 --- a/arch/um/include/asm/Kbuild +++ b/arch/um/include/asm/Kbuild @@ -12,7 +12,6 @@ generic-y += ftrace.h generic-y += futex.h generic-y += hardirq.h generic-y += hw_irq.h -generic-y += io.h generic-y += irq_regs.h generic-y += irq_work.h generic-y += kdebug.h diff --git a/arch/um/include/asm/io-mock-shared.h b/arch/um/include/asm/io-mock-shared.h new file mode 100644 index 0000000000000..6baf59cb17a58 --- /dev/null +++ b/arch/um/include/asm/io-mock-shared.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_UM_IO_MOCK_SHARED_H +#define _ASM_UM_IO_MOCK_SHARED_H + +#define readb readb +u8 readb(const volatile void __iomem *); + +#define readw readw +u16 readw(const volatile void __iomem *); + +#define readl readl +u32 readl(const volatile void __iomem *); + +#ifdef CONFIG_64BIT +#define readq readq +u64 readq(const volatile void __iomem *); +#endif /* CONFIG_64BIT */ + +#define writeb writeb +void writeb(u8, const volatile void __iomem *); + +#define writew writew +void writew(u16, const volatile void __iomem *); + +#define writel writel +void writel(u32, const volatile void __iomem *); + +#ifdef CONFIG_64BIT +#define writeq writeq +void writeq(u64, const volatile void __iomem *); +#endif /* CONFIG_64BIT */ + +#endif /* _ASM_UM_IO_MOCK_SHARED_H */ diff --git a/arch/um/include/asm/io-mock.h b/arch/um/include/asm/io-mock.h new file mode 100644 index 0000000000000..bdc5cd1d4e33c --- /dev/null +++ b/arch/um/include/asm/io-mock.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Mock IO functions. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#ifndef _ASM_UM_IO_MOCK_H +#define _ASM_UM_IO_MOCK_H + +#include <asm/io-mock-shared.h> +#include <kunit/mock.h> + +DECLARE_FUNCTION_MOCK(readb, + RETURNS(u8), PARAMS(const volatile void __iomem *)); + +DECLARE_FUNCTION_MOCK(readw, + RETURNS(u16), PARAMS(const volatile void __iomem *)); + +DECLARE_FUNCTION_MOCK(readl, + RETURNS(u32), PARAMS(const volatile void __iomem *)); + +#ifdef CONFIG_64BIT +DECLARE_FUNCTION_MOCK(readq, + RETURNS(u64), PARAMS(const volatile void __iomem *)); +#endif /* CONFIG_64BIT */ + +DECLARE_FUNCTION_MOCK_VOID_RETURN(writeb, + PARAMS(u8, const volatile void __iomem *)); + +DECLARE_FUNCTION_MOCK_VOID_RETURN(writew, + PARAMS(u16, const volatile void __iomem *)); + +DECLARE_FUNCTION_MOCK_VOID_RETURN(writel, + PARAMS(u32, const volatile void __iomem *)); + +#ifdef CONFIG_64BIT +DECLARE_FUNCTION_MOCK_VOID_RETURN(writeq, + PARAMS(u64, const volatile void __iomem *)); +#endif /* CONFIG_64BIT */ + +#endif /* _ASM_UM_IO_MOCK_H */ diff --git a/arch/um/include/asm/io.h b/arch/um/include/asm/io.h index 96f77b5232aaf..a7f61cf963756 100644 --- a/arch/um/include/asm/io.h +++ b/arch/um/include/asm/io.h @@ -2,11 +2,19 @@ #ifndef _ASM_UM_IO_H #define _ASM_UM_IO_H
+#include <linux/types.h> +#include <asm/byteorder.h> + +#if IS_ENABLED(CONFIG_PLATFORM_MOCK) +#include <asm/io-mock-shared.h> +#endif + #define ioremap ioremap static inline void __iomem *ioremap(phys_addr_t offset, size_t size) { return (void __iomem *)(unsigned long)offset; } +#define ioremap_nocache ioremap
#define iounmap iounmap static inline void iounmap(void __iomem *addr) diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile index 2f36d515762ec..770c480d5a101 100644 --- a/arch/um/kernel/Makefile +++ b/arch/um/kernel/Makefile @@ -25,6 +25,7 @@ obj-$(CONFIG_GPROF) += gprof_syms.o obj-$(CONFIG_GCOV) += gmon_syms.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o obj-$(CONFIG_STACKTRACE) += stacktrace.o +obj-$(CONFIG_PLATFORM_MOCK) += io-mock.o
USER_OBJS := config.o
diff --git a/arch/um/kernel/io-mock.c b/arch/um/kernel/io-mock.c new file mode 100644 index 0000000000000..e0d4648e97a6c --- /dev/null +++ b/arch/um/kernel/io-mock.c @@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Mock IO functions. + * + * Copyright (C) 2018, Google LLC. + * Author: Brendan Higgins brendanhiggins@google.com + */ + +#include <linux/mm.h> +#include <linux/io.h> +#include <linux/kernel.h> +#include <kunit/mock.h> + +DEFINE_FUNCTION_MOCK(readb, + RETURNS(u8), PARAMS(const volatile void __iomem *)); + +DEFINE_FUNCTION_MOCK(readw, + RETURNS(u16), PARAMS(const volatile void __iomem *)); + +DEFINE_FUNCTION_MOCK(readl, + RETURNS(u32), PARAMS(const volatile void __iomem *)); + +#ifdef CONFIG_64BIT +DEFINE_FUNCTION_MOCK(readq, + RETURNS(u64), PARAMS(const volatile void __iomem *)); +#endif /* CONFIG_64BIT */ + +DEFINE_FUNCTION_MOCK_VOID_RETURN(writeb, + PARAMS(u8, const volatile void __iomem *)); + +DEFINE_FUNCTION_MOCK_VOID_RETURN(writew, + PARAMS(u16, const volatile void __iomem *)); + +DEFINE_FUNCTION_MOCK_VOID_RETURN(writel, + PARAMS(u32, const volatile void __iomem *)); + +#ifdef CONFIG_64BIT +DEFINE_FUNCTION_MOCK_VOID_RETURN(writeq, + PARAMS(u64, const volatile void __iomem *)); +#endif /* CONFIG_64BIT */ diff --git a/kunit/Kconfig b/kunit/Kconfig index 5cb500355c873..9d4b7cfff9d92 100644 --- a/kunit/Kconfig +++ b/kunit/Kconfig @@ -6,6 +6,7 @@ menu "KUnit support"
config KUNIT bool "Enable support for unit tests (KUnit)" + select HAS_IOMEM help Enables support for kernel unit tests (KUnit), a lightweight unit testing and mocking framework for the Linux kernel. These tests are
On Tue, Oct 16, 2018 at 6:54 PM Brendan Higgins brendanhiggins@google.com wrote:
This mocks out some iomem functions (functions like readl and writel), for mocking hardware interfaces.
Signed-off-by: Brendan Higgins brendanhiggins@google.com
arch/um/Kconfig.common | 8 +++++- arch/um/Kconfig.um | 5 ++++ arch/um/include/asm/Kbuild | 1 - arch/um/include/asm/io-mock-shared.h | 33 +++++++++++++++++++++ arch/um/include/asm/io-mock.h | 43 ++++++++++++++++++++++++++++ arch/um/include/asm/io.h | 8 ++++++ arch/um/kernel/Makefile | 1 + arch/um/kernel/io-mock.c | 40 ++++++++++++++++++++++++++ kunit/Kconfig | 1 + 9 files changed, 138 insertions(+), 2 deletions(-) create mode 100644 arch/um/include/asm/io-mock-shared.h create mode 100644 arch/um/include/asm/io-mock.h create mode 100644 arch/um/kernel/io-mock.c
diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common index 07f84c842cc31..72e7efb74f7fd 100644 --- a/arch/um/Kconfig.common +++ b/arch/um/Kconfig.common @@ -19,7 +19,13 @@ config MMU default y
config NO_IOMEM
def_bool y
booldefault y if !KUNIT+config HAS_IOMEM
HAS_IOMEM is essentially a disable flag for lots of drivers on UML. Ignoring drivers, it doesn't really control a significant amount of code (albeit small amount of code you need for this series). As a driver disable, it does a poor job as lots of drivers aren't MMIO and aren't disabled. I think we should decouple these 2 things. Perhaps get rid of the driver disable part altogether. We already do 'depends on ARCH_FOO || COMPILE_TEST' for lots of drivers.
Also, I wouldn't be surprised if turning on HAS_IOMEM causes UML randconfig failures. Arnd does lots of randconfig testing and might be willing to help check.
bool "Turns on fake IOMEM support for KUnit"depends on KUNITselect MOCK_IOMEMconfig ISA bool diff --git a/arch/um/Kconfig.um b/arch/um/Kconfig.um index 20da5a8ca9490..8d35e0e2c23d1 100644 --- a/arch/um/Kconfig.um +++ b/arch/um/Kconfig.um @@ -122,3 +122,8 @@ config SECCOMP defined by each seccomp mode.
If unsure, say Y.
+config PLATFORM_MOCK
bool "Enable a mock architecture used for unit testing."depends on KUNIT && OFdefault ndiff --git a/arch/um/include/asm/Kbuild b/arch/um/include/asm/Kbuild index b10dde6cb793b..9fd2827ab76d1 100644 --- a/arch/um/include/asm/Kbuild +++ b/arch/um/include/asm/Kbuild @@ -12,7 +12,6 @@ generic-y += ftrace.h generic-y += futex.h generic-y += hardirq.h generic-y += hw_irq.h -generic-y += io.h generic-y += irq_regs.h generic-y += irq_work.h generic-y += kdebug.h diff --git a/arch/um/include/asm/io-mock-shared.h b/arch/um/include/asm/io-mock-shared.h new file mode 100644 index 0000000000000..6baf59cb17a58 --- /dev/null +++ b/arch/um/include/asm/io-mock-shared.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_UM_IO_MOCK_SHARED_H +#define _ASM_UM_IO_MOCK_SHARED_H
+#define readb readb +u8 readb(const volatile void __iomem *);
What about all the other flavors of MMIO accessors like __raw_readb, readb_relaxed, etc.? readX/writeX is intended for PCI based drivers which doesn't seem to be what you are targeting in this series.
I think it would be good if this capability was not just on UML. I could also imagine wanting to run tests on real h/w. Perhaps you just want to log and/or check i/o accesses. Or you need some dependencies in place rather than trying to fake everything (clocks, gpios, pinmux, irq, etc.). That being said, I'm not trying to add a bunch of things for you to do. Though maybe it makes sense to split this series some. How many of the patches are needed to convert the DT unittests for example?
Rob
On Wed, Oct 17, 2018 at 3:28 PM Rob Herring robh@kernel.org wrote: <snip>
diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common index 07f84c842cc31..72e7efb74f7fd 100644 --- a/arch/um/Kconfig.common +++ b/arch/um/Kconfig.common @@ -19,7 +19,13 @@ config MMU default y
config NO_IOMEM
def_bool y
booldefault y if !KUNIT+config HAS_IOMEM
HAS_IOMEM is essentially a disable flag for lots of drivers on UML. Ignoring drivers, it doesn't really control a significant amount of code (albeit small amount of code you need for this series). As a driver disable, it does a poor job as lots of drivers aren't MMIO and aren't disabled. I think we should decouple these 2 things. Perhaps get rid of the driver disable part altogether. We already do 'depends on ARCH_FOO || COMPILE_TEST' for lots of drivers.
I think that makes sense. Any code that can build should be able to build under KUnit, but we probably want to turn that on on a per driver basis as we write tests for them.
Also, I wouldn't be surprised if turning on HAS_IOMEM causes UML randconfig failures. Arnd does lots of randconfig testing and might be willing to help check.
It almost certainly would fail randconfig. As you point out below, I don't implement everything that's required, just enough to show off KUnit in a couple examples.
bool "Turns on fake IOMEM support for KUnit"depends on KUNITselect MOCK_IOMEMconfig ISA bool diff --git a/arch/um/Kconfig.um b/arch/um/Kconfig.um index 20da5a8ca9490..8d35e0e2c23d1 100644 --- a/arch/um/Kconfig.um +++ b/arch/um/Kconfig.um @@ -122,3 +122,8 @@ config SECCOMP defined by each seccomp mode.
If unsure, say Y.
+config PLATFORM_MOCK
bool "Enable a mock architecture used for unit testing."depends on KUNIT && OFdefault ndiff --git a/arch/um/include/asm/Kbuild b/arch/um/include/asm/Kbuild index b10dde6cb793b..9fd2827ab76d1 100644 --- a/arch/um/include/asm/Kbuild +++ b/arch/um/include/asm/Kbuild @@ -12,7 +12,6 @@ generic-y += ftrace.h generic-y += futex.h generic-y += hardirq.h generic-y += hw_irq.h -generic-y += io.h generic-y += irq_regs.h generic-y += irq_work.h generic-y += kdebug.h diff --git a/arch/um/include/asm/io-mock-shared.h b/arch/um/include/asm/io-mock-shared.h new file mode 100644 index 0000000000000..6baf59cb17a58 --- /dev/null +++ b/arch/um/include/asm/io-mock-shared.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_UM_IO_MOCK_SHARED_H +#define _ASM_UM_IO_MOCK_SHARED_H
+#define readb readb +u8 readb(const volatile void __iomem *);
What about all the other flavors of MMIO accessors like __raw_readb, readb_relaxed, etc.? readX/writeX is intended for PCI based drivers which doesn't seem to be what you are targeting in this series.
Those need to be done too. I just mostly wanted to illustrate that it can be done, and what is needed to support mocking MMIO. I wasn't sure how controversial my approach would be, so I didn't want to put any more work than was necessary for illustration without getting some feedback.
I think it would be good if this capability was not just on UML. I could also imagine wanting to run tests on real h/w. Perhaps you just
I think that's reasonable.
want to log and/or check i/o accesses. Or you need some dependencies in place rather than trying to fake everything (clocks, gpios, pinmux, irq, etc.). That being said, I'm not trying to add a bunch of things for you to do. Though maybe it makes sense to split this series some.
Almost definitely. I figured this patchset, as is, is a good illustration of what I am trying to do, what is possible, and the kind of work that is necessary to get there. If people like what I am doing and want more of this type of thing, I think focussing on getting base support in and then working on features separately is the way to go.
How many of the patches are needed to convert the DT unittests for example?
At first glance, just you probably only need the stuff in the first 9 patches for that. You don't appear to be doing any IO of any sort, so you don't need this stuff. You appear to depend only on some fake data; everything else seems pretty self contained, so you don't need any of the mocking support for that. So if I understand correctly you just need the base support needed for bare bones unit tests, all that stuff is in the first 9 patches.
Cheers
- Added intro and usage guide for KUnit - Added API reference
Signed-off-by: Felix Guo felixguoxiuping@gmail.com Signed-off-by: Brendan Higgins brendanhiggins@google.com --- Documentation/index.rst | 1 + .../kunit/api/class-and-function-mocking.rst | 68 ++ Documentation/kunit/api/index.rst | 21 + Documentation/kunit/api/platform-mocking.rst | 36 + Documentation/kunit/api/test.rst | 15 + Documentation/kunit/faq.rst | 46 + Documentation/kunit/index.rst | 84 ++ Documentation/kunit/start.rst | 185 ++++ Documentation/kunit/usage.rst | 876 ++++++++++++++++++ 9 files changed, 1332 insertions(+) create mode 100644 Documentation/kunit/api/class-and-function-mocking.rst create mode 100644 Documentation/kunit/api/index.rst create mode 100644 Documentation/kunit/api/platform-mocking.rst create mode 100644 Documentation/kunit/api/test.rst create mode 100644 Documentation/kunit/faq.rst create mode 100644 Documentation/kunit/index.rst create mode 100644 Documentation/kunit/start.rst create mode 100644 Documentation/kunit/usage.rst
diff --git a/Documentation/index.rst b/Documentation/index.rst index fdc585703498e..9415b6536d04b 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -66,6 +66,7 @@ merged much easier. kernel-hacking/index trace/index maintainer/index + kunit/index
Kernel API documentation ------------------------ diff --git a/Documentation/kunit/api/class-and-function-mocking.rst b/Documentation/kunit/api/class-and-function-mocking.rst new file mode 100644 index 0000000000000..3e53291145f1f --- /dev/null +++ b/Documentation/kunit/api/class-and-function-mocking.rst @@ -0,0 +1,68 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +Class and Function Mocking +========================== + +This file documents class and function mocking features. + +.. note:: + If possible, prefer class mocking over arbitrary function mocking. Class + mocking has a much more limited scope and provides more control. + This file documents class mocking and most mocking features that do not + depend on function or platform mocking. + +Readability Macros +------------------ +When defining and declaring mock stubs, use these readability macros. + +.. code-block:: c + + #define CLASS(struct_name) struct_name + #define HANDLE_INDEX(index) index + #define METHOD(method_name) method_name + #define RETURNS(return_type) return_type + #define PARAMS(...) __VA_ARGS__ + +Consider a ``struct Foo`` with a member function +``int add(struct Foo*, int a, int b);`` + +When generating a mock stub with :c:func:`DEFINE_STRUCT_CLASS_MOCK`, which +takes a method name, struct name, return type, and method parameters, the +arguments should be passed in with the readability macros. + +.. code-block:: c + + DEFINE_STRUCT_CLASS_MOCK( + METHOD(add), + CLASS(Foo), + RETURNS(int), + PARAMS(struct Foo *, int, int) + ); + +For a more detailed example of this, take a look at the example in +:doc:`../start` + +These macros should only be used in the context of the mock stub generators. + + +Built in Matchers +----------------- + +.. kernel-doc:: include/kunit/mock.h + :doc: Built In Matchers + +Mock Returns +------------ +These functions can be used to specify a value to be returned (``ret``) when a +mocked function is intercepted via :c:func:`EXPECT_CALL`. + +.. code-block:: c + + struct mock_action *int_return(struct test *test, int ret); + struct mock_action *u32_return(struct test *test, u32 ret); + +API +--- +.. kernel-doc:: include/kunit/mock.h + :internal: diff --git a/Documentation/kunit/api/index.rst b/Documentation/kunit/api/index.rst new file mode 100644 index 0000000000000..a4fdc35b32c5c --- /dev/null +++ b/Documentation/kunit/api/index.rst @@ -0,0 +1,21 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +API Reference +============= +.. toctree:: + + test + class-and-function-mocking + platform-mocking + +This section documents the KUnit kernel testing API. It is divided into 3 +sections: + +================================= ============================================== +:doc:`test` documents all of the standard testing API + excluding mocking or mocking related features. +:doc:`class-and-function-mocking` documents class and function mocking features. +:doc:`platform-mocking` documents mocking libraries that mock out + platform specific features. +================================= ============================================== diff --git a/Documentation/kunit/api/platform-mocking.rst b/Documentation/kunit/api/platform-mocking.rst new file mode 100644 index 0000000000000..72555eb5b1de1 --- /dev/null +++ b/Documentation/kunit/api/platform-mocking.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================ +Platform Mocking +================ + +This file documents *platform mocking*, mocking libraries that mock out platform +specific features and aid in writing mocks for platform drivers and other low +level kernel code. + +Enable Platform Mocking +----------------------- +``CONFIG_PLATFORM_MOCK`` needs to be added to the .config (or kunitconfig) to +enable platform mocking. + +Mocked IO Functions +------------------- +The following functions have been mocked for convenience. + +.. code-block:: c + + u8 readb(const volatile void __iomem *); + u16 readw(const volatile void __iomem *); + u32 readl(const volatile void __iomem *); + u64 readq(const volatile void __iomem *); + void writeb(u8, const volatile void __iomem *); + void writew(u16, const volatile void __iomem *); + void writel(u32, const volatile void __iomem *); + void writeq(u64, const volatile void __iomem *); + +.. note:: These functions do not have any non-mocked behaviour in UML. + +API +--- +.. kernel-doc:: include/linux/platform_device_mock.h + :internal: diff --git a/Documentation/kunit/api/test.rst b/Documentation/kunit/api/test.rst new file mode 100644 index 0000000000000..7f22db32536eb --- /dev/null +++ b/Documentation/kunit/api/test.rst @@ -0,0 +1,15 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======== +Test API +======== + +This file documents all of the standard testing API excluding mocking or mocking +related features. + +.. kernel-doc:: include/kunit/test.h + :internal: + +.. kernel-doc:: include/kunit/test-stream.h + :internal: + diff --git a/Documentation/kunit/faq.rst b/Documentation/kunit/faq.rst new file mode 100644 index 0000000000000..fce128d804b49 --- /dev/null +++ b/Documentation/kunit/faq.rst @@ -0,0 +1,46 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +Frequently Asked Questions +========================================= + +How is this different from Autotest, kselftest, etc? +==================================================== +KUnit is a unit testing framework. Autotest, kselftest (and some others) are +not. + +A `unit test http://softwaretestingfundamentals.com/unit-testing/`_ is +supposed to test a single unit of code in isolation, hence the name. A unit +test should be the finest granularity of testing and as such should allow all +possible code paths to be tested in the code under test; this is only possible +if the code under test is very small and does not have any external +dependencies outside of the test's control like hardware. + +There are no testing frameworks currently available for the kernel that do not +require installing the kernel on a test machine or in a VM and all require +tests to be written in userspace and run on the kernel under test; this is true +for Autotest, kselftest, and some others, disqualifying any of them from being +considered unit testing frameworks. + +What is the difference between a unit test and these other kinds of tests? +========================================================================== +Most existing tests for the Linux kernel would be categorized as an integration +test, or an end-to-end test. + +- A unit test is supposed to test a single unit of code in isolation, hence the + name. A unit test should be the finest granularity of testing and as such + should allow all possible code paths to be tested in the code under test; this + is only possible if the code under test is very small and does not have any + external dependencies outside of the test's control like hardware. +- An integration test tests the interaction between a minimal set of components, + usually just two or three. For example, someone might write an integration + test to test the interaction between a driver and a piece of hardware, or to + test the interaction between the userspace libraries the kernel provides and + the kernel itself; however, one of these tests would probably not test the + entire kernel along with hardware interactions and interactions with the + userspace. +- An end-to-end test usually tests the entire system from the perspective of the + code under test. For example, someone might write an end-to-end test for the + kernel by installing a production configuration of the kernel on production + hardware with a production userspace and then trying to exercise some behavior + that depends on interactions between the hardware, the kernel, and userspace. diff --git a/Documentation/kunit/index.rst b/Documentation/kunit/index.rst new file mode 100644 index 0000000000000..fc2716f155d74 --- /dev/null +++ b/Documentation/kunit/index.rst @@ -0,0 +1,84 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +KUnit - Unit Testing for the Linux Kernel +========================================= + +.. toctree:: + :maxdepth: 2 + + start + usage + api/index + faq + +What is KUnit? +============== + +KUnit is a lightweight unit testing and mocking framework for the Linux kernel. +These tests are able to be run locally on a developer's workstation without a VM +or special hardware. + +KUnit is heavily inspired by JUnit, Python's ``unittest.mock``, and +Googletest/Googlemock for C++. They have the same structure for defining test +suites and test cases. KUnit defines a way to mock out C style classes and +functions and create expectations on methods called within the code under test. + +Get started now: :doc:`start` + +Why KUnit? +========== + +Aside from KUnit there is no true unit testing framework for the Linux kernel. +Autotest and kselftest are sometimes cited as unit testing frameworks; however, +they are not by most reasonable definitions of unit tests. + +A unit test is supposed to test a single unit of code in isolation, hence the +name. A unit test should be the finest granularity of testing and as such should +allow all possible code paths to be tested in the code under test; this is only +possible if the code under test is very small and does not have any external +dependencies outside of the test's control like hardware. + +Outside of KUnit, there are no testing frameworks currently +available for the kernel that do not require installing the kernel on a test +machine or in a VM and all require tests to be written in userspace running on +the kernel; this is true for Autotest, and kselftest, disqualifying +any of them from being considered unit testing frameworks. + +KUnit addresses the problem of being able to run tests without needing a virtual +machine or actual hardware with User Mode Linux. User Mode Linux is a Linux +architecture, like ARM or x86; however, unlike other architectures it compiles +to a standalone program that can be run like any other program directly inside +of a host operating system; to be clear, it does not require any virtualization +support; it is just a regular program. + +KUnit is fast. Excluding build time, from invocation to completion KUnit can run +several dozen tests in only 10 to 20 seconds; this might not sound like a big +deal to some people, but having such fast and easy to run tests fundamentally +changes the way you go about testing and even writing code in the first place. +Linus himself said in his `git talk at Google +https://gist.github.com/lorn/1272686/revisions#diff-53c65572127855f1b003db4064a94573R874`_: + + "... a lot of people seem to think that performance is about doing the + same thing, just doing it faster, and that is not true. That is not what + performance is all about. If you can do something really fast, really + well, people will start using it differently." + +In this context Linus was talking about branching and merging, +but this point also applies to testing. If your tests are slow, unreliable, are +difficult to write, and require a special setup or special hardware to run, +then you wait a lot longer to write tests, and you wait a lot longer to run +tests; this means that tests are likely to break, unlikely to test a lot of +things, and are unlikely to be rerun once they pass. If your tests are really +fast, you run them all the time, every time you make a change, and every time +someone sends you some code. Why trust that someone ran all their tests +correctly on every change when you can just run them yourself in less time than +it takes to read his / her test log? + +How do I use it? +=================== + +* :doc:`start` - for new users of KUnit +* :doc:`usage` - for a more detailed explanation of KUnit features +* :doc:`api/index` - for the list of KUnit APIs used for testing + diff --git a/Documentation/kunit/start.rst b/Documentation/kunit/start.rst new file mode 100644 index 0000000000000..14d1e4bd02f58 --- /dev/null +++ b/Documentation/kunit/start.rst @@ -0,0 +1,185 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +Installing dependencies +======================= +KUnit has the same dependencies as the Linux kernel. As long as you can build +the kernel, you can run KUnit. + +KUnit Wrapper +============= +Included with KUnit is a simple Python wrapper that helps format the output to +easily use and read KUnit output. It handles building and running the kernel, as +well as formatting the output. + +The wrapper can be run with: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py + +Creating a kunitconfig +====================== +The Python script is a thin wrapper around Kbuild as such, it needs to be +configured with a ``kunitconfig`` file. This file essentially contains the +regular Kernel config, with the specific test targets as well. + +.. code-block:: bash + + git clone -b master https://kunit.googlesource.com/kunitconfig $PATH_TO_KUNITCONFIG_REPO + cd $PATH_TO_LINUX_REPO + ln -s $PATH_TO_KUNIT_CONFIG_REPO/kunitconfig kunitconfig + +You may want to add kunitconfig to your local gitignore. + +Verifying KUnit Works +------------------------- + +To make sure that everything is set up correctly, simply invoke the Python +wrapper from your kernel repo: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py + +.. note:: + You may want to run ``make mrproper`` first. + +If everything worked correctly, you should see the following: + +.. code-block:: bash + + Generating .config ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +followed by a list of tests that are run. All of them should be passing. + +.. note:: + Because it is building a lot of sources for the first time, the ``Building + kunit kernel`` step may take a while. + +Writing your first test +========================== + +In your kernel repo let's add some code that we can test. Create a file +``drivers/misc/example.h`` with the contents: + +.. code-block:: c + + int misc_example_add(int left, int right); + +create a file ``drivers/misc/example.c``: + +.. code-block:: c + + #include <linux/errno.h> + + #include "example.h" + + int misc_example_add(int left, int right) + { + return left + right; + } + +Now add the following lines to ``drivers/misc/Kconfig``: + +.. code-block:: kconfig + + config MISC_EXAMPLE + bool "My example" + +and the following lines to ``drivers/misc/Makefile``: + +.. code-block:: make + + obj-$(CONFIG_MISC_EXAMPLE) += example.o + +Now we are ready to write the test. The test will be in +``drivers/misc/example-test.c``: + +.. code-block:: c + + #include <linux/test.h> + #include <linux/mock.h> + #include "example.h" + + /* Define the test cases. */ + + static void misc_example_add_test_basic(struct test *test) + { + TEST_EXPECT_EQ(test, 1, misc_example_add(1, 0)); + TEST_EXPECT_EQ(test, 2, misc_example_add(1, 1)); + TEST_EXPECT_EQ(test, 0, misc_example_add(-1, 1)); + TEST_EXPECT_EQ(test, INT_MAX, misc_example_add(0, INT_MAX)); + TEST_EXPECT_EQ(test, -1, misc_example_add(INT_MAX, INT_MIN)); + } + + static void misc_example_test_failure(struct test *test) + { + TEST_FAIL(test, "This test never passes."); + } + + static struct test_case misc_example_test_cases[] = { + TEST_CASE(misc_example_add_test_basic), + TEST_CASE(misc_example_test_failure), + {}, + }; + + static struct test_module misc_example_test_module = { + .name = "misc-example", + .test_cases = misc_example_test_cases, + }; + module_test(misc_example_test_module); + +Now add the following to ``drivers/misc/Kconfig``: + +.. code-block:: kconfig + + config MISC_EXAMPLE_TEST + bool "Test for my example" + depends on MISC_EXAMPLE && TEST + +and the following to ``drivers/misc/Makefile``: + +.. code-block:: make + + obj-$(CONFIG_MISC_EXAMPLE_TEST) += example-test.o + +Now add it to your ``kunitconfig``: + +.. code-block:: none + + CONFIG_MISC_EXAMPLE=y + CONFIG_MISC_EXAMPLE_TEST=y + +Now you can run the test: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py + +You should see the following failure: + +.. TODO(brendanhiggins@google.com): update me!!! + +.. code-block:: none + + ... + kunit misc-example: misc_example_bar_test_success passed + kunit misc-example: EXPECTATION FAILED at drivers/misc/example-test.c:48 + Expected -22 == misc_example_bar(example), but + -22 == -22 + misc_example_bar(example) == -5 + kunit misc-example: misc_example_bar_test_failure failed + kunit misc-example: one or more tests failed + +Congrats! You just wrote your first KUnit test! + +Next Steps +============= +* Check out the :doc:`usage` page for a more + in-depth explanation of KUnit. diff --git a/Documentation/kunit/usage.rst b/Documentation/kunit/usage.rst new file mode 100644 index 0000000000000..605bc4d1165a7 --- /dev/null +++ b/Documentation/kunit/usage.rst @@ -0,0 +1,876 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +Using KUnit +============= + +The purpose of this document is to describe what KUnit is, how it works, how it +is intended to be used, and all the concepts and terminology that are needed to +understand it. This guide assumes a working knowledge of the Linux kernel and +some basic knowledge of testing. + +For a high level introduction to KUnit, including setting up KUnit for your +project, see :doc:`start`. + +Organization of this document +================================= + +This document is organized into two main sections: Testing and Isolating +Behavior. The first covers what a unit test is and how to use KUnit to write +them. The second covers how to use KUnit to isolate code and make it possible +to unit test code that was otherwise un-unit-testable. + +Testing +========== + +What is KUnit? +------------------ + +"K" is short for "kernel" so "KUnit" is the "(Linux) Kernel Unit Testing +Framework." KUnit is intended first and foremost for writing unit tests; it is +general enough that it can be used to write integration tests; however, this is +a secondary goal. KUnit has no ambition of being the only testing framework for +the kernel; for example, it does not intend to be an end-to-end testing +framework. + +What is Unit Testing? +------------------------- + +A `unit test http://softwaretestingfundamentals.com/unit-testing/`_ is a test +that tests code at the smallest possible scope, a *unit* of code. In the C +programming language that's a function. + +Unit tests should be written for all the publicly exposed functions in a +compilation unit; so that is all the functions that are exported in either a +*class* (defined below) or all functions which are **not** static. + +Writing Tests +------------- + +Test Cases +~~~~~~~~~~ + +The fundamental unit in KUnit is the test case. A test case is a function with +the signature ``void (*)(struct test *test)``. It calls a function to be tested +and then sets *expectations* for what should happen. For example: + +.. code-block:: c + + void example_test_success(struct test *test) + { + } + + void example_test_failure(struct test *test) + { + TEST_FAIL(test, "This test never passes."); + } + +In the above example ``example_test_success`` always passes because it does +nothing; no expectations are set, so all expectations pass. On the other hand +``example_test_failure`` always fails because it calls ``TEST_FAIL``, which is a +special expectation that logs a message and causes the test case to fail. + +Expectations +~~~~~~~~~~~~ +An *expectation* is a way to specify that you expect a piece of code to do +something in a test. An expectation is called like a function. A test is made +by setting expectations about the behavior of a piece of code under test; when +one or more of the expectations fail, the test case fails and information about +the failure is logged. For example: + +.. code-block:: c + + void add_test_basic(struct test *test) + { + TEST_EXPECT_EQ(test, 1, add(1, 0)); + TEST_EXPECT_EQ(test, 2, add(1, 1)); + } + +In the above example ``add_test_basic`` makes a number of assertions about the +behavior of a function called ``add``; the first parameter is always of type +``struct test *``, which contains information about the current test context; +the second parameter, in this case, is what the value is expected to be; the +last value is what the value actually is. If ``add`` passes all of these +expectations, the test case, ``add_test_basic`` will pass; if any one of these +expectations fail, the test case will fail. + +It is important to understand that a test case *fails* when any expectation is +violated; however, the test will continue running, potentially trying other +expectations until the test case ends or is otherwise terminated. This is as +opposed to *assertions* which are discussed later. + +To learn about more expectations supported by KUnit, see :doc:`api/test`. + +.. note:: + A single test case should be pretty short, pretty easy to understand, + focused on a single behavior. + +For example, if we wanted to properly test the add function above, we would +create additional tests cases which would each test a different property that an +add function should have like this: + +.. code-block:: c + + void add_test_basic(struct test *test) + { + TEST_EXPECT_EQ(test, 1, add(1, 0)); + TEST_EXPECT_EQ(test, 2, add(1, 1)); + } + + void add_test_negative(struct test *test) + { + TEST_EXPECT_EQ(test, 0, add(-1, 1)); + } + + void add_test_max(struct test *test) + { + TEST_EXPECT_EQ(test, INT_MAX, add(0, INT_MAX)); + TEST_EXPECT_EQ(test, -1, add(INT_MAX, INT_MIN)); + } + + void add_test_overflow(struct test *test) + { + TEST_EXPECT_EQ(test, INT_MIN, add(INT_MAX, 1)); + } + +Notice how it is immediately obvious what all the properties that we are testing +for are. + +Assertions +~~~~~~~~~~ + +KUnit also has the concept of an *assertion*. An assertion is just like an +expectation except the assertion immediately terminates the test case if it is +not satisfied. + +For example: + +.. code-block:: c + + static void mock_test_do_expect_default_return(struct test *test) + { + struct mock_test_context *ctx = test->priv; + struct mock *mock = ctx->mock; + int param0 = 5, param1 = -5; + const char *two_param_types[] = {"int", "int"}; + const void *two_params[] = {¶m0, ¶m1}; + const void *ret; + + ret = mock->do_expect(mock, + "test_printk", test_printk, + two_param_types, two_params, + ARRAY_SIZE(two_params)); + TEST_ASSERT_NOT_ERR_OR_NULL(test, ret); + TEST_EXPECT_EQ(test, -4, *((int *) ret)); + } + +In this example, the method under test should return a pointer to a value, so +if the pointer returned by the method is null or an errno, we don't want to +bother continuing the test since the following expectation could crash the test +case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case if +the appropriate conditions have not been satisfied to complete the test. + +Modules / Test Suites +~~~~~~~~~~~~~~~~~~~~~ + +Now obviously one unit test isn't very helpful; the power comes from having +many test cases covering all of your behaviors. Consequently it is common to +have many *similar* tests; in order to reduce duplication in these closely +related tests most unit testing frameworks provide the concept of a *test +suite*, in KUnit we call it a *test module*; all it is is just a collection of +test cases for a unit of code with a set up function that gets invoked before +every test cases and then a tear down function that gets invoked after every +test case completes. + +Example: + +.. code-block:: c + + static struct test_case example_test_cases[] = { + TEST_CASE(example_test_foo), + TEST_CASE(example_test_bar), + TEST_CASE(example_test_baz), + {}, + }; + + static struct test_module example_test_module[] = { + .name = "example", + .init = example_test_init, + .exit = example_test_exit, + .test_cases = example_test_cases, + }; + module_test(example_test_module); + +In the above example the test suite, ``example_test_module``, would run the test +cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``, each +would have ``example_test_init`` called immediately before it and would have +``example_test_exit`` called immediately after it. +``module_test(example_test_module)`` registers the test suite with the KUnit +test framework. + +.. note:: + A test case will only be run if it is associated with a test suite. + +For a more information on these types of things see the :doc:`api/test`. + +Isolating Behavior +================== + +The most important aspect of unit testing that other forms of testing do not +provide is the ability to limit the amount of code under test to a single unit. +In practice, this is only possible by being able to control what code gets run +when the unit under test calls a function and this is usually accomplished +through some sort of indirection where a function is exposed as part of an API +such that the definition of that function can be changed without affecting the +rest of the code base. In the kernel this primarily comes from two constructs, +classes, structs that contain function pointers that are provided by the +implementer, and architecture specific functions which have definitions selected +at compile time. + +Classes +------- + +Classes are not a construct that is built into the C programming language; +however, it is an easily derived concept. Accordingly, pretty much every project +that does not use a standardized object oriented library (like GNOME's GObject) +has their own slightly different way of doing object oriented programming; the +Linux kernel is no exception. + +The central concept in kernel object oriented programming is the class. In the +kernel, a *class* is a struct that contains function pointers. This creates a +contract between *implementers* and *users* since it forces them to use the +same function signature without having to call the function directly. In order +for it to truly be a class, the function pointers must specify that a pointer +to the class, known as a *class handle*, be one of the parameters; this makes +it possible for the member functions (also known as *methods*) to have access +to member variables (more commonly known as *fields*) allowing the same +implementation to have multiple *instances*. + +Typically a class can be *overridden* by *child classes* by embedding the +*parent class* in the child class. Then when a method provided by the child +class is called, the child implementation knows that the pointer passed to it is +of a parent contained within the child; because of this, the child can compute +the pointer to itself because the pointer to the parent is always a fixed offset +from the pointer to the child; this offset is the offset of the parent contained +in the child struct. For example: + +.. code-block:: c + + struct shape { + int (*area)(struct shape *this); + }; + + struct rectangle { + struct shape parent; + int length; + int width; + }; + + int rectangle_area(struct shape *this) + { + struct rectangle *self = container_of(this, struct shape, parent); + + return self->length * self->width; + }; + + void rectangle_new(struct rectangle *self, int length, int width) + { + self->parent.area = rectangle_area; + self->length = length; + self->width = width; + } + +In this example (as in most kernel code) the operation of computing the pointer +to the child from the pointer to the parent is done by ``container_of``. + +Faking Classes +~~~~~~~~~~~~~~ + +In order to unit test a piece of code that calls a method in a class, the +behavior of the method must be controllable, otherwise the test ceases to be a +unit test and becomes an integration test. + +A fake just provides an implementation of a piece of code that is different than +what runs in a production instance, but behaves identically from the standpoint +of the callers; this is usually done to replace a dependency that is hard to +deal with, or is slow. + +A good example for this might be implementing a fake EEPROM that just stores the +"contents" in an internal buffer. For example, let's assume we have a class that +represents an EEPROM: + +.. code-block:: c + + struct eeprom { + ssize_t (*read)(struct eeprom *this, size_t offset, char *buffer, size_t count); + ssize_t (*write)(struct eeprom *this, size_t offset, const char *buffer, size_t count); + }; + +And we want to test some code that buffers writes to the EEPROM: + +.. code-block:: c + + struct eeprom_buffer { + ssize_t (*write)(struct eeprom_buffer *this, const char *buffer, size_t count); + int flush(struct eeprom_buffer *this); + size_t flush_count; /* Flushes when buffer exceeds flush_count. */ + }; + + struct eeprom_buffer *new_eeprom_buffer(struct eeprom *eeprom); + void destroy_eeprom_buffer(struct eeprom *eeprom); + +We can easily test this code by *faking out* the underlying EEPROM: + +.. code-block:: c + + struct fake_eeprom { + struct eeprom parent; + char contents[FAKE_EEPROM_CONTENTS_SIZE]; + }; + + ssize_t fake_eeprom_read(struct eeprom *parent, size_t offset, char *buffer, size_t count) + { + struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent); + + count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset); + memcpy(buffer, this->contents + offset, count); + + return count; + } + + ssize_t fake_eeprom_write(struct eeprom *this, size_t offset, const char *buffer, size_t count) + { + struct fake_eeprom *this = container_of(parent, struct fake_eeprom, parent); + + count = min(count, FAKE_EEPROM_CONTENTS_SIZE - offset); + memcpy(this->contents + offset, buffer, count); + + return count; + } + + void fake_eeprom_init(struct fake_eeprom *this) + { + this->parent.read = fake_eeprom_read; + this->parent.write = fake_eeprom_write; + memset(this->contents, 0, FAKE_EEPROM_CONTENTS_SIZE); + } + +We can now use it to test ``struct eeprom_buffer``: + +.. code-block:: c + + struct eeprom_buffer_test { + struct fake_eeprom *fake_eeprom; + struct eeprom_buffer *eeprom_buffer; + }; + + static void eeprom_buffer_test_does_not_write_until_flush(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer; + struct fake_eeprom *fake_eeprom = ctx->fake_eeprom; + char buffer[] = {0xff}; + + eeprom_buffer->flush_count = SIZE_MAX; + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0); + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + TEST_EXPECT_EQ(test, fake_eeprom->contents[1], 0); + + eeprom_buffer->flush(eeprom_buffer); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff); + TEST_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff); + } + + static void eeprom_buffer_test_flushes_after_flush_count_met(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer; + struct fake_eeprom *fake_eeprom = ctx->fake_eeprom; + char buffer[] = {0xff}; + + eeprom_buffer->flush_count = 2; + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0); + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff); + TEST_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff); + } + + static void eeprom_buffer_test_flushes_increments_of_flush_count(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer; + struct fake_eeprom *fake_eeprom = ctx->fake_eeprom; + char buffer[] = {0xff, 0xff}; + + eeprom_buffer->flush_count = 2; + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0); + + eeprom_buffer->write(eeprom_buffer, buffer, 2); + TEST_EXPECT_EQ(test, fake_eeprom->contents[0], 0xff); + TEST_EXPECT_EQ(test, fake_eeprom->contents[1], 0xff); + /* Should have only flushed the first two bytes. */ + TEST_EXPECT_EQ(test, fake_eeprom->contents[2], 0); + } + + static int eeprom_buffer_test_init(struct test *test) + { + struct eeprom_buffer_test *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + ASSERT_NOT_ERR_OR_NULL(test, ctx); + + ctx->fake_eeprom = test_kzalloc(test, sizeof(*ctx->fake_eeprom), GFP_KERNEL); + ASSERT_NOT_ERR_OR_NULL(test, ctx->fake_eeprom); + + ctx->eeprom_buffer = new_eeprom_buffer(&ctx->fake_eeprom->parent); + ASSERT_NOT_ERR_OR_NULL(test, ctx->eeprom_buffer); + + test->priv = ctx; + + return 0; + } + + static void eeprom_buffer_test_exit(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + + destroy_eeprom_buffer(ctx->eeprom_buffer); + } + +Mocking Classes +~~~~~~~~~~~~~~~ + +Sometimes the easiest way to make assertions about behavior is to verify +certain methods or functions were called with appropriate arguments. KUnit +allows classes to be *mocked* which means that it generates subclasses whose +behavior can be specified in a test case. KUnit accomplishes this with two sets +of macros: the mock generation macros and the ``TEST_EXPECT_CALL`` macro. + +For example, let's go back to the EEPROM example; instead of faking the EEPROM, +we could have *mocked it out* with the following code: + +.. code-block:: c + + DECLARE_STRUCT_CLASS_MOCK_PREREQS(eeprom); + + DEFINE_STRUCT_CLASS_MOCK(METHOD(read), CLASS(eeprom), + RETURNS(ssize_t), + PARAMS(struct eeprom *, size_t, char *, size_t)); + + DEFINE_STRUCT_CLASS_MOCK(METHOD(write), CLASS(eeprom), + RETURNS(ssize_t), + PARAMS(struct eeprom *, size_t, const char *, size_t)); + + static int eeprom_init(struct MOCK(eeprom) *mock_eeprom) + { + struct eeprom *eeprom = mock_get_trgt(mock_eeprom); + + eeprom->read = read; + eeprom->write = write; + + return 0; + } + + DEFINE_STRUCT_CLASS_MOCK_INIT(eeprom, eeprom); + +We could use the mock in a test as follows: + +.. code-block:: c + + struct eeprom_buffer_test { + struct MOCK(eeprom) *mock_eeprom; + struct eeprom_buffer *eeprom_buffer; + }; + + static void eeprom_buffer_test_does_not_write_until_flush(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer; + struct MOCK(eeprom) *mock_eeprom = ctx->mock_eeprom; + struct mock_expectation *expectation; + char buffer[] = {0xff, 0xff}; + + eeprom_buffer->flush_count = SIZE_MAX; + + expectation = TEST_EXPECT_CALL(write(mock_get_ctrl(mock_eeprom), + test_any(test), + test_any(test), + test_any(test))); + expectation->max_calls_expected = 0; + expectation->min_calls_expected = 0; + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + eeprom_buffer->write(eeprom_buffer, buffer, 1); + + mock_validate_expectations(mock_get_ctrl(mock_eeprom)); + + expectation = TEST_EXPECT_CALL(write(mock_get_ctrl(mock_eeprom), + test_any(test), + test_memeq(test, + buffer, + ARRAY_SIZE(buffer)), + test_ulong_eq(test, 2))); + expectation->max_calls_expected = 1; + expectation->min_calls_expected = 1; + expectation->action = test_long_return(test, 2); + + eeprom_buffer->flush(eeprom_buffer); + } + + static int eeprom_buffer_test_init(struct test *test) + { + struct eeprom_buffer_test *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + ASSERT_NOT_ERR_OR_NULL(test, ctx); + + ctx->mock_eeprom = CONSTRUCT_MOCK(eeprom, test); + ASSERT_NOT_ERR_OR_NULL(test, ctx->fake_eeprom); + + ctx->eeprom_buffer = new_eeprom_buffer(mock_get_trgt(ctx->mock_eeprom)); + ASSERT_NOT_ERR_OR_NULL(test, ctx->eeprom_buffer); + + test->priv = ctx; + + return 0; + } + + static void eeprom_buffer_test_exit(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + + destroy_eeprom_buffer(ctx->eeprom_buffer); + } + +This test case tests the same thing as the +``eeprom_buffer_test_does_not_write_until_flush`` test case from the example in +the faking section. Observe that in this test case you specify how you expect +the mock to be called (technically this is both stubbing and mocking `which are +different things +https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs`_, +but KUnit combines them as many other xUnit testing libraries do) and also how +the mock should behave when those expectations are met (see +``test_long_return``). + +Mocks are extremely powerful as they allow you the finest possible granularity +for verifying how units interact, and allows the injection of arbitrary +behavior. But as Uncle Ben said, "Great power comes with great responsibility." +Mocks are not to be used lightly; they make it possible to test things which are +otherwise difficult or impossible to test, but when used improperly they have a +much higher maintenance burden than using the real thing or even a high quality +fake. + +Compare the ``eeprom_buffer_test_does_not_write_until_flush`` in the faking +example to the above version that uses mocking. It is pretty clear that the +version that uses faking is easier to read. It is also pretty clear that common +behavior between test cases would have to be duplicated with the mocking +version; the fake has the advantage of implementing desired behavior in a single +place. Finally, it is pretty clear that the fake would be much easier to +maintain. Of course what's even easier than having to maintain a fake is not +not having to maintain anything at all. Thus, + +.. important:: + Always prefer high quality fakes over mocks, and always prefer "real" code to + fakes. + +Fakes should generally be used when there is an external dependency that there +is no way around; in the kernel that usually means hardware. If you write a fake +you have to make sure it can be maintained; consequently, it is just as +important as real code and it should get its own tests to verify it works as +expected. Yes, we are telling you to write tests for your fakes. + +Of course sometimes faking something out is infeasible, or there is some code +that is just otherwise impossible to reach; generally this means that your code +should be refactored, but not always. Either way, well tested code in need of +refactoring is better than code that needs refactoring but has no tests. This +leads to the single most important testing principle that overrides all others: + +.. important:: + **Always prefer tests over no tests, no matter what!** + +For more information on class mocking see :doc:`api/class-and-function-mocking`. + +Mocking Arbitrary Functions +--------------------------- + +.. important:: + Always prefer class mocking over arbitrary function mocking where possible. + Class mocking has a much more limited scope and provides more control. + +Sometimes it is necessary to mock a function that does not use any class style +indirection. First and foremost, if you encounter this in your own code, please +rewrite it so that uses class style indirection discussed above, but if this is +in some code that is outside of your control you may use KUnit's function +mocking features. + +KUnit provides macros to allow arbitrary functions to be overridden so that the +original definition is replaced with a mock stub. For most functions, all you +have to do is label the function ``__mockable``: + +.. code-block:: c + + int __mockable example(int arg) {...} + +If a function is ``__mockable`` and a mock is defined: + +.. code-block:: c + + DEFINE_FUNCTION_MOCK(example, RETURNS(int), PARAMS(int)); + +When the function is called, the mock stub will actually be called. + +.. note:: + There is no performance penalty or potential side effects from doing this. + When not compiling for testing, ``__mockable`` compiles away. + +.. note:: + ``__mockable`` does not work on inlined functions. + +Spying +~~~~~~ + +Sometimes it is desirable to have a mock function that delegates to the original +definition in some or all circumstances. This is called *spying*: + +.. code-block:: c + + DEFINE_SPYABLE(i2c_add_adapter, RETURNS(int), PARAMS(struct i2c_adapter *)); + int REAL_ID(i2c_add_adapter)(struct i2c_adapter *adapter) + { + ... + } + +This allows the function to be overridden by a mock as with ``__mockable``; +however, it associates the original definition of the function with an alternate +symbol that KUnit can still reference. This makes it possible to mock the +function and then have the mock delegate to the original function definition +with the ``INVOKE_REAL(...)`` action: + +.. code-block:: c + + static int aspeed_i2c_test_init(struct test *test) + { + struct mock_param_capturer *adap_capturer; + struct mock_expectation *handle; + struct aspeed_i2c_test *ctx; + int ret; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + test->priv = ctx; + + handle = TEST_EXPECT_CALL( + i2c_add_adapter(capturer_to_matcher(adap_capturer))); + handle->action = INVOKE_REAL(test, i2c_add_adapter); + ret = of_fake_probe_platform_by_name(test, + "aspeed-i2c-bus", + "test-i2c-bus"); + if (ret < 0) + return ret; + + ASSERT_PARAM_CAPTURED(test, adap_capturer); + ctx->adap = mock_capturer_get(adap_capturer, struct i2c_adapter *); + + return 0; + } + +For more information on function mocking see +:doc:`api/class-and-function-mocking`. + +Platform Mocking +---------------- +The Linux kernel generally forbids normal code from accessing architecture +specific features. Instead, low level hardware features are usually abstracted +so that architecture specific code can live in the ``arch/`` directory and all +other code relies on APIs exposed by it. + +KUnit provides a mock architecture that currently allows mocking basic IO memory +accessors and in the future will provide even more. A major use case for +platform mocking is unit testing platform drivers, so KUnit also provides +helpers for this as well. + +In order to use platform mocking, ``CONFIG_PLATFORM_MOCK`` must be enabled in +your ``kunitconfig``. + +For more information on platform mocking see :doc:`api/platform-mocking`. + +Method Call Expectations +======================== +Once we have classes and methods mocked, we can place more advanced +expectations. Previously, we could only place expectations on simple return +values. With the :c:func:`TEST_EXPECT_CALL` macro, which allows you to make +assertions that a certain mocked function is called with specific arguments +given some code to be run. + +Basic Usage +----------- +Imagine we had some kind of dependency like this: + +.. code-block:: c + + struct Printer { + void (*print)(int arg); + }; + + // Printer's print + void printer_print(int arg) + { + do_something_to_print_to_screen(arg); + } + + struct Foo { + struct Printer *internal_printer; + void (*print_add_two)(struct Foo*, int); + }; + + // Foo's print_add_two: + void foo_print_add_two(struct Foo *this, int arg) + { + internal_printer->print(arg + 2); + } + +and we wanted to test ``struct Foo``'s behaviour, that ``foo->print_add_two`` +actually adds 2 to the argument passed. To properly unit test this, we create +mocks for all of ``struct Foo``'s dependencies, like ``struct Printer``. +We first setup stubs for ``MOCK(Printer)`` and its ``print`` function. + +In the real code, we'd assign a real ``struct Printer`` to the +``internal_printer`` variable in our ``struct Foo`` object, but in the +test, we'd construct a ``struct Foo`` with our ``MOCK(Printer)``. + +Finally, we can place expectations on the ``MOCK(Printer)``. + +For example: + +.. code-block:: c + + static int test_foo_add_two(struct test *test) + { + struct MOCK(Printer) *mock_printer = get_mocked_printer(); + struct Foo *foo = initialize_foo(mock_printer); + + // print() is a mocked method stub + TEST_EXPECT_CALL(print(test_any(test), test_int_eq(test, 12))); + + foo->print_add_two(foo, 10); + } + +Here, we expect that the printer's print function will be called (by default, +once), and that it will be called with the argument ``12``. Once we've placed +expectations, we can call the function we want to test to see that it behaves +as we expected. + +Matchers +-------- +Above, we see ``test_any`` and ``test_int_eq``, which are matchers. A matcher +simply asserts that the argument passed to that function call fulfills some +condition. In this case, ``test_any()`` matches any argument, and +``test_int_eq(12)`` asserts that the argument passed to that function must +equal 12. If we had called: ``foo->print_add_two(foo, 9)`` instead, the +expectation would not have been fulfilled. There are a variety of built-in +matchers: :doc:`api/class-and-function-mocking` has a section about these +matchers. + +.. note:: + :c:func:`TEST_EXPECT_CALL` only works with mocked functions and methods. + Matchers may only be used within the function inside the + :c:func:`TEST_EXPECT_CALL`. + +Additional :c:func:`EXPECT_CALL` Properties +------------------------------------------- + +The return value of :c:func:`TEST_EXPECT_CALL` is a ``struct +mock_expectation``. We can capture the value and add extra properties to it as +defined by the ``struct mock_expectation`` interface. + +Times Called +~~~~~~~~~~~~ +In the previous example, if we wanted assert that the method is never called, +we could write: + +.. code-block:: c + + ... + struct mock_expectation* handle = TEST_EXPECT_CALL(...); + handle->min_calls_expected = 0; + handle->max_calls_expected = 0; + ... + +Both those fields are set to 1 by default and can be changed to assert a range +of times that the method or function is called. + +Mocked Actions +~~~~~~~~~~~~~~ +Because ``mock_printer`` is a mock, it doesn't actually perform any task. If +the function had some side effect that ``struct Foo`` requires to have been +done, such as modifying some state, we could mock that as well. + +Each expectation has an associated ``struct mock_action`` which can be set with +``handle->action``. By default, there are two actions that mock return values. +Those can also be found in :doc:`api/class-and-function-mocking`. + +Custom actions can be defined by simply creating a ``struct mock_action`` and +assigning the appropriate function to ``do_action``. Mocked actions have access +to the parameters passed to the mocked function, as well as have the ability to +change / set the return value. + + +The Nice, the Strict, and the Naggy +=================================== +KUnit has three different mock types that can be set on a mocked class: nice +mocks, strict mocks, and naggy mocks. These are set via the corresponding macros +:c:func:`NICE_MOCK`, :c:func:`STRICT_MOCK`, and :c:func:`NAGGY_MOCK`, with naggy +mocks being the default. + +The type of mock simply dictates the behaviour the mock exhibits when +expectations are placed on it. + ++-----------------------+------------+--------------------+--------------------+ +| | **Nice** | **Naggy (default)**| **Strict** | ++-----------------------+------------+--------------------+--------------------+ +| Method called with no | Do nothing | Prints warning for | Fails test, prints | +| expectations on it | | uninteresting call | warning | +| | | | uninteresting call | ++-----------------------+------------+--------------------+--------------------+ +| Method called with no | Fails test, prints warnings, prints tried | +| matching expectations | expectations | +| on it | | ++-----------------------+------------------------------------------------------+ +| Test ends with an | Fail test, print warning | +| unfulfilled | | +| expectation | | ++-----------------------+------------------------------------------------------+ + +These macros take a ``MOCK(struct_name)`` and so should be used when retrieving +the mocked object. Following the example in :doc:`start`, there was this test +case: + +.. code-block:: c + + static void misc_example_bar_test_success(struct test *test) + { + struct MOCK(misc_example) *mock_example = test->priv; + struct misc_example *example = mock_get_trgt(mock_example); + struct mock_expectation *handle; + + handle = TEST_EXPECT_CALL(misc_example_foo(mock_get_ctrl(mock_example), + test_int_eq(test, 5))); + handle->action = int_return(test, 0); + + TEST_EXPECT_EQ(test, 0, misc_example_bar(example)); + } + +If we wanted ``mock_example`` to be a nice mock instead, we would simply write: + +.. code-block:: c + + struct MOCK(misc_example) *mock_example = NICE_MOCK(test->priv);
kunit_config.py: - parses .config and Kconfig files
kunit_kernel.py: provides helper functions to: - configure the kernel using kunitconfig - builds the kernel with the correct architecture - provides function to invoke the kernel and stream the output back
The kernel invocation is wrapped in a subprocess call within the module because regular invocation of the kernel (./linux) may modify TTY settings.
Signed-off-by: Felix Guo felixguoxiuping@gmail.com Signed-off-by: Brendan Higgins brendanhiggins@google.com --- tools/testing/kunit/.gitignore | 3 + tools/testing/kunit/kunit_config.py | 60 ++++++++++++++ tools/testing/kunit/kunit_kernel.py | 123 ++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+) create mode 100644 tools/testing/kunit/.gitignore create mode 100644 tools/testing/kunit/kunit_config.py create mode 100644 tools/testing/kunit/kunit_kernel.py
diff --git a/tools/testing/kunit/.gitignore b/tools/testing/kunit/.gitignore new file mode 100644 index 0000000000000..c791ff59a37a9 --- /dev/null +++ b/tools/testing/kunit/.gitignore @@ -0,0 +1,3 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] \ No newline at end of file diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py new file mode 100644 index 0000000000000..183bd5e758762 --- /dev/null +++ b/tools/testing/kunit/kunit_config.py @@ -0,0 +1,60 @@ +# SPDX-License-Identifier: GPL-2.0 + +import collections +import re + +CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_\w+ is not set$' +CONFIG_PATTERN = r'^CONFIG_\w+=\S+$' + +KconfigEntryBase = collections.namedtuple('KconfigEntry', ['raw_entry']) + + +class KconfigEntry(KconfigEntryBase): + + def __str__(self) -> str: + return self.raw_entry + + +class KconfigParseError(Exception): + """Error parsing Kconfig defconfig or .config.""" + + +class Kconfig(object): + """Represents defconfig or .config specified using the Kconfig language.""" + + def __init__(self): + self._entries = [] + + def entries(self): + return set(self._entries) + + def add_entry(self, entry: KconfigEntry) -> None: + self._entries.append(entry) + + def is_subset_of(self, other: "Kconfig") -> bool: + return self.entries().issubset(other.entries()) + + def write_to_file(self, path: str) -> None: + with open(path, 'w') as f: + for entry in self.entries(): + f.write(str(entry) + '\n') + + def parse_from_string(self, blob: str) -> None: + """Parses a string containing KconfigEntrys and populates this Kconfig.""" + self._entries = [] + is_not_set_matcher = re.compile(CONFIG_IS_NOT_SET_PATTERN) + config_matcher = re.compile(CONFIG_PATTERN) + for line in blob.split('\n'): + line = line.strip() + if not line: + continue + elif config_matcher.match(line) or is_not_set_matcher.match(line): + self._entries.append(KconfigEntry(line)) + elif line[0] == '#': + continue + else: + raise KconfigParseError('Failed to parse: ' + line) + + def read_from_file(self, path: str) -> None: + with open(path, 'r') as f: + self.parse_from_string(f.read()) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py new file mode 100644 index 0000000000000..87abaede50513 --- /dev/null +++ b/tools/testing/kunit/kunit_kernel.py @@ -0,0 +1,123 @@ +# SPDX-License-Identifier: GPL-2.0 + +import logging +import subprocess +import os + +import kunit_config + +KCONFIG_PATH = '.config' + +class ConfigError(Exception): + """Represents an error trying to configure the Linux kernel.""" + + +class BuildError(Exception): + """Represents an error trying to build the Linux kernel.""" + + +class LinuxSourceTreeOperations(object): + """An abstraction over command line operations performed on a source tree.""" + + def make_mrproper(self): + try: + subprocess.check_output(['make', 'mrproper']) + except OSError as e: + raise ConfigError('Could not call make command: ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + + def make_olddefconfig(self): + try: + subprocess.check_output(['make', 'ARCH=um', 'olddefconfig']) + except OSError as e: + raise ConfigError('Could not call make command: ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + + def make(self): + try: + subprocess.check_output(['make', 'ARCH=um']) + except OSError as e: + raise BuildError('Could not call execute make: ' + e) + except subprocess.CalledProcessError as e: + raise BuildError(e.output) + + def linux_bin(self, params, timeout): + """Runs the Linux UML binary. Must be named 'linux'.""" + process = subprocess.Popen( + ['./linux'] + params, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + process.wait(timeout=timeout) + return process + + +class LinuxSourceTree(object): + """Represents a Linux kernel source tree with KUnit tests.""" + + def __init__(self): + self._kconfig = kunit_config.Kconfig() + self._kconfig.read_from_file('kunitconfig') + self._ops = LinuxSourceTreeOperations() + + def clean(self): + try: + self._ops.make_mrproper() + except ConfigError as e: + logging.error(e) + return False + return True + + def build_config(self): + self._kconfig.write_to_file(KCONFIG_PATH) + try: + self._ops.make_olddefconfig() + except ConfigError as e: + logging.error(e) + return False + validated_kconfig = kunit_config.Kconfig() + validated_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(validated_kconfig): + logging.error('Provided Kconfig is not contained in validated .config!') + return False + return True + + def build_reconfig(self): + """Creates a new .config if it is not a subset of the kunitconfig.""" + if os.path.exists(KCONFIG_PATH): + existing_kconfig = kunit_config.Kconfig() + existing_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(existing_kconfig): + print('Regenerating .config ...') + os.remove(KCONFIG_PATH) + return self.build_config() + else: + return True + else: + print('Generating .config ...') + return self.build_config() + + def build_um_kernel(self): + try: + self._ops.make_olddefconfig() + self._ops.make() + except (ConfigError, BuildError) as e: + logging.error(e) + return False + used_kconfig = kunit_config.Kconfig() + used_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(used_kconfig): + logging.error('Provided Kconfig is not contained in final config!') + return False + return True + + def run_kernel(self, args=[]): + timeout = None + args.extend(['mem=256M']) + process = self._ops.linux_bin(args, timeout) + with open('test.log', 'w') as f: + for line in process.stdout: + f.write(line.rstrip().decode('ascii') + '\n') + yield line.rstrip().decode('ascii')
The KUnit wrapper script interfaces with the two modules (kunit_config.py and kunit_kernel.py) and provides a command line interface for running KUnit tests. This interface allows the caller to specify options like test timeouts. The script handles configuring, building and running the kernel and tests.
The output parser (kunit_parser.py) simply strips out all the output from the kernel that is outputted as part of it's initialization sequence. This ensures that only the output from KUnit is displayed on the screen.
A full version of the output is written to test.log, or can be seen by passing --raw_output to the wrapper script.
Signed-off-by: Felix Guo felixguoxiuping@gmail.com Signed-off-by: Brendan Higgins brendanhiggins@google.com --- tools/testing/kunit/kunit.py | 40 +++++++++++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 3 +-- tools/testing/kunit/kunit_parser.py | 24 +++++++++++++++++ 3 files changed, 65 insertions(+), 2 deletions(-) create mode 100755 tools/testing/kunit/kunit.py create mode 100644 tools/testing/kunit/kunit_parser.py
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py new file mode 100755 index 0000000000000..1356be404996b --- /dev/null +++ b/tools/testing/kunit/kunit.py @@ -0,0 +1,40 @@ +#!/usr/bin/python3 +# SPDX-License-Identifier: GPL-2.0 + +# A thin wrapper on top of the KUnit Kernel + +import argparse +import sys +import os + +import kunit_config +import kunit_kernel +import kunit_parser + +parser = argparse.ArgumentParser(description='Runs KUnit tests.') + +parser.add_argument('--raw_output', help='don't format output from kernel', + action='store_true') + +parser.add_argument('--timeout', help='maximum number of seconds to allow for ' + 'all tests to run. This does not include time taken to ' + 'build the tests.', type=int, default=300, + metavar='timeout') + +cli_args = parser.parse_args() +linux = kunit_kernel.LinuxSourceTree() + +success = linux.build_reconfig() +if not success: + quit() + +print('Building KUnit Kernel ...') +success = linux.build_um_kernel() +if not success: + quit() + +print('Starting KUnit Kernel ...') +if cli_args.raw_output: + kunit_parser.raw_output(linux.run_kernel(timeout=cli_args.timeout)) +else: + kunit_parser.parse_run_tests(linux.run_kernel(timeout=cli_args.timeout)) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 87abaede50513..c1259b174f7d4 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -113,8 +113,7 @@ class LinuxSourceTree(object): return False return True
- def run_kernel(self, args=[]): - timeout = None + def run_kernel(self, args=[], timeout=None): args.extend(['mem=256M']) process = self._ops.linux_bin(args, timeout) with open('test.log', 'w') as f: diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py new file mode 100644 index 0000000000000..1dff3adb73bd3 --- /dev/null +++ b/tools/testing/kunit/kunit_parser.py @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 + +import re + +kunit_start_re = re.compile('console .* enabled') +kunit_end_re = re.compile('List of all partitions:') + +def isolate_kunit_output(kernel_output): + started = False + for line in kernel_output: + if kunit_start_re.match(line): + started = True + elif kunit_end_re.match(line): + break + elif started: + yield line + +def raw_output(kernel_output): + for line in kernel_output: + print(line) + +def parse_run_tests(kernel_output): + for output in isolate_kunit_output(kernel_output): + print(output)
- added colors to displayed output - added timing and summary
Signed-off-by: Felix Guo felixguoxiuping@gmail.com Signed-off-by: Brendan Higgins brendanhiggins@google.com --- tools/testing/kunit/kunit.py | 20 ++++++- tools/testing/kunit/kunit_parser.py | 93 ++++++++++++++++++++++++++++- 2 files changed, 109 insertions(+), 4 deletions(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 1356be404996b..b36c7b7924567 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -6,6 +6,7 @@ import argparse import sys import os +import time
import kunit_config import kunit_kernel @@ -24,17 +25,32 @@ parser.add_argument('--timeout', help='maximum number of seconds to allow for ' cli_args = parser.parse_args() linux = kunit_kernel.LinuxSourceTree()
+config_start = time.time() success = linux.build_reconfig() +config_end = time.time() if not success: quit()
-print('Building KUnit Kernel ...') +kunit_parser.print_with_timestamp('Building KUnit Kernel ...') + +build_start = time.time() success = linux.build_um_kernel() +build_end = time.time() if not success: quit()
-print('Starting KUnit Kernel ...') +kunit_parser.print_with_timestamp('Starting KUnit Kernel ...') +test_start = time.time() + if cli_args.raw_output: kunit_parser.raw_output(linux.run_kernel(timeout=cli_args.timeout)) else: kunit_parser.parse_run_tests(linux.run_kernel(timeout=cli_args.timeout)) + +test_end = time.time() + +kunit_parser.print_with_timestamp(( + "Elapsed time: %.3fs total, %.3fs configuring, %.3fs " + + "building, %.3fs running.\n") % (test_end - config_start, + config_end - config_start, build_end - build_start, + test_end - test_start)) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 1dff3adb73bd3..d9051e407d5a7 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0
import re +from datetime import datetime
kunit_start_re = re.compile('console .* enabled') kunit_end_re = re.compile('List of all partitions:') @@ -19,6 +20,94 @@ def raw_output(kernel_output): for line in kernel_output: print(line)
+DIVIDER = "=" * 30 + +RESET = '\033[0;0m' + +def red(text): + return '\033[1;31m' + text + RESET + +def yellow(text): + return '\033[1;33m' + text + RESET + +def green(text): + return '\033[1;32m' + text + RESET + +def print_with_timestamp(message): + print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) + +def print_log(log): + for m in log: + print_with_timestamp(m) + def parse_run_tests(kernel_output): - for output in isolate_kunit_output(kernel_output): - print(output) + test_case_output = re.compile('^kunit .*?: (.*)$') + + test_module_success = re.compile('^kunit .*: all tests passed') + test_module_fail = re.compile('^kunit .*: one or more tests failed') + + test_case_success = re.compile('^kunit (.*): (.*) passed') + test_case_fail = re.compile('^kunit (.*): (.*) failed') + test_case_crash = re.compile('^kunit (.*): (.*) crashed') + + total_tests = set() + failed_tests = set() + crashed_tests = set() + + def get_test_name(match): + return match.group(1) + ":" + match.group(2) + + current_case_log = [] + def end_one_test(match, log): + log.clear() + total_tests.add(get_test_name(match)) + + print_with_timestamp(DIVIDER) + for line in isolate_kunit_output(kernel_output): + # Ignore module output: + if (test_module_success.match(line) or + test_module_fail.match(line)): + print_with_timestamp(DIVIDER) + continue + + match = re.match(test_case_success, line) + if match: + print_with_timestamp(green("[PASSED] ") + + get_test_name(match)) + end_one_test(match, current_case_log) + continue + + match = re.match(test_case_fail, line) + # Crashed tests will report as both failed and crashed. We only + # want to show and count it once. + if match and get_test_name(match) not in crashed_tests: + failed_tests.add(get_test_name(match)) + print_with_timestamp(red("[FAILED] " + + get_test_name(match))) + print_log(map(yellow, current_case_log)) + print_with_timestamp("") + end_one_test(match, current_case_log) + continue + + match = re.match(test_case_crash, line) + if match: + crashed_tests.add(get_test_name(match)) + print_with_timestamp(yellow("[CRASH] " + + get_test_name(match))) + print_log(current_case_log) + print_with_timestamp("") + end_one_test(match, current_case_log) + continue + + # Strip off the `kunit module-name:` prefix + match = re.match(test_case_output, line) + if match: + current_case_log.append(match.group(1)) + else: + current_case_log.append(line) + + fmt = green if (len(failed_tests) + len(crashed_tests) == 0) else red + print_with_timestamp( + fmt("Testing complete. %d tests run. %d failed. %d crashed." % + (len(total_tests), len(failed_tests), len(crashed_tests)))) +
Added myself as maintainer of KUnit, the Linux kernel's unit testing framework.
Signed-off-by: Brendan Higgins brendanhiggins@google.com --- MAINTAINERS | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index 544cac829cf44..9c3d34f0062ad 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7801,6 +7801,21 @@ S: Maintained F: tools/testing/selftests/ F: Documentation/dev-tools/kselftest*
+KERNEL UNIT TESTING FRAMEWORK (KUnit) +M: Brendan Higgins brendanhiggins@google.com +L: kunit-dev@googlegroups.com +W: https://google.github.io/kunit-docs/third_party/kernel/docs/ +S: Maintained +F: Documentation/kunit/ +F: arch/um/include/asm/io-mock-shared.h +F: arch/um/include/asm/io-mock.h +F: arch/um/kernel/io-mock.c +F: drivers/base/platform-mock.c +F: include/linux/platform_device_mock.h +F: include/kunit/ +F: kunit/ +F: tools/testing/kunit/ + KERNEL USERMODE HELPER M: "Luis R. Rodriguez" mcgrof@kernel.org L: linux-kernel@vger.kernel.org
On Tue, Oct 16, 2018 at 04:50:49PM -0700, Brendan Higgins wrote:
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in under a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more.
## What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code.
## Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed.
## More information on KUnit
There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here: https://google.github.io/kunit-docs/third_party/kernel/docs/
We've started our own unit test framework (very early, and not any real infrastructure yet) under drivers/gpu/drm/selftests. They're all meant to test functions and small pieces of functionality of our libraries in isolation, without any need for a real (or virtual) gpu driver. It kinda integrates both with kselftests and with our graphics test suite, but directly running tests using UML as part of the build process sounds much, much better. Having proper and standardized infrastructure for kernel unit tests sounds terrific.
In other words: I want.
Please keep dri-devel@lists.freedesktop.org in the loop on future versions.
Cheers, Daniel
-----Original Message----- From: Brendan Higgins
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
I'm interested in this, and think the kernel might benefit from this, but I have lots of questions.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel.
This is stated here and a few places in the documentation. Just to clarify, KUnit works by compiling the unit under test, along with the test code itself, and then runs it on the machine where the compilation took place? Is this right? How does cross-compiling enter into the equation? If not what I described, then what exactly is happening?
Sorry - I haven't had time to look through the patches in detail.
Another issue is, what requirements does this place on the tested code? Is extra instrumentation required? I didn't see any, but I didn't look exhaustively at the code.
Are all unit tests stored separately from the unit-under-test, or are they expected to be in the same directory? Who is expected to maintain the unit tests? How often are they expected to change? (Would it be every time the unit-under-test changed?)
Does the test code require the same level of expertise to write and maintain as the unit-under-test code? That is, could this be a new opportunity for additional developers (especially relative newcomers) to add value to the kernel by writing and maintaining test code, or does this add to the already large burden of code maintenance for our existing maintainers.
Thanks, -- Tim
...
On Wed, Oct 17, 2018 at 10:49 AM Tim.Bird@sony.com wrote:
-----Original Message----- From: Brendan Higgins
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
I'm interested in this, and think the kernel might benefit from this, but I have lots of questions.
Awesome!
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel.
This is stated here and a few places in the documentation. Just to clarify, KUnit works by compiling the unit under test, along with the test code itself, and then runs it on the machine where the compilation took place? Is this right? How does cross-compiling enter into the equation? If not what I described, then what exactly is happening?
Yep, that's exactly right!
The test and the code under test are linked together in the same binary and are compiled under Kbuild. Right now I am linking everything into a UML kernel, but I would ultimately like to make tests compile into completely independent test binaries. So each test file would get compiled into its own test binary and would link against only the code needed to run the test, but we are a bit of a ways off from that.
For now, tests compile as part of a UML kernel and a test script boots the UML kernel, tests run as part of the boot process, and the script extracts test results and reports them.
I intentionally made it so the KUnit test libraries could be relatively easily ported to other architectures, but in the long term, tests that depend on being built into a real kernel that boots on real hardware would be a lot more difficult to maintain and we would never be able to provide the kind of resources and infrastructure as we could for tests that run as normal user space binaries.
Does that answer your question?
Sorry - I haven't had time to look through the patches in detail.
Another issue is, what requirements does this place on the tested code? Is extra instrumentation required? I didn't see any, but I didn't look exhaustively at the code.
Nope, no special instrumentation. As long as the code under tests can be compiled under COMPILE_TEST for the host architecture, you should be able to use KUnit.
Are all unit tests stored separately from the unit-under-test, or are they expected to be in the same directory? Who is expected to maintain the unit tests? How often are they expected to change? (Would it be every time the unit-under-test changed?)
Tests are in the same directory as the code under test. For example, if I have a driver drivers/i2c/busses/i2c-aspeed.c, I would write a test drivers/i2c/busses/i2c-aspeed-test.c (that's my opinion anyway).
Unit tests should be the responsibility of the person who is responsible for the code. So one way to do this would be that unit tests should be the responsibility of the maintainer who would in turn require that new tests are written for any new code added, and that all tests should pass for every patch sent for review.
A well written unit test tests public interfaces (by public I just mean functions exported outside of a .c file, so non-static functions and functions which are shared as a member of a struct) so a unit test should change at a slower rate than the code under test, but you would likely have to change the test anytime the public interface changes (intended behavior changes, function signature changes, new public feature added, etc). More succinctly, if the contract that your code provide changes your test should probably change, if the contract doesn't change, your test probably shouldn't change. Does that make sense?
Does the test code require the same level of expertise to write and maintain as the unit-under-test code? That is, could this be a new opportunity for additional developers (especially relative newcomers) to add value to the kernel by writing and maintaining test code, or does this add to the already large burden of code maintenance for our existing maintainers.
So a couple things, in order to write a unit test, the person who writes the test must understand what the code they are testing is supposed to do. To some extent that will probably require someone with some expertise to ensure that the test makes sense, and indeed a change that breaks a test should be accompanied by a update to the test.
On the other hand, I think understanding what pre-existing code does and is supposed to do is much easier than writing new code from scratch, and probably doesn't require too much expertise. I actually did a bit of an experiment internally on this: I had some people with no prior knowledge of the kernel write some tests for existing kernel code and they were able to do it with only minimal guidance. I was so happy with the result that I was already thinking that it might have some potential for onboarding newcomers.
Now, how much burden does this add to maintainers? As someone who pretty regularly reviews code that come in with unit tests and code that comes in without unit tests. I find it much easier to review code that comes in with unit tests. I would actually say that from the standpoint of being an owner of a code base, unit tests actually reduce the amount of work I have to do overall. Code with unit tests is usually cleaner, the tests tell me exactly what the code is supposed to do, and I can run the tests (or ideally have an automated service run the tests) that tell me that the code actually does what the tests say it should. Even when it comes to writing code I find that writing code with unit tests ends up saving me time overall.
On Tue, Oct 16, 2018 at 6:53 PM Brendan Higgins brendanhiggins@google.com wrote:
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in under a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more.
I very much like this. The DT code too has unit tests with our own, simple infrastructure. They too can run under UML (and every other arch).
Rob
On 10/16/18 4:50 PM, Brendan Higgins wrote:
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Hi,
Just a general comment:
Documentation/process/submitting-patches.rst says: <<Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.>>
That also means saying things like: ... test: add instead of ... test: added and "enable" instead of "enabled" and "improve" instead of "improved" and "implement" instead of "implemented".
thanks.
On Wed, Oct 17, 2018 at 4:12 PM Randy Dunlap rdunlap@infradead.org wrote:
On 10/16/18 4:50 PM, Brendan Higgins wrote:
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Hi,
Just a general comment:
Documentation/process/submitting-patches.rst says: <<Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.>>
Thanks! I will fix this in the next revision.
On Tue, Oct 16, 2018 at 4:54 PM Brendan Higgins brendanhiggins@google.com wrote:
This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in under a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more.
## What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code.
## Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed.
## More information on KUnit
There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here: https://google.github.io/kunit-docs/third_party/kernel/docs/
Nice! I've been using mocking techniques in kernel code for the libnvdimm test infrastructure in tools/testing/nvdimm/. It's part unit test infrastructure, part emulation, and I've always had the feeling it's all a bit too adhoc. I'm going to take a look and see what can be converted to kunit. Please include linux-nvdimm@lists.01.org on future postings.
I'll shamelessly plug my lwn article about unit testing https://lwn.net/Articles/654071/ because it's always good to find fellow co-travelers to compare notes and advocate for more test oriented kernel development.
On Wed, Oct 17, 2018 at 8:55 PM Dan Williams dan.j.williams@intel.com wrote: <snip>
## More information on KUnit
There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here: https://google.github.io/kunit-docs/third_party/kernel/docs/
Nice! I've been using mocking techniques in kernel code for the libnvdimm test infrastructure in tools/testing/nvdimm/. It's part unit test infrastructure, part emulation, and I've always had the feeling it's all a bit too adhoc. I'm going to take a look and see what can be converted to kunit. Please include linux-nvdimm@lists.01.org on future postings.
Great to hear!
Interesting, is this kind of like the nfsim stuff?
I'll shamelessly plug my lwn article about unit testing https://lwn.net/Articles/654071/ because it's always good to find fellow co-travelers to compare notes and advocate for more test oriented kernel development.
Most definitely! I will take a look, and be in touch.
Cheers!
linux-kselftest-mirror@lists.linaro.org
-
Brendan Higgins -
Dan Williams -
Daniel Vetter -
Joe Perches -
Kieran Bingham -
Randy Dunlap -
Rob Herring -
Rob Herring -
Tim.Bird＠sony.com