Nowadays, there are increasing requirements to benchmark the performance
of dma_map and dma_unmap particually while the device is attached to an
IOMMU.
This patchset provides the benchmark infrastruture for streaming DMA
mapping. The architecture of the code is pretty much similar with GUP
benchmark:
* mm/gup_benchmark.c provides kernel interface;
* tools/testing/selftests/vm/gup_benchmark.c provides user program to
call the interface provided by mm/gup_benchmark.c.
In our case, kernel/dma/map_benchmark.c is like mm/gup_benchmark.c;
tools/testing/selftests/dma/dma_map_benchmark.c is like tools/testing/
selftests/vm/gup_benchmark.c
A major difference with GUP benchmark is DMA_MAP benchmark needs to run
on a device. Considering one board with below devices and IOMMUs
device A ------- IOMMU 1
device B ------- IOMMU 2
device C ------- non-IOMMU
Different devices might attach to different IOMMU or non-IOMMU. To make
benchmark run, we can either
* create a virtual device and hack the kernel code to attach the virtual
device to IOMMU1, IOMMU2 or non-IOMMU.
* use the existing driver_override mechinism, unbind device A,B, or c from
their original driver and bind them to "dma_map_benchmark" platform_driver
or pci_driver for benchmarking.
In this patchset, I prefer to use the driver_override and avoid the various
hack in kernel. We can dynamically switch devices behind different IOMMUs
to get the performance of dma map on IOMMU or non-IOMMU.
Barry Song (2):
dma-mapping: add benchmark support for streaming DMA APIs
selftests/dma: add test application for DMA_MAP_BENCHMARK
MAINTAINERS | 6 +
kernel/dma/Kconfig | 8 +
kernel/dma/Makefile | 1 +
kernel/dma/map_benchmark.c | 202 ++++++++++++++++++
tools/testing/selftests/dma/Makefile | 6 +
tools/testing/selftests/dma/config | 1 +
.../testing/selftests/dma/dma_map_benchmark.c | 72 +++++++
7 files changed, 296 insertions(+)
create mode 100644 kernel/dma/map_benchmark.c
create mode 100644 tools/testing/selftests/dma/Makefile
create mode 100644 tools/testing/selftests/dma/config
create mode 100644 tools/testing/selftests/dma/dma_map_benchmark.c
--
2.25.1
For simplcity, strip all trailing whitespace from parsed output.
I imagine no one is printing out meaningful trailing whitespace via
KUNIT_FAIL() or similar, and that if they are, they really shouldn't.
`isolate_kunit_output()` yielded liens with trailing \n, which results
in artifacty output like this:
$ ./tools/testing/kunit/kunit.py run
[16:16:46] [FAILED] example_simple_test
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
After this change:
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
We should *not* be expecting lines to end with \n in kunit_tool_test.py
for this reason.
Do the same for `raw_output()` as well which suffers from the same
issue.
This is a followup to [1], but rebased onto kunit-fixes to pick up the
other raw_output() fix and fixes for kunit_tool_test.py.
[1] https://lore.kernel.org/linux-kselftest/20201020233219.4146059-1-dlatypov@g…
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit_parser.py | 3 ++-
tools/testing/kunit/kunit_tool_test.py | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 84a1af2581f5..edd6fbd1cf18 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -54,6 +54,7 @@ kunit_end_re = re.compile('(List of all partitions:|'
def isolate_kunit_output(kernel_output):
started = False
for line in kernel_output:
+ line = line.rstrip() # line always has a trailing \n
if kunit_start_re.search(line):
prefix_len = len(line.split('TAP version')[0])
started = True
@@ -65,7 +66,7 @@ def isolate_kunit_output(kernel_output):
def raw_output(kernel_output):
for line in kernel_output:
- print(line)
+ print(line.rstrip())
DIVIDER = '=' * 60
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 0b60855fb819..497ab51bc170 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -102,7 +102,7 @@ class KUnitParserTest(unittest.TestCase):
'test_data/test_output_isolated_correctly.log')
file = open(log_path)
result = kunit_parser.isolate_kunit_output(file.readlines())
- self.assertContains('TAP version 14\n', result)
+ self.assertContains('TAP version 14', result)
self.assertContains(' # Subtest: example', result)
self.assertContains(' 1..2', result)
self.assertContains(' ok 1 - example_simple_test', result)
@@ -115,7 +115,7 @@ class KUnitParserTest(unittest.TestCase):
'test_data/test_pound_sign.log')
with open(log_path) as file:
result = kunit_parser.isolate_kunit_output(file.readlines())
- self.assertContains('TAP version 14\n', result)
+ self.assertContains('TAP version 14', result)
self.assertContains(' # Subtest: kunit-resource-test', result)
self.assertContains(' 1..5', result)
self.assertContains(' ok 1 - kunit_resource_test_init_resources', result)
base-commit: cab67acc8a18d7c6f1850313e3da1a030abe8fc4
--
2.29.1.341.ge80a0c044ae-goog
For simplcity, strip all trailing whitespace from parsed output.
I imagine no one is printing out meaningful trailing whitespace via
KUNIT_FAIL() or similar, and that if they are, they really shouldn't.
At some point, the lines from `isolate_kunit_output()` started having
trailing \n, which results in artifacty output like this:
$ ./tools/testing/kunit/kunit.py run
[16:16:46] [FAILED] example_simple_test
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
After this change:
[16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
[16:16:46] Expected 1 + 1 == 3, but
[16:16:46] 1 + 1 == 2
[16:16:46] 3 == 3
[16:16:46] not ok 1 - example_simple_test
[16:16:46]
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit_parser.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 8019e3dd4c32..e68b1c66a73f 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -342,7 +342,8 @@ def parse_run_tests(kernel_output) -> TestResult:
total_tests = 0
failed_tests = 0
crashed_tests = 0
- test_result = parse_test_result(list(isolate_kunit_output(kernel_output)))
+ test_result = parse_test_result(list(
+ l.rstrip() for l in isolate_kunit_output(kernel_output)))
if test_result.status == TestStatus.NO_TESTS:
print(red('[ERROR] ') + yellow('no tests run!'))
elif test_result.status == TestStatus.FAILURE_TO_PARSE_TESTS:
base-commit: c4d6fe7311762f2e03b3c27ad38df7c40c80cc93
--
2.29.0.rc1.297.gfa9743e501-goog
When JSON support was added in [1], the KunitParseRequest tuple was
updated to contain a 'build_dir' field, but kunit.py parse doesn't
accept --build_dir as an option. The code nevertheless tried to access
it, resulting in this error:
AttributeError: 'Namespace' object has no attribute 'build_dir'
Given that the parser only uses the build_dir variable to set the
'build_environment' json field, we set it to None (which gives the JSON
'null') for now. Ultimately, we probably do want to be able to set this,
but since it's new functionality which (for the parse subcommand) never
worked, this is the quickest way of getting it back up and running.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/c…
Fixes: 21a6d1780d5bbfca0ce9b8104ca6233502fcbf86 ("kunit: tool: allow generating test results in JSON")
Signed-off-by: David Gow <davidgow(a)google.com>
---
This is a quick fix because kunit.py parse is completely broken: it
appears it was introduced in the rebase of the JSON parser after the
separation of concerns patch.
tools/testing/kunit/kunit.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index ebf5f5763dee..a6d5f219f714 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -337,7 +337,7 @@ def main(argv, linux=None):
kunit_output = f.read().splitlines()
request = KunitParseRequest(cli_args.raw_output,
kunit_output,
- cli_args.build_dir,
+ None,
cli_args.json)
result = parse_tests(request)
if result.status != KunitStatus.SUCCESS:
base-commit: c4d6fe7311762f2e03b3c27ad38df7c40c80cc93
--
2.29.0.rc1.297.gfa9743e501-goog
Currently --raw_output means nothing gets shown.
Why?
Because `raw_output()` has a `yield` and therefore is a generator, which
means it only executes when you ask it for a value.
Given no one actually is using it as a generator (checked via the added
type annotation), drop the yield so we actually print the output.
Also strip off the trailing \n (and any other whitespace) to avoid
[<601d6d3a>] ? printk+0x0/0x9b
[<601e5058>] ? kernel_init+0x23/0x14b
[<600170d2>] ? new_thread_handler+0x82/0xc0
making the output unreadable.
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit_parser.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 8019e3dd4c32..c44bb7c27ce6 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -63,10 +63,9 @@ def isolate_kunit_output(kernel_output):
elif started:
yield line[prefix_len:] if prefix_len > 0 else line
-def raw_output(kernel_output):
+def raw_output(kernel_output) -> None:
for line in kernel_output:
- print(line)
- yield line
+ print(line.rstrip())
DIVIDER = '=' * 60
base-commit: 07e0887302450a62f51dba72df6afb5fabb23d1c
--
2.29.1.341.ge80a0c044ae-goog
This is a bit of a mixed bag.
The background is that I have some sort() and list_sort() rework
planned, but as part of that series I want to extend their their test
suites somewhat to make sure I don't goof up - and I want to use lots
of random list lengths with random contents to increase the chance of
somebody eventually hitting "hey, sort() is broken when the length is
3 less than a power of 2 and only the last two elements are out of
order". But when such a case is hit, it's vitally important that the
developer can reproduce the exact same test case, which means using a
deterministic sequence of random numbers.
Since Petr noticed [1] the non-determinism in test_printf in
connection with Arpitha's work on rewriting it to kunit, this prompted
me to use test_printf as a first place to apply that principle, and
get the infrastructure in place that will avoid repeating the "module
parameter/seed the rnd_state/report the seed used" boilerplate in each
module.
Shuah, assuming the kselftest_module.h changes are ok, I think it's
most natural if you carry these patches, though I'd be happy with any
other route as well.
[1] https://lore.kernel.org/lkml/20200821113710.GA26290@alley/
Rasmus Villemoes (4):
prandom.h: add *_state variant of prandom_u32_max
kselftest_module.h: unconditionally expand the KSTM_MODULE_GLOBALS()
macro
kselftest_module.h: add struct rnd_state and seed parameter
lib/test_printf.c: use deterministic sequence of random numbers
Documentation/dev-tools/kselftest.rst | 2 --
include/linux/prandom.h | 29 ++++++++++++++++
lib/test_bitmap.c | 3 --
lib/test_printf.c | 13 ++++---
lib/test_strscpy.c | 2 --
tools/testing/selftests/kselftest_module.h | 40 ++++++++++++++++++----
6 files changed, 72 insertions(+), 17 deletions(-)
--
2.23.0
Currently the tool redirects make stdout + stderr, and only shows them
if the make command fails.
This means build warnings aren't shown to the user.
This change prints the contents of stderr even if make succeeds, under
the assumption these are only build warnings or other messages the user
likely wants to see.
We drop stdout from the raised exception since we can no longer easily
collate stdout and stderr and just showing the stderr seems fine.
Example with a warning:
[14:56:35] Building KUnit Kernel ...
../lib/kunit/kunit-test.c: In function ‘kunit_test_successful_try’:
../lib/kunit/kunit-test.c:19:6: warning: unused variable ‘unused’ [-Wunused-variable]
19 | int unused;
| ^~~~~~
[14:56:40] Starting KUnit Kernel ...
Note the stderr has a trailing \n, and since we use print, we add
another, but it helps separate make and kunit.py output.
Example with a build error:
[15:02:45] Building KUnit Kernel ...
ERROR:root:../lib/kunit/kunit-test.c: In function ‘kunit_test_successful_try’:
../lib/kunit/kunit-test.c:19:2: error: unknown type name ‘invalid_type’
19 | invalid_type *test = data;
| ^~~~~~~~~~~~
...
Signed-off-by: Daniel Latypov <dlatypov(a)google.com>
---
tools/testing/kunit/kunit_kernel.py | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index b557b1e93f98..326e82746d41 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -82,11 +82,16 @@ class LinuxSourceTreeOperations(object):
if build_dir:
command += ['O=' + build_dir]
try:
- subprocess.check_output(command, stderr=subprocess.STDOUT)
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
except OSError as e:
- raise BuildError('Could not call execute make: ' + str(e))
- except subprocess.CalledProcessError as e:
- raise BuildError(e.output.decode())
+ raise BuildError('Could not call make command: ' + str(e))
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
def linux_bin(self, params, timeout, build_dir, outfile):
"""Runs the Linux UML binary. Must be named 'linux'."""
base-commit: 07e0887302450a62f51dba72df6afb5fabb23d1c
--
2.29.1.341.ge80a0c044ae-goog