- Linux-kselftest-mirror - lists.linaro.org

[PATCH kunit-next] kunit: kunit_tool: Separate out config/build/exec/parse

by David Gow

Add new subcommands to kunit.py to allow stages of the existing 'run' subcommand to be run independently: - 'config': Verifies that .config is a subset of .kunitconfig - 'build': Compiles a UML kernel for KUnit - 'exec': Runs the kernel, and outputs the test results. - 'parse': Parses test results from a file or stdin The 'run' command continues to behave as before. Signed-off-by: David Gow <davidgow(a)google.com> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- Changes since the RFC[1]: - Rebased on top of kselftest/kunit, adding support for --alltests and --make_options - Fixed an issue where the help text wouldn't print properly if an invalid subcommand were specified. - Changed 'parse' subcommand to make the filename optional (and default to stdin) as suggested by Brendan. [1]: https://lkml.org/lkml/2020/3/11/1225 tools/testing/kunit/kunit.py | 293 ++++++++++++++++++++----- tools/testing/kunit/kunit_tool_test.py | 63 +++++- 2 files changed, 296 insertions(+), 60 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 7dca74774dd2..b01838b6f5f9 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -20,8 +20,17 @@ import kunit_config import kunit_kernel import kunit_parser -KunitResult = namedtuple('KunitResult', ['status','result']) +KunitResult = namedtuple('KunitResult', ['status','result','elapsed_time']) +KunitConfigRequest = namedtuple('KunitConfigRequest', + ['build_dir', 'defconfig', 'make_options']) +KunitBuildRequest = namedtuple('KunitBuildRequest', + ['jobs', 'build_dir', 'alltests', + 'make_options']) +KunitExecRequest = namedtuple('KunitExecRequest', + ['timeout', 'build_dir', 'alltests']) +KunitParseRequest = namedtuple('KunitParseRequest', + ['raw_output', 'input_data']) KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir', 'defconfig', 'alltests', 'make_options']) @@ -46,14 +55,25 @@ def get_kernel_root_path(): sys.exit(1) return parts[0] -def run_tests(linux: kunit_kernel.LinuxSourceTree, - request: KunitRequest) -> KunitResult: +def config_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitConfigRequest) -> KunitResult: + kunit_parser.print_with_timestamp('Configuring KUnit Kernel ...') + config_start = time.time() + if request.defconfig: + create_default_kunitconfig() success = linux.build_reconfig(request.build_dir, request.make_options) config_end = time.time() if not success: - return KunitResult(KunitStatus.CONFIG_FAILURE, 'could not configure kernel') + return KunitResult(KunitStatus.CONFIG_FAILURE, + 'could not configure kernel', + config_end - config_start) + return KunitResult(KunitStatus.SUCCESS, + 'configured kernel successfully', + config_end - config_start) +def build_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitBuildRequest) -> KunitResult: kunit_parser.print_with_timestamp('Building KUnit Kernel ...') build_start = time.time() @@ -64,78 +84,167 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, build_end = time.time() if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') + if not success: + return KunitResult(KunitStatus.BUILD_FAILURE, + 'could not build kernel', + build_end - build_start) + return KunitResult(KunitStatus.SUCCESS, + 'built kernel successfully', + build_end - build_start) +def exec_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitExecRequest) -> KunitResult: kunit_parser.print_with_timestamp('Starting KUnit Kernel ...') test_start = time.time() - kunit_output = linux.run_kernel( + result = linux.run_kernel( timeout=None if request.alltests else request.timeout, build_dir=request.build_dir) + + test_end = time.time() + + return KunitResult(KunitStatus.SUCCESS, + result, + test_end - test_start) + +def parse_tests(request: KunitParseRequest) -> KunitResult: + parse_start = time.time() + + test_result = kunit_parser.TestResult(kunit_parser.TestStatus.SUCCESS, + [], + 'Tests not Parsed.') if request.raw_output: - raw_output = kunit_parser.raw_output(kunit_output) - isolated = list(kunit_parser.isolate_kunit_output(raw_output)) - test_result = kunit_parser.parse_test_result(isolated) + kunit_parser.raw_output(request.input_data) else: - test_result = kunit_parser.parse_run_tests(kunit_output) - test_end = time.time() + test_result = kunit_parser.parse_run_tests(request.input_data) + parse_end = time.time() + + if test_result.status != kunit_parser.TestStatus.SUCCESS: + return KunitResult(KunitStatus.TEST_FAILURE, test_result, + parse_end - parse_start) + + return KunitResult(KunitStatus.SUCCESS, test_result, + parse_end - parse_start) + + +def run_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitRequest) -> KunitResult: + run_start = time.time() + + config_request = KunitConfigRequest(request.build_dir, + request.defconfig, + request.make_options) + config_result = config_tests(linux, config_request) + if config_result.status != KunitStatus.SUCCESS: + return config_result + + build_request = KunitBuildRequest(request.jobs, request.build_dir, + request.alltests, + request.make_options) + build_result = build_tests(linux, build_request) + if build_result.status != KunitStatus.SUCCESS: + return build_result + + exec_request = KunitExecRequest(request.timeout, request.build_dir, + request.alltests) + exec_result = exec_tests(linux, exec_request) + if exec_result.status != KunitStatus.SUCCESS: + return exec_result + + parse_request = KunitParseRequest(request.raw_output, + exec_result.result) + parse_result = parse_tests(parse_request) + + run_end = time.time() kunit_parser.print_with_timestamp(( 'Elapsed time: %.3fs total, %.3fs configuring, %.3fs ' + 'building, %.3fs running\n') % ( - test_end - config_start, - config_end - config_start, - build_end - build_start, - test_end - test_start)) + run_end - run_start, + config_result.elapsed_time, + build_result.elapsed_time, + exec_result.elapsed_time)) + return parse_result + +def add_common_opts(parser): + parser.add_argument('--build_dir', + help='As in the make command, it specifies the build ' + 'directory.', + type=str, default='', metavar='build_dir') + parser.add_argument('--make_options', + help='X=Y make option, can be repeated.', + action='append') + parser.add_argument('--alltests', + help='Run all KUnit tests through allyesconfig', + action='store_true') + +def add_config_opts(parser): + parser.add_argument('--defconfig', + help='Uses a default .kunitconfig.', + action='store_true') + +def add_build_opts(parser): + parser.add_argument('--jobs', + help='As in the make command, "Specifies the number of ' + 'jobs (commands) to run simultaneously."', + type=int, default=8, metavar='jobs') + +def add_exec_opts(parser): + parser.add_argument('--timeout', + help='maximum number of seconds to allow for all tests ' + 'to run. This does not include time taken to build the ' + 'tests.', + type=int, + default=300, + metavar='timeout') + +def add_parse_opts(parser): + parser.add_argument('--raw_output', help='don\'t format output from kernel', + action='store_true') - if test_result.status != kunit_parser.TestStatus.SUCCESS: - return KunitResult(KunitStatus.TEST_FAILURE, test_result) - else: - return KunitResult(KunitStatus.SUCCESS, test_result) def main(argv, linux=None): parser = argparse.ArgumentParser( description='Helps writing and running KUnit tests.') subparser = parser.add_subparsers(dest='subcommand') + # The 'run' command will config, build, exec, and parse in one go. run_parser = subparser.add_parser('run', help='Runs KUnit tests.') - run_parser.add_argument('--raw_output', help='don\'t format output from kernel', - action='store_true') - - run_parser.add_argument('--timeout', - help='maximum number of seconds to allow for all tests ' - 'to run. This does not include time taken to build the ' - 'tests.', - type=int, - default=300, - metavar='timeout') - - run_parser.add_argument('--jobs', - help='As in the make command, "Specifies the number of ' - 'jobs (commands) to run simultaneously."', - type=int, default=8, metavar='jobs') - - run_parser.add_argument('--build_dir', - help='As in the make command, it specifies the build ' - 'directory.', - type=str, default='', metavar='build_dir') - - run_parser.add_argument('--defconfig', - help='Uses a default .kunitconfig.', - action='store_true') - - run_parser.add_argument('--alltests', - help='Run all KUnit tests through allyesconfig', - action='store_true') - - run_parser.add_argument('--make_options', - help='X=Y make option, can be repeated.', - action='append') + add_common_opts(run_parser) + add_config_opts(run_parser) + add_build_opts(run_parser) + add_exec_opts(run_parser) + add_parse_opts(run_parser) + + config_parser = subparser.add_parser('config', + help='Ensures that .config contains all of ' + 'the options in .kunitconfig') + add_common_opts(config_parser) + add_config_opts(config_parser) + + build_parser = subparser.add_parser('build', help='Builds a kernel with KUnit tests') + add_common_opts(build_parser) + add_build_opts(build_parser) + + exec_parser = subparser.add_parser('exec', help='Run a kernel with KUnit tests') + add_common_opts(exec_parser) + add_exec_opts(exec_parser) + add_parse_opts(exec_parser) + + # The 'parse' option is special, as it doesn't need the kernel source + # (therefore there is no need for a build_dir, hence no add_common_opts) + # and the '--file' argument is not relevant to 'run', so isn't in + # add_parse_opts() + parse_parser = subparser.add_parser('parse', + help='Parses KUnit results from a file, ' + 'and parses formatted results.') + add_parse_opts(parse_parser) + parse_parser.add_argument('file', + help='Specifies the file to read results from.', + type=str, nargs='?', metavar='input_file') cli_args = parser.parse_args(argv) if cli_args.subcommand == 'run': - if get_kernel_root_path(): - os.chdir(get_kernel_root_path()) - if cli_args.build_dir: if not os.path.exists(cli_args.build_dir): os.mkdir(cli_args.build_dir) @@ -143,9 +252,6 @@ def main(argv, linux=None): cli_args.build_dir, kunit_kernel.kunitconfig_path) - if cli_args.defconfig: - create_default_kunitconfig() - if not linux: linux = kunit_kernel.LinuxSourceTree() @@ -159,6 +265,81 @@ def main(argv, linux=None): result = run_tests(linux, request) if result.status != KunitStatus.SUCCESS: sys.exit(1) + elif cli_args.subcommand == 'config': + if cli_args.build_dir: + if not os.path.exists(cli_args.build_dir): + os.mkdir(cli_args.build_dir) + kunit_kernel.kunitconfig_path = os.path.join( + cli_args.build_dir, + kunit_kernel.kunitconfig_path) + + if not linux: + linux = kunit_kernel.LinuxSourceTree() + + request = KunitConfigRequest(cli_args.build_dir, + cli_args.defconfig, + cli_args.make_options) + result = config_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'build': + if cli_args.build_dir: + if not os.path.exists(cli_args.build_dir): + os.mkdir(cli_args.build_dir) + kunit_kernel.kunitconfig_path = os.path.join( + cli_args.build_dir, + kunit_kernel.kunitconfig_path) + + if not linux: + linux = kunit_kernel.LinuxSourceTree() + + request = KunitBuildRequest(cli_args.jobs, + cli_args.build_dir, + cli_args.alltests, + cli_args.make_options) + result = build_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'exec': + if cli_args.build_dir: + if not os.path.exists(cli_args.build_dir): + os.mkdir(cli_args.build_dir) + kunit_kernel.kunitconfig_path = os.path.join( + cli_args.build_dir, + kunit_kernel.kunitconfig_path) + + if not linux: + linux = kunit_kernel.LinuxSourceTree() + + exec_request = KunitExecRequest(cli_args.timeout, + cli_args.build_dir, + cli_args.alltests) + exec_result = exec_tests(linux, exec_request) + parse_request = KunitParseRequest(cli_args.raw_output, + exec_result.result) + result = parse_tests(parse_request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + exec_result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'parse': + if cli_args.file == None: + kunit_output = sys.stdin + else: + with open(cli_args.file, 'r') as f: + kunit_output = f.read().splitlines() + request = KunitParseRequest(cli_args.raw_output, + kunit_output) + result = parse_tests(request) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) else: parser.print_help() diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 984588d6ba95..5bb7b118ebd9 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -239,6 +239,24 @@ class KUnitMainTest(unittest.TestCase): self.print_patch.stop() pass + def test_config_passes_args_pass(self): + kunit.main(['config'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_build_passes_args_pass(self): + kunit.main(['build'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '', None) + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_exec_passes_args_pass(self): + kunit.main(['exec'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + assert self.linux_source_mock.run_kernel.call_count == 1 + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_run_passes_args_pass(self): kunit.main(['run'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 @@ -247,6 +265,13 @@ class KUnitMainTest(unittest.TestCase): build_dir='', timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_exec_passes_args_fail(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + with self.assertRaises(SystemExit) as e: + kunit.main(['exec'], self.linux_source_mock) + assert type(e.exception) == SystemExit + assert e.exception.code == 1 + def test_run_passes_args_fail(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) with self.assertRaises(SystemExit) as e: @@ -257,14 +282,28 @@ class KUnitMainTest(unittest.TestCase): assert self.linux_source_mock.run_kernel.call_count == 1 self.print_mock.assert_any_call(StrContains(' 0 tests run')) + def test_exec_raw_output(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + kunit.main(['exec', '--raw_output'], self.linux_source_mock) + assert self.linux_source_mock.run_kernel.call_count == 1 + for kall in self.print_mock.call_args_list: + assert kall != mock.call(StrContains('Testing complete.')) + assert kall != mock.call(StrContains(' 0 tests run')) + def test_run_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) - with self.assertRaises(SystemExit) as e: - kunit.main(['run', '--raw_output'], self.linux_source_mock) - assert type(e.exception) == SystemExit - assert e.exception.code == 1 + kunit.main(['run', '--raw_output'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 assert self.linux_source_mock.run_kernel.call_count == 1 + for kall in self.print_mock.call_args_list: + assert kall != mock.call(StrContains('Testing complete.')) + assert kall != mock.call(StrContains(' 0 tests run')) + + def test_exec_timeout(self): + timeout = 3453 + kunit.main(['exec', '--timeout', str(timeout)], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=timeout) + self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_run_timeout(self): timeout = 3453 @@ -282,5 +321,21 @@ class KUnitMainTest(unittest.TestCase): build_dir=build_dir, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_config_builddir(self): + build_dir = '.kunit' + kunit.main(['config', '--build_dir', build_dir], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + + def test_build_builddir(self): + build_dir = '.kunit' + kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None) + + def test_exec_builddir(self): + build_dir = '.kunit' + kunit.main(['exec', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + if __name__ == '__main__': unittest.main() -- 2.26.2.526.g744177e7f7-goog

5 years, 1 month

1
0
0 0

Re: kprobe/kprobe_syntax_errors.tc faild on dash due to sh I/O error

by Masami Hiramatsu

Hi Liu, On Wed, 29 Apr 2020 15:13:20 +0800 Liu Yiding <yidingx.liu(a)intel.com> wrote: > Hi, all. > > > I met a funny error when i run kprobe_syntax_errors > > ``` > > root@vm-snb-35 > /usr/src/linux-selftests-x86_64-rhel-7.6-kselftests-ae83d0b416db002fe95601e7f97f64b59514d936/tools/testing/selftests/ftrace# > ./ftracetest -vvv test.d/kprobe/kprobe_syntax_errors.tc [snip] + expr 13 > + 0 + test 13 -eq 13 + echo p:kprobes/testevent _do_fork abcd=\1 sh: > echo: I/O error > > ``` > > This error only happend on dash, use bash run this test is ok. > > backslash ('\1') will be transferred into empty in dash. Oops, good catch! I found that came from the built-in echo command behavior. Bash's echo doesn't interpret the backslash escape, but dash's POSIX-compliant echo does. ----<dash>---- $ a='\1' $ b=`echo $a` $ test "$a" = "$b" && echo "same!" || echo "different" different ----<bash>---- $ a='\1' $ b=`echo $a` $ test "$a" = "$b" && echo "same!" || echo "different" same! OK, I'll fix it. Thank you! -- Masami Hiramatsu <mhiramat(a)kernel.org>

5 years, 1 month

1
0
0 0

[GIT PULL] Kselftest update for Linux 5.7-rc4

by Shuah Khan

Hi Linus, Please pull the following Kselftest update for Linux 5.7-rc4. This kselftest update for Linux 5.7-rc4 consists of: - ftrace test fixes to check for required filter files and kprobe args. - Kselftest build/cross-build dependency check script to make it easier for test ring admins/users to configure build systems correctly for build/cross-build kselftests. Currently checks library dependencies. - Checks if Kselftests can be built/cross-built on a system running compile test on a trivial C file with LDLIBS specified for each individual test in their Makefiles. - Prints suggested target list for a system filtering out tests failed the build dependency check from the TARGETS in Selftests the main Makefile when optional -p is specified. - Prints pass/fail dependency check for each tests/sub-test. - Prints pass/fail targets and libraries. - Default: runs dependency checks on all tests. - Optional test name can be specified to check dependencies for it. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit b87080eab4c1377706c113fc9c0157f19ea8fed1: selftests/ipc: Fix test failure seen after initial test run (2020-04-14 10:24:28 -0600) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-5.7-rc4 for you to fetch changes up to f0c0d0cf590f71b2213b29a7ded2cde3d0a1a0ba: selftests/ftrace: Check the first record for kprobe_args_type.tc (2020-04-24 09:39:26 -0600) ---------------------------------------------------------------- linux-kselftest-5.7-rc4 This kselftest update for Linux 5.7-rc4 consists of: - ftrace test fixes to check for required filter files and kprobe args. - Kselftest build/cross-build dependency check script to make it easier for test ring admins/users to configure build systems correctly for build/cross-build kselftests. Currently checks library dependencies. - Checks if Kselftests can be built/cross-built on a system running compile test on a trivial C file with LDLIBS specified for each individual test in their Makefiles. - Prints suggested target list for a system filtering out tests failed the build dependency check from the TARGETS in Selftests the main Makefile when optional -p is specified. - Prints pass/fail dependency check for each tests/sub-test. - Prints pass/fail targets and libraries. - Default: runs dependency checks on all tests. - Optional test name can be specified to check dependencies for it. ---------------------------------------------------------------- Shuah Khan (1): selftests: add build/cross-build dependency check script Xiao Yang (2): selftests/ftrace: Check required filter files before running test selftests/ftrace: Check the first record for kprobe_args_type.tc .../ftrace/test.d/ftrace/fgraph-filter-stack.tc | 5 +- .../ftrace/test.d/ftrace/fgraph-filter.tc | 2 + .../ftrace/test.d/ftrace/func-filter-glob.tc | 2 + .../test.d/ftrace/func-filter-notrace-pid.tc | 5 +- .../ftrace/test.d/ftrace/func-filter-pid.tc | 5 +- .../ftrace/test.d/ftrace/func-filter-stacktrace.tc | 2 +- .../ftrace/test.d/ftrace/func_event_triggers.tc | 5 +- .../ftrace/test.d/ftrace/func_mod_trace.tc | 2 +- .../ftrace/test.d/ftrace/func_profiler.tc | 5 +- .../ftrace/test.d/ftrace/func_set_ftrace_file.tc | 5 +- .../ftrace/test.d/ftrace/func_stack_tracer.tc | 2 + .../test.d/ftrace/func_traceonoff_triggers.tc | 5 +- tools/testing/selftests/ftrace/test.d/functions | 6 + .../ftrace/test.d/kprobe/kprobe_args_type.tc | 2 +- .../ftrace/test.d/kprobe/kprobe_ftrace.tc | 2 + tools/testing/selftests/gpio/Makefile | 12 +- tools/testing/selftests/intel_pstate/Makefile | 2 +- tools/testing/selftests/kselftest_deps.sh | 272 +++++++++++++++++++++ tools/testing/selftests/memfd/Makefile | 14 +- 19 files changed, 315 insertions(+), 40 deletions(-) create mode 100755 tools/testing/selftests/kselftest_deps.sh ----------------------------------------------------------------

5 years, 1 month

2
1
0 0

[GIT PULL] Kunit update for Linux 5.7-rc4

by Shuah Khan

Hi Linus, Please pull the following Kunit update for Linux 5.7-rc4. This Kunit update for Linux 5.7-rc4 consists of a single fix to flush the test summary to the console log without delay. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit 8f3d9f354286745c751374f5f1fcafee6b3f3136: Linux 5.7-rc1 (2020-04-12 12:35:55 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-kunit-5.7-rc4 for you to fetch changes up to 6cb1818798812fe7e2c8fe3f489ad1c86adfd6c4: kunit: Add missing newline in summary message (2020-04-23 15:42:00 -0600) ---------------------------------------------------------------- linux-kselftest-kunit-5.7-rc4 This Kunit update for Linux 5.7-rc4 consists of a single fix to flush the test summary to the console log without delay. ---------------------------------------------------------------- Marco Elver (1): kunit: Add missing newline in summary message lib/kunit/test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ----------------------------------------------------------------

5 years, 1 month

2
1
0 0

[PATCH] selftests/ftrace: Add CONFIG_SAMPLE_FTRACE_DIRECT=m kconfig

by Xiao Yang

ftrace-direct.tc and kprobe-direct.tc require CONFIG_SAMPLE_FTRACE_DIRECT=m so add it to config file which is used by merge_config.sh. Signed-off-by: Xiao Yang <yangx.jy(a)cn.fujitsu.com> --- tools/testing/selftests/ftrace/config | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config index c2c8de4fafff..e59d985eeff0 100644 --- a/tools/testing/selftests/ftrace/config +++ b/tools/testing/selftests/ftrace/config @@ -11,5 +11,6 @@ CONFIG_PREEMPTIRQ_DELAY_TEST=m CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_SAMPLES=y +CONFIG_SAMPLE_FTRACE_DIRECT=m CONFIG_SAMPLE_TRACE_PRINTK=m CONFIG_KALLSYMS_ALL=y -- 2.25.1

5 years, 1 month

3
3
0 0

[PATCH v6 00/12] mm/gup: track FOLL_PIN pages

by John Hubbard

Hi, Jan and Kirill: I've tentatively removed your review and ACK, respectively, for patch 12 (the last dump_page patch), because even though they are logically the same as what you reviewed in v5, the base is Matthew's new patch instead of my earlier patch. (Trying to err on the side of caution with these tags.) There is a git repo and branch, for convenience in reviewing: git@github.com:johnhubbard/linux.git track_user_pages_v6 ============================================================ Changes since v5: * Rebased onto Linux 5.6.0-rc1. * Swapped in Matthew Wilcox's more comprehensive dump_page() patch, and moved it later in this series so that it immediately precedes my subsequent dump_page() patch, for slightly easier reviews and commit log history. * Fixed "the last bug!" in the /proc/vmstat patch, by moving the mod_node_page_state() call in put_compound_page() so that it only happens in the FOLL_PIN case. * Added a couple more ACKs from Kirill. * Tweaked the "Future steps" in this cover letter to add a little detail about what comes next. ============================================================ Changes since v4: * Added documentation about the huge page behavior of the new /proc/vmstat items. * Added a missing mode_node_page_state() call to put_compound_head(). * Fixed a tracepoint call in page_ref_sub_return(). * Added a trailing underscore to a URL in pin_user_pages.rst, to fix a broken generated link. * Added ACKs and reviewed-by's from Jan Kara and Kirill Shutemov. * Rebased onto today's linux.git, and * I am experimenting here with "git format-patch --base=<commit>". This generated the "base-commit:" tag you'll see at the end of this cover letter. I was inspired to do so after trying out a new get-lore-mbox.py tool (it's very nice), mentioned in a recent LWN article (https://lwn.net/Articles/811528/ ). That tool relies on the base-commit tag for some things. ============================================================ Changes since v3: * Rebased onto latest linux.git * Added ACKs and reviewed-by's from Kirill Shutemov and Jan Kara. * /proc/vmstat: * Renamed items, after realizing that I hate the previous names: nr_foll_pin_requested --> nr_foll_pin_acquired nr_foll_pin_returned --> nr_foll_pin_released * Removed the CONFIG_DEBUG_VM guard, and collapsed away a wrapper routine: now just calls mod_node_page_state() directly. * Tweaked the WARN_ON_ONCE() statements in mm/hugetlb.c to be more informative, and added comments above them as well. * Fixed gup_benchmark: signed int --> unsigned long. * One or two minor formatting changes. ============================================================ Changes since v2: * Rebased onto linux.git, because the akpm tree for 5.6 has been merged. * Split the tracking patch into even more patches, as requested. * Merged Matthew Wilcox's dump_page() changes into mine, as part of the first patch. * Renamed: page_dma_pinned() --> page_maybe_dma_pinned(), in response to Kirill Shutemov's review. * Moved a WARN to the top of a routine, and fixed a typo in the commit description of patch #7, also as suggested by Kirill. ============================================================ Changes since v1: * Split the tracking patch into 6 smaller patches * Rebased onto today's linux-next/akpm (there weren't any conflicts). * Fixed an "unsigned int" vs. "int" problem in gup_benchmark, reported by Nathan Chancellor. (I don't see it in my local builds, probably because they use gcc, but an LLVM test found the mismatch.) * Fixed a huge page pincount problem (add/subtract vs. increment/decrement), spotted by Jan Kara. ============================================================ There is a reasonable case to be made for merging two of the patches (patches 7 and 8), given that patch 7 provides tracking that has upper limits on the number of pins that can be done with huge pages. Let me know if anyone wants those merged, but unless there is some weird chance of someone grabbing patch 7 and not patch 8, I don't really see the need. Meanwhile, it's easier to review in this form. Also, patch 3 has been revived. Earlier reviewers asked for it to be merged into the tracking patch (one cannot please everyone, heh), but now it's back out on it's own. This activates tracking of FOLL_PIN pages. This is in support of fixing the get_user_pages()+DMA problem described in [1]-[4]. FOLL_PIN support is now in the main linux tree. However, the patch to use FOLL_PIN to track pages was *not* submitted, because Leon saw an RDMA test suite failure that involved (I think) page refcount overflows when huge pages were used. This patch definitively solves that kind of overflow problem, by adding an exact pincount, for compound pages (of order > 1), in the 3rd struct page of a compound page. If available, that form of pincounting is used, instead of the GUP_PIN_COUNTING_BIAS approach. Thanks again to Jan Kara for that idea. Other interesting changes: * dump_page(): added one, or two new things to report for compound pages: head refcount (for all compound pages), and map_pincount (for compound pages of order > 1). * Documentation/core-api/pin_user_pages.rst: removed the "TODO" for the huge page refcount upper limit problems, and added notes about how it works now. Also added a note about the dump_page() enhancements. * Added some comments in gup.c and mm.h, to explain that there are two ways to count pinned pages: exact (for compound pages of order > 1) and fuzzy (GUP_PIN_COUNTING_BIAS: for all other pages). ============================================================ General notes about the tracking patch: This is a prerequisite to solving the problem of proper interactions between file-backed pages, and [R]DMA activities, as discussed in [1], [2], [3], [4] and in a remarkable number of email threads since about 2017. :) In contrast to earlier approaches, the page tracking can be incrementally applied to the kernel call sites that, until now, have been simply calling get_user_pages() ("gup"). In other words, opt-in by changing from this: get_user_pages() (sets FOLL_GET) put_page() to this: pin_user_pages() (sets FOLL_PIN) unpin_user_page() ============================================================ Future steps: * Convert more subsystems from get_user_pages() to pin_user_pages(). The first probably needs to be bio/biovecs, because any filesystem testing is too difficult without those in place. * Change VFS and filesystems to respond appropriately when encountering dma-pinned pages. * Work with Ira and others to connect this all up with file system leases. [1] Some slow progress on get_user_pages() (Apr 2, 2019): https://lwn.net/Articles/784574/ [2] DMA and get_user_pages() (LPC: Dec 12, 2018): https://lwn.net/Articles/774411/ [3] The trouble with get_user_pages() (Apr 30, 2018): https://lwn.net/Articles/753027/ [4] LWN kernel index: get_user_pages() https://lwn.net/Kernel/Index/#Memory_management-get_user_pages John Hubbard (11): mm/gup: split get_user_pages_remote() into two routines mm/gup: pass a flags arg to __gup_device_* functions mm: introduce page_ref_sub_return() mm/gup: pass gup flags to two more routines mm/gup: require FOLL_GET for get_user_pages_fast() mm/gup: track FOLL_PIN pages mm/gup: page->hpage_pinned_refcount: exact pin counts for huge pages mm/gup: /proc/vmstat: pin_user_pages (FOLL_PIN) reporting mm/gup_benchmark: support pin_user_pages() and related calls selftests/vm: run_vmtests: invoke gup_benchmark with basic FOLL_PIN coverage mm: dump_page(): additional diagnostics for huge pinned pages Matthew Wilcox (Oracle) (1): mm: Improve dump_page() for compound pages Documentation/core-api/pin_user_pages.rst | 86 ++-- include/linux/mm.h | 108 ++++- include/linux/mm_types.h | 7 +- include/linux/mmzone.h | 2 + include/linux/page_ref.h | 9 + mm/debug.c | 44 +- mm/gup.c | 451 ++++++++++++++++----- mm/gup_benchmark.c | 71 +++- mm/huge_memory.c | 29 +- mm/hugetlb.c | 60 ++- mm/page_alloc.c | 2 + mm/rmap.c | 6 + mm/vmstat.c | 2 + tools/testing/selftests/vm/gup_benchmark.c | 15 +- tools/testing/selftests/vm/run_vmtests | 22 + 15 files changed, 734 insertions(+), 180 deletions(-) base-commit: bb6d3fb354c5ee8d6bde2d576eb7220ea09862b9 -- 2.25.0

5 years, 1 month

5
27
0 0

[PATCH v4 0/7] firmware: add partial read support in request_firmware_into_buf

by Scott Branden

This patch series adds partial read support in request_firmware_into_buf. In order to accept the enhanced API it has been requested that kernel selftests and upstreamed driver utilize the API enhancement and so are included in this patch series. Also in this patch series is the addition of a new Broadcom VK driver utilizing the new request_firmware_into_buf enhanced API. Changes from v3: - fix sparse warnings - fix printf format specifiers for size_t - fix 32-bit cross-compiling reports 32-bit shifts - use readl/writel,_relaxed to access pci ioremap memory, removed memory barriers and volatile keyword with such change - driver optimizations for interrupt/poll functionalities Changes from v2: - remove unnecessary code and mutex locks in lib/test_firmware.c - remove VK_IOCTL_ACCESS_BAR support from driver and use pci sysfs instead - remove bitfields - remove Kconfig default m - adjust formatting and some naming based on feedback - fix error handling conditions - use appropriate return codes - use memcpy_toio instead of direct access to PCIE bar Scott Branden (7): fs: introduce kernel_pread_file* support firmware: add offset to request_firmware_into_buf test_firmware: add partial read support for request_firmware_into_buf firmware: test partial file reads of request_firmware_into_buf bcm-vk: add bcm_vk UAPI misc: bcm-vk: add Broadcom VK driver MAINTAINERS: bcm-vk: add maintainer for Broadcom VK Driver MAINTAINERS | 7 + drivers/base/firmware_loader/firmware.h | 5 + drivers/base/firmware_loader/main.c | 52 +- drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/bcm-vk/Kconfig | 29 + drivers/misc/bcm-vk/Makefile | 12 + drivers/misc/bcm-vk/bcm_vk.h | 412 +++++ drivers/misc/bcm-vk/bcm_vk_dev.c | 1211 ++++++++++++++ drivers/misc/bcm-vk/bcm_vk_legacy.c | 89 ++ drivers/misc/bcm-vk/bcm_vk_msg.c | 1414 +++++++++++++++++ drivers/misc/bcm-vk/bcm_vk_msg.h | 195 +++ drivers/misc/bcm-vk/bcm_vk_sg.c | 271 ++++ drivers/misc/bcm-vk/bcm_vk_sg.h | 60 + drivers/misc/bcm-vk/bcm_vk_tty.c | 352 ++++ drivers/soc/qcom/mdt_loader.c | 7 +- fs/exec.c | 96 +- include/linux/firmware.h | 8 +- include/linux/fs.h | 20 + include/uapi/linux/misc/bcm_vk.h | 99 ++ lib/test_firmware.c | 144 +- .../selftests/firmware/fw_filesystem.sh | 80 + 22 files changed, 4516 insertions(+), 49 deletions(-) create mode 100644 drivers/misc/bcm-vk/Kconfig create mode 100644 drivers/misc/bcm-vk/Makefile create mode 100644 drivers/misc/bcm-vk/bcm_vk.h create mode 100644 drivers/misc/bcm-vk/bcm_vk_dev.c create mode 100644 drivers/misc/bcm-vk/bcm_vk_legacy.c create mode 100644 drivers/misc/bcm-vk/bcm_vk_msg.c create mode 100644 drivers/misc/bcm-vk/bcm_vk_msg.h create mode 100644 drivers/misc/bcm-vk/bcm_vk_sg.c create mode 100644 drivers/misc/bcm-vk/bcm_vk_sg.h create mode 100644 drivers/misc/bcm-vk/bcm_vk_tty.c create mode 100644 include/uapi/linux/misc/bcm_vk.h -- 2.17.1

5 years, 1 month

1
6
0 0

Canada Greetings

by Fel

Greetings, My name is Felix,I am contacting you in respect of an urgent matter (Deal) regarding funds in excess of Nine Million US Dollars which resulted from a liquidated BTC account belonging to a deceased account holder. I will let you in on my plan and why I chose to contact you in the first place after I have received your reply and gaining your trust. Many thanks and looking forward to your reply. Felix.

5 years, 1 month

1
0
0 0

Canada Greetings

by Fel

Greetings, My name is Felix,I am contacting you in respect of an urgent matter (Deal) regarding funds in excess of Nine Million US Dollars which resulted from a liquidated BTC account belonging to a deceased account holder. I will let you in on my plan and why I chose to contact you in the first place after I have received your reply and gaining your trust. Many thanks and looking forward to your reply. Felix.

5 years, 1 month

1
0
0 0

[PATCH net-next v6 0/5] kselftest: add fixture parameters

by Jakub Kicinski

Hi! This set is an attempt to make running tests for different sets of data easier. The direct motivation is the tls test which we'd like to run for TLS 1.2 and TLS 1.3, but currently there is no easy way to invoke the same tests with different parameters. Tested all users of kselftest_harness.h. Dave, would it be possible to take these via net-next? It seems we're failing to get Shuah's attention. v2: - don't run tests by fixture - don't pass params as an explicit argument v3: - go back to the orginal implementation with an extra parameter, and running by fixture (Kees); - add LIST_APPEND helper (Kees); - add a dot between fixture and param name (Kees); - rename the params to variants (Tim); v4: - whitespace fixes. v5 (Kees): - move a comment; - remove a temporary variable; - reword the commit message on patch 4. v6: - resend for net-next. v1: https://lore.kernel.org/netdev/20200313031752.2332565-1-kuba@kernel.org/ v2: https://lore.kernel.org/netdev/20200314005501.2446494-1-kuba@kernel.org/ v3: https://lore.kernel.org/netdev/20200316225647.3129354-1-kuba@kernel.org/ v4: https://lore.kernel.org/netdev/20200317010419.3268916-1-kuba@kernel.org/ v5: https://lore.kernel.org/netdev/20200318010153.40797-1-kuba@kernel.org/ Jakub Kicinski (5): kselftest: factor out list manipulation to a helper kselftest: create fixture objects kselftest: run tests by fixture kselftest: add fixture variants selftests: tls: run all tests for TLS 1.2 and TLS 1.3 Documentation/dev-tools/kselftest.rst | 3 +- tools/testing/selftests/kselftest_harness.h | 234 +++++++++++++++----- tools/testing/selftests/net/tls.c | 93 ++------ 3 files changed, 202 insertions(+), 128 deletions(-) -- 2.25.4

5 years, 1 month

2
6
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror