selftests: net: udpgso_bench.sh failed on 4.19, 4.14, 4.9 and 4.4 branches. PASS on stable branch 5.1, mainline and next. This failure is started happening on 4.19 and older kernel branches after kselftest upgrade to version 5.1
Is there any possibilities to backport ?
Error: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524
Test output: ----------------- selftests: net: udpgso_bench.sh ipv4 tcp tcp rx: 469 MB/s 7930 calls/s tcp tx: 469 MB/s 7961 calls/s 7961 msg/s tcp rx: 470 MB/s 7941 calls/s tcp tx: 470 MB/s 7977 calls/s 7977 msg/s tcp rx: 470 MB/s 7933 calls/s tcp tx: 470 MB/s 7975 calls/s 7975 msg/s tcp zerocopy tcp tx: 357 MB/s 6064 calls/s 6064 msg/s tcp rx: 357 MB/s 6052 calls/s tcp tx: 352 MB/s 5981 calls/s 5981 msg/s tcp rx: 352 MB/s 5979 calls/s tcp tx: 350 MB/s 5937 calls/s 5937 msg/s tcp rx: 350 MB/s 5938 calls/s udp udp rx: 23 MB/s 16505 calls/s udp tx: 23 MB/s 16464 calls/s 392 msg/s udp rx: 23 MB/s 16500 calls/s udp tx: 23 MB/s 16506 calls/s 393 msg/s udp rx: 23 MB/s 16396 calls/s udp gso udp rx: 536 MB/s 9097 calls/s udp tx: 545 MB/s 9246 calls/s 9246 msg/s udp rx: 545 MB/s 9256 calls/s udp tx: 545 MB/s 9256 calls/s 9256 msg/s udp rx: 545 MB/s 9259 calls/s udp tx: 545 MB/s 9258 calls/s 9258 msg/s udp rx: 545 MB/s 9252 calls/s udp gso zerocopy ./udpgso_bench_tx: setsockopt zerocopy: Unknown error 524 ipv6 tcp tcp tx: 470 MB/s 7979 calls/s 7979 msg/s tcp rx: 470 MB/s 7947 calls/s tcp rx: 471 MB/s 7979 calls/s tcp tx: 514 MB/s 8721 calls/s 8721 msg/s tcp zerocopy tcp tx: 392 MB/s 6658 calls/s 6658 msg/s tcp rx: 392 MB/s 6399 calls/s tcp rx: 350 MB/s 5936 calls/s tcp tx: 350 MB/s 5945 calls/s 5945 msg/s tcp rx: 350 MB/s 5937 calls/s tcp tx: 350 MB/s 5940 calls/s 5940 msg/s udp udp rx: 20 MB/s 14802 calls/s udp tx: 20 MB/s 14921 calls/s 347 msg/s udp rx: 24 MB/s 17797 calls/s udp tx: 24 MB/s 17802 calls/s 414 msg/s udp rx: 17 MB/s 12453 calls/s udp tx: 17 MB/s 12470 calls/s 290 msg/s udp rx: 17 MB/s 12409 calls/s udp tx: 545 MB/s 9257 calls/s 9257 msg/s udp rx: 545 MB/s 9249 calls/s udp tx: 545 MB/s 9248 calls/s 9248 msg/s udp rx: 545 MB/s 9254 calls/s udp tx: 545 MB/s 9254 calls/s 9254 msg/s udp rx: 545 MB/s 9260 calls/s udp gso zerocopy ./udpgso_bench_tx: setsockopt zerocopy: Unknown error 524 not ok 1.. selftests: net: udpgso_bench.sh [FAIL] selftests: net_udpgso_bench.sh [FAIL]
Best regards Naresh Kamboju
On Tue, Jun 18, 2019 at 7:27 AM Naresh Kamboju naresh.kamboju@linaro.org wrote:
selftests: net: udpgso_bench.sh failed on 4.19, 4.14, 4.9 and 4.4 branches. PASS on stable branch 5.1, mainline and next. This failure is started happening on 4.19 and older kernel branches after kselftest upgrade to version 5.1
Does version 5.1 here mean running tests from Linux 5.1, against older kernels?
Is there any possibilities to backport ?
Error: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524
MSG_ZEROCOPY for UDP was added in commit b5947e5d1e71 ("udp: msg_zerocopy") in Linux 5.0.
The selftest was expanded with this feature in commit db63e489c7aa ("selftests: extend zerocopy tests to udp"), also in Linux 5.0.
Those tests are not expected to pass on older kernels.
On Tue, Jun 18, 2019 at 08:31:16AM -0400, Willem de Bruijn wrote:
On Tue, Jun 18, 2019 at 7:27 AM Naresh Kamboju naresh.kamboju@linaro.org wrote:
selftests: net: udpgso_bench.sh failed on 4.19, 4.14, 4.9 and 4.4 branches. PASS on stable branch 5.1, mainline and next. This failure is started happening on 4.19 and older kernel branches after kselftest upgrade to version 5.1
Does version 5.1 here mean running tests from Linux 5.1, against older kernels?
Is there any possibilities to backport ?
Error: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524
MSG_ZEROCOPY for UDP was added in commit b5947e5d1e71 ("udp: msg_zerocopy") in Linux 5.0.
The selftest was expanded with this feature in commit db63e489c7aa ("selftests: extend zerocopy tests to udp"), also in Linux 5.0.
Those tests are not expected to pass on older kernels.
Any way to degrade gracefully if the feature is not present at all in the kernel under test? People run the latest version of kselftests on older kernels all the time.
thanks,
greg k-h
On Tue, Jun 18, 2019 at 12:10 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 08:31:16AM -0400, Willem de Bruijn wrote:
On Tue, Jun 18, 2019 at 7:27 AM Naresh Kamboju naresh.kamboju@linaro.org wrote:
selftests: net: udpgso_bench.sh failed on 4.19, 4.14, 4.9 and 4.4 branches. PASS on stable branch 5.1, mainline and next. This failure is started happening on 4.19 and older kernel branches after kselftest upgrade to version 5.1
Does version 5.1 here mean running tests from Linux 5.1, against older kernels?
Is there any possibilities to backport ?
Error: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524
MSG_ZEROCOPY for UDP was added in commit b5947e5d1e71 ("udp: msg_zerocopy") in Linux 5.0.
The selftest was expanded with this feature in commit db63e489c7aa ("selftests: extend zerocopy tests to udp"), also in Linux 5.0.
Those tests are not expected to pass on older kernels.
Any way to degrade gracefully if the feature is not present at all in the kernel under test? People run the latest version of kselftests on older kernels all the time.
We add new tests along with new features and bug fixes all the time. All of those will fail on older kernels, as expected.
I'm honestly surprised to hear that we run newer tests against older kernels. Is the idea to validate fixes in stable branches? If so, should we instead backport the relevant tests to those stable branches? Only the tests that verify fixes, leaving out those for new features, of course.
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
thanks,
greg k-h
On Tue, Jun 18, 2019 at 1:15 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
I was not aware that kselftest had this feature.
But it appears that exit code KSFT_SKIP (4) will achieve this. Okay, I'll send a patch and will keep that in mind for future tests.
On Tue, Jun 18, 2019 at 01:27:14PM -0400, Willem de Bruijn wrote:
On Tue, Jun 18, 2019 at 1:15 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
I was not aware that kselftest had this feature.
But it appears that exit code KSFT_SKIP (4) will achieve this. Okay, I'll send a patch and will keep that in mind for future tests.
Wonderful, thanks for doing that!
greg k-h
On Tue, Jun 18, 2019 at 1:39 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 01:27:14PM -0400, Willem de Bruijn wrote:
On Tue, Jun 18, 2019 at 1:15 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
I was not aware that kselftest had this feature.
But it appears that exit code KSFT_SKIP (4) will achieve this. Okay, I'll send a patch and will keep that in mind for future tests.
Wonderful, thanks for doing that!
One complication: an exit code works for a single test, but here multiple test variants are run from a single shell script.
I see that in similar such cases that use the test harness (ksft_test_result_skip) the overall test returns success as long as all individual cases return either success or skip.
I think it's preferable to return KSFT_SKIP if any of the cases did so (and none returned an error). I'll do that unless anyone objects.
On Tue, Jun 18, 2019 at 2:59 PM Willem de Bruijn willemdebruijn.kernel@gmail.com wrote:
On Tue, Jun 18, 2019 at 1:39 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 01:27:14PM -0400, Willem de Bruijn wrote:
On Tue, Jun 18, 2019 at 1:15 PM Greg KH gregkh@linuxfoundation.org wrote:
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
I was not aware that kselftest had this feature.
But it appears that exit code KSFT_SKIP (4) will achieve this. Okay, I'll send a patch and will keep that in mind for future tests.
Wonderful, thanks for doing that!
One complication: an exit code works for a single test, but here multiple test variants are run from a single shell script.
I see that in similar such cases that use the test harness (ksft_test_result_skip) the overall test returns success as long as all individual cases return either success or skip.
I think it's preferable to return KSFT_SKIP if any of the cases did so (and none returned an error). I'll do that unless anyone objects.
http://patchwork.ozlabs.org/patch/1118309/
The shell script scaffolding can perhaps be reused for other similar tests.
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 14:58:26 -0400
I see that in similar such cases that use the test harness (ksft_test_result_skip) the overall test returns success as long as all individual cases return either success or skip.
I think it's preferable to return KSFT_SKIP if any of the cases did so (and none returned an error). I'll do that unless anyone objects.
I guess this is a question of semantics.
I mean, if you report skip at the top level does that mean that all sub tests were skipped? People may think so... :)
On Tue, Jun 18, 2019 at 6:44 PM David Miller davem@davemloft.net wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 14:58:26 -0400
I see that in similar such cases that use the test harness (ksft_test_result_skip) the overall test returns success as long as all individual cases return either success or skip.
I think it's preferable to return KSFT_SKIP if any of the cases did so (and none returned an error). I'll do that unless anyone objects.
I guess this is a question of semantics.
I mean, if you report skip at the top level does that mean that all sub tests were skipped? People may think so... :)
Yes, it's not ideal. Erring on the side of caution? Unlike pass, it is a signal that an admin may or may not choose to act on. I run a selected subset of tests from tools/testing that are all expected to pass, so if one returns skip, I would want to take a closer look.
From: Greg KH gregkh@linuxfoundation.org Date: Tue, 18 Jun 2019 19:15:16 +0200
On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
From: Willem de Bruijn willemdebruijn.kernel@gmail.com Date: Tue, 18 Jun 2019 12:37:33 -0400
Specific to the above test, I can add a check command testing setsockopt SO_ZEROCOPY return value. AFAIK kselftest has no explicit way to denote "skipped", so this would just return "pass". Sounds a bit fragile, passing success when a feature is absent.
Especially since the feature might be absent because the 'config' template forgot to include a necessary Kconfig option.
That is what the "skip" response is for, don't return "pass" if the feature just isn't present. That lets people run tests on systems without the config option enabled as you say, or on systems without the needed userspace tools present.
Ok I see how skip works, thanks for explaining.
It would just be nice if it could work in a way such that we could distinguish "too old kernel for feature" from "missing Kconfig symbol in selftest config template". :-)
linux-kselftest-mirror@lists.linaro.org
-
David Miller -
Greg KH -
Naresh Kamboju -
Willem de Bruijn