Fix userfaultfd_hugetlb on hosts which have more than 64 cpus.
---------------------------
running userfaultfd_hugetlb
---------------------------
invalid MiB
Usage: <MiB> <bounces>
[FAIL]
Via userfaultfd.c we can know, hugetlb_size needs to meet hugetlb_size >=
nr_cpus * hugepage_size. hugepage_size is often 2M, so when host cpus >
64, it requires more than 128M.
Changes since v1:
- update changelog/comments and variable name to make code more easier to
read/understand(stolen from Mike Kravetz)
Link: http://lkml.kernel.org/r/20180302024356.83359-1-zhijianx.li@intel.com
Signed-off-by: Li Zhijian <zhijianx.li(a)intel.com>
Cc: Shuah Khan <shuah(a)kernel.org>
Cc: SeongJae Park <sj38.park(a)gmail.com>
Cc: Philippe Ombredanne <pombredanne(a)nexb.com>
Cc: Aneesh Kumar K.V <aneesh.kumar(a)linux.vnet.ibm.com>
Cc: Mike Kravetz <mike.kravetz(a)oracle.com>
Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org>
---
tools/testing/selftests/vm/run_vmtests | 25 +++++++++++++++++--------
1 file changed, 17 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests
index d2561895a021a..22d5646738302 100755
--- a/tools/testing/selftests/vm/run_vmtests
+++ b/tools/testing/selftests/vm/run_vmtests
@@ -2,25 +2,33 @@
# SPDX-License-Identifier: GPL-2.0
#please run as root
-#we need 256M, below is the size in kB
-needmem=262144
mnt=./huge
exitcode=0
-#get pagesize and freepages from /proc/meminfo
+#get huge pagesize and freepages from /proc/meminfo
while read name size unit; do
if [ "$name" = "HugePages_Free:" ]; then
freepgs=$size
fi
if [ "$name" = "Hugepagesize:" ]; then
- pgsize=$size
+ hpgsize_KB=$size
fi
done < /proc/meminfo
+# Simple hugetlbfs tests have a hardcoded minimum requirement of
+# huge pages totaling 256MB (262144KB) in size. The userfaultfd
+# hugetlb test requires a minimum of 2 * nr_cpus huge pages. Take
+# both of these requirements into account and attempt to increase
+# number of huge pages available.
+nr_cpus=$(nproc)
+hpgsize_MB=$((hpgsize_KB / 1024))
+half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
+needmem_KB=$((half_ufd_size_MB * 2 * 1024))
+
#set proper nr_hugepages
-if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then
+if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
nr_hugepgs=`cat /proc/sys/vm/nr_hugepages`
- needpgs=`expr $needmem / $pgsize`
+ needpgs=$((needmem_KB / hpgsize_KB))
tries=2
while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do
lackpgs=$(( $needpgs - $freepgs ))
@@ -107,8 +115,9 @@ fi
echo "---------------------------"
echo "running userfaultfd_hugetlb"
echo "---------------------------"
-# 256MB total huge pages == 128MB src and 128MB dst
-./userfaultfd hugetlb 128 32 $mnt/ufd_test_file
+# Test requires source and destination huge pages. Size of source
+# (half_ufd_size_MB) is passed as argument to test.
+./userfaultfd hugetlb $half_ufd_size_MB 32 $mnt/ufd_test_file
if [ $? -ne 0 ]; then
echo "[FAIL]"
exitcode=1
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
this patch is to fix running userfaultfd_hugetlb failed on the host which has
more than 64 cpus
---------------------------
running userfaultfd_hugetlb
---------------------------
invalid MiB
Usage: <MiB> <bounces>
[FAIL]
>From userfaultfd.c we can know, hugetlb_size need to meet hugetlb_size >= nr_cpus * hugepage_size
hugepage_size is often 2M, so when host cpus > 64, it requires more than 128M.
Signed-off-by: Li Zhijian <zhijianx.li(a)intel.com>
---
tools/testing/selftests/vm/run_vmtests | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests
index d2561895a021a..c440fb972afe9 100755
--- a/tools/testing/selftests/vm/run_vmtests
+++ b/tools/testing/selftests/vm/run_vmtests
@@ -2,8 +2,6 @@
# SPDX-License-Identifier: GPL-2.0
#please run as root
-#we need 256M, below is the size in kB
-needmem=262144
mnt=./huge
exitcode=0
@@ -17,6 +15,13 @@ while read name size unit; do
fi
done < /proc/meminfo
+nr_cpus=$(nproc)
+pgsize_MB=$((pgsize/1024))
+# rule: nr_cpus * pgsize_MB <= hugetlb_size(round to 128M for testing)
+hugetlb_size=$((((nr_cpus*pgsize_MB+127)/128)*128))
+# needmem depends on the nr_cpus, below is the size in kB
+needmem=$((hugetlb_size*2*1024))
+
#set proper nr_hugepages
if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then
nr_hugepgs=`cat /proc/sys/vm/nr_hugepages`
@@ -107,8 +112,8 @@ fi
echo "---------------------------"
echo "running userfaultfd_hugetlb"
echo "---------------------------"
-# 256MB total huge pages == 128MB src and 128MB dst
-./userfaultfd hugetlb 128 32 $mnt/ufd_test_file
+# 256MB total huge pages == 128MB src and 128MB dst when nr_cpus <= 64
+./userfaultfd hugetlb $hugetlb_size 32 $mnt/ufd_test_file
if [ $? -ne 0 ]; then
echo "[FAIL]"
exitcode=1
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Commit 16c513b13477
("selftests: memory-hotplug: silence test command echo")
introduced regression in emit_tests and results in the following
failure when selftests are installed and run. Fix it.
Running tests in memory-hotplug
========================================
./run_kselftest.sh: line 121: @./mem-on-off-test.sh: No such file or
directory
selftests: memory-hotplug [FAIL]
Fixes: 16c513b13477 (selftests: memory-hotplug: silence test command echo")
Reported-by: Naresh Kamboju <naresh.kamboju(a)linaro.org>
Signed-off-by: Shuah Khan <shuahkh(a)osg.samsung.com>
---
tools/testing/selftests/memory-hotplug/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/memory-hotplug/Makefile b/tools/testing/selftests/memory-hotplug/Makefile
index 183b46883875..686da510f989 100644
--- a/tools/testing/selftests/memory-hotplug/Makefile
+++ b/tools/testing/selftests/memory-hotplug/Makefile
@@ -5,7 +5,8 @@ include ../lib.mk
TEST_PROGS := mem-on-off-test.sh
override RUN_TESTS := @./mem-on-off-test.sh -r 2 && echo "selftests: memory-hotplug [PASS]" || echo "selftests: memory-hotplug [FAIL]"
-override EMIT_TESTS := echo "$(RUN_TESTS)"
+
+override EMIT_TESTS := echo "$(subst @,,$(RUN_TESTS))"
run_full_test:
@/bin/bash ./mem-on-off-test.sh && echo "memory-hotplug selftests: [PASS]" || echo "memory-hotplug selftests: [FAIL]"
--
2.14.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bug fixes and an enhancement for the recent forwarding tests:
- only check tc version on tc tests
- handle multipath tests failing with 0 packet count
- fix ping command for IPv6 on Debian jessie
- improve summary of multipath tests
v2
- add CHECK_TC to bridge_vlan_aware.sh (Ido)
- dropped patch 2; always check for mz given its use
- fixed commit message for the last patch (Multipath: was dropped)
David Ahern (4):
selftests: forwarding: Only check tc version for tc tests
selftests: forwarding: Handle 0 for packet difference in multipath
tests
selftests: forwarding: Use PING6 instead of ping for ipv6 multipath
test
selftests: forwarding: Add description to the multipath tests
.../selftests/net/forwarding/bridge_vlan_aware.sh | 1 +
tools/testing/selftests/net/forwarding/lib.sh | 29 ++++++++------
.../selftests/net/forwarding/router_multipath.sh | 46 +++++++++++++---------
.../testing/selftests/net/forwarding/tc_actions.sh | 2 +-
.../testing/selftests/net/forwarding/tc_chains.sh | 2 +-
.../testing/selftests/net/forwarding/tc_common.sh | 2 +
.../testing/selftests/net/forwarding/tc_flower.sh | 2 +-
.../selftests/net/forwarding/tc_shblocks.sh | 2 +-
8 files changed, 53 insertions(+), 33 deletions(-)
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bug fixes and an enhancement for the recent forwarding tests:
- only check tc version on tc tests
- only check for mz for tests that will use it
- handle multipath tests failing with 0 packet count
- fix ping command for IPv6 on Debian jessie
- improve summary of multipath tests
BTW: who is the cruel person that made whitespace 7 spaces in these files?
David Ahern (5):
selftests: forwarding: Only check tc version for tc tests
selftests: forwarding: Only check for mz when it is needed
selftests: forwarding: Handle 0 for packet difference in multipath
tests
selftests: forwarding: Use PING6 instead of ping for ipv6 multipath
test
selftests: forwarding: Add description to the multipath tests
tools/testing/selftests/net/forwarding/lib.sh | 37 ++++++++++-------
.../selftests/net/forwarding/router_multipath.sh | 47 +++++++++++++---------
.../testing/selftests/net/forwarding/tc_actions.sh | 2 +-
.../testing/selftests/net/forwarding/tc_chains.sh | 2 +-
.../testing/selftests/net/forwarding/tc_common.sh | 3 ++
.../testing/selftests/net/forwarding/tc_flower.sh | 2 +-
.../selftests/net/forwarding/tc_shblocks.sh | 2 +-
7 files changed, 59 insertions(+), 36 deletions(-)
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 02/28/2018 08:57 AM, Daniel Borkmann wrote:
> Hi Tushar,
>
> On 02/28/2018 01:33 AM, Tushar Dave wrote:
>> Using bpf_probe_read_str() from samples/bpf causes compiler warning.
>> e.g.
>> warning: implicit declaration of function 'bpf_probe_read_str' is invalid in C99
>> [-Wimplicit-function-declaration]
>> num = bpf_probe_read_str(buf, sizeof(buf), ctx->di);
>> ^
>> 1 warning generated.
>>
>> Add bpf_probe_read_str() to bpf_helpers.h so it can be used by
>> samples/bpf programs.
>>
>> Signed-off-by: Tushar Dave <tushar.n.dave(a)oracle.com>
>
> In general no objections to it, but it would need an in-tree
> user first:
>
> $ git grep -n bpf_probe_read_str tools/
> tools/include/uapi/linux/bpf.h:596: * int bpf_probe_read_str(void *dst, int size, const void *unsafe_ptr)
> $
>
> Why not adding this along with a sample?
Okay, I will send new patch along with new sample or add usage of
bpf_probe_read_str() in one of our exiting sample :)
Thanks.
-Tushar
PS: adding correct mail-list this time linux-kselftest(a)vger.kernel.org
>
> Thanks,
> Daniel
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Linus,
Please pull the following Kselftest update for 4.16-rc4.
This kselftest fixes update for 4.16-rc4 consists of of fixes for
various problems in test output, compile errors, and missing configs.
Diff for the update is attached.
thanks,
-- Shuah
-----------------------------------------------------------------------------------
The following changes since commit 7928b2cbe55b2a410a0f5c1f154610059c57b1b2:
Linux 4.16-rc1 (2018-02-11 15:04:29 -0800)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-4.16-rc4
for you to fetch changes up to f6869826de700bce59e2cef14974f99836e34e4f:
selftests: vm: update .gitignore with new test (2018-02-26 16:09:50 -0700)
----------------------------------------------------------------
linux-kselftest-4.16-rc4
This kselftest fixes update for 4.16-rc4 consists of of fixes for
various problems in test output, compile errors, and missing configs.
----------------------------------------------------------------
Anders Roxell (2):
selftests: sync: missing CFLAGS while compiling
selftests: memfd: add config fragment for fuse
Daniel Díaz (2):
selftests/android: Fix line continuation in Makefile
selftests/futex: Fix line continuation in Makefile
Dominik Brodowski (1):
selftest/vDSO: fix O=
Naresh Kamboju (1):
selftests: pstore: Adding config fragment CONFIG_PSTORE_RAM=m
Shuah Khan (2):
selftests: memory-hotplug: silence test command echo
selftests: vm: update .gitignore with new test
tools/testing/selftests/android/Makefile | 8 ++++----
tools/testing/selftests/futex/Makefile | 6 +++---
tools/testing/selftests/memfd/config | 1 +
tools/testing/selftests/memory-hotplug/Makefile | 2 +-
tools/testing/selftests/pstore/config | 1 +
tools/testing/selftests/sync/Makefile | 2 +-
tools/testing/selftests/vDSO/Makefile | 14 +++++++-------
tools/testing/selftests/vm/.gitignore | 1 +
8 files changed, 19 insertions(+), 16 deletions(-)
create mode 100644 tools/testing/selftests/memfd/config
-----------------------------------------------------------------------------------
One of the nice things about network namespaces is that they allow one
to easily create and test complex environments.
Unfortunately, these namespaces can not be used with actual switching
ASICs, as their ports can not be migrated to other network namespaces
(NETIF_F_NETNS_LOCAL) and most of them probably do not support the
L1-separation provided by namespaces.
However, a similar kind of flexibility can be achieved by using VRFs and
by looping the switch ports together. For example:
br0
+
vrf-h1 | vrf-h2
+ +---+----+ +
| | | |
192.0.2.1/24 + + + + 192.0.2.2/24
swp1 swp2 swp3 swp4
+ + + +
| | | |
+--------+ +--------+
The VRFs act as lightweight namespaces representing hosts connected to
the switch.
This approach for testing switch ASICs has several advantages over the
traditional method that requires multiple physical machines, to name a
few:
1. Only the device under test (DUT) is being tested without noise from
other system.
2. Ability to easily provision complex topologies. Testing bridging
between 4-ports LAGs or 8-way ECMP requires many physical links that are
not always available. With the VRF-based approach one merely needs to
loopback more ports.
These tests are written with switch ASICs in mind, but they can be run
on any Linux box using veth pairs to emulate physical loopbacks.
v2:
* Order local variables declaration according to function arguments
order (Petr)
v1:
* Change location to net/forwarding instead of forwarding/
* Add ability to pause on failure
* Add ability to pause on cleanup
* Make configuration file optional
* Make ping/ping6/mz configurable
* Add more tc tests
Ido Schimmel (7):
selftests: forwarding: Add initial testing framework
selftests: forwarding: Add a test for FDB learning
selftests: forwarding: Add a test for flooded traffic
selftests: forwarding: Add a test for basic IPv4 and IPv6 routing
selftests: forwarding: Create test topology for multipath routing
selftests: forwarding: Test IPv4 weighted nexthops
selftests: forwarding: Test IPv6 weighted nexthops
Jiri Pirko (7):
selftests: forwarding: Add tc offload check helper
selftests: forwarding: Add MAC get helper
selftests: forwarding: Allow to get netdev interfaces names from
commandline
selftests: forwarding: Introduce tc flower matching tests
selftests: forwarding: Introduce tc actions tests
selftests: forwarding: Introduce basic tc chains tests
selftests: forwarding: Introduce basic shared blocks tests
tools/testing/selftests/net/forwarding/.gitignore | 1 +
tools/testing/selftests/net/forwarding/README | 56 +++
.../selftests/net/forwarding/bridge_vlan_aware.sh | 87 ++++
tools/testing/selftests/net/forwarding/config | 12 +
.../net/forwarding/forwarding.config.sample | 31 ++
tools/testing/selftests/net/forwarding/lib.sh | 533 +++++++++++++++++++++
tools/testing/selftests/net/forwarding/router.sh | 125 +++++
.../selftests/net/forwarding/router_multipath.sh | 322 +++++++++++++
.../testing/selftests/net/forwarding/tc_actions.sh | 195 ++++++++
.../testing/selftests/net/forwarding/tc_chains.sh | 122 +++++
.../testing/selftests/net/forwarding/tc_common.sh | 23 +
.../testing/selftests/net/forwarding/tc_flower.sh | 196 ++++++++
.../selftests/net/forwarding/tc_shblocks.sh | 122 +++++
13 files changed, 1825 insertions(+)
create mode 100644 tools/testing/selftests/net/forwarding/.gitignore
create mode 100644 tools/testing/selftests/net/forwarding/README
create mode 100755 tools/testing/selftests/net/forwarding/bridge_vlan_aware.sh
create mode 100644 tools/testing/selftests/net/forwarding/config
create mode 100644 tools/testing/selftests/net/forwarding/forwarding.config.sample
create mode 100644 tools/testing/selftests/net/forwarding/lib.sh
create mode 100755 tools/testing/selftests/net/forwarding/router.sh
create mode 100755 tools/testing/selftests/net/forwarding/router_multipath.sh
create mode 100755 tools/testing/selftests/net/forwarding/tc_actions.sh
create mode 100755 tools/testing/selftests/net/forwarding/tc_chains.sh
create mode 100644 tools/testing/selftests/net/forwarding/tc_common.sh
create mode 100755 tools/testing/selftests/net/forwarding/tc_flower.sh
create mode 100755 tools/testing/selftests/net/forwarding/tc_shblocks.sh
--
2.14.3
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html