Hi,
This is basically a resend, with a rebase onto today's latest Linux
main, in order to show that the patches are still relevant and correct.
Changes since v3:
1. Rebased onto Linux 6.10-rc6+.
Cover letter for v3:
Jason A. Donenfeld, I've added you because I ended up looking through
your latest "implement getrandom() in vDSO" series [1], which also
touches this Makefile, so just a heads up about upcoming (minor) merge
conflicts.
Changes since v2:
1. Added two patches, both of which apply solely to the Makefile.
These provide a smaller, cleaner, and more accurate Makefile.
2. Added Reviewed-by and Tested-by tags for the original patch, which
fixes all of the clang errors and warnings for this selftest.
3. Removed an obsolete blurb from the commit description of the original
patch, now that Valentin Obst LLVM build fix has been merged.
John Hubbard (3):
selftests/vDSO: fix clang build errors and warnings
selftests/mm: remove partially duplicated "all:" target in Makefile
selftests/vDSO: remove duplicate compiler invocations from Makefile
tools/testing/selftests/vDSO/Makefile | 29 ++++++++-----------
tools/testing/selftests/vDSO/parse_vdso.c | 16 ++++++----
.../selftests/vDSO/vdso_standalone_test_x86.c | 18 ++++++++++--
3 files changed, 39 insertions(+), 24 deletions(-)
base-commit: 8a9c6c40432e265600232b864f97d7c675e8be52
--
2.45.2
When building with clang, via:
make LLVM=1 -C tools/testing/selftest
...clang warns about an unused irqcount variable. clang is correct: the
variable is incremented and then ignored.
Fix this by deleting the irqcount variable.
Signed-off-by: John Hubbard <jhubbard(a)nvidia.com>
---
Changes since v2:
1) Rebased onto Linux 6.10-rc6+
Changes since the first version:
1) Rebased onto Linux 6.10-rc1
thanks,
John Hubbard
tools/testing/selftests/timers/rtcpie.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/timers/rtcpie.c b/tools/testing/selftests/timers/rtcpie.c
index 4ef2184f1558..7c07edd0d450 100644
--- a/tools/testing/selftests/timers/rtcpie.c
+++ b/tools/testing/selftests/timers/rtcpie.c
@@ -29,7 +29,7 @@ static const char default_rtc[] = "/dev/rtc0";
int main(int argc, char **argv)
{
- int i, fd, retval, irqcount = 0;
+ int i, fd, retval;
unsigned long tmp, data, old_pie_rate;
const char *rtc = default_rtc;
struct timeval start, end, diff;
@@ -120,7 +120,6 @@ int main(int argc, char **argv)
fprintf(stderr, " %d",i);
fflush(stderr);
- irqcount++;
}
/* Disable periodic interrupts */
base-commit: 8a9c6c40432e265600232b864f97d7c675e8be52
--
2.45.2
These patches aim to make using the openvswitch testsuite more reliable.
These should address the major sources of flakiness in the openvswitch
test suite allowing the CI infrastructure to exercise the openvswitch
module for patch series. There should be no change for users who simply
run the tests (except that patch 3/3 does make some of the debugging a bit
easier by making some output more verbose).
Aaron Conole (3):
selftests: openvswitch: Bump timeout to 15 minutes.
selftests: openvswitch: Attempt to autoload module.
selftests: openvswitch: Be more verbose with selftest debugging.
.../selftests/net/openvswitch/openvswitch.sh | 23 ++++++++++++-------
.../selftests/net/openvswitch/settings | 1 +
2 files changed, 16 insertions(+), 8 deletions(-)
create mode 100644 tools/testing/selftests/net/openvswitch/settings
--
2.45.1
On Fri, Jul 05, 2024 at 11:49:28AM GMT, Adrián Moreno wrote:
> On Thu, Jul 04, 2024 at 10:56:51AM GMT, Adrian Moreno wrote:
> > ** Background **
> > Currently, OVS supports several packet sampling mechanisms (sFlow,
> > per-bridge IPFIX, per-flow IPFIX). These end up being translated into a
> > userspace action that needs to be handled by ovs-vswitchd's handler
> > threads only to be forwarded to some third party application that
> > will somehow process the sample and provide observability on the
> > datapath.
> >
> > A particularly interesting use-case is controller-driven
> > per-flow IPFIX sampling where the OpenFlow controller can add metadata
> > to samples (via two 32bit integers) and this metadata is then available
> > to the sample-collecting system for correlation.
> >
> > ** Problem **
> > The fact that sampled traffic share netlink sockets and handler thread
> > time with upcalls, apart from being a performance bottleneck in the
> > sample extraction itself, can severely compromise the datapath,
> > yielding this solution unfit for highly loaded production systems.
> >
> > Users are left with little options other than guessing what sampling
> > rate will be OK for their traffic pattern and system load and dealing
> > with the lost accuracy.
> >
> > Looking at available infrastructure, an obvious candidated would be
> > to use psample. However, it's current state does not help with the
> > use-case at stake because sampled packets do not contain user-defined
> > metadata.
> >
> > ** Proposal **
> > This series is an attempt to fix this situation by extending the
> > existing psample infrastructure to carry a variable length
> > user-defined cookie.
> >
> > The main existing user of psample is tc's act_sample. It is also
> > extended to forward the action's cookie to psample.
> >
> > Finally, a new OVS action (OVS_SAMPLE_ATTR_PSAMPLE) is created.
> > It accepts a group and an optional cookie and uses psample to
> > multicast the packet and the metadata.
> >
> > --
> > v8 -> v9:
> > - Rebased.
> >
> > v7 -> v8:
> > - Rebased
> > - Redirect flow insertion to /dev/null to avoid spat in test.
> > - Removed inline keyword in stub execute_psample_action function.
> >
> > v6 -> v7:
> > - Rebased
> > - Fixed typo in comment.
> >
> > v5 -> v6:
> > - Renamed emit_sample -> psample
> > - Addressed unused variable and conditionally compilation of function.
> >
> > v4 -> v5:
> > - Rebased.
> > - Removed lefover enum value and wrapped some long lines in selftests.
> >
> > v3 -> v4:
> > - Rebased.
> > - Addressed Jakub's comment on private and unused nla attributes.
> >
> > v2 -> v3:
> > - Addressed comments from Simon, Aaron and Ilya.
> > - Dropped probability propagation in nested sample actions.
> > - Dropped patch v2's 7/9 in favor of a userspace implementation and
> > consume skb if emit_sample is the last action, same as we do with
> > userspace.
> > - Split ovs-dpctl.py features in independent patches.
> >
> > v1 -> v2:
> > - Create a new action ("emit_sample") rather than reuse existing
> > "sample" one.
> > - Add probability semantics to psample's sampling rate.
> > - Store sampling probability in skb's cb area and use it in emit_sample.
> > - Test combining "emit_sample" with "trunc"
> > - Drop group_id filtering and tracepoint in psample.
> >
> > rfc_v2 -> v1:
> > - Accommodate Ilya's comments.
> > - Split OVS's attribute in two attributes and simplify internal
> > handling of psample arguments.
> > - Extend psample and tc with a user-defined cookie.
> > - Add a tracepoint to psample to facilitate troubleshooting.
> >
> > rfc_v1 -> rfc_v2:
> > - Use psample instead of a new OVS-only multicast group.
> > - Extend psample and tc with a user-defined cookie.
> >
> > Adrian Moreno (10):
> > net: psample: add user cookie
> > net: sched: act_sample: add action cookie to sample
> > net: psample: skip packet copy if no listeners
> > net: psample: allow using rate as probability
> > net: openvswitch: add psample action
> > net: openvswitch: store sampling probability in cb.
> > selftests: openvswitch: add psample action
> > selftests: openvswitch: add userspace parsing
> > selftests: openvswitch: parse trunc action
> > selftests: openvswitch: add psample test
> >
> > Documentation/netlink/specs/ovs_flow.yaml | 17 ++
> > include/net/psample.h | 5 +-
> > include/uapi/linux/openvswitch.h | 31 +-
> > include/uapi/linux/psample.h | 11 +-
> > net/openvswitch/Kconfig | 1 +
> > net/openvswitch/actions.c | 66 ++++-
> > net/openvswitch/datapath.h | 3 +
> > net/openvswitch/flow_netlink.c | 32 ++-
> > net/openvswitch/vport.c | 1 +
> > net/psample/psample.c | 16 +-
> > net/sched/act_sample.c | 12 +
> > .../selftests/net/openvswitch/openvswitch.sh | 115 +++++++-
> > .../selftests/net/openvswitch/ovs-dpctl.py | 272 +++++++++++++++++-
> > 13 files changed, 566 insertions(+), 16 deletions(-)
> >
> > --
> > 2.45.2
> >
>
> Hi,
>
> Simon Horman has spotted that openvswitch.sh tests are failing in the
> debug executor:
>
> https://netdev.bots.linux.dev/contest.html?test=openvswitch-sh
>
> The failing tests are two: psample and upcall_interfaces. These two
> tests have a known source of instability (they use "sleep") that make
> them specially unreliable in slow systems.
>
> Aaron and I already discussed this and I'm working on a patch to make
> both tests more robust by adding a wait-and-retry mechanism.
>
> I hope this series can be considered regardless of this flaky tests.
>
Adding more context to explain our situation.
This series has a counterpart in OVS [1]. The state of this other series
is still RFC just because the kernel bits have not yet been merged.
OVS 3.4 "softfreeze" was declared last monday, which excludes from the
release any series that is stil in RFC state.
Given the kernel parts seemed very close to be merged, an exception was
given to the series so we can consider it for inclusion [2].
I hate to put any pressure on already busy maintainers but I would also
dislike missing this OVS release by just one or two days and having
to wait 6 months (OVS release cadence) for it to be available.
Again, I don't want to put pressure on maintainers. If it's not
possible, that's it. I just wanted to voice our timeline constraints.
Thanks for your understanding.
Adrián
[1] https://patchwork.ozlabs.org/project/openvswitch/cover/20240704085710.35384…
[2] https://mail.openvswitch.org/pipermail/ovs-dev/2024-July/415261.html
in the do_setcpu, this function does not need to have a return value,
which is meaningless
Signed-off-by: Liu Jing <liujing(a)cmss.chinamobile.com>
---
tools/testing/selftests/net/msg_zerocopy.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/net/msg_zerocopy.c b/tools/testing/selftests/net/msg_zerocopy.c
index bdc03a2097e8..0b54f2011449 100644
--- a/tools/testing/selftests/net/msg_zerocopy.c
+++ b/tools/testing/selftests/net/msg_zerocopy.c
@@ -118,7 +118,7 @@ static uint16_t get_ip_csum(const uint16_t *start, int num_words)
return ~sum;
}
-static int do_setcpu(int cpu)
+static void do_setcpu(int cpu)
{
cpu_set_t mask;
@@ -129,7 +129,6 @@ static int do_setcpu(int cpu)
else if (cfg_verbose)
fprintf(stderr, "cpu: %u\n", cpu);
- return 0;
}
static void do_setsockopt(int fd, int level, int optname, int val)
--
2.33.0
Recent CI failures brought my attention to the fact that pmu_counters_test
sometimes fails because it doesn't get any LLC cache misses.
It apparently happens because CLFLUSH can race with CPU prediction.
To attempt to fix this, implement a more aggressive cache flushing - now it is flushed
on each iteration of the measured loop which should at least reduce by order
of magnitude the chance of this happening.
This patch survived more that a day of running in a loop on a Comet Lake machine,
where the test used to fail after about 10-20 minites.
Best regards,
Maxim Levitsky
Maxim Levitsky (1):
KVM: selftests: pmu_counters_test: increase robustness of LLC cache
misses
.../selftests/kvm/x86_64/pmu_counters_test.c | 20 +++++++++----------
1 file changed, 9 insertions(+), 11 deletions(-)
--
2.26.3
From: Geliang Tang <tanggeliang(a)kylinos.cn>
v7:
- address Martin's comments in v6. (thanks)
- use MAX(opts->backlog, 0) instead of opts->backlog.
- use connect_to_fd_opts instead connect_to_fd.
- more ASSERT_* to check errors.
v6:
- update patch 6 as Daniel suggested. (thanks)
v5:
- keep make_server and make_client as Eduard suggested.
v4:
- a new patch to use make_sockaddr in sockmap_ktls.
- a new patch to close fd in error path in drop_on_reuseport.
- drop make_server() in patch 7.
- drop make_client() too in patch 9.
v3:
- a new patch to add backlog for network_helper_opts.
- use start_server_str in sockmap_ktls now, not start_server.
v2:
- address Eduard's comments in v1. (thanks)
- fix errors reported by CI.
This patch set uses network helpers in sockmap_ktls and sk_lookup, and
drop three local helpers tcp_server(), inetaddr_len() and make_socket()
in them.
Geliang Tang (9):
selftests/bpf: Add backlog for network_helper_opts
selftests/bpf: Use start_server_str in sockmap_ktls
selftests/bpf: Use connect_to_fd_opts in sockmap_ktls
selftests/bpf: Use make_sockaddr in sockmap_ktls
selftests/bpf: Close fd in error path in drop_on_reuseport
selftests/bpf: Use start_server_str in sk_lookup
selftests/bpf: Use connect_to_fd_opts in sk_lookup
selftests/bpf: Use connect_to_addr in sk_lookup
selftests/bpf: Drop make_socket in sk_lookup
tools/testing/selftests/bpf/network_helpers.c | 2 +-
tools/testing/selftests/bpf/network_helpers.h | 4 +
.../selftests/bpf/prog_tests/sk_lookup.c | 150 ++++++++----------
.../selftests/bpf/prog_tests/sockmap_ktls.c | 53 ++-----
4 files changed, 77 insertions(+), 132 deletions(-)
--
2.43.0
Hi Linus,
This PR fixes a few kselftests [1]. This has been in linux-next for a week and
rebased to add Mark Brown's Tested-by. The race condition found while writing
this fix is not new and seems specific to UML's hostfs (I also tested against
ext4 and btrfs without being able to trigger this issue).
Feel free to take this PR if you see fit.
Regards,
Mickaël
[1] https://lore.kernel.org/r/9341d4db-5e21-418c-bf9e-9ae2da7877e1@sirena.org.uk
--
The following changes since commit f2661062f16b2de5d7b6a5c42a9a5c96326b8454:
Linux 6.10-rc5 (2024-06-23 17:08:54 -0400)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/mic/linux.git tags/kselftest-fix-2024-07-04
for you to fetch changes up to 130e42806773013e9cf32d211922c935ae2df86c:
selftests/harness: Fix tests timeout and race condition (2024-06-28 16:06:03 +0200)
----------------------------------------------------------------
Fix Kselftests timeout and race condition
----------------------------------------------------------------
Mickaël Salaün (1):
selftests/harness: Fix tests timeout and race condition
tools/testing/selftests/kselftest_harness.h | 43 ++++++++++++++++-------------
1 file changed, 24 insertions(+), 19 deletions(-)
Current practice in the selftests Makefiles is to use $(LLVM) as a way
to decide if clang is being used as the compiler (and/or the linker
front end). Unfortunately, this does not cover all of the use cases:
1) CC could have been set within selftests/lib.mk, by inferring it from
LLVM==1, or
2) CC could have been set externally, such as when cross compiling.
Solution: In order to allow subsystem selftests to more accurately
control clang-specific behavior, such as compiler options, provide a new
Makefile variable: SELFTESTS_CC_IS_CLANG. If $(CC) contains an
invocation of clang in any form, then SELFTESTS_CC_IS_CLANG will be
non-empty.
SELFTESTS_CC_IS_CLANG does not specify which linker is being used.
However, it can still help with linker options, because $(CC) is often
used to do both the compile and link steps (often in the same step).
Signed-off-by: John Hubbard <jhubbard(a)nvidia.com>
---
Hi,
If this looks reasonable, I'll break it up into separate patches and
post it as a non-RFC.
thanks,
John Hubbard
tools/testing/selftests/bpf/Makefile | 2 +-
tools/testing/selftests/fchmodat2/Makefile | 12 +++++++-----
tools/testing/selftests/hid/Makefile | 2 +-
tools/testing/selftests/lib.mk | 15 +++++++++++++++
tools/testing/selftests/openat2/Makefile | 16 +++++++++-------
5 files changed, 33 insertions(+), 14 deletions(-)
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index dd49c1d23a60..6b924297ab71 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -55,7 +55,7 @@ progs/test_sk_lookup.c-CFLAGS := -fno-strict-aliasing
progs/timer_crash.c-CFLAGS := -fno-strict-aliasing
progs/test_global_func9.c-CFLAGS := -fno-strict-aliasing
-ifneq ($(LLVM),)
+ifeq ($(SELFTESTS_CC_IS_CLANG),)
# Silence some warnings when compiled with clang
CFLAGS += -Wno-unused-command-line-argument
endif
diff --git a/tools/testing/selftests/fchmodat2/Makefile b/tools/testing/selftests/fchmodat2/Makefile
index 4373cea79b79..d00b01be5d96 100644
--- a/tools/testing/selftests/fchmodat2/Makefile
+++ b/tools/testing/selftests/fchmodat2/Makefile
@@ -2,14 +2,16 @@
CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined $(KHDR_INCLUDES)
+TEST_GEN_PROGS := fchmodat2_test
+
+include ../lib.mk
+
# gcc requires -static-libasan in order to ensure that Address Sanitizer's
# library is the first one loaded. However, clang already statically links the
# Address Sanitizer if -fsanitize is specified. Therefore, simply omit
# -static-libasan for clang builds.
-ifeq ($(LLVM),)
+# This check must be done after including ../lib.mk, in order to pick up the
+# correct value of SELFTESTS_CC_IS_CLANG.
+ifeq ($(SELFTESTS_CC_IS_CLANG),)
CFLAGS += -static-libasan
endif
-
-TEST_GEN_PROGS := fchmodat2_test
-
-include ../lib.mk
diff --git a/tools/testing/selftests/hid/Makefile b/tools/testing/selftests/hid/Makefile
index 2b5ea18bde38..734a53dc8ad9 100644
--- a/tools/testing/selftests/hid/Makefile
+++ b/tools/testing/selftests/hid/Makefile
@@ -27,7 +27,7 @@ CFLAGS += -I$(OUTPUT)/tools/include
LDLIBS += -lelf -lz -lrt -lpthread
# Silence some warnings when compiled with clang
-ifneq ($(LLVM),)
+ifeq ($(SELFTESTS_CC_IS_CLANG),)
CFLAGS += -Wno-unused-command-line-argument
endif
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 429535816dbd..f321ad5a1d0c 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -43,6 +43,21 @@ else
CC := $(CROSS_COMPILE)gcc
endif # LLVM
+# SELFTESTS_CC_IS_CLANG allows subsystem selftests to more accurately control
+# clang-specific behavior, such as compiler options. If CC is an invocation of
+# clang in any form, then SELFTESTS_CC_IS_CLANG will be non-empty. Notes:
+#
+# 1) CC could have been set above, by inferring it from LLVM==1, or externally,
+# from the CC shell environment variable.
+#
+# 2) SELFTESTS_CC_IS_CLANG does not specify which linker is being used. However,
+# it can still help with linker options, if clang or gcc is used for the
+# linker front end.
+SELFTESTS_CC_IS_CLANG :=
+ifeq ($(findstring clang,$(CC)),clang)
+ SELFTESTS_CC_IS_CLANG := 1
+endif
+
ifeq (0,$(MAKELEVEL))
ifeq ($(OUTPUT),)
OUTPUT := $(shell pwd)
diff --git a/tools/testing/selftests/openat2/Makefile b/tools/testing/selftests/openat2/Makefile
index 185dc76ebb5f..7acb85a8f2ac 100644
--- a/tools/testing/selftests/openat2/Makefile
+++ b/tools/testing/selftests/openat2/Makefile
@@ -3,16 +3,18 @@
CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined
TEST_GEN_PROGS := openat2_test resolve_test rename_attack_test
+LOCAL_HDRS += helpers.h
+
+include ../lib.mk
+
+$(TEST_GEN_PROGS): helpers.c
+
# gcc requires -static-libasan in order to ensure that Address Sanitizer's
# library is the first one loaded. However, clang already statically links the
# Address Sanitizer if -fsanitize is specified. Therefore, simply omit
# -static-libasan for clang builds.
-ifeq ($(LLVM),)
+# This check must be done after including ../lib.mk, in order to pick up the
+# correct value of SELFTESTS_CC_IS_CLANG.
+ifeq ($(SELFTESTS_CC_IS_CLANG),)
CFLAGS += -static-libasan
endif
-
-LOCAL_HDRS += helpers.h
-
-include ../lib.mk
-
-$(TEST_GEN_PROGS): helpers.c
base-commit: 9a5cd459be8a425d70cda1fa1c89af7875a35d17
--
2.45.2