*Changes in v10*
- Add specific condition to return error if hugetlb is used with wp
async
- Move changes in tools/include/uapi/linux/fs.h to separate patch
- Add documentation
*Changes in v9:*
- Correct fault resolution for userfaultfd wp async
- Fix build warnings and errors which were happening on some configs
- Simplify pagemap ioctl's code
*Changes in v8:*
- Update uffd async wp implementation
- Improve PAGEMAP_IOCTL implementation
*Changes in v7:*
- Add uffd wp async
- Update the IOCTL to use uffd under the hood instead of soft-dirty
flags
Hello,
Note:
Soft-dirty pages and pages which have been written-to are synonyms. As
kernel already has soft-dirty feature inside which we have given up to
use, we are using written-to terminology while using UFFD async WP under
the hood.
This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear
the info about page table entries. The following operations are
supported in this ioctl:
- Get the information if the pages have been written-to (PAGE_IS_WRITTEN),
file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped
(PAGE_IS_SWAPPED).
- Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which
pages have been written-to.
- Find pages which have been written-to and write protect the pages
(atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE)
It is possible to find and clear soft-dirty pages entirely in userspace.
But it isn't efficient:
- The mprotect and SIGSEGV handler for bookkeeping
- The userfaultfd wp (synchronous) with the handler for bookkeeping
Some benchmarks can be seen here[1]. This series adds features that weren't
present earlier:
- There is no atomic get soft-dirty/Written-to status and clear present in
the kernel.
- The pages which have been written-to can not be found in accurate way.
(Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty
pages than there actually are.)
Historically, soft-dirty PTE bit tracking has been used in the CRIU
project. The procfs interface is enough for finding the soft-dirty bit
status and clearing the soft-dirty bit of all the pages of a process.
We have the use case where we need to track the soft-dirty PTE bit for
only specific pages on-demand. We need this tracking and clear mechanism
of a region of memory while the process is running to emulate the
getWriteWatch() syscall of Windows.
*(Moved to using UFFD instead of soft-dirtyi feature to find pages which
have been written-to from v7 patch series)*:
Stop using the soft-dirty flags for finding which pages have been
written to. It is too delicate and wrong as it shows more soft-dirty
pages than the actual soft-dirty pages. There is no interest in
correcting it [2][3] as this is how the feature was written years ago.
It shouldn't be updated to changed behaviour. Peter Xu has suggested
using the async version of the UFFD WP [4] as it is based inherently
on the PTEs.
So in this patch series, I've added a new mode to the UFFD which is
asynchronous version of the write protect. When this variant of the
UFFD WP is used, the page faults are resolved automatically by the
kernel. The pages which have been written-to can be found by reading
pagemap file (!PM_UFFD_WP). This feature can be used successfully to
find which pages have been written to from the time the pages were
write protected. This works just like the soft-dirty flag without
showing any extra pages which aren't soft-dirty in reality.
The information related to pages if the page is file mapped, present and
swapped is required for the CRIU project [5][6]. The addition of the
required mask, any mask, excluded mask and return masks are also required
for the CRIU project [5].
The IOCTL returns the addresses of the pages which match the specific
masks. The page addresses are returned in struct page_region in a compact
form. The max_pages is needed to support a use case where user only wants
to get a specific number of pages. So there is no need to find all the
pages of interest in the range when max_pages is specified. The IOCTL
returns when the maximum number of the pages are found. The max_pages is
optional. If max_pages is specified, it must be equal or greater than the
vec_size. This restriction is needed to handle worse case when one
page_region only contains info of one page and it cannot be compacted.
This is needed to emulate the Windows getWriteWatch() syscall.
The patch series include the detailed selftest which can be used as an
example for the uffd async wp test and PAGEMAP_IOCTL. It shows the
interface usages as well.
[1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora…
[2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.…
[3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.…
[4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n
[5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/
[6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/
Regards,
Muhammad Usama Anjum
Muhammad Usama Anjum (6):
userfaultfd: Add UFFD WP Async support
userfaultfd: update documentation to describe UFFD_FEATURE_WP_ASYNC
fs/proc/task_mmu: Implement IOCTL to get and/or the clear info about
PTEs
tools headers UAPI: Update linux/fs.h with the kernel sources
mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL
selftests: vm: add pagemap ioctl tests
Documentation/admin-guide/mm/pagemap.rst | 24 +
Documentation/admin-guide/mm/userfaultfd.rst | 7 +
fs/proc/task_mmu.c | 290 ++++++
fs/userfaultfd.c | 20 +-
include/linux/userfaultfd_k.h | 11 +
include/uapi/linux/fs.h | 50 ++
include/uapi/linux/userfaultfd.h | 10 +-
mm/memory.c | 23 +-
tools/include/uapi/linux/fs.h | 50 ++
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 5 +-
tools/testing/selftests/vm/pagemap_ioctl.c | 881 +++++++++++++++++++
12 files changed, 1364 insertions(+), 8 deletions(-)
create mode 100644 tools/testing/selftests/vm/pagemap_ioctl.c
--
2.30.2
Patch 1 fixes a possible deadlock in subflow_error_report() reported by
lockdep. The report was in fact a false positive but the modification
makes sense and silences lockdep to allow syzkaller to find real issues.
The regression has been introduced in v5.12.
Patch 2 is a refactoring needed to be able to fix the two next issues.
It improves the situation and can be backported up to v6.0.
Patches 3 and 4 fix UaF reported by KASAN. It fixes issues potentially
visible since v5.7 and v5.19 but only reproducible until recently
(v6.0). These two patches depend on patch 2/7.
Patch 5 fixes the order of the printed values: expected vs seen values.
The regression has been introduced recently: present in Linus' tree but
not in a tagged version yet.
Patch 6 adds missing ro_after_init flags. A previous patch added them
for other functions but these two have been missed. This previous patch
has been backported to stable versions (up to v5.12) so probably better
to do the same here.
Patch 7 fixes tcp_set_state() being called twice in a row since v5.10.
Signed-off-by: Matthieu Baerts <matthieu.baerts(a)tessares.net>
---
Geliang Tang (1):
mptcp: add ro_after_init for tcp{,v6}_prot_override
Matthieu Baerts (2):
selftests: mptcp: userspace pm: fix printed values
mptcp: avoid setting TCP_CLOSE state twice
Paolo Abeni (4):
mptcp: fix possible deadlock in subflow_error_report
mptcp: refactor passive socket initialization
mptcp: use the workqueue to destroy unaccepted sockets
mptcp: fix UaF in listener shutdown
net/mptcp/protocol.c | 44 +++-----
net/mptcp/protocol.h | 4 +-
net/mptcp/subflow.c | 122 +++++++---------------
tools/testing/selftests/net/mptcp/userspace_pm.sh | 2 +-
4 files changed, 59 insertions(+), 113 deletions(-)
---
base-commit: aaa3c08ee0653beaa649d4adfb27ad562641cfd8
change-id: 20230227-upstream-net-20230227-mptcp-fixes-cc78f3a2f5b2
Best regards,
--
Matthieu Baerts <matthieu.baerts(a)tessares.net>
Hello there,
I ran the static analyser cppcheck over the linux-6.2 source code and got this:
linux-6.2/tools/testing/selftests/powerpc/pmu/sampling_tests/mmcra_thresh_marked_sample_test.c:68:10: style: Same expression '0x3' found multiple times in chain of '&' operators. [duplicateExpression]
Source code is
FAIL_IF(EV_CODE_EXTRACT(event.attr.config, sample & 0x3) !=
get_mmcra_sample_mode(get_reg_value(intr_regs, "MMCRA"), 4));
but
#define EV_CODE_EXTRACT(x, y) \
((x >> ev_shift_##y) & ev_mask_##y)
Given the token pasting, I very much doubt an expression like "sample & 0x3"
will work correctly. Same thing on the line above
FAIL_IF(EV_CODE_EXTRACT(event.attr.config, sample >> 2) !=
get_mmcra_rand_samp_elig(get_reg_value(intr_regs, "MMCRA"), 4));
"sample >> 2" doesn't look like a valid token to me.
Regards
David Binderman
KUnit's 'hooks.o' file need to be built-in whenever KUnit is enabled
(even if CONFIG_KUNIT=m). We'd previously attemtped to do this by adding
'kunit/hooks.o' to obj-y in lib/Makefile, but this caused hooks.c to be
rebuilt even when it was unchanged.
Instead, always recurse into lib/kunit using obj-y when KUnit is
enabled, and add the hooks there.
Fixes: 7170b7ed6acb ("kunit: Add "hooks" to call into KUnit when it's built as a module").
Reported-by: Linus Torvalds <torvalds(a)linux-foundation.org>
Link: https://lore.kernel.org/linux-kselftest/CAHk-=wiEf7irTKwPJ0jTMOF3CS-13UXmF6…
Signed-off-by: David Gow <davidgow(a)google.com>
---
I like this way of handling the makefiles much better. I had tried it
when originally writing the hooks patch and not managed to get it
working. Not sure what's changed now, but it works in all of the usual
cases (CONFIG_KUNIT={n,y,m}, kunit.py run, etc).
Linus, Shuah: Let me know if you want this to go via the KUnit branch,
or if you just want to apply it directly and get rid of the annoyances
ASAP.
---
lib/Makefile | 12 ++++--------
lib/kunit/Makefile | 2 +-
2 files changed, 5 insertions(+), 9 deletions(-)
diff --git a/lib/Makefile b/lib/Makefile
index 469be6240523..baf2821f7a00 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -127,14 +127,10 @@ CFLAGS_test_fpu.o += $(FPU_CFLAGS)
obj-$(CONFIG_TEST_LIVEPATCH) += livepatch/
-obj-$(CONFIG_KUNIT) += kunit/
-# Include the KUnit hooks unconditionally. They'll compile to nothing if
-# CONFIG_KUNIT=n, otherwise will be a small table of static data (static key,
-# function pointers) which need to be built-in even when KUnit is a module.
-ifeq ($(CONFIG_KUNIT), m)
-obj-y += kunit/hooks.o
-else
-obj-$(CONFIG_KUNIT) += kunit/hooks.o
+# Some KUnit files (hooks.o) need to be built-in even when KUnit is a module,
+# so we can't just use obj-$(CONFIG_KUNIT).
+ifdef CONFIG_KUNIT
+obj-y += kunit/
endif
ifeq ($(CONFIG_DEBUG_KOBJECT),y)
diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile
index da665cd4ea12..cb417f504996 100644
--- a/lib/kunit/Makefile
+++ b/lib/kunit/Makefile
@@ -13,7 +13,7 @@ kunit-objs += debugfs.o
endif
# KUnit 'hooks' are built-in even when KUnit is built as a module.
-lib-y += hooks.o
+obj-y += hooks.o
obj-$(CONFIG_KUNIT_TEST) += kunit-test.o
--
2.39.2.637.g21b0678d19-goog
Dear maintainers,
This series is following up on the last fix [2]. I thought I could
forget about it with that. But, I was wrong because now this was
realized as an incomplete solution -- my bad. Here is some context for
this series:
The last fix [3] has resolved the case when copying the initialized
dynamic state from init_fpstate to the user buffer in
__copy_xstate_to_uabi_buf(). (This was intended to resolve the fallout
of the init_fpstate fix [1].)
But, when copying the *non-initialized* dynamic state from the task
xstate, the code [4] unconditionally retrieves the address in
init_fpstate which is needless. Consequently, this triggers a
false-positive warning as shown in [5] which meaninglessly confuses
users.
With these repetitive surgeries, a more solid and comprehensive
solution is more helpful I thought. Considerably removing init_fpstate
from this loop is not impossible here because dynamic states have an
all-zeros init state. Then, zeroing the user buffer instead of
retrieving init_fpstate resolves the issue and simplifies the code.
These issues were discovered from the KVM execution with launching a
guest and running the KVM self-test as __copy_xstate_to_uabi_buf() was
called. But, the negligibly missing ptrace test could have disclosed
them too. So that case is included here.
Thanks,
Chang
[1] https://lore.kernel.org/lkml/20220824191223.1248-1-chang.seok.bae@intel.com/
[2] https://lore.kernel.org/lkml/20221018221349.4196-1-chang.seok.bae@intel.com/
[3] https://lore.kernel.org/lkml/20221021185844.13472-1-chang.seok.bae@intel.co…
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arc…
[5] https://lore.kernel.org/kvm/20230221163655.920289-2-mizhang@google.com/
Chang S. Bae (2):
x86/fpu/xstate: Prevent false-positive warning in
__copy_xstate_uabi_buf()
selftests/x86/amx: Add a ptrace test
arch/x86/kernel/fpu/xstate.c | 30 ++++-----
tools/testing/selftests/x86/amx.c | 108 +++++++++++++++++++++++++++++-
2 files changed, 119 insertions(+), 19 deletions(-)
base-commit: 7fa08de735e41001a70c8ca869b2b159d74c2339
--
2.17.1
From: Rong Tao <rongtao(a)cestc.cn>
commit bc292ab00f6c("mm: introduce vma->vm_flags wrapper functions")
turns the vm_flags into a const variable.
Added bpf_find_vma test in commit f108662b27c9("selftests/bpf: Add tests
for bpf_find_vma") to assign values to variables that declare const in
find_vma_fail1.c programs, which is an error to the compiler and does not
test BPF verifiers. It is better to replace 'const vm_flags_t vm_flags'
with 'unsigned long vm_start' for testing.
$ make -C tools/testing/selftests/bpf/ -j8
...
progs/find_vma_fail1.c:16:16: error: cannot assign to non-static data
member 'vm_flags' with const-qualified type 'const vm_flags_t' (aka
'const unsigned long')
vma->vm_flags |= 0x55;
~~~~~~~~~~~~~ ^
../tools/testing/selftests/bpf/tools/include/vmlinux.h:1898:20:
note: non-static data member 'vm_flags' declared const here
const vm_flags_t vm_flags;
~~~~~~~~~~~`~~~~~~^~~~~~~~
Signed-off-by: Rong Tao <rongtao(a)cestc.cn>
---
v2: Add more useful commit information
v1: https://lore.kernel.org/lkml/tencent_FC8827062142CF5936974B2A30AF6CA3C408@q…
---
tools/testing/selftests/bpf/progs/find_vma_fail1.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/progs/find_vma_fail1.c b/tools/testing/selftests/bpf/progs/find_vma_fail1.c
index b3b326b8e2d1..47d5dedff554 100644
--- a/tools/testing/selftests/bpf/progs/find_vma_fail1.c
+++ b/tools/testing/selftests/bpf/progs/find_vma_fail1.c
@@ -13,7 +13,7 @@ static long write_vma(struct task_struct *task, struct vm_area_struct *vma,
struct callback_ctx *data)
{
/* writing to vma, which is illegal */
- vma->vm_flags |= 0x55;
+ vma->vm_start = 0xffffffffff600000;
return 0;
}
--
2.39.2
From: Chuck Lever <chuck.lever(a)oracle.com>
Allow the new GSS Kerberos encryption type test suites to run
outside of the kunit infrastructure. Replace the assertion that
fires when lookup_enctype() so that the case is skipped instead of
failing outright.
Reported-by: Geert Uytterhoeven <geert(a)linux-m68k.org>
Signed-off-by: Chuck Lever <chuck.lever(a)oracle.com>
---
Hey Geert -
This patch addresses part of your concern: with this patch applied,
you should be able to run only the tests that target the encryption
types that are enabled in your kernel. The other tests will be
skipped rather than fail outright.
net/sunrpc/auth_gss/gss_krb5_test.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/net/sunrpc/auth_gss/gss_krb5_test.c b/net/sunrpc/auth_gss/gss_krb5_test.c
index c287ce15c419..0a7c5280e4e3 100644
--- a/net/sunrpc/auth_gss/gss_krb5_test.c
+++ b/net/sunrpc/auth_gss/gss_krb5_test.c
@@ -49,7 +49,8 @@ static void kdf_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
derivedkey.data = kunit_kzalloc(test, param->expected_result->len,
GFP_KERNEL);
@@ -83,7 +84,8 @@ static void checksum_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
Kc.len = gk5e->Kc_length;
Kc.data = kunit_kzalloc(test, Kc.len, GFP_KERNEL);
@@ -725,7 +727,8 @@ static void rfc3962_encrypt_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
cbc_tfm = crypto_alloc_sync_skcipher(gk5e->aux_cipher, 0, 0);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, cbc_tfm);
@@ -1319,7 +1322,8 @@ static void rfc6803_encrypt_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
usage.data[3] = param->constant;
@@ -1810,7 +1814,8 @@ static void rfc8009_encrypt_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
*(__be32 *)usage.data = cpu_to_be32(2);
@@ -1975,7 +1980,8 @@ static void encrypt_selftest_case(struct kunit *test)
/* Arrange */
gk5e = gss_krb5_lookup_enctype(param->enctype);
- KUNIT_ASSERT_NOT_NULL(test, gk5e);
+ if (!gk5e)
+ kunit_skip(test, "Encryption type is not available");
cbc_tfm = crypto_alloc_sync_skcipher(gk5e->aux_cipher, 0, 0);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, cbc_tfm);
Add basic support to run m68k under QEMU via kunit_tool.
Signed-off-by: Geert Uytterhoeven <geert(a)linux-m68k.org>
---
Does this need CONFIG_VIRTIO_CONSOLE=y? It seems to work fine without.
drivers/clk/.kunitconfig:
Fails because m68k uses big-endian I/O ops
drivers/gpu/drm/tests/.kunitconfig
kernel BUG at drivers/gpu/drm/drm_buddy.c:21!
(also on arm, ignored)
drivers/gpu/drm/vc4/tests/.kunitconfig
Depends on arm, works on arm
drivers/hid/.kunitconfig
Unable to handle kernel NULL pointer dereference at virtual address (ptrval)
Oops: 00000000
PC: [<001d7380>] uclogic_params_ugee_v2_has_battery+0x6/0x68
(also on arm, ignored)
fs/ext4/.kunitconfig: OK
fs/fat/.kunitconfig: OK
kernel/kcsan/.kunitconfig
Needs HAVE_ARCH_KCSAN and SMP
(the former also on arm, ignored)
lib/kunit/.kunitconfig: OK
mm/kfence/.kunitconfig
Needs HAVE_ARCH_KFENCE
(fails on arm due to missing CONFIG_KFENCE_KUNIT_TEST=y, which
depends on TRACEPOINTS)
net/sunrpc/.kunitconfig
After dropping CONFIG_STACKTRACE=y (why is this needed?) from
net/sunrpc/.kunitconfig:
Unable to handle kernel NULL pointer dereference at virtual address (ptrval)
(also on arm, ignored)
---
tools/testing/kunit/qemu_configs/m68k.py | 10 ++++++++++
1 file changed, 10 insertions(+)
create mode 100644 tools/testing/kunit/qemu_configs/m68k.py
diff --git a/tools/testing/kunit/qemu_configs/m68k.py b/tools/testing/kunit/qemu_configs/m68k.py
new file mode 100644
index 0000000000000000..287fc386f8a7ff16
--- /dev/null
+++ b/tools/testing/kunit/qemu_configs/m68k.py
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='m68k',
+ kconfig='''
+CONFIG_VIRT=y''',
+ qemu_arch='m68k',
+ kernel_path='vmlinux',
+ kernel_command_line='console=hvc0',
+ extra_qemu_params=['-machine', 'virt'])
--
2.34.1