Thank you Michał.
On 7/21/23 12:28 AM, Michał Mirosław wrote:
> b. rename match "flags" to 'page categories' everywhere - this makes
> it easier to differentiate the ioctl()s categorisation of pages
> from struct page flags;
> c. change {required + excluded} to {inverted + required}. This was
> rejected before, but I'd like to illustrate the difference.
> Old interface can be translated to the new by:
> categories_inverted = excluded_mask
> categories_mask = required_mask | excluded_mask
> categories_anyof_mask = anyof_mask
> The new way allows filtering by: A & (B | !C)
> categories_inverted = C
> categories_mask = A
> categories_anyof_mask = B | C
Andrei and Danylo,
Are you okay with these masks? It were you two who had proposed these.
--
BR,
Muhammad Usama Anjum
This series demonstrates how KTAP output can be used by nolibc-test to
make the test results better to read for people and machines.
Especially when running multiple invocations for different architectors
or build configurations we can make use of the kernels TAP parser to
automatically provide aggregated test reports.
The code is very hacky and incomplete and mostly meant to validate if
the output format is useful.
Start with the last patch of the series to actually see the generated
format, or run it for yourself.
Signed-off-by: Thomas Weißschuh <linux(a)weissschuh.net>
---
Thomas Weißschuh (7):
selftests/nolibc: statically calculate number of testsuites
selftests/nolibc: use unsigned indices for testcases
selftests/nolibc: replace repetitive test structure with macro
selftests/nolibc: count subtests
kselftest: support KTAP format
kselftest: support skipping tests with testname
selftests/nolibc: proof of concept for TAP output
tools/testing/selftests/kselftest.h | 20 +++
tools/testing/selftests/nolibc/nolibc-test.c | 197 ++++++++++--------------
tools/testing/selftests/nolibc/run-all-tests.sh | 22 +++
3 files changed, 127 insertions(+), 112 deletions(-)
---
base-commit: dfef4fc45d5713eb23d87f0863aff9c33bd4bfaf
change-id: 20230718-nolibc-ktap-tmp-4408f505408d
Best regards,
--
Thomas Weißschuh <linux(a)weissschuh.net>
Hi Michał,
kernel test robot noticed the following build errors:
[auto build test ERROR on akpm-mm/mm-everything]
[also build test ERROR on linus/master v6.5-rc2 next-20230721]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Micha-Miros-aw/Re-fs-proc-ta…
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/a0b5c6776b2ed91f78a7575649f8b100e58bd3a9.16898810…
patch subject: Re: fs/proc/task_mmu: Implement IOCTL for efficient page table scanning
config: powerpc-randconfig-r015-20230720 (https://download.01.org/0day-ci/archive/20230721/202307211507.xOl45LiR-lkp@…)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
reproduce: (https://download.01.org/0day-ci/archive/20230721/202307211507.xOl45LiR-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307211507.xOl45LiR-lkp@intel.com/
All errors (new ones prefixed by >>):
>> fs/proc/task_mmu.c:1921:6: error: call to undeclared function 'userfaultfd_wp_async'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
1921 | if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
| ^
>> fs/proc/task_mmu.c:2200:12: error: call to undeclared function 'uffd_wp_range'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
2200 | int err = uffd_wp_range(vma, addr, end - addr, true);
| ^
2 errors generated.
vim +/userfaultfd_wp_async +1921 fs/proc/task_mmu.c
1913
1914 static int pagemap_scan_test_walk(unsigned long start, unsigned long end,
1915 struct mm_walk *walk)
1916 {
1917 struct pagemap_scan_private *p = walk->private;
1918 struct vm_area_struct *vma = walk->vma;
1919 unsigned long vma_category = 0;
1920
> 1921 if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
1922 vma_category |= PAGE_IS_WPASYNC;
1923 else if (p->arg.flags & PM_SCAN_CHECK_WPASYNC)
1924 return -EPERM;
1925
1926 if (vma->vm_flags & VM_PFNMAP)
1927 return 1;
1928
1929 if (!pagemap_scan_is_interesting_vma(vma_category, p))
1930 return 1;
1931
1932 p->cur_vma_category = vma_category;
1933 return 0;
1934 }
1935
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Michał,
kernel test robot noticed the following build errors:
[auto build test ERROR on akpm-mm/mm-everything]
[also build test ERROR on linus/master v6.5-rc2 next-20230720]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Micha-Miros-aw/Re-fs-proc-ta…
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/a0b5c6776b2ed91f78a7575649f8b100e58bd3a9.16898810…
patch subject: Re: fs/proc/task_mmu: Implement IOCTL for efficient page table scanning
config: i386-randconfig-r022-20230720 (https://download.01.org/0day-ci/archive/20230721/202307211337.5dwCMeHb-lkp@…)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230721/202307211337.5dwCMeHb-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307211337.5dwCMeHb-lkp@intel.com/
All errors (new ones prefixed by >>):
>> fs/proc/task_mmu.c:1921:6: error: call to undeclared function 'userfaultfd_wp_async'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
^
>> fs/proc/task_mmu.c:2200:12: error: call to undeclared function 'uffd_wp_range'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
int err = uffd_wp_range(vma, addr, end - addr, true);
^
2 errors generated.
vim +/userfaultfd_wp_async +1921 fs/proc/task_mmu.c
1913
1914 static int pagemap_scan_test_walk(unsigned long start, unsigned long end,
1915 struct mm_walk *walk)
1916 {
1917 struct pagemap_scan_private *p = walk->private;
1918 struct vm_area_struct *vma = walk->vma;
1919 unsigned long vma_category = 0;
1920
> 1921 if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
1922 vma_category |= PAGE_IS_WPASYNC;
1923 else if (p->arg.flags & PM_SCAN_CHECK_WPASYNC)
1924 return -EPERM;
1925
1926 if (vma->vm_flags & VM_PFNMAP)
1927 return 1;
1928
1929 if (!pagemap_scan_is_interesting_vma(vma_category, p))
1930 return 1;
1931
1932 p->cur_vma_category = vma_category;
1933 return 0;
1934 }
1935
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
In order to select a nexthop for multipath routes, fib_select_multipath()
is used with legacy nexthops and nexthop_select_path_hthr() is used with
nexthop objects. Those two functions perform a validity test on the
neighbor related to each nexthop but their logic is structured differently.
This causes a divergence in behavior and nexthop_select_path_hthr() may
return a nexthop that failed the neighbor validity test even if there was
one that passed.
Refactor nexthop_select_path_hthr() to make it more similar to
fib_select_multipath() and fix the problem mentioned above.
v2:
Removed unnecessary "first" variable in "nexthop: Do not return invalid
nexthop object during multipath selection".
v1:
https://lore.kernel.org/netdev/20230529201914.69828-1-bpoirier@nvidia.com/
---
Benjamin Poirier (4):
nexthop: Factor out hash threshold fdb nexthop selection
nexthop: Factor out neighbor validity check
nexthop: Do not return invalid nexthop object during multipath selection
selftests: net: Add test cases for nexthop groups with invalid neighbors
net/ipv4/nexthop.c | 61 +++++++++----
tools/testing/selftests/net/fib_nexthops.sh | 129 ++++++++++++++++++++++++++++
2 files changed, 171 insertions(+), 19 deletions(-)
---
base-commit: 36395b2efe905650cd179d67411ffee3b770268b
change-id: 20230719-nh_select-0303d55a1fb0
Best regards,
--
Benjamin Poirier <bpoirier(a)nvidia.com>
Hi Michał,
kernel test robot noticed the following build errors:
[auto build test ERROR on akpm-mm/mm-everything]
[also build test ERROR on linus/master v6.5-rc2 next-20230720]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Micha-Miros-aw/Re-fs-proc-ta…
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/a0b5c6776b2ed91f78a7575649f8b100e58bd3a9.16898810…
patch subject: Re: fs/proc/task_mmu: Implement IOCTL for efficient page table scanning
config: arc-randconfig-r035-20230720 (https://download.01.org/0day-ci/archive/20230721/202307211030.2CJH6TkM-lkp@…)
compiler: arceb-elf-gcc (GCC) 12.3.0
reproduce: (https://download.01.org/0day-ci/archive/20230721/202307211030.2CJH6TkM-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307211030.2CJH6TkM-lkp@intel.com/
All errors (new ones prefixed by >>):
fs/proc/task_mmu.c: In function 'pagemap_scan_test_walk':
fs/proc/task_mmu.c:1921:13: error: implicit declaration of function 'userfaultfd_wp_async'; did you mean 'userfaultfd_wp'? [-Werror=implicit-function-declaration]
1921 | if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
| ^~~~~~~~~~~~~~~~~~~~
| userfaultfd_wp
fs/proc/task_mmu.c: In function 'pagemap_scan_pte_hole':
>> fs/proc/task_mmu.c:2200:19: error: implicit declaration of function 'uffd_wp_range' [-Werror=implicit-function-declaration]
2200 | int err = uffd_wp_range(vma, addr, end - addr, true);
| ^~~~~~~~~~~~~
fs/proc/task_mmu.c: In function 'pagemap_scan_init_bounce_buffer':
fs/proc/task_mmu.c:2290:22: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
2290 | p->vec_out = (void __user *)p->arg.vec;
| ^
fs/proc/task_mmu.c: At top level:
fs/proc/task_mmu.c:1967:13: warning: 'pagemap_scan_backout_range' defined but not used [-Wunused-function]
1967 | static void pagemap_scan_backout_range(struct pagemap_scan_private *p,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/uffd_wp_range +2200 fs/proc/task_mmu.c
2182
2183 static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
2184 int depth, struct mm_walk *walk)
2185 {
2186 struct pagemap_scan_private *p = walk->private;
2187 struct vm_area_struct *vma = walk->vma;
2188 int ret;
2189
2190 if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p))
2191 return 0;
2192
2193 ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end);
2194 if (addr == end)
2195 return ret;
2196
2197 if (~p->arg.flags & PM_SCAN_WP_MATCHING)
2198 return ret;
2199
> 2200 int err = uffd_wp_range(vma, addr, end - addr, true);
2201 if (err < 0)
2202 ret = err;
2203
2204 return ret;
2205 }
2206
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Michał,
kernel test robot noticed the following build warnings:
[auto build test WARNING on akpm-mm/mm-everything]
[also build test WARNING on linus/master v6.5-rc2 next-20230720]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Micha-Miros-aw/Re-fs-proc-ta…
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/a0b5c6776b2ed91f78a7575649f8b100e58bd3a9.16898810…
patch subject: Re: fs/proc/task_mmu: Implement IOCTL for efficient page table scanning
config: mips-allyesconfig (https://download.01.org/0day-ci/archive/20230721/202307210528.2qgK1vwi-lkp@…)
compiler: mips-linux-gcc (GCC) 12.3.0
reproduce: (https://download.01.org/0day-ci/archive/20230721/202307210528.2qgK1vwi-lkp@…)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp(a)intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307210528.2qgK1vwi-lkp@intel.com/
All warnings (new ones prefixed by >>):
fs/proc/task_mmu.c: In function 'pagemap_scan_test_walk':
fs/proc/task_mmu.c:1921:13: error: implicit declaration of function 'userfaultfd_wp_async'; did you mean 'userfaultfd_wp'? [-Werror=implicit-function-declaration]
1921 | if (userfaultfd_wp_async(vma) && userfaultfd_wp_use_markers(vma))
| ^~~~~~~~~~~~~~~~~~~~
| userfaultfd_wp
fs/proc/task_mmu.c: In function 'pagemap_scan_init_bounce_buffer':
>> fs/proc/task_mmu.c:2290:22: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
2290 | p->vec_out = (void __user *)p->arg.vec;
| ^
fs/proc/task_mmu.c: At top level:
fs/proc/task_mmu.c:1967:13: warning: 'pagemap_scan_backout_range' defined but not used [-Wunused-function]
1967 | static void pagemap_scan_backout_range(struct pagemap_scan_private *p,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +2290 fs/proc/task_mmu.c
2264
2265 static int pagemap_scan_init_bounce_buffer(struct pagemap_scan_private *p)
2266 {
2267 if (!p->arg.vec_len) {
2268 /*
2269 * An arbitrary non-page-aligned sentinel value for
2270 * pagemap_scan_push_range().
2271 */
2272 p->cur_buf.start = p->cur_buf.end = ULLONG_MAX;
2273 return 0;
2274 }
2275
2276 /*
2277 * Allocate a smaller buffer to get output from inside the page
2278 * walk functions and walk the range in PAGEMAP_WALK_SIZE chunks.
2279 * The last range is always stored in p.cur_buf to allow coalescing
2280 * consecutive ranges that have the same categories returned across
2281 * walk_page_range() calls.
2282 */
2283 p->vec_buf_len = min_t(size_t, PAGEMAP_WALK_SIZE >> PAGE_SHIFT,
2284 p->arg.vec_len - 1);
2285 p->vec_buf = kmalloc_array(p->vec_buf_len, sizeof(*p->vec_buf),
2286 GFP_KERNEL);
2287 if (!p->vec_buf)
2288 return -ENOMEM;
2289
> 2290 p->vec_out = (void __user *)p->arg.vec;
2291
2292 return 0;
2293 }
2294
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
=== Context ===
In the context of a middlebox, fragmented packets are tricky to handle.
The full 5-tuple of a packet is often only available in the first
fragment which makes enforcing consistent policy difficult. There are
really only two stateless options, neither of which are very nice:
1. Enforce policy on first fragment and accept all subsequent fragments.
This works but may let in certain attacks or allow data exfiltration.
2. Enforce policy on first fragment and drop all subsequent fragments.
This does not really work b/c some protocols may rely on
fragmentation. For example, DNS may rely on oversized UDP packets for
large responses.
So stateful tracking is the only sane option. RFC 8900 [0] calls this
out as well in section 6.3:
Middleboxes [...] should process IP fragments in a manner that is
consistent with [RFC0791] and [RFC8200]. In many cases, middleboxes
must maintain state in order to achieve this goal.
=== BPF related bits ===
Policy has traditionally been enforced from XDP/TC hooks. Both hooks
run before kernel reassembly facilities. However, with the new
BPF_PROG_TYPE_NETFILTER, we can rather easily hook into existing
netfilter reassembly infra.
The basic idea is we bump a refcnt on the netfilter defrag module and
then run the bpf prog after the defrag module runs. This allows bpf
progs to transparently see full, reassembled packets. The nice thing
about this is that progs don't have to carry around logic to detect
fragments.
=== Changelog ===
Changes from v4:
* Refactor module handling code to not sleep in rcu_read_lock()
* Also unify the v4 and v6 hook structs so they can share codepaths
* Fixed some checkpatch.pl formatting warnings
Changes from v3:
* Correctly initialize `addrlen` stack var for recvmsg()
Changes from v2:
* module_put() if ->enable() fails
* Fix CI build errors
Changes from v1:
* Drop bpf_program__attach_netfilter() patches
* static -> static const where appropriate
* Fix callback assignment order during registration
* Only request_module() if callbacks are missing
* Fix retval when modprobe fails in userspace
* Fix v6 defrag module name (nf_defrag_ipv6_hooks -> nf_defrag_ipv6)
* Simplify priority checking code
* Add warning if module doesn't assign callbacks in the future
* Take refcnt on module while defrag link is active
[0]: https://datatracker.ietf.org/doc/html/rfc8900
Daniel Xu (5):
netfilter: defrag: Add glue hooks for enabling/disabling defrag
netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link
bpf: selftests: Support not connecting client socket
bpf: selftests: Support custom type and proto for client sockets
bpf: selftests: Add defrag selftests
include/linux/netfilter.h | 10 +
include/uapi/linux/bpf.h | 5 +
net/ipv4/netfilter/nf_defrag_ipv4.c | 17 +-
net/ipv6/netfilter/nf_defrag_ipv6_hooks.c | 11 +
net/netfilter/core.c | 6 +
net/netfilter/nf_bpf_link.c | 116 ++++++-
tools/include/uapi/linux/bpf.h | 5 +
tools/testing/selftests/bpf/Makefile | 4 +-
.../selftests/bpf/generate_udp_fragments.py | 90 ++++++
.../selftests/bpf/ip_check_defrag_frags.h | 57 ++++
tools/testing/selftests/bpf/network_helpers.c | 26 +-
tools/testing/selftests/bpf/network_helpers.h | 3 +
.../bpf/prog_tests/ip_check_defrag.c | 283 ++++++++++++++++++
.../selftests/bpf/progs/ip_check_defrag.c | 104 +++++++
14 files changed, 715 insertions(+), 22 deletions(-)
create mode 100755 tools/testing/selftests/bpf/generate_udp_fragments.py
create mode 100644 tools/testing/selftests/bpf/ip_check_defrag_frags.h
create mode 100644 tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c
create mode 100644 tools/testing/selftests/bpf/progs/ip_check_defrag.c
--
2.41.0