From: Zi Yan <ziy(a)nvidia.com>
Hi all,
With Matthew's THP in pagecache patches[1], we will be able to handle any size
pagecache THPs, but currently split_huge_page can only split a THP to order-0
pages. This can easily erase the benefit of having pagecache THPs, when
operations like truncate might want to keep pages larger than order-0. In
response, here is the patches to add support for splitting a THP to any lower
order pages. In addition, this patchset prepares for my PUD THP patchset[2],
since splitting a PUD THP to multiple PMD THPs can be handled by
split_huge_page_to_list_to_order function added by this patchset, which reduces
a lot of redundant code without just replicating split_huge_page for PUD THP.
To help the tests of splitting huge pages, I added a new debugfs interface
at <debugfs>/split_huge_pages_in_range_pid, so developers can split THPs in a
given range from a process with the given pid by writing
"<pid>,<vaddr_start>,<vaddr_end>,<to_order>" to the interface. I also added a
new test program to test 1) splitting PMD THPs, 2) splitting PTE-mapped THPs,
3) splitting pagecache THPs to any lower order, 4) truncating a pagecache
THP to a page with a lower order, and 5) punching holes in a pagecache THP to
cause splitting THPs to lower order THPs.
The patchset is on top of Matthew's pagecache/next tree[3].
* Patch 1 is cherry-picked from Matthew's recent xarray fix [4] just to make sure
Patch 3 to 7 can run without problem. I let Matthew decide how it should get
picked up.
* Patch 2 is self-contained and can be merged if it looks OK.
Comments and/or suggestions are welcome.
ChangeLog
===
>From RFC:
1. Fixed debugfs to handle splitting PTE-mapped THPs properly and added stats
for split THPs.
2. Added a new test case for splitting PTE-mapped THPs. Each of the four PTEs
points to a different subpage from four THPs and used kpageflags to check
whether a PTE points to a THP or not (AnonHugePages from smap does not show
PTE-mapped THPs).
3. mem_cgroup_split_huge_fixup() takes order instead of nr.
4. split_page_owner takes old_order and new_order instead of nr and new_order.
5. Corrected __split_page_owner declaration and fixed its implementation when
splitting a THP to a new order.
6. Renamed left to remaining in truncate_inode_partial_page().
7. Use VM_BUG_ON instead of WARN_ONCE when splitting a THP to the unsupported
order-0 and splitting anonymous THPs to non-zero orders.
8. Added punching holes in a file as a new pagecache THP split test case, which
uncovered an xarray bug.
[1] https://lore.kernel.org/linux-mm/20201029193405.29125-1-willy@infradead.org/
[2] https://lore.kernel.org/linux-mm/20200928175428.4110504-1-zi.yan@sent.com/
[3] https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/next
[4] https://git.infradead.org/users/willy/xarray.git
Matthew Wilcox (Oracle) (1):
XArray: Fix splitting to non-zero orders
Zi Yan (6):
mm: huge_memory: add new debugfs interface to trigger split huge page
on any page range.
mm: memcg: make memcg huge page split support any order split.
mm: page_owner: add support for splitting to any order in split
page_owner.
mm: thp: split huge page to any lower order pages.
mm: truncate: split thp to a non-zero order if possible.
mm: huge_memory: enable debugfs to split huge pages to any order.
include/linux/huge_mm.h | 8 +
include/linux/memcontrol.h | 5 +-
include/linux/page_owner.h | 10 +-
lib/test_xarray.c | 26 +-
lib/xarray.c | 4 +-
mm/huge_memory.c | 219 ++++++--
mm/internal.h | 1 +
mm/memcontrol.c | 6 +-
mm/migrate.c | 2 +-
mm/page_alloc.c | 2 +-
mm/page_owner.c | 13 +-
mm/swap.c | 1 -
mm/truncate.c | 29 +-
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 1 +
.../selftests/vm/split_huge_page_test.c | 479 ++++++++++++++++++
16 files changed, 742 insertions(+), 65 deletions(-)
create mode 100644 tools/testing/selftests/vm/split_huge_page_test.c
--
2.28.0
On Tue, 17 Nov 2020 at 19:02, Greg Kroah-Hartman
<gregkh(a)linuxfoundation.org> wrote:
>
> This is the start of the stable review cycle for the 5.9.9 release.
> There are 255 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Thu, 19 Nov 2020 12:20:51 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.9.9-rc1.…
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.9.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
Results from Linaro’s test farm.
No regressions on arm64, arm, x86_64, and i386.
Tested-by: Linux Kernel Functional Testing <lkft(a)linaro.org>
NOTE:
1)
BUG: Invalid wait context on arm64 db410c device while booting.
This issue has not reproduced after several testing loops.
https://lore.kernel.org/stable/CA+G9fYsk54r9Re4E9BWpqsoxLjpCvxRKFWRgdiKVcPo…
2)
kselftest test suite version upgrade to v5.9
3)
While running kselftest netfilter on x86, i386, arm64 and arm devices
the following kernel warning was noticed.
WARNING: at net/netfilter/nf_tables_api.c:622
lockdep_nfnl_nft_mutex_not_held+0x19/0x20 [nf_tables]
https://lore.kernel.org/linux-kselftest/CA+G9fYvFUpODs+NkSYcnwKnXm62tmP=ksL…
4)
>From this release we have started building kernels with clang-10 toolchain
and testing LTP testsuite on qemu_arm64, qemu_arm, qemu_x86_64 and qemu_i386.
Summary
------------------------------------------------------------------------
kernel: 5.9.9-rc1
git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
git branch: linux-5.9.y
git commit: fb1622495321923cbb1ae2c6cf2da1e9ca286800
git describe: v5.9.8-256-gfb1622495321
Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-5.9.y/build/v5.9.8…
No regressions (compared to build v5.9.8)
No fixes (compared to build v5.9.8)
Ran 52946 total tests in the following environments and test suites.
Environments
--------------
- dragonboard-410c
- hi6220-hikey
- i386
- juno-r2
- juno-r2-compat
- juno-r2-kasan
- nxp-ls2088
- qemu-arm-clang
- qemu-arm64-clang
- qemu-arm64-kasan
- qemu-i386-clang
- qemu-x86_64-clang
- qemu-x86_64-kasan
- qemu_arm
- qemu_arm64
- qemu_arm64-compat
- qemu_i386
- qemu_x86_64
- qemu_x86_64-compat
- x15
- x86
- x86-kasan
Test Suites
-----------
* build
* install-android-platform-tools-r2600
* libhugetlbfs
* linux-log-parser
* ltp-cap_bounds-tests
* ltp-commands-tests
* ltp-containers-tests
* ltp-cpuhotplug-tests
* ltp-crypto-tests
* ltp-dio-tests
* ltp-fcntl-locktests-tests
* ltp-filecaps-tests
* ltp-fs-tests
* ltp-fs_bind-tests
* ltp-fs_perms_simple-tests
* ltp-fsx-tests
* ltp-hugetlb-tests
* ltp-io-tests
* ltp-ipc-tests
* ltp-math-tests
* ltp-mm-tests
* ltp-nptl-tests
* ltp-pty-tests
* ltp-sched-tests
* ltp-securebits-tests
* ltp-syscalls-tests
* ltp-tracing-tests
* perf
* v4l2-compliance
* ltp-controllers-tests
* ltp-cve-tests
* network-basic-tests
* kselftest
* ltp-open-posix-tests
* kvm-unit-tests
* kunit
* kselftest-vsyscall-mode-native
* kselftest-vsyscall-mode-none
--
Linaro LKFT
https://lkft.linaro.org
On Wed, 18 Nov 2020 16:43:33 -0500 Willem de Bruijn wrote:
> On Tue, Nov 17, 2020 at 12:19 PM Jakub Kicinski <kuba(a)kernel.org> wrote:
> > Sorry I misremembered it's 4. We can leave that as is.
>
> Instead of having to remember, maybe we should have a file in
> tools/testing/selftest to define constants?
>
> I defined them one-off in tools/testing/selftests/net/udpgso_bench.sh
>
> readonly KSFT_PASS=0
> readonly KSFT_FAIL=1
> readonly KSFT_SKIP=4
>
> along with some other kselftest shell support infra. But having each
> test figure this out independently is duplicative and error prone.
Sounds like a good idea, I was surprised it wasn't already defined in
any lib.
CCing the selftest ML.
Hi Linus,
Please pull the following Kunit fixes update for Linux 5.10-rc5.
This Kunit update for Linux 5.10-rc5 consists of several fixes Kunit
documentation, tool, compile time fixes not pollute source directory,
and fix to remove tools/testing/kunit/.gitattributes file.
diff is attached.
Brendan fixed the weirdness with tools/testing/kunit/.gitattributes
file. Thanks for noticing it.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 0d0d245104a42e593adcf11396017a6420c08ba8:
kunit: tools: fix kunit_tool tests for parsing test plans (2020-10-26
13:25:40 -0600)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-kunit-fixes-5.10-rc5
for you to fetch changes up to 3084db0e0d5076cd48408274ab0911cd3ccdae88:
kunit: fix display of failed expectations for strings (2020-11-10
13:45:15 -0700)
----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.10-rc5
This Kunit update for Linux 5.10-rc5 consists of several fixes Kunit
documentation, tool, compile time fixes not pollute source directory,
and fix to remove tools/testing/kunit/.gitattributes file.
----------------------------------------------------------------
Andy Shevchenko (2):
kunit: Do not pollute source directory with generated files
(.kunitconfig)
kunit: Do not pollute source directory with generated files
(test.log)
Brendan Higgins (1):
kunit: tool: unmark test_data as binary blobs
Daniel Latypov (4):
kunit: tool: fix pre-existing python type annotation errors
kunit: tool: print out stderr from make (like build warnings)
kunit: tool: fix extra trailing \n in raw + parsed test output
kunit: fix display of failed expectations for strings
David Gow (1):
kunit: Fix kunit.py parse subcommand (use null build_dir)
Randy Dunlap (3):
KUnit: Docs: fix a wording typo
KUnit: Docs: style: fix some Kconfig example issues
KUnit: Docs: usage: wording fixes
Documentation/dev-tools/kunit/faq.rst | 2 +-
Documentation/dev-tools/kunit/style.rst | 18 +++++------
Documentation/dev-tools/kunit/usage.rst | 10 +++----
include/kunit/test.h | 2 +-
tools/testing/kunit/.gitattributes | 1 -
tools/testing/kunit/kunit.py | 27 ++++++++---------
tools/testing/kunit/kunit_kernel.py | 53
+++++++++++++++++++++++++--------
tools/testing/kunit/kunit_parser.py | 17 ++++++-----
tools/testing/kunit/kunit_tool_test.py | 4 +--
9 files changed, 80 insertions(+), 54 deletions(-)
delete mode 100644 tools/testing/kunit/.gitattributes
----------------------------------------------------------------