When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com --- tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); - return timeout == -1; }
@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size);
+ madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com --- tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh index 33fc7fafa8f9..a38c984103ce 100755 --- a/tools/testing/selftests/mm/run_vmtests.sh +++ b/tools/testing/selftests/mm/run_vmtests.sh @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
CATEGORY="thp" run_test ./khugepaged -s 2
+CATEGORY="thp" run_test ./khugepaged all:shmem + +CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem + CATEGORY="thp" run_test ./transhuge-stress -d 20
# Try to create XFS if not provided
collpase in the subject line should be collapse -- Tim
-----Original Message----- From: Baolin Wang baolin.wang@linux.alibaba.com Subject: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases. Signed-off-by: Baolin Wang <baolin. wang@ linux. alibaba. com> --- tools/testing/selftests/mm/run_vmtests. sh
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh index 33fc7fafa8f9..a38c984103ce 100755 --- a/tools/testing/selftests/mm/run_vmtests.sh +++ b/tools/testing/selftests/mm/run_vmtests.sh @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
CATEGORY="thp" run_test ./khugepaged -s 2
+CATEGORY="thp" run_test ./khugepaged all:shmem
+CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
CATEGORY="thp" run_test ./transhuge-stress -d 20
# Try to create XFS if not provided
2.43.5
On 2025/6/12 12:20, Bird, Tim wrote:
collpase in the subject line should be collapse
Good catch. Will fix the typo. Thanks.
-----Original Message----- From: Baolin Wang baolin.wang@linux.alibaba.com Subject: [PATCH 2/2] selftests: mm: add shmem collpase as a default test item
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases. Signed-off-by: Baolin Wang <baolin. wang@ linux. alibaba. com> --- tools/testing/selftests/mm/run_vmtests. sh
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh index 33fc7fafa8f9..a38c984103ce 100755 --- a/tools/testing/selftests/mm/run_vmtests.sh +++ b/tools/testing/selftests/mm/run_vmtests.sh @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged
CATEGORY="thp" run_test ./khugepaged -s 2
+CATEGORY="thp" run_test ./khugepaged all:shmem
+CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
CATEGORY="thp" run_test ./transhuge-stress -d 20
# Try to create XFS if not provided
-- 2.43.5
On 12/06/25 9:24 am, Baolin Wang wrote:
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh index 33fc7fafa8f9..a38c984103ce 100755 --- a/tools/testing/selftests/mm/run_vmtests.sh +++ b/tools/testing/selftests/mm/run_vmtests.sh @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged CATEGORY="thp" run_test ./khugepaged -s 2 +CATEGORY="thp" run_test ./khugepaged all:shmem
+CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
- CATEGORY="thp" run_test ./transhuge-stress -d 20
# Try to create XFS if not provided
Reviewed-by: Dev Jain dev.jain@arm.com Tested-by: Dev Jain dev.jain@arm.com
On 12.06.25 05:54, Baolin Wang wrote:
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh index 33fc7fafa8f9..a38c984103ce 100755 --- a/tools/testing/selftests/mm/run_vmtests.sh +++ b/tools/testing/selftests/mm/run_vmtests.sh @@ -485,6 +485,10 @@ CATEGORY="thp" run_test ./khugepaged CATEGORY="thp" run_test ./khugepaged -s 2 +CATEGORY="thp" run_test ./khugepaged all:shmem
+CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
Ahh, there we have it already, nice :)
Acked-by: David Hildenbrand david@redhat.com
On 11 Jun 2025, at 23:54, Baolin Wang wrote:
Currently, we only test anonymous memory collapse by default. We should also add shmem collapse as a default test item to catch issues that could break the test cases.
Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/run_vmtests.sh | 4 ++++ 1 file changed, 4 insertions(+)
Acked-by: Zi Yan ziy@nvidia.com
Best Regards, Yan, Zi
On 12/06/25 9:24 am, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
- return timeout == -1; }
@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size);
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
The idea looks sane to me, but do we need to add the madvise call to madvise_retracted_page_tables() too, since that also calls wait_for_scan()?
On 2025/6/12 13:10, Dev Jain wrote:
On 12/06/25 9:24 am, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); } - madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
return timeout == -1; } @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size); + madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
The idea looks sane to me, but do we need to add the madvise call to madvise_retracted_page_tables() too, since that also calls wait_for_scan()?
Yes, I also realized this after sending the patches:) Thanks.
To keep the original logic:
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index d462f62d8116..3452763e2fe3 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -1074,6 +1074,7 @@ static void madvise_retracted_page_tables(struct collapse_context *c, return; } success("OK"); + madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); c->collapse("Install huge PMD from page cache", p, nr_hpages, ops, true); validate_memory(p, 0, size);
On 12.06.25 05:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem),
Hmm, this combination is not run automatically through run_tests.sh, right? IIUC, it only runs "./khugepaged" which tests anon only ...
Should we add it there? Then I would probably have noticed that myself earlier :)
I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
Right. It's always problematic when we have some contradicting information in the VMA vs. pagecache.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
- return timeout == -1; }
@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size);
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
It's a shame we have this weird interface: there is no way we can clear VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll almost immediately later call cleanup_area() where we munmap(), right?
On 2025/6/12 18:08, David Hildenbrand wrote:
On 12.06.25 05:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem),
Hmm, this combination is not run automatically through run_tests.sh, right? IIUC, it only runs "./khugepaged" which tests anon only ...
Should we add it there? Then I would probably have noticed that myself earlier :)
Yes, see patch 2.
I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
Right. It's always problematic when we have some contradicting information in the VMA vs. pagecache.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); } - madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
return timeout == -1; } @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size); + madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
It's a shame we have this weird interface: there is no way we can clear VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
Right.
But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll almost immediately later call cleanup_area() where we munmap(), right?
I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test cases all passed.
However, a potential impact of removing MADV_NOHUGEPAGE is that, khugepaged might report 'timeout', but check_huge() would still report 'success' (assuming khugepaged tries to scan the VMA and successfully collapses it after the timeout). Such test result could be confusing.
I know this kind of case is very rare and may not even be detectable in tests. At least, I couldn't reproduce it. But I prefer to keep the original test logic and fix the issue without making further changes.
On 12.06.25 13:37, Baolin Wang wrote:
On 2025/6/12 18:08, David Hildenbrand wrote:
On 12.06.25 05:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem),
Hmm, this combination is not run automatically through run_tests.sh, right? IIUC, it only runs "./khugepaged" which tests anon only ...
Should we add it there? Then I would probably have noticed that myself earlier :)
Yes, see patch 2.
Yes, was pleasantly surprised when I found that :)
I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
Right. It's always problematic when we have some contradicting information in the VMA vs. pagecache.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); } - madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
return timeout == -1; } @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size); + madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
It's a shame we have this weird interface: there is no way we can clear VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
Right.
But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll almost immediately later call cleanup_area() where we munmap(), right?
I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test cases all passed.
However, a potential impact of removing MADV_NOHUGEPAGE is that, khugepaged might report 'timeout', but check_huge() would still report 'success' (assuming khugepaged tries to scan the VMA and successfully collapses it after the timeout). Such test result could be confusing.
If we run into the timeout, we return "true" from wait_for_scan(), and in khugepaged_collapse() returns immediately.
So we wouldn't issue another check_huge() call in khugepaged_collapse().
Did I miss something?
On 2025/6/12 19:45, David Hildenbrand wrote:
On 12.06.25 13:37, Baolin Wang wrote:
On 2025/6/12 18:08, David Hildenbrand wrote:
On 12.06.25 05:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem),
Hmm, this combination is not run automatically through run_tests.sh, right? IIUC, it only runs "./khugepaged" which tests anon only ...
Should we add it there? Then I would probably have noticed that myself earlier :)
Yes, see patch 2.
Yes, was pleasantly surprised when I found that :)
I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
Right. It's always problematic when we have some contradicting information in the VMA vs. pagecache.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); } - madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
return timeout == -1; } @@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size); + madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else
It's a shame we have this weird interface: there is no way we can clear VM_HUGEPAGE without setting VM_NOHUGEPAGE :(
Right.
But, do we even care about setting MADV_NOHUGEPAGE at all? IIUC, we'll almost immediately later call cleanup_area() where we munmap(), right?
I tested removing the MADV_NOHUGEPAGE setting, and the khugepaged test cases all passed.
However, a potential impact of removing MADV_NOHUGEPAGE is that, khugepaged might report 'timeout', but check_huge() would still report 'success' (assuming khugepaged tries to scan the VMA and successfully collapses it after the timeout). Such test result could be confusing.
If we run into the timeout, we return "true" from wait_for_scan(), and in khugepaged_collapse() returns immediately.
So we wouldn't issue another check_huge() call in khugepaged_collapse().
Did I miss something?
Ah, right. Sorry for the wrong example. Now I'm fine to drop the MADV_NOHUGEPAGE settiing. Thanks.
On 11 Jun 2025, at 23:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
Can you add the title of the commit? It is easier to understand the context.
2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
- return timeout == -1;
}
I assume you are going to just remove this madvise based on your discussion with David. With that, feel free to add Reviewed-by: Zi Yan ziy@nvidia.com
Thanks.
Best Regards, Yan, Zi
On 12.06.25 17:46, Zi Yan wrote:
On 11 Jun 2025, at 23:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
Can you add the title of the commit? It is easier to understand the context.
2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
Probably checkpatch.pl would point out the same :)
On 2025/6/12 23:46, Zi Yan wrote:
On 11 Jun 2025, at 23:54, Baolin Wang wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
Can you add the title of the commit? It is easier to understand the context.
2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma")
Sure.
set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);
- return timeout == -1; }
I assume you are going to just remove this madvise based on your discussion with David. With that, feel free to add Reviewed-by: Zi Yan ziy@nvidia.com
Thanks.
This patch has been successfully tested. All the khugepaged shmem tests passed.
# ./khugepaged khugepaged:shmem Save THP and khugepaged settings... OK Allocate huge page on fault... OK Split huge PMD on MADV_DONTNEED... OK
Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... OK
Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... OK
Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... OK Restore THP and khugepaged settings... OK
Tested-by: Mario Casquero mcasquer@redhat.com
On Thu, Jun 12, 2025 at 5:55 AM Baolin Wang baolin.wang@linux.alibaba.com wrote:
When running the khugepaged selftest for shmem (./khugepaged all:shmem), I encountered the following test failures: " Run test: collapse_full (khugepaged:shmem) Collapse multiple fully populated PTE table.... Fail ... Run test: collapse_single_pte_entry (khugepaged:shmem) Collapse PTE table with single PTE entry present.... Fail ... Run test: collapse_full_of_compound (khugepaged:shmem) Allocate huge page... OK Split huge page leaving single PTE page table full of compound pages... OK Collapse PTE table full of compound pages.... Fail "
The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent khugepaged from continuing to scan shmem VMA after khugepaged finishes scanning in the wait_for_scan() function. Moreover, shmem requires a refault to establish PMD mappings.
However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during refault.
To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault. With this fix, the shmem test case passes.
Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the hw/process/vma") Signed-off-by: Baolin Wang baolin.wang@linux.alibaba.com
tools/testing/selftests/mm/khugepaged.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c index 8a4d34cce36b..d462f62d8116 100644 --- a/tools/testing/selftests/mm/khugepaged.c +++ b/tools/testing/selftests/mm/khugepaged.c @@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int nr_hpages, usleep(TICK); }
madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);return timeout == -1;}
@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages, if (ops != &__anon_ops) ops->fault(p, 0, nr_hpages * hpage_pmd_size);
madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE); if (ops->check_huge(p, expect ? nr_hpages : 0)) success("OK"); else-- 2.43.5
linux-kselftest-mirror@lists.linaro.org
-
Baolin Wang -
Bird, Tim -
David Hildenbrand -
Dev Jain -
Mario Casquero -
Zi Yan