We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org --- include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) { - return page_count(virt_to_page(pte)) > 1; + return ptdesc_pmd_is_shared(virt_to_ptdesc(pte)); } #else static inline bool hugetlb_pmd_shared(pte_t *pte)
On Fri, 2025-12-05 at 22:35 +0100, David Hildenbrand (Red Hat) wrote:
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Reviewed-by: Rik van Riel riel@surriel.com
From: Lance Yang lance.yang@linux.dev
On Fri, 5 Dec 2025 22:35:55 +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Good catch! Feel free to add:
Reviewed-by: Lance yang lance.yang@linux.dev
include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) {
- return page_count(virt_to_page(pte)) > 1;
- return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
} #else static inline bool hugetlb_pmd_shared(pte_t *pte)
On 2025/12/6 13:55, Lance Yang wrote:
From: Lance Yang lance.yang@linux.dev
On Fri, 5 Dec 2025 22:35:55 +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Good catch! Feel free to add:
Reviewed-by: Lance yang lance.yang@linux.dev
Actually:
Reviewed-by: Lance Yang lance.yang@linux.dev
From: Lance Yang lance.yang@linux.dev
On Fri, 5 Dec 2025 22:35:55 +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Tested on x86 with two independent processes sharing a 1GiB hugetlbfs file (aligned a 1GiB boundary).
Before the fix, even though PMD sharing worked (pt_share_count=1), hugetlb_pmd_shared() returned false because page_count() was still 1, causing smaps to report it as "Private" and pagemap to set it PM_MMAP_EXCLUSIVE incorrectly :(
After the fix, hugetlb_pmd_shared() correctly detects the sharing, smaps reports it as "Shared", and PM_MMAP_EXCLUSIVE is cleared ;)
Tested-by: Lance Yang lance.yang@linux.dev
Cheers!
include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) {
- return page_count(virt_to_page(pte)) > 1;
- return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
} #else static inline bool hugetlb_pmd_shared(pte_t *pte)
On Fri, Dec 05, 2025 at 10:35:55PM +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Oops, didn't notice there's still missing conversion!
Looks good to me, Reviewed-by: Harry Yoo harry.yoo@oracle.com
On 12/8/25 03:32, Lance Yang wrote:
From: Lance Yang lance.yang@linux.dev
On Fri, 5 Dec 2025 22:35:55 +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Tested on x86 with two independent processes sharing a 1GiB hugetlbfs file (aligned a 1GiB boundary).
Before the fix, even though PMD sharing worked (pt_share_count=1), hugetlb_pmd_shared() returned false because page_count() was still 1, causing smaps to report it as "Private" and pagemap to set it PM_MMAP_EXCLUSIVE incorrectly :(
After the fix, hugetlb_pmd_shared() correctly detects the sharing, smaps reports it as "Shared", and PM_MMAP_EXCLUSIVE is cleared ;)
Tested-by: Lance Yang lance.yang@linux.dev
Thanks a lot Lance for the testing and thanks to everybody for the review!
On Mon, Dec 08, 2025 at 10:32:31AM +0800, Lance Yang wrote:
From: Lance Yang lance.yang@linux.dev
On Fri, 5 Dec 2025 22:35:55 +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Tested on x86 with two independent processes sharing a 1GiB hugetlbfs file (aligned a 1GiB boundary).
Before the fix, even though PMD sharing worked (pt_share_count=1), hugetlb_pmd_shared() returned false because page_count() was still 1, causing smaps to report it as "Private" and pagemap to set it PM_MMAP_EXCLUSIVE incorrectly :(
After the fix, hugetlb_pmd_shared() correctly detects the sharing, smaps reports it as "Shared", and PM_MMAP_EXCLUSIVE is cleared ;)
Yikes yikes yikes...
I wonder what else might be broken in this stuff :/
Tested-by: Lance Yang lance.yang@linux.dev
Cheers!
include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) {
- return page_count(virt_to_page(pte)) > 1;
- return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
} #else static inline bool hugetlb_pmd_shared(pte_t *pte)
On Fri, Dec 05, 2025 at 10:35:55PM +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Yikes this seems pretty serious!!
How did we not pick up on this before...
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Esp. given Lance's testing... LGTM so:
Reviewed-by: Lorenzo Stoakes lorenzo.stoakes@oracle.com
include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) {
- return page_count(virt_to_page(pte)) > 1;
- return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
} #else static inline bool hugetlb_pmd_shared(pte_t *pte) -- 2.52.0
On Fri, Dec 05, 2025 at 10:35:55PM +0100, David Hildenbrand (Red Hat) wrote:
We switched from (wrongly) using the page count to an independent shared count. Now, shared page tables have a refcount of 1 (excluding speculative references) and instead use ptdesc->pt_share_count to identify sharing.
We didn't convert hugetlb_pmd_shared(), so right now, we would never detect a shared PMD table as such, because sharing/unsharing no longer touches the refcount of a PMD table.
Page migration, like mbind() or migrate_pages() would allow for migrating folios mapped into such shared PMD tables, even though the folios are not exclusive. In smaps we would account them as "private" although they are "shared", and we would be wrongly setting the PM_MMAP_EXCLUSIVE in the pagemap interface.
Fix it by properly using ptdesc_pmd_is_shared() in hugetlb_pmd_shared().
Fixes: 59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count") Cc: stable@vger.kernel.org Cc: Liu Shixin liushixin2@huawei.com Signed-off-by: David Hildenbrand (Red Hat) david@kernel.org
Good catch David,
Acked-by: Oscar Salvador osalvador@suse.de
include/linux/hugetlb.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 019a1c5281e4e..03c8725efa289 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1326,7 +1326,7 @@ static inline __init void hugetlb_cma_reserve(int order) #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING static inline bool hugetlb_pmd_shared(pte_t *pte) {
- return page_count(virt_to_page(pte)) > 1;
- return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
} #else static inline bool hugetlb_pmd_shared(pte_t *pte) -- 2.52.0
linux-stable-mirror@lists.linaro.org
-
David Hildenbrand (Red Hat) -
Harry Yoo -
Lance Yang -
Lance Yang -
Lorenzo Stoakes -
Oscar Salvador -
Rik van Riel