[PATCH v3 0/7] Split a folio to any lower order folios

List overview All Threads
Download

newer

older

[PATCH v1 0/2] KVM: s390: Fix AR...

[PATCH v17 6/6]...

Zi Yan

3 Apr 2023 3 Apr '23

8:18 p.m.

From: Zi Yan ziy@nvidia.com

Hi all,

File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

The patchset is on top of mm-everything-2023-03-27-21-20.

Changelog === Since v2 --- 1. Fixed an issue in __split_page_owner() introduced during my rebase

Since v1 --- 1. Changed split_page_memcg() and split_page_owner() parameter to use order 2. Used folio_test_pmd_mappable() in place of the equivalent code

Details ===

* Patch 1 changes split_page_memcg() to use order instead of nr_pages * Patch 2 changes split_page_owner() to use order instead of nr_pages * Patch 3 and 4 add new_order parameter split_page_memcg() and split_page_owner() and prepare for upcoming changes. * Patch 5 adds split_huge_page_to_list_to_order() to split a huge page to any lower order. The original split_huge_page_to_list() calls split_huge_page_to_list_to_order() with new_order = 0. * Patch 6 uses split_huge_page_to_list_to_order() in large pagecache folio truncation instead of split the large folio all the way down to order-0. * Patch 7 adds a test API to debugfs and test cases in split_huge_page_test selftests.

Comments and/or suggestions are welcome.

[1] https://lore.kernel.org/linux-mm/Y%2FblF0GIunm+pRIC@casper.infradead.org/

Zi Yan (7): mm/memcg: use order instead of nr in split_page_memcg() mm/page_owner: use order instead of nr in split_page_owner() mm: memcg: make memcg huge page split support any order split. mm: page_owner: add support for splitting to any order in split page_owner. mm: thp: split huge page to any lower order pages. mm: truncate: split huge page cache page to a non-zero order if possible. mm: huge_memory: enable debugfs to split huge pages to any order.

-- 2.39.2

Show replies by date

Zi Yan

3 Apr 3 Apr

8:18 p.m.

New subject: [PATCH v3 1/7] mm/memcg: use order instead of nr in split_page_memcg()

From: Zi Yan ziy@nvidia.com

We do not have non power of two pages, using nr is error prone if nr is not power-of-two. Use page order instead.

Signed-off-by: Zi Yan ziy@nvidia.com --- include/linux/memcontrol.h | 4 ++-- mm/huge_memory.c | 3 ++- mm/memcontrol.c | 3 ++- mm/page_alloc.c | 4 ++-- 4 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index aa69ea98e2d8..e06a61ea4fc1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1151,7 +1151,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, rcu_read_unlock(); }

-void split_page_memcg(struct page *head, unsigned int nr); +void split_page_memcg(struct page *head, int order);

unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, gfp_t gfp_mask, @@ -1588,7 +1588,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx) { }

-static inline void split_page_memcg(struct page *head, unsigned int nr) +static inline void split_page_memcg(struct page *head, int order) { }

diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 81a5689806af..3bb003eb80a3 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2512,10 +2512,11 @@ static void __split_huge_page(struct page *page, struct list_head *list, struct address_space *swap_cache = NULL; unsigned long offset = 0; unsigned int nr = thp_nr_pages(head); + int order = folio_order(folio); int i;

/* complete memcg works before add pages to LRU */ - split_page_memcg(head, nr); + split_page_memcg(head, order);

if (PageAnon(head) && PageSwapCache(head)) { swp_entry_t entry = { .val = page_private(head) }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 681e7528a714..cab2828e188d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3414,11 +3414,12 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size) /* * Because page_memcg(head) is not set on tails, set it now. */ -void split_page_memcg(struct page *head, unsigned int nr) +void split_page_memcg(struct page *head, int order) { struct folio *folio = page_folio(head); struct mem_cgroup *memcg = folio_memcg(folio); int i; + unsigned int nr = 1 << order;

if (mem_cgroup_disabled() || !memcg) return; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0767dd6bc5ba..d84b121d1e03 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2781,7 +2781,7 @@ void split_page(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); split_page_owner(page, 1 << order); - split_page_memcg(page, 1 << order); + split_page_memcg(page, order); } EXPORT_SYMBOL_GPL(split_page);

@@ -4997,7 +4997,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *last = page + nr;

split_page_owner(page, 1 << order); - split_page_memcg(page, 1 << order); + split_page_memcg(page, order); while (page < --last) set_page_refcounted(last);

-- 2.39.2

Zi Yan

8:18 p.m.

New subject: [PATCH v3 2/7] mm/page_owner: use order instead of nr in split_page_owner()

From: Zi Yan ziy@nvidia.com

We do not have non power of two pages, using nr is error prone if nr is not power-of-two. Use page order instead.

Signed-off-by: Zi Yan ziy@nvidia.com --- include/linux/page_owner.h | 8 ++++---- mm/huge_memory.c | 2 +- mm/page_alloc.c | 4 ++-- mm/page_owner.c | 3 ++- 4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h index 119a0c9d2a8b..d7878523adfc 100644 --- a/include/linux/page_owner.h +++ b/include/linux/page_owner.h @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops; extern void __reset_page_owner(struct page *page, unsigned short order); extern void __set_page_owner(struct page *page, unsigned short order, gfp_t gfp_mask); -extern void __split_page_owner(struct page *page, unsigned int nr); +extern void __split_page_owner(struct page *page, int order); extern void __folio_copy_owner(struct folio *newfolio, struct folio *old); extern void __set_page_owner_migrate_reason(struct page *page, int reason); extern void __dump_page_owner(const struct page *page); @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page, __set_page_owner(page, order, gfp_mask); }

-static inline void split_page_owner(struct page *page, unsigned int nr) +static inline void split_page_owner(struct page *page, int order) { if (static_branch_unlikely(&page_owner_inited)) - __split_page_owner(page, nr); + __split_page_owner(page, order); } static inline void folio_copy_owner(struct folio *newfolio, struct folio *old) { @@ -60,7 +60,7 @@ static inline void set_page_owner(struct page *page, { } static inline void split_page_owner(struct page *page, - unsigned short order) + int order) { } static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3bb003eb80a3..a21921c90b21 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2557,7 +2557,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, unlock_page_lruvec(lruvec); /* Caller disabled irqs, so they are still disabled here */

- split_page_owner(head, nr); + split_page_owner(head, order);

/* See comment in __split_huge_page_tail() */ if (PageAnon(head)) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d84b121d1e03..d537828bc4be 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2780,7 +2780,7 @@ void split_page(struct page *page, unsigned int order)

for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); - split_page_owner(page, 1 << order); + split_page_owner(page, order); split_page_memcg(page, order); } EXPORT_SYMBOL_GPL(split_page); @@ -4996,7 +4996,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *page = virt_to_page((void *)addr); struct page *last = page + nr;

- split_page_owner(page, 1 << order); + split_page_owner(page, order); split_page_memcg(page, order); while (page < --last) set_page_refcounted(last); diff --git a/mm/page_owner.c b/mm/page_owner.c index 31169b3e7f06..64233b5b09d5 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -211,11 +211,12 @@ void __set_page_owner_migrate_reason(struct page *page, int reason) page_ext_put(page_ext); }

-void __split_page_owner(struct page *page, unsigned int nr) +void __split_page_owner(struct page *page, int order) { int i; struct page_ext *page_ext = page_ext_get(page); struct page_owner *page_owner; + unsigned int nr = 1 << order;

if (unlikely(!page_ext)) return;

-- 2.39.2

Zi Yan

8:18 p.m.

New subject: [PATCH v3 3/7] mm: memcg: make memcg huge page split support any order split.

From: Zi Yan ziy@nvidia.com

It sets memcg information for the pages after the split. A new parameter new_order is added to tell the order of subpages in the new page, always 0 for now. It prepares for upcoming changes to support split huge page to any lower order.

Signed-off-by: Zi Yan ziy@nvidia.com --- include/linux/memcontrol.h | 4 ++-- mm/huge_memory.c | 2 +- mm/memcontrol.c | 11 ++++++----- mm/page_alloc.c | 4 ++-- 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e06a61ea4fc1..1633c00fe393 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1151,7 +1151,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, rcu_read_unlock(); }

-void split_page_memcg(struct page *head, int order); +void split_page_memcg(struct page *head, int old_order, int new_order);

unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, gfp_t gfp_mask, @@ -1588,7 +1588,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx) { }

-static inline void split_page_memcg(struct page *head, int order) +static inline void split_page_memcg(struct page *head, int old_order, int new_order) { }

diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a21921c90b21..106cde74d933 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2516,7 +2516,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, int i;

/* complete memcg works before add pages to LRU */ - split_page_memcg(head, order); + split_page_memcg(head, order, 0);

if (PageAnon(head) && PageSwapCache(head)) { swp_entry_t entry = { .val = page_private(head) }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index cab2828e188d..93ae37f90c84 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3414,23 +3414,24 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size) /* * Because page_memcg(head) is not set on tails, set it now. */ -void split_page_memcg(struct page *head, int order) +void split_page_memcg(struct page *head, int old_order, int new_order) { struct folio *folio = page_folio(head); struct mem_cgroup *memcg = folio_memcg(folio); int i; - unsigned int nr = 1 << order; + unsigned int old_nr = 1 << old_order; + unsigned int new_nr = 1 << new_order;

if (mem_cgroup_disabled() || !memcg) return;

- for (i = 1; i < nr; i++) + for (i = new_nr; i < old_nr; i += new_nr) folio_page(folio, i)->memcg_data = folio->memcg_data;

if (folio_memcg_kmem(folio)) - obj_cgroup_get_many(__folio_objcg(folio), nr - 1); + obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1); else - css_get_many(&memcg->css, nr - 1); + css_get_many(&memcg->css, old_nr / new_nr - 1); }

#ifdef CONFIG_SWAP diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d537828bc4be..ef559795525b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2781,7 +2781,7 @@ void split_page(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); split_page_owner(page, order); - split_page_memcg(page, order); + split_page_memcg(page, order, 0); } EXPORT_SYMBOL_GPL(split_page);

@@ -4997,7 +4997,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *last = page + nr;

split_page_owner(page, order); - split_page_memcg(page, order); + split_page_memcg(page, order, 0); while (page < --last) set_page_refcounted(last);

-- 2.39.2

Zi Yan

8:18 p.m.

New subject: [PATCH v3 4/7] mm: page_owner: add support for splitting to any order in split page_owner.

From: Zi Yan ziy@nvidia.com

It adds a new_order parameter to set new page order in page owner. It prepares for upcoming changes to support split huge page to any lower order.

Signed-off-by: Zi Yan ziy@nvidia.com --- include/linux/page_owner.h | 10 +++++----- mm/huge_memory.c | 2 +- mm/page_alloc.c | 4 ++-- mm/page_owner.c | 9 +++++---- 4 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h index d7878523adfc..a784ba69f67f 100644 --- a/include/linux/page_owner.h +++ b/include/linux/page_owner.h @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops; extern void __reset_page_owner(struct page *page, unsigned short order); extern void __set_page_owner(struct page *page, unsigned short order, gfp_t gfp_mask); -extern void __split_page_owner(struct page *page, int order); +extern void __split_page_owner(struct page *page, int old_order, int new_order); extern void __folio_copy_owner(struct folio *newfolio, struct folio *old); extern void __set_page_owner_migrate_reason(struct page *page, int reason); extern void __dump_page_owner(const struct page *page); @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page, __set_page_owner(page, order, gfp_mask); }

-static inline void split_page_owner(struct page *page, int order) +static inline void split_page_owner(struct page *page, int old_order, int new_order) { if (static_branch_unlikely(&page_owner_inited)) - __split_page_owner(page, order); + __split_page_owner(page, old_order, new_order); } static inline void folio_copy_owner(struct folio *newfolio, struct folio *old) { @@ -56,11 +56,11 @@ static inline void reset_page_owner(struct page *page, unsigned short order) { } static inline void set_page_owner(struct page *page, - unsigned int order, gfp_t gfp_mask) + unsigned short order, gfp_t gfp_mask) { } static inline void split_page_owner(struct page *page, - int order) + int old_order, int new_order) { } static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 106cde74d933..f8a8a72b207d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2557,7 +2557,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, unlock_page_lruvec(lruvec); /* Caller disabled irqs, so they are still disabled here */

- split_page_owner(head, order); + split_page_owner(head, order, 0);

/* See comment in __split_huge_page_tail() */ if (PageAnon(head)) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ef559795525b..4845ff6c4223 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2780,7 +2780,7 @@ void split_page(struct page *page, unsigned int order)

for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); - split_page_owner(page, order); + split_page_owner(page, order, 0); split_page_memcg(page, order, 0); } EXPORT_SYMBOL_GPL(split_page); @@ -4996,7 +4996,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *page = virt_to_page((void *)addr); struct page *last = page + nr;

- split_page_owner(page, order); + split_page_owner(page, order, 0); split_page_memcg(page, order, 0); while (page < --last) set_page_refcounted(last); diff --git a/mm/page_owner.c b/mm/page_owner.c index 64233b5b09d5..72244a4f1a31 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -211,19 +211,20 @@ void __set_page_owner_migrate_reason(struct page *page, int reason) page_ext_put(page_ext); }

-void __split_page_owner(struct page *page, int order) +void __split_page_owner(struct page *page, int old_order, int new_order) { int i; struct page_ext *page_ext = page_ext_get(page); struct page_owner *page_owner; - unsigned int nr = 1 << order; + unsigned int old_nr = 1 << old_order; + unsigned int new_nr = 1 << new_order;

if (unlikely(!page_ext)) return;

- for (i = 0; i < nr; i++) { + for (i = 0; i < old_nr; i += new_nr) { page_owner = get_page_owner(page_ext); - page_owner->order = 0; + page_owner->order = new_order; page_ext = page_ext_next(page_ext); } page_ext_put(page_ext);

-- 2.39.2

Zi Yan

8:18 p.m.

New subject: [PATCH v3 5/7] mm: thp: split huge page to any lower order pages.

From: Zi Yan ziy@nvidia.com

To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list corruption when checking the page_deferred_list again.

It has many uses, like minimizing the number of pages after truncating a huge pagecache page. For anonymous THPs, we can only split them to order-0 like before until we add support for any size anonymous THPs.

Signed-off-by: Zi Yan ziy@nvidia.com --- include/linux/huge_mm.h | 10 ++-- mm/huge_memory.c | 102 +++++++++++++++++++++++++++++----------- 2 files changed, 81 insertions(+), 31 deletions(-)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 20284387b841..32c91e1b59cd 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -147,10 +147,11 @@ void prep_transhuge_page(struct page *page); void free_transhuge_page(struct page *page);

bool can_split_folio(struct folio *folio, int *pextra_pins); -int split_huge_page_to_list(struct page *page, struct list_head *list); +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order); static inline int split_huge_page(struct page *page) { - return split_huge_page_to_list(page, NULL); + return split_huge_page_to_list_to_order(page, NULL, 0); } void deferred_split_folio(struct folio *folio);

@@ -297,7 +298,8 @@ can_split_folio(struct folio *folio, int *pextra_pins) return false; } static inline int -split_huge_page_to_list(struct page *page, struct list_head *list) +split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order) { return 0; } @@ -397,7 +399,7 @@ static inline bool thp_migration_supported(void) static inline int split_folio_to_list(struct folio *folio, struct list_head *list) { - return split_huge_page_to_list(&folio->page, list); + return split_huge_page_to_list_to_order(&folio->page, list, 0); }

static inline int split_folio(struct folio *folio) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f8a8a72b207d..619d25278340 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2359,11 +2359,13 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,

static void unmap_folio(struct folio *folio) { - enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD | - TTU_SYNC; + enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC;

VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);

+ if (folio_test_pmd_mappable(folio)) + ttu_flags |= TTU_SPLIT_HUGE_PMD; + /* * Anon pages need migration entries to preserve them, but file * pages can simply be left unmapped, then faulted back on demand. @@ -2395,7 +2397,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail, struct lruvec *lruvec, struct list_head *list) { VM_BUG_ON_PAGE(!PageHead(head), head); - VM_BUG_ON_PAGE(PageCompound(tail), head); VM_BUG_ON_PAGE(PageLRU(tail), head); lockdep_assert_held(&lruvec->lru_lock);

@@ -2416,7 +2417,7 @@ static void lru_add_page_tail(struct page *head, struct page *tail, }

static void __split_huge_page_tail(struct page *head, int tail, - struct lruvec *lruvec, struct list_head *list) + struct lruvec *lruvec, struct list_head *list, unsigned int new_order) { struct page *page_tail = head + tail;

@@ -2483,10 +2484,15 @@ static void __split_huge_page_tail(struct page *head, int tail, * which needs correct compound_head(). */ clear_compound_head(page_tail); + if (new_order) { + prep_compound_page(page_tail, new_order); + prep_transhuge_page(page_tail); + }

/* Finally unfreeze refcount. Additional reference from page cache. */ - page_ref_unfreeze(page_tail, 1 + (!PageAnon(head) || - PageSwapCache(head))); + page_ref_unfreeze(page_tail, 1 + ((!PageAnon(head) || + PageSwapCache(head)) ? + thp_nr_pages(page_tail) : 0));

if (page_is_young(head)) set_page_young(page_tail); @@ -2504,7 +2510,7 @@ static void __split_huge_page_tail(struct page *head, int tail, }

static void __split_huge_page(struct page *page, struct list_head *list, - pgoff_t end) + pgoff_t end, unsigned int new_order) { struct folio *folio = page_folio(page); struct page *head = &folio->page; @@ -2512,11 +2518,12 @@ static void __split_huge_page(struct page *page, struct list_head *list, struct address_space *swap_cache = NULL; unsigned long offset = 0; unsigned int nr = thp_nr_pages(head); + unsigned int new_nr = 1 << new_order; int order = folio_order(folio); int i;

/* complete memcg works before add pages to LRU */ - split_page_memcg(head, order, 0); + split_page_memcg(head, order, new_order);

if (PageAnon(head) && PageSwapCache(head)) { swp_entry_t entry = { .val = page_private(head) }; @@ -2531,14 +2538,14 @@ static void __split_huge_page(struct page *page, struct list_head *list,

ClearPageHasHWPoisoned(head);

- for (i = nr - 1; i >= 1; i--) { - __split_huge_page_tail(head, i, lruvec, list); + for (i = nr - new_nr; i >= new_nr; i -= new_nr) { + __split_huge_page_tail(head, i, lruvec, list, new_order); /* Some pages can be beyond EOF: drop them from page cache */ if (head[i].index >= end) { struct folio *tail = page_folio(head + i);

if (shmem_mapping(head->mapping)) - shmem_uncharge(head->mapping->host, 1); + shmem_uncharge(head->mapping->host, new_nr); else if (folio_test_clear_dirty(tail)) folio_account_cleaned(tail, inode_to_wb(folio->mapping->host)); @@ -2548,29 +2555,38 @@ static void __split_huge_page(struct page *page, struct list_head *list, __xa_store(&head->mapping->i_pages, head[i].index, head + i, 0); } else if (swap_cache) { + /* + * split anonymous THPs (including swapped out ones) to + * non-zero order not supported + */ + VM_WARN_ONCE(new_order, + "Split swap-cached anon folio to non-0 order not supported"); __xa_store(&swap_cache->i_pages, offset + i, head + i, 0); } }

- ClearPageCompound(head); + if (!new_order) + ClearPageCompound(head); + else + set_compound_order(head, new_order); unlock_page_lruvec(lruvec); /* Caller disabled irqs, so they are still disabled here */

- split_page_owner(head, order, 0); + split_page_owner(head, order, new_order);

/* See comment in __split_huge_page_tail() */ if (PageAnon(head)) { /* Additional pin to swap cache */ if (PageSwapCache(head)) { - page_ref_add(head, 2); + page_ref_add(head, 1 + new_nr); xa_unlock(&swap_cache->i_pages); } else { page_ref_inc(head); } } else { /* Additional pin to page cache */ - page_ref_add(head, 2); + page_ref_add(head, 1 + new_nr); xa_unlock(&head->mapping->i_pages); } local_irq_enable(); @@ -2583,7 +2599,15 @@ static void __split_huge_page(struct page *page, struct list_head *list, split_swap_cluster(entry); }

- for (i = 0; i < nr; i++) { + /* + * set page to its compound_head when split to non order-0 pages, so + * we can skip unlocking it below, since PG_locked is transferred to + * the compound_head of the page and the caller will unlock it. + */ + if (new_order) + page = compound_head(page); + + for (i = 0; i < nr; i += new_nr) { struct page *subpage = head + i; if (subpage == page) continue; @@ -2617,29 +2641,31 @@ bool can_split_folio(struct folio *folio, int *pextra_pins) }

/* - * This function splits huge page into normal pages. @page can point to any - * subpage of huge page to split. Split doesn't change the position of @page. + * This function splits huge page into pages in @new_order. @page can point to + * any subpage of huge page to split. Split doesn't change the position of + * @page. * * Only caller must hold pin on the @page, otherwise split fails with -EBUSY. * The huge page must be locked. * * If @list is null, tail pages will be added to LRU list, otherwise, to @list. * - * Both head page and tail pages will inherit mapping, flags, and so on from - * the hugepage. + * Pages in new_order will inherit mapping, flags, and so on from the hugepage. * - * GUP pin and PG_locked transferred to @page. Rest subpages can be freed if - * they are not mapped. + * GUP pin and PG_locked transferred to @page or the compound page @page belongs + * to. Rest subpages can be freed if they are not mapped. * * Returns 0 if the hugepage is split successfully. * Returns -EBUSY if the page is pinned or if anon_vma disappeared from under * us. */ -int split_huge_page_to_list(struct page *page, struct list_head *list) +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, + unsigned int new_order) { struct folio *folio = page_folio(page); struct deferred_split *ds_queue = get_deferred_split_queue(folio); - XA_STATE(xas, &folio->mapping->i_pages, folio->index); + /* reset xarray order to new order after split */ + XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); struct anon_vma *anon_vma = NULL; struct address_space *mapping = NULL; int extra_pins, ret; @@ -2649,6 +2675,18 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);

+ /* Cannot split THP to order-1 (no order-1 THPs) */ + if (new_order == 1) { + VM_WARN_ONCE(1, "Cannot split to order-1 folio"); + return -EINVAL; + } + + /* Split anonymous folio to non-zero order not support */ + if (folio_test_anon(folio) && new_order) { + VM_WARN_ONCE(1, "Split anon folio to non-0 order not support"); + return -EINVAL; + } + is_hzp = is_huge_zero_page(&folio->page); VM_WARN_ON_ONCE_FOLIO(is_hzp, folio); if (is_hzp) @@ -2744,7 +2782,13 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (folio_ref_freeze(folio, 1 + extra_pins)) { if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; - list_del(&folio->_deferred_list); + /* + * Reinitialize page_deferred_list after removing the + * page from the split_queue, otherwise a subsequent + * split will see list corruption when checking the + * page_deferred_list. + */ + list_del_init(&folio->_deferred_list); } spin_unlock(&ds_queue->split_queue_lock); if (mapping) { @@ -2754,14 +2798,18 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (folio_test_swapbacked(folio)) { __lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, -nr); - } else { + } else if (!new_order) { + /* + * Decrease THP stats only if split to normal + * pages + */ __lruvec_stat_mod_folio(folio, NR_FILE_THPS, -nr); filemap_nr_thps_dec(mapping); } }

- __split_huge_page(page, list, end); + __split_huge_page(page, list, end, new_order); ret = 0; } else { spin_unlock(&ds_queue->split_queue_lock);

-- 2.39.2

Hugh Dickins

16 Apr 16 Apr

7:25 p.m.

New subject: [PATCH v3 5/7] mm: thp: split huge page to any lower order pages.

On Mon, 3 Apr 2023, Zi Yan wrote:

...

From: Zi Yan ziy@nvidia.com

To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list corruption when checking the page_deferred_list again.

It has many uses, like minimizing the number of pages after truncating a huge pagecache page. For anonymous THPs, we can only split them to order-0 like before until we add support for any size anonymous THPs.

Signed-off-by: Zi Yan ziy@nvidia.com

...

@@ -2754,14 +2798,18 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (folio_test_swapbacked(folio)) { __lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, -nr);
	} else {
	} else if (!new_order) {
		/*
		 * Decrease THP stats only if split to normal
		 * pages
		 */
	__lruvec_stat_mod_folio(folio, NR_FILE_THPS,
				-nr);
	filemap_nr_thps_dec(mapping);
}
}

This part is wrong. The problem I've had is /proc/sys/vm/stat_refresh warning of negative nr_shmem_hugepages (which then gets shown as 0 in vmstat or meminfo, even though there actually are shmem hugepages).

At first I thought that the fix needed (which I'm running with) is:

--- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2797,17 +2797,16 @@ int split_huge_page_to_list_to_order(str int nr = folio_nr_pages(folio);

xas_split(&xas, folio, folio_order(folio)); - if (folio_test_swapbacked(folio)) { - __lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, - -nr); - } else if (!new_order) { - /* - * Decrease THP stats only if split to normal - * pages - */ - __lruvec_stat_mod_folio(folio, NR_FILE_THPS, - -nr); - filemap_nr_thps_dec(mapping); + if (folio_test_pmd_mappable(folio) && + new_order < HPAGE_PMD_ORDER) { + if (folio_test_swapbacked(folio)) { + __lruvec_stat_mod_folio(folio, + NR_SHMEM_THPS, -nr); + } else { + __lruvec_stat_mod_folio(folio, + NR_FILE_THPS, -nr); + filemap_nr_thps_dec(mapping); + } } }

because elsewhere the maintenance of NR_SHMEM_THPS or NR_FILE_THPS is rightly careful to be dependent on folio_test_pmd_mappable() (and, so far as I know, we shall not be seeing folios of order higher than HPAGE_PMD_ORDER yet in mm/huge_memory.c - those would need more thought).

But it may be more complicated than that, given that patch 7/7 appears (I haven't tried) to allow splitting to other orders on a file opened for reading - that might be a bug.

The complication here is that we now have four kinds of large folio in mm/huge_memory.c, and the rules are a bit different for each.

Anonymous THPs: okay, I think I've seen you exclude those with -EINVAL at a higher level (and they wouldn't be getting into this "if (mapping) {" block anyway).

Shmem (swapbacked) THPs: we are only allocating shmem in 0-order or HPAGE_PMD_ORDER at present. I can imagine that in a few months or a year-or-so's time, we shall want to follow Matthew's folio readahead, and generalize to other orders in shmem; but right now I'd really prefer not to have truncation or debugfs introducing the surprise of other orders there. Maybe there's little that needs to be fixed, only the THP_SWPOUT and THP_SWPOUT_FALLBACK statistics have come to mind so far (would need to be limited to folio_test_pmd_mappable()); though I've no idea how well intermediate orders will work with or against THP swapout.

CONFIG_READ_ONLY_THP_FOR_FS=y file THPs: those need special care, and their filemap_nr_thps_dec(mapping) above may not be good enough. So long as it's working as intended, it does exclude the possibility of truncation splitting here; but if you allow splitting via debugfs to reach them, then the accounting needs to be changed - for them, any order higher than 0 has to be counted in nr_thps - so splitting one HPAGE_PMD_ORDER THP into multiple large folios will need to add to that count, not decrement it. Otherwise, a filesystem unprepared for large folios or compound pages is in danger of meeting them by surprise. Better just disable that possibility, along with shmem.

mapping_large_folio_support() file THPs: this category is the one you're really trying to address with this series, they can already come in various orders, and it's fair for truncation to make a different choice of orders - but is what it's doing worth doing? I'll say more on 6/7.

Hugh

Zi Yan

17 Apr 17 Apr

2:49 p.m.

New subject: [PATCH v3 5/7] mm: thp: split huge page to any lower order pages.

On 16 Apr 2023, at 15:25, Hugh Dickins wrote:

...

On Mon, 3 Apr 2023, Zi Yan wrote:

...
From: Zi Yan ziy@nvidia.com

To split a THP to any lower order pages, we need to reform THPs on subpages at given order and add page refcount based on the new page order. Also we need to reinitialize page_deferred_list after removing the page from the split_queue, otherwise a subsequent split will see list corruption when checking the page_deferred_list again.

It has many uses, like minimizing the number of pages after truncating a huge pagecache page. For anonymous THPs, we can only split them to order-0 like before until we add support for any size anonymous THPs.

Signed-off-by: Zi Yan ziy@nvidia.com

...

...
@@ -2754,14 +2798,18 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (folio_test_swapbacked(folio)) { __lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, -nr);
	} else {
	} else if (!new_order) {
		/*
		 * Decrease THP stats only if split to normal
		 * pages
		 */
	__lruvec_stat_mod_folio(folio, NR_FILE_THPS,
				-nr);
	filemap_nr_thps_dec(mapping);
}
}
This part is wrong. The problem I've had is /proc/sys/vm/stat_refresh warning of negative nr_shmem_hugepages (which then gets shown as 0 in vmstat or meminfo, even though there actually are shmem hugepages).

At first I thought that the fix needed (which I'm running with) is:

--- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2797,17 +2797,16 @@ int split_huge_page_to_list_to_order(str int nr = folio_nr_pages(folio);
	xas_split(&xas, folio, folio_order(folio));
	if (folio_test_swapbacked(folio)) {
		__lruvec_stat_mod_folio(folio, NR_SHMEM_THPS,
					-nr);
	} else if (!new_order) {
		/*
		 * Decrease THP stats only if split to normal
		 * pages
		 */
		__lruvec_stat_mod_folio(folio, NR_FILE_THPS,
					-nr);
		filemap_nr_thps_dec(mapping);
	if (folio_test_pmd_mappable(folio) &&
	    new_order < HPAGE_PMD_ORDER) {
		if (folio_test_swapbacked(folio)) {
			__lruvec_stat_mod_folio(folio,
					NR_SHMEM_THPS, -nr);
		} else {
			__lruvec_stat_mod_folio(folio,
					NR_FILE_THPS, -nr);
			filemap_nr_thps_dec(mapping);
		}
}
}
because elsewhere the maintenance of NR_SHMEM_THPS or NR_FILE_THPS is rightly careful to be dependent on folio_test_pmd_mappable() (and, so far as I know, we shall not be seeing folios of order higher than HPAGE_PMD_ORDER yet in mm/huge_memory.c - those would need more thought).

But it may be more complicated than that, given that patch 7/7 appears (I haven't tried) to allow splitting to other orders on a file opened for reading - that might be a bug.

The complication here is that we now have four kinds of large folio in mm/huge_memory.c, and the rules are a bit different for each.

Anonymous THPs: okay, I think I've seen you exclude those with -EINVAL at a higher level (and they wouldn't be getting into this "if (mapping) {" block anyway).

Shmem (swapbacked) THPs: we are only allocating shmem in 0-order or HPAGE_PMD_ORDER at present. I can imagine that in a few months or a year-or-so's time, we shall want to follow Matthew's folio readahead, and generalize to other orders in shmem; but right now I'd really prefer not to have truncation or debugfs introducing the surprise of other orders there. Maybe there's little that needs to be fixed, only the THP_SWPOUT and THP_SWPOUT_FALLBACK statistics have come to mind so far (would need to be limited to folio_test_pmd_mappable()); though I've no idea how well intermediate orders will work with or against THP swapout.

CONFIG_READ_ONLY_THP_FOR_FS=y file THPs: those need special care, and their filemap_nr_thps_dec(mapping) above may not be good enough. So long as it's working as intended, it does exclude the possibility of truncation splitting here; but if you allow splitting via debugfs to reach them, then the accounting needs to be changed - for them, any order higher than 0 has to be counted in nr_thps - so splitting one HPAGE_PMD_ORDER THP into multiple large folios will need to add to that count, not decrement it. Otherwise, a filesystem unprepared for large folios or compound pages is in danger of meeting them by surprise. Better just disable that possibility, along with shmem.

OK. I will disable for these two cases. I will come back to them once I figure the details out.

...

mapping_large_folio_support() file THPs: this category is the one you're really trying to address with this series, they can already come in various orders, and it's fair for truncation to make a different choice of orders - but is what it's doing worth doing? I'll say more on 6/7.

Hugh

-- Best Regards, Yan, Zi

Zi Yan

3 Apr 3 Apr

8:18 p.m.

New subject: [PATCH v3 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

From: Zi Yan ziy@nvidia.com

To minimize the number of pages after a huge page truncation, we do not need to split it all the way down to order-0. The huge page has at most three parts, the part before offset, the part to be truncated, the part remaining at the end. Find the greatest common divisor of them to calculate the new page order from it, so we can split the huge page to this order and keep the remaining pages as large and as few as possible.

Signed-off-by: Zi Yan ziy@nvidia.com --- mm/truncate.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/mm/truncate.c b/mm/truncate.c index 86de31ed4d32..817efd5e94b4 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -22,6 +22,7 @@ #include <linux/buffer_head.h> /* grr. try_to_release_page */ #include <linux/shmem_fs.h> #include <linux/rmap.h> +#include <linux/gcd.h> #include "internal.h"

/* @@ -211,7 +212,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio) bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) { loff_t pos = folio_pos(folio); - unsigned int offset, length; + unsigned int offset, length, remaining; + unsigned int new_order = folio_order(folio);

if (pos < start) offset = start - pos; @@ -222,6 +224,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) length = length - offset; else length = end + 1 - pos - offset; + remaining = folio_size(folio) - offset - length;

folio_wait_writeback(folio); if (length == folio_size(folio)) { @@ -236,11 +239,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) */ folio_zero_range(folio, offset, length);

+ /* + * Use the greatest common divisor of offset, length, and remaining + * as the smallest page size and compute the new order from it. So we + * can truncate a subpage as large as possible. Round up gcd to + * PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0. + */ + new_order = ilog2(round_up(gcd(gcd(offset, length), remaining), + PAGE_SIZE) / PAGE_SIZE); + + /* order-1 THP not supported, downgrade to order-0 */ + if (new_order == 1) + new_order = 0; + + if (folio_has_private(folio)) folio_invalidate(folio, offset, length); if (!folio_test_large(folio)) return true; - if (split_folio(folio) == 0) + if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0) return true; if (folio_test_dirty(folio)) return false;

-- 2.39.2

Hugh Dickins

16 Apr 16 Apr

7:44 p.m.

New subject: [PATCH v3 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

On Mon, 3 Apr 2023, Zi Yan wrote:

...

From: Zi Yan ziy@nvidia.com

To minimize the number of pages after a huge page truncation, we do not need to split it all the way down to order-0. The huge page has at most three parts, the part before offset, the part to be truncated, the part remaining at the end. Find the greatest common divisor of them to calculate the new page order from it, so we can split the huge page to this order and keep the remaining pages as large and as few as possible.

Signed-off-by: Zi Yan ziy@nvidia.com

mm/truncate.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/mm/truncate.c b/mm/truncate.c index 86de31ed4d32..817efd5e94b4 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -22,6 +22,7 @@ #include <linux/buffer_head.h> /* grr. try_to_release_page */ #include <linux/shmem_fs.h> #include <linux/rmap.h> +#include <linux/gcd.h>

Really?

...

#include "internal.h" /* @@ -211,7 +212,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio) bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) { loff_t pos = folio_pos(folio);

unsigned int offset, length;

unsigned int offset, length, remaining;

unsigned int new_order = folio_order(folio);

if (pos < start) offset = start - pos; @@ -222,6 +224,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) length = length - offset; else length = end + 1 - pos - offset;

remaining = folio_size(folio) - offset - length;

folio_wait_writeback(folio); if (length == folio_size(folio)) { @@ -236,11 +239,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) */ folio_zero_range(folio, offset, length);
/*
* Use the greatest common divisor of offset, length, and remaining
* as the smallest page size and compute the new order from it. So we
* can truncate a subpage as large as possible. Round up gcd to
* PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
*/
new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
		   PAGE_SIZE) / PAGE_SIZE);

Gosh. In mm/readahead.c I can see "order = __ffs(index)", and I think something along those lines would be more appropriate here.

But, if there's any value at all to choosing intermediate orders here in truncation, I don't think choosing a single order is the right approach - more easily implemented, yes, but is it worth doing?

What you'd actually want (if anything) is to choose the largest orders possible, with smaller and smaller orders filling in the rest (I expect there's a technical name for this, but I don't remember - bin packing is something else, I think).

As this code stands, truncate a 2M huge page at 1M and you get two 1M pieces (one then discarded) - nice; but truncate it at 1M+1 and you get lots of order 2 (forced up from 1) pieces. Seems weird, and not worth the effort.

Hugh

...

/* order-1 THP not supported, downgrade to order-0 */

if (new_order == 1)
new_order = 0;
if (folio_has_private(folio)) folio_invalidate(folio, offset, length); if (!folio_test_large(folio)) return true;
if (split_folio(folio) == 0)

if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0) return true; if (folio_test_dirty(folio)) return false;

-- 2.39.2

Hugh Dickins

7:51 p.m.

New subject: [PATCH v3 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

...

As this code stands, truncate a 2M huge page at 1M and you get two 1M pieces (one then discarded) - nice; but truncate it at 1M+1 and you get lots of order 2 (forced up from 1) pieces. Seems weird, and not worth the effort.

I've probably said that wrong: truncate at 1M+1 and you'd get lots of order 0 pieces.

Hugh

Zi Yan

17 Apr 17 Apr

3:20 p.m.

New subject: [PATCH v3 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

On 16 Apr 2023, at 15:44, Hugh Dickins wrote:

...

On Mon, 3 Apr 2023, Zi Yan wrote:

...
From: Zi Yan ziy@nvidia.com

To minimize the number of pages after a huge page truncation, we do not need to split it all the way down to order-0. The huge page has at most three parts, the part before offset, the part to be truncated, the part remaining at the end. Find the greatest common divisor of them to calculate the new page order from it, so we can split the huge page to this order and keep the remaining pages as large and as few as possible.

Signed-off-by: Zi Yan ziy@nvidia.com

mm/truncate.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/mm/truncate.c b/mm/truncate.c index 86de31ed4d32..817efd5e94b4 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -22,6 +22,7 @@ #include <linux/buffer_head.h> /* grr. try_to_release_page */ #include <linux/shmem_fs.h> #include <linux/rmap.h> +#include <linux/gcd.h>

Really?

...
#include "internal.h"

/* @@ -211,7 +212,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio) bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) { loff_t pos = folio_pos(folio);

unsigned int offset, length;

unsigned int offset, length, remaining;

unsigned int new_order = folio_order(folio);

if (pos < start) offset = start - pos;

@@ -222,6 +224,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) length = length - offset; else length = end + 1 - pos - offset;

remaining = folio_size(folio) - offset - length;

folio_wait_writeback(folio); if (length == folio_size(folio)) {

@@ -236,11 +239,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end) */ folio_zero_range(folio, offset, length);
/*
* Use the greatest common divisor of offset, length, and remaining
* as the smallest page size and compute the new order from it. So we
* can truncate a subpage as large as possible. Round up gcd to
* PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
*/
new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
		   PAGE_SIZE) / PAGE_SIZE);
Gosh. In mm/readahead.c I can see "order = __ffs(index)", and I think something along those lines would be more appropriate here.

But, if there's any value at all to choosing intermediate orders here in truncation, I don't think choosing a single order is the right approach - more easily implemented, yes, but is it worth doing?

What you'd actually want (if anything) is to choose the largest orders possible, with smaller and smaller orders filling in the rest (I expect there's a technical name for this, but I don't remember - bin packing is something else, I think).

As this code stands, truncate a 2M huge page at 1M and you get two 1M pieces (one then discarded) - nice; but truncate it at 1M+1 and you get lots of order 2 (forced up from 1) pieces. Seems weird, and not worth the effort.

The approach I am adding here is the simplest way of splitting a folio and trying to get >0 order folios after the split.

Yes, I agree that using "__ffs(index)" can create more >0 order folios, but it comes with either more runtime overheads or more code changes. Like your "1MB + 1" page split example, using "__ffs(index)", ideally, you will split 2MB into 2 1MBs, then 1MB into 2 512KBs, ..., 8KB into 2 4KBs and at the end of the day, we will have 1MB, 512KB, ..., 8KB each and two 2 4KBs, maximizing the number of >0 order folios. But what is the cost?

1. To minimizing code changes, we then need to call split_huge_page_to_list_to_order() 9 times from new_order=8 to new_order=0. Since each split needs to unmap and remap the target folio, we shall see 9 TLB shutdowns. I am not sure it is worth the cost.

2. To get rid of the unmap and remap overheads, we probably can unmap the folio, then do all the 9 splits, then remap all the split pages. But this would make split_huge_page() a lot more complicated and I am not sure a good way of passing split order information and the corresponding to-be-split subpages. Do we need a dynamic list to store them, making new memory allocations a prerequisite of split_huge_page()? Maybe we can encode in the split information in two ints? In the first one, each bit tells which order to split the page (like order=__ffs(index)) and in the second one, each bit tells which subpage to split next (0 means the left subpage, 1 means the right subpage). So your "1MB + 1" page split will be encoded as 0b111111111 (first int), 0b1000000 (second int and it has 1 fewer bit, since first split does not need to know which subpage to split).

What do you think? If you have a better idea, I am all ears. And if you are willing to help me review the more complicated code changes, I am more than happy to implement it in the next version. :)

Thank you for your comments. They are very helpful!

...

Hugh

...
/* order-1 THP not supported, downgrade to order-0 */

if (new_order == 1)
new_order = 0;
if (folio_has_private(folio)) folio_invalidate(folio, offset, length); if (!folio_test_large(folio)) return true;
if (split_folio(folio) == 0)

if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0) return true; if (folio_test_dirty(folio)) return false;

-- 2.39.2

-- Best Regards, Yan, Zi

Hugh Dickins

18 Apr 18 Apr

1:05 a.m.

New subject: [PATCH v3 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

On Mon, 17 Apr 2023, Zi Yan wrote:

...

What do you think? If you have a better idea, I am all ears. And if you are willing to help me review the more complicated code changes, I am more than happy to implement it in the next version. :)

Sorry, no, not me. You'll have to persuade someone else that "optimizing" truncation is a worthwhile path to pursue (and to pursue now) - I was just trying to illustrate that the current patchset didn't seem very useful.

But don't throw your work away. I expect some of it can become useful later e.g. once most of the main filesystems support large folios, and the complication of CONFIG_READ_ONLY_THP_FOR_FS can be deleted.

I doubt my "minimizing the number of folios" approach would be worth the effort; I think more likely that we shall settle on an intermediate folio size (between 4K and THP: maybe 64K, but not necessarily the same on all machines or all workloads) to aim for, and then maybe truncation of THP would split to those units. But it's not a job I shall get into - I'll just continue to report and try to fix what bugs I see.

Hugh

Zi Yan

3 Apr 3 Apr

8:18 p.m.

New subject: [PATCH v3 7/7] mm: huge_memory: enable debugfs to split huge pages to any order.

From: Zi Yan ziy@nvidia.com

It is used to test split_huge_page_to_list_to_order for pagecache THPs. Also add test cases for split_huge_page_to_list_to_order via both debugfs, truncating a file, and punching holes in a file.

Signed-off-by: Zi Yan ziy@nvidia.com --- mm/huge_memory.c | 34 ++- .../selftests/mm/split_huge_page_test.c | 225 +++++++++++++++++- 2 files changed, 242 insertions(+), 17 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 619d25278340..ad5b29558a51 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3023,7 +3023,7 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma) }

static int split_huge_pages_pid(int pid, unsigned long vaddr_start, - unsigned long vaddr_end) + unsigned long vaddr_end, unsigned int new_order) { int ret = 0; struct task_struct *task; @@ -3085,13 +3085,19 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start, goto next;

total++; - if (!can_split_folio(page_folio(page), NULL)) + /* + * For folios with private, split_huge_page_to_list_to_order() + * will try to drop it before split and then check if the folio + * can be split or not. So skip the check here. + */ + if (!folio_test_private(page_folio(page)) && + !can_split_folio(page_folio(page), NULL)) goto next;

if (!trylock_page(page)) goto next;

- if (!split_huge_page(page)) + if (!split_huge_page_to_list_to_order(page, NULL, new_order)) split++;

unlock_page(page); @@ -3109,7 +3115,7 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start, }

static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start, - pgoff_t off_end) + pgoff_t off_end, unsigned int new_order) { struct filename *file; struct file *candidate; @@ -3148,7 +3154,7 @@ static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start, if (!folio_trylock(folio)) goto next;

- if (!split_folio(folio)) + if (!split_huge_page_to_list_to_order(&folio->page, NULL, new_order)) split++;

folio_unlock(folio); @@ -3173,10 +3179,14 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf, { static DEFINE_MUTEX(split_debug_mutex); ssize_t ret; - /* hold pid, start_vaddr, end_vaddr or file_path, off_start, off_end */ + /* + * hold pid, start_vaddr, end_vaddr, new_order or + * file_path, off_start, off_end, new_order + */ char input_buf[MAX_INPUT_BUF_SZ]; int pid; unsigned long vaddr_start, vaddr_end; + unsigned int new_order = 0;

ret = mutex_lock_interruptible(&split_debug_mutex); if (ret) @@ -3205,29 +3215,29 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf, goto out; }

- ret = sscanf(buf, "0x%lx,0x%lx", &off_start, &off_end); - if (ret != 2) { + ret = sscanf(buf, "0x%lx,0x%lx,%d", &off_start, &off_end, &new_order); + if (ret != 2 && ret != 3) { ret = -EINVAL; goto out; } - ret = split_huge_pages_in_file(file_path, off_start, off_end); + ret = split_huge_pages_in_file(file_path, off_start, off_end, new_order); if (!ret) ret = input_len;

goto out; }

- ret = sscanf(input_buf, "%d,0x%lx,0x%lx", &pid, &vaddr_start, &vaddr_end); + ret = sscanf(input_buf, "%d,0x%lx,0x%lx,%d", &pid, &vaddr_start, &vaddr_end, &new_order); if (ret == 1 && pid == 1) { split_huge_pages_all(); ret = strlen(input_buf); goto out; - } else if (ret != 3) { + } else if (ret != 3 && ret != 4) { ret = -EINVAL; goto out; }

- ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end); + ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end, new_order); if (!ret) ret = strlen(input_buf); out: diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c index b8558c7f1a39..cbb5e6893cbf 100644 --- a/tools/testing/selftests/mm/split_huge_page_test.c +++ b/tools/testing/selftests/mm/split_huge_page_test.c @@ -16,6 +16,7 @@ #include <sys/mount.h> #include <malloc.h> #include <stdbool.h> +#include <time.h> #include "vm_util.h"

uint64_t pagesize; @@ -23,10 +24,12 @@ unsigned int pageshift; uint64_t pmd_pagesize;

#define SPLIT_DEBUGFS "/sys/kernel/debug/split_huge_pages" +#define SMAP_PATH "/proc/self/smaps" +#define THP_FS_PATH "/mnt/thp_fs" #define INPUT_MAX 80

-#define PID_FMT "%d,0x%lx,0x%lx" -#define PATH_FMT "%s,0x%lx,0x%lx" +#define PID_FMT "%d,0x%lx,0x%lx,%d" +#define PATH_FMT "%s,0x%lx,0x%lx,%d"

#define PFN_MASK ((1UL<<55)-1) #define KPF_THP (1UL<<22) @@ -113,7 +116,7 @@ void split_pmd_thp(void)

/* split all THPs */ write_debugfs(PID_FMT, getpid(), (uint64_t)one_page, - (uint64_t)one_page + len); + (uint64_t)one_page + len, 0);

for (i = 0; i < len; i++) if (one_page[i] != (char)i) { @@ -203,7 +206,7 @@ void split_pte_mapped_thp(void)

/* split all remapped THPs */ write_debugfs(PID_FMT, getpid(), (uint64_t)pte_mapped, - (uint64_t)pte_mapped + pagesize * 4); + (uint64_t)pte_mapped + pagesize * 4, 0);

/* smap does not show THPs after mremap, use kpageflags instead */ thp_size = 0; @@ -269,7 +272,7 @@ void split_file_backed_thp(void) }

/* split the file-backed THP */ - write_debugfs(PATH_FMT, testfile, pgoff_start, pgoff_end); + write_debugfs(PATH_FMT, testfile, pgoff_start, pgoff_end, 0);

status = unlink(testfile); if (status) @@ -290,20 +293,232 @@ void split_file_backed_thp(void) printf("file-backed THP split test done, please check dmesg for more information\n"); }

+void create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd, char **addr) +{ + size_t i; + int dummy; + + srand(time(NULL)); + + *fd = open(testfile, O_CREAT | O_RDWR, 0664); + if (*fd == -1) { + perror("Failed to create a file at "THP_FS_PATH); + exit(EXIT_FAILURE); + } + + for (i = 0; i < fd_size; i++) { + unsigned char byte = (unsigned char)i; + + write(*fd, &byte, sizeof(byte)); + } + close(*fd); + sync(); + *fd = open("/proc/sys/vm/drop_caches", O_WRONLY); + if (*fd == -1) { + perror("open drop_caches"); + goto err_out_unlink; + } + if (write(*fd, "3", 1) != 1) { + perror("write to drop_caches"); + goto err_out_unlink; + } + close(*fd); + + *fd = open(testfile, O_RDWR); + if (*fd == -1) { + perror("Failed to open a file at "THP_FS_PATH); + goto err_out_unlink; + } + + *addr = mmap(NULL, fd_size, PROT_READ|PROT_WRITE, MAP_SHARED, *fd, 0); + if (*addr == (char *)-1) { + perror("cannot mmap"); + goto err_out_close; + } + madvise(*addr, fd_size, MADV_HUGEPAGE); + + for (size_t i = 0; i < fd_size; i++) + dummy += *(*addr + i); + + if (!check_huge_file(*addr, fd_size / pmd_pagesize, pmd_pagesize)) { + printf("No pagecache THP generated, please mount a filesystem supporting pagecache THP at "THP_FS_PATH"\n"); + goto err_out_close; + } + return; +err_out_close: + close(*fd); +err_out_unlink: + unlink(testfile); + exit(EXIT_FAILURE); +} + +void split_thp_in_pagecache_to_order(size_t fd_size, int order) +{ + int fd; + char *addr; + size_t i; + const char testfile[] = THP_FS_PATH "/test"; + int err = 0; + + create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr); + + printf("split %ld kB PMD-mapped pagecache page to order %d ... ", fd_size >> 10, order); + write_debugfs(PID_FMT, getpid(), (uint64_t)addr, (uint64_t)addr + fd_size, order); + + for (i = 0; i < fd_size; i++) + if (*(addr + i) != (char)i) { + printf("%lu byte corrupted in the file\n", i); + err = EXIT_FAILURE; + goto out; + } + + if (!check_huge_file(addr, 0, pmd_pagesize)) { + printf("Still FilePmdMapped not split\n"); + err = EXIT_FAILURE; + goto out; + } + + printf("done\n"); +out: + close(fd); + unlink(testfile); + if (err) + exit(err); +} + +void truncate_thp_in_pagecache_to_order(size_t fd_size, int order) +{ + int fd; + char *addr; + size_t i; + const char testfile[] = THP_FS_PATH "/test"; + int err = 0; + + create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr); + + printf("truncate %ld kB PMD-mapped pagecache page to size %lu kB ... ", + fd_size >> 10, 4UL << order); + ftruncate(fd, pagesize << order); + + for (i = 0; i < (pagesize << order); i++) + if (*(addr + i) != (char)i) { + printf("%lu byte corrupted in the file\n", i); + err = EXIT_FAILURE; + goto out; + } + + if (!check_huge_file(addr, 0, pmd_pagesize)) { + printf("Still FilePmdMapped not split after truncate\n"); + err = EXIT_FAILURE; + goto out; + } + + printf("done\n"); +out: + close(fd); + unlink(testfile); + if (err) + exit(err); +} + +void punch_hole_in_pagecache_thp(size_t fd_size, off_t offset[], off_t len[], + int n, int num_left_thps) +{ + int fd, j; + char *addr; + size_t i; + const char testfile[] = THP_FS_PATH "/test"; + int err = 0; + + create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr); + + for (j = 0; j < n; j++) { + printf("punch a hole to %ld kB PMD-mapped pagecache page at addr: %lx, offset %ld, and len %ld ...\n", + fd_size >> 10, (unsigned long)addr, offset[j], len[j]); + fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, offset[j], len[j]); + } + + for (i = 0; i < fd_size; i++) { + int in_hole = 0; + + for (j = 0; j < n; j++) + if (i >= offset[j] && i < (offset[j] + len[j])) { + in_hole = 1; + break; + } + + if (in_hole) { + if (*(addr + i)) { + printf("%lu byte non-zero after punch\n", i); + err = EXIT_FAILURE; + goto out; + } + continue; + } + if (*(addr + i) != (char)i) { + printf("%lu byte corrupted in the file\n", i); + err = EXIT_FAILURE; + goto out; + } + } + + if (!check_huge_file(addr, num_left_thps, pmd_pagesize)) { + printf("Still FilePmdMapped not split after punch\n"); + goto out; + } + printf("done\n"); +out: + close(fd); + unlink(testfile); + if (err) + exit(err); +} + int main(int argc, char **argv) { + int i; + size_t fd_size; + off_t offset[2], len[2]; + if (geteuid() != 0) { printf("Please run the benchmark as root\n"); exit(EXIT_FAILURE); }

+ setbuf(stdout, NULL); + pagesize = getpagesize(); pageshift = ffs(pagesize) - 1; pmd_pagesize = read_pmd_pagesize(); + fd_size = 2 * pmd_pagesize;

split_pmd_thp(); split_pte_mapped_thp(); split_file_backed_thp();

+ for (i = 8; i >= 0; i--) + if (i != 1) + split_thp_in_pagecache_to_order(fd_size, i); + + /* + * for i is 1, truncate code in the kernel should create order-0 pages + * instead of order-1 THPs, since order-1 THP is not supported. No error + * is expected. + */ + for (i = 8; i >= 0; i--) + truncate_thp_in_pagecache_to_order(fd_size, i); + + offset[0] = 123; + offset[1] = 4 * pagesize; + len[0] = 200 * pagesize; + len[1] = 16 * pagesize; + punch_hole_in_pagecache_thp(fd_size, offset, len, 2, 1); + + offset[0] = 259 * pagesize + pagesize / 2; + offset[1] = 33 * pagesize; + len[0] = 129 * pagesize; + len[1] = 16 * pagesize; + punch_hole_in_pagecache_thp(fd_size, offset, len, 2, 1); + return 0; }

-- 2.39.2

Andrew Morton

4 Apr 4 Apr

9:47 p.m.

On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...

File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Hugh Dickins

16 Apr 16 Apr

6:11 p.m.

On Tue, 4 Apr 2023, Andrew Morton wrote:

...

On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

Thanks, Hugh

Andrew Morton

6:45 p.m.

On Sun, 16 Apr 2023 11:11:49 -0700 (PDT) Hugh Dickins hughd@google.com wrote:

...

On Tue, 4 Apr 2023, Andrew Morton wrote:

...
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I have it still parked awaiting some reviewer input.

...

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

OK, I'll drop the series for this cycle.

David Hildenbrand

17 Apr 17 Apr

2:20 p.m.

On 16.04.23 20:11, Hugh Dickins wrote:

...

On Tue, 4 Apr 2023, Andrew Morton wrote:

...
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without.

Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation.

-- Thanks, David / dhildenb

Zi Yan

7:26 p.m.

On 17 Apr 2023, at 10:20, David Hildenbrand wrote:

...

On 16.04.23 20:11, Hugh Dickins wrote:

...
On Tue, 4 Apr 2023, Andrew Morton wrote:

...
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without.

Ryan is working on large folio for anonymous pages and has shown promising performance results[1]. This patchset would avoid getting base pages during split if possible.

...

Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation.

Come on. 225 out of 365 insertions are self tests code. We need motivation to add testing code?

[1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.c...

-- Best Regards, Yan, Zi

David Hildenbrand

18 Apr 18 Apr

10:29 a.m.

On 17.04.23 21:26, Zi Yan wrote:

...

On 17 Apr 2023, at 10:20, David Hildenbrand wrote:

...
On 16.04.23 20:11, Hugh Dickins wrote:

...
On Tue, 4 Apr 2023, Andrew Morton wrote:

...
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without.

Ryan is working on large folio for anonymous pages and has shown promising performance results[1]. This patchset would avoid getting base pages during split if possible.

Yes, I know. And it would be great to get some actual numbers with/without your patches to show that this optimization actually matters in practice.

Unrelated, your cover letter mentions "file folio truncate operations.". Would it also apply to FALLOC_FL_PUNCH_HOLE, when partially zapping a THP?

...

...
Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation.

Come on. 225 out of 365 insertions are self tests code. We need motivation to add testing code?

Well, if you add feature X and the tests target feature X, then certainly having the tests require the same motivation as feature X. But yeah, the actual kernel code change is smaller than it looks at first sight.

-- Thanks, David / dhildenb

Zi Yan

2 p.m.

On 18 Apr 2023, at 6:29, David Hildenbrand wrote:

...

On 17.04.23 21:26, Zi Yan wrote:

...
On 17 Apr 2023, at 10:20, David Hildenbrand wrote:

...
On 16.04.23 20:11, Hugh Dickins wrote:

...
On Tue, 4 Apr 2023, Andrew Morton wrote:

...
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan zi.yan@sent.com wrote:

...
File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.

Andrew, please don't let this series drift into 6.4-rc1.

I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises.

The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch.

I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without.

Ryan is working on large folio for anonymous pages and has shown promising performance results[1]. This patchset would avoid getting base pages during split if possible.

Yes, I know. And it would be great to get some actual numbers with/without your patches to show that this optimization actually matters in practice.

Sure. Will try to add perf numbers in the next version.

...

Unrelated, your cover letter mentions "file folio truncate operations.". Would it also apply to FALLOC_FL_PUNCH_HOLE, when partially zapping a THP?

Yes. In the self tests, I have fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, offset[j], len[j]); and it uses the truncate operation I updated in patch 6.

...

...
...
Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation.

Come on. 225 out of 365 insertions are self tests code. We need motivation to add testing code?

Well, if you add feature X and the tests target feature X, then certainly having the tests require the same motivation as feature X. But yeah, the actual kernel code change is smaller than it looks at first sight.

-- Thanks,

David / dhildenb

-- Best Regards, Yan, Zi

Pankaj Raghav (Samsung)

13 Feb 13 Feb

12:30 p.m.

Hi Zi yan,

...

From: Zi Yan ziy@nvidia.com

Hi all,

File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

I recently posted patches to enable block size > page size(Large Block Sizes) in XFS[1]. The main idea of LBS is to have a notion of minimum order in the page cache that corresponds to the filesystem block size.

Ability to split a folio based on a given order is something that would definitely optimize the LBS implementation.

The current implementation refuses to split a large folio if it has a minimum order set in the page cache [2]. What we would like to have instead is to split it based on the minimum order. The main use is of course being able to free some folios during partial truncate operation.

Your patch was also suggested by willy during our LPC talk[3].

I tried rebasing your patch and there were a lot of non-trivial conflicts. Is there any plans on sending a new version?

[1] https://lore.kernel.org/linux-xfs/20240213093713.1753368-1-kernel@pankajragh... [2] https://lore.kernel.org/linux-xfs/20240213093713.1753368-9-kernel@pankajragh... [3] https://youtu.be/ar72r5Xf7x4?si=XDb-g7SSIgS-5TkP&t=1457

-- Pankaj

Zi Yan

1:46 p.m.

On 13 Feb 2024, at 7:30, Pankaj Raghav (Samsung) wrote:

...

Hi Zi yan,

...
From: Zi Yan ziy@nvidia.com

Hi all,

File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations.

I recently posted patches to enable block size > page size(Large Block Sizes) in XFS[1]. The main idea of LBS is to have a notion of minimum order in the page cache that corresponds to the filesystem block size.

Ability to split a folio based on a given order is something that would definitely optimize the LBS implementation.

The current implementation refuses to split a large folio if it has a minimum order set in the page cache [2]. What we would like to have instead is to split it based on the minimum order. The main use is of course being able to free some folios during partial truncate operation.

Your patch was also suggested by willy during our LPC talk[3].

I tried rebasing your patch and there were a lot of non-trivial conflicts. Is there any plans on sending a new version?

Sure. I am going to rebase and send a new version out.

...

[1] https://lore.kernel.org/linux-xfs/20240213093713.1753368-1-kernel@pankajragh... [2] https://lore.kernel.org/linux-xfs/20240213093713.1753368-9-kernel@pankajragh... [3] https://youtu.be/ar72r5Xf7x4?si=XDb-g7SSIgS-5TkP&t=1457

-- Pankaj

-- Best Regards, Yan, Zi

312

days inactive

628

days old

linux-kselftest-mirror@lists.linaro.org

22 comments

participants

tags (0)

participants (6)

Andrew Morton
David Hildenbrand
Hugh Dickins
Pankaj Raghav (Samsung)
Zi Yan
Zi Yan