- Linux-stable-mirror - lists.linaro.org

[PATCH] filesystem-dax: Fix dax_layout_busy_page() livelock

by Dan Williams

In the presence of multi-order entries the typical pagevec_lookup_entries() pattern may loop forever: while (index < end && pagevec_lookup_entries(&pvec, mapping, index, min(end - index, (pgoff_t)PAGEVEC_SIZE), indices)) { ... for (i = 0; i < pagevec_count(&pvec); i++) { index = indices[i]; ... } index++; /* BUG */ } The loop updates 'index' for each index found and then increments to the next possible page to continue the lookup. However, if the last entry in the pagevec is multi-order then the next possible page index is more than 1 page away. Fix this locally for the filesystem-dax case by checking for dax-multi-order entries. Going forward new users of multi-order entries need to be similarly careful, or we need a generic way to report the page increment in the radix iterator. Fixes: 5fac7408d828 ("mm, fs, dax: handle layout changes to pinned dax...") Cc: <stable(a)vger.kernel.org> Cc: Jan Kara <jack(a)suse.cz> Cc: Ross Zwisler <zwisler(a)kernel.org> Cc: Matthew Wilcox <willy(a)infradead.org> Signed-off-by: Dan Williams <dan.j.williams(a)intel.com> --- fs/dax.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 4becbf168b7f..c1472eede1f7 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -666,6 +666,8 @@ struct page *dax_layout_busy_page(struct address_space *mapping) while (index < end && pagevec_lookup_entries(&pvec, mapping, index, min(end - index, (pgoff_t)PAGEVEC_SIZE), indices)) { + pgoff_t nr_pages = 1; + for (i = 0; i < pagevec_count(&pvec); i++) { struct page *pvec_ent = pvec.pages[i]; void *entry; @@ -680,8 +682,11 @@ struct page *dax_layout_busy_page(struct address_space *mapping) xa_lock_irq(&mapping->i_pages); entry = get_unlocked_mapping_entry(mapping, index, NULL); - if (entry) + if (entry) { page = dax_busy_page(entry); + /* account for multi-order entries */ + nr_pages = 1UL << dax_radix_order(entry); + } put_unlocked_mapping_entry(mapping, index, entry); xa_unlock_irq(&mapping->i_pages); if (page) @@ -696,7 +701,7 @@ struct page *dax_layout_busy_page(struct address_space *mapping) */ pagevec_remove_exceptionals(&pvec); pagevec_release(&pvec); - index++; + index += nr_pages; if (page) break;

7 years, 1 month

1
1
0 0

please apply d41aa5252394 ("mm: madvise(MADV_DODUMP): allow hugetlbfs pages")

by Daniel Black

This is a bugfix on 0103bd16fb90 which exists in 3.16+ stable trees Mike Kravetz confirmed this as a regression: https://marc.info/?l=linux-mm&m=153843190414206&w=2 Thanks -- Daniel Black

7 years, 1 month

2
1
0 0

[PATCH stable 0/2] termios: Alpha BOTHER/IBSHIFT, tty_baudrate fix

by H. Peter Anvin

From: "H. Peter Anvin (Intel)" <hpa(a)zytor.com> It turns out that Alpha is the only architecture that never implemented BOTHER and IBSHIFT, which is otherwise ages old. This is one thing that has held up glibc support for this feature (all other architectures have supported these for about a decade, at least before the current 3.2 glibc cutoff.) Furthermore, in the process of dealing with this, I discovered that the current code in tty_baudrate.c can read past the end of the baud_table[] on Alpha and PowerPC. The second patch in this series fixes that, but it also cleans up the code substantially by auto-generating the table and, since all architectures now have them, removing all conditionals for BOTHER and IBSHIFT existing. Tagging for stable because these are concrete problems. I have a much bigger update in process, nearly done, which will clean up a lot of duplicated code and make the uapi headers usable for libc, but that is not critical on the same level. Tested on x86, compile-tested on Alpha. Signed-off-by: H. Peter Anvin (Intel) <hpa(a)zytor.com> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Cc: Jiri Slaby <jslaby(a)suse.com> Cc: Al Viro <viro(a)zeniv.linux.org.uk> Cc: Richard Henderson <rth(a)twiddle.net> Cc: Ivan Kokshaysky <ink(a)jurassic.park.msu.ru> Cc: Matt Turner <mattst88(a)gmail.com> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Kate Stewart <kstewart(a)linuxfoundation.org> Cc: Philippe Ombredanne <pombredanne(a)nexb.com> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Cc: Eugene Syromiatnikov <esyr(a)redhat.com> Cc: <linux-alpha(a)vger.kernel.org> Cc: <linux-serial(a)vger.kernel.org> Cc: Johan Hovold <johan(a)kernel.org> Cc: Alan Cox <alan(a)lxorguk.ukuu.org.uk> Cc: Benjamin Herrenschmidt <benh(a)kernel.crashing.org> Cc: Paul Mackerras <paulus(a)samba.org> Cc: Michael Ellerman <mpe(a)ellerman.id.au> Cc: <stable(a)vger.kernel.org> --- arch/alpha/include/asm/termios.h | 8 +- arch/alpha/include/uapi/asm/ioctls.h | 5 + arch/alpha/include/uapi/asm/termbits.h | 17 +++ drivers/tty/.gitignore | 1 + drivers/tty/Makefile | 16 +++ drivers/tty/bmacros.c | 2 + drivers/tty/tty_baudrate.c | 190 +++++++++++++-------------------- 7 files changed, 122 insertions(+), 117 deletions(-)

7 years, 1 month

1
2
0 0

[PATCH] remoteproc: qcom: q6v5: Propagate EPROBE_DEFER

by Bjorn Andersson

In the case that the interrupts fail to result because of the interrupt-controller not yet being registered the platform_get_irq_byname() call will fail with -EPROBE_DEFER, but passing this into devm_request_threaded_irq() will result in -EINVAL being returned, the driver is therefor not reprobed later. Fixes: 3b415c8fb263 ("remoteproc: q6v5: Extract common resource handling") Cc: stable(a)vger.kernel.org Signed-off-by: Bjorn Andersson <bjorn.andersson(a)linaro.org> --- drivers/remoteproc/qcom_q6v5.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/remoteproc/qcom_q6v5.c b/drivers/remoteproc/qcom_q6v5.c index 61a760ee4aac..e9ab90c19304 100644 --- a/drivers/remoteproc/qcom_q6v5.c +++ b/drivers/remoteproc/qcom_q6v5.c @@ -198,6 +198,9 @@ int qcom_q6v5_init(struct qcom_q6v5 *q6v5, struct platform_device *pdev, } q6v5->fatal_irq = platform_get_irq_byname(pdev, "fatal"); + if (q6v5->fatal_irq == -EPROBE_DEFER) + return -EPROBE_DEFER; + ret = devm_request_threaded_irq(&pdev->dev, q6v5->fatal_irq, NULL, q6v5_fatal_interrupt, IRQF_TRIGGER_RISING | IRQF_ONESHOT, @@ -208,6 +211,9 @@ int qcom_q6v5_init(struct qcom_q6v5 *q6v5, struct platform_device *pdev, } q6v5->ready_irq = platform_get_irq_byname(pdev, "ready"); + if (q6v5->ready_irq == -EPROBE_DEFER) + return -EPROBE_DEFER; + ret = devm_request_threaded_irq(&pdev->dev, q6v5->ready_irq, NULL, q6v5_ready_interrupt, IRQF_TRIGGER_RISING | IRQF_ONESHOT, @@ -218,6 +224,9 @@ int qcom_q6v5_init(struct qcom_q6v5 *q6v5, struct platform_device *pdev, } q6v5->handover_irq = platform_get_irq_byname(pdev, "handover"); + if (q6v5->handover_irq == -EPROBE_DEFER) + return -EPROBE_DEFER; + ret = devm_request_threaded_irq(&pdev->dev, q6v5->handover_irq, NULL, q6v5_handover_interrupt, IRQF_TRIGGER_RISING | IRQF_ONESHOT, @@ -229,6 +238,9 @@ int qcom_q6v5_init(struct qcom_q6v5 *q6v5, struct platform_device *pdev, disable_irq(q6v5->handover_irq); q6v5->stop_irq = platform_get_irq_byname(pdev, "stop-ack"); + if (q6v5->stop_irq == -EPROBE_DEFER) + return -EPROBE_DEFER; + ret = devm_request_threaded_irq(&pdev->dev, q6v5->stop_irq, NULL, q6v5_stop_interrupt, IRQF_TRIGGER_RISING | IRQF_ONESHOT, -- 2.18.0

7 years, 1 month

2
2
0 0

[PATCH] jbd2: Fix use after free in jbd2_log_do_checkpoint()

by Jan Kara

The code cleaning transaction's lists of checkpoint buffers has a bug where it increases bh refcount only after releasing journal->j_list_lock. Thus the following race is possible: CPU0 CPU1 jbd2_log_do_checkpoint() jbd2_journal_try_to_free_buffers() __journal_try_to_free_buffer(bh) ... while (transaction->t_checkpoint_io_list) ... if (buffer_locked(bh)) { <-- IO completes now, buffer gets unlocked --> spin_unlock(&journal->j_list_lock); spin_lock(&journal->j_list_lock); __jbd2_journal_remove_checkpoint(jh); spin_unlock(&journal->j_list_lock); try_to_free_buffers(page); get_bh(bh) <-- accesses freed bh Fix the problem by grabbing bh reference before unlocking journal->j_list_lock. Fixes: dc6e8d669cf5cb3ff84707c372c0a2a8a5e80845 Fixes: be1158cc615fd723552f0d9912087423c7cadda5 Reported-by: syzbot+7f4a27091759e2fe7453(a)syzkaller.appspotmail.com CC: stable(a)vger.kernel.org Signed-off-by: Jan Kara <jack(a)suse.cz> --- fs/jbd2/checkpoint.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/jbd2/checkpoint.c b/fs/jbd2/checkpoint.c index c125d662777c..26f8d7e46462 100644 --- a/fs/jbd2/checkpoint.c +++ b/fs/jbd2/checkpoint.c @@ -251,8 +251,8 @@ int jbd2_log_do_checkpoint(journal_t *journal) bh = jh2bh(jh); if (buffer_locked(bh)) { - spin_unlock(&journal->j_list_lock); get_bh(bh); + spin_unlock(&journal->j_list_lock); wait_on_buffer(bh); /* the journal_head may have gone by now */ BUFFER_TRACE(bh, "brelse"); @@ -333,8 +333,8 @@ int jbd2_log_do_checkpoint(journal_t *journal) jh = transaction->t_checkpoint_io_list; bh = jh2bh(jh); if (buffer_locked(bh)) { - spin_unlock(&journal->j_list_lock); get_bh(bh); + spin_unlock(&journal->j_list_lock); wait_on_buffer(bh); /* the journal_head may have gone by now */ BUFFER_TRACE(bh, "brelse"); -- 2.16.4

7 years, 1 month

4
8
0 0

[patch 13/14] ocfs2: fix locking for res->tracking and dlm->tracking_list

by akpm＠linux-foundation.org

From: Ashish Samant <ashish.samant(a)oracle.com> Subject: ocfs2: fix locking for res->tracking and dlm->tracking_list In dlm_init_lockres() we access and modify res->tracking and dlm->tracking_list without holding dlm->track_lock. This can cause list corruptions and can end up in kernel panic. Fix this by locking res->tracking and dlm->tracking_list with dlm->track_lock instead of dlm->spinlock. Link: http://lkml.kernel.org/r/1529951192-4686-1-git-send-email-ashish.samant@ora… Signed-off-by: Ashish Samant <ashish.samant(a)oracle.com> Reviewed-by: Changwei Ge <ge.changwei(a)h3c.com> Acked-by: Joseph Qi <jiangqi903(a)gmail.com> Acked-by: Jun Piao <piaojun(a)huawei.com> Cc: Mark Fasheh <mark(a)fasheh.com> Cc: Joel Becker <jlbec(a)evilplan.org> Cc: Junxiao Bi <junxiao.bi(a)oracle.com> Cc: Changwei Ge <ge.changwei(a)h3c.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- fs/ocfs2/dlm/dlmmaster.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/ocfs2/dlm/dlmmaster.c~ocfs2-fix-locking-for-res-tracking-and-dlm-tracking_list +++ a/fs/ocfs2/dlm/dlmmaster.c @@ -584,9 +584,9 @@ static void dlm_init_lockres(struct dlm_ res->last_used = 0; - spin_lock(&dlm->spinlock); + spin_lock(&dlm->track_lock); list_add_tail(&res->tracking, &dlm->tracking_list); - spin_unlock(&dlm->spinlock); + spin_unlock(&dlm->track_lock); memset(res->lvb, 0, DLM_LVB_LEN); memset(res->refmap, 0, sizeof(res->refmap)); _

7 years, 1 month

1
0
0 0

[patch 11/14] mm/vmstat.c: skip NR_TLB_REMOTE_FLUSH* properly

by akpm＠linux-foundation.org

From: Jann Horn <jannh(a)google.com> Subject: mm/vmstat.c: skip NR_TLB_REMOTE_FLUSH* properly 5dd0b16cdaff ("mm/vmstat: Make NR_TLB_REMOTE_FLUSH_RECEIVED available even on UP") made the availability of the NR_TLB_REMOTE_FLUSH* counters inside the kernel unconditional to reduce #ifdef soup, but (either to avoid showing dummy zero counters to userspace, or because that code was missed) didn't update the vmstat_array, meaning that all following counters would be shown with incorrect values. This only affects kernel builds with CONFIG_VM_EVENT_COUNTERS=y && CONFIG_DEBUG_TLBFLUSH=y && CONFIG_SMP=n. Link: http://lkml.kernel.org/r/20181001143138.95119-2-jannh@google.com Fixes: 5dd0b16cdaff ("mm/vmstat: Make NR_TLB_REMOTE_FLUSH_RECEIVED available even on UP") Signed-off-by: Jann Horn <jannh(a)google.com> Reviewed-by: Kees Cook <keescook(a)chromium.org> Reviewed-by: Andrew Morton <akpm(a)linux-foundation.org> Acked-by: Michal Hocko <mhocko(a)suse.com> Acked-by: Roman Gushchin <guro(a)fb.com> Cc: Davidlohr Bueso <dave(a)stgolabs.net> Cc: Oleg Nesterov <oleg(a)redhat.com> Cc: Christoph Lameter <clameter(a)sgi.com> Cc: Kemi Wang <kemi.wang(a)intel.com> Cc: Andy Lutomirski <luto(a)kernel.org> Cc: Ingo Molnar <mingo(a)kernel.org> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/vmstat.c | 3 +++ 1 file changed, 3 insertions(+) --- a/mm/vmstat.c~mm-vmstat-skip-nr_tlb_remote_flush-properly +++ a/mm/vmstat.c @@ -1275,6 +1275,9 @@ const char * const vmstat_text[] = { #ifdef CONFIG_SMP "nr_tlb_remote_flush", "nr_tlb_remote_flush_received", +#else + "", /* nr_tlb_remote_flush */ + "", /* nr_tlb_remote_flush_received */ #endif /* CONFIG_SMP */ "nr_tlb_local_flush_all", "nr_tlb_local_flush_one", _

7 years, 1 month

1
0
0 0

[patch 09/14] proc: restrict kernel stack dumps to root

by akpm＠linux-foundation.org

From: Jann Horn <jannh(a)google.com> Subject: proc: restrict kernel stack dumps to root Currently, you can use /proc/self/task/*/stack to cause a stack walk on a task you control while it is running on another CPU. That means that the stack can change under the stack walker. The stack walker does have guards against going completely off the rails and into random kernel memory, but it can interpret random data from your kernel stack as instruction pointers and stack pointers. This can cause exposure of kernel stack contents to userspace. Restrict the ability to inspect kernel stacks of arbitrary tasks to root in order to prevent a local attacker from exploiting racy stack unwinding to leak kernel task stack contents. See the added comment for a longer rationale. There don't seem to be any users of this userspace API that can't gracefully bail out if reading from the file fails. Therefore, I believe that this change is unlikely to break things. In the case that this patch does end up needing a revert, the next-best solution might be to fake a single-entry stack based on wchan. Link: http://lkml.kernel.org/r/20180927153316.200286-1-jannh@google.com Fixes: 2ec220e27f50 ("proc: add /proc/*/stack") Signed-off-by: Jann Horn <jannh(a)google.com> Acked-by: Kees Cook <keescook(a)chromium.org> Cc: Alexey Dobriyan <adobriyan(a)gmail.com> Cc: Ken Chen <kenchen(a)google.com> Cc: Will Deacon <will.deacon(a)arm.com> Cc: Laura Abbott <labbott(a)redhat.com> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Ingo Molnar <mingo(a)redhat.com> Cc: "H . Peter Anvin" <hpa(a)zytor.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- fs/proc/base.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) --- a/fs/proc/base.c~proc-restrict-kernel-stack-dumps-to-root +++ a/fs/proc/base.c @@ -407,6 +407,20 @@ static int proc_pid_stack(struct seq_fil unsigned long *entries; int err; + /* + * The ability to racily run the kernel stack unwinder on a running task + * and then observe the unwinder output is scary; while it is useful for + * debugging kernel issues, it can also allow an attacker to leak kernel + * stack contents. + * Doing this in a manner that is at least safe from races would require + * some work to ensure that the remote task can not be scheduled; and + * even then, this would still expose the unwinder as local attack + * surface. + * Therefore, this interface is restricted to root. + */ + if (!file_ns_capable(m->file, &init_user_ns, CAP_SYS_ADMIN)) + return -EACCES; + entries = kmalloc_array(MAX_STACK_TRACE_DEPTH, sizeof(*entries), GFP_KERNEL); if (!entries) _

7 years, 1 month

1
0
0 0

[patch 04/14] mm, thp: fix mlocking THP page with migration enabled

by akpm＠linux-foundation.org

From: "Kirill A. Shutemov" <kirill.shutemov(a)linux.intel.com> Subject: mm, thp: fix mlocking THP page with migration enabled A transparent huge page is represented by a single entry on an LRU list. Therefore, we can only make unevictable an entire compound page, not individual subpages. If a user tries to mlock() part of a huge page, we want the rest of the page to be reclaimable. We handle this by keeping PTE-mapped huge pages on normal LRU lists: the PMD on border of VM_LOCKED VMA will be split into PTE table. Introduction of THP migration breaks[1] the rules around mlocking THP pages. If we had a single PMD mapping of the page in mlocked VMA, the page will get mlocked, regardless of PTE mappings of the page. For tmpfs/shmem it's easy to fix by checking PageDoubleMap() in remove_migration_pmd(). Anon THP pages can only be shared between processes via fork(). Mlocked page can only be shared if parent mlocked it before forking, otherwise CoW will be triggered on mlock(). For Anon-THP, we can fix the issue by munlocking the page on removing PTE migration entry for the page. PTEs for the page will always come after mlocked PMD: rmap walks VMAs from oldest to newest. Test-case: #include <unistd.h> #include <sys/mman.h> #include <sys/wait.h> #include <linux/mempolicy.h> #include <numaif.h> int main(void) { unsigned long nodemask = 4; void *addr; addr = mmap((void *)0x20000000UL, 2UL << 20, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, -1, 0); if (fork()) { wait(NULL); return 0; } mlock(addr, 4UL << 10); mbind(addr, 2UL << 20, MPOL_PREFERRED | MPOL_F_RELATIVE_NODES, &nodemask, 4, MPOL_MF_MOVE); return 0; } [1] https://lkml.kernel.org/r/CAOMGZ=G52R-30rZvhGxEbkTw7rLLwBGadVYeo--iizcD3upL… Link: http://lkml.kernel.org/r/20180917133816.43995-1-kirill.shutemov@linux.intel… Fixes: 616b8371539a ("mm: thp: enable thp migration in generic path") Signed-off-by: Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> Reported-by: Vegard Nossum <vegard.nossum(a)oracle.com> Reviewed-by: Zi Yan <zi.yan(a)cs.rutgers.edu> Cc: Naoya Horiguchi <n-horiguchi(a)ah.jp.nec.com> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: Andrea Arcangeli <aarcange(a)redhat.com> Cc: <stable(a)vger.kernel.org> [4.14+] Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- mm/huge_memory.c | 2 +- mm/migrate.c | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) --- a/mm/huge_memory.c~mm-thp-fix-mlocking-thp-page-with-migration-enabled +++ a/mm/huge_memory.c @@ -2931,7 +2931,7 @@ void remove_migration_pmd(struct page_vm else page_add_file_rmap(new, true); set_pmd_at(mm, mmun_start, pvmw->pmd, pmde); - if (vma->vm_flags & VM_LOCKED) + if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new)) mlock_vma_page(new); update_mmu_cache_pmd(vma, address, pvmw->pmd); } --- a/mm/migrate.c~mm-thp-fix-mlocking-thp-page-with-migration-enabled +++ a/mm/migrate.c @@ -275,6 +275,9 @@ static bool remove_migration_pte(struct if (vma->vm_flags & VM_LOCKED && !PageTransCompound(new)) mlock_vma_page(new); + if (PageTransHuge(page) && PageMlocked(page)) + clear_page_mlock(page); + /* No need to invalidate - it was non-present before */ update_mmu_cache(vma, pvmw.address, pvmw.pte); } _

7 years, 1 month

1
0
0 0

[patch 01/14] mm: migration: fix migration of huge PMD shared pages

by akpm＠linux-foundation.org

From: Mike Kravetz <mike.kravetz(a)oracle.com> Subject: mm: migration: fix migration of huge PMD shared pages The page migration code employs try_to_unmap() to try and unmap the source page. This is accomplished by using rmap_walk to find all vmas where the page is mapped. This search stops when page mapcount is zero. For shared PMD huge pages, the page map count is always 1 no matter the number of mappings. Shared mappings are tracked via the reference count of the PMD page. Therefore, try_to_unmap stops prematurely and does not completely unmap all mappings of the source page. This problem can result is data corruption as writes to the original source page can happen after contents of the page are copied to the target page. Hence, data is lost. This problem was originally seen as DB corruption of shared global areas after a huge page was soft offlined due to ECC memory errors. DB developers noticed they could reproduce the issue by (hotplug) offlining memory used to back huge pages. A simple testcase can reproduce the problem by creating a shared PMD mapping (note that this must be at least PUD_SIZE in size and PUD_SIZE aligned (1GB on x86)), and using migrate_pages() to migrate process pages between nodes while continually writing to the huge pages being migrated. To fix, have the try_to_unmap_one routine check for huge PMD sharing by calling huge_pmd_unshare for hugetlbfs huge pages. If it is a shared mapping it will be 'unshared' which removes the page table entry and drops the reference on the PMD page. After this, flush caches and TLB. mmu notifiers are called before locking page tables, but we can not be sure of PMD sharing until page tables are locked. Therefore, check for the possibility of PMD sharing before locking so that notifiers can prepare for the worst possible case. Link: http://lkml.kernel.org/r/20180823205917.16297-2-mike.kravetz@oracle.com [mike.kravetz(a)oracle.com: make _range_in_vma() a static inline] Link: http://lkml.kernel.org/r/6063f215-a5c8-2f0c-465a-2c515ddc952d@oracle.com Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz <mike.kravetz(a)oracle.com> Acked-by: Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> Reviewed-by: Naoya Horiguchi <n-horiguchi(a)ah.jp.nec.com> Acked-by: Michal Hocko <mhocko(a)suse.com> Cc: Vlastimil Babka <vbabka(a)suse.cz> Cc: Davidlohr Bueso <dave(a)stgolabs.net> Cc: Jerome Glisse <jglisse(a)redhat.com> Cc: Mike Kravetz <mike.kravetz(a)oracle.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> --- include/linux/hugetlb.h | 14 ++++++++++++ include/linux/mm.h | 6 +++++ mm/hugetlb.c | 37 +++++++++++++++++++++++++++++++-- mm/rmap.c | 42 +++++++++++++++++++++++++++++++++++--- 4 files changed, 94 insertions(+), 5 deletions(-) --- a/include/linux/hugetlb.h~mm-migration-fix-migration-of-huge-pmd-shared-pages +++ a/include/linux/hugetlb.h @@ -140,6 +140,8 @@ pte_t *huge_pte_alloc(struct mm_struct * pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long sz); int huge_pmd_unshare(struct mm_struct *mm, unsigned long *addr, pte_t *ptep); +void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, + unsigned long *start, unsigned long *end); struct page *follow_huge_addr(struct mm_struct *mm, unsigned long address, int write); struct page *follow_huge_pd(struct vm_area_struct *vma, @@ -170,6 +172,18 @@ static inline unsigned long hugetlb_tota return 0; } +static inline int huge_pmd_unshare(struct mm_struct *mm, unsigned long *addr, + pte_t *ptep) +{ + return 0; +} + +static inline void adjust_range_if_pmd_sharing_possible( + struct vm_area_struct *vma, + unsigned long *start, unsigned long *end) +{ +} + #define follow_hugetlb_page(m,v,p,vs,a,b,i,w,n) ({ BUG(); 0; }) #define follow_huge_addr(mm, addr, write) ERR_PTR(-EINVAL) #define copy_hugetlb_page_range(src, dst, vma) ({ BUG(); 0; }) --- a/include/linux/mm.h~mm-migration-fix-migration-of-huge-pmd-shared-pages +++ a/include/linux/mm.h @@ -2455,6 +2455,12 @@ static inline struct vm_area_struct *fin return vma; } +static inline bool range_in_vma(struct vm_area_struct *vma, + unsigned long start, unsigned long end) +{ + return (vma && vma->vm_start <= start && end <= vma->vm_end); +} + #ifdef CONFIG_MMU pgprot_t vm_get_page_prot(unsigned long vm_flags); void vma_set_page_prot(struct vm_area_struct *vma); --- a/mm/hugetlb.c~mm-migration-fix-migration-of-huge-pmd-shared-pages +++ a/mm/hugetlb.c @@ -4545,13 +4545,41 @@ static bool vma_shareable(struct vm_area /* * check on proper vm_flags and page table alignment */ - if (vma->vm_flags & VM_MAYSHARE && - vma->vm_start <= base && end <= vma->vm_end) + if (vma->vm_flags & VM_MAYSHARE && range_in_vma(vma, base, end)) return true; return false; } /* + * Determine if start,end range within vma could be mapped by shared pmd. + * If yes, adjust start and end to cover range associated with possible + * shared pmd mappings. + */ +void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, + unsigned long *start, unsigned long *end) +{ + unsigned long check_addr = *start; + + if (!(vma->vm_flags & VM_MAYSHARE)) + return; + + for (check_addr = *start; check_addr < *end; check_addr += PUD_SIZE) { + unsigned long a_start = check_addr & PUD_MASK; + unsigned long a_end = a_start + PUD_SIZE; + + /* + * If sharing is possible, adjust start/end if necessary. + */ + if (range_in_vma(vma, a_start, a_end)) { + if (a_start < *start) + *start = a_start; + if (a_end > *end) + *end = a_end; + } + } +} + +/* * Search for a shareable pmd page for hugetlb. In any case calls pmd_alloc() * and returns the corresponding pte. While this is not necessary for the * !shared pmd case because we can allocate the pmd later as well, it makes the @@ -4648,6 +4676,11 @@ int huge_pmd_unshare(struct mm_struct *m { return 0; } + +void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, + unsigned long *start, unsigned long *end) +{ +} #define want_pmd_share() (0) #endif /* CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ --- a/mm/rmap.c~mm-migration-fix-migration-of-huge-pmd-shared-pages +++ a/mm/rmap.c @@ -1362,11 +1362,21 @@ static bool try_to_unmap_one(struct page } /* - * We have to assume the worse case ie pmd for invalidation. Note that - * the page can not be free in this function as call of try_to_unmap() - * must hold a reference on the page. + * For THP, we have to assume the worse case ie pmd for invalidation. + * For hugetlb, it could be much worse if we need to do pud + * invalidation in the case of pmd sharing. + * + * Note that the page can not be free in this function as call of + * try_to_unmap() must hold a reference on the page. */ end = min(vma->vm_end, start + (PAGE_SIZE << compound_order(page))); + if (PageHuge(page)) { + /* + * If sharing is possible, start and end will be adjusted + * accordingly. + */ + adjust_range_if_pmd_sharing_possible(vma, &start, &end); + } mmu_notifier_invalidate_range_start(vma->vm_mm, start, end); while (page_vma_mapped_walk(&pvmw)) { @@ -1409,6 +1419,32 @@ static bool try_to_unmap_one(struct page subpage = page - page_to_pfn(page) + pte_pfn(*pvmw.pte); address = pvmw.address; + if (PageHuge(page)) { + if (huge_pmd_unshare(mm, &address, pvmw.pte)) { + /* + * huge_pmd_unshare unmapped an entire PMD + * page. There is no way of knowing exactly + * which PMDs may be cached for this mm, so + * we must flush them all. start/end were + * already adjusted above to cover this range. + */ + flush_cache_range(vma, start, end); + flush_tlb_range(vma, start, end); + mmu_notifier_invalidate_range(mm, start, end); + + /* + * The ref count of the PMD page was dropped + * which is part of the way map counting + * is done for shared PMDs. Return 'true' + * here. When there is no other sharing, + * huge_pmd_unshare returns false and we will + * unmap the actual page and drop map count + * to zero. + */ + page_vma_mapped_walk_done(&pvmw); + break; + } + } if (IS_ENABLED(CONFIG_MIGRATION) && (flags & TTU_MIGRATION) && _

7 years, 1 month

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror