From: Hugh Dickins hughd@google.com Subject: mm/khugepaged: collapse_shmem() do not crash on Compound

collapse_shmem()'s VM_BUG_ON_PAGE(PageTransCompound) was unsafe: before it holds page lock of the first page, racing truncation then extension might conceivably have inserted a hugepage there already. Fail with the SCAN_PAGE_COMPOUND result, instead of crashing (CONFIG_DEBUG_VM=y) or otherwise mishandling the unexpected hugepage - though later we might code up a more constructive way of handling it, with SCAN_SUCCESS.

Link: http://lkml.kernel.org/r/alpine.LSU.2.11.1811261529310.2275@eggly.anvils Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins hughd@google.com Cc: Kirill A. Shutemov kirill.shutemov@linux.intel.com Cc: Jerome Glisse jglisse@redhat.com Cc: Konstantin Khlebnikov khlebnikov@yandex-team.ru Cc: Matthew Wilcox willy@infradead.org Cc: stable@vger.kernel.org [4.8+] Signed-off-by: Andrew Morton akpm@linux-foundation.org ---

mm/khugepaged.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)

--- a/mm/khugepaged.c~mm-khugepaged-collapse_shmem-do-not-crash-on-compound +++ a/mm/khugepaged.c @@ -1399,7 +1399,15 @@ static void collapse_shmem(struct mm_str */ VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(!PageUptodate(page), page); - VM_BUG_ON_PAGE(PageTransCompound(page), page); + + /* + * If file was truncated then extended, or hole-punched, before + * we locked the first page, then a THP might be there already. + */ + if (PageTransCompound(page)) { + result = SCAN_PAGE_COMPOUND; + goto out_unlock; + }

if (page_mapping(page) != mapping) { result = SCAN_TRUNCATED; _