在 2024/7/3 20:02, David Hildenbrand 写道:
On 22.06.24 08:48, yangge1116@126.com wrote:
From: yangge yangge1116@126.com
If a large number of CMA memory are configured in system (for example, the CMA memory accounts for 50% of the system memory), starting a virtual virtual machine, it will call pin_user_pages_remote(..., FOLL_LONGTERM, ...) to pin memory. Normally if a page is present and in CMA area, pin_user_pages_remote() will migrate the page from CMA area to non-CMA area because of FOLL_LONGTERM flag. But the current code will cause the migration failure due to unexpected page refcounts, and eventually cause the virtual machine fail to start.
If a page is added in LRU batch, its refcount increases one, remove the page from LRU batch decreases one. Page migration requires the page is not referenced by others except page mapping. Before migrating a page, we should try to drain the page from LRU batch in case the page is in it, however, folio_test_lru() is not sufficient to tell whether the page is in LRU batch or not, if the page is in LRU batch, the migration will fail.
To solve the problem above, we modify the logic of adding to LRU batch. Before adding a page to LRU batch, we clear the LRU flag of the page so that we can check whether the page is in LRU batch by folio_test_lru(page). Seems making the LRU flag of the page invisible a long time is no problem, because a new page is allocated from buddy and added to the lru batch, its LRU flag is also not visible for a long time.
I think we need to describe the impact of this change in a better way. This example here is certainly interesting, but if pages are new they are also not candidate for immediate reclaim (tail of the LRU).
The positive thing is that we can more reliably identify pages that are on an LRU batch.
Further, a page can now only be on exactly one LRU batch.
But, as long as a page is on a LRU batch, we cannot isolate it, and we cannot check if it's an LRU page. The latter can currently already happen for a shorter time when moving LRU pages, and temporarily clearing the flag.
I shared some examples where we don't care, because we'd check for additional folio references either way (and the one from the LRU batch).
But I think we have to identify if there are any LRU folio/page checks that could now be impacted "more". At least we should document it properly to better understand the possible impact (do we maybe have to flush more often?).
Thanks. I have reviewed a lot of paths using LRU folio/page checks and haven't seen more impact. I will documnt possible impact in next version, thanks.