On Wed, Dec 11, 2024 at 11:53 AM SeongJae Park sj@kernel.org wrote:
On Fri, 6 Dec 2024 11:57:55 -0800 Yuanchu Xie yuanchu@google.com wrote:
Thanks for the response Johannes. Some replies inline.
On Tue, Nov 26, 2024 at 11:26\u202fPM Johannes Weiner hannes@cmpxchg.org wrote:
On Tue, Nov 26, 2024 at 06:57:19PM -0800, Yuanchu Xie wrote:
This patch series provides workingset reporting of user pages in lruvecs, of which coldness can be tracked by accessed bits and fd references. However, the concept of workingset applies generically to all types of memory, which could be kernel slab caches, discardable userspace caches (databases), or CXL.mem. Therefore, data sources might come from slab shrinkers, device drivers, or the userspace. Another interesting idea might be hugepage workingset, so that we can measure the proportion of hugepages backing cold memory. However, with architectures like arm, there may be too many hugepage sizes leading to a combinatorial explosion when exporting stats to the userspace. Nonetheless, the kernel should provide a set of workingset interfaces that is generic enough to accommodate the various use cases, and extensible to potential future use cases.
Doesn't DAMON already provide this information?
CCing SJ.
Thanks for the CC. DAMON was really good at visualizing the memory access frequencies last time I tried it out!
Thank you for this kind acknowledgement, Yuanchu!
For server use cases, DAMON would benefit from integrations with cgroups. The key then would be a standard interface for exporting a cgroup's working set to the user.
I show two ways to make DAMON supports cgroups for now. First way is making another DAMON operations set implementation for cgroups. I shared a rough idea for this before, probably on kernel summit. But I haven't had a chance to prioritize this so far. Please let me know if you need more details. The second way is extending DAMOS filter to provide more detailed statistics per DAMON-region, and adding another DAMOS action that does nothing but only accounting the detailed statistics. Using the new DAMOS action, users will be able to know how much of specific DAMON-found regions are filtered out by the given filter. Because we have DAMOS filter type for cgroups, we can know how much of workingset (or, warm memory) belongs to specific groups. This can be applied to not only cgroups, but for any DAMOS filter types that exist (e.g., anonymous page, young page).
I believe the second way is simpler to implement while providing information that sufficient for most possible use cases. I was anyway planning to do this.
For a container orchestrator like kubernetes, the node agents need to be able to gather the working set stats at a per-job level. Some jobs can create sub-hierarchies as well, so it's important that we have hierarchical stats.
Do you think it's a good idea to integrate DAMON to provide some aggregate stats in a memory controller file? With the DAMOS cgroup filter, there can be some kind of interface that a DAMOS action or the damo tool could call into. I feel that would be a straightforward and integrated way to support cgroups.
Yuanchu