Shared-memory allocator services (e.g., Android's gralloc) allocate dma-bufs on behalf of client processes in different cgroups. Today those allocations are charged to the allocator's own cgroup, leaving client cgroups with no visibility into their actual memory consumption and defeating resource limits.
This series adds memory cgroup accounting for dma-buf allocations, building on T.J. Mercier's earlier series [1] and incorporating feedback from previous versions and LPC discussions [2].
Patch 1 introduces the MEMCG_DMABUF stat and the obj_cgroup_reserve_dmabuf()/obj_cgroup_cancel_dmabuf() and mem_cgroup_commit_dmabuf()/mem_cgroup_uncharge_dmabuf() helpers in mm/memcontrol.c
Patch 2 wires up the charging policy: dma_buf_export() stores exp_info.objcg and updates the stat, a new charge_fd field in DMA_HEAP_IOCTL_ALLOC accepts a pidfd or cgroupfd to identify the target cgroup, and __GFP_ACCOUNT is removed from system-heap allocations to avoid double counting.
Patch 3 adds DMA_BUF_IOCTL_XFER_CHARGE, a general ioctl on any dma-buf fd to transfer charges post-allocation. This covers the double-hop allocator case (SurfaceFlinger -> gralloc -> kernel) and works for buffers from any exporter that opts in. Charging is gated by dmabuf->objcg being set at export time, ensuring that transfer is only applied for buffers backed by system memory where size reflects actual bytes. If the receiving cgroup lacks capacity for the charge, the ioctl returns -ENOMEM and the original charge is preserved.
Patches 4-7 add access control for cross-cgroup charging. Letting userspace decide who to charge needs to be a privileged operation because it has material impact on OOM decision and resource limits. Charging outside the caller's own memcg subtree requires CAP_SYS_RESOURCE. On top of that, LSM hooks gate the two charge_fd flavours: security_memcg_charge_task() gates the pidfd path (target identified by task credentials) and security_memcg_charge_cgroup() gates the cgroupfd path (target identified by cgroup directory label), with SELinux implementations using process2:charge_to and cgroup:charge_to permissions.
Patch 8 adds selftests. The tests and vmtest script were mostly written by an LLM, closely following the style of the surrounding tests and the virtme-ng setup. All code has been reviewed and tested.
Known limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. - Only memcg accounting. CMA heaps would additionally need dmem controller integration. - If a transfer of an uncharged buffer fails, the buffer remains charged to the original cgroup. The allocator (userspace) is expected to handle this (retry, free, ...). - The LSM hooks are intentionally narrow. Broader cgroup access controls (e.g. for the memfd/kexec handover case [2]) could be a separate effort reusing the same pattern. - Only system heap sets exp_info.objcg today. Other exporters (DRM, V4L2, etc.) can opt in by setting it, but are not covered by this series. - Related to the above: lazy backing-store allocation is not supported. This can be extended later, e.g., by adding a flag on the dma-buf struct for explicitly marking when memory is committed.
Note on TTM/GPU accounting [3]: that effort charges system memory allocated for GPU buffers per page, with its own stats (gpu_active and gpu_reclaim). This series charges at the dma-buf level, for memory with no memory.stat category of its own; exporters whose memory is already charged by their allocator must not opt in, so the two do not overlap.
[1] https://lore.kernel.org/cgroups/20230109213809.418135-1-tjmercier@google.com... [2] https://lpc.events/event/20/contributions/2416/ [3] https://lore.kernel.org/all/20260706052330.1110909-1-airlied@gmail.com/
Signed-off-by: Albert Esteve aesteve@redhat.com --- Changes in v5: - Reject dma-buf reservations above INT_MAX pages to avoid nr_pages truncation (Sashiko) - Require CAP_SYS_RESOURCE for charge requests targeting outside caller's own memcg subtree (Sashiko) - Hold an obj_cgroup instead of a memcg in the dma-buf struct (Thomas Hellström) - Restore mem_cgroup_commit_dmabuf() so dma_buf_export() does not reach into memcg stat internals - Resolve the cgroup and run the LSM check against the same file in get_obj_cgroup_from_fd() - The cgroupfd hook now takes the file and checks the inode SID, so genfscon-labeled cgroups are covered - Rename the LSM hooks to security_memcg_charge_{task,cgroup}() - Replace dma_buf object class in SELinux with process2:charge_to and cgroup:charge_to - Link to v4: https://lore.kernel.org/r/20261104-v2_20230123_tjmercier_google_com-v4-0-181...
Changes in v4: - Remove memcg_account flag; exp_info.memcg != NULL is the opt-in - Centralize memcg bookkeeping in dma_buf_export() - Drop mem_cgroup_commit_dmabuf() - Make mem_cgroup_transfer_charge() generic (not dma-buf specific) - Add EXPORT_SYMBOL_GPL for reserve/cancel (system_heap is tristate) - Reorder: move xfer ioctl before LSM patches - Serialize dmabuf->memcg updates with the dma-buf resv lock instead of cmpxchg - Link to v3: https://lore.kernel.org/r/20261027-v2_20230123_tjmercier_google_com-v3-0-78d...
Changes in v3: - Split mem_cgroup_charge_dmabuf() into reserve/commit/cancel three-phase API - Extract mem_cgroup_get_from_fd() helper shared by allocate and xfer paths - Rename __padding to flags in struct dma_buf_transfer_charge - Fix UAF race: move mod_memcg_state() before cmpxchg in transfer - Add memcg_account opt-in flag; xfer returns -EOPNOTSUPP if unset - Wrap dma_buf_charge_cgroup hook in #ifdef CONFIG_CGROUPS - Fix TOCTOU in SELinux charge_cgroup xattr length handling - Link to v2: https://lore.kernel.org/r/20261021-v2_20230123_tjmercier_google_com-v2-0-ef6...
Changes in v2: - Move charge_pid_fd to charge_fd, accepting cgroupfd in addition to pidfd per Christian König. - New DMA_BUF_IOCTL_XFER_CHARGE ioctl for post-allocation charge transfer on any dma-buf, not just dma-heap buffers per Christian König and T.J. Mercier. - Moved __GFP_ACCOUNT removal to patch 1 per Barry Song. - dma-buf charge/uncharge helpers take struct dma_buf * directly to avoid the need for caller to manage size calculations. - Added test_xfer_charge_cgroupfd_self to dmabuf-heap selftests. - Added LSM and SELinux hooks for cgroupfd-based charging. - Rebased to v7.3-rc7. - Link to v1: https://lore.kernel.org/r/20260512-v2_20230123_tjmercier_google_com-v1-0-632...
--- Albert Esteve (7): dma-heap: charge dma-buf memory via explicit memcg dma-buf: transfer objcg charges via DMA_BUF_IOCTL_XFER_CHARGE lsm: Add memcg_charge_task LSM hook lsm: Add memcg_charge_cgroup LSM hook selinux: Restrict cross-cgroup memcg charging selinux: Restrict memcg charging to a cgroupfd target selftests/dmabuf-heaps: Add dma-buf memcg accounting tests
T.J. Mercier (1): memcg: Add MEMCG_DMABUF accounting infrastructure
Documentation/admin-guide/cgroup-v2.rst | 6 + drivers/dma-buf/dma-buf.c | 45 +++ drivers/dma-buf/dma-heap.c | 27 +- drivers/dma-buf/heaps/cma_heap.c | 3 +- drivers/dma-buf/heaps/system_heap.c | 19 +- drivers/tee/tee_heap.c | 3 +- include/linux/dma-buf.h | 10 + include/linux/dma-heap.h | 4 +- include/linux/lsm_hook_defs.h | 2 + include/linux/memcontrol.h | 56 +++ include/linux/security.h | 15 + include/uapi/linux/dma-buf.h | 26 ++ include/uapi/linux/dma-heap.h | 7 + mm/memcontrol.c | 271 +++++++++++++ security/security.c | 35 ++ security/selinux/hooks.c | 17 + security/selinux/include/classmap.h | 3 +- tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 447 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 237 ++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 +++++++++++ 22 files changed, 1446 insertions(+), 16 deletions(-) --- base-commit: 05441d697f7f9a927264f4058cbbc6a03dfcac9d change-id: 20260508-v2_20230123_tjmercier_google_com-f44fcfb16530
Best regards, -- Albert Esteve aesteve@redhat.com
From: "T.J. Mercier" tjmercier@google.com
Introduce a MEMCG_DMABUF stat counter and helpers for charging, cancelling, and uncharging dma-buf memory against a memory cgroup.
The charge is tracked on struct dma_buf via a new objcg pointer. The object cgroup reference is stored into (or cleared from) dmabuf->objcg by the helpers, removing the need for callers to manage those details. The release path in dma_buf_release() is reduced to a single mem_cgroup_uncharge_dmabuf() call.
obj_cgroup_reserve_dmabuf() takes nr_pages as unsigned long and rejects values above INT_MAX. This differs from the rest of memcg API. Specifically, dma-buf sizes can be userspace-controlled and long-sized (size_t), while memcg charge and stat updates are int-sized. Charges that pass the reservation are therefore safe to handle as unsigned int in the export, transfer, and uncharge paths.
Rather than charging at the page level via __GFP_ACCOUNT, which would require exporters to expose their backing pages for cross-cgroup transfer via new dma_buf_ops, charges are tracked externally on struct dma_buf through a single accounting path.
The next patch wires up the charging policy in the dma-heap ioctl.
Signed-off-by: T.J. Mercier tjmercier@google.com Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 4 ++ drivers/dma-buf/dma-buf.c | 3 + include/linux/dma-buf.h | 4 ++ include/linux/memcontrol.h | 32 +++++++++ mm/memcontrol.c | 112 ++++++++++++++++++++++++++++++++ 5 files changed, 155 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 9e3c67b02ae2..f1f6808c5600 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1666,6 +1666,10 @@ The following nested keys are defined. Amount of memory used for storing in-kernel data structures.
+ dmabuf (npn) + Amount of memory used for exported DMA buffers allocated by the cgroup. + Stays with the allocating cgroup regardless of how the buffer is shared. + workingset_refault_anon Number of refaults of previously evicted anonymous pages.
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 36cdc342627e..06462faf762e 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -14,6 +14,7 @@ #include <linux/fs.h> #include <linux/slab.h> #include <linux/dma-buf.h> +#include <linux/memcontrol.h> #include <linux/dma-fence.h> #include <linux/dma-fence-unwrap.h> #include <linux/anon_inodes.h> @@ -182,6 +183,8 @@ static void dma_buf_release(struct dentry *dentry)
dmabuf->ops->release(dmabuf);
+ mem_cgroup_uncharge_dmabuf(dmabuf); + if (dmabuf->resv == (struct dma_resv *)&dmabuf[1]) dma_resv_fini(dmabuf->resv);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index d15b2b31d3c9..2f56afd1f830 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -27,6 +27,7 @@ struct device; struct dma_buf; struct dma_buf_attachment; +struct obj_cgroup;
/** * struct dma_buf_ops - operations possible on struct dma_buf @@ -429,6 +430,9 @@ struct dma_buf {
__poll_t active; } cb_in, cb_out; + + /** @objcg: object cgroup to which this buffer is currently attributed */ + struct obj_cgroup *objcg; };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 69e1bcc67c58..46fbb418bf39 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -10,6 +10,9 @@
#ifndef _LINUX_MEMCONTROL_H #define _LINUX_MEMCONTROL_H + +struct dma_buf; + #include <linux/cgroup.h> #include <linux/vm_event_item.h> #include <linux/hardirq.h> @@ -41,6 +44,7 @@ enum memcg_stat_item { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, MEMCG_NR_STAT, };
@@ -688,6 +692,9 @@ int mem_cgroup_charge_hugetlb(struct folio *folio, struct mm_struct *mm, int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, struct mm_struct *mm, gfp_t gfp);
+bool obj_cgroup_reserve_dmabuf(struct obj_cgroup *objcg, gfp_t gfp, + unsigned long nr_pages); + void __mem_cgroup_uncharge(struct folio *folio);
/** @@ -703,6 +710,11 @@ static inline void mem_cgroup_uncharge(struct folio *folio) __mem_cgroup_uncharge(folio); }
+void obj_cgroup_cancel_dmabuf(struct obj_cgroup *objcg, unsigned long nr_pages); + +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf); +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf); + void __mem_cgroup_uncharge_folios(struct folio_batch *folios); static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { @@ -1203,10 +1215,30 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; }
+static inline bool obj_cgroup_reserve_dmabuf(struct obj_cgroup *objcg, + gfp_t gfp, + unsigned long nr_pages) +{ + return true; +} + +static inline void obj_cgroup_cancel_dmabuf(struct obj_cgroup *objcg, + unsigned long nr_pages) +{ +} + +static inline void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf) +{ +} + static inline void mem_cgroup_uncharge(struct folio *folio) { }
+static inline void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ +} + static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a05d9a8baa1b..45e82cda7c44 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -70,6 +70,7 @@ #include <net/ip.h> #include "slab.h" #include "memcontrol-v1.h" +#include <linux/dma-buf.h>
#include <linux/uaccess.h>
@@ -459,6 +460,7 @@ static const unsigned int memcg_stat_items[] = { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, };
#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) @@ -1644,6 +1646,7 @@ static const struct memory_stat memory_stats[] = { #ifdef CONFIG_HUGETLB_PAGE { "hugetlb", NR_HUGETLB }, #endif + { "dmabuf", MEMCG_DMABUF },
/* The memory events */ { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, @@ -5821,6 +5824,115 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); }
+/** + * obj_cgroup_reserve_dmabuf - charge an object cgroup for a dma-buf allocation + * @objcg: object cgroup to charge (may be %NULL) + * @gfp_mask: reclaim mode + * @nr_pages: number of pages to charge + * + * Charges @nr_pages against @objcg. Call this before allocating pages so + * the cgroup limit is enforced up front. On success the caller must + * finalize the charge with the MEMCG_DMABUF stat update, or call + * obj_cgroup_cancel_dmabuf() to undo it. + * + * @nr_pages is derived from the buffer size, which may be controlled by + * userspace, so it is taken as unsigned long and rejected above what + * a single memcg charge can account. + * + * Returns %true on success, %false if @nr_pages is too large or the + * charge would exceed the limit. + */ +bool obj_cgroup_reserve_dmabuf(struct obj_cgroup *objcg, gfp_t gfp_mask, + unsigned long nr_pages) +{ + struct mem_cgroup *memcg; + int ret; + + if (!objcg || mem_cgroup_disabled()) + return true; + + /* a single memcg charge accounts at most INT_MAX pages */ + if (nr_pages > INT_MAX) + return false; + + memcg = get_mem_cgroup_from_objcg(objcg); + ret = try_charge(memcg, gfp_mask, nr_pages); + mem_cgroup_put(memcg); + return !ret; +} +EXPORT_SYMBOL_GPL(obj_cgroup_reserve_dmabuf); + +/** + * obj_cgroup_cancel_dmabuf - undo a dma-buf charge + * @objcg: object cgroup to uncharge (may be %NULL) + * @nr_pages: same value passed to obj_cgroup_reserve_dmabuf() + * + * Returns @nr_pages of charge to @objcg. Use when the allocation fails + * after obj_cgroup_reserve_dmabuf(). + */ +void obj_cgroup_cancel_dmabuf(struct obj_cgroup *objcg, unsigned long nr_pages) +{ + struct mem_cgroup *memcg; + + if (!objcg) + return; + + memcg = get_mem_cgroup_from_objcg(objcg); + if (!mem_cgroup_is_root(memcg)) + refill_stock(memcg, nr_pages); + mem_cgroup_put(memcg); +} +EXPORT_SYMBOL_GPL(obj_cgroup_cancel_dmabuf); + +/** + * mem_cgroup_commit_dmabuf - finalize a dma-buf charge + * @dmabuf: buffer whose charge is being finalized + * + * Adds @dmabuf's size to the %MEMCG_DMABUF stat of the cgroup its object + * cgroup currently resolves to. Call after obj_cgroup_reserve_dmabuf() + * succeeded and the buffer is about to be exported. Safe to call when + * @dmabuf->objcg is %NULL. + */ +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf) +{ + struct mem_cgroup *memcg; + + if (!dmabuf->objcg) + return; + + memcg = get_mem_cgroup_from_objcg(dmabuf->objcg); + mod_memcg_state(memcg, MEMCG_DMABUF, + PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT); + mem_cgroup_put(memcg); +} + +/** + * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its object cgroup + * @dmabuf: buffer to uncharge + * + * Reverses a finalized dma-buf charge. Drops the object cgroup + * reference stored in @dmabuf->objcg and clears the pointer. + * Safe to call when @dmabuf->objcg is %NULL. + */ +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ + struct obj_cgroup *objcg = dmabuf->objcg; + struct mem_cgroup *memcg; + unsigned long nr_pages; + + if (!objcg) + return; + + memcg = get_mem_cgroup_from_objcg(objcg); + nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + mod_memcg_state(memcg, MEMCG_DMABUF, -nr_pages); + if (!mem_cgroup_is_root(memcg)) + refill_stock(memcg, nr_pages); + mem_cgroup_put(memcg); + obj_cgroup_put(objcg); + dmabuf->objcg = NULL; +} + static int __init cgroup_memory(char *s) { char *token;
Both DMA_HEAP_IOCTL_ALLOC and the new DMA_BUF_IOCTL_XFER_CHARGE ioctl accept a charge_fd field that, when set, causes the allocation to be charged to an arbitrary process's cgroup rather than the caller's.
Charging is currently limited to the caller's own memcg subtree unless the caller holds CAP_SYS_RESOURCE. But the capability is all-or-nothing. A process holding it, along with a dma-heap device node or a dma-buf fd can charge unlimited memory to any other process's cgroup, potentially exhausting that cgroup's limit and triggering OOM kills independent of the victim's own activity or privileges.
Add security_memcg_charge_task(), called in get_obj_cgroup_from_fd() when charge_fd refers to another process. The hook receives the credentials of the allocating process (from) and the credentials of the process whose cgroup will be charged (to), giving security modules a controlled enforcement point for cross-cgroup objcg attribution policy.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 1 + include/linux/security.h | 7 +++++++ mm/memcontrol.c | 11 +++++++++++ security/security.c | 15 +++++++++++++++ 4 files changed, 34 insertions(+)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 87d9d06710cb..93c8bb5642f5 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -43,6 +43,7 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, const kernel_cap_t *permitted) LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) +LSM_HOOK(int, 0, memcg_charge_task, const struct cred *from, const struct cred *to) LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index 40831fab5470..b21823225ed0 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -352,6 +352,7 @@ int security_capable(const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts); +int security_memcg_charge_task(const struct cred *from, const struct cred *to); int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -710,6 +711,12 @@ static inline int security_capable(const struct cred *cred, return cap_capable(cred, ns, cap, opts); }
+static inline int security_memcg_charge_task(const struct cred *from, + const struct cred *to) +{ + return 0; +} + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 03cd90d5e623..2e6fcbe14d43 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -60,6 +60,7 @@ #include <linux/lockdep.h> #include <linux/resume_user_mode.h> #include <linux/psi.h> +#include <linux/security.h> #include <linux/seq_buf.h> #include <linux/sched/isolation.h> #include <linux/kmemleak.h> @@ -3198,10 +3199,12 @@ static bool mem_cgroup_may_charge(struct mem_cgroup *memcg) struct obj_cgroup *get_obj_cgroup_from_fd(int fd) { struct cgroup_subsys_state *css; + const struct cred *tcred; struct obj_cgroup *objcg; struct mem_cgroup *memcg; struct task_struct *task; unsigned int pidfd_flags; + int ret;
if (mem_cgroup_disabled()) return ERR_PTR(-EOPNOTSUPP); @@ -3226,6 +3229,14 @@ struct obj_cgroup *get_obj_cgroup_from_fd(int fd) if (IS_ERR(task)) return ERR_CAST(task);
+ tcred = get_task_cred(task); + ret = security_memcg_charge_task(current_cred(), tcred); + put_cred(tcred); + if (ret) { + put_task_struct(task); + return ERR_PTR(ret); + } + rcu_read_lock(); css = task_css(task, memory_cgrp_id); put_task_struct(task); diff --git a/security/security.c b/security/security.c index f6583d507395..e198e00b6a40 100644 --- a/security/security.c +++ b/security/security.c @@ -674,6 +674,21 @@ int security_capable(const struct cred *cred, return call_int_hook(capable, cred, ns, cap, opts); }
+/** + * security_memcg_charge_task() - Check if cross-cgroup memory charging is allowed + * @from: credentials of the process requesting the charge + * @to: credentials of the process to charge + * + * Check whether the process with credentials @from is allowed to charge + * memory to the cgroup of the process with credentials @to. + * + * Return: Returns 0 if permission is granted. + */ +int security_memcg_charge_task(const struct cred *from, const struct cred *to) +{ + return call_int_hook(memcg_charge_task, from, to); +} + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
Similar to how security_memcg_charge_task adds an access-control point for pidfd for the charge_fd of a DMA_HEAP_IOCTL_ALLOC or DMA_BUF_IOCTL_XFER_CHARGE ioctl, we need a new access-control point for the case where charge_fd represents a cgroupfd.
Add security_memcg_charge_cgroup(), called in get_obj_cgroup_from_fd() when charge_fd refers to a cgroup2. The hook receives the credentials of the requesting process (cred) and the open file of the cgroup directory to charge (file), giving security modules a controlled enforcement point for cross-cgroup memcg attribution policy.
The file is passed instead of the cgroup so that a module can label-check the object the cgroupfd actually names. get_obj_cgroup_from_fd() resolves the memory css from that same file with css_tryget_online_from_dir(), so the fd cannot be swapped between the check and the charge.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 1 + include/linux/security.h | 8 ++++++++ mm/memcontrol.c | 9 ++++++++- security/security.c | 20 ++++++++++++++++++++ 4 files changed, 37 insertions(+), 1 deletion(-)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 93c8bb5642f5..3b254b4e60ec 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -44,6 +44,7 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) LSM_HOOK(int, 0, memcg_charge_task, const struct cred *from, const struct cred *to) +LSM_HOOK(int, 0, memcg_charge_cgroup, const struct cred *cred, const struct file *file) LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index b21823225ed0..d1cd6c324ffd 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -353,6 +353,8 @@ int security_capable(const struct cred *cred, int cap, unsigned int opts); int security_memcg_charge_task(const struct cred *from, const struct cred *to); +int security_memcg_charge_cgroup(const struct cred *cred, + const struct file *file); int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -717,6 +719,12 @@ static inline int security_memcg_charge_task(const struct cred *from, return 0; }
+static inline int security_memcg_charge_cgroup(const struct cred *cred, + const struct file *file) +{ + return 0; +} + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2e6fcbe14d43..693f0b2877bf 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3220,7 +3220,14 @@ struct obj_cgroup *get_obj_cgroup_from_fd(int fd) css = ERR_PTR(-EBADF); }
- if (IS_ERR(css)) { + if (!IS_ERR(css)) { + /* fd is a cgroupfd */ + ret = security_memcg_charge_cgroup(current_cred(), fd_file(f)); + if (ret) { + css_put(css); + return ERR_PTR(ret); + } + } else { if (PTR_ERR(css) != -EBADF) return ERR_CAST(css);
diff --git a/security/security.c b/security/security.c index e198e00b6a40..925b6c003059 100644 --- a/security/security.c +++ b/security/security.c @@ -689,6 +689,26 @@ int security_memcg_charge_task(const struct cred *from, const struct cred *to) return call_int_hook(memcg_charge_task, from, to); }
+/** + * security_memcg_charge_cgroup() - Check if charging memory to a cgroup is allowed + * @cred: credentials of the process requesting the charge + * @file: open file of the cgroup directory to charge + * + * Check whether the process with credentials @cred is allowed to charge + * memory to the cgroup referenced by @file. Called when userspace passes a + * cgroupfd to identify the charge target instead of a pidfd. @file is the + * file the cgroup was resolved from, so the check applies to the object + * that is actually charged. It must have been already validated by the + * caller. + * + * Return: Returns 0 if permission is granted. + */ +int security_memcg_charge_cgroup(const struct cred *cred, + const struct file *file) +{ + return call_int_hook(memcg_charge_cgroup, cred, file); +} + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
The security_memcg_charge_cgroup() hook allows security modules to control which processes may charge memcg memory to an arbitrary cgroup named by a cgroupfd passed in charge_fd. Without a policy implementation, the hook is a no-op, so a domain that has both dma-heap access and CAP_SYS_RESOURCE can charge any cgroup outside its own subtree, and SELinux cannot say which targets are acceptable.
Implement selinux_memcg_charge_cgroup() using avc_has_perm() with a new cgroup object class and a charge_to permission. This is the counterpart of the process2 charge_to permission used for the pidfd path.
The target SID is the SID of the inode of the cgroup directory the cgroupfd refers to, so access control follows the cgroup filesystem labeling. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t cgroup_t:cgroup charge_to;
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 10 ++++++++++ security/selinux/include/classmap.h | 1 + 2 files changed, 11 insertions(+)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 70de6094b2b4..fcf0b03a588d 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2210,6 +2210,15 @@ static int selinux_memcg_charge_task(const struct cred *from, const struct cred SECCLASS_PROCESS2, PROCESS2__CHARGE_TO, NULL); }
+static int selinux_memcg_charge_cgroup(const struct cred *cred, + const struct file *file) +{ + struct inode_security_struct *isec = inode_security(file_inode(file)); + + return avc_has_perm(cred_sid(cred), isec->sid, SECCLASS_CGROUP, + CGROUP__CHARGE_TO, NULL); +} + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7688,6 +7697,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), LSM_HOOK_INIT(memcg_charge_task, selinux_memcg_charge_task), + LSM_HOOK_INIT(memcg_charge_cgroup, selinux_memcg_charge_cgroup), LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index 4004c05aae5b..32dd95de509a 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -181,6 +181,7 @@ const struct security_class_mapping secclass_map[] = { { "user_namespace", { "create", NULL } }, { "memfd_file", { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } }, + { "cgroup", { "charge_to", NULL } }, /* last one */ { NULL, {} } };
Add tests for the new charge_fd field in struct dma_heap_allocation_data.
When the charge_fd feature is absent (unpatched kernel), the probe in pidfd_alloc_supported() detects this and the tests are skipped gracefully.
Add vmtest.sh similar to other subsystem suites, to orchestrate building the selftests (optionally with a freshly compiled kernel) inside a virtme-ng VM, so the tests can be run without modifying the host system. Add a config fragment with required Kconfig symbols.
Also extend test_memcontrol suite to cover cross-cgroup dma-buf accounting:
- test_memcg_dmabuf: a buffer charged via pidfd is accounted to the target cgroup's memory.stat - test_memcg_dmabuf_xfer_at_limit: transferring a charge to a cgroup at its memory.max limit fails with ENOMEM and leaves the charge in the source cgroup - test_memcg_dmabuf_dead_cgroup: a cgroup removed while one of its charged buffers is still open and is released instead of lingering as a dying descendant, and the charge moves to the parent and is released when the buffer is closed.
All of them are skipped on unpatched kernels or when /dev/dma_heap/system is absent.
Signed-off-by: Albert Esteve aesteve@redhat.com --- tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 447 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 237 ++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 +++++++++++ 5 files changed, 909 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index e01584c2189a..9edfc9f1de5c 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -CFLAGS += -Wall -pthread +CFLAGS += -Wall -pthread $(KHDR_INCLUDES)
all: ${HELPER_PROGS}
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index bb673012dcf0..f612726bd9af 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -18,8 +18,15 @@ #include <netdb.h> #include <errno.h> #include <sys/mman.h> +#include <sys/prctl.h> #include <sys/sysinfo.h>
+#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <signal.h> +#include <sys/ioctl.h> + +#include "../pidfd/pidfd.h" #include "kselftest.h" #include "cgroup_util.h"
@@ -27,6 +34,13 @@ #define MEMCG_FLUSH_BYTES \ (2 * sysconf(_SC_PAGESIZE) * MEMCG_CHARGE_BATCH * get_nprocs())
+#define DMA_HEAP_SYSTEM "/dev/dma_heap/system" +#define ONE_MEG (1024 * 1024) + +#define MEMCG_SOCKSTAT_WAIT_RETRIES 30 +#define MEMCG_DMABUF_WAIT_RETRIES 30 +#define MEMCG_DMABUF_DEAD_WAIT_RETRIES 80 + static bool has_localevents; static bool has_recursiveprot; static int page_size; @@ -1797,6 +1811,417 @@ static int test_memcg_inotify_delete_dir(const char *root) return ret; }
+static int memcg_dmabuf_child(const char *cgroup, void *arg) +{ + pid_t *parent_pid = arg; + + prctl(PR_SET_PDEATHSIG, SIGKILL); + if (getppid() != *parent_pid) + return 0; + + pause(); + return 0; +} + +/* + * This test allocates a dma-buf via DMA_HEAP_IOCTL_ALLOC with a pidfd + * pointing to a child process in a separate cgroup, then checks that + * memory.stat[dmabuf] in the child's cgroup rises by the allocation size + * and returns to zero after the buffer fd is closed. + */ +static int test_memcg_dmabuf(const char *root) +{ + char *parent = NULL, *child_cg = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, pidfd = -1; + pid_t child_pid, parent_pid = getpid(); + int child_status; + long dmabuf_stat; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_memcg_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + child_cg = cg_name(parent, "child"); + if (!child_cg) + goto cleanup_parent; + + if (cg_create(child_cg)) + goto cleanup_parent; + + child_pid = cg_run_nowait(child_cg, memcg_dmabuf_child, &parent_pid); + if (child_pid < 0) + goto cleanup_child; + + if (cg_wait_for_proc_count(child_cg, 1)) + goto cleanup_kill; + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_pidfd; + } + + alloc.charge_fd = (__u32)pidfd; + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) { + if (errno == EINVAL || errno == ENOTTY || errno == EOPNOTSUPP) + ret = KSFT_SKIP; + goto cleanup_heap; + } + dmabuf_fd = (int)alloc.fd; + + dmabuf_stat = cg_read_key_long(child_cg, "memory.stat", "dmabuf "); + if (dmabuf_stat == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (dmabuf_stat != ONE_MEG) + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != ONE_MEG) { + fprintf(stderr, "Expected dmabuf stat %d, got %ld\n", + ONE_MEG, dmabuf_stat); + goto cleanup_dmabuf; + } + + close(dmabuf_fd); + dmabuf_fd = -1; + + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", 0, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != 0) { + fprintf(stderr, "Expected dmabuf stat 0 after close, got %ld\n", + dmabuf_stat); + goto cleanup_heap; + } + + ret = KSFT_PASS; + +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_pidfd: + close(pidfd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_child: + cg_destroy(child_cg); + free(child_cg); +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + +/* + * Allocate a dma-buf in cgroup A, set memory.max on cgroup B below the + * buffer size, then attempt DMA_BUF_IOCTL_XFER_CHARGE to B. The transfer + * must fail with ENOMEM and the charge must remain in A. + */ +static int test_memcg_dmabuf_xfer_at_limit(const char *root) +{ + char *parent = NULL, *cg_a = NULL, *cg_b = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, cg_b_fd = -1; + pid_t child_pid, parent_pid = getpid(); + int child_status; + long stat_a; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + struct dma_buf_transfer_charge xfer = { 0 }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_xfer_limit_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + cg_a = cg_name(parent, "cg_a"); + cg_b = cg_name(parent, "cg_b"); + if (!cg_a || !cg_b) + goto cleanup_cgs; + + if (cg_create(cg_a) || cg_create(cg_b)) + goto cleanup_cgs; + + if (cg_write(cg_b, "memory.max", "524288")) + goto cleanup_cgs; + + child_pid = cg_run_nowait(cg_a, memcg_dmabuf_child, &parent_pid); + if (child_pid < 0) + goto cleanup_cgs; + + if (cg_wait_for_proc_count(cg_a, 1)) + goto cleanup_kill; + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + alloc.charge_fd = (__u32)sys_pidfd_open(child_pid, 0); + if ((int)alloc.charge_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_heap; + } + + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) { + close(alloc.charge_fd); + goto cleanup_heap; + } + close(alloc.charge_fd); + dmabuf_fd = (int)alloc.fd; + + stat_a = cg_read_key_long(cg_a, "memory.stat", "dmabuf "); + if (stat_a == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (stat_a != ONE_MEG) + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge not in A after alloc: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_dmabuf; + } + + cg_b_fd = open(cg_b, O_RDONLY | O_DIRECTORY); + if (cg_b_fd < 0) + goto cleanup_dmabuf; + + xfer.charge_fd = (__u32)cg_b_fd; + if (ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer) == 0) { + fprintf(stderr, "xfer_charge should have failed at limit\n"); + goto cleanup_cgfd; + } + if (errno != ENOMEM) { + fprintf(stderr, "Expected ENOMEM, got %s\n", strerror(errno)); + goto cleanup_cgfd; + } + + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge leaked from A: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_cgfd; + } + + ret = KSFT_PASS; + +cleanup_cgfd: + close(cg_b_fd); +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_cgs: + if (cg_a) { + cg_destroy(cg_a); + free(cg_a); + } + if (cg_b) { + cg_destroy(cg_b); + free(cg_b); + } +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + +/* + * Allocate a dma-buf charged to a cgroup, then remove that cgroup while + * the buffer is still open. The charge is held by an object cgroup, + * which is reparented, so the removed cgroup must be released instead of + * lingering as a dying descendant until the buffer is closed. + */ +static int test_memcg_dmabuf_dead_cgroup(const char *root) +{ + char *parent = NULL, *child_cg = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, cg_fd = -1; + long dmabuf_stat, dying; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_dead_cgroup_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + child_cg = cg_name(parent, "child"); + if (!child_cg) + goto cleanup_parent; + + if (cg_create(child_cg)) + goto cleanup_child; + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_child; + } + + cg_fd = open(child_cg, O_RDONLY | O_DIRECTORY); + if (cg_fd < 0) + goto cleanup_heap; + + alloc.charge_fd = (__u32)cg_fd; + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) { + if (errno == EINVAL || errno == ENOTTY || errno == EOPNOTSUPP) + ret = KSFT_SKIP; + goto cleanup_cgfd; + } + /* The buffer holds the charge; the fd must not hold the cgroup. */ + close(cg_fd); + cg_fd = -1; + dmabuf_fd = (int)alloc.fd; + + dmabuf_stat = cg_read_key_long(child_cg, "memory.stat", "dmabuf "); + if (dmabuf_stat == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (dmabuf_stat != ONE_MEG) + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != ONE_MEG) { + fprintf(stderr, "Expected dmabuf stat %d, got %ld\n", + ONE_MEG, dmabuf_stat); + goto cleanup_dmabuf; + } + + if (cg_destroy(child_cg)) { + fprintf(stderr, "Failed to remove charged cgroup\n"); + goto cleanup_dmabuf; + } + + dying = cg_read_key_long_poll(parent, "cgroup.stat", + "nr_dying_descendants ", 0, + MEMCG_DMABUF_DEAD_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dying) { + fprintf(stderr, "Removed cgroup still dying with buffer open\n"); + goto cleanup_dmabuf; + } + + /* The charge must have moved to the parent with the cgroup. */ + dmabuf_stat = cg_read_key_long(parent, "memory.stat", "dmabuf "); + if (dmabuf_stat != ONE_MEG) + dmabuf_stat = cg_read_key_long_poll(parent, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != ONE_MEG) { + fprintf(stderr, "Charge not in parent after removal: expected %d, got %ld\n", + ONE_MEG, dmabuf_stat); + goto cleanup_dmabuf; + } + + /* Uncharge through the reparented object cgroup. */ + close(dmabuf_fd); + dmabuf_fd = -1; + + dmabuf_stat = cg_read_key_long(parent, "memory.stat", "dmabuf "); + if (dmabuf_stat != 0) + dmabuf_stat = cg_read_key_long_poll(parent, "memory.stat", + "dmabuf ", 0, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != 0) { + fprintf(stderr, "Expected dmabuf stat 0 after close, got %ld\n", + dmabuf_stat); + goto cleanup_heap; + } + + ret = KSFT_PASS; + +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_cgfd: + if (cg_fd >= 0) + close(cg_fd); +cleanup_heap: + close(heap_fd); +cleanup_child: + if (child_cg) { + cg_destroy(child_cg); + free(child_cg); + } +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + #define T(x) { x, #x } struct memcg_test { int (*fn)(const char *root); @@ -1818,13 +2243,20 @@ struct memcg_test { T(test_memcg_oom_group_score_events), T(test_memcg_inotify_delete_file), T(test_memcg_inotify_delete_dir), + T(test_memcg_dmabuf), + T(test_memcg_dmabuf_xfer_at_limit), + T(test_memcg_dmabuf_dead_cgroup), }; #undef T
int main(int argc, char **argv) { char root[PATH_MAX]; - int i, proc_status; + int i, proc_status, count = ARRAY_SIZE(tests); + const char *filter = NULL; + + if (argc > 1) + filter = argv[1];
page_size = sysconf(_SC_PAGE_SIZE); if (page_size <= 0) @@ -1855,8 +2287,19 @@ int main(int argc, char **argv) ksft_exit_skip("Failed to query cgroup mount option\n"); has_localevents = proc_status;
- ksft_set_plan(ARRAY_SIZE(tests)); + if (filter) { + count = 0; + for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (!strcmp(tests[i].name, filter)) + count++; + } + if (!count) + ksft_exit_fail_msg("unknown test '%s'\n", filter); + } + ksft_set_plan(count); for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (filter && strcmp(tests[i].name, filter)) + continue; switch (tests[i].fn(root)) { case KSFT_PASS: ksft_test_result_pass("%s\n", tests[i].name); diff --git a/tools/testing/selftests/dmabuf-heaps/config b/tools/testing/selftests/dmabuf-heaps/config index be091f1cdfa0..94c8f33b71a2 100644 --- a/tools/testing/selftests/dmabuf-heaps/config +++ b/tools/testing/selftests/dmabuf-heaps/config @@ -1,3 +1,4 @@ +CONFIG_MEMCG=y CONFIG_DMABUF_HEAPS=y CONFIG_DMABUF_HEAPS_SYSTEM=y CONFIG_DRM_VGEM=y diff --git a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c index 1d49df671919..44e3f33acf87 100644 --- a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c +++ b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c @@ -3,6 +3,7 @@ #include <dirent.h> #include <errno.h> #include <fcntl.h> +#include <signal.h> #include <stdio.h> #include <stdlib.h> #include <stdint.h> @@ -10,11 +11,16 @@ #include <unistd.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/prctl.h> +#include <sys/syscall.h> #include <sys/types.h> +#include <sys/wait.h>
#include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/limits.h> #include <drm/drm.h> +#include "../pidfd/pidfd.h" #include "kselftest.h"
#define DEVPATH "/dev/dma_heap" @@ -320,6 +326,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; __u64 garbage1; __u64 garbage2; __u64 garbage3; @@ -328,6 +336,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, .fd = 0, .fd_flags = O_RDWR | O_CLOEXEC, .heap_flags = flags, + .charge_fd = 0, + .__padding = 0, .garbage1 = 0xffffffff, .garbage2 = 0x88888888, .garbage3 = 0x11111111, @@ -496,6 +506,227 @@ static void test_alloc_no_fd_leak_on_efault(char *heap_name) ksft_test_result(fd_before == fd_after, "fd leak on EFAULT: before=%d after=%d\n", fd_before, fd_after); + +out: + close(heap_fd); +} + +static int dmabuf_heap_alloc_pidfd(int fd, size_t len, unsigned int heap_flags, + unsigned int charge_fd, int *dmabuf_fd) +{ + struct dma_heap_allocation_data data = { + .len = len, + .fd = 0, + .fd_flags = O_RDWR | O_CLOEXEC, + .heap_flags = heap_flags, + .charge_fd = charge_fd, + }; + int ret; + + if (!dmabuf_fd) + return -EINVAL; + + ret = ioctl(fd, DMA_HEAP_IOCTL_ALLOC, &data); + if (ret < 0) + return ret; + *dmabuf_fd = (int)data.fd; + return ret; +} + +/* + * Probe whether the kernel honours charge_fd in DMA_HEAP_IOCTL_ALLOC. + */ +static bool pidfd_alloc_supported(int heap_fd) +{ + int devnull_fd, dmabuf_fd = -1, ret; + + devnull_fd = open("/dev/null", O_RDONLY); + if (devnull_fd < 0) + return false; + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, devnull_fd, &dmabuf_fd); + if (dmabuf_fd >= 0) { + close(dmabuf_fd); + dmabuf_fd = -1; + } + close(devnull_fd); + return ret < 0; +} + +/* + * Test: allocate charging the calling process's own cgroup via a self pidfd. + */ +static void test_alloc_pidfd_self(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + pidfd = sys_pidfd_open(getpid(), 0); + if (pidfd < 0) { + ksft_test_result_skip("pidfd_open not available\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with self pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); +out: + close(heap_fd); +} + +/* + * Test: allocate charging a child process's cgroup via a child pidfd. + */ +static void test_alloc_pidfd_child(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1; + pid_t child_pid, parent_pid; + int status, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + parent_pid = getpid(); + child_pid = fork(); + if (child_pid == 0) { + prctl(PR_SET_PDEATHSIG, SIGKILL); + if (getppid() != parent_pid) + _exit(0); + pause(); + _exit(0); + } + if (child_pid < 0) + ksft_exit_fail_msg("fork failed: %s\n", strerror(errno)); + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); + ksft_test_result_skip("pidfd_open for child failed\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with child pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); +out: + close(heap_fd); +} + +static int open_self_cgroupfd(void) +{ + char cgroup_path[PATH_MAX]; + char line[PATH_MAX]; + int ret = -1; + FILE *f; + + f = fopen("/proc/self/cgroup", "r"); + if (!f) + return -1; + while (fgets(line, sizeof(line), f)) { + if (strncmp(line, "0::", 3)) + continue; + line[strcspn(line, "\n")] = '\0'; + if (snprintf(cgroup_path, sizeof(cgroup_path), + "/sys/fs/cgroup%s", line + 3) < PATH_MAX) + ret = 0; + break; + } + fclose(f); + + if (ret) + return -1; + + return open(cgroup_path, O_RDONLY | O_DIRECTORY); +} + +/* + * Test: allocate charging the calling process's own cgroup via a cgroupfd. + */ +static void test_alloc_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, cgroup_fd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with cgroupfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(cgroup_fd); +out: + close(heap_fd); +} + +/* + * Test: transfer a dma-buf's charge to self cgroup via DMA_BUF_IOCTL_XFER_CHARGE. + */ +static void test_xfer_charge_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + struct dma_buf_transfer_charge xfer = { 0 }; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + ret = dmabuf_heap_alloc(heap_fd, ONE_MEG, 0, &dmabuf_fd); + if (ret) { + ksft_test_result_fail("Allocation failed %d\n", ret); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + close(dmabuf_fd); + goto out; + } + + xfer.charge_fd = cgroup_fd; + ret = ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); + if (!ret) + ksft_test_result_pass("XFER_CHARGE to self cgroupfd\n"); + else if (errno == EOPNOTSUPP) + ksft_test_result_skip("buffer not memcg-charged (mem_accounting=0?)\n"); + else if (errno == ENOTTY || errno == EINVAL) + ksft_test_result_skip("XFER_CHARGE not supported: %s\n", + strerror(errno)); + else + ksft_test_result_fail("XFER_CHARGE failed: %s\n", strerror(errno)); + + close(cgroup_fd); + close(dmabuf_fd); out: close(heap_fd); } @@ -530,7 +761,7 @@ int main(void) return KSFT_SKIP; }
- ksft_set_plan(12 * numer_of_heaps()); + ksft_set_plan(16 * numer_of_heaps());
while ((dir = readdir(d))) { if (!strncmp(dir->d_name, ".", 2)) @@ -546,6 +777,10 @@ int main(void) test_alloc_compat(dir->d_name); test_alloc_errors(dir->d_name); test_alloc_no_fd_leak_on_efault(dir->d_name); + test_alloc_pidfd_self(dir->d_name); + test_alloc_pidfd_child(dir->d_name); + test_alloc_cgroupfd_self(dir->d_name); + test_xfer_charge_cgroupfd_self(dir->d_name); } closedir(d);
diff --git a/tools/testing/selftests/dmabuf-heaps/vmtest.sh b/tools/testing/selftests/dmabuf-heaps/vmtest.sh new file mode 100755 index 000000000000..3574b71ca83e --- /dev/null +++ b/tools/testing/selftests/dmabuf-heaps/vmtest.sh @@ -0,0 +1,226 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (c) 2026 Red Hat +# +# Dependencies: +# * virtme-ng +# * qemu (used by virtme-ng) + +readonly SCRIPT_DIR="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)" +readonly KERNEL_CHECKOUT=$(realpath "${SCRIPT_DIR}"/../../../../) +readonly CGROUP_DIR="${KERNEL_CHECKOUT}/tools/testing/selftests/cgroup" + +source "${SCRIPT_DIR}"/../kselftest/ktap_helpers.sh + +readonly DMABUF_HEAP_TEST="${SCRIPT_DIR}"/dmabuf-heap +readonly MEMCONTROL_TEST="${CGROUP_DIR}"/test_memcontrol +readonly TMP_DIR=$(mktemp -d /tmp/dmabuf-vmtest.XXXXXXXX) + +VERBOSE=false +BUILD=false +BUILD_HOST="" +BUILD_HOST_PODMAN_CONTAINER_NAME="" +SHELL_MODE=false + +usage() { + echo + echo "$0 [OPTIONS]" + echo + echo "Options" + echo " -b: build the kernel from the current source tree and use it for the VM" + echo " -H: hostname for remote build host (used with -b)" + echo " -p: podman container name for remote build host (used with -b)" + echo " Example: -H beefyserver -p vng" + echo " -s: drop into an interactive shell in the VM instead of running tests" + echo " -v: enable verbose vng/qemu output" + echo + + exit 1 +} + +die() { + echo "$*" >&2 + exit "${KSFT_FAIL}" +} + +cleanup() { + rm -rf "${TMP_DIR}" +} + +check_deps() { + for dep in vng make; do + if [[ ! -x $(command -v "${dep}") ]]; then + echo -e "skip: dependency ${dep} not found!\n" + exit "${KSFT_SKIP}" + fi + done + + if ${SHELL_MODE}; then + return + fi + + if [[ ! -x "${DMABUF_HEAP_TEST}" ]]; then + printf "skip: %s not found!" "${DMABUF_HEAP_TEST}" + printf " Please build the kselftest dmabuf-heaps target (or use -b).\n" + exit "${KSFT_SKIP}" + fi + + if [[ ! -x "${MEMCONTROL_TEST}" ]]; then + printf "skip: %s not found!" "${MEMCONTROL_TEST}" + printf " Please build the kselftest cgroup target (or use -b).\n" + exit "${KSFT_SKIP}" + fi +} + +check_vng() { + local tested_versions=("1.36" "1.37") + local version + local ok=0 + + version="$(vng --version)" + for tv in "${tested_versions[@]}"; do + if [[ "${version}" == *"${tv}"* ]]; then + ok=1 + break + fi + done + + if [[ "${ok}" -eq 0 ]]; then + printf "warning: vng version '%s' has not been tested and may " "${version}" >&2 + printf "not function properly.\n\tThe following versions have been tested: " >&2 + echo "${tested_versions[@]}" >&2 + fi +} + +build_selftests() { + make -C "${KERNEL_CHECKOUT}" headers_install \ + INSTALL_HDR_PATH="${TMP_DIR}/usr" -j"$(nproc)" + + local khdr="-isystem ${TMP_DIR}/usr/include" + + if ! make -C "${SCRIPT_DIR}" KHDR_INCLUDES="${khdr}" -j"$(nproc)"; then + die "failed to build dmabuf-heaps selftests" + fi + + if ! make -C "${CGROUP_DIR}" KHDR_INCLUDES="${khdr}" \ + "${MEMCONTROL_TEST}" -j"$(nproc)"; then + die "failed to build cgroup/test_memcontrol selftest" + fi +} + +handle_build() { + if ! ${BUILD}; then + return + fi + + if [[ ! -d "${KERNEL_CHECKOUT}" ]]; then + echo "-b requires vmtest.sh called from the kernel source tree" >&2 + exit 1 + fi + + pushd "${KERNEL_CHECKOUT}" &>/dev/null + + if ! vng --kconfig --config "${SCRIPT_DIR}/config"; then + die "failed to generate .config for kernel source tree (${KERNEL_CHECKOUT})" + fi + + local vng_args=("-v" "--config" "${SCRIPT_DIR}/config" "--build") + + if [[ -n "${BUILD_HOST}" ]]; then + vng_args+=("--build-host" "${BUILD_HOST}") + fi + + if [[ -n "${BUILD_HOST_PODMAN_CONTAINER_NAME}" ]]; then + vng_args+=("--build-host-exec-prefix" \ + "podman exec -ti ${BUILD_HOST_PODMAN_CONTAINER_NAME}") + fi + + if ! vng "${vng_args[@]}"; then + die "failed to build kernel from source tree (${KERNEL_CHECKOUT})" + fi + + build_selftests + + popd &>/dev/null +} + +make_runner() { + # virtme-ng shares the host filesystem, so TMP_DIR is accessible + # inside the VM at the same absolute path. + cat > "${TMP_DIR}/run_tests.sh" <<-EOF + #!/bin/sh + set -u + + # Enable memory controller for child cgroups + echo +memory > /sys/fs/cgroup/cgroup.subtree_control 2>/dev/null + + PASS=0; FAIL=0; SKIP=0; N=0 + + run() { + name="$1"; shift + N=$((N+1)) + "$@"; rc=$? + if [ $rc -eq 0 ]; then echo "ok $N $name"; PASS=$((PASS+1)) + elif [ $rc -eq 4 ]; then echo "ok $N $name # SKIP"; SKIP=$((SKIP+1)) + else echo "not ok $N $name"; FAIL=$((FAIL+1)) + fi + } + + run "dmabuf-heap charge_fd ioctl" ${DMABUF_HEAP_TEST} + run "memcontrol dma-buf memcg" ${MEMCONTROL_TEST} test_memcg_dmabuf + run "memcontrol dma-buf xfer limit" \ + ${MEMCONTROL_TEST} test_memcg_dmabuf_xfer_at_limit + echo "# PASS=$PASS SKIP=$SKIP FAIL=$FAIL" + [ $FAIL -eq 0 ] + EOF + chmod +x "${TMP_DIR}/run_tests.sh" +} + +run_vm() { + local verbose_opt="" + local kernel_opt="" + + ${VERBOSE} && verbose_opt="--verbose" + + # If we are running from within the kernel source tree, use the kernel + # source tree as the kernel to boot, otherwise use the running kernel. + if [[ "$(realpath "$(pwd)")" == "${KERNEL_CHECKOUT}"* ]]; then + kernel_opt="${KERNEL_CHECKOUT}" + fi + + if ${SHELL_MODE}; then + echo "Starting interactive shell in VM. Exit to stop VM." + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + -- bash --noprofile --norc + exit "${KSFT_PASS}" + fi + + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + --exec "${TMP_DIR}/run_tests.sh" +} + +while getopts :hvsbH:p: o +do + case $o in + v) VERBOSE=true;; + b) BUILD=true;; + s) SHELL_MODE=true;; + H) BUILD_HOST=$OPTARG;; + p) BUILD_HOST_PODMAN_CONTAINER_NAME=$OPTARG;; + h|*) usage;; + esac +done +shift $((OPTIND-1)) + +trap cleanup EXIT + +check_deps +check_vng +handle_build +make_runner + +echo "Booting VM and running tests..." +run_vm
On embedded platforms a central process often allocates dma-buf memory on behalf of client applications. Without a way to attribute the charge to the requesting client's cgroup, the cost lands on the allocator, making per-cgroup memory limits ineffective for the actual consumers.
Add charge_fd to struct dma_heap_allocation_data. When set to a valid pidfd or cgroupfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's or cgroup's memcg and charges the buffer there via obj_cgroup_reserve_dmabuf() inside heap's allocate() callback. In the system heap, the object cgroup is passed through dma_buf_export_info so that dma_buf_export() stores it and updates the MEMCG_DMABUF stat. Without charge_fd, and with the mem_accounting module parameter enabled, the buffer is charged to the allocator's own cgroup. The __GFP_ACCOUNT flag is removed from system heap allocations to avoid double counting. CMA and TEE heaps accept the objcg in the callback but ignore it.
Resolving a charge target outside the caller's own memcg subtree requires CAP_SYS_RESOURCE. Without it, any process could charge the root cgroup to escape its own limits, or push reclaim and OOM onto unrelated cgroups. Charging within the caller's subtree cannot escape limits, since descendant usage counts toward the ancestor, and needs no privilege.
Usage examples:
1. Central allocator charging to a client at allocation time. The allocator knows the client's PID (e.g., from binder's sender_pid) and uses pidfd to attribute the charge:
pid_t client_pid = txn->sender_pid; int pidfd = pidfd_open(client_pid, 0);
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, .charge_fd = pidfd, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); close(pidfd); /* alloc.fd is now charged to client's cgroup */
2. Default allocation (no pidfd, mem_accounting=1). When charge_fd is not set and the mem_accounting module parameter is enabled, the buffer is charged to the allocator's own cgroup:
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); /* charged to current process's cgroup */
Current limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. Means only the first owner (and exporter) of the shared buffer bears the charge. - Only memcg accounting supported. While this makes sense for system heap buffers, other heaps (e.g., CMA heaps) will require selectively charging also for the dmem controller.
Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 6 ++- drivers/dma-buf/dma-buf.c | 3 ++ drivers/dma-buf/dma-heap.c | 27 +++++++++-- drivers/dma-buf/heaps/cma_heap.c | 3 +- drivers/dma-buf/heaps/system_heap.c | 19 ++++++-- drivers/tee/tee_heap.c | 3 +- include/linux/dma-buf.h | 5 ++ include/linux/dma-heap.h | 4 +- include/linux/memcontrol.h | 11 +++++ include/uapi/linux/dma-heap.h | 7 +++ mm/memcontrol.c | 83 +++++++++++++++++++++++++++++++++ 11 files changed, 158 insertions(+), 13 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index f1f6808c5600..c6bf2e435d74 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1667,8 +1667,10 @@ The following nested keys are defined. structures.
dmabuf (npn) - Amount of memory used for exported DMA buffers allocated by the cgroup. - Stays with the allocating cgroup regardless of how the buffer is shared. + Amount of memory used for exported DMA buffers allocated by or on + behalf of the cgroup, this currently includes system heap + allocations. Stays with the cgroup regardless of how the buffer + is shared.
workingset_refault_anon Number of refaults of previously evicted anonymous pages. diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 06462faf762e..88e0802fa05f 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -750,6 +750,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner; + dmabuf->objcg = exp_info->objcg; spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; @@ -771,6 +772,8 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
DMA_BUF_TRACE(trace_dma_buf_export, dmabuf);
+ mem_cgroup_commit_dmabuf(dmabuf); + return dmabuf;
err_file: diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index 8a6c2b6e0cc6..2f53bc95a45a 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -10,6 +10,7 @@ #include <linux/device.h> #include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/memcontrol.h> #include <linux/err.h> #include <linux/export.h> #include <linux/list.h> @@ -78,6 +79,7 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) struct dma_heap_allocation_data *heap_allocation = data; struct dma_heap *heap = file->private_data; struct dma_buf *dmabuf; + struct obj_cgroup *objcg = NULL; int fd; size_t len;
@@ -90,15 +92,34 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS) return ERR_PTR(-EINVAL);
+ if (heap_allocation->__padding != 0) + return ERR_PTR(-EINVAL); + + if (heap_allocation->charge_fd) { + objcg = get_obj_cgroup_from_fd(heap_allocation->charge_fd); + if (IS_ERR(objcg)) + return ERR_CAST(objcg); + } + + if (!objcg && mem_accounting && !mem_cgroup_disabled()) + objcg = get_obj_cgroup_from_current(); + len = PAGE_ALIGN(heap_allocation->len); - if (!len) + if (!len) { + obj_cgroup_put(objcg); return ERR_PTR(-EINVAL); + }
dmabuf = heap->ops->allocate(heap, len, heap_allocation->fd_flags, - heap_allocation->heap_flags); + heap_allocation->heap_flags, objcg);
- if (IS_ERR(dmabuf)) + if (IS_ERR(dmabuf)) { + obj_cgroup_put(objcg); return dmabuf; + } + + if (!dmabuf->objcg) + obj_cgroup_put(objcg);
fd = get_unused_fd_flags(heap_allocation->fd_flags); if (fd < 0) { diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c index 3fb4b946c91a..6c08dc25b177 100644 --- a/drivers/dma-buf/heaps/cma_heap.c +++ b/drivers/dma-buf/heaps/cma_heap.c @@ -281,7 +281,8 @@ static const struct dma_buf_ops cma_heap_buf_ops = { static struct dma_buf *cma_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct obj_cgroup *objcg) { struct cma_heap *cma_heap = dma_heap_get_drvdata(heap); struct cma_heap_buffer *buffer; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index c8959eadc71d..ad94bb5be736 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -17,6 +17,7 @@ #include <linux/err.h> #include <linux/highmem.h> #include <linux/mem_encrypt.h> +#include <linux/memcontrol.h> #include <linux/mm.h> #include <linux/set_memory.h> #include <linux/module.h> @@ -388,8 +389,6 @@ static struct page *alloc_largest_available(unsigned long size, if (max_order < orders[i]) continue; flags = order_flags[i]; - if (mem_accounting) - flags |= __GFP_ACCOUNT; page = alloc_pages(flags, orders[i]); if (!page) continue; @@ -401,11 +400,13 @@ static struct page *alloc_largest_available(unsigned long size, static struct dma_buf *system_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct obj_cgroup *objcg) { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining = len; + unsigned long nr_pages = PAGE_ALIGN(len) >> PAGE_SHIFT; unsigned int max_order = orders[0]; struct system_heap_priv *priv = dma_heap_get_drvdata(heap); bool cc_shared = priv->cc_shared; @@ -416,10 +417,15 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, struct page *page, *tmp_page; int i, ret = -ENOMEM;
- buffer = kzalloc_obj(*buffer); - if (!buffer) + if (!obj_cgroup_reserve_dmabuf(objcg, GFP_KERNEL, nr_pages)) return ERR_PTR(-ENOMEM);
+ buffer = kzalloc_obj(*buffer); + if (!buffer) { + ret = -ENOMEM; + goto cancel_charge; + } + INIT_LIST_HEAD(&buffer->attachments); mutex_init(&buffer->lock); buffer->heap = heap; @@ -473,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer; + exp_info.objcg = objcg; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf); @@ -498,6 +505,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, list_for_each_entry_safe(page, tmp_page, &pages, lru) __free_pages(page, compound_order(page)); kfree(buffer); +cancel_charge: + obj_cgroup_cancel_dmabuf(objcg, nr_pages);
return ERR_PTR(ret); } diff --git a/drivers/tee/tee_heap.c b/drivers/tee/tee_heap.c index 007c00156520..83fd14a0aaec 100644 --- a/drivers/tee/tee_heap.c +++ b/drivers/tee/tee_heap.c @@ -167,7 +167,8 @@ static const struct dma_buf_ops tee_heap_buf_ops = {
static struct dma_buf *tee_dma_heap_alloc(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct obj_cgroup *objcg) { struct tee_dma_heap *h = dma_heap_get_drvdata(heap); DEFINE_DMA_BUF_EXPORT_INFO(exp_info); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 2f56afd1f830..ff4678d00d8c 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -505,6 +505,10 @@ struct dma_buf_attachment { * @flags: mode flags for the file * @resv: reservation-object, NULL to allocate default one * @priv: Attach private data of allocator to this buffer + * @objcg: object cgroup to charge for this buffer's memory. Set only for + * buffers backed by system memory, not accounted elsewhere, where + * @size reflects the actual allocation in bytes. The buffer takes + * over the caller's reference. * * This structure holds the information required to export the buffer. Used * with dma_buf_export() only. @@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv; + struct obj_cgroup *objcg; };
/** diff --git a/include/linux/dma-heap.h b/include/linux/dma-heap.h index 648328a64b27..b5a42b49c68d 100644 --- a/include/linux/dma-heap.h +++ b/include/linux/dma-heap.h @@ -12,6 +12,7 @@ #include <linux/types.h>
struct dma_heap; +struct obj_cgroup;
/** * struct dma_heap_ops - ops to operate on a given heap @@ -23,7 +24,8 @@ struct dma_heap_ops { struct dma_buf *(*allocate)(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags); + u64 heap_flags, + struct obj_cgroup *objcg); };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 46fbb418bf39..8e016d9b91bc 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1812,6 +1812,7 @@ void __memcg_kmem_uncharge_page(struct page *page, int order); */ struct obj_cgroup *current_obj_cgroup(void); struct obj_cgroup *get_obj_cgroup_from_folio(const struct folio *folio); +struct obj_cgroup *get_obj_cgroup_from_fd(int fd);
static inline struct obj_cgroup *get_obj_cgroup_from_current(void) { @@ -1919,6 +1920,16 @@ static inline struct obj_cgroup *get_obj_cgroup_from_folio(const struct folio *f return NULL; }
+static inline struct obj_cgroup *get_obj_cgroup_from_fd(int fd) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline struct obj_cgroup *get_obj_cgroup_from_current(void) +{ + return NULL; +} + static inline bool memcg_bpf_enabled(void) { return false; diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h index a4cf716a49fa..bfcd1c25bc7e 100644 --- a/include/uapi/linux/dma-heap.h +++ b/include/uapi/linux/dma-heap.h @@ -29,6 +29,11 @@ * handle to the allocated dma-buf * @fd_flags: file descriptor flags used when allocating * @heap_flags: flags passed to heap + * @charge_fd: optional pidfd or cgroupfd whose cgroup should be + * charged for this allocation; 0 means charge the calling + * process's cgroup when the mem_accounting module parameter + * is enabled, or skip charging otherwise + * @__padding: reserved, must be zero * * Provided by userspace as an argument to the ioctl */ @@ -37,6 +42,8 @@ struct dma_heap_allocation_data { __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; };
#define DMA_HEAP_IOC_MAGIC 'H' diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 45e82cda7c44..adc0abcf80fa 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -49,6 +49,7 @@ #include <linux/slab.h> #include <linux/swapops.h> #include <linux/spinlock.h> +#include <linux/file.h> #include <linux/fs.h> #include <linux/seq_file.h> #include <linux/vmpressure.h> @@ -3168,6 +3169,88 @@ struct obj_cgroup *get_obj_cgroup_from_folio(const struct folio *folio) return objcg; }
+/* + * Charging outside the caller's own memcg subtree can escape its limits + * or push reclaim and OOM onto other cgroups. + */ +static bool mem_cgroup_may_charge(struct mem_cgroup *memcg) +{ + struct mem_cgroup *self = get_mem_cgroup_from_current(); + bool ret = mem_cgroup_is_descendant(memcg, self); + + mem_cgroup_put(self); + return ret || capable(CAP_SYS_RESOURCE); +} + +/** + * get_obj_cgroup_from_fd - resolve an object cgroup from a cgroupfd or pidfd + * @fd: file descriptor (cgroupfd or pidfd) + * + * Tries @fd as a cgroupfd first; if that fails, tries it as a pidfd. Only + * cgroup2 directories are accepted as cgroupfds, any other fd is tried as a + * pidfd. + * + * Resolving a memcg outside the caller's own subtree requires CAP_SYS_RESOURCE. + * + * On success returns a referenced objcg pointer. On failure returns + * an ERR_PTR. + */ +struct obj_cgroup *get_obj_cgroup_from_fd(int fd) +{ + struct cgroup_subsys_state *css; + struct obj_cgroup *objcg; + struct mem_cgroup *memcg; + struct task_struct *task; + unsigned int pidfd_flags; + + if (mem_cgroup_disabled()) + return ERR_PTR(-EOPNOTSUPP); + + CLASS(fd_raw, f)(fd); + if (fd_empty(f)) + return ERR_PTR(-EBADF); + + css = css_tryget_online_from_dir(fd_file(f)->f_path.dentry, + &memory_cgrp_subsys); + if (!IS_ERR(css) && !cgroup_on_dfl(css->cgroup)) { + css_put(css); + css = ERR_PTR(-EBADF); + } + + if (IS_ERR(css)) { + if (PTR_ERR(css) != -EBADF) + return ERR_CAST(css); + + /* fd is a pidfd */ + task = pidfd_get_task(fd, &pidfd_flags); + if (IS_ERR(task)) + return ERR_CAST(task); + + rcu_read_lock(); + css = task_css(task, memory_cgrp_id); + put_task_struct(task); + if (!css || !css_tryget_online(css)) { + rcu_read_unlock(); + return ERR_PTR(-ENOENT); + } + rcu_read_unlock(); + } + + memcg = mem_cgroup_from_css(css); + if (!mem_cgroup_may_charge(memcg)) { + mem_cgroup_put(memcg); + return ERR_PTR(-EPERM); + } + + objcg = get_obj_cgroup_from_memcg(memcg, numa_node_id()); + mem_cgroup_put(memcg); + + if (!objcg) + return ERR_PTR(-ENOENT); + + return objcg; +} + #ifdef CONFIG_MEMCG_NMI_SAFETY_REQUIRES_ATOMIC static inline void account_kmem_nmi_safe(struct mem_cgroup *memcg, int val) {
The security_memcg_charge_task() hook allows security modules to control which processes may charge/transfer memcg memory to another process's cgroup outside their tree. Without a policy implementation, the hook is a no-op and the restriction is not enforced.
On SELinux-managed systems the only limit is CAP_SYS_RESOURCE. A domain that holds both dma-heap access (today's only exporter that opts-in) and that capability, can charge memory to any cgroup outside its own subtree. Since the capability is all-or-nothing, SELinux policy cannot express which targets such a domain may charge to.
Implement selinux_memcg_charge_task() using avc_has_perm() from the process2 object class and a new charge_to permission. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t client_app_t:process2 charge_to;
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 7 +++++++ security/selinux/include/classmap.h | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 6168dd5f91c2..70de6094b2b4 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2204,6 +2204,12 @@ static int selinux_capable(const struct cred *cred, struct user_namespace *ns, return cred_has_capability(cred, cap, opts, ns == &init_user_ns); }
+static int selinux_memcg_charge_task(const struct cred *from, const struct cred *to) +{ + return avc_has_perm(cred_sid(from), cred_sid(to), + SECCLASS_PROCESS2, PROCESS2__CHARGE_TO, NULL); +} + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7681,6 +7687,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capget, selinux_capget), LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), + LSM_HOOK_INIT(memcg_charge_task, selinux_memcg_charge_task), LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index 90cb61b16425..4004c05aae5b 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -60,7 +60,7 @@ const struct security_class_mapping secclass_map[] = { "siginh", "setrlimit", "rlimitinh", "dyntransition", "setcurrent", "execmem", "execstack", "execheap", "setkeycreate", "setsockcreate", "getrlimit", NULL } }, - { "process2", { "nnp_transition", "nosuid_transition", NULL } }, + { "process2", { "nnp_transition", "nosuid_transition", "charge_to", NULL } }, { "system", { "ipc_info", "syslog_read", "syslog_mod", "syslog_console", "module_request", "module_load", "firmware_load",
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's objcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. Concurrent transfers are serialized by the dma-buf reservation lock, so lockdep covers the locking order of the charge path early. The transfer implementation is generic (obj_cgroup_transfer_charge) and not dma-buf specific, so it can be reused by other subsystems.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. The xfer ioctl returns -EOPNOTSUPP for buffers whose objcg is not set. Exporters opt in by passing a objcg through dma_buf_export_info at export time. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Reviewed-by: Christian König christian.koenig@amd.com Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 2 +- drivers/dma-buf/dma-buf.c | 39 ++++++++++++++++++++++ include/linux/dma-buf.h | 3 +- include/linux/memcontrol.h | 13 ++++++++ include/uapi/linux/dma-buf.h | 26 +++++++++++++++ mm/memcontrol.c | 58 +++++++++++++++++++++++++++++++++ 6 files changed, 139 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index c6bf2e435d74..8d10ff7ac745 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1670,7 +1670,7 @@ The following nested keys are defined. Amount of memory used for exported DMA buffers allocated by or on behalf of the cgroup, this currently includes system heap allocations. Stays with the cgroup regardless of how the buffer - is shared. + is shared, unless moved with DMA_BUF_IOCTL_XFER_CHARGE.
workingset_refault_anon Number of refaults of previously evicted anonymous pages. diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 88e0802fa05f..5d4b55b67e99 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,42 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif
+/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf, + const void __user *user_data) +{ + struct dma_buf_transfer_charge arg; + struct obj_cgroup *objcg; + unsigned int nr_pages; + int ret = 0; + + if (copy_from_user(&arg, user_data, sizeof(arg))) + return -EFAULT; + + if (arg.flags != 0) + return -EINVAL; + + objcg = get_obj_cgroup_from_fd(arg.charge_fd); + if (IS_ERR(objcg)) + return PTR_ERR(objcg); + + dma_resv_lock(dmabuf->resv, NULL); + + if (!dmabuf->objcg) { + ret = -EOPNOTSUPP; + goto out; + } + + nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + if (!obj_cgroup_transfer_charge(&dmabuf->objcg, objcg, MEMCG_DMABUF, nr_pages)) + ret = -ENOMEM; + +out: + dma_resv_unlock(dmabuf->resv); + obj_cgroup_put(objcg); + return ret; +} + static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +626,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
+ case DMA_BUF_IOCTL_XFER_CHARGE: + return dma_buf_xfer_charge(dmabuf, (const void __user *)arg); + default: return -ENOTTY; } diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index ff4678d00d8c..916d73ae6a51 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -508,7 +508,8 @@ struct dma_buf_attachment { * @objcg: object cgroup to charge for this buffer's memory. Set only for * buffers backed by system memory, not accounted elsewhere, where * @size reflects the actual allocation in bytes. The buffer takes - * over the caller's reference. + * over the caller's reference. Enables charge transfer via + * DMA_BUF_IOCTL_XFER_CHARGE. * * This structure holds the information required to export the buffer. Used * with dma_buf_export() only. diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 8e016d9b91bc..37c81be349b8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1827,6 +1827,11 @@ static inline struct obj_cgroup *get_obj_cgroup_from_current(void) int obj_cgroup_charge(struct obj_cgroup *objcg, gfp_t gfp, size_t size); void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size);
+bool obj_cgroup_transfer_charge(struct obj_cgroup **ptr_objcg, + struct obj_cgroup *new_objcg, + enum memcg_stat_item item, + unsigned int nr_pages); + extern struct static_key_false memcg_bpf_enabled_key; static inline bool memcg_bpf_enabled(void) { @@ -1930,6 +1935,14 @@ static inline struct obj_cgroup *get_obj_cgroup_from_current(void) return NULL; }
+static inline bool obj_cgroup_transfer_charge(struct obj_cgroup **ptr_objcg, + struct obj_cgroup *new_objcg, + enum memcg_stat_item item, + unsigned int nr_pages) +{ + return true; +} + static inline bool memcg_bpf_enabled(void) { return false; diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..0aaf2e7d593b 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; };
+/** + * struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge + * + * Any holder of a dma-buf file descriptor can perform a + * DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory + * cost to a specific cgroup, or move it from one cgroup to + * another. The ioctl charges the new cgroup before uncharging + * the old one atomically. On failure the original charge is + * preserved. If the buffer has no existing memcg charge, the + * ioctl returns -EOPNOTSUPP. + */ +struct dma_buf_transfer_charge { + /** @flags: Reserved, must be zero */ + __u32 flags; /* unused for now */ + /** + * @charge_fd: File descriptor identifying the target cgroup. + * + * May be either a pidfd, in which case the charge moves to the + * memory cgroup of the referenced process, or a cgroupfd obtained + * by opening a cgroup v2 directory, in which case the charge moves + * directly to that cgroup's memory controller. + */ + __u32 charge_fd; +}; + #define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
@@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge)
#endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index adc0abcf80fa..03cd90d5e623 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3251,6 +3251,64 @@ struct obj_cgroup *get_obj_cgroup_from_fd(int fd) return objcg; }
+/** + * obj_cgroup_transfer_charge - move a charge between object cgroups + * @ptr_objcg: pointer to the object cgroup to transfer from + * @new_objcg: target object cgroup; caller must hold a reference + * @item: memcg stat counter to adjust; must not be a node stat + * @nr_pages: number of pages to transfer + * + * Transfers @nr_pages of charge from *@ptr_objcg to @new_objcg and, on + * success, stores @new_objcg in *@ptr_objcg. Does nothing if both resolve + * to the same memcg; *@ptr_objcg is left untouched in that case. + * + * Updates to *@ptr_objcg must be serialized by the caller. + * Returns %true on success, %false if the charge against @new_objcg + * failed. + */ +bool obj_cgroup_transfer_charge(struct obj_cgroup **ptr_objcg, + struct obj_cgroup *new_objcg, + enum memcg_stat_item item, + unsigned int nr_pages) +{ + struct obj_cgroup *old_objcg = *ptr_objcg; + struct mem_cgroup *old_memcg = NULL, *new_memcg; + bool ret = true; + + if (!new_objcg || mem_cgroup_disabled()) + return true; + + new_memcg = get_mem_cgroup_from_objcg(new_objcg); + if (old_objcg) + old_memcg = get_mem_cgroup_from_objcg(old_objcg); + + if (old_memcg == new_memcg) + goto out; + + if (try_charge(new_memcg, GFP_KERNEL, nr_pages)) { + ret = false; + goto out; + } + + mod_memcg_state(new_memcg, item, nr_pages); + + if (old_objcg) { + /* Uncharge old cgroup */ + mod_memcg_state(old_memcg, item, -nr_pages); + if (!mem_cgroup_is_root(old_memcg)) + refill_stock(old_memcg, nr_pages); + } + + obj_cgroup_get(new_objcg); + *ptr_objcg = new_objcg; + obj_cgroup_put(old_objcg); + +out: + mem_cgroup_put(new_memcg); + mem_cgroup_put(old_memcg); + return ret; +} + #ifdef CONFIG_MEMCG_NMI_SAFETY_REQUIRES_ATOMIC static inline void account_kmem_nmi_safe(struct mem_cgroup *memcg, int val) {
linaro-mm-sig@lists.linaro.org