On Thu, Apr 03, 2025 at 09:39:52AM +0200, Christian König wrote:
For the UMA GPU case where there is no device memory or eviction problem, perhaps a configurable option to just say account memory in memcg for all allocations done by this process, and state yes you can work around it with allocation servers or whatever but the behaviour for well behaved things is at least somewhat defined.
We can have that as a workaround, but I think we should approach that differently.
With upcoming CXL even coherent device memory is exposed to the core OS as NUMA memory with just a high latency.
So both in the CXL and UMA case it actually doesn't make sense to allocate the memory through the driver interfaces any more. With AMDGPU for example we are just replicating mbind()/madvise() within the driver.
Instead what the DRM subsystem should aim for is to allocate memory using the normal core OS functionality and then import it into the driver.
AMD, NVidia and Intel have HMM working for quite a while now but it has some limitations, especially on the performance side.
So for AMDGPU we are currently evaluating udmabuf as alternative. That seems to be working fine with different NUMA nodes, is perfectly memcg accounted and gives you a DMA-buf which can be imported everywhere.
The only show stopper might be the allocation performance, but even if that's the case I think the ongoing folio work will properly resolve that.
I mean, no, the showstopper to that is that using udmabuf has the assumption that you have an IOMMU for every device doing DMA, which is absolutely not true on !x86 platforms.
It might be true for all GPUs, but it certainly isn't for display controllers, and it's not either for codecs, ISPs, and cameras.
And then there's the other assumption that all memory is under the memory allocator control, which isn't the case on most recent platforms either.
We *need* to take CMA into account there, all the carved-out, device specific memory regions, and the memory regions that aren't even under Linux supervision like protected memory that is typically handled by the firmware and all you get is a dma-buf.
Saying that it's how you want to workaround it on AMD is absolutely fine, but DRM as a whole should certainly not aim for that, because it can't.
Maxime