This discussion reminded me of a very cool use-case thats looming in the future:
video analytics
In video analytics you'd: capture analyze (probably from user space or in a DSP - either way some data is flowing up the stack) encode to a texture possibly send back through analyze
To make this fast, one buffer may transition between many owners - and having multiple mappings may make that easier to deal with.
I think people are suggesting a multiple map scenario, where only one mapper is live at a time. We could make this on-demand where the appropriate fixups for a given mapping are done when that mapping becomes live.
On 21 April 2011 03:04, Hans Verkuil hverkuil@xs4all.nl wrote:
On Wednesday, April 20, 2011 23:24:14 Zach Pfeffer wrote:
That's true.
I think there's some value in splitting the discussion up into 3 areas:
Allocation Mapping Buffer flows
I agree with this split. They are for the most part independent problems, but solutions for all three are needed to make a zero-copy pipeline a reality.
With regards to V4L: there is currently no buffer sharing possible at all (except in the trivial case of buffers in virtual memory, and of course when using CMEM-type solutions). Even between a V4L capture and output device you cannot share buffers. The main reason is that the videobuf framework used in drivers was a horrid piece of code that prevented any sharing. In 2.6.39 the videobuf2 replacement framework was merged that will make such sharing much, much easier.
This functionality is not yet implemented though, partially because this is all still very new, and partially because we would like to wait and see if a general solution will be created.
It should be fairly easy to implement such a solution in the videobuf2 framework. Buffer 'handles' can be either 32 or 64 bit, so we have no restriction there. File descriptors would be fine as handle.
A requirement to enable buffer sharing between V4L and anything else would probably be that the V4L driver must use videobuf2. Since this is brand new, the initial set of compatible drivers will be very small, but that should improve over time. Anyway, that's our problem :-)
A separate problem that I would like to discuss is the interaction of GPU/FB and V4L display devices. There is overlap there and we need to figure out how to handle that. However, this is unrelated to the memory summit. It might be a good topic for a meeting on Tuesday and/or Wednesday morning, though.
Regards,
Hans
There seems to be a general design disconnect between the way Linux deals with buffer mapping (one mapper at a time, buffers are mapped and unmapped as they flow through the system) and the way users actually want things to work (sharing a global buffer that one entity writes to and another reads without unmapping each time).
Perhaps there's a solution that will give users the allusion of shared mappings while ensuring correctness if those mappings are different (something on-demand perhaps).
-Zach
On 20 April 2011 10:13, Jordan Crouse jcrouse@codeaurora.org wrote:
On 04/19/2011 10:12 PM, Zach Pfeffer wrote:
Speaking of Graphics and Multimedia - we may want to discuss IOMMU APIs and distributed memory management. These devices are becoming more prevalent and having a standard way of working with them would be useful.
I did a little of this work at Qualcomm and pushed some soundly rejected patches to the kernel, see "mm: iommu: An API to unify IOMMU, CPU and device memory management."
-Zach
As we talked during the meeting at ELC, IOMMU is important, but I think that there is broad agreement to consolidate (eventually) on the standard APIs. I still think that the memory allocation problem is the more interesting one because it affects everybody equally, MMU or not. Not that I want to shut down debate or anything, I just don't want to distract us from the larger problem that we face.
Jordan
On 19 April 2011 20:52, Clark, Robrob@ti.com wrote:
On Mon, Apr 18, 2011 at 9:45 AM, Sree Kumarsreeon@gmail.com wrote:
Thanks Jesse for initiating the mailing list.
We need to address the requirements of Graphics and Multimedia Accelerators (IPs). What we really need is a permanent solution (at upstream) which accommodates the following requirements and conforms to Graphics and Multimedia use cases.
1.Mechanism to map/unmap the memory. Some of the IPs’ have the ability to address virtual memory and some can address only physically contiguous address space. We need to address both these cases. 2.Mechanism to allocate and release memory. 3.Method to share the memory (ZERO copy is a MUST for better performance) between different device drivers (example output of camera to multimedia encoder). 4.Method to share the memory with different processes in userspace. The sharing mechanism should include built-in security features.
Are there any special requirements from V4L or DRM perspectives?
From DRI perspective.. I guess the global buffer name is restricted to a 4 byte integer, unless you change the DRI proto..
Authentication hooks for the driver (on x11 driver side) are for a single authentication covering all buffers shared between client and server, and is done by 4 byte token exchange between client and server. I've not had time yet to look more closely at the authentication aspect of ION.
Those are just things off the top of my head, hopefully someone else from X11 world chimes in with whatever else I missed. But I guess most important thing is whether or not it can fit within existing DRI protocol. If it does, then the drivers on client and server side could use whatever..
BR, -R
Thanks, Sree
Linaro-mm-sig mailing list Linaro-mm-sig@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-mm-sig
Linaro-mm-sig mailing list Linaro-mm-sig@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-mm-sig
Linaro-mm-sig mailing list Linaro-mm-sig@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-mm-sig
-- Jordan Crouse Qualcomm Innovation Center Qualcomm Innovation Center is a member of Code Aurora Forum
Linaro-mm-sig mailing list Linaro-mm-sig@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-mm-sig