Hi Mauro,
Mauro Carvalho Chehab wrote:
After the mm panels, I had a few discussions with Hans, Rob and Daniel, among others, during the V4L and KMS discussions and after that. Based on those discussions, I'm pretty much convinced that the normal MMAP way of streaming (VIDIOC_[REQBUF|STREAMON|STREAMOFF|QBUF|DQBUF ioctl's) are not the best way to share data with framebuffers. We probably need something that it is close to VIDIOC_FBUF/VIDIOC_OVERLAY, but it is still not the same thing.
I suspect that working on such API is somewhat orthogonal to the decision of using a file pointer based or a bufer ID based based kABI for passing the buffer parameters to the newly V4L calls, but we cannot decide about the type of buffer ID that we'll use if we not finish working at an initial RFC for the V4L API, as the way the buffers will be passed into it will depend on how we design such API.
It should be also noticed that, while in the shared buffers some definitions can be postponed to happen later (as it is basically a Kernelspace-only ABI - at least initially), the V4L API should be designed to consider all possible scenarios, as "diamonds and userspace API's are forever"(tm).
It seems to me that the proper way to develop such API is starting working with Xorg V4L driver, changing it to work with KMS and with the new API (probably porting some parts of it to kernelspace).
One of the problems with a shared framebuffer is that an overlayed V4L stream may, at the worse case, be sent to up to 4 different GPU's and/or displays, like:
===================+=================== | | | | D1 +----|---+ D2 | | | V4L| | | +-------------|----+---|--------------| | | | | | | D3 +----+---+ D4 | | | | =======================================
Where D1, D2, D3 and D4 are 4 different displays, and the same V4L framebuffer is partially shared between them (the above is an example of a V4L input, although the reverse scenario of having one frame buffer divided into 4 V4L outputs also seems to be possible).
As the same image may be divided into 4 monitors, the buffer filling should be synced with all of them, in order to avoid flipping effects. Also, the buffer can't be re-used until all displays finish reading.
If this were four different GPUs it would be quite difficult to map the memory so that a V4L2 device would be able to use a GPU display memory as its output the way you described above --- assuming that D[1-4] memories are separate. Typically hardware requires that each line is contiguous in memory (either physical or virtual) and there's a static offset between each two consecutive lines. (Please correct me if I've misunderstood some of the above.)
If one was to implement such a case, X will handle (hopefully!) the drawing of the buffer to the $DISPLAY, and passing the buffer objects between V4L2 and XV would be handled by a user space process.
As Laurent pointed out, being able to share buffers between various subsystems in a generic way is what we're really missing right now.
Albeit a special case, the omapxvsink GStreamer element is a very good example of this: it maps physically contiguous FBDEV (system) memory to user space which then uses that as V4L2 USERPTR buffers and achieves zero copy from the ISP to the display.
Currently the user has to be aware that the memory must be allocated by the omapxvsink (and not a camera source GStreamer plugin, for example) so that this can be done. Being able to make this type of use cases more generic and efficient is important.
The existing V4L2 ioctls are good for this --- what we might need is just a new memory type for the generic DMA buffers.
Cheers,