Hi Alan and Rob,
On Monday 19 September 2011 02:09:36 Rob Clark wrote:
On Sun, Sep 18, 2011 at 5:23 PM, Alan Cox alan@lxorguk.ukuu.org.uk wrote:
This would leave us with the issue of controlling formats and other parameters on the pipelines. We could keep separate DRM, KMS, FB and V4L APIs for that,
There are some other differences that matter. The exact state and behaviour of memory, sequencing of accesses, cache control and management are a critical part of DRM for most GPUs, as is the ability to have them in swap backed objects and to do memory management of them. Fences and the like are a big part of the logic of many renderers and the same fencing has to be applied between capture and GPU, and also in some cases between playback accelerators (eg MP4 playback) and GPU.
That's why I believe the DRM API is our best solution to address all those issues.
I'm not advocating merging the DRM, FB and V4L APIs for memory management. What I would like to investigate is whether we can use a common API for the common needs, which are (in my opinion):
- reporting the entities that make up the graphics pipeline (such as planes, overlays, compositors, transmitters, connectors, ...), especially when pipelines get more complex than the plane->crtc->encoder->connector DRM model
- configuring data routing in those complex pipelines
- and possibly configuring formats (pixel format, frame size, crop rectangle, composition rectangle, ...) on those entities
To glue them together I think you'd need to support the use of GEM objects (maybe extended) in V4L. That may actually make life cleaner and simpler in some respects because GEM objects are refcounted nicely and have handles.
fwiw, I think the dmabuf proposal that linaro GWG is working on should be sufficient for V4L to capture directly into a GEM buffer that can be scanned out (overlay) or composited by GPU, etc, in cases where the different dma initiators can all access some common memory:
http://lists.linaro.org/pipermail/linaro-mm-sig/2011-September/000616.html
The idea is that you could allocate a GEM buffer, export a dmabuf handle for that buffer that could be passed to v4l2 camera device (ie. V4L2_MEMORY_DMABUF), video encoder, etc.. the importing device should bracket DMA to/from the buffer w/ get/put_scatterlist() so an unused buffer could be unpinned if needed.
I second Rob here, I think that API should be enough to solve our memory sharing problems between different devices. This is a bit out of scope though, as neither the low-level Linux display framework proposal nor my comments target that, but it's an important topic worth mentioning.
DRM and KMS abstract out stuff into what is akin to V4L subdevices for the various objects the video card has that matter for display - from scanout buffers to the various video outputs, timings and the like.
I don't know what it's like with OMAP but for some of the x86 stuff particularly low speed/power stuff the capture devices, GPU and overlays tend to be fairly incestuous in order to do things like 1080i/p preview while recording from the camera.
We don't like extra memcpy's, but something like dmabuf fits us nicely.. and I expect it would work well in any sort of UMA system where camera, encoder, GPU, overlay, etc all can share the same memory and formats. I suspect the situation is similar in the x86 SoC world.. but would be good to get some feedback on the proposal. (I guess next version of the RFC would go out to more mailing lists for broader review.)
GPU is also a bit weird in some ways because while its normally nonsensical to do things like use the capture facility one card to drive part of another, it's actually rather useful (although not supported really by DRM) to do exactly that with GPUs. A simple example is a dual headed box with a dumb frame buffer and an accelerated output both of which are using memory that can be hit by the accelerated card. Classic example being a USB plug in monitor.