Hi Rob,
- flipping between multiple back buffers, perhaps not in order (to handle video formats with B-frames)
Oh, yes please. The closed source drivers seems to do this also all the time, and I never really understood why DRI is limiting the buffers to the OGL attachment points.
...
Current solutions use the GPU to do a scaled/colorconvert into a DRI2 buffer from the client context. The goal of this protocol change is to push the decision to use overlay or GPU blit to the xorg driver.
You left one corner case out, HDMI allows for the framebuffer to be in YUV 4:4:4 format. So it is possible to send YUV data to the display (usually a TV) without color conversion at all, but I think with the current state of X we are years away from that.
...
In many cases, an overlay will avoid several passes through memory (blit/scale/colorconvert to DRI back-buffer on client side, blit to front and fake-front, and then whatever compositing is done by the window manager). On the other hand, overlays can often be handled directly by the scanout engine in the display hardware, with the GPU switched off.
Actually AMD has thrown out the hardware support for overlay with the R7xx (or was it evergreen?) series, because they got support for turning shader pipes off separately and figured out that it use less power to turn off all shaders except one and then use this one for color conversion and scaling, compared to having a distinct hardware block doing the job. But there are tendencies to get a distinct color conversion block back again.
...
Note: video is not exactly the same as 3d, there are a number of other things to consider (scaling, colorconvert, multi-planar formats). But on the other hand the principle is similar (direct rendering from hw video codecs). And a lot infrastructure of connection, authentication, is same. So there are two options, either extend DRI2 or add a new protocol which duplicates some parts. I'd like to consider extending DRI2 first, but if people think the requirements for video are too much different from 3d, then I could split this into a new protocol.
If you ask me extending things seems the better way to do this.
..
@@ -184,6 +185,11 @@ DRI2ATTACHMENT { DRI2BufferFrontLeft These values describe various attachment points for DRI2 buffers.
- In the case of video driver (DRI2DriverXV) the attachment,
- other than DRI2BufferFrontLeft, just indicates buffer
- number and has no other special significance. There is no
- automatic maintenance of DRI2BufferFakeFrontLeft.
I think that will created compatibility problems with existing implementations, because the DDX side doesn't know if it's talking to a video or 3D client side.
...
- The "XV_OSD" attribute specifies the XID of a pixmap containing
- ARGB data to be non-destructively overlayed over the video. This
- could be used to implement subtiles, on-screen-menus, etc.
Why an XID? I'm not 100% sure about it, but using a DRI buffer name directly here seems to be the better alternative.
Are you targeting/limiting this to a particular API (or the customary limitations of overlay HW)? I ask because VDPAU allows clients to pass in an arbitrary colour conversion matrix rather than color standard/hue/sat/bri/con, so it wouldn't be possible to use this in that context.
Ideally it would something that could be used either from device-dependent VDPAU or VAAPI driver back-end, or something that could be used in a generic way, for example GStreamer sink element that could be used with software codecs.
AFAIK DRI mostly isn't a device driver dependent protocol, and the client side doesn't necessary know to which hardware it is talking, just look at how gallium 3D is working, talking with X over the DRI protocol is part of the driver independent state tracker, and NOT part of the driver itself.
So having this driver independent is just a must have, and not optional.
Well, this is the goal anyways. There is one slight other complication for use in a generic way.. it would need to be a bit better defined what the buffer 'name' is, so that the client side would know how to interpret it, mmap it if needed. But I think there is a solution brewing: http://lists.linaro.org/pipermail/linaro-mm-sig/2011-August/000509.html
That's indeed true, but currently it is the only parameter that must be interpreted in a driver dependent fashion. And I really think it should stay that way.
As far as color conversion matrix... well, the attribute system can have arbitrary device-dependent attributes. In the VDPAU case, I suppose the implementation on the client side knows which xorg driver it is talking to, and could introduce it's own attributes. Perhaps a bit awkward for communicating a matrix, but you could in theory have 4*3 different attributes (ie. XV_M00, XV_M01, ... XV_M23) for each entry in the matrix.
Yes, but you should define that clearly in the protocol, we also need something to make the client side know what is supported: individual values or matrix or both?
Also in general, their compositing API is a lot more flexible and allows for a background + multiple layers, rather than just a single layer. I suppose you could pre-flatten the layers into a single one, but the background would be problematic.
Yeah, pre-flatten into a single layer, I think. I mean, we *could* push that to xorg driver side too, but I was trying to not make things overly complicated in the protocol.
I'm not sure I caught the issue about background.. or are you thinking about video w/ AYUV? Is there any hw out there that supports overlays w/ YUV that has an alpha channel? If this is enough of a weird edge case, maybe it is ok to fall back in these cases to the old way of doing the blending on the client side and just looking like a 3d app to the xorg side. (I suspect in this sort of case you'd end up falling back to the GPU on the xorg side otherwise.) But I'm always interested to hear any other suggestions.
VDPAU indeed defines two YUVA formats you can use, but currently I haven't seen anybody using the background picture functionality so far, because there are just not so much YUVA videos around.
Christian.