Am 01.11.22 um 22:09 schrieb Nicolas Dufresne:
[SNIP]
But the client is just a video player. It doesn't understand how to allocate BOs for Panfrost or AMD or etnaviv. So without a universal allocator (again ...), 'just allocate on the GPU' isn't a useful response to the client.
Well exactly that's the point I'm raising: The client *must* understand that!
See we need to be able to handle all restrictions here, coherency of the data is just one of them.
For example the much more important question is the location of the data and for this allocating from the V4L2 device is in most cases just not going to fly.
It feels like this is a generic statement and there is no reason it could not be the other way around.
And exactly that's my point. You always need to look at both ways to share the buffer and can't assume that one will always work.
As far as I can see it you guys just allocate a buffer from a V4L2 device, fill it with data and send it to Wayland for displaying.
To be honest I'm really surprised that the Wayland guys hasn't pushed back on this practice already.
This only works because the Wayland as well as X display pipeline is smart enough to insert an extra copy when it find that an imported buffer can't be used as a framebuffer directly.
I have colleague who integrated PCIe CODEC (Blaize Xplorer X1600P PCIe Accelerator) hosting their own RAM. There was large amount of ways to use it. Of course, in current state of DMABuf, you have to be an exporter to do anything fancy, but it did not have to be like this, its a design choice. I'm not sure in the end what was the final method used, the driver isn't yet upstream, so maybe that is not even final. What I know is that there is various condition you may use the CODEC for which the optimal location will vary. As an example, using the post processor or not, see my next comment for more details.
Yeah, and stuff like this was already discussed multiple times. Local memory of devices can only be made available by the exporter, not the importer.
So in the case of separated camera and encoder you run into exactly the same limitation that some device needs the allocation to happen on the camera while others need it on the encoder.
The more common case is that you need to allocate from the GPU and then import that into the V4L2 device. The background is that all dGPUs I know of need the data inside local memory (VRAM) to be able to scan out from it.
The reality is that what is common to you, might not be to others. In my work, most ARM SoC have display that just handle direct scannout from cameras and codecs.
The only case the commonly fails is whenever we try to display UVC created dmabuf,
Well, exactly that's not correct! The whole x86 use cases of direct display for dGPUs are broken because media players think they can do the simple thing and offload all the problematic cases to the display server.
This is absolutely *not* the common use case you describe here, but rather something completely special to ARM.
which have dirty CPU write cache and this is the type of thing we'd like to see solved. I think this series was addressing it in principle, but failing the import and the raised point is that this wasn't the optimal way.
There is a community project called LibreELEC, if you aren't aware, they run Khodi with direct scanout of video stream on a wide variety of SoC and they use the CODEC as exporter all the time. They simply don't have cases were the opposite is needed (or any kind of remote RAM to deal with). In fact, FFMPEG does not really offer you any API to reverse the allocation.
Ok, let me try to explain it once more. It sounds like I wasn't able to get my point through.
That we haven't heard anybody screaming that x86 doesn't work is just because we handle the case that a buffer isn't directly displayable in X/Wayland anyway, but this is absolutely not the optimal solution.
The argument that you want to keep the allocation on the codec side is completely false as far as I can see.
We already had numerous projects where we reported this practice as bugs to the GStreamer and FFMPEG project because it won't work on x86 with dGPUs.
This is just a software solution which works because of coincident and not because of engineering.
Regards, Christian.