On Thursday, February 10, 2011 08:17:31 Linus Walleij wrote:
Thanks for the help Harald, much appreciated.
On Wed, Feb 9, 2011 at 8:44 PM, Harald Gustafsson harald.gustafsson@ericsson.com wrote:
OMX main purpose is to handle multimedia hardware and offer an interface to that HW that looks identical indenpendent of the vendor delivering that hardware, much like the v4l2 or USB subsystems tries to do. And yes optimally it should be implemented in drivers/omx in Linux and a user space library on top of that.
Thanks for clarifying this part, it was unclear to me. The reason being that it seems OMX does not imply userspace/kernelspace separation, and I was thinking more of it as a userspace lib. Now my understanding is that if e.g. OpenMAX defines a certain data structure, say for a PCM frame or whatever, then that exact struct is supposed to be used by the kernelspace/userspace interface, and defined in the include file exported by the kernel.
It might be that some alignment also needs to be made between 4vl2 and other OS's implementation, to ease developing drivers for many OSs (sorry I don't know these details, but you ST-E guys should know).
The basic conflict I would say is that Linux has its own API+ABI, which is defined by V4L and ALSA through a community process without much thought about any existing standard APIs. (In some cases also predating them.)
By the way IL is about to finalize version 1.2 of OpenMAX IL which is more than a years work of aligning all vendors and fixing unclear and buggy parts.
I suspect that the basic problem with Khronos OpenMAX right now is how to handle communities - for example the X consortium had something like the same problem a while back, only member companies could partake in the standard process, and they need of course to pay an upfront fee for that, and the majority of these companies didn't exactly send Linux community members to the meetings.
And now all the companies who took part in OpenMAX somehow end up having to do a lot of upfront community work if they want to drive the API:s in a certain direction, discuss it again with the V4L and ALSA maintainers and so on. Which takes a lot of time and patience with uncertain outcome, since this process is autonomous from Khronos. Nobody seems to be doing this, I javen't seen a single patch aimed at trying to unify the APIs so far. I don't know if it'd be welcome.
This coupled with strict delivery deadlines and a marketing will to state conformance to OpenMAX of course leads companies into solutions breaking the Linux kernelspace API to be able to present this.
Now I think we have a pretty clear view of the problem, I don't know what could be done about it though :-/
One option might be to create a OMX wrapper library around the V4L2 API. Something similar is already available for the old V4L1 API (now removed from the kernel) that allows apps that still speak V4L1 only to use the V4L2 API. This is done in the libv4l1 library. The various v4l libraries are maintained here: http://git.linuxtv.org/v4l-utils.git
Adding a libomx might not be such a bad idea. Linaro might be the appropriate organization to look into this. Any missing pieces in V4L2 needed to create a fully functioning omx API can be discussed and solved.
Making this part of v4l-utils means that it is centrally maintained and automatically picked up by distros.
It will certainly be a non-trivial exercise, but it is a one-time job that should solve a lot of problems. But someone has to do it...
Regarding using V4L to communicate with DSPs/other processors: that too could be something for Linaro to pick up: experiment with it for one particular board, see what (if anything) is needed to make this work. I expect it to be pretty easy, but again, nobody has actually done the initial work.
Once you have an example driver, then it should be much easier for others to follow.
As Linus said, companies are unlikely to start doing this by themselves, but it seems that this work would exactly fit the Linaro purpose. From the Linaro homepage:
"Linaro™ brings together the open source community and the electronics industry to work on key projects, deliver great tools, reduce industry wide fragmentation and provide common foundations for Linux software distributions and stacks to land on."
Spot on, I'd say :-)
Just for the record, let me say again they the V4L2 community will be very happy to assist with this when it comes to extending/improving the V4L2 API to make all this possible.
Regards,
Hans