Hi Ben,

Sorry overlooked this mail.

On Mon, Apr 11, 2011 at 11:18 PM, Benjamin Gaignard benjamin.gaignard@linaro.org wrote:

Hi,

Could you provide us Jason's detailed analysis about V4L2 API gap/issues ?

Do you already have in mind a list of requirements for this API (frame based, asynchronous, memory management/allocation, power ...) ?

I don't have a specific list of requirements, just wondering what is the suggestion from multimedia team, provided so many APIs are out there.

So below is basically Jason's analysis of using V4L2 API for the video encoding and decoding:

===========>8===========

1. Architecture

It's a simple and easy understand arch which based on v4l2 capture framework. It uses file io write/read to put/get bitstream, and file io open/close to support multi-instance. V4l2 is used to manage video buffers of decode and encode data.

2. Flow

For decoding:

1) vpu_write(1st frame) -- get decoder info 2) v4l2_g_fmt (width/height) 3) v4l2_reqbufs 4) v4l2_qbuf all bufs and streamon 5) vpu_write(bit stream) 6) poll_interrupt 7) v4l2_dqbuf 8) v4l2_qbuf 9) loop to step 5)

A vpu_start_frame operation could be actived by vpu_write, v4l2_qbuf or in interrupt handler if there is no active v4l2_buffer in processing.

For encoding:

1)v4l2_s_fmt (width/height) 2)v4l2_reqbufs 3)streamon 4)v4l2_qbuf 5)v4l2_dqbuf 6)wait queue active by interrupt 7)vpu_read(bit stream) 8)loop to step 4)

A vpu_start_frame operation could be actived by v4l2_qbuf or in interrupt handler if there is no active v4l2_buffer in processing.

3. Limitations

* direct rendering requirement

There is only V4L2_MEMORY_MMAP support which make a memcopy must, V4L2_MEMORY_USERPTR needed to support buffers allocated from gst sink plugin

* some tricky modes support

e.g. it can not set/get frame based option, for example, during decoding, after every frame decode finish, MMF should could get whether it's an I-frame which is useful to seek mode. Based on this framework, it's not easy to add such feature. (such output info should be attached on each video buf)

* H264 reorder or IPBB sequence decode

the driver do decoding according to the input sequence, it says output frame will follow the input frame sequence, under cases of h264 reorder or clip encoded with IPBB order, the output may be out of order, RET_DEC_PIC_FRAME_IDX (display index) and RET_DEC_PIC_CUR_IDX(decoded index) need to be supported during decoding.

* multi-instances

VPU can not start another decoding/encoding operation before current decoding/encoding frame finish, the driver needs to take care of this

* file play mode support

For file play mode enable, we need to set CMD_DEC_PIC_CHUNK_SIZE and CMD_DEC_PIC_BB_START registers before decoding, that’s for the indication of input bitstream and stream size of this frame.

* error handling

How errors during encoding/decoding could be handled

I think 3) is not easy to enabled with V4L2 framework, and it's very important.

Benjamin

2011/4/11 Eric Miao eric.miao@linaro.org

...

Hi All,

Not really sure how this is being discussed in the Multimedia WG. But do we yet have a standardized kernel driver framework for video acceleration?

A bit investigation showed a chaos in this area:

1. Different SoC vendors are using their specific APIs

2. The freescale case, there is a proposed driver from Sascha using V4L2

interface, yet our analysis showed some issues with this API to support a full featured usage, and the API itself doesn't just seem to be right for this (thanks for Jason for a detailed analysis)

3. XvMC (obsolete I believe)

4. nVidia's VDPAU, which is supported by XBMC and many others though

5. VA API (Intel proposed and currently available on Intel graphics), can

use VDPAU as a backend

6. XvBA - AMD is proposing this for the ATI video HW

• eric

---

linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev

---

linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev