Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3: - Introduced two wrappers with explicit names to handle destroy internal buffers (Nicolas) - Used sub state check instead of introducing new boolean (Vikash) - Addressed other comments (Vikash) - Reorderd patches to have all fixes patches first (Dmitry) - Link to v2: https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2: - Added Changes to make sure all buffers are released in session close (bryna) - Added tracking for flush responses to fix a timing issue. - Added a handling to fix timing issue in reconfig - Splitted patch 06/20 in two patches (Bryan) - Added missing fixes tag (bryan) - Updated fluster report (Nicolas) - Link to v1: https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC: - Added additional fixes to address issues identified during further testing. - Moved typo fix to a seperate patch [Neil] - Reordered the patches for better logical flow and clarity [Neil, Dmitry] - Added fixes tag wherever applicable [Neil, Dmitry] - Removed the default case in the switch statement for codecs [Bryan] - Replaced if-else statements with switch-case [Bryan] - Added comments for mbpf [Bryan] - RFC: https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-) --- base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
Firmware might hold the DPB buffers for reference in case of sequence change, so skip destroying buffers for which QUEUED flag is not removed. Also, make sure that all buffers are released during streamoff.
Cc: stable@vger.kernel.org Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue internal buffers") Reviewed-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++- drivers/media/platform/qcom/iris/iris_buffer.h | 3 ++- drivers/media/platform/qcom/iris/iris_vdec.c | 4 ++-- drivers/media/platform/qcom/iris/iris_vidc.c | 33 ++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index e5c5a564fcb8..981fedb000ed 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf return 0; }
-int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) +static int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane, bool force) { const struct iris_platform_data *platform_data = inst->core->iris_platform_data; struct iris_buffer *buf, *next; @@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) for (i = 0; i < len; i++) { buffers = &inst->buffers[internal_buf_type[i]]; list_for_each_entry_safe(buf, next, &buffers->list, list) { + /* + * during stream on, skip destroying internal(DPB) buffer + * if firmware did not return it. + * during close, destroy all buffers irrespectively. + */ + if (!force && buf->attr & BUF_ATTR_QUEUED) + continue; + ret = iris_destroy_internal_buffer(inst, buf); if (ret) return ret; @@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) return 0; }
+int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane) +{ + return iris_destroy_internal_buffers(inst, plane, true); +} + +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane) +{ + return iris_destroy_internal_buffers(inst, plane, false); +} + static int iris_release_internal_buffers(struct iris_inst *inst, enum iris_buffer_type buffer_type) { diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index c36b6347b077..00825ad2dc3a 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst *inst, u32 plane); int iris_create_internal_buffers(struct iris_inst *inst, u32 plane); int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 4143acedfc57..9c049b9671cc 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst)
iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret;
@@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst)
iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); if (ret) return ret;
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index ca0f4e310f77..a8144595cc78 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst) iris_wait_for_session_response(inst, false); }
+static void iris_check_num_queued_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data = inst->core->iris_platform_data; + struct iris_buffer *buf, *next; + struct iris_buffers *buffers; + const u32 *internal_buf_type; + u32 internal_buffer_count, i; + u32 count = 0; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + internal_buffer_count = platform_data->dec_op_int_buf_tbl_size; + } + + for (i = 0; i < internal_buffer_count; i++) { + buffers = &inst->buffers[internal_buf_type[i]]; + list_for_each_entry_safe(buf, next, &buffers->list, list) + count++; + if (count) + dev_err(inst->core->dev, "%d buffer of type %d not released", + count, internal_buf_type[i]); + } +} + int iris_close(struct file *filp) { struct iris_inst *inst = iris_get_inst(filp, NULL); @@ -233,8 +260,10 @@ int iris_close(struct file *filp) iris_session_close(inst); iris_inst_change_state(inst, IRIS_INST_DEINIT); iris_v4l2_fh_deinit(inst); - iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); - iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); iris_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock);
On 01/05/2025 20:13, Dikshita Agarwal wrote:
Firmware might hold the DPB buffers for reference in case of sequence change, so skip destroying buffers for which QUEUED flag is not removed. Also, make sure that all buffers are released during streamoff.
Cc: stable@vger.kernel.org Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue internal buffers") Reviewed-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++- drivers/media/platform/qcom/iris/iris_buffer.h | 3 ++- drivers/media/platform/qcom/iris/iris_vdec.c | 4 ++-- drivers/media/platform/qcom/iris/iris_vidc.c | 33 ++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index e5c5a564fcb8..981fedb000ed 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf return 0; } -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) +static int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane, bool force) { const struct iris_platform_data *platform_data = inst->core->iris_platform_data; struct iris_buffer *buf, *next; @@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) for (i = 0; i < len; i++) { buffers = &inst->buffers[internal_buf_type[i]]; list_for_each_entry_safe(buf, next, &buffers->list, list) {
/** during stream on, skip destroying internal(DPB) buffer* if firmware did not return it.* during close, destroy all buffers irrespectively.*/if (!force && buf->attr & BUF_ATTR_QUEUED)continue;ret = iris_destroy_internal_buffer(inst, buf); if (ret) return ret;@@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) return 0; } +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane) +{
- return iris_destroy_internal_buffers(inst, plane, true);
+}
+int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane) +{
- return iris_destroy_internal_buffers(inst, plane, false);
+}
- static int iris_release_internal_buffers(struct iris_inst *inst, enum iris_buffer_type buffer_type) {
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index c36b6347b077..00825ad2dc3a 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst *inst, u32 plane); int iris_create_internal_buffers(struct iris_inst *inst, u32 plane); int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 4143acedfc57..9c049b9671cc 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst) iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret;
@@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst) iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); if (ret) return ret;
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index ca0f4e310f77..a8144595cc78 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst) iris_wait_for_session_response(inst, false); } +static void iris_check_num_queued_internal_buffers(struct iris_inst *inst, u32 plane) +{
- const struct iris_platform_data *platform_data = inst->core->iris_platform_data;
- struct iris_buffer *buf, *next;
- struct iris_buffers *buffers;
- const u32 *internal_buf_type;
- u32 internal_buffer_count, i;
- u32 count = 0;
- if (V4L2_TYPE_IS_OUTPUT(plane)) {
internal_buf_type = platform_data->dec_ip_int_buf_tbl;internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size;- } else {
internal_buf_type = platform_data->dec_op_int_buf_tbl;internal_buffer_count = platform_data->dec_op_int_buf_tbl_size;- }
- for (i = 0; i < internal_buffer_count; i++) {
buffers = &inst->buffers[internal_buf_type[i]];list_for_each_entry_safe(buf, next, &buffers->list, list)count++;if (count)dev_err(inst->core->dev, "%d buffer of type %d not released",count, internal_buf_type[i]);- }
+}
- int iris_close(struct file *filp) { struct iris_inst *inst = iris_get_inst(filp, NULL);
@@ -233,8 +260,10 @@ int iris_close(struct file *filp) iris_session_close(inst); iris_inst_change_state(inst, IRIS_INST_DEINIT); iris_v4l2_fh_deinit(inst);
- iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); iris_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock);
I left some comments in the previous cycle
https://lore.kernel.org/linux-arm-msm/a056266e-612d-4abf-916f-3db49b00dbde@l...
I don't see those addressed in this cycle. Can you give some feedback from your POV.
--- bod
On 5/2/2025 5:44 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
Firmware might hold the DPB buffers for reference in case of sequence change, so skip destroying buffers for which QUEUED flag is not removed. Also, make sure that all buffers are released during streamoff.
Cc: stable@vger.kernel.org Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue internal buffers") Reviewed-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
drivers/media/platform/qcom/iris/iris_buffer.c | 20 +++++++++++++++- drivers/media/platform/qcom/iris/iris_buffer.h | 3 ++- drivers/media/platform/qcom/iris/iris_vdec.c | 4 ++-- drivers/media/platform/qcom/iris/iris_vidc.c | 33 ++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index e5c5a564fcb8..981fedb000ed 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -376,7 +376,7 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf return 0; } -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) +static int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane, bool force) { const struct iris_platform_data *platform_data = inst->core->iris_platform_data; struct iris_buffer *buf, *next; @@ -396,6 +396,14 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) for (i = 0; i < len; i++) { buffers = &inst->buffers[internal_buf_type[i]]; list_for_each_entry_safe(buf, next, &buffers->list, list) { + /* + * during stream on, skip destroying internal(DPB) buffer + * if firmware did not return it. + * during close, destroy all buffers irrespectively. + */ + if (!force && buf->attr & BUF_ATTR_QUEUED) + continue;
ret = iris_destroy_internal_buffer(inst, buf); if (ret) return ret; @@ -405,6 +413,16 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) return 0; } +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane) +{ + return iris_destroy_internal_buffers(inst, plane, true); +}
+int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane) +{ + return iris_destroy_internal_buffers(inst, plane, false); +}
static int iris_release_internal_buffers(struct iris_inst *inst, enum iris_buffer_type buffer_type) { diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index c36b6347b077..00825ad2dc3a 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -106,7 +106,8 @@ void iris_get_internal_buffers(struct iris_inst *inst, u32 plane); int iris_create_internal_buffers(struct iris_inst *inst, u32 plane); int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); -int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_all_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_dequeued_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 4143acedfc57..9c049b9671cc 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -408,7 +408,7 @@ int iris_vdec_streamon_input(struct iris_inst *inst) iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); - ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret; @@ -496,7 +496,7 @@ int iris_vdec_streamon_output(struct iris_inst *inst) iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); - ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + ret = iris_destroy_dequeued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); if (ret) return ret; diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index ca0f4e310f77..a8144595cc78 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -221,6 +221,33 @@ static void iris_session_close(struct iris_inst *inst) iris_wait_for_session_response(inst, false); } +static void iris_check_num_queued_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data = inst->core->iris_platform_data; + struct iris_buffer *buf, *next; + struct iris_buffers *buffers; + const u32 *internal_buf_type; + u32 internal_buffer_count, i; + u32 count = 0;
+ if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + internal_buffer_count = platform_data->dec_op_int_buf_tbl_size; + }
+ for (i = 0; i < internal_buffer_count; i++) { + buffers = &inst->buffers[internal_buf_type[i]]; + list_for_each_entry_safe(buf, next, &buffers->list, list) + count++; + if (count) + dev_err(inst->core->dev, "%d buffer of type %d not released", + count, internal_buf_type[i]); + } +}
int iris_close(struct file *filp) { struct iris_inst *inst = iris_get_inst(filp, NULL); @@ -233,8 +260,10 @@ int iris_close(struct file *filp) iris_session_close(inst); iris_inst_change_state(inst, IRIS_INST_DEINIT); iris_v4l2_fh_deinit(inst); - iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); - iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + iris_destroy_all_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + iris_check_num_queued_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); iris_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock);
I left some comments in the previous cycle
https://lore.kernel.org/linux-arm-msm/a056266e-612d-4abf-916f-3db49b00dbde@l...
I don't see those addressed in this cycle. Can you give some feedback from your POV.
I responded to your comments, pls check here https://lore.kernel.org/linux-arm-msm/2c431232-e0d6-1e6c-cd22-a912b5f08f7a@q...
Thanks, Dikshita
bod
Update the width, height and buffer size of CAPTURE based on the resolution set to OUTPUT via VIDIOC_S_FMT. This is required to set the updated capture resolution to firmware when S_FMT is called only for OUTPUT.
Cc: stable@vger.kernel.org Fixes: b530b95de22c ("media: iris: implement s_fmt, g_fmt and try_fmt ioctls") Reviewed-by: Bryan O'Donoghue bryan.odonoghue@linaro.org Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_vdec.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 9c049b9671cc..d342f733feb9 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -171,6 +171,11 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f) output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+ /* Update capture format based on new ip w/h */ + output_fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, 128); + output_fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, 32); + inst->buffers[BUF_OUTPUT].size = iris_get_buffer_size(inst, BUF_OUTPUT); + inst->crop.left = 0; inst->crop.top = 0; inst->crop.width = f->fmt.pix_mp.width;
During reconfig, the firmware sends the resolution aligned to 8 bytes. If the driver sends the same resolution back to the firmware the resolution will be aligned to 16 bytes not 8.
The alignment mismatch would then subsequently cause the firmware to send another redundant sequence change event.
Fix this by not setting the resolution property during reconfig.
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c | 15 ++++++++------- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + 3 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 64f887d9a17d..2a86c27443ea 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -546,14 +546,15 @@ static int iris_hfi_gen1_set_resolution(struct iris_inst *inst) struct hfi_framesize fs; int ret;
- fs.buffer_type = HFI_BUFFER_INPUT; - fs.width = inst->fmt_src->fmt.pix_mp.width; - fs.height = inst->fmt_src->fmt.pix_mp.height; - - ret = hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs)); - if (ret) - return ret; + if (!iris_drc_pending(inst)) { + fs.buffer_type = HFI_BUFFER_INPUT; + fs.width = inst->fmt_src->fmt.pix_mp.width; + fs.height = inst->fmt_src->fmt.pix_mp.height;
+ ret = hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs)); + if (ret) + return ret; + } fs.buffer_type = HFI_BUFFER_OUTPUT2; fs.width = inst->fmt_dst->fmt.pix_mp.width; fs.height = inst->fmt_dst->fmt.pix_mp.height; diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index 5976e926c83d..104e1687ad39 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -245,7 +245,7 @@ int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane) return iris_inst_change_sub_state(inst, 0, set_sub_state); }
-static inline bool iris_drc_pending(struct iris_inst *inst) +bool iris_drc_pending(struct iris_inst *inst) { return inst->sub_state & IRIS_INST_SUB_DRC && inst->sub_state & IRIS_INST_SUB_DRC_LAST; diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index 78c61aac5e7e..e718386dbe04 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -140,5 +140,6 @@ int iris_inst_sub_state_change_drain_last(struct iris_inst *inst); int iris_inst_sub_state_change_drc_last(struct iris_inst *inst); int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane); bool iris_allow_cmd(struct iris_inst *inst, u32 cmd); +bool iris_drc_pending(struct iris_inst *inst);
#endif
On 01/05/2025 20:13, Dikshita Agarwal wrote:
During reconfig, the firmware sends the resolution aligned to 8 bytes. If the driver sends the same resolution back to the firmware the resolution will be aligned to 16 bytes not 8.
My question here is why there's an alignment mismatch between the APSS and firmware at all ?
--- bod
On 5/2/2025 5:48 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
During reconfig, the firmware sends the resolution aligned to 8 bytes. If the driver sends the same resolution back to the firmware the resolution will be aligned to 16 bytes not 8.
My question here is why there's an alignment mismatch between the APSS and firmware at all ?
It's not about the mismatch between APPS and firmware. It's about how firmware handles this, it's not expecting the driver to set the resolution after reconfig.
Thanks, Dikshita
bod
Currently, port check enforces that session property response must arrive only on the BITSTREAM port. However, firmware can send some responses on other port as well.
Remove the strict port validation to correctly handle session property responses from the firmware.
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 --- 1 file changed, 3 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index b75a01641d5d..d1a2a497a7b2 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -636,9 +636,6 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst, { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst);
- if (pkt->port != HFI_PORT_BITSTREAM) - return 0; - if (pkt->flags & HFI_FW_FLAGS_INFORMATION) return 0;
The current check only considers the core error state before allowing writes to the HFI queues. However, the core can also transition to the deinit state due to a system error triggered by the response thread. In such cases, writing to the HFI queues should not be allowed.
Fix this by adding a check for the core deinit state, ensuring that writes are rejected when core is not in a valid state.
Cc: stable@vger.kernel.org Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.c b/drivers/media/platform/qcom/iris/iris_hfi_queue.c index fac7df0c4d1a..221dcd09e1e1 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_queue.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.c @@ -113,7 +113,7 @@ int iris_hfi_queue_cmd_write_locked(struct iris_core *core, void *pkt, u32 pkt_s { struct iris_iface_q_info *q_info = &core->command_queue;
- if (core->state == IRIS_CORE_ERROR) + if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT) return -EINVAL;
if (!iris_hfi_queue_write(q_info, pkt, pkt_size)) {
On 01/05/2025 20:13, Dikshita Agarwal wrote:
- if (core->state == IRIS_CORE_ERROR)
- if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT) return -EINVAL;
Instead of checking for 2/3 of the states why not just check for the 1/3 ?
enum iris_core_state { IRIS_CORE_DEINIT, IRIS_CORE_INIT, IRIS_CORE_ERROR, };
if (core->state != IRIS_CORE_INIT) return -EINVAL;
Cleaner and more explicit - declaring the state you must be in, as opposed to a list of states you should not be in.
Assuming you accept that suggested change:
Reviewed-by: Bryan O'Donoghue bryan.odonoghue@linaro.org
On 5/2/25 2:22 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
- if (core->state == IRIS_CORE_ERROR) + if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT) return -EINVAL;
Instead of checking for 2/3 of the states why not just check for the 1/3 ?
enum iris_core_state { IRIS_CORE_DEINIT, IRIS_CORE_INIT, IRIS_CORE_ERROR, };
if (core->state != IRIS_CORE_INIT) return -EINVAL;
Cleaner and more explicit - declaring the state you must be in, as opposed to a list of states you should not be in.
Being explicit in state machines helps maintainability - if we get e.g. IRIS_CORE_LIGHT_SLEEP down the line, this could easily fail
Konrad
On 5/2/2025 7:24 PM, Konrad Dybcio wrote:
On 5/2/25 2:22 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
- if (core->state == IRIS_CORE_ERROR) + if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT) return -EINVAL;
Instead of checking for 2/3 of the states why not just check for the 1/3 ?
enum iris_core_state { IRIS_CORE_DEINIT, IRIS_CORE_INIT, IRIS_CORE_ERROR, };
if (core->state != IRIS_CORE_INIT) return -EINVAL;
Cleaner and more explicit - declaring the state you must be in, as opposed to a list of states you should not be in.
Being explicit in state machines helps maintainability - if we get e.g. IRIS_CORE_LIGHT_SLEEP down the line, this could easily fail
Agree, would prefer to keep the check as is if (core->state == IRIS_CORE_ERROR || core->state == IRIS_CORE_DEINIT)
Thanks, Dikshita
Konrad
HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer supported on current firmware, remove setting the same to firmware.
At the same time, remove the check for non-zero number of v4l2 controls as some SOC might not expose any capability which requires v4l2 control.
Cc: stable@vger.kernel.org Fixes: 79865252acb6 ("media: iris: enable video driver probe of SM8250 SoC") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_ctrls.c | 6 ------ drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c | 8 -------- drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h | 1 - drivers/media/platform/qcom/iris/iris_platform_common.h | 2 +- drivers/media/platform/qcom/iris/iris_platform_sm8250.c | 9 --------- 5 files changed, 1 insertion(+), 25 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c index b690578256d5..915de101fcba 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -17,8 +17,6 @@ static inline bool iris_valid_cap_id(enum platform_inst_fw_cap_type cap_id) static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id) { switch (id) { - case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER: - return DEBLOCK; case V4L2_CID_MPEG_VIDEO_H264_PROFILE: return PROFILE; case V4L2_CID_MPEG_VIDEO_H264_LEVEL: @@ -34,8 +32,6 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id) return 0;
switch (cap_id) { - case DEBLOCK: - return V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER; case PROFILE: return V4L2_CID_MPEG_VIDEO_H264_PROFILE; case LEVEL: @@ -84,8 +80,6 @@ int iris_ctrls_init(struct iris_inst *inst) if (iris_get_v4l2_id(cap[idx].cap_id)) num_ctrls++; } - if (!num_ctrls) - return -EINVAL;
/* Adding 1 to num_ctrls to include V4L2_CID_MIN_BUFFERS_FOR_CAPTURE */
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 2a86c27443ea..ce855a20ce4b 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -490,14 +490,6 @@ iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *p packet->shdr.hdr.size += sizeof(u32) + sizeof(*wm); break; } - case HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER: { - struct hfi_enable *en = prop_data; - u32 *in = pdata; - - en->enable = *in; - packet->shdr.hdr.size += sizeof(u32) + sizeof(*en); - break; - } default: return -EINVAL; } diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 9f246816a286..e178604855c1 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -65,7 +65,6 @@
#define HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS 0x202001
-#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 #define HFI_PROPERTY_PARAM_VDEC_DPB_COUNTS 0x120300e #define HFI_PROPERTY_CONFIG_VDEC_ENTROPY 0x1204004
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index ac76d9e1ef9c..1dab276431c7 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -89,7 +89,7 @@ enum platform_inst_fw_cap_type { CODED_FRAMES, BIT_DEPTH, RAP_FRAME, - DEBLOCK, + TIER, INST_FW_CAP_MAX, };
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 5c86fd7b7b6f..543fa2661539 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -30,15 +30,6 @@ static struct platform_inst_fw_cap inst_fw_cap_sm8250[] = { .hfi_id = HFI_PROPERTY_PARAM_WORK_MODE, .set = iris_set_stage, }, - { - .cap_id = DEBLOCK, - .min = 0, - .max = 1, - .step_or_mask = 1, - .value = 0, - .hfi_id = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER, - .set = iris_set_u32, - }, };
static struct platform_inst_caps platform_inst_cap_sm8250 = {
On 01/05/2025 20:13, Dikshita Agarwal wrote:
HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer supported on current firmware, remove setting the same to firmware.
What about older firmware - what's the effect there ?
At the same time, remove the check for non-zero number of v4l2 controls as some SOC might not expose any capability which requires v4l2 control.
Please break up patches like this one patch removing your legacy functionality - assuming that makes sense another patch amending your non-zero number of v4l2 controls.
Generally any commit log or patch title that requires "do this and do that" should be broken up @ the and.
--- bod
On 5/2/2025 5:57 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER is deprecated and no longer supported on current firmware, remove setting the same to firmware.
What about older firmware - what's the effect there ?
if you are referring to firmware being used in venus, its the same. This property was never supported on SM8250 firmware.
At the same time, remove the check for non-zero number of v4l2 controls as some SOC might not expose any capability which requires v4l2 control.
Please break up patches like this one patch removing your legacy functionality - assuming that makes sense another patch amending your non-zero number of v4l2 controls.
Ok, will split.
Thanks, Dikshita
Generally any commit log or patch title that requires "do this and do that" should be broken up @ the and.
bod
The function pointers responsible for setting firmware properties were never initialized in the instance capability structure, causing it to remain NULL. As a result, the firmware properties were not being set correctly.
Fix this by properly assigning the function pointers from the core capability to the instance capability, ensuring that the properties are correctly applied to the firmware.
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_ctrls.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c index 915de101fcba..13f5cf0d0e8a 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -157,6 +157,7 @@ void iris_session_init_caps(struct iris_core *core) core->inst_fw_caps[cap_id].value = caps[i].value; core->inst_fw_caps[cap_id].flags = caps[i].flags; core->inst_fw_caps[cap_id].hfi_id = caps[i].hfi_id; + core->inst_fw_caps[cap_id].set = caps[i].set; } }
On 01/05/2025 20:13, Dikshita Agarwal wrote:
The function pointers responsible for setting firmware properties were never initialized in the instance capability structure, causing it to remain NULL. As a result, the firmware properties were not being set correctly.
Fix this by properly assigning the function pointers from the core capability to the instance capability, ensuring that the properties are correctly applied to the firmware.
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
drivers/media/platform/qcom/iris/iris_ctrls.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c index 915de101fcba..13f5cf0d0e8a 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -157,6 +157,7 @@ void iris_session_init_caps(struct iris_core *core) core->inst_fw_caps[cap_id].value = caps[i].value; core->inst_fw_caps[cap_id].flags = caps[i].flags; core->inst_fw_caps[cap_id].hfi_id = caps[i].hfi_id;
core->inst_fw_caps[cap_id].set = caps[i].set;
Reviewed-by: Bryan O'Donoghue bryan.odonoghue@linaro.org
A warning reported by smatch indicated a possible null pointer dereference where one of the arguments to API "iris_hfi_gen2_handle_system_error" could sometimes be null.
To fix this, add a check to validate that the argument passed is not null before accessing its members.
Cc: stable@vger.kernel.org Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks") Reported-by: Dan Carpenter dan.carpenter@linaro.org Closes: https://lore.kernel.org/linux-media/634cc9b8-f099-4b54-8556-d879fb2b5169@sta... Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index d1a2a497a7b2..4488540d1d41 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -265,7 +265,8 @@ static int iris_hfi_gen2_handle_system_error(struct iris_core *core, { struct iris_inst *instance;
- dev_err(core->dev, "received system error of type %#x\n", pkt->type); + if (pkt) + dev_err(core->dev, "received system error of type %#x\n", pkt->type);
core->state = IRIS_CORE_ERROR;
On 01/05/2025 20:13, Dikshita Agarwal wrote:
A warning reported by smatch indicated a possible null pointer dereference where one of the arguments to API "iris_hfi_gen2_handle_system_error" could sometimes be null.
To fix this, add a check to validate that the argument passed is not null before accessing its members.
Cc: stable@vger.kernel.org Fixes: fb583a214337 ("media: iris: introduce host firmware interface with necessary hooks") Reported-by: Dan Carpenter dan.carpenter@linaro.org Closes: https://lore.kernel.org/linux-media/634cc9b8-f099-4b54-8556-d879fb2b5169@sta... Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index d1a2a497a7b2..4488540d1d41 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -265,7 +265,8 @@ static int iris_hfi_gen2_handle_system_error(struct iris_core *core, { struct iris_inst *instance;
- dev_err(core->dev, "received system error of type %#x\n", pkt->type);
- if (pkt)
dev_err(core->dev, "received system error of type %#x\n", pkt->type);core->state = IRIS_CORE_ERROR;
Reviewed-by: Bryan O'Donoghue bryan.odonoghue@linaro.org
Correct a typo from "dpeth" to "depth".
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index a908b41e2868..802fa62c26eb 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -178,7 +178,7 @@ static int iris_hfi_gen2_set_crop_offsets(struct iris_inst *inst) sizeof(u64)); }
-static int iris_hfi_gen2_set_bit_dpeth(struct iris_inst *inst) +static int iris_hfi_gen2_set_bit_depth(struct iris_inst *inst) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); @@ -378,7 +378,7 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p {HFI_PROP_BITSTREAM_RESOLUTION, iris_hfi_gen2_set_bitstream_resolution }, {HFI_PROP_CROP_OFFSETS, iris_hfi_gen2_set_crop_offsets }, {HFI_PROP_CODED_FRAMES, iris_hfi_gen2_set_coded_frames }, - {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, iris_hfi_gen2_set_bit_dpeth }, + {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, iris_hfi_gen2_set_bit_depth }, {HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, iris_hfi_gen2_set_min_output_count }, {HFI_PROP_PIC_ORDER_CNT_TYPE, iris_hfi_gen2_set_picture_order_count }, {HFI_PROP_SIGNAL_COLOR_INFO, iris_hfi_gen2_set_colorspace },
On 01/05/2025 20:13, Dikshita Agarwal wrote:
Correct a typo from "dpeth" to "depth".
Cc: stable@vger.kernel.org Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") Acked-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index a908b41e2868..802fa62c26eb 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -178,7 +178,7 @@ static int iris_hfi_gen2_set_crop_offsets(struct iris_inst *inst) sizeof(u64)); } -static int iris_hfi_gen2_set_bit_dpeth(struct iris_inst *inst) +static int iris_hfi_gen2_set_bit_depth(struct iris_inst *inst) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); @@ -378,7 +378,7 @@ static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 p {HFI_PROP_BITSTREAM_RESOLUTION, iris_hfi_gen2_set_bitstream_resolution }, {HFI_PROP_CROP_OFFSETS, iris_hfi_gen2_set_crop_offsets }, {HFI_PROP_CODED_FRAMES, iris_hfi_gen2_set_coded_frames },
{HFI_PROP_LUMA_CHROMA_BIT_DEPTH, iris_hfi_gen2_set_bit_dpeth },
{HFI_PROP_LUMA_CHROMA_BIT_DEPTH, iris_hfi_gen2_set_bit_depth },
Reviewed-by: Bryan O'Donoghue bryan.odonoghue@linaro.org
Currently, two types of flush commands are queued to the firmware, the first flush queued as part of sequence change, does not wait for a response, while the second flush queued as part of stop, expects a completion response before proceeding further.
Due to timing issue, the flush response corresponding to the first command could arrive after the second flush is issued. This casuses the driver to incorrectly assume that the second flush has completed, leading to the premature signaling of flush_completion.
To address this, introduce a counter to track the number of pending flush responses and signal flush completion only when all expected responses are received.
Cc: stable@vger.kernel.org Fixes: 11712ce70f8e ("media: iris: implement vb2 streaming ops") Reviewed-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- .../media/platform/qcom/iris/iris_hfi_gen1_command.c | 4 +++- .../media/platform/qcom/iris/iris_hfi_gen1_response.c | 17 +++++++++++------ drivers/media/platform/qcom/iris/iris_instance.h | 2 ++ 3 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index ce855a20ce4b..bd9d86220e61 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -208,8 +208,10 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) flush_pkt.flush_type = flush_type;
ret = iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size); - if (!ret) + if (!ret) { + inst->flush_responses_pending++; ret = iris_wait_for_session_response(inst, true); + } }
return ret; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index b72d503dd740..271e14469223 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -207,7 +207,8 @@ static void iris_hfi_gen1_event_seq_changed(struct iris_inst *inst, flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH; flush_pkt.shdr.session_id = inst->session_id; flush_pkt.flush_type = HFI_FLUSH_OUTPUT; - iris_hfi_queue_cmd_write(inst->core, &flush_pkt, flush_pkt.shdr.hdr.size); + if (!iris_hfi_queue_cmd_write(inst->core, &flush_pkt, flush_pkt.shdr.hdr.size)) + inst->flush_responses_pending++; }
iris_vdec_src_change(inst); @@ -408,7 +409,9 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet) flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH; flush_pkt.shdr.session_id = inst->session_id; flush_pkt.flush_type = HFI_FLUSH_OUTPUT; - iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size); + if (!iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size)) + inst->flush_responses_pending++; + iris_inst_sub_state_change_drain_last(inst);
return; @@ -558,7 +561,6 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response const struct iris_hfi_gen1_response_pkt_info *pkt_info; struct device *dev = core->dev; struct hfi_session_pkt *pkt; - struct completion *done; struct iris_inst *inst; bool found = false; u32 i; @@ -619,9 +621,12 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response if (shdr->error_type != HFI_ERR_NONE) iris_inst_change_state(inst, IRIS_INST_ERROR);
- done = pkt_info->pkt == HFI_MSG_SESSION_FLUSH ? - &inst->flush_completion : &inst->completion; - complete(done); + if (pkt_info->pkt == HFI_MSG_SESSION_FLUSH) { + if (!(--inst->flush_responses_pending)) + complete(&inst->flush_completion); + } else { + complete(&inst->completion); + } } mutex_unlock(&inst->lock);
diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index caa3c6507006..06a7f1174ad5 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -27,6 +27,7 @@ * @crop: structure of crop info * @completion: structure of signal completions * @flush_completion: structure of signal completions for flush cmd + * @flush_responses_pending: counter to track number of pending flush responses * @fw_caps: array of supported instance firmware capabilities * @buffers: array of different iris buffers * @fw_min_count: minimnum count of buffers needed by fw @@ -57,6 +58,7 @@ struct iris_inst { struct iris_hfi_rect_desc crop; struct completion completion; struct completion flush_completion; + u32 flush_responses_pending; struct platform_inst_fw_cap fw_caps[INST_FW_CAP_MAX]; struct iris_buffers buffers[BUF_TYPE_MAX]; u32 fw_min_count;
On 01/05/2025 20:13, Dikshita Agarwal wrote:
Currently, two types of flush commands are queued to the firmware, the first flush queued as part of sequence change, does not wait for a response, while the second flush queued as part of stop, expects a completion response before proceeding further.
Due to timing issue, the flush response corresponding to the first command could arrive after the second flush is issued. This casuses the driver to incorrectly assume that the second flush has completed, leading to the premature signaling of flush_completion.
To address this, introduce a counter to track the number of pending flush responses and signal flush completion only when all expected responses are received.
Is there no other way to correlate command/response than a counter on the APSS side ?
Usually command/response protocols have some kind of identifier field or sequence number embedded in the protocol headers.
--- bod
On 5/2/2025 6:10 PM, Bryan O'Donoghue wrote:
On 01/05/2025 20:13, Dikshita Agarwal wrote:
Currently, two types of flush commands are queued to the firmware, the first flush queued as part of sequence change, does not wait for a response, while the second flush queued as part of stop, expects a completion response before proceeding further.
Due to timing issue, the flush response corresponding to the first command could arrive after the second flush is issued. This casuses the driver to incorrectly assume that the second flush has completed, leading to the premature signaling of flush_completion.
To address this, introduce a counter to track the number of pending flush responses and signal flush completion only when all expected responses are received.
Is there no other way to correlate command/response than a counter on the APSS side ?
Usually command/response protocols have some kind of identifier field or sequence number embedded in the protocol headers.
There is no such identifier field for flush command and response at the moment.
Thanks, Dikshita
bod
When the resolution changes, the driver internally updates the width and height, but the client continue to queue buffers with the older resolution until the last flag is received. This results in a mismatch when the buffers are prepared, causing failure due to outdated size.
Introduce a check to prevent size validation during buffer preparation if a resolution reconfiguration is in progress, to handle this.
Cc: stable@vger.kernel.org Fixes: 17f2a485ca67 ("media: iris: implement vb2 ops for buf_queue and firmware response") Reviewed-by: Vikash Garodia quic_vgarodia@quicinc.com Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com --- drivers/media/platform/qcom/iris/iris_vb2.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index cdf11feb590b..b3bde10eb6d2 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -259,13 +259,14 @@ int iris_vb2_buf_prepare(struct vb2_buffer *vb) return -EINVAL; }
- if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && - vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_OUTPUT)) - return -EINVAL; - if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && - vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_INPUT)) - return -EINVAL; - + if (!(inst->sub_state & IRIS_INST_SUB_DRC)) { + if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_OUTPUT)) + return -EINVAL; + if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && + vb2_plane_size(vb, 0) < iris_get_buffer_size(inst, BUF_INPUT)) + return -EINVAL; + } return 0; }
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2: https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case:
- 10 testcases failed due to unsupported 10 bit format.
- DBLK_A_MAIN10_VIXS_4
- INITQP_B_Main10_Sony_1
- TSUNEQBD_A_MAIN10_Technicolor_2
- WP_A_MAIN10_Toshiba_3
- WP_MAIN10_B_Toshiba_3
- WPP_A_ericsson_MAIN10_2
- WPP_B_ericsson_MAIN10_2
- WPP_C_ericsson_MAIN10_2
- WPP_E_ericsson_MAIN10_2
- WPP_F_ericsson_MAIN10_2
- 4 testcase failed due to unsupported resolution
- PICSIZE_A_Bossen_1
- PICSIZE_B_Bossen_1
- WPP_D_ericsson_MAIN10_2
- WPP_D_ericsson_MAIN_2
- 2 testcase failed due to CRC mismatch
- RAP_A_docomo_6
- RAP_B_Bossen_2
- BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case:
- 64 testcases failed due to unsupported resolution
- vp90-2-02-size-08x08.webm
- vp90-2-02-size-08x10.webm
- vp90-2-02-size-08x16.webm
- vp90-2-02-size-08x18.webm
- vp90-2-02-size-08x32.webm
- vp90-2-02-size-08x34.webm
- vp90-2-02-size-08x64.webm
- vp90-2-02-size-08x66.webm
- vp90-2-02-size-10x08.webm
- vp90-2-02-size-10x10.webm
- vp90-2-02-size-10x16.webm
- vp90-2-02-size-10x18.webm
- vp90-2-02-size-10x32.webm
- vp90-2-02-size-10x34.webm
- vp90-2-02-size-10x64.webm
- vp90-2-02-size-10x66.webm
- vp90-2-02-size-16x08.webm
- vp90-2-02-size-16x10.webm
- vp90-2-02-size-16x16.webm
- vp90-2-02-size-16x18.webm
- vp90-2-02-size-16x32.webm
- vp90-2-02-size-16x34.webm
- vp90-2-02-size-16x64.webm
- vp90-2-02-size-16x66.webm
- vp90-2-02-size-18x08.webm
- vp90-2-02-size-18x10.webm
- vp90-2-02-size-18x16.webm
- vp90-2-02-size-18x18.webm
- vp90-2-02-size-18x32.webm
- vp90-2-02-size-18x34.webm
- vp90-2-02-size-18x64.webm
- vp90-2-02-size-18x66.webm
- vp90-2-02-size-32x08.webm
- vp90-2-02-size-32x10.webm
- vp90-2-02-size-32x16.webm
- vp90-2-02-size-32x18.webm
- vp90-2-02-size-32x32.webm
- vp90-2-02-size-32x34.webm
- vp90-2-02-size-32x64.webm
- vp90-2-02-size-32x66.webm
- vp90-2-02-size-34x08.webm
- vp90-2-02-size-34x10.webm
- vp90-2-02-size-34x16.webm
- vp90-2-02-size-34x18.webm
- vp90-2-02-size-34x32.webm
- vp90-2-02-size-34x34.webm
- vp90-2-02-size-34x64.webm
- vp90-2-02-size-34x66.webm
- vp90-2-02-size-64x08.webm
- vp90-2-02-size-64x10.webm
- vp90-2-02-size-64x16.webm
- vp90-2-02-size-64x18.webm
- vp90-2-02-size-64x32.webm
- vp90-2-02-size-64x34.webm
- vp90-2-02-size-64x64.webm
- vp90-2-02-size-64x66.webm
- vp90-2-02-size-66x08.webm
- vp90-2-02-size-66x10.webm
- vp90-2-02-size-66x16.webm
- vp90-2-02-size-66x18.webm
- vp90-2-02-size-66x32.webm
- vp90-2-02-size-66x34.webm
- vp90-2-02-size-66x64.webm
- vp90-2-02-size-66x66.webm
- 2 testcases failed due to unsupported format
- vp91-2-04-yuv422.webm
- vp91-2-04-yuv444.webm
- 1 testcase failed with CRC mismatch
- vp90-2-22-svc_1280x720_3.ivf
- Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
- 2 testcase failed due to unsupported resolution after sequence change
- vp90-2-21-resize_inter_320x180_5_1-2.webm
- vp90-2-21-resize_inter_320x180_7_1-2.webm
- 1 testcase failed due to unsupported stream
- vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case:
- 10 testcases failed due to unsupported 10 bit format.
- DBLK_A_MAIN10_VIXS_4
- INITQP_B_Main10_Sony_1
- TSUNEQBD_A_MAIN10_Technicolor_2
- WP_A_MAIN10_Toshiba_3
- WP_MAIN10_B_Toshiba_3
- WPP_A_ericsson_MAIN10_2
- WPP_B_ericsson_MAIN10_2
- WPP_C_ericsson_MAIN10_2
- WPP_E_ericsson_MAIN10_2
- WPP_F_ericsson_MAIN10_2
- 4 testcase failed due to unsupported resolution
- PICSIZE_A_Bossen_1
- PICSIZE_B_Bossen_1
- WPP_D_ericsson_MAIN10_2
- WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case:
- 64 testcases failed due to unsupported resolution
- vp90-2-02-size-08x08.webm
- vp90-2-02-size-08x10.webm
- vp90-2-02-size-08x16.webm
- vp90-2-02-size-08x18.webm
- vp90-2-02-size-08x32.webm
- vp90-2-02-size-08x34.webm
- vp90-2-02-size-08x64.webm
- vp90-2-02-size-08x66.webm
- vp90-2-02-size-10x08.webm
- vp90-2-02-size-10x10.webm
- vp90-2-02-size-10x16.webm
- vp90-2-02-size-10x18.webm
- vp90-2-02-size-10x32.webm
- vp90-2-02-size-10x34.webm
- vp90-2-02-size-10x64.webm
- vp90-2-02-size-10x66.webm
- vp90-2-02-size-16x08.webm
- vp90-2-02-size-16x10.webm
- vp90-2-02-size-16x16.webm
- vp90-2-02-size-16x18.webm
- vp90-2-02-size-16x32.webm
- vp90-2-02-size-16x34.webm
- vp90-2-02-size-16x64.webm
- vp90-2-02-size-16x66.webm
- vp90-2-02-size-18x08.webm
- vp90-2-02-size-18x10.webm
- vp90-2-02-size-18x16.webm
- vp90-2-02-size-18x18.webm
- vp90-2-02-size-18x32.webm
- vp90-2-02-size-18x34.webm
- vp90-2-02-size-18x64.webm
- vp90-2-02-size-18x66.webm
- vp90-2-02-size-32x08.webm
- vp90-2-02-size-32x10.webm
- vp90-2-02-size-32x16.webm
- vp90-2-02-size-32x18.webm
- vp90-2-02-size-32x32.webm
- vp90-2-02-size-32x34.webm
- vp90-2-02-size-32x64.webm
- vp90-2-02-size-32x66.webm
- vp90-2-02-size-34x08.webm
- vp90-2-02-size-34x10.webm
- vp90-2-02-size-34x16.webm
- vp90-2-02-size-34x18.webm
- vp90-2-02-size-34x32.webm
- vp90-2-02-size-34x34.webm
- vp90-2-02-size-34x64.webm
- vp90-2-02-size-34x66.webm
- vp90-2-02-size-64x08.webm
- vp90-2-02-size-64x10.webm
- vp90-2-02-size-64x16.webm
- vp90-2-02-size-64x18.webm
- vp90-2-02-size-64x32.webm
- vp90-2-02-size-64x34.webm
- vp90-2-02-size-64x64.webm
- vp90-2-02-size-64x66.webm
- vp90-2-02-size-66x08.webm
- vp90-2-02-size-66x10.webm
- vp90-2-02-size-66x16.webm
- vp90-2-02-size-66x18.webm
- vp90-2-02-size-66x32.webm
- vp90-2-02-size-66x34.webm
- vp90-2-02-size-66x64.webm
- vp90-2-02-size-66x66.webm
- 2 testcases failed due to unsupported format
- vp91-2-04-yuv422.webm
- vp91-2-04-yuv444.webm
- 1 testcase failed with CRC mismatch
- vp90-2-22-svc_1280x720_3.ivf
- Bug raised:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371
- 5 testcase failed due to unsupported resolution after sequence change
- vp90-2-21-resize_inter_320x180_5_1-2.webm
- vp90-2-21-resize_inter_320x180_7_1-2.webm
- vp90-2-21-resize_inter_320x240_5_1-2.webm
- vp90-2-21-resize_inter_320x240_7_1-2.webm
- vp90-2-18-resize.ivf
- 1 testcase failed with CRC mismatch
Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
- vp90-2-16-intra-only.webm
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +} + #endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I would not recommend adding the HEVC and VP9 for 8650 unless it is functional. Please drop the HEVC and VP9 from 8650 platform data, and once it is validated, they can be added in a separate series.
Regards, Vikash
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 5/2/2025 2:01 PM, Vikash Garodia wrote:
On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I would not recommend adding the HEVC and VP9 for 8650 unless it is functional. Please drop the HEVC and VP9 from 8650 platform data, and once it is validated, they can be added in a separate series.
Agree, I would not want to hold the enablement of HEVC and VP9 codecs. So, I will drop support for HEVC and VP9 codecs and retain only H.264 support in the 8650 caps in the next version.
Thanks, Dikshita
Regards, Vikash
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 02/05/2025 10:41, Dikshita Agarwal wrote:
On 5/2/2025 2:01 PM, Vikash Garodia wrote:
On 5/2/2025 1:04 PM, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I would not recommend adding the HEVC and VP9 for 8650 unless it is functional. Please drop the HEVC and VP9 from 8650 platform data, and once it is validated, they can be added in a separate series.
Agree, I would not want to hold the enablement of HEVC and VP9 codecs. So, I will drop support for HEVC and VP9 codecs and retain only H.264 support in the 8650 caps in the next version.
Sure, thx
Neil
Thanks, Dikshita
Regards, Vikash
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 02/05/2025 09:34, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I found the fix, it was a difference in buffer calculation, but not for vpu33, but a typo since v2 was right:
========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h index 2272f0c21683..ee95fd20b794 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -33,7 +33,7 @@ struct iris_inst; #define H264_DISPLAY_BUF_SIZE 3328 #define H264_NUM_FRM_INFO 66 #define H265_NUM_TILE_COL 32 -#define H265_NUM_TILE_ROW 12 +#define H265_NUM_TILE_ROW 128 #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1) #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32)) ========================================><======================================
This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm suprised this still works on SM8550 !
So please keep VP9 and HEVC enabled on v4.
Thanks, Neil
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 5/5/2025 10:10 PM, neil.armstrong@linaro.org wrote:
On 02/05/2025 09:34, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I found the fix, it was a difference in buffer calculation, but not for vpu33, but a typo since v2 was right:
========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h index 2272f0c21683..ee95fd20b794 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -33,7 +33,7 @@ struct iris_inst; #define H264_DISPLAY_BUF_SIZE 3328 #define H264_NUM_FRM_INFO 66 #define H265_NUM_TILE_COL 32 -#define H265_NUM_TILE_ROW 12 +#define H265_NUM_TILE_ROW 128 #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1) #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32)) ========================================><======================================
This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm suprised this still works on SM8550 !
Thanks for checking this. but this value has been the same since v1 https://lore.kernel.org/linux-media/20250408-iris-dec-hevc-vp9-v1-19-acd2587... And yes, it works on SM8550, I believe you also tested v3 and confirmed the same. But anyways, the correct value should be 128 and I confirmed that SM8550 works with the fix as well.
So please keep VP9 and HEVC enabled on v4.
Ok
Thanks, Dikshita
Thanks, Neil
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
On 06/05/2025 10:06, Dikshita Agarwal wrote:
On 5/5/2025 10:10 PM, neil.armstrong@linaro.org wrote:
On 02/05/2025 09:34, Dikshita Agarwal wrote:
On 5/2/2025 12:55 PM, Neil Armstrong wrote:
Hi,
On 01/05/2025 21:13, Dikshita Agarwal wrote:
Hi All,
This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs.
These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions.
Changes in v3:
- Introduced two wrappers with explicit names to handle destroy internal
buffers (Nicolas)
- Used sub state check instead of introducing new boolean (Vikash)
- Addressed other comments (Vikash)
- Reorderd patches to have all fixes patches first (Dmitry)
- Link to v2:
https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quic...
Changes in v2:
- Added Changes to make sure all buffers are released in session close
(bryna)
- Added tracking for flush responses to fix a timing issue.
- Added a handling to fix timing issue in reconfig
- Splitted patch 06/20 in two patches (Bryan)
- Added missing fixes tag (bryan)
- Updated fluster report (Nicolas)
- Link to v1:
https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quici...
Changes sinces RFC:
- Added additional fixes to address issues identified during further
testing.
- Moved typo fix to a seperate patch [Neil]
- Reordered the patches for better logical flow and clarity [Neil,
Dmitry]
- Added fixes tag wherever applicable [Neil, Dmitry]
- Removed the default case in the switch statement for codecs [Bryan]
- Replaced if-else statements with switch-case [Bryan]
- Added comments for mbpf [Bryan]
- RFC:
https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@q...
This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020... [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@...
These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected.
Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@...
The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame.
235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm
The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2
232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame.
Signed-off-by: Dikshita Agarwal quic_dikshita@quicinc.com
Dikshita Agarwal (23): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Add handling for corrupt and drop frames media: iris: Send V4L2_BUF_FLAG_ERROR for buffers with 0 filled length media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Skip flush on first sequence change media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling
drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 57 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1160 insertions(+), 211 deletions(-)
base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250428-qcom-iris-hevc-vp9-eb31f30c3390 prerequisite-message-id: 20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-message-id: 20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: feda620f147ca14a958c92afdc85a1dc507701ac prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea
Best regards,
HEVC & VP9 works fine on HDK8550.
But on SM8650-QRD & SM8650-HDK while decoding HEVC, I get: [ 44.741670] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.755724] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.776462] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.797179] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.816630] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.837387] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.856812] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.877576] qcom-iris aa00000.video-codec: session error received 0x1000005: unknown [ 44.897000] qcom-iris aa00000.video-codec: session error received 0x4000005: insufficient resources [ 44.917801] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 44.937254] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.959128] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 44.981025] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.000459] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.022376] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.041816] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.063736] qcom-iris aa00000.video-codec: session error received 0x1000009: unknown [ 45.083167] qcom-iris aa00000.video-codec: session error received 0x4000004: invalid operation for current state [ 45.105459] ------------[ cut here ]------------ [ 45.121152] WARNING: CPU: 6 PID: 573 at drivers/media/common/videobuf2/videobuf2-core.c:1827 vb2_start_streaming+0x100/0x178 [videobuf2_common] while VP9 works fine.
Is it a firmware issue ?
Looks like resources set to firmware are not sufficient. I suspect, internal buffers set to firmware are less than what it requires, this can change for different VPUs. Pls check if there is any difference in internal buffer calculations between vpu3 and vpu33.
I found the fix, it was a difference in buffer calculation, but not for vpu33, but a typo since v2 was right:
========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h index 2272f0c21683..ee95fd20b794 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -33,7 +33,7 @@ struct iris_inst; #define H264_DISPLAY_BUF_SIZE 3328 #define H264_NUM_FRM_INFO 66 #define H265_NUM_TILE_COL 32 -#define H265_NUM_TILE_ROW 12 +#define H265_NUM_TILE_ROW 128 #define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1) #define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32)) ========================================><======================================
This fixes HEVC on SM8650, so with this change VP9 and HEVC works fine, I'm suprised this still works on SM8550 !
Thanks for checking this. but this value has been the same since v1 https://lore.kernel.org/linux-media/20250408-iris-dec-hevc-vp9-v1-19-acd2587...
Yeah indeed I found that in v2 of iris patchset patch 20 the value is right: https://lore.kernel.org/all/1702899149-21321-21-git-send-email-quic_dikshita...
Seems the typo appeared on the vp9 & hevc patchset.
And yes, it works on SM8550, I believe you also tested v3 and confirmed the same. But anyways, the correct value should be 128 and I confirmed that SM8550 works with the fix as well.
Good !
So please keep VP9 and HEVC enabled on v4.
Ok
Thanks! Neil
Thanks, Dikshita
Thanks, Neil
Thanks, Dikshita
I've added: ========================================><====================================== diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c index d3026b2bcb70..8c0ab00ab435 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c +++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c @@ -400,7 +400,7 @@ struct iris_platform_data sm8650_data = { .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .vpu_ops = &iris_vpu33_ops, - .set_preset_registers = iris_set_sm8550_preset_registers, + .set_preset_registers = iris_set_sm8650_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8650_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h index 75e9d572e788..9e2d23f12f75 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8650.h +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8650.h @@ -10,4 +10,20 @@ static const char * const sm8650_clk_reset_table[] = { "bus", "core" };
static const char * const sm8650_controller_reset_table[] = { "xo" };
+static void iris_set_sm8650_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); + writel(0x33332222, core->reg_base + 0x13030); + writel(0x44444444, core->reg_base + 0x13034); + writel(0x1022, core->reg_base + 0x13038); + writel(0x0, core->reg_base + 0x13040); + writel(0xFFFF, core->reg_base + 0x13048); + writel(0x33332222, core->reg_base + 0x13430); + writel(0x44444444, core->reg_base + 0x13434); + writel(0x1022, core->reg_base + 0x13438); + writel(0x0, core->reg_base + 0x13440); + writel(0xFFFF, core->reg_base + 0x13448); + writel(0x99, core->reg_base + 0xA013C); +}
#endif ========================================><====================================== and no change, error still occurs with HEVC decoding.
Thanks, Neil
linux-stable-mirror@lists.linaro.org
-
Bryan O'Donoghue -
Dikshita Agarwal -
Konrad Dybcio -
Neil Armstrong -
neil.armstrong＠linaro.org
-
Vikash Garodia