Hi Himanshu,

Le mardi 30 septembre 2025 à 09:33 +0530, Himanshu Dewangan a écrit :

Exynos MFC new driver upstream proposal +++++++++++++++++++++++++++++++++++++++

The s5p-mfc driver in the kernel is quite outdated and has not kept up with the hardware advances of the MFC IP. Going forward, we want to provide support for all the later versions of MFC in open source (both mobile and ExynosAuto would be supported by Samsung)

We (Samsung) would like to propose a new driver for the latest generation of Samsung MFC (Multi-Format Codec) hardware. Although the kernel already contains a Samsung MFC driver (drivers/media/platform/s5p-mfc/), the newer hardware diverges significantly from the previous generations, in features supported, register interface and in system integration. This has necessitated a new driver, rather than updating the existing one.

I wont deny that sometimes a new driver is needed, and I encourage you to defend your point. It does just justify any form of copy paste, also understand that device descriptions should be independent from the driver implementation. This is also why this is reviewed indecently, and DT nack won't ever be ignored by us.

From quick walk through, I'm not convince this new driver actually makes use of everything new in Linux and Linux Media since 2011. It will be important for you to research what other modern drivers do, and what could apply to your code base. I'm quite skeptical with all the custom memory handling code, which generally indicates the miss-use of dma APIs. Realistically, you will have to split your driver in smaller parts and submit in a way we can see your design principle, and not just randomly unused files being added. Considering the state of it, we are up to 20 or 30 revision, with 29 patches, you won't get to the end of that process.

V4L2 wise, one of the common mistakes is that you are adding a lot of new controls, most of them just slammed as vendor control without even trying to generalize. It did make sense in 2011, since there was no decoder to compare against, but today there is a lot of example and overlap between various brand. Omitting documentation and not keeping the API addition separate is also a no go for me. Make it very clear whenever something that will live in our API forever is to be added.

We also added rules since 2011 that enforce drivers submitter to show that proper testing have occurred. Its starts with the very basic v4l2-compliance report which you have omitted. Since its codec driver, we require a summary of your conformance decoding testing done with fluster[] with the supported framework of your choice (for v4l2 stateful, ffmpeg and gstreamer are supported, you can add more, as long as they are open source).

We also expect that your firmware has been submitted to linux-firmware with a link to that submission. Without that, we can only consider your set as an RFC to show people what is coming.

regards, Nicolas

Current driver vs targetted driver comparison +++++++++++++++++++++++++++++++++++++++++++++

Target Feature Current mainline : Simple V4L2/ VB2 interface based encoder/ decoder New proposed driver: Dual core support, NAL-q support, LLC, Coredump, performance measurement, plugin driver architecture, hardware manager(meerkat), DRM/OTF, Resource manager, bandwidth compression, New Pixel formats (NV12N,  YV12, P010, SBWC,  RGB family), 10 bit support, HDR support, Prio and RR Process Scheduler support,

Register map Current mainline : Fixed register layout, compatible across s5p variants New proposed driver: Completely redesigned register set, incompatible with old layout

Command model Current mainline : Mailbox-style command queue New proposed driver: Ring-buffer command queue with different signaling

Memory management Current mainline : CMA-backed contiguous buffers, ION legacy New proposed driver: dma-heap / scatter-gather buffers with strict alignment rules

Firmware Current mainline : Supports up to Firmware v12, Legacy binary format, loaded via shared mechanism New proposed driver: New firmware format, different protocol for control messages. Support Firmware v13 onwards

DT bindings Current mainline : Single clock + reset, simple PM New proposed driver: Multiple clocks, reset domains, runtime PM dependencies

Error handling Current mainline : Simple IRQ error flags New proposed driver: Detailed error codes, recovery sequences required

Code impact Current mainline :~8k LOC, minimal conditionals New proposed driver: ~65KLOC with full features

Plan for supporting latest MFC (Best balance between code clarity and long-term kernel health, while still respecting ABI and legacy users.) 1 We propose a new driver (exynos-mfc) for the latest generation of Samsung MFC (Multi-Format Codec) hardware. 2 MFC FW V6 to V12 will be supported by existing mainline (s5p-mfc) and later versions by Exynos-mfc driver 3 The existing MFC driver will remain available for older SoC’s and will not be broken. The new driver will only bind to new compatible strings in DT, avoiding regressions for legacy hardware. Samsung will take responsibility for maintaining both drivers until the old one can be formally marked as “legacy.” 4 Keep it buildable for old SoCs but clearly say new SoCs should use the new driver. 5 VIDEO_SAMSUNG_S5P_MFC and VIDEO_EXYNOS_MFC will both be supported for an agreed time 6 Long term Support for new driver  - commitment from Samsung 7 The hardware has diverged to the point where conditionally supporting both in one codebase is worse for long-term kernel health 8 Splitting keeps the code clean, testable, and reviewable, while ensuring legacy users aren’t broken due to user space ABI  guarantees. This seems more clean and easier approach Both can coexist without code spaghetti.

Features and description supported ++++++++++++++++++++++++++++++++++

1 MFC driver registration Probe functionality and video nodes registration 2 MFC driver open & close support MFC firmware loading,  "ls" & "cat" on the video nodes  and udev rules on the video nodes 3 H264 decoding H264 decoding support - HW supported profiles & levels (YUV420 Semi-Planar, 8-bit, min/max resolution) 4 HEVC decoding HEVC decoding support - HW supported profiles & levels (YUV420 Semi-Planar, 8-bit, min/max resolution) 5 VP8 decoding VP8 decoding support - HW supported profiles & levels (YUV420 Semi-Planar, 8-bit, min/max resolution) 6 VP9 decoding VP9 decoding support - HW supported profiles & levels (YUV420 Semi-Planar, 8-bit, min/max resolution) 7 AV1 decoding AV1 decoding support - HW supported profiles & levels (YUV420 Semi-Planar, 8-bit, min/max resolution) 8 H264 encoding H264 encoding support - Basic encoding (YUV420 Semi-Planar, 8-bit, min/max resolution) 9 HEVC encoding HEVC encoding support - Basic encoding (YUV420 Semi-Planar, 8-bit, min/max resolution) 10 VP8 encoding VP8 encoding support - Basic encoding (YUV420 Semi-Planar, 8-bit, min/max resolution) 11 VP9 encoding VP9 encoding support - Basic encoding (YUV420 Semi-Planar, 8-bit, min/max resolution) 12 Debug fs support Debug fs to control the MFC functionalities 13 Debug log support Debug logs to get on the dmesg prompt for debug purpose 14 Debug SFR dump support MFC SFR dumps during failures 15 NAL Queue mode control NAL Queue mode support 16 Batch mode control Batch mode support 17 HW supported YUV formats YUV 420, 422, 444 (planar, semi-planar) formats support as per MFC HW specification 18 HW supported RGB formats RGB format support as per MFC HW specification 19 Multi instance decoding/encoding Multi instance decoding/encoding 20 Suspend and resume Suspend and resume support 21 Runtime suspend and resume Runtime suspend and resume support 22 Priority decoding (decoders) Priority decoding support 23 Frame delay configuration (decoders) Frame delay configuration support 24 Error handling and conceal control (decoders) HW/Platform limitations should be gracefully exited and, bit stream errors should be reported or concealed with warning indication 25 Reference frame control (encoders) Reference frame control support 26 SPS/PPS control (encoders) SPS/PPS control support 27 Loop filter control (encoders) Loop filter control support 28 B frame support (encoders) B frame support 29 GOP control (encoders) GOP control support 30 Frame rate control (encoders) Frame rate control support

Features to be  supported in 2026 +++++++++++++++++++++++++++++++++

31 Bit rate control (encoders) Bit rate control support 32 QP control (encoders) QP control support 33 I/IDR control (encoders) I/IDR control support 34 Scalable control (encoders) Scalable control support 35 ROI control (encoders) ROI control support 36 Multi slice control (encoders) Multi slice control support 37 Profile/Level control (encoders) Profile/Level control support 38 Padding control (encoders) Padding control support 39 Aspect ratio control (encoders) Aspect ratio control support 40 Buffer full handling (encoders) Buffer full handling support 41 SBWC SBWC bandwidth compression support 42 10-bit support 10-bit support as per MFC HW specification 43 Secure signed FW support Security signed FW support 44 Secure playback DRM content playback for decoders 45 HDR Support (decoders) HDR metadata support 46 Dynamic resolution change Dynamic resolution change support

Nagaraju Siddineni (29):   dt-bindings: media: mfc: Add Exynos MFC devicetree binding   arm64: dts: mfc: Add MFC device tree for Auto V920 SoC   media: mfc: Add MFC driver data structures and debugging macros   media: mfc: Add full register map and bit definitions for MFC hardware   media: mfc: Add MFC driver header files and core utilities   media: mfc: Add MFC core hardware register and debugfs APIs   media: mfc: Add MFC core command, hwlock, ISR and run functionalities   media: mfc: Add Exynos‑MFC driver probe support   media: mfc: Add bus‑devfreq, QoS, multi‑view and control     infrastructure   media: mfc: Add buffer‑queue and IOVMM support   media: mfc: Add rate‑calculation framework and memory utilities   media: mfc: Introduce QoS support and instance context handling   media: mfc: Add decoder core sync functions   media: mfc: Add buffer‑control framework   media: mfc: Add decoder resource‑management (RM) support and     load‑balancing   media: mfc: Enhance HW‑lock handling, scheduling and error recovery   media: mfc: Add VB2 decoder support   media: mfc: Add V4L2 decoder driver   media: mfc: Add QoS, Butler workqueue, and priority‑based scheduling   media: mfc: Add H264 decoder support   media: mfc: Add multi‑codec support & QoS improvements   media: mfc: Add H.264 encoder support with buffer and QoS improvements   media: mfc: Add encoder parameters, ROI & QoS support   media: mfc: Add encoder VB2 support to driver   media: mfc: Add encoder v4l2 driver interface   media: mfc: Add full encoder support   media: mfc: Add H.264 encoder support   media: mfc: Add AVC, VP8, VP9, and HEVC encoding support   media: mfc: Hardware‑accelerated encoding support

.../bindings/media/samsung,exynos-mfc.yaml    |   77 +  MAINTAINERS                                   |   10 +  .../dts/exynos/exynosautov920-evt2-mfc.dtsi   |  630 +++  .../arm64/boot/dts/exynos/exynosautov920.dtsi |    1 +  drivers/media/platform/samsung/Kconfig        |    7 +  drivers/media/platform/samsung/Makefile       |    1 +  .../media/platform/samsung/exynos-mfc/Kconfig |   34 +  .../platform/samsung/exynos-mfc/Makefile      |   26 +  .../samsung/exynos-mfc/base/mfc_buf.c         |  765 +++  .../samsung/exynos-mfc/base/mfc_buf.h         |   43 +  .../samsung/exynos-mfc/base/mfc_common.h      |  444 ++  .../samsung/exynos-mfc/base/mfc_data_struct.h | 2014 +++++++  .../samsung/exynos-mfc/base/mfc_debug.h       |  247 +  .../samsung/exynos-mfc/base/mfc_format.h      |  316 ++  .../samsung/exynos-mfc/base/mfc_macros.h      |   95 +  .../samsung/exynos-mfc/base/mfc_media.h       |  554 ++  .../samsung/exynos-mfc/base/mfc_mem.c         |  995 ++++  .../samsung/exynos-mfc/base/mfc_mem.h         |  155 +  .../samsung/exynos-mfc/base/mfc_qos.c         | 1070 ++++  .../samsung/exynos-mfc/base/mfc_qos.h         |   99 +  .../samsung/exynos-mfc/base/mfc_queue.c       |  966 ++++  .../samsung/exynos-mfc/base/mfc_queue.h       |  158 +  .../exynos-mfc/base/mfc_rate_calculate.c      |  640 +++  .../exynos-mfc/base/mfc_rate_calculate.h      |  106 +  .../samsung/exynos-mfc/base/mfc_regs.h        |   58 +  .../samsung/exynos-mfc/base/mfc_regs_mfc.h    | 1002 ++++  .../samsung/exynos-mfc/base/mfc_sched.h       |   30 +  .../samsung/exynos-mfc/base/mfc_utils.c       |  401 ++  .../samsung/exynos-mfc/base/mfc_utils.h       |  481 ++  .../media/platform/samsung/exynos-mfc/mfc.c   | 1366 +++++  .../platform/samsung/exynos-mfc/mfc_core.c    |  703 +++  .../samsung/exynos-mfc/mfc_core_buf_ctrl.c    |  543 ++  .../samsung/exynos-mfc/mfc_core_cmd.c         |  576 ++  .../samsung/exynos-mfc/mfc_core_cmd.h         |   41 +  .../samsung/exynos-mfc/mfc_core_enc_param.c   | 1463 ++++++  .../samsung/exynos-mfc/mfc_core_enc_param.h   |   23 +  .../samsung/exynos-mfc/mfc_core_hw_reg_api.c  |  122 +  .../samsung/exynos-mfc/mfc_core_hw_reg_api.h  |  144 +  .../samsung/exynos-mfc/mfc_core_hwlock.c      |  761 +++  .../samsung/exynos-mfc/mfc_core_hwlock.h      |   80 +  .../samsung/exynos-mfc/mfc_core_intlock.c     |   98 +  .../samsung/exynos-mfc/mfc_core_intlock.h     |   20 +  .../samsung/exynos-mfc/mfc_core_isr.c         | 2104 ++++++++  .../samsung/exynos-mfc/mfc_core_isr.h         |   25 +  .../samsung/exynos-mfc/mfc_core_ops.c         |  813 +++  .../samsung/exynos-mfc/mfc_core_ops.h         |   16 +  .../platform/samsung/exynos-mfc/mfc_core_pm.c |  209 +  .../platform/samsung/exynos-mfc/mfc_core_pm.h |   33 +  .../samsung/exynos-mfc/mfc_core_reg_api.c     |  597 +++  .../samsung/exynos-mfc/mfc_core_reg_api.h     |  380 ++  .../samsung/exynos-mfc/mfc_core_run.c         |  541 ++  .../samsung/exynos-mfc/mfc_core_run.h         |   35 +  .../samsung/exynos-mfc/mfc_core_sched_prio.c  |  431 ++  .../samsung/exynos-mfc/mfc_core_sync.c        |  423 ++  .../samsung/exynos-mfc/mfc_core_sync.h        |   34 +  .../samsung/exynos-mfc/mfc_ctx_ctrl.c         | 1469 ++++++  .../platform/samsung/exynos-mfc/mfc_debugfs.c |  248 +  .../platform/samsung/exynos-mfc/mfc_debugfs.h |   20 +  .../samsung/exynos-mfc/mfc_dec_v4l2.c         | 1739 +++++++  .../samsung/exynos-mfc/mfc_dec_v4l2.h         |   20 +  .../platform/samsung/exynos-mfc/mfc_dec_vb2.c |  394 ++  .../platform/samsung/exynos-mfc/mfc_dec_vb2.h |   19 +  .../samsung/exynos-mfc/mfc_enc_v4l2.c         | 4614 +++++++++++++++++  .../samsung/exynos-mfc/mfc_enc_v4l2.h         |   20 +  .../platform/samsung/exynos-mfc/mfc_enc_vb2.c |  443 ++  .../platform/samsung/exynos-mfc/mfc_enc_vb2.h |   19 +  .../platform/samsung/exynos-mfc/mfc_rm.c      | 2652 ++++++++++  .../platform/samsung/exynos-mfc/mfc_rm.h      |  110 +  68 files changed, 34773 insertions(+)  create mode 100644 Documentation/devicetree/bindings/media/samsung,exynos-mfc.yaml  create mode 100644 arch/arm64/boot/dts/exynos/exynosautov920-evt2-mfc.dtsi  create mode 100644 drivers/media/platform/samsung/exynos-mfc/Kconfig  create mode 100644 drivers/media/platform/samsung/exynos-mfc/Makefile  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_buf.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_buf.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_common.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_data_struct.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_debug.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_format.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_macros.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_media.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_mem.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_mem.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_qos.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_qos.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_queue.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_queue.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_rate_calculate.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_rate_calculate.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_regs.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_regs_mfc.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_sched.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_utils.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/base/mfc_utils.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_buf_ctrl.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_cmd.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_cmd.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_enc_param.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_enc_param.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_hw_reg_api.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_hw_reg_api.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_hwlock.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_hwlock.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_intlock.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_intlock.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_isr.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_isr.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_ops.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_ops.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_pm.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_pm.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_reg_api.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_reg_api.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_run.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_run.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_sched_prio.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_sync.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_core_sync.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_ctx_ctrl.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_debugfs.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_debugfs.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_dec_v4l2.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_dec_v4l2.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_dec_vb2.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_dec_vb2.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_enc_v4l2.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_enc_v4l2.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_enc_vb2.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_enc_vb2.h  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_rm.c  create mode 100644 drivers/media/platform/samsung/exynos-mfc/mfc_rm.h