On Mon, Oct 31, 2022 at 11:52 AM Hector Martin marcan@marcan.st wrote:
On 01/11/2022 01.15, Justin Forbes wrote:
On Thu, Oct 27, 2022 at 8:57 AM Hector Martin marcan@marcan.st wrote:
drm_fb_build_fourcc_list() currently returns all emulated formats unconditionally as long as the native format is among them, even though not all combinations have conversion helpers. Although the list is arguably provided to userspace in precedence order, userspace can pick something out-of-order (and thus break when it shouldn't), or simply only support a format that is unsupported (and thus think it can work, which results in the appearance of a hang as FB blits fail later on, instead of the initialization error you'd expect in this case).
Add checks to filter the list of emulated formats to only those supported for conversion to the native format. This presumes that there is a single native format (only the first is checked, if there are multiple). Refactoring this API to drop the native list or support it properly (by returning the appropriate emulated->native mapping table) is left for a future patch.
The simpledrm driver is left as-is with a full table of emulated formats. This keeps all currently working conversions available and drops all the broken ones (i.e. this a strict bugfix patch, adding no new supported formats nor removing any actually working ones). In order to avoid proliferation of emulated formats, future drivers should advertise only XRGB8888 as the sole emulated format (since some userspace assumes its presence).
This fixes a real user regression where the ?RGB2101010 support commit started advertising it unconditionally where not supported, and KWin decided to start to use it over the native format and broke, but also the fixes the spurious RGB565/RGB888 formats which have been wrongly unconditionally advertised since the dawn of simpledrm.
Fixes: 6ea966fca084 ("drm/simpledrm: Add [AX]RGB210101
Cc: stable@vger.kernel.org Signed-off-by: Hector Martin marcan@marcan.st
There is a CC for stable on here, but this patch does not apply in any way on 6.0 or older kernels as the fourcc bits and considerable churn came in with the 6.1 merge window. You don't happen to have a backport of this to 6.0 do you?
v1 is probably closer to such a backport, and I offered to figure it out on Matrix but I heard you're already working on it ;)
i am, but I didn't want to be, so I thought I would ask.
Justin