From: Ville Syrjälä ville.syrjala@linux.intel.com
[ Upstream commit 404c3acda4b65924c05bc63242e94f954f84c165 ]
Our current limited range matrix is a bit off. I think it was originally calculated with rounding, as if we wanted the normal pixel replication type of behaviour. That is, since the 8bpc max value is 0xeb we assumed the 16bpc max value should be 0xebeb, but what the HDMI spec actually says it should be is 0xeb00.
So to get what we want we make the formula out = in * (235-16) << (12-8) / in_max + 16 << (12-8), with 12 being precision of the csc, 8 being the precision of the constants we used.
The hardware takes its coefficients as floating point values, but the (235−16)/255 = ~.86, so exponent 0 is what we want anyway, so it works out perfectly without having to hardcode it in hex or start playing with floats.
In terms of raw numbers we are feeding the hardware the post offset changes from 0x101 to 0x100, and the coefficient changes from 0xdc0 to 0xdb0 (~.860->~.855). So this should make everything come out just a tad darker.
I already used better constants in lut_limited_range() earlier so the output of the two paths should be closer now.
Signed-off-by: Ville Syrjälä ville.syrjala@linux.intel.com Link: https://patchwork.freedesktop.org/patch/msgid/20230329135002.3096-2-ville.sy... Reviewed-by: Ankit Nautiyal ankit.k.nautiyal@intel.com Stable-dep-of: 19db2062094c ("drm/i915: No 10bit gamma on desktop gen3 parts") Signed-off-by: Sasha Levin sashal@kernel.org --- drivers/gpu/drm/i915/display/intel_color.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_color.c b/drivers/gpu/drm/i915/display/intel_color.c index 36aac88143ac1..3c3e2f5a5cdec 100644 --- a/drivers/gpu/drm/i915/display/intel_color.c +++ b/drivers/gpu/drm/i915/display/intel_color.c @@ -116,10 +116,9 @@ struct intel_color_funcs { #define ILK_CSC_COEFF_FP(coeff, fbits) \ (clamp_val(((coeff) >> (32 - (fbits) - 3)) + 4, 0, 0xfff) & 0xff8)
-#define ILK_CSC_COEFF_LIMITED_RANGE 0x0dc0 #define ILK_CSC_COEFF_1_0 0x7800 - -#define ILK_CSC_POSTOFF_LIMITED_RANGE (16 * (1 << 12) / 255) +#define ILK_CSC_COEFF_LIMITED_RANGE ((235 - 16) << (12 - 8)) /* exponent 0 */ +#define ILK_CSC_POSTOFF_LIMITED_RANGE (16 << (12 - 8))
/* Nop pre/post offsets */ static const u16 ilk_csc_off_zero[3] = {};