If two or more suitable entries with the same filename are found in __uc_fw_auto_select's fw_blobs, and that filename fails to load in the first attempt and in the retry, when __uc_fw_auto_select is called for the third time, the coincidence of strings will cause it to clear file_selected.path at the first hit, so it will return the second hit over and over again, indefinitely.
Of course this doesn't occur with the pristine blob lists, but a modified version could run into this, e.g., patching in a duplicate entry, or (as in our case) disarming blob loading by remapping their names to "/*(DEBLOBBED)*/", given a toolchain that unifies identical string literals.
Of course I'm ready to carry a patchlet to avoid this problem triggered by our (GNU Linux-libre's) intentional changes, but I figured you might be interested in fail-safing it even in accidental backporting circumstances. I realize it's not entirely foolproof: if the same string appears in two entries separated by a different one, the infinite loop might still occur. Catching that even more unlikely situation seemed too expensive.
Link: https://www.fsfla.org/pipermail/linux-libre/2023-March/003506.html Cc: intel-gfx@lists.freedesktop.org Cc: stable@vger.kernel.org # 6.[12].x Signed-off-by: Alexandre Oliva lxoliva@fsfla.org --- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c index 9d6f571097e6..2b7564a3ed82 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c @@ -259,7 +259,10 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw) uc_fw->file_selected.path = NULL;
continue; - } + } else if (uc_fw->file_wanted.path == blob->path) + /* Avoid retrying forever when neighbor + entries point to the same path. */ + continue;
uc_fw->file_selected.path = blob->path; uc_fw->file_wanted.path = blob->path;
On Sun, Mar 12, 2023 at 04:56:23PM -0300, Alexandre Oliva wrote:
If two or more suitable entries with the same filename are found in __uc_fw_auto_select's fw_blobs, and that filename fails to load in the first attempt and in the retry, when __uc_fw_auto_select is called for the third time, the coincidence of strings will cause it to clear file_selected.path at the first hit, so it will return the second hit over and over again, indefinitely.
Of course this doesn't occur with the pristine blob lists, but a modified version could run into this, e.g., patching in a duplicate entry, or (as in our case) disarming blob loading by remapping their names to "/*(DEBLOBBED)*/", given a toolchain that unifies identical string literals.
Of course I'm ready to carry a patchlet to avoid this problem triggered by our (GNU Linux-libre's) intentional changes, but I figured you might be interested in fail-safing it even in accidental backporting circumstances. I realize it's not entirely foolproof: if the same string appears in two entries separated by a different one, the infinite loop might still occur. Catching that even more unlikely situation seemed too expensive.
Link: https://www.fsfla.org/pipermail/linux-libre/2023-March/003506.html Cc: intel-gfx@lists.freedesktop.org Cc: stable@vger.kernel.org # 6.[12].x Signed-off-by: Alexandre Oliva lxoliva@fsfla.org
Cc: Daniele Ceraolo Spurio daniele.ceraolospurio@intel.com Cc: John Harrison John.C.Harrison@Intel.com
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c index 9d6f571097e6..2b7564a3ed82 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c @@ -259,7 +259,10 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw)
Since __uc_fw_auto_select is also called from another place, intel_uc_fw_init_early out of the intel_uc_fw_fetch infinite loop, I'm afraid this proposal below could have some side-effect.
I hope Daniele and John have a better understanding and can provide some guidance or acks here.
uc_fw->file_selected.path = NULL;continue;
}
} else if (uc_fw->file_wanted.path == blob->path)/* Avoid retrying forever when neighborentries point to the same path. */continue;uc_fw->file_selected.path = blob->path; uc_fw->file_wanted.path = blob->path; -- 2.25.1
-- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer Disinformation flourishes because many people care deeply about injustice but very few check the facts. Ask me about https://stallmansupport.org
On Mar 22, 2023, Rodrigo Vivi rodrigo.vivi@intel.com wrote:
On Sun, Mar 12, 2023 at 04:56:23PM -0300, Alexandre Oliva wrote:
Since __uc_fw_auto_select is also called from another place, intel_uc_fw_init_early out of the intel_uc_fw_fetch infinite loop,
That other place is conceptually, sort of, the first iteration of the infinite loop. Before that first separate early-init call, *uc_fw is returned by devm_drm_dev_alloc to i915_driver_create, so zero-initialized I presume, in both the guc and the huc cases.
Only if this first call finds a matching entry (setting both file_{selected,wanted}.path), and the selected entry fails to load, do we even enter the loop (provided that other conditionals are satisfied) and look for other entries, using file_selected.path to find how far the previous call got (and, with the proposed patch, file_wanted.path to avoid retrying the path we've just tried).
I hope Daniele and John have a better understanding and can provide some guidance or acks here.
I surely appreciate additional eyes and minds that are more acquainted with the code at hand than I am. Thanks,
On 3/12/2023 12:56, Alexandre Oliva wrote:
If two or more suitable entries with the same filename are found in __uc_fw_auto_select's fw_blobs, and that filename fails to load in the first attempt and in the retry, when __uc_fw_auto_select is called for the third time, the coincidence of strings will cause it to clear file_selected.path at the first hit, so it will return the second hit over and over again, indefinitely.
Of course this doesn't occur with the pristine blob lists, but a modified version could run into this, e.g., patching in a duplicate entry, or (as in our case) disarming blob loading by remapping their names to "/*(DEBLOBBED)*/", given a toolchain that unifies identical string literals.
Not sure what you mean by disarming?
I think what you are saying is that you made a change similar to this? #define __MAKE_UC_FW_PATH_MMP(prefix_, name_, major_, minor_, patch_) "i915/invalid_file_name.bin"
So all entries in the table have the exact same filename. And with the toolchain unification comment, that means not just a matching string but the same string pointer. Thus, the search code is getting confused.
I'm not sure that is really a valid use case that the driver code should be expected to support. I'm not even sure what the purpose of your testing is? Even without the infinite loop, the driver is not going to load because you have removed the firmware files?
However, I think you are saying that the problem would also exist if there was some kind of genuine duplication in the table? Can you give an example of a genuine use case problem? If the same string is used for different platforms, I believe that should be fine. E.g. there are already a bunch of different platforms that all use the same TGL firmware file. Even with the string unification, that should not be an issue because the search is within a platform only. So there can only be a problem if a single platform specifies the same filename multiple times? Which would be a bug in the table because why? It would be redundant entries that have no purpose.
Note that I'm not saying we don't want to take your change. But I would like to understand if there is a genuine issue that maybe needs a better fix. E.g. should the table verification code be enhanced to just reject the table entirely if there are such errors present.
Also, is this string unification thing a part of the current gcc toolchain? Or are you saying that is a new feature that is not generally available yet? Or maybe only exists in some non-gcc toolchain?
Thanks, John.
Of course I'm ready to carry a patchlet to avoid this problem triggered by our (GNU Linux-libre's) intentional changes, but I figured you might be interested in fail-safing it even in accidental backporting circumstances. I realize it's not entirely foolproof: if the same string appears in two entries separated by a different one, the infinite loop might still occur. Catching that even more unlikely situation seemed too expensive.
Link: https://www.fsfla.org/pipermail/linux-libre/2023-March/003506.html Cc: intel-gfx@lists.freedesktop.org Cc: stable@vger.kernel.org # 6.[12].x Signed-off-by: Alexandre Oliva lxoliva@fsfla.org
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c index 9d6f571097e6..2b7564a3ed82 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c @@ -259,7 +259,10 @@ __uc_fw_auto_select(struct drm_i915_private *i915, struct intel_uc_fw *uc_fw) uc_fw->file_selected.path = NULL; continue;
}
} else if (uc_fw->file_wanted.path == blob->path)/* Avoid retrying forever when neighborentries point to the same path. */continue;uc_fw->file_selected.path = blob->path; uc_fw->file_wanted.path = blob->path;
Hello, John,
On Mar 24, 2023, John Harrison john.c.harrison@intel.com wrote:
On 3/12/2023 12:56, Alexandre Oliva wrote:
If two or more suitable entries with the same filename are found in __uc_fw_auto_select's fw_blobs, and that filename fails to load in the first attempt and in the retry, when __uc_fw_auto_select is called for the third time, the coincidence of strings will cause it to clear file_selected.path at the first hit, so it will return the second hit over and over again, indefinitely.
Of course this doesn't occur with the pristine blob lists, but a modified version could run into this, e.g., patching in a duplicate entry, or (as in our case) disarming blob loading by remapping their names to "/*(DEBLOBBED)*/", given a toolchain that unifies identical string literals.
Not sure what you mean by disarming?
Our users find loading nonfree firmware harmful.
I think what you are saying is that you made a change similar to this? #define __MAKE_UC_FW_PATH_MMP(prefix_, name_, major_, minor_, patch_) "i915/invalid_file_name.bin"
Yeah, that's the jist of it. The name we use is "/*(DEBLOBBED)*/", so that it can't possibly be satisfied.
So all entries in the table have the exact same filename.
*nod*
And with the toolchain unification comment, that means not just a matching string but the same string pointer. Thus, the search code is getting confused.
Exactly
I'm not sure that is really a valid use case that the driver code should be expected to support.
It's most certainly not. As I wrote, I'd be happy to keep on carrying the patch that adjusts the code to cope with our changes. I just thought the same issue could come up by, say, mistakenly applying a patch twice to add support for a new card, a circumstance in which one might not have the card readily available to try it out.
Even without the infinite loop, the driver is not going to load because you have removed the firmware files?
Oh, no, the driver loads just fine even without those blobs, and that's much nicer of you than other drivers for hardware that doesn't really require blobs, but that insist on bailing out if the firmware can't be loaded. i915 hasn't been hostile like that.
When you override the firmware filenames, and it fails to load, the driver makes it a (reasonable IMHO) hard fail, but when it just fails to find the regular firmware files, it's nice that it proceeds that does the best it can.
However, I think you are saying that the problem would also exist if there was some kind of genuine duplication in the table?
Yes. Not the kind you mention, for different platforms, but an actual duplicate entry, such as what you might get if you applied a patch that added an entry for a new card, and then applied it again, resolving the conflicts in a way that retained the duplicate entries.
So there can only be a problem if a single platform specifies the same filename multiple times? Which would be a bug in the table because why? It would be redundant entries that have no purpose.
Agreed.
Note that I'm not saying we don't want to take your change. But I would like to understand if there is a genuine issue that maybe needs a better fix. E.g. should the table verification code be enhanced to just reject the table entirely if there are such errors present.
Table verification might wish to detect and report duplicate filenames for the same platform, to catch even alternating duplicates (e.g. "a", then "b", then "a" again), but it would be kind if you didn't make that a hard error, otherwise we'd have to tweak it to cope with our own "/*(DEBLOBBED)*/" duplicates.
Another approach, that would probably be more efficient as the table grows, is to store in uc_fw a pointer to or index of the current or next entry to be searched, so that the code doesn't have to iterate over the table at every try (O(n^2)), and instead takes it from exactly where it left off, running overall a single time over the whole table (O(n)), at the cost of a pointer or index in uc_fw. Then, duplicates in the table wouldn't matter at all.
Also, is this string unification thing a part of the current gcc toolchain?
Yeah, compilers and linkers have been unifying (read-only) string literals for a very long time.
Thanks,
On 3/26/2023 02:46, Alexandre Oliva wrote:
Hello, John,
On Mar 24, 2023, John Harrison john.c.harrison@intel.com wrote:
On 3/12/2023 12:56, Alexandre Oliva wrote:
If two or more suitable entries with the same filename are found in __uc_fw_auto_select's fw_blobs, and that filename fails to load in the first attempt and in the retry, when __uc_fw_auto_select is called for the third time, the coincidence of strings will cause it to clear file_selected.path at the first hit, so it will return the second hit over and over again, indefinitely.
Of course this doesn't occur with the pristine blob lists, but a modified version could run into this, e.g., patching in a duplicate entry, or (as in our case) disarming blob loading by remapping their names to "/*(DEBLOBBED)*/", given a toolchain that unifies identical string literals.
Not sure what you mean by disarming?
Our users find loading nonfree firmware harmful.
I think what you are saying is that you made a change similar to this? #define __MAKE_UC_FW_PATH_MMP(prefix_, name_, major_, minor_, patch_) "i915/invalid_file_name.bin"
Yeah, that's the jist of it. The name we use is "/*(DEBLOBBED)*/", so that it can't possibly be satisfied.
So all entries in the table have the exact same filename.
*nod*
And with the toolchain unification comment, that means not just a matching string but the same string pointer. Thus, the search code is getting confused.
Exactly
I'm not sure that is really a valid use case that the driver code should be expected to support.
It's most certainly not. As I wrote, I'd be happy to keep on carrying the patch that adjusts the code to cope with our changes. I just thought the same issue could come up by, say, mistakenly applying a patch twice to add support for a new card, a circumstance in which one might not have the card readily available to try it out.
Not following this argument. You can't add support for a card that you don't have access to. GuC firmware is produced internally by Intel so it isn't going to be added by some third party person. And internally, we have CI systems up and running for each platform before the patches to support that platform land in the upstream tree. So any such error most certainly should be caught by pre-merge CI.
Even without the infinite loop, the driver is not going to load because you have removed the firmware files?
Oh, no, the driver loads just fine even without those blobs, and that's much nicer of you than other drivers for hardware that doesn't really require blobs, but that insist on bailing out if the firmware can't be loaded. i915 hasn't been hostile like that.
That situation won't last...
When you override the firmware filenames, and it fails to load, the driver makes it a (reasonable IMHO) hard fail, but when it just fails to find the regular firmware files, it's nice that it proceeds that does the best it can.
However, I think you are saying that the problem would also exist if there was some kind of genuine duplication in the table?
Yes. Not the kind you mention, for different platforms, but an actual duplicate entry, such as what you might get if you applied a patch that added an entry for a new card, and then applied it again, resolving the conflicts in a way that retained the duplicate entries.
I would consider that a bug that should never make it past either pre-merge CI or code review.
Also, that is what we have the table verification code for - ensuring that bugs don't creep in to the table. So if you have spotted a hole in that verification then I do think it needs plugging.
Unfortunately for you, I think that is the best way forward for i915/Intel. Enhancing the verification step to ensure that such bugs can't happen before it gets to do the search. However, I think there are easier ways for you to modify the driver to prevent firmware loading. E.g. rather than modifying the table, just force an early exit from the loading code itself. And if you really do need to remove the firmware files from the compiled binary completely, then replacing them with unique names would also work - '/*(DEBLOBBED_1)*/', '/*(DEBLOBBED_2)*/', etc.
So there can only be a problem if a single platform specifies the same filename multiple times? Which would be a bug in the table because why? It would be redundant entries that have no purpose.
Agreed.
Note that I'm not saying we don't want to take your change. But I would like to understand if there is a genuine issue that maybe needs a better fix. E.g. should the table verification code be enhanced to just reject the table entirely if there are such errors present.
Table verification might wish to detect and report duplicate filenames for the same platform, to catch even alternating duplicates (e.g. "a", then "b", then "a" again), but it would be kind if you didn't make that a hard error, otherwise we'd have to tweak it to cope with our own "/*(DEBLOBBED)*/" duplicates.
Another approach, that would probably be more efficient as the table grows, is to store in uc_fw a pointer to or index of the current or next entry to be searched, so that the code doesn't have to iterate over the table at every try (O(n^2)), and instead takes it from exactly where it left off, running overall a single time over the whole table (O(n)), at the cost of a pointer or index in uc_fw. Then, duplicates in the table wouldn't matter at all.
Also, is this string unification thing a part of the current gcc toolchain?
Yeah, compilers and linkers have been unifying (read-only) string literals for a very long time.
That's what I would have assumed. Which is why I was confused that you were saying 'if you use a toolchain that does this'. It seemed that you were implying that most don't and this was a special situation.
John.
Thanks,
On Mar 31, 2023, John Harrison john.c.harrison@intel.com wrote:
I'm not sure that is really a valid use case that the driver code should be expected to support.
It's most certainly not. As I wrote, I'd be happy to keep on carrying the patch that adjusts the code to cope with our changes. I just thought the same issue could come up by, say, mistakenly applying a patch twice to add support for a new card, a circumstance in which one might not have the card readily available to try it out.
Not following this argument.
I was talking about downstream backporting, e.g. random users or small-distro maintainers attempting to backport support for certain cards without realizing it's already there.
Oh, no, the driver loads just fine even without those blobs, and that's much nicer of you than other drivers for hardware that doesn't really require blobs, but that insist on bailing out if the firmware can't be loaded. i915 hasn't been hostile like that.
That situation won't last...
:-(
I would consider that a bug that should never make it past either pre-merge CI or code review.
Agreed.
And if you really do need to remove the firmware files from the compiled binary completely, then replacing them with unique names would also work - '/*(DEBLOBBED_1)*/', '/*(DEBLOBBED_2)*/', etc.
That is not doable with our current deblob-check implementation, unfortunately. There are long-term plans to switch to a different approach, but we're not there yet. So I guess we'll have to use custom code to disable blob loading on i915, while that's still possible, when the stricter table checking hits.
Also, is this string unification thing a part of the current gcc toolchain?
Yeah, compilers and linkers have been unifying (read-only) string literals for a very long time.
That's what I would have assumed. Which is why I was confused that you were saying 'if you use a toolchain that does this'. It seemed that you were implying that most don't and this was a special situation.
Oh, no, sorry, it's just that compilers can't be dependent on for string literal unification. It's an optimization, not a language-imposed requirement.
Thanks for considering the patch, and for the heads up about darker days coming for users of Intel video cards! :-(
linux-stable-mirror@lists.linaro.org
-
Alexandre Oliva -
John Harrison -
Rodrigo Vivi