From: Mauro Carvalho Chehab mchehab+huawei@kernel.org
[ Upstream commit 2b16b71a05a7f056221751b906c13f8809656b1f ]
The logic there which adds a dependency note to Sphinx cache is not taking into account that the build dir may not be the source dir. This causes a performance regression:
$ time make O=/tmp/foo SPHINXDIRS=admin-guide htmldocs
[OUTDATED] Added: set() Changed: {'abi-obsolete', 'abi-removed', 'abi-stable-files', 'abi-obsolete-files', 'abi-stable', 'abi', 'abi-removed-files', 'abi-testing-files', 'abi-testing', 'gpio/index', 'gpio/obsolete'} Removed: set() All docs count: 385 Found docs count: 385
real 0m11,324s user 0m15,783s sys 0m1,164s
To get the root cause of the problem (ABI files reported as changed), I used this changeset:
diff --git a/Documentation/conf.py b/Documentation/conf.py index e8766e689c1b..ab486623bd8b 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -571,3 +571,16 @@ def setup(app): """Patterns need to be updated at init time on older Sphinx versions"""
app.connect('config-inited', update_patterns) + app.connect('env-get-outdated', on_outdated) + +def on_outdated(app, env, added, changed, removed): + """Track cache outdated due to added/changed/removed files""" + print("\n[OUTDATED]") + print(f"Added: {added}") + print(f"Changed: {changed}") + print(f"Removed: {removed}") + print(f"All docs count: {len(env.all_docs)}") + print(f"Found docs count: {len(env.found_docs)}") + + # Just return what we have + return added | changed | removed
Reported-by: Akira Yokosawa akiyks@gmail.com Closes: https://lore.kernel.org/linux-doc/c174f7c5-ec21-4eae-b1c3-f643cca90d9d@gmail... Signed-off-by: Mauro Carvalho Chehab mchehab+huawei@kernel.org Tested-by: Akira Yokosawa akiyks@gmail.com Signed-off-by: Jonathan Corbet corbet@lwn.net Link: https://lore.kernel.org/r/e25673d87357457bc54ee863e97ff8f75956580d.175275221... Signed-off-by: Sasha Levin sashal@kernel.org ---
LLM Generated explanations, may be completely bogus:
Now, let me understand the actual bug better. The issue is that when building with `O=<dir>`, the `f` variable contains a relative path from the source tree, but when we use `os.path.abspath(f)`, it resolves relative to the current working directory (which is the build directory), not the source tree. This causes Sphinx to think files have changed when they haven't.
**Backport Status: YES**
## Extensive Explanation:
This commit is a strong candidate for backporting to stable kernel trees for the following reasons:
### 1. **Clear Bug Fix for Performance Regression** The commit fixes a concrete performance regression in the documentation build system when using out-of-tree builds (`O=<dir>`). The commit message shows the issue causes unnecessary rebuilds with a ~11 second penalty due to incorrectly tracking file dependencies.
### 2. **Small and Contained Change** The code change is minimal and focused - only 4 lines are modified: - Line 149-150 (old): `env.note_dependency(os.path.abspath(f))` - Line 150-152 (new): Adds a proper path construction and file existence check
The fix correctly constructs the absolute path by joining `srctree` with the relative path `f`, ensuring the dependency tracking works correctly regardless of build directory location.
### 3. **Root Cause is Clear** The bug occurs because `os.path.abspath(f)` resolves relative paths from the current working directory (the build directory when using `O=`), not from the source tree. This causes Sphinx to track non-existent or wrong file paths as dependencies, leading to false positives in change detection.
### 4. **No Architectural Changes** This is a straightforward bugfix that: - Doesn't introduce new features - Doesn't change any APIs or interfaces - Only fixes incorrect path resolution logic - Adds a defensive file existence check
### 5. **Affects Common Use Case** Out-of-tree builds with `O=<dir>` are a common workflow in kernel development, especially for: - CI/CD systems - Clean builds - Testing multiple configurations - Keeping source trees pristine
### 6. **Low Risk of Regression** The change: - Only affects documentation builds - Adds a safety check (`os.path.isfile()`) that prevents tracking non- existent files - The logic is more correct than before (properly resolving paths relative to source tree) - Has been tested by the reporter (Akira Yokosawa)
### 7. **Meets Stable Criteria** According to stable kernel rules, this fix: - Fixes a real bug that users hit (performance regression in documentation builds) - Is obviously correct and tested - Is small (under 100 lines) - Fixes only one thing - Has no dependencies on other changes
The commit directly addresses a performance issue that affects kernel documentation builds, making it slower and less efficient when using out-of-tree builds. This is the type of bug fix that stable kernels should receive to maintain quality and performance for users who build documentation.
Documentation/sphinx/kernel_abi.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index db6f0380de94..4c4375201b9e 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -146,8 +146,10 @@ class KernelCmd(Directive): n += 1
if f != old_f: - # Add the file to Sphinx build dependencies - env.note_dependency(os.path.abspath(f)) + # Add the file to Sphinx build dependencies if the file exists + fname = os.path.join(srctree, f) + if os.path.isfile(fname): + env.note_dependency(fname)
old_f = f