Progress
* UM-2 [QEMU upstream maintainership]
+ Respin of a linux-user cleanup patchset
+ Code review, as usual
* QEMU-406 [QEMU support for MVE (M-profile Vector Extension; Helium)]
+ Working on version 2 of the "optimized code gen for MVE" patchset;
this now covers all the insns that have an easy optimized version.
-- PMM
Successfully identified regression in *linux* in CI configuration tcwg_kernel/gnu-master-arm-mainline-allmodconfig. So far, this commit has regressed CI configurations:
- tcwg_kernel/gnu-master-arm-mainline-allmodconfig
Culprit:
<cut>
commit 3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151
Author: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sun Sep 5 11:24:05 2021 -0700
Enable '-Werror' by default for all kernel builds
... but make it a config option so that broken environments can disable
it when required.
We really should always have a clean build, and will disable specific
over-eager warnings as required, if we can't fix them. But while I
fairly religiously enforce that in my own tree, it doesn't get enforced
by various build robots that don't necessarily report warnings.
So this just makes '-Werror' a default compiler flag, but allows people
to disable it for their configuration if they have some particular
issues.
Occasionally, new compiler versions end up enabling new warnings, and it
can take a while before we have them fixed (or the warnings disabled if
that is what it takes), so the config option allows for that situation.
Hopefully this will mean that I get fewer pull requests that have new
warnings that were not noticed by various automation we have in place.
Knock wood.
Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org>
</cut>
Results regressed to (for first_bad == 3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
21769
# First few build errors in logs:
from (for last_good == fd47ff55c9c31101fcc06d20cb381da3d4089bd5)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1:
-5
# build_abe qemu:
-2
# linux_n_obj:
29880
# linux build successful:
all
Artifacts of last_good build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Build top page/logs: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Configuration details:
Reproduce builds:
<cut>
mkdir investigate-linux-3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151
cd investigate-linux-3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151
git clone https://git.linaro.org/toolchain/jenkins-scripts
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_kernel-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /linux/ ./ ./bisect/baseline/
cd linux
# Reproduce first_bad build
git checkout --detach 3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach fd47ff55c9c31101fcc06d20cb381da3d4089bd5
../artifacts/test.sh
cd ..
</cut>
History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/…
Artifacts: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Build log: https://ci.linaro.org/job/tcwg_kernel-gnu-bisect-gnu-master-arm-mainline-al…
Full commit (up to 1000 lines):
<cut>
commit 3fe617ccafd6f5bb33c2391d6f4eeb41c1fd0151
Author: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sun Sep 5 11:24:05 2021 -0700
Enable '-Werror' by default for all kernel builds
... but make it a config option so that broken environments can disable
it when required.
We really should always have a clean build, and will disable specific
over-eager warnings as required, if we can't fix them. But while I
fairly religiously enforce that in my own tree, it doesn't get enforced
by various build robots that don't necessarily report warnings.
So this just makes '-Werror' a default compiler flag, but allows people
to disable it for their configuration if they have some particular
issues.
Occasionally, new compiler versions end up enabling new warnings, and it
can take a while before we have them fixed (or the warnings disabled if
that is what it takes), so the config option allows for that situation.
Hopefully this will mean that I get fewer pull requests that have new
warnings that were not noticed by various automation we have in place.
Knock wood.
Signed-off-by: Linus Torvalds <torvalds(a)linux-foundation.org>
---
Makefile | 3 +++
init/Kconfig | 14 ++++++++++++++
2 files changed, 17 insertions(+)
diff --git a/Makefile b/Makefile
index 6bc1c5b17a62..d45fc2edf186 100644
--- a/Makefile
+++ b/Makefile
@@ -785,6 +785,9 @@ stackp-flags-$(CONFIG_STACKPROTECTOR_STRONG) := -fstack-protector-strong
KBUILD_CFLAGS += $(stackp-flags-y)
+KBUILD_CFLAGS-$(CONFIG_WERROR) += -Werror
+KBUILD_CFLAGS += $(KBUILD_CFLAGS-y)
+
ifdef CONFIG_CC_IS_CLANG
KBUILD_CPPFLAGS += -Qunused-arguments
# The kernel builds with '-std=gnu89' so use of GNU extensions is acceptable.
diff --git a/init/Kconfig b/init/Kconfig
index e708180e9a59..8cb97f141b70 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -137,6 +137,20 @@ config COMPILE_TEST
here. If you are a user/distributor, say N here to exclude useless
drivers to be distributed.
+config WERROR
+ bool "Compile the kernel with warnings as errors"
+ default y
+ help
+ A kernel build should not cause any compiler warnings, and this
+ enables the '-Werror' flag to enforce that rule by default.
+
+ However, if you have a new (or very old) compiler with odd and
+ unusual warnings, or you have some architecture with problems,
+ you may need to disable this config option in order to
+ successfully build the kernel.
+
+ If in doubt, say Y.
+
config UAPI_HEADER_TEST
bool "Compile test UAPI headers"
depends on HEADERS_INSTALL && CC_CAN_LINK
</cut>
Successfully identified regression in *llvm* in CI configuration tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3_LTO. So far, this commit has regressed CI configurations:
- tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3_LTO
Culprit:
<cut>
commit 19dc02e99f802922a3af69e802465bee0723b57a
Author: Nikita Popov <nikita.ppv(a)gmail.com>
Date: Sun Aug 22 18:15:55 2021 +0200
[MergeICmps] Allow sinking past non-load/store
This is a followup to D106591. MergeICmps currently only allows
sinking the loads past either instructions that don't write to
memory at all, or simple loads/stores that don't modify the memory
the loads access.
The "simple loads/stores" part of this check doesn't seem necessary
to me -- AA isModRef() already accurately models any operation
that may clobber the memory. For example, in the adjusted test case
the transform is still fine if the call to @foo() isn't readonly,
but inaccessiblememonly -- in both cases, the call cannot modify
the loaded memory.
Differential Revision: https://reviews.llvm.org/D108517
</cut>
Results regressed to (for first_bad == 19dc02e99f802922a3af69e802465bee0723b57a)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O3_LTO artifacts/build-19dc02e99f802922a3af69e802465bee0723b57a/results_id:
1
# 464.h264ref,h264ref_base.default regressed by 105
from (for last_good == da12d88b1c5fc42b49b92fcf94917ca489dd677f)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O3_LTO artifacts/build-da12d88b1c5fc42b49b92fcf94917ca489dd677f/results_id:
1
Artifacts of last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of last_good: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O3_LTO/4822
Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of first_bad: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O3_LTO/4807
Build top page/logs: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Configuration details:
Reproduce builds:
<cut>
mkdir investigate-llvm-19dc02e99f802922a3af69e802465bee0723b57a
cd investigate-llvm-19dc02e99f802922a3af69e802465bee0723b57a
git clone https://git.linaro.org/toolchain/jenkins-scripts
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /llvm/ ./ ./bisect/baseline/
cd llvm
# Reproduce first_bad build
git checkout --detach 19dc02e99f802922a3af69e802465bee0723b57a
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach da12d88b1c5fc42b49b92fcf94917ca489dd677f
../artifacts/test.sh
cd ..
</cut>
History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/…
Artifacts: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Build log: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Full commit (up to 1000 lines):
<cut>
commit 19dc02e99f802922a3af69e802465bee0723b57a
Author: Nikita Popov <nikita.ppv(a)gmail.com>
Date: Sun Aug 22 18:15:55 2021 +0200
[MergeICmps] Allow sinking past non-load/store
This is a followup to D106591. MergeICmps currently only allows
sinking the loads past either instructions that don't write to
memory at all, or simple loads/stores that don't modify the memory
the loads access.
The "simple loads/stores" part of this check doesn't seem necessary
to me -- AA isModRef() already accurately models any operation
that may clobber the memory. For example, in the adjusted test case
the transform is still fine if the call to @foo() isn't readonly,
but inaccessiblememonly -- in both cases, the call cannot modify
the loaded memory.
Differential Revision: https://reviews.llvm.org/D108517
---
llvm/lib/Transforms/Scalar/MergeICmps.cpp | 14 +-------------
.../Transforms/MergeICmps/X86/split-block-does-work.ll | 2 +-
2 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/llvm/lib/Transforms/Scalar/MergeICmps.cpp b/llvm/lib/Transforms/Scalar/MergeICmps.cpp
index f13f24ad2027..34465c76dd3d 100644
--- a/llvm/lib/Transforms/Scalar/MergeICmps.cpp
+++ b/llvm/lib/Transforms/Scalar/MergeICmps.cpp
@@ -66,15 +66,6 @@ namespace {
#define DEBUG_TYPE "mergeicmps"
-// Returns true if the instruction is a simple load or a simple store
-static bool isSimpleLoadOrStore(const Instruction *I) {
- if (const LoadInst *LI = dyn_cast<LoadInst>(I))
- return LI->isSimple();
- if (const StoreInst *SI = dyn_cast<StoreInst>(I))
- return SI->isSimple();
- return false;
-}
-
// A BCE atom "Binary Compare Expression Atom" represents an integer load
// that is a constant offset from a base value, e.g. `a` or `o.c` in the example
// at the top.
@@ -244,10 +235,7 @@ bool BCECmpBlock::canSinkBCECmpInst(const Instruction *Inst,
// If this instruction may clobber the loads and is in middle of the BCE cmp
// block instructions, then bail for now.
if (Inst->mayWriteToMemory()) {
- // Bail if this is not a simple load or store
- if (!isSimpleLoadOrStore(Inst))
- return false;
- // Disallow stores that might alias the BCE operands
+ // Disallow instructions that might modify the BCE operands
MemoryLocation LLoc = MemoryLocation::get(Cmp.Lhs.LoadI);
MemoryLocation RLoc = MemoryLocation::get(Cmp.Rhs.LoadI);
if (isModSet(AA.getModRefInfo(Inst, LLoc)) ||
diff --git a/llvm/test/Transforms/MergeICmps/X86/split-block-does-work.ll b/llvm/test/Transforms/MergeICmps/X86/split-block-does-work.ll
index 0b9663f44980..1e341b92918d 100644
--- a/llvm/test/Transforms/MergeICmps/X86/split-block-does-work.ll
+++ b/llvm/test/Transforms/MergeICmps/X86/split-block-does-work.ll
@@ -3,7 +3,7 @@
%S = type { i32, i32, i32, i32 }
-declare void @foo(...) readonly
+declare void @foo(...) inaccessiblememonly
; We can split %entry and create a memcmp(16 bytes).
define zeroext i1 @opeq1(
</cut>
Successfully identified regression in *llvm* in CI configuration tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O2. So far, this commit has regressed CI configurations:
- tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O2
Culprit:
<cut>
commit d39d3a327b1303012370e47d991459ffbfce45ef
Author: Peyton, Jonathan L <jonathan.l.peyton(a)intel.com>
Date: Fri Aug 20 16:06:13 2021 -0500
[OpenMP][test] fix omp_get_wtime.c test to be more accommodating
The omp_get_wtime.c test fails intermittently if the recorded times are
off by too much which can happen when many tests are run in parallel.
Instead of failing if one timing is a little off, take average of 100
timings minus the 10 worst.
Differential Revision: https://reviews.llvm.org/D108488
</cut>
Results regressed to (for first_bad == d39d3a327b1303012370e47d991459ffbfce45ef)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O2 artifacts/build-d39d3a327b1303012370e47d991459ffbfce45ef/results_id:
1
# 447.dealII,dealII_base.default regressed by 105
from (for last_good == f77174d4b8cfba3c0a53c78e53edbbaf57e37fc5)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O2 artifacts/build-f77174d4b8cfba3c0a53c78e53edbbaf57e37fc5/results_id:
1
Artifacts of last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of last_good: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O2/4734
Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of first_bad: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O2/4757
Build top page/logs: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Configuration details:
Reproduce builds:
<cut>
mkdir investigate-llvm-d39d3a327b1303012370e47d991459ffbfce45ef
cd investigate-llvm-d39d3a327b1303012370e47d991459ffbfce45ef
git clone https://git.linaro.org/toolchain/jenkins-scripts
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /llvm/ ./ ./bisect/baseline/
cd llvm
# Reproduce first_bad build
git checkout --detach d39d3a327b1303012370e47d991459ffbfce45ef
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach f77174d4b8cfba3c0a53c78e53edbbaf57e37fc5
../artifacts/test.sh
cd ..
</cut>
History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/…
Artifacts: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Build log: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Full commit (up to 1000 lines):
<cut>
commit d39d3a327b1303012370e47d991459ffbfce45ef
Author: Peyton, Jonathan L <jonathan.l.peyton(a)intel.com>
Date: Fri Aug 20 16:06:13 2021 -0500
[OpenMP][test] fix omp_get_wtime.c test to be more accommodating
The omp_get_wtime.c test fails intermittently if the recorded times are
off by too much which can happen when many tests are run in parallel.
Instead of failing if one timing is a little off, take average of 100
timings minus the 10 worst.
Differential Revision: https://reviews.llvm.org/D108488
---
openmp/runtime/test/api/omp_get_wtime.c | 75 ++++++++++++++++++++++++++-------
1 file changed, 59 insertions(+), 16 deletions(-)
diff --git a/openmp/runtime/test/api/omp_get_wtime.c b/openmp/runtime/test/api/omp_get_wtime.c
index e2bb211e0ce4..a862e07fc5a2 100644
--- a/openmp/runtime/test/api/omp_get_wtime.c
+++ b/openmp/runtime/test/api/omp_get_wtime.c
@@ -4,30 +4,73 @@
#include "omp_testsuite.h"
#include "omp_my_sleep.h"
-int test_omp_get_wtime()
-{
+#define NTIMES 100
+
+// This is the error % threshold. Be generous with the error threshold since
+// this test may be run in parallel with many other tests it may throw off the
+// sleep timing.
+#define THRESHOLD 33.0
+
+double test_omp_get_wtime(double desired_wait_time) {
double start;
double end;
- double measured_time;
- double wait_time = 0.2;
start = 0;
end = 0;
start = omp_get_wtime();
- my_sleep (wait_time);
+ my_sleep(desired_wait_time);
end = omp_get_wtime();
- measured_time = end-start;
- return ((measured_time > 0.97 * wait_time) && (measured_time < 1.03 * wait_time)) ;
+ return end - start;
}
-int main()
-{
- int i;
- int num_failed=0;
+int compare_times(const void *lhs, const void *rhs) {
+ const double *a = (const double *)lhs;
+ const double *b = (const double *)rhs;
+ return *a - *b;
+}
+
+int main() {
+ int i, final_count;
+ double percent_off;
+ double *begin, *end, *ptr;
+ double wait_time = 0.01;
+ double average = 0.0;
+ double n = 0.0;
+ double *times = (double *)malloc(sizeof(double) * NTIMES);
+
+ // Get each timing
+ for (i = 0; i < NTIMES; i++) {
+ times[i] = test_omp_get_wtime(wait_time);
+ }
+
+ // Remove approx the "worst" tenth of the timings
+ qsort(times, NTIMES, sizeof(double), compare_times);
+ begin = times;
+ end = times + NTIMES;
+ for (i = 0; i < NTIMES / 10; ++i) {
+ if (i % 2 == 0)
+ begin++;
+ else
+ end--;
+ }
+
+ // Get the average of the remaining timings
+ for (ptr = begin, final_count = 0; ptr != end; ++ptr, ++final_count)
+ average += times[i];
+ average /= (double)final_count;
+ free(times);
+
+ // Calculate the percent off of desired wait time
+ percent_off = (average - wait_time) / wait_time * 100.0;
+ // Should always be positive, but just in case
+ if (percent_off < 0)
+ percent_off = -percent_off;
- for(i = 0; i < REPETITIONS; i++) {
- if(!test_omp_get_wtime()) {
- num_failed++;
- }
+ if (percent_off > (double)THRESHOLD) {
+ fprintf(stderr, "error: average of %d runs (%lf) is of by %lf%%\n", NTIMES,
+ average, percent_off);
+ return EXIT_FAILURE;
}
- return num_failed;
+ printf("pass: average of %d runs (%lf) is only off by %lf%%\n", NTIMES,
+ average, percent_off);
+ return EXIT_SUCCESS;
}
</cut>
Successfully identified regression in *linux* in CI configuration tcwg_kernel/llvm-master-aarch64-mainline-allyesconfig. So far, this commit has regressed CI configurations:
- tcwg_kernel/llvm-master-aarch64-mainline-allyesconfig
Culprit:
<cut>
commit 342f43af70dbc74f8629381998f92c060e1763a2
Author: Maurizio Lombardi <mlombard(a)redhat.com>
Date: Thu Jul 29 15:52:50 2021 +0200
iscsi_ibft: fix crash due to KASLR physical memory remapping
Starting with commit a799c2bd29d1
("x86/setup: Consolidate early memory reservations")
memory reservations have been moved earlier during the boot process,
before the execution of the Kernel Address Space Layout Randomization code.
setup_arch() calls the iscsi_ibft's find_ibft_region() function
to find and reserve the memory dedicated to the iBFT and this function
also saves a virtual pointer to the iBFT table for later use.
The problem is that if KALSR is active, the physical memory gets
remapped somewhere else in the virtual address space and the pointer is
no longer valid, this will cause a kernel panic when the iscsi driver tries
to dereference it.
iBFT detected.
BUG: unable to handle page fault for address: ffff888000099fd8
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
..snip..
Call Trace:
? ibft_create_kobject+0x1d2/0x1d2 [iscsi_ibft]
do_one_initcall+0x44/0x1d0
? kmem_cache_alloc_trace+0x119/0x220
do_init_module+0x5c/0x270
__do_sys_init_module+0x12e/0x1b0
do_syscall_64+0x40/0x80
entry_SYSCALL_64_after_hwframe+0x44/0xae
Fix this bug by saving the address of the physical location
of the ibft; later the driver will use isa_bus_to_virt() to get
the correct virtual address.
N.B. On each reboot KASLR randomizes the virtual addresses so
assuming phys_to_virt before KASLR does its deed is incorrect.
Simplify the code by renaming find_ibft_region()
to reserve_ibft_region() and remove all the wrappers.
Signed-off-by: Maurizio Lombardi <mlombard(a)redhat.com>
Reviewed-by: Mike Rapoport <rppt(a)linux.ibm.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad(a)kernel.org>
</cut>
Results regressed to (for first_bad == 342f43af70dbc74f8629381998f92c060e1763a2)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_llvm:
-5
# build_abe qemu:
-2
# linux_n_obj:
19722
# First few build errors in logs:
from (for last_good == 62fb9874f5da54fdb243003b386128037319b219)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_llvm:
-5
# build_abe qemu:
-2
# linux_n_obj:
19795
# linux build successful:
all
Artifacts of last_good build: https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl…
Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl…
Build top page/logs: https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl…
Configuration details:
Reproduce builds:
<cut>
mkdir investigate-linux-342f43af70dbc74f8629381998f92c060e1763a2
cd investigate-linux-342f43af70dbc74f8629381998f92c060e1763a2
git clone https://git.linaro.org/toolchain/jenkins-scripts
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_kernel-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /linux/ ./ ./bisect/baseline/
cd linux
# Reproduce first_bad build
git checkout --detach 342f43af70dbc74f8629381998f92c060e1763a2
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach 62fb9874f5da54fdb243003b386128037319b219
../artifacts/test.sh
cd ..
</cut>
History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/…
Artifacts: https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl…
Build log: https://ci.linaro.org/job/tcwg_kernel-llvm-bisect-llvm-master-aarch64-mainl…
Full commit (up to 1000 lines):
<cut>
commit 342f43af70dbc74f8629381998f92c060e1763a2
Author: Maurizio Lombardi <mlombard(a)redhat.com>
Date: Thu Jul 29 15:52:50 2021 +0200
iscsi_ibft: fix crash due to KASLR physical memory remapping
Starting with commit a799c2bd29d1
("x86/setup: Consolidate early memory reservations")
memory reservations have been moved earlier during the boot process,
before the execution of the Kernel Address Space Layout Randomization code.
setup_arch() calls the iscsi_ibft's find_ibft_region() function
to find and reserve the memory dedicated to the iBFT and this function
also saves a virtual pointer to the iBFT table for later use.
The problem is that if KALSR is active, the physical memory gets
remapped somewhere else in the virtual address space and the pointer is
no longer valid, this will cause a kernel panic when the iscsi driver tries
to dereference it.
iBFT detected.
BUG: unable to handle page fault for address: ffff888000099fd8
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] SMP PTI
..snip..
Call Trace:
? ibft_create_kobject+0x1d2/0x1d2 [iscsi_ibft]
do_one_initcall+0x44/0x1d0
? kmem_cache_alloc_trace+0x119/0x220
do_init_module+0x5c/0x270
__do_sys_init_module+0x12e/0x1b0
do_syscall_64+0x40/0x80
entry_SYSCALL_64_after_hwframe+0x44/0xae
Fix this bug by saving the address of the physical location
of the ibft; later the driver will use isa_bus_to_virt() to get
the correct virtual address.
N.B. On each reboot KASLR randomizes the virtual addresses so
assuming phys_to_virt before KASLR does its deed is incorrect.
Simplify the code by renaming find_ibft_region()
to reserve_ibft_region() and remove all the wrappers.
Signed-off-by: Maurizio Lombardi <mlombard(a)redhat.com>
Reviewed-by: Mike Rapoport <rppt(a)linux.ibm.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad(a)kernel.org>
---
arch/x86/kernel/setup.c | 10 --------
drivers/firmware/iscsi_ibft.c | 10 +++++---
drivers/firmware/iscsi_ibft_find.c | 48 ++++++++++++++------------------------
include/linux/iscsi_ibft.h | 18 ++++++--------
4 files changed, 32 insertions(+), 54 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 1e720626069a..b6a62af06a9f 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -571,16 +571,6 @@ void __init reserve_standard_io_resources(void)
}
-static __init void reserve_ibft_region(void)
-{
- unsigned long addr, size = 0;
-
- addr = find_ibft_region(&size);
-
- if (size)
- memblock_reserve(addr, size);
-}
-
static bool __init snb_gfx_workaround_needed(void)
{
#ifdef CONFIG_PCI
diff --git a/drivers/firmware/iscsi_ibft.c b/drivers/firmware/iscsi_ibft.c
index 7127a04bca19..612a59e213df 100644
--- a/drivers/firmware/iscsi_ibft.c
+++ b/drivers/firmware/iscsi_ibft.c
@@ -84,8 +84,10 @@ MODULE_DESCRIPTION("sysfs interface to BIOS iBFT information");
MODULE_LICENSE("GPL");
MODULE_VERSION(IBFT_ISCSI_VERSION);
+static struct acpi_table_ibft *ibft_addr;
+
#ifndef CONFIG_ISCSI_IBFT_FIND
-struct acpi_table_ibft *ibft_addr;
+phys_addr_t ibft_phys_addr;
#endif
struct ibft_hdr {
@@ -858,11 +860,13 @@ static int __init ibft_init(void)
int rc = 0;
/*
- As on UEFI systems the setup_arch()/find_ibft_region()
+ As on UEFI systems the setup_arch()/reserve_ibft_region()
is called before ACPI tables are parsed and it only does
legacy finding.
*/
- if (!ibft_addr)
+ if (ibft_phys_addr)
+ ibft_addr = isa_bus_to_virt(ibft_phys_addr);
+ else
acpi_find_ibft_region();
if (ibft_addr) {
diff --git a/drivers/firmware/iscsi_ibft_find.c b/drivers/firmware/iscsi_ibft_find.c
index 64bb94523281..a0594590847d 100644
--- a/drivers/firmware/iscsi_ibft_find.c
+++ b/drivers/firmware/iscsi_ibft_find.c
@@ -31,8 +31,8 @@
/*
* Physical location of iSCSI Boot Format Table.
*/
-struct acpi_table_ibft *ibft_addr;
-EXPORT_SYMBOL_GPL(ibft_addr);
+phys_addr_t ibft_phys_addr;
+EXPORT_SYMBOL_GPL(ibft_phys_addr);
static const struct {
char *sign;
@@ -47,13 +47,24 @@ static const struct {
#define VGA_MEM 0xA0000 /* VGA buffer */
#define VGA_SIZE 0x20000 /* 128kB */
-static int __init find_ibft_in_mem(void)
+/*
+ * Routine used to find and reserve the iSCSI Boot Format Table
+ */
+void __init reserve_ibft_region(void)
{
unsigned long pos;
unsigned int len = 0;
void *virt;
int i;
+ ibft_phys_addr = 0;
+
+ /* iBFT 1.03 section 1.4.3.1 mandates that UEFI machines will
+ * only use ACPI for this
+ */
+ if (efi_enabled(EFI_BOOT))
+ return;
+
for (pos = IBFT_START; pos < IBFT_END; pos += 16) {
/* The table can't be inside the VGA BIOS reserved space,
* so skip that area */
@@ -70,35 +81,12 @@ static int __init find_ibft_in_mem(void)
/* if the length of the table extends past 1M,
* the table cannot be valid. */
if (pos + len <= (IBFT_END-1)) {
- ibft_addr = (struct acpi_table_ibft *)virt;
- pr_info("iBFT found at 0x%lx.\n", pos);
- goto done;
+ ibft_phys_addr = pos;
+ memblock_reserve(ibft_phys_addr, PAGE_ALIGN(len));
+ pr_info("iBFT found at 0x%lx.\n", ibft_phys_addr);
+ return;
}
}
}
}
-done:
- return len;
-}
-/*
- * Routine used to find the iSCSI Boot Format Table. The logical
- * kernel address is set in the ibft_addr global variable.
- */
-unsigned long __init find_ibft_region(unsigned long *sizep)
-{
- ibft_addr = NULL;
-
- /* iBFT 1.03 section 1.4.3.1 mandates that UEFI machines will
- * only use ACPI for this */
-
- if (!efi_enabled(EFI_BOOT))
- find_ibft_in_mem();
-
- if (ibft_addr) {
- *sizep = PAGE_ALIGN(ibft_addr->header.length);
- return (u64)virt_to_phys(ibft_addr);
- }
-
- *sizep = 0;
- return 0;
}
diff --git a/include/linux/iscsi_ibft.h b/include/linux/iscsi_ibft.h
index b7b45ca82bea..790e7fcfc1a6 100644
--- a/include/linux/iscsi_ibft.h
+++ b/include/linux/iscsi_ibft.h
@@ -13,26 +13,22 @@
#ifndef ISCSI_IBFT_H
#define ISCSI_IBFT_H
-#include <linux/acpi.h>
+#include <linux/types.h>
/*
- * Logical location of iSCSI Boot Format Table.
- * If the value is NULL there is no iBFT on the machine.
+ * Physical location of iSCSI Boot Format Table.
+ * If the value is 0 there is no iBFT on the machine.
*/
-extern struct acpi_table_ibft *ibft_addr;
+extern phys_addr_t ibft_phys_addr;
/*
* Routine used to find and reserve the iSCSI Boot Format Table. The
- * mapped address is set in the ibft_addr variable.
+ * physical address is set in the ibft_phys_addr variable.
*/
#ifdef CONFIG_ISCSI_IBFT_FIND
-unsigned long find_ibft_region(unsigned long *sizep);
+void reserve_ibft_region(void);
#else
-static inline unsigned long find_ibft_region(unsigned long *sizep)
-{
- *sizep = 0;
- return 0;
-}
+static inline void reserve_ibft_region(void) {}
#endif
#endif /* ISCSI_IBFT_H */
</cut>
Successfully identified regression in *llvm* in CI configuration tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3. So far, this commit has regressed CI configurations:
- tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3
Culprit:
<cut>
commit 4cd8dd3fe05e099792e1494dedd074eb5ba289b6
Author: Amy Kwan <amy.kwan1(a)ibm.com>
Date: Sun Aug 22 13:46:52 2021 -0500
[scudo][standalone] Link tests against libatomic if libatomic exists
It is possible that libatomic does not exist on some systems. This patch updates
the scudo standalone tests to link against libatomic if the library exists.
This is an update to the original patch: https://reviews.llvm.org/D64134 and
aims to resolve https://bugs.llvm.org/show_bug.cgi?id=51431.
Differential Revision: https://reviews.llvm.org/D108503
</cut>
Results regressed to (for first_bad == 4cd8dd3fe05e099792e1494dedd074eb5ba289b6)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O3 artifacts/build-4cd8dd3fe05e099792e1494dedd074eb5ba289b6/results_id:
1
# 447.dealII,dealII_base.default regressed by 103
from (for last_good == d8d84c9df82fc114f2b22a533a8183065ca1a2e0)
# reset_artifacts:
-10
# build_abe binutils:
-9
# build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
-8
# build_abe linux:
-7
# build_abe glibc:
-6
# build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
-5
# build_llvm true:
-3
# true:
0
# benchmark -- -O3 artifacts/build-d8d84c9df82fc114f2b22a533a8183065ca1a2e0/results_id:
1
Artifacts of last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of last_good: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O3/4515
Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Results ID of first_bad: tx1_64/tcwg_bmk_llvm_tx1/bisect-llvm-master-aarch64-spec2k6-O3/4510
Build top page/logs: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Configuration details:
Reproduce builds:
<cut>
mkdir investigate-llvm-4cd8dd3fe05e099792e1494dedd074eb5ba289b6
cd investigate-llvm-4cd8dd3fe05e099792e1494dedd074eb5ba289b6
git clone https://git.linaro.org/toolchain/jenkins-scripts
mkdir -p artifacts/manifests
curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-… --fail
chmod +x artifacts/test.sh
# Reproduce the baseline build (build all pre-requisites)
./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
# Save baseline build state (which is then restored in artifacts/test.sh)
mkdir -p ./bisect
rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /llvm/ ./ ./bisect/baseline/
cd llvm
# Reproduce first_bad build
git checkout --detach 4cd8dd3fe05e099792e1494dedd074eb5ba289b6
../artifacts/test.sh
# Reproduce last_good build
git checkout --detach d8d84c9df82fc114f2b22a533a8183065ca1a2e0
../artifacts/test.sh
cd ..
</cut>
History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/…
Artifacts: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Build log: https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-…
Full commit (up to 1000 lines):
<cut>
commit 4cd8dd3fe05e099792e1494dedd074eb5ba289b6
Author: Amy Kwan <amy.kwan1(a)ibm.com>
Date: Sun Aug 22 13:46:52 2021 -0500
[scudo][standalone] Link tests against libatomic if libatomic exists
It is possible that libatomic does not exist on some systems. This patch updates
the scudo standalone tests to link against libatomic if the library exists.
This is an update to the original patch: https://reviews.llvm.org/D64134 and
aims to resolve https://bugs.llvm.org/show_bug.cgi?id=51431.
Differential Revision: https://reviews.llvm.org/D108503
---
compiler-rt/lib/scudo/standalone/tests/CMakeLists.txt | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/compiler-rt/lib/scudo/standalone/tests/CMakeLists.txt b/compiler-rt/lib/scudo/standalone/tests/CMakeLists.txt
index f4186eba1688..eaa47a04a179 100644
--- a/compiler-rt/lib/scudo/standalone/tests/CMakeLists.txt
+++ b/compiler-rt/lib/scudo/standalone/tests/CMakeLists.txt
@@ -39,7 +39,10 @@ foreach(lib ${SANITIZER_TEST_CXX_LIBRARIES})
endforeach()
list(APPEND LINK_FLAGS -pthread)
# Linking against libatomic is required with some compilers
-list(APPEND LINK_FLAGS -latomic)
+check_library_exists(atomic __atomic_load_8 "" COMPILER_RT_HAS_LIBATOMIC)
+if (COMPILER_RT_HAS_LIBATOMIC)
+ list(APPEND LINK_FLAGS -latomic)
+endif()
set(SCUDO_TEST_HEADERS
scudo_unit_test.h
</cut>