Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
Hello Tamar,
It looks like this failure happens because qemu-aarch64 segfaults when trying to run the testcase binary:
Running on qemu: timeout -k 30s 330s setarch x86_64 -R qemu-aarch64 -cpu any -R 0 -L /home/tcwg-buildslave/workspace/tcwg_gnu_1/abe/builds/destdir/x86_64-pc-linux-gnu/aarch64-linux-gnu/libc ./mask_struct_load_3_run.exe spawn [open ...] timeout: the monitored command dumped core FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test
Which would be a QEMU bug. I'm working on reproducing it and will report there if appropriate.
However, our CI also found some size and performance regressions with this commit, as detailed in https://linaro.atlassian.net/browse/GNU-1040:
- tcwg_bmk-code_size-cpu2017fast - gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r - gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r
- tcwg_bmk-code_size-spec2k6 - gnu-aarch64-master-O2: grew in size by 2% - 447.dealII - gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer
- tcwg_bmk-code_speed-cpu2017rate - gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r - gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r
- tcwg_bmk-fujitsu_speed-cpu2017speed - gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.] exp@@GLIBC_2.29
The link above has the full results.
ci_notify@linaro.org writes:
Dear contributor, our automatic CI has detected problems related to your patch(es). Please find some details below. If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or ping your favourite Linaro toolchain developer on the usual project channel.
In master-aarch64 after:
| commit gcc-14-5673-g33c2b70dbab | Author: Tamar Christina tamar.christina@arm.com | Date: Tue Nov 21 13:20:39 2023 +0000 | | AArch64: Add new generic-armv8-a CPU and make it the default. | | This patch adds a new generic scheduling model "generic-armv8-a" and makes it | the default for all Armv8 architectures. | | -mcpu=generic and -mtune=generic is kept around for those that really want the | previous cost model. | ... 34 lines of the commit log omitted.
FAIL: 1 regressions: 8 progressions
regressions.sum: === gcc tests ===
Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test
=== Results Summary ===progressions.sum: === gcc tests ===
Running gcc:gcc.dg/vect/vect.exp ... FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects (test for excess errors) UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed to produce executable UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects compilation failed to produce executable FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to produce executable UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects compilation failed to produce executable FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) ... and 3 more entries
You can find the failure logs in *.log.1.xz files in
The full lists of regressions and progressions are in
The list of [ignored] baseline and flaky failures are in
The configuration of this build is: CI config tcwg_gnu_cross_check_gcc master-aarch64
-----------------8<--------------------------8<--------------------------8<-------------------------- The information below can be used to reproduce a debug environment:
Current build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/110... Reference build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/110...
Reproduce last good and first bad builds: https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha...
Full commit : https://github.com/gcc-mirror/gcc/commit/33c2b70dbabc02788caabcbc66b7baeafeb...
Latest bug report status : https://linaro.atlassian.net/browse/GNU-1040
List of configurations that regressed due to this commit :
- tcwg_gnu_cross_check_gcc
** master-aarch64 *** FAIL: 1 regressions: 8 progressions *** https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha... *** https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/110...
Hello Thaigo,
-----Original Message----- From: Thiago Jung Bauermann thiago.bauermann@linaro.org Sent: Saturday, December 2, 2023 12:15 AM To: Tamar Christina Tamar.Christina@arm.com Cc: linaro-toolchain@lists.linaro.org; gcc-regression@gcc.gnu.org; tcwg- validation@linaro.org Subject: Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
Hello Tamar,
It looks like this failure happens because qemu-aarch64 segfaults when trying to run the testcase binary:
Running on qemu: timeout -k 30s 330s setarch x86_64 -R qemu-aarch64 -cpu any -R 0 -L /home/tcwg- buildslave/workspace/tcwg_gnu_1/abe/builds/destdir/x86_64-pc-linux- gnu/aarch64-linux-gnu/libc ./mask_struct_load_3_run.exe spawn [open ...] timeout: the monitored command dumped core FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test
Which would be a QEMU bug. I'm working on reproducing it and will report there if appropriate.
However, our CI also found some size and performance regressions with this commit, as detailed in https://linaro.atlassian.net/browse/GNU-1040:
tcwg_bmk-code_size-cpu2017fast
- gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r
- gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r
tcwg_bmk-code_size-spec2k6
- gnu-aarch64-master-O2: grew in size by 2% - 447.dealII
- gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer
Size gains are expected. The cost model turns on throughput based optimizations. Since will grow because among other things it unrolls.
- tcwg_bmk-code_speed-cpu2017rate
- gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r
- gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r
As the commit message says, there are outstanding costing issues that will be resolved at stage 4 when the remaining infrastructure are in place.
- tcwg_bmk-fujitsu_speed-cpu2017speed
- gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.]
exp@@GLIBC_2.29
This is suspect, it says the slowdown is in exp in glibc, that would be unrelated to my patch.
Tamar
The link above has the full results.
ci_notify@linaro.org writes:
Dear contributor, our automatic CI has detected problems related to your
patch(es). Please
find some details below. If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or
ping
your favourite Linaro toolchain developer on the usual project channel.
In master-aarch64 after:
| commit gcc-14-5673-g33c2b70dbab | Author: Tamar Christina tamar.christina@arm.com | Date: Tue Nov 21 13:20:39 2023 +0000 | | AArch64: Add new generic-armv8-a CPU and make it the default. | | This patch adds a new generic scheduling model "generic-armv8-a" and
makes it
| the default for all Armv8 architectures. | | -mcpu=generic and -mtune=generic is kept around for those that really want
the
| previous cost model. | ... 34 lines of the commit log omitted.
FAIL: 1 regressions: 8 progressions
regressions.sum: === gcc tests ===
Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test
=== Results Summary ===progressions.sum: === gcc tests ===
Running gcc:gcc.dg/vect/vect.exp ... FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed
to produce executable
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-
objects compilation failed to produce executable
FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to
produce executable
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects
compilation failed to produce executable
FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) ... and 3 more entries
You can find the failure logs in *.log.1.xz files in
build/1102/artifact/artifacts/00-sumfiles/ .
The full lists of regressions and progressions are in
build/1102/artifact/artifacts/notify/ .
The list of [ignored] baseline and flaky failures are in
build/1102/artifact/artifacts/sumfiles/xfails.xfail .
The configuration of this build is: CI config tcwg_gnu_cross_check_gcc master-aarch64
-----------------8<--------------------------8<--------------------------8<-------------------
The information below can be used to reproduce a debug environment:
Current build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-
aarch64-build/1102/artifact/artifacts
Reference build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-
aarch64-build/1101/artifact/artifacts
Reproduce last good and first bad builds: https://git-
us.linaro.org/toolchain/ci/interesting- commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw g_gnu_cross_check_gcc/master-aarch64/reproduction_instructions.txt
Full commit : https://github.com/gcc-
mirror/gcc/commit/33c2b70dbabc02788caabcbc66b7baeafeb95bcf
Latest bug report status : https://linaro.atlassian.net/browse/GNU-1040
List of configurations that regressed due to this commit :
- tcwg_gnu_cross_check_gcc
** master-aarch64 *** FAIL: 1 regressions: 8 progressions *** https://git-us.linaro.org/toolchain/ci/interesting-
commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw g_gnu_cross_check_gcc/master-aarch64/details.txt
*** https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-
build/1102/
-- Thiago
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hello Tamar,
Thank you for your clarifications. They're very helpful.
Tamar Christina Tamar.Christina@arm.com writes:
tcwg_bmk-code_size-cpu2017fast
- gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r
- gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r
tcwg_bmk-code_size-spec2k6
- gnu-aarch64-master-O2: grew in size by 2% - 447.dealII
- gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer
Size gains are expected. The cost model turns on throughput based optimizations. Since will grow because among other things it unrolls.
Ok, that is good to know.
- tcwg_bmk-code_speed-cpu2017rate
- gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r
- gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r
As the commit message says, there are outstanding costing issues that will be resolved at stage 4 when the remaining infrastructure are in place.
Nice.
- tcwg_bmk-fujitsu_speed-cpu2017speed
- gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.]
exp@@GLIBC_2.29
This is suspect, it says the slowdown is in exp in glibc, that would be unrelated to my patch.
Indeed, thanks for pointing out.
On Dec 4, 2023, at 12:25, Tamar Christina Tamar.Christina@arm.com wrote:
...
- tcwg_bmk-fujitsu_speed-cpu2017speed
- gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.]
exp@@GLIBC_2.29
This is suspect, it says the slowdown is in exp in glibc, that would be unrelated to my patch.
Hi Tamar,
Linaro benchmarking builds the whole sysroot with the "new" compiler, including glibc. It may be interesting to double-check code-gen differences on glibc's exp() and make sure they are no obvious bad choices.
-- Maxim Kuvyrkov https://www.linaro.org
Tamar
The link above has the full results.
ci_notify@linaro.org writes:
Dear contributor, our automatic CI has detected problems related to your
patch(es). Please
find some details below. If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or
ping
your favourite Linaro toolchain developer on the usual project channel.
In master-aarch64 after:
| commit gcc-14-5673-g33c2b70dbab | Author: Tamar Christina tamar.christina@arm.com | Date: Tue Nov 21 13:20:39 2023 +0000 | | AArch64: Add new generic-armv8-a CPU and make it the default. | | This patch adds a new generic scheduling model "generic-armv8-a" and
makes it
| the default for all Armv8 architectures. | | -mcpu=generic and -mtune=generic is kept around for those that really want
the
| previous cost model. | ... 34 lines of the commit log omitted.
FAIL: 1 regressions: 8 progressions
regressions.sum: === gcc tests ===
Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test
=== Results Summary ===progressions.sum: === gcc tests ===
Running gcc:gcc.dg/vect/vect.exp ... FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects (test
for excess errors)
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed
to produce executable
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-
objects compilation failed to produce executable
FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to
produce executable
UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects
compilation failed to produce executable
FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) ... and 3 more entries
You can find the failure logs in *.log.1.xz files in
build/1102/artifact/artifacts/00-sumfiles/ .
The full lists of regressions and progressions are in
build/1102/artifact/artifacts/notify/ .
The list of [ignored] baseline and flaky failures are in
build/1102/artifact/artifacts/sumfiles/xfails.xfail .
The configuration of this build is: CI config tcwg_gnu_cross_check_gcc master-aarch64
-----------------8<--------------------------8<--------------------------8<-------------------
The information below can be used to reproduce a debug environment:
Current build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-
aarch64-build/1102/artifact/artifacts
Reference build : https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-
aarch64-build/1101/artifact/artifacts
Reproduce last good and first bad builds: https://git-
us.linaro.org/toolchain/ci/interesting- commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw g_gnu_cross_check_gcc/master-aarch64/reproduction_instructions.txt
Full commit : https://github.com/gcc-
mirror/gcc/commit/33c2b70dbabc02788caabcbc66b7baeafeb95bcf
Latest bug report status : https://linaro.atlassian.net/browse/GNU-1040
List of configurations that regressed due to this commit :
- tcwg_gnu_cross_check_gcc
** master-aarch64 *** FAIL: 1 regressions: 8 progressions *** https://git-us.linaro.org/toolchain/ci/interesting-
commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw g_gnu_cross_check_gcc/master-aarch64/details.txt
*** https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-
build/1102/
-- Thiago
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org To unsubscribe send an email to linaro-toolchain-leave@lists.linaro.org
linaro-toolchain@lists.linaro.org
-
Maxim Kuvyrkov -
Tamar Christina -
Thiago Jung Bauermann