I think all of these actually fall under
"I suspect there are still some further issues to address here, since the framework does not correctly test that the multilibs and startup code enable alternative format; but this is still an improvement over what we had before."
All the failures are execution test failures due to the fact that we don't check the available hardware/multilibs for running the test; so blindly adding options and then running the test is incorrect. But we currently lack such a test in the framework.
It's also less than clear exactly what these tests are checking and which part of what they are checking that really requires the options they add. I suspect that they previously passed only by accident (they didn't really add enough flags to enable what they author thought they were checking).
R.
On 10/02/2024 02:43, ci_notify@linaro.org wrote:
Dear contributor, our automatic CI has detected problems related to your patch(es). Please find some details below. If you have any questions, please follow up on linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg channel, or ping your favourite Linaro toolchain developer on the usual project channel.
We appreciate that it might be difficult to find the necessary logs or reproduce the issue locally. If you can't get what you need from our CI within minutes, let us know and we will be happy to help.
We track this report status in https://linaro.atlassian.net/browse/GNU-1149 https://linaro.atlassian.net/browse/GNU-1149 , please let us know if you are looking at the problem and/or when you have a fix.
In arm-eabi cortex-m33 hard after:
| commit gcc-14-8887-gd9459129ea8 | Author: Richard Earnshaw rearnsha@arm.com | Date: Mon Feb 5 17:16:45 2024 +0000 | | arm: testsuite: fix issues relating to fp16 alternative testing | | The v*_fp16_xN_1.c tests on Arm have been unstable since they were | added. This is not a problem with the tests themselves, or even the | patches that were added, but with the testsuite infrastructure. It | turned out that another set of dg- tests for fp16 were corrupting the | cached set of options used by the new tests, leading to running the | ... 45 lines of the commit log omitted.
FAIL: 29 regressions
regressions.sum: === g++ tests ===
Running g++:g++.dg/dg.exp ... FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-3.C -std=c++14 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-3.C -std=c++17 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-3.C -std=c++20 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-3.C -std=c++98 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-4.C -std=gnu++14 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-4.C -std=gnu++17 execution test FAIL: g++.dg/ext/arm-fp16/arm-fp16-ops-4.C -std=gnu++20 execution test ... and 26 more entries
You can find the failure logs in *.log.1.xz files in - https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/363/artifact/artifacts/00-sumfiles/ The full lists of regressions and progressions as well as configure and make commands are in - https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/363/artifact/artifacts/notify/ The list of [ignored] baseline and flaky failures are in - https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/363/artifact/artifacts/sumfiles/xfails.xfail
The configuration of this build is: CI config tcwg_gnu_embed_check_gcc arm-eabi -mthumb -march=armv8-m.main+dsp+fp -mtune=cortex-m33 -mfloat-abi=hard -mfpu=auto
-----------------8<--------------------------8<--------------------------8<-------------------------- The information below can be used to reproduce a debug environment:
Current build : https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/363/artifact/artifacts Reference build : https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/362/artifact/artifacts
Reproduce last good and first bad builds: https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha... https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha1/d9459129ea8f8c3cbd6150b90e842decba7952a3/tcwg_gnu_embed_check_gcc/master-thumb_m33_eabi/reproduction_instructions.txt
Full commit : https://github.com/gcc-mirror/gcc/commit/d9459129ea8f8c3cbd6150b90e842decba7... https://github.com/gcc-mirror/gcc/commit/d9459129ea8f8c3cbd6150b90e842decba7952a3
List of configurations that regressed due to this commit :
- tcwg_gnu_embed_check_gcc
** master-thumb_m33_eabi *** FAIL: 29 regressions
https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha... https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha1/d9459129ea8f8c3cbd6150b90e842decba7952a3/tcwg_gnu_embed_check_gcc/master-thumb_m33_eabi/details.txt
https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-bu... https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m33_eabi-build/363/artifact/artifacts