Thanks for your report.
I think you’d need to add 0640_0214, 0640_0215, 0640_0223 and 0640_0224 to Fujitsu/Fortran/0640/fpconfig.cmake with TEST_SUITE_FP_TOLERANCE_FP32. Alternatively, applying -fno-reciprocal-math to these tests should also take care of the discrepancy in results.
All four print REAL*4 values but are currently graded against the directory default FP_TOLERANCE, which is derived from the FP64 constants, so 1.0e-14 with TEST_SUITE_FUJITSU_WITH_FAST_MATH=ON and 0x1.0p-51 without. Testing on aarch64 with the fast-math configuration, I see a single REAL*4 value change in each of them, from .9 to .90000003. That is one ULP of binary32 arising from reciprocal substitution on 9.0/10.0: the correctly rounded quotient is 0x3F666666, or 0.899999976, while 9.0f * (1.0f/10.0f) rounds to 0x3F666667, or 0.900000036. The relative difference is 6.6e-8, roughly six orders of magnitude above the tolerance being applied, so the comparison fails even though the result is within one ULP. Some of the outputs also show REAL*8 values shifting by one ULP, for instance .6 to .6000000000000001, but those are around 1.8e-16 and pass comfortably.
Since -freciprocal-math is implied by -ffast-math, which this configuration uses, the substitution is permitted and I don't think the new value should be treated as incorrect. TEST_SUITE_FP_TOLERANCE_FP32 is 2.38e-7, which covers the new test results.
It appears the tests have been passing just because the reciprocal substitution happened not to fire. My recent change to Flang enables the FIR loop invariant code motion pass by default. The pass perturbs the IR enough that the cost model's reciprocal decision flips, resulting in the slight difference to reference output.
Best, Caroline
From: ci_notify@linaro.org ci_notify@linaro.org Date: Thursday, September 3, 2026 at 5:25 PM To: contact-ci-compiler-test-suite@cs.jp.fujitsu.com contact-ci-compiler-test-suite@cs.jp.fujitsu.com Cc: Newcombe, Caroline caroline.newcombe@hpe.com; maxim.kuvyrkov@linaro.org maxim.kuvyrkov@linaro.org Subject: [Linaro-TCWG-CI] llvmorg-24-init-6625-gedbf358d0149: 4 regressions on aarch64
Dear contributor,
Our automatic CI has detected problems related to your patch(es). Please find some details below.
In tcwg_flang_test/main-aarch64-Ofast-sve_vla, after: | commit llvmorg-24-init-6625-gedbf358d0149 | Author: Caroline Newcombe caroline.newcombe@hpe.com | Date: Mon Aug 31 11:20:03 2026 -0400 | | [flang] Enable FIR loop invariant code motion by default (#218703) | | The FIR loop invariant code motion pass was added in #173438, where it | was disabled by default. This enables it for optimization levels above | `-O0`. The option `-disable-fir-licm` has been added in place of | ... 5 lines of the commit log omitted.
Produces 4 regressions: | | regressions.sum: | Running test-suite:Fujitsu/Fortran/0640 ... | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0214.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0215.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0223.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0224.test | # "FAIL" means : the execution of the compiled binary failed / output of the binary differs from the expected one
Used configuration : * Toolchain : cmake -G Ninja ../llvm/llvm "-DLLVM_ENABLE_PROJECTS=clang;lld;flang;clang-tools-extra" "-DLLVM_ENABLE_RUNTIMES=openmp" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=True -DCMAKE_INSTALL_PREFIX=../llvm-install "-DLLVM_TARGETS_TO_BUILD=AArch64" -DCLANG_DEFAULT_LINKER=lld * Testsuite : export LD_LIBRARY_PATH=$\WORKSPACE/llvm-install/lib/aarch64-unknown-linux-gnu${LD_LIBRARY_PATH:+:$\LD_LIBRARY_PATH} cmake -GNinja -DCMAKE_C_COMPILER="$\WORKSPACE/llvm-install/bin2/clang" -DCMAKE_CXX_COMPILER="$\WORKSPACE/llvm-install/bin2/clang++" -DCMAKE_Fortran_COMPILER="$\WORKSPACE/llvm-install/bin2/flang-new" -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= -DCMAKE_Fortran_FLAGS= -DCMAKE_C_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DCMAKE_CXX_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DCMAKE_Fortran_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DTEST_SUITE_FORTRAN=ON -DTEST_SUITE_SUBDIRS=Fujitsu -DTEST_SUITE_FUJITSU_WITH_FAST_MATH=ON "$\WORKSPACE/test/test-suite"
We track this bug report under https://urldefense.com/v3/__https://linaro.atlassian.net/browse/LLVM-2272__%... . (Require Linaro login) Please let us know if you have a fix.
If you have any questions regarding this report, please ask on linaro-toolchain@lists.linaro.org mailing list.
-----------------8<--------------------------8<--------------------------8<--------------------------
The information below contains the details of the failures, and the ways to reproduce a debug environment:
You can find the failure logs in *.log.1.xz files in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... The full lists of regressions and improvements as well as configure and make commands are in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... The list of [ignored] baseline and flaky failures are in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--...
Fujitsu testsuite : https://github.com/fujitsu/compiler-test-suite/
Current build : https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... Reference build : https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--...
Instruction to reproduce the build : https://urldefense.com/v3/__https://gitlab.com/LinaroLtd/tcwg/ci/interesting...
Full commit : https://github.com/llvm/llvm-project/commit/edbf358d014942a17953c3ccfcf9f206...
Thank you for your thorough investigation. As you pointed out, there was a problem with the tests. We will fix the tests.
The tests fail because the comparison uses a relative tolerance of 1.0e-14, which is too strict for single-precision floating-point calculations, causing valid FP32 results such as 0.90000003 to be rejected when compared with the reference value 0.9.
This issue is not due to actual computational errors caused by code generation or optimization, but rather a problem with the test configuration resulting from a double-precision tolerance being set for single-precision operations. The observed discrepancy is a rounding error of approximately 1 ULP, which falls within the normal range for single-precision operations. Therefore, it is appropriate to relax the comparison tolerance using TEST_SUITE_FP_TOLERANCE_FP32 (0x1.0p-22).
Please ignore the mail. We'll update the tests. We apologize for the false detection.
Best regards, Yasuyuki Ohno
Software Development Div., Advanced Technology Development Unit, Fujitsu Research, Fujitsu Limited, JAPAN.
________________________________ 差出人: Newcombe, Caroline caroline.newcombe@hpe.com 送信: 2026 年 9 月 5 日 (土曜日) 5:12 宛先: linaro-toolchain@lists.linaro.org linaro-toolchain@lists.linaro.org; contact-ci-compiler-test-suite@cs.jp.fujitsu.com contact-ci-compiler-test-suite@cs.jp.fujitsu.com Cc: maxim.kuvyrkov@linaro.org maxim.kuvyrkov@linaro.org 件名: Re: [Linaro-TCWG-CI] llvmorg-24-init-6625-gedbf358d0149: 4 regressions on aarch64
Thanks for your report.
I think you’d need to add 0640_0214, 0640_0215, 0640_0223 and 0640_0224 to Fujitsu/Fortran/0640/fpconfig.cmake with TEST_SUITE_FP_TOLERANCE_FP32. Alternatively, applying -fno-reciprocal-math to these tests should also take care of the discrepancy in results.
All four print REAL*4 values but are currently graded against the directory default FP_TOLERANCE, which is derived from the FP64 constants, so 1.0e-14 with TEST_SUITE_FUJITSU_WITH_FAST_MATH=ON and 0x1.0p-51 without. Testing on aarch64 with the fast-math configuration, I see a single REAL*4 value change in each of them, from .9 to .90000003. That is one ULP of binary32 arising from reciprocal substitution on 9.0/10.0: the correctly rounded quotient is 0x3F666666, or 0.899999976, while 9.0f * (1.0f/10.0f) rounds to 0x3F666667, or 0.900000036. The relative difference is 6.6e-8, roughly six orders of magnitude above the tolerance being applied, so the comparison fails even though the result is within one ULP. Some of the outputs also show REAL*8 values shifting by one ULP, for instance .6 to .6000000000000001, but those are around 1.8e-16 and pass comfortably.
Since -freciprocal-math is implied by -ffast-math, which this configuration uses, the substitution is permitted and I don't think the new value should be treated as incorrect. TEST_SUITE_FP_TOLERANCE_FP32 is 2.38e-7, which covers the new test results.
It appears the tests have been passing just because the reciprocal substitution happened not to fire. My recent change to Flang enables the FIR loop invariant code motion pass by default. The pass perturbs the IR enough that the cost model's reciprocal decision flips, resulting in the slight difference to reference output.
Best, Caroline
From: ci_notify@linaro.org ci_notify@linaro.org Date: Thursday, September 3, 2026 at 5:25 PM To: contact-ci-compiler-test-suite@cs.jp.fujitsu.com contact-ci-compiler-test-suite@cs.jp.fujitsu.com Cc: Newcombe, Caroline caroline.newcombe@hpe.com; maxim.kuvyrkov@linaro.org maxim.kuvyrkov@linaro.org Subject: [Linaro-TCWG-CI] llvmorg-24-init-6625-gedbf358d0149: 4 regressions on aarch64
Dear contributor,
Our automatic CI has detected problems related to your patch(es). Please find some details below.
In tcwg_flang_test/main-aarch64-Ofast-sve_vla, after: | commit llvmorg-24-init-6625-gedbf358d0149 | Author: Caroline Newcombe caroline.newcombe@hpe.com | Date: Mon Aug 31 11:20:03 2026 -0400 | | [flang] Enable FIR loop invariant code motion by default (#218703) | | The FIR loop invariant code motion pass was added in #173438, where it | was disabled by default. This enables it for optimization levels above | `-O0`. The option `-disable-fir-licm` has been added in place of | ... 5 lines of the commit log omitted.
Produces 4 regressions: | | regressions.sum: | Running test-suite:Fujitsu/Fortran/0640 ... | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0214.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0215.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0223.test | FAIL: test-suite :: Fujitsu/Fortran/0640/Fujitsu-Fortran-0640_0224.test | # "FAIL" means : the execution of the compiled binary failed / output of the binary differs from the expected one
Used configuration : * Toolchain : cmake -G Ninja ../llvm/llvm "-DLLVM_ENABLE_PROJECTS=clang;lld;flang;clang-tools-extra" "-DLLVM_ENABLE_RUNTIMES=openmp" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=True -DCMAKE_INSTALL_PREFIX=../llvm-install "-DLLVM_TARGETS_TO_BUILD=AArch64" -DCLANG_DEFAULT_LINKER=lld * Testsuite : export LD_LIBRARY_PATH=$\WORKSPACE/llvm-install/lib/aarch64-unknown-linux-gnu${LD_LIBRARY_PATH:+:$\LD_LIBRARY_PATH} cmake -GNinja -DCMAKE_C_COMPILER="$\WORKSPACE/llvm-install/bin2/clang" -DCMAKE_CXX_COMPILER="$\WORKSPACE/llvm-install/bin2/clang++" -DCMAKE_Fortran_COMPILER="$\WORKSPACE/llvm-install/bin2/flang-new" -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= -DCMAKE_Fortran_FLAGS= -DCMAKE_C_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DCMAKE_CXX_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DCMAKE_Fortran_FLAGS_RELEASE="-O3 -ffast-math -march=armv8.4-a+sve -msve-vector-bits=scalable -mllvm -scalable-vectorization=preferred -DNDEBUG" -DTEST_SUITE_FORTRAN=ON -DTEST_SUITE_SUBDIRS=Fujitsu -DTEST_SUITE_FUJITSU_WITH_FAST_MATH=ON "$\WORKSPACE/test/test-suite"
We track this bug report under https://urldefense.com/v3/__https://linaro.atlassian.net/browse/LLVM-2272__%... . (Require Linaro login) Please let us know if you have a fix.
If you have any questions regarding this report, please ask on linaro-toolchain@lists.linaro.org mailing list.
-----------------8<--------------------------8<--------------------------8<--------------------------
The information below contains the details of the failures, and the ways to reproduce a debug environment:
You can find the failure logs in *.log.1.xz files in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... The full lists of regressions and improvements as well as configure and make commands are in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... The list of [ignored] baseline and flaky failures are in * https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--...
Fujitsu testsuite : https://github.com/fujitsu/compiler-test-suite/
Current build : https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--... Reference build : https://urldefense.com/v3/__http://54.172.246.49:9090/jobs/tcwg_flang_test--...
Instruction to reproduce the build : https://urldefense.com/v3/__https://gitlab.com/LinaroLtd/tcwg/ci/interesting...
Full commit : https://github.com/llvm/llvm-project/commit/edbf358d014942a17953c3ccfcf9f206...
linaro-toolchain@lists.linaro.org
-
Newcombe, Caroline -
Yasuyuki Ohno (Fujitsu)