Successfully identified regression in *gcc* in CI configuration tcwg_bmk_gnu_eabi_stm32/gnu_eabi-release-arm_eabi-coremark-Os_LTO. So far, this commit has regressed CI configurations: - tcwg_bmk_gnu_eabi_stm32/gnu_eabi-release-arm_eabi-coremark-Os_LTO

Culprit: <cut> commit e9cc5cc589ca3fac2738c9def311e6079172f79f Author: Martin Liska mliska@suse.cz Date: Wed Apr 14 15:39:10 2021 +0200

Sync gcc-changelog scripts.

contrib/ChangeLog:

* gcc-changelog/git_commit.py: Sync with master. * gcc-changelog/git_email.py: Likewise. * gcc-changelog/git_update_version.py: Likewise. * gcc-changelog/setup.cfg: Likewise. * gcc-changelog/test_email.py: Likewise. * gcc-changelog/test_patches.txt: Likewise. </cut>

Results regressed to (for first_bad == e9cc5cc589ca3fac2738c9def311e6079172f79f) # reset_artifacts: -10 # build_abe binutils: -9 # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard: -8 # build_abe newlib: -6 # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard: -5 # true: 0 # benchmark -Os_LTO_mthumb -- artifacts/build-e9cc5cc589ca3fac2738c9def311e6079172f79f/results_id: 1

from (for last_good == 062b65404f0d139d683411d88e46f972cdd3b372) # reset_artifacts: -10 # build_abe binutils: -9 # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard: -8 # build_abe newlib: -6 # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard: -5 # true: 0 # benchmark -Os_LTO_mthumb -- artifacts/build-baseline/results_id: 1

Artifacts of last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... Results ID of last_good: stm32_STM32L476RGTx/tcwg_bmk_gnu_eabi_stm32/baseline-gnu_eabi-release-arm_eabi-coremark-Os_LTO/2198 Artifacts of first_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... Results ID of first_bad: stm32_STM32L476RGTx/tcwg_bmk_gnu_eabi_stm32/bisect-gnu_eabi-release-arm_eabi-coremark-Os_LTO/2212 Build top page/logs: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab...

Configuration details:

Reproduce builds: <cut> mkdir investigate-gcc-e9cc5cc589ca3fac2738c9def311e6079172f79f cd investigate-gcc-e9cc5cc589ca3fac2738c9def311e6079172f79f

git clone https://git.linaro.org/toolchain/jenkins-scripts

mkdir -p artifacts/manifests curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... --fail curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... --fail curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... --fail chmod +x artifacts/test.sh

# Reproduce the baseline build (build all pre-requisites) ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh

# Save baseline build state (which is then restored in artifacts/test.sh) mkdir -p ./bisect rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /gcc/ ./ ./bisect/baseline/

cd gcc

# Reproduce first_bad build git checkout --detach e9cc5cc589ca3fac2738c9def311e6079172f79f ../artifacts/test.sh

# Reproduce last_good build git checkout --detach 062b65404f0d139d683411d88e46f972cdd3b372 ../artifacts/test.sh

cd .. </cut>

History of pending regressions and results: https://git.linaro.org/toolchain/ci/base-artifacts.git/log/?h=linaro-local/c...

Artifacts: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab... Build log: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eab...

Full commit (up to 1000 lines): <cut> commit e9cc5cc589ca3fac2738c9def311e6079172f79f Author: Martin Liska mliska@suse.cz Date: Wed Apr 14 15:39:10 2021 +0200

Sync gcc-changelog scripts.

contrib/ChangeLog:

* gcc-changelog/git_commit.py: Sync with master. * gcc-changelog/git_email.py: Likewise. * gcc-changelog/git_update_version.py: Likewise. * gcc-changelog/setup.cfg: Likewise. * gcc-changelog/test_email.py: Likewise. * gcc-changelog/test_patches.txt: Likewise. --- contrib/gcc-changelog/git_commit.py | 51 ++++++++-- contrib/gcc-changelog/git_email.py | 2 +- contrib/gcc-changelog/git_update_version.py | 4 + contrib/gcc-changelog/setup.cfg | 3 + contrib/gcc-changelog/test_email.py | 20 ++++ contrib/gcc-changelog/test_patches.txt | 147 ++++++++++++++++++++++++++++ 6 files changed, 219 insertions(+), 8 deletions(-)

diff --git a/contrib/gcc-changelog/git_commit.py b/contrib/gcc-changelog/git_commit.py index ee1973371be..b28f7deac23 100755 --- a/contrib/gcc-changelog/git_commit.py +++ b/contrib/gcc-changelog/git_commit.py @@ -214,6 +214,7 @@ class ChangeLogEntry: self.lines = [] self.files = [] self.file_patterns = [] + self.opened_parentheses = 0

def parse_file_names(self): # Whether the content currently processed is between a star prefix the @@ -223,8 +224,14 @@ class ChangeLogEntry: for line in self.lines: # If this line matches the star prefix, start the location # processing on the information that follows the star. + # Note that we need to skip macro names that can be in form of: + # + # * config/i386/i386.md (*fix_trunc<mode>_i387_1, + # *add<mode>3_ne, *add<mode>3_eq_0, *add<mode>3_ne_0, + # *fist<mode>2_<rounding>_1, *<code><mode>3_1): + # m = star_prefix_regex.match(line) - if m: + if m and len(m.group('spaces')) == 1: in_location = True line = m.group('content')

@@ -307,8 +314,9 @@ class GitCommit: if self.revert_commit: self.info = self.commit_to_info_hook(self.revert_commit)

+ # Allow complete deletion of ChangeLog files in a commit project_files = [f for f in self.info.modified_files - if self.is_changelog_filename(f[0]) + if (self.is_changelog_filename(f[0], allow_suffix=True) and f[1] != 'D') or f[0] in misc_files] ignored_files = [f for f in self.info.modified_files if self.in_ignored_location(f[0])] @@ -328,6 +336,7 @@ class GitCommit: self.parse_changelog() self.parse_file_names() self.check_for_empty_description() + self.check_for_broken_parentheses() self.deduce_changelog_locations() self.check_file_patterns() if not self.errors: @@ -343,8 +352,14 @@ class GitCommit: return [x[0] for x in self.info.modified_files if x[1] == 'A']

@classmethod - def is_changelog_filename(cls, path): - return path.endswith('/ChangeLog') or path == 'ChangeLog' + def is_changelog_filename(cls, path, allow_suffix=False): + basename = os.path.basename(path) + if basename == 'ChangeLog': + return True + elif allow_suffix and basename.startswith('ChangeLog'): + return True + else: + return False

@classmethod def find_changelog_location(cls, name): @@ -400,8 +415,10 @@ class GitCommit: if line != line.rstrip(): self.errors.append(Error('trailing whitespace', line)) if len(line.replace('\t', ' ' * TAB_WIDTH)) > LINE_LIMIT: - self.errors.append(Error('line exceeds %d character limit' - % LINE_LIMIT, line)) + # support long filenames + if not line.startswith('\t* ') or not line.endswith(':') or ' ' in line[3:-1]: + self.errors.append(Error('line exceeds %d character limit' + % LINE_LIMIT, line)) m = changelog_regex.match(line) if m: last_entry = ChangeLogEntry(m.group(1).rstrip('/'), @@ -490,7 +507,8 @@ class GitCommit: else: m = star_prefix_regex.match(line) if m: - if len(m.group('spaces')) != 1: + if (len(m.group('spaces')) != 1 and + last_entry.opened_parentheses == 0): msg = 'one space should follow asterisk' self.errors.append(Error(msg, line)) else: @@ -502,6 +520,7 @@ class GitCommit: msg = f'empty group "{needle}" found' self.errors.append(Error(msg, line)) last_entry.lines.append(line) + self.process_parentheses(last_entry, line) else: if last_entry.is_empty: msg = 'first line should start with a tab, ' \ @@ -509,6 +528,18 @@ class GitCommit: self.errors.append(Error(msg, line)) else: last_entry.lines.append(line) + self.process_parentheses(last_entry, line) + + def process_parentheses(self, last_entry, line): + for c in line: + if c == '(': + last_entry.opened_parentheses += 1 + elif c == ')': + if last_entry.opened_parentheses == 0: + msg = 'bad wrapping of parenthesis' + self.errors.append(Error(msg, line)) + else: + last_entry.opened_parentheses -= 1

def parse_file_names(self): for entry in self.changelog_entries: @@ -532,6 +563,12 @@ class GitCommit: msg = 'missing description of a change' self.errors.append(Error(msg, line))

+ def check_for_broken_parentheses(self): + for entry in self.changelog_entries: + if entry.opened_parentheses != 0: + msg = 'bad parentheses wrapping' + self.errors.append(Error(msg, entry.lines[0])) + def get_file_changelog_location(self, changelog_file): for file in self.info.modified_files: if file[0] == changelog_file: diff --git a/contrib/gcc-changelog/git_email.py b/contrib/gcc-changelog/git_email.py index 00ad00458f4..b0547b363aa 100755 --- a/contrib/gcc-changelog/git_email.py +++ b/contrib/gcc-changelog/git_email.py @@ -66,7 +66,7 @@ class GitEmail(GitCommit): t = 'A' else: t = 'M' - modified_files.append((target, t)) + modified_files.append((target if t != 'D' else source, t)) git_info = GitInfo(None, date, author, body, modified_files) super().__init__(git_info, strict=strict, commit_to_info_hook=lambda x: None) diff --git a/contrib/gcc-changelog/git_update_version.py b/contrib/gcc-changelog/git_update_version.py index d2cadb8811c..1e2b22b008b 100755 --- a/contrib/gcc-changelog/git_update_version.py +++ b/contrib/gcc-changelog/git_update_version.py @@ -26,6 +26,9 @@ from git_repository import parse_git_revisions

current_timestamp = datetime.datetime.now().strftime('%Y%m%d\n')

+# Skip the following commits, they cannot be correctly processed +IGNORED_COMMITS = ('c2be82058fb40f3ae891c68d185ff53e07f14f45') +

def read_timestamp(path): with open(path) as f: @@ -98,6 +101,7 @@ def update_current_branch(): head = head.parents[1] commits = parse_git_revisions(args.git_path, '%s..%s' % (commit.hexsha, head.hexsha)) + commits = [c for c in commits if c.info.hexsha not in IGNORED_COMMITS] for git_commit in reversed(commits): prepend_to_changelog_files(repo, args.git_path, git_commit, not args.dry_mode) diff --git a/contrib/gcc-changelog/setup.cfg b/contrib/gcc-changelog/setup.cfg index 9e4a0f6479c..efc313f6d52 100644 --- a/contrib/gcc-changelog/setup.cfg +++ b/contrib/gcc-changelog/setup.cfg @@ -1,2 +1,5 @@ +[flake8] +max-line-length = 120 + [tool:pytest] addopts = -vv --flake8 diff --git a/contrib/gcc-changelog/test_email.py b/contrib/gcc-changelog/test_email.py index 5db56caef9e..8abf5c37487 100755 --- a/contrib/gcc-changelog/test_email.py +++ b/contrib/gcc-changelog/test_email.py @@ -404,3 +404,23 @@ class TestGccChangelog(unittest.TestCase): email = self.from_patch_glob('0001-Add-horse2.patch') assert not email.errors assert email.changelog_entries[0].files == ['koníček.txt'] + + def test_modification_of_old_changelog(self): + email = self.from_patch_glob('0001-fix-old-ChangeLog.patch') + assert not email.errors + + def test_multiline_parentheses(self): + email = self.from_patch_glob('0001-Add-macro.patch') + assert not email.errors + + def test_multiline_bad_parentheses(self): + email = self.from_patch_glob('0002-Wrong-macro-changelog.patch') + assert email.errors[0].message == 'bad parentheses wrapping' + + def test_changelog_removal(self): + email = self.from_patch_glob('0001-ChangeLog-removal.patch', strict=True) + assert not email.errors + + def test_long_filenames(self): + email = self.from_patch_glob('0001-long-filenames') + assert not email.errors diff --git a/contrib/gcc-changelog/test_patches.txt b/contrib/gcc-changelog/test_patches.txt index ffd13682d5c..3f9806dc076 100644 --- a/contrib/gcc-changelog/test_patches.txt +++ b/contrib/gcc-changelog/test_patches.txt @@ -3398,4 +3398,151 @@ index 00000000000..56c67f58752 -- 2.29.2

+=== 0001-fix-old-ChangeLog.patch === +From fd498465b2801203089616be9a0e3c1f4fc065a0 Mon Sep 17 00:00:00 2001 +From: Martin Liska mliska@suse.cz +Date: Wed, 13 Jan 2021 11:45:37 +0100 +Subject: [PATCH] Fix a changelog. + +--- + gcc/ChangeLog-2020 | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/gcc/ChangeLog-2020 b/gcc/ChangeLog-2020 +index 6553720acad..2c170ef014a 100644 +--- a/gcc/ChangeLog-2020 ++++ b/gcc/ChangeLog-2020 +@@ -1 +1,2 @@ + ++ + +-- +2.29.2 +=== 0001-Add-macro.patch === +From 9b7eedc932fe594547fb060b36dfd9e4178c4f9b Mon Sep 17 00:00:00 2001 +From: Martin Liska mliska@suse.cz +Date: Wed, 13 Jan 2021 16:26:45 +0100 +Subject: [PATCH 1/2] Add macro. + +gcc/ChangeLog: + + * config/i386/i386.md (*fix_trunc<mode>_i387_1, *add<mode>3_eq, + *add<mode>3_ne, *add<mode>3_eq_0, *add<mode>3_ne_0, *add<mode>3_eq, + *fist<mode>2_<rounding>_1, *<code><mode>3_1, *<code>di3_doubleword): + Use ix86_pre_reload_split instead of can_create_pseudo_p in condition. + * config/i386/sse.md + (*fix_trunc<mode>_i387_1, *add<mode>3_eq, + *add<mode>3_ne, *add<mode>3_eq_0, *add<mode>3_ne_0, *add<mode>3_eq, + *fist<mode>2_<rounding>_1): This should also work. +--- + gcc/config/i386/i386.md | 1 + + gcc/config/i386/sse.md | 1 + + 2 files changed, 2 insertions(+) + +diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md +index b60784a2908..ac63591b33f 100644 +--- a/gcc/config/i386/i386.md ++++ b/gcc/config/i386/i386.md +@@ -1 +1,2 @@ + ++ + +diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md +index 7f03fc491c3..0e17997db26 100644 +--- a/gcc/config/i386/sse.md ++++ b/gcc/config/i386/sse.md +@@ -1 +1,2 @@ + ++ + +-- +2.29.2 + +=== 0002-Wrong-macro-changelog.patch === +From 3542802111d4c6752ac7233ef96655b7fb78aae4 Mon Sep 17 00:00:00 2001 +From: Martin Liska mliska@suse.cz +Date: Wed, 13 Jan 2021 16:54:58 +0100 +Subject: [PATCH 2/2] Wrong macro changelog + +gcc/ChangeLog:

+ * config/i386/i386.md (*fix_trunc<mode>_i387_1, + (foo): Change it. +--- + gcc/config/i386/i386.md | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md +index ac63591b33f..ff4d61764e7 100644 +--- a/gcc/config/i386/i386.md ++++ b/gcc/config/i386/i386.md +@@ -1 +1,2 @@ + ++ +-- +2.29.2 + +=== 0001-ChangeLog-removal.patch === +From b39fadf9df1a9510afcab0a391182da7dc68de24 Mon Sep 17 00:00:00 2001 +From: Martin Liska mliska@suse.cz +Date: Fri, 12 Mar 2021 09:10:55 +0100 +Subject: [PATCH] Test ChangeLog removal. + +gcc/ChangeLog: + + * ipa-icf.c (make_pass_ipa_icf): Add line. +--- +diff --git a/gcc/analyzer/ChangeLog b/gcc/analyzer/ChangeLog +deleted file mode 100644 +index 94e87f6bcde..00000000000 +--- a/gcc/analyzer/ChangeLog ++++ /dev/null +@@ -1,1 +0,0 @@ +- foo +diff --git a/gcc/ipa-icf.c b/gcc/ipa-icf.c +index 5dd33a75c3a..c4ce432cb98 100644 +--- a/gcc/ipa-icf.c ++++ b/gcc/ipa-icf.c +@@ -3655,3 +3655,4 @@ make_pass_ipa_icf (gcc::context *ctxt) + { + return new ipa_icf::pass_ipa_icf (ctxt); + } ++ +-- +2.30.1 + +=== 0001-long-filenames === +From 0a5b3f87bdac5e61f9a626c795d30f9e93198585 Mon Sep 17 00:00:00 2001 +From: Martin Liska mliska@suse.cz +Date: Mon, 12 Apr 2021 13:10:14 +0200 +Subject: [PATCH] libstdc++: Fix some tests that fail in C++20 mode + +The linear_congruential_engine negative tests fail with a different +error in C++20 mode, because double is no longer an invalid type for +NTTP. Adjust the expected errors. + +libstdc++-v3/ChangeLog: + + * testsuite/26_numerics/random/linear_congruential_engine/requirements/non_uint_neg.cc: + Adjust expected error for C++20 mode. + * testsuite/tr1/5_numerical_facilities/random/linear_congruential/requirements/non_uint_neg.cc: + Likewise. +--- + .../linear_congruential_engine/requirements/non_uint_neg.cc | 4 +++- + .../random/linear_congruential/requirements/non_uint_neg.cc | 3 ++- + 2 files changed, 5 insertions(+), 2 deletions(-) + +diff --git a/libstdc++-v3/testsuite/26_numerics/random/linear_congruential_engine/requirements/non_uint_neg.cc b/libstdc++-v3/testsuite/26_numerics/random/linear_congruential_engine/requirements/non_uint_neg.cc +index e04e8ca6972..a36d63c6c7b 100644 +--- a/libstdc++-v3/testsuite/26_numerics/random/linear_congruential_engine/requirements/non_uint_neg.cc ++++ b/libstdc++-v3/testsuite/26_numerics/random/linear_congruential_engine/requirements/non_uint_neg.cc +@@ -1 +1,2 @@ + ++ +diff --git a/libstdc++-v3/testsuite/tr1/5_numerical_facilities/random/linear_congruential/requirements/non_uint_neg.cc b/libstdc++-v3/testsuite/tr1/5_numerical_facilities/random/linear_congruential/requirements/non_uint_neg.cc +index 5ad82db1def..53b15f32516 100644 +--- a/libstdc++-v3/testsuite/tr1/5_numerical_facilities/random/linear_congruential/requirements/non_uint_neg.cc ++++ b/libstdc++-v3/testsuite/tr1/5_numerical_facilities/random/linear_congruential/requirements/non_uint_neg.cc +@@ -1 +1,2 @@ + ++ </cut>