On Tue, Feb 11, 2020 at 2:10 PM Nicolas Dechesne nicolas.dechesne@linaro.org wrote:
On Tue, Feb 11, 2020 at 1:29 PM Ryan Harkin ryan.harkin@linaro.org wrote:
On Tue, 11 Feb 2020, 12:19 Nicolas Dechesne, nicolas.dechesne@linaro.org wrote:
hey,
On Tue, Feb 11, 2020 at 12:59 PM Ryan Harkin ryan.harkin@linaro.org wrote:
Hi Nico,
On Tue, 11 Feb 2020 at 08:34, Nicolas Dechesne nicolas.dechesne@linaro.org wrote:
hi,
we are hitting some errors with repo init since last night.
Yes, I was investigating this failure too. Thanks very much for working it out.
After looking into it , I came up with the following explanation and fix.. If I am correct, it will impact all our OE RPB build setup..
It definitely impacts on my WaRP7 builds, which are failing since last night too.
Can you please review that carefully?
The PR is on github: https://github.com/96boards/oe-rpb-manifest/pull/121
I added myself as a reviewer and approved it.
thanks. Looks like Fathi merged it.
Good. That's a quick turnaround.
I will backport to other branches and push the same fix. You are probably using your own branch anyways,
argh.. now it gets funny.. our builds have restarted, but still fail... the reason is because of this line:
https://git.linaro.org/ci/job/configs.git/tree/rpb-openembedded/builders.sh#...
so basically we fetch the 'previous' manifest, and try to deduce the changes since the last build. However the old manifest contains the symlink with relative path, so the new repo command fails
00:05:27.092 + repo diffmanifests /srv/oe/pinned-manifest-latest.xml /srv/oe/pinned-manifest.xml 00:05:27.233 Traceback (most recent call last): 00:05:27.233 File "/srv/oe/.repo/repo/main.py", line 555, in <module> 00:05:27.233 _Main(sys.argv[1:]) 00:05:27.233 File "/srv/oe/.repo/repo/main.py", line 530, in _Main 00:05:27.233 result = run() 00:05:27.233 File "/srv/oe/.repo/repo/main.py", line 523, in <lambda> 00:05:27.233 run = lambda: repo._Run(name, gopts, argv) or 0 00:05:27.233 File "/srv/oe/.repo/repo/main.py", line 212, in _Run 00:05:27.233 result = cmd.Execute(copts, cargs) 00:05:27.233 File "/srv/oe/.repo/repo/subcmds/diffmanifests.py", line 195, in Execute 00:05:27.233 manifest1.Override(args[0], load_local_manifests=False) 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 174, in Override 00:05:27.233 self._Load() 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 480, in _Load 00:05:27.233 self._ParseManifest(nodes) 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 588, in _ParseManifest 00:05:27.233 project = self._ParseProject(node) 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 910, in _ParseProject 00:05:27.233 self._ParseLinkFile(project, n) 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 1045, in _ParseLinkFile 00:05:27.233 self._ValidateFilePaths('linkfile', src, dest) 00:05:27.233 File "/srv/oe/.repo/repo/manifest_xml.py", line 1026, in _ValidateFilePaths 00:05:27.233 '<%s> invalid "src": %s: %s' % (element, src, msg)) 00:05:27.233 error.ManifestInvalidPathError: <linkfile> invalid "src": ../../.repo/manifests/setup-environment: bad component: ..
not sure how to best fix that... maybe we should check the old manifest and try to detect the symlink with relative path and skip repo diffmanifest.. it really gets ugly...
I am not proud ;-) but I think this should fix the problem for all our builds? Can you please check: https://review.linaro.org/c/ci/job/configs/+/34254
Yes, I have a clone with some small patches on top.
so you should cherry-pick for your project(s) !
Same difference, but I rebase my branch when anything new is committed.
You've highlighted problems below, but in the end, your fix is better than no fix. And anything else we do could be on top of this fix.
I put a copy of the commit message below to bring it to your attention!
cheers nico
== All along we have been using a relative symlink for setup-environment which points outside of the project path. While we've enjoyed this hack for several years, a recent change was in repo, and repo init now fails with:
error.ManifestInvalidPathError: <linkfile> invalid "src": ../../.repo/manifests/setup-environment: bad component: .repo
The change in repo is this one: https://gerrit.googlesource.com/git-repo/+/07392ed32662006c029299bc06617310c...
Yes, I also worked out this was the culprit.
The repo documentation states: The symlink is created at “dest” (relative to the top of the tree) and points to the path specified by “src” which is a path in the project.
As such, repo clearly expects that 'src' is in the project path, which is not the cause when using ../../.repo/ path.
This patch changes our folder structure in such a way that:
- the manifest git is checked out as ./conf/ folder in the top level
directory
- setup-environment link is created by repo, as a link to
conf/setup-environment-internal
The main drawback is that it feels non standard to duplicate the whole manifest project again (since it's already checked out in .repo) however using .repo content also meant we were making assumptions on
implementation details of repo itself.
I don't see an obvious way around this. We are using the repo contents, but we're not allowed to use it directly.
So unless we re-work how we do setup-environment, I don't see a way to avoid this. The only nagging thought I have is that setup-environment and default.xml are two distinct things, so we could split it into two repos. But as the repo is so small, I don't see the added value of doing this. It seems to be more overhead. That said, when I clone the manifest repo for my own projects, all I'm doing is editing default.xml, and nothing else. So it might make sense anyway, so I can get changes to things like setup-env without have to rebase my manifest repo. But again, it's such a small repo, it's not really a problem to have it like this. And probably makes life easier in the long term.
Cheers, Ryan.
Signed-off-by: Nicolas Dechesne nicolas.dechesne@linaro.org _______________________________________________ OpenEmbedded mailing list OpenEmbedded@lists.linaro.org https://lists.linaro.org/mailman/listinfo/openembedded