On Mon, Nov 06, 2023 at 11:09:44AM -0600, Rob Herring wrote:
On Thu, Nov 2, 2023 at 12:36 PM Mark Brown broonie@kernel.org wrote:
On Thu, Nov 02, 2023 at 07:15:58PM +0530, Naresh Kamboju wrote:
On Thu, 2 Nov 2023 at 17:41, Aishwarya TCV aishwarya.tcv@arm.com wrote:
https://storage.kernelci.org/mainline/master/v6.6-9152-gdeefd5024f07/arm64/d...
...
May be due to, A loop of symlinks that are pointing to self / same files ?
Right, it does look like something bad is going on with symlinks:
'/tmp/kci/linux/tools/testing/selftests/../../../build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/tools/testing/selftests/powerpc/vphn/vphn.c'
Please build by using tuxmake and validate builds are working.
Note that tuxmake does an in tree build of kselftest:
make --silent --keep-going --jobs=8 O=/home/tuxbuild/.cache/tuxmake/builds/1/build INSTALL_PATH=/home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- 'CC=sccache aarch64-linux-gnu-gcc' 'HOSTCC=sccache gcc' kselftest-install
and does it's own tarball build too, whereas kernelci does an out of tree build and uses kselftest-gen_tar:
make KBUILD_BUILD_USER=KernelCI FORMAT=.xz ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- CC="ccache aarch64-linux-gnu-gcc" O=/tmp/kci/linux/build -C/tmp/kci/linux -j10 kselftest-gen_tar
and that the error is in the dt-extract-compatibles program which is part of the kernel (well, imported into the kernel from dtc upstream):
File "/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles", line 107, in <module> compat_ignore_list.extend(parse_compatibles_to_ignore(f))
This all suggests that something to do with how the build is set up is resulting in the source symlink that gets created for out of tree builds blowing up, I guess it's not specifically the DT stuff that's blowing it up but rather that it's tripping over an existing bug. Really does look like a legitimate bug though, the source link is set up by the in tree kernel build infrastructure.
I did poke a bit at reproducing outside of the KernelCI scripts but didn't manage to yet.
I can repro with "make dt_compatible_check". The problem is with an 'out of tree' build within the tree. That's my normal setup, but the difference is I have ".build" directories. If I use "build" instead, then I can repro. The issue is the iglob will recurse into "build" but not hidden directories (by default). There's no option to not follow symlinks which would solve this (there is an open python issue since 2017 to add it). I don't see a simple solution in python other than getting a full list with glob(), convert to absolute paths, and remove duplicates. I imagine that will be somewhat slow.
Hi, sorry for the delay, I was on vacation last week.
I was able to reproduce the issue the way you described. And I also suspected an alternative approach would be slower, but after trying it out it ran just as fast as the current one, even on cold cache, so I sent it out:
https://lore.kernel.org/all/20231107225624.9811-1-nfraprado@collabora.com
Let me know your thoughts there.
Thanks, Nícolas