Hi Zhangjin,
On Tue, Jul 11, 2023 at 11:55:08AM +0800, Zhangjin Wu wrote:
Hi, Willy and Thomas
I'm planning to renew this patchset, so, require more discuss.
The main goal of this minimal kernel config support here is faster build and therefore faster test for the run target, but this old patchset is not enough, especially for the 'mrproper' operation during every config:
defconfig: $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare +tinyconfig: + $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare +
It will depend on the number of runs, because the mrproper will only be run before recreating a config. Logically once you have your kernel you're not going to reconfigure it.
In order to avoid this 'mrproper', one solution is create an arch specicifc output directory for all of the kernel targets to avoid they share the same build output (by default, it is the same as the source code directory and therefore pollute the source code tree), it is required for both tools/nolibc and selftests/nolibc:
This could provide an improvement when building for multiple archs in a row, indeed. But keep in mind that an important aspect is also to be able to test the kernel you're currently working on, and not to have to rebuild it again for nolibc into a subdir.
And given that the makefile doesn't know how to iterate over multiple archs, it would make sense to permit to pass the build output as an argument to the makefile so that the external script calling make for each arch would just pass the build directory (as is sometimes done for the normal kernel).
The "O" variable could be used to pass an arch specific output for every arch:
# OUTPUT is only set when run from the main makefile, otherwise # it defaults to this nolibc directory. OUTPUT ?= $(CURDIR)/ # Architecture specific directory ARCH_OUTPUT ?= $(CURDIR)/build/$(ARCH)/With "O":
// tools/nolibc KERNEL_BUILD ?= $(ARCH_OUTPUT)linux/ MAKE_KERNEL = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH) headers_standalone: headers $(Q)$(MAKE_KERNEL) headers $(Q)$(MAKE_KERNEL) headers_install INSTALL_HDR_PATH=$(ARCH_OUTPUT)sysroot// selftests/nolibc RUN_OUT = $(ARCH_OUTPUT)run.out ... NOLIBC_INITRAMFS = $(ARCH_OUTPUT)initramfs/ ... KERNEL_BUILD ?= $(ARCH_OUTPUT)linux/ MAKE_KERNEL = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
Yes but this one will break reuse of the already built kernel. Also for those trying all archs for a single test it will eat a lot of disk space (when enabling certain debugging symbols you can end up with extremely large build dirs, I already ran out of space a few times).
I think that instead we should just let the user pass this "O=" to the make command and let the lower ones inherit it. That makes it closer to what it usually done. I'm also fine with using another variable if there's a good reason for this, but given that it's already called "O" it's not something unexpected to the user. Thus we could imagine just running:
for arch in "${ARCHS[@]}"; do make -j$(nproc) O=$PWD/kernel-$arch ARCH=$arch \ CROSS_COMPILE=/path/to/cc/$arch/bin/$arch- \ run done
Which is simple enough without requiring a user to have to figure what build option to hack on for just a regular build.
And further, the output of the other targets should be put in the $(ARCH_OUTPUT) directory too, include the sysroot, nolibc-test, initramfs, run.out and even libc-test, the whole Makefile of selftests/nolibc will be touched.
It's a matter of taste but there are pros and cons. I personally spend my time using grep -r on the sysroots, having gigabytes of .o there would quickly made me regret that approach. I would also argue that when you pass "O=" to the kernel build, it serves as a prefix and the directory hierarchy is preserved, so if you would want to put the sysroots etc under $O then it would be more logical that it appears in $O/tools/testing/selftests/nolibc/sysroot/$arch, which would then remain compatible with today when O corresponds to the kernel's CURDIR. Then maybe that would be the solution to making O per-arch for those who want to re-run over many kernels, while keeping it easy for those who want to test their own kernel or want to grep in sysroot.
I have prepared and verified such a solution locally, before send the patches, welcome your suggestions.
My local patchset now provides such features:
- Allow build everything for a target arch in the arch specific directory
- Allow clean everything of a target arch
- Allow trigger a config target automatically for the run target, use the new tinyconfig by default, be able to choose defconfig as before
- Allow configure additional config options for both defconfig and tinyconfig
- Allow configure the default CROSS_COMPILE by default, only the ARCH variable is required
- Allow download the missing cross toolchains automatically (from mirrrors.edge.kernel.org)
- Allow download the missing bios automatically
- Allow download and build musl automatically
- Allow run tests for all of the supported architectures automatically with one command with minimal preparation, may be only the installation of qemu-user and qemu-system
There can be good things there, but I'm seeing a lot of "by default" and "automatically" which worry me, because when you're debugging something, there's nothing more annoying than all the stuff happening in your back and fooling your analysis. On the opposite I do value the ability to proceed through steps that I don't need to revalidate, and these ones can be done automatically just by chaining a few commands from a script.
And no download should ever be done automatically. Any download must always be deliberately and consciously initiated by the user. How many times we've seen some scripts stall with no information, just to discover a SYN_SENT socket in netstat! You can have a "download-toolchain", "download-bios", "download-musl" etc if you want, but quite frankly, I'm seriously doubting it will be *that* useful to anyone working on the kernel since by definition they already have these toolchains, and sometimes different ones (e.g. linaro, bootlin etc) and in different versions. In fact if some expresses the need for having a simplified command to download a toolchain, then it wouldn't be specific to nolibc-test but would be used as a general tool for the kernel since it would be shared with all the rest of the code. You could also imagine downloading and building qemu, but it also takes a while and we're very far from the simple build script whose purpose was to validate that a few syscalls appeared to work fine.
With the new patchset, it is able to rebuild and rerun everything in several minutes, it may make people very happy to develop nolibc itself and also make people to work on linux with nolibc, especially for developing and testing a new kernel feature ;-)
I doubt it. Rebuilding everything is not what most developers need. What they need is to just fix a missing parenthesis that broke the build and rebuild the strict minimum, restarting from where they left. And they need to be able to figure what caused that "FAILED" output, disassemble the output, re-run it manually under qemu-user, rebuild the .c manually after copy-pasting the whole command line via "V=1", etc.
Keep in mind that the purpose of a selftest is not to optimize the case where it works fine, but to optimize the developer's work when it fails. This is something that a lot of people tend to get wrong. They're proud of a "make world" that does everything including delivering them pizzas to wait for the builds to complete, and they say "hey, impressed?". But when someone else reports "I'm getting this strange error I don't understand", they can hardly suggest more than "hmmm run make clean, try again, or use the same distro as me because it works for me".
What I've tried to focus on from the beginning with this tiny test suite precisely is debugging. That's why I wanted to support alternate libcs, to help compare, making it easy to use a different toolchain and other cflags, running in userland, building just the binary so that I can scp it to a remote native machine etc. All of these are important, and your approach above makes me feel like I would lose most of it, or still be able to do that but in a very complicated way (i.e. constantly have to reverse-engineer a complicated makefile to figure where I can inject an option).
However one thing I mentioned in the past is the usefulness of some batching scripts (I even shared one). And I think that helping the user prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!
Thanks, Willy