Hi, Willy
Here is the v5, purely include the ppc parts, with two critical fixups for the latest gcc 13.1.0 toolchain, now, both run and run-user pass.
Here is the run-user test report:
// with local toolchains $ for arch in ppc ppc64 ppc64le; do make run-user XARCH=$arch | grep "status: "; done 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning
// with latest toolchains $ for arch in ppc ppc64 ppc64le; do make run-user XARCH=$arch CC=/path/to/gcc-13.1.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc | grep status; file nolibc-test; done 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning nolibc-test: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), statically linked, stripped 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning nolibc-test: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, stripped 165 test(s): 157 passed, 8 skipped, 0 failed => status: warning nolibc-test: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, stripped
Since the missing serial console enabling patch [1] for ppc32 has already gotten a Reviewed-by line from the ppc maintainer, now, the ppc defconfig aligns with the others', and it is able to simply move the nolibc-test-config related stuff to the next tinyconfig series.
Based on v4 [2], beside removing several nolibc-test-config related patches, two bugs with the latest gcc 13.1.0 have been fixed.
Changes from v4 --> v5:
* tools/nolibc: add support for powerpc64 selftests/nolibc: add XARCH and ARCH mapping support selftests/nolibc: add test support for ppc64 selftests/nolibc: allow customize CROSS_COMPILE by architecture selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
Almost the same as v4.
* tools/nolibc: add support for powerpc
For 32-bit PowerPC, with newer gcc compilers (e.g. gcc 13.1.0), "omit-frame-pointer" fails with __attribute__((no_stack_protector)) but works with __attribute__((__optimize__("-fno-stack-protector")))
Using the later for ppc32 to workaround the issue.
* selftests/nolibc: add test support for ppc
Add default CFLAGS for ppc to allow build with the latest powerpc64-linux-gcc toolchain from https://mirrors.edge.kernel.org/pub/tools/crosstool/
* selftests/nolibc: add test support for ppc64le
Align with kernel, prefer elfv2 ABI to elfv1 ABI when the toolchain support, otherwise, ABI mismatched binary will not run.
Best regards, Zhangjin Wu --- [1]: https://lore.kernel.org/lkml/bb7b5f9958b3e3a20f6573ff7ce7c5dc566e7e32.169098... [2]: https://lore.kernel.org/lkml/cover.1690916314.git.falcon@tinylab.org/
Zhangjin Wu (8): tools/nolibc: add support for powerpc tools/nolibc: add support for powerpc64 selftests/nolibc: add XARCH and ARCH mapping support selftests/nolibc: add test support for ppc selftests/nolibc: add test support for ppc64le selftests/nolibc: add test support for ppc64 selftests/nolibc: allow customize CROSS_COMPILE by architecture selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
tools/include/nolibc/arch-powerpc.h | 213 ++++++++++++++++++++++++ tools/include/nolibc/arch.h | 2 + tools/testing/selftests/nolibc/Makefile | 74 ++++++-- 3 files changed, 277 insertions(+), 12 deletions(-) create mode 100644 tools/include/nolibc/arch-powerpc.h
Both syscall declarations and _start code definition are added for powerpc to nolibc.
Like mips, powerpc uses a register (exactly, the summary overflow bit) to record the error occurred, and uses another register to return the value [1]. So, the return value of every syscall declaration must be normalized to match the __sysret() helper, return -value when there is an error, otheriwse, return value directly.
Glibc and musl use different methods to check the summary overflow bit, glibc (sysdeps/unix/sysv/linux/powerpc/sysdep.h) saves the cr register to r0 at first, and then check the summary overflow bit in cr0:
mfcr r0 r0 & (1 << 28) ? -r3 : r3
-->
10003c14: 7c 00 00 26 mfcr r0 10003c18: 74 09 10 00 andis. r9,r0,4096 10003c1c: 41 82 00 08 beq 0x10003c24 10003c20: 7c 63 00 d0 neg r3,r3
Musl (arch/powerpc/syscall_arch.h) directly checks the summary overflow bit with the 'bns' instruction, it is smaller:
/* no summary overflow bit means no error, return value directly */ bns+ 1f /* otherwise, return negated value */ neg r3, r3 1:
-->
10000418: 40 a3 00 08 bns 0x10000420 1000041c: 7c 63 00 d0 neg r3,r3
Like musl, Linux (arch/powerpc/include/asm/vdso/gettimeofday.h) uses the same method for do_syscall_2() too.
Here applies the second method to get smaller size.
[1]: https://man7.org/linux/man-pages/man2/syscall.2.html
Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/include/nolibc/arch-powerpc.h | 197 ++++++++++++++++++++++++++++ tools/include/nolibc/arch.h | 2 + 2 files changed, 199 insertions(+) create mode 100644 tools/include/nolibc/arch-powerpc.h
diff --git a/tools/include/nolibc/arch-powerpc.h b/tools/include/nolibc/arch-powerpc.h new file mode 100644 index 000000000000..8332c9d3e5d6 --- /dev/null +++ b/tools/include/nolibc/arch-powerpc.h @@ -0,0 +1,197 @@ +/* SPDX-License-Identifier: LGPL-2.1 OR MIT */ +/* + * PowerPC specific definitions for NOLIBC + * Copyright (C) 2023 Zhangjin Wu falcon@tinylab.org + */ + +#ifndef _NOLIBC_ARCH_POWERPC_H +#define _NOLIBC_ARCH_POWERPC_H + +#include "compiler.h" +#include "crt.h" + +/* Syscalls for PowerPC : + * - stack is 16-byte aligned + * - syscall number is passed in r0 + * - arguments are in r3, r4, r5, r6, r7, r8, r9 + * - the system call is performed by calling "sc" + * - syscall return comes in r3, and the summary overflow bit is checked + * to know if an error occurred, in which case errno is in r3. + * - the arguments are cast to long and assigned into the target + * registers which are then simply passed as registers to the asm code, + * so that we don't have to experience issues with register constraints. + */ + +#define _NOLIBC_SYSCALL_CLOBBERLIST \ + "memory", "cr0", "r12", "r11", "r10", "r9" + +#define my_syscall0(num) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num) \ + : \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8", "r7", "r6", "r5", "r4" \ + ); \ + _ret; \ +}) + +#define my_syscall1(num, arg1) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8", "r7", "r6", "r5", "r4" \ + ); \ + _ret; \ +}) + + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + register long _arg2 __asm__ ("r4") = (long)(arg2); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num), "+r"(_arg2) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8", "r7", "r6", "r5" \ + ); \ + _ret; \ +}) + + +#define my_syscall3(num, arg1, arg2, arg3) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + register long _arg2 __asm__ ("r4") = (long)(arg2); \ + register long _arg3 __asm__ ("r5") = (long)(arg3); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num), "+r"(_arg2), "+r"(_arg3) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8", "r7", "r6" \ + ); \ + _ret; \ +}) + + +#define my_syscall4(num, arg1, arg2, arg3, arg4) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + register long _arg2 __asm__ ("r4") = (long)(arg2); \ + register long _arg3 __asm__ ("r5") = (long)(arg3); \ + register long _arg4 __asm__ ("r6") = (long)(arg4); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num), "+r"(_arg2), "+r"(_arg3), \ + "+r"(_arg4) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8", "r7" \ + ); \ + _ret; \ +}) + + +#define my_syscall5(num, arg1, arg2, arg3, arg4, arg5) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + register long _arg2 __asm__ ("r4") = (long)(arg2); \ + register long _arg3 __asm__ ("r5") = (long)(arg3); \ + register long _arg4 __asm__ ("r6") = (long)(arg4); \ + register long _arg5 __asm__ ("r7") = (long)(arg5); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num), "+r"(_arg2), "+r"(_arg3), \ + "+r"(_arg4), "+r"(_arg5) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST, "r8" \ + ); \ + _ret; \ +}) + +#define my_syscall6(num, arg1, arg2, arg3, arg4, arg5, arg6) \ +({ \ + register long _ret __asm__ ("r3"); \ + register long _num __asm__ ("r0") = (num); \ + register long _arg1 __asm__ ("r3") = (long)(arg1); \ + register long _arg2 __asm__ ("r4") = (long)(arg2); \ + register long _arg3 __asm__ ("r5") = (long)(arg3); \ + register long _arg4 __asm__ ("r6") = (long)(arg4); \ + register long _arg5 __asm__ ("r7") = (long)(arg5); \ + register long _arg6 __asm__ ("r8") = (long)(arg6); \ + \ + __asm__ volatile ( \ + " sc\n" \ + " bns+ 1f\n" \ + " neg %0, %0\n" \ + "1:\n" \ + : "=r"(_ret), "+r"(_num), "+r"(_arg2), "+r"(_arg3), \ + "+r"(_arg4), "+r"(_arg5), "+r"(_arg6) \ + : "0"(_arg1) \ + : _NOLIBC_SYSCALL_CLOBBERLIST \ + ); \ + _ret; \ +}) + +/* FIXME: For 32-bit PowerPC, with newer gcc compilers (e.g. gcc 13.1.0), + * "omit-frame-pointer" fails with __attribute__((no_stack_protector)) but + * works with __attribute__((__optimize__("-fno-stack-protector"))) + */ +#ifdef __no_stack_protector +#undef __no_stack_protector +#define __no_stack_protector __attribute__((__optimize__("-fno-stack-protector"))) +#endif + +/* startup code */ +void __attribute__((weak, noreturn, optimize("Os", "omit-frame-pointer"))) __no_stack_protector _start(void) +{ + __asm__ volatile ( + "mr 3, 1\n" /* save stack pointer to r3, as arg1 of _start_c */ + "clrrwi 1, 1, 4\n" /* align the stack to 16 bytes */ + "li 0, 0\n" /* zero the frame pointer */ + "stwu 1, -16(1)\n" /* the initial stack frame */ + "bl _start_c\n" /* transfer to c runtime */ + ); + __builtin_unreachable(); +} + +#endif /* _NOLIBC_ARCH_POWERPC_H */ diff --git a/tools/include/nolibc/arch.h b/tools/include/nolibc/arch.h index 82b43935650f..e276fb0680af 100644 --- a/tools/include/nolibc/arch.h +++ b/tools/include/nolibc/arch.h @@ -25,6 +25,8 @@ #include "arch-aarch64.h" #elif defined(__mips__) && defined(_ABIO32) #include "arch-mips.h" +#elif defined(__powerpc__) +#include "arch-powerpc.h" #elif defined(__riscv) #include "arch-riscv.h" #elif defined(__s390x__)
This follows the 64-bit PowerPC ABI [1], refers to the slides: "A new ABI for little-endian PowerPC64 Design & Implementation" [2] and the musl code in arch/powerpc64/crt_arch.h.
First, stdu and clrrdi are used instead of stwu and clrrwi for powerpc64.
Second, the stack frame size is increased to 32 bytes for powerpc64, 32 bytes is the minimal stack frame size supported described in [2].
Besides, the TOC pointer (GOT pointer) must be saved to r2.
This works on both little endian and big endian 64-bit PowerPC.
[1]: https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.pdf [2]: https://www.llvm.org/devmtg/2014-04/PDFs/Talks/Euro-LLVM-2014-Weigand.pdf
Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/include/nolibc/arch-powerpc.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
diff --git a/tools/include/nolibc/arch-powerpc.h b/tools/include/nolibc/arch-powerpc.h index 8332c9d3e5d6..76c3784f9dc7 100644 --- a/tools/include/nolibc/arch-powerpc.h +++ b/tools/include/nolibc/arch-powerpc.h @@ -172,6 +172,7 @@ _ret; \ })
+#ifndef __powerpc64__ /* FIXME: For 32-bit PowerPC, with newer gcc compilers (e.g. gcc 13.1.0), * "omit-frame-pointer" fails with __attribute__((no_stack_protector)) but * works with __attribute__((__optimize__("-fno-stack-protector"))) @@ -180,10 +181,24 @@ #undef __no_stack_protector #define __no_stack_protector __attribute__((__optimize__("-fno-stack-protector"))) #endif +#endif /* !__powerpc64__ */
/* startup code */ void __attribute__((weak, noreturn, optimize("Os", "omit-frame-pointer"))) __no_stack_protector _start(void) { +#ifdef __powerpc64__ + /* On 64-bit PowerPC, save TOC/GOT pointer to r2 */ + extern char TOC __asm__ (".TOC."); + register volatile long r2 __asm__ ("r2") = (void *)&TOC - (void *)_start; + + __asm__ volatile ( + "mr 3, 1\n" /* save stack pointer to r3, as arg1 of _start_c */ + "clrrdi 1, 1, 4\n" /* align the stack to 16 bytes */ + "li 0, 0\n" /* zero the frame pointer */ + "stdu 1, -32(1)\n" /* the initial stack frame */ + "bl _start_c\n" /* transfer to c runtime */ + ); +#else __asm__ volatile ( "mr 3, 1\n" /* save stack pointer to r3, as arg1 of _start_c */ "clrrwi 1, 1, 4\n" /* align the stack to 16 bytes */ @@ -191,6 +206,7 @@ void __attribute__((weak, noreturn, optimize("Os", "omit-frame-pointer"))) __no_ "stwu 1, -16(1)\n" /* the initial stack frame */ "bl _start_c\n" /* transfer to c runtime */ ); +#endif __builtin_unreachable(); }
Most of the CPU architectures have different variants, but kernel usually only accepts parts of them via the ARCH variable, the others should be customized via kernel config files.
To simplify testing, a new XARCH variable is added to extend the kernel's ARCH with a few variants of the same architecture, and it is used to customize variant specific variables, at last XARCH is converted to the kernel's ARCH:
e.g. make run XARCH=<one of the supported variants> | \ | `-> variant specific variables: | IMAGE, DEFCONFIG, QEMU_ARCH, QEMU_ARGS, CFLAGS ... \ `---> kernel's ARCH
XARCH and ARCH are carefully mapped to allow users to pass architecture variants via XARCH or pass architecture via ARCH from cmdline.
PowerPC is the first user and also a very good reference architecture of this mapping, it has variants with different combinations of 32-bit/64-bit and bit endian/little endian.
To use this mapping, the other architectures can refer to PowerPC, If the target architecture only has one variant, XARCH is simply an alias of ARCH, no additional mapping required.
Suggested-by: Willy Tarreau w@1wt.eu Link: https://lore.kernel.org/lkml/20230702171715.GD16233@1wt.eu/ Link: https://lore.kernel.org/lkml/20230730061801.GA7690@1wt.eu/ Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 46 ++++++++++++++++++------- 1 file changed, 34 insertions(+), 12 deletions(-)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index f42adef87e12..46a2312b06f5 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -14,6 +14,27 @@ include $(srctree)/scripts/subarch.include ARCH = $(SUBARCH) endif
+# XARCH extends the kernel's ARCH with a few variants of the same +# architecture that only differ by the configuration, the toolchain +# and the Qemu program used. It is copied as-is into ARCH except for +# a few specific values which are mapped like this: +# +# XARCH | ARCH | config +# -------------|-----------|------------------------- +# ppc | powerpc | 32 bits +# ppc64 | powerpc | 64 bits big endian +# ppc64le | powerpc | 64 bits little endian +# +# It is recommended to only use XARCH, though it does not harm if +# ARCH is already set. For simplicity, ARCH is sufficient for all +# architectures where both are equal. + +# configure default variants for target kernel supported architectures +XARCH = $(or $(XARCH_$(ARCH)),$(ARCH)) + +# map from user input variants to their kernel supported architectures +ARCH := $(or $(ARCH_$(XARCH)),$(XARCH)) + # kernel image names by architecture IMAGE_i386 = arch/x86/boot/bzImage IMAGE_x86_64 = arch/x86/boot/bzImage @@ -24,7 +45,7 @@ IMAGE_mips = vmlinuz IMAGE_riscv = arch/riscv/boot/Image IMAGE_s390 = arch/s390/boot/bzImage IMAGE_loongarch = arch/loongarch/boot/vmlinuz.efi -IMAGE = $(IMAGE_$(ARCH)) +IMAGE = $(IMAGE_$(XARCH)) IMAGE_NAME = $(notdir $(IMAGE))
# default kernel configurations that appear to be usable @@ -37,7 +58,7 @@ DEFCONFIG_mips = malta_defconfig DEFCONFIG_riscv = defconfig DEFCONFIG_s390 = defconfig DEFCONFIG_loongarch = defconfig -DEFCONFIG = $(DEFCONFIG_$(ARCH)) +DEFCONFIG = $(DEFCONFIG_$(XARCH))
# optional tests to run (default = all) TEST = @@ -52,7 +73,7 @@ QEMU_ARCH_mips = mipsel # works with malta_defconfig QEMU_ARCH_riscv = riscv64 QEMU_ARCH_s390 = s390x QEMU_ARCH_loongarch = loongarch64 -QEMU_ARCH = $(QEMU_ARCH_$(ARCH)) +QEMU_ARCH = $(QEMU_ARCH_$(XARCH))
# QEMU_ARGS : some arch-specific args to pass to qemu QEMU_ARGS_i386 = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)" @@ -64,7 +85,7 @@ QEMU_ARGS_mips = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_riscv = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_s390 = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_loongarch = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)" -QEMU_ARGS = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_EXTRA) +QEMU_ARGS = $(QEMU_ARGS_$(XARCH)) $(QEMU_ARGS_EXTRA)
# OUTPUT is only set when run from the main makefile, otherwise # it defaults to this nolibc directory. @@ -81,7 +102,7 @@ CFLAGS_mips = -EL CFLAGS_STACKPROTECTOR ?= $(call cc-option,-mstack-protector-guard=global $(call cc-option,-fstack-protector-all)) CFLAGS ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 \ $(call cc-option,-fno-stack-protector) \ - $(CFLAGS_$(ARCH)) $(CFLAGS_STACKPROTECTOR) + $(CFLAGS_$(XARCH)) $(CFLAGS_STACKPROTECTOR) LDFLAGS := -s
REPORT ?= awk '/[OK][\r]*$$/{p++} /[FAIL][\r]*$$/{if (!f) printf("\n"); f++; print;} /[SKIPPED][\r]*$$/{s++} \ @@ -96,24 +117,25 @@ help: @echo " sysroot create the nolibc sysroot here (uses $$ARCH)" @echo " nolibc-test build the executable (uses $$CC and $$CROSS_COMPILE)" @echo " libc-test build an executable using the compiler's default libc instead" - @echo " run-user runs the executable under QEMU (uses $$ARCH, $$TEST)" + @echo " run-user runs the executable under QEMU (uses $$XARCH, $$TEST)" @echo " initramfs prepare the initramfs with nolibc-test" - @echo " defconfig create a fresh new default config (uses $$ARCH)" - @echo " kernel (re)build the kernel with the initramfs (uses $$ARCH)" - @echo " run runs the kernel in QEMU after building it (uses $$ARCH, $$TEST)" - @echo " rerun runs a previously prebuilt kernel in QEMU (uses $$ARCH, $$TEST)" + @echo " defconfig create a fresh new default config (uses $$XARCH)" + @echo " kernel (re)build the kernel with the initramfs (uses $$XARCH)" + @echo " run runs the kernel in QEMU after building it (uses $$XARCH, $$TEST)" + @echo " rerun runs a previously prebuilt kernel in QEMU (uses $$XARCH, $$TEST)" @echo " clean clean the sysroot, initramfs, build and output files" @echo "" @echo "The output file is "run.out". Test ranges may be passed using $$TEST." @echo "" @echo "Currently using the following variables:" @echo " ARCH = $(ARCH)" + @echo " XARCH = $(XARCH)" @echo " CROSS_COMPILE = $(CROSS_COMPILE)" @echo " CC = $(CC)" @echo " OUTPUT = $(OUTPUT)" @echo " TEST = $(TEST)" - @echo " QEMU_ARCH = $(if $(QEMU_ARCH),$(QEMU_ARCH),UNKNOWN_ARCH) [determined from $$ARCH]" - @echo " IMAGE_NAME = $(if $(IMAGE_NAME),$(IMAGE_NAME),UNKNOWN_ARCH) [determined from $$ARCH]" + @echo " QEMU_ARCH = $(if $(QEMU_ARCH),$(QEMU_ARCH),UNKNOWN_ARCH) [determined from $$XARCH]" + @echo " IMAGE_NAME = $(if $(IMAGE_NAME),$(IMAGE_NAME),UNKNOWN_ARCH) [determined from $$XARCH]" @echo ""
all: run
Kernel uses ARCH=powerpc for both 32-bit and 64-bit PowerPC, here adds a ppc variant for 32-bit PowerPC and uses it as the default variant of powerpc architecture.
Users can pass XARCH=ppc (or ARCH=powerpc) to test 32-bit PowerPC.
The default qemu-system-ppc g3beige machine [1] is used to run 32-bit powerpc kernel with pmac32_defconfig. The missing PMACZILOG serial tty and console are enabled in another patch [2].
Note,
- zImage doesn't boot due to "qemu-system-ppc: Some ROM regions are overlapping" error, so, vmlinux is used instead.
- since the VSX support may be disabled in kernel side, to avoid "illegal instruction" errors due to missing VSX kernel support, let's simply let compiler not generate vector/scalar (VSX) instructions via the '-mno-vsx' option.
- as 'man gcc' shows, '-mmultiple' is used to generate code that uses the load multiple word instructions and the store multiple word instructions. Those instructions do not work when the processor is in little-endian mode (except PPC740/PPC750), so, we only enable it for big endian powerpc.
[1]: https://qemu.readthedocs.io/en/latest/system/ppc/powermac.html [2]: https://lore.kernel.org/lkml/bb7b5f9958b3e3a20f6573ff7ce7c5dc566e7e32.169098...
Suggested-by: Willy Tarreau w@1wt.eu Link: https://lore.kernel.org/lkml/ZL9leVOI25S2+0+g@1wt.eu/ Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 46a2312b06f5..53f55aad8f5c 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -30,9 +30,11 @@ endif # architectures where both are equal.
# configure default variants for target kernel supported architectures +XARCH_powerpc = ppc XARCH = $(or $(XARCH_$(ARCH)),$(ARCH))
# map from user input variants to their kernel supported architectures +ARCH_ppc = powerpc ARCH := $(or $(ARCH_$(XARCH)),$(XARCH))
# kernel image names by architecture @@ -42,6 +44,7 @@ IMAGE_x86 = arch/x86/boot/bzImage IMAGE_arm64 = arch/arm64/boot/Image IMAGE_arm = arch/arm/boot/zImage IMAGE_mips = vmlinuz +IMAGE_ppc = vmlinux IMAGE_riscv = arch/riscv/boot/Image IMAGE_s390 = arch/s390/boot/bzImage IMAGE_loongarch = arch/loongarch/boot/vmlinuz.efi @@ -55,6 +58,7 @@ DEFCONFIG_x86 = defconfig DEFCONFIG_arm64 = defconfig DEFCONFIG_arm = multi_v7_defconfig DEFCONFIG_mips = malta_defconfig +DEFCONFIG_ppc = pmac32_defconfig DEFCONFIG_riscv = defconfig DEFCONFIG_s390 = defconfig DEFCONFIG_loongarch = defconfig @@ -70,6 +74,7 @@ QEMU_ARCH_x86 = x86_64 QEMU_ARCH_arm64 = aarch64 QEMU_ARCH_arm = arm QEMU_ARCH_mips = mipsel # works with malta_defconfig +QEMU_ARCH_ppc = ppc QEMU_ARCH_riscv = riscv64 QEMU_ARCH_s390 = s390x QEMU_ARCH_loongarch = loongarch64 @@ -82,6 +87,7 @@ QEMU_ARGS_x86 = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $( QEMU_ARGS_arm64 = -M virt -cpu cortex-a53 -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_arm = -M virt -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_mips = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" +QEMU_ARGS_ppc = -M g3beige -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_riscv = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_s390 = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_loongarch = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)" @@ -97,6 +103,7 @@ else Q=@ endif
+CFLAGS_ppc = -m32 -mbig-endian -Wl,-EB,-melf32ppc -mmultiple -mno-vsx CFLAGS_s390 = -m64 CFLAGS_mips = -EL CFLAGS_STACKPROTECTOR ?= $(call cc-option,-mstack-protector-guard=global $(call cc-option,-fstack-protector-all))
Kernel uses ARCH=powerpc for both 32-bit and 64-bit PowerPC, here adds a ppc64le variant for little endian 64-bit PowerPC, users can pass XARCH=ppc64le to test it.
The powernv machine of qemu-system-ppc64le is used for there is just a working powernv_defconfig.
As the document [1] shows:
PowerNV (as Non-Virtualized) is the “bare metal” platform using the OPAL firmware. It runs Linux on IBM and OpenPOWER systems and it can be used as an hypervisor OS, running KVM guests, or simply as a host OS.
Notes,
- since the VSX support may be disabled in kernel side, to avoid "illegal instruction" errors due to missing VSX kernel support, let's simply let compiler not generate vector/scalar (VSX) instructions via the '-mno-vsx' option.
- little endian ppc64 prefers elfv2 to elfv1 if the toolchain (e.g. gcc 13.1.0) supports it, let's align with kernel, otherwise, our elfv1 binary will not run on kernel with elfv2 ABI.
[1]: https://qemu.readthedocs.io/en/latest/system/ppc/powernv.html
Suggested-by: Willy Tarreau w@1wt.eu Link: https://lore.kernel.org/lkml/20230722120747.GC17311@1wt.eu/ Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 53f55aad8f5c..4d71393e75ef 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -35,6 +35,7 @@ XARCH = $(or $(XARCH_$(ARCH)),$(ARCH))
# map from user input variants to their kernel supported architectures ARCH_ppc = powerpc +ARCH_ppc64le = powerpc ARCH := $(or $(ARCH_$(XARCH)),$(XARCH))
# kernel image names by architecture @@ -45,6 +46,7 @@ IMAGE_arm64 = arch/arm64/boot/Image IMAGE_arm = arch/arm/boot/zImage IMAGE_mips = vmlinuz IMAGE_ppc = vmlinux +IMAGE_ppc64le = arch/powerpc/boot/zImage IMAGE_riscv = arch/riscv/boot/Image IMAGE_s390 = arch/s390/boot/bzImage IMAGE_loongarch = arch/loongarch/boot/vmlinuz.efi @@ -59,6 +61,7 @@ DEFCONFIG_arm64 = defconfig DEFCONFIG_arm = multi_v7_defconfig DEFCONFIG_mips = malta_defconfig DEFCONFIG_ppc = pmac32_defconfig +DEFCONFIG_ppc64le = powernv_defconfig DEFCONFIG_riscv = defconfig DEFCONFIG_s390 = defconfig DEFCONFIG_loongarch = defconfig @@ -75,6 +78,7 @@ QEMU_ARCH_arm64 = aarch64 QEMU_ARCH_arm = arm QEMU_ARCH_mips = mipsel # works with malta_defconfig QEMU_ARCH_ppc = ppc +QEMU_ARCH_ppc64le = ppc64le QEMU_ARCH_riscv = riscv64 QEMU_ARCH_s390 = s390x QEMU_ARCH_loongarch = loongarch64 @@ -88,6 +92,7 @@ QEMU_ARGS_arm64 = -M virt -cpu cortex-a53 -append "panic=-1 $(TEST:%=NOLIBC QEMU_ARGS_arm = -M virt -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_mips = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_ppc = -M g3beige -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" +QEMU_ARGS_ppc64le = -M powernv -append "console=hvc0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_riscv = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_s390 = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_loongarch = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)" @@ -104,6 +109,7 @@ Q=@ endif
CFLAGS_ppc = -m32 -mbig-endian -Wl,-EB,-melf32ppc -mmultiple -mno-vsx +CFLAGS_ppc64le = -m64 -mlittle-endian -Wl,-EL,-melf64lppc -mno-vsx $(call cc-option,-mabi=elfv2) CFLAGS_s390 = -m64 CFLAGS_mips = -EL CFLAGS_STACKPROTECTOR ?= $(call cc-option,-mstack-protector-guard=global $(call cc-option,-fstack-protector-all))
Kernel uses ARCH=powerpc for both 32-bit and 64-bit PowerPC, here adds a ppc64 variant for big endian 64-bit PowerPC, users can pass XARCH=ppc64 to test it.
The powernv machine of qemu-system-ppc64 is used with powernv_be_defconfig.
As the document [1] shows:
PowerNV (as Non-Virtualized) is the “bare metal” platform using the OPAL firmware. It runs Linux on IBM and OpenPOWER systems and it can be used as an hypervisor OS, running KVM guests, or simply as a host OS.
Notes,
- differs from little endian 64-bit PowerPC, vmlinux is used instead of zImage, because big endian zImage [2] only boot on qemu with x-vof=on (added from qemu v7.0) and a fixup patch [3] for qemu v7.0.51:
- since the VSX support may be disabled in kernel side, to avoid "illegal instruction" errors due to missing VSX kernel support, let's simply let compiler not generate vector/scalar (VSX) instructions via the '-mno-vsx' option.
- as 'man gcc' shows, '-mmultiple' is used to generate code that uses the load multiple word instructions and the store multiple word instructions. Those instructions do not work when the processor is in little-endian mode (except PPC740/PPC750), so, we only enable it for big endian powerpc.
- for big endian ppc64, as the help message from arch/powerpc/Kconfig shows, the V2 ABI is standard for 64-bit little-endian, but for big-endian it is less well tested by kernel and toolchain, so, use elfv1 as-is, no need to explicitly ask toolchain to use elfv2 here.
[1]: https://qemu.readthedocs.io/en/latest/system/ppc/powernv.html [2]: https://github.com/linuxppc/issues/issues/402 [3]: https://lore.kernel.org/qemu-devel/20220504065536.3534488-1-aik@ozlabs.ru/
Suggested-by: Willy Tarreau w@1wt.eu Link: https://lore.kernel.org/lkml/20230722121019.GD17311@1wt.eu/ Link: https://lore.kernel.org/lkml/20230719043353.GC5331@1wt.eu/ Reviewed-by: Thomas Weißschuh linux@weissschuh.net Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 4d71393e75ef..ec290cd82e21 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -35,6 +35,7 @@ XARCH = $(or $(XARCH_$(ARCH)),$(ARCH))
# map from user input variants to their kernel supported architectures ARCH_ppc = powerpc +ARCH_ppc64 = powerpc ARCH_ppc64le = powerpc ARCH := $(or $(ARCH_$(XARCH)),$(XARCH))
@@ -46,6 +47,7 @@ IMAGE_arm64 = arch/arm64/boot/Image IMAGE_arm = arch/arm/boot/zImage IMAGE_mips = vmlinuz IMAGE_ppc = vmlinux +IMAGE_ppc64 = vmlinux IMAGE_ppc64le = arch/powerpc/boot/zImage IMAGE_riscv = arch/riscv/boot/Image IMAGE_s390 = arch/s390/boot/bzImage @@ -61,6 +63,7 @@ DEFCONFIG_arm64 = defconfig DEFCONFIG_arm = multi_v7_defconfig DEFCONFIG_mips = malta_defconfig DEFCONFIG_ppc = pmac32_defconfig +DEFCONFIG_ppc64 = powernv_be_defconfig DEFCONFIG_ppc64le = powernv_defconfig DEFCONFIG_riscv = defconfig DEFCONFIG_s390 = defconfig @@ -78,6 +81,7 @@ QEMU_ARCH_arm64 = aarch64 QEMU_ARCH_arm = arm QEMU_ARCH_mips = mipsel # works with malta_defconfig QEMU_ARCH_ppc = ppc +QEMU_ARCH_ppc64 = ppc64 QEMU_ARCH_ppc64le = ppc64le QEMU_ARCH_riscv = riscv64 QEMU_ARCH_s390 = s390x @@ -92,6 +96,7 @@ QEMU_ARGS_arm64 = -M virt -cpu cortex-a53 -append "panic=-1 $(TEST:%=NOLIBC QEMU_ARGS_arm = -M virt -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_mips = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_ppc = -M g3beige -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" +QEMU_ARGS_ppc64 = -M powernv -append "console=hvc0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_ppc64le = -M powernv -append "console=hvc0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_riscv = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" QEMU_ARGS_s390 = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)" @@ -109,6 +114,7 @@ Q=@ endif
CFLAGS_ppc = -m32 -mbig-endian -Wl,-EB,-melf32ppc -mmultiple -mno-vsx +CFLAGS_ppc64 = -m64 -mbig-endian -Wl,-EB,-melf64ppc -mmultiple -mno-vsx CFLAGS_ppc64le = -m64 -mlittle-endian -Wl,-EL,-melf64lppc -mno-vsx $(call cc-option,-mabi=elfv2) CFLAGS_s390 = -m64 CFLAGS_mips = -EL
Some cross compilers may not just be prefixed with ARCH, customize them by architecture may simplify the test a lot, especially, when iterate with ARCH.
After customizing this for every architecture, the minimal test argument will be architecture itself, no CROSS_COMPILE required to be passed.
If the prefix of installed cross compiler is not the same as the one customized, we can also pass CROSS_COMPILE as before or even pass CROSS_COMPILE_<ARCH>.
Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index ec290cd82e21..6e34d364a177 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -55,6 +55,12 @@ IMAGE_loongarch = arch/loongarch/boot/vmlinuz.efi IMAGE = $(IMAGE_$(XARCH)) IMAGE_NAME = $(notdir $(IMAGE))
+# CROSS_COMPILE: cross toolchain prefix by architecture +CROSS_COMPILE ?= $(CROSS_COMPILE_$(XARCH)) + +# make sure CC is prefixed with CROSS_COMPILE +$(call allow-override,CC,$(CROSS_COMPILE)gcc) + # default kernel configurations that appear to be usable DEFCONFIG_i386 = defconfig DEFCONFIG_x86_64 = defconfig
The little-endian powerpc64le compilers provided by Ubuntu and Fedora are able to compile big endian kernel and big endian nolibc-test [1].
These default CROSS_COMPILE settings allow to test target architectures with:
$ cd /path/to/tools/testing/selftests/nolibc/
$ for arch in ppc ppc64 ppc64le; do \ make run-user XARCH=$arch | grep "status: "; \ done
If want to use another cross compiler, please simply pass CROSS_COMPILE or CC as before.
For example, it is able to build 64-bit nolibc-test with the big endian powerpc64-linux-gcc crosstool from [2]:
$ wget -c https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/13.1.0/... $ tar xvf x86_64-gcc-13.1.0-nolibc-powerpc64-linux.tar.xz $ export PATH=$PWD/gcc-13.1.0-nolibc/powerpc64-linux/bin/:$PATH
$ export CROSS_COMPILE_ppc=powerpc64-linux- $ export CROSS_COMPILE_ppc64=powerpc64-linux- $ export CROSS_COMPILE_ppc64le=powerpc64-linux- $ for arch in ppc ppc64 ppc64le; do \ make run-user XARCH=$arch | grep "status: "; \ done
Or specify CC directly with full path:
$ export CC=$PWD/gcc-13.1.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc $ for arch in ppc ppc64 ppc64le; do \ make run-user XARCH=$arch | grep "status: "; \ done
[1]: https://github.com/open-power/skiboot [2]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
Signed-off-by: Zhangjin Wu falcon@tinylab.org --- tools/testing/selftests/nolibc/Makefile | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile index 6e34d364a177..16cc33db54e3 100644 --- a/tools/testing/selftests/nolibc/Makefile +++ b/tools/testing/selftests/nolibc/Makefile @@ -56,6 +56,9 @@ IMAGE = $(IMAGE_$(XARCH)) IMAGE_NAME = $(notdir $(IMAGE))
# CROSS_COMPILE: cross toolchain prefix by architecture +CROSS_COMPILE_ppc ?= powerpc-linux-gnu- +CROSS_COMPILE_ppc64 ?= powerpc64le-linux-gnu- +CROSS_COMPILE_ppc64le ?= powerpc64le-linux-gnu- CROSS_COMPILE ?= $(CROSS_COMPILE_$(XARCH))
# make sure CC is prefixed with CROSS_COMPILE
linux-kselftest-mirror@lists.linaro.org