- A new output binary (board.axf) has been added exclusively for board usage, which uses a different linker script (board.lds.S). - BootMonitor has already placed secondary CPUs in wfi state, generate a software interrupt first and point other cores to jump to the right location by writing the address to SYS_FLAGS. - Make room for atags, which are populated from BootMonitor's loader, at memory address 0x80000100. - Code relocation for secondary cores in wfe state, has been moved further back to a safer location than the initrd region.
Usage example in BootMonitor (atags passing idea by Marc Zyngier): - load zImageDT LOAD_ADRESS 0x80008000 - flash linux boot board earlyprintk console=ttyAMA0 mem=...
Or run the image itself, with default atags being set via config.mk: - load zImageDT LOAD_ADDRESS 0x80008000 - flash run board
Tested on: - Dual/single cluster FastModels - V2P-CA15-TC1 CoreTile - V2P-CA15-A7-TC2 CoreTile (single cluster and big.LITTLE operation)
Signed-off-by: Alexander Spyridakis a.spyridakis@virtualopensystems.com --- Makefile | 19 +++++++++--- board.lds.S | 21 +++++++++++++ boot.S | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++--- config-default.mk | 2 +- 4 files changed, 118 insertions(+), 9 deletions(-) create mode 100644 board.lds.S
diff --git a/Makefile b/Makefile index 0997dcf..c0f10ca 100644 --- a/Makefile +++ b/Makefile @@ -21,8 +21,9 @@ KERNEL = uImage
IMAGE = linux-system.axf SEMIIMG = linux-system-semi.axf +BOARD = board.axf LD_SCRIPT = model.lds.S - +B_LD_SCRIPT = board.lds.S
CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld @@ -31,14 +32,15 @@ LD = $(CROSS_COMPILE)ld export CROSS_COMPILE ARCH
# Build all wrappers -all: $(IMAGE) $(SEMIIMG) +all: $(IMAGE) $(SEMIIMG) $(BOARD)
# Build just the semihosting wrapper semi: $(SEMIIMG)
clean distclean: - rm -f $(IMAGE) $(SEMIIMG) \ - model.lds modelsemi.lds $(OBJS) $(KERNEL) + rm -f $(IMAGE) $(SEMIIMG) $(BOARD) \ + model.lds modelsemi.lds board.lds \ + board.o $(OBJS) $(KERNEL)
$(KERNEL): $(KERNEL_SRC)/arch/arm/boot/uImage cp $< $@ @@ -49,9 +51,15 @@ $(IMAGE): $(OBJS) model.lds $(KERNEL) $(FILESYSTEM) Makefile $(SEMIIMG): $(OBJS) modelsemi.lds $(LD) -o $@ $(OBJS) --script=modelsemi.lds
+$(BOARD): board.o board.lds Makefile + $(LD) -o board.axf --script=board.lds + boot.o: $(BOOTLOADER) $(CC) $(CPPFLAGS) -DKCMD='$(KCMD)' -c -o $@ $<
+board.o: $(BOOTLOADER) + $(CC) $(CPPFLAGS) -DKCMD='$(KCMD)' -DVE_BOARD -c -o $@ $< + %.o: %.c $(CC) $(CPPFLAGS) -O2 -ffreestanding -I. -Ilibfdt -c -o $@ $<
@@ -61,6 +69,9 @@ model.lds: $(LD_SCRIPT) Makefile modelsemi.lds: $(LD_SCRIPT) Makefile $(CC) $(CPPFLAGS) -DSEMIHOSTING=1 -E -P -C -o $@ $<
+board.lds: $(B_LD_SCRIPT) Makefile + $(CC) $(CPPFLAGS) -DBOARD -E -P -C -o $@ $< + $(KERNEL_SRC)/arch/arm/boot/uImage: force $(MAKE) -C $(KERNEL_SRC) -j4 uImage
diff --git a/board.lds.S b/board.lds.S new file mode 100644 index 0000000..560f65e --- /dev/null +++ b/board.lds.S @@ -0,0 +1,21 @@ +OUTPUT_FORMAT("elf32-littlearm") +OUTPUT_ARCH(arm) +TARGET(binary) + +INPUT(./board.o) + +PHYS_OFFSET = 0x80000000; +MON_OFFSET = 0xf0000000; + +SECTIONS +{ + . = PHYS_OFFSET; + + .text : { board.o } + + . = PHYS_OFFSET + 0x8000; + kernel = .; + + . = PHYS_OFFSET + 0x00d00000; + fs_start = .; +} diff --git a/boot.S b/boot.S index dd453e3..5d6e6a1 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,9 @@ .arch_extension virt .text
+ .equ GICD_CTLR, 0x2c001000 + .equ VE_SYS_FLAGS, 0x1c010030 + .macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -68,11 +71,60 @@ vectors: @ ...and return to calling code in NS state movs pc, lr
- .globl start start: #ifdef SMP #ifdef VEXPRESS +#ifdef VE_BOARD + @ On the VE board only CPU0 should be awake at this stage + @ BootMonitor's linux loader, will pass machine id and atags to r1 and r2 + @ Save those two registers, instead of passing our own defaults + cmp r2, #0 + ldrne r0, =bootregs + stmne r0, {r1, r2} + + @ Update SYS_FLAGS with the address that secondary cores will jump to + ldr r0, =VE_SYS_FLAGS + ldr r1, =SEC_CORE_ENTRY + str r1, [r0] + + @ Generate Software Interrupt to wake secondary cores + ldr r0, =GICD_CTLR + ldr r1, =(1 << 24) + str r1, [r0, #0xf00] + + b SEC_CORE_ENTRY + +.org 0x100 +atags: + @ ATAG_CORE + .long 2 + .long 0x54410001 + + @ ATAG_CMDLINE + .long (1f - .) >> 2 + .long 0x54410009 + /* The kernel boot command line is defined in config.mk */ + .asciz KCMD + .align 2 +1: + +#ifdef USE_INITRD + @ ATAG_INITRD2 + .long 4 + .long 0x54420005 + .long fs_start + .long FS_SIZE +#endif + + @ ATAG_NONE + .long 0 + .long 0x00000000 + +@ BootMonitor will write the atags at 0x80000100, move code as far as we can. +.org 0xe00 +SEC_CORE_ENTRY: +#endif @ @ Program architected timer frequency @ @@ -95,7 +147,7 @@ start: @
@ Set all interrupts to be non-secure - ldr r0, =0x2c001000 @ Dist GIC base + ldr r0, =GICD_CTLR @ Dist GIC base ldr r1, [r0, #0x04] @ Type Register cmp r4, #0 andeq r1, r1, #0x1f @@ -130,12 +182,19 @@ start: @ enter_hyp
- ldr r1, =fs_start - 0x100 +#ifdef VE_BOARD + @ Clear SYS_FLAGS to let the kernel wake the rest of the cores properly + ldr r0, =VE_SYS_FLAGS + mvn r1, #0 + str r1, [r0, 0x04] +#endif + + ldr r1, =MON_OFFSET - 0x100 adr r2, 1f ldmia r2, {r3 - r7} @ move the code to a location stmia r1, {r3 - r7} @ less likely to be overridden #ifdef VEXPRESS - ldr r0, =0x1c010030 @ VE SYS_FLAGS register + ldr r0, =VE_SYS_FLAGS @ VE SYS_FLAGS register #else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif @@ -151,6 +210,7 @@ start: #endif
2: +#ifndef VE_BOARD @ @ UART initialisation (38400 8N1) @ @@ -214,3 +274,20 @@ kernel_cmd: .asciz KCMD #endif kernel_cmd_end: +#else /* VE_BOARD */ + enter_hyp + + @ + @ Kernel parameters + @ + ldr r0, =bootregs + ldm r0, {r1, r2} + mov r0, #0 + mov r3, #0 + ldr lr, =kernel + mov pc, lr @ jump to the kernel + +bootregs: + .long 2272 @ r1 + .long atags @ r2 +#endif /* VE_BOARD */ diff --git a/config-default.mk b/config-default.mk index 6c73934..0ba1a06 100644 --- a/config-default.mk +++ b/config-default.mk @@ -28,7 +28,7 @@ SYSTEM ?= vexpress # Turn this on to use an initrd whose contents are in filesystem.cpio.gz USE_INITRD ?= no ifeq ($(USE_INITRD),yes) -CPPFLAGS += -DUSE_INITRD +CPPFLAGS += -DUSE_INITRD -DFS_SIZE=$(shell stat -c %s $(FILESYSTEM)) FILESYSTEM ?= filesystem.cpio.gz else FILESYSTEM =
Hi Alexander,
Could you explain with a bit more detail what's the intended usage scenario for this code?
If you have BootMonitor, it is already capable of booting the kernel.
Regards, Liviu
On Wed, Dec 12, 2012 at 08:35:10PM +0000, Alexander Spyridakis wrote:
- A new output binary (board.axf) has been added exclusively for board usage, which uses a different linker script (board.lds.S).
- BootMonitor has already placed secondary CPUs in wfi state, generate a software interrupt first and point other cores to jump to the right location by writing the address to SYS_FLAGS.
- Make room for atags, which are populated from BootMonitor's loader, at memory address 0x80000100.
- Code relocation for secondary cores in wfe state, has been moved further back to a safer location than the initrd region.
Usage example in BootMonitor (atags passing idea by Marc Zyngier):
- load zImageDT LOAD_ADRESS 0x80008000
- flash linux boot board earlyprintk console=ttyAMA0 mem=...
Or run the image itself, with default atags being set via config.mk:
- load zImageDT LOAD_ADDRESS 0x80008000
- flash run board
Tested on:
- Dual/single cluster FastModels
- V2P-CA15-TC1 CoreTile
- V2P-CA15-A7-TC2 CoreTile (single cluster and big.LITTLE operation)
Signed-off-by: Alexander Spyridakis a.spyridakis@virtualopensystems.com
Makefile | 19 +++++++++--- board.lds.S | 21 +++++++++++++ boot.S | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++--- config-default.mk | 2 +- 4 files changed, 118 insertions(+), 9 deletions(-) create mode 100644 board.lds.S
diff --git a/Makefile b/Makefile index 0997dcf..c0f10ca 100644 --- a/Makefile +++ b/Makefile @@ -21,8 +21,9 @@ KERNEL = uImage IMAGE = linux-system.axf SEMIIMG = linux-system-semi.axf +BOARD = board.axf LD_SCRIPT = model.lds.S
+B_LD_SCRIPT = board.lds.S CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld @@ -31,14 +32,15 @@ LD = $(CROSS_COMPILE)ld export CROSS_COMPILE ARCH # Build all wrappers -all: $(IMAGE) $(SEMIIMG) +all: $(IMAGE) $(SEMIIMG) $(BOARD) # Build just the semihosting wrapper semi: $(SEMIIMG) clean distclean:
- rm -f $(IMAGE) $(SEMIIMG) \
- model.lds modelsemi.lds $(OBJS) $(KERNEL)
- rm -f $(IMAGE) $(SEMIIMG) $(BOARD) \
- model.lds modelsemi.lds board.lds \
- board.o $(OBJS) $(KERNEL)
$(KERNEL): $(KERNEL_SRC)/arch/arm/boot/uImage cp $< $@ @@ -49,9 +51,15 @@ $(IMAGE): $(OBJS) model.lds $(KERNEL) $(FILESYSTEM) Makefile $(SEMIIMG): $(OBJS) modelsemi.lds $(LD) -o $@ $(OBJS) --script=modelsemi.lds +$(BOARD): board.o board.lds Makefile
- $(LD) -o board.axf --script=board.lds
boot.o: $(BOOTLOADER) $(CC) $(CPPFLAGS) -DKCMD='$(KCMD)' -c -o $@ $< +board.o: $(BOOTLOADER)
- $(CC) $(CPPFLAGS) -DKCMD='$(KCMD)' -DVE_BOARD -c -o $@ $<
%.o: %.c $(CC) $(CPPFLAGS) -O2 -ffreestanding -I. -Ilibfdt -c -o $@ $< @@ -61,6 +69,9 @@ model.lds: $(LD_SCRIPT) Makefile modelsemi.lds: $(LD_SCRIPT) Makefile $(CC) $(CPPFLAGS) -DSEMIHOSTING=1 -E -P -C -o $@ $< +board.lds: $(B_LD_SCRIPT) Makefile
- $(CC) $(CPPFLAGS) -DBOARD -E -P -C -o $@ $<
$(KERNEL_SRC)/arch/arm/boot/uImage: force $(MAKE) -C $(KERNEL_SRC) -j4 uImage diff --git a/board.lds.S b/board.lds.S new file mode 100644 index 0000000..560f65e --- /dev/null +++ b/board.lds.S @@ -0,0 +1,21 @@ +OUTPUT_FORMAT("elf32-littlearm") +OUTPUT_ARCH(arm) +TARGET(binary)
+INPUT(./board.o)
+PHYS_OFFSET = 0x80000000; +MON_OFFSET = 0xf0000000;
+SECTIONS +{
- . = PHYS_OFFSET;
- .text : { board.o }
- . = PHYS_OFFSET + 0x8000;
- kernel = .;
- . = PHYS_OFFSET + 0x00d00000;
- fs_start = .;
+} diff --git a/boot.S b/boot.S index dd453e3..5d6e6a1 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,9 @@ .arch_extension virt .text
- .equ GICD_CTLR, 0x2c001000
- .equ VE_SYS_FLAGS, 0x1c010030
.macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -68,11 +71,60 @@ vectors: @ ...and return to calling code in NS state movs pc, lr
- .globl start
start: #ifdef SMP #ifdef VEXPRESS +#ifdef VE_BOARD
- @ On the VE board only CPU0 should be awake at this stage
- @ BootMonitor's linux loader, will pass machine id and atags to r1 and r2
- @ Save those two registers, instead of passing our own defaults
- cmp r2, #0
- ldrne r0, =bootregs
- stmne r0, {r1, r2}
- @ Update SYS_FLAGS with the address that secondary cores will jump to
- ldr r0, =VE_SYS_FLAGS
- ldr r1, =SEC_CORE_ENTRY
- str r1, [r0]
- @ Generate Software Interrupt to wake secondary cores
- ldr r0, =GICD_CTLR
- ldr r1, =(1 << 24)
- str r1, [r0, #0xf00]
- b SEC_CORE_ENTRY
+.org 0x100 +atags:
- @ ATAG_CORE
- .long 2
- .long 0x54410001
- @ ATAG_CMDLINE
- .long (1f - .) >> 2
- .long 0x54410009
/* The kernel boot command line is defined in config.mk */.asciz KCMD- .align 2
+1:
+#ifdef USE_INITRD
- @ ATAG_INITRD2
- .long 4
- .long 0x54420005
- .long fs_start
- .long FS_SIZE
+#endif
- @ ATAG_NONE
- .long 0
- .long 0x00000000
+@ BootMonitor will write the atags at 0x80000100, move code as far as we can. +.org 0xe00 +SEC_CORE_ENTRY: +#endif @ @ Program architected timer frequency @ @@ -95,7 +147,7 @@ start: @ @ Set all interrupts to be non-secure
- ldr r0, =0x2c001000 @ Dist GIC base
- ldr r0, =GICD_CTLR @ Dist GIC base ldr r1, [r0, #0x04] @ Type Register cmp r4, #0 andeq r1, r1, #0x1f
@@ -130,12 +182,19 @@ start: @ enter_hyp
- ldr r1, =fs_start - 0x100
+#ifdef VE_BOARD
- @ Clear SYS_FLAGS to let the kernel wake the rest of the cores properly
- ldr r0, =VE_SYS_FLAGS
- mvn r1, #0
- str r1, [r0, 0x04]
+#endif
- ldr r1, =MON_OFFSET - 0x100 adr r2, 1f ldmia r2, {r3 - r7} @ move the code to a location stmia r1, {r3 - r7} @ less likely to be overridden
#ifdef VEXPRESS
- ldr r0, =0x1c010030 @ VE SYS_FLAGS register
- ldr r0, =VE_SYS_FLAGS @ VE SYS_FLAGS register
#else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif @@ -151,6 +210,7 @@ start: #endif 2: +#ifndef VE_BOARD @ @ UART initialisation (38400 8N1) @ @@ -214,3 +274,20 @@ kernel_cmd: .asciz KCMD #endif kernel_cmd_end: +#else /* VE_BOARD */
- enter_hyp
- @
- @ Kernel parameters
- @
- ldr r0, =bootregs
- ldm r0, {r1, r2}
- mov r0, #0
- mov r3, #0
- ldr lr, =kernel
- mov pc, lr @ jump to the kernel
+bootregs:
- .long 2272 @ r1
- .long atags @ r2
+#endif /* VE_BOARD */ diff --git a/config-default.mk b/config-default.mk index 6c73934..0ba1a06 100644 --- a/config-default.mk +++ b/config-default.mk @@ -28,7 +28,7 @@ SYSTEM ?= vexpress # Turn this on to use an initrd whose contents are in filesystem.cpio.gz USE_INITRD ?= no ifeq ($(USE_INITRD),yes) -CPPFLAGS += -DUSE_INITRD +CPPFLAGS += -DUSE_INITRD -DFS_SIZE=$(shell stat -c %s $(FILESYSTEM)) FILESYSTEM ?= filesystem.cpio.gz else FILESYSTEM = -- 1.7.9.5
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On 18 December 2012 12:10, Liviu Dudau Liviu.Dudau@arm.com wrote:
Hi Alexander,
Could you explain with a bit more detail what's the intended usage scenario for this code?
If you have BootMonitor, it is already capable of booting the kernel.
The major difference is that BootMonitor doesn't initialize Hyp mode and without it we can't start KVM.
Initially (back in the TC1 era), I tried to run linux-system.axf as is, but BootMonitor wasn't (and still isn't I think) able to execute elf files with multiple program headers. The solution for me at the time was to load the kernel image separately and run a simplified version of the bootwrapper. Of course this would only work for CPU0 as the rest of the cores are already set to sleep by BootMonitor (contrary to FastModels).
This patch tries to unify usage of the bootwrapper for both FastModels and the Versatile Express board, without being intrusive to the FastModels code path. Actually any changes to boot.S are bound for the board binary only.
Regards.
On 18 December 2012 16:42, Alexander Spyridakis a.spyridakis@virtualopensystems.com wrote:
On 18 December 2012 12:10, Liviu Dudau Liviu.Dudau@arm.com wrote:
If you have BootMonitor, it is already capable of booting the kernel.
The major difference is that BootMonitor doesn't initialize Hyp mode and without it we can't start KVM.
It would be good to (at some point) enhance BootMonitor so it does boot kernels in Hyp mode, incidentally.
-- PMM
On Tue, Dec 18, 2012 at 04:42:08PM +0000, Alexander Spyridakis wrote:
On 18 December 2012 12:10, Liviu Dudau <Liviu.Dudau@arm.commailto:Liviu.Dudau@arm.com> wrote: Hi Alexander,
Could you explain with a bit more detail what's the intended usage scenario for this code?
If you have BootMonitor, it is already capable of booting the kernel.
The major difference is that BootMonitor doesn't initialize Hyp mode and without it we can't start KVM.
Initially (back in the TC1 era), I tried to run linux-system.axf as is, but BootMonitor wasn't (and still isn't I think) able to execute elf files with multiple program headers. The solution for me at the time was to load the kernel image separately and run a simplified version of the bootwrapper. Of course this would only work for CPU0 as the rest of the cores are already set to sleep by BootMonitor (contrary to FastModels).
This patch tries to unify usage of the bootwrapper for both FastModels and the Versatile Express board, without being intrusive to the FastModels code path. Actually any changes to boot.S are bound for the board binary only.
Regards.
Hi Alexander,
Thanks for the explanation.
As for having an unified boot process, using UEFI it's more likely to lead to that rather than waiting on BootMon to support HYP mode.
Regards, Liviu
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
-
Alexander Spyridakis -
Liviu Dudau -
Peter Maydell