To enable CPU hotplug the need to provide some boot code at the reset vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to place the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
We then modify the boot protocol slightly to allow hot-plugging of any CPU, including CPU #0, when the system is already booted. This is done by checking if SYS_FLAGS is already set before the normal check for CPU0 and the boot-or-wait decision made.
This patch is based on work by Nicolas Pitre.
Signed-off-by: Nicolas Pitre nico@linaro.org Signed-off-by: Jon Medhurst tixy@linaro.org --- boot.S | 41 ++++++++++++++++++++++++----------------- model.lds.S | 3 ++- 2 files changed, 26 insertions(+), 18 deletions(-)
diff --git a/boot.S b/boot.S index dd453e3..fb56693 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,8 @@ .arch_extension virt .text
+ b start @ Must be first instruction for power on reset vector + .macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -119,27 +121,32 @@ start: orr r0, r0, r1 mcr p15, 0, r0, c1, c1, 2
- @ Check CPU nr again - mrc p15, 0, r0, c0, c0, 5 @ MPIDR (ARMv7 only) - bfc r0, #24, #8 @ CPU number, taking multicluster into account - cmp r0, #0 @ primary CPU? - beq 2f - - @ - @ Secondary CPUs (following the RealView SMP booting protocol) - @ - enter_hyp - - ldr r1, =fs_start - 0x100 - adr r2, 1f - ldmia r2, {r3 - r7} @ move the code to a location - stmia r1, {r3 - r7} @ less likely to be overridden + /* + * If SYS_FLAGS is already set, this is a warm boot and we blindly + * branch to the indicated address right away, irrespective of the + * CPU we are. + */ #ifdef VEXPRESS ldr r0, =0x1c010030 @ VE SYS_FLAGS register #else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif - mov pc, r1 @ branch to the relocated code + ldr r1, [r0] + cmp r1, #0 + beq 1f + enter_hyp + bx r1 +1: + /* + * Otherwise this is a cold boot. In this case it depends if + * we are the primary CPU or not. The primary CPU boots the system + * while the secondaries wait for the primary to set SYS_FLAGS. + */ + mrc p15, 0, r1, c0, c0, 5 + bics r1, #0xff000000 + beq 2f + + enter_hyp 1: #ifdef VEXPRESS wfe @@ -147,7 +154,7 @@ start: ldr r1, [r0] cmp r1, #0 beq 1b - mov pc, r1 @ branch to the given address + bx r1 @ branch to the given address #endif
2: diff --git a/model.lds.S b/model.lds.S index 793df89..f37824e 100644 --- a/model.lds.S +++ b/model.lds.S @@ -27,7 +27,8 @@ STACKTOP = 0xff000000;
SECTIONS { - . = PHYS_OFFSET; + . = 0; + .boot : { boot.o }
. = PHYS_OFFSET + 0x8000 - 0x40;
Adding some named people to the 'To' list to try and elicit review comments... :-)
On Fri, 2012-12-07 at 11:35 +0000, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the reset vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to place the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
We then modify the boot protocol slightly to allow hot-plugging of any CPU, including CPU #0, when the system is already booted. This is done by checking if SYS_FLAGS is already set before the normal check for CPU0 and the boot-or-wait decision made.
This patch is based on work by Nicolas Pitre.
Signed-off-by: Nicolas Pitre nico@linaro.org Signed-off-by: Jon Medhurst tixy@linaro.org
boot.S | 41 ++++++++++++++++++++++++----------------- model.lds.S | 3 ++- 2 files changed, 26 insertions(+), 18 deletions(-)
diff --git a/boot.S b/boot.S index dd453e3..fb56693 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,8 @@ .arch_extension virt .text
- b start @ Must be first instruction for power on reset vector
.macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -119,27 +121,32 @@ start: orr r0, r0, r1 mcr p15, 0, r0, c1, c1, 2
- @ Check CPU nr again
- mrc p15, 0, r0, c0, c0, 5 @ MPIDR (ARMv7 only)
- bfc r0, #24, #8 @ CPU number, taking multicluster into account
- cmp r0, #0 @ primary CPU?
- beq 2f
- @
- @ Secondary CPUs (following the RealView SMP booting protocol)
- @
- enter_hyp
- ldr r1, =fs_start - 0x100
- adr r2, 1f
- ldmia r2, {r3 - r7} @ move the code to a location
- stmia r1, {r3 - r7} @ less likely to be overridden
- /*
* If SYS_FLAGS is already set, this is a warm boot and we blindly* branch to the indicated address right away, irrespective of the* CPU we are.*/#ifdef VEXPRESS ldr r0, =0x1c010030 @ VE SYS_FLAGS register #else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif
- mov pc, r1 @ branch to the relocated code
- ldr r1, [r0]
- cmp r1, #0
- beq 1f
- enter_hyp
- bx r1
+1:
- /*
* Otherwise this is a cold boot. In this case it depends if* we are the primary CPU or not. The primary CPU boots the system* while the secondaries wait for the primary to set SYS_FLAGS.*/- mrc p15, 0, r1, c0, c0, 5
- bics r1, #0xff000000
- beq 2f
- enter_hyp
1: #ifdef VEXPRESS wfe @@ -147,7 +154,7 @@ start: ldr r1, [r0] cmp r1, #0 beq 1b
- mov pc, r1 @ branch to the given address
- bx r1 @ branch to the given address
#endif 2: diff --git a/model.lds.S b/model.lds.S index 793df89..f37824e 100644 --- a/model.lds.S +++ b/model.lds.S @@ -27,7 +27,8 @@ STACKTOP = 0xff000000; SECTIONS {
- . = PHYS_OFFSET;
- . = 0;
- .boot : { boot.o }
. = PHYS_OFFSET + 0x8000 - 0x40;
Hi,
I will get back to you on this one at the end of the week - hang in there:)
-Christoffer
On Tue, Dec 11, 2012 at 4:52 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
Adding some named people to the 'To' list to try and elicit review comments... :-)
On Fri, 2012-12-07 at 11:35 +0000, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the reset vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to place the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
We then modify the boot protocol slightly to allow hot-plugging of any CPU, including CPU #0, when the system is already booted. This is done by checking if SYS_FLAGS is already set before the normal check for CPU0 and the boot-or-wait decision made.
This patch is based on work by Nicolas Pitre.
Signed-off-by: Nicolas Pitre nico@linaro.org Signed-off-by: Jon Medhurst tixy@linaro.org
boot.S | 41 ++++++++++++++++++++++++----------------- model.lds.S | 3 ++- 2 files changed, 26 insertions(+), 18 deletions(-)
diff --git a/boot.S b/boot.S index dd453e3..fb56693 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,8 @@ .arch_extension virt .text
b start @ Must be first instruction for power on reset vector.macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -119,27 +121,32 @@ start: orr r0, r0, r1 mcr p15, 0, r0, c1, c1, 2
@ Check CPU nr againmrc p15, 0, r0, c0, c0, 5 @ MPIDR (ARMv7 only)bfc r0, #24, #8 @ CPU number, taking multicluster into accountcmp r0, #0 @ primary CPU?beq 2f@@ Secondary CPUs (following the RealView SMP booting protocol)@enter_hypldr r1, =fs_start - 0x100adr r2, 1fldmia r2, {r3 - r7} @ move the code to a locationstmia r1, {r3 - r7} @ less likely to be overridden
/** If SYS_FLAGS is already set, this is a warm boot and we blindly* branch to the indicated address right away, irrespective of the* CPU we are.*/#ifdef VEXPRESS ldr r0, =0x1c010030 @ VE SYS_FLAGS register #else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif
mov pc, r1 @ branch to the relocated code
ldr r1, [r0]cmp r1, #0beq 1fenter_hypbx r1+1:
/** Otherwise this is a cold boot. In this case it depends if* we are the primary CPU or not. The primary CPU boots the system* while the secondaries wait for the primary to set SYS_FLAGS.*/mrc p15, 0, r1, c0, c0, 5bics r1, #0xff000000beq 2fenter_hyp1: #ifdef VEXPRESS wfe @@ -147,7 +154,7 @@ start: ldr r1, [r0] cmp r1, #0 beq 1b
mov pc, r1 @ branch to the given address
bx r1 @ branch to the given address#endif
2: diff --git a/model.lds.S b/model.lds.S index 793df89..f37824e 100644 --- a/model.lds.S +++ b/model.lds.S @@ -27,7 +27,8 @@ STACKTOP = 0xff000000;
SECTIONS {
- . = PHYS_OFFSET;
. = 0;
.boot : { boot.o }
. = PHYS_OFFSET + 0x8000 - 0x40;
kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the reset vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to place the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0 and that the first part of the physical memory space is reserved for secure access?
Otherwise I can't see any issues with this code.
Acked-by: Christoffer Dall c.dall@virtualopensystems.com
We then modify the boot protocol slightly to allow hot-plugging of any CPU, including CPU #0, when the system is already booted. This is done by checking if SYS_FLAGS is already set before the normal check for CPU0 and the boot-or-wait decision made.
This patch is based on work by Nicolas Pitre.
Signed-off-by: Nicolas Pitre <nico@linaro.org javascript:;> Signed-off-by: Jon Medhurst <tixy@linaro.org javascript:;>
boot.S | 41 ++++++++++++++++++++++++----------------- model.lds.S | 3 ++- 2 files changed, 26 insertions(+), 18 deletions(-)
diff --git a/boot.S b/boot.S index dd453e3..fb56693 100644 --- a/boot.S +++ b/boot.S @@ -12,6 +12,8 @@ .arch_extension virt .text
b start @ Must be first instruction for power on resetvector
.macro enter_hyp @ We assume we're entered in Secure Supervisor mode. To @ get to Hyp mode we have to pass through Monitor mode @@ -119,27 +121,32 @@ start: orr r0, r0, r1 mcr p15, 0, r0, c1, c1, 2
@ Check CPU nr againmrc p15, 0, r0, c0, c0, 5 @ MPIDR (ARMv7 only)bfc r0, #24, #8 @ CPU number, takingmulticluster into account
cmp r0, #0 @ primary CPU?beq 2f@@ Secondary CPUs (following the RealView SMP booting protocol)@enter_hypldr r1, =fs_start - 0x100adr r2, 1fldmia r2, {r3 - r7} @ move the code to alocation
stmia r1, {r3 - r7} @ less likely to beoverridden
/** If SYS_FLAGS is already set, this is a warm boot and we blindly* branch to the indicated address right away, irrespective of the* CPU we are.*/#ifdef VEXPRESS ldr r0, =0x1c010030 @ VE SYS_FLAGS register #else ldr r0, =0x10000030 @ RealView SYS_FLAGS register #endif
mov pc, r1 @ branch to the relocatedcode
ldr r1, [r0]cmp r1, #0beq 1fenter_hypbx r1+1:
/** Otherwise this is a cold boot. In this case it depends if* we are the primary CPU or not. The primary CPU boots the system* while the secondaries wait for the primary to set SYS_FLAGS.*/mrc p15, 0, r1, c0, c0, 5bics r1, #0xff000000beq 2fenter_hyp1: #ifdef VEXPRESS wfe @@ -147,7 +154,7 @@ start: ldr r1, [r0] cmp r1, #0 beq 1b
mov pc, r1 @ branch to the givenaddress
bx r1 @ branch to the givenaddress #endif
2: diff --git a/model.lds.S b/model.lds.S index 793df89..f37824e 100644 --- a/model.lds.S +++ b/model.lds.S @@ -27,7 +27,8 @@ STACKTOP = 0xff000000;
SECTIONS {
- . = PHYS_OFFSET;
. = 0;
.boot : { boot.o }
. = PHYS_OFFSET + 0x8000 - 0x40;
-- 1.7.10.4
kvmarm mailing list kvmarm@lists.cs.columbia.edu javascript:; https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
Otherwise I can't see any issues with this code.
Acked-by: Christoffer Dall c.dall@virtualopensystems.com
Thanks for taking the time to review this.
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
-Christoffer
On Fri, 2012-12-14 at 10:10 -0500, Christoffer Dall wrote:
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
In the RTSM manual, the first 64Mb of physical memory space is marked as "NOR FLASH0", as it is on real hardware. This is flash memory which is read-only under normal circumstances, and not anything Linux would go poking about in, unless you configured an MTD device and used it for a file system.
DRAM is in the top 2GB and is what Linux uses for memory, and where most of the bootwrapper still lives. The only bit I've relocated to the bottom of the address space is everything in boot.S which includes all the bits we need to survive after Linux boots in order to handle a cpu startup again after being powered down then up again.
Are you convinced, or do you still have concerns?
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
On 14/12/12 15:54, Jon Medhurst (Tixy) wrote:
On Fri, 2012-12-14 at 10:10 -0500, Christoffer Dall wrote:
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
In the RTSM manual, the first 64Mb of physical memory space is marked as "NOR FLASH0", as it is on real hardware. This is flash memory which is read-only under normal circumstances, and not anything Linux would go poking about in, unless you configured an MTD device and used it for a file system.
DRAM is in the top 2GB and is what Linux uses for memory, and where most of the bootwrapper still lives. The only bit I've relocated to the bottom of the address space is everything in boot.S which includes all the bits we need to survive after Linux boots in order to handle a cpu startup again after being powered down then up again.
Are you convinced, or do you still have concerns?
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
In the horrible hack that I use to boot both TC2 and RTSM, I use the static RAM located at 0x2E000000. It is only 64kB though.
M.
On Fri, 2012-12-14 at 16:00 +0000, Marc Zyngier wrote:
On 14/12/12 15:54, Jon Medhurst (Tixy) wrote:
On Fri, 2012-12-14 at 10:10 -0500, Christoffer Dall wrote:
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
In the RTSM manual, the first 64Mb of physical memory space is marked as "NOR FLASH0", as it is on real hardware. This is flash memory which is read-only under normal circumstances, and not anything Linux would go poking about in, unless you configured an MTD device and used it for a file system.
DRAM is in the top 2GB and is what Linux uses for memory, and where most of the bootwrapper still lives. The only bit I've relocated to the bottom of the address space is everything in boot.S which includes all the bits we need to survive after Linux boots in order to handle a cpu startup again after being powered down then up again.
Are you convinced, or do you still have concerns?
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
In the horrible hack that I use to boot both TC2 and RTSM, I use the static RAM located at 0x2E000000. It is only 64kB though.
We also want to get some code located so it gets executed when the core is reset (to support cpu hotplug), that's why locating the code to address zero was used. Or is there some magic 'redirect reset address' register?
Perhaps we need to create a separate NOR flash image to give RTSM to support this or reconsider whether it's worth the effort to get RTSM supporting cpu hotplug at all.
On Fri, 14 Dec 2012, Marc Zyngier wrote:
On 14/12/12 15:54, Jon Medhurst (Tixy) wrote:
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
In the horrible hack that I use to boot both TC2 and RTSM, I use the static RAM located at 0x2E000000. It is only 64kB though.
That misses the point.
This patch has to hook some code at the address where the reset vector is, so that we can get control of a CPU when it is brought back out of reset.
So gents, please let stop splitting hairs. This is required for CPU hotplug to work, and by extension the b.L switcher.
Nicolas
On Fri, 2012-12-14 at 15:54 +0000, Jon Medhurst (Tixy) wrote:
On Fri, 2012-12-14 at 10:10 -0500, Christoffer Dall wrote:
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
In the RTSM manual, the first 64Mb of physical memory space is marked as "NOR FLASH0", as it is on real hardware. This is flash memory which is read-only under normal circumstances, and not anything Linux would go poking about in, unless you configured an MTD device and used it for a file system.
DRAM is in the top 2GB and is what Linux uses for memory, and where most of the bootwrapper still lives. The only bit I've relocated to the bottom of the address space is everything in boot.S which includes all the bits we need to survive after Linux boots in order to handle a cpu startup again after being powered down then up again.
Are you convinced, or do you still have concerns?
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
I have now spoken to someone in ARM's Fast Models team about this and have been told that having an ELF section in the application which is located at the NOR flash address is a legitimate and supported way of setting the contents of flash.
Does anyone have any further outstanding concerns or comments about my proposed patch?
On Thu, Dec 20, 2012 at 12:23:48PM +0000, Jon Medhurst (Tixy) wrote:
On Fri, 2012-12-14 at 15:54 +0000, Jon Medhurst (Tixy) wrote:
On Fri, 2012-12-14 at 10:10 -0500, Christoffer Dall wrote:
On Fri, Dec 14, 2012 at 3:49 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Thu, 2012-12-13 at 13:52 -0800, Christoffer Dall wrote:
On Friday, December 7, 2012, Jon Medhurst (Tixy) wrote:
To enable CPU hotplug the need to provide some boot code at the
reset
vector and which survives after the kernel has booted without being overwritten. We achieve this by the getting the linker script to
place
the code in boot.S at address zero. This now means we can delete the code that relocates the secondary CPU pen code to "a location less likely to be overridden".
not knowing these dev boards's memory maps I assume that the sysflags are initialized to 0
That's what the TRM says and what is observed in practice.
and that the first part of the physical memory space is reserved for secure access?
Wasn't quite sure what you meant my that, so looked at the RTSM manual for 'security' and spotted that if you enable the 'secure_memory' option, then the NOR flash at address 0 is "Secure RO, aborts on non-secure accesses". So if we need to support that option (do we?) then my patch is broken.
we probably don't need this for vexpress in any situation that uses the boot-wrapper would be my guess, and we can always fix later then.
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
In the RTSM manual, the first 64Mb of physical memory space is marked as "NOR FLASH0", as it is on real hardware. This is flash memory which is read-only under normal circumstances, and not anything Linux would go poking about in, unless you configured an MTD device and used it for a file system.
DRAM is in the top 2GB and is what Linux uses for memory, and where most of the bootwrapper still lives. The only bit I've relocated to the bottom of the address space is everything in boot.S which includes all the bits we need to survive after Linux boots in order to handle a cpu startup again after being powered down then up again.
Are you convinced, or do you still have concerns?
I must admit, my patch is based on someone elses work, and I've not seen documentation which says that NOR flash can be written to simply by the application being loaded by the models having code located at the relevant address. I do know, that this code works on both A15 and A15xA7 models to boot Linux and Android, and that the new support it provides for power down and up cores works.
I'll try and see if I can find out if locating code at address zero is officially supported...
I have now spoken to someone in ARM's Fast Models team about this and have been told that having an ELF section in the application which is located at the NOR flash address is a legitimate and supported way of setting the contents of flash.
Does anyone have any further outstanding concerns or comments about my proposed patch?
Since this doesn't seem to be merged yet, I'll just comment that this all looks sensible to me.
For the bootwrapper to provide the reset vector code in (simulated) NOR flash feels like exactly the right thing to do here.
(Minor point: you can merge the "b start" with the first entry of the vectors, because this slot is never used for any other purpose in any vector table. The code works anyway, though.)
Cheers ---Dave
On 7 January 2013 13:47, Dave Martin dave.martin@linaro.org wrote:
On Thu, Dec 20, 2012 at 12:23:48PM +0000, Jon Medhurst (Tixy) wrote:
Does anyone have any further outstanding concerns or comments about my proposed patch?
Since this doesn't seem to be merged yet, I'll just comment that this all looks sensible to me.
I'm not in a position to do anything with this until next week at earliest, but since it seems to have had sufficient review I'm happy for somebody (Tixy? Dave?) with commit access to the boot-wrapper repo on git.linaro.org to go ahead and commit it if they want.
-- PMM
On Mon, 2013-01-07 at 13:59 +0000, Peter Maydell wrote:
On 7 January 2013 13:47, Dave Martin dave.martin@linaro.org wrote:
On Thu, Dec 20, 2012 at 12:23:48PM +0000, Jon Medhurst (Tixy) wrote:
Does anyone have any further outstanding concerns or comments about my proposed patch?
Since this doesn't seem to be merged yet, I'll just comment that this all looks sensible to me.
I'm not in a position to do anything with this until next week at earliest, but since it seems to have had sufficient review I'm happy for somebody (Tixy? Dave?) with commit access to the boot-wrapper repo on git.linaro.org to go ahead and commit it if they want.
Thanks, I've now pushed the patch to the repo.
On Mon, 2013-01-07 at 13:47 +0000, Dave Martin wrote:
(Minor point: you can merge the "b start" with the first entry of the vectors, because this slot is never used for any other purpose in any vector table. The code works anyway, though.)
I considered that when I wrote the patch but thought I would keep things separate for clarity and avoid tripping up anyone modifying the code later. We can spare 4 bytes of memory :-)
On 14 December 2012 16:10, Christoffer Dall c.dall@virtualopensystems.comwrote:
What I was trying to figure out was why you can get rid of the "a location less likely to be overridden" and why you're sure this memory stays unmodified - I thought that was because you were loading the code in the beginning of the physical memory space, which wouldn't be touched by the linux kernel (i.e. reserved for the secure side), but my guess may have been wrong and I was never a whiz with linker scripts.
I have to comment on this that the relocation address "ldr r1, =fs_start - 0x100" doesn't work for the board. Secondary CPUs will never wake and I had to move the relocation part under the monitor offset instead (not needed to relocate at all in the end if boot.o is put at the start of memory, I guess...).
-
Alexander Spyridakis -
Christoffer Dall -
Dave Martin -
Jon Medhurst (Tixy) -
Marc Zyngier -
Nicolas Pitre -
Peter Maydell