Kevin,
As we discussed at ELC, it would be nice to test ARM big endian in the board farm. Here are some instructions on how to get this running:
1 Take the Buildroot branch at http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=for-ke.... It is Buildroot 2014.02 + one patch for the AArch64 symlink + one patch for the ARM big endian Linaro toolchain.
2 Build a Buildroot configuration such as:
BR2_armeb=y BR2_cortex_a8=y BR2_ARM_EABIHF=y BR2_TOOLCHAIN_EXTERNAL=y BR2_TARGET_ROOTFS_CPIO=y # BR2_TARGET_ROOTFS_TAR is not set
3 Build mvebu_v7_defconfig or multi_v7_defconfig with CONFIG_CPU_BIG_ENDIAN=y (no other change is needed, except your own configuration stuff maybe, like initrd support, devtmpfs and so on).
4 Boot
I've tested this right now on Armada XP OpenBlocks AX3-4. It is worth noting that in BE mode, the ATAGS are not used, so if your OpenBlocks has only 1 GB of RAM, the boot will crash (with some weird tty related panic). That's because the Device Tree for the OpenBlocks AX3-4 encodes a memory size of 3 GB, which some OpenBlocks have. If you fall into this issue, then change the memory node in the DT to use 1 GB only.
Normally, this should also work on Armada 370 Mirabox, though I haven't tested right now.
Do you think you could add this big endian test to the board farm?
In addition to this, it would be interesting if you could also test LPAE on Armada XP OpenBlocks AX3-4 (it should work on little endian, however I have never tested LPAE/big-endian).
Thanks a lot!
Thomas
Thomas Petazzoni thomas.petazzoni@free-electrons.com writes:
Kevin,
As we discussed at ELC, it would be nice to test ARM big endian in the board farm. Here are some instructions on how to get this running:
Excellent, thanks.
1 Take the Buildroot branch at http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=for-ke.... It is Buildroot 2014.02 + one patch for the AArch64 symlink + one patch for the ARM big endian Linaro toolchain.
2 Build a Buildroot configuration such as:
BR2_armeb=y BR2_cortex_a8=y BR2_ARM_EABIHF=y BR2_TOOLCHAIN_EXTERNAL=y BR2_TARGET_ROOTFS_CPIO=y # BR2_TARGET_ROOTFS_TAR is not set
3 Build mvebu_v7_defconfig or multi_v7_defconfig with CONFIG_CPU_BIG_ENDIAN=y (no other change is needed, except your own configuration stuff maybe, like initrd support, devtmpfs and so on).
OK, I'm still putting the final touches on support in my automated builder for base defconfig + kconfig fragments. After that, I'll be able to build these kind of kconfig variants easily.
4 Boot
I've tested this right now on Armada XP OpenBlocks AX3-4. It is worth noting that in BE mode, the ATAGS are not used, so if your OpenBlocks has only 1 GB of RAM, the boot will crash (with some weird tty related panic). That's because the Device Tree for the OpenBlocks AX3-4 encodes a memory size of 3 GB, which some OpenBlocks have. If you fall into this issue, then change the memory node in the DT to use 1 GB only.
Hmm, if atags are not used, then u-boot will not be able to pass an initramfs in memory. That's a bit of a problem for my current setup.
I rely on initramfs because MMC and NFS rootfs setups break rather often, so I like to rely on initramfs, and then switch_root to MMC/NFS.
Normally, this should also work on Armada 370 Mirabox, though I haven't tested right now.
Do you think you could add this big endian test to the board farm?
Yes, I'll add it to my TODO list.
In addition to this, it would be interesting if you could also test LPAE on Armada XP OpenBlocks AX3-4 (it should work on little endian, however I have never tested LPAE/big-endian).
OK, I'll add the obsax3 to the list of LPAE capable boards. This won't start getting published though until I finish support for the kconfig fragments.
Kevin
Dear Kevin Hilman,
On Wed, 07 May 2014 12:59:41 -0700, Kevin Hilman wrote:
3 Build mvebu_v7_defconfig or multi_v7_defconfig with CONFIG_CPU_BIG_ENDIAN=y (no other change is needed, except your own configuration stuff maybe, like initrd support, devtmpfs and so on).
OK, I'm still putting the final touches on support in my automated builder for base defconfig + kconfig fragments. After that, I'll be able to build these kind of kconfig variants easily.
Ok.
I've tested this right now on Armada XP OpenBlocks AX3-4. It is worth noting that in BE mode, the ATAGS are not used, so if your OpenBlocks has only 1 GB of RAM, the boot will crash (with some weird tty related panic). That's because the Device Tree for the OpenBlocks AX3-4 encodes a memory size of 3 GB, which some OpenBlocks have. If you fall into this issue, then change the memory node in the DT to use 1 GB only.
Hmm, if atags are not used, then u-boot will not be able to pass an initramfs in memory. That's a bit of a problem for my current setup.
I rely on initramfs because MMC and NFS rootfs setups break rather often, so I like to rely on initramfs, and then switch_root to MMC/NFS.
Hum, I see. Unfortunately, my understanding is that there is no support in the kernel for reading little-endian ATAGs when the kernel is built big-endian. There are some patches available to do that, but I think the general opinion was that BE support on ARMv7 is sufficiently new that we expected a DT-capable bootloader. It turns out that this is not really true :/
For those platforms, maybe you could use an initramfs built into the kernel, using CONFIG_INITRAMFS_SOURCE ? This is what I use all the time.
In addition to this, it would be interesting if you could also test LPAE on Armada XP OpenBlocks AX3-4 (it should work on little endian, however I have never tested LPAE/big-endian).
OK, I'll add the obsax3 to the list of LPAE capable boards. This won't start getting published though until I finish support for the kconfig fragments.
Great, thanks!
Thomas
On Wed, May 7, 2014 at 3:05 PM, Thomas Petazzoni thomas.petazzoni@free-electrons.com wrote:
On Wed, 07 May 2014 12:59:41 -0700, Kevin Hilman wrote:
[...]
Hmm, if atags are not used, then u-boot will not be able to pass an initramfs in memory. That's a bit of a problem for my current setup.
I rely on initramfs because MMC and NFS rootfs setups break rather often, so I like to rely on initramfs, and then switch_root to MMC/NFS.
Hum, I see. Unfortunately, my understanding is that there is no support in the kernel for reading little-endian ATAGs when the kernel is built big-endian. There are some patches available to do that, but I think the general opinion was that BE support on ARMv7 is sufficiently new that we expected a DT-capable bootloader. It turns out that this is not really true :/
For those platforms, maybe you could use an initramfs built into the kernel, using CONFIG_INITRAMFS_SOURCE ? This is what I use all the time.
That's not as easy as it sounds because my boot testing is totally de-coupled from the builder. IOW, at build time, I have no idea what boards the zImage will be run on, and I don't want all kernels to have an initramfs built in. For that matter, my automated kernel builder would also have to have an initramfs laying around, or build one, and I don't want that either.
Maybe on those boards, I'll have to rely on MMC rootfs and hope that's supported (and stable) in mainline.
Kevin
On Wed, 7 May 2014, Kevin Hilman wrote:
Hmm, if atags are not used, then u-boot will not be able to pass an initramfs in memory. That's a bit of a problem for my current setup.
I rely on initramfs because MMC and NFS rootfs setups break rather often, so I like to rely on initramfs, and then switch_root to MMC/NFS.
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtb
Obviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
Nicolas
Dear Nicolas Pitre,
On Wed, 7 May 2014 23:30:47 -0400 (EDT), Nicolas Pitre wrote:
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtb
Obviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
Thanks,
Thomas
On Thu, 8 May 2014, Thomas Petazzoni wrote:
Dear Nicolas Pitre,
On Wed, 7 May 2014 23:30:47 -0400 (EDT), Nicolas Pitre wrote:
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtb
Obviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
Something like that, yes. I think there is even a binding specific to initrd in that node as well.
Nicolas
On Thu, May 8, 2014 at 11:49 AM, Thomas Petazzoni thomas.petazzoni@free-electrons.com wrote:
Dear Nicolas Pitre,
On Wed, 7 May 2014 23:30:47 -0400 (EDT), Nicolas Pitre wrote:
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtbObviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
I'm finally getting around to testing big endian, and I can confirm that using initrd= on the command-line (via DT chosen/bootargs) is working fine for loading an initrd.
The problem now is how to automate this when I'm given a zImage, a pre-built DTB (binary), and a separate ramdisk. Especially when that same unmodified DTB can boot fine for other kernels.
It would be much easier if I could just change the cmdline via u-boot, but since the ATAGs dont' work, the kernel doesn't get the u-boot command-line either (even with CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y).
That leaves me with needing to modify a (binary) DTB. Ugh.
Kevin
Dear Kevin Hilman,
On Tue, 20 May 2014 11:37:01 -0700, Kevin Hilman wrote:
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
I'm finally getting around to testing big endian, and I can confirm that using initrd= on the command-line (via DT chosen/bootargs) is working fine for loading an initrd.
The problem now is how to automate this when I'm given a zImage, a pre-built DTB (binary), and a separate ramdisk. Especially when that same unmodified DTB can boot fine for other kernels.
It would be much easier if I could just change the cmdline via u-boot, but since the ATAGs dont' work, the kernel doesn't get the u-boot command-line either (even with CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y).
That leaves me with needing to modify a (binary) DTB. Ugh.
Unfortunately, there aren't much solutions:
*) As far as I'm aware there is no DT-capable bootloader for this platform (OpenBlocks AX-3). So having the bootloader modify the DT is not possible.
*) There is no support for ATAGs compatibility in BE mode.
Possible solutions:
*) Use a shim layer between the bootloader and the kernel to adjust the DTB as needed. Jason Cooper has worked on an impedance matcher to do such sort of DTB tweaking.
*) Change the way you do things, and assemble a bundled kernel image that contains the initramfs (using CONFIG_INITRAMFS_SOURCE) and is appended with the DTB. That's the solution I'm using.
Best regards,
Thomas
On Tue, 20 May 2014, Kevin Hilman wrote:
On Thu, May 8, 2014 at 11:49 AM, Thomas Petazzoni thomas.petazzoni@free-electrons.com wrote:
Dear Nicolas Pitre,
On Wed, 7 May 2014 23:30:47 -0400 (EDT), Nicolas Pitre wrote:
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtbObviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
I'm finally getting around to testing big endian, and I can confirm that using initrd= on the command-line (via DT chosen/bootargs) is working fine for loading an initrd.
The problem now is how to automate this when I'm given a zImage, a pre-built DTB (binary), and a separate ramdisk. Especially when that same unmodified DTB can boot fine for other kernels.
It would be much easier if I could just change the cmdline via u-boot, but since the ATAGs dont' work, the kernel doesn't get the u-boot command-line either (even with CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y).
That leaves me with needing to modify a (binary) DTB. Ugh.
You could standardize around an identical ramdisk address for all your targets where that same unmodified DTB is used. The ramdisk address passed by U-Boot on those platforms where U-Boot is able to do so would be overwritten by the one specified on the kernel cmdline, but if they are the same address then you'll be fine.
Nicolas
On Tue, May 20, 2014 at 12:12 PM, Nicolas Pitre nicolas.pitre@linaro.org wrote:
On Tue, 20 May 2014, Kevin Hilman wrote:
On Thu, May 8, 2014 at 11:49 AM, Thomas Petazzoni thomas.petazzoni@free-electrons.com wrote:
Dear Nicolas Pitre,
On Wed, 7 May 2014 23:30:47 -0400 (EDT), Nicolas Pitre wrote:
You still can pass initrd=<phys_addr>,<size> to the kernel via the cmdline string. And if neither ATAGS nor DT are provided you'll have to use the cmdline to pass the memory information as well. This can be hardcoded into the kernel with CONFIG_CMDLINE.
However a better solution that doesn't involve hardcoding anything in the kernel if you want to use a multi-platform kernel is to use CONFIG_ARM_APPENDED_DTB and:
make zImage cat arch/arm/boot/zImage your_board.dtb > zImage_with_dtb mkimage ... # to create uImage using zImage_with_dtbObviously you shouldn't have to rebuild your_board.dtb each time you build a new kernel for the same board.
I believe Kevin is already using appended DTB, because the bootloader on the OpenBlocks platform does not have DT support at all, so appended DTB is really the only way to get a mainline kernel to work on this platform.
So what you suggest is that Kevin could use the /chosen/bootargs DT node to encode the initrd address, and then make sure he loads the initrd from the bootloader at the corresponding address?
I'm finally getting around to testing big endian, and I can confirm that using initrd= on the command-line (via DT chosen/bootargs) is working fine for loading an initrd.
The problem now is how to automate this when I'm given a zImage, a pre-built DTB (binary), and a separate ramdisk. Especially when that same unmodified DTB can boot fine for other kernels.
It would be much easier if I could just change the cmdline via u-boot, but since the ATAGs dont' work, the kernel doesn't get the u-boot command-line either (even with CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y).
That leaves me with needing to modify a (binary) DTB. Ugh.
You could standardize around an identical ramdisk address for all your targets where that same unmodified DTB is used. The ramdisk address passed by U-Boot on those platforms where U-Boot is able to do so would be overwritten by the one specified on the kernel cmdline, but if they are the same address then you'll be fine.
I really don't like building in these kinds of assumptions. And even if I did, I still need to modify the DTB somplace. Either boot-time patching, or at build time. Build time is ruled out because I don't have control over all the builds that I'm trying to boot test.
After a little tinkering though, it turns out runtime patching the DTB isn't that hard. DTC can un-compile a dtb to the original .dts. I can then append a dts fragment with a new chosen node, (re)compile that and I get a new dtb with an updated chosen node. I'm playing with that now and it seems workable.
Kevin
On Tue, 20 May 2014, Kevin Hilman wrote:
On Tue, May 20, 2014 at 12:12 PM, Nicolas Pitre nicolas.pitre@linaro.org wrote:
You could standardize around an identical ramdisk address for all your targets where that same unmodified DTB is used. The ramdisk address passed by U-Boot on those platforms where U-Boot is able to do so would be overwritten by the one specified on the kernel cmdline, but if they are the same address then you'll be fine.
I really don't like building in these kinds of assumptions. And even if I did, I still need to modify the DTB somplace. Either boot-time patching, or at build time. Build time is ruled out because I don't have control over all the builds that I'm trying to boot test.
I thought you mentioned you didn't want to modify the DTB.
After a little tinkering though, it turns out runtime patching the DTB isn't that hard. DTC can un-compile a dtb to the original .dts. I can then append a dts fragment with a new chosen node, (re)compile that and I get a new dtb with an updated chosen node. I'm playing with that now and it seems workable.
Good.
Nicolas
On 05/20/2014 12:37 PM, Kevin Hilman wrote: ...
That leaves me with needing to modify a (binary) DTB. Ugh.
It's actually pretty easy to do that, assuming you have hooks in your build/... process to actually plug that in easily.
For example, take a look at line 202 and 227 of:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/tegra-uboo...
On Tue, May 20, 2014 at 3:00 PM, Stephen Warren swarren@wwwdotorg.org wrote:
On 05/20/2014 12:37 PM, Kevin Hilman wrote: ...
That leaves me with needing to modify a (binary) DTB. Ugh.
It's actually pretty easy to do that, assuming you have hooks in your build/... process to actually plug that in easily.
For example, take a look at line 202 and 227 of:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/tegra-uboo...
Hmm, interesting. I didn't know about fdtput. fdtput -t s foo.dtb /chosen bootargs "new command-line args" does exactly what I need. Looks like it replaces the set of hacks I just whipped up (I used dtc to de-compile a dtb to a dts, append a new chosen node, and dtc to re-compile and I was able to insert the command-line that way. Looks like fdtput does that much better. Thanks for the pointer.
Kevin
Dear Kevin Hilman,
On Tue, 20 May 2014 22:47:37 -0700, Kevin Hilman wrote:
It's actually pretty easy to do that, assuming you have hooks in your build/... process to actually plug that in easily.
For example, take a look at line 202 and 227 of:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/tegra-uboo...
Hmm, interesting. I didn't know about fdtput. fdtput -t s foo.dtb /chosen bootargs "new command-line args" does exactly what I need. Looks like it replaces the set of hacks I just whipped up (I used dtc to de-compile a dtb to a dts, append a new chosen node, and dtc to re-compile and I was able to insert the command-line that way. Looks like fdtput does that much better. Thanks for the pointer.
Ah, indeed, really interesting. I also didn't know about fdtput. I was talking with Kevin about this big endian issue yesterday, and pointed to dtc as a possible solution, but I was hoping for a tool that allows to edit the DTB without having to extract the full DTS, patch it, and re-create the DTB.
It appears that such a tool already exists, thanks a lot for pointing it out!
Best regards,
Thomas
kernel-build-reports@lists.linaro.org
-
Kevin Hilman -
Nicolas Pitre -
Stephen Warren -
Thomas Petazzoni