Hi folks
This is my first public mail to our own validation mailing list ;-). I'm CCing a few people that could be interested in this topic.
Now to the topic at hand. I'm trying to make it possible to use fast models (in any way) by the end of this week. I'm tracking this blueprint [1] but right now it is just an empty shell as I'm still wrapping my head around this.
A few observations so far:
1) Running a fast model on a 32bit VM is not practical, it seems to crash after a few / dozen minutes due to insufficient address space 2) I still have not managed to get autologin to work. This is mildly annoying but should be possible to get over. 3) A poweroff in the machine causes kernel panic 4) A reboot in the machine seems to just hang and do nothing 5) After much tinkering (installing additional packages, etc, etc) I managed to run stream, I've attached the bundle if anyone is interested. 6) If you need to modify the rootfs in any way it is about 10x faster to do it via chroot + qemu than to run the fast model (on core i7) 7) The system boots with nfsroot, the IP of the server and the directory to mount is burned in the .axf file 8) An axf file is created from a kernel image and some other bits that I don't yet understand. It seems that boot-wrapper is a kind of bootloader for linux in the fast model. I've found at least two sources for that [2] (from the wiki page by Peter Maydell) and [3] (from Michael Hope) 9) I'm not at all sure why we need to cross compile qemu for arm (as instructed in [4]) - is this so that we can test visualization support in a15? 10) The instructions did miss a few things and I had to run apt-get -f install / install extra cross packages. In the end I'm not sure how much of this is only required to build qemu, how much is required to build the rootfs and how much is required to build the kernel. It seems to be too messy ATM, I'd like to split this into separate problems that could be automated with standalone scripts we could fire in a VM or chroot (so that we are sure we got the dependencies right and everything really works fine) 11) I'm not at all sure what's the difference between model_shell's terminal_[0123] and uart[0123]. That is, I'd like to know what those options do with a little bit more precision. Looking at the pdfs that came with the program did not help.
The major roadblock:
*) I'm still trying to parametrize the kernel command line. There seem to be a few possible ways to do it. One based on just rebuilding the axf file (longest/ugilest), one on changing the boot-wrapper bootloader to accept some extra data at the end and pass that down to the kernel (not my specialty, could take a moment to get to work), looking at model_shell -l output it seems there is a way to pass something known as 'cluster.cpu0.semihosting-cmd_line='. Perhaps that could be used to put some data that the kernel can read at runtime.
That's it. Keep fighting
ZK
[1]: https://blueprints.launchpad.net/lava-dispatcher/+spec/lava-fast-model-clien... [2]: git://git.ncl.cs.columbia.edu/pub/git/boot-wrapper [3]: https://github.com/virtualopensystems/boot-wrapper [4]: https://wiki.linaro.org/PeterMaydell/A15OnFastModels#Cross_compile_QEMU
On 24 January 2012 11:08, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
- Running a fast model on a 32bit VM is not practical, it seems to
crash after a few / dozen minutes due to insufficient address space
It kind of depends what workload you're giving it. A plain 32 bit guest should generally be OK. However the KVM work will require more address space than that anyway so we might as well standardise on 64 bit hosts from the start.
- I still have not managed to get autologin to work. This is mildly
annoying but should be possible to get over.
Issue on our side.
- A poweroff in the machine causes kernel panic
- A reboot in the machine seems to just hang and do nothing
We should be filing this kind of thing as bugs/enhancement requests through our official ARM support channel (for the avoidance of doubt, that's not me :-)).
- The system boots with nfsroot, the IP of the server and the
directory to mount is burned in the .axf file
I have a plan here, see below.
- An axf file is created from a kernel image and some other bits that
I don't yet understand. It seems that boot-wrapper is a kind of bootloader for linux in the fast model. I've found at least two sources for that [2] (from the wiki page by Peter Maydell) and [3] (from Michael Hope)
The official boot-wrapper sources come from git://git.ncl.cs.columbia.edu/pub/git/boot-wrapper
The boot-wrapper basically wraps up: * kernel * initrd (optionally) * kernel command line string * some trivial boot code to set up and launch the kernel into one AXF (ELF) file.
- I'm not at all sure why we need to cross compile qemu for arm (as
instructed in [4]) - is this so that we can test visualization support in a15?
This is because that wiki page is instructions for getting set up to test/develop KVM. So you don't need it to just run a simple system under the fast model. However we will need to set up something similar for doing validation testing of KVM: we will need to be able to build a QEMU, build a kvm-guest kernel, install the kvm-guest kernel and QEMU into the filesystem of the fast-model-guest and then launch the guest and then run LAVA tests on the guest.
- The instructions did miss a few things and I had to run apt-get -f
install / install extra cross packages. In the end I'm not sure how much of this is only required to build qemu, how much is required to build the rootfs and how much is required to build the kernel. It seems to be too messy ATM, I'd like to split this into separate problems that could be automated with standalone scripts we could fire in a VM or chroot (so that we are sure we got the dependencies right and everything really works fine)
Yes, automation is the general end plan. If there are bits of the wiki instructions that are not right please edit the wiki to correct them...
- I'm not at all sure what's the difference between model_shell's
terminal_[0123] and uart[0123]. That is, I'd like to know what those options do with a little bit more precision.
uart[0123] model the UARTs, ie the PL011 PrimeCells on a Versatile Express motherboard. terminal_[0123] are modelling a virtual terminal connected to the uart[0123] via a virtual RS232 cable.
From the user's point of view it doesn't make much difference,
it's just that some parameters happen to be on one component and some on the other depending on where the functionality they are controlling is implemented.
The major roadblock:
*) I'm still trying to parametrize the kernel command line. There seem to be a few possible ways to do it. One based on just rebuilding the axf file (longest/ugilest), one on changing the boot-wrapper bootloader to accept some extra data at the end and pass that down to the kernel (not my specialty, could take a moment to get to work), looking at model_shell -l output it seems there is a way to pass something known as 'cluster.cpu0.semihosting-cmd_line='. Perhaps that could be used to put some data that the kernel can read at runtime.
Yes, I have a plan to improve this. Rather than having bootwrapper include the kernel/initrd/commandline, we can have it use semihosting to read the files at runtime. So you would then be able to use the semihosting-cmd_line parameter to pass a string like "--kernel /path/to/uImage --initrd /path/to/initrd -- args for kernel" Then you would be able to build a single AXF file as a one-off step, and use the same AXF file for loading any kernel/initrd.
Might take me a few days to knock this together, though.
-- PMM
On Tue, Jan 24, 2012 at 12:51 PM, Peter Maydell peter.maydell@linaro.org wrote:
On 24 January 2012 11:08, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
- Running a fast model on a 32bit VM is not practical, it seems to
crash after a few / dozen minutes due to insufficient address space
It kind of depends what workload you're giving it. A plain 32 bit guest should generally be OK. However the KVM work will require more address space than that anyway so we might as well standardise on 64 bit hosts from the start.
- I still have not managed to get autologin to work. This is mildly
annoying but should be possible to get over.
Issue on our side.
- A poweroff in the machine causes kernel panic
- A reboot in the machine seems to just hang and do nothing
We should be filing this kind of thing as bugs/enhancement requests through our official ARM support channel (for the avoidance of doubt, that's not me :-)).
- The system boots with nfsroot, the IP of the server and the
directory to mount is burned in the .axf file
I have a plan here, see below.
- An axf file is created from a kernel image and some other bits that
I don't yet understand. It seems that boot-wrapper is a kind of bootloader for linux in the fast model. I've found at least two sources for that [2] (from the wiki page by Peter Maydell) and [3] (from Michael Hope)
The official boot-wrapper sources come from git://git.ncl.cs.columbia.edu/pub/git/boot-wrapper
The boot-wrapper basically wraps up: * kernel * initrd (optionally) * kernel command line string * some trivial boot code to set up and launch the kernel into one AXF (ELF) file.
Can we reserve a few more bytes and patch the command line before launching? This would allow me to write the infrastructure code without waiting on your (better) fix with one-axf-to-rulle-them-all (see the bottom of this message).
- I'm not at all sure why we need to cross compile qemu for arm (as
instructed in [4]) - is this so that we can test visualization support in a15?
This is because that wiki page is instructions for getting set up to test/develop KVM. So you don't need it to just run a simple system under the fast model. However we will need to set up something similar for doing validation testing of KVM: we will need to be able to build a QEMU, build a kvm-guest kernel, install the kvm-guest kernel and QEMU into the filesystem of the fast-model-guest and then launch the guest and then run LAVA tests on the guest.
- The instructions did miss a few things and I had to run apt-get -f
install / install extra cross packages. In the end I'm not sure how much of this is only required to build qemu, how much is required to build the rootfs and how much is required to build the kernel. It seems to be too messy ATM, I'd like to split this into separate problems that could be automated with standalone scripts we could fire in a VM or chroot (so that we are sure we got the dependencies right and everything really works fine)
Yes, automation is the general end plan. If there are bits of the wiki instructions that are not right please edit the wiki to correct them...
I did post a few updates, the one that is still missing is the moment when you install a few more pacakges built with xapt (I actually installed all of them). In the end I think this should be split and specialized (basic 15 working, qemu-in-a15, etc etc). To let people validate the instructions are accurate.
- I'm not at all sure what's the difference between model_shell's
terminal_[0123] and uart[0123]. That is, I'd like to know what those options do with a little bit more precision.
uart[0123] model the UARTs, ie the PL011 PrimeCells on a Versatile Express motherboard. terminal_[0123] are modelling a virtual terminal connected to the uart[0123] via a virtual RS232 cable. From the user's point of view it doesn't make much difference, it's just that some parameters happen to be on one component and some on the other depending on where the functionality they are controlling is implemented.
I'm trying to figure out the 'optimal' set of options that I should pass. I was thinking about spawning two ttys. One for just grabbing the boot messages (and syslog/console) later on and one for interactive shell. It seems that I don't really care and terminal[01] are just as good.
The major roadblock:
*) I'm still trying to parametrize the kernel command line. There seem to be a few possible ways to do it. One based on just rebuilding the axf file (longest/ugilest), one on changing the boot-wrapper bootloader to accept some extra data at the end and pass that down to the kernel (not my specialty, could take a moment to get to work), looking at model_shell -l output it seems there is a way to pass something known as 'cluster.cpu0.semihosting-cmd_line='. Perhaps that could be used to put some data that the kernel can read at runtime.
Yes, I have a plan to improve this. Rather than having bootwrapper include the kernel/initrd/commandline, we can have it use semihosting to read the files at runtime. So you would then be able to use the semihosting-cmd_line parameter to pass a string like "--kernel /path/to/uImage --initrd /path/to/initrd -- args for kernel" Then you would be able to build a single AXF file as a one-off step, and use the same AXF file for loading any kernel/initrd.
Might take me a few days to knock this together, though.
Excellent description, thanks! If that could work we should fall-back to making .axf a private implementation detail and allow people to run their kernel/initrd?/rootfs directly. Can I assist you with this task in any way? Can we put more priority on it (Paul? Fabo?)
Thanks ZK
On 24 January 2012 12:11, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Tue, Jan 24, 2012 at 12:51 PM, Peter Maydell peter.maydell@linaro.org wrote:
The boot-wrapper basically wraps up: * kernel * initrd (optionally) * kernel command line string * some trivial boot code to set up and launch the kernel into one AXF (ELF) file.
Can we reserve a few more bytes and patch the command line before launching? This would allow me to write the infrastructure code without waiting on your (better) fix with one-axf-to-rulle-them-all (see the bottom of this message).
It's not quite as easy as just adding some spare NULs after the string, because the command line is embedded in an ATAG list. If you look at the bottom of boot.S you can see this. So you'd need to stick the NULs at the end of the ATAG list, move the ATAGS after ATAG_CMDLINE up/down when you edited the command line, and update the length field in the ATAG_CMDLINE node itself.
I did post a few updates, the one that is still missing is the moment when you install a few more pacakges built with xapt (I actually installed all of them). In the end I think this should be split and specialized (basic 15 working, qemu-in-a15, etc etc). To let people validate the instructions are accurate.
I specifically avoided installing all the packages xapt generates because the general recommendation for xapt is not to pollute your system with them, so I cut the list down to the bare minimum that worked. If you're installing in a dev-use-only chroot or vm then it's less important to keep it clean, of course.
-- PMM
On Tue, Jan 24, 2012 at 1:25 PM, Peter Maydell peter.maydell@linaro.org wrote:
On 24 January 2012 12:11, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Tue, Jan 24, 2012 at 12:51 PM, Peter Maydell peter.maydell@linaro.org wrote:
The boot-wrapper basically wraps up: * kernel * initrd (optionally) * kernel command line string * some trivial boot code to set up and launch the kernel into one AXF (ELF) file.
Can we reserve a few more bytes and patch the command line before launching? This would allow me to write the infrastructure code without waiting on your (better) fix with one-axf-to-rulle-them-all (see the bottom of this message).
It's not quite as easy as just adding some spare NULs after the string, because the command line is embedded in an ATAG list. If you look at the bottom of boot.S you can see this. So you'd need to stick the NULs at the end of the ATAG list, move the ATAGS after ATAG_CMDLINE up/down when you edited the command line, and update the length field in the ATAG_CMDLINE node itself.
While that could work for the moment I'll focus on the barebones API. If you can create a a15 boot trampoline axf then we will adopt that. Otherwise we'll fall-back to patching axfs.
I'll start a repository with the initial code I'm working on and post back.
I did post a few updates, the one that is still missing is the moment when you install a few more pacakges built with xapt (I actually installed all of them). In the end I think this should be split and specialized (basic 15 working, qemu-in-a15, etc etc). To let people validate the instructions are accurate.
I specifically avoided installing all the packages xapt generates because the general recommendation for xapt is not to pollute your system with them, so I cut the list down to the bare minimum that worked. If you're installing in a dev-use-only chroot or vm then it's less important to keep it clean, of course.
I'm sure I had to install at least two more and I still was missing more. Still, I understand the reasons for keeping systems tidy.
ZK
On 24 January 2012 11:51, Peter Maydell peter.maydell@linaro.org wrote:
Yes, I have a plan to improve this. Rather than having bootwrapper include the kernel/initrd/commandline, we can have it use semihosting to read the files at runtime. So you would then be able to use the semihosting-cmd_line parameter to pass a string like "--kernel /path/to/uImage --initrd /path/to/initrd -- args for kernel" Then you would be able to build a single AXF file as a one-off step, and use the same AXF file for loading any kernel/initrd.
I've posted boot-wrapper patches to do this to the android-virt list. http://patches.linaro.org/6379/ http://patches.linaro.org/6380/
(unfortunately linaro patchwork doesn't seem to have the 'download mbox' link which stock patchwork does, so those links aren't as useful as they could be...)
-- PMM
linaro-validation@lists.linaro.org
-
Peter Maydell -
Zygmunt Krynicki