I am trying to set up a Xen CI job on lava. The components are built (Xen hypervisor, xen tools deb package, Xen-enabled kernel, standard minimal rootfs). Now I'm working out what the lava job should look like.
Xen requires a non-standard boot with the hypervisor loaded as well as the kernel before booting (and rootfs unpacked).
Julien says we need to do (at least some of) the config on this page before running XEN: http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress which involves disabling the A7 (required), and one way of loading the stuff in the right places, which there may be uboot/uefi-based alternatives to?). This config is done in images.txt and board.txt files.
Those files apparently live on the second, internal, microSD card. I presume that deploy_linaro_image only writes to the main SD card. Certainly the hwpack (http://releases.linaro.org/13.08/ubuntu/vexpress/hwpack_linaro-vexpress_2013...) does not contain an images.txt or board.txt file)
The card can apparently be exposed as a fat filesystem over USB (usb-storage). Is that cable plugged in? Is there a lava mechanism for accessing this and updating it?
What firmware version (boot monitor) is in the machine in the lab? What is currently in the images.txt, board.txt files it is booting with? Is 'sys_flags' bringup already selected or not?
Who knows about this stuff?
If we can't change those files in lava jobs then I believe that the xen CI task is currently blocked until one of these becomes true: * Xen runs on the standard vexpress config * Xen runs on arndale properly * A mechanism for changing these files is developed
The Xen card: https://cards.linaro.org/browse/VIRT-75
Wookey
Scriptable boot sequences have already landed to an extent, but I have a few changes landing soon (I hope) that will make them a bit more reliable. It is all basically expect/send to some approximation. If you can use that sort of logic Tyler and I both have experience with it and can help (not sure about the rest of the LAVA team).
I currently use this to flash new images onto boards using uboot and tftp in my home lab then hack around the fact that the image I just flashed doesn't auto-login.
I hope to change how this works a bit soon so I can configuration separate for boards, images and tests. Currently the extra config I need for an image is either held in the board or test config. This should allow for writing a test that references an image config file. #include all over again :-)
The job looks like this at the moment:
{ "device_type": "some device", "actions": [ { "command": "deploy_linaro_kernel", "parameters": { "kernel": "file:///path/to/zImage", "ramdisk": "file:///path/to/jffs2", "dtb": "... used to crash without this, left it in, unused ..." } }, { "command": "boot_linaro_image", "parameters": { "interactive_boot_cmds": true, "options": [ "setenv ipaddr '192.168.1.50'", "setenv serverip ${lava_server_ip}", "setenv update_k1_lava 'tftp ${lava_kernel} && sha1sum $loadaddr $filesize; nand erase k1; nand write k1 $filesize; set loadaddr 0x08000000; tftp ${lava_ramdisk} && sha1sum $loadaddr $filesize; nand erase r1; nand write r1 $filesize; set loadaddr 0x8000'", "run update_k1_lava", "sendline boot", "expect Machine: Entropic STB Krome platform", "expect login:", "sendline root" ] } }, { "command": "lava_test_shell", "parameters": { "testdef_urls": [ "http://192.168.1.11/tests/ls.yaml" ], "timeout": 1800 } }, { "command": "submit_results", "parameters": { "stream": "/anonymous/dooferlad/", "server": "http://192.168.1.10/RPC2/" } } ], "timeout": 18000, "job_name": "Run ls for ultimate testing!" }
Note the interactive_boot_cmds has some "sendline <string>" lines and some "<string>" lines. The difference is that sendline doesn't wait for the bootloader prompt. Most of the time you don't need to do sendline/expect, but if you want to do something a bit unusual, such as wait for a login prompt and type root (as above), they come in useful.
James
On 3 September 2013 17:44, Wookey wookey@wookware.org wrote:
I am trying to set up a Xen CI job on lava. The components are built (Xen hypervisor, xen tools deb package, Xen-enabled kernel, standard minimal rootfs). Now I'm working out what the lava job should look like.
Xen requires a non-standard boot with the hypervisor loaded as well as the kernel before booting (and rootfs unpacked).
Julien says we need to do (at least some of) the config on this page before running XEN: http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress which involves disabling the A7 (required), and one way of loading the stuff in the right places, which there may be uboot/uefi-based alternatives to?). This config is done in images.txt and board.txt files.
Those files apparently live on the second, internal, microSD card. I presume that deploy_linaro_image only writes to the main SD card. Certainly the hwpack (http://releases.linaro.org/13.08/ubuntu/vexpress/hwpack_linaro-vexpress_2013...) does not contain an images.txt or board.txt file)
The card can apparently be exposed as a fat filesystem over USB (usb-storage). Is that cable plugged in? Is there a lava mechanism for accessing this and updating it?
What firmware version (boot monitor) is in the machine in the lab? What is currently in the images.txt, board.txt files it is booting with? Is 'sys_flags' bringup already selected or not?
Who knows about this stuff?
If we can't change those files in lava jobs then I believe that the xen CI task is currently blocked until one of these becomes true:
- Xen runs on the standard vexpress config
- Xen runs on arndale properly
- A mechanism for changing these files is developed
The Xen card: https://cards.linaro.org/browse/VIRT-75
Wookey
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/
linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
On Tue, Sep 03, 2013 at 05:44:13PM +0100, Wookey wrote:
I am trying to set up a Xen CI job on lava. The components are built (Xen hypervisor, xen tools deb package, Xen-enabled kernel, standard minimal rootfs). Now I'm working out what the lava job should look like.
Xen requires a non-standard boot with the hypervisor loaded as well as the kernel before booting (and rootfs unpacked).
Julien says we need to do (at least some of) the config on this page before running XEN: http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress which involves disabling the A7 (required), and one way of loading the stuff in the right places, which there may be uboot/uefi-based alternatives to?). This config is done in images.txt and board.txt files.
Those files apparently live on the second, internal, microSD card. I presume that deploy_linaro_image only writes to the main SD card. Certainly the hwpack (http://releases.linaro.org/13.08/ubuntu/vexpress/hwpack_linaro-vexpress_2013...) does not contain an images.txt or board.txt file)
The card can apparently be exposed as a fat filesystem over USB (usb-storage). Is that cable plugged in?
It is.
Is there a lava mechanism for accessing this and updating it?
Yes, but at the moment it's only used to replace the UEFI binary with the one that comes with the test disk image. It would be possible to add some code that will also replace other files in there.
What firmware version (boot monitor) is in the machine in the lab?
ARM Versatile Express Boot Monitor Version: V5.1.9
What is currently in the images.txt, board.txt files it is booting with? Is 'sys_flags' bringup already selected or not?
It is not.
Who knows about this stuff?
If we can't change those files in lava jobs then I believe that the xen CI task is currently blocked until one of these becomes true:
- Xen runs on the standard vexpress config
- Xen runs on arndale properly
- A mechanism for changing these files is developed
The Xen card: https://cards.linaro.org/browse/VIRT-75
+++ Antonio Terceiro [2013-09-04 20:41 -0300]:
On Tue, Sep 03, 2013 at 05:44:13PM +0100, Wookey wrote:
Xen requires a non-standard boot with the hypervisor loaded as well as the kernel before booting (and rootfs unpacked).
Julien says we need to do (at least some of) the config on this page before running XEN: http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress which involves disabling the A7 (required), and one way of loading the stuff in the right places, which there may be uboot/uefi-based alternatives to?). This config is done in images.txt and board.txt files.
Those files apparently live on the second, internal, microSD card.
The card can apparently be exposed as a fat filesystem over USB (usb-storage). Is that cable plugged in?
It is.
Is there a lava mechanism for accessing this and updating it?
Yes, but at the moment it's only used to replace the UEFI binary with the one that comes with the test disk image. It would be possible to add some code that will also replace other files in there.
Is this code that we have to add to lava and then release into production, or can we issue interactive_boot_cmds sendline/expect foo to get this done for now?
(Which machine is actually issuing those commands in lava? I haven't got the architecture properly understood in my head, which makes this confusing to think about. Although I guess the idea is that lava wraps all that so I shouldn't need to know. I think I do need to know the device name of this SD-card if we are doing mount;change file;unmount in interactive_boot_cmds. And presumably $something needs to remember to put the default one back again after this job runs, which is the real reason why lava needs to support it properly?
What firmware version (boot monitor) is in the machine in the lab?
ARM Versatile Express Boot Monitor Version: V5.1.9
OK so according to http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress this is not new enough to use SCC: 0x700 0x0032E003 and we have to use SCC: 0x018 0x00001FFF instead, if I have understood that page right.
What is currently in the images.txt, board.txt files it is booting with? Is 'sys_flags' bringup already selected or not?
It is not.
OK. The Xen page says "Xen only supports sys_flags bring up". So what does this mean? Can we turn it on, can we change Xen? (I don't know why I have this job - I have absolutely no idea what I am talking about :-) All I get to do is badger about 5 other people who _do_ have some clue :-)
Cheers for your help so far, the murk may be clearing slowly...
Wookey
On Thu, Sep 05, 2013 at 01:54:58PM +0100, Wookey wrote:
+++ Antonio Terceiro [2013-09-04 20:41 -0300]:
On Tue, Sep 03, 2013 at 05:44:13PM +0100, Wookey wrote:
Xen requires a non-standard boot with the hypervisor loaded as well as the kernel before booting (and rootfs unpacked).
Julien says we need to do (at least some of) the config on this page before running XEN: http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Vexpress which involves disabling the A7 (required), and one way of loading the stuff in the right places, which there may be uboot/uefi-based alternatives to?). This config is done in images.txt and board.txt files.
Those files apparently live on the second, internal, microSD card.
The card can apparently be exposed as a fat filesystem over USB (usb-storage). Is that cable plugged in?
It is.
Is there a lava mechanism for accessing this and updating it?
Yes, but at the moment it's only used to replace the UEFI binary with the one that comes with the test disk image. It would be possible to add some code that will also replace other files in there.
Is this code that we have to add to lava and then release into production, or can we issue interactive_boot_cmds sendline/expect foo to get this done for now?
The interactive_boot_cmds thing is used to interact with the bootloader (e.g. sending uboot/uefi commands) over the serial connection after the board is booted into it, so I don't think that would be useful in this case where you want to run commands on the host to fiddle with the Vexpress USB storage.
(Which machine is actually issuing those commands in lava? I haven't got the architecture properly understood in my head, which makes this confusing to think about. Although I guess the idea is that lava wraps all that so I shouldn't need to know.
So LAVA works like this:
- there is a LAVA master server, which serves the web UI and the XMLRPC API - there are multiple worker servers. Each worker node has potentially many boards connected to it. The worker node is responsible for actually running the jobs that get assigned to their boards.
+---- [worker] ---+--- [board] | +--- [board] | [master] ------+---- [worker] ---+--- [board] | +--- [board] | +---- [worker] ---+--- [board] +--- [board]
Each vexpress board is plugged via USB to the host that controls it, and we have special code that handles the support for reflashing the firmware via the USB mass storage interface. So you are right in which you don't need to care about which the actual connections: whatever worker/board pair that gets the job will be able to deal with it.
I think I do need to know the device name of this SD-card if we are doing mount;change file;unmount in interactive_boot_cmds.
You cannot really send arbitrary commands to run on the LAVA server.
And presumably $something needs to remember to put the default one back again after this job runs, which is the real reason why lava needs to support it properly?
Correct.
So how the UEFI replacing works is that lava will mount the USB storage interface, make a backup of the UEFI binary there, copy over the UEFI binary provided by the test image, and boot. When recovering the board (for a new job for example), LAVA will restore the UEFI binary from the backup.
One easy way to support this use case would be to also read board.txt from the image in the same directory where the UEFI binary already is, and use that to replace the board.txt before booting.
But it would be overkill to require a new image to be built just for this: will the xen testing have its own special image?
If yes, then we are lucky and you could just include the desired boot.txt in the image, and I could write the code on the LAVA side to handle it.
If not (i.e. you will be using stock images), then we need to figure out another mechanism for passing the desired board.txt, which shouldn't be hard, but will require more effort than the easy way.
linaro-validation@lists.linaro.org
-
Antonio Terceiro -
James Tunnicliffe -
Wookey