On Sat, 9 Jan 2016 09:53:58 +0100 Alexander Couzens lynxis@fe80.eu wrote:

Hi,

I would like to use lava for coreboot.

Please clarify your idea of a test plan for this support:

0: Is this a CI loop of coreboot builds? (coreboot binary changes every test) 1: Is this using a fixed version of coreboot to test performance of coreboot? (coreboot binary changes at admin intervention, possibly with changes to the test jobs) 2: Is this using coreboot to test the kernel? (making coreboot devices available for testing)

coreboot needs to installed on flash which is soldered on the mainboard. To flash/install coreboot you need an external SPI flasher e.g. the Beagleboneblack can do this. Using a SPI Testclip, it's possible to flash the SPI chip in-circuit. The SPI chip and partially the mainboard needs to be powered externally.

How does the board get reset to a known working version, can it also be set to a different bootloader entirely?

The important thing for LAVA is that, no matter how badly test 1 goes, test 2 can start with a clean slate. The device must come into a usable mode even if test 1 'bricks' the device.

My concrete test setup is a thinkpad x60. To flash coreboot, I have to do the following:

1 disconnect the power to DUT 2 connect external 3.3V to the SPI chip (and partially DUT) 3 connect the beagleboneblack's SPI pins to the DUT 4 flash coreboot via beagleboneblack 5 disconnect beagleboneblack

Depending on how that is done, you may need hardware to automate that.

6 disconnect external power supply 7 power the x60 8 press the power button of the DUT

For LAVA, you need to bypass that button.

9 run test on linux

I've managed to do all the hardware work.

• control the power to the DUT via pdudaemon using a ubiquity power

[1].

• control external 3.3V via raspberry gpio's on a relais card
• control powerbutton via raspberry gpio's on a relais card
• put beagleboneblack's spi pins into high-z via dts-overlays [2]

So does that mean that you've got relays to automate all of the steps? The job could now run unattended?

TLDR;

I'm looking for feedback about implementing a lava for coreboot.

What's a good way to integrate the coreboot flashing flow into lava? I started writing a pipeline driver for flashrom [3].

OK, the pipeline is definitely the right way to do it.

How should I integrate the raspberrypi's gpios controlling important things? Using pdudaemon again?

PDU is useful as a way of scheduling power events where there are multiple jobs but each "PDU" can only act on one operation at a time. You could use any scripting method - the point is that you need a call in the pipeline dispatcher to poke the pi at the right points. In the pipeline, that would be a protocol (like vland). Actions in your deploy code then make calls as dictated by the test job.

Do you use safeguards to protect damage from devices?

We don't have GPIO code in pipeline dispatcher or the old dispatcher. It is going to need to be behind some other interface that can manage things to prevent damage, as much as is possible.

E.g. It's possible to enable both powersupply for DUT and might kill one.

PDU daemon can't reasonably work with that but the pipeline can. You don't have to expose the protocol API to the test writer, the actions can poke it directly, particularly to do setup and other "sensitive" operations. e.g. vland is just such a protocol - it takes parameters from the test writer but it makes the API calls itself and will fail the job if it considers that those parameters will result in a bad set of calls. (The remote vland daemon also has a set of safeguards and the protocol also has to handle the daemon refusing to execute a call.)

[1] https://github.com/mattface/pdudaemon/pull/1 [2] https://github.com/lynxis/bbb_highz_spi [3] https://github.com/lynxis/lava-dispatcher/blob/master/lava_dispatcher/pipeli...

I must say, it's quite a validation of the refactoring that you've got to this stage on your own. Congratulations.

The issue of GPIO management could be done in a couple of ways:

0: The protocol code itself could do it. Create a new protocol which uses parameters in the testjob and device configuration to prevent collisions and implement the safeguards. The protocol code runs on the dispatcher, so needs some way of poking the actual pins, e.g. a TCP/IP socket or a call to a daemon. That daemon should be fairly dumb - it does what it is told, immediately or flat out refuses. To cope with multiple jobs and multiple calls, these operations need to be fast.

1: You could borrow from lavapdu and write a custom daemon to implement the safeguards (this is just vland operates) - the protocol then simply calls that daemon which runs locally and pokes the pins *only* if the safeguards in the protocol and the daemon are met.

2: Write a protocol which implements the safeguards and then pokes lavapdu. This, I think, is the most risky as it hands off the critical issue of timing to a separate daemon with it's own intrinsic latency and scheduling. You want the scheduling of the actual operations to be under the sole control of the process making the decisions about whether the operation should happen *now* or wait until something else has completed.

Deciding between 0 and 1 depends somewhat on how *long* each operation will take. vland is a daemon (case 1) because it has to telnet onto each switch and setup ports - that can take minutes per vlan. Therefore, vland has to cope with multiple requests arriving simultaneously from different jobs and then schedule the actions. To get the scheduling right, it has to be a daemon itself, with state, not rely on a daemon with scheduling priorities of it's own like lavapdu.

vland is not merged into master yet, the pipeline protocol code is here: https://review.linaro.org/#/c/7719/

The actual vland daemon code is here: http://git.linaro.org/lava/vland.git

Hope this helps.

On Mon, 1 Feb 2016 17:49:39 +0100 Alexander Couzens lynxis@fe80.eu wrote:

Hi Neil,

thanks for your help.

I got now to the state of basic testing. But there are some hacks, which needs to be solved ;).

A test run includes now the external flashing of the BIOS chip. Afterwards the system will boots from hard-drive and lava can connect via ssh.

deploy: to: flashrom is ignoring the os:, but lava requires an os: field.

The 'os' field will be important for testing - you need to hook into the Overlay classes. The flashrom deploy (or a second deploy) will need to put the overlay onto that hard-drive and unpack it. It is the 'os' of the rootfs on the hard-drive that matters.

Therefore, you should be using a second deploy that creates the rootfs to avoid problems of persistence. These are the same problems as having a persistent NFS.

http://localhost/static/docs/dispatcher-design.html#persistence

How is it possible to capture the serial output while it's booting?

Compare with the UBoot Boot classes. There will be a Connection and the actions use that to wait for the prompt. The Connection is a pexpect.spawn, so whilst waiting, the output is captured.

Or does it require another boot method.

Yes - or more likely a reuse or inheritance from the existing classes. Compare with the iPXE support.

At the moment the flashrom deployment "boots" the DUT, because "boot: ssh" expects a fully booted device.

Yes - you'd need something like the secondary media support or a second deploy to actually get a rootfs into which can be installed openssh-server (and doing the installation will require the 'os' field).

You may simply need the ExpectShellSession and AutoLogin support.