On 21/12/17 16:09, Guillaume Tucker wrote:

On 21/12/17 15:53, Neil Williams wrote:

On 21 December 2017 at 15:37, Guillaume Tucker <
guillaume.tucker@collabora.com> wrote:

On 21/12/17 15:02, Neil Williams wrote:

On 21 December 2017 at 14:51, Guillaume Tucker <
guillaume.tucker@collabora.com> wrote:

On 21/12/17 14:49, Guillaume Tucker wrote:

On 19/12/17 19:02, Guillaume Tucker wrote:

[...]

I've now added some patches to generate the FIT image using

mkimage using a similar approach as for uImage, see all the
current reviews here:

https://review.linaro.org/#/q/topic:T6321-depthcharge

As Tomeu pointed out, the "mkimage -f auto" option can be used to
automatically generate a .its structure without actually using
any intermediate file.  There are a few things that can't be done
with this simple mode but as far as I can tell it should cover
our needs for Depthcharge.

The LXC support remains available if particular tests need customised

handling or options (or if a patched version of mkimage is needed).

The FIT format has been around for long enough for it to work
with standard mkimage, I would hope.  But yes, if some LAVA labs
use an older distro with an incompatible version then LXC would
be a way to keep the version of mkimage under control.  It would
however make the test definitions a bit more complicated and
depthcharge-specific, which is something we would prefer to avoid
for kernelci.org in particular.

So I'll see how it goes with running mkimage on the dispatcher
and with various versions of mkimage.

I still need to compare various versions of mkimage.

So Debian Stretch has mkimage 2016.11 which supports the "-f
auto" option.  However, Debian Jessie has mkimage 2014.10 which
does not and really needs an its file to generate an FIT image.

There's no backport to jessie-backports currently.

Add a check the validate() based on infrastructure_check() and see if the
version can be checked for compliance - raise InfrastructureError
exception
if not. Then it will be up to admins to base their dispatchers on Stretch.
We'll be moving to Stretch being the recommended base for LAVA in the New
Year and dropping support for Jessie at some point in 2018.

Is it OK to require a recent enough version of mkimage as in

Stretch in order to generate these FIT images, or should we
generate the its file to support older versions?

I started to look at options to generate the its file, it can be
a bit cumbersome but should not be technically challenging...
Would be nice to avoid it if possible to keep the code simpler.

Alternatively, it should hopefully be easy enough to backport
u-boot-tools from Stretch to a local Jessie repository.

Also, dtc is not installed on ci.linaro.org so the Gerrit

Compare with existing unit tests - if you need a tool in the path which is
not always installed, use a @skipIf decorator with the
infrastructure_check
on that binary.


automated unit tests are now failing since I added a check:

    14:35:27 JobError: Invalid job data: ["Cannot find command 'dtc' in
$PATH"]

OK so it should all be good now, I've sent new patch sets that
addressed all the existing comments in Gerrit.  I'm hoping to get
the Depthcharge tftpboot command merged upstream soon, probably
in January though.  Meanwhile I guess I can start updating the
LAVA documentation.

OK, so I remember you saying not to merge until the support is upstream. Do
you need to rely on u-boot-tools from Stretch with the latest changes?

Yes, the changes I've sent require the mkimage (u-boot-tools)
version from Stretch.  This one does not depend on dtc.

We're looking into the issues around Stretch because:

0: The Linaro lab are moving to Stretch in January 2018
1: Jessie is losing security support in July 2018
2: A number of packages needed for python3 support of lava-server are only
currently in Stretch
3: Django 2.0 is on the horizon without python2 support.

So, as announced, we will be moving to Stretch during 2018 - and ending
support for Jessie before July 2018. That may help you decide on what
support you need from mkimage and friends.

I guess it's good timing to do this while LAVA support for Jessie
is being dropped.  I would prefer to see this merged before July,
but if need be we can keep patching our lab for a few releases.

A quick update: all the changes have now been merged and are in
the LAVA 2018.1 release.  Thanks!

I've backported the changes on top of 2017.9 as that's what we're
using in our lab and added a veryon-jaq device.  It's now being
added to kernelci.org as well.

As we're currently running Jessie on the dispatcher, I rebuilt
the Stretch package version of u-boot-tools (2016.11) for Jessie.
This was trivial, no porting effort was required and it's working
fine.

Here's an example run:

  https://lava.collabora.co.uk/scheduler/job/1040962

Guillaume