On Mon, 19 Dec 2016 11:12:13 +0000 Neil Williams codehelp@debian.org wrote:
On Mon, 19 Dec 2016 10:35:58 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi,
Since HiKey is the only board (that we care about) supported in V2 right now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be deployed as V2 only instances in staging.
That's a question for the lab team, there are hardware implications beyond just the extra HiKey boards due to the extra Cambrionix hub support.
That would allow us to start migration work without waiting for Juno/X15, get some hands-on with ImageReports2 replacements, etc.
The 1 HiKey on staging is not particularly busy and LXC HiKey V2 test jobs are quicker than V1. This can be further improved by not repeating the same tests every time when all that's happening is development of the test definition and supporting scripts.
However, there are issues with how adb is used within these test definitions. https://github.com/mwasilew/AndroidViewClient.git has issues with certain builds of adb. I have results from this test job but that was using Debian unstable: https://staging.validation.linaro.org/scheduler/job/157649/definition Re-submissions of this testjob have failed due to a newer version of adb becoming available inside the LXC which caused the view client to fail.
Note also that even if you want to use a custom-built adb during the test shell (which is supported), adb and fastboot must still exist in the packages: list to the LXC as these binaries are essential to deploying the test images onto the HiKey in the first place.
Any feedback on improving the documentation of how to develop test jobs based on HiKey, Android and LXC will be appreciated but the core of how LXC works, what it does and how it differs from a full VM is beyond the scope of the LAVA documentation. (The docs do include links for further reading on LXC.)
Also - how is it with the database? Is there some dev/playground database that we could submit results to during migration so that we do not pollute database with errors and mistakes?
V2 supports omitting specific results from queries and charts but the simplest way to handle this is to use the metadata support to mark the V2 jobs as "in development" and then match for or against that piece of metadata in queries based on the V2 results. Deleting data is the wrong option here.
staging.validation.linaro.org is not intended to provide CI for teams other than the LAVA software team. However it is useful in this complex situation where the V1 requirements are so different to the V2 support. Once a set of V2 test definitions have been developed, the Lab team can prepare to add / divide the HiKey devices in production to support release testing using V2.
Please use staging as a way to develop the V2 test definitions - once a test job is working, work can move on to another definition until there is enough coverage to allocate HiKey devices in production to V2 only. It is vital during this process that only one element is changed at a time, so stick to the one build (I've been using 238) and re-test all V2 definitions when changing either the build or the LXC support or the scripts which are common to multiple test jobs.
The files from build 238 are preserved here: http://images.validation.linaro.org/functional-test-images/hikey/ (To cover for when snapshots eventually removes those files.)