On Mon, 19 Dec 2016 10:35:58 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi,
Since HiKey is the only board (that we care about) supported in V2 right now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be deployed as V2 only instances in staging.
That's a question for the lab team, there are hardware implications beyond just the extra HiKey boards due to the extra Cambrionix hub support.
That would allow us to start migration work without waiting for Juno/X15, get some hands-on with ImageReports2 replacements, etc.
The 1 HiKey on staging is not particularly busy and LXC HiKey V2 test jobs are quicker than V1. This can be further improved by not repeating the same tests every time when all that's happening is development of the test definition and supporting scripts.
However, there are issues with how adb is used within these test definitions. https://github.com/mwasilew/AndroidViewClient.git has issues with certain builds of adb. I have results from this test job but that was using Debian unstable: https://staging.validation.linaro.org/scheduler/job/157649/definition Re-submissions of this testjob have failed due to a newer version of adb becoming available inside the LXC which caused the view client to fail.
Note also that even if you want to use a custom-built adb during the test shell (which is supported), adb and fastboot must still exist in the packages: list to the LXC as these binaries are essential to deploying the test images onto the HiKey in the first place.
Any feedback on improving the documentation of how to develop test jobs based on HiKey, Android and LXC will be appreciated but the core of how LXC works, what it does and how it differs from a full VM is beyond the scope of the LAVA documentation. (The docs do include links for further reading on LXC.)
Also - how is it with the database? Is there some dev/playground database that we could submit results to during migration so that we do not pollute database with errors and mistakes?
V2 supports omitting specific results from queries and charts but the simplest way to handle this is to use the metadata support to mark the V2 jobs as "in development" and then match for or against that piece of metadata in queries based on the V2 results. Deleting data is the wrong option here.
staging.validation.linaro.org is not intended to provide CI for teams other than the LAVA software team. However it is useful in this complex situation where the V1 requirements are so different to the V2 support. Once a set of V2 test definitions have been developed, the Lab team can prepare to add / divide the HiKey devices in production to support release testing using V2.
Please use staging as a way to develop the V2 test definitions - once a test job is working, work can move on to another definition until there is enough coverage to allocate HiKey devices in production to V2 only. It is vital during this process that only one element is changed at a time, so stick to the one build (I've been using 238) and re-test all V2 definitions when changing either the build or the LXC support or the scripts which are common to multiple test jobs.
On 15 December 2016 at 12:35, Neil Williams codehelp@debian.org wrote:
On Thu, 15 Dec 2016 10:49:19 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi Neil/guys
thanks for the update. In your presentation you talk about pre-requisites for the V2 migration and one of them is "Make all devices on all instances usable with V2". When will this happen for Juno, HiKey and X15?
HiKey V2 support using LXC is now available on staging but there are lab changes required to allow HiKeys in production to support V2. Any one HiKey board cannot support V1 and V2 simultaneously due to limitations of the HiKey hardware. Tests using Android and LXC are running on staging to support migrating the test definitions away from MultiNode to V2 using LXC: https://staging.validation.linaro.org/results/157649
It's up to the lab team to manage the upgrade of hikey devices in production to V2 support, dependent on expanding the initial test jobs on staging to the full range of tests required for LMG.
Juno Android support is in development.
X15 support in LAVA V2 has not been requested and has hardware requirements which need lab work. I'm anticipating that X15 support is desired for LAVA V2 but no work has yet been done on V2 support for X15.
I'm guessing that LMG would need few months on top of that milestone to migrate our test jobs, re-establish dashboards/trends, setup pre-merge testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions entirely) happens.
As shown in the slides and agreed at LAS16, 3 months V2 data will be required for all teams before the decision is made to disable V1 submissions. You can listen to the presentation again if you want to hear the discussion, using the link provided.
http://connect.linaro.org/resource/las16/las16-503/ (Note: the video will auto-start, be aware.)
--
Neil Williams
-- With best regards,
Jakub Pavelek
Linaro Mobile Group project manager Linaro.org│Open source software for ARM SoCs