Hi Neil/guys
thanks for the update. In your presentation you talk about pre-requisites for the V2 migration and one of them is "Make all devices on all instances usable with V2". When will this happen for Juno, HiKey and X15?
I'm guessing that LMG would need few months on top of that milestone to migrate our test jobs, re-establish dashboards/trends, setup pre-merge testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions entirely) happens.
On 14 December 2016 at 17:03, Neil Williams neil.williams@linaro.org wrote:
Everyone using LAVA should be aware of V1 and V2 and that V1 is going to be turned off. The plan for how to do that has been discussed within the team and within Linaro. The most recent presentation on the migration can be seen here:
http://connect.linaro.org/resource/las16/las16-503/ (Note: the video will auto-start, be aware.)
The slides are available on slideshare:
http://www.slideshare.net/linaroorg/las16503-lava-v2-migration
The dates for each step in the migration are not finalised but will occur during 2017, this is advance notice.
The highlights of the migration are:
0: During 2017, regular announcements will be made on this list before each major change. In addition, the regular release announcements will include details of the major changes in that release.
1: The first major change will be that a new release of lava-server and lava-dispatcher will *disable* V1 submissions entirely. Any queued V1 test jobs will become invalid upon this upgrade,
2: A later release will remove V1 documentation and start removing the V1 source code.
3: A release will be made which *forcibly deletes V1 test jobs, bundles, filters and image reports*. This is required to complete the removal of the V1 source code.
4: The final step of the migration is a release containing no V1 source code at all. Only V2 support will remain. This is essential as the V1 source code is currently blocking some useful additions to V2 as well as plans for future development. The data cannot remain accessible without the V1 source code. There is not and can not be any support for converting V1 data to V2.
Please take this chance to read up on the V2 documentation, including how Results, Queries and Charts replace the deprecated V1 support from filters and image reports.
Work has started upstream on making sure the migration runs smoothly, especially when deleting large numbers of V1 test cases.
Remember: Once the first major step is released, upgrades will stop running any V1 test jobs. Subsequent releases will include changes which force the deletion of your V1 test data.
If you want to preserve your V1 data, consider setting up a read-only reference instance which is then pinned at a particular release of LAVA to prevent deletion of the V1 data.
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-announce mailing list Lava-announce@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-announce
On Thu, 15 Dec 2016 10:49:19 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi Neil/guys
thanks for the update. In your presentation you talk about pre-requisites for the V2 migration and one of them is "Make all devices on all instances usable with V2". When will this happen for Juno, HiKey and X15?
HiKey V2 support using LXC is now available on staging but there are lab changes required to allow HiKeys in production to support V2. Any one HiKey board cannot support V1 and V2 simultaneously due to limitations of the HiKey hardware. Tests using Android and LXC are running on staging to support migrating the test definitions away from MultiNode to V2 using LXC: https://staging.validation.linaro.org/results/157649
It's up to the lab team to manage the upgrade of hikey devices in production to V2 support, dependent on expanding the initial test jobs on staging to the full range of tests required for LMG.
Juno Android support is in development.
X15 support in LAVA V2 has not been requested and has hardware requirements which need lab work. I'm anticipating that X15 support is desired for LAVA V2 but no work has yet been done on V2 support for X15.
I'm guessing that LMG would need few months on top of that milestone to migrate our test jobs, re-establish dashboards/trends, setup pre-merge testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions entirely) happens.
As shown in the slides and agreed at LAS16, 3 months V2 data will be required for all teams before the decision is made to disable V1 submissions. You can listen to the presentation again if you want to hear the discussion, using the link provided.
http://connect.linaro.org/resource/las16/las16-503/ (Note: the video will auto-start, be aware.)
Hi,
Since HiKey is the only board (that we care about) supported in V2 right now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be deployed as V2 only instances in staging. That would allow us to start migration work without waiting for Juno/X15, get some hands-on with ImageReports2 replacements, etc.
Also - how is it with the database? Is there some dev/playground database that we could submit results to during migration so that we do not pollute database with errors and mistakes?
On 15 December 2016 at 12:35, Neil Williams codehelp@debian.org wrote:
On Thu, 15 Dec 2016 10:49:19 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi Neil/guys
thanks for the update. In your presentation you talk about pre-requisites for the V2 migration and one of them is "Make all devices on all instances usable with V2". When will this happen for Juno, HiKey and X15?
HiKey V2 support using LXC is now available on staging but there are lab changes required to allow HiKeys in production to support V2. Any one HiKey board cannot support V1 and V2 simultaneously due to limitations of the HiKey hardware. Tests using Android and LXC are running on staging to support migrating the test definitions away from MultiNode to V2 using LXC: https://staging.validation.linaro.org/results/157649
It's up to the lab team to manage the upgrade of hikey devices in production to V2 support, dependent on expanding the initial test jobs on staging to the full range of tests required for LMG.
Juno Android support is in development.
X15 support in LAVA V2 has not been requested and has hardware requirements which need lab work. I'm anticipating that X15 support is desired for LAVA V2 but no work has yet been done on V2 support for X15.
I'm guessing that LMG would need few months on top of that milestone to migrate our test jobs, re-establish dashboards/trends, setup pre-merge testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions entirely) happens.
As shown in the slides and agreed at LAS16, 3 months V2 data will be required for all teams before the decision is made to disable V1 submissions. You can listen to the presentation again if you want to hear the discussion, using the link provided.
http://connect.linaro.org/resource/las16/las16-503/ (Note: the video will auto-start, be aware.)
--
Neil Williams
On Mon, 19 Dec 2016 10:35:58 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi,
Since HiKey is the only board (that we care about) supported in V2 right now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be deployed as V2 only instances in staging.
That's a question for the lab team, there are hardware implications beyond just the extra HiKey boards due to the extra Cambrionix hub support.
That would allow us to start migration work without waiting for Juno/X15, get some hands-on with ImageReports2 replacements, etc.
The 1 HiKey on staging is not particularly busy and LXC HiKey V2 test jobs are quicker than V1. This can be further improved by not repeating the same tests every time when all that's happening is development of the test definition and supporting scripts.
However, there are issues with how adb is used within these test definitions. https://github.com/mwasilew/AndroidViewClient.git has issues with certain builds of adb. I have results from this test job but that was using Debian unstable: https://staging.validation.linaro.org/scheduler/job/157649/definition Re-submissions of this testjob have failed due to a newer version of adb becoming available inside the LXC which caused the view client to fail.
Note also that even if you want to use a custom-built adb during the test shell (which is supported), adb and fastboot must still exist in the packages: list to the LXC as these binaries are essential to deploying the test images onto the HiKey in the first place.
Any feedback on improving the documentation of how to develop test jobs based on HiKey, Android and LXC will be appreciated but the core of how LXC works, what it does and how it differs from a full VM is beyond the scope of the LAVA documentation. (The docs do include links for further reading on LXC.)
Also - how is it with the database? Is there some dev/playground database that we could submit results to during migration so that we do not pollute database with errors and mistakes?
V2 supports omitting specific results from queries and charts but the simplest way to handle this is to use the metadata support to mark the V2 jobs as "in development" and then match for or against that piece of metadata in queries based on the V2 results. Deleting data is the wrong option here.
staging.validation.linaro.org is not intended to provide CI for teams other than the LAVA software team. However it is useful in this complex situation where the V1 requirements are so different to the V2 support. Once a set of V2 test definitions have been developed, the Lab team can prepare to add / divide the HiKey devices in production to support release testing using V2.
Please use staging as a way to develop the V2 test definitions - once a test job is working, work can move on to another definition until there is enough coverage to allocate HiKey devices in production to V2 only. It is vital during this process that only one element is changed at a time, so stick to the one build (I've been using 238) and re-test all V2 definitions when changing either the build or the LXC support or the scripts which are common to multiple test jobs.
On 15 December 2016 at 12:35, Neil Williams codehelp@debian.org wrote:
On Thu, 15 Dec 2016 10:49:19 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi Neil/guys
thanks for the update. In your presentation you talk about pre-requisites for the V2 migration and one of them is "Make all devices on all instances usable with V2". When will this happen for Juno, HiKey and X15?
HiKey V2 support using LXC is now available on staging but there are lab changes required to allow HiKeys in production to support V2. Any one HiKey board cannot support V1 and V2 simultaneously due to limitations of the HiKey hardware. Tests using Android and LXC are running on staging to support migrating the test definitions away from MultiNode to V2 using LXC: https://staging.validation.linaro.org/results/157649
It's up to the lab team to manage the upgrade of hikey devices in production to V2 support, dependent on expanding the initial test jobs on staging to the full range of tests required for LMG.
Juno Android support is in development.
X15 support in LAVA V2 has not been requested and has hardware requirements which need lab work. I'm anticipating that X15 support is desired for LAVA V2 but no work has yet been done on V2 support for X15.
I'm guessing that LMG would need few months on top of that milestone to migrate our test jobs, re-establish dashboards/trends, setup pre-merge testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions entirely) happens.
As shown in the slides and agreed at LAS16, 3 months V2 data will be required for all teams before the decision is made to disable V1 submissions. You can listen to the presentation again if you want to hear the discussion, using the link provided.
http://connect.linaro.org/resource/las16/las16-503/ (Note: the video will auto-start, be aware.)
--
Neil Williams
-- With best regards,
Jakub Pavelek
Linaro Mobile Group project manager Linaro.org│Open source software for ARM SoCs
On Mon, 19 Dec 2016 11:12:13 +0000 Neil Williams codehelp@debian.org wrote:
On Mon, 19 Dec 2016 10:35:58 +0100 Jakub Pavelek jakub.pavelek@linaro.org wrote:
Hi,
Since HiKey is the only board (that we care about) supported in V2 right now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be deployed as V2 only instances in staging.
That's a question for the lab team, there are hardware implications beyond just the extra HiKey boards due to the extra Cambrionix hub support.
That would allow us to start migration work without waiting for Juno/X15, get some hands-on with ImageReports2 replacements, etc.
The 1 HiKey on staging is not particularly busy and LXC HiKey V2 test jobs are quicker than V1. This can be further improved by not repeating the same tests every time when all that's happening is development of the test definition and supporting scripts.
However, there are issues with how adb is used within these test definitions. https://github.com/mwasilew/AndroidViewClient.git has issues with certain builds of adb. I have results from this test job but that was using Debian unstable: https://staging.validation.linaro.org/scheduler/job/157649/definition Re-submissions of this testjob have failed due to a newer version of adb becoming available inside the LXC which caused the view client to fail.
Note also that even if you want to use a custom-built adb during the test shell (which is supported), adb and fastboot must still exist in the packages: list to the LXC as these binaries are essential to deploying the test images onto the HiKey in the first place.
Any feedback on improving the documentation of how to develop test jobs based on HiKey, Android and LXC will be appreciated but the core of how LXC works, what it does and how it differs from a full VM is beyond the scope of the LAVA documentation. (The docs do include links for further reading on LXC.)
Also - how is it with the database? Is there some dev/playground database that we could submit results to during migration so that we do not pollute database with errors and mistakes?
V2 supports omitting specific results from queries and charts but the simplest way to handle this is to use the metadata support to mark the V2 jobs as "in development" and then match for or against that piece of metadata in queries based on the V2 results. Deleting data is the wrong option here.
staging.validation.linaro.org is not intended to provide CI for teams other than the LAVA software team. However it is useful in this complex situation where the V1 requirements are so different to the V2 support. Once a set of V2 test definitions have been developed, the Lab team can prepare to add / divide the HiKey devices in production to support release testing using V2.
Please use staging as a way to develop the V2 test definitions - once a test job is working, work can move on to another definition until there is enough coverage to allocate HiKey devices in production to V2 only. It is vital during this process that only one element is changed at a time, so stick to the one build (I've been using 238) and re-test all V2 definitions when changing either the build or the LXC support or the scripts which are common to multiple test jobs.
The files from build 238 are preserved here: http://images.validation.linaro.org/functional-test-images/hikey/ (To cover for when snapshots eventually removes those files.)
linaro-validation@lists.linaro.org
-
Jakub Pavelek -
Neil Williams