Lava-announce

lava-announce@lists.linaro.org

68 discussions

2016.4 backports uploaded - a note on django and django-tables

by Neil Williams

Updates of existing backports are not usually announced in this way but there is a note to consider, so I'm covering that here. Independent of the update of the lava-server and lava-dispatcher backports to 2016.4, admins using jessie-backports do need to be aware of a issue which is still pending related to django and django-tables. A backport of django-tables is pending, waiting for these two packages to also be accepted into jessie-backports: https://ftp-master.debian.org/new/django-haystack_2.4.1-1~bpo8%2B1.html https://ftp-master.debian.org/new/python-fudge_1.1.0-1~bpo8%2B1.html These are required before django-tables can build inside jessie-backports and therefore before I can upload a build of django-tables for jessie-backports. Irrespective of the version of LAVA being used, if django itself is installed from jessie-backports, the version of django-tables in jessie will cause a HTTP500. The fix is to either downgrade django back to the version in jessie or to pull in the version of python-django-tables2 from *testing*.: https://packages.debian.org/stretch/python-django-tables2. It is this upstream version which is to be backported to jessie-backports. Once these last two dependencies clear the queue for backports, I'll make the backport of django-tables and then admins can have django and django-tables from jessie-backports which will work. i.e. the installed django needs to match up with the installed django-tables - either both from jessie or, once available, both from jessie-backports. This isn't a problem in LAVA as such, it affects LAVA and I'll have the fix available just as soon as the queue clears within Debian. Finally, LAVA is likely to move to using django from jessie-backports once this issue is fixed so that instances gain the security fixes in the django 1.8 release - at that point, LAVA will ensure that the updated django-tables is pulled in at the same time. LAVA has been running with django 1.8 (and django1.9) for developers for some time, testing with 1.8 will continue in the meantime. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

9 years, 7 months

2016.4 production release

by Neil Williams

lava-server changes: Add support for python-django-debug-toolbar Deleting V1 filters now cascade delete image chart filters. Reduce the number of SQL queries used on common pages. Improve scheduler iterative performance. Add support for deleting unused tokens Stop runaway healthchecks in V2. Migrate option_list to argparse for django 1.8 and later. Allow authentication with result export in V2 Drop references to Ubuntu beyond 2016.9.post1 Implement omitting individual results from queries in V2 Indicate omitted results and allow including them back in. Add a management command for refreshing queries Change V1 measurement field to be float only. Clean up top-level documentation Introduce limit to queries in V2. * Suggest python-django-debug-toolbar * Refresh all V2 queries during package postinst to ensure materialized views are available. lava-dispatcher changes: Allow for adb not being available on some arches Support ramdisks that are not compressed Move the ramdisk header requirements into the device template Fix calling the Bzr unit tests Use tar flags from deployment data when unpacking overlay kvm-aarch32: introduce device type for 32-bit guest on 64-bit KVM host kvm-aarch64: expose a virtual random number generator (RNG) to the guest Provide an apache2 config for V2 workers Initial support for device using GRUB bootloader in V2 Add in the ability to halt the V1 master image before powering off No longer require ssh options or identity_file Don't use -p port in SCP options https://validation.linaro.org/ will be updated in due course (probably at the start of next week). Packages have been uploaded to Debian (unstable) and will appear on the mirrors in due course. Once these versions migrate into stretch, backports can be made to jessie-backports. https://tracker.debian.org/pkg/lava-server https://tracker.debian.org/pkg/lava-dispatcher https://qa.debian.org/developer.php?email=pkg-linaro-lava-devel%40lists.ali… The production repo also has this release. https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava… https://validation.linaro.org/static/docs/v1/installing_on_debian.html#lava… Information will also appear at http://releases.linaro.org/components/lava/ in due course. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

9 years, 8 months

Deprecating the old dispatcher

by Neil Williams

The 2016.2 release marks the start of the migration to the pipeline dispatcher. The previous dispatcher code, the documentation for that code and the support for JSON job formats are now *deprecated*. As a result, the corresponding lava-server code has also been *deprecated*, this includes Bundles, Filters and Image Reports (original and 2.0). The deprecated Dashboard is replaced by Results with Queries and Charts which have been ported to the data produced by the pipeline testjobs. Future development within LAVA will be based solely on the pipeline dispatcher and associated server side objects. Support for the deprecated dispatcher will be maintained for future releases only during the migration and *will be removed* once the migration is complete. The migration is expected to take most of 2016. Individual devices and particular items of job support will be gradually disabled until the migration completes for all devices and all jobs. The migration process involves: 0: Adding pipeline support for devices and deployment methods 1: Migrating user submissions and automated submissions to the pipeline support. For the Linaro LAVA instances, this will happen within the particular teams inside Linaro. 2: Making selected devices exclusive to the pipeline so that JSON jobs can no longer be submitted for those devices. At this point, support for JSON jobs which rely on those devices will cease. 3: Once all devices are exclusive, reject all JSON submissions. 4: Disable the old scheduler daemon on the LAVA instance. At some point, probably in 2017 - a month or two after JSON submissions cease - the migration will complete by executing: 5: Removal of code support for the old dispatcher in lava-dispatcher and lava-tool codebases and associated documentation. 6: Removal of code support for database objects specific to the old dispatcher, like Bundle, in lava-server with associated documentation. This will involve the deletion of the Bundle data as well as image reports, filters and BundleStreams. TestJob objects which are not pipeline jobs will also be deleted. 7: Removal of the rest of the code which is still dependent upon or only used to support deprecated objects and functions. 8: Modification of code which only exists to isolate the deprecated objects from the pipeline objects. e.g. newly created jobs or devices will default to pipeline support. LAVA instances outside Linaro will need to manage their own migrations during 2016 if updates are to be applied during 2017. It is not possible to retain the database objects without the deprecated code, so owners of individual instances may choose to create an archive instance which has no online devices, accepts no submissions, just provides access to a snapshot of the data at the time that JSON submissions ceased. To maintain access to the archived data, this instance must not upgrade to LAVA releases after the archive is created and the original devices should persist in the database but be kept in the Offline state. Devices should also have submissions restricted to a single administrator account for which no API token exists. More information on the details of the migration and the state of support for jobs using the old dispatcher will be announced on this mailing list. As a reminder - lava-announce is a read-only list. Posts to this list are only made by the LAVA software team. Replies need to be directed to linaro-validation(a)lists.linaro.org - your email client should do this for you, using the Reply-To header added by mailman. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

9 years, 10 months

Issues with Django OpenID Authentication

by Neil Williams

Please reply to linaro-validation(a)lists.linaro.org Please let us know if you are using OpenID authentication with LAVA. Newer versions of django will make it impossible to support django-openid-auth in Debian unstable and testing. The version of django-openid-auth in Jessie can continue to be used, so we would like to know how many users want to continue with this support. OpenID as a protocol has been dying for some time and Linaro has moved over to LDAP, which is fine if LDAP is already available. The time pressure for this change is coming from the schedule to get the latest django and the latest lava packages into Ubuntu Xenial 16.04LTS which means that support needs to be implemented in the 2015.12 or 2016.1 LAVA releases. This is why this is quickly following the trusty change. We have been aware of the issues with django-openid-auth for some time, it was only when we had completed the move of the Cambridge lab to LDAP that changes involving django-openid-auth could be considered. If you are using OpenID authentication (e.g. using Launchpad or Google OpenID), please let us know. If you would like to see some other forms of authentication supported, also let us know. We can investigate Python Social Auth (http://psa.matiasaguirre.net/), if there is interest. If we don't hear from users who want django-openid-auth support for use on Debian Jessie, we will drop django-openid-auth support from all lava builds. This will leave LDAP and local Django accounts in 2015.12. If anyone has experience of other django authentication modules, also let us know. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

10 years

Freezing support for LAVA on Ubuntu Trusty 14.04LTS

by Neil Williams

See also: https://lists.linaro.org/pipermail/lava-announce/2015-November/000002.html which was also sent to these lists (except linaro-dev). So far, nobody has come forward as a Trusty user. The only Trusty instance of which we are aware is already due to migrate to Debian Jessie. The LAVA software team are now applying updates which will freeze LAVA software support for Ubuntu Trusty at 2015.9 for lava-dispatcher and 2015.9.post1 for lava-server due to the complexities of supporting both django1.6 and the current django1.7 in Jessie and django1.8, possibly django1.9 by the time Debian Stretch is released. The last packages for Ubuntu Trusty 14.04LTS will be: lava-server 2015.9.post1 lava-dispatcher 2015.9 Once these changes are applied, the Debian packaging used to build future versions of LAVA packages will prevent builds against django1.6 and prevent installation if django1.6 is found, in order to prevent database corruption. This means that Trusty users will not be able to use the results of the dispatcher refactoring. Ubuntu Xenial Xerus - which is planned to be the 16.04LTS in April 2016 - is expected to pick up LAVA software releases from Debian up until the 2016.1 release (possibly 2016.2) and is also expected to be using django1.8. The next Debian stable release (Stretch), for which no date has yet been set, may use django1.9. Initial attempts at migrating a test instance from Trusty to django1.7 did not go well and the migration from Trusty to Xenial cannot be supported by the LAVA software team - the recommendation is to go directly from 2015.9 on Trusty to the same version available for Debian Jessie but there will still be work to be done to prepare and implement the migration which will be instance-dependent. Documentation is being added to assist with this migration but there will remain risks of data loss which will need to be managed for each instance. It is imperative that anyone using Trusty has an up to date backup of the postgresql database dump before considering any migration. If the existing data is to be dropped, a new install on Debian Jessie is recommended. It is not possible for the LAVA software team to support all versions of django from 1.6 to 1.9 - particular problems are known when going from django1.6 to django1.7 as the methods to migrate the lava-server database changed fundamentally in django1.7. Notes are being added to the documentation on the trusty branch based on 2015.9 to be released within lava-server 2015.9.post1 and to the documentation in the master branch (which will go into 2015.12). All future builds of LAVA software will now be made and uploaded only to Debian and releases.linaro.org. So far, nobody has come forward who is willing to maintain packaging for LAVA software on any distribution other than Debian. As the refactoring proceeds, we expect that it will become easier to package LAVA for other distributions but the migration to the refactoring must be complete first. Everyone interested in or using LAVA is encouraged to subscribe to the lava-announce mailing list which is low volume and only used for substantial changes like this. https://lists.linaro.org/mailman/listinfo/lava-announce See also https://validation.linaro.org/static/docs/support.html -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

10 years

Calling all Trusty users of LAVA

by Neil Williams

This is a call to *all* users of LAVA on Trusty - please let us know who you are, what you're needs are and whether you are able to migrate to Debian Jessie instead of going from Trusty 14.04LTS to Xenial Xerus 16.04LTS. If you are using LAVA on any release of Ubuntu older than Trusty, the only advice from the LAVA team is to immediately migrate to Debian Jessie. If you are using LAVA on Wily Werewolf or Vivid Vervet, you should also consider testing the migration to Xenial and comparing with a migration to Debian Jessie, as below. LAVA is looking at a fix for the 2015.9 release but Django1.9 is in beta release already. There are issues with django1.8 which are in development. Currently, we are proposing that this update will be applied to 2015.9 and made into a frozen release. The master branch of LAVA will continue to develop and will need to use more features only available in Django1.7 and later. Documentation relating to installing Trusty would then be deprecated and removed in subsequent releases from the master branch. Equally, future database migrations on the master branch would no longer provide south support, using the django migration support provided by django1.7 and later. So these releases would not be built for Trusty - leaving only the frozen branch. Due to the complexity of supporting django1.6, it is unlikely that updates will be available for the frozen branch once this happens.. The changes in the lava-server due to the ongoing refactoring will mean that users of Trusty will be unable to migrate to pipeline support until the server has also been migrated to Xenial 16.04LTS. In addition to this, there is concern that migrating from Trusty and django1.6 all the way to django1.8 or possibly django1.9 in Ubuntu Xenial 16.04LTS is going to be problematic and the LAVA team will be unable to assist in most cases. The alternative is for someone with a reasonably complex lab running Ubuntu to take up a role as tester of the frozen branch *and* responsibility for patches which can maintain trusty support and migration to Xenial 16.04LTS. The problem then will be that it will be a very large transition when 16.04LTS actually becomes available - only for the same lag to start all over again. I'm unsure when Xenial will close the window for migrations from Debian into Xenial - I expect that the 2015.12 release of LAVA will migrate, I expect that 2016.1 will migrate too but I cannot be sure about 2016.2 or 2016.3. That migration is completely outside the control of the LAVA software team. https://launchpad.net/ubuntu/xenial/+source/lava-server Everyone considering staying on Ubuntu is advised to try a migration to Xenial *now* - in a VM, with and without a recent backup of your database and logs. Xenial currently has 2015.11. Also compare with a migration to Debian Jessie by dumping and reimporting the database. In each case, ensure that the permissions on /var/lib/lava-server and sub-directories are retained from the original. Please talk to us and test out what you are going to do. https://validation.linaro.org/static/docs/support.html -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

10 years, 1 month

Skipping 2015.11 availability on Trusty

by Neil Williams

It has become impractical to maintain support for the latest production releases on Ubuntu Trusty 14.04LTS due to behaviour required by the server side changes needed by the dispatcher refactoring. These are important changes and are impractical to backport to the version of python-django available in Trusty. I have tried working around the issues but it involves pulling more dependencies from Jessie or Vivid and in my tests, the existing installation became confused and failed to install due to problems with the database migrations. The LAVA software team do not have the resources to maintain backports of the complete dependency chain in Ubuntu, it is a much more manageable set in Debian Jessie (where the same migration went without issues on multiple machines). The changes in 2015.11 are almost exclusively related to the dispatcher refactoring and therefore there is expected to be limited impact on current installations on Trusty. For the next release, we will investigate code changes which omit the new support for Trusty builds but this will have the effect of preventing those installs from working with the pipeline and the new YAML job submissions. The documentation for Trusty will also be updated, if necessary as a hotfix to 2015.9 just for Trusty. If the code changes do not work cleanly, then the current 2015.9 (and the documentation hotfix to follow) would become the last LAVA releases for Ubuntu Trusty 14.04LTS. Users of Ubuntu Trusty 14.04LTS are reminded that a new LTS is due for release in April 2016 but that the LAVA software team are not using Trusty any longer. Any issues with the migration from Trusty to the new LTS are outside of our control. Migrating an instance from Ubuntu to Debian would involve dumping and re-importing the database - bear in mind that Trusty has postgres9.3 and Jessie has 9.4, so care will be needed to migrate the postgresql cluster correctly - there is documentation for that migration in the lava-server docs. It is not recommended to attempt to migrate the OS itself from Ubuntu to Debian (or vice-versa), a reinstall is a better option, so make a backup and ensure that the backup works. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

10 years, 1 month

New mailing lists and refactoring changes

by Neil Williams

After discussions at the LAVA Users Forum at Connect SFO15 [0], the LAVA team have arranged for some changes in the mailing lists to make it easier to announce and manage the upcoming changes from the ongoing refactoring. https://lists.linaro.org/mailman/listinfo/lava-users - a discussion list amongst users for users, with some admins listening in and helping out. Users in this sense includes test writers and administrators of LAVA instances outside Linaro. It is particularly aimed at those running LAVA in places where Linaro is not necessarily aware of the install. I'm not asking for LAVA instances to declare themselves, it is free software after all, it is an offer of assistance and a request that users take advantage of the support available to manage the implications of the refactoring for their own use cases. Please feel free to initiate threads and discuss issues and problems on the lava-users list. As a reminder, for those who use LAVA but have not recently been to a Connect, the dispatcher is being refactored and this had led to advancements and modifications in the lava-server as well as a completely re-written job submission format. LAVA is retaining compatibility *only* with the Lava Test Shell Definitions (the YAML files people are currently using) and there can be no automated way of converting existing JSON job submissions to the new job submission format (which uses YAML to allow for comments, amongst other improvements). The timetable for these changes is expected to cover most of 2016. After https://validation.linaro.org/ has migrated to the new code, the current dispatcher code will start to be removed from the upstream in 2017. One result of these changes is that as the Linaro production instance migrates away from the current dispatcher codebase, there is a likelihood of some breakage in some parts of the current dispatcher. (This is one of the primary reasons why the dispatcher is being refactored.) The refactoring introduces a lot of benefits, including much more robust communication between the workers and the master, removal of configuration on the workers so that admins only change things in one place, a lot of new methods within the dispatcher to support new types of test and a much cleaner, more modular, codebase for future development. Also, the Linaro Validation Mailman List <linaro-validation(a)lists.linaro.org> is to be renamed (keeping linaro-validation as an alias) to lava-devel for consistency with lava-users and lava-announce. There will be a separate email to the linaro-validation list about these changes shortly. These new mailing lists will be included in the documentation included in next release of the LAVA software (2015.10). Documentation of the new code is already in the master branch of lava-server and installed via the lava-server-doc package and the initial code for the refactoring is being developed in parallel with the current code. This list will be used to announce when the refactoring codebase is ready for admins to begin migrating tests and test writers to the new code and methods - currently, the new dispatcher is developer only. At SFO15, there was also an update on the state of the refactoring. [1] lava-announce will continue to be a low volume list. Reply-To has been set to the linaro-validation mailing list (transparently migrating to lava-devel once the linaro-validation alias is in place). The LAVA developers would ask that everyone running LAVA would subscribe to at least the lava-announce mailing list, to help with the migration to the new support. Videos will be available for most meetings at SFO15 in due course. The slides are already available. [0] https://sfo15.pathable.com/meetings/302656 [1] https://sfo15.pathable.com/meetings/303074 -- Neil Williams ============= Tech Lead, LAVA software team. neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

10 years, 2 months

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Lava-announce