linaro-validation June 2012

linaro-validation@lists.linaro.org

26 participants
41 discussions

Re: [Linaro-validation] Switching LAVA Deployment to buildout

by Michael Hudson-Doyle

Abner Silva <abner(a)collabora.co.uk> writes: > Hi Michael, > > On Tue, 2012-06-12 at 17:07 +1200, Michael Hudson-Doyle wrote: >> Those of you who have been around for a while will know that we used >> to package LAVA as Debian packages. This had its problems and we >> switched over to the world of pip and virtualenv and >> lava-deployment-tool. >> >> Unfortunately, pip and virtualenv have not really given us the control >> we need to be confident in our deployments, and so I'm proposing to >> change again, this time to zc.buildout. This is less of a change than >> the switch away from debs -- instances of LAVA will still be a concept >> and the lava-deployment-tool script will still be used, at least for now. >> >> The way buildout works is that there is a config file that specifies >> how -- reproducibly! -- to assemble various pieces into a deployment. >> I have a branch of lava-server at lp:~mwhudson/lava-server/use-buildout >> that contains such a config file (and various other related bits and >> pieces), and a branch of lava-deployment-tool that creates a >> buildout-based deployment. >> >> An instance on disk in the new world is similar to but not the same as >> an instance is today. In particular it is not a virtual environment >> -- although for compatibility, it looks like one. There is still a >> $LAVA_INSTANCE/bin/activate script that you can source in bash to put >> that instance's scripts at the front of your $PATH. Inside an >> instance, the etc, run, tmp and var directories server the same >> function as they do today. There is also a new directory, code. The >> code directory contains a number of subdirectories, one for each revno >> of lava-server that has been installed into that instance, and a >> symlink 'current' that points to the currently active one. >> >> There are now two merge proposals on LP: >> >> https://code.launchpad.net/~mwhudson/lava-server/use-buildout/+merge/109768 >> https://code.launchpad.net/~mwhudson/lava-deployment-tool/use-buildout/+mer… >> >> I've made an effort to update the documentation in l-d-t's README, and >> I've put an htmlified version of this file online at >> http://people.linaro.org/~mwh/ldt.html so that might be a good place >> to start. > > As I never worked with buildout before I was wondering if there was a > discussion around this topic before deciding to adopt this new > deployment approach. There was some internal discussion, but this is the first time it's been mentioned on a mailing list I guess. > I would like to read that and see what was discussed and what are the > main advantages of changing how to deploy Lava once again. Is it > available somewhere? No. Basically, as I've been the only person to do major deployments to v.l.o for several months now, I feel I'm in a pretty good position to make decisions about this! And I hate using the current scripts. I wanted to spend a little time with buildout before proposing that we use it to make sure it was worth proposing, and probably spent a little too long: I basically implemented everything except migration from existing deployments. > In case this change really happen, I believe that would be nice to have > some sort of tutorial teaching how to migrate your current setup and > deploy it with buildout, without loosing your database, etc. _Of course_ we will do this sort of thing (we need it for v.l.o!). Obviously, I don't know all the details of your deployment and so it's going to be a slightly stressful time when the change comes to be made -- I hope you can do lots of testing. I'm also happy to learn about details of your deployment and advise you on how to make the transition. > Also, I remember to have long conversations on IRC with you guys and > IIRC there was an agreement that Lava would also be released as deb > packages again, I don't know which conversations you're referring to here, I don't think I was involved. Certainly, in my position as owner of the technical plan, "package server components as debs again" is not on there. > so we would have an option to use pip or debs. How is that going? It's not. > Please, forgive me if I missed any important discussion. Well, you weren't on the phone calls Andy and I had to talk about this. I guess it's just part of the fun of open source: I am most motivated by the problems _I_ see, and fragility of deployment was one of those. I should say that nothing I am doing will prevent you from maintaining your own version of lava-deployment-tool that uses virtualenv & pip but as we won't be using this approach ourselves this might be more work for you in the long run. Cheers, mwh

12 years, 6 months

initial android monkeyrunner tests arrived

by Alexander Sack

Hi, I am very happy to see all those monkeyrunner tests to suddenly show up on the panda side. Thanks for everybody who worked on making those happen! https://android-build.linaro.org/builds/~linaro-android/panda-ics-gcc47-til… Now I am curious if and how we can get a description on what they do? What's the idea? I felt that having them documented on our https://wiki.linaro.org/Platform/QA/TestCases/Android page in a new "automation" section would be one option to keep all our tests documented in a single place. Other ideas? Also, in this build I see two fail for that build. Any idea what's going on? -- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

12 years, 6 months

Re: [Linaro-validation] tip build mmtest failures in lava

by Alexander Sack

On Tue, Jun 12, 2012 at 7:10 PM, Zach Pfeffer <zach.pfeffer(a)linaro.org> wrote: > On 12 June 2012 11:55, Alexander Sack <asac(a)linaro.org> wrote: >> On Tue, Jun 12, 2012 at 6:54 PM, Zach Pfeffer <zach.pfeffer(a)linaro.org> wrote: >>> On 12 June 2012 09:02, Alexander Sack <asac(a)linaro.org> wrote: >>>> Hi, >>>> >>>> looking at origen and snowball lava results from our tip builds I >>>> realized that mmtest's are failing: >>>> >>>> origen: https://android-build.linaro.org/builds/~linaro-android/origen-ics-gcc47-sa… >>>> -> http://validation.linaro.org/lava-server/dashboard/streams/private/team/lin… >>> >>> I get a 404 error for this one ^^^^^^^^^^^^^ >> >> >> you have to be logged into validation.linaro.org to see any raw >> results nowadays. try that. > > That worked better. How can I download the log? Every time I try and > save it its giving me a download.html file? > Good question. Only thing I know is how to open the "inline" view ... I guess there is a secret link to download attachment, but yes, I also have a hard time spotting the download link for it (e.g. stdout.log) :/ Andy? LAVA folks? -- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

12 years, 6 months

Re: [Linaro-validation] LAVA Core

by Andy Doan

lava-core will be very slow progress. lava-core only has untested support for fast-models at the moment. We won't be porting over all our other components in the near term. On 06/12/2012 08:59 AM, Abner Silva wrote: > Hi, > > I haven't heard about Lava Core effort for a while already. What is the > current status? Any news on this front? > > BR, > > -Abner > > On Sat, 2012-04-14 at 20:14 +0200, Zygmunt Krynicki wrote: >> Hi >> >> I'd like to announce a new project in the LAVA family. >> >> (README file follows) >> http://xkcd.com/927/ >> >> Seriously, lava has way to many tiny pieces. It all started when we >> renamed launch-control to lava-dashboard and created lava-server in >> the process. Now it's getting in the way. >> >> So here we are, lava-core is the start of the merge. Subsequent >> projects will slowly merge into the core. Then one day we may just get >> a package that is called 'lava', that ships a program called 'lava' >> that does useful stuff. >> (End of readme file) >> >> The project has been registered on Launchpad already. I plan on >> proposing merge requests that move parts of the non-client pieces >> there. Existing projects shall remain as backwards compatible code >> imports from lava-core. >> >> Thanks >> ZK >> _______________________________________________ >> linaro-validation mailing list >> linaro-validation(a)lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/linaro-validation >

12 years, 6 months

suggestion: let's display the lava test result details inline on android-build

by Alexander Sack

Hi, if you check the android-build LAVA result table for https://android-build.linaro.org/builds/~linaro-android/snowball-ics-gcc46-… you will notice we see quite some details for the benchmarks. However, we don't see the individual pass/fails for the "simple pass/fail" tests. For example: 1. lava, mmtest etc. - don't display what tests succeeded/failed - just the aggregated number 2. glmark2 etc. - display all the individual measurements What can we do to display details for all test runs on android-build? I am in particular keen about seeing the individual "lava" results inline on android-build. -- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

12 years, 6 months

Re: [Linaro-validation] Switching LAVA Deployment to buildout

by Andy Doan

On 06/12/2012 09:10 AM, Abner Silva wrote: > Hi Michael, > > On Tue, 2012-06-12 at 17:07 +1200, Michael Hudson-Doyle wrote: >> Those of you who have been around for a while will know that we used >> to package LAVA as Debian packages. This had its problems and we >> switched over to the world of pip and virtualenv and >> lava-deployment-tool. >> >> Unfortunately, pip and virtualenv have not really given us the control >> we need to be confident in our deployments, and so I'm proposing to >> change again, this time to zc.buildout. This is less of a change than >> the switch away from debs -- instances of LAVA will still be a concept >> and the lava-deployment-tool script will still be used, at least for now. >> >> The way buildout works is that there is a config file that specifies >> how -- reproducibly! -- to assemble various pieces into a deployment. >> I have a branch of lava-server at lp:~mwhudson/lava-server/use-buildout >> that contains such a config file (and various other related bits and >> pieces), and a branch of lava-deployment-tool that creates a >> buildout-based deployment. >> >> An instance on disk in the new world is similar to but not the same as >> an instance is today. In particular it is not a virtual environment >> -- although for compatibility, it looks like one. There is still a >> $LAVA_INSTANCE/bin/activate script that you can source in bash to put >> that instance's scripts at the front of your $PATH. Inside an >> instance, the etc, run, tmp and var directories server the same >> function as they do today. There is also a new directory, code. The >> code directory contains a number of subdirectories, one for each revno >> of lava-server that has been installed into that instance, and a >> symlink 'current' that points to the currently active one. >> >> There are now two merge proposals on LP: >> >> https://code.launchpad.net/~mwhudson/lava-server/use-buildout/+merge/109768 >> https://code.launchpad.net/~mwhudson/lava-deployment-tool/use-buildout/+mer… >> >> I've made an effort to update the documentation in l-d-t's README, and >> I've put an htmlified version of this file online at >> http://people.linaro.org/~mwh/ldt.html so that might be a good place >> to start. > > As I never worked with buildout before I was wondering if there was a > discussion around this topic before deciding to adopt this new > deployment approach. I would like to read that and see what was > discussed and what are the main advantages of changing how to deploy > Lava once again. Is it available somewhere? There hasn't been a big dialog. Michael and I had been talking about how our current pip/virtualenv has been a ticking time bomb and isn't very reproducible. Michael looked at buildout because its used by the Launchpad team who has similar but more complex requirements. As per the discussion - I think this is the place to have it and is why Michael sent this email. > In case this change really happen, I believe that would be nice to have > some sort of tutorial teaching how to migrate your current setup and > deploy it with buildout, without loosing your database, etc. We'll get there, one requirement Michael has is to make this still work with a "lava-deployment-tool upgrade". > Also, I remember to have long conversations on IRC with you guys and > IIRC there was an agreement that Lava would also be released as deb > packages again, so we would have an option to use pip or debs. How is > that going? I don't think we've ever agreed to release the server components as .debs. For the client stuff, we have and will continue to release them this way. > Please, forgive me if I missed any important discussion. > > Thanks in advance. > > BR, > > -Abner > >> Cheers, >> mwh >> >> _______________________________________________ >> linaro-validation mailing list >> linaro-validation(a)lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/linaro-validation >

12 years, 6 months

Lava Install Notes

by William Mills

All, Last week (mostly the weekend) I started a project to set up my own lava instance. I use EC2 for this sort of thing so I don't trash my own systems. Below is a walk through of my experience. Take it for what it is worth. I did get something running at http://test3.arago-project.org/ I'll leave it running over the weekend. I was hoping to do this all again clean but that does not look like it will happen anytime soon. Rather than letting this get stale, I will share it in its rather "raw" form. 0) What Ubuntu version? I started with 12.04 but Lava seems to default to python2.6 and that is not even an install candidate in 12.04 so I backed up to 11.10. Not sure if it really matters or not as the virtualenv seems to have used 2.7 even though lava-deployment-tool hard codes 2.6. Did not figure out what was going on there. 1) lava-tool, lava-scheduler-tool, do not work behind an http proxy workaround: go home to play w/ lava 2) lava-deployment-tool does not handle uwsgi download correctly tarball has moved to old/ some places fixed others not workaround: patch to centralize definition better fix: look in both "" and "/old" better'ist fix: also move this to bundle (otherwise this defeats the whole purpose of all the bundle overhead) 3) lava-deployment-tool wizards are a pain in the neck better fix: have separate config step that creates values install then takes bundle and config 4) no admin account and don't know how to install instructions tell you to login to admin but no info on how to lava-deployment-tool manage test1 createsuperuser example should give URL for those not familiar with django conventions 5) deployment does not enable https:// protocol but examples use it hack workaround: use http: for everything shouldn't things that require authorization only work over https://?? at least as an option? 6) need to add schedule job permissions for users prerequisite: login to site w/ your launchpad account log in to site admin w/ superuser account find your launchpad account in Users add scheduler* permissions better fix: create labuser and labadmin groups and give appropriate permissions to each, then just add users to appropriate group 7) keyring.core needed but not available source /srv/lava/instances/test1/bin/activate pip install keyring deactivate 8) no devicetypes docs talk about adding device but does not say you need to add device type as well cant add a device w/o first adding a device type dashboard has a device type also, do I need to add there as well? [Seems like No] what about kernel ci? this all seams very manual (tried to submit without any device or device type, did get 404 error) in django admin panel under scheduler add device type panda [can submit a job now] 9) added device config file to wrong directory, needs to go into /srv/lava/instances/test1/etc/lava-dispatcher/devices/ not up one level 10) need to install conmux, & missing dependency liburi-perl (for real HW this would be needed. I got it working but it was not clear where they got the apc control app from and how it was configured. need command hardreset) [switched to qemu focus] 11) Add beagle-xm device type & beagle-xm01 device to scheduler add devices/beagle-xm01.conf device_type = beagle-xm hostname = beagle-xm01 client_type = qemu 12) download of hwpack and rootfs not working getting html for root dir of release server wget of same url works ok on same machine I suspect that url2 opener does not handle redirects that releases.linaro.com make the user do (even thought wget works fine) squid proxy probably hides this from dispatcher result: squid is not really optional workaround: put copy of images needed into my /images/ref dir and change job to refer to those. works 13) need to install qemu installed qemu-user-static and qemu-system supported -M beaglexm however, EC2 tiny instance does not have enough memory to allocate 512M RAM qemu-system-arm won't honor a -m 128 w/ a -M beaglexm in either order (I consider that a bug) installed 4GB swap file and got past this. (booting linaro image on qemu did not cause any noticeable swapping) 14) qemu command does not work default_qemu_command was qemu, set it to qemu-system-arm in beagle-xm01.conf (should be in beagle-xm.conf) 15) qemu boot times out in udev or wait for root. EC2 tiny instance is probably too underpowered to run this Need to start over with a bigger machine Bill

12 years, 6 months

LAVA Core

by Zygmunt Krynicki

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi I'd like to announce a new project in the LAVA family. (README file follows) http://xkcd.com/927/ Seriously, lava has way to many tiny pieces. It all started when we renamed launch-control to lava-dashboard and created lava-server in the process. Now it's getting in the way. So here we are, lava-core is the start of the merge. Subsequent projects will slowly merge into the core. Then one day we may just get a package that is called 'lava', that ships a program called 'lava' that does useful stuff. (End of readme file) The project has been registered on Launchpad already. I plan on proposing merge requests that move parts of the non-client pieces there. Existing projects shall remain as backwards compatible code imports from lava-core. Thanks ZK -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJPib6QAAoJEOvx/vtfL8ZXXgQP+QHhT/4beTTW7+yCFehDwe6i iDNZVPAhjrXxeh8TFisfdnPP+8z9khkcQTyXOgJnC3T4vbmWNBTjDQTpkSg/0lzS O8a0YBkcdQ5RPDr1NNiLG7PoAJDUts8yY2DXk2C1ykWmqOZBw7NcRVaX/gQ1Ym75 JODbrtcO1vT+E5rBNYiEgMEELStJZfMGbfGzFI+d4ONz702SJNVDHcY7FReKuTDc nvOZ8zfW03fHFNobAAR5F57lGBz6oLZjz/nblsCJjA9tf20LPsQTUxkv6EkkEyd1 0CTkjQt60kHQB/FYoxLofJ76CR12gGGNLauAf+fQPBa4CG5MvMsd7fdDbMVa3iZh MKzOv206SPTmB7GHRVTfTqoeFwsXNEc7CnFvEWA9GbCUboqBxuYnEw95m1137+/7 B3XHqsBmuJC1KJrwNIAFTl41oRuHchwEeZv72wBxoMU+4MKVpHFz6F0P3aR9da4s Uty9z+Nk8lUyQBGMOy6GMzqlPnNNYPM/v3f0CBZkA+D9mtdD/pFJGkhm/xmyE23S 5NskNi7CXZ+m+kyONAzL3JagLnB7PE+L8tHbap3IKbxkPbB5kzz/3eWqK6Evq+3t eQJZzuEq8gmIzyHVYjtnRvWXigv3m/vQrkcFC9Gs36tNXG0lEO07DbLpj3ZTlQ03 K+VSjvjsNhMvdhitIY7I =DKzA -----END PGP SIGNATURE-----

12 years, 6 months

new installation

by novello

I have made a new installation from lava-deployment-tools after that I have try tu use this command: lava-tool auth-add http://novello@192.168.1.5/RPC2/ Paste token for http://novello@192.168.1.5/RPC2/: ERROR: Token rejected by server for user novello. I Have find this error. sorry but what i have to do to solve it. Best regards Novello G.

12 years, 6 months

SORRY AGAIN BUT IT DON'T WORK.

by Novello Giampiero

I HAVE MADE THE INSTALLATION USING LAVA-DEPLOIMENT-TOOLS thaN i WOULDLIKE TO INSTALL A NEW NEST. ===> # download example test defintion:curl http://people.linaro.org/~doanac/pass_fail.py > lava_test/test_definitions/pass_fail.pyDONE OK# patch lava_test/core/providers.py to include this test in the# _builtin_tests arrayDONE OK# "install" the lava-test:lava-test install pass_failtest)novello@qc:/srv/lava/instances/test/build/lava-android-test$ lava-test install pass_fail <LAVA_TEST>2012-06-11 09:45:24 AM WARNING: Unable to load test definition from u'pass_fail': ValueError(u'unknown url type: pass_fail',) <LAVA_TEST>2012-06-11 09:45:24 AM WARNING: Unable to load test definition from u'pass_fail': ValueError(u'unknown url type: pass_fail',) ERROR: There is no test with the specified ID AND HERE I HAVE STOPPED THE TEST. THEN I HAVE TRY lava-tool auth-add https://nicola@192.168.1.5/RPC2/ ==> TOKEN REJECTED BY SERVER FROM USER NICOLA sorry how i can add a user? I have made some superuser so i beleave that nicola that is a superuser could be used Best Regards Novello G.

12 years, 6 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

linaro-validation June 2012