Hi all,
As I wrote in a previous mail, I kindly ask you to point me for some straight documentation on how we can import some external tests results directly to lava dashboard via JSON file ? I search for 2 days for documentation on how to do it but couldn't find any. Is there some step by step documentation/examples ?
Thank you for the help, Ionut C
On Tue, 10 Jun 2014 07:24:37 +0000 "Chisanovici, IonutX" ionutx.chisanovici@intel.com wrote:
Hi all,
As I wrote in a previous mail, I kindly ask you to point me for some straight documentation on how we can import some external tests results directly to lava dashboard via JSON file ? I search for 2 days for documentation on how to do it but couldn't find any. Is there some step by step documentation/examples ?
LAVA doesn't support this explicitly. LAVA results are of LAVA tests - (LAVA isn't primarily a reporting engine, it's designed to run the tests and have a core set of report views but also provide export to other tools which can do a much more detailed analysis).
What do you expect to gain by having results with no supporting data in the dashboard? It's not actually helpful to have result bundles which cannot link back to a test job - you lose the ability to re-test and you lose all the context of the results.
There are times when developers will pull sets of bundles from a server or manipulate bundles directly but this is for testing. It makes no sense for a CI loop to have results without context.
Hi Neil,
Actually, the setup is like this (I have detailed this also):
1. One of Fedora/Ubuntu/OpenSuse/CentOS machine which should be as device in lava server (on this machine will run the tests). I have to have these os'es and not to have a lava prebuilt image. 2. The above machine has to send the test results to lava server some how (JSON/YAML) 3. See the test results in lava dashboard.
I'm pretty sure that this scenario can happen and can be implemented using lava server.
Thanks, Ionut C ________________________________________ From: linaro-validation-bounces@lists.linaro.org [linaro-validation-bounces@lists.linaro.org] on behalf of Neil Williams [codehelp@debian.org] Sent: Tuesday, June 10, 2014 11:32 AM To: linaro-validation@lists.linaro.org Subject: Re: [Linaro-validation] Lava server - external tests results imported into dashboard
On Tue, 10 Jun 2014 07:24:37 +0000 "Chisanovici, IonutX" ionutx.chisanovici@intel.com wrote:
Hi all,
As I wrote in a previous mail, I kindly ask you to point me for some straight documentation on how we can import some external tests results directly to lava dashboard via JSON file ? I search for 2 days for documentation on how to do it but couldn't find any. Is there some step by step documentation/examples ?
LAVA doesn't support this explicitly. LAVA results are of LAVA tests - (LAVA isn't primarily a reporting engine, it's designed to run the tests and have a core set of report views but also provide export to other tools which can do a much more detailed analysis).
What do you expect to gain by having results with no supporting data in the dashboard? It's not actually helpful to have result bundles which cannot link back to a test job - you lose the ability to re-test and you lose all the context of the results.
There are times when developers will pull sets of bundles from a server or manipulate bundles directly but this is for testing. It makes no sense for a CI loop to have results without context.
--
Neil Williams ============= http://www.linux.codehelp.co.uk/
On Tue, 10 Jun 2014 08:51:29 +0000 "Chisanovici, IonutX" ionutx.chisanovici@intel.com wrote:
Actually, the setup is like this (I have detailed this also):
- One of Fedora/Ubuntu/OpenSuse/CentOS machine which should be as
device in lava server (on this machine will run the tests). I have to have these os'es and not to have a lava prebuilt image.
The OS can be inside a VM (and therefore deployed inside LAVA) and it only needs a LAVA overlay to run the tests. The image would be created by you, to your needs, just downloaded by LAVA. It wouldn't have to be an image built by Linaro. This would be a KVM device using a purpose-built image. (Using KVM means you only need one machine, as long as that machine has enough cores and enough RAM.) Alternatively, LAVA can just use a connection to one existing machine for each OS, via the dummy devices.
- The above
machine has to send the test results to lava server some how (JSON/YAML) 3. See the test results in lava dashboard.
Without the rest of the build log, the results aren't going to be useful.
I'm pretty sure that this scenario can happen and can be implemented using lava server.
Probably via the dummy-ssh device. The machine would already be running, a test would login, run the build so that the logs appear in LAVA and then collect the results as normal. No need to have external test results, just openssh-server running.
Any command used to generate the results externally can be executed within LAVA - all it needs is a connection to that box. If you don't want a cleanly booted image, then use a connection to a dirty environment which is already running but a clean environment is preferable.
LAVA can support multiple nodes per test job, so you could do all the tests at the same time.
Hi Neil,
Ok, I will try the dummy-ssh part but I need more examples on how to do that. Based on 'https://validation.linaro.org/static/docs/dummy-deploy.html', that is very little. Please remind that I'm newby/beginner in lava. Where should I put that ssh01.conf (folder location under lava installation dir) ? How can I add a dummy device ? How can I test a full connection between the server and the dummy-ssh device ?
Thank you, Ionut C ________________________________________ From: linaro-validation-bounces@lists.linaro.org [linaro-validation-bounces@lists.linaro.org] on behalf of Neil Williams [codehelp@debian.org] Sent: Tuesday, June 10, 2014 1:44 PM To: linaro-validation@lists.linaro.org Subject: Re: [Linaro-validation] Lava server - external tests results imported into dashboard
On Tue, 10 Jun 2014 08:51:29 +0000 "Chisanovici, IonutX" ionutx.chisanovici@intel.com wrote:
Actually, the setup is like this (I have detailed this also):
- One of Fedora/Ubuntu/OpenSuse/CentOS machine which should be as
device in lava server (on this machine will run the tests). I have to have these os'es and not to have a lava prebuilt image.
The OS can be inside a VM (and therefore deployed inside LAVA) and it only needs a LAVA overlay to run the tests. The image would be created by you, to your needs, just downloaded by LAVA. It wouldn't have to be an image built by Linaro. This would be a KVM device using a purpose-built image. (Using KVM means you only need one machine, as long as that machine has enough cores and enough RAM.) Alternatively, LAVA can just use a connection to one existing machine for each OS, via the dummy devices.
- The above
machine has to send the test results to lava server some how (JSON/YAML) 3. See the test results in lava dashboard.
Without the rest of the build log, the results aren't going to be useful.
I'm pretty sure that this scenario can happen and can be implemented using lava server.
Probably via the dummy-ssh device. The machine would already be running, a test would login, run the build so that the logs appear in LAVA and then collect the results as normal. No need to have external test results, just openssh-server running.
Any command used to generate the results externally can be executed within LAVA - all it needs is a connection to that box. If you don't want a cleanly booted image, then use a connection to a dirty environment which is already running but a clean environment is preferable.
LAVA can support multiple nodes per test job, so you could do all the tests at the same time.
--
Neil Williams ============= http://www.linux.codehelp.co.uk/
On Tue, 10 Jun 2014 13:26:40 +0000 "Chisanovici, IonutX" ionutx.chisanovici@intel.com wrote:
Hi Neil,
Ok, I will try the dummy-ssh part but I need more examples on how to do that. Based on 'https://validation.linaro.org/static/docs/dummy-deploy.html', that is very little.
Combine that with:
https://git.linaro.org/lava/lava-lab.git/blob/HEAD:/lava/devices/dispatcher0...
(a working dummy-ssh in production)
Adding to the dispatcher:
https://validation.linaro.org/static/docs/lava-image-creation.html#adding-to...
Adding to the database:
https://validation.linaro.org/static/docs/lava-image-creation.html#adding-to...
(Yes, that section could do with some more content but once you've done it once, it will make sense.)
Please remind that I'm newby/beginner in lava. Where should I put that ssh01.conf (folder location under lava installation dir) ? How can I add a dummy device ? How can I test a full connection between the server and the dummy-ssh device ?
If you can ssh to the machine as root from the LAVA dispatcher without a password prompt, things should be fine.
A JSON snippet to connect would be:
"actions": [ { "command": "dummy_deploy", "parameters": { "target_type": "ubuntu" } },
The target_type only matters if you are installing packages on the ssh host or if you're using Android. You should be able to use a "ubuntu" target_type even when connecting to Fedora if all the packages you need are already installed. We don't have Fedora users in the team, so various bits of Fedora deployment are untested and possibly incomplete. Feedback and fixes for this would be useful. ;-)
https://validation.linaro.org/scheduler/job/117750/definition
linaro-validation@lists.linaro.org
-
Chisanovici, IonutX -
Neil Williams