*
Hi,
Greetings from LAVA Team!
We are very happy to announce you that there are new features getting
added to validation.linaro.org. In order to serve you better we are
trying to collect metadata from lava-test-shell tests that you are
running in validation.linaro.org.
On 2013-04-16 we will be deploying a new version of lava-dashboard in
validation.linaro.org which requires all lava-test-shell test
definitions, which you run should have metadata to be complete.
When we say metadata to be complete, following is a sample test
definition we refer to:
metadata:
name: device-tree
format: "Lava-Test-Shell Test Definition 1.0"
description: "Device tree test to check folder structure."
os:
- ubuntu
devices:
- origen
- snowball
- panda
- panda-es
- vexpress
environment:
- lava-test-shell
install:
bzr-repos:
- lp:~linaro-foundations/linaro-ubuntu/lava-test-device-tree
run:
steps:
- "cd lava-test-device-tree; sudo bash -x ./run-test.sh"
parse:
pattern: "(?P<test_case_id>[a-zA-Z0-9_-]+):\s(?P<result>\w+)"
However, the sections such as os, devices, environment are optional in
the above, but other parameters such as name, format, description are
mandatory in the metadata section.
If your test definition is not part of a bzr or git repository then it
is mandatory to have a 'version' parameter in metadata section. The
following example shows how a test definition metadata section will
look like for a test definition which is not part of bzr or git
repository:
metadata:
name: device-tree
format: "Lava-Test-Shell Test Definition 1.0"
version: 1.0
description: "Device tree test to check folder structure."
os:
- ubuntu
devices:
- origen
- snowball
- panda
- panda-es
- vexpress
environment:
- lava-test-shell
The documentation has been updated to reflect this change:
https://lava-dispatcher.readthedocs.org/en/latest/lava_test_shell.html
If you have any doubts in the above, please approach linaro-validation
linaro-validation@lists.linaro.org. Remember test definitions that
do not adhere to the above notification will fail when run on
validation.linaro.org after 2013-04-16.
If you want to test the correctness of your test definitions before
2013-04-16 feel free to use staging.validation.linaro.org.
Thank You.
LAVA Team!
*-- Tyler Baker Technical Architect, Automation & CI Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
W dniu 12.04.2013 22:16, Tyler Baker pisze:
[removed everyone@ to not get Mike complaining again about sending email to >180 people]
We are very happy to announce you that there are new features getting added to validation.linaro.org
In order to serve you better we are trying to collect metadata from lava-test-shell tests that you are running in validation.linaro.org http://validation.linaro.org.
metadata: name: device-tree format: "Lava-Test-Shell Test Definition 1.0" description: "Device tree test to check folder structure."
os:
- ubuntu
Doc says: ==== Handling Dependencies (Ubuntu)¶
If your test requires some packages to be installed before its run it can express that in the install section with:
install: deps: - linux-libc-dev - build-essential ====
How I can write one test which will cover different distributions with names of dependencies written for each of them?
For example I want to run "hrw-test-umbaumbau" test under Fedora, OpenEmbedded, Ubuntu. It requires working toolchain on device and SLang development headers. Each distro needs other names... Today I would write a copy for each distribution.
devices:
- origen
- snowball
- panda
- panda-es
- vexpress
Can I run someone's test on my umbaumba device without making copy of test? With old LAVA I could. With new it looks like I can not. Regression?
On Sun, Apr 14, 2013 at 10:31:55PM +0200, Marcin Juszkiewicz wrote:
W dniu 12.04.2013 22:16, Tyler Baker pisze:
[removed everyone@ to not get Mike complaining again about sending email to >180 people]
We are very happy to announce you that there are new features getting added to validation.linaro.org
In order to serve you better we are trying to collect metadata from lava-test-shell tests that you are running in validation.linaro.org http://validation.linaro.org.
metadata: name: device-tree format: "Lava-Test-Shell Test Definition 1.0" description: "Device tree test to check folder structure."
os:
- ubuntu
Doc says:
Handling Dependencies (Ubuntu)¶
If your test requires some packages to be installed before its run it can express that in the install section with:
install: deps: - linux-libc-dev - build-essential ====
How I can write one test which will cover different distributions with names of dependencies written for each of them?
For example I want to run "hrw-test-umbaumbau" test under Fedora, OpenEmbedded, Ubuntu. It requires working toolchain on device and SLang development headers. Each distro needs other names... Today I would write a copy for each distribution.
There is ongoing to work to push dependency installation to a helper script that is provided by the distro-specific support scripts:
https://code.launchpad.net/~terceiro/lava-dispatcher/modularize-distro-suppo...
Basically, when you list deps p1, p2 and p3, lava will call `lava-install-packages p1 p2 p3` on the target, where lava-install-packages is a script that does the right thing for that distro.
When you have packages with different names, though, this approach does not work. In this case you have to remember that lava-test-shell allows you to run whatever you want on the target, and handle the different cases for each distro in your test script.
In that case, perhaps we could provide a binary called `lava-distro` that prints the name of the distro where the test is being run, so that nobody needs to duplicate distro-detection logic. What do you think?
devices:
- origen
- snowball
- panda
- panda-es
- vexpress
Can I run someone's test on my umbaumba device without making copy of test? With old LAVA I could. With new it looks like I can not. Regression?
this metadata is just informative, aimed at allowing use to collect data about the test definitions, and which devices are supposed to be supported by then. Nothing will stop you from submitting that test to another device.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
W dniu 15.04.2013 15:32, Antonio Terceiro pisze:
How I can write one test which will cover different distributions with names of dependencies written for each of them?
For example I want to run "hrw-test-umbaumbau" test under Fedora, OpenEmbedded, Ubuntu. It requires working toolchain on device and SLang development headers. Each distro needs other names... Today I would write a copy for each distribution.
There is ongoing to work to push dependency installation to a helper script that is provided by the distro-specific support scripts:
https://code.launchpad.net/~terceiro/lava-dispatcher/modularize-distro-suppo...
Basically, when you list deps p1, p2 and p3, lava will call `lava-install-packages p1 p2 p3` on the target, where lava-install-packages is a script that does the right thing for that distro.
When you have packages with different names, though, this approach does not work. In this case you have to remember that lava-test-shell allows you to run whatever you want on the target, and handle the different cases for each distro in your test script.
In that case, perhaps we could provide a binary called `lava-distro` that prints the name of the distro where the test is being run, so that nobody needs to duplicate distro-detection logic. What do you think?
What about this:
====================== metadata: name: hrw-umba-umba2 format: "Lava-Test-Shell Test Definition 1.0" description: "Test to check Umba on Umba Umba"
os: - ubuntu - fedora - openembedded
install: deps:
deps-ubuntu: - linux-libc-dev - build-essential
deps-fedora: - glibc-devel - build-essential-like-package
devices: - vexpress64 - umbaumba
run: steps: - "sh run-test.sh" - "sh gather-results.sh" ======================
LAVA has list of supported systems (ubuntu, fedora, openembedded in that test) and has a list of deps for ubuntu, separate one for fedora and empty fallback just in case (can be omitted). OpenEmbedded skips install part because all required components are in image already.
This way if we will add for example Slackware to the list you have to add it to 'lava-install-package' script and test authors may decide do they want to support it or not.
On Mon, Apr 15, 2013 at 04:14:36PM +0200, Marcin Juszkiewicz wrote:
What about this:
======================
[snip]
install: deps:
deps-ubuntu: - linux-libc-dev - build-essential deps-fedora: - glibc-devel - build-essential-like-package
[...]
======================
LAVA has list of supported systems (ubuntu, fedora, openembedded in that test) and has a list of deps for ubuntu, separate one for fedora and empty fallback just in case (can be omitted). OpenEmbedded skips install part because all required components are in image already.
This way if we will add for example Slackware to the list you have to add it to 'lava-install-package' script and test authors may decide do they want to support it or not.
That makes a lot of sense
I have implemented this in my distro support modularization branch: http://bazaar.launchpad.net/~terceiro/lava-dispatcher/modularize-distro-supp...
That should land Soon™.
Thanks for your input.
linaro-validation@lists.linaro.org
-
Antonio Terceiro -
Marcin Juszkiewicz -
Tyler Baker