Hello
As you know I am working on OpenEmbedded for AArch64 and ARMv7a setups. Part of this work is adding LAVA tests. And this is where problem starts so after reading some scripts I decided that it will be better to just write an email and discuss it on ML.
First thing is structure of tests and their data. We have git repo ssh://git.linaro.org/srv/git.linaro.org/git/qa/test-definitions.git with test definitions split to android/oe/ubuntu directories. Nicely done.
But those test definitions fetch another set of repositories/files. Some are git, some are bzr and many of them are just single files. Can we move them to /qa/test-definitions.git to have one repository will all tests?
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA... And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
What do you think?
On Thu, Mar 21, 2013 at 10:19 AM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Hello
As you know I am working on OpenEmbedded for AArch64 and ARMv7a setups. Part of this work is adding LAVA tests. And this is where problem starts so after reading some scripts I decided that it will be better to just write an email and discuss it on ML.
First thing is structure of tests and their data. We have git repo ssh://git.linaro.org/srv/git.linaro.org/git/qa/test-definitions.git with test definitions split to android/oe/ubuntu directories. Nicely done.
But those test definitions fetch another set of repositories/files. Some are git, some are bzr and many of them are just single files. Can we move them to /qa/test-definitions.git to have one repository will all tests?
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA... And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
What do you think?
I think that these are very good questions.
linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
Hi Marcin,
On Thursday 21 March 2013 02:49 PM, Marcin Juszkiewicz wrote:
But those test definitions fetch another set of repositories/files. Some are git, some are bzr and many of them are just single files. Can we move them to /qa/test-definitions.git to have one repository will all tests?
Yes all the scripts available as part of lava-test are from bzr repos. This is maintained by different teams, hence I left it that way. But ideally they should all be brought into the new git repository.
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA... And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
I am unsure where install-ubuntu.sh script is available? This is because I do not know the image generation process. If you could point me to this stuff, I can consolidate it if possible.
Thank You.
On 21.03.2013 12:45, Senthil Kumaran wrote:
On Thursday 21 March 2013 02:49 PM, Marcin Juszkiewicz wrote:
Yes all the scripts available as part of lava-test are from bzr repos. This is maintained by different teams, hence I left it that way. But ideally they should all be brought into the new git repository.
Would be lovely.
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA... And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
I am unsure where install-ubuntu.sh script is available?
Sorry, I meant that it could be split to such script.
This is because I do not know the image generation process. If you could point me to this stuff, I can consolidate it if possible.
Open a bug against linaro-ci with list of Ubuntu packages you need in images and subscribe Fathi and me. We can take care of adding such packages.
Hello Marcin,
Thanks for raising this point, we really need to get rid of distro-specific code, or at least keep them at a minimum so that we can easily add support for new ones.
On Thu, Mar 21, 2013 at 10:19:40AM +0100, Marcin Juszkiewicz wrote:
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA...
IMO including everything needed for every test in the images does not scale. When we have 100 tests, each one requiring 1 package, do we want to have 100 extra packages in every image? Also, everyone creating new tests would have to coordinate with the teams that create images, which increases the communications overhead and adds bottlenecks.
And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
I think we should provide means for tests to install dependencies in a cross-distro way, by having the tests declare a list of packages that need to need to be installed a before it is executed.
In Ubuntu, that list of packages would be passed to `apt-get install`, while in OE it would be passed into `opkg install`, etc. The LAVA dispatcher could them install a lava-install script that does the right thing on each playform and would be called to install test dependencies.
The worst case is having a package with different names in different distros, so in that case we could implement a way of specifying separate per-distro dependencies list, or advising test authors to install the tests as part of the test steps, which are shell commands anyway.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
W dniu 22.03.2013 20:41, Antonio Terceiro pisze:
On Thu, Mar 21, 2013 at 10:19:40AM +0100, Marcin Juszkiewicz wrote:
Next thing is distro dependencies. For example perf test done for Ubuntu. First thing it does is finding out which exactly perf package it needs and install it. Why this is not done on image generation time? Especially when we know that all rootfs images we make end in LAVA...
IMO including everything needed for every test in the images does not scale. When we have 100 tests, each one requiring 1 package, do we want to have 100 extra packages in every image?
Good point.
Also, everyone creating new tests would have to coordinate with the teams that create images, which increases the communications overhead and adds bottlenecks.
+1
And can this part be moved to install-ubuntu.sh script so for OE I could just reuse script instead of creating a copy of it? Probably similar would be with others.
I think we should provide means for tests to install dependencies in a cross-distro way, by having the tests declare a list of packages that need to need to be installed a before it is executed.
In Ubuntu, that list of packages would be passed to `apt-get install`, while in OE it would be passed into `opkg install`, etc.
We can not use 'opkg install' cause with OE we do not have software repositories. Of course we could create them but in this case I prefer to add packages into image rather than maintain repository.
linaro-validation@lists.linaro.org
-
Alexander Sack -
Antonio Terceiro -
Marcin Juszkiewicz -
Senthil Kumaran