On Thu, 10 May 2012 10:21:31 -0430, Luis Araujo luis.araujo@collabora.co.uk wrote:
Hello,
I am sending this email to bring up a topic that has been discussed before[0] , and that I believe it is still relevant and worth considering; about moving away the test definitions from the lava-test package into its own project/package.
We also had a discussion at #linaro-lava yesterday, some points were raised in favour or regarding separating the test definitions, among them:
- A separated test definitions package would allow to upgrade, modify
existing tests and/or release new ones without the need of releasing a whole new lava-test package; and the other way around (it would make maintenance easier and more flexible, something that current test definitions seem to be lacking).
- It would promote more specialized tests definitions; since we have an
independent test definitions package, it should be easier to tailor and branch for specific platforms or projects (and even encourage that).
- It is cleaner, from a development point of view, to keep test
definitions separated from the lava-test tool. Test definitions are not components, but test files, similar in functionality to json job files. This also would improve maintenance and collaboration efforts.
- A possible test definitions package should use a kind of versioning to
keep compliance with lava-test API, and hence avoiding any breakage, but at the same time making the packages independent enough so we can upgrade/modify one without affecting the other.
- We still could keep some minimal test definitions in the lava-test
package, though these would be more like 'simple' test cases, serving more like examples for the given lava-test API/Core version.
Apart of the above discussed issues, there are also some valid points at [1]. Of special interest are the maintenance problem and to keep proper cross-platform support in the tests.
My initial idea was to have a possible 'lava-test-definitions' package that could initially contain all the current available tests from the latest lava-test package, update/fix the existing broken tests (if any), then get a launchpad project following the TODO items from the blueprint at [1] , and that should be enough to get us on the way.
This email is intended to start a discussion that hopefully could bring some technical decision about the subject, hence it is an open door for ideas and comments, so please, share yours :)
It sounds OK to me, I guess I'd like to know what Paul Larson thinks, seeing as we'd basically be wanting to shove the work of maintaining lava-test-definitions over to his team :-)
Cheers, mwh