Hi Dan,
Here's an example of generating lava templates using jinja2. It's probably more complicated than you need.
See https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/lava-job-de... for a directory of templates, and see https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/submit_for_... for the python code that does the generation. Lastly, you can run https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/test_submit... and it will generate all of the YAML files into "./tmp".
Like I said, this is likely more complicated than you are looking for. I suggest starting simpler using https://pypi.python.org/pypi/jinja2-cli/0.6.0.
Thank you, that really helps a lot. We want to trigger the LAVA jobs from Jenkins as well, and we need lots of metadata because the tests will be part of our release notes in the end. So no, this is definitely not more complicated than we need. In fact, this is a very good starting point for us.
You should think about adding a chapter to the LAVA documentation about how to generate YAML job submissions via templates. I assume we are not the only ones facing this problem. And I had basically no experience with jinja2 at all before evaluating LAVA, so I would not have come up with the idea of using this mechanism for the job submissions myself.
Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks@garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz
On 20 March 2018 at 10:18, Tim Jaacks tim.jaacks@garz-fricke.com wrote:
Hi Dan,
Here's an example of generating lava templates using jinja2. It's
probably more complicated than you need.
openembedded-lkft/lava-job-definitions
for a directory of templates, and see https://git.linaro.org/ci/job/configs.git/tree/
openembedded-lkft/submit_for_testing.py
for the python code that does the generation. Lastly, you can run
https://git.linaro.org/ci/job/configs.git/tree/openembedded- lkft/test_submit_for_testing.sh
and it will generate all of the YAML files into "./tmp".
Like I said, this is likely more complicated than you are looking for. I
suggest starting simpler using https://pypi.python.org/pypi/ jinja2-cli/0.6.0.
Thank you, that really helps a lot. We want to trigger the LAVA jobs from Jenkins as well, and we need lots of metadata because the tests will be part of our release notes in the end. So no, this is definitely not more complicated than we need. In fact, this is a very good starting point for us.
You should think about adding a chapter to the LAVA documentation about how to generate YAML job submissions via templates. I assume we are not the only ones facing this problem. And I had basically no experience with jinja2 at all before evaluating LAVA, so I would not have come up with the idea of using this mechanism for the job submissions myself.
Story added to track this addition: https://projects.linaro.org/browse/LAVA-1260
Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks@garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Tue, Mar 20, 2018 at 10:18:17AM +0000, Tim Jaacks wrote:
Hi Dan,
Here's an example of generating lava templates using jinja2. It's probably more complicated than you need.
See https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/lava-job-de... for a directory of templates, and see https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/submit_for_... for the python code that does the generation. Lastly, you can run https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/test_submit... and it will generate all of the YAML files into "./tmp".
Like I said, this is likely more complicated than you are looking for. I suggest starting simpler using https://pypi.python.org/pypi/jinja2-cli/0.6.0.
Thank you, that really helps a lot. We want to trigger the LAVA jobs from Jenkins as well, and we need lots of metadata because the tests will be part of our release notes in the end. So no, this is definitely not more complicated than we need. In fact, this is a very good starting point for us.
In that case, I can explain a bit more about the layout. We run about 20 lava jobs for each of about 9 different device types. Instead of maintaining 20*9 lava job yaml files with lots of duplication, we have one template file per device (/devices/), and one template file per test type (/testplan/). For the tests, there's still some duplication between similar tests so we abstracted it out into a couple master test files that take an argument.
This allows us to keep the job templates DRY which is critical for maintenance and consistency concerns over time.
It is a little goofy to write out structured data files using a templating language, but it actually does work quite well, and isn't something we invented here. Ansible, for example, uses yaml+jinja2 extensively. There are also filters in jinja2 such as to_json and to_yaml that can be used to dump larger data structures directly.
Dan
You should think about adding a chapter to the LAVA documentation about how to generate YAML job submissions via templates. I assume we are not the only ones facing this problem. And I had basically no experience with jinja2 at all before evaluating LAVA, so I would not have come up with the idea of using this mechanism for the job submissions myself.
Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks@garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz
Hi Dan, hi Neil,
Here's an example of generating lava templates using jinja2. It's probably more complicated than you need.
See https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/lava -job-definitions for a directory of templates, and see https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/subm it_for_testing.py for the python code that does the generation. Lastly, you can run https://git.linaro.org/ci/job/configs.git/tree/openembedded-lkft/test _submit_for_testing.sh and it will generate all of the YAML files into "./tmp".
Like I said, this is likely more complicated than you are looking for. I suggest starting simpler using https://pypi.python.org/pypi/jinja2-cli/0.6.0.
Thank you, that really helps a lot. We want to trigger the LAVA jobs from Jenkins as well, and we need lots of metadata because the tests will be part of our release notes in the end. So no, this is definitely not more complicated than we need. In fact, this is a very good starting point for us.
In that case, I can explain a bit more about the layout. We run about 20 lava jobs for each of about 9 different device types. Instead of maintaining 20*9 lava job yaml files with lots of duplication, we have one template file per device (/devices/), and one template file per test type (/testplan/). For the tests, there's still some duplication between similar tests so we abstracted it out into a couple master test files that take an argument.
This allows us to keep the job templates DRY which is critical for maintenance and consistency concerns over time.
It is a little goofy to write out structured data files using a templating language, but it actually does work quite well, and isn't something we invented here. Ansible, for example, uses yaml+jinja2 extensively. There are also filters in jinja2 such as to_json and to_yaml that can be used to dump larger data structures directly.
Dan
in the meantime I managed to set up a template structure along with a script to construct job definitions suiting our needs. Thanks again for your help.
I have a further question: As far as I see, everyone seems to keep job definitions (or templates for those) and test definitions in separate repositories. Is there a certain reason for this? Why shouldn't I put job templates and test definitions that belong together side by side in the same folder?
Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks@garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz
On 10 July 2018 at 12:52, Tim Jaacks tim.jaacks@garz-fricke.com wrote:
in the meantime I managed to set up a template structure along with a script to construct job definitions suiting our needs. Thanks again for your help.
I have a further question: As far as I see, everyone seems to keep job definitions (or templates for those) and test definitions in separate repositories. Is there a certain reason for this? Why shouldn't I put job templates and test definitions that belong together side by side in the same folder?
I don't know of anything that should stop you from doing that. In our case we try to keep tests 'executor agnostic'. So they can be ran outside of LAVA and don't depend on any LAVA wrapper scripts. But that isn't something mandatory. So if you feel like locking your tests to LAVA you can do that.
Best Regards, milosz
On Tue, 10 Jul 2018 at 13:00, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
On 10 July 2018 at 12:52, Tim Jaacks tim.jaacks@garz-fricke.com wrote:
in the meantime I managed to set up a template structure along with a
script to construct job definitions suiting our needs. Thanks again for your help.
I have a further question: As far as I see, everyone seems to keep job
definitions (or templates for those) and test definitions in separate repositories. Is there a certain reason for this? Why shouldn't I put job templates and test definitions that belong together side by side in the same folder?
I don't know of anything that should stop you from doing that. In our case we try to keep tests 'executor agnostic'. So they can be ran outside of LAVA and don't depend on any LAVA wrapper scripts. But that isn't something mandatory. So if you feel like locking your tests to LAVA you can do that.
LAVA documentation does recommend the portability that Milosz describes.
https://staging.validation.linaro.org/static/docs/v2/writing-tests.html#writ...
The git repository location doesn't have to affect portability but portability is a worthwhile goal in and of itself. We are looking at a future version of the Lava Test Shell Definition which would remove as many of the current wrapper scripts as we can - especially the installer helpers and OS-specific wrappers.
More of a potential problem is that the git history can get large if there are a lot of unrelated changes - template changes and test job changes and test definition changes. LAVA software supports shallow clones by default and removing the git history entirely in each test definition but there are still situations where the test definition repository is too large, when cloned, to fit into the ramdisk or other system used on the DUT.
So it's not that you shouldn't keep the files together, it's more that separation has useful benefits and you are future proofing yourself more effectively by using separation. Just imagine having to separate these out at some point in the future. Lots of small repos are more efficient during testing than repeatedly cloning one huge repository for every test job. (This is also an argument for not only separating templates from test definitions but separating different types of test definitions into their own repositories.)
Best Regards, milosz
-
Dan Rue -
Milosz Wasilewski -
Neil Williams -
Tim Jaacks