Hello everyone,
I know from the LAVA documentation how to add metadata to jobs and test suites. When I look at test results, I see that test cases have metadata, too. E.g. https://validation.linaro.org/results/testcase/9759970 shows the following metadata:
case: linux-linaro-ubuntu-lscpu definition: 0_smoke-tests-lxc result: pass
Is there a possibility to add custom metadata to test cases?
Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks@garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE!
Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz
On 12 June 2018 at 16:05, Tim Jaacks tim.jaacks@garz-fricke.com wrote:
Hello everyone,
I know from the LAVA documentation how to add metadata to jobs and test suites. When I look at test results, I see that test cases have metadata, too. E.g. https://validation.linaro.org/results/testcase/9759970 shows the following metadata:
case: linux-linaro-ubuntu-lscpu definition: 0_smoke-tests-lxc result: pass
That data is retrieved directly from the test case. case is the string passed to lava-test-case as the test case ID. definition is the currently running test definition (so an integral part of how the test case is reported) and then there is the result.
Actions can supply metadata, e.g. https://validation.linaro.org/results/testcase/9759948
This metadata comes from the Python3 code running the action and covers a range of dynamic data which is not otherwise recorded. This data is not modifiable.
Test suites gain metadata from the operation of the test job itself. The content of the Lava Test Shell Definition is not used, other than to identify the test defiinition.
The user-controllable metadata needs to be in the Test Job Submission, to then appear in the top level results:
e.g. in
https://validation.linaro.org/results/1882533
path: health-checks/dragonboard-820c.yaml source: https://git.linaro.org/lava-team/refactoring.git
come from the test job submission.
Is there a possibility to add custom metadata to test cases?
Only as a URL: https://validation.linaro.org/static/docs/v2/writing-tests.html#recording-te...
Once the test job has completed, bug links are also available for test jobs, test suites and test cases.
Extra data can be recorded as separate test cases using the test case name, e.g.
https://staging.validation.linaro.org/results/testcase/5557198
(Although I really should tweak that to drop the Python3 string handling artifacts in the test shell scripts. It should read code-version-2018.5+12261.50e347aca )
We are considering new features for the Lava Test Shell Definition but changes won't be made any time soon, it's likely to be 2019 before we have a clear plan for that work.
-
Neil Williams -
Tim Jaacks