On 06/30/2015 09:39 PM, Milosz Wasilewski wrote:
On 30 June 2015 at 10:05, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 25 June 2015 at 16:32, Luis Araujo luis.araujo@collabora.co.uk wrote:
I checked and it doesn't work (doesn't detect missing results). Here is example job: https://validation.linaro.org/scheduler/job/382325 (I'm not sure it's publicly available)
I cannot access it (even logged in with my launchpad account).
Can you send me a link publicly available with this same problem?, I really would like to check this out.
I'll try to reproduce it with some other device.
Luis,
Not exactly the same case, but I'm able to show that the missing results are not detected. Here is the job: https://validation.linaro.org/scheduler/job/410147
Here is my output: Generating lqa report for job(s): 410147
Report for job(s) (Tue Jun 30 14:33:11 2015): 410147 1 test job(s) ran: 1 complete (0 fully successful, 1 wih failures), 0 incomplete
--- Failed Jobs --- *
(F) Jobs with failed tests:
410147: https://ci.linaro.org/jenkins/job/linux-linaro-stable-lsk-v3.18/hwpack=beagl...
319 passed, 17 failed, 27 skipped, 0 unknown FAILED | rcutorture:rcutorture-start FAILED | rcutorture:lava-test-shell-run FAILED | pwrmgmt:cputopology_01 FAILED | usb-test-basic:list-all-usb-devices FAILED | usb-test-basic:examine-all-usb-devices FAILED | usb-test-basic:print-supported-protocols FAILED | usb-test-basic:print-supported-speeds FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 FAILED | lava:test_kernel_exception_1 Job: https://validation.linaro.org/scheduler/job/410147 Bundle: https://validation.linaro.org/dashboard/permalink/bundle/f022ec4a6ce822e154f...
In this case the LTP entries are there in the bundle. However one is missing the parameters. Anyway, I'll try to reproduce the original problem in some other test (maybe an artificial one, so you). In case of the original problem the entry for one LTP shell is missing from the bundle.
milosz
Right, the method won't report missing results for this job since all the testdef's specified in the yaml are available in the bundle:
https://validation.linaro.org/scheduler/job/410147/definition https://validation.linaro.org/dashboard/streams/anonymous/mwasilew/bundles/f...
The _find_missing_tests method is actually used only to check for those testdef from the lava_test_shell command that for some reason were not properly reported in the bundle stream. Now, if these tests have missing results, certainly, the method won't check for those, but this could be a later improvement for sure.
milosz