Chase- I haven't seen any new AOSP results since your mail that said it is now stable. Shouldn't we be seeing regular reports for lkft/android-hikey-linaro-4.4-aosp like we do for lkft/linaro-hikey-linaro-4.4-oe?
On Fri, Aug 18, 2017 at 5:57 AM, Chase QI chase.qi@linaro.org wrote:
On 17 August 2017 at 10:21, Chase QI chase.qi@linaro.org wrote:
Hi Todd,
Milosz is on holiday.
On 17 August 2017 at 08:26, Todd Kjos tkjos@google.com wrote:
Milosz,
I was looking at the -aosp tests that just ran. Looking at a comparison of the 4.4/4.9 runs at:
https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandroid-hikey-linaro...
I noticed that most CTS/VTS tests don't run. Why is that? Are we deliberately pruning the tests? Or is a failure causing the test runs to stop before getting to all the tests?
It is the later case. Some of the test jobs finished completely, but have no real test results. I am investigating these issues.
With latest fixes on test suite[1] and Jenkins project configuration[2], AOSP testing is stable now. Refer to the below reports. CtsLibcoreTestCases is the only incomplete job, we still need fixes to handle its 45k+ test results.
https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-aosp/build/v4.9.... https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.4-aosp/build/v4.4....
The above separate reports look fine; while comparison between them still doesn't look good[3]. One reason is, for the same test module, the number of test case varies across branches. We probably should only compare the common results and only show the differs of them. Further work is required on this, I will raise a task in QAS team.
[1] https://git.linaro.org/qa/test-definitions.git [2] https://git.linaro.org/ci/job/configs.git [3] https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandroid-hikey-linaro...
-Todd
Lts-dev mailing list Lts-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lts-dev
-- Thanks, Chase
-- Thanks, Chase