On Aug 25, 2017 22:56, "Chase Qi" chase.qi@linaro.org wrote:
Hi Todd,
Here is the latest testing status for v4.9 https://qa-reports.linaro.org/ lkft/android-hikey-linaro-4.9-aosp/ . There are 17 jobs set on each build, 16 jobs finished correctly on the recent 6 builds. CtsLibcoreTestCases is the missing one, we still need fixes for the timeout on result pulling.
Hi Milosz,
I guess we can go without CtsLibcoreTestCases for the moment. Is it ok to reduce 'Expected test runs' to 16 so that qa-reports generates and sends test report for AOSP testing?
I guess we can give it a try.
milosz
-- Thanks, Chase
On 26 August 2017 at 01:40, Todd Kjos tkjos@google.com wrote:
Chase- I haven't seen any new AOSP results since your mail that said it is now stable. Shouldn't we be seeing regular reports for lkft/android-hikey-linaro-4.4-aosp like we do for lkft/linaro-hikey-linaro-4.4-oe?
On Fri, Aug 18, 2017 at 5:57 AM, Chase QI chase.qi@linaro.org wrote:
On 17 August 2017 at 10:21, Chase QI chase.qi@linaro.org wrote:
Hi Todd,
Milosz is on holiday.
On 17 August 2017 at 08:26, Todd Kjos tkjos@google.com wrote:
Milosz,
I was looking at the -aosp tests that just ran. Looking at a
comparison of
the 4.4/4.9 runs at:
https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandr
oid-hikey-linaro-4.4-aosp&project=lkft%2Fandroid-hikey-linaro-4.9-aosp
I noticed that most CTS/VTS tests don't run. Why is that? Are we deliberately pruning the tests? Or is a failure causing the test runs
to
stop before getting to all the tests?
It is the later case. Some of the test jobs finished completely, but have no real test results. I am investigating these issues.
With latest fixes on test suite[1] and Jenkins project configuration[2], AOSP testing is stable now. Refer to the below reports. CtsLibcoreTestCases is the only incomplete job, we still need fixes to handle its 45k+ test results.
https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-
aosp/build/v4.9.43-638384-g17c33fe8a2e3/
https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.4-
aosp/build/v4.4.82-568129-gdc23078e4e65-rebuild1/
The above separate reports look fine; while comparison between them still doesn't look good[3]. One reason is, for the same test module, the number of test case varies across branches. We probably should only compare the common results and only show the differs of them. Further work is required on this, I will raise a task in QAS team.
[1] https://git.linaro.org/qa/test-definitions.git [2] https://git.linaro.org/ci/job/configs.git [3] https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandr
oid-hikey-linaro-4.4-aosp&project=lkft%2Fandroid-hikey-linaro-4.9-aosp
-Todd
Lts-dev mailing list Lts-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lts-dev
-- Thanks, Chase
-- Thanks, Chase