Hi Todd,Here is the latest testing status for v4.9 https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9- . There are 17 jobs set on each build, 16 jobs finished correctly on the recent 6 builds. CtsLibcoreTestCases is the missing one, we still need fixes for the timeout on result pulling.aosp/ Hi Milosz,I guess we can go without CtsLibcoreTestCases for the moment. Is it ok to reduce 'Expected test runs' to 16 so that qa-reports generates and sends test report for AOSP testing?
--Thanks,ChaseOn 26 August 2017 at 01:40, Todd Kjos <tkjos@google.com> wrote:Chase- I haven't seen any new AOSP results since your mail that said
it is now stable. Shouldn't we be seeing regular reports for
lkft/android-hikey-linaro-4.4-aosp like we do for
lkft/linaro-hikey-linaro-4.4-oe?
On Fri, Aug 18, 2017 at 5:57 AM, Chase QI <chase.qi@linaro.org> wrote:
> On 17 August 2017 at 10:21, Chase QI <chase.qi@linaro.org> wrote:
>> Hi Todd,
>>
>> Milosz is on holiday.
>>
>> On 17 August 2017 at 08:26, Todd Kjos <tkjos@google.com> wrote:
>>> Milosz,
>>>
>>> I was looking at the -aosp tests that just ran. Looking at a comparison of
>>> the 4.4/4.9 runs at:
>>>
>>> https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandr oid-hikey-linaro-4.4-aosp& project=lkft%2Fandroid-hikey- linaro-4.9-aosp
>>>
>>> I noticed that most CTS/VTS tests don't run. Why is that? Are we
>>> deliberately pruning the tests? Or is a failure causing the test runs to
>>> stop before getting to all the tests?
>>>
>>
>> It is the later case. Some of the test jobs finished completely, but
>> have no real test results. I am investigating these issues.
>>
>
> With latest fixes on test suite[1] and Jenkins project
> configuration[2], AOSP testing is stable now. Refer to the below
> reports. CtsLibcoreTestCases is the only incomplete job, we still need
> fixes to handle its 45k+ test results.
>
> https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9- aosp/build/v4.9.43-638384-g17c 33fe8a2e3/
> https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.4- aosp/build/v4.4.82-568129-gdc2 3078e4e65-rebuild1/
>
> The above separate reports look fine; while comparison between them
> still doesn't look good[3]. One reason is, for the same test module,
> the number of test case varies across branches. We probably should
> only compare the common results and only show the differs of them.
> Further work is required on this, I will raise a task in QAS team.
>
> [1] https://git.linaro.org/qa/test-definitions.git
> [2] https://git.linaro.org/ci/job/configs.git
> [3] https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandr oid-hikey-linaro-4.4-aosp& project=lkft%2Fandroid-hikey- linaro-4.9-aosp
>
>>> -Todd
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Lts-dev mailing list
>>> Lts-dev@lists.linaro.org
>>> https://lists.linaro.org/mailman/listinfo/lts-dev
>>>
>>
>>
>>
>> --
>> Thanks,
>> Chase
>
>
>
> --
> Thanks,
> Chase