Chase- I haven't seen any new AOSP results since your mail that said
it is now stable. Shouldn't we be seeing regular reports for
lkft/android-hikey-linaro-4.4-aosp like we do for
lkft/linaro-hikey-linaro-4.4-oe?

On Fri, Aug 18, 2017 at 5:57 AM, Chase QI <chase.qi@linaro.org> wrote:
> On 17 August 2017 at 10:21, Chase QI <chase.qi@linaro.org> wrote:
>> Hi Todd,
>>
>> Milosz is on holiday.
>>
>> On 17 August 2017 at 08:26, Todd Kjos <tkjos@google.com> wrote:
>>> Milosz,
>>>
>>> I was looking at the -aosp tests that just ran. Looking at a comparison of
>>> the 4.4/4.9 runs at:
>>>
>>> https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandroid-hikey-linaro-4.4-aosp&project=lkft%2Fandroid-hikey-linaro-4.9-aosp
>>>
>>> I noticed that most CTS/VTS tests don't run. Why is that? Are we
>>> deliberately pruning the tests? Or is a failure causing the test runs to
>>> stop before getting to all the tests?
>>>
>>
>> It is the later case. Some of the test jobs finished completely, but
>> have no real test results. I am investigating these issues.
>>
>
> With latest fixes on test suite[1] and Jenkins project
> configuration[2], AOSP testing is stable now. Refer to the below
> reports. CtsLibcoreTestCases is the only incomplete job, we still need
> fixes to handle its 45k+ test results.
>
> https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-aosp/build/v4.9.43-638384-g17c33fe8a2e3/
> https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.4-aosp/build/v4.4.82-568129-gdc23078e4e65-rebuild1/
>
> The above separate reports look fine; while comparison between them
> still doesn't look good[3]. One reason is, for the same test module,
> the number of test case varies across branches. We probably should
> only compare the common results and only show the differs of them.
> Further work is required on this, I will raise a task in QAS team.
>
> [1] https://git.linaro.org/qa/test-definitions.git
> [2] https://git.linaro.org/ci/job/configs.git
> [3] https://qa-reports.linaro.org/_/compare/?project=lkft%2Fandroid-hikey-linaro-4.4-aosp&project=lkft%2Fandroid-hikey-linaro-4.9-aosp
>
>>> -Todd
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Lts-dev mailing list
>>> Lts-dev@lists.linaro.org
>>> https://lists.linaro.org/mailman/listinfo/lts-dev
>>>
>>
>>
>>
>> --
>> Thanks,
>> Chase
>
>
>
> --
> Thanks,
> Chase