Re: [Lts-dev] [Lkft-maintainers] android-hikey-linaro v4.4.114-568662-ga01f7678d0a9/a01f7678d0a9: regressions detected in project android-lkft/android-hikey-linaro-4.4-aosp

14 Feb 2018

      On 13 February 2018 at 10:26, Milosz Wasilewski
milosz.wasilewski@linaro.org wrote:
...
Currently it's summary per module. I'm not sure if we can easily do a
more generic one without major changes. I'll check that.
I guess if we can eliminate the
"CtsLibcoreTestCases/libcore.java.time.zone.IcuZoneRulesProviderTest.*"
which is a bug in reporting, the list would be much shorter already.
I sent fix for review:
https://github.com/Linaro/squad/pull/227
milosz
...
milosz
On 2 February 2018 at 18:07, Todd Kjos tkjos@google.com wrote:
...
In the email, I like the info that is presented now, but would just add a
summary per suite-type (eg, rollup results for each of boot, vts, cts). For
cts, I'd be fine with a rollup on a per abi basis or combining the abis,
whatever is easier. Seeing the rolled-up pass/fail numbers helps give
confidence that all the tests actually ran and the results are in line with
expectations without having scroll down and add things up in my head.
On Thu, Feb 1, 2018 at 12:13 PM, Dan Rue dan.rue@linaro.org wrote:
...
We can format it however you'd like.
Other projects did not want to see their persistent failures, so they're
hidden unless they're "new", in which case they show up in the
regressions list.
So, I can add in the list of failures below the list of regressions, and
above the "Boards, architectures and test suites" section. Any other
changes you'd like to see?
Dan
On Thu, Feb 01, 2018 at 12:05:29PM -0800, Todd Kjos wrote:
...
Yes, that's better, but its a pretty long list so its hard to quickly
tell
if there are a lot of failures. How hard would it be to have a summary
statement for cts and vts that give the overall pass/fail (so we don't
have
to scan down the whole list of tests to get a feel for how many failures
there are)?
On Thu, Feb 1, 2018 at 9:06 AM, Tom Gall tom.gall@linaro.org wrote:
...
Is this in keeping what you were thinking Todd from your prior email?
Tom
...
On Feb 1, 2018, at 11:04 AM, Linaro QA (beta)
qa-reports@linaro.org
wrote:
...
Summary

git repo: https://android.googlesource.com/kernel/hikey-linaro
git branch: android-hikey-linaro-4.4
git commit: a01f7678d0a99aadc2d76e1ab9d3bc692d3b7ad8
git describe: v4.4.114-568662-ga01f7678d0a9
Test details:
https://qa-reports.linaro.org/android-lkft/android-hikey-
linaro-4.4-aosp/build/v4.4.114-568662-ga01f7678d0a9
...
Regressions (compared to build v4.4.113-568657-g7af7b22459e6)

hi6220-hikey - arm64 (AOSP master):
 vts-test/arm64-v8a.VtsKernelLtp:

VtsKernelLtp.syscalls.nanosleep01_64bit

test src: not informed

Boards, architectures and test suites:
hi6220-hikey - arm64 (AOSP master)

boot - pass: 6,
boottime-test - pass: 3,
cts-lkft-arm64-v8a - pass: 2,
cts-lkft-arm64-v8a/arm64-v8a.CtsAslrMallocTestCases - pass: 1,
cts-lkft-arm64-v8a/arm64-v8a.CtsBionicTestCases - pass: 2618,
cts-lkft-arm64-v8a/arm64-v8a.CtsBluetoothTestCases - pass: 42,

fail: 3

cts-lkft-arm64-v8a/arm64-v8a.CtsCameraTestCases - pass: 224,
cts-lkft-arm64-v8a/arm64-v8a.CtsDisplayTestCases - pass: 13,
cts-lkft-arm64-v8a/arm64-v8a.CtsDrmTestCases - pass: 42,
cts-lkft-arm64-v8a/arm64-v8a.CtsGraphicsTestCases - pass: 1553,
cts-lkft-arm64-v8a/arm64-v8a.CtsHardwareTestCases - pass: 12,
cts-lkft-arm64-v8a/arm64-v8a.CtsJankDeviceTestCases - pass: 1,
cts-lkft-arm64-v8a/arm64-v8a.CtsJniTestCases - pass: 65,
cts-lkft-arm64-v8a/arm64-v8a.CtsLibcoreLegacy22TestCases - pass:

5,

cts-lkft-arm64-v8a/arm64-v8a.CtsLibcoreTestCases - pass: 17845,

fail:
2
...