On 23 December 2017 at 15:43, Milosz Wasilewski <milosz.wasilewski@linaro.org> wrote:
+Vishal

On 22 December 2017 at 17:56, Todd Kjos <tkjos@google.com> wrote:
> In this view: https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-aosp/tests/vts-test/arm64-v8a.VtsKernelConfig/VtsKernelConfigTest.testKernelConfigs
>
> What does it mean that it passes "hi6220-hikey" and fails
> "hi6220-hikeyaosp-stable"? Does it mean that the latest kernel passes,
> but the kernel prebuilt for aosp doesn't?

hi6220-hikey uses AOSP master userspace
hi6220-hikeyaosp-stable uses Android O userspace
Beside that kernels "should" be the same. Vishal might know why it's
not the case.

Milosz is right about the userspace. The kernel is the same in both the builds. The
issue reported as regression happens intermittently.

This is the reasoning for the issue from Martijn on other mail discussion:
This is because of another binder change that delays handling certain
events to improve latency - and the test had certain assumptions about the
arrival time of these events it shouldn't have.

I don't have the kernel commit triggering this (am on mobile), but the test
should be fixed instead - fix is here:

https://android-review.googlesource.com/563488

The test has been updated but we haven't updated VTS to pick this fix. I'll be updating VTS next week
as part of monthly VTS + userspace update. 

milosz

> On Thu, Dec 21, 2017 at 7:49 PM, Linaro QA <qa-reports@linaro.org> wrote:
>> Summary
>> ------------------------------------------------------------------------
>>
>> git repo: https://android.googlesource.com/kernel/hikey-linaro
>> git branch: android-hikey-linaro-4.9
>> git commit: 19675ab872e1b45820822489b65cd6c950b8b679
>> git describe: v4.9.71-638768-g19675ab872e1
>> Test details: https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-aosp/build/v4.9.71-638768-g19675ab872e1
>>
>> Regressions (compared to build v4.9.71-638766-ga0caa11e6b11)
>> ------------------------------------------------------------------------
>>
>> hi6220-hikey - arm64:
>>   vts-test/armeabi-v7a.VtsKernelBinderTest:
>>     * VtsKernelBinderTest.BinderDriverInterfaceTest.Transaction_IPC64_32bit
>>
>>     * test src: not informed
>>
>>
>> Boards, architectures and test suites:
>> -------------------------------------
>>
>> hi6220-hikey - arm64
>> * boot - pass: 5,
>> * boottime-test - pass: 3,
>> * vts-test - pass: 8,
>> * vts-test/arm64-v8a.KernelProcFileApiTest - fail: 1, pass: 23,
>> * vts-test/arm64-v8a.SyscallExistenceTest - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelApiSysfsTest - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelBinderTest - pass: 53,
>> * vts-test/arm64-v8a.VtsKernelConfig - pass: 1,
>> * vts-test/arm64-v8a.VtsKernelHwBinder - pass: 1,
>> * vts-test/arm64-v8a.VtsKernelLibcutilsTest - pass: 6,
>> * vts-test/arm64-v8a.VtsKernelLinuxKselftest - pass: 11,
>> * vts-test/arm64-v8a.VtsKernelLinuxKselftestPresubmit - pass: 5,
>> * vts-test/arm64-v8a.VtsKernelLtp - pass: 777,
>> * vts-test/arm64-v8a.VtsKernelNetdTest - pass: 32,
>> * vts-test/arm64-v8a.VtsKernelSelinuxFileApi - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.BasicRead - pass: 2,
>> * vts-test/arm64-v8a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.OpenAndClose - pass: 2,
>> * vts-test/arm64-v8a.VtsQtaguidTest - pass: 20,
>> * vts-test/armeabi-v7a.KernelProcFileApiTest - fail: 1, pass: 23,
>> * vts-test/armeabi-v7a.SyscallExistenceTest - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelApiSysfsTest - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelBinderTest - fail: 1, pass: 52,
>> * vts-test/armeabi-v7a.VtsKernelConfig - pass: 1,
>> * vts-test/armeabi-v7a.VtsKernelHwBinder - pass: 1,
>> * vts-test/armeabi-v7a.VtsKernelLibcutilsTest - pass: 6,
>> * vts-test/armeabi-v7a.VtsKernelLinuxKselftest - pass: 9,
>> * vts-test/armeabi-v7a.VtsKernelLinuxKselftestPresubmit - pass: 6,
>> * vts-test/armeabi-v7a.VtsKernelLtp - pass: 786,
>> * vts-test/armeabi-v7a.VtsKernelNetdTest - pass: 32,
>> * vts-test/armeabi-v7a.VtsKernelSelinuxFileApi - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.BasicRead - pass: 2,
>> * vts-test/armeabi-v7a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.OpenAndClose - pass: 2,
>> * vts-test/armeabi-v7a.VtsQtaguidTest - pass: 20,
>>
>> hi6220-hikey - arm64 (Android O)
>> * boot - pass: 4,
>> * boottime-test - pass: 3,
>> * vts-test - pass: 6,
>> * vts-test/arm64-v8a.KernelProcFileApiTest - pass: 20,
>> * vts-test/arm64-v8a.SyscallExistenceTest - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelApiSysfsTest - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelBinderTest - fail: 1, pass: 48,
>> * vts-test/arm64-v8a.VtsKernelConfig - fail: 1,
>> * vts-test/arm64-v8a.VtsKernelHwBinder - pass: 1,
>> * vts-test/arm64-v8a.VtsKernelLibcutilsTest - pass: 5,
>> * vts-test/arm64-v8a.VtsKernelLinuxKselftest - pass: 11,
>> * vts-test/arm64-v8a.VtsKernelLinuxKselftestPresubmit - pass: 5,
>> * vts-test/arm64-v8a.VtsKernelLtp - fail: 1, pass: 809,
>> * vts-test/arm64-v8a.VtsKernelSelinuxFileApi - pass: 3,
>> * vts-test/arm64-v8a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.BasicRead - pass: 2,
>> * vts-test/arm64-v8a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.OpenAndClose - pass: 2,
>> * vts-test/arm64-v8a.VtsQtaguidTest - pass: 21,
>> * vts-test/armeabi-v7a.KernelProcFileApiTest - pass: 20,
>> * vts-test/armeabi-v7a.SyscallExistenceTest - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelApiSysfsTest - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelBinderTest - fail: 1, pass: 48,
>> * vts-test/armeabi-v7a.VtsKernelConfig - fail: 1,
>> * vts-test/armeabi-v7a.VtsKernelHwBinder - pass: 1,
>> * vts-test/armeabi-v7a.VtsKernelLibcutilsTest - pass: 5,
>> * vts-test/armeabi-v7a.VtsKernelLinuxKselftest - pass: 9,
>> * vts-test/armeabi-v7a.VtsKernelLinuxKselftestPresubmit - pass: 6,
>> * vts-test/armeabi-v7a.VtsKernelSelinuxFileApi - pass: 3,
>> * vts-test/armeabi-v7a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.BasicRead - pass: 2,
>> * vts-test/armeabi-v7a.VtsKernelTunTest/GtestBinaryTest.Basic/VtsKernelTunTest.OpenAndClose - pass: 2,
>> * vts-test/armeabi-v7a.VtsQtaguidTest - pass: 21,
>>
>>
>>
>> Documentation - https://collaborate.linaro.org/display/LKFT/Email+Reports
>>
>> --
>> Linaro QA (beta)
>> https://qa-reports.linaro.org
>> _______________________________________________
>> Lts-dev mailing list
>> Lts-dev@lists.linaro.org
>> https://lists.linaro.org/mailman/listinfo/lts-dev
> _______________________________________________
> Lts-dev mailing list
> Lts-dev@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lts-dev