Hi Todd,
I want to dive into slightly more detail here as I think this is important. Some of this might be repetitive so forgive me for that but I think we're probably due for a sync so I guess this note might help serve as that.
For VTS we aren't running the HAL tests nor the benchmarks. Is this still the correct strategy?
For CTS, we've been running a subset.
4.4 Ex: https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.4-aosp/build/v4.4.... 4.9 Ex: https://qa-reports.linaro.org/lkft/android-hikey-linaro-4.9-aosp/build/v4.9....
I have a couple of the LMG guys working on full CTS. That's in progress and we're starting to run those test jobs, establish a baseline, fix bugs that are being found etc as Anmar noted. When we run CTS this way we're sharding it so it can span multiple boards and thus decrease the time before we have results. Here's an example run of one of the shards https://lkft.validation.linaro.org/scheduler/job/29416#bottom
All that said, as I recall in the past, there had been discussion on CTS tests to run. Looking at it from a strategic point of view, do we want to go forward with running a full CTS as part of LKFT or is the current subset what we want to use for LKFT going forward?
Thoughts?
On Fri, Sep 15, 2017 at 10:03 AM, Anmar Oueja anmar.oueja@linaro.org wrote:
Hello Todd,
On Tue, Sep 12, 2017 at 10:38 AM, Todd Kjos tkjos@google.com wrote:
It looks like all 17 jobs finish now, is there another reason why there have been no aosp results sent out via email since 8/16?
We had a configuration problem that was preventing us from sending e-mails. That was fixed yesterday.
Please keep in mind that we are baselining CTS test suite and will include it back into the LKFT bttery of tests once we are done. VTS will continue to run though.
anmar _______________________________________________ Lts-dev mailing list Lts-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lts-dev