LAVA and Android Toolchain Benchmarks

[Alexander Sack]

On Wed, Jan 18, 2012 at 5:59 AM, Andy Doan <andy.doan at linaro.org> wrote:
> Sorry for the wide distribution, but I wasn't sure who all would be
> interested.
>
> I spent time over the last month updating the Android monthly toolchain
> benchmark process[1] to pull its benchmark data from LAVA tests that are
> stored in validation.linaro.org. Here's an example test run[2].

Looks like an awesome pitch. So far you are using lava to store your
local results and then render them in the dashboard, right?

> I wound up using the "LAVA kernel CI views" project as a skeleton to
> create something for Android. I've got a local prototype that's starting
> to do just about everything I want (I'm fighting some issues the the
> javascript FLOT library for my charts). I'm attaching a screenshot so
> you can get a rough idea.
>
> Before I really invest time, I wanted to get people's thoughts. Some big
> questions for me:
>
> 1) Is anyone against doing this?

No, that's great and having a great report for such things is exactly
what we are looking for. Of course, later with automated builds for
toolchain tip branches and lava runs to validate and get benchmark
results.

> 2) The project is currently called "Android Benchmarks". However, I'm
> wondering if we should make more of a generic view for "Android".

IIRC, organizing the dashboard around teams and boards and/or
products/baselines was discussed at connect and makes sense imo.

My take though is that a team dashboard is basically putting multiple
views on a single page, so starting with one detailed view (toolchain
benchmarking) sounds fine.

> "Toolchain Benchmarks" could then be one part of this, but we'd have a
> spot to add other things if needed/wanted. Due to how projects are shown
> across the top of validation.l.o, I think we need pretty concise entries.

I agree. For navigation hierarchy it makes sense to think about what
areas would have such view. I would think this view should be
accessible from an Android, Toolchain WG and Benchmarking area. (or
only one of those to start).

> 3) If the general thought is "this looks okay", then are you guys okay
> with targeting it for next month?

I have no direct say on that, but I am certainly happy if blueprints
for such cool views get scheduled. So +1 from me.

-- 
Alexander Sack
Technical Director, Linaro Platform Teams
http://www.linaro.org | Open source software for ARM SoCs
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog