On 01/18/2012 11:24 AM, Zygmunt Krynicki wrote:
On Wed, Jan 18, 2012 at 5:44 PM, Andy Doan andy.doan@linaro.org wrote:
On 01/18/2012 10:21 AM, Zygmunt Krynicki wrote:
Again, could you share the code?
Okay, here's a drop:
https://code.launchpad.net/~doanac/+junk/lava_android_benchmark_views
The commit message details some of the problems the code currently has.
Looking at it now
Some quick comments:
- We should strive to use the database to compute averages/stdev
- We should materialize such changes over time to keep performance
sensible (scanning everything is going to be too slow)
This was just a proof-of-concept, so things are sloppy. Still, I think you are right. It will probably make building up representations like you noted in comment 4 easier.
I haven't fully read into the dashboard_app code. But it seems to have a concept of storing the bundle and then "analyzing it" which I think moves it to a database representation. That type of concept might work well for this.
I think there might even be a chance to simply make the data a "super bundle" ie combine the bundles into one. With the "measurement" fields just being the averages. The only other thing I'd want to add to each measurement is the standard deviation - not sure if that can fit in the existing data model or not (or if I'm over-thinking this)
- Associating BenchmarkRun with Bundle is odd, why not with TestRun?
A BenchmarkRun contains multiple test runs. For example it may have 4 skia, 4 v8, and 4 0xbench Test Runs. We can then take the averages of each of type to get better average results. So a BenchmarkRun can be viewed as one job that was submitted to the validation to run against a given android build. Then we can compare say a linaro-gcc-4.6 run with an Android-gcc-4.4 run.
- In benchmark_run.get_summary(jso) you iterate over deserialized
bundle. You may want to iterate over the database models instead, computing sanitized_bundle() is pricy and it was just a hack for the dashboard (I should have made it private)
Sure - the original code I had written before this django app was based on pulling streams from the validation server so it was json based.
I'm still unsure how the computation model work ATM, I'll post again.
Sure - the code needs some cleaning up and optimizations. Thanks for your input.llllll