linaro-validation March 2012

linaro-validation@lists.linaro.org

21 participants
36 discussions

Re: [Linaro-validation] UI driven test framework for Android

by Paul Larson

On Thu, Mar 22, 2012 at 9:26 AM, Le.chi Thu <le.chi.thu(a)linaro.org> wrote: > Hi Paul, > > I have worked with a UI driven test framework for Android earlier in > STE and would like to use it for Linaro Android test (ok from STE > side). We modified the code from open source TEMA project to work with > ICS and would like to upload it the changes to the community. > > I created a blueprint for that > https://blueprints.launchpad.net/lava-test/+spec/ui-drive-test-for-android > to has time to do it. > Can you tell me how this differs from monkey runner? Sounds similar, and Yongqin is already working on monkeyrunner support. > > Please priority. > > Here is the blueprints I would like to work with in next cycle and the > highest first > > * health-check-job-vexpress (very small task) > * Include STE kernel test suite in LAVA and CI > The above two are good ones, I know the STE kernel suite has some issues to sort out wrt building though. > * ui-drive-test-for-android > See above question. > * manually-health-check > What is this about?

12 years, 10 months

Android/Panda Failures

by Andy Doan

Hey Guys, I've noticed my own Panda/Android jobs were failing. ex: <http://validation.linaro.org/lava-server/scheduler/job/17089> I've also noticed the official builds aren't working either: <http://validation.linaro.org/lava-server/scheduler/job/17092> I've personally booted my image, so I think we have a problem. Is anyone aware of this or looking into it already? -andy

12 years, 11 months

Re: [Linaro-validation] deploy timeouts

by Fathi Boudra

On 20 March 2012 16:22, Dave Pigott wrote: > Hi, > > I think we need to seriously look at allowing a timeout parameter around > "deploy_linaro_image", because vexpress is going to need around 4 hours to > deploy a full desktop, and it seems crazy that we should hard code that big > a timeout into master.py for *all* platforms. > > I realise there is an issue as to which part the timeout is applied to and > how it's apportioned, but even if we just allowed to pass through the whole > timeout to each stage it would be better than what we have at the moment. > > Thoughts? Follow up to Dave initial thread, we didn't have reached a conclusion. He reported https://bugs.launchpad.net/lava-dispatcher/+bug/961926 Could we plan (and assign someone) to land a fix for this bug? Cheers, Fathi

12 years, 11 months

De-emphasize serial connection for LAVA Dispatcher, introduce LAVA Master Agent, LAVA Test Agent

by Zygmunt Krynicki

I've registered another blueprint. This time on LAVA Dispatcher. The goal is similar as the previous blueprint about device manager, to gather feedback and to propose some small steps sub-blueprints that could be scheduled in 2012.04. The general goal is to improve the way we run tests by making them more reliable and more featureful (richer in state) at the same time. Please read and comment on the mailing list https://blueprints.launchpad.net/lava-dispatcher/+spec/lava-dispatcher-de-e… Thanks ZK -- Zygmunt Krynicki Linaro Validation Team

12 years, 11 months

Re: [Linaro-validation] Pluggable LAVA bundle specification

by Zygmunt Krynicki

W dniu 30.03.2012 02:00, Michael Hudson-Doyle pisze: > On Thu, 29 Mar 2012 10:49:21 +0200, Zygmunt Krynicki <zygmunt.krynicki(a)linaro.org> wrote: >> W dniu 29.03.2012 06:36, Michael Hudson-Doyle pisze: >>> On Tue, 27 Mar 2012 23:10:37 +0200, Zygmunt Krynicki <zygmunt.krynicki(a)linaro.org> wrote: >>>> Hi. >>>> >>>> Another specification for the backlog and your feedback. Please read >>>> this as I'd like to move towards planning dependency blueprints (even if >>>> they also end up in a backlog). >>>> >>>> https://blueprints.launchpad.net/linaro-python-dashboard-bundle/+spec/plugg… >>> >>> Hi, >>> >>> Can you motivate this one a bit? It seems sort of vaguely reasonable >>> (although I feel the creeping horrors of SGML and DTDs and things on the >>> horizon), but I would like to know the problem you are solving here. >> >> I actually wanted to avoid SGML that feels to be slowly creeping in. The >> motivation is to allow the core format to remain small and lean and to >> allow users to experiment with custom data sets. For example, if I want >> to include accurate kernel data it could go into a linux-specific >> section. If I want to include apt/dpkg data it could go into a >> dpkg-specific section. Likewise if I want to include some trace / >> profiler data specific to my application I could create my own >> representation and store the data there. >> >> This goes hand in hand with plugs support in lava-test. I'd like to be >> able to say, run that test and capture this side band data with plugs A, >> B and C. Some ideas are: sampling CPU, memory, network and IO load, >> capturing GPU performance data, power measurement data. Capturing >> package data for non-apt systems. > > I like the idea in general, although I can't convince myself it's a > priority. It's not, it's pretty low now. I just need an excuse to release a new format with this big empty "plugs" dictionary that does not have additionalProperties: false. > I'm also not sure how some details will work in practice, like how would > the data associated with a plug be stored in postgres? Also > experimentation sort of implies versioning, would you have a version for > a plug, or would the diplay of older data sometimes break when you > install a new version of a plug? So you are right and the only solution is, we don't. We'll keep the raw data as it was provided. Specialized extensions can track any deserialized / denormalized / normalized models of that data. As for versions, we keep our current policy, every third party extension can do whatever the author desires, including shooting themselves in the foot if they want. We'll give people a sandbox to play in, I hope. ZK PS: CC-ing the list again -- Zygmunt Krynicki Linaro Validation Team

12 years, 11 months

Deployment report/fallout

by Michael Hudson-Doyle

Hi, Just a quick note about today's deployment. The good news is that things are all working :) There were a few wrinkles along the way. A few were due to the fact that replacing which package provides a certain module (lava.utils.interface in this case, which moved from lava-server to its own package) appears to confuse things a bit. Once the new code was in place, I prepared to run health check jobs. I used the new admin action to mark all 'passing' boards as 'unknown' health state, cleared out /linaro/image enough to have plenty of disk space while the health jobs ran and then turned the scheduler back on. As desired, all boards started running health check jobs. This meant that there were 30 odd processes uncompressing disk images, loopback mounting them and tarring bits of them up -- load went up to 90, and stayed there for a few hours! Although the machine was still quite usable in this state, it's probably not a good situation. I filed a couple of bugs about this: https://bugs.launchpad.net/lava-scheduler/+bug/967849 https://bugs.launchpad.net/lava-dispatcher/+bug/967877 Thankfully, the large majority of health jobs passed. We had a few issues: 3 jobs on panda hit the serial truncation thing: http://validation.linaro.org/lava-server/scheduler/job/16932/log_file http://validation.linaro.org/lava-server/scheduler/job/16946/log_file http://validation.linaro.org/lava-server/scheduler/job/16956/log_file The last was a different command though -- "mount /dev/disk/by-label/testbo" -- could it be getting worse? 1 job on origen02 (had deployment time out for reasons that are not clear: http://validation.linaro.org/lava-server/scheduler/job/16961/log_file 1 job on origen03 failed: http://validation.linaro.org/lava-server/scheduler/job/16951 but because the board was spamming the serial line when the job started, I haven't actually loaded enough of the log to find the problem yet... (maybe we should hard reset the board when we terminate a badly behaving job?). Cheers, mwh

12 years, 11 months

Fwd: Networking on the fast model

by Paul Larson

Here's the other way to do it... ---------- Forwarded message ---------- From: Vishal Bhoj <vishal.bhoj(a)linaro.org> Date: Tue, Mar 27, 2012 at 11:52 AM Subject: Fwd: Networking on the fast model To: Paul Larson <paul.larson(a)linaro.org>, Zygmunt Krynicki < zygmunt.krynicki(a)linaro.org> ---------- Forwarded message ---------- From: Dave Martin <dave.martin(a)linaro.org> Date: 22 March 2012 16:56 Subject: Networking on the fast model To: Vishal Bhoj <vishal.bhoj(a)linaro.org> git clone git://git.linaro.org/people/dmart/tunsetup.git cd tunsetup make sudo ./tunsetup -c -t tap -o `id -u` tap%d >/tmp/tunsetup.conf # ^ this creates a fake ethernet device which can be accessed by you . /tmp/tunsetup.conf -C motherboard.hostbridge.interfaceName=$TUN_IF Random link-local addresses are generated automatically. You can configure boot-time networking on the model by putting: ip=$TUN_REMOTEIP::$TUN_LOCALIP:$TUN_NETMASK ...on the kernel command line If you want to allow the model to talk to the Internet, you also need to enable NAT: sudo /bin/sh -c 'echo 1 >/proc/sys/net/ipv4/ip_forward' sudo iptables -t nat -A POSTROUTING -s $TUN_REMOTEIP -j MASQUERADE ...and copy/paste the contents of /etc/resolv.conf into the model filesystem's /etc/resolv.conf The IP and DNS setup stuff could be handled by having a local DHCP server, but I was too lazy to set that up... Let me know how you get on! Cheers ---Dave

12 years, 11 months

Fwd: [Linaro-big-little] RTSM user networking support

by Paul Larson

Forwarded email 1/2 - usermode networking in android on fast models. This should cover 99% of what we want, unfortunately if you ever want to ping something, it will not work. ---------- Forwarded message ---------- From: Chris Redpath <Chris.Redpath(a)arm.com> Date: Thu, Mar 29, 2012 at 5:24 AM Subject: [Linaro-big-little] RTSM user networking support To: "linaro-big-little(a)lists.linaro.org" <linaro-big-little(a)lists.linaro.org > Hi all,**** ** ** I don’t know if its generally known about, but you can use user mode networking with models from v 7 onwards. There are some limitations and (at least in my experience so far) connections occasionally drop, but personally I could not configure TAP and a bridge without my desktop locking up, so it was a vast improvement for me. When it connects, ADB works over this connection and you can use that to set up other forwarding tunnels as necessary.**** ** ** More details at the arm infocenter here: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.arn0001a/index.… **** ** ** This is the setup I’ve been using:**** ** ** additional model command line options:**** -C motherboard.hostbridge.userNetworking=true -C motherboard.hostbridge.userNetPorts="5555=6565" -C motherboard.smsc_91c111.enabled=1**** ** ** adb options on guest os:**** (taken from https://wiki.linaro.org/Platform/Android/SetupAdbOverTcp - can be added to the init script too)**** stop adbd**** setprop service.adb.tcp.port 6565**** start adbd**** ** ** adb options on host OS:**** export ADBHOST=127.0.0.1**** sudo android-sdk-linux_x86/platform-tools/adb start-server**** adb connect localhost**** ** ** ** ** Best Regards,**** Chris**** -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ linaro-big-little mailing list linaro-big-little(a)lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-big-little

12 years, 11 months

Syntax for custom LAVA tests

by Paul Sokolovsky

Hello, Yong Qin is working on the blueprint https://blueprints.launchpad.net/lava-android-test/+spec/modify-android-bui… to add arbitrary custom client-side scripts to Android Build. He submitted first implementation of it as https://code.launchpad.net/~liuyq0307/linaro-android-build-tools/run-custom… and documented as https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration . Unfortunately, I'm not thrilled with that implementation, more specifically, its "user interface" (i.e. any parts which user directly faces) by the following reasons: 1. The idea behind Android Build's build config was that they're short and easy for human to parse, essentially one glance-over would enough to get a good idea what is built here, even for outsider. Consequently, the configs should not be overloaded with details not related to building. If there's a need for integration with other systems, we have good pattern of externalizing such details and then just referring to them with a single variable in a build config. 2. The whole approach in https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration seems like trying to encode hierarchical structure in the shell syntax, which is not much supporting of that. The end result looks pretty much like representation of graph structure in raw assembler - it's spaghetti mix of data pieces and labels, requiring long time to wrap hand around to understand it, and cumbersome and error-prone to write. So, I would like to propose alternative syntax solving the issues above. I probably should start with saying that if the talk is about LAVA, then using native LAVA JSON request immediately comes to mind. Well, I guess human-writability wasn't a design goal for that, so I skip it. It still makes sense to stick to general-purpose hierarchical structure syntax though. Except that JSON has 2 problems: a) it doesn't support comments natively, so we'll need to pre-process it; b) error reporting/localization may be still no ideal. Anyway, here's my try, it is presented as a comment to https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration and then full example at https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration/pfalc… Let's discuss if that covers our needs and constraints. Thanks, Paul Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

12 years, 11 months

Pluggable LAVA bundle specification

by Zygmunt Krynicki

Hi. Another specification for the backlog and your feedback. Please read this as I'd like to move towards planning dependency blueprints (even if they also end up in a backlog). https://blueprints.launchpad.net/linaro-python-dashboard-bundle/+spec/plugg… Many thanks ZK -- Zygmunt Krynicki Linaro Validation Team

12 years, 11 months

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

linaro-validation March 2012