linaro-validation November 2012

linaro-validation@lists.linaro.org

17 participants
58 discussions

Ability to pull a git repository that includes the test definition

by Senthil Kumaran S

Hi, We now have support for pulling git repositories which holds the test definition, by defining it in the job file. The code is available here - https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-from-repo A sample job file which works is attached with this email. The git repository which I used is here - http://git.linaro.org/gitweb?p=people/stylesen/sampletestdefs.git;a=summary Please have a look at the whiteboard here - https://blueprints.launchpad.net/lava-dispatcher/+spec/test-case-managment-… - which gives more information about the change. Will shortly push support for bzr. PS: The code available in the above branch is still in development and not production ready. Any attempt to merge it with production will prove FATAL :) Thank You. -- Senthil Kumaran S http://www.stylesen.org/ http://www.sasenthilkumaran.com/

11 years, 5 months

Urgent TC2 request

by Dave Pigott

Hi all, David Zinman contacted me with an urgent request. Apparently Mathieu's TC2 is playing up and he has an urgent need for it. Could we liberate one from the lab while Mathieu returns it, and then deal with either fixing it (if it's the motherboard, I have a spare; if it's the tile, we'll get ARM to swap it out)? Looking at the TC2 load, it's not so bad that we're going to block on it. Thanks Dave

11 years, 5 months

Weekend health check failures

by Dave Pigott

Three over the weekend: ---------------- panda-es03 ---------------- http://validation.linaro.org/lava-server/scheduler/job/39026 Looks like some nasty kernel panic during wget. Went on the board, rebooted it. All seems ok, so put back online to retest. ---------------- snowball02 ---------------- http://validation.linaro.org/lava-server/scheduler/job/39129 ---------------- snowball03 ---------------- http://validation.linaro.org/lava-server/scheduler/job/39125 Genuine problem - looks like they aren't being recognised on their USB port. Not much I can do from home, so will look at them in the morning. Thanks Dave

11 years, 5 months

Overnight failures

by Dave Pigott

Just the one, although the report says two. Did someone else put one online that had failed? --------------------- vexpress-tc2-01 --------------------- http://validation.linaro.org/lava-server/scheduler/job/39042 This *looks* like there was an existing telnet session on the board that hand't closed. I went on the board and it was fine, so whatever/whoever it was had disconnected. Rebooted and put back online. Thanks Dave

11 years, 5 months

health check images

by Andy Doan

Hey Guys, We currently use ubuntu-desktop RFS's for our daily health checks in LAVA. Its my understanding the dev-platform team will be sunsetting ubuntu-desktop images. Due to this, I think we (the LAVA team) need to think about moving to a new set of images for our health checks so that they better reflect reality. 2012.11 should be producing some pre-built images for both server and nano. I think picking the server pre-built images probably makes more sense since it gives us a little more coverage than nano, but I don't have a strong sense on whether it makes much difference since we basically just do boot-testing in our health check. = So what does this mean? I think we'll need to file a 12.12 blueprint for doing health check investigation work. Essentially, we should build a job for each device-type in production based off its health job. We then update the image URL to be the new candidate. We then submit it 100 times and do a failure analysis. At this point we cross our fingers and hope the failures aren't worse than what we currently see. If so, we can switch the image over. If its not the case, we'll need to work with dev-platform team on getting new issues addressed. make sense? who wants to help? :) -andy

11 years, 5 months

lava-test-shell tweaks and audio capture

by Michael Hudson-Doyle

I was recently making some additions to the lava-test-shell code and in the process more or less had to write some documentation on how things work. Here is what I wrote (actually what I wrote includes stuff about test result attachments but that's not in trunk yet): # LAVA Test Shell implementation details # ====================================== # # The idea of lava-test-shell is a YAML test definition is "compiled" into a # job that is run when the device under test boots and then the output of this # job is retrieved and analyzed and turned into a bundle of results. # # In practice, this means a hierarchy of directories and files is created # during test installation, a sub-hierarchy is created during execution to # hold the results and these latter sub-hierarchy whole lot is poked at on the # host during analysis. # # On Ubuntu and OpenEmbedded, the hierarchy is rooted at /lava. / is mounted # read-only on Android, so there we root the hierarchy at /data/lava. I'll # assume Ubuntu paths from here for simplicity. # # The directory tree that is created during installation looks like this: # # /lava/ # bin/ This directory is put on the path when the # test code is running -- these binaries can # be viewed as a sort of device-side "API" # for test authors. # lava-test-runner The job that runs the tests on boot. # lava-test-shell A helper to run a test suite. # tests/ # ${IDX}_${TEST_ID}/ One directory per test to be executed. # testdef.yml The test definition. # install.sh The install steps. # run.sh The run steps. # [repos] The test definition can specify bzr or git # repositories to clone into this directory. # # In addition, a file /etc/lava-test-runner.conf is created containing the # names of the directories in /lava/tests/ to execute. # # During execution, the following files are created: # # /lava/ # results/ # cpuinfo.txt Hardware info. # meminfo.txt Ditto. # build.txt Software info. # pkgs.txt Ditto # ${IDX}_${TEST_ID}-${TIMESTAMP}/ # testdef.yml Attached to the test run in the bundle for # archival purposes. # install.sh Ditto. # run.sh Ditto. # stdout.log The standard output of run.sh. # stderr.log The standard error of run.sh (actually not # created currently) # return_code The exit code of run.sh. # # After the test run has completed, the /lava/results directory is pulled over # to the host and turned into a bundle for submission to the dashboard. Now clearly the work Senthil is doing on getting testdefs from a repo is going to make some changes to the files that get created during installation (I had envisioned that the repo that the job specifies would be cloned to /lava/tests/${IDX}_${TEST_ID} but that might not be completely safe -- if the repo contains a file called run.sh we don't want to clobber that! Not sure what to do here). But the tweaks I want to propose are more to do with what goes into the /lava/bin and /lava/results directories. For a start, I don't think it's especially useful that the tests run with lava-test-runner or lava-test-shell on $PATH -- there's no reason for a test author to call either of those functions! However I want to add helpers -- called lava-test-case and lava-test-case-attach in my brain currently: * lava-test-case will send the test case started signal, run a shell command, interpret its exit code as a test result and send the test case stopped signal * lava-test-case-attach arranges to attach a file to the test result for a particular test case id So you could imagine some run steps for an audio capture test: run: steps: - lava-fft -t 3 > generated.wav - lava-test-case-attach sine-wave generated.wav audio/wav - lava-test-case sine-wave aplay generated.wav and appropriate hooks on the host side: * test case start/stop hooks that would capture audio * an "analysis hook" that would compare the captured sample with the attached wav file (and attach the captured wav) Semi-relatedly, I would like to (at least optionally) store the test result data more explicitly in the /lava/results/${IDX}_${TEST_ID}-${TIMESTAMP} directory. Maybe something like this (in the above style): # /lava/ # results/ # ... As before # ${IDX}_${TEST_ID}-${TIMESTAMP}/ # ... All the stuff we had before. # cases/ # ${TEST_CASE_ID}/ # result This would contain pass/fail/skip/unknown # units Mb/s, V, W, Hz, whatever # measurement Self explanatory I expect. # attachments/ # ${FILENAME} The content of the attachment # ${FILENAME}.mimetype The mime-type for the attachment # attributes/ # ${KEY} The content of the file would be the # value of the attachment. # ... other things you can stick on test results ... Basically this would be defining another representation for test results: on the file system, in addition to the existing two: as JSON or in a postgres DB. The reason for doing this is twofold: 1) it's more amenable than JSON to being incrementally built up by a bunch of shell scripts as a lava-test-shell test runs and 2) this directory could be presented to an "analysis hook" written in shell (earlier on today I told Andy Doan that I though writing hooks in shell would be impractical; now I'm not so sure). Also: 3) (noone expects the spanish inquisition!) it would allow us to write a lava-test-shell test that does not depend on parsing stdout.log. Apologies for the log ramble! Cheers, mwh

11 years, 5 months

VExpress A5

by Dave Pigott

Hi all, I've been soak testing A5 in staging for the last few days, and I've seen 6 failures out of 26 runs. They are all down to "MciReadBlockData(): TIMEOUT! Response:0xA404012 Status:0x48" when trying to boot a test image. The only sensible way to fix this is to add a retry on booting the test image, so until that's done I don't think we can deploy it in production. Thanks Dave

11 years, 5 months

Out of office

by Dave Pigott

Hi all, I have some odds and ends to sort out tomorrow, so short of a major problem in the lab, I'll be working from home. Thanks Dave

11 years, 5 months

Health check failures

by Dave Pigott

None! \o/

11 years, 5 months

stupid daylight savings

by Andy Doan

Hey Guys Sorry I missed the weekly meeting. I had forgotten that it had changed from 8am to 7am for me due to daylight savings.

11 years, 5 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

linaro-validation November 2012