Hi, All
In my afternoon, the jobs on staging failed to run due to the following error:
http://staging.validation.linaro.org/scheduler/job/34124/log_file
Traceback (most recent call last):
File "/srv/lava/branches/lava-dispatcher/lava_dispatcher/actions/boot_control.py",
line 63, in run
client.boot_linaro_image()
File "/srv/lava/branches/lava-dispatcher/lava_dispatcher/client/base.py",
line 365, in boot_linaro_image
self._boot_linaro_image()
File "/srv/lava/branches/lava-dispatcher/lava_dispatcher/client/targetdevice.py",
line 68, in _boot_linaro_image
self.proc = self.target_device.power_on()
File "/srv/lava/branches/lava-dispatcher/lava_dispatcher/device/master.py",
line 77, in power_on
self._boot_linaro_image()
File "/srv/lava/branches/lava-dispatcher/lava_dispatcher/device/master.py",
line 403, in _boot_linaro_image
boot_cmds = self.deployment_data['boot_cmds']
KeyError: 'boot_cmds'
I can run the jobs in my morning.
--
Thanks,
Yongqin Liu
---------------------------------------------------------------
#mailing list
linaro-android(a)lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-validation(a)lists.linaro.org
http://lists.linaro.org/pipermail/linaro-validation
Hi, Michael
I found there is a web application
http://validation.linaro.org/collectd/ deployed on control, but not on
staging.
And Dave said it's probably deployed by you.
It seems that we can use that web application to check the cpu usage
of a period.
And I guess it will be helpful for me to check the "CTS consumes CPU" problem.
So could you help to deploy it on staging, or I can deployed myself if
you can give me the instructions or link of instructions.
Thanks,
Yongqin Liu
---------------------------------------------------------------
#mailing list
linaro-android(a)lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-validation(a)lists.linaro.org
http://lists.linaro.org/pipermail/linaro-validation
Hi, Michael
When I use lava-tool submit-job to submit a job to my local lava instance
(with the buildout format),
I get the "Specified device not found" error like this:
(trunk2)18:54:05 liuyq:code$ lava-tool submit-job
http://admin:b9w73ap9lox1nxt6rwgsiz17wjsfcr8yu2i7qm7r7eztcj5kpfkhna0lq9rbx4…
EXPERIMENTAL - SUBJECT TO CHANGE (See --experimental-notice for more info)
ERROR: <Fault 404: 'Specified device not found.'>
(trunk2)18:54:18 liuyq:code$
When I access the administration web page to check the device, I get the
"column lava_scheduler_app_devicetype.use_celery does not exist" error like
follwoing:
DatabaseError at /admin/lava_scheduler_app/devicetype/
column lava_scheduler_app_devicetype.use_celery does not exist
LINE 1: ...ava_scheduler_app_devicetype"."health_check_job", "lava_sche...
^
Request Method: GET
Request URL:
http://192.168.1.127/buildout/admin/lava_scheduler_app/devicetype/
Django Version: 1.4.1
Exception Type: DatabaseError
Exception Value:
column lava_scheduler_app_devicetype.use_celery does not exist
LINE 1: ...ava_scheduler_app_devicetype"."health_check_job", "lava_sche...
^
Exception Location:
/srv/lava/.cache/eggs/Django-1.4.1-py2.7.egg/django/db/backends/postgresql_psycopg2/base.py
in execute, line 52
Python Executable:
/srv/lava/instances/buildout/var/lib/bootstrap-venv/bin/uwsgi
Python Version: 2.7.3
Python Path:
['/srv/lava/instances/buildout/code/r67',
'/srv/lava/code/lava-server/versiontools-1.9.1-py2.7.egg',
'/srv/lava/code/lava-server',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg',
'/srv/lava/.cache/eggs/lava_tool-0.5-py2.7.egg',
'/srv/lava/code/lava-dispatcher',
'/srv/lava/code/lava-dashboard',
'/srv/lava/code/lava-scheduler',
'/srv/lava/code/lava-android-test',
'/srv/lava/.cache/eggs/lava_celery-0.4.1-py2.7.egg',
'/srv/lava/.cache/eggs/psycopg2-2.4.5-py2.7-linux-x86_64.egg',
'/srv/lava/.cache/eggs/python_dateutil-1.5-py2.7.egg',
'/srv/lava/.cache/eggs/keyring-0.9.1-py2.7.egg',
'/srv/lava/.cache/eggs/django_testscenarios-0.7.2-py2.7.egg',
'/srv/lava/.cache/eggs/django_devserver-0.4.0-py2.7.egg',
'/srv/lava/.cache/eggs/mocker-1.1.1-py2.7.egg',
'/srv/lava/.cache/eggs/testscenarios-0.3-py2.7.egg',
'/srv/lava/.cache/eggs/testtools-0.9.16-py2.7.egg',
'/srv/lava/.cache/eggs/django_testproject-0.1.1-py2.7.egg',
'/srv/lava/.cache/eggs/Django-1.4.1-py2.7.egg',
'/srv/lava/.cache/eggs/celery-2.5.5-py2.7.egg',
'/srv/lava/.cache/eggs/pexpect-2.4-py2.7.egg',
'/srv/lava/.cache/eggs/linaro_dashboard_bundle-1.7.2-py2.7.egg',
'/srv/lava/.cache/eggs/Twisted-12.1.0-py2.7-linux-x86_64.egg',
'/srv/lava/.cache/eggs/South-0.7.5-py2.7.egg',
'/srv/lava/.cache/eggs/simplejson-2.4.0-py2.7-linux-x86_64.egg',
'/srv/lava/.cache/eggs/django_tables2-0.11.0-py2.7.egg',
'/srv/lava/.cache/eggs/django_restricted_resource-0.2.7-py2.7.egg',
'/srv/lava/.cache/eggs/Pygments-1.5-py2.7.egg',
'/srv/lava/.cache/eggs/linaro_django_xmlrpc-0.6-py2.7.egg',
'/srv/lava/.cache/eggs/linaro_django_pagination-2.0.2-py2.7.egg',
'/srv/lava/.cache/eggs/docutils-0.9.1-py2.7.egg',
'/srv/lava/.cache/eggs/django_staticfiles-0.3.4-py2.7.egg',
'/srv/lava/.cache/eggs/configglue-1.0.3-py2.7.egg',
'/srv/lava/.cache/eggs/lava_utils_interface-1.0-py2.7.egg',
'/srv/lava/.cache/eggs/json_schema_validator-2.3-py2.7.egg',
'/usr/lib/python2.7',
'/srv/lava/.cache/eggs/Markdown-2.1.1-py2.7.egg',
'/srv/lava/.cache/eggs/python_openid-2.2.4-py2.7.egg',
'/srv/lava/.cache/eggs/django_openid_auth-0.4-py2.7.egg',
'/srv/lava/.cache/eggs/django_debian-0.10.2-py2.7.egg',
'/srv/lava/.cache/eggs/zc.recipe.egg-1.3.2-py2.7.egg',
'/srv/lava/.cache/eggs/kombu-2.1.8-py2.7.egg',
'/srv/lava/.cache/eggs/anyjson-0.3.1-py2.7.egg',
'/srv/lava/.cache/eggs/zope.interface-4.0.1-py2.7-linux-x86_64.egg',
'/srv/lava/.cache/eggs/pyxdg-0.23-py2.7.egg',
'/srv/lava/.cache/eggs/django_seatbelt-1.0.1-py2.7.egg',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/site-packages/pip-1.1-py2.7.egg',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/site-packages',
'/srv/lava/.cache/eggs/amqplib-1.0.2-py2.7.egg',
'.',
'',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/site-packages/pip-1.1-py2.7.egg',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/plat-linux2',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/lib-tk',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/lib-old',
'/srv/lava/instances/buildout/var/lib/bootstrap-venv/lib/python2.7/lib-dynload',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk']
Server time: Wed, 10 Oct 2012 10:55:45 +0000
I guess this is because I did the the update with wrong method,
Do you know if I can fix this problem with some commands?
I guess if we can make the django create the database correctly, this
problem will go.
but I don't know how to do that:(
--
Thanks,
Yongqin Liu
---------------------------------------------------------------
#mailing list
linaro-android(a)lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-validation(a)lists.linaro.org
http://lists.linaro.org/pipermail/linaro-validation
Hi all,
I found an interesting health failure today on origen07
http://validation.linaro.org/lava-server/scheduler/job/35016/log_file
When you look at the log, you see that the board starts off at the u-boot prompt. It then tries to do a "reboot", which (obviously) fails. So naturally, it then does a hard reset, and this is where it does something very odd: It interrupts the boot and tries to boot the previously installed test image. I haven't yet looked at the dispatcher code to figure out why (that's my next job).
What then started alarm bells ringing was that I saw this:
1261680 bytes read
reading uInitrd
1532597 bytes read
reading board.dtb
** Unable to read "board.dtb" from mmc 0:5 **
So whatever the test image was, it was expecting a device tree blob, which I would have assumed would have to have been installed during deploy_linaro_image() being that if there is one it should just be part of the test boot deployment.
So I looked at the log from the previous job:
http://validation.linaro.org/lava-server/scheduler/job/34938/log_file#entry…
and sure enough, you'll see at that mark the same issue.
So there are two things:
1) There's some twisted logic in the dispatcher that's making it do odd things if it starts off in u-boot
2) Do we have an issue with dtbs not being handled properly by lava, or is it just that the hwpack was incomplete?
Thanks
Dave
FYI,
I just made some pretty big updates to our dispatcher documentation.
This stuff is hard to view as a code review, so I just merged it (it
can't be worse than what we had) and it can be viewed at:
http://lava-dispatcher.readthedocs.org/en/latest/
Please let me know if something looks bad (or even better make the
change yourself)
If you don't follow dispatcher commit or review mail, you might not be
aware than Andy and I have been working an implementation of the "black
box" tests we've been talking about for a while. This mail is just a
summary of the things that have been mentioned in review that would be
good to fix "in the next branch" so that we don't forget about them.
The list probably doesn't make a great deal of sense if you haven't read
the review mail, sorry about that.
In no particular order:
* explode if an android test has deps rather than trying to install
via apt-get
* need to have lava-test-runner be a proper service on android
* make power_off() == 'in the master image' consistently
* make cmd_android_install_binaries work for !master deployments
* think about suppressing the root shell on console on ubuntu
* remove separation between LavaClient and TargetBasedClient,
consider slimming (and renaming?) interface
* fix retreive_results return interface
* rename download_image
* look at how boot_options works
* define common target superclass for FastModelTarget and QEMUTarget
* set up automated dispatcher validation
I've just thought of one more thing, which is that running a
lava_test_shell action followed by a lava_test_run action in the same
job will likely not work well because the lava test shell test will be
run again when the lava_test_run action boots the image -- maybe the
lava-test-runner should clean up after itself so that the tests only run
on *the* next boot rather than every following boot? If we do do the
"disable auto-login-shell" action, then it should probably reenable that
too...
Cheers,
mwh
Hi all,
Just a quick note to say that I'm restoring the staging database from
the latest production snapshot. Staging will be down for however long
this takes (maybe an hour).
Cheers,
mwh
Hi All,
Finally, after months of delays, our leased line is stable and usable. I have the router configured and ready to switch over, which I plan to do on Monday. As a consequence, I will be putting all the boards in LAVA offline on Sunday morning (UK time). Currently running jobs will be allowed to complete, but any new jobs will get queued, and not run until the switchover is complete.
While the lab is down, I will also be upgrading control to precise and attaching the WiFi AP so that it has access to the outside world.
Obviously, with the switch over, the IP address of v.l.o will change, and this will take some time to permeate through to DNS. I will make sure that this has happened before bringing all the boards back online, but it will probably mean that LAVA will not be running jobs until Tuesday morning, and that v.l.o will actually disappear for a couple of hours on Monday.
Once the transition has happened, we should see a significant improvement in the response time when accessing v.l.o and utilising LAVA. Of course, if anyone sees anything wrong or odd, please alert me immediately.
Thanks
Dave
Dave Pigott
Validation Engineer
T: +44 1223 40 00 63 | M +44 7940 45 93 44
Linaro.org │ Open source software for ARM SoCs
Follow Linaro: Facebook | Twitter | Blog
