Axel,
Happy New Year!
You've done most of the setup it seems. Does the LAVA user you created have an API token? You create these using either LAVA UI (https://validation.linaro.org/api/tokens/) or REST API if you have fresh enough LAVA version (https://master.lavasoftware.org/static/docs/v2/data-export.html#index-3). This token should be than added in SQUAD admin for backend you created.
For using SQUAD to proxy your job submissions you need web frontend with API and SQUAD worker. Our ansible scripts start frontend as systemd service with apache2 as proxy: https://github.com/Linaro/qa-reports.linaro.org/blob/master/ansible/roles/fr... Worker is an instance of celery process that talks to RabbitMQ. Our example systemd service: https://github.com/Linaro/qa-reports.linaro.org/blob/master/ansible/roles/wo...
You can also set this up locally for testing without systemd. Just use the ExecStart commands from systemd service files.
Submitting testjob to LAVA via SQUAD proxy is pretty easy. Here is an example: https://squad.readthedocs.io/en/latest/ci.html#submitting-test-job-requests
Make sure the URL for the request is in form: /api/submitjob/<team>/<project>/<version>/<ci-env>
ci-env doesn't have to exist in advance. It will be created on submission. Also make sure that 'backend' parameter matches the name of your backend in SQUAD (name field in admin). 'definition' has to be a valid YAML test job definition that works with your LAVA instance. You can try to submit it directly using LAVA interface to confirm it's valid and does what you expect.
The above setup will get you as far as submitting the job to LAVA. It will not however immediately fetch the results once the test job is finished. To do that you also need a SQUAD listener service and LAVA ZMQ publisher. SQUAD listener connects to LAVA's ZMQ publisher and monitors events related to the test job it submitted. Example listener systemd service file: https://github.com/Linaro/qa-reports.linaro.org/blob/master/ansible/roles/ma... In the local setup it's enough to issue 'listen' admin command. You can also force fetch the test jobs form admin panel. I use it for debugging but it's not a good idea to rely on this method in production setup.
Hope this helps. If you have any further questions, please don't hesitate to ask.
Best Regards, milosz
On Wed, 2 Jan 2019 at 11:09, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
First of all, I wish everyone an happy new year.
So, I'm currently trying to use Squad. I can access the web UI, I added a "group" and a "project". Now I want to submit a job to LAVA through Squad. Note that I created a "backend" through Squad administration and a Squad user through LAVA administration. I don't get how to submit a job, can you provide me a more explicit example ?
Best regards, Axel
On Mon, 17 Dec 2018 at 17:18, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Mon, 17 Dec 2018 at 15:49, Axel Lebourhis axel.lebourhis@linaro.org wrote:
On Mon, 17 Dec 2018 at 14:58, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Mon, 17 Dec 2018 at 13:52, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I was trying to install on a debian. I tried to deploy a LXC with Ubuntu 18.04 like you, now it's working fine. Is it impossible to run it on debian ?
I just tried on debian stable (9.6 in docker) and again it worked. No idea why it didn't work for you.
I found the issue, ~/.local/bin wasn't in my PATH...
I'll add this to docs. Thanks for reporting.
Also, if I understood correctly, I need to install squad in a different host than LAVA ?
I didn't try the same host as lava master, but it shouldn't be a problem. SQUAD connects to LAVA over API so it should not matter where it's installed as long as the API calls can go through.
Ok I see. After reading the documentation, I don't really understand how to connect SQUAD and LAVA. What do you think is better between using the same host and a different one ?
Let me try to explain. In SQUAD convention LAVA is a 'ci backend'. In order to connect to your instance you need to create a CI backend object in SQUAD. You can do it from /admin/ci/backend/add/ (or from REST API with POST request). The fields should be filled in as follows:
- name: this is the unique name you will use in the API calls when
submitting/retrieving jobs from LAVA instance. We use the TLD of the instance
- url: https://<your instance TLD>/RPC2
- username: name of the user in LAVA instance (one that can log into LAVA)
- token: LAVA API token for the username above
Other settings are described in the docs.
There are several ways of letting SQUAD know that there are LAVA jobs it should look for.
- submitjob API
https://squad.readthedocs.io/en/latest/ci.html#submitting-test-job-requests This is the one we use in Linaro most often. It will use SQUAD as a proxy to submit jobs to LAVA. 'backend' is the 'name' you used in the CI backend settings above. 'definition' is the same YAML that is submitted to LAVA for job execution. From this point SQUAD will track jobs submitted to LAVA. When the job is completed SQUAD will pull the results to SQUAD. For this to work you will need a background worker (celery) and listener (started with squad-admin command).
- watchjob API
https://squad.readthedocs.io/en/latest/ci.html#submitting-test-job-watch-req... You submit jobs to LAVA on your own and than ask SQUAD to listen to LAVA job status. It's not widely used and there might be issues especially with jobs that run Android CTS. If you decide to use this method, I'll need to fix these problems.
It's a good idea to check our ansible playbook for qa-reports.linaro.org. It's in github here: https://github.com/Linaro/qa-reports.linaro.org/tree/master/ansible
Hope this helps. If you still have issues please don't hesitate to ask
milosz
Regards, Axel
milosz
Regards, Axel
On Mon, 17 Dec 2018 at 14:28, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Mon, 17 Dec 2018 at 12:59, Axel Lebourhis axel.lebourhis@linaro.org wrote: > > Hi everyone, > > I'm using LAVA at NXP and after talking with Neil and Milosz, I'd like to > use SQUAD. > I followed the instructions here > https://squad.readthedocs.io/en/latest/install.html > But when I try to run the command "squad", bash tells me the command is not > found.
I just pulled a fresh ubuntu 18.04 container and repeated the steps from the docs. Everything works OK. Did you use python3? SQUAD requires p3 and doesn't work on p2. Here is the list of installed pip packages:
# pip3 freeze amqp==2.3.2 asn1crypto==0.24.0 billiard==3.5.0.5 celery==4.2.1 certifi==2018.11.29 chardet==3.0.4 coreapi==2.3.3 coreschema==0.0.4 cryptography==2.1.4 Django==1.11.17 django-cors-headers==2.4.0 django-filter==1.1.0 django-simple-history==2.6.0 djangorestframework==3.9.0 djangorestframework-filters==0.10.2.post0 gunicorn==19.9.0 idna==2.6 itypes==1.1.0 Jinja2==2.10 keyring==10.6.0 keyrings.alt==3.0 kombu==4.2.2 Markdown==3.0.1 MarkupSafe==1.1.0 msgpack==0.6.0 psycopg2==2.7.6.1 pycrypto==2.6.1 pygobject==3.26.1 pyparsing==2.3.0 python-dateutil==2.7.5 pytz==2018.7 pyxdg==0.25 PyYAML==3.13 pyzmq==17.1.2 requests==2.21.0 SecretStorage==2.3.1 six==1.11.0 sqlparse==0.2.4 squad==0.56 svgwrite==1.2.1 uritemplate==3.0.0 urllib3==1.24.1 vine==1.1.4 whitenoise==4.1.2
> I tried to run it through Python3 shell, but didn't work either. > Maybe I'm missing something obvious ?
Did you try to use virtualenv? After installing I'm able to import 'squad' in the python3 shell.
# python3 Python 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import squad >>>
So I've no idea what went wrong :( I'm available on #linaro-qa on freenode if you have a few moments to debug.
milosz
> > Best regards, > Axel > _______________________________________________ > Squad-dev mailing list > Squad-dev@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/squad-dev