On Tue, 22 Jan 2019 at 12:50, Axel Lebourhis axel.lebourhis@linaro.org wrote:
On Tue, 22 Jan 2019 at 11:38, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Tue, 22 Jan 2019 at 09:43, Axel Lebourhis axel.lebourhis@linaro.org wrote:
On Mon, 21 Jan 2019 at 18:20, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Mon, 21 Jan 2019 at 15:52, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi everyone,
I'm using Bamboo to build new binaries. My goal is to setup my builder to trigger Squad to submit tests to LAVA using the latest binaries. This part is done but I still have a problem. If there are regressions with new binaries, Bamboo will still say the last build is good because it only performs curl request and that's it.
My question is, is there a way to get a feedback from Squad concerning tests results ? For example the number of fail/pass. This would be enough to me, I'd just have to write something to compare 2 builds.
you already can compare 2 builds in SQUAD using 'report' API: https://squad.readthedocs.io/en/latest/api.html#builds-api-builds
Hi Milosz,
I tested this API, it returns the url of the report, I guess this is what it is supposed to do. I opened the url returned, I get some output but it doesn't give me the results. Maybe I'm missing something ?
It's expected. Report generating might be a lengthy process so the API call is non blocking. There are several options here:
- poll the URL you got
You need to check the field status_code. It follows HTTP status codes so you're looking for 200 in this field for properly generated report. If it's empty it means report is still in cooking. If it's sth else than 200 you should check error_message for details
I checked and I get "status_code": null, "error_message": null Maybe it's related to the issue I submitted here : https://github.com/Linaro/squad/issues/457
I'm not sure. Could you check if your worker consumes messages from reporting_queue?
- setup callback URL
SQUAD can issue HTTP request on the callback URL. It can also authorize itself with the token you provide. Some details in the docs: https://squad.readthedocs.io/en/latest/api.html#builds-api-builds
If I understand well, Squad will request the URL set in "callback" option when the report is ready ?
yes, correct
- setup email notification for the report
SQUAD will send the generated report to the email address provided (you can set only one email address)
This is one of my objectives but atm I don't know how to write email templates.
You don't need a new template for this to work. SQUAD will use it's default if there is no alternative set. The only thing is to provide email address to send report to. Of course SQUAD's email sending settings have to be correct as well: https://docs.djangoproject.com/en/2.1/topics/email/
Please note that report generation tasks are sent to different queue than all other tasks. So you need to setup your worker to be aware of both queues. In our instance it's like this: /bin/celery -A squad worker -Q celery,reporting_queue --max-tasks-per-child=5000 --max-memory-per-child=1500000 --loglevel INFO
I use the same command, I took it from your qa-reports.linaro.org repo on github.
that should work then. Could you check your message queue on the message broker to see if there are any messages pending?
milosz
This however will only be triggered by request, not by any event in SQUAD. Could you specify whether you would like to a) query squad about the data after you push all your results b) have squad trigger a notification on receiving all your results? The latter only sends email. We don't have any option for callback yet.
My usecase would fit the first choice. I'd like to be able to query Squad and get data like number of pass/fail. With these data I would be able to mark my build as failed is there are regressions.
api/builds/<id>/status should help you with this. There is boolean 'finished' field which indicates whether SQUAD considers the build completed or not.
What makes Squad consider the build is completed ?
milosz
Regards, Axel
You can also use a badge: https://squad.readthedocs.io/en/latest/api.html#badges Example here (under test results header): https://lkft.linaro.org/
milosz
Regards, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev