On Tue, 22 Jan 2019 at 11:38, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
On Tue, 22 Jan 2019 at 09:43, Axel Lebourhis axel.lebourhis@linaro.org wrote:
On Mon, 21 Jan 2019 at 18:20, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
On Mon, 21 Jan 2019 at 15:52, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi everyone,
I'm using Bamboo to build new binaries. My goal is to setup my
builder to
trigger Squad to submit tests to LAVA using the latest binaries. This part is done but I still have a problem. If there are
regressions with
new binaries, Bamboo will still say the last build is good because it only performs
curl
request and that's it.
My question is, is there a way to get a feedback from Squad concerning tests results ? For example the number of fail/pass. This would be
enough
to me, I'd just have to write something to compare 2 builds.
you already can compare 2 builds in SQUAD using 'report' API: https://squad.readthedocs.io/en/latest/api.html#builds-api-builds
Hi Milosz,
I tested this API, it returns the url of the report, I guess this is
what it is supposed to do.
I opened the url returned, I get some output but it doesn't give me the
results.
Maybe I'm missing something ?
It's expected. Report generating might be a lengthy process so the API call is non blocking. There are several options here:
- poll the URL you got
You need to check the field status_code. It follows HTTP status codes so you're looking for 200 in this field for properly generated report. If it's empty it means report is still in cooking. If it's sth else than 200 you should check error_message for details
I checked and I get "status_code": null, "error_message": null Maybe it's related to the issue I submitted here : https://github.com/Linaro/squad/issues/457
- setup callback URL
SQUAD can issue HTTP request on the callback URL. It can also authorize itself with the token you provide. Some details in the docs: https://squad.readthedocs.io/en/latest/api.html#builds-api-builds
If I understand well, Squad will request the URL set in "callback" option when the report is ready ?
- setup email notification for the report
SQUAD will send the generated report to the email address provided (you can set only one email address)
This is one of my objectives but atm I don't know how to write email templates.
Please note that report generation tasks are sent to different queue than all other tasks. So you need to setup your worker to be aware of both queues. In our instance it's like this: /bin/celery -A squad worker -Q celery,reporting_queue --max-tasks-per-child=5000 --max-memory-per-child=1500000 --loglevel INFO
I use the same command, I took it from your qa-reports.linaro.org repo on
github.
This however will only be triggered by request, not by any event in SQUAD. Could you specify whether you would like to a) query squad about the data after you push all your results b) have squad trigger a notification on receiving all your results? The latter only sends email. We don't have any option for callback yet.
My usecase would fit the first choice. I'd like to be able to query
Squad and get data like number of pass/fail.
With these data I would be able to mark my build as failed is there are
regressions.
api/builds/<id>/status should help you with this. There is boolean 'finished' field which indicates whether SQUAD considers the build completed or not.
What makes Squad consider the build is completed ?
milosz
Regards, Axel
You can also use a badge: https://squad.readthedocs.io/en/latest/api.html#badges Example here (under test results header): https://lkft.linaro.org/
milosz
Regards, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev