Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ? Do I need to use a specific tradefed test definition to interface it with the plugin ? Does the job submission works the same way ? using the submitjob API ?
Thank you in advance, Axel
On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it with the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. The main thing that was missing in my test definition was the "lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it with the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions'
Am I missing something ?
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. The main thing that was missing in my test definition was the "lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it
with
the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. The main thing that was missing in my test definition was the "lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it with the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
Hi Milosz,
Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed
plugin is triggered.
The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40
+0000] [INFO] Starting CTS/VTS plugin for test job: 1717
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40
+0000] [ERROR] Plugin postprocessing error: 'definitions'
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent
call last):
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition
in test_action['test']['definitions']:
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test
definition (based on yours) to match my setup.
The main thing that was missing in my test definition was the
"lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it
with
the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API
?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
[2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. The main thing that was missing in my test definition was the "lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I saw you have a repository with tradefed and ltp plugins for SQUAD. If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to attach the error logs to failed test cases.
My questions are : How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
Do I need to use a specific tradefed test definition to interface it with the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of LAVA. Tradefed creates a zip file that is later uploaded to 'artifactorial' (https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following form: curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
Does the job submission works the same way ? using the submitjob API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
Thank you in advance, Axel _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one stored
by SQUAD, retrieved with api/testjobs/<id>/definition/.
I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed
plugin is triggered.
The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40
+0000] [INFO] Starting CTS/VTS plugin for test job: 1717
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40
+0000] [ERROR] Plugin postprocessing error: 'definitions'
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most
recent call last):
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test
definition (based on yours) to match my setup.
The main thing that was missing in my test definition was the
"lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > Hi all, > > I saw you have a repository with tradefed and ltp plugins for
SQUAD.
> If I understand correctly, those plugins retrieve CTS/VTS/LTP
logs to
> attach the error logs to failed test cases. > > My questions are : > How can I setup those plugins to work with my SQUAD instance ?
you need to install the package from pip (https://pypi.org/project/squad-linaro-plugins/) to your squad instance. That should do the trick. Each project has to have the plugin enabled manually.
> Do I need to use a specific tradefed test definition to interface
it with
> the plugin ?
yes, tradefed plugin assumes that all logs are stored outside of
LAVA.
Tradefed creates a zip file that is later uploaded to
'artifactorial'
(https://archive.validation.linaro.org/). You can use some other service but it needs to support upload request in the following
form:
curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" ATTACHMENT is the zip file that tradefed created and ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the content. This step is done automatically when you use tradefed [1] or noninteractive-tradefed [2] from test-definitions
When running in squad post processing step plugin downloads the zip file and extracts logs from it. Failed tests are updated with logs.
> Does the job submission works the same way ? using the submitjob
API ?
Test job submission works the same way. Plugins work as a post processing step after LAVA jobs are completed.
[1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
[2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
milosz
> > Thank you in advance, > Axel > _______________________________________________ > Squad-dev mailing list > Squad-dev@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/squad-dev
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Thank you for your email. It is now working fine. I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. The main thing that was missing in my test definition was the "lava-test-reference".
Thank you for you help.
Regards, Axel
On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: > > On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: > > > > Hi all, > > > > I saw you have a repository with tradefed and ltp plugins for SQUAD. > > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to > > attach the error logs to failed test cases. > > > > My questions are : > > How can I setup those plugins to work with my SQUAD instance ? > > you need to install the package from pip > (https://pypi.org/project/squad-linaro-plugins/) to your squad > instance. That should do the trick. Each project has to have the > plugin enabled manually. > > > Do I need to use a specific tradefed test definition to interface it with > > the plugin ? > > yes, tradefed plugin assumes that all logs are stored outside of LAVA. > Tradefed creates a zip file that is later uploaded to 'artifactorial' > (https://archive.validation.linaro.org/). You can use some other > service but it needs to support upload request in the following form: > curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > ATTACHMENT is the zip file that tradefed created and > ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the > content. > This step is done automatically when you use tradefed [1] or > noninteractive-tradefed [2] from test-definitions > > When running in squad post processing step plugin downloads the zip > file and extracts logs from it. Failed tests are updated with logs. > > > Does the job submission works the same way ? using the submitjob API ? > > Test job submission works the same way. Plugins work as a post > processing step after LAVA jobs are completed. > > [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... > [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... > > milosz > > > > > Thank you in advance, > > Axel > > _______________________________________________ > > Squad-dev mailing list > > Squad-dev@lists.linaro.org > > https://lists.linaro.org/mailman/listinfo/squad-dev
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one
stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Hi all,
I get back to this topic as I have a new issue now. Squad worker return a "Plugin postprocessing error" when the
tradefed plugin is triggered.
The logs :
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most
recent call last):
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is
expected
and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
Best regards, Axel
On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > Hi Milosz, > > Thank you for your email. It is now working fine. > I setup my own local HTTP server and updated my tradefed test
definition (based on yours) to match my setup.
> The main thing that was missing in my test definition was the
"lava-test-reference".
> > Thank you for you help. > > Regards, > Axel > > On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> > >> > Hi all, >> > >> > I saw you have a repository with tradefed and ltp plugins for
SQUAD.
>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP
logs to
>> > attach the error logs to failed test cases. >> > >> > My questions are : >> > How can I setup those plugins to work with my SQUAD instance ? >> >> you need to install the package from pip >> (https://pypi.org/project/squad-linaro-plugins/) to your squad >> instance. That should do the trick. Each project has to have the >> plugin enabled manually. >> >> > Do I need to use a specific tradefed test definition to
interface it with
>> > the plugin ? >> >> yes, tradefed plugin assumes that all logs are stored outside of
LAVA.
>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
>> (https://archive.validation.linaro.org/). You can use some other >> service but it needs to support upload request in the following
form:
>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> ATTACHMENT is the zip file that tradefed created and >> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading
the
>> content. >> This step is done automatically when you use tradefed [1] or >> noninteractive-tradefed [2] from test-definitions >> >> When running in squad post processing step plugin downloads the
zip
>> file and extracts logs from it. Failed tests are updated with
logs.
>> >> > Does the job submission works the same way ? using the
submitjob API ?
>> >> Test job submission works the same way. Plugins work as a post >> processing step after LAVA jobs are completed. >> >> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
>> >> milosz >> >> > >> > Thank you in advance, >> > Axel >> > _______________________________________________ >> > Squad-dev mailing list >> > Squad-dev@lists.linaro.org >> > https://lists.linaro.org/mailman/listinfo/squad-dev
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote: > > Hi all, > > I get back to this topic as I have a new issue now. > Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. > The logs : > > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions' > > Am I missing something ?
Could you show your test job definition? CTS plugin will first check whether it's supposed to work on the test job by inspecting the definition. Apparently there is some mismatch between what is expected and what is actually there. You can hide/obfuscate all URLs, they're not that important.
milosz
> > Best regards, > Axel > > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote: >> >> Hi Milosz, >> >> Thank you for your email. It is now working fine. >> I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. >> The main thing that was missing in my test definition was the "lava-test-reference". >> >> Thank you for you help. >> >> Regards, >> Axel >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>> > >>> > Hi all, >>> > >>> > I saw you have a repository with tradefed and ltp plugins for SQUAD. >>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to >>> > attach the error logs to failed test cases. >>> > >>> > My questions are : >>> > How can I setup those plugins to work with my SQUAD instance ? >>> >>> you need to install the package from pip >>> (https://pypi.org/project/squad-linaro-plugins/) to your squad >>> instance. That should do the trick. Each project has to have the >>> plugin enabled manually. >>> >>> > Do I need to use a specific tradefed test definition to interface it with >>> > the plugin ? >>> >>> yes, tradefed plugin assumes that all logs are stored outside of LAVA. >>> Tradefed creates a zip file that is later uploaded to 'artifactorial' >>> (https://archive.validation.linaro.org/). You can use some other >>> service but it needs to support upload request in the following form: >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >>> ATTACHMENT is the zip file that tradefed created and >>> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the >>> content. >>> This step is done automatically when you use tradefed [1] or >>> noninteractive-tradefed [2] from test-definitions >>> >>> When running in squad post processing step plugin downloads the zip >>> file and extracts logs from it. Failed tests are updated with logs. >>> >>> > Does the job submission works the same way ? using the submitjob API ? >>> >>> Test job submission works the same way. Plugins work as a post >>> processing step after LAVA jobs are completed. >>> >>> [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >>> [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >>> >>> milosz >>> >>> > >>> > Thank you in advance, >>> > Axel >>> > _______________________________________________ >>> > Squad-dev mailing list >>> > Squad-dev@lists.linaro.org >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Hi Milosz,
Yes sure, you will find it attached. This definition is the one
stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
I hid the sensible information with "****".
Regards, Axel
On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> > On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > > > Hi all, > > > > I get back to this topic as I have a new issue now. > > Squad worker return a "Plugin postprocessing error" when the
tradefed plugin is triggered.
> > The logs : > > > > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback
(most recent call last):
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
> > > > Am I missing something ? > > Could you show your test job definition? CTS plugin will first
check
> whether it's supposed to work on the test job by inspecting the > definition. Apparently there is some mismatch between what is
expected
> and what is actually there. You can hide/obfuscate all URLs,
they're
> not that important. > > milosz > > > > > Best regards, > > Axel > > > > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> > >> Hi Milosz, > >> > >> Thank you for your email. It is now working fine. > >> I setup my own local HTTP server and updated my tradefed test
definition (based on yours) to match my setup.
> >> The main thing that was missing in my test definition was the
"lava-test-reference".
> >> > >> Thank you for you help. > >> > >> Regards, > >> Axel > >> > >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >>> > >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >>> > > >>> > Hi all, > >>> > > >>> > I saw you have a repository with tradefed and ltp plugins
for SQUAD.
> >>> > If I understand correctly, those plugins retrieve
CTS/VTS/LTP logs to
> >>> > attach the error logs to failed test cases. > >>> > > >>> > My questions are : > >>> > How can I setup those plugins to work with my SQUAD
instance ?
> >>> > >>> you need to install the package from pip > >>> (https://pypi.org/project/squad-linaro-plugins/) to your
squad
> >>> instance. That should do the trick. Each project has to have
the
> >>> plugin enabled manually. > >>> > >>> > Do I need to use a specific tradefed test definition to
interface it with
> >>> > the plugin ? > >>> > >>> yes, tradefed plugin assumes that all logs are stored outside
of LAVA.
> >>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
> >>> (https://archive.validation.linaro.org/). You can use some
other
> >>> service but it needs to support upload request in the
following form:
> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > >>> ATTACHMENT is the zip file that tradefed created and > >>> ARTIFACTORIAL_TOKEN is used to authenticate the user
uploading the
> >>> content. > >>> This step is done automatically when you use tradefed [1] or > >>> noninteractive-tradefed [2] from test-definitions > >>> > >>> When running in squad post processing step plugin downloads
the zip
> >>> file and extracts logs from it. Failed tests are updated with
logs.
> >>> > >>> > Does the job submission works the same way ? using the
submitjob API ?
> >>> > >>> Test job submission works the same way. Plugins work as a post > >>> processing step after LAVA jobs are completed. > >>> > >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
> >>> > >>> milosz > >>> > >>> > > >>> > Thank you in advance, > >>> > Axel > >>> > _______________________________________________ > >>> > Squad-dev mailing list > >>> > Squad-dev@lists.linaro.org > >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
I think it's interactive test section that makes the plugin unhappy. I'll fix that and upload new version.
milosz
On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > Hi Milosz, > > Yes sure, you will find it attached. This definition is the one
stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
> I hid the sensible information with "****". > > Regards, > Axel > > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> > >> > Hi all, >> > >> > I get back to this topic as I have a new issue now. >> > Squad worker return a "Plugin postprocessing error" when the
tradefed plugin is triggered.
>> > The logs : >> > >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback
(most recent call last):
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
>> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
>> > >> > Am I missing something ? >> >> Could you show your test job definition? CTS plugin will first
check
>> whether it's supposed to work on the test job by inspecting the >> definition. Apparently there is some mismatch between what is
expected
>> and what is actually there. You can hide/obfuscate all URLs,
they're
>> not that important. >> >> milosz >> >> > >> > Best regards, >> > Axel >> > >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >> >> >> Hi Milosz, >> >> >> >> Thank you for your email. It is now working fine. >> >> I setup my own local HTTP server and updated my tradefed test
definition (based on yours) to match my setup.
>> >> The main thing that was missing in my test definition was the
"lava-test-reference".
>> >> >> >> Thank you for you help. >> >> >> >> Regards, >> >> Axel >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >>> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >>> > >> >>> > Hi all, >> >>> > >> >>> > I saw you have a repository with tradefed and ltp plugins
for SQUAD.
>> >>> > If I understand correctly, those plugins retrieve
CTS/VTS/LTP logs to
>> >>> > attach the error logs to failed test cases. >> >>> > >> >>> > My questions are : >> >>> > How can I setup those plugins to work with my SQUAD
instance ?
>> >>> >> >>> you need to install the package from pip >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your
squad
>> >>> instance. That should do the trick. Each project has to have
the
>> >>> plugin enabled manually. >> >>> >> >>> > Do I need to use a specific tradefed test definition to
interface it with
>> >>> > the plugin ? >> >>> >> >>> yes, tradefed plugin assumes that all logs are stored
outside of LAVA.
>> >>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
>> >>> (https://archive.validation.linaro.org/). You can use some
other
>> >>> service but it needs to support upload request in the
following form:
>> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> >>> ATTACHMENT is the zip file that tradefed created and >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user
uploading the
>> >>> content. >> >>> This step is done automatically when you use tradefed [1] or >> >>> noninteractive-tradefed [2] from test-definitions >> >>> >> >>> When running in squad post processing step plugin downloads
the zip
>> >>> file and extracts logs from it. Failed tests are updated
with logs.
>> >>> >> >>> > Does the job submission works the same way ? using the
submitjob API ?
>> >>> >> >>> Test job submission works the same way. Plugins work as a
post
>> >>> processing step after LAVA jobs are completed. >> >>> >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
>> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
>> >>> >> >>> milosz >> >>> >> >>> > >> >>> > Thank you in advance, >> >>> > Axel >> >>> > _______________________________________________ >> >>> > Squad-dev mailing list >> >>> > Squad-dev@lists.linaro.org >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Hi Milosz,
Great thank you.
Axel
On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> > I think it's interactive test section that makes the plugin
unhappy.
> I'll fix that and upload new version. > > milosz > > On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > > > Hi Milosz, > > > > Yes sure, you will find it attached. This definition is the one
stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
> > I hid the sensible information with "****". > > > > Regards, > > Axel > > > > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >> > >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> > > >> > Hi all, > >> > > >> > I get back to this topic as I have a new issue now. > >> > Squad worker return a "Plugin postprocessing error" when the
tradefed plugin is triggered.
> >> > The logs : > >> > > >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback
(most recent call last):
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
> >> > > >> > Am I missing something ? > >> > >> Could you show your test job definition? CTS plugin will first
check
> >> whether it's supposed to work on the test job by inspecting the > >> definition. Apparently there is some mismatch between what is
expected
> >> and what is actually there. You can hide/obfuscate all URLs,
they're
> >> not that important. > >> > >> milosz > >> > >> > > >> > Best regards, > >> > Axel > >> > > >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> >> > >> >> Hi Milosz, > >> >> > >> >> Thank you for your email. It is now working fine. > >> >> I setup my own local HTTP server and updated my tradefed
test definition (based on yours) to match my setup.
> >> >> The main thing that was missing in my test definition was
the "lava-test-reference".
> >> >> > >> >> Thank you for you help. > >> >> > >> >> Regards, > >> >> Axel > >> >> > >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >> >>> > >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> >>> > > >> >>> > Hi all, > >> >>> > > >> >>> > I saw you have a repository with tradefed and ltp plugins
for SQUAD.
> >> >>> > If I understand correctly, those plugins retrieve
CTS/VTS/LTP logs to
> >> >>> > attach the error logs to failed test cases. > >> >>> > > >> >>> > My questions are : > >> >>> > How can I setup those plugins to work with my SQUAD
instance ?
> >> >>> > >> >>> you need to install the package from pip > >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your
squad
> >> >>> instance. That should do the trick. Each project has to
have the
> >> >>> plugin enabled manually. > >> >>> > >> >>> > Do I need to use a specific tradefed test definition to
interface it with
> >> >>> > the plugin ? > >> >>> > >> >>> yes, tradefed plugin assumes that all logs are stored
outside of LAVA.
> >> >>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
> >> >>> (https://archive.validation.linaro.org/). You can use some
other
> >> >>> service but it needs to support upload request in the
following form:
> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > >> >>> ATTACHMENT is the zip file that tradefed created and > >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user
uploading the
> >> >>> content. > >> >>> This step is done automatically when you use tradefed [1] or > >> >>> noninteractive-tradefed [2] from test-definitions > >> >>> > >> >>> When running in squad post processing step plugin downloads
the zip
> >> >>> file and extracts logs from it. Failed tests are updated
with logs.
> >> >>> > >> >>> > Does the job submission works the same way ? using the
submitjob API ?
> >> >>> > >> >>> Test job submission works the same way. Plugins work as a
post
> >> >>> processing step after LAVA jobs are completed. > >> >>> > >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
> >> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
> >> >>> > >> >>> milosz > >> >>> > >> >>> > > >> >>> > Thank you in advance, > >> >>> > Axel > >> >>> > _______________________________________________ > >> >>> > Squad-dev mailing list > >> >>> > Squad-dev@lists.linaro.org > >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
https://github.com/Linaro/squadplugins/pull/9 Antonio, could you take a look?
milosz
On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > Hi Milosz, > > Great thank you. > > Axel > > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> I think it's interactive test section that makes the plugin
unhappy.
>> I'll fix that and upload new version. >> >> milosz >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> > >> > Hi Milosz, >> > >> > Yes sure, you will find it attached. This definition is the one
stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
>> > I hid the sensible information with "****". >> > >> > Regards, >> > Axel >> > >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >> > >> >> > Hi all, >> >> > >> >> > I get back to this topic as I have a new issue now. >> >> > Squad worker return a "Plugin postprocessing error" when the
tradefed plugin is triggered.
>> >> > The logs : >> >> > >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06
03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback
(most recent call last):
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
>> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
>> >> > >> >> > Am I missing something ? >> >> >> >> Could you show your test job definition? CTS plugin will first
check
>> >> whether it's supposed to work on the test job by inspecting the >> >> definition. Apparently there is some mismatch between what is
expected
>> >> and what is actually there. You can hide/obfuscate all URLs,
they're
>> >> not that important. >> >> >> >> milosz >> >> >> >> > >> >> > Best regards, >> >> > Axel >> >> > >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >> >> >> >> >> Hi Milosz, >> >> >> >> >> >> Thank you for your email. It is now working fine. >> >> >> I setup my own local HTTP server and updated my tradefed
test definition (based on yours) to match my setup.
>> >> >> The main thing that was missing in my test definition was
the "lava-test-reference".
>> >> >> >> >> >> Thank you for you help. >> >> >> >> >> >> Regards, >> >> >> Axel >> >> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> >>> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >> >>> > >> >> >>> > Hi all, >> >> >>> > >> >> >>> > I saw you have a repository with tradefed and ltp
plugins for SQUAD.
>> >> >>> > If I understand correctly, those plugins retrieve
CTS/VTS/LTP logs to
>> >> >>> > attach the error logs to failed test cases. >> >> >>> > >> >> >>> > My questions are : >> >> >>> > How can I setup those plugins to work with my SQUAD
instance ?
>> >> >>> >> >> >>> you need to install the package from pip >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your
squad
>> >> >>> instance. That should do the trick. Each project has to
have the
>> >> >>> plugin enabled manually. >> >> >>> >> >> >>> > Do I need to use a specific tradefed test definition to
interface it with
>> >> >>> > the plugin ? >> >> >>> >> >> >>> yes, tradefed plugin assumes that all logs are stored
outside of LAVA.
>> >> >>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
>> >> >>> (https://archive.validation.linaro.org/). You can use
some other
>> >> >>> service but it needs to support upload request in the
following form:
>> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> >> >>> ATTACHMENT is the zip file that tradefed created and >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user
uploading the
>> >> >>> content. >> >> >>> This step is done automatically when you use tradefed [1]
or
>> >> >>> noninteractive-tradefed [2] from test-definitions >> >> >>> >> >> >>> When running in squad post processing step plugin
downloads the zip
>> >> >>> file and extracts logs from it. Failed tests are updated
with logs.
>> >> >>> >> >> >>> > Does the job submission works the same way ? using the
submitjob API ?
>> >> >>> >> >> >>> Test job submission works the same way. Plugins work as a
post
>> >> >>> processing step after LAVA jobs are completed. >> >> >>> >> >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
>> >> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
>> >> >>> >> >> >>> milosz >> >> >>> >> >> >>> > >> >> >>> > Thank you in advance, >> >> >>> > Axel >> >> >>> > _______________________________________________ >> >> >>> > Squad-dev mailing list >> >> >>> > Squad-dev@lists.linaro.org >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
Hi Milosz,
I saw your fix has been merged, is it available now with pip ?
Regards, Axel
On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> > https://github.com/Linaro/squadplugins/pull/9 > Antonio, could you take a look? > > milosz > > On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> > > > Hi Milosz, > > > > Great thank you. > > > > Axel > > > > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >> > >> I think it's interactive test section that makes the plugin
unhappy.
> >> I'll fix that and upload new version. > >> > >> milosz > >> > >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> > > >> > Hi Milosz, > >> > > >> > Yes sure, you will find it attached. This definition is the
one stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
> >> > I hid the sensible information with "****". > >> > > >> > Regards, > >> > Axel > >> > > >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >> >> > >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> >> > > >> >> > Hi all, > >> >> > > >> >> > I get back to this topic as I have a new issue now. > >> >> > Squad worker return a "Plugin postprocessing error" when
the tradefed plugin is triggered.
> >> >> > The logs : > >> >> > > >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback
(most recent call last):
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for
test_definition in test_action['test']['definitions']:
> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError:
'definitions'
> >> >> > > >> >> > Am I missing something ? > >> >> > >> >> Could you show your test job definition? CTS plugin will
first check
> >> >> whether it's supposed to work on the test job by inspecting
the
> >> >> definition. Apparently there is some mismatch between what is
expected
> >> >> and what is actually there. You can hide/obfuscate all URLs,
they're
> >> >> not that important. > >> >> > >> >> milosz > >> >> > >> >> > > >> >> > Best regards, > >> >> > Axel > >> >> > > >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> >> >> > >> >> >> Hi Milosz, > >> >> >> > >> >> >> Thank you for your email. It is now working fine. > >> >> >> I setup my own local HTTP server and updated my tradefed
test definition (based on yours) to match my setup.
> >> >> >> The main thing that was missing in my test definition was
the "lava-test-reference".
> >> >> >> > >> >> >> Thank you for you help. > >> >> >> > >> >> >> Regards, > >> >> >> Axel > >> >> >> > >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> >> >> >>> > >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
> >> >> >>> > > >> >> >>> > Hi all, > >> >> >>> > > >> >> >>> > I saw you have a repository with tradefed and ltp
plugins for SQUAD.
> >> >> >>> > If I understand correctly, those plugins retrieve
CTS/VTS/LTP logs to
> >> >> >>> > attach the error logs to failed test cases. > >> >> >>> > > >> >> >>> > My questions are : > >> >> >>> > How can I setup those plugins to work with my SQUAD
instance ?
> >> >> >>> > >> >> >>> you need to install the package from pip > >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your
squad
> >> >> >>> instance. That should do the trick. Each project has to
have the
> >> >> >>> plugin enabled manually. > >> >> >>> > >> >> >>> > Do I need to use a specific tradefed test definition to
interface it with
> >> >> >>> > the plugin ? > >> >> >>> > >> >> >>> yes, tradefed plugin assumes that all logs are stored
outside of LAVA.
> >> >> >>> Tradefed creates a zip file that is later uploaded to
'artifactorial'
> >> >> >>> (https://archive.validation.linaro.org/). You can use
some other
> >> >> >>> service but it needs to support upload request in the
following form:
> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > >> >> >>> ATTACHMENT is the zip file that tradefed created and > >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user
uploading the
> >> >> >>> content. > >> >> >>> This step is done automatically when you use tradefed [1]
or
> >> >> >>> noninteractive-tradefed [2] from test-definitions > >> >> >>> > >> >> >>> When running in squad post processing step plugin
downloads the zip
> >> >> >>> file and extracts logs from it. Failed tests are updated
with logs.
> >> >> >>> > >> >> >>> > Does the job submission works the same way ? using the
submitjob API ?
> >> >> >>> > >> >> >>> Test job submission works the same way. Plugins work as a
post
> >> >> >>> processing step after LAVA jobs are completed. > >> >> >>> > >> >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
> >> >> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
> >> >> >>> > >> >> >>> milosz > >> >> >>> > >> >> >>> > > >> >> >>> > Thank you in advance, > >> >> >>> > Axel > >> >> >>> > _______________________________________________ > >> >> >>> > Squad-dev mailing list > >> >> >>> > Squad-dev@lists.linaro.org > >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
OK, found the culprit. It's the VTS test name: arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: > > Hi Milosz, > > I saw your fix has been merged, is it available now with pip ? > > Regards, > Axel > > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> https://github.com/Linaro/squadplugins/pull/9 >> Antonio, could you take a look? >> >> milosz >> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> > >> > Hi Milosz, >> > >> > Great thank you. >> > >> > Axel >> > >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >> I think it's interactive test section that makes the plugin unhappy. >> >> I'll fix that and upload new version. >> >> >> >> milosz >> >> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> > >> >> > Hi Milosz, >> >> > >> >> > Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >> >> > I hid the sensible information with "****". >> >> > >> >> > Regards, >> >> > Axel >> >> > >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> > >> >> >> > Hi all, >> >> >> > >> >> >> > I get back to this topic as I have a new issue now. >> >> >> > Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. >> >> >> > The logs : >> >> >> > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions' >> >> >> > >> >> >> > Am I missing something ? >> >> >> >> >> >> Could you show your test job definition? CTS plugin will first check >> >> >> whether it's supposed to work on the test job by inspecting the >> >> >> definition. Apparently there is some mismatch between what is expected >> >> >> and what is actually there. You can hide/obfuscate all URLs, they're >> >> >> not that important. >> >> >> >> >> >> milosz >> >> >> >> >> >> > >> >> >> > Best regards, >> >> >> > Axel >> >> >> > >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> >> >> >> >> >> Hi Milosz, >> >> >> >> >> >> >> >> Thank you for your email. It is now working fine. >> >> >> >> I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. >> >> >> >> The main thing that was missing in my test definition was the "lava-test-reference". >> >> >> >> >> >> >> >> Thank you for you help. >> >> >> >> >> >> >> >> Regards, >> >> >> >> Axel >> >> >> >> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> >>> > >> >> >> >>> > Hi all, >> >> >> >>> > >> >> >> >>> > I saw you have a repository with tradefed and ltp plugins for SQUAD. >> >> >> >>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to >> >> >> >>> > attach the error logs to failed test cases. >> >> >> >>> > >> >> >> >>> > My questions are : >> >> >> >>> > How can I setup those plugins to work with my SQUAD instance ? >> >> >> >>> >> >> >> >>> you need to install the package from pip >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your squad >> >> >> >>> instance. That should do the trick. Each project has to have the >> >> >> >>> plugin enabled manually. >> >> >> >>> >> >> >> >>> > Do I need to use a specific tradefed test definition to interface it with >> >> >> >>> > the plugin ? >> >> >> >>> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored outside of LAVA. >> >> >> >>> Tradefed creates a zip file that is later uploaded to 'artifactorial' >> >> >> >>> (https://archive.validation.linaro.org/). You can use some other >> >> >> >>> service but it needs to support upload request in the following form: >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the >> >> >> >>> content. >> >> >> >>> This step is done automatically when you use tradefed [1] or >> >> >> >>> noninteractive-tradefed [2] from test-definitions >> >> >> >>> >> >> >> >>> When running in squad post processing step plugin downloads the zip >> >> >> >>> file and extracts logs from it. Failed tests are updated with logs. >> >> >> >>> >> >> >> >>> > Does the job submission works the same way ? using the submitjob API ? >> >> >> >>> >> >> >> >>> Test job submission works the same way. Plugins work as a post >> >> >> >>> processing step after LAVA jobs are completed. >> >> >> >>> >> >> >> >>> [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >> >> >> >>> [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >> >> >> >>> >> >> >> >>> milosz >> >> >> >>> >> >> >> >>> > >> >> >> >>> > Thank you in advance, >> >> >> >>> > Axel >> >> >> >>> > _______________________________________________ >> >> >> >>> > Squad-dev mailing list >> >> >> >>> > Squad-dev@lists.linaro.org >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
> I forgot to release it. Will do it today. > > milosz > > On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > > > > Hi Milosz, > > > > I saw your fix has been merged, is it available now with pip ? > > > > Regards, > > Axel > > > > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> > >> https://github.com/Linaro/squadplugins/pull/9 > >> Antonio, could you take a look? > >> > >> milosz > >> > >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> > > >> > Hi Milosz, > >> > > >> > Great thank you. > >> > > >> > Axel > >> > > >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> > >> >> I think it's interactive test section that makes the plugin > unhappy. > >> >> I'll fix that and upload new version. > >> >> > >> >> milosz > >> >> > >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> > > >> >> > Hi Milosz, > >> >> > > >> >> > Yes sure, you will find it attached. This definition is the > one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. > >> >> > I hid the sensible information with "****". > >> >> > > >> >> > Regards, > >> >> > Axel > >> >> > > >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> >> > >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> > > >> >> >> > Hi all, > >> >> >> > > >> >> >> > I get back to this topic as I have a new issue now. > >> >> >> > Squad worker return a "Plugin postprocessing error" when > the tradefed plugin is triggered. > >> >> >> > The logs : > >> >> >> > > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: > 1717 > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: > 'definitions' > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > Traceback (most recent call last): > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File > "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in > __postprocess_testjob__ > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > plugin.postprocess_testjob(test_job) > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File > "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in > postprocess_testjob > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for > test_definition in test_action['test']['definitions']: > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: > KeyError: 'definitions' > >> >> >> > > >> >> >> > Am I missing something ? > >> >> >> > >> >> >> Could you show your test job definition? CTS plugin will > first check > >> >> >> whether it's supposed to work on the test job by inspecting > the > >> >> >> definition. Apparently there is some mismatch between what > is expected > >> >> >> and what is actually there. You can hide/obfuscate all > URLs, they're > >> >> >> not that important. > >> >> >> > >> >> >> milosz > >> >> >> > >> >> >> > > >> >> >> > Best regards, > >> >> >> > Axel > >> >> >> > > >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> >> > >> >> >> >> Hi Milosz, > >> >> >> >> > >> >> >> >> Thank you for your email. It is now working fine. > >> >> >> >> I setup my own local HTTP server and updated my tradefed > test definition (based on yours) to match my setup. > >> >> >> >> The main thing that was missing in my test definition > was the "lava-test-reference". > >> >> >> >> > >> >> >> >> Thank you for you help. > >> >> >> >> > >> >> >> >> Regards, > >> >> >> >> Axel > >> >> >> >> > >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> >> >> >>> > >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> >> >> >>> > > >> >> >> >>> > Hi all, > >> >> >> >>> > > >> >> >> >>> > I saw you have a repository with tradefed and ltp > plugins for SQUAD. > >> >> >> >>> > If I understand correctly, those plugins retrieve > CTS/VTS/LTP logs to > >> >> >> >>> > attach the error logs to failed test cases. > >> >> >> >>> > > >> >> >> >>> > My questions are : > >> >> >> >>> > How can I setup those plugins to work with my SQUAD > instance ? > >> >> >> >>> > >> >> >> >>> you need to install the package from pip > >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to > your squad > >> >> >> >>> instance. That should do the trick. Each project has to > have the > >> >> >> >>> plugin enabled manually. > >> >> >> >>> > >> >> >> >>> > Do I need to use a specific tradefed test definition > to interface it with > >> >> >> >>> > the plugin ? > >> >> >> >>> > >> >> >> >>> yes, tradefed plugin assumes that all logs are stored > outside of LAVA. > >> >> >> >>> Tradefed creates a zip file that is later uploaded to > 'artifactorial' > >> >> >> >>> (https://archive.validation.linaro.org/). You can use > some other > >> >> >> >>> service but it needs to support upload request in the > following form: > >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F > >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" > >> >> >> >>> ATTACHMENT is the zip file that tradefed created and > >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user > uploading the > >> >> >> >>> content. > >> >> >> >>> This step is done automatically when you use tradefed > [1] or > >> >> >> >>> noninteractive-tradefed [2] from test-definitions > >> >> >> >>> > >> >> >> >>> When running in squad post processing step plugin > downloads the zip > >> >> >> >>> file and extracts logs from it. Failed tests are > updated with logs. > >> >> >> >>> > >> >> >> >>> > Does the job submission works the same way ? using > the submitjob API ? > >> >> >> >>> > >> >> >> >>> Test job submission works the same way. Plugins work as > a post > >> >> >> >>> processing step after LAVA jobs are completed. > >> >> >> >>> > >> >> >> >>> [1] > https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... > >> >> >> >>> [2] > https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... > >> >> >> >>> > >> >> >> >>> milosz > >> >> >> >>> > >> >> >> >>> > > >> >> >> >>> > Thank you in advance, > >> >> >> >>> > Axel > >> >> >> >>> > _______________________________________________ > >> >> >> >>> > Squad-dev mailing list > >> >> >> >>> > Squad-dev@lists.linaro.org > >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev >
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
> Great, thanks! > > On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < > milosz.wasilewski@linaro.org> wrote: > >> I forgot to release it. Will do it today. >> >> milosz >> >> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> > >> > Hi Milosz, >> > >> > I saw your fix has been merged, is it available now with pip ? >> > >> > Regards, >> > Axel >> > >> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < >> milosz.wasilewski@linaro.org> wrote: >> >> >> >> https://github.com/Linaro/squadplugins/pull/9 >> >> Antonio, could you take a look? >> >> >> >> milosz >> >> >> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >> > >> >> > Hi Milosz, >> >> > >> >> > Great thank you. >> >> > >> >> > Axel >> >> > >> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < >> milosz.wasilewski@linaro.org> wrote: >> >> >> >> >> >> I think it's interactive test section that makes the plugin >> unhappy. >> >> >> I'll fix that and upload new version. >> >> >> >> >> >> milosz >> >> >> >> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >> >> > >> >> >> > Hi Milosz, >> >> >> > >> >> >> > Yes sure, you will find it attached. This definition is the >> one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >> >> >> > I hid the sensible information with "****". >> >> >> > >> >> >> > Regards, >> >> >> > Axel >> >> >> > >> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < >> milosz.wasilewski@linaro.org> wrote: >> >> >> >> >> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >> >> >> > >> >> >> >> > Hi all, >> >> >> >> > >> >> >> >> > I get back to this topic as I have a new issue now. >> >> >> >> > Squad worker return a "Plugin postprocessing error" when >> the tradefed plugin is triggered. >> >> >> >> > The logs : >> >> >> >> > >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >> [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: >> 1717 >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >> [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: >> 'definitions' >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >> Traceback (most recent call last): >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File >> "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in >> __postprocess_testjob__ >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >> plugin.postprocess_testjob(test_job) >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File >> "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in >> postprocess_testjob >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for >> test_definition in test_action['test']['definitions']: >> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >> KeyError: 'definitions' >> >> >> >> > >> >> >> >> > Am I missing something ? >> >> >> >> >> >> >> >> Could you show your test job definition? CTS plugin will >> first check >> >> >> >> whether it's supposed to work on the test job by >> inspecting the >> >> >> >> definition. Apparently there is some mismatch between what >> is expected >> >> >> >> and what is actually there. You can hide/obfuscate all >> URLs, they're >> >> >> >> not that important. >> >> >> >> >> >> >> >> milosz >> >> >> >> >> >> >> >> > >> >> >> >> > Best regards, >> >> >> >> > Axel >> >> >> >> > >> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >> >> >> >> >> >> >> >> >> Hi Milosz, >> >> >> >> >> >> >> >> >> >> Thank you for your email. It is now working fine. >> >> >> >> >> I setup my own local HTTP server and updated my >> tradefed test definition (based on yours) to match my setup. >> >> >> >> >> The main thing that was missing in my test definition >> was the "lava-test-reference". >> >> >> >> >> >> >> >> >> >> Thank you for you help. >> >> >> >> >> >> >> >> >> >> Regards, >> >> >> >> >> Axel >> >> >> >> >> >> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < >> milosz.wasilewski@linaro.org> wrote: >> >> >> >> >>> >> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >> >> >> >>> > >> >> >> >> >>> > Hi all, >> >> >> >> >>> > >> >> >> >> >>> > I saw you have a repository with tradefed and ltp >> plugins for SQUAD. >> >> >> >> >>> > If I understand correctly, those plugins retrieve >> CTS/VTS/LTP logs to >> >> >> >> >>> > attach the error logs to failed test cases. >> >> >> >> >>> > >> >> >> >> >>> > My questions are : >> >> >> >> >>> > How can I setup those plugins to work with my SQUAD >> instance ? >> >> >> >> >>> >> >> >> >> >>> you need to install the package from pip >> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to >> your squad >> >> >> >> >>> instance. That should do the trick. Each project has >> to have the >> >> >> >> >>> plugin enabled manually. >> >> >> >> >>> >> >> >> >> >>> > Do I need to use a specific tradefed test definition >> to interface it with >> >> >> >> >>> > the plugin ? >> >> >> >> >>> >> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored >> outside of LAVA. >> >> >> >> >>> Tradefed creates a zip file that is later uploaded to >> 'artifactorial' >> >> >> >> >>> (https://archive.validation.linaro.org/). You can use >> some other >> >> >> >> >>> service but it needs to support upload request in the >> following form: >> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user >> uploading the >> >> >> >> >>> content. >> >> >> >> >>> This step is done automatically when you use tradefed >> [1] or >> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >> >> >> >> >>> >> >> >> >> >>> When running in squad post processing step plugin >> downloads the zip >> >> >> >> >>> file and extracts logs from it. Failed tests are >> updated with logs. >> >> >> >> >>> >> >> >> >> >>> > Does the job submission works the same way ? using >> the submitjob API ? >> >> >> >> >>> >> >> >> >> >>> Test job submission works the same way. Plugins work >> as a post >> >> >> >> >>> processing step after LAVA jobs are completed. >> >> >> >> >>> >> >> >> >> >>> [1] >> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >> >> >> >> >>> [2] >> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >> >> >> >> >>> >> >> >> >> >>> milosz >> >> >> >> >>> >> >> >> >> >>> > >> >> >> >> >>> > Thank you in advance, >> >> >> >> >>> > Axel >> >> >> >> >>> > _______________________________________________ >> >> >> >> >>> > Squad-dev mailing list >> >> >> >> >>> > Squad-dev@lists.linaro.org >> >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev >> >
Yes, I will submit patch to test definitions. My previous version of plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
> Hi Milosz, > > Today I faced a new issue, it may be a bug. I ran a full VTS run, > the plugin returned the following error : > > Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 > +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 > +0000] [ERROR] Plugin postprocessing error: too many values to unpack > (expected 2) > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most > recent call last): > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File > "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in > __postprocess_testjob__ > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: > plugin.postprocess_testjob(test_job) > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File > "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in > postprocess_testjob > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: > self._assign_test_log(results.test_results.contents, failed) > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File > "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in > _assign_test_log > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, > test_suite_name = test_suite_name.split(".") > Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many > values to unpack (expected 2) > > Do you have any idea ? > > Regards, > Axel > > On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> Great, thanks! >> >> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < >> milosz.wasilewski@linaro.org> wrote: >> >>> I forgot to release it. Will do it today. >>> >>> milosz >>> >>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> > >>> > Hi Milosz, >>> > >>> > I saw your fix has been merged, is it available now with pip ? >>> > >>> > Regards, >>> > Axel >>> > >>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < >>> milosz.wasilewski@linaro.org> wrote: >>> >> >>> >> https://github.com/Linaro/squadplugins/pull/9 >>> >> Antonio, could you take a look? >>> >> >>> >> milosz >>> >> >>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> >> > >>> >> > Hi Milosz, >>> >> > >>> >> > Great thank you. >>> >> > >>> >> > Axel >>> >> > >>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < >>> milosz.wasilewski@linaro.org> wrote: >>> >> >> >>> >> >> I think it's interactive test section that makes the plugin >>> unhappy. >>> >> >> I'll fix that and upload new version. >>> >> >> >>> >> >> milosz >>> >> >> >>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> >> >> > >>> >> >> > Hi Milosz, >>> >> >> > >>> >> >> > Yes sure, you will find it attached. This definition is >>> the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >>> >> >> > I hid the sensible information with "****". >>> >> >> > >>> >> >> > Regards, >>> >> >> > Axel >>> >> >> > >>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < >>> milosz.wasilewski@linaro.org> wrote: >>> >> >> >> >>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> >> >> >> > >>> >> >> >> > Hi all, >>> >> >> >> > >>> >> >> >> > I get back to this topic as I have a new issue now. >>> >> >> >> > Squad worker return a "Plugin postprocessing error" >>> when the tradefed plugin is triggered. >>> >> >> >> > The logs : >>> >> >> >> > >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: >>> 1717 >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: >>> 'definitions' >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> Traceback (most recent call last): >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File >>> "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in >>> __postprocess_testjob__ >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> plugin.postprocess_testjob(test_job) >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File >>> "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in >>> postprocess_testjob >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> for test_definition in test_action['test']['definitions']: >>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>> KeyError: 'definitions' >>> >> >> >> > >>> >> >> >> > Am I missing something ? >>> >> >> >> >>> >> >> >> Could you show your test job definition? CTS plugin will >>> first check >>> >> >> >> whether it's supposed to work on the test job by >>> inspecting the >>> >> >> >> definition. Apparently there is some mismatch between >>> what is expected >>> >> >> >> and what is actually there. You can hide/obfuscate all >>> URLs, they're >>> >> >> >> not that important. >>> >> >> >> >>> >> >> >> milosz >>> >> >> >> >>> >> >> >> > >>> >> >> >> > Best regards, >>> >> >> >> > Axel >>> >> >> >> > >>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> >> >> >> >> >>> >> >> >> >> Hi Milosz, >>> >> >> >> >> >>> >> >> >> >> Thank you for your email. It is now working fine. >>> >> >> >> >> I setup my own local HTTP server and updated my >>> tradefed test definition (based on yours) to match my setup. >>> >> >> >> >> The main thing that was missing in my test definition >>> was the "lava-test-reference". >>> >> >> >> >> >>> >> >> >> >> Thank you for you help. >>> >> >> >> >> >>> >> >> >> >> Regards, >>> >> >> >> >> Axel >>> >> >> >> >> >>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < >>> milosz.wasilewski@linaro.org> wrote: >>> >> >> >> >>> >>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < >>> axel.lebourhis@linaro.org> wrote: >>> >> >> >> >>> > >>> >> >> >> >>> > Hi all, >>> >> >> >> >>> > >>> >> >> >> >>> > I saw you have a repository with tradefed and ltp >>> plugins for SQUAD. >>> >> >> >> >>> > If I understand correctly, those plugins retrieve >>> CTS/VTS/LTP logs to >>> >> >> >> >>> > attach the error logs to failed test cases. >>> >> >> >> >>> > >>> >> >> >> >>> > My questions are : >>> >> >> >> >>> > How can I setup those plugins to work with my SQUAD >>> instance ? >>> >> >> >> >>> >>> >> >> >> >>> you need to install the package from pip >>> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to >>> your squad >>> >> >> >> >>> instance. That should do the trick. Each project has >>> to have the >>> >> >> >> >>> plugin enabled manually. >>> >> >> >> >>> >>> >> >> >> >>> > Do I need to use a specific tradefed test >>> definition to interface it with >>> >> >> >> >>> > the plugin ? >>> >> >> >> >>> >>> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored >>> outside of LAVA. >>> >> >> >> >>> Tradefed creates a zip file that is later uploaded to >>> 'artifactorial' >>> >> >> >> >>> (https://archive.validation.linaro.org/). You can >>> use some other >>> >> >> >> >>> service but it needs to support upload request in the >>> following form: >>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" >>> -F >>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >>> >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user >>> uploading the >>> >> >> >> >>> content. >>> >> >> >> >>> This step is done automatically when you use tradefed >>> [1] or >>> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >>> >> >> >> >>> >>> >> >> >> >>> When running in squad post processing step plugin >>> downloads the zip >>> >> >> >> >>> file and extracts logs from it. Failed tests are >>> updated with logs. >>> >> >> >> >>> >>> >> >> >> >>> > Does the job submission works the same way ? using >>> the submitjob API ? >>> >> >> >> >>> >>> >> >> >> >>> Test job submission works the same way. Plugins work >>> as a post >>> >> >> >> >>> processing step after LAVA jobs are completed. >>> >> >> >> >>> >>> >> >> >> >>> [1] >>> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >>> >> >> >> >>> [2] >>> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >>> >> >> >> >>> >>> >> >> >> >>> milosz >>> >> >> >> >>> >>> >> >> >> >>> > >>> >> >> >> >>> > Thank you in advance, >>> >> >> >> >>> > Axel >>> >> >> >> >>> > _______________________________________________ >>> >> >> >> >>> > Squad-dev mailing list >>> >> >> >> >>> > Squad-dev@lists.linaro.org >>> >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev >>> >>
Hi Milosz,
Will you make a new release of the plugin package today with your recent patch ?
Thanks, Axel
On Thu, 13 Jun 2019 at 15:32, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Yes, I will submit patch to test definitions. My previous version of plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis < axel.lebourhis@linaro.org> wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
> Could you share your testjob YAML and test-results.xml from VTS (may > be privately). I think I know what's going on. Thanks for reporting this. > > milosz > > On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis < > axel.lebourhis@linaro.org> wrote: > >> Hi Milosz, >> >> Today I faced a new issue, it may be a bug. I ran a full VTS run, >> the plugin returned the following error : >> >> Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 >> 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 >> 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to >> unpack (expected 2) >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most >> recent call last): >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File >> "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in >> __postprocess_testjob__ >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: >> plugin.postprocess_testjob(test_job) >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File >> "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in >> postprocess_testjob >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: >> self._assign_test_log(results.test_results.contents, failed) >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File >> "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in >> _assign_test_log >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, >> test_suite_name = test_suite_name.split(".") >> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too >> many values to unpack (expected 2) >> >> Do you have any idea ? >> >> Regards, >> Axel >> >> On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis < >> axel.lebourhis@linaro.org> wrote: >> >>> Great, thanks! >>> >>> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < >>> milosz.wasilewski@linaro.org> wrote: >>> >>>> I forgot to release it. Will do it today. >>>> >>>> milosz >>>> >>>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> > >>>> > Hi Milosz, >>>> > >>>> > I saw your fix has been merged, is it available now with pip ? >>>> > >>>> > Regards, >>>> > Axel >>>> > >>>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < >>>> milosz.wasilewski@linaro.org> wrote: >>>> >> >>>> >> https://github.com/Linaro/squadplugins/pull/9 >>>> >> Antonio, could you take a look? >>>> >> >>>> >> milosz >>>> >> >>>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> >> > >>>> >> > Hi Milosz, >>>> >> > >>>> >> > Great thank you. >>>> >> > >>>> >> > Axel >>>> >> > >>>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < >>>> milosz.wasilewski@linaro.org> wrote: >>>> >> >> >>>> >> >> I think it's interactive test section that makes the plugin >>>> unhappy. >>>> >> >> I'll fix that and upload new version. >>>> >> >> >>>> >> >> milosz >>>> >> >> >>>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> >> >> > >>>> >> >> > Hi Milosz, >>>> >> >> > >>>> >> >> > Yes sure, you will find it attached. This definition is >>>> the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >>>> >> >> > I hid the sensible information with "****". >>>> >> >> > >>>> >> >> > Regards, >>>> >> >> > Axel >>>> >> >> > >>>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < >>>> milosz.wasilewski@linaro.org> wrote: >>>> >> >> >> >>>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> >> >> >> > >>>> >> >> >> > Hi all, >>>> >> >> >> > >>>> >> >> >> > I get back to this topic as I have a new issue now. >>>> >> >> >> > Squad worker return a "Plugin postprocessing error" >>>> when the tradefed plugin is triggered. >>>> >> >> >> > The logs : >>>> >> >> >> > >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: >>>> 1717 >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: >>>> 'definitions' >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> Traceback (most recent call last): >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line >>>> 132, in __postprocess_testjob__ >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> plugin.postprocess_testjob(test_job) >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line >>>> 185, in postprocess_testjob >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> for test_definition in test_action['test']['definitions']: >>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: >>>> KeyError: 'definitions' >>>> >> >> >> > >>>> >> >> >> > Am I missing something ? >>>> >> >> >> >>>> >> >> >> Could you show your test job definition? CTS plugin will >>>> first check >>>> >> >> >> whether it's supposed to work on the test job by >>>> inspecting the >>>> >> >> >> definition. Apparently there is some mismatch between >>>> what is expected >>>> >> >> >> and what is actually there. You can hide/obfuscate all >>>> URLs, they're >>>> >> >> >> not that important. >>>> >> >> >> >>>> >> >> >> milosz >>>> >> >> >> >>>> >> >> >> > >>>> >> >> >> > Best regards, >>>> >> >> >> > Axel >>>> >> >> >> > >>>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> >> >> >> >> >>>> >> >> >> >> Hi Milosz, >>>> >> >> >> >> >>>> >> >> >> >> Thank you for your email. It is now working fine. >>>> >> >> >> >> I setup my own local HTTP server and updated my >>>> tradefed test definition (based on yours) to match my setup. >>>> >> >> >> >> The main thing that was missing in my test definition >>>> was the "lava-test-reference". >>>> >> >> >> >> >>>> >> >> >> >> Thank you for you help. >>>> >> >> >> >> >>>> >> >> >> >> Regards, >>>> >> >> >> >> Axel >>>> >> >> >> >> >>>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < >>>> milosz.wasilewski@linaro.org> wrote: >>>> >> >> >> >>> >>>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < >>>> axel.lebourhis@linaro.org> wrote: >>>> >> >> >> >>> > >>>> >> >> >> >>> > Hi all, >>>> >> >> >> >>> > >>>> >> >> >> >>> > I saw you have a repository with tradefed and ltp >>>> plugins for SQUAD. >>>> >> >> >> >>> > If I understand correctly, those plugins retrieve >>>> CTS/VTS/LTP logs to >>>> >> >> >> >>> > attach the error logs to failed test cases. >>>> >> >> >> >>> > >>>> >> >> >> >>> > My questions are : >>>> >> >> >> >>> > How can I setup those plugins to work with my >>>> SQUAD instance ? >>>> >> >> >> >>> >>>> >> >> >> >>> you need to install the package from pip >>>> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to >>>> your squad >>>> >> >> >> >>> instance. That should do the trick. Each project has >>>> to have the >>>> >> >> >> >>> plugin enabled manually. >>>> >> >> >> >>> >>>> >> >> >> >>> > Do I need to use a specific tradefed test >>>> definition to interface it with >>>> >> >> >> >>> > the plugin ? >>>> >> >> >> >>> >>>> >> >> >> >>> yes, tradefed plugin assumes that all logs are >>>> stored outside of LAVA. >>>> >> >> >> >>> Tradefed creates a zip file that is later uploaded >>>> to 'artifactorial' >>>> >> >> >> >>> (https://archive.validation.linaro.org/). You can >>>> use some other >>>> >> >> >> >>> service but it needs to support upload request in >>>> the following form: >>>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" >>>> -F >>>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >>>> >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >>>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user >>>> uploading the >>>> >> >> >> >>> content. >>>> >> >> >> >>> This step is done automatically when you use >>>> tradefed [1] or >>>> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >>>> >> >> >> >>> >>>> >> >> >> >>> When running in squad post processing step plugin >>>> downloads the zip >>>> >> >> >> >>> file and extracts logs from it. Failed tests are >>>> updated with logs. >>>> >> >> >> >>> >>>> >> >> >> >>> > Does the job submission works the same way ? using >>>> the submitjob API ? >>>> >> >> >> >>> >>>> >> >> >> >>> Test job submission works the same way. Plugins work >>>> as a post >>>> >> >> >> >>> processing step after LAVA jobs are completed. >>>> >> >> >> >>> >>>> >> >> >> >>> [1] >>>> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >>>> >> >> >> >>> [2] >>>> https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >>>> >> >> >> >>> >>>> >> >> >> >>> milosz >>>> >> >> >> >>> >>>> >> >> >> >>> > >>>> >> >> >> >>> > Thank you in advance, >>>> >> >> >> >>> > Axel >>>> >> >> >> >>> > _______________________________________________ >>>> >> >> >> >>> > Squad-dev mailing list >>>> >> >> >> >>> > Squad-dev@lists.linaro.org >>>> >> >> >> >>> > >>>> https://lists.linaro.org/mailman/listinfo/squad-dev >>>> >>>
Axel,
I'll try it out first with some test jobs I have. I finished late yesterday and didn't have time to do that. I only ran unit tests that don't cover all possible cases.
milosz
On Fri, 14 Jun 2019 at 09:17, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Will you make a new release of the plugin package today with your recent patch ?
Thanks, Axel
On Thu, 13 Jun 2019 at 15:32, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
Yes, I will submit patch to test definitions. My previous version of plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
OK, found the culprit. It's the VTS test name: arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote: > > Milosz, > > I privately shared with you the archive containing what you asked for. > > On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >> >> Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this. >> >> milosz >> >> On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>> >>> Hi Milosz, >>> >>> Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error : >>> >>> Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") >>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2) >>> >>> Do you have any idea ? >>> >>> Regards, >>> Axel >>> >>> On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>> >>>> Great, thanks! >>>> >>>> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>> >>>>> I forgot to release it. Will do it today. >>>>> >>>>> milosz >>>>> >>>>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> > >>>>> > Hi Milosz, >>>>> > >>>>> > I saw your fix has been merged, is it available now with pip ? >>>>> > >>>>> > Regards, >>>>> > Axel >>>>> > >>>>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>> >> >>>>> >> https://github.com/Linaro/squadplugins/pull/9 >>>>> >> Antonio, could you take a look? >>>>> >> >>>>> >> milosz >>>>> >> >>>>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >> > >>>>> >> > Hi Milosz, >>>>> >> > >>>>> >> > Great thank you. >>>>> >> > >>>>> >> > Axel >>>>> >> > >>>>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>> >> >> >>>>> >> >> I think it's interactive test section that makes the plugin unhappy. >>>>> >> >> I'll fix that and upload new version. >>>>> >> >> >>>>> >> >> milosz >>>>> >> >> >>>>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >> >> > >>>>> >> >> > Hi Milosz, >>>>> >> >> > >>>>> >> >> > Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >>>>> >> >> > I hid the sensible information with "****". >>>>> >> >> > >>>>> >> >> > Regards, >>>>> >> >> > Axel >>>>> >> >> > >>>>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>> >> >> >> >>>>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >> >> >> > >>>>> >> >> >> > Hi all, >>>>> >> >> >> > >>>>> >> >> >> > I get back to this topic as I have a new issue now. >>>>> >> >> >> > Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. >>>>> >> >> >> > The logs : >>>>> >> >> >> > >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: >>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions' >>>>> >> >> >> > >>>>> >> >> >> > Am I missing something ? >>>>> >> >> >> >>>>> >> >> >> Could you show your test job definition? CTS plugin will first check >>>>> >> >> >> whether it's supposed to work on the test job by inspecting the >>>>> >> >> >> definition. Apparently there is some mismatch between what is expected >>>>> >> >> >> and what is actually there. You can hide/obfuscate all URLs, they're >>>>> >> >> >> not that important. >>>>> >> >> >> >>>>> >> >> >> milosz >>>>> >> >> >> >>>>> >> >> >> > >>>>> >> >> >> > Best regards, >>>>> >> >> >> > Axel >>>>> >> >> >> > >>>>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >> >> >> >> >>>>> >> >> >> >> Hi Milosz, >>>>> >> >> >> >> >>>>> >> >> >> >> Thank you for your email. It is now working fine. >>>>> >> >> >> >> I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. >>>>> >> >> >> >> The main thing that was missing in my test definition was the "lava-test-reference". >>>>> >> >> >> >> >>>>> >> >> >> >> Thank you for you help. >>>>> >> >> >> >> >>>>> >> >> >> >> Regards, >>>>> >> >> >> >> Axel >>>>> >> >> >> >> >>>>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>> >> >> >> >>> >>>>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >> >> >> >>> > >>>>> >> >> >> >>> > Hi all, >>>>> >> >> >> >>> > >>>>> >> >> >> >>> > I saw you have a repository with tradefed and ltp plugins for SQUAD. >>>>> >> >> >> >>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to >>>>> >> >> >> >>> > attach the error logs to failed test cases. >>>>> >> >> >> >>> > >>>>> >> >> >> >>> > My questions are : >>>>> >> >> >> >>> > How can I setup those plugins to work with my SQUAD instance ? >>>>> >> >> >> >>> >>>>> >> >> >> >>> you need to install the package from pip >>>>> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your squad >>>>> >> >> >> >>> instance. That should do the trick. Each project has to have the >>>>> >> >> >> >>> plugin enabled manually. >>>>> >> >> >> >>> >>>>> >> >> >> >>> > Do I need to use a specific tradefed test definition to interface it with >>>>> >> >> >> >>> > the plugin ? >>>>> >> >> >> >>> >>>>> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored outside of LAVA. >>>>> >> >> >> >>> Tradefed creates a zip file that is later uploaded to 'artifactorial' >>>>> >> >> >> >>> (https://archive.validation.linaro.org/). You can use some other >>>>> >> >> >> >>> service but it needs to support upload request in the following form: >>>>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >>>>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >>>>> >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >>>>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the >>>>> >> >> >> >>> content. >>>>> >> >> >> >>> This step is done automatically when you use tradefed [1] or >>>>> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >>>>> >> >> >> >>> >>>>> >> >> >> >>> When running in squad post processing step plugin downloads the zip >>>>> >> >> >> >>> file and extracts logs from it. Failed tests are updated with logs. >>>>> >> >> >> >>> >>>>> >> >> >> >>> > Does the job submission works the same way ? using the submitjob API ? >>>>> >> >> >> >>> >>>>> >> >> >> >>> Test job submission works the same way. Plugins work as a post >>>>> >> >> >> >>> processing step after LAVA jobs are completed. >>>>> >> >> >> >>> >>>>> >> >> >> >>> [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >>>>> >> >> >> >>> [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >>>>> >> >> >> >>> >>>>> >> >> >> >>> milosz >>>>> >> >> >> >>> >>>>> >> >> >> >>> > >>>>> >> >> >> >>> > Thank you in advance, >>>>> >> >> >> >>> > Axel >>>>> >> >> >> >>> > _______________________________________________ >>>>> >> >> >> >>> > Squad-dev mailing list >>>>> >> >> >> >>> > Squad-dev@lists.linaro.org >>>>> >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Ok sure no problem !
Axel
On Fri, 14 Jun 2019 at 10:21, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Axel,
I'll try it out first with some test jobs I have. I finished late yesterday and didn't have time to do that. I only ran unit tests that don't cover all possible cases.
milosz
On Fri, 14 Jun 2019 at 09:17, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Will you make a new release of the plugin package today with your recent
patch ?
Thanks, Axel
On Thu, 13 Jun 2019 at 15:32, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
Yes, I will submit patch to test definitions. My previous version of
plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I
wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
OK, found the culprit. It's the VTS test name:
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit
VTS decided to use "/" at the end of the test name which SQUAD isn't
very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always
recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> > Thank Axel, debugging now. I don't see anything that stands out in
any obvious way.
> > milosz > > On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>> >> Milosz, >> >> I privately shared with you the archive containing what you asked
for.
>> >> On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>> >>> Could you share your testjob YAML and test-results.xml from VTS
(may be privately). I think I know what's going on. Thanks for reporting this.
>>> >>> milosz >>> >>> On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>> >>>> Hi Milosz, >>>> >>>> Today I faced a new issue, it may be a bug. I ran a full VTS
run, the plugin returned the following error :
>>>> >>>> Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12
20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12
20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2)
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most
recent call last):
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
plugin.postprocess_testjob(test_job)
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
self._assign_test_log(results.test_results.contents, failed)
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
test_suite_abi, test_suite_name = test_suite_name.split(".")
>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too
many values to unpack (expected 2)
>>>> >>>> Do you have any idea ? >>>> >>>> Regards, >>>> Axel >>>> >>>> On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>> >>>>> Great, thanks! >>>>> >>>>> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>> >>>>>> I forgot to release it. Will do it today. >>>>>> >>>>>> milosz >>>>>> >>>>>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> > >>>>>> > Hi Milosz, >>>>>> > >>>>>> > I saw your fix has been merged, is it available now with pip
?
>>>>>> > >>>>>> > Regards, >>>>>> > Axel >>>>>> > >>>>>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>> >> >>>>>> >> https://github.com/Linaro/squadplugins/pull/9 >>>>>> >> Antonio, could you take a look? >>>>>> >> >>>>>> >> milosz >>>>>> >> >>>>>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >> > >>>>>> >> > Hi Milosz, >>>>>> >> > >>>>>> >> > Great thank you. >>>>>> >> > >>>>>> >> > Axel >>>>>> >> > >>>>>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>> >> >> >>>>>> >> >> I think it's interactive test section that makes the
plugin unhappy.
>>>>>> >> >> I'll fix that and upload new version. >>>>>> >> >> >>>>>> >> >> milosz >>>>>> >> >> >>>>>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >> >> > >>>>>> >> >> > Hi Milosz, >>>>>> >> >> > >>>>>> >> >> > Yes sure, you will find it attached. This definition
is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
>>>>>> >> >> > I hid the sensible information with "****". >>>>>> >> >> > >>>>>> >> >> > Regards, >>>>>> >> >> > Axel >>>>>> >> >> > >>>>>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>> >> >> >> >>>>>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >> >> >> > >>>>>> >> >> >> > Hi all, >>>>>> >> >> >> > >>>>>> >> >> >> > I get back to this topic as I have a new issue now. >>>>>> >> >> >> > Squad worker return a "Plugin postprocessing error"
when the tradefed plugin is triggered.
>>>>>> >> >> >> > The logs : >>>>>> >> >> >> > >>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
Traceback (most recent call last):
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
plugin.postprocess_testjob(test_job)
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
for test_definition in test_action['test']['definitions']:
>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
KeyError: 'definitions'
>>>>>> >> >> >> > >>>>>> >> >> >> > Am I missing something ? >>>>>> >> >> >> >>>>>> >> >> >> Could you show your test job definition? CTS plugin
will first check
>>>>>> >> >> >> whether it's supposed to work on the test job by
inspecting the
>>>>>> >> >> >> definition. Apparently there is some mismatch between
what is expected
>>>>>> >> >> >> and what is actually there. You can hide/obfuscate
all URLs, they're
>>>>>> >> >> >> not that important. >>>>>> >> >> >> >>>>>> >> >> >> milosz >>>>>> >> >> >> >>>>>> >> >> >> > >>>>>> >> >> >> > Best regards, >>>>>> >> >> >> > Axel >>>>>> >> >> >> > >>>>>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >> >> >> >> >>>>>> >> >> >> >> Hi Milosz, >>>>>> >> >> >> >> >>>>>> >> >> >> >> Thank you for your email. It is now working fine. >>>>>> >> >> >> >> I setup my own local HTTP server and updated my
tradefed test definition (based on yours) to match my setup.
>>>>>> >> >> >> >> The main thing that was missing in my test
definition was the "lava-test-reference".
>>>>>> >> >> >> >> >>>>>> >> >> >> >> Thank you for you help. >>>>>> >> >> >> >> >>>>>> >> >> >> >> Regards, >>>>>> >> >> >> >> Axel >>>>>> >> >> >> >> >>>>>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>> >> >> >> >>> >>>>>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >> >> >> >>> > >>>>>> >> >> >> >>> > Hi all, >>>>>> >> >> >> >>> > >>>>>> >> >> >> >>> > I saw you have a repository with tradefed and
ltp plugins for SQUAD.
>>>>>> >> >> >> >>> > If I understand correctly, those plugins
retrieve CTS/VTS/LTP logs to
>>>>>> >> >> >> >>> > attach the error logs to failed test cases. >>>>>> >> >> >> >>> > >>>>>> >> >> >> >>> > My questions are : >>>>>> >> >> >> >>> > How can I setup those plugins to work with my
SQUAD instance ?
>>>>>> >> >> >> >>> >>>>>> >> >> >> >>> you need to install the package from pip >>>>>> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/)
to your squad
>>>>>> >> >> >> >>> instance. That should do the trick. Each project
has to have the
>>>>>> >> >> >> >>> plugin enabled manually. >>>>>> >> >> >> >>> >>>>>> >> >> >> >>> > Do I need to use a specific tradefed test
definition to interface it with
>>>>>> >> >> >> >>> > the plugin ? >>>>>> >> >> >> >>> >>>>>> >> >> >> >>> yes, tradefed plugin assumes that all logs are
stored outside of LAVA.
>>>>>> >> >> >> >>> Tradefed creates a zip file that is later
uploaded to 'artifactorial'
>>>>>> >> >> >> >>> (https://archive.validation.linaro.org/). You
can use some other
>>>>>> >> >> >> >>> service but it needs to support upload request in
the following form:
>>>>>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F
"path=@${ATTACHMENT}" -F
>>>>>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}"
"${ARTIFACTORIAL_URL}"
>>>>>> >> >> >> >>> ATTACHMENT is the zip file that tradefed created
and
>>>>>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the
user uploading the
>>>>>> >> >> >> >>> content. >>>>>> >> >> >> >>> This step is done automatically when you use
tradefed [1] or
>>>>>> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >>>>>> >> >> >> >>> >>>>>> >> >> >> >>> When running in squad post processing step plugin
downloads the zip
>>>>>> >> >> >> >>> file and extracts logs from it. Failed tests are
updated with logs.
>>>>>> >> >> >> >>> >>>>>> >> >> >> >>> > Does the job submission works the same way ?
using the submitjob API ?
>>>>>> >> >> >> >>> >>>>>> >> >> >> >>> Test job submission works the same way. Plugins
work as a post
>>>>>> >> >> >> >>> processing step after LAVA jobs are completed. >>>>>> >> >> >> >>> >>>>>> >> >> >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
>>>>>> >> >> >> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
>>>>>> >> >> >> >>> >>>>>> >> >> >> >>> milosz >>>>>> >> >> >> >>> >>>>>> >> >> >> >>> > >>>>>> >> >> >> >>> > Thank you in advance, >>>>>> >> >> >> >>> > Axel >>>>>> >> >> >> >>> > _______________________________________________ >>>>>> >> >> >> >>> > Squad-dev mailing list >>>>>> >> >> >> >>> > Squad-dev@lists.linaro.org >>>>>> >> >> >> >>> >
1.8 now released and uploaded. I checked a few CTS and VTS jobs locally and all worked well. Please try with your VTS job as you run more tests than I have available. Should work (fingers crossed)!
milosz
On Fri, 14 Jun 2019 at 09:41, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Ok sure no problem !
Axel
On Fri, 14 Jun 2019 at 10:21, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
Axel,
I'll try it out first with some test jobs I have. I finished late yesterday and didn't have time to do that. I only ran unit tests that don't cover all possible cases.
milosz
On Fri, 14 Jun 2019 at 09:17, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Will you make a new release of the plugin package today with your recent patch ?
Thanks, Axel
On Thu, 13 Jun 2019 at 15:32, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
Yes, I will submit patch to test definitions. My previous version of plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
I wrote this patch: https://github.com/Linaro/squadplugins/pull/10
In my local testing it fixes the problem and a few other issues I wansn't aware before.
milosz
On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: > > OK, found the culprit. It's the VTS test name: > arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit > VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions. > > As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin. > > milosz > > > On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >> >> Thank Axel, debugging now. I don't see anything that stands out in any obvious way. >> >> milosz >> >> On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>> >>> Milosz, >>> >>> I privately shared with you the archive containing what you asked for. >>> >>> On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>> >>>> Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this. >>>> >>>> milosz >>>> >>>> On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>> >>>>> Hi Milosz, >>>>> >>>>> Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error : >>>>> >>>>> Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") >>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2) >>>>> >>>>> Do you have any idea ? >>>>> >>>>> Regards, >>>>> Axel >>>>> >>>>> On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>> >>>>>> Great, thanks! >>>>>> >>>>>> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>>>> >>>>>>> I forgot to release it. Will do it today. >>>>>>> >>>>>>> milosz >>>>>>> >>>>>>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> > >>>>>>> > Hi Milosz, >>>>>>> > >>>>>>> > I saw your fix has been merged, is it available now with pip ? >>>>>>> > >>>>>>> > Regards, >>>>>>> > Axel >>>>>>> > >>>>>>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>>>> >> >>>>>>> >> https://github.com/Linaro/squadplugins/pull/9 >>>>>>> >> Antonio, could you take a look? >>>>>>> >> >>>>>>> >> milosz >>>>>>> >> >>>>>>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> >> > >>>>>>> >> > Hi Milosz, >>>>>>> >> > >>>>>>> >> > Great thank you. >>>>>>> >> > >>>>>>> >> > Axel >>>>>>> >> > >>>>>>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>>>> >> >> >>>>>>> >> >> I think it's interactive test section that makes the plugin unhappy. >>>>>>> >> >> I'll fix that and upload new version. >>>>>>> >> >> >>>>>>> >> >> milosz >>>>>>> >> >> >>>>>>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> >> >> > >>>>>>> >> >> > Hi Milosz, >>>>>>> >> >> > >>>>>>> >> >> > Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >>>>>>> >> >> > I hid the sensible information with "****". >>>>>>> >> >> > >>>>>>> >> >> > Regards, >>>>>>> >> >> > Axel >>>>>>> >> >> > >>>>>>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>>>> >> >> >> >>>>>>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> >> >> >> > >>>>>>> >> >> >> > Hi all, >>>>>>> >> >> >> > >>>>>>> >> >> >> > I get back to this topic as I have a new issue now. >>>>>>> >> >> >> > Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. >>>>>>> >> >> >> > The logs : >>>>>>> >> >> >> > >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: >>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions' >>>>>>> >> >> >> > >>>>>>> >> >> >> > Am I missing something ? >>>>>>> >> >> >> >>>>>>> >> >> >> Could you show your test job definition? CTS plugin will first check >>>>>>> >> >> >> whether it's supposed to work on the test job by inspecting the >>>>>>> >> >> >> definition. Apparently there is some mismatch between what is expected >>>>>>> >> >> >> and what is actually there. You can hide/obfuscate all URLs, they're >>>>>>> >> >> >> not that important. >>>>>>> >> >> >> >>>>>>> >> >> >> milosz >>>>>>> >> >> >> >>>>>>> >> >> >> > >>>>>>> >> >> >> > Best regards, >>>>>>> >> >> >> > Axel >>>>>>> >> >> >> > >>>>>>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> >> >> >> >> >>>>>>> >> >> >> >> Hi Milosz, >>>>>>> >> >> >> >> >>>>>>> >> >> >> >> Thank you for your email. It is now working fine. >>>>>>> >> >> >> >> I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. >>>>>>> >> >> >> >> The main thing that was missing in my test definition was the "lava-test-reference". >>>>>>> >> >> >> >> >>>>>>> >> >> >> >> Thank you for you help. >>>>>>> >> >> >> >> >>>>>>> >> >> >> >> Regards, >>>>>>> >> >> >> >> Axel >>>>>>> >> >> >> >> >>>>>>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski milosz.wasilewski@linaro.org wrote: >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis axel.lebourhis@linaro.org wrote: >>>>>>> >> >> >> >>> > >>>>>>> >> >> >> >>> > Hi all, >>>>>>> >> >> >> >>> > >>>>>>> >> >> >> >>> > I saw you have a repository with tradefed and ltp plugins for SQUAD. >>>>>>> >> >> >> >>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to >>>>>>> >> >> >> >>> > attach the error logs to failed test cases. >>>>>>> >> >> >> >>> > >>>>>>> >> >> >> >>> > My questions are : >>>>>>> >> >> >> >>> > How can I setup those plugins to work with my SQUAD instance ? >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> you need to install the package from pip >>>>>>> >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your squad >>>>>>> >> >> >> >>> instance. That should do the trick. Each project has to have the >>>>>>> >> >> >> >>> plugin enabled manually. >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> > Do I need to use a specific tradefed test definition to interface it with >>>>>>> >> >> >> >>> > the plugin ? >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored outside of LAVA. >>>>>>> >> >> >> >>> Tradefed creates a zip file that is later uploaded to 'artifactorial' >>>>>>> >> >> >> >>> (https://archive.validation.linaro.org/). You can use some other >>>>>>> >> >> >> >>> service but it needs to support upload request in the following form: >>>>>>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >>>>>>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >>>>>>> >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >>>>>>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the >>>>>>> >> >> >> >>> content. >>>>>>> >> >> >> >>> This step is done automatically when you use tradefed [1] or >>>>>>> >> >> >> >>> noninteractive-tradefed [2] from test-definitions >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> When running in squad post processing step plugin downloads the zip >>>>>>> >> >> >> >>> file and extracts logs from it. Failed tests are updated with logs. >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> > Does the job submission works the same way ? using the submitjob API ? >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> Test job submission works the same way. Plugins work as a post >>>>>>> >> >> >> >>> processing step after LAVA jobs are completed. >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >>>>>>> >> >> >> >>> [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> milosz >>>>>>> >> >> >> >>> >>>>>>> >> >> >> >>> > >>>>>>> >> >> >> >>> > Thank you in advance, >>>>>>> >> >> >> >>> > Axel >>>>>>> >> >> >> >>> > _______________________________________________ >>>>>>> >> >> >> >>> > Squad-dev mailing list >>>>>>> >> >> >> >>> > Squad-dev@lists.linaro.org >>>>>>> >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev
Great !
I'm going to run VTS/CTS during the weekend, I'll get back to you on Monday :)
Thanks Milosz, Axel
On Fri, 14 Jun 2019 at 13:07, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
1.8 now released and uploaded. I checked a few CTS and VTS jobs locally and all worked well. Please try with your VTS job as you run more tests than I have available. Should work (fingers crossed)!
milosz
On Fri, 14 Jun 2019 at 09:41, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Ok sure no problem !
Axel
On Fri, 14 Jun 2019 at 10:21, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
Axel,
I'll try it out first with some test jobs I have. I finished late yesterday and didn't have time to do that. I only ran unit tests that don't cover all possible cases.
milosz
On Fri, 14 Jun 2019 at 09:17, Axel Lebourhis axel.lebourhis@linaro.org
wrote:
Hi Milosz,
Will you make a new release of the plugin package today with your
recent patch ?
Thanks, Axel
On Thu, 13 Jun 2019 at 15:32, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
Yes, I will submit patch to test definitions. My previous version of
plugin patch didn't actually work. I think I have a better version now. Adding tests to verify special cases now.
milosz
On Thu, 13 Jun 2019 at 14:29, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
Great !
So are you still going to submit a patch to test-definitions ?
Axel
On Thu, 13 Jun 2019 at 14:41, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
> > I wrote this patch: > https://github.com/Linaro/squadplugins/pull/10 > > In my local testing it fixes the problem and a few other issues I
wansn't aware before.
> > milosz > > > On Thu, 13 Jun 2019 at 12:58, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>> >> OK, found the culprit. It's the VTS test name: >>
arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit
>> VTS decided to use "/" at the end of the test name which SQUAD
isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
>> >> As a side note, I found that the plugin code doesn't always
recognize ABI properly. I'll fix that in the plugin.
>> >> milosz >> >> >> On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>> >>> Thank Axel, debugging now. I don't see anything that stands out
in any obvious way.
>>> >>> milosz >>> >>> On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>> >>>> Milosz, >>>> >>>> I privately shared with you the archive containing what you
asked for.
>>>> >>>> On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>> >>>>> Could you share your testjob YAML and test-results.xml from
VTS (may be privately). I think I know what's going on. Thanks for reporting this.
>>>>> >>>>> milosz >>>>> >>>>> On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>> >>>>>> Hi Milosz, >>>>>> >>>>>> Today I faced a new issue, it may be a bug. I ran a full VTS
run, the plugin returned the following error :
>>>>>> >>>>>> Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12
20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12
20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2)
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback
(most recent call last):
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
plugin.postprocess_testjob(test_job)
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
self._assign_test_log(results.test_results.contents, failed)
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File
"/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]:
test_suite_abi, test_suite_name = test_suite_name.split(".")
>>>>>> Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError:
too many values to unpack (expected 2)
>>>>>> >>>>>> Do you have any idea ? >>>>>> >>>>>> Regards, >>>>>> Axel >>>>>> >>>>>> On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>> >>>>>>> Great, thanks! >>>>>>> >>>>>>> On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>>>> >>>>>>>> I forgot to release it. Will do it today. >>>>>>>> >>>>>>>> milosz >>>>>>>> >>>>>>>> On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> > >>>>>>>> > Hi Milosz, >>>>>>>> > >>>>>>>> > I saw your fix has been merged, is it available now with
pip ?
>>>>>>>> > >>>>>>>> > Regards, >>>>>>>> > Axel >>>>>>>> > >>>>>>>> > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>>>> >> >>>>>>>> >> https://github.com/Linaro/squadplugins/pull/9 >>>>>>>> >> Antonio, could you take a look? >>>>>>>> >> >>>>>>>> >> milosz >>>>>>>> >> >>>>>>>> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> >> > >>>>>>>> >> > Hi Milosz, >>>>>>>> >> > >>>>>>>> >> > Great thank you. >>>>>>>> >> > >>>>>>>> >> > Axel >>>>>>>> >> > >>>>>>>> >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>>>> >> >> >>>>>>>> >> >> I think it's interactive test section that makes the
plugin unhappy.
>>>>>>>> >> >> I'll fix that and upload new version. >>>>>>>> >> >> >>>>>>>> >> >> milosz >>>>>>>> >> >> >>>>>>>> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> >> >> > >>>>>>>> >> >> > Hi Milosz, >>>>>>>> >> >> > >>>>>>>> >> >> > Yes sure, you will find it attached. This
definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/.
>>>>>>>> >> >> > I hid the sensible information with "****". >>>>>>>> >> >> > >>>>>>>> >> >> > Regards, >>>>>>>> >> >> > Axel >>>>>>>> >> >> > >>>>>>>> >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski <
milosz.wasilewski@linaro.org> wrote:
>>>>>>>> >> >> >> >>>>>>>> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> >> >> >> > >>>>>>>> >> >> >> > Hi all, >>>>>>>> >> >> >> > >>>>>>>> >> >> >> > I get back to this topic as I have a new issue
now.
>>>>>>>> >> >> >> > Squad worker return a "Plugin postprocessing
error" when the tradefed plugin is triggered.
>>>>>>>> >> >> >> > The logs : >>>>>>>> >> >> >> > >>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
[2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions'
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
Traceback (most recent call last):
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com
celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com
celery[26318]: plugin.postprocess_testjob(test_job)
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com
celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com
celery[26318]: for test_definition in test_action['test']['definitions']:
>>>>>>>> >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]:
KeyError: 'definitions'
>>>>>>>> >> >> >> > >>>>>>>> >> >> >> > Am I missing something ? >>>>>>>> >> >> >> >>>>>>>> >> >> >> Could you show your test job definition? CTS
plugin will first check
>>>>>>>> >> >> >> whether it's supposed to work on the test job by
inspecting the
>>>>>>>> >> >> >> definition. Apparently there is some mismatch
between what is expected
>>>>>>>> >> >> >> and what is actually there. You can hide/obfuscate
all URLs, they're
>>>>>>>> >> >> >> not that important. >>>>>>>> >> >> >> >>>>>>>> >> >> >> milosz >>>>>>>> >> >> >> >>>>>>>> >> >> >> > >>>>>>>> >> >> >> > Best regards, >>>>>>>> >> >> >> > Axel >>>>>>>> >> >> >> > >>>>>>>> >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> >> >> >> >> >>>>>>>> >> >> >> >> Hi Milosz, >>>>>>>> >> >> >> >> >>>>>>>> >> >> >> >> Thank you for your email. It is now working
fine.
>>>>>>>> >> >> >> >> I setup my own local HTTP server and updated my
tradefed test definition (based on yours) to match my setup.
>>>>>>>> >> >> >> >> The main thing that was missing in my test
definition was the "lava-test-reference".
>>>>>>>> >> >> >> >> >>>>>>>> >> >> >> >> Thank you for you help. >>>>>>>> >> >> >> >> >>>>>>>> >> >> >> >> Regards, >>>>>>>> >> >> >> >> Axel >>>>>>>> >> >> >> >> >>>>>>>> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski
milosz.wasilewski@linaro.org wrote:
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis <
axel.lebourhis@linaro.org> wrote:
>>>>>>>> >> >> >> >>> > >>>>>>>> >> >> >> >>> > Hi all, >>>>>>>> >> >> >> >>> > >>>>>>>> >> >> >> >>> > I saw you have a repository with tradefed
and ltp plugins for SQUAD.
>>>>>>>> >> >> >> >>> > If I understand correctly, those plugins
retrieve CTS/VTS/LTP logs to
>>>>>>>> >> >> >> >>> > attach the error logs to failed test cases. >>>>>>>> >> >> >> >>> > >>>>>>>> >> >> >> >>> > My questions are : >>>>>>>> >> >> >> >>> > How can I setup those plugins to work with
my SQUAD instance ?
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> you need to install the package from pip >>>>>>>> >> >> >> >>> (
https://pypi.org/project/squad-linaro-plugins/) to your squad
>>>>>>>> >> >> >> >>> instance. That should do the trick. Each
project has to have the
>>>>>>>> >> >> >> >>> plugin enabled manually. >>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> > Do I need to use a specific tradefed test
definition to interface it with
>>>>>>>> >> >> >> >>> > the plugin ? >>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> yes, tradefed plugin assumes that all logs are
stored outside of LAVA.
>>>>>>>> >> >> >> >>> Tradefed creates a zip file that is later
uploaded to 'artifactorial'
>>>>>>>> >> >> >> >>> (https://archive.validation.linaro.org/). You
can use some other
>>>>>>>> >> >> >> >>> service but it needs to support upload request
in the following form:
>>>>>>>> >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F
"path=@${ATTACHMENT}" -F
>>>>>>>> >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}"
"${ARTIFACTORIAL_URL}"
>>>>>>>> >> >> >> >>> ATTACHMENT is the zip file that tradefed
created and
>>>>>>>> >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate
the user uploading the
>>>>>>>> >> >> >> >>> content. >>>>>>>> >> >> >> >>> This step is done automatically when you use
tradefed [1] or
>>>>>>>> >> >> >> >>> noninteractive-tradefed [2] from
test-definitions
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> When running in squad post processing step
plugin downloads the zip
>>>>>>>> >> >> >> >>> file and extracts logs from it. Failed tests
are updated with logs.
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> > Does the job submission works the same way ?
using the submitjob API ?
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> Test job submission works the same way.
Plugins work as a post
>>>>>>>> >> >> >> >>> processing step after LAVA jobs are completed. >>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> [1]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
>>>>>>>> >> >> >> >>> [2]
https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint...
>>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> milosz >>>>>>>> >> >> >> >>> >>>>>>>> >> >> >> >>> > >>>>>>>> >> >> >> >>> > Thank you in advance, >>>>>>>> >> >> >> >>> > Axel >>>>>>>> >> >> >> >>> >
>>>>>>>> >> >> >> >>> > Squad-dev mailing list >>>>>>>> >> >> >> >>> > Squad-dev@lists.linaro.org >>>>>>>> >> >> >> >>> >
-
Axel Lebourhis -
Milosz Wasilewski