OK, found the culprit. It's the VTS test name: arm64-v8a.VtsHalCameraProviderV2_4Target/VtsHalCameraProviderV2_4Target.CameraHidlTest.autoFocus(legacy/0)_64bit VTS decided to use "/" at the end of the test name which SQUAD isn't very happy about (the legacy/0 part). I added a fix for [] some time ago but it looks like VTS also uses () in some cases :( I'm not sure whether we should fix it in SQUAD or in test-definitions. IMHO adding to many 'special cases' to SQUAD isn't a good idea. I'll propose a patch to test-definitions.
As a side note, I found that the plugin code doesn't always recognize ABI properly. I'll fix that in the plugin.
milosz
On Thu, 13 Jun 2019 at 11:51, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Thank Axel, debugging now. I don't see anything that stands out in any obvious way.
milosz
On Thu, 13 Jun 2019 at 11:21, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Milosz,
I privately shared with you the archive containing what you asked for.
On Thu, 13 Jun 2019 at 12:06, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
Could you share your testjob YAML and test-results.xml from VTS (may be privately). I think I know what's going on. Thanks for reporting this.
milosz
On Thu, 13 Jun 2019 at 10:58, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi Milosz,
Today I faced a new issue, it may be a bug. I ran a full VTS run, the plugin returned the following error :
Jun 12 20:25:36 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:36 +0000] [INFO] Starting CTS/VTS plugin for test job: 1841 Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: [2019-06-12 20:25:38 +0000] [ERROR] Plugin postprocessing error: too many values to unpack (expected 2) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: Traceback (most recent call last): Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: plugin.postprocess_testjob(test_job) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 207, in postprocess_testjob Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: self._assign_test_log(results.test_results.contents, failed) Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 50, in _assign_test_log Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: test_suite_abi, test_suite_name = test_suite_name.split(".") Jun 12 20:25:38 squad.sw.nxp.com celery[6499]: ValueError: too many values to unpack (expected 2)
Do you have any idea ?
Regards, Axel
On Tue, 11 Jun 2019 at 13:27, Axel Lebourhis axel.lebourhis@linaro.org wrote:
Great, thanks!
On Tue, 11 Jun 2019 at 12:42, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
I forgot to release it. Will do it today.
milosz
On Tue, 11 Jun 2019 at 11:03, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: > > Hi Milosz, > > I saw your fix has been merged, is it available now with pip ? > > Regards, > Axel > > On Fri, 7 Jun 2019 at 14:39, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> https://github.com/Linaro/squadplugins/pull/9 >> Antonio, could you take a look? >> >> milosz >> >> On Fri, 7 Jun 2019 at 13:34, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> > >> > Hi Milosz, >> > >> > Great thank you. >> > >> > Axel >> > >> > On Fri, 7 Jun 2019 at 14:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >> I think it's interactive test section that makes the plugin unhappy. >> >> I'll fix that and upload new version. >> >> >> >> milosz >> >> >> >> On Thu, 6 Jun 2019 at 12:45, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> > >> >> > Hi Milosz, >> >> > >> >> > Yes sure, you will find it attached. This definition is the one stored by SQUAD, retrieved with api/testjobs/<id>/definition/. >> >> > I hid the sensible information with "****". >> >> > >> >> > Regards, >> >> > Axel >> >> > >> >> > On Thu, 6 Jun 2019 at 12:45, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >> >> >> On Thu, 6 Jun 2019 at 09:38, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> > >> >> >> > Hi all, >> >> >> > >> >> >> > I get back to this topic as I have a new issue now. >> >> >> > Squad worker return a "Plugin postprocessing error" when the tradefed plugin is triggered. >> >> >> > The logs : >> >> >> > >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [INFO] Starting CTS/VTS plugin for test job: 1717 >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: [2019-06-06 03:35:40 +0000] [ERROR] Plugin postprocessing error: 'definitions' >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: Traceback (most recent call last): >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/squad/ci/models.py", line 132, in __postprocess_testjob__ >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: plugin.postprocess_testjob(test_job) >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: File "/usr/local/lib/python3.5/dist-packages/tradefed/__init__.py", line 185, in postprocess_testjob >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: for test_definition in test_action['test']['definitions']: >> >> >> > Jun 06 03:35:40 squad.sw.nxp.com celery[26318]: KeyError: 'definitions' >> >> >> > >> >> >> > Am I missing something ? >> >> >> >> >> >> Could you show your test job definition? CTS plugin will first check >> >> >> whether it's supposed to work on the test job by inspecting the >> >> >> definition. Apparently there is some mismatch between what is expected >> >> >> and what is actually there. You can hide/obfuscate all URLs, they're >> >> >> not that important. >> >> >> >> >> >> milosz >> >> >> >> >> >> > >> >> >> > Best regards, >> >> >> > Axel >> >> >> > >> >> >> > On Mon, 3 Jun 2019 at 10:19, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> >> >> >> >> >> Hi Milosz, >> >> >> >> >> >> >> >> Thank you for your email. It is now working fine. >> >> >> >> I setup my own local HTTP server and updated my tradefed test definition (based on yours) to match my setup. >> >> >> >> The main thing that was missing in my test definition was the "lava-test-reference". >> >> >> >> >> >> >> >> Thank you for you help. >> >> >> >> >> >> >> >> Regards, >> >> >> >> Axel >> >> >> >> >> >> >> >> On Wed, 29 May 2019 at 11:26, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote: >> >> >> >>> >> >> >> >>> On Wed, 29 May 2019 at 08:27, Axel Lebourhis < axel.lebourhis@linaro.org> wrote: >> >> >> >>> > >> >> >> >>> > Hi all, >> >> >> >>> > >> >> >> >>> > I saw you have a repository with tradefed and ltp plugins for SQUAD. >> >> >> >>> > If I understand correctly, those plugins retrieve CTS/VTS/LTP logs to >> >> >> >>> > attach the error logs to failed test cases. >> >> >> >>> > >> >> >> >>> > My questions are : >> >> >> >>> > How can I setup those plugins to work with my SQUAD instance ? >> >> >> >>> >> >> >> >>> you need to install the package from pip >> >> >> >>> (https://pypi.org/project/squad-linaro-plugins/) to your squad >> >> >> >>> instance. That should do the trick. Each project has to have the >> >> >> >>> plugin enabled manually. >> >> >> >>> >> >> >> >>> > Do I need to use a specific tradefed test definition to interface it with >> >> >> >>> > the plugin ? >> >> >> >>> >> >> >> >>> yes, tradefed plugin assumes that all logs are stored outside of LAVA. >> >> >> >>> Tradefed creates a zip file that is later uploaded to 'artifactorial' >> >> >> >>> (https://archive.validation.linaro.org/). You can use some other >> >> >> >>> service but it needs to support upload request in the following form: >> >> >> >>> curl "${CURL_VERBOSE_FLAG}" -F "path=@${ATTACHMENT}" -F >> >> >> >>> "token=${ARTIFACTORIAL_TOKEN}" "${ARTIFACTORIAL_URL}" >> >> >> >>> ATTACHMENT is the zip file that tradefed created and >> >> >> >>> ARTIFACTORIAL_TOKEN is used to authenticate the user uploading the >> >> >> >>> content. >> >> >> >>> This step is done automatically when you use tradefed [1] or >> >> >> >>> noninteractive-tradefed [2] from test-definitions >> >> >> >>> >> >> >> >>> When running in squad post processing step plugin downloads the zip >> >> >> >>> file and extracts logs from it. Failed tests are updated with logs. >> >> >> >>> >> >> >> >>> > Does the job submission works the same way ? using the submitjob API ? >> >> >> >>> >> >> >> >>> Test job submission works the same way. Plugins work as a post >> >> >> >>> processing step after LAVA jobs are completed. >> >> >> >>> >> >> >> >>> [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef... >> >> >> >>> [2] https://git.linaro.org/qa/test-definitions.git/tree/automated/android/nonint... >> >> >> >>> >> >> >> >>> milosz >> >> >> >>> >> >> >> >>> > >> >> >> >>> > Thank you in advance, >> >> >> >>> > Axel >> >> >> >>> > _______________________________________________ >> >> >> >>> > Squad-dev mailing list >> >> >> >>> > Squad-dev@lists.linaro.org >> >> >> >>> > https://lists.linaro.org/mailman/listinfo/squad-dev