Hi,
I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases?
I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
I will appreciate any help, thanks.
Josue
Hi Josue,
On Jul 28, 2016, at 11:05 PM, Albarran, Josue j-albarran@ti.com wrote:
Hi, I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases? I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
The parse pattern is good, it matches your ltp output. I checked with python re module manually, the output shown as below. So if you don’t see the result in LAVA results, I think it is not the problem of the pattern, might be sth else.
Maybe your are using LAVA V2, I haven’t tried to run ltp with LAVA V2.
import re print re.match("^(?!.+ED)(?P<test_case_id>\w+)\s+(?P<result>PASS|FAIL)\s+\d+", "ETH_XS_FUNC_PING PASS 0").groupdict()
{'test_case_id': 'ETH_XS_FUNC_PING', 'result': 'PASS'}
I will appreciate any help, thanks. Josue _______________________________________________ linaro-validation mailing list linaro-validation@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-validation
-- Thanks, Chase
Yes, I'm currently using LAVA v2. Thanks for the information.
-----Original Message----- From: Chase Qi [mailto:chase.qi@linaro.org] Sent: Thursday, July 28, 2016 10:40 AM To: Albarran, Josue Cc: linaro-validation@lists.linaro.org Subject: Re: [Linaro-validation] Parse output
Hi Josue,
On Jul 28, 2016, at 11:05 PM, Albarran, Josue j-albarran@ti.com wrote:
Hi, I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases? I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
The parse pattern is good, it matches your ltp output. I checked with python re module manually, the output shown as below. So if you don’t see the result in LAVA results, I think it is not the problem of the pattern, might be sth else.
Maybe your are using LAVA V2, I haven’t tried to run ltp with LAVA V2.
import re print re.match("^(?!.+ED)(?P<test_case_id>\w+)\s+(?P<result>PASS|FAIL)\s+\d+", "ETH_XS_FUNC_PING PASS 0").groupdict()
{'test_case_id': 'ETH_XS_FUNC_PING', 'result': 'PASS'}
I will appreciate any help, thanks. Josue _______________________________________________ linaro-validation mailing list linaro-validation@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-validation
-- Thanks, Chase
On Thu, 28 Jul 2016 15:05:05 +0000 "Albarran, Josue" j-albarran@ti.com wrote:
Hi,
I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases?
V2 docs are going to recommend that the implicit pattern at the end of the test definition is hard to get right and this is exactly how it presents. It is a python regex but it has to go through a variety of format tools and debugging the pattern in LAVA takes a lot of time.
The simplest solution here is to take a log of the output, write a custom script that reliably parses the output and which you can test locally. Then either wrap the ltp in that or write the ltp output to a file and process the file. It saves a lot of time in the long run. Scripts like this will remain compatible with V1 and V2.
This doesn't involve changing the test cases and you've done most of the work already by creating a pattern that matches locally - the same process you used to create that pattern is the process to use to create the wrapper script.
(This is what we're doing with the lava-server unit tests.)
I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
I will appreciate any help, thanks.
Josue
Ok, so you're saying to use something like this script https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/common/scripts/ltp... to wrap the test call and parse the output in that script?
I see this script is being called in the following test definition: https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/ubuntu/ltp-ddt.yam....
My goal is to have this wrapper script to be able to see a list of pass or fail with the test cases that were called.
Also, I see lava-test-case-attach is used in the script, isn't it unsupported for now in LAVA v2?
Thanks,
Josue
-----Original Message----- From: linaro-validation [mailto:linaro-validation-bounces@lists.linaro.org] On Behalf Of Neil Williams Sent: Thursday, July 28, 2016 11:08 AM To: linaro-validation@lists.linaro.org Subject: Re: [Linaro-validation] Parse output
On Thu, 28 Jul 2016 15:05:05 +0000 "Albarran, Josue" j-albarran@ti.com wrote:
Hi,
I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases?
V2 docs are going to recommend that the implicit pattern at the end of the test definition is hard to get right and this is exactly how it presents. It is a python regex but it has to go through a variety of format tools and debugging the pattern in LAVA takes a lot of time.
The simplest solution here is to take a log of the output, write a custom script that reliably parses the output and which you can test locally. Then either wrap the ltp in that or write the ltp output to a file and process the file. It saves a lot of time in the long run. Scripts like this will remain compatible with V1 and V2.
This doesn't involve changing the test cases and you've done most of the work already by creating a pattern that matches locally - the same process you used to create that pattern is the process to use to create the wrapper script.
(This is what we're doing with the lava-server unit tests.)
I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
I will appreciate any help, thanks.
Josue
On Fri, 29 Jul 2016 16:56:06 +0000 "Albarran, Josue" j-albarran@ti.com wrote:
Ok, so you're saying to use something like this script https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/common/scripts/ltp... to wrap the test call and parse the output in that script?
A lot of these things are easier in languages other than shell, but yes. That particular script only seems to do part of the job - I'm not familiar with LTP, so can't really tell what it is trying to do but it doesn't call lava-test-case.
From a cursory glance, something like this https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/common/scripts/ltp... may be a better example. Where that is printing test_case_id: it may as well simple call lava-test-case with the same values.
Simplest thing to do is to get a complete log and run tests locally with your own script.
I see this script is being called in the following test definition: https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/ubuntu/ltp-ddt.yam....
My goal is to have this wrapper script to be able to see a list of pass or fail with the test cases that were called.
.... which means that it can then call lava-test-case <name> --result directly.
Also, I see lava-test-case-attach is used in the script, isn't it unsupported for now in LAVA v2?
In this script, lava-test-case-attach is making a tarball available - that's not the results but a log file or other artefacts. In LAVA V2, we are implementing a publishing API which allows test jobs to push files to configured servers for this kind of purpose.
The problem with lava-test-case-attach and V1 is that it needs special handling on the device and is not stored in a sane way on the server.
Thanks,
Josue
-----Original Message----- From: linaro-validation [mailto:linaro-validation-bounces@lists.linaro.org] On Behalf Of Neil Williams Sent: Thursday, July 28, 2016 11:08 AM To: linaro-validation@lists.linaro.org Subject: Re: [Linaro-validation] Parse output
On Thu, 28 Jul 2016 15:05:05 +0000 "Albarran, Josue" j-albarran@ti.com wrote:
Hi,
I ran a simple LTP-DDT test case that produces the following output http://pastebin.ubuntu.com/21280373/. Is there a way to parse this output so that it shows up in results section of the dashboard as a pass. I know you can call lava-test-case from within the script and do the parsing inside the script, but I want to do this for many different test cases. Is there an easier way to do it without changing the test cases?
V2 docs are going to recommend that the implicit pattern at the end of the test definition is hard to get right and this is exactly how it presents. It is a python regex but it has to go through a variety of format tools and debugging the pattern in LAVA takes a lot of time.
The simplest solution here is to take a log of the output, write a custom script that reliably parses the output and which you can test locally. Then either wrap the ltp in that or write the ltp output to a file and process the file. It saves a lot of time in the long run. Scripts like this will remain compatible with V1 and V2.
This doesn't involve changing the test cases and you've done most of the work already by creating a pattern that matches locally - the same process you used to create that pattern is the process to use to create the wrapper script.
(This is what we're doing with the lava-server unit tests.)
I tried the parsing in the test definition as follows http://pastebin.ubuntu.com/21280894/.
I will appreciate any help, thanks.
Josue
--
Neil Williams
linaro-validation@lists.linaro.org
-
Albarran, Josue -
Chase Qi -
Neil Williams