On 3 November 2017 at 08:29, Loys Ollivier lollivier@baylibre.com wrote:

Hi all!

I am working on writing LAVA test definitions and using the callback method to send test results to kernelCI.

I noticed a sort of bug (?) when the test definitions are reporting a lot of elements. The job finishes and the callback gets triggered before the end of the log parsing. I think the callback method is not waiting for the parser to finish before sending the event. The callback output is then missing some test results.

I made a simple test script that reports 300 test elements and returns. I can see in the LAVA log that they are all detected. But in the callback object there is around 80 test results. If I add a sleep (15 seconds) to hold the script before returning, the callback has the 300 test results bundled in the json.

Did anyone experienced this before ?

This sounds familiar. I had similar issue when reporting a big number of results. However I didn't use the callback. In my case it was notification on ZMQ publisher followed by XML-RPC call. Also some results were missing. Subsequent XML-RPC call a few seconds later returned all results. However I thought the problem was fixed. https://projects.linaro.org/browse/LAVA-1030

milosz

Everyone,

Thanks for your feedback. So this is something you experienced as well :) Maybe the callback was not in the picture at the time of the fix (ZMQ/XML-RPC) ? I wasn't around at that time so I can't tell.

Stevan, Here is my test definition: github https://github.com/loysollivier/agl-test-definitions/blob/master/test-suites/yocto-ptest.yaml and the test script: github https://github.com/loysollivier/agl-test-definitions/blob/master/common/scripts/lava-test.sh. It is very basic but feel free to use it if you want to replicate my procedure.

I'll keep an eye on the CTT request (if I manage to log in jira)!

Thanks

On Fri, Nov 3, 2017 at 11:07 AM, Remi Duraffort remi.duraffort@linaro.org wrote:

...

However I thought the problem was fixed. https://projects.linaro.org/browse/LAVA-1030

You are right, this has been fixed for zmq events.

-- Rémi Duraffort LAVA Team

Hello,

unfortunately I do not have the code anymore. But it is fairly easy to reproduce: - Specify a callback instance (URL + token) - Call a test script - Within the test script do something like: for i in {1..1000};do echo "lava-test-case test-$i --result pass"; done - Exit the script - When the LAVA test definition execution finishes: verify that the callback happens - Look in the callback result, test number != 1000

Hope this helps,

Best,

Loys

On Tue, Apr 3, 2018 at 10:59 AM, Chetan Sharma chetan.sharma@timesys.com wrote:

Hi Team Can you share an example code or reference to understand this scenario more.

Look forward to hear from you with positive response.

On Fri, Nov 3, 2017 at 8:52 PM, Loys Ollivier lollivier@baylibre.com wrote:

...

Everyone,

Thanks for your feedback. So this is something you experienced as well :) Maybe the callback was not in the picture at the time of the fix (ZMQ/XML-RPC) ? I wasn't around at that time so I can't tell.

Stevan, Here is my test definition: github https://github.com/loysollivier/agl-test-definitions/blob/master/test-suites/yocto-ptest.yaml and the test script: github https://github.com/loysollivier/agl-test-definitions/blob/master/common/scripts/lava-test.sh. It is very basic but feel free to use it if you want to replicate my procedure.

I'll keep an eye on the CTT request (if I manage to log in jira)!

Thanks

On Fri, Nov 3, 2017 at 11:07 AM, Remi Duraffort < remi.duraffort@linaro.org> wrote:

...

...

However I thought the problem was fixed. https://projects.linaro.org/browse/LAVA-1030

You are right, this has been fixed for zmq events.

-- Rémi Duraffort LAVA Team

-- Loys OLLIVIER Embedded Software Engineer | Baylibre

---

Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users

-- Thanks & Regards Chetan Sharma