Hello everyone,
I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block
Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete.
Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server
Is there anything I'm doing incorrectly ? Or are you aware of this bug ?
Thanks !
Hi Loys,
There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago.
For using it, you need to enable event notification on your server first. Please follow these instructions:
https://staging.validation.linaro.org/static/docs/v2/advanced-installation.h...
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example.
To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/
In either case, lava-tool should exit when it receives job 'completed' event from the server.
HTH,
On 12/13/2017 02:54 PM, Loys Ollivier wrote:
Hello everyone,
I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block
Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete.
Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server
Is there anything I'm doing incorrectly ? Or are you aware of this bug ?
Thanks !
-- Loys OLLIVIER
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic < stevan.radakovic@linaro.org> wrote:
Hi Loys,
There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago.
For using it, you need to enable event notification on your server first. Please follow these instructions:
https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example.
To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/
In either case, lava-tool should exit when it receives job 'completed' event from the server.
HTH,
On 12/13/2017 02:54 PM, Loys Ollivier wrote:
Hello everyone,
I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block
Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete.
Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server
Is there anything I'm doing incorrectly ? Or are you aware of this bug ?
Thanks !
-- Loys OLLIVIER
Lava-users mailing listLava-users@lists.linaro.orghttps://lists.linaro.org/mailman/listinfo/lava-users
-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on your server first. Please follow these instructions: https://staging.validation.linaro.org/static/docs/v2/advanced-installation.html#configuring-event-notifications <https://staging.validation.linaro.org/static/docs/v2/advanced-installation.html#configuring-event-notifications> After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.org> https://lists.linaro.org/mailman/listinfo/lava-users <https://lists.linaro.org/mailman/listinfo/lava-users>-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org <http://www.linaro.org>> │ Open source software for ARM SoCs-- Loys OLLIVIER Baylibre
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on your server first. Please follow these instructions:https://staging.validation.linaro.org/static/docs/v2/advanced-installation.h...
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.org> https://lists.linaro.org/mailman/listinfo/lava-users <https://lists.linaro.org/mailman/listinfo/lava-users>-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org <http://www.linaro.org>> │ Open sourcesoftware for ARM SoCs
-- Loys OLLIVIER Baylibre
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on your server first. Please follow these instructions:https://staging.validation.linaro.org/static/docs/v2/advanced-installation.h...
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.org> https://lists.linaro.org/mailman/listinfo/lava-users <https://lists.linaro.org/mailman/listinfo/lava-users>-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org <http://www.linaro.org>> │ Open sourcesoftware for ARM SoCs
-- Loys OLLIVIER Baylibre
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080 --job-definition job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528 Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts', 'lava-tool')() File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run() File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py", line 820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on your server first. Please follow these instructions:https://staging.validation.linaro.org/static/docs/v2/advanced-installation.h...
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.org> https://lists.linaro.org/mailman/listinfo/lava-users <https://lists.linaro.org/mailman/listinfo/lava-users>-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org <http://www.linaro.org>> │ Open sourcesoftware for ARM SoCs
-- Loys OLLIVIER Baylibre
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080 --job-definition job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if it was just another django project, it is not. The django internal support is not supported for LAVA. It needs to be packaged (using the developer build scripts if you want to build latest) and have apache properly configured. (It's also advisable to use https:// to protect the authentication tokens.)
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts', 'lava-tool')() File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run() File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py", line 820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on yourserver
first. Please follow these instructions:advanced-installation.html#configuring-event-notifications
advanced-installation.html#configuring-event-notifications>
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed inthe
next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.orghttps://lists.linaro.org/mailman/listinfo/lava-users <https://lists.linaro.org/mailman/listinfo/lava-users>-- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org <http://www.linaro.org>> │ Opensource
software for ARM SoCs
-- Loys OLLIVIER Baylibre
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080 --job-definition job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if it was just another django project, it is not. The django internal support is not supported for LAVA. It needs to be packaged (using the developer build scripts if you want to build latest) and have apache properly configured. (It's also advisable to use https:// to protect the authentication tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months. As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts', 'lava-tool')() File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run() File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py", line 820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and file a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote:
Hello Stevan,
Thanks for the help on that. I enabled the server side notifications, I can see them using a publisher + python on my host computer. Unfortunately lava-tool still doesn't return even when the job finishes....
Also, my server is running on a specific port so to access it I am using: lava-tool wait-job-events http://user@server:port/RPC2 It works perfectly well to submit jobs. I also checked that the port number 5500 is opened.
Is there anything else I should do on the lava-tool side to get this notifications ? Again I can receive them on the host using python but for some reason lava-tool doesn't work.
Thanks !
Loys
On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> wrote:
Hi Loys, There's definitely bug with the message you get from the "submit-job --block" and we will open a ticket for this - wait-for-job does not exist it was replaced with wait-job-events long time ago. For using it, you need to enable event notification on yourserver
first. Please follow these instructions:advanced-installation.html#configuring-event-notifications
advanced-installation.html#configuring-event-notifications>
After that, using lava-tool, you can either submit and wait for the job events or track the events of the existing job. Example you provided is good for the latter example. To submit and wait you can use: lava-tool wait-job-events --job-definition /path/to/job.yaml http://your.server/RPC2/ In either case, lava-tool should exit when it receives job 'completed' event from the server. HTH, On 12/13/2017 02:54 PM, Loys Ollivier wrote:> Hello everyone, > > I am using lava-tool to monitor my jobs. Previously I used: > $ lava-tool submit-job --block > > Using version of lava-tool 0.23 I now have this message: > --> This kind of polling is deprecated and will be removed in
the
> next release. Please use "wait-for-job" command. > > But "wait-for-job" doesn't exist. > There is a "wait-job-events" option though. I tried this one and > it doesn't return even once the job has finished. If I manually > stop it and restart it with the same job number I get as output: > --> Job already finished with status Complete. > > Command I'm using: > $ lava-tool wait-job-events --job-id 20 http://user@lava-server > > Is there anything I'm doing incorrectly ? Or are you aware of > this bug ? > > Thanks ! > > > -- Loys OLLIVIER > > > > > _______________________________________________ > Lava-users mailing list > Lava-users@lists.linaro.org <mailto:Lava-users@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lava-users > https://lists.linaro.org/mailman/listinfo/lava-users -- Stevan Radaković | LAVA Engineer Linaro.org <www.linaro.org http://www.linaro.org> │ Open
source
software for ARM SoCs
-- Loys OLLIVIER Baylibre
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if it was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly configured. (It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from packages and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject support?
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance, it's not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-===================-================================================================ ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote:
Hi Loys,
There's auth-add command which is used to add credentials for authenticatation of lava-tool to the lava-server instance. But if you can submit a job via lava-tool you've probably already done that.
Are you receiving no output after submitting the job via lava-tool wait-job-events?
Exactly.
It's possibly a bug in lava-tool I'm also seeing job completed event missing when trying it out locally. I'll investigate some more and
file
a bug.
lava-tool exits when it receives job completed event, that's the most likely reason that it hangs.
Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
Cheers,
On 12/14/2017 03:35 PM, Loys Ollivier wrote: > Hello Stevan, > > Thanks for the help on that. I enabled the server side
notifications,
> I can see them using a publisher + python on my host computer. > Unfortunately lava-tool still doesn't return even when the job > finishes.... > > Also, my server is running on a specific port so to access it I am > using: lava-tool wait-job-events http://user@server:port/RPC2 > It works perfectly well to submit jobs. I also checked that the port > number 5500 is opened. > > Is there anything else I should do on the lava-tool side to get this > notifications ? Again I can receive them on the host using python
but
> for some reason lava-tool doesn't work. > > Thanks ! > > Loys > > On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic > <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> > wrote: > > Hi Loys, > > There's definitely bug with the message you get from the > "submit-job --block" and we will open a ticket for this - > wait-for-job does not exist it was replaced with
wait-job-events
> long time ago. > > For using it, you need to enable event notification on your
server
> first. Please follow these instructions: > > > https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications
> > > > <https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications>
> > > > After that, using lava-tool, you can either submit and wait for > the job events or track the events of the existing job. Example > you provided is good for the latter example. > > To submit and wait you can use: > lava-tool wait-job-events --job-definition /path/to/job.yaml > http://your.server/RPC2/ > > In either case, lava-tool should exit when it receives job > 'completed' event from the server. > > > HTH, > > > > On 12/13/2017 02:54 PM, Loys Ollivier wrote: >> Hello everyone, >> >> I am using lava-tool to monitor my jobs. Previously I used: >> $ lava-tool submit-job --block >> >> Using version of lava-tool 0.23 I now have this message: >> --> This kind of polling is deprecated and will be removed in
the
>> next release. Please use "wait-for-job" command. >> >> But "wait-for-job" doesn't exist. >> There is a "wait-job-events" option though. I tried this one
and
>> it doesn't return even once the job has finished. If I
manually
>> stop it and restart it with the same job number I get as
output:
>> --> Job already finished with status Complete. >> >> Command I'm using: >> $ lava-tool wait-job-events --job-id 20
>> >> Is there anything I'm doing incorrectly ? Or are you aware of >> this bug ? >> >> Thanks ! >> >> >> -- Loys OLLIVIER >> >> >> >> >> _______________________________________________ >> Lava-users mailing list >> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
>> https://lists.linaro.org/mailman/listinfo/lava-users >> https://lists.linaro.org/mailman/listinfo/lava-users > -- Stevan Radaković | LAVA Engineer > Linaro.org <www.linaro.org http://www.linaro.org> │ Open
source
> software for ARM SoCs > > > > > -- > Loys OLLIVIER > Baylibre >
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if it was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly configured. (It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from packages and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance, it's not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-===================-================================================================ ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ? I will try to upgrade lava and test with the new instance.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote: > Hi Loys, > > There's auth-add command which is used to add credentials for > authenticatation of lava-tool to the lava-server instance. But if you > can submit a job via lava-tool you've probably already done that. > > Are you receiving no output after submitting the job via lava-tool > wait-job-events? > Exactly. > It's possibly a bug in lava-tool I'm also seeing job completed event > missing when trying it out locally. I'll investigate some more and
file
> a bug. > > lava-tool exits when it receives job completed event, that's the most > likely reason that it hangs. Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
> Cheers, > > On 12/14/2017 03:35 PM, Loys Ollivier wrote: >> Hello Stevan, >> >> Thanks for the help on that. I enabled the server side
notifications,
>> I can see them using a publisher + python on my host computer. >> Unfortunately lava-tool still doesn't return even when the job >> finishes.... >> >> Also, my server is running on a specific port so to access it I am >> using: lava-tool wait-job-events http://user@server:port/RPC2 >> It works perfectly well to submit jobs. I also checked that the port >> number 5500 is opened. >> >> Is there anything else I should do on the lava-tool side to get this >> notifications ? Again I can receive them on the host using python
but
>> for some reason lava-tool doesn't work. >> >> Thanks ! >> >> Loys >> >> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >> <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> >> wrote: >> >> Hi Loys, >> >> There's definitely bug with the message you get from the >> "submit-job --block" and we will open a ticket for this - >> wait-for-job does not exist it was replaced with
wait-job-events
>> long time ago. >> >> For using it, you need to enable event notification on your
server
>> first. Please follow these instructions: >> >> >> https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications
>> >> >> >> <https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications>
>> >> >> >> After that, using lava-tool, you can either submit and wait for >> the job events or track the events of the existing job. Example >> you provided is good for the latter example. >> >> To submit and wait you can use: >> lava-tool wait-job-events --job-definition /path/to/job.yaml >> http://your.server/RPC2/ >> >> In either case, lava-tool should exit when it receives job >> 'completed' event from the server. >> >> >> HTH, >> >> >> >> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>> Hello everyone, >>> >>> I am using lava-tool to monitor my jobs. Previously I used: >>> $ lava-tool submit-job --block >>> >>> Using version of lava-tool 0.23 I now have this message: >>> --> This kind of polling is deprecated and will be removed in
the
>>> next release. Please use "wait-for-job" command. >>> >>> But "wait-for-job" doesn't exist. >>> There is a "wait-job-events" option though. I tried this one
and
>>> it doesn't return even once the job has finished. If I
manually
>>> stop it and restart it with the same job number I get as
output:
>>> --> Job already finished with status Complete. >>> >>> Command I'm using: >>> $ lava-tool wait-job-events --job-id 20
>>> >>> Is there anything I'm doing incorrectly ? Or are you aware of >>> this bug ? >>> >>> Thanks ! >>> >>> >>> -- Loys OLLIVIER >>> >>> >>> >>> >>> _______________________________________________ >>> Lava-users mailing list >>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
>>> https://lists.linaro.org/mailman/listinfo/lava-users >>> https://lists.linaro.org/mailman/listinfo/lava-users >> -- Stevan Radaković | LAVA Engineer >> Linaro.org <www.linaro.org http://www.linaro.org> │ Open
source
>> software for ARM SoCs >> >> >> >> >> -- >> Loys OLLIVIER >> Baylibre >>
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17 January 2018 at 10:47, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com
wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if
it
was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly
configured.
(It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from
packages
and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject
support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Right, that makes sense. OK - it was a throwback to another issue where the test instance was poorly configured. The redirection happened to use the same kind of URL.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance,
it's
not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/
trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-======
=============-==============================================
ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ?
Normally, we would do that on the server side through the XML-RPC interface - deprecate the old call and add a new one. lava-tool is beginning to show problems from a lot of historical legacy code and it's not at all easy to remove as we did with the removal of V1. We're looking at a new tool with a clean Python3 start as a replacement. Once we have moved forward a bit more with that one, I'll make the announcement.
In the meantime, it's only this particular functionality in lava-tool which is affected. Are you using this particular support frequently? We can look at using the XML-RPC call system.api_version from lava-tool to check the particular instance being used for the wait-job-events support.
I will try to upgrade lava and test with the new instance.
Thanks, sorry for any confusion - let us know the results either way.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py",
line
49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which
causes
a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking
ticket
for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote: > > On 14/12/2017 16:18, Stevan Radakovic wrote: >> Hi Loys, >> >> There's auth-add command which is used to add credentials for >> authenticatation of lava-tool to the lava-server instance. But if
you
>> can submit a job via lava-tool you've probably already done that. >> >> Are you receiving no output after submitting the job via lava-tool >> wait-job-events? >> > Exactly. >> It's possibly a bug in lava-tool I'm also seeing job completed
event
>> missing when trying it out locally. I'll investigate some more and
file
>> a bug. >> >> lava-tool exits when it receives job completed event, that's the
most
>> likely reason that it hangs. > Not sure to get your point here. But I think we're on the same page: > lava-tool hangs... > > Thanks for help ! > > Is there any way I can follow the evolution / resolution of this
bug ?
> >> Cheers, >> >> On 12/14/2017 03:35 PM, Loys Ollivier wrote: >>> Hello Stevan, >>> >>> Thanks for the help on that. I enabled the server side
notifications,
>>> I can see them using a publisher + python on my host computer. >>> Unfortunately lava-tool still doesn't return even when the job >>> finishes.... >>> >>> Also, my server is running on a specific port so to access it I am >>> using: lava-tool wait-job-events http://user@server:port/RPC2 >>> It works perfectly well to submit jobs. I also checked that the
port
>>> number 5500 is opened. >>> >>> Is there anything else I should do on the lava-tool side to get
this
>>> notifications ? Again I can receive them on the host using python
but
>>> for some reason lava-tool doesn't work. >>> >>> Thanks ! >>> >>> Loys >>> >>> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >>> <stevan.radakovic@linaro.org <mailto:stevan.radakovic@linaro.org
>>> wrote: >>> >>> Hi Loys, >>> >>> There's definitely bug with the message you get from the >>> "submit-job --block" and we will open a ticket for this - >>> wait-for-job does not exist it was replaced with
wait-job-events
>>> long time ago. >>> >>> For using it, you need to enable event notification on your
server
>>> first. Please follow these instructions: >>> >>> >>> https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications
>>> >>> >>> >>> <https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications>
>>> >>> >>> >>> After that, using lava-tool, you can either submit and wait
for
>>> the job events or track the events of the existing job.
Example
>>> you provided is good for the latter example. >>> >>> To submit and wait you can use: >>> lava-tool wait-job-events --job-definition /path/to/job.yaml >>> http://your.server/RPC2/ >>> >>> In either case, lava-tool should exit when it receives job >>> 'completed' event from the server. >>> >>> >>> HTH, >>> >>> >>> >>> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>>> Hello everyone, >>>> >>>> I am using lava-tool to monitor my jobs. Previously I used: >>>> $ lava-tool submit-job --block >>>> >>>> Using version of lava-tool 0.23 I now have this message: >>>> --> This kind of polling is deprecated and will be removed
in
the
>>>> next release. Please use "wait-for-job" command. >>>> >>>> But "wait-for-job" doesn't exist. >>>> There is a "wait-job-events" option though. I tried this one
and
>>>> it doesn't return even once the job has finished. If I
manually
>>>> stop it and restart it with the same job number I get as
output:
>>>> --> Job already finished with status Complete. >>>> >>>> Command I'm using: >>>> $ lava-tool wait-job-events --job-id 20
>>>> >>>> Is there anything I'm doing incorrectly ? Or are you aware
of
>>>> this bug ? >>>> >>>> Thanks ! >>>> >>>> >>>> -- Loys OLLIVIER >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Lava-users mailing list >>>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>> https://lists.linaro.org/mailman/listinfo/lava-users >>> -- Stevan Radaković | LAVA Engineer >>> Linaro.org <www.linaro.org http://www.linaro.org> │ Open
source
>>> software for ARM SoCs >>> >>> >>> >>> >>> -- >>> Loys OLLIVIER >>> Baylibre >>>
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17/01/2018 12:02, Neil Williams wrote:
On 17 January 2018 at 10:47, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com
wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if
it
was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly
configured.
(It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from
packages
and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject
support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Right, that makes sense. OK - it was a throwback to another issue where the test instance was poorly configured. The redirection happened to use the same kind of URL.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance,
it's
not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/
trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-======
=============-==============================================
ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ?
Normally, we would do that on the server side through the XML-RPC interface
- deprecate the old call and add a new one. lava-tool is beginning to show
problems from a lot of historical legacy code and it's not at all easy to remove as we did with the removal of V1. We're looking at a new tool with a clean Python3 start as a replacement. Once we have moved forward a bit more with that one, I'll make the announcement.
I understand thanks.
In the meantime, it's only this particular functionality in lava-tool which is affected. Are you using this particular support frequently? We can look at using the XML-RPC call system.api_version from lava-tool to check the particular instance being used for the wait-job-events support.
Here's what I am using in my CI loop: # lava-tool submit-job http://loys@lava-test:10080 job.yaml --block submitted as job: http://lava-test:10080/scheduler/job/75
This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
Hence my question about wait-job-events. As I did not want to use a deprecated command. I guess I will stick to that one until the new announcement. I do not have particular issues with the submit-job command.
I will try to upgrade lava and test with the new instance.
Thanks, sorry for any confusion - let us know the results either way.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py",
line
49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote: > Hi Loys, > > After some more testing I realized it's a bug in lava-tool which
causes
> a problem when you're using a specific port for the lava-server > endpoint. It's a pretty straightforward fix, here's the tracking
ticket
> for it: https://projects.linaro.org/servicedesk/customer/portal/1/ CTT-819 > > Thanks, > > On 12/14/2017 05:56 PM, Loys Ollivier wrote: >> >> On 14/12/2017 16:18, Stevan Radakovic wrote: >>> Hi Loys, >>> >>> There's auth-add command which is used to add credentials for >>> authenticatation of lava-tool to the lava-server instance. But if
you
>>> can submit a job via lava-tool you've probably already done that. >>> >>> Are you receiving no output after submitting the job via lava-tool >>> wait-job-events? >>> >> Exactly. >>> It's possibly a bug in lava-tool I'm also seeing job completed
event
>>> missing when trying it out locally. I'll investigate some more and
file
>>> a bug. >>> >>> lava-tool exits when it receives job completed event, that's the
most
>>> likely reason that it hangs. >> Not sure to get your point here. But I think we're on the same page: >> lava-tool hangs... >> >> Thanks for help ! >> >> Is there any way I can follow the evolution / resolution of this
bug ?
>> >>> Cheers, >>> >>> On 12/14/2017 03:35 PM, Loys Ollivier wrote: >>>> Hello Stevan, >>>> >>>> Thanks for the help on that. I enabled the server side
notifications,
>>>> I can see them using a publisher + python on my host computer. >>>> Unfortunately lava-tool still doesn't return even when the job >>>> finishes.... >>>> >>>> Also, my server is running on a specific port so to access it I am >>>> using: lava-tool wait-job-events http://user@server:port/RPC2 >>>> It works perfectly well to submit jobs. I also checked that the
port
>>>> number 5500 is opened. >>>> >>>> Is there anything else I should do on the lava-tool side to get
this
>>>> notifications ? Again I can receive them on the host using python
but
>>>> for some reason lava-tool doesn't work. >>>> >>>> Thanks ! >>>> >>>> Loys >>>> >>>> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >>>> <stevan.radakovic@linaro.org <mailto:stevan.radakovic@linaro.org
>>>> wrote: >>>> >>>> Hi Loys, >>>> >>>> There's definitely bug with the message you get from the >>>> "submit-job --block" and we will open a ticket for this - >>>> wait-for-job does not exist it was replaced with
wait-job-events
>>>> long time ago. >>>> >>>> For using it, you need to enable event notification on your server >>>> first. Please follow these instructions: >>>> >>>> >>>> https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications >>>> >>>> >>>> >>>> https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications >>>> >>>> >>>> >>>> After that, using lava-tool, you can either submit and wait
for
>>>> the job events or track the events of the existing job.
Example
>>>> you provided is good for the latter example. >>>> >>>> To submit and wait you can use: >>>> lava-tool wait-job-events --job-definition /path/to/job.yaml >>>> http://your.server/RPC2/ >>>> >>>> In either case, lava-tool should exit when it receives job >>>> 'completed' event from the server. >>>> >>>> >>>> HTH, >>>> >>>> >>>> >>>> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>>>> Hello everyone, >>>>> >>>>> I am using lava-tool to monitor my jobs. Previously I used: >>>>> $ lava-tool submit-job --block >>>>> >>>>> Using version of lava-tool 0.23 I now have this message: >>>>> --> This kind of polling is deprecated and will be removed
in
the >>>>> next release. Please use "wait-for-job" command. >>>>> >>>>> But "wait-for-job" doesn't exist. >>>>> There is a "wait-job-events" option though. I tried this one
and
>>>>> it doesn't return even once the job has finished. If I
manually
>>>>> stop it and restart it with the same job number I get as
output:
>>>>> --> Job already finished with status Complete. >>>>> >>>>> Command I'm using: >>>>> $ lava-tool wait-job-events --job-id 20
>>>>> >>>>> Is there anything I'm doing incorrectly ? Or are you aware
of
>>>>> this bug ? >>>>> >>>>> Thanks ! >>>>> >>>>> >>>>> -- Loys OLLIVIER >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Lava-users mailing list >>>>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
> >>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>> -- Stevan Radaković | LAVA Engineer >>>> Linaro.org <www.linaro.org http://www.linaro.org> │ Open source >>>> software for ARM SoCs >>>> >>>> >>>> >>>> >>>> -- >>>> Loys OLLIVIER >>>> Baylibre >>>> > _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 17 January 2018 at 12:28, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 12:02, Neil Williams wrote:
On 17 January 2018 at 10:47, Loys Ollivier lollivier@baylibre.com
wrote:
On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com
wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
> > Hi Stevan, all, > > Following on that here's the output of the lava-tool using version > 0.24-1. Seems like there's a new bug: > > --8> > > $ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
> job.yaml > submitted as job(s): > http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it
will
work on a random port - make sure you're not trying to run LAVA as if
it
was just another django project, it is not. The django internal
support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly
configured.
(It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for
several
months.
There is no guarantee that it will continue to work and bugs or issues
in
that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to
the
outside world.
That answers for the https:// but there is still a problem with the
test
instance setup using a random port. Make sure it is installed from
packages
and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject
support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Right, that makes sense. OK - it was a throwback to another issue where
the
test instance was poorly configured. The redirection happened to use the same kind of URL.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging,
random
port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': '
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': '
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc',
'submit_time':
'2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance,
it's
not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/
trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-======
=============-==============================================
ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ?
Normally, we would do that on the server side through the XML-RPC
interface
- deprecate the old call and add a new one. lava-tool is beginning to
show
problems from a lot of historical legacy code and it's not at all easy to remove as we did with the removal of V1. We're looking at a new tool
with a
clean Python3 start as a replacement. Once we have moved forward a bit
more
with that one, I'll make the announcement.
I understand thanks.
In the meantime, it's only this particular functionality in lava-tool
which
is affected. Are you using this particular support frequently? We can
look
at using the XML-RPC call system.api_version from lava-tool to check the particular instance being used for the wait-job-events support.
Here's what I am using in my CI loop: # lava-tool submit-job http://loys@lava-test:10080 job.yaml --block submitted as job: http://lava-test:10080/scheduler/job/75
This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
I've created https://projects.linaro.org/browse/CTT-889 which can be tracked via the linked story https://projects.linaro.org/browse/LAVA-1223 to use the system.api call in lava-tool to provide support for the older versions.
Hence my question about wait-job-events. As I did not want to use a deprecated command. I guess I will stick to that one until the new announcement. I do not have particular issues with the submit-job command.
I will try to upgrade lava and test with the new instance.
Thanks, sorry for any confusion - let us know the results either way.
> Now waiting for job events... > Traceback (most recent call last): > File "/usr/bin/lava-tool", line 11, in <module> > load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
> File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py",
line
> 49, in main > LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
> 150, in run > raise SystemExit(cls().dispatch(args)) > File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
> 140, in dispatch > return command.invoke() > File > "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
> 820, in invoke > if data["state"] == self.FINISHED_JOB_STATE: > KeyError: 'state' >
Make sure the LAVA instance itself is running 2018.1 as there have
been
changes to the scheduler state machine.
> > > On 16/12/2017 10:28, Stevan Radakovic wrote: >> Hi Loys, >> >> After some more testing I realized it's a bug in lava-tool which
causes
>> a problem when you're using a specific port for the lava-server >> endpoint. It's a pretty straightforward fix, here's the tracking
ticket
>> for it: https://projects.linaro.org/servicedesk/customer/portal/1/ > CTT-819 >> >> Thanks, >> >> On 12/14/2017 05:56 PM, Loys Ollivier wrote: >>> >>> On 14/12/2017 16:18, Stevan Radakovic wrote: >>>> Hi Loys, >>>> >>>> There's auth-add command which is used to add credentials for >>>> authenticatation of lava-tool to the lava-server instance. But if
you
>>>> can submit a job via lava-tool you've probably already done that. >>>> >>>> Are you receiving no output after submitting the job via
lava-tool
>>>> wait-job-events? >>>> >>> Exactly. >>>> It's possibly a bug in lava-tool I'm also seeing job completed
event
>>>> missing when trying it out locally. I'll investigate some more
and
file
>>>> a bug. >>>> >>>> lava-tool exits when it receives job completed event, that's the
most
>>>> likely reason that it hangs. >>> Not sure to get your point here. But I think we're on the same
page:
>>> lava-tool hangs... >>> >>> Thanks for help ! >>> >>> Is there any way I can follow the evolution / resolution of this
bug ?
>>> >>>> Cheers, >>>> >>>> On 12/14/2017 03:35 PM, Loys Ollivier wrote: >>>>> Hello Stevan, >>>>> >>>>> Thanks for the help on that. I enabled the server side
notifications,
>>>>> I can see them using a publisher + python on my host computer. >>>>> Unfortunately lava-tool still doesn't return even when the job >>>>> finishes.... >>>>> >>>>> Also, my server is running on a specific port so to access it I
am
>>>>> using: lava-tool wait-job-events http://user@server:port/RPC2 >>>>> It works perfectly well to submit jobs. I also checked that the
port
>>>>> number 5500 is opened. >>>>> >>>>> Is there anything else I should do on the lava-tool side to get
this
>>>>> notifications ? Again I can receive them on the host using
python
but
>>>>> for some reason lava-tool doesn't work. >>>>> >>>>> Thanks ! >>>>> >>>>> Loys >>>>> >>>>> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >>>>> <stevan.radakovic@linaro.org <mailto:stevan.radakovic@
linaro.org
>>>>> wrote: >>>>> >>>>> Hi Loys, >>>>> >>>>> There's definitely bug with the message you get from the >>>>> "submit-job --block" and we will open a ticket for this - >>>>> wait-for-job does not exist it was replaced with
wait-job-events
>>>>> long time ago. >>>>> >>>>> For using it, you need to enable event notification on your > server >>>>> first. Please follow these instructions: >>>>> >>>>> >>>>> https://staging.validation.linaro.org/static/docs/v2/ > advanced-installation.html#configuring-event-notifications >>>>> >>>>> >>>>> >>>>> https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications> >>>>> >>>>> >>>>> >>>>> After that, using lava-tool, you can either submit and wait
for
>>>>> the job events or track the events of the existing job.
Example
>>>>> you provided is good for the latter example. >>>>> >>>>> To submit and wait you can use: >>>>> lava-tool wait-job-events --job-definition
/path/to/job.yaml
>>>>> http://your.server/RPC2/ >>>>> >>>>> In either case, lava-tool should exit when it receives job >>>>> 'completed' event from the server. >>>>> >>>>> >>>>> HTH, >>>>> >>>>> >>>>> >>>>> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>>>>> Hello everyone, >>>>>> >>>>>> I am using lava-tool to monitor my jobs. Previously I
used:
>>>>>> $ lava-tool submit-job --block >>>>>> >>>>>> Using version of lava-tool 0.23 I now have this message: >>>>>> --> This kind of polling is deprecated and will be removed
in
> the >>>>>> next release. Please use "wait-for-job" command. >>>>>> >>>>>> But "wait-for-job" doesn't exist. >>>>>> There is a "wait-job-events" option though. I tried this
one
and
>>>>>> it doesn't return even once the job has finished. If I
manually
>>>>>> stop it and restart it with the same job number I get as
output:
>>>>>> --> Job already finished with status Complete. >>>>>> >>>>>> Command I'm using: >>>>>> $ lava-tool wait-job-events --job-id 20
>>>>>> >>>>>> Is there anything I'm doing incorrectly ? Or are you aware
of
>>>>>> this bug ? >>>>>> >>>>>> Thanks ! >>>>>> >>>>>> >>>>>> -- Loys OLLIVIER >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Lava-users mailing list >>>>>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
>> >>>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>>> -- Stevan Radaković | LAVA Engineer >>>>> Linaro.org <www.linaro.org http://www.linaro.org> │ Open > source >>>>> software for ARM SoCs >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Loys OLLIVIER >>>>> Baylibre >>>>> >> > _______________________________________________ > Lava-users mailing list > Lava-users@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/lava-users >
-
Loys Ollivier -
Neil Williams -
Stevan Radakovic