Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser. However, if I drop the superuser privileges and add just the privileges for adding workers, it fails with:
Unable to call 'workers.add': <Fault 403: "User 'testuser' is not superuser."
we even tried enabling all the permissions for that user, but leaving the superuser flag off, and it still fails.
Why does this require superuser and the specific permissions related to workers don't work?
Kevin
On Tue, 21 Aug 2018 at 22:46, Kevin Hilman khilman@baylibre.com wrote:
Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser.
Adding remote workers to an instance would be an easy way to DDOS an instance by swamping the ZMQ ports with fake attempts - the process needs to be under the control of the admins of the instance.
If the remote worker is properly configured, it will register itself automatically - this is why encryption of the master:slave communication is so important. A LAVA master which is visible to the internet should always use encryption.
In most cases, the lavacli workers add command isn't required.
However, if I drop the superuser privileges and add just the privileges for adding workers, it fails with:
Unable to call 'workers.add': <Fault 403: "User 'testuser' is not superuser."
we even tried enabling all the permissions for that user, but leaving the superuser flag off, and it still fails.
The check is made at the remote end, in the XMLRPC.
Why does this require superuser and the specific permissions related to workers don't work?
Kevin _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Tue, Aug 28, 2018 at 12:41 AM Neil Williams neil.williams@linaro.org wrote:
On Tue, 21 Aug 2018 at 22:46, Kevin Hilman khilman@baylibre.com wrote:
Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser.
Adding remote workers to an instance would be an easy way to DDOS an instance by swamping the ZMQ ports with fake attempts - the process needs to be under the control of the admins of the instance.
If the remote worker is properly configured, it will register itself automatically - this is why encryption of the master:slave communication is so important. A LAVA master which is visible to the internet should always use encryption.
All masters and slaves are in control of the admins and encryption is enabled. The per-user permissions still do not work.
However, all of this still begs the question: why do those per-user permissions even exist if they don't do anything because superuser privileges are required? If that's a hard requirement, shouldn't those permissions just be removed so it's not confusing for admins?
In most cases, the lavacli workers add command isn't required.
Ahh... so, IIUC, when a new worker connects, it automatically adds itself. so a "workers add" command isn't needed?
What are the cases where a "workers add" is actually needed then?
Kevin
On Tue, 28 Aug 2018 at 19:49, Kevin Hilman khilman@baylibre.com wrote:
On Tue, Aug 28, 2018 at 12:41 AM Neil Williams neil.williams@linaro.org wrote:
On Tue, 21 Aug 2018 at 22:46, Kevin Hilman khilman@baylibre.com wrote:
Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser.
Adding remote workers to an instance would be an easy way to DDOS an
instance by swamping the ZMQ ports with fake attempts - the process needs to be under the control of the admins of the instance.
If the remote worker is properly configured, it will register itself
automatically - this is why encryption of the master:slave communication is so important. A LAVA master which is visible to the internet should always use encryption.
All masters and slaves are in control of the admins and encryption is enabled. The per-user permissions still do not work.
However, all of this still begs the question: why do those per-user permissions even exist if they don't do anything because superuser privileges are required? If that's a hard requirement, shouldn't those permissions just be removed so it's not confusing for admins?
In most cases, the lavacli workers add command isn't required.
Ahh... so, IIUC, when a new worker connects, it automatically adds itself. so a "workers add" command isn't needed?
Yes - with an up to date lava-master, (2018.5 and later IIRC, possibly a release or two earlier, I'd have to check) , the process is automatic.
What are the cases where a "workers add" is actually needed then?
lavacli needs to cope with older versions of lava-master.
We can update the manpage and --help output of lavacli - will also check to see if the XMLRPC call itself should become a no-op in the next release.
https://projects.linaro.org/browse/LAVA-1404
On Tue, Aug 28, 2018 at 11:46 PM Neil Williams neil.williams@linaro.org wrote:
On Tue, 28 Aug 2018 at 19:49, Kevin Hilman khilman@baylibre.com wrote:
On Tue, Aug 28, 2018 at 12:41 AM Neil Williams neil.williams@linaro.org wrote:
On Tue, 21 Aug 2018 at 22:46, Kevin Hilman khilman@baylibre.com wrote:
Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser.
Adding remote workers to an instance would be an easy way to DDOS an instance by swamping the ZMQ ports with fake attempts - the process needs to be under the control of the admins of the instance.
If the remote worker is properly configured, it will register itself automatically - this is why encryption of the master:slave communication is so important. A LAVA master which is visible to the internet should always use encryption.
All masters and slaves are in control of the admins and encryption is enabled. The per-user permissions still do not work.
However, all of this still begs the question: why do those per-user permissions even exist if they don't do anything because superuser privileges are required? If that's a hard requirement, shouldn't those permissions just be removed so it's not confusing for admins?
In most cases, the lavacli workers add command isn't required.
Ahh... so, IIUC, when a new worker connects, it automatically adds itself. so a "workers add" command isn't needed?
Yes - with an up to date lava-master, (2018.5 and later IIRC, possibly a release or two earlier, I'd have to check) , the process is automatic.
Thanks, that seems to work.
The next blocker to having a non-superuser remote worker is the adding the dispatcher_ip, which also requires superuser privileges[1], and doesn't appear to have an individual user permission ACL.
Assuming ZMQ encryption between master/slave, is it possible to have a remote worker without admin privileges? Is this something that has been validated?
Kevin
[1] lab-slave-0_1 | Add dispatcher_ip 192.168.66.1 to lab-slave-0 lab-slave-0_1 | Traceback (most recent call last): lab-slave-0_1 | File "/usr/local/bin/setdispatcherip.py", line 11, in <module> lab-slave-0_1 | server.scheduler.workers.set_config("%s" % sys.argv[2], "dispatcher_ip: %s" % sys.argv[3]) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in _call_ lab-slave-0_1 | return self._send(self._name, args) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request lab-slave-0_1 | verbose=self.__verbose lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request lab-slave-0_1 | return self.single_request(host, handler, request_body, verbose) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request lab-slave-0_1 | return self.parse_response(response) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response lab-slave-0_1 | return u.close() lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close lab-slave-0_1 | raise Fault(**self._stack[0]) lab-slave-0_1 | xmlrpclib.Fault: <Fault 403: "User 'nonadminuser' is not superuser.">
On Wed, 29 Aug 2018 at 17:11, Kevin Hilman khilman@baylibre.com wrote:
On Tue, Aug 28, 2018 at 11:46 PM Neil Williams neil.williams@linaro.org wrote:
On Tue, 28 Aug 2018 at 19:49, Kevin Hilman khilman@baylibre.com wrote:
On Tue, Aug 28, 2018 at 12:41 AM Neil Williams <
neil.williams@linaro.org> wrote:
On Tue, 21 Aug 2018 at 22:46, Kevin Hilman khilman@baylibre.com
wrote:
Hello,
When trying to use lavacli to add a remote worker, it works fine if the user is a superuser.
Adding remote workers to an instance would be an easy way to DDOS an
instance by swamping the ZMQ ports with fake attempts - the process needs to be under the control of the admins of the instance.
If the remote worker is properly configured, it will register itself
automatically - this is why encryption of the master:slave communication is so important. A LAVA master which is visible to the internet should always use encryption.
All masters and slaves are in control of the admins and encryption is enabled. The per-user permissions still do not work.
However, all of this still begs the question: why do those per-user permissions even exist if they don't do anything because superuser privileges are required? If that's a hard requirement, shouldn't those permissions just be removed so it's not confusing for admins?
In most cases, the lavacli workers add command isn't required.
Ahh... so, IIUC, when a new worker connects, it automatically adds itself. so a "workers add" command isn't needed?
Yes - with an up to date lava-master, (2018.5 and later IIRC, possibly a
release or two earlier, I'd have to check) , the process is automatic.
Thanks, that seems to work.
The next blocker to having a non-superuser remote worker is the adding the dispatcher_ip, which also requires superuser privileges[1], and doesn't appear to have an individual user permission ACL.
Assuming ZMQ encryption between master/slave, is it possible to have a remote worker without admin privileges? Is this something that has been validated?
No. Adding and managing workers is solely a superuser task because such operations can fundamentally change the topology of the lab and undermine ongoing CI.
Kevin
[1] lab-slave-0_1 | Add dispatcher_ip 192.168.66.1 to lab-slave-0 lab-slave-0_1 | Traceback (most recent call last): lab-slave-0_1 | File "/usr/local/bin/setdispatcherip.py", line 11, in <module> lab-slave-0_1 | server.scheduler.workers.set_config("%s" % sys.argv[2], "dispatcher_ip: %s" % sys.argv[3]) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in _call_ lab-slave-0_1 | return self._send(self._name, args) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request lab-slave-0_1 | verbose=self.__verbose lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request lab-slave-0_1 | return self.single_request(host, handler, request_body, verbose) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request lab-slave-0_1 | return self.parse_response(response) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response lab-slave-0_1 | return u.close() lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close lab-slave-0_1 | raise Fault(**self._stack[0]) lab-slave-0_1 | xmlrpclib.Fault: <Fault 403: "User 'nonadminuser' is not superuser.">
Hi all!
Am 30.08.2018 um 09:38 schrieb Neil Williams:
> Yes - with an up to date lava-master, (2018.5 and later IIRC, possibly a release or two earlier, I'd have to check) , the process is automatic. Thanks, that seems to work. The next blocker to having a non-superuser remote worker is the adding the dispatcher_ip, which also requires superuser privileges[1], and doesn't appear to have an individual user permission ACL. Assuming ZMQ encryption between master/slave, is it possible to have a remote worker without admin privileges? Is this something that has been validated?No. Adding and managing workers is solely a superuser task because such operations can fundamentally change the topology of the lab and undermine ongoing CI.
JSM: @Neil: but for this we have the ACL's . So why not *enable* ppl to do this if they choose so by granting the ACL. The superuser flag is IMHO just a shortcut for 'ALL ACL'.
Here is the use-case: - Remove lab being brought up. Keys exchanged, the remote lab should be able to register and manager its internal settings (like dispatcher_ip) .
dispatcher_ip is a good example. B/C the *server* admin does not need to know this. It is in the domain of the admin of the *worker lab*.
Jan-Simon
Kevin [1] lab-slave-0_1 | Add dispatcher_ip 192.168.66.1 to lab-slave-0 lab-slave-0_1 | Traceback (most recent call last): lab-slave-0_1 | File "/usr/local/bin/setdispatcherip.py", line 11, in <module> lab-slave-0_1 | server.scheduler.workers.set_config("%s" % sys.argv[2], "dispatcher_ip: %s" % sys.argv[3]) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in _call_ lab-slave-0_1 | return self._send(self._name, args) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request lab-slave-0_1 | verbose=self.__verbose lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request lab-slave-0_1 | return self.single_request(host, handler, request_body, verbose) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request lab-slave-0_1 | return self.parse_response(response) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response lab-slave-0_1 | return u.close() lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close lab-slave-0_1 | raise Fault(**self._stack[0]) lab-slave-0_1 | xmlrpclib.Fault: <Fault 403: "User 'nonadminuser' is not superuser.">--
Neil Williams
neil.williams@linaro.org mailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Hello,
LAVA and more precisely the xmlrpc api in that case, are not using the Django fine grained permissions ( https://docs.djangoproject.com/en/2.1/topics/auth/default/#default-permissio... ). We are only using the is_superuser flag to know if a user is able or not to make the corresponding calls.
That's in fact a good idea to use the django permissions (like adding a decorator that calls has_perm("lava_scheduler_app.add_worker")) in the xmlrpc api. If you have some time to write a patch about it, I will be happy to review it.
Cheers
2018-08-30 13:11 GMT+02:00 Jan-Simon Moeller jsmoeller@linuxfoundation.org :
Hi all!
Am 30.08.2018 um 09:38 schrieb Neil Williams:
> Yes - with an up to date lava-master, (2018.5 and later IIRC, possibly a release or two earlier, I'd have to check) , the process is automatic. Thanks, that seems to work. The next blocker to having a non-superuser remote worker is the adding the dispatcher_ip, which also requires superuser privileges[1], and doesn't appear to have an individual user permission ACL. Assuming ZMQ encryption between master/slave, is it possible to have a remote worker without admin privileges? Is this something that has been validated?No. Adding and managing workers is solely a superuser task because such operations can fundamentally change the topology of the lab and undermine ongoing CI.
JSM: @Neil: but for this we have the ACL's . So why not *enable* ppl to do this if they choose so by granting the ACL. The superuser flag is IMHO just a shortcut for 'ALL ACL'.
Here is the use-case:
- Remove lab being brought up. Keys exchanged, the remote lab should be
able to register and manager its internal settings (like dispatcher_ip) .
dispatcher_ip is a good example. B/C the *server* admin does not need to know this. It is in the domain of the admin of the *worker lab*.
Jan-Simon
Kevin [1] lab-slave-0_1 | Add dispatcher_ip 192.168.66.1 to lab-slave-0 lab-slave-0_1 | Traceback (most recent call last): lab-slave-0_1 | File "/usr/local/bin/setdispatcherip.py", line 11, in <module> lab-slave-0_1 | server.scheduler.workers.set_config("%s" % sys.argv[2], "dispatcher_ip: %s" % sys.argv[3]) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in _call_ lab-slave-0_1 | return self._send(self._name, args) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request lab-slave-0_1 | verbose=self.__verbose lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request lab-slave-0_1 | return self.single_request(host, handler, request_body, verbose) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request lab-slave-0_1 | return self.parse_response(response) lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response lab-slave-0_1 | return u.close() lab-slave-0_1 | File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close lab-slave-0_1 | raise Fault(**self._stack[0]) lab-slave-0_1 | xmlrpclib.Fault: <Fault 403: "User 'nonadminuser' is not superuser.">--
Neil Williams
neil.williams@linaro.org mailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-
Jan-Simon Moeller -
Kevin Hilman -
Neil Williams -
Remi Duraffort