-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello guys.
I'd like to ask for your advice in for implementing authentication mechanism for the XML-RPC interface to the dashboard. I believe this problem has two aspects: user needs and client program needs. I was about to implement HTTP Digest authentication that will authorize the user against the Django user database. James pointed out that we should discuss this topic more. I'd like to know what you think.
What users need to have. ========================
I think that for our internal use cases a best option would be launchpad integration. We don't mind using launchpad, we already have accounts there. We could use oauth for authentication. I'm not an expect on oauth so I'm not sure what kind of impact it has on the API itself, if any (do we need to pass some special argument to each call?). I suspect it has a negotiation phase where the client and oauth provider exchange some messages after which the client can authenticate by providing some extra header in each request. In any way that's one option.
For a centralized installation this would be pretty good. For ad-hoc installations it could be difficult to setup but I'm just guessing here.
Other options are (off the top of my head): - - Basic/Digest HTTP authentication. Digest is pretty good (or so wikipedia tells me) and has a very nice property of not requiring any changes to the API. User identity is provided as out-of-band HTTP header. - - Custom authentication via special XML-RPC methods and some user/pass/token/whatever arguments. This is IMHO pretty ugly as it affects the interface. The upside is that we can build any authentication options we like. - - No authentication at all. All XML-RPC interfaces are public and anyone may call them. Surprisingly this is not such a bad option. Since all request would be anonymous all the users would be allowed to do is upload data to public "directories". All other interaction would have to be performed via the website or the administration panel.
What clients need to implement. ===============================
So this is about everyone who has to interact with the API, currently that's just dashboard itself (we have a command-line client that can use each exposed method) and of course abrek. While I don't mind changing the API or security method to make it better we need to consider that each client will have to adapt as well. Worse, if one client has to interact with two servers it may be required to discover/negotiate supported authentication before we can even connect.
Thanks Zygmunt
On Sat, 04 Sep 2010 18:32:30 +0200, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello guys.
I'd like to ask for your advice in for implementing authentication mechanism for the XML-RPC interface to the dashboard. I believe this problem has two aspects: user needs and client program needs. I was about to implement HTTP Digest authentication that will authorize the user against the Django user database. James pointed out that we should discuss this topic more. I'd like to know what you think.
What users need to have.
I think that for our internal use cases a best option would be launchpad integration. We don't mind using launchpad, we already have accounts there.
That works for now, yes. That's a distinct question from the auth mechanism of the API though. We can use openid for users to authenticate themselves via LP, or any openid provider as we like. We can then choose an authentication mechanism basically independent of that for the API.
We could use oauth for authentication. I'm not an expect on oauth so I'm not sure what kind of impact it has on the API itself, if any (do we need to pass some special argument to each call?). I suspect it has a negotiation phase where the client and oauth provider exchange some messages after which the client can authenticate by providing some extra header in each request. In any way that's one option.
That's correct. There are no changes to the API needed.
The main difference to digest is that you don't give your password to the app, you give it a token that stands in for your password, but can be revoked independently of your password. There are other ways to do that than oauth.
For a centralized installation this would be pretty good. For ad-hoc installations it could be difficult to setup but I'm just guessing here.
Other options are (off the top of my head):
- Basic/Digest HTTP authentication. Digest is pretty good (or so
wikipedia tells me) and has a very nice property of not requiring any changes to the API. User identity is provided as out-of-band HTTP header.
- Custom authentication via special XML-RPC methods and some
user/pass/token/whatever arguments. This is IMHO pretty ugly as it affects the interface. The upside is that we can build any authentication options we like.
No, let's not do this.
- No authentication at all. All XML-RPC interfaces are public and anyone
may call them. Surprisingly this is not such a bad option. Since all request would be anonymous all the users would be allowed to do is upload data to public "directories". All other interaction would have to be performed via the website or the administration panel.
What clients need to implement.
So this is about everyone who has to interact with the API, currently that's just dashboard itself (we have a command-line client that can use each exposed method) and of course abrek. While I don't mind changing the API or security method to make it better we need to consider that each client will have to adapt as well. Worse, if one client has to interact with two servers it may be required to discover/negotiate supported authentication before we can even connect.
That's not such a big issue in my eyes. launch-control should just have one method, at least for now.
What's missing before we can sensibly discuss this is a discussion of how we would like the user interaction to work.
Do we want to support setting up clients without a connection to the server? I don't see why as if you have no connection you can't submit results, except if:
Do we want to use the authentication token to identify a user + machine combination?
Do we want users to be able to revoke access for certain tokens (and hence clients) without revoking them all?
Basically, we need to be able to complete this paragraph as a minimum.
A new developer joins linaro and is shipped a board that they wish to submit test results for. To do this they "apt-get install abrek" and then...
Thanks,
James
First up, let me apologize for the slightly waffly nature of this email. I'm not an expert in this area, but I hope I know enough to be of some use...
On Sat, 04 Sep 2010 18:32:30 +0200, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello guys.
I'd like to ask for your advice in for implementing authentication mechanism for the XML-RPC interface to the dashboard. I believe this problem has two aspects: user needs and client program needs. I was about to implement HTTP Digest authentication that will authorize the user against the Django user database. James pointed out that we should discuss this topic more. I'd like to know what you think.
What users need to have.
I think that for our internal use cases a best option would be launchpad integration. We don't mind using launchpad, we already have accounts there. We could use oauth for authentication.
I may be talking out of my behind here, but do you really mean oauth? OAuth is more about authorization -- should this request be permitted -- and the subject of this mail talks about "authentication" -- who is this request on behalf of. The two questions are related of course, but Launchpad supports *openid*, which is a way of delegating authentication. Launchpad also supports oauth for authenticating API requests, but I don't think that's relevant to the issue at hand. AIUI launch-control would still need to implement the server end of oauth and not be able to leverage Launchpad's implementation -- I may be wrong about this though.
Using Launchpad's openid opens the door to doing *authorization* by Launchpad team membership -- users who belong to team ~X can upload results to this method.
If you really meant oauth, then it probably makes sense -- but it doesn't really handle authentication. The typical workflow would be:
1) you'd run some configuration command for abrek 2) launch-control would pop up in your browser 3) you would authenticate yourself to launch-control, any which way 4) you would click a button saying "yes, allow abrek to upload results" 5) the configuration command would receive a token that it could include in subsequent requests
I'm not an expect on oauth so I'm not sure what kind of impact it has on the API itself, if any (do we need to pass some special argument to each call?). I suspect it has a negotiation phase where the client and oauth provider exchange some messages after which the client can authenticate by providing some extra header in each request. In any way that's one option.
I think that's more or less right, but generally the exchange happens outside your application. There have been some conversations about ways to do the token generation without heading to a web browser, but I didn't really pay much attention to them.
For a centralized installation this would be pretty good. For ad-hoc installations it could be difficult to setup but I'm just guessing here.
There are simple-ish openid servers around, I think. Not many of them support launchpad's team extension though.
Other options are (off the top of my head):
- Basic/Digest HTTP authentication. Digest is pretty good (or so
wikipedia tells me) and has a very nice property of not requiring any changes to the API. User identity is provided as out-of-band HTTP header.
Yeah, this sounds fine to me, basically. Less sexy than the above suggestions of course :-) If we delegate to Launchpad for identity, then I would not be especially comfortable putting my LP password on every device that I might want to talk to launch-control.
- Custom authentication via special XML-RPC methods and some
user/pass/token/whatever arguments. This is IMHO pretty ugly as it affects the interface. The upside is that we can build any authentication options we like.
This sounds pointless and ugly. Let's use an existing standard of some kind.
- No authentication at all. All XML-RPC interfaces are public and anyone
may call them. Surprisingly this is not such a bad option. Since all request would be anonymous all the users would be allowed to do is upload data to public "directories". All other interaction would have to be performed via the website or the administration panel.
If this is a realistic option, it certainly has a certain appeal :-)
What clients need to implement.
So this is about everyone who has to interact with the API, currently that's just dashboard itself (we have a command-line client that can use each exposed method) and of course abrek. While I don't mind changing the API or security method to make it better we need to consider that each client will have to adapt as well. Worse, if one client has to interact with two servers it may be required to discover/negotiate supported authentication before we can even connect.
OAuth and OpenID intend to be standards in this area, so using one of these shouldn't be too bad in this respect.
Hope this helped.
Cheers, mwh
On Sun, 05 Sep 2010 17:25:14 +1200, Michael Hudson michael.hudson@linaro.org wrote:
I may be talking out of my behind here, but do you really mean oauth? OAuth is more about authorization -- should this request be permitted -- and the subject of this mail talks about "authentication" -- who is this request on behalf of. The two questions are related of course, but Launchpad supports *openid*, which is a way of delegating authentication. Launchpad also supports oauth for authenticating API requests, but I don't think that's relevant to the issue at hand. AIUI launch-control would still need to implement the server end of oauth and not be able to leverage Launchpad's implementation -- I may be wrong about this though.
Yes. I think that Zygmunt conflated two thoughts in that paragraph.
We have the question of identity. For the linaro instance (and maybe all), we could use openid for this. This allows us to identify a user (and presumably have their openid provider authenticate them, though there's no guarantees of that.) In particular we could support Launchpad openid.
openid is all but useless for authenticating API requests to launch-control. We need both identification and authentication here (and there will be authorization too, but that's beside the point here in my opinion, once you have identity you can do authorization anyway you like.) oauth is one candidate for this step, the client signs the requests on behalf of the user, and by checking that signature we get identification and authentication.
Using Launchpad's openid opens the door to doing *authorization* by Launchpad team membership -- users who belong to team ~X can upload results to this method.
Yes.
If you really meant oauth, then it probably makes sense -- but it doesn't really handle authentication. The typical workflow would be:
- you'd run some configuration command for abrek
- launch-control would pop up in your browser
- you would authenticate yourself to launch-control, any which way
- you would click a button saying "yes, allow abrek to upload results"
- the configuration command would receive a token that it could include in subsequent requests
In my mind oauth does handle authentication, at the request level at least, as it authenticates you to the API when you make requests.
Other options are (off the top of my head):
- Basic/Digest HTTP authentication. Digest is pretty good (or so
wikipedia tells me) and has a very nice property of not requiring any changes to the API. User identity is provided as out-of-band HTTP header.
Yeah, this sounds fine to me, basically. Less sexy than the above suggestions of course :-) If we delegate to Launchpad for identity, then I would not be especially comfortable putting my LP password on every device that I might want to talk to launch-control.
That's not really feasible anyway.
If we want HTTP digest auth, then we have to do the following:
1. Have users create an account on the system (possibly using openid against LP or any other provider, this provides them with one method to log in.) 2. Have them enter a password/generate one for them that they can discover. 3. Allow that password to be used to authenticate API requests (and possibly web requests too.)
We are not limited to a single password here, and could have e.g. one per device if we wanted.
At no point though should we ask the user for their LP/SSO password.
- Custom authentication via special XML-RPC methods and some
user/pass/token/whatever arguments. This is IMHO pretty ugly as it affects the interface. The upside is that we can build any authentication options we like.
This sounds pointless and ugly. Let's use an existing standard of some kind.
+1
[ Just so we are clear about terms, some definitions for how I am using them:
Identification - who the user is. If you don't have this then you can't tell users apart. It is possible to have a system with just identification, for instance lots of tills behind bars do this, where a server presses their name before they start entering a transaction, so all transactions are attributed a person, but can easily be spoofed.
Authentication - whether the user is who they say they are. This prevents the spoofing in the bar case. This is the model we all know and love from Unix, where you enter your username and password. There are systems where you have authentication, but not identification, which is a sort of authenticated anonymous model. Then you know the user is allowed, but can't tell the users apart. Think of the "warthogs" account on the Canonical RT. - Authentication implying identification is a bad idea. Usually our passwords will be different, so on a multi-user Ubuntu machine we could just request the password and log in whoever that matches. However, what happens if two people choose the same password? Also, it makes brute-force attacks much easier, as you could just type in "password" and will probably get logged in.
Authorization - whether the user is allowed to perform a particular task. This one is fairly obvious, can the user (perhaps anonymous, perhaps identified but not authenticated) perform whatever action they are trying to do? - Without identification then you basically have a boolean system, whether the anonymous user can do something. - Without authentication you have easy impersonation, and so the controls are more advisory than mandatory.
]
Thanks,
James
Here's my $0.02 (adjusted for inflation):
On Sat, Sep 4, 2010 at 11:32 AM, Zygmunt Krynicki < zygmunt.krynicki@linaro.org> wrote:
I think that for our internal use cases a best option would be launchpad integration. We don't mind using launchpad, we already have accounts there. We could use oauth for authentication. I'm not an expect on oauth
...
For a centralized installation this would be pretty good. For ad-hoc installations it could be difficult to setup but I'm just guessing here.
As you mention, for ad-hoc installations, this could be a problem. While
this does have a certain appeal to it, it also looks like it would take quite a bit more time and effort to implement this, with no compelling need visible at this point. We can revisit whether that is still the case down the road, and possibly support it as an option in the future, but for now I think it better to take a simpler approach.
Other options are (off the top of my head):
- Basic/Digest HTTP authentication. Digest is pretty good (or so
I think this would fit our needs just fine.
- Custom authentication via special XML-RPC methods and some
I think we all agree this is not desirable
- - No authentication at all. All XML-RPC interfaces are public and anyone
may call them. Surprisingly this is not such a bad option. Since all request would be anonymous all the users would be allowed to do is upload data to public "directories". All other interaction would have to be performed via the website or the administration panel.
I can certainly see the logic here, and discounting the possibility of a nefarious spammer flooding our server with junk, I think it's worth consideration. My concerns revolve around what it might complicate on the back end. If we want to go this route, I think the impact needs to be better defined. - With lots of possible submissions, without any kind of identification attached to them, how do I find my submission in the haystack? - You mentioned uploading to "directories", which could fix the above partially, but what prevents someone from uploading to my directory, even by accident (for instance, if they have a similar name and just happen to choose the same "path" as me) - Are results publicly automatically? If not, who has the authority to approve them? And if you restrict access to this, aren't we back to providing an authentication mechanism again, but just complicating the process by which results are posted? ... You can see my line of thinking here I hope. I'm not saying it's a bad option, just that there are some implementation details that span farther than just the auth stage that need to be clearly defined first, if we want to go in this direction.
Thanks, Paul Larson
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 07/09/10 17:23, Paul Larson wrote:
Other options are (off the top of my head):
- Basic/Digest HTTP authentication. Digest is pretty good (or so
I think this would fit our needs just fine.
I agree, and for the moment I'd like to pursue this route. It's the fastest to develop/deploy and solves an important problem - having access to secure APIs in a not-unreasonable manner (security-wise)
- No authentication at all. All XML-RPC interfaces are public and anyone
may call them. Surprisingly this is not such a bad option. Since all request would be anonymous all the users would be allowed to do is upload data to public "directories". All other interaction would have to be performed via the website or the administration panel.
I can certainly see the logic here, and discounting the possibility of a nefarious spammer flooding our server with junk, I think it's worth consideration. My concerns revolve around what it might complicate on the back end. If we want to go this route, I think the impact needs to be better defined.
It's actually already done, there is no impact on the server. Simply, all methods behave as if the user is not known. To add support for user identification (authentication) you would have to tweak with the xml_rpc_handler view and pass this down to each method as some kind of argument. The latter is surprisingly more complicated. I still did not figure out a pretty way of passing this non-parameter to the function. There are ways of doing it but they don't seem clean.
- With lots of possible submissions, without any kind of identification
attached to them, how do I find my submission in the haystack?
Well each submission gets an ID (so you know it when you submit) but it's a difficult thing to solve in a *big* stream of bundles.
- You mentioned uploading to "directories", which could fix the above
partially, but what prevents someone from uploading to my directory, even by accident (for instance, if they have a similar name and just happen to choose the same "path" as me)
I have a simple mechanism for this. You can create bundle streams which are named containers for bundles. A bundle must be in exactly one container. There are three kinds of containers: - public, they have a pathname like: /anonymous/ or /anonymous/NAME/ (where NAME is any single word that does not contain '/'). They are open for reading/writing by anyone (note that there is no API for removing bundles yet so there is no problem about who can remove something from this stream) - private, they have a pathname like /personal/USERNAME/ or /personal/USERNAME/NAME/ (NAME behaves as above). Only the owner may access the contents. - - team, like private but they belong to a group, the pathname is /team/TEAMNAME/
- Are results publicly automatically?
Read access is determined by the bundle stream and accessing user identity.
If not, who has the authority to
approve them?
You'd have to define "approve". I suspect you are interested in being able to mark some as "valid" and being included in reports. I suspect that you could do such a thing by having two streams like /something/incoming/ and /something/checked/ and moving bundles between them as you see fit. If more control is needed please tell me what use case you'd like to accommodate.
And if you restrict access to this, aren't we back to
providing an authentication mechanism again, but just complicating the process by which results are posted?
I'm not sure what's the answer to this. I think that posting results is easy and what is offered now is sufficient. If you find a use case that cannot be solved with current technology we can design a solution for that.
... You can see my line of thinking here I hope. I'm not saying it's a bad option, just that there are some implementation details that span farther than just the auth stage that need to be clearly defined first, if we want to go in this direction.
Mmm. Right, the whole how do we use this in practice thing. I think that it's going to be usable and initially we will use digest authentication and team streams where a group of trusted members can shuffle the data so that it makes sense.
Thanks Zygmunt