Hello Validation team,
Proceeding further with CBuild/LAVA integration work, it's time to think how to publish gcc, etc. binary tarballs as produced by the builds. Initially it seemed like an easy step, but the more I think about it, the more it seems like complicated to do "right".
The tarballs in question are ~100Mb, so I guess there cannot be talk about publishing them as attachment to a result bundle. Let me know if that's ok though, because it would resolve all complexities below.
Otherwise, is there a way to set some custom data per user, which would be available to all jobs submitted by that user - in a secure manner? The idea, that would be an access credential to use for publishing (so far we usually use SSH privkey to auth to publishing host).
If not, is there a way to pass such information during job submission, but again in a secure manner (i.e. it shouldn't be part of job definition, as that's dumped in plain text).
Any other infrastructure in LAVA (like, pre-authed publishing host) or suggestions how to solve this?
Thanks, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On 11/27/2012 10:51 AM, Paul Sokolovsky wrote:
Hello Validation team,
Proceeding further with CBuild/LAVA integration work, it's time to think how to publish gcc, etc. binary tarballs as produced by the builds. Initially it seemed like an easy step, but the more I think about it, the more it seems like complicated to do "right".
The tarballs in question are ~100Mb, so I guess there cannot be talk about publishing them as attachment to a result bundle. Let me know if that's ok though, because it would resolve all complexities below.
The attachment approach indeed seems like a bad idea.
Otherwise, is there a way to set some custom data per user, which would be available to all jobs submitted by that user - in a secure manner? The idea, that would be an access credential to use for publishing (so far we usually use SSH privkey to auth to publishing host).
If not, is there a way to pass such information during job submission, but again in a secure manner (i.e. it shouldn't be part of job definition, as that's dumped in plain text).
Any other infrastructure in LAVA (like, pre-authed publishing host) or suggestions how to solve this?
We currently have an SSH key in place to pull restricted builds from mombin. My idea after thinking on this for 60 seconds is that we create a new "publish" action for our jobs. This publish option would then push these files somewhere. In our validation lab specific setup we could publish somewhere on mombin using our current ssh key.
Does this sound sensible?
У уто, 27. 11 2012. у 11:25 -0600, Andy Doan пише:
We currently have an SSH key in place to pull restricted builds from mombin. My idea after thinking on this for 60 seconds is that we create a new "publish" action for our jobs. This publish option would then push these files somewhere. In our validation lab specific setup we could publish somewhere on mombin using our current ssh key.
Does this sound sensible?
So, the push would happen from the LAVA server node, not from individual boards? IOW, this means the SSH key would not show up on anything other than the LAVA master, thus being unlikely for it to be exported.
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
Cheers, Danilo
On 11/27/2012 12:17 PM, Danilo Šegan wrote:
У уто, 27. 11 2012. у 11:25 -0600, Andy Doan пише:
We currently have an SSH key in place to pull restricted builds from mombin. My idea after thinking on this for 60 seconds is that we create a new "publish" action for our jobs. This publish option would then push these files somewhere. In our validation lab specific setup we could publish somewhere on mombin using our current ssh key.
Does this sound sensible?
So, the push would happen from the LAVA server node, not from individual boards? IOW, this means the SSH key would not show up on anything other than the LAVA master, thus being unlikely for it to be exported.
yes, we have ways of transferring a file from target->host in our dispatcher. We could use that so that our private key only has to live on our actual server(s).
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
So you are saying we might need a new "publishing key". That seems fine, just a slightly different config option for our setup I'd think.
Heya Andy,
У уто, 27. 11 2012. у 12:24 -0600, Andy Doan пише:
yes, we have ways of transferring a file from target->host in our dispatcher. We could use that so that our private key only has to live on our actual server(s).
Cool.
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
So you are saying we might need a new "publishing key". That seems fine, just a slightly different config option for our setup I'd think.
Yeah, mostly for the sanity (and symmetry) of our existing set up. The way it's currently set up is that our publishing framework accepts SSH connections with very limited permissions:
- upload step which only allows sftp-ing to /srv/snapshots.linaro.org/uploads which is not publicly accessible - trigger step which reshuffles the files into appropriate location (restricted to running publish_to_snapshots.py script)
We use separate user accounts on mombin with two different SSH keys (this was requested by IS so they could limit possible actions for these passphrase-less SSH keys). Two-step process also ensures we do not offer incomplete files for download and allows us to do some pre-processing before publishing (eg. since jenkins publishes our username_buildname combo into a single directory, we split that into username/buildname in this step).
Full docs on the publishing setup currently are at
http://bazaar.launchpad.net/~linaro-infrastructure/linaro-license-protection...
(might be slightly out of date regarding paths on the system for scripts that are shared between users)
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Cheers, Danilo
Danilo Šegan danilo.segan@linaro.org writes:
Heya Andy,
У уто, 27. 11 2012. у 12:24 -0600, Andy Doan пише:
yes, we have ways of transferring a file from target->host in our dispatcher. We could use that so that our private key only has to live on our actual server(s).
Cool.
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
So you are saying we might need a new "publishing key". That seems fine, just a slightly different config option for our setup I'd think.
Yeah, mostly for the sanity (and symmetry) of our existing set up. The way it's currently set up is that our publishing framework accepts SSH connections with very limited permissions:
- upload step which only allows sftp-ing
to /srv/snapshots.linaro.org/uploads which is not publicly accessible
- trigger step which reshuffles the files into appropriate location
(restricted to running publish_to_snapshots.py script)
[..]
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Sounds like all the more reason to keep the job file at the fairly abstract "publish" level then.
Cheers, mwh
У сре, 28. 11 2012. у 08:31 +1300, Michael Hudson-Doyle пише:
Sounds like all the more reason to keep the job file at the fairly abstract "publish" level then.
Indeed. I suppose we'd like to get suggestions from your side on the design and implementation tips on how to best do this: encapsulate the publishing knowledge in the lava-server (I suppose) and extend jobs to support abstract 'publish this-and-that' step.
Cheers, Danilo
Hello,
On Wed, 28 Nov 2012 08:31:41 +1300 Michael Hudson-Doyle michael.hudson@linaro.org wrote:
[]
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Sounds like all the more reason to keep the job file at the fairly abstract "publish" level then.
Yes, I also fully agree with this idea. And it also clear that we need to support different methods of publishing - test/devel and production, old and new, etc. So, publish section in job JSON def would accept identifier of publishing method, like "method"/"location"/"destination", whatever:
{ "command": "publish", "parameters": { "location": "snapshots", "pattern": "build/product/*/out/*.tar.*" } }
That location key also shouldn't be treated as just a hostname, but rather as identifier of publisher config, which can be for example a subdirectory like:
publisher/<id>/: publish - this gets called by lava with the list of files to do actual publishing ssh_key - a file used by publish script, can be any other files as needed
Of course, the above is just a guess how it can be done, please use whatever suits LAVA paradigm well, but it would be nice to get "devel" publishing to mombin with existing LAVA setup soon, and "production" publishing later, so some way of abstracting publisher support is definitely needed. Please let us now if Infrastructure can help with org side of this, like submitting a ticket or blueprint for this.
Cheers, mwh
Thanks, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On 11/29/2012 04:34 AM, Paul Sokolovsky wrote:
Hello,
On Wed, 28 Nov 2012 08:31:41 +1300 Michael Hudson-Doyle michael.hudson@linaro.org wrote:
[]
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Sounds like all the more reason to keep the job file at the fairly abstract "publish" level then.
Yes, I also fully agree with this idea. And it also clear that we need to support different methods of publishing - test/devel and production, old and new, etc. So, publish section in job JSON def would accept identifier of publishing method, like "method"/"location"/"destination", whatever:
{ "command": "publish", "parameters": { "location": "snapshots", "pattern": "build/product/*/out/*.tar.*" } }
Seems about right - without me *really* thinking into the issue :)
That location key also shouldn't be treated as just a hostname, but rather as identifier of publisher config, which can be for example a subdirectory like:
publisher/<id>/: publish - this gets called by lava with the list of files to do actual publishing ssh_key - a file used by publish script, can be any other files as needed
I'm not a huge fan of this, but I don't really have a better suggestion to offer.
Of course, the above is just a guess how it can be done, please use whatever suits LAVA paradigm well, but it would be nice to get "devel" publishing to mombin with existing LAVA setup soon, and "production" publishing later, so some way of abstracting publisher support is definitely needed. Please let us now if Infrastructure can help with org side of this, like submitting a ticket or blueprint for this.
I think you are on the right track. I think the best way you could help here, would be to submit a patch for this.
Andy Doan andy.doan@linaro.org writes:
On 11/29/2012 04:34 AM, Paul Sokolovsky wrote:
Hello,
On Wed, 28 Nov 2012 08:31:41 +1300 Michael Hudson-Doyle michael.hudson@linaro.org wrote:
[]
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Sounds like all the more reason to keep the job file at the fairly abstract "publish" level then.
Yes, I also fully agree with this idea. And it also clear that we need to support different methods of publishing - test/devel and production, old and new, etc. So, publish section in job JSON def would accept identifier of publishing method, like "method"/"location"/"destination", whatever:
{ "command": "publish", "parameters": { "location": "snapshots", "pattern": "build/product/*/out/*.tar.*" } }
Seems about right - without me *really* thinking into the issue :)
That location key also shouldn't be treated as just a hostname, but rather as identifier of publisher config, which can be for example a subdirectory like:
publisher/<id>/: publish - this gets called by lava with the list of files to do actual publishing ssh_key - a file used by publish script, can be any other files as needed
I'm not a huge fan of this, but I don't really have a better suggestion to offer.
Yeah, I think this seems OK to me.
Cheers, mwh
Of course, the above is just a guess how it can be done, please use whatever suits LAVA paradigm well, but it would be nice to get "devel" publishing to mombin with existing LAVA setup soon, and "production" publishing later, so some way of abstracting publisher support is definitely needed. Please let us now if Infrastructure can help with org side of this, like submitting a ticket or blueprint for this.
I think you are on the right track. I think the best way you could help here, would be to submit a patch for this.
Hello Andy,
On Fri, 30 Nov 2012 11:24:13 -0600 Andy Doan andy.doan@linaro.org wrote:
[]
Of course, the above is just a guess how it can be done, please use whatever suits LAVA paradigm well, but it would be nice to get "devel" publishing to mombin with existing LAVA setup soon, and "production" publishing later, so some way of abstracting publisher support is definitely needed. Please let us now if Infrastructure can help with org side of this, like submitting a ticket or blueprint for this.
I think you are on the right track. I think the best way you could help here, would be to submit a patch for this.
Ok, I got the hint, filed it as https://bugs.launchpad.net/lava-dispatcher/+bug/1086753 so far to not be forgotten, will see when I can get to it.
Hello,
On Tue, 27 Nov 2012 20:15:55 +0100 Danilo Šegan danilo.segan@linaro.org wrote:
Heya Andy,
У уто, 27. 11 2012. у 12:24 -0600, Andy Doan пише:
yes, we have ways of transferring a file from target->host in our dispatcher. We could use that so that our private key only has to live on our actual server(s).
Cool.
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
So you are saying we might need a new "publishing key". That seems fine, just a slightly different config option for our setup I'd think.
Yeah, mostly for the sanity (and symmetry) of our existing set up. The way it's currently set up is that our publishing framework accepts SSH connections with very limited permissions:
- upload step which only allows sftp-ing
to /srv/snapshots.linaro.org/uploads which is not publicly accessible
- trigger step which reshuffles the files into appropriate location
(restricted to running publish_to_snapshots.py script)
We use separate user accounts on mombin with two different SSH keys (this was requested by IS so they could limit possible actions for these passphrase-less SSH keys).
It should be also added that these SSH keys for additional security allow login only from a specific IP address. So, we indeed would need to publish from LAVA master, not directly from target boards (we have the same thing in Jenkins).
Two-step process also ensures we do not offer incomplete files for download and allows us to do some pre-processing before publishing (eg. since jenkins publishes our username_buildname combo into a single directory, we split that into username/buildname in this step).
Full docs on the publishing setup currently are at
http://bazaar.launchpad.net/~linaro-infrastructure/linaro-license-protection...
(might be slightly out of date regarding paths on the system for scripts that are shared between users)
I am hoping we can find some time to improve this with an API on https://snapshots.linaro.org that would be authenticated directly, but we can't make promises on when that's going to be around.
Cheers, Danilo
Paul Sokolovsky paul.sokolovsky@linaro.org writes:
Hello,
On Tue, 27 Nov 2012 20:15:55 +0100 Danilo Šegan danilo.segan@linaro.org wrote:
Heya Andy,
У уто, 27. 11 2012. у 12:24 -0600, Andy Doan пише:
yes, we have ways of transferring a file from target->host in our dispatcher. We could use that so that our private key only has to live on our actual server(s).
Cool.
Since we'd like to switch to API-based publishing as well, I suppose that means we could also have a key stored in the database for pushing stuff over: does that make sense?
So you are saying we might need a new "publishing key". That seems fine, just a slightly different config option for our setup I'd think.
Yeah, mostly for the sanity (and symmetry) of our existing set up. The way it's currently set up is that our publishing framework accepts SSH connections with very limited permissions:
- upload step which only allows sftp-ing
to /srv/snapshots.linaro.org/uploads which is not publicly accessible
- trigger step which reshuffles the files into appropriate location
(restricted to running publish_to_snapshots.py script)
We use separate user accounts on mombin with two different SSH keys (this was requested by IS so they could limit possible actions for these passphrase-less SSH keys).
It should be also added that these SSH keys for additional security allow login only from a specific IP address. So, we indeed would need to publish from LAVA master, not directly from target boards (we have the same thing in Jenkins).
While I agree publishing from the master makes sense, the conclusion you draw here is not valid -- all requests from the lab appear to come from the same IP address by the wonders of NAT. We have 5 IP addresses from our ISP but currently only use one.
Cheers, mwh
linaro-validation@lists.linaro.org
-
Andy Doan -
Danilo Šegan -
Michael Hudson-Doyle -
Paul Sokolovsky