I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
--john
On Wed, May 23, 2012 at 4:17 AM, John Rigby john.rigby@linaro.org wrote:
I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
Yes, having a precloned repository on master would help the faster builds. CI Maintainers job https://ci.linaro.org/jenkins/view/Linux%20Maintainers/job/linux-maintainers... makes use of this. We have a clone available on master @ http://ci.linaro.org/kernel_git_repo/kernel/linux.git. Please let me know your requirement so that I can make the improvements further if required. Right now you cannot access http://ci.linaro.org/kernel_git_repo/kernel/linux.git as it needs apache restart and I cannot do that instantly as there are jobs running on jenkins . I will fix it as soon as the jenkins have no further jobs running.
--john
On Wed, May 23, 2012 at 8:37 AM, Deepti Kalakeri deepti.kalakeri@linaro.org wrote:
On Wed, May 23, 2012 at 4:17 AM, John Rigby john.rigby@linaro.org wrote:
I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
Yes, having a precloned repository on master would help the faster builds. CI Maintainers job https://ci.linaro.org/jenkins/view/Linux%20Maintainers/job/linux-maintainers... already makes use of this. We have a clone available on master @ http://ci.linaro.org/kernel_git_repo/kernel/linux.git. Please let me know your requirement so that I can make the improvements further if required. Right now you cannot access http://ci.linaro.org/kernel_git_repo/kernel/linux.git%C2%A0 as it needs apache restart and I cannot do that instantly as there are jobs running on jenkins . I will fix it as soon as the jenkins have no further jobs running.
--john
remember that our http: proxy is set up in a way that it should not make much of a diff...
The thing is that we have to transfer a complete linux tree to the slave node no matter what.
Whether you --reference something on master or use the master hosted squid shouldn't make any significant net difference.
So bottom line: I don't think you will win much, but I am happy to be proofen wrong.
Lets discuss this next week. I understand your point about the squid proxy.
On Wed, May 23, 2012 at 12:42 AM, Alexander Sack asac@linaro.org wrote:
On Wed, May 23, 2012 at 8:37 AM, Deepti Kalakeri deepti.kalakeri@linaro.org wrote:
On Wed, May 23, 2012 at 4:17 AM, John Rigby john.rigby@linaro.org wrote:
I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
Yes, having a precloned repository on master would help the faster builds. CI Maintainers job https://ci.linaro.org/jenkins/view/Linux%20Maintainers/job/linux-maintainers... already makes use of this. We have a clone available on master @ http://ci.linaro.org/kernel_git_repo/kernel/linux.git. Please let me know your requirement so that I can make the improvements further if required. Right now you cannot access http://ci.linaro.org/kernel_git_repo/kernel/linux.git%C2%A0 as it needs apache restart and I cannot do that instantly as there are jobs running on jenkins . I will fix it as soon as the jenkins have no further jobs running.
--john
remember that our http: proxy is set up in a way that it should not make much of a diff...
The thing is that we have to transfer a complete linux tree to the slave node no matter what.
Whether you --reference something on master or use the master hosted squid shouldn't make any significant net difference.
So bottom line: I don't think you will win much, but I am happy to be proofen wrong.
-- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On Wed, May 23, 2012 at 5:03 PM, John Rigby john.rigby@linaro.org wrote:
Lets discuss this next week. I understand your point about the squid proxy.
Sounds good. Is there a particular session where this could fit in?
I think there is a kernel ci session.
On Wed, May 23, 2012 at 10:09 AM, Alexander Sack asac@linaro.org wrote:
On Wed, May 23, 2012 at 5:03 PM, John Rigby john.rigby@linaro.org wrote:
Lets discuss this next week. I understand your point about the squid proxy.
Sounds good. Is there a particular session where this could fit in?
-- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
http://summit.linaro.org/lcq2-12/meeting/20856/ubuntu-leb-review-current-ci-...
is not specific to kernel ci but seems to be the best match
On Wed, May 23, 2012 at 10:41 AM, John Rigby john.rigby@linaro.org wrote:
I think there is a kernel ci session.
On Wed, May 23, 2012 at 10:09 AM, Alexander Sack asac@linaro.org wrote:
On Wed, May 23, 2012 at 5:03 PM, John Rigby john.rigby@linaro.org wrote:
Lets discuss this next week. I understand your point about the squid proxy.
Sounds good. Is there a particular session where this could fit in?
-- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
Alexander Sack asac@linaro.org writes:
On Wed, May 23, 2012 at 8:37 AM, Deepti Kalakeri deepti.kalakeri@linaro.org wrote:
On Wed, May 23, 2012 at 4:17 AM, John Rigby john.rigby@linaro.org wrote:
I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
Yes, having a precloned repository on master would help the faster builds. CI Maintainers job https://ci.linaro.org/jenkins/view/Linux%20Maintainers/job/linux-maintainers... already makes use of this. We have a clone available on master @ http://ci.linaro.org/kernel_git_repo/kernel/linux.git. Please let me know your requirement so that I can make the improvements further if required. Right now you cannot access http://ci.linaro.org/kernel_git_repo/kernel/linux.git as it needs apache restart and I cannot do that instantly as there are jobs running on jenkins . I will fix it as soon as the jenkins have no further jobs running.
--john
remember that our http: proxy is set up in a way that it should not make much of a diff...
The thing is that we have to transfer a complete linux tree to the slave node no matter what.
One could make an AMI that contained some version of Linus tip that you could use with --reference. It's a bunch of work though, and larger AMIs are slower to boot so I don't know if it's worth persuing.
Cheers, mwh
On Wed, May 23, 2012 at 11:30 PM, Michael Hudson-Doyle michael.hudson@linaro.org wrote:
Alexander Sack asac@linaro.org writes:
On Wed, May 23, 2012 at 8:37 AM, Deepti Kalakeri deepti.kalakeri@linaro.org wrote:
On Wed, May 23, 2012 at 4:17 AM, John Rigby john.rigby@linaro.org wrote:
I believe from experience on a local host that having a precloned tree of for example current upstream linus on jenkins to use with --reference could be a huge win. We can discuss this in the kernel-ci session at connect.
Yes, having a precloned repository on master would help the faster builds. CI Maintainers job https://ci.linaro.org/jenkins/view/Linux%20Maintainers/job/linux-maintainers... already makes use of this. We have a clone available on master @ http://ci.linaro.org/kernel_git_repo/kernel/linux.git. Please let me know your requirement so that I can make the improvements further if required. Right now you cannot access http://ci.linaro.org/kernel_git_repo/kernel/linux.git as it needs apache restart and I cannot do that instantly as there are jobs running on jenkins . I will fix it as soon as the jenkins have no further jobs running.
--john
remember that our http: proxy is set up in a way that it should not make much of a diff...
The thing is that we have to transfer a complete linux tree to the slave node no matter what.
One could make an AMI that contained some version of Linus tip that you could use with --reference. It's a bunch of work though, and larger AMIs are slower to boot so I don't know if it's worth persuing.
Ack. I sent the same idea to Danilo etc. outside of the thread.
I believe it makes sense, especially since we found that custom AMIs will be helpful and probably useful for other issues.
Hello,
On Wed, 23 May 2012 08:42:59 +0200 Alexander Sack asac@linaro.org wrote:
[]
remember that our http: proxy is set up in a way that it should not make much of a diff...
The thing is that we have to transfer a complete linux tree to the slave node no matter what.
Whether you --reference something on master or use the master hosted squid shouldn't make any significant net difference.
Speaking of squid, we have problems with it starting up after a reboot, I filed https://bugs.launchpad.net/linaro-ci/+bug/1003835 for this.
And well, we probably should do more formal testing of it - when I looked at it during ci.linaro.org migration to a bigger machine, I didn't see stellar hit rate.
So bottom line: I don't think you will win much, but I am happy to be proofen wrong.
-
Alexander Sack -
Deepti Kalakeri -
John Rigby -
Michael Hudson-Doyle -
Paul Sokolovsky