Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200 minutes ?
Here's a link to the log [1].
Thanks, Mathieu.
[1]. https://android-build.linaro.org/jenkins/job/linaro-android_snowball-ics-gcc...
On Wed, May 16, 2012 at 1:28 AM, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200 minutes ?
Nothing against timeout bump, maybe we can try 170?
Any idea why we see this slowdown? Did you add something significant to recent snowball builds?
On 16 May 2012 02:27, Alexander Sack asac@linaro.org wrote:
On Wed, May 16, 2012 at 1:28 AM, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Gents,
I've had this error twice in a row now while compiling Android on
jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200
minutes ?
Nothing against timeout bump, maybe we can try 170?
Any idea why we see this slowdown? Did you add something significant to recent snowball builds?
We have added a lot of code and binaries in the vendor tarball to support multimedia on snowball.
/Patrik
-- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
Hello,
On Wed, 16 May 2012 02:27:06 +0200 Alexander Sack asac@linaro.org wrote:
On Wed, May 16, 2012 at 1:28 AM, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200 minutes ?
Nothing against timeout bump, maybe we can try 170?
Any idea why we see this slowdown? Did you add something significant to recent snowball builds?
Well, the were no response from Android folks, so I assume no changes were done on the code side. I bumped timeout to 170mins the same day, but was shown at least 2 builds still timed out. We now have more builds to do stats, and I looked into that with more detail.
This shows it very well:
https://android-build.linaro.org/jenkins/job/doanac_tracking-panda-linaro4.6...
#9 - 2h51m, timed out #10, few days later - 2h01m, among the best of our times
I looked also in build logs for details stats to see if some specific stage (like seed update) is responsible for slowdown, and no, it's just roughly proportional slowdown for all stages (more details are at https://bugs.launchpad.net/linaro-android-infrastructure/+bug/997551). So this can be attributed only to EC2 disturbances - we had "bad week" when instances underperformed, leading to timeouts. Now, it's back to normal. We have similar "bad weeks" wrt to hanged builds, where that happens twice a day vs 1-2 a week as usual.
So, I bumped timeout to 3hrs (180mins), which should account for worst cases known so far.
On 12-05-22 09:10 AM, Paul Sokolovsky wrote:
Hello,
On Wed, 16 May 2012 02:27:06 +0200 Alexander Sack asac@linaro.org wrote:
On Wed, May 16, 2012 at 1:28 AM, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200 minutes ?
Nothing against timeout bump, maybe we can try 170?
Any idea why we see this slowdown? Did you add something significant to recent snowball builds?
Well, the were no response from Android folks, so I assume no changes were done on the code side. I bumped timeout to 170mins the same day, but was shown at least 2 builds still timed out. We now have more builds to do stats, and I looked into that with more detail.
This shows it very well:
https://android-build.linaro.org/jenkins/job/doanac_tracking-panda-linaro4.6...
#9 - 2h51m, timed out #10, few days later - 2h01m, among the best of our times
I looked also in build logs for details stats to see if some specific stage (like seed update) is responsible for slowdown, and no, it's just roughly proportional slowdown for all stages (more details are at https://bugs.launchpad.net/linaro-android-infrastructure/+bug/997551). So this can be attributed only to EC2 disturbances - we had "bad week" when instances underperformed, leading to timeouts. Now, it's back to normal. We have similar "bad weeks" wrt to hanged builds, where that happens twice a day vs 1-2 a week as usual.
So, I bumped timeout to 3hrs (180mins), which should account for worst cases known so far.
Hey Paul,
Thanks for that - on my side I haven't experienced a timeout since you bumped the limit to 170mins.
Regards, Mathieu.
On Tue, 2012-05-15 at 17:28 -0600, Mathieu Poirier wrote:
Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
A Versatile Express build also had this [1].
On Tue, 15 May 2012 17:28:40 -0600 Mathieu Poirier mathieu.poirier@linaro.org wrote:
Gents,
I've had this error twice in a row now while compiling Android on jenkins:
Build timed out (after 150 minutes). Marking the build as aborted.
This is tracked as https://bugs.launchpad.net/linaro-android-infrastructure/+bug/997551 , hope to get to it soon. As Alexander suggest, I'm going to try smaller bump for now.
The thing is, the build was still alive and in the last steps of packaging the system partition. Can we increase the limit to 200 minutes ?
Here's a link to the log [1].
Thanks, Mathieu.
[1]. https://android-build.linaro.org/jenkins/job/linaro-android_snowball-ics-gcc...
linaro-android@lists.linaro.org
-
Alexander Sack -
Jon Medhurst (Tixy) -
Mathieu Poirier -
Patrik Ryd -
Paul Sokolovsky