Hello,

One of the issues with https://android-build.linaro.org/ is that, if build fails, it's not easy to tell if it happened because of compilation error ("real" failure) or due to non-deterministic error with setting up infrastructure for build (e.g. during source checkout). The latter not so uncommon due to vast source size of Android and complexity of cloud infrastructure.

This became especially problematic with start of automated CI loop, where it leads to false negatives when doing pre-merge testing.

Improving this situation was subject of few blueprints in which Infrastructure team worked, and recently, a solution was deployed. With it, if a build fails due to non-deterministic infrastructure error, it will get status of "NOT BUILT", meaning that a build never actually got to compilation phase. Suggested action in such case is to retry.

Please note that Jenkins there is some inconsistency within the Jenkins itself regarding NOT BUILT status - "Pending" is used as a display name in places, and the same gray icon used as for ABORTED builds. So, please keep that in mind, or yet better use Build Frontend.

Unfortunately, even short weekend testing showed that error separation achieved is not ideal (folks who participated in Connect Q4.11 session dedicated to this, may remember that I said that it would take adding AI to do that properly ;-) ).

In particular, if there was an issue with manifest file (essentially, an error in Android source code), it will be reported as NOT BUILT instead of FAILED. Here's example of such build: https://android-build.linaro.org/jenkins/job/linaro-android_panda-ics-gcc44-...

Opposite miscategorization may also happen: very early infra error may be reported as FAILED instead of NOT BUILT. Example: https://android-build.linaro.org/jenkins/job/linaro-android_panda-ics-gcc46-...

So, Infrastructure team will continue to work on improving reliability of builds, but in the meantime please keep looking in the build logs for actual cause of the failure (feel free to report unexpected build status to https://bugs.launchpad.net/linaro-android-infrastructure)