Hi Fathi,
Given the results of last week's weekly testing, do you think we should now flip the switch to move to using the images built by ubuntu-build.linaro.org (offspring.linaro.org as was)?
If not, what are the blockers to doing so?
Thanks,
James
Hi,
I am compiling 11.05 release linux kernel using linaro gcc-4.5.1.
Getting following errors:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- -j 8 CHK include/linux/version.h CHK include/generated/utsrelease.h make[1]: `include/generated/mach-types.h' is up to date. CALL scripts/checksyscalls.sh CHK include/generated/compile.h Building modules, stage 2. Kernel: arch/arm/boot/Image is ready SHIPPED arch/arm/boot/compressed/lib1funcs.S AS arch/arm/boot/compressed/lib1funcs.o MODPOST 1736 modules LD arch/arm/boot/compressed/vmlinux OBJCOPY arch/arm/boot/zImage Kernel: arch/arm/boot/zImage is ready ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8974.ko] undefined! ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8940.ko] undefined! ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8510.ko] undefined! WARNING: modpost: Found 2 section mismatch(es). To see full details build your kernel with: 'make CONFIG_DEBUG_SECTION_MISMATCH=y' make[1]: *** [__modpost] Error 1 make: *** [modules] Error 2
Division symbols seems missing.
On Tue, Jun 21, 2011 at 4:31 PM, Amit Mahajan amit.mahajan@b-labs.com wrote:
Hi,
I am compiling 11.05 release linux kernel using linaro gcc-4.5.1.
Getting following errors:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- -j 8 CHK include/linux/version.h CHK include/generated/utsrelease.h make[1]: `include/generated/mach-types.h' is up to date. CALL scripts/checksyscalls.sh CHK include/generated/compile.h Building modules, stage 2. Kernel: arch/arm/boot/Image is ready SHIPPED arch/arm/boot/compressed/lib1funcs.S AS arch/arm/boot/compressed/lib1funcs.o MODPOST 1736 modules LD arch/arm/boot/compressed/vmlinux OBJCOPY arch/arm/boot/zImage Kernel: arch/arm/boot/zImage is ready ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8974.ko] undefined! ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8940.ko] undefined! ERROR: "__aeabi_uldivmod" [sound/soc/codecs/snd-soc-wm8510.ko] undefined! WARNING: modpost: Found 2 section mismatch(es). To see full details build your kernel with: 'make CONFIG_DEBUG_SECTION_MISMATCH=y' make[1]: *** [__modpost] Error 1 make: *** [modules] Error 2
Division symbols seems missing.
Hi Amit. This is a known issue with the compiler. See LP: #771832 at: https://bugs.launchpad.net/gcc-linaro/+bug/771832
The problem exists in all 4.6 based compilers. The work around is to change the kernel configuration and disable Device Drivers -> Sound card support -> Advanced Linux Sound Architecture -> ALSA for SoC audio support -> Build all ASoC CODEC drivers.
-- Michael
Hi,
On 20 June 2011 16:10, James Westby james.westby@linaro.org wrote:
Hi Fathi,
Given the results of last week's weekly testing, do you think we should now flip the switch to move to using the images built by ubuntu-build.linaro.org (offspring.linaro.org as was)?
I tend to say -1.
If not, what are the blockers to doing so?
Today, the builders are unstable. It requires: 1. check the build status everyday as the notifications are broken. 2. manual intervention to reboot the builders when they're stuck (only IS can do it). 3. trigger manually a rebuild.
I can live with this manual steps in the short term but it should be fixed ASAP. It isn't a hard blocker as we have a (manual) workaround. Any other opinion?
Cheers,
Fathi
On Tue, Jun 21, 2011 at 6:04 AM, Fathi Boudra fathi.boudra@linaro.org wrote:
Hi,
On 20 June 2011 16:10, James Westby james.westby@linaro.org wrote:
Hi Fathi,
Given the results of last week's weekly testing, do you think we should now flip the switch to move to using the images built by ubuntu-build.linaro.org (offspring.linaro.org as was)?
I tend to say -1.
If not, what are the blockers to doing so?
Today, the builders are unstable. It requires:
- check the build status everyday as the notifications are broken.
- manual intervention to reboot the builders when they're stuck (only
IS can do it). 3. trigger manually a rebuild.
I can live with this manual steps in the short term but it should be fixed ASAP. It isn't a hard blocker as we have a (manual) workaround. Any other opinion?
Are we expecting to use our Offspring instance to make the 11.06 release? If so, I believe this will end up as a blocker, as the builders are not reliable at all atm.
Cheers,
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable. It requires:
- check the build status everyday as the notifications are broken.
Notifications are now working.
I can live with this manual steps in the short term but it should be fixed ASAP. It isn't a hard blocker as we have a (manual) workaround. Any other opinion?
IS are looking in to this problem. It is not confined to our slaves (at least they have found plenty of problems with unstable ARM boards.) The only info I have currently on it is the following kern.log snippet:
Jun 13 23:10:01 miraclefruit kernel: [67719.667022] Unhandled fault: external abort on non-linefetch (0x1018) at 0x40000000 Jun 14 23:10:01 miraclefruit kernel: [23044.748687] Unhandled fault: external abort on non-linefetch (0x1018) at 0x40313000 Jun 15 10:39:07 miraclefruit kernel: [64388.888458] usb 1-2.3: reset high speed USB device using ehci-omap and address 4
I've heard a rumour that the problems are related to the USB subsytem and the USB hard disks, but nothing more concrete than that. They are running beagleXMs for what it is worth.
I'll update as I get more information.
Thanks,
James
On Tue, Jun 21, 2011 at 10:56 PM, James Westby james.westby@linaro.org wrote:
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable. It requires:
- check the build status everyday as the notifications are broken.
Notifications are now working.
I can live with this manual steps in the short term but it should be fixed ASAP. It isn't a hard blocker as we have a (manual) workaround. Any other opinion?
IS are looking in to this problem. It is not confined to our slaves (at least they have found plenty of problems with unstable ARM boards.) The only info I have currently on it is the following kern.log snippet:
Jun 13 23:10:01 miraclefruit kernel: [67719.667022] Unhandled fault: external abort on non-linefetch (0x1018) at 0x40000000 Jun 14 23:10:01 miraclefruit kernel: [23044.748687] Unhandled fault: external abort on non-linefetch (0x1018) at 0x40313000 Jun 15 10:39:07 miraclefruit kernel: [64388.888458] usb 1-2.3: reset high speed USB device using ehci-omap and address 4
The unhandled fault shouldn't be related with the usb errors. From my experience with Beagle xM the USB port will trigger the reset once the disk is consuming more power than beagle can provide. As I know this is probably a powered USB disk, it'd be good to check the same hardware on another beagle to see if we're able to reproduce the problem, as in the end it could be just a normal hardware failure.
I've heard a rumour that the problems are related to the USB subsytem and the USB hard disks, but nothing more concrete than that. They are running beagleXMs for what it is worth.
Do you know why we're using beagle xM and not panda for the builders? It'd also be good to have the correct board revision ID to know if the xM is a prototype or a final release (A, A2, B or even C now).
Cheers,
On Wed, 22 Jun 2011 00:16:27 -0300, Ricardo Salveti ricardo.salveti@linaro.org wrote:
The unhandled fault shouldn't be related with the usb errors. From my experience with Beagle xM the USB port will trigger the reset once the disk is consuming more power than beagle can provide. As I know this is probably a powered USB disk, it'd be good to check the same hardware on another beagle to see if we're able to reproduce the problem, as in the end it could be just a normal hardware failure.
I believe we have seen this on more than one of our builders (they have identical setups as far as I know). I think it has been confined mainly to one or two builders as they are the most frequently used.
Would you still recommend a particular test here?
I've heard a rumour that the problems are related to the USB subsytem and the USB hard disks, but nothing more concrete than that. They are running beagleXMs for what it is worth.
Do you know why we're using beagle xM and not panda for the builders?
Pandas weren't readily available when we set this up.
It'd also be good to have the correct board revision ID to know if the xM is a prototype or a final release (A, A2, B or even C now).
I'll ask.
Thanks,
James
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable.
They have just had new hard disk enclosures added, and should be more stable now. Let's monitor for a few days and see if they are more stable. On what day should we re-assess the situation and make another go/no-go call?
Thanks,
James
On Wed, Jun 22, 2011 at 11:16 PM, James Westby james.westby@linaro.org wrote:
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable.
They have just had new hard disk enclosures added, and should be more stable now. Let's monitor for a few days and see if they are more stable. On what day should we re-assess the situation and make another go/no-go call?
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
On 23 June 2011 10:06, Alexander Sack asac@linaro.org wrote:
On Wed, Jun 22, 2011 at 11:16 PM, James Westby james.westby@linaro.org wrote:
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable.
They have just had new hard disk enclosures added, and should be more stable now. Let's monitor for a few days and see if they are more stable. On what day should we re-assess the situation and make another go/no-go call?
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
we can stress the system by increasing the build cadence (temporary modify the daily build cron to trigger a build every 3h).
On Thu, Jun 23, 2011 at 12:44 PM, Fathi Boudra fathi.boudra@linaro.org wrote:
On 23 June 2011 10:06, Alexander Sack asac@linaro.org wrote:
On Wed, Jun 22, 2011 at 11:16 PM, James Westby james.westby@linaro.org wrote:
On Tue, 21 Jun 2011 09:04:41 +0000, Fathi Boudra fathi.boudra@linaro.org wrote:
If not, what are the blockers to doing so?
Today, the builders are unstable.
They have just had new hard disk enclosures added, and should be more stable now. Let's monitor for a few days and see if they are more stable. On what day should we re-assess the situation and make another go/no-go call?
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
we can stress the system by increasing the build cadence (temporary modify the daily build cron to trigger a build every 3h).
couldn't we inject a big amount of test builds in the queue to achieve full utilization for 24 or 48 hours or so? of course, queuing them as low priority so they don't make our "real" builds the cycles. Lets see what james says ;).
On Thu, 23 Jun 2011 14:28:42 +0200, Alexander Sack asac@linaro.org wrote:
couldn't we inject a big amount of test builds in the queue to achieve full utilization for 24 or 48 hours or so? of course, queuing them as low priority so they don't make our "real" builds the cycles. Lets see what james says ;).
We can do that, but it's easier for us to just let cron do it.
There's no facility that I know of for priority in the build queue.
We should be careful not to fill up the disk space by trying to do too many builds at once.
Fathi, have you got anywhere on defining the build retention policy? If so we could implement that at the same time to free up some disk space.
Thanks,
James
On Thu, 23 Jun 2011 12:06:46 +0200, Alexander Sack asac@linaro.org wrote:
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
We didn't add any extra stress, but all everything has been built twice now with no builder issues. Do we have the confidence to switch now?
Thanks,
James
On Fri, Jun 24, 2011 at 11:19 AM, James Westby james.westby@linaro.org wrote:
On Thu, 23 Jun 2011 12:06:46 +0200, Alexander Sack asac@linaro.org wrote:
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
We didn't add any extra stress, but all everything has been built twice now with no builder issues. Do we have the confidence to switch now?
I've being following the builds and even requesting some more extra ones, and all seems to be working fine at the moment.
Guess it'll be OK for the switch and release, unless something else breaks and burn :-)
Cheers,
On 24 June 2011 17:19, James Westby james.westby@linaro.org wrote:
On Thu, 23 Jun 2011 12:06:46 +0200, Alexander Sack asac@linaro.org wrote:
can we somehow smoke test the infrastructure continuously till tomorrow and then reassess tomorrow afternoon?
We didn't add any extra stress, but all everything has been built twice now with no builder issues. Do we have the confidence to switch now?
Yes. 3 consecutive days without any glitches. +1 on my side.
Cheers,
Fathi
On Sun, 26 Jun 2011 11:06:08 +0300, Fathi Boudra fathi.boudra@linaro.org wrote:
Yes. 3 consecutive days without any glitches. +1 on my side.
Fathi,
Do you want to switch this week? (I can't tell you exactly when we can switch as it relies on the availability of an IS sysadmin to make the changes.)
Were you planning on taking the 11.06 release from ubuntu-build images or the OEM-built images?
Thanks,
James
On 27 June 2011 19:40, James Westby james.westby@linaro.org wrote:
On Sun, 26 Jun 2011 11:06:08 +0300, Fathi Boudra fathi.boudra@linaro.org wrote:
Yes. 3 consecutive days without any glitches. +1 on my side.
Fathi,
Do you want to switch this week? (I can't tell you exactly when we can switch as it relies on the availability of an IS sysadmin to make the changes.)
asap, this week or next week.
Were you planning on taking the 11.06 release from ubuntu-build images or the OEM-built images?
from ubuntu-build.linaro.org
Cheers,
Fathi
-
Alexander Sack -
Amit Mahajan -
Fathi Boudra -
James Westby -
Michael Hope -
Ricardo Salveti