Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Currently trying some other workarounds, such as adding -mno-unaligned-access to the kernel compiler flags, checking if it's already fixed on head, trying the generic arm-eabi toolchain for the kernel, etc.
ttyl bero
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Currently trying some other workarounds, such as adding -mno-unaligned-access to the kernel compiler flags, checking if it's already fixed on head, trying the generic arm-eabi toolchain for the kernel, etc.
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior? If so I think John has a JTAG that he could trace with.
We could look at both u-boot and the kernel binary compiled with 11.11 and 11.12 and see if we can spot any differences. I can look at this later today.
ttyl bero
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
We could look at both u-boot and the kernel binary compiled with 11.11 and 11.12 and see if we can spot any differences. I can look at this later today.
u-boot with 11.12 works, it's just the kernel acting up. https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui... is userland+uboot built with 11.12, kernel built w/ whatever toolchain is installed on the system.
ttyl bero
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
+1 to that. I wonder if the kernel guys have any hints for tracking this type of problem down? Early printk() or similar?
-- Michael
2011/12/18 Michael Hope michael.hope@linaro.org:
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
+1 to that. I wonder if the kernel guys have any hints for tracking this type of problem down? Early printk() or similar?
Michael, is there anything else we can do or check?
Bero,
Would you list the current status of each test build and what you may have discovered in testing each one?
-- Michael
On Mon, Dec 19, 2011 at 11:05 AM, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/18 Michael Hope michael.hope@linaro.org:
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
+1 to that. I wonder if the kernel guys have any hints for tracking this type of problem down? Early printk() or similar?
Michael, is there anything else we can do or check?
Nothing that I know of, sorry. I've looked through the commits and the most likely problem is unaligned loads and stores which could have exposed a latent bug in the kernel or compiler. The -mno-unaligned-access flag will turn this code off and should help point the finger.
-- Michael
Hi, good news, I have a build that works.
-mno-unaligned-access did the trick -- so we have a workaround for tomorrow's release. It would still be good to get some more info on the hang itself to help debug the issue.
Another interesting bit is that the (3.0.1 based) landing-panda kernel boots without needing to add -mno-unaligned-access, while the (3.1.0 based) staging-panda kernel doesn't. So the problem must be triggered by something that differs between those 2 kernels in the early bootup phase.
I'll look into it a bit more when I have some time, right now I'll add the workaround to our builds so we have something releasable tomorrow.
ttyl bero
On 18 December 2011 14:05, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/18 Michael Hope michael.hope@linaro.org:
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
+1 to that. I wonder if the kernel guys have any hints for tracking this type of problem down? Early printk() or similar?
Michael, is there anything else we can do or check?
Bero,
Would you list the current status of each test build and what you may have discovered in testing each one?
-- Michael
-- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
Bero's
https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
build has the workaround in it.
wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... bzr branch lp:linaro-image-tools ./linaro-image-tools/linaro-android-media-create --mmc /dev/sdc --dev panda --system system.tar.bz2 --userdata userdata.tar.bz2 --boot boot.tar.bz2
...to try it out.
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, good news, I have a build that works.
-mno-unaligned-access did the trick -- so we have a workaround for tomorrow's release. It would still be good to get some more info on the hang itself to help debug the issue.
Another interesting bit is that the (3.0.1 based) landing-panda kernel boots without needing to add -mno-unaligned-access, while the (3.1.0 based) staging-panda kernel doesn't. So the problem must be triggered by something that differs between those 2 kernels in the early bootup phase.
I'll look into it a bit more when I have some time, right now I'll add the workaround to our builds so we have something releasable tomorrow.
ttyl bero
On 18 December 2011 14:05, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/18 Michael Hope michael.hope@linaro.org:
2011/12/18 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 17 December 2011 06:30, Zach Pfeffer zach.pfeffer@linaro.org wrote:
2011/12/17 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
Hi, I've verified that toolchain 11.12 compiles our userland ok, the only problem is w/ the kernel. A build using the 11.12 toolchain for userland and an older toolchain for the kernel is ok.
Is is possible to create an android-build with this configuration?
Yes, I already did: https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
Those sound good. Ricardo have you compiled and booted a kernel with the generic 11.12 arm-eabi- toolchain?
Or with any other 11.12 toolchain for that matter? We currently build arm-eabi- and arm-linux-androideabi- toolchains, so arm-linux-gnueabi- is untested on our side. But that's likely what you're using. I don't think there's enough of a difference between arm-linux-androideabi- and arm-linux-gnueabi- to make the theory that one boots while the other doesn't plausible.
Maybe John Rigby can help. I saw the failure at boot, do you still see the same behavior?
Yes
If so I think John has a JTAG that he could trace with.
That would definitely be useful
+1 to that. I wonder if the kernel guys have any hints for tracking this type of problem down? Early printk() or similar?
Michael, is there anything else we can do or check?
Bero,
Would you list the current status of each test build and what you may have discovered in testing each one?
-- Michael
-- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
2011/12/19 Zach Pfeffer zach.pfeffer@linaro.org:
Bero's
https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
build has the workaround in it.
wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... bzr branch lp:linaro-image-tools ./linaro-image-tools/linaro-android-media-create --mmc /dev/sdc --dev panda --system system.tar.bz2 --userdata userdata.tar.bz2 --boot boot.tar.bz2
...to try it out.
Hmm. There's a CPU flag to say if you should trap or just do it when you see an unaligned access. I wonder if the trap is turned on during early init?
-- Michael
On 18 December 2011 14:58, Michael Hope michael.hope@linaro.org wrote:
Hmm. There's a CPU flag to say if you should trap or just do it when you see an unaligned access. I wonder if the trap is turned on during early init?
Good question... Do you know the details off the top of your head? (I don't, so nothing to grep for without googling first ;) )
ttyl bero
2011/12/19 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 18 December 2011 14:58, Michael Hope michael.hope@linaro.org wrote:
Hmm. There's a CPU flag to say if you should trap or just do it when you see an unaligned access. I wonder if the trap is turned on during early init?
Good question... Do you know the details off the top of your head? (I don't, so nothing to grep for without googling first ;) )
According to the A9 TRM, it's bit A of SCTLR "Enables strict alignment of data to detect alignment faults in data accesses".
-- Michael
On Mon, 2011-12-19 at 11:58 +1300, Michael Hope wrote:
2011/12/19 Zach Pfeffer zach.pfeffer@linaro.org:
Bero's
https://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/#bui...
build has the workaround in it.
wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... wget http://android-build.linaro.org/builds/~berolinux/panda-toolchain-test/11/ta... bzr branch lp:linaro-image-tools ./linaro-image-tools/linaro-android-media-create --mmc /dev/sdc --dev panda --system system.tar.bz2 --userdata userdata.tar.bz2 --boot boot.tar.bz2
...to try it out.
Hmm. There's a CPU flag to say if you should trap or just do it when you see an unaligned access. I wonder if the trap is turned on during early init?
I think a recent patch proposed on the linux-arm-kernel list is germane: http://comments.gmane.org/gmane.linux.ports.arm.kernel/138900
Coincidentally, I was planning today to see if I could disable CONFIG_ALIGNMENT_TRAP on vexpress as when testing the DS-5 Streamline tool I noticed it was showing about 1000 alignment faults a second, which can't be doing performance any good.
On 19 December 2011 01:33, Jon Medhurst (Tixy) tixy@linaro.org wrote:
I think a recent patch proposed on the linux-arm-kernel list is germane: http://comments.gmane.org/gmane.linux.ports.arm.kernel/138900
Coincidentally, I was planning today to see if I could disable CONFIG_ALIGNMENT_TRAP on vexpress as when testing the DS-5 Streamline tool I noticed it was showing about 1000 alignment faults a second, which can't be doing performance any good.
The panda kernel has CONFIG_ALIGNMENT_TRAP=y
ttyl bero
Adding Andy Green directly who may be interested in this.
2011/12/19 Bernhard Rosenkränzer bernhard.rosenkranzer@linaro.org:
On 19 December 2011 01:33, Jon Medhurst (Tixy) tixy@linaro.org wrote:
I think a recent patch proposed on the linux-arm-kernel list is germane: http://comments.gmane.org/gmane.linux.ports.arm.kernel/138900
Coincidentally, I was planning today to see if I could disable CONFIG_ALIGNMENT_TRAP on vexpress as when testing the DS-5 Streamline tool I noticed it was showing about 1000 alignment faults a second, which can't be doing performance any good.
The panda kernel has CONFIG_ALIGNMENT_TRAP=y
ttyl bero
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-android@lists.linaro.org
-
Bernhard Rosenkränzer -
Jon Medhurst (Tixy) -
Michael Hope -
Zach Pfeffer