Hey folks,
Continuing on from the discussions at LC last week, here are my two patches. They work OK for me in testing on armhf and armel builds here, so I hope they are helpful for other people.
Background: in bi-arch and multi-arch configuration, soft-float and hard-float libraries do not play well together at all:
* ldconfig (currently) does not distinguish between the two ABIs when scanning libraries and putting their details in ld.so.cache, so if you have libs of both ABIs installed on your system then there is no guarantee that the right one will be used when needed.
* ld.so (currently) cannot verify at run-time whether a given lib uses the hard-float or soft-float ABI, which can lead to similar problems.
I previously had just a patch against ld.so for this, causing it to fail to load a library when the wrong ABI was detected. However, more testing has shown that this is not a sufficient solution. ld.so trusts the contents of the cache, so I've had to add logic into ldconfig itself and add extra information in the cache to describe the ABI of a library. There's already similar code in there to recognise 32-bit libs on a 64-bit system, so I've extended things in what I hope is a clean way. Unfortunately, there are also paths through ld.so that *don't* use the cache anyway so we still need the checks in there as well. If the wrong lib is loaded by mistake, return ENOENT so that a fallback is attempted.
For now, there's a fair amount of duplication in the code in each patch. I'd be happier to see it cleaned up and shared better, but my initial efforts in that direction have been thwarted by the internals of eglibc - the (very minimal) infrastructure available in ld.so is very different to that in ldconfig (an entirely normal and boring C application). If anybody fancies trying that cleanup, please feel free... :-)
As the list drops attachments, here are links to the two patches:
http://people.linaro.org/~stevemcintyre/armhf/unsubmitted-ldconfig-cache-abi... http://people.linaro.org/~stevemcintyre/armhf/unsubmitted-ldso-abi-check.dif...
Cheers,
On Thu, Nov 10, 2011 at 05:33:52PM +0000, Steve McIntyre wrote:
Hey folks,
Continuing on from the discussions at LC last week, here are my two patches. They work OK for me in testing on armhf and armel builds here, so I hope they are helpful for other people.
Background: in bi-arch and multi-arch configuration, soft-float and hard-float libraries do not play well together at all:
ldconfig (currently) does not distinguish between the two ABIs when scanning libraries and putting their details in ld.so.cache, so if you have libs of both ABIs installed on your system then there is no guarantee that the right one will be used when needed.
ld.so (currently) cannot verify at run-time whether a given lib uses the hard-float or soft-float ABI, which can lead to similar problems.
Talking with doko on IRC, it's become clear that the second patch needs extending a little more to allow ldd to work on a mixed/multi-arch system. Here's the addition:
steve@panda-smcintyre:~/linker/eglibc-2.13/elf$ diff -u dl-load.c~ dl-load.c --- dl-load.c~ 2011-11-23 15:53:24.341835279 +0000 +++ dl-load.c 2011-11-23 16:21:27.005896511 +0000 @@ -1715,7 +1715,15 @@ ret = check_arm_attributes_hfabi(fd, ehdr, &is_hf); if (ret != 0) return ret; - + +#ifdef __ARM_PCS_VFP + if (!is_hf) + _exit(1); +#else + if (is_hf) + _exit(1); +#endif + if (all_hf == -1) { if (is_hf)
Cheers,
On 11/23/2011 06:39 PM, Steve McIntyre wrote:
On Thu, Nov 10, 2011 at 05:33:52PM +0000, Steve McIntyre wrote:
Hey folks,
Continuing on from the discussions at LC last week, here are my two patches. They work OK for me in testing on armhf and armel builds here, so I hope they are helpful for other people.
Background: in bi-arch and multi-arch configuration, soft-float and hard-float libraries do not play well together at all:
ldconfig (currently) does not distinguish between the two ABIs when scanning libraries and putting their details in ld.so.cache, so if you have libs of both ABIs installed on your system then there is no guarantee that the right one will be used when needed.
ld.so (currently) cannot verify at run-time whether a given lib uses the hard-float or soft-float ABI, which can lead to similar problems.
Talking with doko on IRC, it's become clear that the second patch needs extending a little more to allow ldd to work on a mixed/multi-arch system. Here's the addition:
steve@panda-smcintyre:~/linker/eglibc-2.13/elf$ diff -u dl-load.c~ dl-load.c --- dl-load.c~ 2011-11-23 15:53:24.341835279 +0000 +++ dl-load.c 2011-11-23 16:21:27.005896511 +0000 @@ -1715,7 +1715,15 @@ ret = check_arm_attributes_hfabi(fd, ehdr, &is_hf); if (ret != 0) return ret;
+#ifdef __ARM_PCS_VFP
- if (!is_hf)
- _exit(1);
+#else
- if (is_hf)
- _exit(1);
+#endif
- if (all_hf == -1) { if (is_hf)
this seems to work, except for ldd, which needs to be made aware of the hf dynamic linker. Without this, you see on armel:
$ ldd <armhf a.out> not a dynamic executable
changing RTLDLIST in ldd to RTLDLIST="/lib/ld-linux.so.3 /lib/arm-linux-gnueabihf/ld-linux.so.3"
$ ldd a.out libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x40241000) /lib/arm-linux-gnueabihf/ld-linux.so.3 (0x4008c000)
setting a specific RTLDLIST in glibc is usually done in sysdeps/unix/sysv/linux/*/ldd-rewrite.sed
however in this case you would need to add the armhf linker for a armel default build, and add the armel (sf) linker for an armhf build with a fixed pattern. currently setting RTLDLIST within the packaging. An eglibc built this way for armel can be found in the ubuntu-toolchain-r test PPA (installs into oneiric and precise).
Matthias
On Sat, Nov 26, 2011 at 03:38:18PM +0100, Matthias Klose wrote:
On 11/23/2011 06:39 PM, Steve McIntyre wrote:
On Thu, Nov 10, 2011 at 05:33:52PM +0000, Steve McIntyre wrote:
Hey folks,
Continuing on from the discussions at LC last week, here are my two patches. They work OK for me in testing on armhf and armel builds here, so I hope they are helpful for other people.
Background: in bi-arch and multi-arch configuration, soft-float and hard-float libraries do not play well together at all:
ldconfig (currently) does not distinguish between the two ABIs when scanning libraries and putting their details in ld.so.cache, so if you have libs of both ABIs installed on your system then there is no guarantee that the right one will be used when needed.
ld.so (currently) cannot verify at run-time whether a given lib uses the hard-float or soft-float ABI, which can lead to similar problems.
Talking with doko on IRC, it's become clear that the second patch needs extending a little more to allow ldd to work on a mixed/multi-arch system. Here's the addition:
steve@panda-smcintyre:~/linker/eglibc-2.13/elf$ diff -u dl-load.c~ dl-load.c --- dl-load.c~ 2011-11-23 15:53:24.341835279 +0000 +++ dl-load.c 2011-11-23 16:21:27.005896511 +0000 @@ -1715,7 +1715,15 @@ ret = check_arm_attributes_hfabi(fd, ehdr, &is_hf); if (ret != 0) return ret;
+#ifdef __ARM_PCS_VFP
- if (!is_hf)
- _exit(1);
+#else
- if (is_hf)
- _exit(1);
+#endif
- if (all_hf == -1) { if (is_hf)
this seems to work, except for ldd.
ldd is the reason for this change, of course - to stop the situation where ldd would say ENOENT for all dependencies due to trying the wrong linker.
which needs to be made aware of the hf dynamic linker. Without this, you see on armel:
$ ldd <armhf a.out> not a dynamic executable
Hmmm, yes. In much the same way as amd64 and i386 list both the amd64 and i386 linkers there. Sorry, from discussions on IRC I thought that was happening already.
changing RTLDLIST in ldd to RTLDLIST="/lib/ld-linux.so.3 /lib/arm-linux-gnueabihf/ld-linux.so.3"
$ ldd a.out libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x40241000) /lib/arm-linux-gnueabihf/ld-linux.so.3 (0x4008c000)
setting a specific RTLDLIST in glibc is usually done in sysdeps/unix/sysv/linux/*/ldd-rewrite.sed
however in this case you would need to add the armhf linker for a armel default build, and add the armel (sf) linker for an armhf build with a fixed pattern. currently setting RTLDLIST within the packaging. An eglibc built this way for armel can be found in the ubuntu-toolchain-r test PPA (installs into oneiric and precise).
OK.
Cheers,
-
Matthias Klose -
Steve McIntyre