[This was also posted to debian-devel but I got the linaro addesss wrong: http://lists.debian.org/debian-devel/2011/02/msg00196.html]

Libtool is intended to make library linking 'just work' whatever the details of your build mechanisms are.

However in Debian/Ubuntu cross-building it seems to go out of its way to make it not-work. The problem is papered-over when using i386 machines, or amd64 machines and a new-enough (or old enough) version of binutils, but this seems to me to be a very poor solution as it has to be put into the linker for every cross-combination people might choose to build between.

Making libtool actually do the right thing would be much better. In fact it would be correct, and we like correctness.

This is not a new problem. In a bit of research I found a post suggesting it was actually broken in 2002 (I've not yet looked into this).

(a long list of previous posts on the issue from OE, Debian, Ubuntu, astlinux and buildroot people is at the end of this mail).

Now, before I get too carried away I want to check that I'm not just misunderstanding something. It seems faintly crazy that this very straightforward cross-comiling problem is not already solved.

And I have to admit that I have always done my best to avoid understanding the details of libtool (like most people). However I think the time has come to sort this out properly, if in fact it hasn't been already.

The problem -----------

We are crossbuilding Debian packages, that build libraries, which link to other libraries. There are lots of those, but I'm using unixodbc and libprce3 here as they seem fairly typical examples.

The libraries to link against are installed into a directory, historically /usr/<triplet>/lib, but moving to /usr/lib/<host-multiarchtuple> using multiarch mechanisms. They could also be in sysroot locations if using sysroot mechanisms). The point is that they are _not_ in /usr/lib (or /usr/lib/<build-multiarchtuple> in multiarch world).

libtool gets most of this process right (or at least not wrong), until the library is installed into the package location typically (debian/tmp/usr/lib) using libtool's 'install' mode, which then calls libtool with mode 'relink'.

At this points it calls the linker and adds -L/usr/lib on the front - thereby adding this path in front of the default cross-compiler path.

If called without this -L, or called with -L/usr/<triplet>/lib (i.e the correct path, wherever that is), then everything is fine because the libraries for the correct (host) architecture are found. So there are 2 ways to get this right and one way to get it wrong, which is the one libtool picks.

The workarounds which mean this has not yet been fixed are that the linker used to just issue a warning and ignore libraries which it didn't understand: /usr/lib/libgcc_s.so: file not recognized: File format not recognized

And more recently it has been taught to understand specific architectures enough to decide they are the wrong ones: /usr/arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc

However on amd64 machine with armel toolchain and binutils-arm-linux-gnueabi_2.20.51.20100908-0ubuntu2cross1.52 the skipping fails for static libs: /usr/lib/libc.a: could not read symbols: File format not recognized and the build fails.

on i386, or later versions of binutils, these are skipped the same as the .so s.

However as stated above I don't think relying on this linker behaviour is any real sort of solution - we should get libtool to either leave it up to the toolchain, or pass the correct explicit path.

Here is the example unixodbc case that illustrates the problem: (Built with: xdeb -a armel --apt-source unixodbc, which runs: dpkg-buildpackage -rfakeroot -d -us -uc -aarmel -b -tc )

make[4]: Entering directory /home/chroot-user/build/unixodbc-2.2.14p2/samples' test -z "/usr/lib" || mkdir -p -- "/home/chroot-user/build/unixodbc-2.2.14p2/debian/tmp/usr/lib" /bin/bash ../libtool --mode=install /usr/bin/install -c 'libboundparam.la' '/home/chroot-user/build/unixodbc-2.2.14p2/debian/tmp/usr/lib/libboundparam.la' libtool: install: warning: relinking libboundparam.la' libtool: install: (cd /home/chroot-user/build/unixodbc-2.2.14p2/samples; /bin/bash /home/chroot-user/build/unixodbc-2.2.14p2/libtool --tag CC --mode=relink arm-l$ libtool: relink: arm-linux-gnueabi-gcc -shared .libs/boundparam.o .libs/helper.o .libs/cboundtimestampparam.o -Wl,--whole-archive ../autotest/.libs/libgtrtstlc.$ /usr/lib/gcc/arm-linux-gnueabi/4.5.1/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/lib/gcc/arm-linux-gnueabi/4.5.1/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/lib/gcc/arm-linux-gnueabi/4.5.1/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc /usr/lib/libc.a: could not read symbols: File format not recognized collect2: ld returned 1 exit status libtool: install: error: relink libboundparam.la' with the above command before installing it make[4]: *** [install-libLTLIBRARIES] Error 1

I note that when doing this libtool --config shows that it does have an idea of where the right place is: maverick)wookey@kh:~/ubuntu/maverick/build/pcre3$ ./libtool --config | grep sys_lib sys_lib_search_path_spec="/usr/lib/gcc/arm-linux-gnueabi/4.4.5 /usr/arm-linux-gnueabi/lib" sys_lib_dlsearch_path_spec="/usr/arm-linux-gnueabi/lib /lib /usr/lib /usr/local/lib"

I haven't worked out exactly when and where libtool applies those, but as it seems to get the compiling and linking part right they may be being used (or the compiler defaults are just working as they should). It's only the install/relink phase that is bust (so far as I can tell).

pcre3 provides a simpler and quicker-to-build example of the same problem.

Here are various previous postings on the subject: Simon Richter in march last year, to libtool list: http://lists.gnu.org/archive/html/libtool/2010-03/msg00013.html (no response)

Loic Minier in October last year, to libtool list http://thread.gmane.org/gmane.comp.gnu.libtool.bugs/7626 (response: yes that seems to be a bug, no time to fix now, does sysroot option fix it? 'Not for me' said lool)

Philip Prindeville from astlinux, July 2010, to libtool list http://lists.gnu.org/archive/html/libtool/2010-07/msg00018.html (response: send us some reproducible examples, and 'heres a ptch for the sysroot-and-deleting-.la-files case')

Martin Panter from Openembedded in December 2010, to libtool-patches-gnu http://osdir.com/ml/libtool-patches-gnu/2011-01/msg00025.html which includes a patch to fix it. (response: useful discussion of possible patches)

Opinions on the best way to make libtool do the right thing both now and in the future are welcome. Ther are quite a few patches in the above threads providing different solutions, and I haven't yet determined which of them have been included where.

Wookey