I probably know the answer to this already but ...
For shared libs one can define and use something like:
void __attribute__ ((constructor)) my_init(void); void __attribute__ ((destructor)) my_fini(void);
Which of course allows your lib to run code just after the library is loaded and just before the library is going to be unloaded. This helps keep out cruft such as the following out of your design:
PleaseCallThisLibraryFunctionFirstOrThereWillBeAnErrorWhichYouWillHitCausingYouToPostToTheMailingListAskingTheSameQuestionThatHasBeenAsked1000sOfTimes();
Yeah .. you know the function. I don't like it either.
Unfortunately this doesn't work when people link in the .a from your lib. Libs like libjpeg-turbo in theory should never ever need to be linked in that fashion but consider the browsers who link to the universe instead of using system shared libs.
If we accept that linking .a's valid use, and we don't want to promote the use of the function that won't be named in the design and use of libraries, then I'd like to think for the good of humanity the toolchain should allow for the use of __attribute__ ((constructor)) and __attribute__ ((destructor)) to continue to work in the static link case.
Maybe this has already been solved, in which case where is that fine manual as I clearly have some reading to do!
Thoughts?
Thanks!
Tom
"Where's the kaboom!? There was supposed to be an earth-shattering kaboom!" Marvin Martian Multimedia Tech Lead | Linaro.org │ Open source software for ARM SoCs w) tom.gall att linaro.org w) tom_gall att vnet.ibm.com h) tom_gall att mac.com
Here is some triky way for this problem, you can put the constructor and destructor to the source file which contain necessary function call in your libraries to enforce the linker to archive your constructor and destructor.
However if this solution is not work for your situation, you can apply the patch in attach for build script to enable the LOCAL_WHOLE_STATIC_LIBRARIES for executable,
After patch you can just add a line in your Android.mk :
LOCAL_WHOLE_STATIC_LIBRARIES += libfoo
The most disadvantage of this way is you should always link libfoo by LOCAL_WHOLE_STATIC_LIBRARIES...and this patch don't send to linaro and aosp yet.
On Mon, Dec 5, 2011 at 1:40 AM, Tom Gall tom.gall@linaro.org wrote:
I probably know the answer to this already but ...
For shared libs one can define and use something like:
void __attribute__ ((constructor)) my_init(void); void __attribute__ ((destructor)) my_fini(void);
Which of course allows your lib to run code just after the library is loaded and just before the library is going to be unloaded. This helps keep out cruft such as the following out of your design:
PleaseCallThisLibraryFunctionFirstOrThereWillBeAnErrorWhichYouWillHitCausingYouToPostToTheMailingListAskingTheSameQuestionThatHasBeenAsked1000sOfTimes();
Yeah .. you know the function. I don't like it either.
Unfortunately this doesn't work when people link in the .a from your lib. Libs like libjpeg-turbo in theory should never ever need to be linked in that fashion but consider the browsers who link to the universe instead of using system shared libs.
If we accept that linking .a's valid use, and we don't want to promote the use of the function that won't be named in the design and use of libraries, then I'd like to think for the good of humanity the toolchain should allow for the use of __attribute__ ((constructor)) and __attribute__ ((destructor)) to continue to work in the static link case.
Maybe this has already been solved, in which case where is that fine manual as I clearly have some reading to do!
Thoughts?
Thanks!
Tom
"Where's the kaboom!? There was supposed to be an earth-shattering kaboom!" Marvin Martian Multimedia Tech Lead | Linaro.org │ Open source software for ARM SoCs w) tom.gall att linaro.org w) tom_gall att vnet.ibm.com h) tom_gall att mac.com
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Mon, Dec 5, 2011 at 1:40 AM, Tom Gall tom.gall@linaro.org wrote:
I probably know the answer to this already but ...
For shared libs one can define and use something like:
void __attribute__ ((constructor)) my_init(void); void __attribute__ ((destructor)) my_fini(void);
Which of course allows your lib to run code just after the library is loaded and just before the library is going to be unloaded. This helps keep out cruft such as the following out of your design:
PleaseCallThisLibraryFunctionFirstOrThereWillBeAnErrorWhichYouWillHitCausingYouToPostToTheMailingListAskingTheSameQuestionThatHasBeenAsked1000sOfTimes();
Yeah .. you know the function. I don't like it either.
Unfortunately this doesn't work when people link in the .a from your lib. Libs like libjpeg-turbo in theory should never ever need to be linked in that fashion but consider the browsers who link to the universe instead of using system shared libs.
On Mon, Dec 05, 2011 at 04:19:11PM +0800, Kito Cheng wrote:
Here is some triky way for this problem, you can put the constructor and destructor to the source file which contain necessary function call in your libraries to enforce the linker to archive your constructor and destructor.
However if this solution is not work for your situation, you can apply the patch in attach for build script to enable the LOCAL_WHOLE_STATIC_LIBRARIES for executable,
After patch you can just add a line in your Android.mk :
LOCAL_WHOLE_STATIC_LIBRARIES += libfoo
The most disadvantage of this way is you should always link libfoo by LOCAL_WHOLE_STATIC_LIBRARIES...and this patch don't send to linaro and aosp yet.
[...]
Part of the problem here is that .a libraries lack the dependency and linkage metadata that shared libraries have.
-2)
Put up with the need to call an explicit initialisation function for the library. A lot of commonly-used libraries require an initialisation call, and I'm not sure it causes that much of a problem in practice...
-1)
Put a C++ wrapper around just enough of your library such that your constructor/destructor code is recognised as a needed static constructor/descructor by the toolchain.
I can't think of a very nice way of doing this, so I won't elaborate on it...
It's also not really a solution, since you still need to pull in a dummy static object from somewhere in order to cause the construcor and descructor to get called.
0)
libtool or similar may help solve this problem, but I don't know much about this -- also, for solving the problem, that approach only works if uses of your library link via libtool.
1)
One hacky approach is to rename your library to libmylib-real.a, and then make replace libmylib.a with a linker script which pulls in the needed constructor as well as the real library:
libmylib.a: EXTERN(__mylib_constructor) INPUT(/path/to/libmylib-real.a)
This works, providing that __mylib_constructor is external (normally, you would be able have the constructor function static, but it needs to be externally visible in order to be pulled in in this way.
2)
Another way of doing a similar thing is to mark __mylib_constructor as undefined in all the objects that make up the library.
Unfortunately, there seems to be no obvious way of doing that: the assembler generates undefined symbol references automatically for unresolved references at assembly time. There's no way for force the existence of an undefined symbol without an actual reference to it. objcopy/elfedit don't seem to support adding such a symbol either. It would be simple to write a tool to add the undefined symbol reference (such tools may exist already), but binutils doesn't seem to provide this for you. The plausible-looking -u option to gcc doesn't do anything unless doing a link.
One other way of doing it without a special tool is to insert a bogus relocation into the text section of each object with an assembler .reloc directive specifying relocation type R_<arch>_NONE.
There isn't really a portable way to do that, though. The name of the relocation changes per-arch, and some arches have other quirks (on ARM for example, .reloc cannot refer to the current location, but seems instead to need to refer to a defined symbol which is non-zero distance away from the location counter).
One advantage to this approach is that your .a file looks just like any other .a file. Also, you can include that dependency in only those objects which really require the library to be initialised (normally, this is not a huge benefit though, since probably most of your objects _do_ require the library to be initialised).
A disadvantage (other than portability problems) is that, like (1), the constructor symbol must be external (not static)... so it pollutes the symbol table and isn't protected against people calling it directly.
You can create a dummy symbol instead of referring to the constructor symbol directly though -- this solves the second problem.
3)
Finally, you can split your contructor/destructor code out into a separate .o file (say mylib-ctors.o), and use the linker script trick for (1) to forcibly include this object when linking:
libmylib.a: INPUT(/path/to/mylib-ctors.o /path/to/mylib-real.a)
This avoids some of the disadvantages of the other approaches, but you still end up with a strange-looking library which is really a linker script.
This is closer to how the C library traditionally solves the problem (i.e., the crt*.o stuff). libc.so also tends to be a linker script, which deals with the fact that some parts of libc must be statically linked from a separate library when linking to -lc.
Obviously, approaches (1)..(3) all suffer from arch or toolchain portability problems (or both). (The GNU/GCC __constructor__ thing is obviously a portability problem in itself, it you're minded to care about it.)
Cheers ---Dave
Dave Martin dave.martin@linaro.org writes:
Another way of doing a similar thing is to mark __mylib_constructor as undefined in all the objects that make up the library.
Unfortunately, there seems to be no obvious way of doing that: the assembler generates undefined symbol references automatically for unresolved references at assembly time. There's no way for force the existence of an undefined symbol without an actual reference to it.
One way of doing this is to create an R_ARM_NONE relocation against it, such as:
.reloc .,R_ARM_NONE,__mylib_constructor
This isn't as hacky as it might sound, because undefined references don't really make much sense without an associated reloc.
Richard
Richard Sandiford richard.sandiford@linaro.org writes:
Dave Martin dave.martin@linaro.org writes:
Another way of doing a similar thing is to mark __mylib_constructor as undefined in all the objects that make up the library.
Unfortunately, there seems to be no obvious way of doing that: the assembler generates undefined symbol references automatically for unresolved references at assembly time. There's no way for force the existence of an undefined symbol without an actual reference to it.
One way of doing this is to create an R_ARM_NONE relocation against it, such as:
.reloc .,R_ARM_NONE,__mylib_constructor
This isn't as hacky as it might sound, because undefined references don't really make much sense without an associated reloc.
Not that I recommend this as a fix for the original problem btw. :-) Just FYI.
As Kito says, the easiest way of treating a static link like a shared one is to use --whole-archive -lfoo --no-whole-archive. If the library is compiled with -ffunction-sections -fdata-sections, --gc-sections should remove the unnecessary code.
Richard
On Mon, Dec 05, 2011 at 04:47:08PM +0000, Richard Sandiford wrote:
Richard Sandiford richard.sandiford@linaro.org writes:
Dave Martin dave.martin@linaro.org writes:
Another way of doing a similar thing is to mark __mylib_constructor as undefined in all the objects that make up the library.
Unfortunately, there seems to be no obvious way of doing that: the assembler generates undefined symbol references automatically for unresolved references at assembly time. There's no way for force the existence of an undefined symbol without an actual reference to it.
One way of doing this is to create an R_ARM_NONE relocation against it, such as:
.reloc .,R_ARM_NONE,__mylib_constructor
This isn't as hacky as it might sound, because undefined references don't really make much sense without an associated reloc.
Not that I recommend this as a fix for the original problem btw. :-) Just FYI.
Agreed -- it's possible, but it doesn't feel like a nice solution.
As Kito says, the easiest way of treating a static link like a shared one is to use --whole-archive -lfoo --no-whole-archive. If the library
Ideally, a lot of unused code can be discarded automatically in a static link (although this may vanish into the noise when considering that _any_ use of stdio will typically pull in over 300K of probably mostly dead code from libc). --whole-archive would completely defeat this, though the degree to which this matters depends somewhat on the size of the library.
is compiled with -ffunction-sections -fdata-sections, --gc-sections should remove the unnecessary code.
Most projects don't use those options though, and a fair number of projects make assumptions which cause them to break if --gc-sections is used.
Requiring people to link with a special option doesn't really solve the problem as stated -- they could link directly with mylib_ctors.o or -u __mylib_constructor after all.
Building a static library which "just works" with regard to running initialisation code is possible, but there seems to be no really good way of doing it.
Where the library has some notion of a session (where you create some kind of heavyweight context, do something with it and then release it), Mans' suggestion of tying the construction/destruction to this context seems the most convenient option. This exposes the link-time dependencies in a way which the linker can understand correctly without extra help, and provides a natural and convenient API to the user.
Provding you're careful to do it in a thread-safe way, you can also tie lazy initialisation of the library (if it needs global initialisation) to take place at the first "create context or session" call.
Libraries which are sessionless seem less likely to need global initialisation anyway, though it will depend on the library.
Cheers ---Dave
On 4 December 2011 17:40, Tom Gall tom.gall@linaro.org wrote:
I probably know the answer to this already but ...
For shared libs one can define and use something like:
void __attribute__ ((constructor)) my_init(void); void __attribute__ ((destructor)) my_fini(void);
Which of course allows your lib to run code just after the library is loaded and just before the library is going to be unloaded. This helps keep out cruft such as the following out of your design:
PleaseCallThisLibraryFunctionFirstOrThereWillBeAnErrorWhichYouWillHitCausingYouToPostToTheMailingListAskingTheSameQuestionThatHasBeenAsked1000sOfTimes();
Yeah .. you know the function. I don't like it either.
Unfortunately this doesn't work when people link in the .a from your lib. Libs like libjpeg-turbo in theory should never ever need to be linked in that fashion but consider the browsers who link to the universe instead of using system shared libs.
If we accept that linking .a's valid use, and we don't want to promote the use of the function that won't be named in the design and use of libraries, then I'd like to think for the good of humanity the toolchain should allow for the use of __attribute__ ((constructor)) and __attribute__ ((destructor)) to continue to work in the static link case.
Maybe this has already been solved, in which case where is that fine manual as I clearly have some reading to do!
There is no standard way to do this in C. The constructor/destructor attributes you mention are of course gcc extensions, and as you say, they only work for shared libraries.
The usual way to solve this is to do any static initialisation required as part of some other function all users would have to call anyway. For libjpeg, the jpeg_create_(de)comress() functions would be the right place.
Tom Gall tom.gall@linaro.org wrote:
For shared libs one can define and use something like:
void __attribute__ ((constructor)) my_init(void); void __attribute__ ((destructor)) my_fini(void);
Which of course allows your lib to run code just after the library is loaded and just before the library is going to be unloaded.
Unfortunately this doesn't work when people link in the .a from your lib.
Irrespectively of whether it is a good idea to design your library such as to require constructor/destructor routines (:-)) these attributes should work just fine for code that is statically linked. In those cases, the constructor is called early during process startup, and the destructor is called during process exit.
So I'm not quite sure why you say this "doesn't work". Absent further details, I can speculate that you might have been running into problems relating to either:
- missing object references
When linking against a shared library, that whole library gets loaded as a unit, and will always have all its constructors. When linking against a static library, only such object files that are actually (directly or indirectly) referenced by the caller will get pulled into the final executable. If your constructor/destructor routines happen to reside in objects that are not otherwise references, they will simply not be present in the executable and thus not called either.
This type of problems can be fixed by keeping constructors/destructors in object files next to routines that rely on them. For example, if you need a constructor to set up a data structure before it can be used, keep the constructor in the same file that provides the main accessor routines to that data structure, such that every user of the data structure must always pull in that object -- and hence the constructor.
- initialization order issues
The constructors will be called during early startup, which might not be the proper place for whatever they're trying to do (e.g. they might call some other library which itself wasn't initialized yet ...).
This is really the same type of problem you also have with shared libraries that are loaded at startup (i.e. not via dlopen), but the details of the constructor call sequence might change in the static case.
Does either of those explain the problems you were seeing? Otherwise, if you have more details of what exactly you tried and what went wrong, I'd be happy to have a look ...
Mit freundlichen Gruessen / Best Regards
Ulrich Weigand
-- Dr. Ulrich Weigand | Phone: +49-7031/16-3727 STSM, GNU compiler and toolchain for Linux on System z and Cell/B.E. IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martin Jetter | Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen | Registergericht: Amtsgericht Stuttgart, HRB 243294