In a recent github thread https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973 it was suggested that I ask this list about what the exact reasons for the lack of C++ support are, and how/if they break down by C++ feature so as to gauge a possible investment in remedying this situation at least partially.
In other words, suppose I changed the build process to include libstdc++, libgcc, and libgcc_eh (and/or other runtime support commonly linked with C++ programs), what features of C++ would work/still fail? And what implementation work would be required to implement the missing features? On a related note, are there intrinsic properties of the secure environment that may conflict with running C++ code, if any?
Thank you.
- Godmar
On Fri, Aug 11, 2017 at 6:53 AM, Godmar Back godmar@gmail.com wrote:
In a recent github thread https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973 it was suggested that I ask this list about what the exact reasons for the lack of C++ support are, and how/if they break down by C++ feature so as to gauge a possible investment in remedying this situation at least partially.
Answering this question would required detailed knowledge of OP-TEE, and I doubt that anyone on this list has such knowledge. I was just reading some FAQs about OP-TEE myself.
I would guess that part of the problem is that libstdc++ is so large, and isn't designed to be pulled apart into small pieces, as there are a lot of features that interact with other features. Just running nm on libstdc++.so, and grepping for GLIBC_, I see that it calls 105 glibc functions. Some of these functions are I/O related and are obvious problems, like fopen, fclose, fread, fwrite. Some of them are clock related and could be an issue, like clock_gettime, gettimeofday. The string and memory routines are probably OK, but libstdc++ may be using more of them than OP-TEE implements. There are also a lot of wide-character and multi-byte character routines being used that may not be available. There are pthread related functions that may or may not be a problem. There are also some environment and locale related routines that are likely a problem, but maybe you can just provide a default environment and locale.
Otherwise, I don't know enough about OP-TEE to guess what might be a problem. The exception handling support just requires the ability to read your own process memory, including the stack, the unwind info, and the ELF headers that get loaded into memory at program start. I don't know if that is a problem for OP-TEE.
Jim
On Fri, Aug 11, 2017 at 4:04 PM, Jim Wilson jim.wilson@linaro.org wrote:
On Fri, Aug 11, 2017 at 6:53 AM, Godmar Back godmar@gmail.com wrote:
In a recent github thread https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973
it
was suggested that I ask this list about what the exact reasons for the lack of C++ support are, and how/if they break down by C++ feature so as
to
gauge a possible investment in remedying this situation at least
partially.
Answering this question would required detailed knowledge of OP-TEE, and I doubt that anyone on this list has such knowledge.
Is this the wrong mailing list to ask the question then?
I would guess that part of the problem is that libstdc++ is so large, and isn't designed to be pulled apart into small pieces, as there are a lot of features that interact with other features. Just running nm on libstdc++.so, and grepping for GLIBC_, I see that it calls 105 glibc functions. Some of these functions are I/O related and are obvious problems, like fopen, fclose, fread, fwrite.
Normally, libraries are decomposed into individual .o files to avoid exactly this phenomenon where an undefined reference to function X pulls in definitions of functions Y, Z, etc. which then cause a chain of dependencies to other .o files the linker must resolve. Is libstdc++ different in this respect? Because otherwise, features would only be pulled in if required, and that's a much more manageable problem.
- Godmar
On Fri, Aug 11, 2017 at 1:34 PM, Godmar Back godmar@gmail.com wrote:
Is this the wrong mailing list to ask the question then?
Perhaps, but maybe you will get a response on Monday from someone in Europe that didn't see it today.
Normally, libraries are decomposed into individual .o files to avoid exactly this phenomenon where an undefined reference to function X pulls in definitions of functions Y, Z, etc. which then cause a chain of dependencies to other .o files the linker must resolve. Is libstdc++ different in this respect? Because otherwise, features would only be pulled in if required, and that's a much more manageable problem.
Yes, of course, if you static link, then you only get the .o files that you need. But as I mentioned before, a lot of C++ features depend on other C++ features. A simple hello world program compiled and statically linked is over 2MB.
palantir:2089$ cat tmp.cc #include <iostream>
int main (void) { std::cout << "Hello world\n"; return 0; } palantir:2090$ g++ -static tmp.cc palantir:2091$ ls -lt a.out -rwxrwxr-x 1 wilson wilson 2181416 Aug 11 14:03 a.out palantir:2092$
We can see all of the object files being pulled in if we pass the --verbose option to the linker. There are 92 of them.
palantir:2092$ g++ -static tmp.cc -Wl,--verbose | grep libstdc attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a succeeded (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios_init.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)globals_io.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ext11-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ostream-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)wlocale-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-wstring-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)functexcept.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-stdexcept.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)streambuf-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_throw.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_exception.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_personality.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)del_op.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_catch.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_aux_runtime.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_typeid.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_cast.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)si_class_type_info.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_call.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_opv.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)pure.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)tinfo.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)class_type_info.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_op.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_handler.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)vmi_class_type_info.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)del_opv.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_alloc.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_terminate.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_globals.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_alloc.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)dyncast.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale_facets.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)time_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale_init.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)stdexcept.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)c++locale.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)monetary_members_cow.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)numeric_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)basic_file.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios_locale.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)streambuf.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)messages_members_cow.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)numeric_members_cow.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)codecvt.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-istream-string.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)codecvt_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)collate_members_cow.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)string-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-locale-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-wlocale-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-string-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)snprintf_lite.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)fstream-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-stdexcept.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)system_error.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-shim_facets.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-locale_init.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype_configure_char.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)functional.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-shim_facets.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)wstring-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)future.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)regex.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)lt1-codecvt.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-ios_failure.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)iostream-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)compatibility.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_array_new.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)guard.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_unex_handler.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_ptr.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_term_handler.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)monetary_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)misc-inst.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream-string.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)collate_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)messages_members.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)futex.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)condition_variable.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)guard_error.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)vterminate.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_type.o (/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cp-demangle.o palantir:2093$
Perhaps there are other things that can be used that have smaller requirements, but in general, the C++ library is not designed for small slices. Everything depends on everything else, and it is best to use shared linking and use the whole thing as a single unit.
Jim
That also depends on what you want to use C++ for. If you just want to use the convenient syntax and language features then you don't need to link against any additional library. In that case you'd disable exceptions and rtti and implement new/delete using malloc/free.
Thanks Michael
On Fri, Aug 11, 2017 at 3:53 PM, Godmar Back godmar@gmail.com wrote:
In a recent github thread https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973 it was suggested that I ask this list about what the exact reasons for the lack of C++ support are, and how/if they break down by C++ feature so as to gauge a possible investment in remedying this situation at least partially.
In other words, suppose I changed the build process to include libstdc++, libgcc, and libgcc_eh (and/or other runtime support commonly linked with C++ programs), what features of C++ would work/still fail? And what implementation work would be required to implement the missing features? On a related note, are there intrinsic properties of the secure environment that may conflict with running C++ code, if any?
Thank you.
- Godmar
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain
On Sat, Aug 12, 2017 at 1:27 AM, Michael Zimmermann sigmaepsilon92@gmail.com wrote:
That also depends on what you want to use C++ for. If you just want to use the convenient syntax and language features then you don't need to link against any additional library. In that case you'd disable exceptions and rtti and implement new/delete using malloc/free.
Instead of pounding a round peg into a square hole with C++, maybe the path to pursue is GCC support for try/finally in C. You get most of the semantics you are looking for, and someone as experienced as you will know exactly what to do with the feature.
Lack of try/finally support means library often do awful things. Asterisk was full of issues, like resource leaks, that try/finally would have fixed. Eventually Asterisk moved to nested/inner functions and trampolines. They silently lost NX stacks, which was a cure as bad as the disease. It just moved problems around rather than fixing them.
Microsoft compilers support try/finally in C. I recall a GCC bug report years ago asking for it, but it was declined.
Jeff
I'm sorry the discussion got off track about the relative virtues of C++ for embedded programming.
Our need is to run parts of existing C++ code (the px4 http://px4.io/ library for drones) under OP-TEE. The C++ features we need support for thus are the ones used there, and we'd like to change it as little as possible.
So far, I've heard the risk that C++ binaries get large if you include libstdc++ features such as cout. Ok. See my footnote (*) below.
Size aside, what other properties of OP-TEE are risking conflict with C++'s runtime that anyone already knows of?
- Godmar
(*): In this context, I've found a FAQ entry describing why they have such large object files https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.size and mentioning a garbage collection feature of the GNU linker, which I hadn't heard of. It reduces the size for the static Hello World program from a roughly 1.7M text section to 1.3M text:
$ cat hello.cc #include <iostream>
int main() { std::cout << "Hello, World\n"; }
$ g++ -static hello.cc -o hello $ size hello text data bss dec hex filename 1711137 33580 18056 1762773 1ae5d5 hello $ ls -l hello -rwxrwxr-x 1 gback gback 2181416 Aug 12 10:34 hello
$ g++ -static -Wl,--gc-sections hello.cc -o hello $ size hello text data bss dec hex filename 1321767 24388 16424 1362579 14ca93 hello $ ls -l hello -rwxrwxr-x 1 gback gback 1567592 Aug 12 10:34 hello
On Sat, Aug 12, 2017 at 1:55 AM, Jeffrey Walton noloader@gmail.com wrote:
On Sat, Aug 12, 2017 at 1:27 AM, Michael Zimmermann sigmaepsilon92@gmail.com wrote:
That also depends on what you want to use C++ for. If you just want to use the convenient syntax and language features then you don't need to link against any additional library. In that case you'd disable exceptions and rtti and implement new/delete using malloc/free.
Instead of pounding a round peg into a square hole with C++, maybe the path to pursue is GCC support for try/finally in C. You get most of the semantics you are looking for, and someone as experienced as you will know exactly what to do with the feature.
Lack of try/finally support means library often do awful things. Asterisk was full of issues, like resource leaks, that try/finally would have fixed. Eventually Asterisk moved to nested/inner functions and trampolines. They silently lost NX stacks, which was a cure as bad as the disease. It just moved problems around rather than fixing them.
Microsoft compilers support try/finally in C. I recall a GCC bug report years ago asking for it, but it was declined.
Jeff
Hello Godmar,
This is a very difficult question to answer as it will depend a lot on the constraints of the OP-TEE environment and what subset of the C++ language and libraries that the developers are willing to support. I can speak a little bit about generic support of C++ in embedded systems via my experience with ARM's commercial toolchain, which is based on a mixture of clang, libc++, libc++abi and libunwind; and ARM's own C-library and linker. This isn't going to directly map to the GNU world but I don't think it will be too far off.
The minimal additional support for C++ over C is support for the language runtime, representing helper functions that the compiler expects to be defined, regardless of whether these are called in the source code; for example the various forms of new and delete. These functions often have a cxa prefix are defined generically by the Itanium C++ ABI and built upon by platform specific ABIs such the ARM C++. In the llvm world these are defined by the libc++abi library with stack unwinding provided by the libunwind library. In general I wouldn't expect there to be any major difficulty in providing such a run-time support library for OP-TEE, but I don't know anything at all about any special restrictions that platform has that might get in the way. For example I don't know whether OP-TEE requires additional libraries to be certified?
Any use of C++ static constructors need to be initialised by the run-time environment, this is typically handled by the C-library start-up.
Support for the C++ standard library is more complicated, particular with C++11 and beyond. There are some areas of functionality that are fairly easy to say this won't work on our platform. For example std::thread may not work if the underlying platform has no support for threads. There are some areas such as the containers and algorithms that will require little additional support beyond the runtime. Some features such as use of iostream will significantly bloat the code as this will pull in all the locale support, require static initialisation of the cin, cout objects.
Perhaps the most difficult decision is whether to support exceptions. For your own code, choosing not to support exceptions isn't a problem as both gcc and clang can compile without exception tables, however many C++ libraries, including the standard library make use of exceptions, and an uncaught exception results in program termination. The libc++ library can be compiled without exceptions but this just means it will terminate the program if an exception would be thrown.
To summarise, if the platform runtime support is limited, there isn't an easy answer to say C++ is supported, and the answer heavily depends on the design choices of the platform holder. The OP-TEE developers would most likely have to define a subset that is supported and constantly monitor it as toolchains change. They may not consider this worth the trade-off against the simpler answer of C++ is not supported.
Hope this is of some use
Peter
On 11 August 2017 at 14:53, Godmar Back godmar@gmail.com wrote:
In a recent github thread https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973 it was suggested that I ask this list about what the exact reasons for the lack of C++ support are, and how/if they break down by C++ feature so as to gauge a possible investment in remedying this situation at least partially.
In other words, suppose I changed the build process to include libstdc++, libgcc, and libgcc_eh (and/or other runtime support commonly linked with C++ programs), what features of C++ would work/still fail? And what implementation work would be required to implement the missing features? On a related note, are there intrinsic properties of the secure environment that may conflict with running C++ code, if any?
Thank you.
- Godmar
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain
On 14 August 2017 at 10:27, Peter Smith peter.smith@linaro.org wrote:
Perhaps the most difficult decision is whether to support exceptions. For your own code, choosing not to support exceptions isn't a problem as both gcc and clang can compile without exception tables, however many C++ libraries, including the standard library make use of exceptions, and an uncaught exception results in program termination. The libc++ library can be compiled without exceptions but this just means it will terminate the program if an exception would be thrown.
Most standard library exceptions can be easily worked around with a bit of kludge, for example out_of_range, bad_cast, overflow_error. Others may require a more complicated setup, like special allocators that don't throw bad_alloc. A few can be simply never used (ex. bad_exception and bad_typeid).
However, from what I gather about OP-TEE, you guys need a secure environment, and relying on the whole C++ standard library (especially a third-party one, even open source) is not feasible.
Honestly, I don't think you should even have threads support, or exceptions, or type-ids or rely on the default allocators, nor you should rely on C/C++ compilers to tell you anything about safe ranges and casts. All those things are riddled with security issues and shouldn't be trifled with.
In the past, I have used C++ with only the C library (for performance purposes) and if you restrict the functionality to a minimum, it should be much easier to control the side effects.
The LLVM project is one example where we control some parts of it and keep most. The European Space Agency has a long document [1] on their own restrictions of the language, showing a different type of restrictions, which may line up closer to your needs.
cheers, --renato
[1] http://www.esa.int/TEC/Software_engineering_and_standardisation/TECRFBUXBQE_...
linaro-toolchain@lists.linaro.org
-
Godmar Back -
Jeffrey Walton -
Jim Wilson -
Michael Zimmermann -
Peter Smith -
Renato Golin