Hi all,
I'd like to kick off discussion on this list with a topic that has come up in the ML working group meetings: dynamic loading of ArmNN binary backends.
What this is
Instead of compiling in support for various hardware into libarmnn.so, backends can be compiled separately into their own objects, and loaded later into ArmNN without recompiling.
For example, libarmnn.so might be compiled with just the reference backend, and the NEON and OpenCL backends could be provided as libarmnn-neon.so and libarmnn-opencl.so. At runtime, there would be some way for ArmNN to discover which backend .so files were available, and load them in to the running process.
Why do it
1. Better user experience when downloading Android apps. Compiled apps can link to libarmnn.so, and then choose which backends to bundle in variant optimised .apks for known devices. This is better than the current system where the application author would either need to be able to compile all the known backends in the world and link them all in to the application, or recompile the application for each desired combination of backend support. 2. Binary backend distribution. Backends might be hard to compile (need esoteric headers, vendor hardware support kits which are hard to work with, etc.) Having separate compilation for backends starts to open up the possibility that users could download ArmNN and the backends that they need without having to compile everything themselves. 3. Out-of-tree builds of backends. Backend developers can build armnn once, then iterate on the build of their backend without recompiling and redeploying the rest of ArmNN.
How This is where the questions are, and I'd like to get some feel for what people think, and work together towards a good design. Here are some discussion points to start with.
Discovery mechanism:
Application provides the path to each backend to "armnn::LoadPlugin(const char* path)"? Application provides a path to a directory and armnn tries to load all the .so files in that directory as plugins? Config file? All of the above? Some other ways too?
Relationship to statically-compiled plugins:
Should this mechanism completely replace statically compiled plugins? How would that affect existing OEM deployments? Is it sensible to make both techniques available?
Android:
Does dlopen() work in Android NDK apps? How does LD_LIBRARY_PATH work for apps distributed in .apks? Where should the backend libraries go in apks, and what about libraries that the backends need (e.g. libFabulousHardwareSupport.so)
Got any ideas about any of these questions, please reply and keep the list on cc for all discussion. We'll try to get a bit of discussion on each point, then pull things together into a design proposal.
Many thanks, Matthew
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.