[Resending due to mail bounce]
Hi Pavel,
Thanks for your email, I've modified this response to more clearly indicate what we would like from a design pov and to more clearly indicate that much of this work is open for community contributions.
It should be fine to move PyArmNN into armnn master with a few small modifications. Ideally, I'd rather not have any generated files checked-in to the repository, however any scripts to execute the generation command can be checked-in. In order to remove the hard dependency on Tox (I think we are in agreement on this), it would be best to move the generation commands into a separate stand-alone (bash or other?) script or scripts which the Tox script then calls directly, and make the generation an optional build step in the cmake (which can also then call these same generation scripts). Users and contributors can then regenerate the bindings as needed without requiring Tox. Tox can therefore remain as a purely optional convenience for testing against multiple python versions (which we will use in our internal CI). So to answer one of your questions, at least initially, we would like PyArmNN to be introduced purely as source which can then be built for the target machine. So the work required would be as follows:
1. Refactor the a) swig code gen commands and b) subsequent python source packaging commands from Tox into stand-alone scripts which Tox then calls.
2. Add an optional build step in CMake to generate PyArmNN source files and python source package. Does that sound reasonable?
One of my other main concerns in the short-term, is that the PyArmNN interface is kept up-to-date and working with the changing code base. We don't currently have tests in place to ensure the PyArmNN interface is a) stable and b) kept up-to-date as new interfaces are added to ArmNN. If possible, I would like to see these tests run on every check-in via the Linaro build-robot. We have not planned or scheduled any effort towards this so if you can pick this up, that would be most valuable.
Making PyArmNN available via the package managers is an aspiration we would like to work towards, but there are some things we'd like to achieve first on that path. Mainly, ArmNN (and the corresponding PyArmNN) are currently strongly bound to a particular release version of the API. I would like to make stronger API & ABI guarantees than we currently do for the ArmNN frontend. This would probably require a stable C like interface in place....similar in fact to what PyArmNN is doing now. This would enable us to introduce semantic versioning which would enable ArmNN to work better as a system library, possibly even distributed as a debian package (we are working on the debian packaging atm). I'd also like to apply the same paradigm to the backend interface, but this is a MUCH larger and scarier endeavor and I'm not satisfied that the backend API is mature/stable enough for that just yet. We haven't scheduled this yet so it is useful work that anyone can pick-up. At the very least, I'd like to start some open dialog to find out a) how important this is for the community, and b) how we can get to that point.
Making PyArmNN available as WHL, as I understand it, requires prebuilt binaries for all the potential targets, which introduces a host of headaches which we would like to avoid at the moment but this might become easier if Arm NN is required as a system library with a stable ABI. I'm keen to get other opinions on this though.
@Alexander can correct me if I'm wrong an any of this, but I believe with the source package, you can use "pip install" already. You just have to point it to the generated source package rather than using the package manager. Given the limitations with ABI stability, I think publishing it on pypi.org will introduce a constant ongoing maintenance cost which we would rather solve by getting some of the fundamentals (ie weaker version dependencies) sorted first.
Also, on the documentation, we plan to make them available via the github pages. I think Alexander and his team have done an awesome job on the PyArmNN docs.
If you have more questions or suggestions, or if any of this doesn't work for you, let us know.
Regards, Derek
From: Matthew Bentham Sent: 12 February 2020 13:38 To: Pavel Macenauer; armnn-dev@lists.linaro.org; Georgios Pinitas; Derek Lamberti; Alexander Efremov Subject: Re: pyarmnn integration
+George, Derek, Alexander,
Please can you guys help Pavel? And keep the list on cc for visibility please.
Many thanks, Matthew ________________________________ From: Armnn-dev armnn-dev-bounces@lists.linaro.org on behalf of Pavel Macenauer pavel.macenauer@nxp.com Sent: 10 February 2020 21:00 To: armnn-dev@lists.linaro.org armnn-dev@lists.linaro.org Subject: [Armnn-dev] pyarmnn integration
Hi!
There is a branch experimental/pyarmnn, created by Matthew Bentham, which contains python wrappers for armnn, which initially seems to work pretty well - building a whl archive works, which can be installed using pip and I was able to write an example, which runs inference on a float/quantized model and using all the supported frameworks - tf, tf-lite, caffe, onnx as well. What is missing is to get the python wrappers integrated, run and check unit tests and write a few examples. We discussed this with Matthew already, but I would be glad to hear more opinions regarding how we proceed and to kick off a discussion.
1. How to integrate pyarmnn?
There are 2 paths initially:
1. Build pyarmnn together with armnn using a single cmake command * By default it would be turned off, otherwise it would be build using e.g. -DBUILD_PYARMNN * The product is either a whl or a src package - so should there be 2 options e.g. -DBUILD_PYARMNN_SRC, -DBUILD_PYARMNN_WHL or only a single one, which would always build both? 2. Separate pyarmnn from armnn into a different repository (and keep it as a separate project) * Additionally to a) options -DARMNN_LIB and -DARMNN_INCLUDE would be required as well, so that it can be "linked" against configurable armnn build
The difference is mainly in maintainability - a) forces to maintain pyarmnn and update the swig files to generate wrappers per every release b) on the other hand keeps the project separate, allows to build pyarmnn with a configurable armnn release and doesn't create a dependency to update the swig files whenever armnn interface changes a little.
1. Remove tox? Yes/No - Tox is a python automation library, which is used to generate the wrappers or to run unit tests. It is not really needed, because the wrappers can be generated directly using swig and the src/whl packages generated using python/setuptools and it just creates another dependency. Unit tests can also be run directly using python. 2. Get pyarmnn published on pypi.org? Yes/No - we would be able to install pyarmnn using "pip install pyarmnn"
Any additional ideas, comments, feedback etc. would be of course appreciated.
Thanks! Pavel M _______________________________________________ Armnn-dev mailing list Armnn-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/armnn-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Pavel,
We updated pyarmnn up to the latest Arm NN API. CR could be found here https://review.mlplatform.org/c/ml/armnn/+/2780. In addition to armnn version bump, tests were brought to working condition (some public NN models used to check parsers functionality were replaced with custom test dummy ones).
Your input would be highly appreciated.
Please feel free to contact us in case of any questions/suggestions.
Best regards, Alexander ________________________________ From: Derek Lamberti Derek.Lamberti@arm.com Sent: 17 February 2020 13:42 To: Matthew Bentham Matthew.Bentham@arm.com; Pavel Macenauer pavel.macenauer@nxp.com; armnn-dev@lists.linaro.org armnn-dev@lists.linaro.org; Georgios Pinitas Georgios.Pinitas@arm.com; Alexander Efremov Alexander.Efremov@arm.com Subject: RE: pyarmnn integration
[Resending due to mail bounce]
Hi Pavel,
Thanks for your email, I’ve modified this response to more clearly indicate what we would like from a design pov and to more clearly indicate that much of this work is open for community contributions.
It should be fine to move PyArmNN into armnn master with a few small modifications. Ideally, I’d rather not have any generated files checked-in to the repository, however any scripts to execute the generation command can be checked-in. In order to remove the hard dependency on Tox (I think we are in agreement on this), it would be best to move the generation commands into a separate stand-alone (bash or other?) script or scripts which the Tox script then calls directly, and make the generation an optional build step in the cmake (which can also then call these same generation scripts). Users and contributors can then regenerate the bindings as needed without requiring Tox. Tox can therefore remain as a purely optional convenience for testing against multiple python versions (which we will use in our internal CI). So to answer one of your questions, at least initially, we would like PyArmNN to be introduced purely as source which can then be built for the target machine. So the work required would be as follows:
1. Refactor the a) swig code gen commands and b) subsequent python source packaging commands from Tox into stand-alone scripts which Tox then calls.
2. Add an optional build step in CMake to generate PyArmNN source files and python source package.
Does that sound reasonable?
One of my other main concerns in the short-term, is that the PyArmNN interface is kept up-to-date and working with the changing code base. We don’t currently have tests in place to ensure the PyArmNN interface is a) stable and b) kept up-to-date as new interfaces are added to ArmNN. If possible, I would like to see these tests run on every check-in via the Linaro build-robot. We have not planned or scheduled any effort towards this so if you can pick this up, that would be most valuable.
Making PyArmNN available via the package managers is an aspiration we would like to work towards, but there are some things we’d like to achieve first on that path. Mainly, ArmNN (and the corresponding PyArmNN) are currently strongly bound to a particular release version of the API. I would like to make stronger API & ABI guarantees than we currently do for the ArmNN frontend. This would probably require a stable C like interface in place….similar in fact to what PyArmNN is doing now. This would enable us to introduce semantic versioning which would enable ArmNN to work better as a system library, possibly even distributed as a debian package (we are working on the debian packaging atm). I’d also like to apply the same paradigm to the backend interface, but this is a MUCH larger and scarier endeavor and I’m not satisfied that the backend API is mature/stable enough for that just yet. We haven’t scheduled this yet so it is useful work that anyone can pick-up. At the very least, I’d like to start some open dialog to find out a) how important this is for the community, and b) how we can get to that point.
Making PyArmNN available as WHL, as I understand it, requires prebuilt binaries for all the potential targets, which introduces a host of headaches which we would like to avoid at the moment but this might become easier if Arm NN is required as a system library with a stable ABI. I’m keen to get other opinions on this though.
@Alexander can correct me if I’m wrong an any of this, but I believe with the source package, you can use “pip install” already. You just have to point it to the generated source package rather than using the package manager. Given the limitations with ABI stability, I think publishing it on pypi.org will introduce a constant ongoing maintenance cost which we would rather solve by getting some of the fundamentals (ie weaker version dependencies) sorted first.
Also, on the documentation, we plan to make them available via the github pages. I think Alexander and his team have done an awesome job on the PyArmNN docs.
If you have more questions or suggestions, or if any of this doesn’t work for you, let us know.
Regards,
Derek
From: Matthew Bentham Sent: 12 February 2020 13:38 To: Pavel Macenauer; armnn-dev@lists.linaro.org; Georgios Pinitas; Derek Lamberti; Alexander Efremov Subject: Re: pyarmnn integration
+George, Derek, Alexander,
Please can you guys help Pavel? And keep the list on cc for visibility please.
Many thanks,
Matthew
________________________________
From: Armnn-dev armnn-dev-bounces@lists.linaro.org on behalf of Pavel Macenauer pavel.macenauer@nxp.com Sent: 10 February 2020 21:00 To: armnn-dev@lists.linaro.org armnn-dev@lists.linaro.org Subject: [Armnn-dev] pyarmnn integration
Hi!
There is a branch experimental/pyarmnn, created by Matthew Bentham, which contains python wrappers for armnn, which initially seems to work pretty well - building a whl archive works, which can be installed using pip and I was able to write an example, which runs inference on a float/quantized model and using all the supported frameworks - tf, tf-lite, caffe, onnx as well. What is missing is to get the python wrappers integrated, run and check unit tests and write a few examples. We discussed this with Matthew already, but I would be glad to hear more opinions regarding how we proceed and to kick off a discussion.
1. How to integrate pyarmnn?
There are 2 paths initially:
1. Build pyarmnn together with armnn using a single cmake command * By default it would be turned off, otherwise it would be build using e.g. -DBUILD_PYARMNN * The product is either a whl or a src package - so should there be 2 options e.g. -DBUILD_PYARMNN_SRC, -DBUILD_PYARMNN_WHL or only a single one, which would always build both? 2. Separate pyarmnn from armnn into a different repository (and keep it as a separate project) * Additionally to a) options -DARMNN_LIB and -DARMNN_INCLUDE would be required as well, so that it can be "linked" against configurable armnn build
The difference is mainly in maintainability - a) forces to maintain pyarmnn and update the swig files to generate wrappers per every release b) on the other hand keeps the project separate, allows to build pyarmnn with a configurable armnn release and doesn't create a dependency to update the swig files whenever armnn interface changes a little.
1. Remove tox? Yes/No - Tox is a python automation library, which is used to generate the wrappers or to run unit tests. It is not really needed, because the wrappers can be generated directly using swig and the src/whl packages generated using python/setuptools and it just creates another dependency. Unit tests can also be run directly using python. 2. Get pyarmnn published on pypi.org? Yes/No - we would be able to install pyarmnn using "pip install pyarmnn"
Any additional ideas, comments, feedback etc. would be of course appreciated.
Thanks! Pavel M _______________________________________________ Armnn-dev mailing list Armnn-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/armnn-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-
Alexander Efremov -
Derek Lamberti