Hi,
We are using ArmNN to cross-compile a standalone C++ application on Linux
that loads a standard onnx model. During the model loading, we see a crash
with the below error output -
terminate called after throwing an instance of
'armnn::InvalidArgumentException'
what(): Tensor numDimensions must be greater than 0
Initially we were on armnn master, and later we switched to tag v19.05, but
the error was same for both.
Below is the code snippet to load the model -
armnnOnnxParser::IOnnxParserPtr parser =
armnnOnnxParser::IOnnxParser::Create();
std::cout << "\nmodel load start";
armnn::INetworkPtr network =
parser->CreateNetworkFromBinaryFile("onnx_3DDFA.onnx");
std::cout << "\nmodel load end";
It crashes after printing "model load start" with the error message printed
above.
A gdb backtrace is also provided below -
(gdb) r
Starting program:
/home/root/Rahul/armnn_onnx/3DDFA_ArmNN_onnx/3ddfa_armnn_onnx
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
terminate called after throwing an instance of
'armnn::InvalidArgumentException'
what(): Tensor numDimensions must be greater than 0
model load start
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at
/usr/src/debug/glibc/2.26-r0/git/sysdeps/unix/sysv/linux/raise.c:51
51 }
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at
/usr/src/debug/glibc/2.26-r0/git/sysdeps/unix/sysv/linux/raise.c:51
#1 0x0000ffffbe41df00 in __GI_abort () at
/usr/src/debug/glibc/2.26-r0/git/stdlib/abort.c:90
#2 0x0000ffffbe6aa0f8 in __gnu_cxx::__verbose_terminate_handler() () from
/usr/lib/libstdc++.so.6
#3 0x0000ffffbe6a7afc in ?? () from /usr/lib/libstdc++.so.6
#4 0x0000ffffbe6a7b50 in std::terminate() () from /usr/lib/libstdc++.so.6
#5 0x0000ffffbe6a7e20 in __cxa_throw () from /usr/lib/libstdc++.so.6
#6 0x0000ffffbefdad84 in armnn::TensorShape::TensorShape(unsigned int,
unsigned int const*) () from /home/root/Rahul/armnn_onnx/build/libarmnn.so
#7 0x0000ffffbe7e34d8 in armnnOnnxParser::(anonymous
namespace)::ToTensorInfo(onnx::ValueInfoProto const&) [clone
.constprop.493] () from
/home/root/Rahul/armnn_onnx/build/libarmnnOnnxParser.so
#8 0x0000ffffbe7e4080 in
armnnOnnxParser::OnnxParser::SetupInfo(google::protobuf::RepeatedPtrField<onnx::ValueInfoProto>
const*) () from /home/root/Rahul/armnn_onnx/build/libarmnnOnnxParser.so
#9 0x0000ffffbe7e41ac in armnnOnnxParser::OnnxParser::LoadGraph() () from
/home/root/Rahul/armnn_onnx/build/libarmnnOnnxParser.so
#10 0x0000ffffbe7e4760 in
armnnOnnxParser::OnnxParser::CreateNetworkFromModel(onnx::ModelProto&) ()
from /home/root/Rahul/armnn_onnx/build/libarmnnOnnxParser.so
#11 0x0000ffffbe7e49b0 in
armnnOnnxParser::OnnxParser::CreateNetworkFromBinaryFile(char const*) ()
from /home/root/Rahul/armnn_onnx/build/libarmnnOnnxParser.so
#12 0x0000000000402290 in main ()
(gdb)
Can someone point out if we are missing something out or doing something
wrong? Any help or input is highly appreciated.
Regards,
Rahul
--
This
message contains confidential information and is intended only
for the
individual(s) named. If you are not the intended
recipient, you are
notified that disclosing, copying, distributing or taking any
action in
reliance on the contents of this mail and attached file/s is strictly
prohibited. Please notify the
sender immediately and delete this e-mail
from your system. E-mail transmission
cannot be guaranteed to be secured or
error-free as information could be
intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain
viruses. The sender therefore does
not accept liability for any errors or
omissions in the contents of this
message, which arise as a result of e-mail
transmission.
Hi Derek,
Thanks for your reply and I'm glad we agree on this.
Are there any tickets/issues which I can use to track the changes that you have suggested?
Thanks,
Rob
From: Derek Lamberti <Derek.Lamberti(a)arm.com>
Sent: 13 August 2019 14:01
To: Matthew Bentham <Matthew.Bentham(a)arm.com>; Robert Hughes <Robert.Hughes(a)arm.com>; Armnn-dev(a)lists.linaro.org
Subject: Re: Validation of inputs
Hi Rob,
Yes, I think this is certainly an area where we should do better:
1. For completeness, there are different levels of validation that need to occur. This can be different from the validation performed by the backend::IsSupported() functions. For example, IsSupported only needs to report what is valid for that backend implementation, which may cover only a subset of the full ArmNN specification for the layer. Also worth mentioning.
2. There are different stages where we should perform the validation.
* On the input graph during Graph building (much like you suggested). This would be validation against the ArmNN spec and would indicate to the user immediately (at the point of error) that they have tried to add an invalid op.
* On the LoadedNetwork during workload creation. This is essentially what the current code does and is a validation against the ArmNN spec. However, it's currently performed during the construction of the workloads, and should instead be called by the ArmNN framework just before, which would be independent of workload implementation. I would also make this check for debug builds only. It's useful to validate that all the graph transformations to this point have been valid.
1. Furthermore, there are different stages where we could perform further validation.
* In the optimizer (post-backend-optimization) to verify the optimized result. This is essentially, the same as 2b (above) but earlier in the pipe and would give better user experience. It is required because backend optimization implementations could produce an invalid graph. If we do it early enough in the optimizer pipeline, we could use it to reject invalid optimizations from the backends, and fall-back to the next backend instead of failing outright.
* It would remain up to the back implementations to verify that the workloads created are compatible with their implementations. Similar to the IsSupported() but during workload creation (like we are doing now). This could also be made a debug only option.
1. InferTensorInfos should certainly be safe code. We will soon be updating this code so that it can be used to actually infer the shapes for tensors where the shape is unknown in the model (rather than just for validation). Suffice it to say, the current implementation could be safer.
Thanks for your feedback and keep it coming. I'm eager to make ArmNN a lot more user friendly in the coming year, so this all helps.
Regards,
~Derek
________________________________
From: Matthew Bentham <Matthew.Bentham(a)arm.com<mailto:Matthew.Bentham@arm.com>>
Sent: 13 August 2019 12:00
To: Robert Hughes <Robert.Hughes(a)arm.com<mailto:Robert.Hughes@arm.com>>; Armnn-dev(a)lists.linaro.org<mailto:Armnn-dev@lists.linaro.org> <Armnn-dev(a)lists.linaro.org<mailto:Armnn-dev@lists.linaro.org>>
Cc: Derek Lamberti <Derek.Lamberti(a)arm.com<mailto:Derek.Lamberti@arm.com>>
Subject: Re: Validation of inputs
Thanks Rob, that does seem wrong. At initial glance it looks to me like QueueDescriptor::Validate should not exist, and all that checking should move to roughly where InferTensorInfos is called now. I'll let Derek comment further.
All the best,
Matthew
________________________________
From: Armnn-dev <armnn-dev-bounces(a)lists.linaro.org<mailto:armnn-dev-bounces@lists.linaro.org>> on behalf of Robert Hughes <Robert.Hughes(a)arm.com<mailto:Robert.Hughes@arm.com>>
Sent: 13 August 2019 11:52
To: Armnn-dev(a)lists.linaro.org<mailto:Armnn-dev@lists.linaro.org> <Armnn-dev(a)lists.linaro.org<mailto:Armnn-dev@lists.linaro.org>>
Subject: [Armnn-dev] Validation of inputs
Hi ArmNN dev team,
I am part of the team developing the ArmNN backend for the Arm NPU and have some concerns about the validation that the ArmNN core library performs on its inputs. Below is a description of how I believe validation is performed within ArmNN and the problems that I see with this. This understanding may be flawed so please correct me where I have misunderstood.
When the user creates an INetwork there is minimal validation of the data provided by the user. For example, the dimensionality of input tensors is not checked at this point. The user then calls Optimize() which performs the following steps:
1. InferTensorInfos() - this calls ValidateTensorShapesFromInputs on each Layer in the Graph which confirms that the output tensor shape set on each Layer during Network construction is consistent with the Layer's inputs. For the example of a FullyConnectedLayer, this uses the shape of the input and the shape of the weights to determine the correct output shape. This code seems to make assumptions about the dimensionality of the inputs tensors, for example FullyConnectedLayer::InferOutputShapes() indexes into the input and weight shapes without checking their dimensionality first.
2. AssignBackends() - this calls each backend's IsLayerSupported() APIs. The only data that has been validated so far is that the output shapes of each layer are correct, so the backend IsLayerSupported() APIs cannot assume anything about the shapes of the tensors. This means the backends must perform additional validation.
3. ApplyBackendOptimizations() - this gives each backend the opportunity to "optimize" each subgraph which has been assigned to it. Again, the layers passed to the backend still have not been properly validated, although the backend has had the chance to reject the layers via the IsLayerSupported() APIs.
The user then creates a LoadedNetwork from the IOptimizedNetwork which creates the Workloads. This is delegated to the backend's IWorkloadFactory which is responsible for returning an object implementing IWorkload. In the case of the default backends (reference, Neon, CL), these workloads derive from BaseWorkload, which calls Validate() on the QueueDescriptor for that workload type. This is the place that seems to perform the "proper" validation of what is supported by ArmNN. In the example of Fully Connected, FullyConnectedQueueDescriptor::Validate checks the dimensionality of all tensors, the quantisastion infos, etc. Note that there seems to be no requirement that this validation code is called at all, in the case that the backend-created workloads do not inherit BaseWorkload (this is always the case for backends which replace subgraphs with PreCompiledLayers).
The problems that this causes are as follows:
* The InferTensorInfos() code could crash as it makes assumptions that have not been validated
* Every backend's IsLayerSupported APIs must duplicate the validation code in ArmNN in order to check that the layer is valid, before they even get to the point of checking if that particular backend supports it.
* The "proper" ArmNN layer validation code may never be run, depending on how the backend processes the graph. Specifically, in the case of backends which replace subgraphs with PreCompiledLayers, the validation code is never run.
* These problems affect both end users of the ArmNN API and backend developers
I would suggest that a better method of validation would be to validate the INetwork completely before it is processed any further. This could be done during construction of the INetwork or as the first step in Optimize(). This would simplify the backend code as it would not need to duplicate ArmNN's validation code and give a more consistent interface to end users.
Please let me know your thoughts,
Thanks,
Rob
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________
Armnn-dev mailing list
Armnn-dev(a)lists.linaro.org<mailto:Armnn-dev@lists.linaro.org>
https://lists.linaro.org/mailman/listinfo/armnn-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi ArmNN dev team,
I am part of the team developing the ArmNN backend for the Arm NPU and have some concerns about the validation that the ArmNN core library performs on its inputs. Below is a description of how I believe validation is performed within ArmNN and the problems that I see with this. This understanding may be flawed so please correct me where I have misunderstood.
When the user creates an INetwork there is minimal validation of the data provided by the user. For example, the dimensionality of input tensors is not checked at this point. The user then calls Optimize() which performs the following steps:
1. InferTensorInfos() - this calls ValidateTensorShapesFromInputs on each Layer in the Graph which confirms that the output tensor shape set on each Layer during Network construction is consistent with the Layer's inputs. For the example of a FullyConnectedLayer, this uses the shape of the input and the shape of the weights to determine the correct output shape. This code seems to make assumptions about the dimensionality of the inputs tensors, for example FullyConnectedLayer::InferOutputShapes() indexes into the input and weight shapes without checking their dimensionality first.
2. AssignBackends() - this calls each backend's IsLayerSupported() APIs. The only data that has been validated so far is that the output shapes of each layer are correct, so the backend IsLayerSupported() APIs cannot assume anything about the shapes of the tensors. This means the backends must perform additional validation.
3. ApplyBackendOptimizations() - this gives each backend the opportunity to "optimize" each subgraph which has been assigned to it. Again, the layers passed to the backend still have not been properly validated, although the backend has had the chance to reject the layers via the IsLayerSupported() APIs.
The user then creates a LoadedNetwork from the IOptimizedNetwork which creates the Workloads. This is delegated to the backend's IWorkloadFactory which is responsible for returning an object implementing IWorkload. In the case of the default backends (reference, Neon, CL), these workloads derive from BaseWorkload, which calls Validate() on the QueueDescriptor for that workload type. This is the place that seems to perform the "proper" validation of what is supported by ArmNN. In the example of Fully Connected, FullyConnectedQueueDescriptor::Validate checks the dimensionality of all tensors, the quantisastion infos, etc. Note that there seems to be no requirement that this validation code is called at all, in the case that the backend-created workloads do not inherit BaseWorkload (this is always the case for backends which replace subgraphs with PreCompiledLayers).
The problems that this causes are as follows:
* The InferTensorInfos() code could crash as it makes assumptions that have not been validated
* Every backend's IsLayerSupported APIs must duplicate the validation code in ArmNN in order to check that the layer is valid, before they even get to the point of checking if that particular backend supports it.
* The "proper" ArmNN layer validation code may never be run, depending on how the backend processes the graph. Specifically, in the case of backends which replace subgraphs with PreCompiledLayers, the validation code is never run.
* These problems affect both end users of the ArmNN API and backend developers
I would suggest that a better method of validation would be to validate the INetwork completely before it is processed any further. This could be done during construction of the INetwork or as the first step in Optimize(). This would simplify the backend code as it would not need to duplicate ArmNN's validation code and give a more consistent interface to end users.
Please let me know your thoughts,
Thanks,
Rob
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
