[WIP] Feature add runSignature function

[ghost]

This pull request adds the ability to run signatures in Flutter TFLite.

List of functions added and their testing status:

• getSignatureInputCount(String signatureKey)
• getSignatureOutputCount(String signatureKey)
• getSignatureInputName(String signatureKey, int index)
• getSignatureOutputName(String signatureKey, int index)
• getSignatureInputTensorShape(String signatureKey, String inputName)
• getSignatureOutputTensorShape(String signatureKey, String outputName)
• runSignature(Map<String, Object> inputs, Map<String, Object> outputs, String signatureKey)
• deleteSignatureRunner(String signatureKey)

[CaptainDario]

@st-duymai are you still working on this? It would be an awesome addition!

[ghost]

@st-duymai are you still working on this? It would be an awesome addition!

I'm still working on it. I'm doing it with the MoviNet sample. When I finish it, it would be great if you could help me review it.

[CaptainDario]

Yes, sure. Ping me once you are ready. Due to some personal matters, I am not really active here, so maybe you need to ping me twice.

[michaeltys]

Hi @st-duymai , thanks for your contribution. I was testing the movinet sample and found that inference takes 1 - 2 seconds to complete for a single frame. I checked with multiple frame dimensions (from 2020 to 10001000) and the result is the same. Do you see a possible way to improve the performance here?

I took android implementation as a reference, the inference there takes 20-50 milliseconds on the same data even using Flutter Chanel for native code integration

[CaptainDario]

In my experience, TF Lite flutter is close to the native c performance.

Are you sure that you only measure the inference latency? Not the frame decoding from the camera?

[michaeltys]

In my experience, TF Lite flutter is close to the native c performance.

Are you sure that you only measure the inference latency? Not the frame decoding from the camera?

Yes, I measured just the inference time. Frame decoding takes really little in comparison to the inference in this case

[CaptainDario]

@michaeltys, did you try running it in the native tf lite Benchmark app?

Also, could you provide a link to the model?

[ghost]

@michaeltys Yes, I know this issue. The reason is when I use isolate, I can't remain all tensors and signatures during each inference.

[ghost]

I am closing this pull request as I will no longer be using this GitHub account. I may create a new pull request later.