Skip to content

Commit 8b62cf8

Browse files
am15ham15h
authored
Merge pull request #26 from am15h/v0.4.2
V0.4.2
2 parents e92502f + 2e30e90 commit 8b62cf8

File tree

4 files changed

+206
-167
lines changed

4 files changed

+206
-167
lines changed

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 0.4.2 (Jul 6, 2020)
2+
* Optimize getTensors and getTensor by Index
3+
* Update readme
4+
15
## 0.4.1 (Jun 23, 2020)
26
* Bug fix, output values copy to bytebuffer
37

‎README.md‎

Lines changed: 110 additions & 141 deletions
Original file line numberDiff line numberDiff line change
@@ -1,167 +1,99 @@
1-
2-
31
# TensorFlow Lite Flutter Plugin
42

5-
TensorFlow Lite plugin provides a dart API for accessing TensorFlow Lite interpreter and performing inference. It binds to TensorFlow Lite C API using dart:ffi.
3+
TensorFlow Lite plugin provides a dart API for accessing TensorFlow Lite interpreter and performing inference. It binds to TensorFlow Lite C API using dart:ffi.
64

7-
## (Most Important) Initial setup
5+
## (Important) Initial setup
86

9-
#### Add dynamic libraries to your app
7+
### Add dynamic libraries to your app
108

119
* **Linux/Mac Users**
1210

13-
Place the script [install.sh](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.sh) at the root of your project.
14-
15-
Execute <pre>sh [install.sh](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.sh)</pre> at the root of your project to automatically download and place binaries at appropriate folders.
11+
Place the script [install.sh](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.sh) at the root of your project.
1612

17-
*The binaries installed will **not** include support for `GpuDelegateV2` and `NnApiDelegate` however `InterpreterOptions().useNnApiForAndroid` can still be used.*
13+
Execute <pre>sh [install.sh](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.sh)</pre> at the root of your project to automatically download and place binaries at appropriate folders.
1814

19-
Use **`install.sh -d`** instead if you wish to use these `GpuDelegateV2` and `NnApiDelegate`.
15+
*The binaries installed will **not** include support for `GpuDelegateV2` and `NnApiDelegate` however `InterpreterOptions().useNnApiForAndroid` can still be used.*
2016

21-
* **Windows users**
17+
Use **`install.sh -d`** instead if you wish to use these `GpuDelegateV2` and `NnApiDelegate`.
2218

23-
Place the script [install.bat](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.bat) at the root of your project.
19+
* **Windows users**
2420

25-
Execute <pre>[install.bat](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.bat)</pre> at the root of your project to automatically download and place binaries at appropriate folders.
21+
Place the script [install.bat](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.bat) at the root of your project.
2622

27-
If you want to use delegate support then execute **`install.bat -d`**.
23+
Execute <pre>[install.bat](https://github.com/am15h/tflite_flutter_plugin/blob/master/install.bat)</pre> at the root of your project to automatically download and place binaries at appropriate folders.
2824

29-
These scripts install pre-built binaries based on latest stable tensorflow release.
25+
If you want to use delegate support then execute **`install.bat -d`**.
3026

27+
These scripts install pre-built binaries based on latest stable tensorflow release.
3128

32-
#### Why do we need to do this?
29+
For info about using other tensorflow versions refer to [this](#use-the-plugin-with-any-tensorflow-version) part of readme.
3330

34-
`tflite_flutter` dynamically links to C APIs which are supplied in the form of `libtensorflowlite_c.so` on Android and `TensorFlowLiteC.framework` on iOS.
35-
36-
For Android, We need to manually download these binaries from release assets and place the libtensorflowlite_c.so files in the `<root>/android/app/src/main/jniLibs/` directory for each arm, arm64, x86, x86_64 architecture as done here in the example app.  
31+
## Import
3732

38-
No setup needed for iOS as of now, `TensorFlowLiteC.framework` is embedded in the plugin itself.
33+
import 'package:tflite_flutter/tflite_flutter.dart';
3934

40-
#### How to build locally ?
35+
## Usage instructions
4136

42-
The pre-built binaries are updated with each stable tensorflow release. However, you many want to use latest unstable tf releases or older tf versions, for that proceed to build locally, if you are unable to find the required version in [release assets](https://github.com/am15h/tflite_flutter_plugin/releases).
37+
### Creating the Interpreter
4338

44-
Make sure you have required version of bazel installed. (Check TF_MIN_BAZEL_VERSION, TF_MAX_BAZEL_VERSION in configure.py)
39+
* **From asset**
4540

46-
* **Android**
41+
Place `your_model.tflite` in `assets` directory. Make sure to include assets in `pubspec.yaml`.
4742

48-
Configure your workspace for android builds as per [these instructions](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/android.md#configure-workspace-and-bazelrc).
43+
```dart
44+
final interpreter = await tfl.Interpreter.fromAsset('your_model.tflite');
45+
```
4946
50-
For TensorFlow >= v2.2
51-
```
52-
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/c:tensorflowlite_c
53-
54-
// similarily for arm64 use --config=android_arm64
55-
```
47+
Refer to the documentation for info on creating interpreter from buffer or file.
5648
57-
For TensorFlow <= v2.1
58-
```
59-
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/experimental/c:libtensorflowlite_c.so
60-
61-
// similarily for arm64 use --config=android_arm64
62-
```
49+
### Performing inference
6350
64-
* **iOS**
51+
See [TFLite Flutter Helper Library](https://www.github.com/am15h/tflite_flutter_helper) for easy processing of input and output.
6552
66-
Refer [instructions on TensorFlow Lite website](https://www.tensorflow.org/lite/guide/build_ios#install_bazel) to build locally for iOS.
53+
* **For single input and output**
6754
68-
Note: You must use macOS for building iOS.
55+
Use `void run(Object input, Object output)`.
56+
```dart
57+
// For ex: if input tensor shape [1,5] and type is float32
58+
var input = [[1.23, 6.54, 7.81. 3.21, 2.22]];
6959
70-
## Import
60+
// if output tensor shape [1,2] and type is float32
61+
var output = List(1*2).reshape([1,2]);
7162
72-
import 'package:tflite_flutter/tflite_flutter.dart';
63+
// inference
64+
interpreter.run(input, output);
7365
74-
## Usage instructions
66+
// print the output
67+
print(output);
68+
```
69+
70+
* **For multiple inputs and outputs**
7571
76-
### Creating the Interpreter
72+
Use `void runForMultipleInputs(List<Object> inputs, Map<int, Object> outputs)`.
7773
74+
```dart
75+
var input0 = [1.23];
76+
var input1 = [2.43];
7877
79-
#### Interpreter can be created in three ways:
80-
81-
* **directly from asset** (easiest)
82-
83-
Place `your_model.tflite` in `assets` directory. Make sure to include assets in `pubspec.yaml`.
84-
85-
```
86-
final interpreter = await tfl.Interpreter.fromAsset('your_model.tflite');
87-
```
88-
89-
* **from buffer**
90-
```
91-
final buffer = await getBuffer('assets/your_model.tflite');
92-
final interpreter = Interpreter.fromBuffer(buffer);
93-
94-
Future<Uint8List> getBuffer(String filePath) async {
95-
final rawAssetFile = await rootBundle.load(filePath);
96-
final rawBytes = rawAssetFile.buffer.asUint8List();
97-
return rawBytes;
98-
}
99-
```
100-
101-
* **from file**
102-
103-
```
104-
final dataFile = await getFile('assets/your_model.tflite');
105-
final interpreter = Interpreter.fromFile(dataFile);
106-
107-
Future<File> getFile(String fileName) async {
108-
final appDir = await getTemporaryDirectory();
109-
final appPath = appDir.path;
110-
final fileOnDevice = File('$appPath/$fileName');
111-
final rawAssetFile = await rootBundle.load(fileName);
112-
final rawBytes = rawAssetFile.buffer.asUint8List();
113-
await fileOnDevice.writeAsBytes(rawBytes, flush: true);
114-
return fileOnDevice;
115-
}
116-
```
78+
// input: List<Object>
79+
var inputs = [input0, input1, input0, input1];
11780
118-
### Performing inference
81+
var output0 = List<double>(1);
82+
var output1 = List<double>(1);
11983
120-
* **For single input and output**
121-
122-
Use `void run(Object input, Object output)`.
123-
```
124-
// For ex: if input tensor shape [1,5] and type is float32
125-
var input = [[1.23, 6.54, 7.81. 3.21, 2.22]];
126-
127-
// if output tensor shape [1,2] and type is float32
128-
var output = List(1*2).reshape([1,2]);
129-
130-
// inference
131-
interpreter.run(input, output);
132-
133-
// print the output
134-
print(output);
135-
```
136-
84+
// output: Map<int, Object>
85+
var outputs = {0: output0, 1: output1};
13786
87+
// inference
88+
interpreter.runForMultipleInputs(inputs, outputs);
13889
139-
* **For multiple inputs and outputs**
140-
141-
Use `void runForMultipleInputs(List<Object> inputs, Map<int, Object> outputs)`.
142-
```
143-
var input0 = [1.23];
144-
var input1 = [2.43];
145-
146-
// input: List<Object>
147-
var inputs = [input0, input1, input0, input1];
148-
149-
var output0 = List<double>(1);
150-
var output1 = List<double>(1);
151-
152-
// output: Map<int, Object>
153-
var outputs = {0: output0, 1: output1};
154-
155-
// inference
156-
interpreter.runForMultipleInputs(inputs, outputs);
157-
158-
// print outputs
159-
print(outputs)
160-
```
90+
// print outputs
91+
print(outputs)
92+
```
16193
16294
### Closing the interpreter
16395
164-
```
96+
```dart
16597
interpreter.close();
16698
```
16799

@@ -171,59 +103,96 @@ interpreter.close();
171103

172104
* **NNAPI delegate for Android**
173105

174-
```
106+
```dart
175107
var interpreterOptions = InterpreterOptions()..useNnApiForAndroid = true;
176108
final interpreter = await Interpreter.fromAsset('your_model.tflite',
177109
options: interpreterOptions);
178-
110+
179111
```
112+
180113
or
181-
182-
```
114+
115+
```dart
183116
var interpreterOptions = InterpreterOptions()..addDelegate(NnApiDelegate());
184117
final interpreter = await Interpreter.fromAsset('your_model.tflite',
185118
options: interpreterOptions);
186-
187-
```
188119
120+
```
189121
190122
* **GPU delegate for Android and iOS**
191123
192-
* **Android** GpuDelegateV2
124+
* **Android** GpuDelegateV2
193125
194-
```
126+
```dart
195127
final gpuDelegateV2 = GpuDelegateV2(
196128
options: GpuDelegateOptionsV2(
197-
false,
198-
TfLiteGpuInferenceUsage.fastSingleAnswer,
199-
TfLiteGpuInferencePriority.minLatency,
200-
TfLiteGpuInferencePriority.auto,
201-
TfLiteGpuInferencePriority.auto,
129+
false,
130+
TfLiteGpuInferenceUsage.fastSingleAnswer,
131+
TfLiteGpuInferencePriority.minLatency,
132+
TfLiteGpuInferencePriority.auto,
133+
TfLiteGpuInferencePriority.auto,
202134
));
203135
204136
var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegateV2);
205137
final interpreter = await Interpreter.fromAsset('your_model.tflite',
206138
options: interpreterOptions);
207139
```
208140
209-
* **iOS** Metal Delegate (GpuDelegate)
141+
* **iOS** Metal Delegate (GpuDelegate)
210142
211-
```
143+
```dart
212144
final gpuDelegate = GpuDelegate(
213145
options: GpuDelegateOptions(true, TFLGpuDelegateWaitType.active),
214146
);
215147
var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegate);
216148
final interpreter = await Interpreter.fromAsset('your_model.tflite',
217149
options: interpreterOptions);
218-
219150
```
220151
221-
222152
Refer [Tests](https://github.com/am15h/tflite_flutter_plugin/blob/master/example/test/tflite_flutter_plugin_example_e2e.dart) to see more example code for each method.
223153
224154
Refer [Text Classification Flutter Example App](https://github.com/am15h/tflite_flutter_plugin/tree/master/example) for demo.
225155
156+
#### Use the plugin with any tensorflow version
157+
158+
The pre-built binaries are updated with each stable tensorflow release. However, you many want to use latest unstable tf releases or older tf versions, for that proceed to build locally, if you are unable to find the required version in [release assets](https://github.com/am15h/tflite_flutter_plugin/releases).
159+
160+
Make sure you have required version of bazel installed. (Check TF_MIN_BAZEL_VERSION, TF_MAX_BAZEL_VERSION in configure.py)
161+
162+
* **Android**
163+
164+
Configure your workspace for android builds as per [these instructions](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/android.md#configure-workspace-and-bazelrc).
165+
166+
For TensorFlow >= v2.2
167+
168+
```
169+
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/c:tensorflowlite_c
170+
171+
// similarily for arm64 use --config=android_arm64
172+
```
173+
174+
For TensorFlow <= v2.1
175+
```
176+
bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/experimental/c:libtensorflowlite_c.so
177+
178+
// similarily for arm64 use --config=android_arm64
179+
```
180+
181+
* **iOS**
182+
183+
Refer [instructions on TensorFlow Lite website](https://www.tensorflow.org/lite/guide/build_ios#install_bazel) to build locally for iOS.
184+
185+
Note: You must use macOS for building iOS.
186+
187+
#### More info on dynamic linking
188+
189+
`tflite_flutter` dynamically links to C APIs which are supplied in the form of `libtensorflowlite_c.so` on Android and `TensorFlowLiteC.framework` on iOS.
190+
191+
For Android, We need to manually download these binaries from release assets and place the libtensorflowlite_c.so files in the `<root>/android/app/src/main/jniLibs/` directory for each arm, arm64, x86, x86_64 architecture as done here in the example app.  
192+
193+
No setup needed for iOS as of now, `TensorFlowLiteC.framework` is embedded in the plugin itself.
194+
226195
## Credits
227196
228197
* Tian LIN, Jared Duke, Andrew Selle, YoungSeok Yoon, Shuangfeng Li from the TensorFlow Lite Team for their invaluable guidance.
229-
* Authors of [dart-lang/tflite_native](https://github.com/dart-lang/tflite_native).
198+
* Authors of [dart-lang/tflite_native](https://github.com/dart-lang/tflite_native).

0 commit comments

Comments
 (0)