Cannot convert to tflite, problem with lambda

[mbartosi]

Hi, I want to convert weights to tflite using tflite_convert --keras_model_file=yolov3-tiny.h5 --output_file=yolov3-tiny.tflite
It fails with the following traceback:

Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0513 14:27:21.003883 140717950494528 deprecation.py:506] From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Zeros.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0513 14:27:21.004970 140717950494528 deprecation.py:506] From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Ones.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:820: UserWarning: yolov3_tf2.models is not loaded, but a Lambda layer uses it. It may cause errors.
, UserWarning)
Traceback (most recent call last):
File "/usr/local/bin/tflite_convert", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 448, in main
app.run(main=run_main, argv=sys.argv[:1])
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.7/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 444, in run_main
_convert_model(tflite_flags)
File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 123, in _convert_model
converter = _get_toco_converter(flags)
File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 110, in _get_toco_converter
return converter_fn(**converter_kwargs)
File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 627, in from_keras_model_file
keras_model = _keras.models.load_model(model_file)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 215, in load_model
custom_objects=custom_objects)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/model_config.py", line 55, in model_from_config
return deserialize(config, custom_objects=custom_objects)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/serialization.py", line 95, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 192, in deserialize_keras_object
list(custom_objects.items())))
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1231, in from_config
process_layer(layer_data)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1215, in process_layer
layer = deserialize_layer(layer_data, custom_objects=custom_objects)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/serialization.py", line 95, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 192, in deserialize_keras_object
list(custom_objects.items())))
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1241, in from_config
process_node(layer, node_data)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1197, in process_node
layer(flat_input_tensors[0], **kwargs)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 612, in call
outputs = self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 768, in call
return self.function(inputs, **arguments)
File "/home/mba/GitHub/yolov3-tf2/yolov3_tf2/models.py", line 139, in
x = Lambda(lambda x: import tensorflow as tf; tf.reshape(x, (-1, tf.shape(x)[1], tf.shape(x)[2], anchors, classes + 5)))(x)
NameError: name 'tf' is not defined

I found similar problem here: https://stackoverflow.com/questions/54347963/tf-is-not-defined-on-load-model-using-lambda

How could I solve it?

[zzh8829]

I tried running tflite_convert in tensorflow 2.0 it says
tflite_convert is currently unsupported in 2.0

Following the instruction on TF2.0 API
export to tfserving saved_model first

python export_tfserving.py --output serving/yolov3/1/

run these in python

import tensorflow as tf
model = tf.saved_model.load("serving/yolov3/1")
converter = tf.lite.TFLiteConverter.from_concrete_function(model.signatures['serving_default'])
tflite_model = converter.convert() 
open("converted_model.tflite", "wb").write(tflite_model)

This should work but i'm getting weird errors

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 261, in convert
    shape_list = tensor.get_shape().as_list()
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 1128, in as_list
    raise ValueError("as_list() is not defined on an unknown TensorShape.")
ValueError: as_list() is not defined on an unknown TensorShape.

¯_(ツ)_/¯

Sorry I can't really help right now, i guess we will have to wait for new updates from tflite 2.0.

[gwestner94]

Is there any update for this?
I would love to try using the model in tflite.

[zzh8829]

I followed the new instruction with model support
https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter

model = YoloV3()
model.load_weights('weights.tf')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

But unfortunately i got this error

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

You can try it too and see if it works

[Lap1n]

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

[ghost]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

[ghost]

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

Hey, Tensorflow 2.0 RC1 is released and all my error messages have now disappeared.
Can anyone confirm that?

[1165048017]

For "NameError: name 'tf' is not defined"
you can add the 'custom_objects' parameter in load_model function:
tf.keras.models.load_model('xxx.h5',custom_objects={"tf":tf})

I also met the problem 'keras is not defined' while load model, the same method to solve it:
tf.keras.models.load_model('xxx.h5',custom_objects={"keras":tf.keras})

[peace195]

Here is my solution: https://github.com/peace195/tensorflow-lite-yolo-v3

The .pb should be in right format (SavedModel).

Please try it. I would appreciate if you give me a star for this project 👍

[JJ306]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

[ghost]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version of Tensorflow do you use?

[JJ306]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,
You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version o

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,
You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version of Tensorflow do you use?

I am using TF2.0 & Anaconda on windows 10.

[dao-kun]

those code is also works for me , but the the tfliite output is totally different, did anyone of you meet this before?
`yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)`

my tflite input :
[{'name': 'input_1', 'index': 3, 'shape': array([ 1, 416, 416, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

output :

[{'name': 'Identity', 'index': 0, 'shape': array([ 1, 13, 13, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_1', 'index': 1, 'shape': array([ 1, 26, 26, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_2', 'index': 2, 'shape': array([ 1, 52, 52, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

[zzh8829]

Looks like TFLite converter has problem with CombinedNonMaxSupression operator, we will have to wait for official support for now

[zzh8829]

tensorflow/tensorflow#34693

[DanKlarman]

Hi, amazing work here!
any update on tflite issue?

[lijiong16]

I tried to convert my model to .tflite but got the following error.

RuntimeError: Encountered unresolved custom op: ResizeNearestNeighbor.Node number 26 (ResizeNearestNeighbor) failed to prepare.

[edurenye]

CombinedNonMaxSupression is not whitelisted: tensorflow/tensorflow#37301

[edurenye]

But as it is suggested here: tensorflow/tensorflow#33059 (comment)

We could replace it for tf.image.non_max_suppression_with_scores

Then everything would just work, tflite, onnx...