[tfjs-tflite] Loading second TFLite model hangs in _emscripten_futex_wait

[reuben]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS 12.1
• TensorFlow.js installed from (npm or script link):

<!-- Import @tensorflow/tfjs-core -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core"></script>
<!-- Adds the CPU backend -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-cpu"></script>
<!--
Import @tensorflow/tfjs-tflite
Note that we need to explicitly load dist/tf-tflite.min.js so that it can
locate WASM module files from their default location (dist/).
-->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-tflite/dist/tf-tflite.min.js"></script>

• Browser version: Chrome 97

Describe the current behavior

Trying to load two TFLite models causes the browser to hang on the second tflite.loadTFLiteModel call.

Describe the expected behavior

Being able to load more than a single model at a time.

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/CodePen/any notebook.

I don't have an easy way to share the model files but here's the entire script: https://gist.github.com/reuben/4d1a77251ad73629f54429705ab022a9

Copy two TFLite model files to a folder and name one of them featurizer.tflite and the other output_graph.tflite, then open taht page, click the model picker and select both files. The script loops forever inside _emscripten_futex_wait on the second tflite.loadTFLiteModel call.

Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.

[josephrocca]

I'm seeing this same issue - I can only load one tflite model. The hang happens at model init, and only if COOP/COEP headers are set. Here's a minimal reproduction - you just need to serve it with COOP/COEP headers:

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@3.20.0/dist/tf.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-tflite@0.0.1-alpha.9/dist/tf-tflite.min.js"></script>
  
<script type="module">
  console.log("Attempting to load models...");
  let model1 = await tflite.loadTFLiteModel("https://huggingface.co/rocca/lyra-v2-soundstream/resolve/main/tflite/soundstream_encoder.tflite");
  let model2 = await tflite.loadTFLiteModel("https://huggingface.co/rocca/lyra-v2-soundstream/resolve/main/tflite/lyragan.tflite");
  console.log("Loaded models.")
</script>

It loads fine if you comment out one of the loadTFLiteModel calls.

I used this tfjs-tflite build by @jinjingforever and paused the execution in DevTools to see the stack trace:

Note: If the page isn't served with COOP/COEP headers, then both models seem to load without errors: https://jsbin.com/pahaxaromo/edit?html,output but these models use threads during inference, and so an error will be thrown when trying to use the models for prediction.

[shmishra99]

Hi @reuben ,
Apology for the late response. I've replicated this issue with latest version of @tensorflow/tfjs@4.4.0. It seems like it is working as expected. For your reference I have added the code snippet below.
Code:

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@3.20.0/dist/tf.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-tflite@0.0.1-alpha.9/dist/tf-tflite.min.js"></script>
  
<script type="module">
  async function x(){
  console.log("Attempting to load models...");
  let model1 = await tflite.loadTFLiteModel( 'https://tfhub.dev/tensorflow/lite-model/mobilenet_v2_1.0_224/1/metadata/1');
  console.log("Loaded model1.")
  console.log("Input shape(s):", model1.inputs.map(input => input.shape));
  console.log("Output shape(s):", model1.outputs.map(output => output.shape));
  let model2 = await tflite.loadTFLiteModel( "https://tfhub.dev/google/lite-model/imagenet/mobilenet_v3_small_075_224/classification/5/metadata/1");
  console.log("Loaded model2.")
  console.log("Loaded model2.");
  console.log("Input shape(s):", model2.inputs.map(input => input.shape));
  console.log("Output shape(s):", model2.outputs.map(output => output.shape));

  }
  x()
</script>

Output:

I have loaded two TFLite models in the same code flow and they have loaded perfectly.
Let me know if have I missed something here. Thank You !

[reuben]

I'm no longer working on this project, so I can't verify with the original models.

[shmishra99]

Alright, thank you for the confirmation. It seems like this issue has been resolved with the latest version of @tensorflow/tfjs@4.4.0. Do you need any further help with this issue, or could you please confirm if it has been resolved for you? Please feel free to close the issue if it is resolved. Thank you!

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.