Stream camera frame to ObjectDetector?

[lucksp]

I have been using React-Native-Vision-Camera + react-native-fast-tflite. Unfortunately, I need to investigate an alternate solution to passing images to the model. This happens seamlessly with RNFastTFLite. I noticed with this MLKIt library, that a uri path "string" is needed to pass to the model. That would require converting the frame from RNVisionCamera's frameProcessor to an image on the device and then create a temp file path. This is going to be very memory & storage intensive, so not a good idea.

Does this MLKIt work well with frame streams, or is it designed to be more for single image processing? I see you can set detectorMode: 'stream' option for when initializing the original model options, but it still requires a string path?

[Rvey]

Yeah same issue

[trevor-coleman]

This is a good question -- I think there is likely a way to implement this, but it'll take some work.

MLKit definitely supports streaming, but we'd need to implement a module interface for streams. We should probably remove the option in the meantime, to prevent confusion.