What's the performance like or the live transcription on a real device?

[chen-rn]

Hey!

Super excited to see this project! I'm going to try and run the example demo when I have some time in the upcoming days.

In the meanwhile, I was wondering if you know off the top of your mind what the performance for the real-time transcription is like, on a modern device!

Any video demos?

[jhen0409]

More documentation or demos will be available in the README, they may be later. I've recorded a one minute demo so you can watch it first:

RPReplay_Final1680223219.MP4

iPhone 13 Pro Max (4 threads, base.en model) transcribed from a snippet of Introducing GPT-4 video)

Also, I've moved this issue to Discussions.

[chefexperte]

For some reason on my Pocophone F1 it takes over a minute to process a second of recording, which is absolutely terrible and not realtime by any means.

[jhen0409]

Since there isn't much info here, I'm not really sure what's going on, but based on the Pocophone F1 specs I searched, I don't think it's that slow. It may be helpful to check what type of lib is loaded.

In my limited testing on Pixel 6, using the tiny.en model in release mode, each transcription just took ~2s.

Also, currently this package is only supported run on CPU for Android, unlike iOS has Accelerate Framework / CoreML support. It maybe support GPU (CLBlast for some devices? or Vulkan?) or NNAPI in the future.