This project integrates Unity3D bindings for whisper.cpp to run OpenAI's Whisper ASR model locally on Meta Quest 3. Based on the original whisper.unity repository by @Macoron.
- High-performance inference of Whisper ASR model
- Supports around 60 languages
- Can translate between languages
- Runs entirely on Meta Quest 3 without Internet connection
- Free and open source
This project uses the Whisper model to transcribe a sample scene with JFK's audio file containing his famous line: "Ask not what your country can do for you – ask what you can do for your country." The transcription tests were conducted to measure latency on the Meta Quest 3 headset.
Clone this repository and open it as a regular Unity project. It comes with examples and a tiny multilingual model.
Alternatively, add this repository to your project as a Unity Package using the following git URL:
https://github.com/Macoron/whisper.unity.git?path=/Packages/com.whisper.unity
You can try different Whisper model weights to improve transcription quality. Download model weights from here and place them in your StreamingAssets folder.
This project is licensed under the MIT License. It uses compiled libraries and model weights from whisper.cpp, also licensed under MIT.