Roughcut, native MacOS app using Whisper.cpp + GPT-3.5

[batlounis]

Couldn't have done this without the awesome whisper.cpp, seriously huge thanks @ggerganov :)

We’re dealing with a lot of videos that are dialogue-heavy. From YouTube clips and podcasts to lectures and recorded meetings, sorting through all this content can be a challenge. And if you want to chop out pieces of the original footage, that can slow everything down.

So I created a Mac app called Roughcut that distills hours of dialogue footage into a meaningful sequence.

Features

1. Transcribe footage using Whisper.cpp.
2. Call gpt-3.5-turbo to choose the most meaningful segments from the transcript. This happens on backend server running python.
3. Infer timestamps by looking up GPT results in original Whisper transcription
4. Generate a new video based on the new segments

Whisper.cpp usage
I use word-level timestamps for later inference. Haven't tried diarization yet. Since whisper.cpp supports CoreML the speed is incredible. Due to the nature of Whisper, timestamps can be inaccurate sometimes, especially at the word-level.

Demo video / landing page

[bobqianic]

If you could provide the option to use LLaMA 2, that would be fantastic. The gpt-3.5-turbo API is not inexpensive, and if you use it extensively, it can easily cost you an arm and a leg.