Whisper slowed down greatly

[ArthurPeabody]

I transcribed 20 hour-long professionally-recorded radio programs that took about 4 hours each. The 21st took 7 hours. The 22nd took 6 hours and transcribed only 5 minutes' worth before I terminated it. That was with the tarball I downloaded February 1. I downloaded again February 21, ran it on the same file, the 22nd, and it seems to be working as quickly as before. I run these tasks over night, one daily. This seems odd.

[ggerganov]

Recent whisper.cpp versions (> 1.1.0) introduced temperature fallback which is enabled by default. In some cases when the decoding fails, if can fallback into a repetitive cycle and consume a lot of time.

You can disable the fallback strategy with the -no-fallback.
Additionally, adding -max-context 0 seems to reduce catastrophic failures, but comes at degraded transcription quality.

[tonytorm]

[tonytorm]

i noticed that setting the no_fallback to true only sets the temperature_increment to 0, and if i print the amount of fallbacks i see there are still some - especially regarding log_p failures, i am guessing that i have no control over the amount of fallbacks to allow?

[ggerganov]

When --no-fallback is specified, it prints the number of fallbacks, but they are never executed.
Can you confirm that this is the case? Otherwise, there might be a bug.

Also, you can control the thresholds via:

  -et N,     --entropy-thold N   [2.40   ] entropy threshold for decoder fail
  -lpt N,    --logprob-thold N   [-1.00  ] log probability threshold for decoder fail

[tonytorm]

you are right, the fallbacks are printed (but i guess are not happening) as I have a much shorter transcription time - and obviously a less accurate output.