Error code: 429 - Whisper AI (Free Tier) Issue and Potential Solution

[BetaMaxHeadroom]

Hi,

I ran into an issue that I think may have been addressed in issue 101 and 84 before.

Details:

• I am using a free OpenAI account.
• I added my API key to the overide.yaml file (the key is sandwiched between single quotes)
• I am using python 3.11.7
• OS: Windows 11
• I am not using this on my main PC at the moment. I downloaded this by extracting the .zip file. I did not download the executable hosted on Google Drive.
  • Please let me know if you think using the .exe would provide data in the right window of the GUI.

Functionality

When I execute the command python main.py in my command prompt, the GUI opens successfully. The console displays the following information:
[INFO] Listening to sound from Microphone: #29 - Microphone (REDACTED)
[INFO] Adjusting for ambient noise from Default Mic. Please make some noise from the Default Mic...
[INFO] Completed ambient noise adjustment for Default Mic.
[INFO] Listening to sound from Speaker: #25 - Speakers (REDACTED) [Loopback]
[INFO] Adjusting for ambient noise from Default Speaker. Please play sound from Default Speaker...
[INFO] Completed ambient noise adjustment for Default Speaker.
[INFO] Whisper using GPU: True
READY

All things seem to start fine. The 'issue' comes when I attempt to use the application in a conversational setting. In the GUI, the window on the

• left displays a transcription (not always correctly) of what was picked up by the microphone.
• right displays "Welcome to Transcribe" at the top, and no text/response is populated. Other than the banner at the top, the window is completely blank.

After the application interprets what was picked up by the microphone, the command prompt displays the response:
Error when attempting to get a response from LLM. Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}} [INFO] Exiting gracefully..

When I view my usage details at https://platform.openai.com/usage, the balance displays $0.00. I take this to mean that generating 'responses' requires a paid account. FWIW, I am not too surprised by this.

Items of Consideration

• It may be helpful to see something in the README.md addressing response generation not displaying without a paid OpenAI/Whisper API key.
  • This is your project. I am not telling you how to run it. I just wanted to provide a way of getting ahead of this question as this project becomes more popular.
• Have you considered using other open source models?
  • When attempting to identify something that could leveraged to avoid paying for Whisper AI, I initially found https://github.com/mudler/LocalAI.
  • This may be useful https://github.com/sindresorhus/awesome-whisper.
  • I also found https://github.com/ideasman42/nerd-dictation, which uses https://github.com/alphacep/vosk-api. I am not seeing anything about pricing. There is a chance that this could be a truly 'FREE' workaround.
    • These may be helpful https://github.com/projecte-aina/oTranscribe-plus-desktop and https://github.com/oTranscribe/oTranscribe.
• Would it be possible to edit/comment on the text that is displayed in the left window of the GUI?
  • To allow users the chance to correct some of the dialogue that may not have been recorded accurately.

[vivekuppal]

When I view my usage details at https://platform.openai.com/usage, the balance displays $0.00. I take this to mean that generating 'responses' requires a paid account. FWIW, I am not too surprised by this.

You are correct. Generating a response requires openAI credits. If your usage is limited, a couple of hours a day, you can get by with spending under USD 5 a month.

Thank you for directions. I will look into the suggestions.

[vivekuppal]

@BetaMaxHeadroom
#119

Thoughts, comments, suggestions, commits in the PR are all welcome

[vivekuppal]

Would it be possible to edit/comment on the text that is displayed in the left window of the GUI?
To allow users the chance to correct some of the dialogue that may not have been recorded accurately.

This will be included in an updated version of the GUI soonish.
Any further feedback on the user Interface is appreciated

[vivekuppal]

Started a discussion about new User Interface
#121

[BetaMaxHeadroom]

Happy holidays. Thank you again for making such an incredible tool. I had another question.

Please spend time with your family instead of responding to me today. Don't spend time on this. I just thought of it now, and I wanted to ask this before I forgot about it. This is likely something I should figure out on my own. I mostly just want to make a note.

• Would it be possible to use a different AI solution? I have recently discovered https://www.perplexity.ai/, and the responses seem better than the responses generated by text from OpenAI (on average).
  • Would I simply need to use the perplexity API key (in lieu of the OpenAI API key) to use perplexity? Or, is there something else that I would need to do?

[vivekuppal]

Happy holidays.
Good suggestion on Perplexity AI.

There are a multitude of potential solutions for LLM responses

1. A local RAG / LLM implementation, E.g. Mistral
2. Perplexity AI
3. Azure hosted OpenAI

For Perplexity, would have to make the following changes

1. Create a new section in parameters.yaml for the specific LLM and store the key there. Would have to plumb the api key appropriately to the necessary parts of the code
2. Make changes in the class class GPTResponder: in gpt_responder.py file. This class currently interacts with chatgpt, make changes to have it interact with Perplexity AI instead.

I believe changes to get Perplexity AI as LLM would be a great addition on many fronts.

Azure Hosted OpenAI I believe should be simple, just need to work out the mechanics of setting it up.
Local RAG / LLM implementation I anticipate would be fairly engaged and might take some doing.

[mang0sw33t]

Azure Hosted OpenAI I believe should be simple, just need to work out the mechanics of setting it up.

Should be able to use Azure Hosted OpenAI or Perplexity with relative ease with the latest code base.

Add the following 2 parameters to override.yaml file

OpenAI:
  api_key: 'API_KEY_FOR_THE_PROVIDER'
  base_url: 'BASE_URL_FOR_PROVIDER'

For OpenAI base_url is https://api.openai.com/v1