Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: AI text to speech for proposal body #244

Merged
merged 10 commits into from
Mar 29, 2024
Merged

feat: AI text to speech for proposal body #244

merged 10 commits into from
Mar 29, 2024

Conversation

wa0x6e
Copy link
Contributor

@wa0x6e wa0x6e commented Mar 23, 2024

AI text to speech

Generate a .mp3 audio file through OpenAI API from the proposal's body

NOTE:

  • Will work only for body with at most 4096 chars (limit on OpenAI side), and at least 500 chars
  • Audio files are cached and saved on AWS after first call

Test

  • Ensure you have OPENAI_API_KEY env var set in your .env
  • (optional) In .env, set STORAGE_ENGINE to file, to save the mp3 file in /tmp locally, for easier access
  • Send a curl POST request to: http://localhost:3005/api/ai/tts/[PROPOSAL-ID] (choose a proposal with a body length between 500 and 4096 chars)
  • It should return a Buffer for an audio file (may take some time if file is not cached yet)
  • If you have set your storage engine to file, go to /tmp to find and listen to the audio file

@wa0x6e wa0x6e self-assigned this Mar 23, 2024
@wa0x6e wa0x6e added the enhancement New feature or request label Mar 23, 2024
Copy link

codecov bot commented Mar 24, 2024

Codecov Report

Attention: Patch coverage is 0% with 93 lines in your changes are missing coverage. Please review.

Files Patch % Lines
src/lib/ai/textToSpeech.ts 0.00% 49 Missing and 1 partial ⚠️
src/api.ts 0.00% 34 Missing ⚠️
src/lib/ai/summary.ts 0.00% 9 Missing ⚠️
Additional details and impacted files

📢 Thoughts on this report? Let us know!

@wa0x6e wa0x6e marked this pull request as ready for review March 24, 2024 08:50
@wa0x6e wa0x6e requested a review from bonustrack March 24, 2024 10:39
@wa0x6e
Copy link
Contributor Author

wa0x6e commented Mar 26, 2024

For reviewers, check the approved sister PR #240 for an idea on the class structure and flow

.env.example Show resolved Hide resolved
Comment on lines +12 to +13
AI_SUMMARY_SUBDIR=ai-summary
AI_TTS_SUBDIR=ai-tts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add default values for these, if someone forget. also not changing in .env.test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default value is undefined, as defined here

export function storageEngine(subDir?: string) {

@wa0x6e wa0x6e requested a review from ChaituVR March 28, 2024 12:47
@ChaituVR
Copy link
Member

Somehow i always get RECORD_NOT_FOUND when i go to http://localhost:3005/api/ai/tts/0x770ff4e02634c77aaa09952345551168920f7878b32ab03fcef92763a5fb70ab

Any idea?

@wa0x6e
Copy link
Contributor Author

wa0x6e commented Mar 28, 2024

Somehow i always get RECORD_NOT_FOUND when i go to http://localhost:3005/api/ai/tts/0x770ff4e02634c77aaa09952345551168920f7878b32ab03fcef92763a5fb70ab

Any idea?

Are you sending a POST request ?

}
});

router.post('/ai/tts/:id', async (req, res) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to use post here instead of get and for summery when we are not using any input from body

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very basic bot protection, to avoid link prefetch or any other script/tools calling this API endpoint

Copy link
Member

@ChaituVR ChaituVR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tAck

But i think we should check if requests are coming from snapshot.org
and also have some rate limit on these routes

@wa0x6e wa0x6e merged commit 5511e22 into main Mar 29, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants