Deepgram streaming #101
Merged
Deepgram streaming #101
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
✅ Deploy Preview for openduck canceled.
|
zachwe
reviewed
Mar 25, 2024
Comment on lines
254
to
271
| self.dg_connection = deepgram.listen.live.v("1") | ||
| options = LiveOptions( | ||
| model="nova-2", | ||
| punctuate=True, | ||
| language="en-US", | ||
| encoding="linear16", | ||
| channels=1, | ||
| sample_rate=WS_SAMPLE_RATE, | ||
| interim_results=True, | ||
| utterance_end_ms="1000", | ||
| vad_events=True, | ||
| ) | ||
|
|
||
| self.dg_connection.on( | ||
| LiveTranscriptionEvents.Transcript, | ||
| lambda x, result, **kwargs: self.on_message(result), | ||
| ) | ||
| self.dg_connection.start(options) |
There was a problem hiding this comment.
Let's put this inside if ASR_METHOD == "deepgram":
zachwe
reviewed
Mar 25, 2024
| @@ -483,3 +504,9 @@ async def speak_response( | |||
| audio=np.frombuffer(audio_chunk_bytes, dtype=np.int16), | |||
| latency=t_styletts - t_normalize, | |||
| ) | |||
|
|
|||
| def on_message(self, result): | |||
There was a problem hiding this comment.
Call it on_deepgram_message or on_streaming_asr_message?
zachwe
reviewed
Mar 25, 2024
| @@ -279,7 +279,7 @@ async def connect_daily( | |||
| session_id=session_id, | |||
| record=record, | |||
| input_audio_format="int16", | |||
| tts_config=TTSConfig(provider="elevenlabs", voice_id=voice_id), | |||
| tts_config=TTSConfig(provider="local", voice_id=voice_id), | |||
There was a problem hiding this comment.
can we revert the changes in this file before merging? in prod it's nice to have recording and 11 labs voice for now
zachwe
reviewed
Mar 25, 2024
| @@ -271,6 +288,9 @@ async def interrupt(self, task: asyncio.Task): | |||
| self.is_responding = False | |||
|
|
|||
| async def receive_audio(self, message: bytes): | |||
|
|
|||
| self.dg_connection.send(message) | |||
There was a problem hiding this comment.
Same thing here, should gate on ASR_METHOD == "deepgram"
zachwe
approved these changes
Mar 25, 2024
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
matthewkennedy5
temporarily deployed
to
production
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
User description
attempting to be backwards compatible with whisper.
when ASR_METHOD == "deepgram", deepgram keeps track of the transcription as it goes in
self.transcription, which is used in start_response().Next, I think deepgram has VAD functionality, so we can get rid of silero vad and call start_response() once deepgram has a transcript ready.
Description
ResponseAgentclass, replacing the previous static transcription setup.transcriptattribute toResponseAgentto store the ongoing transcription result from Deepgram.transcriptattribute accordingly.receive_audiomethod to send audio data to Deepgram's live transcription service.start_responsemethod to use the live transcription result when Deepgram is selected as the ASR method.connect_dailyfunction to not record by default.connect_dailyfunction.Changes walkthrough
response_agent.py
Integrate Deepgram Live Transcriptionopenduck-py/openduck_py/response_agent.py
PrerecordedOptionsandFileSourcewithLiveTranscriptionEventsandLiveOptionsfor DeepgramClient.ASR_METHOD.transcriptattribute to store ongoing transcription.receive_audiomethod to send audio data to Deepgram livetranscription.
start_responseto use the storedtranscriptwhenASR_METHODis set to "deepgram".
on_messagemethod to handle live transcription events and updatethe
transcript.voice.py
Update Voice Router Configuration Defaultsopenduck-py/openduck_py/routers/voice.py
recordfromTruetoFalseinconnect_dailyfunction.TTSConfigprovider from "elevenlabs" to "local" inconnect_dailyfunction.💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.