Skip to content

feat: add ThunderPhone realtime (speech-to-speech) plugin - #7202

Open
kolchinski wants to merge 2 commits into
livekit:mainfrom
kolchinski:add-thunderphone-plugin
Open

feat: add ThunderPhone realtime (speech-to-speech) plugin#7202
kolchinski wants to merge 2 commits into
livekit:mainfrom
kolchinski:add-thunderphone-plugin

Conversation

@kolchinski

Copy link
Copy Markdown

Summary

Adds livekit-plugins-thunderphone: ThunderPhone voice agents as a realtime (speech-to-speech) model.

ThunderPhone's Realtime WebSocket API speaks the OpenAI Realtime protocol, so the plugin is a thin subclass of livekit-plugins-openai's RealtimeModel / RealtimeSession that adds the ThunderPhone-specific parts:

  • the endpoint and its query parameters (saved agent, engine, language, caller/callee numbers, wire audio), sk_live_ keys, provider = "ThunderPhone";
  • saved-agent sessions (RealtimeModel(agent_id=12)): prompt, voice, engine, languages, tools and greeting live on ThunderPhone, so the Agent's instructions/tools are not sent and the capabilities declare instructions and tools immutable;
  • inline sessions (RealtimeModel(product="bolt", voice="olivia")): instructions and tools come from the LiveKit Agent; the first session.update starts the call on ThunderPhone and freezes them, so the plugin sends one fully-configured update instead of the framework's three incremental ones;
  • ThunderPhone's call.* events (hang-up reason, transfer, keypad, ignored speech) as thunderphone_call_event / thunderphone_call_ended, and a clean session end when the call hangs up (no reconnect: a reconnect would start a new, separately billed call).

Docs: https://thunderphone.com/docs/guides/use-with-livekit

Installation

pip install livekit-plugins-thunderphone

Validation

  • tests/test_plugin_thunderphone_realtime.py: 15 passed (URL/config shaping, held first update, call events, call-id mapping).
  • Verified live against the ThunderPhone API: saved-agent and inline sessions, greeting, caller transcript, function tool round trip, barge-in, agent hang-up → call.ended → session closes cleanly, bad key → APIStatusError.
  • uv lock regenerated for the new workspace member.

@kolchinski
kolchinski requested a review from a team as a code owner September 10, 2026 00:44
@CLAassistant

CLAassistant commented Sep 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

@kolchinski

Copy link
Copy Markdown
Author

Pushed fixes for the review findings and CI:

  • Timeout starts another billed call: max_session_duration is now forced to None; a ThunderPhone socket is one call and must never be recycled.
  • Client and server turns collide: capabilities now declare turn_detection=True and can_disable_turn_detection=False, so AgentSession never selects client-side turn handling against ThunderPhone's server-side detector.
  • Option updates miss active sessions: session() registers in _sessions.
  • Call-end reason bypasses log redaction: the call.ended event is logged under lk.pii.event.
  • ruff and mypy are clean locally; tests: 16 passed.

ThunderPhone's Realtime WebSocket API speaks the OpenAI Realtime protocol, so
the plugin subclasses livekit-plugins-openai's RealtimeModel/RealtimeSession
and adds the ThunderPhone-specific parts: the endpoint and its query
parameters (saved agent, engine, language, phone numbers, wire audio), sk_live_
keys, saved-agent sessions whose prompt/voice/tools live on ThunderPhone, one
fully-configured session.update for inline sessions (the first update starts
the call and freezes instructions and tools), ThunderPhone's call.* events, and
a clean session end when the call hangs up.
…on, register sessions, lint/type fixes

- max_session_duration is forced to None: the framework's 20-minute socket
  recycle would hang up and start a second billed ThunderPhone call.
- capabilities declare turn_detection=True / can_disable_turn_detection=False
  so AgentSession never runs its own VAD or turn detector against ThunderPhone's.
- session() registers in _sessions so update_options() reaches live sessions.
- call.ended is logged with the lk.pii.event marker; call_id remap guards non-str.
- ruff import order, unused import; mypy-clean event names.
@kolchinski
kolchinski force-pushed the add-thunderphone-plugin branch from cc0da1c to dca147c Compare September 10, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants