Skip to content

fix: create room IO media streams off the event loop and demote retryable STT errors - #7502

Open
muntazir868 wants to merge 2 commits into
livekit:mainfrom
muntazir868:fix/7441-offloop-stream-create
Open

muntazir868 wants to merge 2 commits into
livekit:mainfrom
muntazir868:fix/7441-offloop-stream-create

Conversation

@muntazir868

Copy link
Copy Markdown

Closes #7441

Summary

1. Room IO: create media streams off the event loop

AudioStream.from_track / VideoStream.from_track run a synchronous FFI request, and _on_track_available called it directly from the track_subscribed handler — initializing noise cancellation (e.g. BVC()) blocked the loop for 106–119ms and tripped the loop monitor.

  • _create_stream is now async; the audio/video implementations run from_track via asyncio.to_thread and pass loop= explicitly so the stream still dispatches frames onto the job loop.
  • Creation runs in a task (_create_and_forward_stream) serialized by a lock, with a generation counter: a track event that supersedes an in-flight creation discards the pending stream instead of installing it, and _close_stream defers processor cleanup to that task while it settles.
  • aclose waits for in-flight creations before tearing down.
  • A creation failure logs and resets _track/_publication, preserving the previous synchronous failure's retry semantics.

2. Inference STT: retryable recv_task errors log at debug

A gateway reject before input end (e.g. code=2006, "Streaming ASR unavailable" during init) is retryable=True and already handled by SpeechStream's retry loop (WARNING) and the session's stream recreation. recv_task logged the same failure as ERROR twice (inline + log_exceptions), blaming the application for a handled gateway retry.

  • Retryable errors now log at debug; terminal errors still log at error.
  • The blanket log_exceptions wrapper on recv_task is replaced with level-aware handling.

Testing

  • make check (ruff format + lint, mypy strict) — clean
  • uv run pytest --unit --audio_eot — 4015 passed, 0 failed
  • New tests:
    • test_audio_stream_created_off_event_loop_and_bound_to_it — creation runs on a worker thread, loop= is the running loop, and the loop keeps ticking while from_track blocks for 150ms
    • test_stream_create_discarded_when_superseded — generation guard discards an overtaken creation
    • test_retryable_error_logs_at_debug_not_error / test_non_retryable_error_still_logs_at_error — log-level behavior
  • tests/test_room_io.py runs to_thread inline under virtual time (the autojump clock races real executor threads); the off-loop tests opt out with the real_time marker.

AudioStream/VideoStream.from_track performs a synchronous FFI request
(initializing noise cancellation can block for >100ms) and was called
directly from the track_subscribed handler, tripping the loop monitor.
Create the stream in a task via asyncio.to_thread, pass the running
loop explicitly so frames still dispatch onto it, and guard against
track events that supersede an in-flight creation with a generation
counter plus a lock around processor ownership.
A gateway reject before input end (e.g. a provider refusing the
session during init, code 2006) is retried by SpeechStream, which
already logs a WARNING, and the session recreates the stream when
retries are exhausted. Logging the same failure as ERROR in recv_task
blamed the application for a handled gateway retry; demote retryable
errors to debug and keep ERROR for terminal ones.
@muntazir868
muntazir868 requested a review from a team as a code owner September 27, 2026 14:26
@CLAassistant

CLAassistant commented Sep 27, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment thread livekit-agents/livekit/agents/voice/room_io/_input.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session start blocks the loop in AudioStream.from_track, and xai/stt-1 init fails with Streaming ASR unavailable (2006)

2 participants