Skip to content

feat(voice): expose a counted SpeechHandle interruption hold - #7203

Open
bharatnpti wants to merge 2 commits into
livekit:mainfrom
bharatnpti:feat/hold-interruptions-public
Open

feat(voice): expose a counted SpeechHandle interruption hold#7203
bharatnpti wants to merge 2 commits into
livekit:mainfrom
bharatnpti:feat/hold-interruptions-public

Conversation

@bharatnpti

@bharatnpti bharatnpti commented Sep 10, 2026

Copy link
Copy Markdown

Summary

Exposes SpeechHandle.hold_interruptions() β€” a public context manager over the counted hold that already exists privately β€” and changes how the count is stored, because storing it by overwriting allow_interruptions loses information in both directions.

Fixes #7191.

handle = session.say("Before we go on, do I have your consent?")
with handle.hold_interruptions():
    await handle.wait_for_playout()

The state model

_hold_interruptions() set allow_interruptions = False and remembered a value to restore. That makes a hold and an assignment fight over one field. Instead, the count lives beside the assigned value:

  • getter returns the effective state β€” _allow_interruptions and holds == 0. Behaviour-preserving for every existing reader: all of them already saw False while held, because the hold assigned False.
  • setter is unchanged and stays honest. An assignment during a hold is recorded and takes effect on the last release.
  • interrupt() consults the effective state, so a hold actually blocks it.
  • interruptions_held is a new property β€” narrower than not allow_interruptions, which is also true for a speech simply created uninterruptible.
  • _interruption_holds_restore and its contextlib.suppress(RuntimeError) are gone.

This fixes a latent bug on main

Because the hold overwrote the value, an assignment during a hold defeated the hold outright:

h = SpeechHandle.create(allow_interruptions=True)
h._hold_interruptions()
assert h.allow_interruptions is False   # held
h.allow_interruptions = True
assert h.allow_interruptions is False   # fails on main -- the hold is gone
h.interrupt()                           # succeeds on main

Nothing in the library assigns True, so it is unreachable in-tree today. But RunContext.disallow_interruptions() already routes user code into that setter, and a public hold makes the combination ordinary. Keeping the two independent removes the possibility instead of guarding it.

The log

A speech refusing the interrupt is expected when someone holds it β€” that is what a hold is for, and interrupt(force=True) still cuts through. So the realtime input_speech_started path logs debug for a held speech, and keeps logger.exception for a speech refusing with no holder, which is the desync the loud log was added for: the server cancelled its own response while the speech still believed it could not be interrupted.

Both directions are tested. I mention it because demoting that log unconditionally also hides the desync.

Why per-handle holds are coherent under server-side turn detection

test_allow_interruptions_false_rejected_with_server_turn_detection refuses session-level allow_interruptions=False, on the grounds that the server cancels its own response on user speech. That reasoning is about model-generated speech. For a TTS-backed session.say() it does not apply β€” AgentActivity.say() only uses the realtime session when there is no TTS, so the audio is LiveKit's own and the server has no response to cancel. AgentActivity.interrupt() also raises from _current_speech.interrupt() before reaching self._rt_session.interrupt(), so no response.cancel is sent for a held speech. That is the case this is for: verbatim compliance wording on an otherwise freely interruptible call.

Two things a hold does not do

Both are in the docstring, because both cost us a live call:

  1. The caller is not heard while a held speech plays. discard_audio_if_uninterruptible defaults to True, so silence is substituted on the paths feeding the STT and the realtime model. Adopters need turn_handling=TurnHandlingOptions(interruption={"discard_audio_if_uninterruptible": False}) or the barge-in is not merely ignored, it is discarded. We shipped the equivalent by hand, protected a welcome and a consent question, and made a caller unhearable for 16 seconds β€” they answered twice and neither utterance produced a transcript.
  2. A user turn completing during a hold gets no reply generated for it (AgentActivity, the "skipping reply to user input" path). A hold is for wording, not for a question whose answer is expected mid-sentence.

Tests

13 new:

  • tests/test_speech_handle_hold_interruptions.py (11) β€” hold/release, overlapping holds, that a hold does not make an uninterruptible speech interruptible, both directions of the assignment-vs-hold interaction, force=True through a hold, release after a forced interrupt, refusing to hold an already-interrupted speech, release on exception, repeated cycles, and interruptions_held vs allow_interruptions.
  • tests/test_realtime_input_speech_interrupt.py (2) β€” through AgentActivity._on_input_speech_started with server-side turn detection on: a held speech survives the barge-in and no response.cancel is sent; and an uninterruptible speech with no holder is still reported loudly.

The second file is the claim that actually matters, and it is the one a pure SpeechHandle unit test cannot make.

Checks

  • uv run ruff format --check / ruff check β€” clean
  • uv run python scripts/check_types.py β€” no new errors (the 3 remaining are missing optional stubs for cv2, loguru, boto3, identical on upstream/main)
  • existing interruption, session and tool suites pass unchanged β€” 298 tests, against a 285 baseline on upstream/main plus the 13 added here

@bharatnpti
bharatnpti requested a review from a team as a code owner September 10, 2026 03:18
@CLAassistant

CLAassistant commented Sep 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

bharatnpti pushed a commit to bharatnpti/agents that referenced this pull request Sep 10, 2026
Devin Review's catch on livekit#7203, and it is right. A hold stops LiveKit
interrupting its own playout. It cannot stop a realtime provider cancelling
audio the provider generated: with no TTS attached and a model whose
`capabilities.supports_say` is True, `say()` goes through `_rt_session.say()`
and the server ends it on its own turn detection, with nothing for the hold to
refuse.

That scope limit was in the PR description and not in the docstring, which is
the wrong way round -- the caveat matters to whoever reaches for the API, not
to whoever reads the discussion. Now listed first of the three, with the fix:
attach a TTS model for wording that has to arrive whole.
bharatnpti added a commit to bharatnpti/agents that referenced this pull request Sep 10, 2026
Devin Review's catch on livekit#7203, and it is right. A hold stops LiveKit
interrupting its own playout. It cannot stop a realtime provider cancelling
audio the provider generated: with no TTS attached and a model whose
`capabilities.supports_say` is True, `say()` goes through `_rt_session.say()`
and the server ends it on its own turn detection, with nothing for the hold to
refuse.

That scope limit was in the PR description and not in the docstring, which is
the wrong way round -- the caveat matters to whoever reaches for the API, not
to whoever reads the discussion. Now listed first of the three, with the fix:
attach a TTS model for wording that has to arrive whole.
@bharatnpti
bharatnpti force-pushed the feat/hold-interruptions-public branch from 014a3f2 to 17519be Compare September 10, 2026 03:24
`_hold_interruptions()` / `_release_interruptions()` already implement per-utterance
barge-in protection, but were private, and the counted state was stored by
overwriting `allow_interruptions` β€” which loses information in both directions.

Expose `SpeechHandle.hold_interruptions()`, a context manager over the existing
counted hold, and keep the count beside the assigned value instead of on top of it:

- `allow_interruptions` (getter) now reports the *effective* state,
  `_allow_interruptions and holds == 0`. Every existing reader already saw False
  while held, because the hold assigned False, so this is behaviour-preserving.
- the setter is unchanged and stays honest: an assignment during a hold is recorded
  and takes effect on the last release.
- `interrupt()` consults the effective state, so a hold actually blocks it.
- `interruptions_held` is a new property, narrower than `not allow_interruptions`.

This fixes a latent bug on main: because the hold overwrote the value, an assignment
during a hold defeated the hold outright.

    h = SpeechHandle.create(allow_interruptions=True)
    h._hold_interruptions()
    h.allow_interruptions = True   # on main: the hold is now gone
    h.interrupt()                  # on main: succeeds

Nothing in the library assigns True today, so it is unreachable in-tree β€” but
`RunContext.disallow_interruptions()` already routes user code into that setter, and
a public hold makes the combination ordinary. Keeping the two independent removes the
possibility rather than guarding it.

Also narrows the realtime `input_speech_started` log. A speech refusing the interrupt
is *expected* when someone holds it, so that case is debug; a speech refusing it with
no holder is still the desync the loud log was added for, and stays at exception
level. Both directions are covered by tests.

Tests: 13 new (11 unit + 2 through `AgentActivity._on_input_speech_started`, which is
the claim that matters β€” a held speech survives a realtime barge-in with server-side
turn detection on, and no `response.cancel` is sent). The existing interruption suite
passes unchanged: 298 tests across the interruption, session and tool files.

The docstring names the two things a hold does not do, both of which cost us a live
call: `discard_audio_if_uninterruptible` defaults to True, so by default the caller is
not heard while a held speech plays; and a user turn completing during a hold gets no
reply generated for it.

Refs livekit#7191
Devin Review's catch on livekit#7203, and it is right. A hold stops LiveKit
interrupting its own playout. It cannot stop a realtime provider cancelling
audio the provider generated: with no TTS attached and a model whose
`capabilities.supports_say` is True, `say()` goes through `_rt_session.say()`
and the server ends it on its own turn detection, with nothing for the hold to
refuse.

That scope limit was in the PR description and not in the docstring, which is
the wrong way round -- the caveat matters to whoever reaches for the API, not
to whoever reads the discussion. Now listed first of the three, with the fix:
attach a TTS model for wording that has to arrive whole.
@bharatnpti

Copy link
Copy Markdown
Author

@u9g β€” you added the counted hold in #6862, so this is your primitive: it keeps the count beside allow_interruptions instead of overwriting it, which also fixes an assignment during a hold defeating the hold. Would value your read on whether that breaks an inline-task case.

@longcw β€” the realtime half touches #6642's reasoning: a hold under server-side turn detection is coherent for a TTS-backed say(), where the provider has no response to cancel. Happy to be told otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make SpeechHandle interruption holds public, so one utterance can survive barge-in under server-side turn detection

2 participants