Skip to content

fix(inference): claim word alignment for xai/stt-2 - #7503

Open
muntazir868 wants to merge 1 commit into
livekit:mainfrom
muntazir868:fix/7467-xai-stt2-word-alignment
Open

muntazir868 wants to merge 1 commit into
livekit:mainfrom
muntazir868:fix/7467-xai-stt2-word-alignment

Conversation

@muntazir868

@muntazir868 muntazir868 commented Sep 27, 2026 •

Copy link
Copy Markdown

Part of #7467 (item 3 of 3 — the word-alignment gap). Items 1 and 2 (keyterm forwarding, smart_turn) need gateway-side xai.ExtraSettings support: the gateway currently rejects those keys with failed to decode extra settings: 'xai.ExtraSettings' has invalid keys, so they can't be fixed from this repo alone.

Summary

  • Add xai/stt-2 to XaiModels so it is a first-class inference.STT model in the type union (STTModels).
  • Add xai/stt-2 to _WORD_ALIGNED_MODELS. The issue verified 48 timed words come back through Inference for xai/stt-2, same as xai/stt-1, but the capability claim still reported False.

What the claim actually gates

Correcting the issue description: the claim does not control adaptive interruption — interruption detection no longer consults STT alignment (decoupled from word timestamps by #6765). What it does feed:

  • AudioRecognition receives it from agent_activity.py:883 / :1319, and when the provider omits speech_end_time, audio_recognition.py:1147-1155 derives it from the word timings. Without the claim, xai/stt-2 fell back to a cruder speech-end estimate despite returning timed words.
  • stt.FallbackAdapter reports the combined claim as "word" only if all fallback models claim alignment, so a single unaligned model degrades the whole adapter's claim for downstream consumers.

Testing

  • make check (ruff format + lint, mypy strict) — clean
  • uv run pytest --unit --audio_eot --ignore=tests/test_room.py — 4012 passed, 3 skipped
  • New assertions in tests/test_inference_stt_aligned_transcript_claim.py:
    • xai/stt-1 and xai/stt-2 both report "word" as the primary model
    • new param case aligned-xai-stt2-fallback: fallback="xai/stt-2" keeps the adapter's claim at "word"

@muntazir868
muntazir868 requested a review from a team as a code owner September 27, 2026 14:26
@CLAassistant

CLAassistant commented Sep 27, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants