Skip to content

feat(workhub): add provider-neutral realtime voice coordination - #5472

Draft
xxhZs wants to merge 3 commits into
apache:mainfrom
xxhZs:codex/workhub-voice-provider-pr
Draft

xxhZs wants to merge 3 commits into
apache:mainfrom
xxhZs:codex/workhub-voice-provider-pr

Conversation

@xxhZs

@xxhZs xxhZs commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Add an experimental, provider-neutral voice integration for WorkHub: realtime transcription, a durable voice-request inbox, a SQLite prepared-speech list, and asynchronous Jev review before supplemental speech is delivered. WorkHub keeps its existing system prompt; voice collaboration instructions are appended only during an active call.

  • Introduce a registered provider contract for WebRTC negotiation, normalized observations, delegation, correlated replies, and prepared speech. Provider wire formats and credentials stay outside the renderer and coordination runtime. No provider ships by default; preparation fails before microphone capture without one.
  • Preserve main's task delegation, child supervision, result handling, ordinary input admission, and follow-up batching. The voice layer does not collect child results or manufacture task-result messages. It transports replies WorkHub explicitly produces through voice_reply; there is no automatic child-completion → WorkHub → voice callback in this integration.
  • Hide internal voice tools and active coordination streams while preserving ordinary WorkHub tool presentation. Persist logs, list updates, review cursors, and delivery receipts; prevent stale review results and uncertain deliveries from replaying speech.
  • Remove the unused tool-result-to-public-chat publication pipeline, uncorrelated reply-to-list insertion, dead provider/test hooks, and semantic replay cases that assumed automatic task results. Reply writes require a registered voice requestId; list writes use the list tool.
  • Accept only normalized turn lifecycle/transcript events. Remove experimental JSON imports, old state migrations, legacy wire/transcript handling, and an unused presentation annotator. Unsupported persisted shapes are rejected without rewriting; existing JSON files are left untouched.
  • Exclude the private Codex/App adapter, patched runtime, build scripts, credentials, and experimental development history. The registration seam is for future trusted adapters, not a completed plugin installer.

Verification

  • npm run build:test and Desktop renderer TypeScript check: passed.
  • Final residual-code cleanup verification: Host WorkHub/voice/inbox, composer and projector suites: 160/160 passed; Desktop voice/provider/public-conversation: 49/49 passed; core recovery, runtime projection/durable boundaries and SQLite storage: 180/180 passed.
  • Earlier broader cleanup verification also covered Host execution/root-turn and Desktop visibility/permission/reconnect. Removed tests belonged to obsolete compatibility/publication behavior.
  • Renderer architecture check, changed-file Biome lint/format, ASF source-header audit, and diff whitespace check: passed.
  • No real-media or paid Jev replay was run against this cleaned provider-neutral tree. Fake-provider tests verify transport isolation and lifecycle behavior, not speech quality.

Remaining work before release

  • Supply and separately validate a voice provider adapter; the core intentionally includes none.
  • Run real-media integration with a provider. A future native child-completion notification contract is required for automatic asynchronous task-result speech; it is not implemented here.
  • Address slow WorkHub maintenance, conversational-stop handling for late correlated replies, and semantic checker misclassifications. This PR does not claim to resolve those known experimental limitations.

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: OpenAI Codex implemented the voice integration, provider extraction, cleanup, main integration, documentation, and tests. The commit includes a Generated-by trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actions github-actions Bot added the effort/XXL Over 2500 readable lines label Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XXL Over 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant