|
| 1 | +# Voice Chat Architecture (Convex-Hosted) |
| 2 | + |
| 3 | +Goals |
| 4 | + |
| 5 | +- Low-latency, reliable voice in rooms with mute, speaking indicators, and reconnection. |
| 6 | +- Scale from huddles to large rooms; enable recording and moderation later. |
| 7 | + |
| 8 | +High-Level |
| 9 | + |
| 10 | +- Control plane (Convex): auth, room state, presence, SFU/TURN token minting. |
| 11 | +- Media/signaling (SFU): LiveKit recommended for scalable audio routing. |
| 12 | +- Client: WebRTC with Opus 48 kHz, AEC/NS/AGC; simple voice UI. |
| 13 | + |
| 14 | +Components |
| 15 | + |
| 16 | +- Convex Voice Module |
| 17 | + - join/leave room; setMute/setDevice; heartbeat. |
| 18 | + - mint SFU (LiveKit) access tokens; issue TURN `iceServers` creds (TTL). |
| 19 | + - enforce org/channel permissions for join/mute/kick. |
| 20 | +- SFU (LiveKit or mediasoup) |
| 21 | + - Handles WS signaling and RTP forwarding; recording and moderation hooks. |
| 22 | +- TURN/STUN |
| 23 | + - `coturn` over TLS (`turns:`) with long‑term/HMAC creds. |
| 24 | + |
| 25 | +Client Flow |
| 26 | + |
| 27 | +1. Call Convex `joinVoiceRoom(roomId)`. |
| 28 | + - Returns: `{ roomState, iceServers, sfu: { url, token } }`. |
| 29 | +2. Create SFU connection and publish mic. |
| 30 | +3. Render participant list (Convex room state + SFU events). |
| 31 | +4. Handle device change + ICE restart; VAD for speaking indicator. |
| 32 | + |
| 33 | +Auth & Security |
| 34 | + |
| 35 | +- Convex mints short‑lived SFU tokens: claims `sub`, `room`, `role`, `exp`. |
| 36 | +- Rate‑limit Convex voice actions (QPS, payload size); audit logs. |
| 37 | +- TLS everywhere; DTLS‑SRTP for media. Optional E2EE via Insertable Streams. |
| 38 | + |
| 39 | +Room Model (Convex) |
| 40 | + |
| 41 | +- `voiceRooms` (orgId, channelId, active, createdAt). |
| 42 | +- `voiceParticipants` (roomId, userId, muted, device, joinedAt, lastSeen, speaking). |
| 43 | +- TTL cleanup on `lastSeen` via periodic task. |
| 44 | + |
| 45 | +Directory Plan |
| 46 | + |
| 47 | +- `packages/convex/src/convex/voice/` |
| 48 | + - `rooms.ts` (create/get/join/leave) |
| 49 | + - `participants.ts` (setMute/setDevice/heartbeat) |
| 50 | + - `tokens.ts` (mint SFU + TURN creds) |
| 51 | + - `cleanup.ts` (TTL tasks) |
| 52 | +- `packages/client/src/features/voice/` |
| 53 | + - `components/` |
| 54 | + - `JoinButton.svelte`, `VoicePanel.svelte`, `DevicePicker.svelte` |
| 55 | + - `livekit.ts` (connect/publish helpers) |
| 56 | +- `servers/livekit/` (livekit definition) |
| 57 | + |
| 58 | +Operations |
| 59 | + |
| 60 | +- Deployment: reuse Convex deployment; run SFU per region; multi‑region TURN. |
| 61 | +- Observability: metrics `rooms_active`, `participants_active`, `join_latency_ms`, `actions_rate`, `sfu_join_failures`, `turn_alloc_failures`. |
| 62 | +- Logs: include `roomId`, `userId`, `callId`, `region`; client periodically posts `getStats()` summaries to Convex. |
| 63 | + |
| 64 | +Milestones |
| 65 | + |
| 66 | +- M1: Convex join/leave + token mint + LiveKit connect; participant list; local mute. |
| 67 | +- M2: TURN enabled; device switch + ICE restart; speaking indicator. |
| 68 | +- M3: Server‑side mute/kick; TTL cleanup; rate‑limits; basic metrics. |
| 69 | +- M4: Recording/transcription path; regional routing; admin tools. |
| 70 | + |
| 71 | +Open Items |
| 72 | + |
| 73 | +- Target max participants per room; recording/E2EE requirements; regional rollout order. |
0 commit comments