[Bug]: Reopening a thread with a large event backlog freezes the UI while it visibly re-replays (quadratic replay)

[colonelpanic8]

Summary

When enableAssistantStreaming is on, every provider chunk is persisted as its own
thread.message-sent event. Reopening a thread whose client state is behind (cold tab, or the
thread atom was torn down after THREAD_STATE_IDLE_TTL_MS = 5 min idle) resumes the subscription
with afterSequence, and the server replays every persisted event since that cursor, one frame
per event, with no cap and no coalescing. The client applies them one at a time, and each apply
publishes a full new thread state that re-runs the entire timeline derivation and re-renders the
streaming message's markdown. Total work is superlinear in the backlog; a few thousand deltas locks
the main thread for minutes.

This is not gated on enableAssistantStreaming. The replay applies to all persisted event
types, and real agentic threads are dominated by thread.activity-appended (tool calls): a sampled
real database (streaming off) has a thread with 4,830 events — 4,170 of them activity-appended
plus 350 message events. The reducer re-sorts the entire activities array on every one of them
(threadReducer.ts:507-513), and every publication re-runs deriveWorkLogEntries over all
activities. Streaming mode just makes the text-delta variant of the same problem catastrophically
worse (one event per provider chunk; buffered mode caps text at one event per ~24KB via
MAX_BUFFERED_ASSISTANT_CHARS).

Measurements

UI-level (real MessagesTimeline + markdown + Shiki, via the /perf.html playground, Firefox
headless, dev build — absolute numbers inflated by dev mode, ratios are the signal):

deltas	batch	wall time	React commits	commit time	worst stall
500	1 (today)	14.7s	1,140	9.5s	105ms
1000	1 (today)	77s	2,330	47.7s	566ms
2000	1 (today)	212s	4,697	135s	443ms
2000	32 (simulated coalescing)	8.1s	179	5.3s	257ms
350 + 4200 activities (real-thread mix, works with streaming OFF)	1 (today)	36.8s	7,114	11.2s	568ms
350 + 4200 activities	32	5.4s	325	2.4s	98ms

4× the backlog ⇒ ~14× the time (superlinear), and batching applications 32-at-a-time is a 26×
speedup — the cost is dominated by per-event state publication, not by the text itself.

No-UI (unit level, packages/client-runtime/src/state/threadReplayPerf.test.ts):

• 2000 replayed deltas produce 2,002 state publications through the real resume path
  (threads.ts applyItem → SubscriptionRef.set per event) — the deterministic signature.
• Pure reducer fold over N deltas: 1000 → ~12ms, 2000 → ~18ms, 4000 → ~40ms, 8000 → ~344ms.

Mechanism

1. Server replays the raw firehose. subscribeThread with afterSequence does
   readEvents(afterSequence, Number.MAX_SAFE_INTEGER) and emits every event individually
   (apps/server/src/ws.ts:1391-1416). The shell stream already solves both halves of this —
   Stream.groupedWithin(512, 50ms) coalescing (ws.ts:786-800) and a SHELL_RESUME_MAX_GAP
   snapshot fallback (ws.ts:1290-1309) — but the thread-detail stream has neither.
2. Client applies one event per state publication. Stream.runForEach(applyItem) with no
   batching (packages/client-runtime/src/state/threads.ts:298); every delta does a
   SubscriptionRef.set → atom emission → React render.
3. Each apply is O(thread size). The delta append re-concatenates the whole accumulated text
   and copies the whole messages array (packages/client-runtime/src/state/threadReducer.ts:246-262);
   thread.activity-appended filters, appends, and re-sorts the entire activities array per
   event (threadReducer.ts:507-513).
4. Each publication re-derives the whole timeline (deriveTimelineEntries full re-sort,
   O(n²) deriveRevertTurnCountByUserMessageId, row rebuild) and re-renders the streaming
   message's markdown; the Shiki highlight cache is deliberately bypassed while
   isStreaming (apps/web/src/components/ChatMarkdown.tsx:672, only stored if (!isStreaming)),
   so replayed deltas re-highlight every code block from scratch each time.

Reproduction (deterministic, no provider/server needed)

• UI playground: vp run dev, open http://localhost:<web-port>/perf.html?deltas=2000&auto=1
  (streaming-delta variant) or /perf.html?deltas=350&activities=4200&auto=1 (real agentic mix,
  reproduces with streaming off). Params: deltas, activities, batch (raise to preview
  client-side coalescing), history, code.
  Added in feat(web): add thread replay perf playground page.
• Unit gauge (CI-able): pnpm --filter @t3tools/client-runtime test src/state/threadReplayPerf.test.ts
  — logs reducer timings and the emissions-per-delta count.
• End-to-end (optional): point providers.cursor.binaryPath at a wrapper around
  apps/server/scripts/acp-mock-agent.ts with T3_ACP_PROMPT_RESPONSE_CHUNK_COUNT=3000 and turn on
  enableAssistantStreaming; run a turn, wait >5 min (or reload), reopen the thread.

A screen recording of the 1000-delta replay is attached (real time and 4× speed).

Suspected fixes (in leverage order)

1. Server: cap the resume gap like the shell path — beyond a threshold send a fresh snapshot
   instead of replaying (ws.ts:1290-1309 pattern).
2. Server/client: coalesce replay events (Stream.groupedWithin) and fold a chunk through the
   reducer before a single SubscriptionRef.set (threads.ts:298).
3. Client: let replayed/catch-up deltas use and populate the Shiki cache (the isStreaming bypass
   is right for live typing, wrong for replay); consider useDeferredValue for streaming markdown.

The batch=32 playground row approximates fix 2 alone: ~26× improvement.

[colonelpanic8]

thread-replay-chug-4x.mp4

Here is a video replay at 4x showing the issue with a reproduction thing that I built

[colonelpanic8]

Update after validating against a real database: this is not streaming-gated. A sampled real state.sqlite (with enableAssistantStreaming off) has a thread carrying 4,830 events — 4,170 of them thread.activity-appended (tool calls). Replaying that exact mix through the playground (/perf.html?deltas=350&activities=4200&auto=1) chugs for ~37s with 7,114 React commits and a 568ms worst stall; batch=32 drops it to ~5.4s / 325 commits. The activity path is also the worst per-event reducer cost: threadReducer.ts:507-513 re-sorts the whole activities array on every event. Title and body updated accordingly; the mechanism and suspected fixes are unchanged.

[colonelpanic8]

thread-replay-activities-2x.mp4

not the best video because it doesnt follow the scroll but you can see how long it takes to play those back with the wall clock time.