Skip to content

feat: reclaimable cost map, in-place recondense, and long-session freeze fix - #621

Open
Ther-zh wants to merge 17 commits into
Tarquinen:masterfrom
Ther-zh:feat/recursive-condense
Open

Ther-zh wants to merge 17 commits into
Tarquinen:masterfrom
Ther-zh:feat/recursive-condense

Conversation

@Ther-zh

@Ther-zh Ther-zh commented Sep 20, 2026

Copy link
Copy Markdown

Steer compression to the largest reclaimable region, stop summary bloat, and fix the long-session freeze

Problem

DCP is model-driven: the nudge only says "you are over the limit", never where the tokens actually are. In long sessions this produced three observable failure modes:

  1. Wrong target. In a real 418-message session, 134 messages (0.95 MB) were covered by blocks while 267 messages (5.64 MB) had never been covered at all. The model kept compressing a few nearby messages instead of the 237-message uncovered prefix. Block count grew; context did not shrink.
  2. Summary bloat. Recursive merges rewrote child summaries into ever-larger parents (b116→b119, each ~31K tokens), and single-block re-wraps stacked bN → bN+1 chains.
  3. Freeze on large sessions. Every transform synchronously re-tokenized the entire history, constructing a fresh tokenizer per call (~35–40 s across ~380 tool outputs), blocking the event loop before the provider request was sent.

What this PR does

1. Tell the model where the tokens are — reclaimable cost map

  • lib/messages/inject/reclaimable.ts: findReclaimableRegions() collects uncovered raw ranges and active summaries, ranks them by token size, and reports each region's share of visible context.
  • renderReclaimableGuidance() renders a compact cost map into the nudge, marking the largest region ← COMPRESS THIS FIRST.
  • The compress range tool description now prioritizes COMPRESS THE LARGEST RECLAIMABLE CONTENT FIRST and treats UNCOVERED RAW CONTENT AS FIRST-CLASS.
  • Real-session result: the map pointed at m0001..m0237 (~364.8K tokens); the model compressed it in a single call — −590.9K removed / +9.4K summary, context 759K → 60K.

2. Stop summary bloat — in-place recondense + recursiveCondense

  • lib/compress/recondense.ts: a single-block re-wrap is re-condensed in place (replaced only when strictly smaller; self-references stripped) instead of creating a new block.
  • recursiveCondense condenses child summaries instead of re-expanding full child text into the parent.
  • enforceSummaryShrink (reject a compression whose summary is not smaller than the replaced content) is available but default off — rejecting inside the tool loop can drive model retry loops.

3. Fix the long-session freeze

  • Shared, reused tokenizer + cached per-message/tool token counts.
  • Context accounting and fork recovery moved off the default transform hot path; both opt-in.
  • Measured on the long session: transform ~38 s → ~374 ms.

4. Fork/inherited recovery (opt-in), structured protected content, cooldowns

  • lib/state/recovery.ts replays completed compress calls to rebuild blocks for forked/inherited sessions.
  • Structured protected content is preserved across recursive merges.
  • Stale-token nudge cooldown after a DCP compression; poll-turn cooldown; "don't compress a single message" guidance.

5. Tooling

  • scripts/gray-replay.mts — offline replay of a real session against a local dist.
  • scripts/gray-instance.ps1 — fully isolated XDG instance that loads a local dist without touching the installed cache.
  • .gitattributes to enforce LF.

Defaults / compatibility

Every new behavior is opt-in or backward-compatible:

Option Default
experimental.contextAccounting false
experimental.recoverInherited false
compress.recursiveCondense false
compress.enforceSummaryShrink false
compress.pollCooldown 3

The reclaimable cost map is text-only guidance and changes no state.

Verification

  • npm test148 pass (new coverage for reclaimable regions, recondense, protected content, recovery, accounting, token usage, perf guards).
  • npm run typecheck — clean.
  • Build (tsup + tsc --emitDeclarationOnly) — clean.
  • prettier --check . — clean.
  • Gray test on the real 418-message session: the model compressed the largest uncovered region in one call (759K → 60K).

Notes

  • Happy to split this into smaller PRs (cost map / recondense / freeze fix / recovery) if that is easier to review.

When recursiveCondense is enabled, merging an already-compressed block
into a new range compression no longer expands the child block's full
summary into the parent. Instead the model condenses the child content
directly into its own summary text, so each merge layer is meaningfully
smaller than the sum of its children.

(bN) placeholders become compact [condensed block bN] references while
still consuming the child block (preserving the compression tree).
Protected content from child blocks is still preserved automatically.

Adds compress.recursiveCondense config, prompt guidance, schema/README
docs, and unit tests for placeholder injection and missing-block
fallback in condense mode.
…y junk

Diagnosed from a real 113-compression session: 84/113 compressions were
single-message folds that each created a new summary block, and 64 of
those leaf summaries were never merged into a parent, so active summary
tokens monotonically grew from 4.5K to ~44.6K - compressing more made
the context fuller.

Add prompt-level strategy guidance so the model:
- merges existing compressed blocks (bN) into one parent block first,
  reclaiming child summary space, instead of folding single messages
- prefers larger closed ranges over one-message compressions
- writes summaries that are much smaller than the content they replace
- avoids creating near-identical recap blocks from single small messages

Changes:
- compress-range prompt: new COMPRESSION STRATEGY section
- context-limit-nudge: MERGE EXISTING COMPRESSED BLOCKS FIRST instruction
- nudge guidance: merging-priority and keep-summaries-lean hints
Real-session replay (fork-1.json) showed prompt guidance alone is not
enough: after one good large merge (Tarquinen#113: -88.8K/+15.3K), the model fell
back to folding a single poll message (Tarquinen#114: -121/+453) and writing a
merge summary 2.6x larger than what it replaced (Tarquinen#115: -6.7K/+17.5K).
Both are net-positive context growth - the exact anti-pattern.

Add a hard code-level gate: when compress.enforceSummaryShrink is on
(default true), the range compress tool rejects any compression whose
summary tokens are not strictly smaller than the tokens it replaces
(raw messages + consumed child block summaries). The model must rewrite
a much more condensed summary or merge existing blocks instead.

Adds config key, range-utils helper, tool integration, and unit test.
…efault

- enforceSummaryShrink now defaults to off (was on); the gate misled the
  model into retry loops when a summary could not shrink an uncovered
  prefix. Keep the config knob available for explicit opt-in.
- Add ContextAccountingSnapshot computed on every messages transform:
  raw vs transformed message/token counts, active block coverage
  (covered vs uncovered message tokens), largest uncovered tool outputs,
  provider-reported token totals, effective max/min thresholds and
  over-limit flags, and whether a compression just completed.
- Surface the snapshot through /dcp context and debug logs so the gap
  between the UI percentage and DCP's own pruning becomes explainable.
Forks inherit the parent's raw messages but not the DCP state file (keyed
by sessionId). opencode records parent_id only for subagents and regenerates
message IDs, so parent-state copying is not possible. Instead, replay
completed compress tool parts from the session's own history to rebuild
blocks deterministically:

- collectCompressCallRecords scans messages for completed compress tool
  parts and extracts their full input args (topic, content ranges, summary).
- replayCompletedCompressions replays each pending call through the normal
  range/message resolution + protected-content pipeline, deduplicated by
  compressCallId so restarts stay idempotent.
- Stale mNNNN references that no longer resolve are skipped with a warning
  and never abort the session.
- Hooks recovery into ensureSessionInitialized (config threaded through
  checkSession, command handler, and compress pipeline) so a forked session
  covers its inherited prefix on first load.
The recursive-condense path previously copied every child block's protected
tail by scanning from the first protected heading to the end of the summary,
which included nested 'previously compressed summaries' blocks and made parent
summaries grow instead of shrink (b116 31K -> b117 30.5K).

- Add ProtectedContent {kind: tool|user|prompt, text} to CompressionBlock.
- appendProtectedUserMessages/PromptInfo/Tools now return both the rendered
  summary text and structured entries; range/message/recovery store them on
  the block.
- parseProtectedContentFromSummary migrates legacy blocks: it extracts only
  the explicit protected headings and stops at 'previously compressed
  summaries', so that tail is never carried forward.
- renderProtectedContent reconstructs the canonical sections with dedup.
- injectBlockPlaceholders / appendMissingBlockSummaries / injectBoundarySummary
  now collect structured protected content from child blocks instead of
  copying summary tails, so recursive merges keep only the real protected
  content and shrink the normal narrative.
- loadPruneMessagesState migrates blocks without the field from their summary.
After a DCP compression the provider-reported token total still describes
the pre-compression request; the only stale-token mitigation was a narrow
early-return in injectCompressNudges when the last assistant message had a
completed compress part. Any other transform re-triggered the emergency max
nudge from stale numbers.

- Track state.lastDcpCompression (set on compress completion in finalizeSession
  and the message.part.updated event handler, persisted and restored).
- isReportedTokensStaleAfterDcpCompression detects whether the newest reported
  assistant total predates the last DCP compression.
- isContextOverLimits now uses the locally estimated transformed token count
  for the max-limit check when the reported total is stale, so a compression
  is not immediately followed by another emergency nudge unless the pruned
  view is genuinely still over the limit.
- estimateTransformedTokens sums non-compacted message content plus active
  summary tokens as the explainable local counterpart.
- Context accounting snapshot now reports lastDcpCompression, reportedStale,
  and estimatedTransformedTokens via /dcp context.
Fork-1 showed the model compressing single small messages on nearly every turn
during poll loops, which added summary tokens instead of freeing context. This
adds a trigger-policy guard and prompt strategy to stop that loop:

- compress.pollCooldown (default 3): number of consecutive tool-only assistant
  turns (no new user message in between) after which the emergency max-context
  nudge is suppressed. isPollOnlyTurn walks back from the newest message,
  counting assistant messages that contain tool calls and stopping at any user
  message or non-tool assistant message.
- injectCompressNudges skips adding contextLimitAnchors during poll-only turns,
  so the model is not re-nudged on every NOT_DONE poll iteration.
- COMPRESS_RANGE and block guidance now explicitly forbid single-message
  compressions, instruct 'reconsider range, not error loop', and add poll-turn
  cooldown wording.
- dcp.schema.json documents compress.pollCooldown.
Root cause: on every chat transform, syncToolCache tokenizes every tool part
not yet cached. countTokens used @anthropic-ai/tokenizer's countTokens, which
constructs + frees a full Tiktoken instance per call (~140ms each). Opening a
long session with ~400-800 uncovered tool parts blocked the Node event loop
for 35-40s before any provider request, freezing opencode.

Fixes:
- token-utils: keep one shared tokenizer instance across countTokens calls
  (same NFKC/'all' encode semantics, no per-call free). Real-session fixture:
  syncToolCache full pass 40s -> 584ms; cache-hit pass ~1ms.
- hooks: context accounting snapshot (5-7 full-content tokenization passes per
  transform) is now opt-in via experimental.contextAccounting (default false).
  Full transform over the 709-message fixture: accounting off 374ms, on 2.3s.
- inject/utils: isContextOverLimits only computes the estimated transformed
  token count when the reported total is stale after a DCP compression; 0
  otherwise.
- recovery replay is opt-in via experimental.recoverInherited (default false)
  to keep session init cheap and free of hidden full-history scans.
- dcp.schema.json documents both experimental flags; perf-guards test locks in
  the defaults; scripts/bench-* replay the real long session.
DCP only replaces what it has already compressed; uncovered raw messages keep entering every request verbatim, yet the model tends to compress small recent ranges and ignore a large early prefix. Add a reclaimable-context map (uncovered regions and active summaries, ranked by size) that is injected into the range-mode nudge and the compress tool description, so the model is told where the reclaimable tokens actually are and is asked to compress the largest region first. Gate the map computation on active nudges so it never runs on the per-request hot path, and thread messages through the manual /dcp-compress path so the map also applies there.
…locks

A compression whose range resolves to exactly one already-active block and adds no new message/tool coverage just re-wraps the same content, growing bN -> bN+1 chains without reclaiming anything. Detect that case and rewrite the existing block's summary in place (only when the new summary is strictly smaller, with self-references stripped), keeping the block id, anchor and coverage unchanged.
gray-replay.mts replays a real session through the transform pipeline (no opencode process, no cache changes) to show the guidance a nudge would inject. gray-instance.ps1 sets up a throwaway XDG home that loads the local build via a file URL so the real opencode/provider path can be exercised without touching the production plugin cache.
Thread state.idFormat through recovery replay paths (parseBlockPlaceholders,
appendMissingBlockSummaries, wrapCompressedSummary) and use .summaryText from
the structured protected-content result in the v2 protection test.
@Ther-zh
Ther-zh force-pushed the feat/recursive-condense branch from 8c7e5db to 407549b Compare September 21, 2026 04:47
v3.2.0 adds IdFormat ("xml" | "compact"), and the new v2 plugin path runs
with createSessionState("compact") while reusing the same lib/messages and
lib/compress code. The reclaimable cost map and the in-place recondense
path were written before that and hardcoded the xml form:

- reclaimable: render active block ids via formatBlockRef(id, idFormat)
  so compact sessions list @bn@ instead of bN.
- recondense: pass state.idFormat to wrapCompressedSummary (compact
  sessions were getting an xml dcp-message-id footer) and teach
  stripSelfReferences to drop compact self references (@bn@ and the
  "### compressed block N" heading).
- nudge / context-limit-nudge: stop hardcoding (bN) and "b1 to b6";
  the merge example now uses the session's real refs and placeholder form.

Adds compact-format regression tests for both paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant