fix: omp/stateless Responses sessions never compress (prompt_cache_key identity) - #248
fix: omp/stateless Responses sessions never compress (prompt_cache_key identity)#248ranxianglei wants to merge 5 commits into
Conversation
… kernel falls to a per-request content fingerprint Stateless full-history replay clients without conversation headers (omp without its plugin) minted a new session every request: the kernel identity chain hashes the ENTIRE input array, which changes as the conversation grows. New session per request meant the nudge always evaluated 0 tokens (usage lands after the first forward), so 90%+-full contexts were never compressed and the upstream affinity churned every turn. prompt_cache_key — the client's own stable conversation id — now replaces only that fingerprint; headers, body session_id and previous_response_id still win.
|
[system] 🔄 ework-daemon picked up this issue — preparing workspace… |
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-248Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr248.tgz
npm install -g packageThis comment is automatically updated on each push. |
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-248Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr248.tgz
npm install -g packageThis comment is automatically updated on each push. |
…ading injectTool log flag omp does not ship the bili plugin; the launcher now rides -e dist/agent/omp.js when the config carries no loadable bili entry (mirrors pi, PR#227). omp keeps wire-injected tools (its fork never surfaces extension tools to the model and emits no before_provider_headers); the injected plugin provides the native /acp command. Server log now prints the effective injectTools flag, not the raw one.
|
Model: qwen3.8-27b (vllm) 追加第二半修复（同事故）： 背景：用户确认 omp 发行版不带 bili 插件——之前 /acp 能用全靠 8-23 的手动 install（条目 8-25 被移除后静默降级）。omp 分支原本没有 改动（7749429 + 5eec4fe）：
设计定论（e2e 实测 omp 17.3.8）：omp 主回合请求从不包含 extension 注册工具（只有其内部 title 请求可见），且不发 before_provider_headers —— 所以 omp 不能像 pi 那样绑 pluginMode（会让模型完全失去 ACP 工具，试过 identity register 后否决）。omp 的正确定位 = wire 工具 + prompt_cache_key 身份（本 PR 前半）+ -e 注入的原生 /acp 命令（本后半）。 验证：headless 两轮任务 4 个 wire 工具全注入、回复正常；tmux 交互 /acp 面板渲染 billion-context@0.1.54 + Context 0%/200k。 |
…oadedFrom to extensions block - preferPromptCacheKeyIdentity only promotes printable-ASCII, ≤128-char keys: the promoted value flows into the upstream x-session-id affinity header, where a control char makes undici throw ERR_INVALID_CHAR on EVERY request, and into the persisted session label (unbounded blob). - ompPluginLoadedFrom only honors entries under the extensions: block (a path-shaped value under another key must not suppress -e), and expands ~ like omp's pi-family loader so a loadable entry is never mis-classified stale (double-load via -e).
… (path.dirname chain instead of '/' literal)
|
Model: qwen3.8-27b (vllm) Double-review follow-up (joint regression with #246):
Known accepted trade-off (documented in code): a client that pins ONE constant pck across different conversations still merges them — that is the field's documented cache-affinity semantics; stronger signals (headers/session_id/previous_response_id) always win. |
1 participant
Model: qwen3.8-27b (vllm)
Symptom (user report)
An omp session sat at 92% context with zero compressions (
Resume this session with omp --resume 01a03971-…). Proxy log showed the nudge permanently idle:Root cause
omp without its plugin loaded sends no conversation headers and replays full history statelessly (no
previous_response_id). The kernel's Responses identity chain then falls tohashId(JSON.stringify(body.input))— a hash of the entire input array, which changes every turn. Every request minted a new session (dozens of session files withrequests: 1, each ~92% full). The nudge evaluates token counts before the first forward of a session, so it always saw 0 tokens → never fired. The upstream cache affinity also churned every turn.omp does send its own stable conversation id as
prompt_cache_keyon every request — we deliberately never read it for identity.Fix
preferPromptCacheKeyIdentity()(src/session-id.ts) promotesprompt_cache_keyover the content-fingerprint fallback only. Real conversation headers, bodysession_id,metadata.session_id, andprevious_response_idall remain stronger signals. Wired in src/server.ts before session-id derivation, affinity, and label computation.Validation
requests: 2, usage accumulated (240k) — turn 2's nudge now sees real tokens and fires.