diff --git a/docs/ACCEPTANCE.md b/docs/ACCEPTANCE.md index 0d6f623c..23c83623 100644 --- a/docs/ACCEPTANCE.md +++ b/docs/ACCEPTANCE.md @@ -173,21 +173,23 @@ installed from the package under test, and a working `~/.codex`. ## `openclaw_capture` **What it proves:** that a conversation held in **OpenClaw** reaches -`ai_gateway_messages` on this machine, by both routes the OpenClaw adapter -offers (live proxy capture through the steering plugin's shadow providers, -and session-transcript backfill), that the rows name the real upstream -rather than the `hypaware-*` shadow the turn resolved to, and that a turn -the plugin refuses to steer passes through and is warned about instead of -vanishing silently. +`ai_gateway_messages` on this machine, by both lanes the OpenClaw adapter +offers: live capture through the local gateway once attached (this adapter +writes the `anthropic`/`openai` provider overrides into `openclaw.json` +itself, no separate package to install or link), and a periodic sweep of +local session transcripts that backfills every OpenClaw provider within the +sweep interval. It proves the rows name the real upstream, that a turn both +lanes observe settles to exactly one row rather than two, and that live +capture is reversible via `hyp detach`. **What it does not prove:** anything about OpenClaw's CLI backends (a Claude Code or Codex turn run through OpenClaw belongs to the sibling adapters, [LLP 0147](../llp/0147-cli-backends-are-transcript-captured.decision.md)), -anything about whether a deferred provider family *would* work if it were -steered ([LLP 0146](../llp/0146-host-signed-providers-out-of-shadow-steering.decision.md) -defers those untested, and step 5 only proves the deferral is reported), anything about fleet forwarding, or anything on a machine other than the -one you ran it on. +one you ran it on. There is no longer a deferred-provider-family ledger to +exercise here: LLP 0171 retires that requirement (R13) outright, since the +sweep gives every provider at least transcript-fidelity coverage, so this +procedure has nothing to assert about a "deferred" turn. **Requires:** @@ -195,118 +197,92 @@ one you ran it on. configured for **both** `anthropic` and `openai`. Both shapes are needed: the `openai` turn is the only observation that proves `x-hypaware-upstream` actually arrives (step 4). -- **OpenClaw 2026.4.24 or newer** (`openclaw --version`). The - `before_model_resolve` hook this plugin steers from arrived in 2026.4.21, - and the `hooks.allowConversationAccess` gate below arrived with the - 2026.4.23 plugin-config schema. On an older build the config key is - rejected outright (`Unrecognized key: "allowConversationAccess"`, and - OpenClaw rolls the file back to last-known-good), which leaves the plugin - loading and registering both providers while steering nothing. Check the - version first: every later step reads as a HypAware failure when the real - cause is the host. +- **OpenClaw 2026.4.24 or newer** (`openclaw --version`), the same floor + the prior procedure required. Lane A no longer depends on any OpenClaw + hook API (no `before_model_resolve`, no `hooks.allowConversationAccess`): + attach only needs `models.providers` to be a schema-valid config key, + which [LLP 0167#verify-results](../llp/0167-openclaw-capture-via-config-provider-override.rfc.md#verify-results) + confirms is stable back to 2026.3.13. The floor is kept here, not + re-derived, so this run re-confirms items 1, 3, and 4 of that + verification (step 7) on a current binary: those facts were established + on 2026.3.13 and this procedure has never re-checked them since. - HypAware installed from the package under test, `@hypaware/openclaw` - enabled, daemon running. -- The steering plugin installed into OpenClaw *from the tree under test*. - It is not on npm at the point this procedure must first run (R12 wants a - human run **before** the adapter ships), so link it from the checkout: - - ```sh - openclaw plugins install --link ./openclaw-steering-plugin --force - openclaw plugins enable hypaware-openclaw-steering - ``` - -- Two entries in `~/.openclaw/openclaw.json`. Both are mandatory and - neither is self-announcing when missing: - - ```json5 - { - env: { HYP_GATEWAY_ENDPOINT: "http://127.0.0.1:18521" }, - plugins: { - entries: { - "hypaware-openclaw-steering": { - hooks: { allowConversationAccess: true }, - }, - }, - }, - } - ``` - - `before_model_resolve` is one of OpenClaw's raw conversation hooks, and - OpenClaw will not run it for a non-bundled plugin without - `allowConversationAccess`: the gate applies only to plugins loaded from - `plugins.load.paths` (which is exactly where `--link` puts this one), and - when it blocks, `api.on(...)` returns without throwing. The providers - still register, the plugin still reports as loaded, no turn is ever - steered, and the only trace is a `pluginDiagnostics` warning in the - gateway log. `HYP_GATEWAY_ENDPOINT` is read - once at plugin load; absent it the plugin assumes the fixed default port - ([LLP 0114](../llp/0114-gateway-default-listen-port-fixed.decision.md)), - which is wrong whenever the daemon fell back to an ephemeral bind. Put - the real value there (step 1 reads it) and restart the OpenClaw gateway - (`openclaw gateway restart`) so plugin code and config both reload. - -**Related:** [LLP 0157](../llp/0157-openclaw-full-capture.spec.md) (the -requirements this procedure checks), [LLP 0161](../llp/0161-openclaw-full-capture.design.md) -(the design), [LLP 0159](../llp/0159-openclaw-route-agreement-by-settlement.decision.md) -(why step 6 passes on zero writes). + enabled, daemon running. Nothing to link from the checkout and nothing + else to install: attach writes the two provider overrides itself + (step 1). +- **Steps 5 and 6 (the sweep and zero-duplicate steps) need PR #552 + (fix/issue-543) merged** into the binary under test. Until it lands, the + LLP 0158 session-file reader still parses OpenClaw v3 records with the + old flat shape, so the sweep and the transcript backfill both project + zero rows from a real transcript, not because Lane B is miswired but + because the reader upstream of it has nothing to hand it. A red step 5 + or 6 against an unmerged #552 is not evidence of a Lane B regression; + confirm the merge before filing anything. +- **The `client_attach` status-row re-confirmation in steps 1 and 7 needs + PR #553 (fix/issue-544) merged.** Without it, a now-probed `openclaw` + (its `attach_probe` is real again as of this change set, for the first + time since [LLP 0143](../llp/0143-openclaw-registers-no-attach-probe.decision.md)) + falls back to whatever pre-#553 `hyp status` did for a client that used + to be probe-less, which this procedure was not written to describe. + +**Related:** [LLP 0167](../llp/0167-openclaw-capture-via-config-provider-override.rfc.md) +(the override design and the verify-results this procedure re-confirms), +[LLP 0169](../llp/0169-openclaw-attach-surface-returns.decision.md) (attach/detach), +[LLP 0170](../llp/0170-openclaw-scheduled-transcript-sweep.decision.md) (the sweep), +[LLP 0171](../llp/0171-openclaw-two-lane-capture.spec.md) (the requirements this +procedure checks, R11 in particular), [LLP 0172](../llp/0172-openclaw-two-lane-capture.design.md#acceptance-onboarding) +(section 8.1, this rewrite's own design), [LLP 0159](../llp/0159-openclaw-route-agreement-by-settlement.decision.md) +(why step 5 passes on zero *new* writes from the sweep). ### Steps -1. Confirm the shadow providers registered and the steering hook is live. - There is no settings marker to grep here: the adapter writes nothing to - `openclaw.json` and declares no `attach_probe` (R7), so `hyp attach - openclaw` has nothing to leave behind. The assertion moves to OpenClaw's - own runtime introspection: +1. Attach OpenClaw and confirm the write, then restart the gateway. Before + restarting, re-confirm LLP 0167#verify-results item 4 (no pickup without + a restart): run one turn first, so there is something to contrast once + the restart step below actually takes effect. + + ```sh + hyp query sql "select count(*) from ai_gateway_messages where conversation_source = 'openclaw'" + hyp attach --client openclaw + openclaw agent --agent --model anthropic/ \ + --message "pre-restart probe, should not route through the gateway" + hyp query sql "select count(*) from ai_gateway_messages where conversation_source = 'openclaw'" + ``` + + Pass condition for item 4: the two counts are equal. `hyp attach` wrote + the config, but a running OpenClaw gateway does not pick up + `models.providers` changes until restarted, so the probe turn above + still went out at OpenClaw's original `baseUrl`, not the gateway's. + + Now run the restart instruction `hyp attach` printed: + + ```sh + openclaw gateway restart + ``` + + Then confirm the write itself and the daemon's view of it: ```sh hyp status - jq -r '.sources[] | select(.plugin == "@hypaware/ai-gateway") - | "http://\(.details.host):\(.details.port)"' \ - "${HYP_HOME:-$HOME/.hyp}/hypaware/run/status.json" - INSPECT=$(openclaw plugins inspect hypaware-openclaw-steering --runtime --json) - printf '%s\n' "$INSPECT" - for token in hypaware-anthropic hypaware-openai before_model_resolve; do - printf '%-24s %s\n' "$token" "$(printf '%s' "$INSPECT" | grep -c -- "$token")" - done + jq '.models.providers | {anthropic, openai}' "${OPENCLAW_HOME:-$HOME/.openclaw}/openclaw.json" ``` - Pass condition: `hyp status` shows a running daemon and `openclaw` among - the clients; the printed gateway URL equals the `HYP_GATEWAY_ENDPOINT` - you configured; the report says the plugin is loaded (not errored, not - disabled); and all three tokens are present in it. `openclaw plugins - list` will not do instead: it is a cold registry read, while `inspect - --runtime` imports the module and reports the tools, hooks, services, - gateway methods and commands a live gateway actually registered. - - Read the report, do not assert against a key path. OpenClaw documents - `--runtime --json` as the machine-readable form of the same report and - describes its *contents* (identity, load status, source, registered - capabilities, hooks, diagnostics) without publishing a key schema, so - the field names are its to change between releases and a `jq` selector - written here would fail as a missing key rather than as a missing - registration - a false negative pointing at the wrong system. Grepping - the three tokens is version-proof by comparison: two are provider ids - this repo owns (`openclaw-steering-plugin/src/index.js`) and the third - is OpenClaw's own documented hook name. None of them is a key name - OpenClaw invents for the shape of its report. - `openclaw plugins inspect hypaware-openclaw-steering --runtime` without - `--json` prints the same report for a human to read. - - If your build has no `--runtime` flag (it is absent from some published - CLI references, and `inspect` alone is a cold manifest and registry - check that cannot prove registration), do not substitute the cold read. - Skip to step 3 and let step 4 carry this assertion instead: an `openai` - row there is only reachable through both shadow providers and a hook - that steered, so it proves at once what this step checks separately. - - Expect `hyp status` to also carry a `client_attach_missing` warning for - `openclaw`, telling you to run `hyp attach openclaw`. It is inert here - and running it changes nothing: the generic attach probe has no marker - to find by design (R7). Note it and move on; it says nothing about - capture either way. - -2. Note the current row count and pin the window, so steps 4 and 6 measure - only new traffic: + Pass condition: `hyp status` shows a running daemon and + `openclaw [configured, attached]` among the clients, with no + `client_attach_missing` diagnostic (this is the PR #553 re-confirmation: + a probe-less `openclaw` used to be stuck reading as `attach n/a` + regardless of what was on disk). The `jq` output shows + `anthropic.baseUrl` as the bare gateway origin and `openai.baseUrl` as + the same origin plus `/v1`, both carrying `headers["x-hypaware-upstream"]` + set to their own key, and **both carrying `models: []`**. That empty + array is LLP 0167#verify-results item 1's caveat, re-confirmed here: a + partial entry without it is schema-invalid and OpenClaw hard-refuses the + config outright. Separately confirm the empty array does not empty the + real catalog: `openclaw models list --all` must still list the full + built-in `anthropic` catalog. + +2. Note the current row count and pin the window, so steps 4, 5, and 6 + measure only new traffic: ```sh SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ) @@ -350,163 +326,167 @@ requirements this procedure checks), [LLP 0161](../llp/0161-openclaw-full-captur Pass condition: two rows, `provider` = `anthropic` and `openai`, both with `client_name` = `openclaw` and `last_seen` inside the last few - minutes, and the count grew against step 2. Neither row may say - `hypaware-anthropic` or `hypaware-openai`: a shadow id in this column is - an R6 failure, not a cosmetic one. + minutes, and the count grew against step 2. Unlike the old + steering-plugin design, there is no shadow provider id to leak here: + attach overrides the existing `anthropic`/`openai` entries' `baseUrl` + rather than registering new provider ids, so any `provider` value other + than `anthropic`/`openai` is unexpected on its own terms, not a specific + named failure mode to check for. The `openai` row is the load-bearing one, and it is this procedure's proof that `x-hypaware-upstream` arrives at the gateway. The projector reads the provider from that header and falls back to `anthropic` when - it is absent, so an `anthropic` row alone cannot tell "steered, header - arrived" from "header lost, fell back". Only a row that says `openai` - proves the header survived the trip from the hook to the projection. - -5. Exercise the warning ledger against a deferred provider family. Pick one - whose declared `api` is `anthropic-messages` or `openai-completions`: the - `deferred` branch only runs for a candidate that *shares a shape* with a - canonical provider, so a Google-family id (also in `DEFERRED_SET`, but - declaring `google-generative-ai`) reports `no_preset` and would verify - the wrong branch. If a real deferred-family provider is configured on - this machine, use it. Otherwise declare one locally, which needs no - cloud credentials at all, because the deferral is decided before any - credential is resolved: - - ```json5 - { - models: { - mode: "merge", - providers: { - "anthropic-vertex": { - baseUrl: "http://127.0.0.1:1/deferred-probe", - apiKey: "not-a-real-key", - api: "anthropic-messages", - models: [{ id: "deferred-probe", name: "Deferred family probe" }], - }, - }, - }, - } - ``` + it is absent, so an `anthropic` row alone cannot tell "routed through + the override, header arrived" from "header lost, fell back". Only a row + that says `openai` proves the header survived the trip from the + override entry's static `headers` value to the projection. + +5. Zero-duplicate assertion: the two turns from step 3 are exactly the case + both lanes observe (live capture already caught them at wire fidelity; + their session file entry is also sitting on disk waiting for a sweep). + Wait past one sweep interval (default 5 minutes) after the quiesce + window (default 3 minutes) has elapsed since step 3, then re-run the + same query: ```sh - openclaw gateway restart - openclaw agent --agent --model anthropic-vertex/deferred-probe \ - --message "ping" - openclaw logs --limit 200 | grep hypaware-openclaw-steering hyp query sql " select count(*) from ai_gateway_messages - where conversation_source = 'openclaw' and provider = 'anthropic-vertex' + where conversation_source = 'openclaw' and message_created_at >= '$SINCE_SQL'" + hyp query sql " + select part_id, count(*) n + from ai_gateway_messages + where conversation_source = 'openclaw' + and message_created_at >= '$SINCE_SQL' + group by part_id + having count(*) > 1" ``` - Pass condition: the log carries one `uncaptured provider turn` record - naming `provider: 'anthropic-vertex'` and `cause: 'deferred'`, and the - query returns `0`. The turn itself failing is expected and is part of - the pass: a deferred candidate must be left on its original provider, - unmodified, and must not be rerouted into the gateway (R5). What is - being checked is that the deferral is *reported* rather than - indistinguishable from a gap, which is what every coverage number - downstream rests on (R13). The ledger is rate limited per - provider+cause per OpenClaw gateway process, so if you repeat this step - inside five minutes, expect no second record: restart the gateway or - wait it out rather than concluding the warning stopped working. - -6. Confirm the backfill route agrees with live capture instead of - duplicating it. First check that the live rows settled onto the session - file's native identity, since that convergence is what the pass - condition below measures: + Pass condition: the first count is unchanged from step 4's total (the + scheduled sweep found the same two turns already settled onto their + session file's native identity and wrote nothing new), and the second + query returns zero rows (no `part_id` in the window appears more than + once). This is R11 proven against the daemon's own automatic scheduler + rather than a manually-invoked `hyp backfill`, which is the whole point + of Lane B being *scheduled*, not just present. + +6. Sweep step: prove a turn Lane A never saw still lands, at transcript + fidelity, within one sweep interval. Detach first, so the turn below + has no live route to travel: ```sh + hyp detach --client openclaw + openclaw gateway restart + SINCE2=$(date -u +%Y-%m-%dT%H:%M:%SZ) + SINCE2_SQL=${SINCE2%Z} + openclaw agent --agent --model anthropic/ \ + --message "In one sentence, what is a hash collision?" hyp query sql " - select count(*) total, - sum(case when json_extract(attributes, '\$.openclaw.match_key') is null - then 1 else 0 end) settled - from ai_gateway_messages + select count(*) from ai_gateway_messages where conversation_source = 'openclaw' - and message_created_at >= '$SINCE_SQL'" + and message_created_at >= '$SINCE2_SQL'" ``` - `settled` should equal `total`: a settled row has spent and dropped its - match key. Record the ratio in the release notes even when it is 1.0, - because it is the live measurement of whether OpenClaw appends its - session JSONL in time for the flush, the open question - [LLP 0159](../llp/0159-openclaw-route-agreement-by-settlement.decision.md) - says would trigger revisiting the route-agreement design. Then import - the same window from disk: + Pass condition immediately after the turn: `0`. Detach means the turn + went straight to OpenClaw's own `anthropic` endpoint, not the gateway, + so nothing reaches Lane A; only the session file records it. + + This is also the LLP 0167#verify-results item 3 re-confirmation + ("no self-heal"): confirm the detach actually purged the derived + caches rather than leaving a stale entry for OpenClaw to keep routing + by, since a cache that self-healed would make this step's absence + claim accidentally true for the wrong reason: ```sh - hyp backfill openclaw --since "$SINCE" --json + grep -rl 'x-hypaware-upstream' "${OPENCLAW_HOME:-$HOME/.openclaw}"/agents/*/agent/models.json ``` - Pass condition: `items_seen >= 1` with `rows_written + rows_skipped >= 1` - for `openclaw`. **`rows_written: 0` with `rows_skipped >= 1` is a pass, - not a failure.** Step 3 already captured these turns live, settlement - gave those rows the session file's own message ids, so the - materializer's `part_id` dedupe suppresses every duplicate. Zero writes - is the expected result and is exactly what proves the two routes agree - (R11). Re-running is likewise safe: identity comes from the session - file, so a second import never duplicates. + Pass condition: no matches. `hyp detach` best-effort purges every + `agents//agent/models.json`; a leftover match here means the purge + missed a cache, not that self-heal happened on its own. -7. Disable the steering plugin and confirm OpenClaw goes back to its own - providers, so capture is provably opt-in and reversible: + Now wait past one sweep interval (default 5 minutes) after the quiesce + window (default 3 minutes) has elapsed since the turn above, then + re-run the same query: ```sh - openclaw plugins disable hypaware-openclaw-steering - openclaw gateway restart - openclaw agent --agent --model anthropic/ \ - --message "In one sentence, what is a hash collision?" hyp query sql " - select count(*) from ai_gateway_messages + select count(*), max(message_created_at) + from ai_gateway_messages where conversation_source = 'openclaw' - and message_created_at >= '$SINCE_SQL'" + and message_created_at >= '$SINCE2_SQL'" ``` - Pass condition: the count is unchanged from step 4's total. The turn - answered normally, and nothing about it reached the gateway. + Pass condition: `1`. The scheduled sweep picked the turn up once its + session file cleared the quiesce window, at transcript fidelity, with + no live route involved at all. - Then, if this is your working machine, re-enable so you do not silently - leave OpenClaw capture off (and remove the step 5 probe provider if you - added one): + Re-attach so this machine is not left silently uncaptured on the live + lane, and restart once more: ```sh - openclaw plugins enable hypaware-openclaw-steering + hyp attach --client openclaw openclaw gateway restart ``` +7. Re-confirm [LLP 0167#verify-results](../llp/0167-openclaw-capture-via-config-provider-override.rfc.md#verify-results) + items 1, 3, and 4 on this OpenClaw version, per **Requires**. This step + is a recap, not new commands: each item was already exercised above. + + - **Item 1** ("merges, but only with `models: []`"): confirmed by step + 1's `jq` check and `openclaw models list --all` still showing the + full catalog. + - **Item 3** ("no self-heal"): confirmed by step 6's cache-purge grep + returning no matches after `hyp detach`. + - **Item 4** ("no pickup without restart"): confirmed by step 1's + pre-restart probe turn producing no new row. + + Record in the release notes that all three still hold on + `openclaw --version` as run, not only that this document says they do. + ### If it fails -- Step 1 finds both provider ids but a zero count for - `before_model_resolve`: the hook was registered and silently dropped, so - look for a `pluginDiagnostics` warning in `openclaw logs` naming this - plugin. The usual cause is an `allowConversationAccess` entry that is - missing or filed under a different key: it belongs to the plugin's - manifest `id`, `hypaware-openclaw-steering`, not the npm package name - (`@hypaware/openclaw-steering-plugin`) and not under `.config`. If the - entry is present and correct, check `openclaw --version` against the - floor in **Requires** before suspecting the plugin. -- Step 4 finds no rows at all: check the printed gateway URL against - `HYP_GATEWAY_ENDPOINT` first (a daemon that fell back to an ephemeral - port leaves the plugin talking to whatever holds the default), then - `hyp status` for a stopped daemon, then step 1's inspect output for a - plugin that failed to load after the last `openclaw gateway restart`. -- Step 4 finds `anthropic` rows but no `openai` row: either no OpenAI - credential is configured in OpenClaw (the candidate warns - `no_credential` and passes through, visible in `openclaw logs` exactly - as in step 5), or the `openai` upstream preset is not registered. Check - the log before touching code. -- Step 4 finds a row whose `provider` is `hypaware-anthropic` or - `hypaware-openai`: the shadow id leaked into the projection. That is an - R6 violation and its own bug; do not work around it by rewriting the - value at query time. -- Step 6 reports `rows_written >= 1`: the live rows did not settle, so - backfill imported the same turns a second time under native identity. - Re-run the settled/total query. A settled count below total means the - session JSONL was not on disk when the flush ran, which is the - real-time-append question above, not a backfill bug. Record the observed - ratio and file it against LLP 0159 rather than editing either route. -- Step 5 logs `cause: 'no_preset'` instead of `'deferred'`: the provider - you used does not share an API shape with a canonical one, so it never - reached the deferred-family check. Pick an `anthropic-messages` or - `openai-completions` shaped provider and repeat. +- Step 1's pre-restart probe finds a *new* row before you restart the + gateway: either the gateway was already running with a stale config that + happened to match, or your OpenClaw binary's config reloader has changed + to pick up `models.providers` hot (LLP 0167#verify-results item 4 notes + this as a real possibility on newer chokidar-based reloaders). Either way + this is a finding worth recording, not a HypAware bug: attach never + relies on hot reload, it only prints the restart instruction. +- Step 1 finds `client_attach_missing` still firing after a successful + attach and restart: check `openclaw --version` against the floor in + **Requires** first, then confirm PR #553 is actually in the binary under + test (a probe-less-client `attach n/a` state is exactly what an unmerged + #553 reproduces here). +- Step 4 finds no rows at all: check `hyp status` for a stopped daemon, + then re-run step 1's `jq` check for a config that did not actually write + (a concurrent edit under `openclaw.json` fails the write's mtime guard + rather than silently overwriting), then confirm + `openclaw gateway restart` actually ran after the last config change. +- Step 4 finds `anthropic` rows but no `openai` row: no OpenAI credential + is configured in OpenClaw for that turn, or the `openai` upstream preset + failed to register at plugin activation. Check `hyp status` for a + `@hypaware/openclaw` activation error before touching code. +- Step 5 or 6 finds nothing after waiting past the sweep interval: confirm + PR #552 is merged into the binary under test first (see **Requires**); a + sweep against the unmerged flat reader silently backfills zero rows from + a real OpenClaw v3 transcript, and this is the expected, documented + effect of that specific gap, not a new bug to chase. +- Step 5's second query returns a `part_id` with `count(*) > 1`: the live + row did not settle onto the session file's native identity before the + sweep imported the same turn, so the two rows never converged. Check + whether the live row still carries `attributes.openclaw.match_key` (an + unsettled row does); a settlement that has not run yet points at + [LLP 0159](../llp/0159-openclaw-route-agreement-by-settlement.decision.md)'s + open question about append timing, not a dedupe bug. +- Step 6 finds a row immediately after the detached turn (should be `0`): + `hyp detach` did not actually remove the override entries, most likely + because the entry on disk was not one this gateway wrote (a hand-edited + `baseUrl`, or `models` non-empty) and the detach backed it up instead of + deleting it, per + [LLP 0163](../llp/0163-attach-backs-up-a-malformed-block.decision.md)'s + backup-not-discard rule. Check the detach command's own warning output + before concluding the turn leaked. --- diff --git a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json index 793b3d6f..7f38bb11 100644 --- a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json +++ b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json @@ -36,7 +36,7 @@ { "name": "claude", "label": "capture Claude Code conversations", - "summary": "Configures Claude Code, installs Claude helper skills, and enriches rows from local Claude transcripts.", + "summary": "Configures Claude Code, installs Claude helper skills, and enriches rows from local Claude transcripts. Captures anything that runs the Claude Code CLI or Claude Agent SDK, including an OpenClaw configured with a claude-cli/ backend (the default on a subscription machine), which lands here as claude with no OpenClaw-side setup.", "detect": { "settings_file": ".claude/settings.json" }, "compose": { "plugin": { "name": "@hypaware/claude", "config": { "proxy": "@hypaware/ai-gateway" } }, diff --git a/hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json b/hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json index 691fa312..9a9b86b7 100644 --- a/hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json +++ b/hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json @@ -2,7 +2,7 @@ "schema_version": 1, "name": "@hypaware/openclaw", "version": "1.0.0", - "description": "OpenClaw client adapter for HypAware. Projects the Anthropic Messages and OpenAI Chat Completions exchanges the local AI gateway captures into ai_gateway_messages, settles them onto the session file's native identity, and backfills local OpenClaw session transcripts. Routing itself is owned by the @hypaware/openclaw-steering-plugin npm package installed on the OpenClaw side; this adapter writes nothing to openclaw.json.", + "description": "OpenClaw client adapter for HypAware. Captures OpenClaw conversations in two tiers: live capture through the local gateway once attached (this adapter writes the anthropic/openai provider overrides into openclaw.json's models.providers itself, no separate package), plus a periodic sweep of local OpenClaw session transcripts that backfills every provider within the sweep interval. Projects the Anthropic Messages and OpenAI Chat Completions exchanges the gateway captures into ai_gateway_messages, and settles them onto the session file's native identity.", "hypaware_api": "^1.0.0", "runtime": "node", "node_engine": ">=20", @@ -21,13 +21,21 @@ "client": { "name": "openclaw", "skill_dir": ".openclaw/skills", + "attach_probe": { + "format": "json_path", + "settings_file": ".openclaw/openclaw.json", + "container_path": "models.providers", + "provider_keys": ["anthropic", "openai"], + "marker_header": "x-hypaware-upstream", + "cache_glob": "agents/*/agent/models.json" + }, "required_upstreams": ["anthropic", "openai"] }, "picker": [ { "name": "openclaw", "label": "capture OpenClaw conversations", - "summary": "Records OpenClaw conversations captured through the local gateway, and backfills local session transcripts. Routing is set up on the OpenClaw side by installing the @hypaware/openclaw-steering-plugin package.", + "summary": "Captures OpenClaw conversations in two tiers: live capture through the local gateway once attached, plus a periodic sweep of local session transcripts that backfills every provider within the sweep interval. No separate package to install.", "detect": { "settings_file": ".openclaw/openclaw.json" }, "compose": { "plugin": { "name": "@hypaware/openclaw" }, diff --git a/hypaware-core/plugins-workspace/openclaw/src/attach.js b/hypaware-core/plugins-workspace/openclaw/src/attach.js new file mode 100644 index 00000000..0a653f33 --- /dev/null +++ b/hypaware-core/plugins-workspace/openclaw/src/attach.js @@ -0,0 +1,471 @@ +// @ts-check + +import fsp from 'node:fs/promises' +import os from 'node:os' + +import { isOwnedProviderEntry } from '../../../../src/core/config/provider_entry_ownership.js' +import { resolveClientSettingsPath } from '../../../../src/core/daemon/client_settings_path.js' +import { Attr, getLogger, withSpan } from '../../../../src/core/observability/index.js' +import { atomicWriteFile, errCode, isPlainObject } from 'hypaware/core/util' + +/** + * @import { AiGatewayClientAttachContext } from '../../../../hypaware-plugin-kernel-types.js' + * @import { OpenclawAttachOptions, OpenclawAttachOutcome } from './types.js' + */ + +const PLUGIN_NAME = '@hypaware/openclaw' +const CLIENT_NAME = 'openclaw' + +/** + * `settings_file` for the OpenClaw config, home-relative exactly as the + * manifest declares it, so this write and the manifest's `attach_probe` + * (and therefore `hyp status` / `hyp detach`) resolve the same file through + * the same core seam, including the `$OPENCLAW_HOME` relocation. + */ +const SETTINGS_FILE = '.openclaw/openclaw.json' + +/** The container the two entries live under, per LLP 0167#override-entries. */ +const CONTAINER_KEYS = ['models', 'providers'] + +/** The probeable marker header the gateway's upstream precedence rung reads. */ +const MARKER_HEADER = 'x-hypaware-upstream' + +/** The two provider keys attach owns, in write order. */ +const PROVIDER_KEYS = ['anthropic', 'openai'] + +const RESTART_COMMAND = 'openclaw gateway restart' + +/** + * The instruction R4 requires both surfaces to end with. A running OpenClaw + * gateway keeps routing turns at the old `baseUrl` until it is restarted + * (verified on 2026.3.13, LLP 0167#verify-results item 4), so an attach that + * does not say this reads as a silent no-op to the user. + */ +const RESTART_INSTRUCTION = + `restart the OpenClaw gateway ('${RESTART_COMMAND}') to apply` + +/** + * Create the `openclaw` client's `attach()` effect. + * + * Writes the two `models.providers` entries of LLP 0167#override-entries into + * `openclaw.json` and nothing else (R1). Attach never merges into a *user's* + * entry: a value at either key that HypAware did not write means the user + * deliberately routed that provider somewhere, and rerouting it silently is + * the surprise R2 exists to prevent, so attach refuses and writes nothing. + * An entry HypAware wrote is not that (it is self-identifying, and the same + * ownership test core's detach applies before deleting one), so attach + * overwrites its own: that is what makes re-attach-on-drift work, which + * `action_attach.js`'s `isCurrent()` requires after an ephemeral-port rebind + * (LLP 0086) or an asset-set change (LLP 0107). + * + * Never displacing a user value also means there is no undo record to write: + * the entries themselves are the marker the manifest's `json_path` + * `attach_probe` reads and core's detach reverses, and deletion is the whole + * undo (LLP 0169). + * + * Split out of `index.js` rather than inlined into `registerClient()` the way + * the Claude adapter's dry-run branch is, because the refuse-then-write + * ordering R2 turns on is the thing worth testing directly, without an + * activation around it. + * + * @param {OpenclawAttachOptions} opts + * @returns {{ attach(attachCtx: AiGatewayClientAttachContext): Promise }} + * @ref LLP 0167#attach-detach [implements]: attach writes exactly the two + * models.providers entries, refuses instead of merging when either key holds + * an entry HypAware did not write, and prints the restart instruction; no + * undo record beyond the entries themselves. + */ +export function createOpenclawAttach(opts) { + const homeDir = opts.homeDir ?? os.homedir() + const env = opts.env + const fs = opts.fs ?? fsp + const logger = opts.logger ?? getLogger('plugin.openclaw') + + return { + /** + * @param {AiGatewayClientAttachContext} attachCtx + * @returns {Promise} + */ + async attach(attachCtx) { + return await withSpan( + 'client.attach', + { + [Attr.PLUGIN]: PLUGIN_NAME, + [Attr.OPERATION]: 'client.attach', + client_name: CLIENT_NAME, + hyp_client: CLIENT_NAME, + dry_run: attachCtx.dryRun === true, + }, + async (span) => { + /** @type {string} */ + let settingsPath + try { + settingsPath = resolveClientSettingsPath(CLIENT_NAME, SETTINGS_FILE, env, homeDir) + } catch (err) { + return fail(span, attachCtx, logger, undefined, errMessage(err), 'settings_path') + } + + const endpoint = normalizeEndpoint(attachCtx.endpoint) + if (endpoint === undefined) { + return fail( + span, + attachCtx, + logger, + settingsPath, + `attach needs the local gateway endpoint; got '${String(attachCtx.endpoint)}'`, + 'endpoint' + ) + } + + // Read first, in every branch including dry-run: the refusal is a + // property of what is on disk, so a dry run that skipped the read + // would cheerfully report a write that the real run refuses. This is + // the whole of "pure read-then-decide, no partial write" (R2): the + // only write in this function is the single atomicWriteFile below, + // and every refusal returns before reaching it. + /** @type {{ value: Record, existed: boolean, mtimeMs: number | undefined }} */ + let read + try { + read = await readOpenclawConfig(settingsPath, fs) + } catch (err) { + // A missing or unparseable file is a hard failure, not a refusal: + // attach cannot reason about a config it cannot read, and writing + // a fresh one would hand OpenClaw a config it never had. + return fail(span, attachCtx, logger, settingsPath, errMessage(err), 'read') + } + + const existing = conflictingProviderKeys(read.value) + if (existing.length > 0) { + const reason = + `models.providers.${existing.join(' and models.providers.')} already ` + + `exists in ${settingsPath} and was not written by HypAware; attach ` + + `refuses to merge (LLP 0167#attach-detach). ` + + `Remove it by hand or run 'hyp detach --client ${CLIENT_NAME}' first.` + return fail(span, attachCtx, logger, settingsPath, reason, 'refused') + } + + if (attachCtx.dryRun === true) { + span.setAttribute('status', 'ok') + span.setAttribute('restored', false) + span.setAttribute('changed', false) + writeAttachOutput(attachCtx, { + status: 'ok', + dryRun: true, + settingsPath, + endpoint, + changed: false, + }) + return { status: 'done' } + } + + const next = withProviderEntries(read.value, endpoint) + try { + await atomicWriteFile(settingsPath, `${JSON.stringify(next, null, 2)}\n`, { + fs, + expectedMtimeMs: read.mtimeMs, + }) + } catch (err) { + return fail(span, attachCtx, logger, settingsPath, errMessage(err), 'write') + } + + span.setAttribute('status', 'ok') + span.setAttribute('restored', false) + span.setAttribute('changed', true) + logger.info('client.attach.write', { + hyp_plugin: PLUGIN_NAME, + hyp_client: CLIENT_NAME, + settings_path: settingsPath, + endpoint, + changed: true, + restart_required: true, + }) + writeAttachOutput(attachCtx, { + status: 'ok', + dryRun: false, + settingsPath, + endpoint, + changed: true, + }) + return { status: 'done' } + }, + { component: 'plugin.openclaw' } + ) + }, + } +} + +/** + * The provider keys of {@link PROVIDER_KEYS} that are occupied by something + * attach must not overwrite: present under `models.providers` and **not** an + * entry HypAware itself wrote. + * + * Ownership, not bare presence. What R2 protects is a *user's* deliberate + * routing decision, and HypAware's own entry is not one: it is self-identifying + * (`baseUrl`, `headers['x-hypaware-upstream']` naming the key it sits at, the + * empty `models` array), which is the same triple `detachJsonPathProviders` + * already uses to decide what it may delete, imported rather than restated so + * the two halves cannot drift. + * + * Refusing on bare presence made `attach()` non-idempotent, which + * re-attach-on-drift needs it to be: `action_attach.js`'s `isCurrent()` returns + * false whenever the daemon rebound to a new ephemeral port (LLP 0086) or the + * contributed asset set changed (LLP 0107), and the reconciler then re-performs. + * Every such re-perform refused over the entry the *previous* attach wrote, so + * the marker churned to `failed` with `attempts` climbing, `hyp attach openclaw` + * exited 1, and `openclaw.json` kept pointing at the dead port while the + * `json_path` probe (which matches the marker header, not the base URL) still + * reported `attached: true`. Anything that fails the ownership test still + * refuses, including a bare `"anthropic": null`, a user entry that happens to + * sit at the key, and a hand-edited one that merely kept the header. + * + * @param {Record} config + * @returns {string[]} + * @ref LLP 0086#re-attach-on-drift [constrained-by]: a done attach at a stale + * endpoint is re-performed, so the write it re-performs has to be idempotent + * over its own previous output + */ +function conflictingProviderKeys(config) { + const container = readPath(config, CONTAINER_KEYS) + if (!isPlainObject(container)) return [] + return PROVIDER_KEYS.filter( + (key) => + Object.hasOwn(container, key) && + // No base-URL set: the endpoint has moved by the time a drift re-attach + // runs, so our own entry carries the *old* origin. See the shared + // predicate's note on why detach passes one and attach does not. + !isOwnedProviderEntry(container[key], key, MARKER_HEADER, undefined) + ) +} + +/** + * The config with the two entries added, structurally shared down the + * `models.providers` spine only: every other key of `models`, of + * `models.providers`, and of the file's top level is carried through by + * reference (R1, nothing else in `openclaw.json` is touched). + * + * The bare-origin/`+/v1` asymmetry is load-bearing and not a typo: OpenClaw's + * Anthropic client appends `/v1/messages` to `baseUrl` itself and wants the + * bare origin, while its OpenAI client appends only `/responses` or + * `/chat/completions` and so needs the `/v1` prefix baked in. Both spellings + * are schema-valid, so getting either wrong produces a config OpenClaw accepts + * and silently fails to route through the gateway (LLP 0167#override-entries). + * + * @param {Record} config + * @param {string} endpoint gateway base URL, already trailing-slash-normalized + * @returns {Record} + */ +function withProviderEntries(config, endpoint) { + const models = isPlainObject(config.models) ? config.models : {} + const providers = isPlainObject(models.providers) ? models.providers : {} + return { + ...config, + models: { + ...models, + providers: { + ...providers, + anthropic: providerEntry(endpoint, 'anthropic'), + openai: providerEntry(`${endpoint}/v1`, 'openai'), + }, + }, + } +} + +/** + * One override entry. `models: []` is mandatory, not decorative: OpenClaw's + * config schema types it as a required array and hard-refuses CLI commands on + * a schema-invalid config, while an empty array validates without emptying the + * built-in catalog. + * + * @param {string} baseUrl + * @param {string} upstream + * @returns {Record} + */ +function providerEntry(baseUrl, upstream) { + return { + baseUrl, + headers: { [MARKER_HEADER]: upstream }, + models: [], + } +} + +/** + * Read and parse `openclaw.json`, with the `mtimeMs` the write is gated on so + * a concurrent edit is detected rather than silently overwritten. + * + * Absent and unparseable are both thrown, not returned as an empty config: + * step 1 of LLP 0172 §1.2 makes them hard failures. + * + * @param {string} settingsPath + * @param {typeof fsp} fs + * @returns {Promise<{ value: Record, existed: boolean, mtimeMs: number | undefined }>} + */ +async function readOpenclawConfig(settingsPath, fs) { + /** @type {string} */ + let raw + try { + raw = await fs.readFile(settingsPath, 'utf8') + } catch (err) { + if (errCode(err) === 'ENOENT') { + throw new Error( + `${settingsPath} does not exist; is OpenClaw installed? ` + + 'attach will not create an OpenClaw config it never had' + ) + } + throw new Error(`failed to read ${settingsPath}: ${errMessage(err)}`, { cause: err }) + } + + const stat = await fs.stat(settingsPath) + + /** @type {unknown} */ + let parsed + try { + parsed = JSON.parse(raw) + } catch (err) { + throw new Error(`malformed JSON in ${settingsPath}: ${errMessage(err)}`, { cause: err }) + } + if (!isPlainObject(parsed)) { + throw new Error(`${settingsPath} is not a JSON object; refuse to modify`) + } + return { value: parsed, existed: true, mtimeMs: stat.mtimeMs } +} + +/** + * @param {Record} value + * @param {string[]} keys + * @returns {unknown} + */ +function readPath(value, keys) { + /** @type {unknown} */ + let cursor = value + for (const key of keys) { + if (!isPlainObject(cursor)) return undefined + cursor = cursor[key] + } + return cursor +} + +/** + * The gateway base URL with any trailing slash removed, so the `openai` + * entry's `+ '/v1'` never produces a doubled separator. + * + * @param {unknown} endpoint + * @returns {string | undefined} + */ +function normalizeEndpoint(endpoint) { + if (typeof endpoint !== 'string') return undefined + const trimmed = endpoint.trim().replace(/\/+$/, '') + return trimmed.length > 0 ? trimmed : undefined +} + +/** + * Record a refusal or hard failure on the span, the log, and the caller's + * chosen output mode, and hand back the `{status:'failed', reason}` shape. + * + * Returning rather than throwing is what makes LLP 0169's join-safety clause + * reachable: the generic `ActionOutcome` contract downgrades a `failed` + * outcome to a recorded, retried warning that does not abort the join's other + * actions, so the only obligation here is to never throw out of this path (and + * to have written nothing before reaching it). Reaching that contract still + * takes one translation: the kernel types the *registered* `attach()` as + * `Promise`, so `index.js`'s wrapper rethrows this outcome and + * `perform()`'s catch turns it back into the same shape (LLP 0172 §1.3). + * + * @param {{ setAttribute(key: string, value: unknown): void }} span + * @param {AiGatewayClientAttachContext} attachCtx + * @param {{ warn(event: string, fields: Record): void }} logger + * @param {string | undefined} settingsPath + * @param {string} reason + * @param {string} errorKind + * @returns {OpenclawAttachOutcome} + * @ref LLP 0169#decision [implements]: a refuse during join warns and never + * fails the join, via the existing ActionOutcome 'failed' contract, not a + * new one. + */ +function fail(span, attachCtx, logger, settingsPath, reason, errorKind) { + span.setAttribute('status', 'failed') + span.setAttribute('restored', false) + span.setAttribute('changed', false) + span.setAttribute(Attr.ERROR_KIND, errorKind) + logger.warn('client.attach.refused', { + hyp_plugin: PLUGIN_NAME, + hyp_client: CLIENT_NAME, + ...(settingsPath !== undefined ? { settings_path: settingsPath } : {}), + [Attr.ERROR_KIND]: errorKind, + reason, + }) + writeAttachOutput(attachCtx, { + status: 'failed', + dryRun: attachCtx.dryRun === true, + settingsPath, + changed: false, + reason, + }) + return { status: 'failed', reason } +} + +/** + * Render attach output: one machine-readable JSON line when `json` is set on + * the attach context, otherwise human prose. Both paths carry the restart + * instruction whenever there is (or would be) something to apply, which is + * what R4 asks for: `--json` callers are as blocked on the restart as a human + * is, so hiding it behind the prose branch would make the automated path the + * one that silently does not work. + * + * @param {AiGatewayClientAttachContext} attachCtx + * @param {{ + * status: 'ok' | 'failed', + * dryRun: boolean, + * settingsPath: string | undefined, + * endpoint?: string, + * changed: boolean, + * reason?: string, + * }} fields + */ +function writeAttachOutput(attachCtx, fields) { + // A refusal changed nothing, so there is nothing to restart for; saying + // otherwise would send the user to bounce a gateway that is already + // correct. + const restart = fields.status === 'ok' + if (attachCtx.json) { + /** @type {Record} */ + const payload = { + status: fields.status, + action: 'attach', + client: CLIENT_NAME, + dry_run: fields.dryRun, + changed: fields.changed, + } + if (fields.settingsPath !== undefined) payload.settings_path = fields.settingsPath + if (fields.endpoint !== undefined) payload.endpoint = fields.endpoint + if (fields.reason !== undefined) payload.reason = fields.reason + if (restart) { + payload.providers = [...PROVIDER_KEYS] + payload.restart_required = true + payload.restart_command = RESTART_COMMAND + payload.message = RESTART_INSTRUCTION + } + attachCtx.stdout.write(JSON.stringify(payload) + '\n') + return + } + if (fields.status === 'failed') { + attachCtx.stdout.write(`! OpenClaw attach did not apply: ${fields.reason ?? 'unknown reason'}\n`) + return + } + const path = fields.settingsPath ?? '(unknown path)' + if (fields.dryRun) { + attachCtx.stdout.write(`(dry-run) Would attach OpenClaw via ${path}\n`) + } else { + attachCtx.stdout.write(`✓ OpenClaw attached (${path})\n`) + } + attachCtx.stdout.write(` models.providers.anthropic baseUrl = ${fields.endpoint}\n`) + attachCtx.stdout.write(` models.providers.openai baseUrl = ${fields.endpoint}/v1\n`) + attachCtx.stdout.write(` ${RESTART_INSTRUCTION}\n`) +} + +/** + * @param {unknown} err + * @returns {string} + */ +function errMessage(err) { + return err instanceof Error ? err.message : String(err) +} diff --git a/hypaware-core/plugins-workspace/openclaw/src/backfill.js b/hypaware-core/plugins-workspace/openclaw/src/backfill.js index 1c13ef24..a305d608 100644 --- a/hypaware-core/plugins-workspace/openclaw/src/backfill.js +++ b/hypaware-core/plugins-workspace/openclaw/src/backfill.js @@ -67,6 +67,32 @@ const DEFAULT_PLUGIN_NAME = '@hypaware/openclaw' const CONVERSATION_SOURCE = 'openclaw' const COMPONENT = 'plugin.openclaw.backfill' +/** + * Default Lane B sweep cadence (every 5 minutes), used when the plugin's + * `backfill.sweep_cron` config key is absent. Mirrors R7: "tunable in the + * plugin's `backfill` config section," default otherwise. + * + * @ref LLP 0172#lane-b-sweep [implements]: the sweep cadence default + */ +const DEFAULT_SWEEP_CRON = '*/5 * * * *' + +/** + * The quiesce window's default width, in milliseconds: the settlement flush + * debounce (`QUERY_FLUSH_DEBOUNCE_MS` in `src/core/cache/spool.js`, 2 + * minutes) plus a one-minute margin. Not a re-guessed number: a sweep run + * that considered a file inside this window could race a session OpenClaw + * is still mid-write on, or a settlement pass still mid-flush against the + * same turn (LLP 0170: "quiesce window = settlement flush interval + + * margin"). `config.backfill.quiesce_ms` overrides it for an operator with a + * slower disk or a longer flush debounce. + * + * @ref LLP 0170#decision [implements]: the sweep skips session files whose + * mtime is inside the quiesce window, sized from the existing settlement + * flush debounce plus margin, not a new invented constant. + * @type {number} + */ +const DEFAULT_QUIESCE_MS = 180_000 + /** * The CLI-backend exclusion (R10), as an explicit ALLOWLIST rather than a * denylist: a record projects only when the backend that served its turn is @@ -131,12 +157,19 @@ const SIBLING_ADAPTER_COVERAGE = [ * pluginName?: string, * resolver?: UsagePolicyResolver, * localOnlyListPath?: string, + * config?: JsonObject, * }} opts * @returns {BackfillContribution} */ export function createOpenclawBackfillProvider(opts) { const clientName = opts.clientName ?? DEFAULT_CLIENT_NAME const pluginName = opts.pluginName ?? DEFAULT_PLUGIN_NAME + // The plugin's own validated `config` slice (LLP 0037), the same shape + // `config.js`'s `validateBackfillSection` checks. Absent in every call + // site that hasn't threaded it through yet (a pre-quiesce test harness, an + // older host), which is exactly "not configured" and resolves to the + // default below, not a throw. + const config = opts.config // Through the LLP 0158 reader's own location helper, never a private // `path.join(homeDir, '.openclaw')`: that second opinion silently ignored // `OPENCLAW_HOME`, so a relocated install backfilled zero sessions while the @@ -154,6 +187,10 @@ export function createOpenclawBackfillProvider(opts) { plugin: pluginName, datasets: [AI_GATEWAY_MESSAGES_DATASET], summary: 'Import local OpenClaw session transcripts into ai_gateway_messages', + // @ref LLP 0172#lane-b-sweep [implements]: opt-in Lane B scheduling + // metadata, tunable via `backfill.sweep_cron` (R7), defaulting to + // every 5 minutes when the config key is absent. + sweep: { cron: resolveSweepCron(config) }, /** * @param {BackfillPlanContext} _ctx * @returns {Promise} @@ -166,7 +203,7 @@ export function createOpenclawBackfillProvider(opts) { } }, async *run(ctx) { - yield* runOpenclawBackfill({ ctx, agentsDir, clientName, resolver }) + yield* runOpenclawBackfill({ ctx, agentsDir, clientName, resolver, config }) }, } } @@ -187,18 +224,28 @@ export function createOpenclawBackfillProvider(opts) { * agentsDir: string, * clientName: string, * resolver: UsagePolicyResolver, + * config?: JsonObject, * }} args * @returns {AsyncGenerator} */ async function* runOpenclawBackfill(args) { - const { ctx, agentsDir, clientName, resolver } = args + const { ctx, agentsDir, clientName, resolver, config } = args const log = ctx.log const window = resolveWindow(ctx) + // Lane B's quiesce window (LLP 0172#45-the-quiesce-window): computed once + // per run, not once per file, so every file in the same run is judged + // against the same instant. This composes with, rather than replaces, the + // effectiveProviders/partitionByBackend forward/backward-fill logic below + // (R10): it is a pre-filter on which files this run even reads, entirely + // orthogonal to which records within a read file project. + const quiesceMs = resolveQuiesceMs(config) + const quiesceBeforeMs = Date.now() - quiesceMs log.info('openclaw.backfill.scan_started', { component: COMPONENT, operation: 'backfill.scan', agents_dir: agentsDir, + quiesce_ms: quiesceMs, ...(window.sinceMs !== undefined ? { since: new Date(window.sinceMs).toISOString() } : {}), ...(window.untilMs !== undefined ? { until: new Date(window.untilMs).toISOString() } : {}), status: 'ok', @@ -210,7 +257,7 @@ async function* runOpenclawBackfill(args) { let messagesProjected = 0 let recordsExcluded = 0 - for (const { agentId, filePath } of await listSessionFiles(agentsDir)) { + for (const { agentId, filePath } of await listSessionFiles(agentsDir, quiesceBeforeMs)) { if (ctx.signal?.aborted) break filesSeen += 1 @@ -616,23 +663,97 @@ function setNumber(target, key, source, aliases) { * result, never a throw: a machine with no OpenClaw install must scan to zero * sessions, not fail the whole `hyp backfill` run. * + * `quiesceBeforeMs` (LLP 0172#45-the-quiesce-window, LLP 0170#decision), + * when given, excludes any file whose `mtimeMs` is more recent than it: a + * session still inside the quiesce window is skipped for THIS run, not + * permanently, so a later run (once the file's mtime has aged past the + * cutoff, or the daemon sweep's next tick) picks it back up. + * + * The parameter is optional for exactly one caller: `plan()`, which counts and + * names what is there rather than importing it, so a window that hides files + * from an estimate would only misreport. Every `run()` applies the window, + * whichever surface drives it - `hyp backfill --client openclaw`, the + * onboarding finale, and the daemon sweep all enter through the same `run()`, + * and `runOpenclawBackfill` computes the cutoff once per run before this is + * ever called. There is no "non-sweep, unfiltered" import path. + * * @param {string} agentsDir + * @param {number} [quiesceBeforeMs] Exclusive upper bound on `mtimeMs`. * @returns {Promise>} */ -async function listSessionFiles(agentsDir) { +async function listSessionFiles(agentsDir, quiesceBeforeMs) { /** @type {Array<{ agentId: string, filePath: string }>} */ const out = [] for (const agentId of await readDirNames(agentsDir, 'dir')) { const sessionsDir = path.join(agentsDir, agentId, 'sessions') for (const name of await readDirNames(sessionsDir, 'file')) { if (!name.endsWith('.jsonl')) continue - out.push({ agentId, filePath: path.join(sessionsDir, name) }) + const filePath = path.join(sessionsDir, name) + if (quiesceBeforeMs !== undefined && !(await isOutsideQuiesceWindow(filePath, quiesceBeforeMs))) continue + out.push({ agentId, filePath }) } } out.sort((a, b) => (a.filePath < b.filePath ? -1 : a.filePath > b.filePath ? 1 : 0)) return out } +/** + * Whether `filePath`'s mtime is old enough to clear the quiesce window: its + * `mtimeMs` is at or before `quiesceBeforeMs`. A file that fails to stat + * (removed between the directory read and this call) is treated as still + * inside the window and excluded, the same fail-closed direction the + * usage-policy gate above already takes for an unresolvable input: a + * vanished file is not evidence a session has settled. + * + * @param {string} filePath + * @param {number} quiesceBeforeMs + * @returns {Promise} + */ +async function isOutsideQuiesceWindow(filePath, quiesceBeforeMs) { + try { + const stat = await fs.stat(filePath) + return stat.mtimeMs <= quiesceBeforeMs + } catch { + return false + } +} + +/** + * `quiesceMs` resolved from the plugin's own validated `config` slice, or + * {@link DEFAULT_QUIESCE_MS} when `config.backfill.quiesce_ms` is absent + * (no `config` supplied, no `backfill` block, or no `quiesce_ms` key). + * `config.js`'s `validateBackfillSection` already rejects a non-integer or + * negative value before it ever reaches here, so this reads the field + * as-is rather than re-validating it. + * + * @param {JsonObject | undefined} config + * @returns {number} + */ +function resolveQuiesceMs(config) { + const backfill = isPlainObject(config) && isPlainObject(config.backfill) ? config.backfill : undefined + const quiesceMs = backfill?.quiesce_ms + return typeof quiesceMs === 'number' ? quiesceMs : DEFAULT_QUIESCE_MS +} + +/** + * The contribution's `sweep.cron`, resolved from the plugin's own validated + * `config` slice, or {@link DEFAULT_SWEEP_CRON} when `config.backfill.sweep_cron` + * is absent. Read through the same `isPlainObject` narrowing `resolveQuiesceMs` + * uses rather than an optional-property chain: the slice is a `JsonObject`, so + * every step below its root is a `JsonValue` to the checker and a bare + * `config?.backfill?.sweep_cron` does not typecheck. `config.js`'s + * `validateBackfillSection` already rejects a malformed cron string before it + * reaches here. + * + * @param {JsonObject | undefined} config + * @returns {string} + */ +function resolveSweepCron(config) { + const backfill = isPlainObject(config) && isPlainObject(config.backfill) ? config.backfill : undefined + const sweepCron = backfill?.sweep_cron + return typeof sweepCron === 'string' ? sweepCron : DEFAULT_SWEEP_CRON +} + /** * @param {string} dir * @param {'dir' | 'file'} kind diff --git a/hypaware-core/plugins-workspace/openclaw/src/config.js b/hypaware-core/plugins-workspace/openclaw/src/config.js index 82caf81a..6bbfcecc 100644 --- a/hypaware-core/plugins-workspace/openclaw/src/config.js +++ b/hypaware-core/plugins-workspace/openclaw/src/config.js @@ -4,17 +4,23 @@ * Config validation for the `@hypaware/openclaw` plugin's own `config` * block. v1 validates the optional `attach` sub-object that drives * attach-on-join, `{ on_join }`, and the optional `backfill` sub-object - * that drives backfill-on-join, `{ on_join, window_days }`. Every other - * key passes through untouched so existing configs keep working; there - * is nothing new for core to validate. + * that drives backfill-on-join and Lane B's scheduled sweep, `{ on_join, + * window_days, sweep_cron, quiesce_ms }`. Every other key passes through + * untouched so existing configs keep working; there is nothing new for + * core to validate. * - * Pure and dependency-free: it returns a `ValidationResult` so it plugs - * straight into `ctx.configRegistry.registerSection` and is callable from - * tests without spinning up observability. + * Pure: it returns a `ValidationResult` so it plugs straight into + * `ctx.configRegistry.registerSection` and is callable from tests without + * spinning up observability. `sweep_cron` reuses core's shared 5-field + * cron grammar (`isCronExpression`) rather than inventing a second + * parser, so a malformed schedule is rejected the same way a sink's + * `config.schedule` is. * * @import { ValidationError, ValidationResult } from '../../../../hypaware-plugin-kernel-types.js' */ +import { isCronExpression } from '../../../../src/core/config/validate.js' + /** Manifest `config_sections[].section` name this validator backs. */ export const OPENCLAW_CONFIG_SECTION = 'openclaw' @@ -84,19 +90,28 @@ export function validateAttachSection(value, pointer) { /** * Validate the optional `backfill` policy block shared by every * backfill-capable source plugin: `on_join` (whether to import on join, - * boolean) and `window_days` (how far back, positive integer). Both are - * optional; unknown keys are rejected so a typo (`window_day`) surfaces - * instead of being silently ignored. Pure: the caller chooses where the - * returned pointers mount. + * boolean), `window_days` (how far back, positive integer), `sweep_cron` + * (Lane B's scheduled-sweep cadence, a 5-field cron expression), and + * `quiesce_ms` (how recently-modified a session file must be to skip a + * sweep, non-negative integer milliseconds). All four are optional; + * unknown keys are rejected so a typo (`window_day`) surfaces instead of + * being silently ignored. Pure: the caller chooses where the returned + * pointers mount. * - * A same-shape copy of `@hypaware/codex`'s validator of the same name, - * not a cross-plugin import: no plugin in this codebase imports another - * plugin's `src/` at runtime, so each backfill-capable plugin holds its - * own byte-identical but independently-editable copy. + * Started as a same-shape copy of `@hypaware/codex`'s validator of the + * same name (no plugin in this codebase imports another plugin's `src/` + * at runtime, so each backfill-capable plugin holds its own + * independently-editable copy); `sweep_cron`/`quiesce_ms` are OpenClaw's + * own Lane B additions (LLP 0170#decision, LLP 0172#4.2) and are not + * mirrored onto codex's copy, so the two are no longer byte-identical. * * @ref LLP 0157#backfill [implements]: the plugin-owned `backfill` policy * (`on_join`, `window_days`) declared and validated in the plugin's own * config section (LLP 0037 [constrained-by]). + * @ref LLP 0170#decision [implements]: `sweep_cron` and `quiesce_ms` are + * tunable in the plugin's own `backfill` config section, validated + * together so the unknown-key rejection loop never sees one land ahead + * of the other. * * @param {unknown} value * @param {string} pointer JSON-pointer prefix for the `backfill` object @@ -123,8 +138,26 @@ export function validateBackfillSection(value, pointer) { }) } } + if (raw.sweep_cron !== undefined) { + const cron = raw.sweep_cron + if (typeof cron !== 'string' || !isCronExpression(cron)) { + errors.push({ + pointer: `${pointer}/sweep_cron`, + message: 'backfill.sweep_cron must be a valid 5-field cron expression', + }) + } + } + if (raw.quiesce_ms !== undefined) { + const ms = raw.quiesce_ms + if (typeof ms !== 'number' || !Number.isInteger(ms) || ms < 0) { + errors.push({ + pointer: `${pointer}/quiesce_ms`, + message: 'backfill.quiesce_ms must be a non-negative integer', + }) + } + } for (const key of Object.keys(raw)) { - if (key !== 'on_join' && key !== 'window_days') { + if (key !== 'on_join' && key !== 'window_days' && key !== 'sweep_cron' && key !== 'quiesce_ms') { errors.push({ pointer: `${pointer}/${key}`, message: `unknown backfill key '${key}'` }) } } diff --git a/hypaware-core/plugins-workspace/openclaw/src/index.js b/hypaware-core/plugins-workspace/openclaw/src/index.js index 41886dd2..ff1c5adb 100644 --- a/hypaware-core/plugins-workspace/openclaw/src/index.js +++ b/hypaware-core/plugins-workspace/openclaw/src/index.js @@ -2,32 +2,22 @@ import os from 'node:os' -import { Attr, getLogger, readObservabilityEnv, withSpan } from '../../../../src/core/observability/index.js' +import { readObservabilityEnv } from '../../../../src/core/observability/index.js' import { localOnlyListPath } from '../../../../src/core/usage-policy/index.js' +import { createOpenclawAttach } from './attach.js' import { createOpenclawBackfillProvider } from './backfill.js' import { OPENCLAW_CONFIG_SECTION, validateOpenclawConfig } from './config.js' import { anthropicUpstreamPreset, createOpenclawExchangeProjector, openaiUpstreamPreset } from './projector.js' import { createOpenclawSettlementEnricher } from './settle.js' /** - * @import { AiGatewayCapability, AiGatewayClientAttachContext, PluginActivationContext } from '../../../../hypaware-plugin-kernel-types.js' + * @import { AiGatewayCapability, PluginActivationContext } from '../../../../hypaware-plugin-kernel-types.js' */ const PLUGIN_NAME = '@hypaware/openclaw' const CLIENT_NAME = 'openclaw' const UPSTREAM_NAME = 'anthropic' const OPENAI_UPSTREAM_NAME = 'openai' -const STEERING_PLUGIN_NAME = '@hypaware/openclaw-steering-plugin' - -/** - * Human-readable message `attach()` prints/logs: routing is owned by the - * OpenClaw-side steering plugin, installed through OpenClaw's own plugin - * manager, not by a HypAware-side settings write. - */ -const ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE = - `OpenClaw routing is owned by the '${STEERING_PLUGIN_NAME}' npm package, ` + - `installed on the OpenClaw side (run 'openclaw plugins install ${STEERING_PLUGIN_NAME}'). ` + - 'This adapter no longer writes to openclaw.json; there is nothing for hyp attach to do here.' /** * The plugin's `config_sections` validator, surfaced as a side-effect-free @@ -53,29 +43,26 @@ export const configSection = { section: OPENCLAW_CONFIG_SECTION, validate: valid * client so `hyp attach openclaw` / `hyp detach openclaw` / `hyp clients * openclaw` keep resolving it. * - * Routing is no longer a HypAware-side settings write (LLP 0152): OpenClaw - * traffic is steered by the `@hypaware/openclaw-steering-plugin` npm - * package the user installs on the OpenClaw side. `attach()` is therefore - * an honest no-op: it writes nothing and only reports that routing lives - * elsewhere. The manifest declares no `attach_probe` (R7), so the generic - * attach-on-join reconciler already skips this client - * (`if (!descriptor.attachProbe) continue`); this no-op only runs for the - * manual `hyp attach openclaw` command, which resolves `getClient()` - * directly and does not gate on `attachProbe`. + * Routing is a HypAware-side settings write again (LLP 0168/0169 reverse + * LLP 0152's steering-plugin premise): `attach()` writes the two + * `models.providers` entries of LLP 0167#override-entries into + * `openclaw.json`. The effect itself lives in `attach.js` so the + * refuse-then-write ordering is testable without an activation around it; + * this function only wires it in. The reversal is the single core disk-driven + * undo (LLP 0045 Part 3), which stays inert until the manifest registers the + * `json_path` attach probe that drives it. * - * `attach()` still emits a `client.attach` span tagged with `hyp_plugin`, - * `client_name`, `status`, and `restored=false` (there is nothing to - * restore). The reversing detach is the single core disk-driven undo - * (LLP 0045 Part 3), which is likewise an honest no-op here since the - * descriptor carries no `attach_probe`. No skills ship in v1 (`skill_dir` - * is declared in the manifest for the follow-up). + * `attach()` emits a `client.attach` span tagged with `hyp_plugin`, + * `client_name`, `status`, and `restored=false` (it never displaces a user's + * entry, only rewrites its own, so there is never anything to restore). No + * skills ship in v1 (`skill_dir` is declared in the manifest for the + * follow-up). * * @param {PluginActivationContext} ctx * @ref LLP 0016#knows-nothing-about-claude-or-codex [implements]: adapter requires the ai-gateway capability; registers client + upstream preset - * @ref LLP 0161#activate-and-client-registration [implements]: keeps - * gateway.registerClient() registered with an honest no-op attach() so - * the manual attach/detach/clients commands keep resolving 'openclaw', - * even though routing moved to the OpenClaw-side steering plugin. + * @ref LLP 0169#decision [implements]: the attach surface returns, so + * gateway.registerClient() carries a real settings write again and the + * LLP 0044 attach-on-join loop covers OpenClaw like Claude and Codex. */ export async function activate(ctx) { ctx.configRegistry.registerSection({ @@ -149,66 +136,55 @@ export async function activate(ctx) { // never re-importing sessions live capture already dropped. The list lives // at the SHARED state root, not the per-plugin `ctx.paths.stateDir` where // the file never exists. + // + // `config: ctx.config` is what makes `backfill.sweep_cron` and + // `backfill.quiesce_ms` mean anything at runtime. `ctx.config` is this + // plugin's own already-validated slice, the exact shape `config.js`'s + // `validateBackfillSection` checks. Omitting it left both keys validated on + // the way in and then silently discarded: the contribution registered the + // hardcoded `*/5 * * * *` and 180000ms defaults no matter what the operator + // configured, with no diagnostic anywhere. + // @ref LLP 0172#lane-b-sweep [implements]: the registered contribution's + // `sweep` is populated from this plugin's own validated config, so a + // configured cadence (and quiesce window) is the one that runs ctx.backfills.register( createOpenclawBackfillProvider({ homeDir: ctx.env.HOME ?? os.homedir(), env: ctx.env, clientName: CLIENT_NAME, pluginName: PLUGIN_NAME, + config: ctx.config, localOnlyListPath: localOnlyListPath(readObservabilityEnv(ctx.env).stateDir), }) ) - const logger = getLogger('plugin.openclaw') + const openclawAttach = createOpenclawAttach({ + homeDir: ctx.env.HOME ?? os.homedir(), + env: ctx.env, + }) - // @ref LLP 0143#decision [constrained-by]: no attach_probe means detach's - // core disk-driven undo is already an honest no-op ({ changed: false }); - // this registerClient() keeps attach() registered (a decorative-marker - // problem LLP 0143 warns against does not apply here, since attach() - // never claims to have written anything) purely so the manual - // attach/detach/clients commands keep resolving 'openclaw' by name. gateway.registerClient({ name: CLIENT_NAME, defaultUpstream: UPSTREAM_NAME, - /** @param {AiGatewayClientAttachContext} attachCtx */ + // The kernel types the registered `attach()` as `Promise`, so both + // callers infer success from "did it throw" and a returned outcome reaches + // neither of them. Translating a `failed` outcome into a throw is the only + // way a refusal is observable at all: in the reconciler it lands in + // `perform()`'s existing catch and becomes the `{status:'failed', reason}` + // marker that is recorded, warned, and retried next pass, while the + // reconciler's other actions for the same join carry on (a failed action is + // surfaced, not fatal); on `hyp attach --client openclaw` it becomes exit 1 + // instead of a refusal printed under exit 0. Returning quietly instead wrote + // a `done` marker whose endpoint and assets_key both matched, so + // `isCurrent()` called it current forever and the join never re-attached + // even after the user removed the conflicting `models.providers` entry. + // The effect has already reported the reason on `attachCtx.stdout`; the + // throw carries the same text to the caller's error path. + // @ref LLP 0172#lane-a-attach [implements]: a refusal is recorded as a + // retryable failure and never aborts the join async attach(attachCtx) { - return withSpan( - 'client.attach', - { - [Attr.PLUGIN]: PLUGIN_NAME, - [Attr.OPERATION]: 'client.attach', - client_name: CLIENT_NAME, - hyp_client: CLIENT_NAME, - dry_run: attachCtx.dryRun === true, - }, - async (span) => { - span.setAttribute('status', 'ok') - span.setAttribute('restored', false) - span.setAttribute('routing_owned_by', STEERING_PLUGIN_NAME) - if (attachCtx.json) { - attachCtx.stdout.write( - JSON.stringify({ - status: 'ok', - action: 'attach', - client: CLIENT_NAME, - dry_run: attachCtx.dryRun === true, - changed: false, - routing_owned_by: STEERING_PLUGIN_NAME, - message: ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE, - }) + '\n' - ) - } else { - attachCtx.stdout.write(`${ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE}\n`) - } - logger.info('client.attach.noop', { - hyp_plugin: PLUGIN_NAME, - hyp_client: CLIENT_NAME, - routing_owned_by: STEERING_PLUGIN_NAME, - dry_run: attachCtx.dryRun === true, - }) - }, - { component: 'plugin.openclaw' } - ) + const outcome = await openclawAttach.attach(attachCtx) + if (outcome.status === 'failed') throw new Error(outcome.reason) }, }) } diff --git a/hypaware-core/plugins-workspace/openclaw/src/projector.js b/hypaware-core/plugins-workspace/openclaw/src/projector.js index ec7103b7..48bb7517 100644 --- a/hypaware-core/plugins-workspace/openclaw/src/projector.js +++ b/hypaware-core/plugins-workspace/openclaw/src/projector.js @@ -28,17 +28,17 @@ const CLIENT_NAME = 'openclaw' const CLIENT_HEADER = 'x-hypaware-client' /** - * Written by the `openclaw-steering-plugin` (the OpenClaw-side npm - * package, not this adapter) on any request it has decided to steer - * through a shadow provider. Names the real upstream provider - * (`'anthropic'` or `'openai'`), never a `hypaware-*` shadow id, so the - * gateway's upstream presets know which static `base_url` to forward to - * regardless of which path the shadow provider's own client happened to - * hit. The header has two independent readers here, the presets' - * `match()` and this projector's `project()`, and both tolerate its - * absence. + * Written into `openclaw.json`'s `models.providers` entry by this adapter's + * own `attach()` (the config-override write, not a shadow-provider steer), + * as a static `headers` field on the provider OpenClaw itself calls with no + * further plugin involved. Names the real upstream provider (`'anthropic'` + * or `'openai'`), never a `hypaware-*` shadow id, so the gateway's upstream + * presets know which static `base_url` to forward to regardless of which + * path the request happened to hit. The header has two independent readers + * here, the presets' `match()` and this projector's `project()`, and both + * tolerate its absence. * - * @ref LLP 0161#upstream-header [implements]: the one wire contract the steering plugin and this adapter agree on byte-for-byte + * @ref LLP 0167#override-entries [implements]: the one wire contract attach's config write and this adapter agree on byte-for-byte */ const UPSTREAM_HEADER = 'x-hypaware-upstream' diff --git a/hypaware-core/plugins-workspace/openclaw/src/types.d.ts b/hypaware-core/plugins-workspace/openclaw/src/types.d.ts index c8fee1ec..8542f490 100644 --- a/hypaware-core/plugins-workspace/openclaw/src/types.d.ts +++ b/hypaware-core/plugins-workspace/openclaw/src/types.d.ts @@ -1,3 +1,38 @@ +import type fsp from 'node:fs/promises' + +/** + * Construction options for `createOpenclawAttach`. Every field has a + * process-wide default so a production caller passes only what it has: + * `index.js` threads the kernel's `ctx.env`/`HOME`, while a test injects a + * temp `homeDir` (or an `OPENCLAW_HOME` in `env`) and reads the file back. + */ +export interface OpenclawAttachOptions { + /** `$HOME` the `.openclaw/openclaw.json` path is resolved against. */ + homeDir?: string + /** Env the `$OPENCLAW_HOME` relocation is read from. */ + env?: NodeJS.ProcessEnv + /** Injectable `node:fs/promises`, for the read and the atomic write. */ + fs?: typeof fsp + /** Injectable logger; defaults to the `plugin.openclaw` logger. */ + logger?: { info(event: string, fields: Record): void, warn(event: string, fields: Record): void } +} + +/** + * What the effect reports back. Deliberately the `ActionOutcome` shape the + * generic client-action reconciler already understands (LLP 0169): a refusal + * is a returned `failed`, never a throw, so nothing is half-written and the + * caller decides what to do with it. + * + * The kernel types the *registered* `attach()` as `Promise`, so + * `index.js`'s wrapper rethrows a `failed` outcome to make it visible at all; + * `perform()`'s catch turns it back into this same shape, which the + * reconciler records, warns about, and retries next pass without failing the + * join (LLP 0172 §1.3). + */ +export type OpenclawAttachOutcome = + | { status: 'done' } + | { status: 'failed', reason: string } + /** * The `type: "session"` header line of an OpenClaw session JSONL file * (`~/.openclaw/agents//sessions/.jsonl`). Each field is diff --git a/hypaware-core/smoke/flows/backfill_openclaw_fixture.js b/hypaware-core/smoke/flows/backfill_openclaw_fixture.js new file mode 100644 index 00000000..77d5af3e --- /dev/null +++ b/hypaware-core/smoke/flows/backfill_openclaw_fixture.js @@ -0,0 +1,456 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' +import process from 'node:process' + +import { Attr, installObservability, runRoot } from '../../../src/core/observability/index.js' +import { dispatch } from '../../../src/core/cli/dispatch.js' +import { createCommandRegistry } from '../../../src/core/registry/commands.js' +import { registerCoreCommands } from '../../../src/core/cli/core_commands.js' +import { createKernelRuntime } from '../../../src/core/runtime/activation.js' +import { activatePlugins } from '../../../src/core/runtime/loader.js' +import { loadManifests } from '../../../src/core/manifest.js' +import { resolveDependencies } from '../../../src/core/dep_graph.js' +import { createBackfillSweepDriver } from '../../../src/core/daemon/backfill_sweep.js' +import { runBackfillProvider } from '../../../src/core/commands/backfill.js' + +/** + * LLP 0173 T12 smoke: OpenClaw Lane B sweep -> quiesce filter -> dedupe. + * + * Boots `@hypaware/ai-gateway` + `@hypaware/openclaw` against a tmp + * `HYP_HOME` with two staged OpenClaw v3 session fixtures under the fake + * HOME's `.openclaw/agents/main/sessions/`, both in the nested-`message`- + * envelope shape PR #552's reader (`session_file.js`) projects, and drives + * `src/core/daemon/backfill_sweep.js`'s real `createBackfillSweepDriver` + * (the same driver `runTick()` wires into the daemon's sink-tick cadence, + * LLP 0172#lane-b-sweep) directly against this boot's own + * `kernel.backfills` / `kernel.backfillMaterializers` / `kernel.storage`, + * so a sweep-written row and a `hyp query`-read row land in and come from + * the exact same tables `hyp backfill openclaw` would use. + * + * Asserts the three properties LLP 0173's T12 brief names: + * + * - **(a) quiesce skip**: a session file whose mtime is inside the + * default 180000ms quiesce window (LLP 0172#45-the-quiesce-window) is + * absent from the sweep's first tick. + * - **(b) quiesce capture**: a session file backdated past the window is + * captured by that same tick, with native message identity. + * - **(c) cross-write dedupe**: a second sweep tick (a fresh `now`, so the + * ai-gateway materializer's dedupe gets its own `devRunId` and is + * forced to re-scan committed partitions rather than reuse an + * in-memory seen set) finds the first tick's part_ids already + * committed and writes ZERO new rows. R11's identity-convergence + * argument (`openclaw/src/backfill.js`'s own module doc) is exactly + * that Lane A (live) and Lane B (backfill) land on the same + * `part_id` for the same turn, so the dedupe this proves for two + * sweep ticks is indistinguishable, at the write layer, from "a + * live-lane row already wrote it before the sweep ran." + * + * The sweep's own `now` is chosen to land on OpenClaw's default + * `sweep.cron` (every 5th minute) so this exercises the real `cronMatches` + * due-check (`src/core/sinks/driver.js`, imported by the sweep driver), + * not a `force: true` bypass: this is the only automated coverage, of any + * tier, for the sweep driver's `cronMatches` wiring and the quiesce + * filter's composition with it before the human acceptance run + * (LLP 0173's "hermetic-smoke decision" section, LLP 0172 Section 9). + * + * @ref LLP 0172#45-the-quiesce-window [tests]: a file inside the default + * quiesce window is skipped, one backdated past it is captured + * @ref LLP 0172#lane-b-sweep [tests]: the sweep driver fires the due, + * sweep-bearing provider through the real `cronMatches` due-check + * @ref LLP 0161#backfill-provider [tests]: native message identity makes a + * sweep-then-rerun (standing in for Lane A already having written the same + * part_id) net zero new rows + * + * @param {{ harness: any, expect: any }} args + */ +export async function run({ harness, expect }) { + const obs = installObservability() + if (!obs.tracer.provider) { + throw new Error( + 'backfill_openclaw_fixture: tracer provider not installed - expected HYP_DEV_TELEMETRY=1' + ) + } + + const cacheRoot = path.join(harness.stateDir, 'cache') + const registry = createCommandRegistry() + registerCoreCommands(registry) + const kernel = createKernelRuntime({ commandRegistry: registry, cacheRoot }) + + const pluginsRoot = path.resolve(import.meta.dirname, '..', '..', 'plugins-workspace') + const pluginDirs = [ + path.join(pluginsRoot, 'ai-gateway'), + path.join(pluginsRoot, 'openclaw'), + ] + + // The OpenClaw provider captures its `agents/` root from `ctx.env.HOME` + // (-> `/.openclaw/agents`, `session_file.js`'s + // `defaultOpenclawAgentsDir`) at activation, so stage both session + // fixtures and point HOME at the fake home BEFORE activating plugins. + const fakeHome = path.join(harness.tmpDir, 'home') + const agentsDir = path.join(fakeHome, '.openclaw', 'agents') + const agentId = 'main' + + const freshSessionId = `oc-fresh-${harness.devRunId}` + const oldSessionId = `oc-old-${harness.devRunId}` + + // Inside the quiesce window: a freshly-written file, left untouched, sits + // well inside the default 180000ms window for the whole duration of this + // smoke. + await writeOpenclawSession({ agentsDir, agentId, sessionId: freshSessionId }) + // Outside the quiesce window: back-dated 4 minutes, mirroring + // `test/plugins/openclaw-backfill.test.js`'s own default-quiesce-window + // precedent (`ageFile`, 4 * 60 * 1000 against the real 180000ms default). + const oldFilePath = await writeOpenclawSession({ agentsDir, agentId, sessionId: oldSessionId }) + await ageFile(oldFilePath, 4 * 60 * 1000) + + const previousHome = process.env.HOME + process.env.HOME = fakeHome + + try { + await runRoot( + 'kernel.boot', + { + [Attr.COMPONENT]: 'kernel', + [Attr.OPERATION]: 'boot', + [Attr.SMOKE_NAME]: harness.smokeName, + [Attr.SMOKE_STEP]: 'sweep_activate', + [Attr.DEV_RUN_ID]: harness.devRunId, + status: 'ok', + }, + async () => { + const { loaded } = await loadManifests(pluginDirs) + if (loaded.length !== pluginDirs.length) { + throw new Error(`backfill_openclaw_fixture: expected ${pluginDirs.length} manifests, got ${loaded.length}`) + } + const resolution = await resolveDependencies(loaded.map((l) => l.manifest)) + if (resolution.unsatisfied.length > 0) { + throw new Error( + `backfill_openclaw_fixture: unsatisfied requirements: ${ + resolution.unsatisfied.map((u) => `${u.plugin}:${u.errorKind}`).join(', ') + }` + ) + } + const byName = new Map(loaded.map((l) => [l.manifest.name, l])) + const entries = resolution.order + .map((name) => byName.get(name)) + .filter((l) => l !== undefined) + .map((l) => ({ manifest: l.manifest, rootDir: l.rootDir, config: {} })) + return activatePlugins({ + plugins: entries, + stateRoot: harness.stateDir, + runId: harness.devRunId, + runtime: kernel, + tmpRoot: path.join(harness.tmpDir, 'plugin-temp'), + }) + } + ) + + const env = { ...process.env, HYP_HOME: harness.hypHome } + + // The sweep driver itself, wired exactly the way the daemon wires it + // (`src/core/daemon/backfill_sweep.js`'s own doc), reusing this boot's + // `kernel.backfills` / `kernel.backfillMaterializers` / `kernel.storage`. + /** @type {Array>} */ + const pendingRuns = [] + const sweep = createBackfillSweepDriver({ + backfills: kernel.backfills, + backfillMaterializers: kernel.backfillMaterializers, + storage: kernel.storage, + query: kernel.query, + env, + config: { version: 2 }, + // Test seam (`src/core/daemon/types.d.ts`'s `BackfillSweepRunner`): + // `tick()` fires this fire-and-forget internally and resolves once + // runs are STARTED, not finished, so the smoke needs its own handle + // on the underlying promise to await completion before it queries or + // reruns. Still the real `runBackfillProvider`, just with its + // promise captured on the way out. + runBackfill: (/** @type {any} */ args) => { + const p = runBackfillProvider(args) + pendingRuns.push(p) + return p + }, + }) + + /** + * Run one sweep tick and await every run it fired. + * + * @param {Date} now + */ + async function tickAndAwait(now) { + pendingRuns.length = 0 + const report = await sweep.tick({ now }) + const results = await Promise.all(pendingRuns) + return { report, result: results[0] } + } + + // A UTC-minute-0 instant is due against OpenClaw's default `sweep.cron` + // (every 5th minute): real `cronMatches`, not a `force: true` bypass. + const tick1Now = new Date(Date.UTC(2026, 0, 1, 0, 0, 0)) + // A later due instant, still on the 5-minute grid, so the second tick + // gets its own `devRunId` and the ai-gateway materializer's + // `createBackfillDedupe` (memoized per `devRunId`) is forced to + // re-scan committed partitions rather than reuse tick 1's in-memory + // seen set. + const tick2Now = new Date(tick1Now.getTime() + 5 * 60 * 1000) + + // ----- 1. First sweep tick: quiesce skip + quiesce capture ((a)/(b)) ----- + const tick1 = await tickAndAwait(tick1Now) + expect.that( + 'tick 1: the openclaw provider fired', + tick1.report.fired, + (v) => Array.isArray(v) && v.includes('openclaw'), + ) + expect.that( + 'tick 1: exactly one session file scanned (only the one outside the quiesce window)', + tick1.result, + (v) => v !== undefined && v.ok === true && v.scanned === 1, + ) + expect.that( + 'tick 1: both rows of the outside-window session were written', + tick1.result, + (v) => v !== undefined && v.rowsWritten === 2, + ) + + /** @param {string} sessionId */ + const sqlFor = (sessionId) => ` + select role, content_text, message_id, part_id, provider, conversation_source, client_name + from ai_gateway_messages + where session_id = '${sessionId}' + order by message_index, part_index + `.trim().replace(/\s+/g, ' ') + + const freshRowsAfterTick1 = await queryRows({ + dispatch, sql: sqlFor(freshSessionId), kernel, registry, env, expect, label: 'fresh session after tick 1', + }) + expect.that( + '(a) a file with mtime inside the quiesce window is skipped by the sweep run', + freshRowsAfterTick1, + (v) => Array.isArray(v) && v.length === 0, + ) + + const oldRowsAfterTick1 = await queryRows({ + dispatch, sql: sqlFor(oldSessionId), kernel, registry, env, expect, label: 'old session after tick 1', + }) + expect.that( + '(b) a file with mtime outside the quiesce window is captured by the sweep run', + oldRowsAfterTick1, + (v) => Array.isArray(v) && v.length === 2, + ) + expect.that( + '(b) every captured row carries native identity and the right client/source', + oldRowsAfterTick1, + (v) => Array.isArray(v) && v.every( + (/** @type {any} */ r) => r.conversation_source === 'openclaw' && r.client_name === 'openclaw' && + r.provider === 'anthropic' && typeof r.message_id === 'string' && r.message_id.length > 0, + ), + ) + + // ----- 2. Second sweep tick: cross-write dedupe (c) ----- + const tick2 = await tickAndAwait(tick2Now) + expect.that( + 'tick 2: the openclaw provider fired again', + tick2.report.fired, + (v) => Array.isArray(v) && v.includes('openclaw'), + ) + expect.that( + '(c) rerunning the sweep after the part_id was already written nets zero new rows', + tick2.result, + (v) => v !== undefined && v.ok === true && v.rowsWritten === 0, + ) + + const oldRowsAfterTick2 = await queryRows({ + dispatch, sql: sqlFor(oldSessionId), kernel, registry, env, expect, label: 'old session after tick 2', + }) + expect.that( + '(c) the rerun did not duplicate rows (still exactly two)', + oldRowsAfterTick2, + (v) => Array.isArray(v) && v.length === 2, + ) + expect.that( + '(c) the rerun\'s row set is byte-identical to tick 1\'s (same part_ids, no drift)', + oldRowsAfterTick2, + (v) => Array.isArray(v) && + JSON.stringify(v.map((/** @type {any} */ r) => r.part_id).sort()) === + JSON.stringify(oldRowsAfterTick1.map((/** @type {any} */ r) => r.part_id).sort()), + ) + + const freshRowsAfterTick2 = await queryRows({ + dispatch, sql: sqlFor(freshSessionId), kernel, registry, env, expect, label: 'fresh session after tick 2', + }) + expect.that( + '(c) the still-quiesced session remains untouched by the rerun', + freshRowsAfterTick2, + (v) => Array.isArray(v) && v.length === 0, + ) + + // ----- 3. Internal telemetry: the sweep driver's own log lines, distinct + // from `hyp backfill`'s CLI-path logs, prove this ran through + // the daemon-facing driver (T9's cronMatches wiring), not just + // the provider underneath it. ----- + await obs.shutdown() + const logs = await expect.logs() + + const dueLogs = logs.filter( + (/** @type {any} */ l) => l.body === 'backfill.sweep_due' && l.attributes?.provider === 'openclaw', + ) + expect.that( + 'logs: backfill.sweep_due fired once per due tick (twice total)', + dueLogs, + (v) => Array.isArray(v) && v.length === 2, + ) + + const finishedLogs = logs.filter( + (/** @type {any} */ l) => l.body === 'backfill.sweep_finished' && l.attributes?.provider === 'openclaw', + ) + expect.that( + 'logs: backfill.sweep_finished (tick 1) reports rows_written=2', + finishedLogs.find((/** @type {any} */ l) => l.attributes?.rows_written === 2), + (v) => v !== undefined, + ) + expect.that( + 'logs: backfill.sweep_finished (tick 2) reports rows_written=0', + finishedLogs.find((/** @type {any} */ l) => l.attributes?.rows_written === 0), + (v) => v !== undefined, + ) + + const scanCompleteLogs = logs.filter( + (/** @type {any} */ l) => l.body === 'openclaw.backfill.scan_complete', + ) + expect.that( + 'logs: openclaw.backfill.scan_complete (tick 1) saw one file, past the quiesce filter', + scanCompleteLogs[0], + (v) => v !== undefined && v.attributes?.files_seen === 1 && v.attributes?.sessions_projected === 1, + ) + } finally { + if (previousHome === undefined) delete process.env.HOME + else process.env.HOME = previousHome + } +} + +/** + * Write one minimal OpenClaw v3 session JSONL under + * `//sessions/.jsonl`: a `type: "session"` + * header line and one user/assistant turn in the nested-`message`-envelope + * shape PR #552's reader (`session_file.js`'s `parseOpenclawSessionMessage` + * / `openclawMessageEnvelope`) actually projects - `role`/`content` and, + * on the assistant turn, `model`/`provider`/`api`/`stopReason`/`usage` + * nested under the record's own `message` object, never flat on the + * record line (a flat fixture would test the reader's now-fixed #543 bug, + * not its fix). No `cwd` on the header: an absent `cwd` reads as "not + * usable" (`openclawSessionCwd`) and the session is simply not + * usage-policy gated, which keeps this fixture independent of the host's + * real filesystem beyond the temp tree it writes. + * + * @param {{ agentsDir: string, agentId: string, sessionId: string }} args + * @returns {Promise} + */ +async function writeOpenclawSession(args) { + const { agentsDir, agentId, sessionId } = args + const dir = path.join(agentsDir, agentId, 'sessions') + await fs.mkdir(dir, { recursive: true }) + const filePath = path.join(dir, `${sessionId}.jsonl`) + const startedAt = new Date().toISOString() + const lines = [ + JSON.stringify({ type: 'session', version: 3, id: sessionId, timestamp: startedAt }), + JSON.stringify(messageLine({ + id: `${sessionId}-user`, + timestamp: startedAt, + role: 'user', + content: [{ type: 'text', text: 'list the files' }], + })), + JSON.stringify(messageLine({ + id: `${sessionId}-asst`, + timestamp: startedAt, + parentId: `${sessionId}-user`, + role: 'assistant', + content: [{ type: 'text', text: 'here they are' }], + model: 'claude-sonnet-4-5', + provider: 'anthropic', + api: 'anthropic-messages', + stopReason: 'end_turn', + usage: { input: 11, output: 7, cacheRead: 3, cacheWrite: 2 }, + })), + ] + await fs.writeFile(filePath, lines.join('\n') + '\n', 'utf8') + return filePath +} + +/** + * One `type: "message"` line in the shape OpenClaw actually appends: `id`, + * `parentId`, and `timestamp` on the record line, and every message field + * nested under `message`. Mirrors `test/plugins/openclaw-backfill.test.js`'s + * own `messageLine` helper, verified there against a live install (record + * keys `['id', 'message', 'parentId', 'timestamp', 'type']`). + * + * @param {Record} fields + * @returns {Record} + */ +function messageLine(fields) { + const { id, timestamp, parentId, ...message } = fields + return { + type: 'message', + ...(id !== undefined ? { id } : {}), + ...(timestamp !== undefined ? { timestamp } : {}), + parentId: parentId ?? null, + message: { ...message, ...(timestamp !== undefined ? { timestamp } : {}) }, + } +} + +/** + * Back-date `filePath`'s mtime by `msAgo` milliseconds, so a quiesce-window + * scenario can control file recency without waiting on the wall clock. + * + * @param {string} filePath + * @param {number} msAgo + */ +async function ageFile(filePath, msAgo) { + const past = new Date(Date.now() - msAgo) + await fs.utimes(filePath, past, past) +} + +/** + * Run a `query sql ... --format json` dispatch and return the parsed rows, + * asserting a clean exit and parseable output. + * + * @param {{ dispatch: any, sql: string, kernel: any, registry: any, env: any, expect: any, label: string }} args + * @returns {Promise} + */ +async function queryRows(args) { + const { dispatch: doDispatch, sql, kernel, registry, env, expect, label } = args + const out = makeBuf() + const err = makeBuf() + const code = await doDispatch( + ['query', 'sql', sql, '--refresh', 'always', '--format', 'json'], + { stdout: out, stderr: err, kernel, registry, env } + ) + expect.that(`dispatch: query (${label}) exited 0`, code, (/** @type {number} */ v) => v === 0) + expect.that(`stderr: query (${label}) had no errors`, err.text(), (/** @type {string} */ v) => typeof v === 'string' && v.length === 0) + try { + return JSON.parse(out.text()) + } catch (e) { + expect.that( + `stdout: query (${label}) was valid JSON (${e instanceof Error ? e.message : String(e)})`, + false, + (/** @type {boolean} */ v) => v === true, + ) + return [] + } +} + +function makeBuf() { + /** @type {string[]} */ + const chunks = [] + return { + /** @param {unknown} chunk */ + write(chunk) { + chunks.push(typeof chunk === 'string' ? chunk : String(chunk)) + return true + }, + text() { + return chunks.join('') + }, + } +} diff --git a/hypaware-plugin-kernel-types.d.ts b/hypaware-plugin-kernel-types.d.ts index fce42918..7aa4fcc4 100644 --- a/hypaware-plugin-kernel-types.d.ts +++ b/hypaware-plugin-kernel-types.d.ts @@ -181,17 +181,27 @@ export interface PluginClientManifest { export interface PluginAttachProbeManifest { /** - * The `json_path` format is gone (LLP 0143 R7): core no longer carries a - * read side in `daemon/status.js` or an undo side in - * `config/client_detach_disk.js` for it. Keeping it in this union would be - * worse than a dead name, because nothing validates `attach_probe` at - * runtime (`src/core/manifest.js` treats `contributes` opaquely): a - * manifest declaring it would type-check, probe as never-attached, and - * then slip past `action_attach.js`'s `!descriptor.attachProbe` orphaning - * guard on reverse, dropping the marker with the client's settings still - * written. That is exactly #212. - */ - format: 'json' | 'toml' + * `json_path` returns here (LLP 0173 T1), reversing LLP 0143's removal. + * LLP 0143 pulled the format because, at the time, nothing validated + * `attach_probe` at runtime (`src/core/manifest.js` treats `contributes` + * opaquely): a manifest declaring `json_path` with no runtime support + * behind it would type-check, probe as never-attached, and then slip + * past `action_attach.js`'s `!descriptor.attachProbe` orphaning guard on + * reverse, dropping the marker with the client's settings still written. + * That was exactly #212. The danger was in the *gap* between declaring + * the format and a runtime that reads/undoes it, not in the format + * itself. LLP 0173 T2 restores the undo side + * (`client_detach_disk.js`'s `detachJsonPathProviders`) and T3 restores + * the read side (`daemon/status.js`'s `json_path` branch) before + * OpenClaw's manifest (T5) declares this format again, so the gap #212 + * warned about is closed by construction: no manifest may reach this + * format without both runtime sides already merged. + * + * @ref LLP 0172#lane-a-detach [implements]: json_path's runtime undo + * (client_detach_disk.js) and read (daemon/status.js) sides, which close + * the #212 gap this format's prior removal warned about. + */ + format: 'json' | 'toml' | 'json_path' /** * The client's settings file, RELATIVE to the user's home (e.g. * `.codex/config.toml`). Its first path segment is the client's config @@ -204,6 +214,26 @@ export interface PluginAttachProbeManifest { settings_file: string marker_key?: string marker_header?: string + /** + * `json_path` only: dotted path, relative to the parsed settings file, + * to the container object the probe/undo navigate (e.g. + * `models.providers`). + */ + container_path?: string + /** + * `json_path` only: the container's keys the probe/undo consider, in + * order (e.g. `['anthropic', 'openai']`). `marker_header` (reused, not + * duplicated) is checked against each key's own header value to decide + * ownership. + */ + provider_keys?: string[] + /** + * `json_path` only: glob, relative to the client's config home, of + * cache files the undo best-effort purges the same `provider_keys` + * entries from after the settings-file write (e.g. + * `agents/*\/agent/models.json`). + */ + cache_glob?: string } /** @@ -2257,6 +2287,21 @@ export interface BackfillContribution { * dataset-materializer registry. */ run(ctx: BackfillRunContext): AsyncIterable + /** + * Opt-in scheduling metadata for the daemon's periodic sweep (LLP 0173 + * T9's `backfill_sweep.js`), not a new mechanism: a contribution with no + * `sweep` field is never ticked by the sweep driver, so this is + * absent-by-default and zero behavior change for every provider that + * doesn't set it (Claude's and Codex's contributions today, and + * OpenClaw's own prior to LLP 0173). `cron` is a standard 5-field cron + * expression, evaluated by `cronMatches` (`src/core/sinks/driver.js`), + * the same schedule shape sinks already use. + * + * @ref LLP 0172#lane-b-sweep [implements]: the daemon-side scheduled + * sweep's opt-in contribution field, absent-by-default for every + * provider that doesn't set it. + */ + sweep?: { cron: string } } export interface BackfillPlanContext { diff --git a/llp/0167-openclaw-capture-via-config-provider-override.rfc.md b/llp/0167-openclaw-capture-via-config-provider-override.rfc.md index 18fbcb27..15622006 100644 --- a/llp/0167-openclaw-capture-via-config-provider-override.rfc.md +++ b/llp/0167-openclaw-capture-via-config-provider-override.rfc.md @@ -108,17 +108,35 @@ proposal.** ### Attach, detach, undo {#attach-detach} -- **Attach is refuse + create-only.** If the user already declares - `models.providers.anthropic` or `.openai`, attach refuses with an - explanation: those keys are purely user-authored (verified: no OpenClaw - code writes them; a default install has none), so their presence means - the user deliberately routed that provider somewhere, and silently - rerouting a deliberate override is the surprise this design family - refuses to allow. Otherwise attach creates the two entries whole. +- **Attach refuses over a user's entry, and rewrites its own.** If the + user already declares `models.providers.anthropic` or `.openai`, + attach refuses with an explanation: those keys are user-authored + (verified: no OpenClaw code writes them; a default install has none), + so their presence means the user deliberately routed that provider + somewhere, and silently rerouting a deliberate override is the + surprise this design family refuses to allow. Otherwise attach writes + the two entries whole. + + The one entry that is *not* a user's is the one attach itself wrote, + and it must be overwritable, because attach re-runs. `isCurrent()` + makes a `done` attach stale whenever the gateway rebound to a new + ephemeral port (LLP 0086) or the contributed asset set changed + (LLP 0107), and the reconciler then re-performs; a presence-only + refusal would fail every one of those passes forever, leaving + `openclaw.json` pinned to a dead port while the marker-header probe + still reported `attached`. So the refusal is **ownership-aware**, on + the same self-identifying triple detach already tests before deleting + (`baseUrl`, `headers['x-hypaware-upstream']` naming the key, empty + `models`), out of one shared predicate + (`src/core/config/provider_entry_ownership.js`) so the two halves + cannot drift. Attach's call passes no expected base URL, deliberately: + on a drift re-attach its own entry carries the *old* origin. + Detach deletes an entry only when its `baseUrl` is the gateway's; a present-but-not-ours or mangled entry is backed up, never discarded - (LLP 0163 precedent). Create-only means there is no prior state to - restore, so no undo record exists anywhere: deletion is the whole undo. + (LLP 0163 precedent). Nothing a user authored is ever displaced, so no + prior state is stored and no undo record exists anywhere: deletion is + the whole undo. - **The marker is the entry itself.** The `x-hypaware-upstream` header inside the created entry is the probeable marker. The manifest regains `contributes.client.attach_probe` in the `json_path` format, and core diff --git a/llp/0169-openclaw-attach-surface-returns.decision.md b/llp/0169-openclaw-attach-surface-returns.decision.md index c39a7465..32d04baf 100644 --- a/llp/0169-openclaw-attach-surface-returns.decision.md +++ b/llp/0169-openclaw-attach-surface-returns.decision.md @@ -9,9 +9,9 @@ > With LLP 0168 writing real config entries, there is again a > reversible settings-file write for the LLP 0044/0045 loop to own. -> Attach is refuse + create-only, the marker is the entry itself, core -> revives the `json_path` probe format, and detach also rewrites the -> per-agent model caches, which do not self-heal. +> Attach refuses over a user's entry and rewrites its own, the marker is +> the entry itself, core revives the `json_path` probe format, and detach +> also rewrites the per-agent model caches, which do not self-heal. ## Context @@ -25,11 +25,18 @@ indefinitely, live for routing. ## Decision -- **Refuse + create-only.** `models.providers.anthropic` and `.openai` - are purely user-authored keys; if either exists, attach refuses with - an explanation instead of merging. Otherwise attach creates the two - LLP 0168 entries whole. There is no undo record anywhere: deletion - is the whole undo. +- **Refuse over a user's entry, rewrite our own.** + `models.providers.anthropic` and `.openai` are user-authored keys; if + either holds a value HypAware did not write, attach refuses with an + explanation instead of merging. Otherwise attach writes the two + LLP 0168 entries whole, *including* over an entry a previous attach + wrote: `isCurrent()` re-performs attach on an ephemeral-port rebind + (LLP 0086) or an asset-set change (LLP 0107), so the write has to be + idempotent over its own output or every drift pass refuses forever. + Ownership is the self-identifying triple detach already tests + (`baseUrl`, marker header naming the key, empty `models`), shared with + it as one predicate. No user value is ever displaced, so there is no + undo record anywhere: deletion is the whole undo. - **The marker is the entry.** The `x-hypaware-upstream` header inside the created entry is the probeable marker. The manifest registers `attach_probe` in the `json_path` format, and core restores the diff --git a/llp/0171-openclaw-two-lane-capture.spec.md b/llp/0171-openclaw-two-lane-capture.spec.md index a9115d14..064bad60 100644 --- a/llp/0171-openclaw-two-lane-capture.spec.md +++ b/llp/0171-openclaw-two-lane-capture.spec.md @@ -35,7 +35,12 @@ reversed by R5 below, R12 is replaced by R11 below, and R13 is retired `openclaw.json`. - **R2.** Attach MUST refuse, with an explanation, when `models.providers.anthropic` or `models.providers.openai` already - exists. A refusal during attach-on-join MUST surface as a warning and + holds an entry HypAware did not write. It MUST overwrite one it did + (the self-identifying `baseUrl` + marker-header + empty-`models` + triple detach tests before deleting), because `isCurrent()` + re-performs attach on endpoint or asset-set drift (LLP 0086, + LLP 0107) and a refusal there would strand `openclaw.json` at a dead + port. A refusal during attach-on-join MUST surface as a warning and MUST NOT fail the join (LLP 0169). - **R3.** Detach MUST delete an entry only when its `baseUrl` is the gateway's, MUST back up rather than discard a present-but-unexpected diff --git a/llp/0172-openclaw-two-lane-capture.design.md b/llp/0172-openclaw-two-lane-capture.design.md new file mode 100644 index 00000000..d88b3a11 --- /dev/null +++ b/llp/0172-openclaw-two-lane-capture.design.md @@ -0,0 +1,835 @@ +# LLP 0172: OpenClaw two-lane capture, technical design + +**Type:** design +**Status:** Active +**Systems:** Plugins, Gateway, Config, Sources +**Generated-by:** neutral +**Related:** LLP 0167, LLP 0171, LLP 0168, LLP 0169, LLP 0170 + +> Technical design for the one deliverable set LLP 0171 specifies: the +> reworked `@hypaware/openclaw` attach/detach module (Lane A), the daemon-side +> scheduled sweep (Lane B), the `json_path` core revival, the +> `openclaw-steering-plugin/` deletion, and the acceptance/onboarding +> rewrites. Named files, exact call shapes, and the two open forks the prior +> design (LLP 0161) left unresolved for this half: the attach-probe/status +> interaction with issue #544's fix (PR #553), and the scheduling seam for a +> periodic in-process backfill run. + +## 0. Scope check {#scope-check} + +LLP 0167 is the accepted RFC; LLP 0171 is its requirements spec and states +"one deliverable set"; LLP 0168/0169/0170 are the decisions already merged +between them. Reading all five confirms the orchestrator's premise: nothing +here partitions into separate change sets. Lane A (attach/detach) and Lane B +(the sweep) share one file (`hypaware-core/plugins-workspace/openclaw/src/backfill.js` +gets touched for the quiesce filter that Lane B needs and that Lane A's +dedupe story depends on), one manifest, one config section, and one +acceptance rewrite. Splitting them would mean landing a manifest with a +`json_path` probe that Lane A's detach code doesn't yet honor, or shipping +the sweep with no attach surface to net its overlap against. This design +stays a single change set. + +## 1. Lane A: attach {#lane-a-attach} + +### 1.1 What gets deleted first + +`hypaware-core/plugins-workspace/openclaw/src/index.js` currently registers +`gateway.registerClient({ name: 'openclaw', defaultUpstream: 'anthropic', +async attach(attachCtx) {...} })` with an honest no-op body: it prints +`ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE` and writes nothing. That whole +body, `STEERING_PLUGIN_NAME`, and `ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE` +go. So does the `@ref LLP 0143#decision` comment block above it (lines +164-169 in the current tree) explaining why a no-op is correct: LLP 0169 +supersedes that reasoning outright. + +### 1.2 The new attach module + +New file: `hypaware-core/plugins-workspace/openclaw/src/attach.js`, mirroring +the shape of `hypaware-core/plugins-workspace/claude/src/index.js`'s +`attach()` registration (same `AiGatewayClientAttachContext` parameter, +same `withSpan('client.attach', ...)` wrapper, same dry-run branch writing +through `attachCtx.stdout`/`attachCtx.json`). It exports +`createOpenclawAttach({ homeDir, fs })` returning an object with one method, +`attach(attachCtx)`, that `index.js`'s `activate()` wires into +`gateway.registerClient()` in place of the deleted no-op. + +`attach(attachCtx)`: + +1. Reads `~/.openclaw/openclaw.json` (or `$OPENCLAW_HOME/openclaw.json` if + that env var is set, matching how the plugin already resolves the + settings file elsewhere). A missing or unparseable file is a hard + failure (`{status: 'failed', reason}`), not a refusal: attach can't + reason about a config it can't read. +2. Checks `config.models?.providers?.anthropic` and + `config.models?.providers?.openai`. If **either** key holds an entry + HypAware did not write, refuse: return `{status: 'failed', reason: + 'models.providers. already exists in openclaw.json and was not + written by HypAware; attach refuses to merge (LLP 0167#attach-detach). + Remove it by hand or run hyp detach --client openclaw first.'}` (R2). + Nothing is written; this is a pure read-then-decide, no partial write + to roll back. + + The test is **ownership, not bare presence**, and the difference is + load-bearing rather than cosmetic. 1.4 gives OpenClaw an + `attach_probe`, which is what makes it eligible for attach-on-join + (`action_attach.js`'s `desired()`), and `isCurrent()` returns false + whenever `marker.endpoint !== ctx.endpoint` (an ephemeral-port + rebind, LLP 0086) or the recorded `assets_key` drifts (LLP 0107). The + reconciler then re-`perform()`s, and a presence-only refusal fails + every one of those passes: the marker churns to `failed` with + `attempts` climbing, `hyp attach openclaw` exits 1 (1.3's rethrow), + and `openclaw.json` stays pinned to the dead port while + `probeClientAttachFromDescriptor`'s `json_path` branch, which matches + the marker header and never the base URL, keeps reporting + `attached: true`. `isCurrent`'s own contract says the opposite + ("`perform()` is idempotent in both halves"), so the write has to be + idempotent over its own previous output. + + The entry attach writes is self-identifying: `baseUrl`, + `headers['x-hypaware-upstream']` naming the key it sits at, and the + empty `models` array. That is exactly the triple 2.2's detach already + applies before it may delete an entry, so the two live in one shared + predicate, `src/core/config/provider_entry_ownership.js` + (`isOwnedProviderEntry` + `ownedBaseUrls`, moved out of + `client_detach_disk.js`), rather than being restated on each side + where they could drift into disagreeing about the same file. The + predicate's base-URL set is optional and attach passes none: on a + drift re-attach its own entry carries the *old* origin by + construction, so pinning the check to the live endpoint would + reintroduce the refusal it exists to avoid. Detach always passes the + set, because there the wrong answer deletes a value HypAware never + wrote. Anything failing the test still refuses, including a bare + `"anthropic": null`, a user entry that happens to sit at the key, and + a hand-edited one that merely kept the header. +3. Otherwise, computes the two entries from `attachCtx.endpoint` (the + proven-bound local gateway base URL the daemon resolves in + `src/core/daemon/runtime.js`'s `resolveClientSeam` today, or the + configured-listen fallback the manual CLI path already uses for `hyp + attach`) exactly per LLP 0167#override-entries: + + ```json + { + "models": { + "providers": { + "anthropic": { + "baseUrl": "", + "headers": { "x-hypaware-upstream": "anthropic" }, + "models": [] + }, + "openai": { + "baseUrl": "/v1", + "headers": { "x-hypaware-upstream": "openai" }, + "models": [] + } + } + } + } + ``` + + The bare-origin vs `+/v1` asymmetry is load-bearing (LLP 0167 + #verify-results): OpenClaw's Anthropic client appends its own path, + its OpenAI client does not. Writing the wrong shape for either + produces a schema-valid but non-functional entry, so this exact split + is the one place in the module worth a dedicated unit test rather than + trusting the acceptance run alone. +4. Writes the merged config back (existing `models` keys the file already + had, if any that aren't `providers.anthropic`/`.openai`, are preserved; + nothing outside these two keys under `models.providers` is touched, R1). +5. Prints the `openclaw gateway restart` instruction on both the human and + `--json` output paths (R4), the same way Claude's `attach()` already + prints its own follow-on instructions when one applies. +6. Returns `{status: 'done'}`. + +``` +@ref LLP 0167#attach-detach [implements]: attach writes exactly the two +models.providers entries, refuses instead of merging when either already +exists, and prints the restart instruction; no undo record beyond the +entries themselves. +``` + +### 1.3 Refusal during join must not fail the join + +R2's join-safety clause needs no new mechanism. `src/core/config/action_backfill.js`'s +`perform()` already establishes the contract every `ActionHandler` in this +reconciler follows: a `{status: 'failed', reason}` outcome is recorded and +retried next pass, it does not throw, and it does not abort the reconciler's +other actions for the same client. The effect in `attach.js` returns exactly +that shape on refusal (step 2 above), so attach-on-join downgrades a refusal +to a warning by the pre-existing generic contract, not by anything this +design adds. The design obligation is narrower than it first looks: make sure +the effect never throws on the refuse path (it returns a status object) and +never partially writes before deciding to refuse (step 2 runs before step 4). +Both are satisfied by the ordering above. + +One translation step is load-bearing and easy to miss. The kernel types the +**registered** `attach()` as `Promise`, so the effect's returned outcome +reaches no caller: `action_attach.js`'s `perform()` and `hyp attach`'s +`runClientLifecycle` both infer success from "did it throw". The registered +wrapper in `index.js` therefore rethrows a `failed` outcome +(`if (outcome.status === 'failed') throw new Error(outcome.reason)`), which is +what actually produces the `{status: 'failed', reason}` outcome above: +`perform()`'s catch converts it, and the reconciler records, warns, and +retries it without touching the join's other actions. Swallowing the outcome +instead is not the join-safety clause but its opposite: `perform()` would +record `done`, `isCurrent()` would match on both the endpoint and the +`assets_key` forever, and the refusal would never be retried even once the +user cleared the conflicting `models.providers` entry (while the `json_path` +attach probe kept reporting `not attached`). The same swallow made +`hyp attach --client openclaw` print a refusal and exit 0, so no script could +tell a refusal from a success. Rethrowing at the wrapper, rather than teaching +`perform()` to parse the adapter's `--json` payload, is what fixes both +callers: `runClientLifecycle` hands the adapter `ctx.stdout` directly and +never captures it, so it has no payload to inspect. + +``` +@ref LLP 0169#decision [implements]: a refuse during join warns and never +fails the join, via the existing ActionOutcome 'failed' contract, not a new +one. +``` + +### 1.4 Manifest registration + +`hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json`'s +`contributes.client` gains: + +```json +"attach_probe": { + "format": "json_path", + "settings_file": ".openclaw/openclaw.json", + "container_path": "models.providers", + "provider_keys": ["anthropic", "openai"], + "marker_header": "x-hypaware-upstream", + "cache_glob": "agents/*/agent/models.json" +} +``` + +`description` and the `picker[0].summary` lose every reference to +`@hypaware/openclaw-steering-plugin` (LLP 0167#onboarding); see 5.2. + +## 2. Lane A: detach {#lane-a-detach} + +### 2.1 The `json_path` format returns to core + +LLP 0143 removed the `probe.format === 'json_path'` branch from both +`src/core/config/client_detach_disk.js` and `src/core/daemon/status.js`, +because LLP 0152 (the now-superseded steering design) left nothing on disk +for that branch to reverse or read. LLP 0168 reverses that premise; LLP 0169 +names the restoration explicitly ("core restores the `json_path` branches +removed by PR #510"). The restored branch is not a resurrection of the old +single-entry, single-`hypaware`-provider shape (that shape belonged to +LLP 0109's design, before LLP 0167 replaced one shadow provider per vendor +family with the two canonical entries of LLP 0168). It is a new +implementation shaped for two entries plus a cache purge, driven by the +manifest fields in 1.4. + +`client_detach_disk.js`'s `detachClientFromDisk` dispatcher gains: + +```js +if (probe.format === 'json_path') { + return await detachJsonPathProviders({ + settingsPath, + containerPath: probe.container_path, + providerKeys: probe.provider_keys, + markerHeader: probe.marker_header, + cacheGlob: probe.cache_glob, + homeDir, + expectedBaseUrl, + fs, + }) +} +``` + +`expectedBaseUrl` is the one new fact the dispatcher needs that it doesn't +already have: the gateway's own currently-resolved base origin, so the +routine can tell "this entry is ours" from "this entry merely looks like +ours." `detachClientFromDisk`'s caller, `detachClientViaCore` in +`src/core/commands/clients.js`, already receives a full `CommandRunContext` +(`ctx`), which carries `ctx.capabilities`. The daemon reconciler resolves +the same fact today via `boot.runtime.capabilities`'s `AiGatewayCapability` +(`clients.localEndpoint()`, described at `src/core/daemon/runtime.js` around +the `resolveClientSeam` helper); the manual CLI path already has a +configured-listen fallback for the case where no daemon is bound (the +comment at that call site notes the daemon's rule, proven-bound only, is +stricter than "that's the manual path's"). `detachClientViaCore` resolves +`expectedBaseUrl` the same way and threads it through. No new capability, +no new context field: this is the existing `AiGatewayCapability` lookup, +called from a second, already-instantiated location. + +### 2.2 `detachJsonPathProviders`: ownership, backup, purge + +New function, same file (`client_detach_disk.js`), generic over the +manifest fields (not OpenClaw-named in the implementation, though OpenClaw +remains the sole consumer exactly as LLP 0143 observed for the prior +format): + +1. Read the settings file. Absent file: `{changed: false}` (nothing to + reverse, matching every other format's absent-file behavior). +2. For each key in `providerKeys` (`anthropic`, `openai`): look up + `containerPath.key` (i.e. `models.providers.anthropic`). Absent: skip + this key, nothing to do. +3. **Present:** compare its `baseUrl` against `expectedBaseUrl` / + `expectedBaseUrl + '/v1'` (the same asymmetry attach wrote) and confirm + `headers[markerHeader]` equals the key name. Both match: this entry is + ours, delete the key from the parsed object (R3's "delete only when its + `baseUrl` is the gateway's"). +4. **Present but mismatched** (wrong `baseUrl`, missing/different marker + header, or a value shape attach wouldn't have produced, e.g. `models` + not an empty array): back it up rather than discard it, following the + `prev_malformed` precedent LLP 0163 established for Claude's malformed + `env`/`hooks` blocks. The backup lands under a sibling key in the same + file (e.g. `models.providers.anthropic` moves to a HypAware-owned + `_hypaware_detach_backup.anthropic` before the live key is removed), + not a side-channel state file, so the same `openclaw.json` a human + reads after detach shows both "this key is gone" and "here is what was + there, in case you need it back." LLP 0163 flagged this exact + json/toml-vs-json_path asymmetry (json/toml back up in-place inside the + marker; OpenClaw's prior format refused rather than backed up) as + "worth its own look." This design closes that gap in the same + direction LLP 0163 already took Claude: never discard a value HypAware + didn't write. +5. Write the modified config back if any key changed or was backed up. +6. **Cache purge (R3, independent of step 3-5's outcome):** glob + `homeDir/.openclaw/` (`agents/*/agent/models.json`), where + the `.openclaw` half is the client's *config home*, derived back out of + the already-resolved `settingsPath` rather than re-joined onto + `homeDir`, so a `$OPENCLAW_HOME` relocation cannot leave the purge + working in a different home than the settings write did. For + each matched file, best-effort parse it and delete `providerKeys` + entries if present, then write it back. A file that fails to parse is + logged and skipped, not fatal: LLP 0169 notes these caches "do not + self-heal," so a best-effort purge is strictly better than the status + quo even when one file is unreadable, and detach must not fail the + whole operation over one cache file. +7. Return `{changed: , settingsPath}`. + +``` +@ref LLP 0169#decision [implements]: detach deletes an entry only when its +baseUrl is the gateway's, backs up a present-but-not-ours or mangled entry +instead of discarding it, and deletes the written provider keys from every +agents//agent/models.json. +``` + +``` +@ref LLP 0163#open-questions [implements]: LLP 0163 left "does OpenClaw +converge?" open, arguing it did not, on the ground that OpenClaw's own +config rejects a top-level marker; this design converges the *outcome* +(backup instead of discard) without adopting the marker-key mechanism LLP +0163 correctly ruled out for OpenClaw. +``` + +Step 5 (write back after step 3/4) and step 6 (cache purge) both end with +the same restart-instruction print as attach (R4). `detachClientViaCore` +already has an output-writing seam (`writeCoreDetachOutput`); the +instruction rides that, not a new print call scattered in the core routine. + +**An unknown `expectedBaseUrl` refuses rather than defaults (added in T2).** +Steps 3 and 4 are one branch on ownership, so an absent gateway base URL +does not leave the routine with a safe default: treating every entry as +ours deletes values HypAware never wrote, and treating none as ours reports +a finished detach while the client stays routed at a port the daemon no +longer serves. There is no third answer available from disk, because this +format's undo record *is* the entry. So when at least one `providerKeys` +entry is present and no base URL was threaded in, `detachJsonPathProviders` +throws `ClientDetachError` (`code: 'EXPECTED_BASE_URL_UNKNOWN'`) instead of +picking one of the two wrong answers. Both callers already degrade +correctly: `reverse()` catches it into `{status:'failed'}` and keeps the +marker (which is the #212-safe outcome, not the orphaning one), and +`hyp detach` prints the reason and exits nonzero. An absent settings file +and a file with neither key present are unaffected: they reverse nothing, +so they need no base URL and stay `{changed:false}`. + +**The ownership predicate is shared with attach (added in review round 2).** +Step 3's test is the same question 1.2's refusal asks, from the other side, +about the same two keys in the same file, so it lives in one module both +import: `src/core/config/provider_entry_ownership.js`, exporting +`isOwnedProviderEntry(entry, key, markerHeader, ours)` and +`ownedBaseUrls(expectedBaseUrl)`. Only the base-URL half differs, and the +parameter carries that difference explicitly: detach passes the set (see the +refusal above), attach passes `undefined` because a drift re-attach meets its +own entry at the *previous* origin. Two copies would have been free to drift +into disagreeing about whether a given entry is HypAware's, which is how the +presence-only refusal survived review round 1 while detach was already +ownership-aware. + +**Where the backup key sits.** "Sibling" is meant literally: the backup +lands at `._hypaware_detach_backup.`, inside the same +container the undo already navigates, not at the file's top level. LLP 0163 +ruled a *top-level* HypAware key out for this client (its config schema +rejects one), and that ruling is the entire reason this format refused where +`json`/`toml` backed up; reintroducing the backup as a top-level key would +walk straight back into it. + +### 2.3 `daemon/status.js`'s `json_path` read branch + +The removed read branch (`probe.format === 'json_path' && probe.marker_header`) +is restored as a pure read, parallel to the existing `json`/`toml` branches +at lines 1066/1083 of the current tree: navigate `container_path` + +`provider_keys[0]` (`models.providers.anthropic`), read +`headers[marker_header]`, and report attached when it equals the expected +marker value for at least one of the two configured keys. This is read-only +and has no ownership/backup concerns; it exists purely to make +`probeClientAttachFromDescriptor` (and therefore `hyp status`'s +`client_attach` row and the `client_attach_missing` diagnostic) true again +for OpenClaw. + +``` +@ref LLP 0171#requirements [implements]: R5, the manifest registers +attach_probe in json_path format and core restores the json_path branches in +client_detach_disk.js and daemon/status.js. +``` + +## 3. The descriptor/attach-probe question (interaction 1) {#interaction-1} + +**Answer, stated explicitly per the task's requirement:** after this change +set, OpenClaw's manifest declares a real `attach_probe` (1.4, format +`json_path`), so `descriptor.attachProbe` is truthy for OpenClaw for the +first time since LLP 0143 landed. Concretely, against PR #553's +`descriptor.attachProbe`-gated `hyp status` logic: + +- **`attachable` flips back to `true`.** PR #553 made a probe-less client + read as `attach n/a`; OpenClaw is no longer probe-less, so it exits that + state and re-enters the same real `attached` / `not attached` derivation + every other `json`/`toml` client already gets, computed by + `probeClientAttachFromDescriptor` reading the 2.3 branch. +- **The `client actions: attach openclaw` row goes back to real + `pending`/`done` semantics**, not the `inert` → n/a path PR #553 gave + probe-less clients. Critically, this `pending` is not the permanently- + stuck state issue #544 was filed against: #544's bug was a probe-less + descriptor wedging in `pending` forever because there was truly nothing + on disk to converge toward. Here, `action_attach.desired()` (which + already skips probe-less descriptors, the exact condition PR #553's fix + targeted) sees a real probe, the join reconciler runs `attach()` from + 1.2, and the marker resolves to `done` on a normal run or `failed` + (retried) on a genuine refusal, same as every other client's row. +- **`client_attach_missing` fires meaningfully again**: since `hyp attach + --client openclaw` now performs a real, reversible write (1.2), the + diagnostic's repair suggestion is no longer a dead end, which was exactly + LLP 0143's "should `hyp status` grow a plugin-registry-derived attach + signal... worth its own LLP" open question. That question is answered by + this change set: no separate plugin-registry signal is needed, because + the disk-driven probe (which PR #553's fix generalized correctly) is + sufficient once OpenClaw has something reversible on disk again. + +No part of PR #553's logic needs to change or special-case OpenClaw: it +already does exactly the right thing for any descriptor with a real probe. +The risk this design has to avoid is only in the manifest and the +attach/detach implementation actually producing a working probe (sections +1.4, 2.1-2.3), not in the status/attach state-machine code itself. + +## 4. Lane B: the scheduled sweep {#lane-b-sweep} + +### 4.1 What Lane B reuses + +`hypaware-core/plugins-workspace/openclaw/src/backfill.js` already +implements `createOpenclawBackfillProvider(opts)`, a `BackfillContribution` +with `plan()`/`run()`, registered via `ctx.backfills.register(...)` in +`index.js`. `src/core/commands/backfill.js` already exports +`runBackfillProvider({ctx, provider, dryRun, retentionDays?, since?, until?, +devRunId?})`, an **in-process** (no subprocess) runner used today by "the +onboarding finale" to import a picked client's history right after config is +written. Internally it calls `runProvider()`, which resolves entrypoint +ownership, builds a `BackfillRunContext` via `buildRunContext()`, iterates +`provider.run(runCtx)`, dispatches each yielded item to +`ctx.backfillMaterializers.get(item.kind)`, writes rows, and flushes every +touched dataset. This is the exact scan-materialize-write-flush pipeline +Lane B needs to run every five minutes; nothing about it is CLI-specific +except the full `CommandRunContext` its outward-facing entrypoint currently +demands. + +### 4.2 Kernel type: an optional `sweep` field, not a new mechanism + +`BackfillContribution` (in `hypaware-plugin-kernel-types.d.ts`) gains one +optional field: + +```ts +sweep?: { cron: string } +``` + +Absent on every contribution today (Claude's, Codex's, and OpenClaw's own +prior to this change): zero behavior change for any provider that doesn't +opt in. OpenClaw's `createOpenclawBackfillProvider()` populates it from its +own validated config: + +```js +sweep: { cron: config.backfill?.sweep_cron ?? '*/5 * * * *' } +``` + +(R7: "tunable in the plugin's `backfill` config section"). This keeps the +schedule a plugin-owned fact expressed through the kernel's own contribution +shape, not a config value the daemon has to know OpenClaw's name to find, +which is the same "kernel stays plugin-agnostic" discipline +`llp/0000-hypaware.explainer.md` states as a cross-cutting invariant. + +`hypaware-core/plugins-workspace/openclaw/src/config.js`'s +`validateBackfillSection` gains a `sweep_cron` key (string, validated as a +5-field cron expression, same validator `cronMatches`'s caller already uses +to reject malformed schedules elsewhere) alongside the existing `on_join` +and `window_days` keys, with the same unknown-key rejection the section +already enforces. + +### 4.3 Narrowing `runProvider`'s context type, not widening the daemon's + +`runProvider()`, `resolveOwnersForRun()`, and the materialize/write/flush +helpers they call (all in `src/core/commands/backfill.js`) only ever read +`ctx.backfills`, `ctx.backfillMaterializers`, `ctx.env`, `ctx.storage`, +`ctx.query` (`writeRows`/`flushDataset` resolve a dataset's registered +table path through it before a row can be committed or a partition +flushed), and (via `resolveOwnersForRun`) `ctx.config` for +plugin-configured resolution. None of `CommandRunContext`'s other fields +(`stdout`, `commands`, `verbs`, `skills`, `agents`, `sources`, `sinks`, +`initPresets`, `capabilities`, `plugins`, `cwd`) are touched anywhere in +this call path. Rather than force the daemon to assemble a full, +mostly-unused `CommandRunContext` just to call `runBackfillProvider`, this +design narrows the type both functions declare their `ctx` parameter as, +to a new, smaller type: + +```ts +// A structural subset of CommandRunContext; every existing +// CommandRunContext satisfies it, so every current call site +// (hyp backfill's CLI path, the onboarding finale) keeps typechecking +// unchanged. +interface BackfillRunnerContext { + env: NodeJS.ProcessEnv + config: HypAwareV2Config + storage: QueryStorageService + query: QueryRegistry + backfills: BackfillRegistry + backfillMaterializers: BackfillMaterializerRegistry +} +``` + +`runBackfillProvider`, `runProvider`, `resolveOwnersForRun`, and the +materialize/write/flush helpers' `ctx` parameters change from +`CommandRunContext` to `BackfillRunnerContext`. This is a pure narrowing: +`CommandRunContext` is structurally a superset, so no existing caller's +argument stops satisfying the (now smaller) parameter type. The daemon can +now build a `BackfillRunnerContext` object out of fields `boot.runtime` +already carries (`env`, `config`, `storage`, `query`, `backfills`, +`backfillMaterializers`, all already referenced at +`src/core/runtime/activation.js`) without touching `CommandRunContext` or +constructing stub versions of fields it doesn't need. + +`query` was not part of this list until LLP 0173 T12's hermetic smoke (the +first caller to drive a real, non-dry-run write through the sweep +driver rather than a mocked `runBackfill` seam) found `writeRows` and +`flushDataset` crash on `ctx.query.getDataset` when the daemon-built +`BackfillRunnerContext` reached them: the field really is on this call +path, this design's original field enumeration just missed it because +T9's own tests never exercised a real write. `BackfillSweepDriverOptions` +(Section 4.4) and the daemon's `createBackfillSweepDriver(...)` call +(`src/core/daemon/runtime.js`) both require `query` now for the same +reason. + +### 4.4 Wiring the tick + +New file: `src/core/daemon/backfill_sweep.js`, exporting +`createBackfillSweepDriver({backfills, backfillMaterializers, env, config, +storage, query})` with one method, `tick({now})`: + +```js +function tick({ now }) { + for (const provider of backfills.list()) { + if (!provider.sweep) continue + if (!cronMatches(provider.sweep.cron, now)) continue + void runBackfillProvider({ + ctx: { env, config, storage, query, backfills, backfillMaterializers }, + provider: provider.name, + dryRun: false, + devRunId: `sweep-${provider.name}-${now.getTime()}`, + }) + } +} +``` + +`cronMatches` is imported from `src/core/sinks/driver.js`, the same +due-check the sink driver already uses (LLP 0170's framing: "the daemon +already runs cron-matched periodic work... so this is scheduling an +existing job, not building a new primitive"). `src/core/daemon/runtime.js`'s +`runTick()` already calls `await driver.tick({now, source: 'daemon'})` for +the sink driver inside its `withSpan('sink.tick', ...)` block on the +existing `DEFAULT_TICK_INTERVAL_MS = 60_000` interval; this design adds one +sibling call, `await sweepDriver.tick({now})`, in the same `runTick()`, +right after the sink tick. It rides the existing 60-second loop rather than +opening a second `setInterval` (as the cache-maintenance `maintenanceHandle` +does): a `*/5 * * * *` schedule only ever needs a due-check once a minute, +and reusing the loop means one fewer timer to start, drain, and account for +at shutdown. `runProvider`'s internal work (scan, materialize, write, +flush) is `await`-ed inside the tick but the sweep call itself is fired +without blocking the sink tick behind it (`void runBackfillProvider(...)` +matches the "must never wedge the daemon tick loop" discipline +`action_backfill.js` already documents for the subprocess case; here there +is no subprocess, but a slow provider `run()` still shouldn't stall +`refreshSourceDetails()`/`persist()` later in the same `runTick()`). + +Not blocking has a consequence the sketch above leaves out: nothing stops a +provider being due again while its previous run is still going. A pass over a +large transcript tree can outlive the default `*/5 * * * *` interval, and +neither `runBackfillProvider` nor `runProvider` carries a lock, so a second +concurrent run would land on the same datasets and the same mid-flush spool. +The driver therefore keeps a `Set` of in-flight provider names in its closure: +a due provider already in the set is **skipped, not queued** (the sweep is +level-triggered, so the next tick that finds it due and idle picks up whatever +this one would have), logged as `backfill.sweep_skipped` with +`error_kind: 'already_running'` under the same `component`/`operation` pair as +every other sweep record, and the entry is cleared in both settlement handlers. +This is the `maintenanceInFlight` guard `src/core/daemon/runtime.js` already +applies to the sibling periodic job, widened to a set because this driver +fires one run per provider rather than one job. + +``` +@ref LLP 0170#decision [implements]: the daemon runs the OpenClaw backfill +provider on a cron-matched schedule by extending the existing sink-tick +cadence, not building a new scheduling primitive. +``` + +### 4.5 The quiesce window + +Entirely internal to `backfill.js`, not threaded through +`BackfillRunnerContext` or any kernel type: it is a filter on which session +files a run considers, not a fact the runner or materializer registry needs +to know about. `listSessionFiles(agentsDir)` (currently: enumerate +`agents/*/sessions/*.jsonl` with no time filtering) gains an optional +`quiesceBeforeMs` parameter; `runOpenclawBackfill()` computes it once per +run as `Date.now() - quiesceMs` and skips any file whose `mtimeMs` is more +recent. `quiesceMs` resolves from `config.backfill?.quiesce_ms`, defaulting +to 180,000 (three minutes): the settlement flush interval +(`QUERY_FLUSH_DEBOUNCE_MS = 2 * 60 * 1000` in `src/core/cache/spool.js`) +plus a one-minute margin, so a sweep never races a session file OpenClaw is +still mid-write on, or a settlement pass still mid-flush against the same +turn (LLP 0170: "quiesce window = settlement flush interval + margin"). +This default is a real, cited constant, not an invented number; the +`quiesce_ms` config key exists precisely so an operator with a slower disk +or a longer flush debounce can widen it. + +``` +@ref LLP 0170#decision [implements]: the sweep skips session files whose +mtime is inside the quiesce window, sized from the existing settlement flush +debounce plus margin, not a new invented constant. +``` + +R8 ("the sweep MUST NOT ship before the issue #543 envelope fix is merged") +is a sequencing constraint on this change set's landing order, not a design +decision: this design is written against the fixed reader (PR #552's +`openclawMessageEnvelope`, reading `role`/`content`/`provider`/`usage` under +a nested `message` key), and the sweep driver in 4.4 has no code path that +degrades gracefully if the old flat reader is still in place, it would +simply project nothing (R8's stated failure mode). The Impl-designer rung +that turns this into tasks needs to sequence the merge of #552 ahead of (or +in the same PR series as) this sweep wiring; this design does not need a +runtime guard for an already-fixed dependency. + +## 5. Deletion inventory {#deletion} + +Per LLP 0167#deletion-inventory and R9, deleted in the same change set: + +- **`openclaw-steering-plugin/`** in full: `src/gateway_endpoint.js`, + `src/index.js`, `src/runtime_auth.js`, `src/steering.js`, + `src/warning_ledger.js`, `src/wire_parity.js`, its `package.json`, + `openclaw.plugin.json`, `.d.ts` files, and its `test/` directory (five + files). This is the credential-borrowing runtime auth shim + (`runtime_auth.js`), the live wire-parity mirror (`wire_parity.js`), the + steering decision logic (`steering.js`), the live warning ledger + (`warning_ledger.js`), and the gateway endpoint resolver + (`gateway_endpoint.js`) that only existed to feed them: none of it has a + purpose once Lane A's config-override entries make OpenClaw route to the + gateway on its own, with no in-process steering to perform. Roughly 1,900 + lines of source plus tests, matching LLP 0167's "~2,100 lines" estimate + once the manifest/package.json/`.d.ts` scaffolding is counted. +- **`test/plugins/openclaw-steering-plugin.test.js`** (R9), the root-test + suite's coverage of the deleted package. +- Inside `hypaware-core/plugins-workspace/openclaw/`: the honest no-op + `attach()` and its `STEERING_PLUGIN_NAME`/ + `ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE` constants (1.1), and every + manifest/doc string naming `@hypaware/openclaw-steering-plugin` (1.4, 5.2). + +**What must survive**, named explicitly because it is easy to mistake for +steering-plugin-only code: the exchange projector +(`createOpenclawExchangeProjector`, `anthropicUpstreamPreset`/ +`openaiUpstreamPreset` in `projector.js`), the settlement enricher +(`createOpenclawSettlementEnricher` in `settle.js`), the match-key module +(`match_key.js`), the session-file reader (`session_file.js`, PR #552's +fixed version), and `backfill.js` in its entirety apart from the 4.5 +addition. None of these read from or write to the steering plugin; they +read gateway-captured rows and local session files, both of which exist +independent of how routing gets set up. R10 states this in the requirements +language ("the gateway, the exchange projector, settlement, match key, +reader, and backfill projection MUST be unchanged"); this section is the +concrete list an implementer checks the deletion against. + +``` +@ref LLP 0167#deletion-inventory [implements]: the steering plugin package +and its test suite are deleted whole; the projector, settlement, match key, +reader, and backfill projection are unchanged survivors, not casualties. +``` + +## 6. Dedupe: R11 nets Lane A/B overlap to zero {#dedupe} + +`backfill.js`'s `projectedMessageFromRecord()` already builds each row's +identity from the session file's own native `message.id` directly, with no +match-key indirection (match-key normalization exists for the live-capture +settlement path, not for backfill, because backfill always reads the +session file itself and so is never in fallback/content-hash identity to +begin with). A turn Lane A captured live and settled onto native identity +(LLP 0027/0159's settlement upgrade) and the same turn Lane B later sweeps +out of the session file resolve to the **same** `part_id` by construction: +both derive it from the identical native `message.id`. The existing +dataset-write dedupe on `part_id` (already relied on for re-running `hyp +backfill` idempotently, and for Lane A's own settlement-driven upgrade not +duplicating the fallback-identity row it replaces) nets this overlap to +zero new rows with no new dedupe code. This is why R11 reads as a +consequence of 4.1's reuse, not a separate mechanism to build: the moment +Lane B reuses the exact backfill pipeline that already writes +native-identity rows idempotently, the "sweep over already-captured turns +nets zero writes" requirement (R7's second sentence) and R11 (identity- +identical routes dedupe to zero) are the same fact observed from two +requirements. + +``` +@ref LLP 0171#requirements [implements]: R11, identity-identical routes from +lane A and lane B dedupe to zero via the existing part_id write-dedupe, since +both lanes resolve identity from the same native message.id. +``` + +## 7. Carried-over requirements from LLP 0157 {#carried-over} + +Per LLP 0171#carried-over, R8/R9/R10/R11/R14 from the prior spec remain +binding, unchanged. Where each is satisfied in the current tree, none of it +touched by this change set: + +- **R8** (projector shapes behind the header gate): `projector.js`'s + `anthropicUpstreamPreset`/`openaiUpstreamPreset` and the exchange + projector still gate on `x-hypaware-upstream`, now sourced from the + config-override entries' static `headers` value (LLP 0168) rather than a + steering-plugin-injected header. The gate itself, and the header name, are + unchanged; only who writes the header changes, which is exactly R10's + "unchanged by this change set" for the projector, read together with R8. +- **R9** (the one LLP 0158 reader): `session_file.js`'s + `openclawMessageEnvelope` (PR #552's fix) stays the sole reader for both + the settlement path and Lane B's sweep; nothing in this design adds a + second reader. +- **R10** (backfill policy gate and CLI-backend exclusion): + `backfill.js`'s `PROJECTABLE_PROVIDERS = new Set(['anthropic', 'openai'])` + and `effectiveProviders()`'s forward/backward fill are untouched; Lane B + reuses `runOpenclawBackfill()` as-is apart from the 4.5 quiesce filter, + which composes with, not around, the existing CLI-backend exclusion. +- **R11**: satisfied per section 6 above, for both requirements sets (LLP + 0157's original R11 and LLP 0171's R11, the same requirement carried + forward, not two separate obligations). +- **R14** (settlement resolves cwd and applies the policy drop): + `settle.js`'s `createOpenclawSettlementEnricher` is untouched by this + design; it still resolves the session's cwd and runs it through the usage + policy resolver (`createUsagePolicyResolver`, `localOnlyListPath`) exactly + as before Lane A/B existed in their current shape. + +## 8. Acceptance and onboarding rewrites {#acceptance-onboarding} + +### 8.1 `docs/ACCEPTANCE.md`'s `openclaw_capture` (R11 of LLP 0171) + +The current procedure (lines 173 onward) requires linking and enabling the +steering plugin from the checkout under test +(`openclaw plugins install --link ./openclaw-steering-plugin --force`), and +its "what it proves" language names "live proxy capture through the +steering plugin's shadow providers." Both go. The rewrite: + +- **Setup** drops the steering-plugin link/enable steps entirely; adds + `hyp attach --client openclaw` followed by the `openclaw gateway restart` + instruction the command itself prints (1.2 step 5), replacing the manual + `openclaw.json` edit the old procedure walked through by hand. +- **A sweep step**: run a turn on a provider whose live capture window has + already closed (or with the daemon's Lane A capture briefly disabled), + confirm the row is absent immediately after, then confirm it lands within + one sweep interval (default five minutes) once the quiesce window (4.5) + has passed. This is the step the old procedure had no equivalent for, + because the old design had no separate sweep, only steering-or-not. +- **A zero-duplicate assertion**: run a turn where both lanes will observe + it (live capture succeeds AND the session file records it), wait past one + sweep interval, and assert exactly one row for that turn's `part_id`, + proving section 6's dedupe claim on a real binary rather than only in + code review. +- **Re-confirmation of LLP 0167#verify-results items 1, 3, and 4** (the + `models.providers` shape, the no-self-heal-on-detach cache behavior, and + the restart-required behavior) on an OpenClaw binary at or above the + 2026.4.24 floor the old procedure already required, since those verified + facts were established against 2026.3.13 and R11 asks for re-confirmation + at the floor version the acceptance run actually gates on. +- Drops the version-gate language specific to `before_model_resolve` and + `hooks.allowConversationAccess` (2026.4.21/2026.4.23 features the steering + plugin depended on): Lane A depends on no OpenClaw hook API at all, only + on `models.providers` being a schema-valid config key, which LLP + 0167#verify-results confirms is stable back to 2026.3.13. +- Per R11, a human must still run this before the adapter ships; nothing in + this design substitutes an automated check for that gate. + +``` +@ref LLP 0171#requirements [implements]: R11 (formerly R12 of LLP 0157, +"replaced" per 0171's carried-over note), the acceptance rewrite: attach-flow +steps, a sweep step, a zero-duplicate assertion, and re-confirmation of the +verified facts on the floor version. +``` + +### 8.2 Picker copy (R12 of LLP 0171, LLP 0167#onboarding) + +`hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json`'s +`picker[0].summary` drops "Routing is set up on the OpenClaw side by +installing the @hypaware/openclaw-steering-plugin package" and states the +two capture tiers directly: live capture through the local gateway (once +attached) plus periodic transcript sweep, no separate package to install. +Claude's own picker entry (`hypaware-core/plugins-workspace/claude/hypaware.plugin.json`) +gains the LLP 0167#onboarding line naming the `claude-cli/` case +OpenClaw's own CLI-backend exclusion (R10, LLP 0147) produces explicitly, so +a user who runs Claude Code through OpenClaw understands which picker entry +their turns actually belong to. + +## 9. Hermetic smoke gap {#smoke-gap} + +No hermetic smoke currently writes an OpenClaw session file in any shape: +there is no `backfill_openclaw_fixture` analog to the Codex/Claude flows +under `hypaware-core/smoke/flows`. This is a real, pre-existing gap, flagged +during review of PR #552, and Lane B makes it more consequential: the sweep +path (4.4/4.5) has no hermetic-smoke coverage today, so a regression in +`listSessionFiles`'s new `quiesceBeforeMs` filter, or in the sweep driver's +`cronMatches` wiring, would only surface in the manual acceptance run (8.1), +not in PR-level smoke confidence. This design does not build that fixture +(out of scope for a design document), but names the gap for the +Impl-designer rung: a `backfill_openclaw_fixture` helper, writing a +minimal OpenClaw v3 session JSONL (nested `message` envelope, matching PR +#552's fixed reader) under a temp `agents//sessions/` tree with a +controllable mtime, would let a hermetic smoke exercise the quiesce filter +and the sweep-then-dedupe path deterministically, the same tier distinction +`/work/hypaware/CLAUDE.md`'s Smoke Test Model section draws between +hermetic smokes (PR confidence) and the acceptance smoke (release gate, +8.1). Whether to build it in this change set or a follow-on is a scoping +call for the plan, not this design; the design only establishes that Lane +B's correctness currently rests entirely on 8.1's human-run procedure. + +## 10. Open questions left for a human {#open-questions} + +None of the decisions in this design required inventing an answer where the +cited RFC/decisions were silent; every fork identified during research +(the attach-probe/status interaction, the scheduling seam, the ownership- +check base URL source, the backup-vs-refuse asymmetry) resolved to an +existing mechanism or a directly-cited decision. Two items are worth a +human's attention regardless, both already flagged in the requirements +rather than newly discovered here: + +- **R11's acceptance rewrite (section 8.1) requires a human run before the + adapter ships.** This design specifies what that run must cover; it does + not and cannot perform the run itself. +- **The hermetic smoke gap (section 9)** is a real coverage hole this + design chooses not to close, on the grounds that building a new smoke + fixture is implementation work for a later rung, not a design decision. + If a human reviewing this design set disagrees with deferring it, that is + the one scoping judgment call in this document worth reconsidering before + planning starts. + +## References + +- LLP 0167, LLP 0171, LLP 0168, LLP 0169, LLP 0170 +- LLP 0157 (carried-over R8/R9/R10/R11/R14), LLP 0158, LLP 0159, LLP 0161 + (prior technical design; steering-plugin sections retired here, projector/ + settlement/backfill sections remain the record of what shipped) +- LLP 0163 (malformed-block backup precedent), LLP 0143 (superseded; + json_path retirement, reversed here), LLP 0144 (shadow-provider-per-shape + rationale, carried over as Lane A's rationale) +- LLP 0044, LLP 0045 (attach/detach design) +- `docs/ACCEPTANCE.md`, issue #543 (PR #552), issue #544 (PR #553) diff --git a/llp/0173-openclaw-two-lane-capture.plan.md b/llp/0173-openclaw-two-lane-capture.plan.md new file mode 100644 index 00000000..0d318d5f --- /dev/null +++ b/llp/0173-openclaw-two-lane-capture.plan.md @@ -0,0 +1,404 @@ +# LLP 0173: OpenClaw two-lane capture, implementation plan + +**Type:** plan +**Status:** Active +**Related:** LLP 0172 +**Generated-by:** neutral + +> [LLP 0172](./0172-openclaw-two-lane-capture.design.md) is the technical +> design for the one deliverable set LLP 0171 specifies: the reworked +> `@hypaware/openclaw` attach/detach module (Lane A), the daemon-side +> scheduled sweep (Lane B), the `json_path` core revival, the +> `openclaw-steering-plugin/` deletion, and the acceptance/onboarding +> rewrites. It already names the files, functions, and call shapes and +> resolves every fork LLP 0171 left open. This plan turns those ten sections +> into a thirteen-task graph with real code-dependency edges, states the two +> external blockers (PR #552, PR #553) that no task's `deps` field may +> absorb, and decides the hermetic-smoke scoping question the design left for +> this rung. + +## How this refines the design + +The design's sections map close to 1:1 onto tasks, with three departures +found while re-verifying the design against the actual tree (not assumed +from the design's prose alone): + +- **The deletion inventory (design Section 5) is incomplete against the real + tree.** `tsconfig.json`'s `include` array still lists + `"openclaw-steering-plugin"` at line 19; deleting the package without + dropping this line leaves a dead include path. + `test/plugins/openclaw-client-registration.test.js` is not named in Section + 5 or R9, but it directly asserts the old no-op `attach()`'s + `routing_owned_by`/`openclaw-steering-plugin` output (lines 39-114) and the + premise that `descriptor.attachProbe` is `undefined` (lines 197-240): both + assertions go false once Lane A and the manifest land, so this file needs + a rewrite, not silent bit-rot, and not a delete (three of its six tests + cover real, still-true resolution/adjacency behavior). Both are their own + task (T10, T11) rather than folded into the design's named deletion, because + neither is optional cleanup once discovered. +- **`hypaware-core/plugins-workspace/openclaw/src/projector.js` line 31** + carries a stale comment ("Written by the `openclaw-steering-plugin`...") + that Section 5's survivor list does not flag, because the design correctly + treats the projector's *behavior* as unchanged; the *comment* describing + who writes the header it gates on is now factually wrong once Lane A owns + that write. Folded into T5 (manifest/copy task) rather than its own task, + since it is a one-line prose fix riding the same "stop naming the steering + plugin" sweep. +- **Sections 1 and 2 (attach, detach) are one design narrative but two + independently shippable units.** `attach.js` (Section 1.2) has no runtime + dependency on `detachJsonPathProviders` (Section 2.2); they only share a + manifest field (Section 1.4) that gates both. Splitting them (T4, T2) lets + either land first and still leaves the tree buildable, at the cost of the + manifest task (T5) needing both as prerequisites. + +Everything else maps directly onto the design's section numbers. + +## The task graph + +**First wave (deps `[]`), three-wide:** + +- **T1**, kernel types (Sections 1.4, 2.1, 4.2): `PluginAttachProbeManifest` + regains `'json_path'` in its format union plus `container_path`, + `provider_keys`, `cache_glob` (reusing the existing `marker_header`), and + `BackfillContribution` gains the optional `sweep?: { cron: string }` field. + Revises, rather than deletes outright, the existing comment at the + `json_path` removal site (`hypaware-plugin-kernel-types.d.ts` around line + 183) that warns re-adding the format without runtime support is dangerous: + that warning is now satisfied by T2/T3, so the comment must say what + changed and point at this plan, not simply vanish. +- **T4**, the new attach module (Section 1.1, 1.2, 1.3): `attach.js` plus + `index.js`'s old no-op removal. Has no code dependency on T1 because + `attach.js` never reads `PluginAttachProbeManifest`; it only writes the + `models.providers` shape the manifest (T5) later declares a probe against. +- **T6**, config validation (Section 4.2, 4.5 second half): `config.js`'s + `validateBackfillSection` gains `sweep_cron` and `quiesce_ms` together, in + one task, because the section's unknown-key rejection loop would otherwise + reject whichever key's task landed second as unrecognized for the window + between the two merges. + +**Second wave:** + +- **T2** (deps `[T1]`), detach core (Section 2.1, 2.2) plus threading + `expectedBaseUrl` through both real callers: `detachClientViaCore` in + `src/core/commands/clients.js` (currently calls `detachClientFromDisk` + with no `expectedBaseUrl`) and `action_attach.js`'s `reverse()` (currently + calls `detach({descriptor, env: ctx.env})`; `ctx.endpoint` is already + present in the same `ActionContext` `perform()` uses). Bundled into one + task because a `detachJsonPathProviders` that no caller threads + `expectedBaseUrl` into is untestable end-to-end and would ship as dead + code for one merge. +- **T3** (deps `[T1]`), the `daemon/status.js` read branch (Section 2.3): + pure read, structurally parallel to the existing `json`/`toml` branches at + the current lines 1066/1083. +- **T7** (deps `[T1, T6]`), Lane B's kernel-facing metadata (Section 4.2 + second half, 4.3): `createOpenclawBackfillProvider`'s `sweep` field reading + `config.backfill?.sweep_cron`, and narrowing `runBackfillProvider`, + `runProvider`, `resolveOwnersForRun` (all in `src/core/commands/backfill.js`) + from `CommandRunContext` to the new, smaller `BackfillRunnerContext` + interface. Depends on T6 because reading `config.backfill.sweep_cron` in + production is only meaningful once the validator accepts the key instead + of rejecting a user's config that sets it. +- **T8** (deps `[T6]`), the quiesce window (Section 4.5, first half): + `listSessionFiles(agentsDir)` gains `quiesceBeforeMs`; `runOpenclawBackfill()` + computes `quiesceMs` from `config.backfill?.quiesce_ms` (default 180000, + cited from `QUERY_FLUSH_DEBOUNCE_MS` in `src/core/cache/spool.js` plus a + one-minute margin) and skips files whose `mtimeMs` is more recent. + Independent of T7: this filter operates on file mtimes, not on the + contribution's scheduling metadata. + +**Third wave:** + +- **T5** (deps `[T2, T3, T4]`), the manifest and copy (Section 1.4, 8.2): + `hypaware.plugin.json` gains the `attach_probe` block, `description` and + `picker[0].summary` lose every `@hypaware/openclaw-steering-plugin` + reference, Claude's own picker entry gains the CLI-backend-routing line + Section 8.2 asks for, and `projector.js`'s stale comment (found above) is + corrected. Depends on all three because declaring a `json_path` probe + before core can read it (T3) or reverse it (T2), or before `attach()` + produces the shape the probe describes (T4), would make `hyp status` / + `hyp detach openclaw` probe a format-shape nothing yet honors correctly. +- **T9** (deps `[T7, T8]`), the daemon sweep driver (Section 4.4): new + `src/core/daemon/backfill_sweep.js`, `createBackfillSweepDriver({backfills, + backfillMaterializers, env, config, storage})`'s `tick({now})` iterating + `backfills.list()`, skipping contributions with no `sweep` field or a + not-yet-due `cronMatches` (imported from `src/core/sinks/driver.js`), and + firing `runBackfillProvider(...)` unblocked (`void`, matching + `action_backfill.js`'s "never wedge the tick loop" discipline) rather than + gating `runTick()` on it. Wired into `runtime.js`'s `runTick()` right after + the existing `await driver.tick({now, source: 'daemon'})` call for the sink + driver, riding the existing `DEFAULT_TICK_INTERVAL_MS = 60_000` loop rather + than opening a second timer. **Flagged in Rating/External blockers below**: + this is the task R8 warns about. + +**Fourth wave:** + +- **T10** (deps `[T5]`), rewrite the stale assertions in + `test/plugins/openclaw-client-registration.test.js`: the two tests + asserting `attach()`'s old no-op output matches `/openclaw-steering-plugin/` + (lines 39-82, 84-114 in the current tree) must assert the new write + (refusal-when-exists, the two-entry shape, the restart-instruction print) + instead; the descriptor test asserting `descriptor?.attachProbe === + undefined` (lines 222-240) must assert the new `json_path` shape instead; + the "honest no-op" detach test (lines 197-220) needs its R7-citing comment + corrected (the no-op it observes on a fresh temp home is now the + absent-settings-file guard, not a no-probe guard) and gains a companion + case with a real `openclaw.json` fixture proving the ownership-based + detach (T2) actually fires. The registration-order test (116-155) and the + generic `hyp attach` resolution test (157-195) are unaffected; leave them. +- **T11** (deps `[T5]`), the steering-plugin deletion (Section 5, R9): + `openclaw-steering-plugin/` in full (source, tests, manifest, + `package.json`), `test/plugins/openclaw-steering-plugin.test.js`, and + `tsconfig.json`'s stray `"openclaw-steering-plugin"` include entry (line + 19, not named in the design, found in this plan's own verification pass). + Deliberately independent of T10 in the dependency graph: neither file + T11 deletes is imported by `openclaw-client-registration.test.js` (it only + contains string-regex assertions about steering-plugin *names*, not + imports), so the two tasks do not block each other, but both must land + after T5 per the "deletion strictly after its replacement" rule. +- **T12** (deps `[T8, T9]`), the hermetic smoke gap (Section 9): a new + `backfill_openclaw_fixture` helper (mirroring `backfill_claude_fixture.js` + / `backfill_codex_fixture.js` under `hypaware-core/smoke/flows`) writing a + minimal OpenClaw v3 session JSONL in the nested-`message`-envelope shape, + under a controllable-mtime `agents//sessions/` tree, plus a smoke flow + exercising the quiesce skip (a file mtime inside the window is absent from + the run) and a sweep-then-rerun dedupe assertion (identical `part_id` + nets to zero new rows on a second sweep). **Externally blocked; see + below.** +- **T13** (deps `[T5, T8, T9]`), `docs/ACCEPTANCE.md`'s `openclaw_capture` + rewrite (Section 8.1): drops the steering-plugin link/enable setup and the + `before_model_resolve`/`hooks.allowConversationAccess` version-gate + language; adds the `hyp attach --client openclaw` setup step, a sweep step + (disable or wait out live capture, confirm the row is absent, confirm it + lands within one sweep interval past the quiesce window), a zero-duplicate + assertion (a turn both lanes observe resolves to exactly one row), and + re-confirms LLP 0167#verify-results items 1/3/4 on the floor OpenClaw + version. **Externally blocked for the sweep/dedupe steps; see below.** + +## Rating complexity: the hard parts, by name + +No task in this plan earns a 5. The design (LLP 0172) resolved every real +fork itself (the attach-probe/status interaction, the scheduling seam, the +ownership-check base URL source, the backup-vs-refuse asymmetry); what is +left is well-specified engineering against precedent, some of it exacting, +none of it open judgement calls the way LLP 0162's `resolveSteering` or +`match_key.js` were. + +Three tasks earn a 4, each because correctness failure here is silent, not a +crash: + +- **T2 (detach ownership/backup/purge): 4.** The ownership check (compare + `baseUrl` against `expectedBaseUrl` and `expectedBaseUrl + '/v1'`, confirm + the marker header) has to get both the bare-origin/`+/v1` asymmetry and + the "present but mismatched -> backup, never discard" precedent (LLP 0163) + exactly right; a wrong branch here silently deletes a value HypAware never + wrote, or silently fails to detect the gateway's own entry. The + best-effort cache purge across `agents/*/agent/models.json` adds a second + place a partial failure must not become a fatal one. +- **T4 (attach.js): 4.** The refusal-vs-write decision (refuse if either + `models.providers` key already exists, R2) must run entirely before any + write, and the two-entry shape's bare-origin-vs-`+v1` split is the one + place the design itself flags as "worth a dedicated unit test rather than + trusting the acceptance run alone": writing the wrong shape for either + entry produces a schema-valid but non-functional config, which is a + failure mode no test framework catches by accident. +- **T9 (daemon sweep driver): 4.** Wiring into the daemon's hot tick loop + without blocking it (`void runBackfillProvider(...)`, matching + `action_backfill.js`'s subprocess-era discipline applied to an in-process + call) while still emitting the structured telemetry CLAUDE.md's Log-Driven + Development section requires (component/operation/status attributes + around a new lifecycle transition) takes real care; a mistake here is a + wedged daemon or a live source of unbounded async work with no visibility, + not a unit-test failure. + +Four tasks earn a 3, applying a well-precedented shape but needing real +reasoning about interaction with existing code: + +- **T7: 3.** `BackfillRunnerContext` is a pure structural narrowing (every + `CommandRunContext` still satisfies it), but it touches three functions + across two call sites and must not regress `hyp backfill`'s existing CLI + path or the onboarding finale's call, both of which keep using the wider + type today. +- **T8: 3.** The quiesce filter is a straightforward mtime comparison, but + it must compose with the existing `effectiveProviders`/`partitionByBackend` + forward/backward-fill logic (R10, untouched) without accidentally + filtering by provider identity instead of file recency, and the + 180000ms default must resolve from the cited constant, not a re-guessed + number. +- **T12: 3.** Mirrors two existing fixture precedents closely, but building + a controllable-mtime tree and a dedupe-proving rerun assertion is new work + in this plugin, not a copy-paste. +- **T13: 3.** Writing acceptance steps a human can actually run (exact + timing against the quiesce window and the sweep interval, exact CLI + invocations matching what T2/T4 actually implement) is more than prose + transcription; a wrong step is discovered only when a human tries it, + which is the failure this document exists to prevent. + +Everything else (T1, T3, T5, T6, T10, T11) is mechanical: type edits, a read +branch mirroring existing branches, manifest/config edits against an exact +spec, and a deletion, each already fully specified by the design with no +fork left for the implementer to resolve. + +## External blockers (not expressible as `deps`) + +Two PRs are green, unmerged, and held for a human's manual merge. Verified +directly against git history: `44c7080` (fix/issue-543, PR #552) and +`daca753` (fix/issue-544, PR #553) exist as real commits reachable via +`git log --all` but are not ancestors of `origin/master` or this integration +branch's HEAD (`eda2598`); `session_file.js`'s `parseOpenclawSessionMessage` +in the current tree still reads fields flat off the raw record, confirming +PR #552 is genuinely unmerged here. Neither blocker is encoded in any task's +`deps`, per this plan's own rule that `deps` are intra-plan code edges only: + +- **PR #552 (fix/issue-543)** fixes the LLP 0158 session-file reader to + project OpenClaw v3's nested `message` envelope via `openclawMessageEnvelope`. + R8 states the sweep "MUST NOT ship before" this merges, because the sweep + has no code path that degrades gracefully against the old flat reader; it + would simply project nothing. + - **T9** (sweep driver): the scheduling code itself does not read session + content and can be implemented and unit-tested now against a mocked + `backfills`/provider. Its real-world effect (turns actually landing from + a sweep) is silently zero until #552 merges. Do not treat T9's tests + passing as proof the sweep captures anything on a real OpenClaw v3 + transcript. + - **T12** (hermetic smoke fixture): genuinely cannot be built correctly + against the current tree. The fixture must write the nested-`message` + envelope shape "matching PR #552's fixed reader" (design Section 9, + verbatim); writing it against the still-unmerged old flat reader would + test behavior the reader doesn't have yet, or worse, pass against the + wrong reader and need to be rewritten once #552 lands. **This task + should be held, not dispatched, until #552 merges into this integration + branch.** + - **T13** (ACCEPTANCE.md rewrite): the doc text itself can be written now, + but the sweep step and the zero-duplicate assertion cannot be + successfully run by a human against a real OpenClaw v3 session until + #552 merges. Flag this in the doc's own "Requires" line so a human + running the procedure early gets a clear reason for the failure, not a + confusing false negative. +- **PR #553 (fix/issue-544)** made `hyp status` treat a probe-less client as + `attach n/a` rather than permanently `pending`. Per design Section 3, this + design requires **no code change** to #553's own logic: once T5 lands a + real `attach_probe`, `descriptor.attachProbe` is truthy for OpenClaw for + the first time since LLP 0143, and #553's existing probe-truthy branch + already produces the correct `attached`/`not attached` derivation. The + only place this plan names #553 as a blocker is **T13**: the acceptance + procedure's re-confirmation of `hyp status`'s `client_attach` row assumes + #553's fix is present in whatever binary the human runs the procedure + against. If #553 is unmerged at acceptance time, the status row's + behavior reverts to whatever pre-#553 `hyp status` did for a + now-probed client, which this design was not written to describe. + +No other task in this plan carries an external blocker. + +## The hermetic-smoke decision + +The design (Section 9) explicitly leaves this as "the one scoping judgment +call in this document worth reconsidering." This plan's decision: **include +it, as T12, rather than defer it whole to issue #555**, for three reasons +that hold even given the PR #552 blocker above: + +1. It is small and fully precedented: `backfill_claude_fixture.js` and + `backfill_codex_fixture.js` already establish the fixture-writer shape + this plugin has never had an analog of. +2. It is the only automated coverage, of any tier, for the quiesce filter + (T8) and the sweep driver's `cronMatches` wiring (T9) before the human + acceptance run (T13). CLAUDE.md's Smoke Test Model section states + hermetic smokes exist precisely for "PR confidence" on "plugin/kernel + wiring checks" like this, as distinct from the acceptance tier's release + gate. +3. Deferring it whole to #555 would mean Lane B ships with zero PR-level + regression coverage between now and whenever #555 is separately + prioritized; a regression in `listSessionFiles`'s new parameter or the + sweep's due-check would only surface in a manual acceptance run. + +The blocker on PR #552 does not argue against including the task in this +plan; it argues for **naming T12 in this plan and holding it externally**, +exactly the mechanism the task instructions describe ("isolate it in the +plan and name it in the return value rather than inventing an answer"). +Building the fixture now against the wrong reader shape would be worse than +deferring it, so T12's own brief states the hold explicitly. If a human +reviewing this plan judges #555 the better home for this work regardless +(for example, if #555 is already staffed and this plan's Lane B work should +ship without waiting on it), that is a legitimate reversal of this decision, +not a mistake this plan is making silently. + +## Carrying the design's open items forward + +Design Section 10 names two items for a human, both carried forward +unchanged rather than resolved here, because resolving them is not this +rung's job: + +- **R11's acceptance rewrite (T13) requires a human run before the adapter + ships.** This plan schedules the document; a human still has to run it, + and per the external-blockers section above, a full successful run needs + PR #552 (and, for the status row, PR #553) merged first. +- **The hermetic smoke gap** is the scoping call the previous section + resolves for this plan (include, as T12, held pending #552); the design's + framing of this as worth a human's reconsideration stands, since this + plan's Impl-designer rung is not the human the design asked to weigh in. + +## Notes for implementers + +- No task here flips any LLP's `Status`: LLP 0172 is already `Active`, and + this plan introduces no design needing a shipped-marker flip. +- `@ref` annotations land with the code that realizes them: T2/T3 cite + LLP 0169#decision and LLP 0163#open-questions; T4 cites LLP 0167#attach-detach + and LLP 0169#decision; T7/T8/T9 cite LLP 0170#decision and LLP 0171#requirements + (R7, R8); T6's the two new config keys land together per the note in "How + this refines the design." Run `/ref-check` on touched files before each + task's PR. +- T5 and T11 both touch `hypaware.plugin.json` in non-overlapping ways (T5 + adds `attach_probe` and rewrites copy; T11's deletion never touches this + file). No merge-order concern between them beyond the dependency already + stated (T11 after T5). +- T2 and T4 both eventually feed T5's manifest edit but touch no common file + with each other (`client_detach_disk.js` vs. new `attach.js`); they can + proceed fully in parallel in the second wave despite both gating T5. +- T9's `void runBackfillProvider(...)` fire-and-forget call means a slow or + failing sweep run must not throw unhandled into the daemon's event loop; + wrap it so a rejected promise is logged (`component: 'backfill-sweep'`, + `operation: 'backfill.sweep'`, `error_kind`) rather than becoming an + unhandled rejection, per CLAUDE.md's Log-Driven Development section. + `component` names the emitting module, not the plugin: the driver is + plugin-agnostic (it fires any contribution carrying a `sweep` field, and + OpenClaw is only the first opt-in), and per-plugin attribution already + rides `hyp_plugin`/`provider` on the same records. +- T12, once unblocked, should confirm against issue #555's own tracked scope + before starting, in case #555 has since grown requirements beyond what + design Section 9 describes. + +## References + +- [LLP 0172](./0172-openclaw-two-lane-capture.design.md): the technical + design this plan schedules +- [LLP 0171](./0171-openclaw-two-lane-capture.spec.md): the requirements + (R1-R12, plus LLP 0157's carried-over R8/R9/R10/R11/R14) this design and + plan implement +- LLP 0167, LLP 0168, LLP 0169, LLP 0170: the accepted RFC/decision set LLP + 0172's sections cite one-by-one +- LLP 0163 (malformed-block backup precedent), LLP 0143 (superseded; + `json_path` retirement, reversed by T2/T3), LLP 0157/0158/0159/0161/0162 + (the prior design/plan; steering-plugin-shaped sections retired by T11, + projector/settlement/backfill sections remain the record of what shipped + and are the survivor list T5/T11 check deletions against) +- `docs/ACCEPTANCE.md`, issue #543 (PR #552, `fix/issue-543`), issue #544 + (PR #553, `fix/issue-544`), issue #555 (the separately-tracked hermetic + smoke gap this plan chooses not to defer to, per the hermetic-smoke + decision above) +- `llp/0162-openclaw-full-capture.plan.md`: format precedent for this plan's + task-graph and complexity-rating structure + +## Tasks + +- id: T1 branch: task/openclaw-two-lane-capture/T1 deps: [] complexity: 2 -- hypaware-plugin-kernel-types.d.ts: restore `'json_path'` to `PluginAttachProbeManifest.format`'s union, add `container_path: string`, `provider_keys: string[]`, `cache_glob: string` (reuse existing `marker_header?`), and revise (not delete) the comment at the current removal site to explain what changed and why the danger it warned about (issue #212) is now addressed by T2/T3's runtime support. Add the optional `sweep?: { cron: string }` field to `BackfillContribution`, absent-by-default for every existing contribution. Test: a type-level check (or JSDoc `@ts-check` compile) that `hypaware-core/plugins-workspace/openclaw`'s existing files still typecheck unchanged, proving the additions are additive. +- id: T2 branch: task/openclaw-two-lane-capture/T2 deps: [T1] complexity: 4 -- src/core/config/client_detach_disk.js: add `detachJsonPathProviders({settingsPath, containerPath, providerKeys, markerHeader, cacheGlob, homeDir, expectedBaseUrl, fs})` implementing the ownership check (baseUrl matches expectedBaseUrl or expectedBaseUrl + '/v1', markerHeader value matches the key name), the backup-not-discard path for a present-but-mismatched entry (mirrors the LLP 0163 prev_malformed precedent, lands under a `_hypaware_detach_backup.` sibling key in the same file), and the best-effort cache purge across `homeDir/.openclaw/` (a file that fails to parse is logged and skipped, not fatal). Wire the dispatcher's `probe.format === 'json_path'` branch to call it. Thread a new `expectedBaseUrl` parameter through both real callers: `detachClientViaCore` in src/core/commands/clients.js (currently calls detachClientFromDisk with no expectedBaseUrl) and action_attach.js's `reverse()` (has `ctx.endpoint` already, in the same ActionContext perform() uses). Test: unit tests proving the four outcomes directly (ours: deleted; mismatched: backed up not discarded; absent file: {changed:false}; cache purge best-effort on a malformed sibling file). +- id: T3 branch: task/openclaw-two-lane-capture/T3 deps: [T1] complexity: 2 -- src/core/daemon/status.js: restore the `probe.format === 'json_path'` read branch (removed by LLP 0143 / PR #510) parallel to the existing json/toml branches at the current lines 1066/1083: navigate `container_path` + each of `provider_keys`, read `headers[marker_header]`, report attached when it equals the expected value for at least one configured key. Pure read, no ownership/backup concerns. Test: `probeClientAttachFromDescriptor` returns attached/not-attached correctly against a fixture openclaw.json with the entry present, absent, and present-but-wrong-header. +- id: T4 branch: task/openclaw-two-lane-capture/T4 deps: [] complexity: 4 -- New hypaware-core/plugins-workspace/openclaw/src/attach.js, `createOpenclawAttach({homeDir, fs})` returning `{attach(attachCtx)}`, mirroring hypaware-core/plugins-workspace/claude/src/index.js's attach() shape (same AiGatewayClientAttachContext param, withSpan('client.attach', ...), dry-run branch). Reads openclaw.json (or $OPENCLAW_HOME), refuses with {status:'failed', reason} if models.providers.anthropic or .openai already exists (R2, pure read-then-decide, no partial write), otherwise writes both entries from attachCtx.endpoint with the bare-origin (anthropic) vs. +'/v1' (openai) asymmetry the design flags as worth its own test, preserves any other existing models keys, prints the `openclaw gateway restart` instruction on both human and --json paths, returns {status:'done'}. hypaware-core/plugins-workspace/openclaw/src/index.js: delete the old no-op attach() body, STEERING_PLUGIN_NAME, ROUTING_OWNED_BY_STEERING_PLUGIN_MESSAGE, and the @ref LLP 0143#decision comment block; wire index.js's activate() to attach.js's attach() instead. Test: refusal-when-exists, the exact two-entry shape (asymmetry included), restart-instruction print on both output modes, and that attach() never throws on refusal. +- id: T5 branch: task/openclaw-two-lane-capture/T5 deps: [T2, T3, T4] complexity: 2 -- hypaware-core/plugins-workspace/openclaw/hypaware.plugin.json: add contributes.client.attach_probe exactly per design 1.4 (format json_path, settings_file .openclaw/openclaw.json, container_path models.providers, provider_keys [anthropic, openai], marker_header x-hypaware-upstream, cache_glob agents/*/agent/models.json); rewrite `description` and `picker[0].summary` to drop every @hypaware/openclaw-steering-plugin reference and state the two capture tiers directly (live gateway capture once attached, plus periodic transcript sweep). hypaware-core/plugins-workspace/claude/hypaware.plugin.json: add the LLP 0167#onboarding line naming the claude-cli/ case OpenClaw's CLI-backend exclusion produces. hypaware-core/plugins-workspace/openclaw/src/projector.js line 31: correct the stale "written by the openclaw-steering-plugin" comment to describe the config-override write. Test: a manifest-shape test asserting attach_probe parses to the exact fields above, and that description/summary strings no longer match /openclaw-steering-plugin/. +- id: T6 branch: task/openclaw-two-lane-capture/T6 deps: [] complexity: 2 -- hypaware-core/plugins-workspace/openclaw/src/config.js: validateBackfillSection gains `sweep_cron` (string, validated as a 5-field cron expression via the same validator cronMatches's caller uses to reject malformed schedules elsewhere) and `quiesce_ms` (non-negative integer), added in the same change so the existing unknown-key rejection loop recognizes both together, alongside the existing on_join/window_days keys. Test: validateOpenclawConfig accepts both keys with valid values, rejects an invalid cron string, rejects a negative quiesce_ms, and still rejects a genuinely unknown key. +- id: T7 branch: task/openclaw-two-lane-capture/T7 deps: [T1, T6] complexity: 3 -- hypaware-core/plugins-workspace/openclaw/src/backfill.js: createOpenclawBackfillProvider populates `sweep: { cron: config.backfill?.sweep_cron ?? '*/5 * * * *' }` on the returned contribution. src/core/commands/backfill.js: declare the new `BackfillRunnerContext` interface `{env, config, storage, backfills, backfillMaterializers}` (a structural subset every CommandRunContext already satisfies) and narrow runBackfillProvider's, runProvider's, and resolveOwnersForRun's `ctx` parameter types to it. Test: existing hyp backfill CLI-path and onboarding-finale call sites still typecheck and pass unchanged (proving the narrowing is non-breaking); a new test asserts the contribution's `sweep.cron` reads the configured value and falls back to the default when absent. +- id: T8 branch: task/openclaw-two-lane-capture/T8 deps: [T6] complexity: 3 -- hypaware-core/plugins-workspace/openclaw/src/backfill.js: listSessionFiles(agentsDir) gains an optional `quiesceBeforeMs` parameter, skipping any file whose mtimeMs is more recent; runOpenclawBackfill() computes it once per run as Date.now() - quiesceMs, where quiesceMs resolves from config.backfill?.quiesce_ms defaulting to 180000 (cited from QUERY_FLUSH_DEBOUNCE_MS in src/core/cache/spool.js plus a one-minute margin). Must compose with, not replace, the existing effectiveProviders/partitionByBackend forward/backward-fill logic (R10, untouched). Test: a session file with mtime inside the quiesce window is excluded from the run; one outside it is included; the default resolves to exactly 180000 when quiesce_ms is absent from config. +- id: T9 branch: task/openclaw-two-lane-capture/T9 deps: [T7, T8] complexity: 4 -- New src/core/daemon/backfill_sweep.js, createBackfillSweepDriver({backfills, backfillMaterializers, env, config, storage}) with tick({now}) iterating backfills.list(), skipping contributions with no sweep field or a not-due cronMatches (imported from src/core/sinks/driver.js), and firing `void runBackfillProvider({ctx: {env, config, storage, backfills, backfillMaterializers}, provider: provider.name, dryRun: false, devRunId: sweep--})` per due contribution, with the fired promise's rejection logged (component openclaw, operation backfill.sweep, error_kind) rather than left unhandled. src/core/daemon/runtime.js's runTick(): call `await sweepDriver.tick({now})` right after the existing sink-driver tick call, inside the same DEFAULT_TICK_INTERVAL_MS=60_000 loop, no new timer. Externally blocked for real capture (PR #552, see External blockers); code and unit tests (mocked backfills/provider) can land now. Test: tick() fires runBackfillProvider only for due, sweep-bearing contributions; a rejected sweep run does not throw out of tick() or block the sink tick's own await. +- id: T10 branch: task/openclaw-two-lane-capture/T10 deps: [T5] complexity: 2 -- test/plugins/openclaw-client-registration.test.js: rewrite the two attach() no-op tests (current lines 39-82, 84-114) to assert the new write-based behavior (refusal-when-exists, the two-entry shape, restart-instruction print) instead of the old /openclaw-steering-plugin/ stdout match; rewrite the descriptor test (lines 222-240) to assert the new json_path attach_probe shape instead of `attachProbe === undefined`; correct the "honest no-op" detach test's (lines 197-220) stale R7 comment and add a companion case using a real openclaw.json fixture proving the ownership-based ` detachJsonPathProviders` (T2) actually fires and reports changed:true. Leave the registration-order test (116-155) and the generic hyp attach resolution test (157-195) unchanged; neither depends on the old no-op shape. +- id: T11 branch: task/openclaw-two-lane-capture/T11 deps: [T5] complexity: 1 -- Delete openclaw-steering-plugin/ in full (src/, test/, package.json, openclaw.plugin.json, .d.ts files) and test/plugins/openclaw-steering-plugin.test.js (R9). Remove tsconfig.json's `"openclaw-steering-plugin"` entry from the `include` array (line 19; not named in the design's own deletion inventory, found verifying the deletion against the real tree). Test: `npm test` and a `tsc --noEmit` (or equivalent checkJs run) pass with no reference to the deleted directory remaining anywhere in the tree (`grep -rl openclaw-steering-plugin` returns nothing outside llp/ history documents). +- id: T12 branch: task/openclaw-two-lane-capture/T12 deps: [T8, T9] complexity: 3 -- New backfill_openclaw_fixture helper under hypaware-core/smoke/flows (mirroring backfill_claude_fixture.js / backfill_codex_fixture.js), writing a minimal OpenClaw v3 session JSONL in the nested-message-envelope shape under a temp agents//sessions/ tree with a controllable mtime, plus a smoke flow asserting (a) a file with mtime inside the quiesce window is skipped by a sweep run, (b) a file outside the window is captured, and (c) rerunning the sweep after a live-lane row already wrote the same part_id nets zero new rows. Externally blocked: hold, do not dispatch, until PR #552 merges into this integration branch (the fixture's envelope shape only matches the reader #552 introduces; building it against the current flat reader would test the wrong, soon-obsolete shape). +- id: T13 branch: task/openclaw-two-lane-capture/T13 deps: [T5, T8, T9] complexity: 3 -- docs/ACCEPTANCE.md's openclaw_capture section (starting at the current line 173): drop the steering-plugin link/enable setup and the before_model_resolve/hooks.allowConversationAccess version-gate language; add a setup step running `hyp attach --client openclaw` followed by the restart instruction it prints; add a sweep step (disable or wait out live capture, confirm the row is absent, confirm it lands within one sweep interval past the quiesce window); add a zero-duplicate assertion (a turn both lanes observe resolves to exactly one row for its part_id); re-confirm LLP 0167#verify-results items 1, 3, 4 on the floor OpenClaw version. State in the section's own Requires line that the sweep/dedupe steps need PR #552 merged, and the client_attach status-row re-confirmation needs PR #553 merged, to run successfully. Test: this is a doc; the test is a human's successful run, which this task's own text cannot perform, only specify accurately against what T2/T4/T5 actually implement. diff --git a/openclaw-steering-plugin/openclaw.plugin.json b/openclaw-steering-plugin/openclaw.plugin.json deleted file mode 100644 index 4804729c..00000000 --- a/openclaw-steering-plugin/openclaw.plugin.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "id": "hypaware-openclaw-steering", - "name": "HypAware OpenClaw Steering", - "description": "Registers the hypaware-anthropic and hypaware-openai shadow providers and steers every steerable OpenClaw model call to them, so OpenClaw conversations are captured through the local HypAware AI gateway without editing openclaw.json.", - "version": "0.1.0", - "providers": ["hypaware-anthropic", "hypaware-openai"], - "syntheticAuthRefs": ["hypaware-anthropic", "hypaware-openai"], - "nonSecretAuthMarkers": ["hypaware-borrowed"], - "activation": { - "onStartup": true, - "onCapabilities": ["provider", "hook"] - }, - "configSchema": { - "type": "object", - "additionalProperties": false, - "properties": {} - } -} diff --git a/openclaw-steering-plugin/package.json b/openclaw-steering-plugin/package.json deleted file mode 100644 index 547d4c30..00000000 --- a/openclaw-steering-plugin/package.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "name": "@hypaware/openclaw-steering-plugin", - "version": "0.1.0", - "private": false, - "description": "OpenClaw plugin that steers every steerable model call through the local HypAware AI gateway via two shadow providers (hypaware-anthropic, hypaware-openai), so OpenClaw conversations are captured without editing openclaw.json.", - "type": "module", - "main": "./src/index.js", - "openclaw": { - "extensions": ["./src/index.js"] - }, - "engines": { - "node": ">=20" - }, - "scripts": { - "test": "node --test" - }, - "files": [ - "src/", - "openclaw.plugin.json" - ] -} diff --git a/openclaw-steering-plugin/src/gateway_endpoint.js b/openclaw-steering-plugin/src/gateway_endpoint.js deleted file mode 100644 index 394cf522..00000000 --- a/openclaw-steering-plugin/src/gateway_endpoint.js +++ /dev/null @@ -1,28 +0,0 @@ -// HypAware's fixed default AI gateway listen address (LLP 0114#decision). -// This package cannot import `src/core/config/gateway_endpoint.js` - it runs -// inside OpenClaw's process, never HypAware's, and is not a HypAware kernel -// plugin (LLP 0161#package-layout) - so the value is mirrored here rather -// than imported. -const DEFAULT_GATEWAY_ENDPOINT = 'http://127.0.0.1:18521' - -/** - * The environment variable carrying the local AI gateway's `localEndpoint()` - * value (LLP 0161#steering-plugin). Nothing on the HypAware side sets it: - * this package runs inside OpenClaw's process, so the operator puts it in - * `~/.openclaw/openclaw.json`'s `env` block (docs/ACCEPTANCE.md - * `## openclaw_capture` states the step). Resolved once at plugin load, - * matching `gateway.localEndpoint()`'s own "ask the live gateway" contract - * as closely as a process with no access to the HypAware kernel can. - */ -export const GATEWAY_ENDPOINT_ENV_VAR = 'HYP_GATEWAY_ENDPOINT' - -/** - * @param {NodeJS.ProcessEnv} [env] - * @returns {string} - */ -export function resolveGatewayEndpoint(env = process.env) { - const configured = env[GATEWAY_ENDPOINT_ENV_VAR]?.trim() - return configured || DEFAULT_GATEWAY_ENDPOINT -} - -export { DEFAULT_GATEWAY_ENDPOINT } diff --git a/openclaw-steering-plugin/src/index.js b/openclaw-steering-plugin/src/index.js deleted file mode 100644 index cf63dfc9..00000000 --- a/openclaw-steering-plugin/src/index.js +++ /dev/null @@ -1,223 +0,0 @@ -// `@hypaware/openclaw-steering-plugin` - an OpenClaw-installed plugin, not a -// HypAware kernel plugin. It never calls `ctx.requireCapability` or any -// `PluginActivationContext` method; its only coupling to the HypAware repo -// is the `x-hypaware-upstream` header contract this file writes and -// `@hypaware/openclaw`'s projector reads (LLP 0161#upstream-header). -// -// @ref LLP 0161#package-layout [implements]: a new top-level package, -// sibling to `hypaware-core/` and `src/`, with its own package.json and its -// own OpenClaw-native plugin manifest (`openclaw.plugin.json`), because it -// is an npm package OpenClaw installs, not a relative-import HypAware -// kernel plugin. -// -// OpenClaw's own plugin entry/manifest shape (`definePluginEntry` from -// `openclaw/plugin-sdk/plugin-entry`, the `id`/`name`/`description`/ -// `register(api)` fields, `api.registerProvider({ id, catalog: { run } })`, -// `api.on('before_model_resolve', handler)`) is verified against OpenClaw's -// published plugin docs (docs.openclaw.ai/plugins/sdk-entrypoints, -// /plugins/sdk-provider-plugins, /plugins/hooks, /plugins/manifest; -// 2026-07-30), the same way LLP 0157/0161 verified -// `extensions/anthropic/stream-wrappers.ts` and -// `openclaw/plugin-sdk/provider-auth-runtime` against the openclaw repo -// before relying on them. Two residual open items, worth confirming against -// a live OpenClaw install before this plugin ships: the exact field names -// `before_model_resolve`'s `event` argument carries for the candidate's -// resolved `provider`/`api` (the docs state only "the current prompt and -// attachment metadata", not a field list), and whether the hook's return -// value supports a metadata channel beyond `providerOverride`/ -// `modelOverride` for carrying `x-hypaware-upstream` (LLP 0161#steering-plugin -// states the hook returns both; the docs page examined here names only the -// two override fields explicitly). - -import { definePluginEntry } from 'openclaw/plugin-sdk/plugin-entry' -import { resolveApiKeyForProvider } from 'openclaw/plugin-sdk/provider-auth-runtime' - -import { resolveGatewayEndpoint } from './gateway_endpoint.js' -import { createPrepareRuntimeAuth, normalizeBorrowedCredential, resolveShadowSyntheticAuth } from './runtime_auth.js' -import { resolveSteering } from './steering.js' -import { createWarningLedger } from './warning_ledger.js' -import { wrapAnthropicShadowStream } from './wire_parity.js' - -/** - * @import { OpenclawPluginApi, ProviderPrepareRuntimeAuthContext, ResolvedProviderAuth } from './types.js' - */ - -/** Must match the `id` in `openclaw.plugin.json` (OpenClaw plugin-manifest requirement). */ -export const PLUGIN_ID = 'hypaware-openclaw-steering' - -export const SHADOW_ANTHROPIC_PROVIDER_ID = 'hypaware-anthropic' -export const SHADOW_OPENAI_PROVIDER_ID = 'hypaware-openai' - -/** - * Adapts OpenClaw's `resolveApiKeyForProvider` to the "one credential or - * nothing" contract `resolveSteering` and `createPrepareRuntimeAuth` both - * branch on. The SDK resolves to a `ResolvedProviderAuth` record - * (`{ apiKey?, profileId?, source, mode }`), never a bare key, so the object - * itself is always truthy and unwrapping it here is what makes "no credential - * for this provider" a distinguishable outcome at all. - * - * @param {{ provider: string, context?: ProviderPrepareRuntimeAuthContext }} params - * @returns {Promise} - */ -function borrowShadowedCredential(params) { - return resolveApiKeyForProvider({ - provider: params.provider, - cfg: params.context?.config, - agentDir: params.context?.agentDir, - workspaceDir: params.context?.workspaceDir, - }) -} - -/** - * Registers both shadow providers (LLP 0144#decision) with `baseUrl` at the - * local HypAware AI gateway, and the credential/wire hooks that make a steered - * turn indistinguishable from an unsteered one. - * - * The registration carries no apiKey: a `hypaware-*` provider has none, and - * must never acquire one. `resolveSyntheticAuth` supplies a non-secret - * placeholder purely so OpenClaw's runtime reaches `prepareRuntimeAuth` - * (which throws `MissingProviderAuthError` first otherwise), and - * `prepareRuntimeAuth` swaps in the shadowed provider's real credential for - * that one request (LLP 0145#decision). - * - * `wrapStreamFn` is Anthropic-only: LLP 0148's scope note makes parity - * mirroring per API shape, and `openai-completions` has no OpenClaw-side - * shaping to mirror in v1. - * - * @ref LLP 0157#steering-plugin [implements]: the two shadow providers are - * registered programmatically with `baseUrl` at the local gateway, never - * writing to the user's `openclaw.json`. - * - * @param {OpenclawPluginApi} api - * @param {string} baseUrl - * @param {ReturnType} ledger - */ -function registerShadowProviders(api, baseUrl, ledger) { - const prepareRuntimeAuth = createPrepareRuntimeAuth({ - baseUrl, - resolveCredential: borrowShadowedCredential, - onError: ({ provider, error }) => { - ledger.warn({ - provider, - cause: 'no_credential', - operation: 'prepare_runtime_auth', - status: 'borrow_failed', - detail: describeError(error), - }) - }, - }) - - api.registerProvider({ - id: SHADOW_ANTHROPIC_PROVIDER_ID, - label: 'HypAware capture (Anthropic)', - catalog: { - order: 'simple', - async run() { - return { - providers: { - [SHADOW_ANTHROPIC_PROVIDER_ID]: { baseUrl, api: 'anthropic-messages' }, - }, - } - }, - }, - resolveSyntheticAuth: resolveShadowSyntheticAuth, - prepareRuntimeAuth, - wrapStreamFn: (ctx) => - wrapAnthropicShadowStream(ctx, { - onSkipped: (reason) => { - ledger.warn({ - provider: SHADOW_ANTHROPIC_PROVIDER_ID, - cause: 'wire_parity_skipped', - operation: 'wrap_stream_fn', - status: 'degraded', - detail: reason, - }) - }, - }), - }) - - api.registerProvider({ - id: SHADOW_OPENAI_PROVIDER_ID, - label: 'HypAware capture (OpenAI)', - catalog: { - order: 'simple', - async run() { - return { - providers: { - [SHADOW_OPENAI_PROVIDER_ID]: { baseUrl, api: 'openai-completions' }, - }, - } - }, - }, - resolveSyntheticAuth: resolveShadowSyntheticAuth, - prepareRuntimeAuth, - }) -} - -/** - * @param {unknown} error - * @returns {string} - */ -function describeError(error) { - return error instanceof Error ? error.message : String(error) -} - -/** - * Registers the `before_model_resolve` steering hook. Re-fires per - * candidate the hook is asked about - primary, fallbacks, per-agent - * overrides, the extra model slots (LLP 0152 Context) - so returning a - * decision per call, rather than caching one, is what makes coverage total - * (LLP 0157 R2). - * - * @ref LLP 0161#steering-precedence [implements]: `resolveSteering`'s - * four-branch precedence wired to OpenClaw's own hook contract. - * - * @param {OpenclawPluginApi} api - * @param {ReturnType} ledger - */ -function registerSteeringHook(api, ledger) { - api.on('before_model_resolve', async (event, ctx) => { - const candidate = { provider: event.provider, api: event.api } - - const result = await resolveSteering(candidate, { - // Same borrow `prepareRuntimeAuth` performs, unwrapped to the bare key. - // `resolveSteering` refuses to steer a provider whose credential cannot - // be resolved (LLP 0157 R3), so the probe and the borrow have to agree - // on what "resolvable" means or the ledger's coverage claim is wrong. - resolveCredential: async (provider) => { - const resolved = await borrowShadowedCredential({ - provider, - context: { provider, agentDir: ctx?.agentDir, workspaceDir: ctx?.workspaceDir }, - }) - return normalizeBorrowedCredential(resolved)?.apiKey - }, - }) - - if (result.steer) { - return { providerOverride: result.providerOverride, requestMeta: result.requestMeta } - } - - ledger.warn({ provider: result.provider, cause: result.cause, session: ctx?.sessionKey }) - // No decision: the candidate passes through on its original provider, - // unmodified (LLP 0157 R5). The user's turn never fails because of - // capture. - return undefined - }) -} - -export default definePluginEntry({ - id: PLUGIN_ID, - name: 'HypAware OpenClaw Steering', - description: - 'Registers the hypaware-anthropic and hypaware-openai shadow providers and steers every steerable OpenClaw model call to them, so OpenClaw conversations are captured through the local HypAware AI gateway without editing openclaw.json.', - // `definePluginEntry`'s own declaration cannot name `OpenclawPluginApi` - // (see `openclaw_plugin_sdk.d.ts`), so the parameter is annotated here. - /** @param {OpenclawPluginApi} api */ - register(api) { - const baseUrl = resolveGatewayEndpoint() - const ledger = createWarningLedger() - - registerShadowProviders(api, baseUrl, ledger) - registerSteeringHook(api, ledger) - }, -}) diff --git a/openclaw-steering-plugin/src/openclaw_plugin_sdk.d.ts b/openclaw-steering-plugin/src/openclaw_plugin_sdk.d.ts deleted file mode 100644 index 4de08516..00000000 --- a/openclaw-steering-plugin/src/openclaw_plugin_sdk.d.ts +++ /dev/null @@ -1,42 +0,0 @@ -// Ambient declarations for the two `openclaw/plugin-sdk/*` subpaths -// `src/index.js` imports. `openclaw` is the host process that installs this -// package, not a dependency of it, so those specifiers never resolve inside -// this repo - which left `index.js`, the entrypoint that registers both -// shadow providers and wires the steering hook, as the one source file the -// typecheck graph could not reach (every other file in the package is -// pulled in through the root test shim, `test/plugins/ -// openclaw-steering-plugin.test.js`). Declaring the two modules here puts -// it back under `npm run typecheck`. Like `types.d.ts`, this asserts -// OpenClaw's API surface rather than importing it, because OpenClaw is the -// host, not a HypAware dependency (LLP 0161#package-layout). -// -// Two constraints shape the form below. The file must stay a script - no -// top-level `import` or `export` - or the blocks are read as augmentations -// of modules that do not exist. And an ambient module body may not import -// through a relative specifier (TS2439), so neither block can reference -// `./types.js`: `definePluginEntry` therefore takes its `register` callback -// loosely and `index.js` annotates the parameter itself, and the auth -// record is restated structurally here. Keep that restatement in step with -// `ResolvedProviderAuth` in `types.d.ts`; it is the same OpenClaw record. - -declare module 'openclaw/plugin-sdk/plugin-entry' { - export function definePluginEntry(entry: { - id: string - name: string - description?: string - register(api: any): void | Promise - }): unknown -} - -declare module 'openclaw/plugin-sdk/provider-auth-runtime' { - /** - * Resolves to a record, never a bare key, so an unwrapped `apiKey` of - * `undefined` is what "no credential for this provider" looks like. - */ - export function resolveApiKeyForProvider(params: { - provider: string - cfg?: unknown - agentDir?: string - workspaceDir?: string - }): Promise<{ apiKey?: string, profileId?: string, source?: string, mode?: string }> -} diff --git a/openclaw-steering-plugin/src/runtime_auth.js b/openclaw-steering-plugin/src/runtime_auth.js deleted file mode 100644 index e25277c2..00000000 --- a/openclaw-steering-plugin/src/runtime_auth.js +++ /dev/null @@ -1,158 +0,0 @@ -// Credential borrowing for the two shadow providers. -// -// A `hypaware-*` provider has no credential of its own and must never acquire -// one: LLP 0145 rejects both "require a vendor env var" and "read OpenClaw's -// credential storage from outside its process", leaving exactly one route - -// resolve the *shadowed* provider's credential in-process, through OpenClaw's -// own public `resolveApiKeyForProvider`, and hand it back for that one -// request. This module is the pure half of that, with the SDK call injected so -// it is testable without an OpenClaw host. -// -// Nothing here writes the borrowed credential anywhere. There is no cache, no -// module-level variable holding it, and no return path other than the value -// OpenClaw asked for: the credential exists only for the duration of the call -// that resolved it (LLP 0145#decision, LLP 0157 R3). -// -// @ref LLP 0145#decision [implements]: the shadow provider borrows the -// shadowed provider's credential inside `prepareRuntimeAuth`, so refresh and -// expiry ride OpenClaw's generic background-refresh path instead of being -// pinned at catalog-build time. - -import { CANONICAL_PROVIDER_FOR_SHAPE, SHADOW_FOR_SHAPE } from './steering.js' - -/** - * @import { ProviderPrepareRuntimeAuthContext, ProviderPreparedRuntimeAuth, ProviderSyntheticAuthResult, ResolvedProviderAuth } from './types.js' - */ - -/** - * Which real provider each shadow stands in for, derived from the two maps - * `resolveSteering` already branches on rather than restated, so a third API - * shape can never be added to steering without also being borrowable here. - * - * @type {Readonly>} - */ -export const REAL_PROVIDER_FOR_SHADOW = Object.freeze( - Object.fromEntries( - Object.entries(SHADOW_FOR_SHAPE).map(([shape, shadow]) => [shadow, CANONICAL_PROVIDER_FOR_SHAPE[shape]]), - ), -) - -/** - * The placeholder credential `resolveSyntheticAuth` hands OpenClaw for a - * `hypaware-*` provider. It is not a secret and never reaches the wire: the - * only thing it does is get the runtime past the `MissingProviderAuthError` - * that `applyApiKeyInfo` throws for a provider with no resolvable credential, - * which is raised *before* `prepareRuntimeAuth` runs (verified against - * openclaw's `src/agents/embedded-agent-runner/run/auth-controller.ts`, - * 2026-07-30). Without it the borrow below would never be reached, and every - * steered turn would fail on the shadow provider's empty credential. LM - * Studio's bundled plugin uses the same seam with `custom-local`. - */ -export const SYNTHETIC_AUTH_MARKER = 'hypaware-borrowed' - -/** - * How long a prepared OAuth borrow is declared good for. - * - * OpenClaw only schedules a background re-preparation when the prepared auth - * carries an `expiresAt`; without one, the borrowed token is pinned for the - * whole run and a long turn can outlive it. OpenClaw's public - * `resolveApiKeyForProvider` returns `{ apiKey, profileId, source, mode }` and - * no expiry, so this value is not a claim about when the token dies - it is a - * *re-resolution deadline*, chosen so OpenClaw comes back and asks again, - * at which point `resolveApiKeyForProvider`'s own OAuth branch refreshes - * under lock if needed. It must stay comfortably above OpenClaw's - * `RUNTIME_AUTH_REFRESH_MARGIN_MS` (5 minutes), since the refresh is scheduled - * at `expiresAt - margin` and clamped to a 5-second floor: a TTL at or below - * the margin would busy-loop. - */ -export const BORROWED_OAUTH_REVALIDATE_MS = 15 * 60 * 1000 - -/** - * Reduces OpenClaw's `ResolvedProviderAuth` to "is there a usable credential, - * and is it an OAuth one". A resolver that returns a bare string (or a - * `MissingProviderAuthError` shaped result with no `apiKey`) is handled too, - * because `resolveSteering`'s credential probe and this borrow share one - * injected resolver and must agree on what "no credential" means. - * - * @param {ResolvedProviderAuth | string | undefined | null} resolved - * @returns {{ apiKey: string, mode?: string, profileId?: string, source?: string } | undefined} - */ -export function normalizeBorrowedCredential(resolved) { - if (typeof resolved === 'string') { - return resolved.trim() ? { apiKey: resolved } : undefined - } - if (!resolved || typeof resolved !== 'object') return undefined - const apiKey = typeof resolved.apiKey === 'string' ? resolved.apiKey.trim() : '' - if (!apiKey) return undefined - return { apiKey, mode: resolved.mode, profileId: resolved.profileId, source: resolved.source } -} - -/** - * `resolveSyntheticAuth` for the shadow providers. Returns a placeholder only - * for a provider this plugin owns, so it can never affect another provider's - * auth resolution. - * - * @param {{ provider: string }} ctx - * @returns {ProviderSyntheticAuthResult | undefined} - */ -export function resolveShadowSyntheticAuth(ctx) { - if (REAL_PROVIDER_FOR_SHADOW[ctx.provider] === undefined) return undefined - return { - apiKey: SYNTHETIC_AUTH_MARKER, - source: 'hypaware-openclaw-steering (borrowed at request time)', - mode: 'api-key', - } -} - -/** - * Builds the `prepareRuntimeAuth` hook shared by both shadow providers. - * - * Per call, it re-resolves the shadowed provider's credential and returns it - * together with the gateway `baseUrl` - the belt-and-braces endpoint override - * LLP 0152's Consequences name, so a request cannot escape the gateway even if - * the catalog entry is stale. It returns `undefined` (rather than throwing) - * when nothing can be borrowed: `resolveSteering` already refused to steer - * that provider, so reaching here means the credential disappeared between the - * steering decision and the request, and a pass-through failure is the honest - * outcome. Capture must never fail a user's turn on its own account - * (LLP 0157 R5). - * - * @ref LLP 0161#credentials-and-wire [implements]: `{ apiKey, baseUrl, - * expiresAt? }` per request, never persisted, re-resolved every call. - * - * @param {{ - * baseUrl: string, - * resolveCredential(params: { provider: string, context: ProviderPrepareRuntimeAuthContext }): Promise | ResolvedProviderAuth | string | undefined | null, - * now?: () => number, - * revalidateMs?: number, - * onError?: (info: { provider: string, error: unknown }) => void, - * }} opts - * @returns {(ctx: ProviderPrepareRuntimeAuthContext) => Promise} - */ -export function createPrepareRuntimeAuth(opts) { - const now = opts.now ?? Date.now - const revalidateMs = opts.revalidateMs ?? BORROWED_OAUTH_REVALIDATE_MS - - return async function prepareRuntimeAuth(ctx) { - const realProvider = REAL_PROVIDER_FOR_SHADOW[ctx.provider] - if (realProvider === undefined) return undefined - - let resolved - try { - resolved = await opts.resolveCredential({ provider: realProvider, context: ctx }) - } catch (error) { - opts.onError?.({ provider: realProvider, error }) - return undefined - } - - const borrowed = normalizeBorrowedCredential(resolved) - if (borrowed === undefined) return undefined - - /** @type {ProviderPreparedRuntimeAuth} */ - const prepared = { apiKey: borrowed.apiKey, baseUrl: opts.baseUrl } - if (borrowed.mode === 'oauth') { - prepared.expiresAt = now() + revalidateMs - } - return prepared - } -} diff --git a/openclaw-steering-plugin/src/steering.js b/openclaw-steering-plugin/src/steering.js deleted file mode 100644 index ab91ea5a..00000000 --- a/openclaw-steering-plugin/src/steering.js +++ /dev/null @@ -1,153 +0,0 @@ -// This module is the pure decision core of the OpenClaw steering plugin. It -// has no dependency on `openclaw/plugin-sdk/*` on purpose, so it is testable -// standalone (LLP 0162 T1) without an OpenClaw host to run inside of. - -/** - * The one real provider `before_model_resolve` ever substitutes a shadow - * for, per API shape (LLP 0144#decision): `hypaware-anthropic` covers - * `anthropic-messages`, `hypaware-openai` covers `openai-completions`. - * - * @ref LLP 0144#decision [implements]: one shadow provider per API shape, - * not per vendor. - * @type {Readonly>} - */ -export const SHADOW_FOR_SHAPE = Object.freeze({ - 'anthropic-messages': 'hypaware-anthropic', - 'openai-completions': 'hypaware-openai', -}) - -/** - * The literal provider id each shape's shadow stands in for. A candidate - * whose declared `api` matches a shape but whose `provider` is not this - * value is never steered directly to that shadow (LLP 0161#steering-precedence: - * the gateway's upstream presets have a static `base_url` per preset, so - * steering anything else would silently redirect it to the wrong vendor). - * - * @type {Readonly>} - */ -export const CANONICAL_PROVIDER_FOR_SHAPE = Object.freeze({ - 'anthropic-messages': 'anthropic', - 'openai-completions': 'openai', -}) - -/** - * Host-signed and per-user-URL provider families that are deferred rather - * than steered, even though they share a shape with a canonical provider - * (LLP 0146#decision). This is a declared list, not a heuristic: a provider - * is steered unless it is named here. - * - * - `amazon-bedrock`, `anthropic-vertex`: signing is scoped to the request - * host, so a retargeted `baseUrl` likely breaks it (LLP 0146 Context). - * - `google`, `google-vertex`, `google-gemini-cli`: LLP 0146's "the Google - * providers" family, named here by OpenClaw's own provider ids; verify - * this trio against a live OpenClaw install before relying on it, and if - * OpenClaw ships another Google-family id later, add it as its own short - * decision LLP citing LLP 0146/0161, not a silent diff here - * (LLP 0161 Section 11). - * - `cloudflare-ai-gateway`, `vercel-ai-gateway`: deferred for a mechanical - * reason, not a signing one - their real base URL is per-user, so no - * static gateway preset can represent them (LLP 0146 Open questions). - * - * @ref LLP 0146#decision [implements]: the declared deferred-family list. - * @type {ReadonlySet} - */ -export const DEFERRED_SET = new Set([ - 'amazon-bedrock', - 'anthropic-vertex', - 'google', - 'google-vertex', - 'google-gemini-cli', - 'cloudflare-ai-gateway', - 'vercel-ai-gateway', -]) - -/** - * The three named pass-through warning causes (LLP 0157 R5, LLP 0149#decision). - * `resolveSteering` never returns a fourth. - */ -export const WARNING_CAUSES = Object.freeze({ - NO_CREDENTIAL: 'no_credential', - NO_PRESET: 'no_preset', - DEFERRED: 'deferred', -}) - -/** - * @param {{ provider: string }} candidate - * @param {string} cause - * @returns {{ steer: false, cause: string, provider: string }} - */ -function warn(candidate, cause) { - return { steer: false, cause, provider: candidate.provider } -} - -/** - * Wraps `openclaw/plugin-sdk/provider-auth-runtime`'s `resolveApiKeyForProvider` - * (LLP 0145#decision) so "no credential available" is a value `resolveSteering` - * can branch on, rather than an exception it would have to catch inline. A - * throw (profile store unavailable, provider unknown to OpenClaw's auth - * runtime, etc.) is treated the same as "no credential": the turn passes - * through and warns, it never fails the user's turn (LLP 0157 R5). - * - * @param {string} provider - * @param {{ resolveCredential(provider: string): Promise|string|undefined|null }} ctx - * @returns {Promise} - */ -export async function tryResolveApiKeyForProvider(provider, ctx) { - try { - const credential = await ctx.resolveCredential(provider) - return credential || undefined - } catch { - return undefined - } -} - -/** - * The four-branch steering precedence (LLP 0161#steering-precedence), - * terminal at each check: - * - * 1. no shadow covers this candidate's `api` shape at all -> `no_preset` - * 2. the candidate's `provider` is not the shape's one canonical vendor: - * - a named deferred family (LLP 0146) -> `deferred` - * - anything else (an unrecognized vendor sharing the shape) -> `no_preset` - * 3. the shadowed provider's credential cannot be resolved -> `no_credential` - * 4. otherwise: steer, carrying the real provider as `x-hypaware-upstream` - * request metadata (LLP 0161#upstream-header) so the gateway and the - * projector can both recover true upstream identity. - * - * @ref LLP 0157#requirements [implements]: R2, R3, R5, R6 collapse into this - * one algorithm, not four independent checks. - * - * @param {{ provider: string, api: string }} candidate - * @param {{ resolveCredential(provider: string): Promise|string|undefined|null }} ctx - * @returns {Promise< - * | { steer: true, providerOverride: string, requestMeta: { 'x-hypaware-upstream': string } } - * | { steer: false, cause: string, provider: string } - * >} - */ -export async function resolveSteering(candidate, ctx) { - const shape = candidate.api - const shadow = SHADOW_FOR_SHAPE[shape] - - if (shadow === undefined) { - return warn(candidate, WARNING_CAUSES.NO_PRESET) - } - - const canonicalProvider = CANONICAL_PROVIDER_FOR_SHAPE[shape] - if (candidate.provider !== canonicalProvider) { - if (DEFERRED_SET.has(candidate.provider)) { - return warn(candidate, WARNING_CAUSES.DEFERRED) - } - return warn(candidate, WARNING_CAUSES.NO_PRESET) - } - - const credential = await tryResolveApiKeyForProvider(candidate.provider, ctx) - if (credential === undefined) { - return warn(candidate, WARNING_CAUSES.NO_CREDENTIAL) - } - - return { - steer: true, - providerOverride: shadow, - requestMeta: { 'x-hypaware-upstream': candidate.provider }, - } -} diff --git a/openclaw-steering-plugin/src/types.d.ts b/openclaw-steering-plugin/src/types.d.ts deleted file mode 100644 index 3848b6c7..00000000 --- a/openclaw-steering-plugin/src/types.d.ts +++ /dev/null @@ -1,147 +0,0 @@ -// Minimal ambient shapes for the slice of OpenClaw's own plugin SDK this -// package touches (`openclaw/plugin-sdk/plugin-entry`, -// `openclaw/plugin-sdk/provider-auth-runtime`). This is OpenClaw API -// surface, not HypAware's - see LLP 0161#package-layout - so it is typed -// locally in this package rather than pulled from -// `hypaware-plugin-kernel-types.d.ts`, which describes the unrelated -// HypAware kernel plugin contract this package never implements. - -export interface ProviderCatalogEntry { - baseUrl: string - api: string -} - -export interface ProviderCatalogRunResult { - providers: Record -} - -/** - * The model descriptor `pi-ai` hands a `StreamFn`. Only the two fields the - * wire-parity mirror branches on are typed here. - */ -export interface WireModel { - id?: string - api?: string - provider?: string - baseUrl?: string -} - -/** - * The slice of `pi-ai`'s `StreamOptions` the mirror reads or replaces. - * `headers` is documented there as "merged with provider defaults; can - * override default headers", and `onPayload` as the hook for "inspecting or - * replacing provider payloads before sending". - */ -export interface StreamOptions { - apiKey?: string - headers?: Record - onPayload?: (payload: unknown, model: WireModel) => unknown - [key: string]: unknown -} - -export type StreamFn = (model: WireModel, context: unknown, options?: StreamOptions) => unknown - -/** `ProviderWrapStreamFnContext`, narrowed to what the mirror reads. */ -export interface ProviderWrapStreamFnContext { - provider: string - modelId?: string - model?: WireModel - extraParams?: Record - streamFn?: StreamFn -} - -/** OpenClaw's `ResolvedProviderAuth`, the return of `resolveApiKeyForProvider`. */ -export interface ResolvedProviderAuth { - apiKey?: string - profileId?: string - source?: string - mode?: 'api-key' | 'oauth' | 'token' | 'aws-sdk' | string -} - -/** OpenClaw's `ProviderPrepareRuntimeAuthContext`. */ -export interface ProviderPrepareRuntimeAuthContext { - provider: string - modelId?: string - model?: WireModel - apiKey?: string - authMode?: string - profileId?: string - agentDir?: string - workspaceDir?: string - config?: unknown - env?: NodeJS.ProcessEnv -} - -/** OpenClaw's `ProviderPreparedRuntimeAuth`. */ -export interface ProviderPreparedRuntimeAuth { - apiKey: string - baseUrl?: string - expiresAt?: number -} - -/** OpenClaw's `ProviderSyntheticAuthResult`. */ -export interface ProviderSyntheticAuthResult { - apiKey: string - source: string - mode: 'api-key' | 'oauth' | 'token' -} - -export interface RegisterProviderOptions { - id: string - label?: string - catalog: { - order?: string - run(): Promise - } - prepareRuntimeAuth?: ( - ctx: ProviderPrepareRuntimeAuthContext, - ) => Promise | ProviderPreparedRuntimeAuth | undefined - resolveSyntheticAuth?: (ctx: { provider: string }) => ProviderSyntheticAuthResult | undefined - wrapStreamFn?: (ctx: ProviderWrapStreamFnContext) => StreamFn | undefined -} - -export interface BeforeModelResolveEvent { - provider: string - api: string -} - -export interface BeforeModelResolveCtx { - sessionKey?: string - agentId?: string - agentDir?: string - workspaceDir?: string -} - -export interface BeforeModelResolveResult { - providerOverride?: string - modelOverride?: string - requestMeta?: Record -} - -/** - * One pass-through warning record the ledger rate-limits and emits. - * `operation`, `status`, and `detail` default to the LLP 0149 pass-through - * ledger's own values; they are per-record so the credential and wire-parity - * hooks (LLP 0161#credentials-and-wire), which are degraded capture rather - * than an uncaptured turn, can share the one rate limiter. - */ -export interface UncapturedTurn { - provider: string - cause: string - session?: string - operation?: string - status?: string - detail?: string -} - -export interface OpenclawPluginApi { - registerProvider(opts: RegisterProviderOptions): void - on( - hookName: 'before_model_resolve', - handler: ( - event: BeforeModelResolveEvent, - ctx: BeforeModelResolveCtx, - ) => Promise | BeforeModelResolveResult | undefined, - opts?: Record, - ): void -} diff --git a/openclaw-steering-plugin/src/warning_ledger.js b/openclaw-steering-plugin/src/warning_ledger.js deleted file mode 100644 index 608fdfde..00000000 --- a/openclaw-steering-plugin/src/warning_ledger.js +++ /dev/null @@ -1,64 +0,0 @@ -// Rate-limited pass-through warning emission (LLP 0149#decision): "rate- -// limited per provider+cause, not per turn, so a misconfigured provider does -// not flood logs." This package runs inside OpenClaw's own process, not the -// HypAware kernel, so it has no access to HypAware's structured-logging -// sink; it emits a structured record naming provider, cause, and session so -// whatever collects OpenClaw's own plugin output can still answer a -// coverage query, matching the fields LLP 0149 names as the coverage ledger. -// -// @ref LLP 0149#decision [implements]: one rate-limited warning per -// provider+cause naming provider, cause, session. - -/** - * @import { UncapturedTurn } from './types.js' - */ - -const DEFAULT_WINDOW_MS = 5 * 60 * 1000 - -/** - * @param {{ - * windowMs?: number, - * now?: () => number, - * emit?: (record: UncapturedTurn & { component: string, operation: string, status: string }) => void, - * }} [opts] - */ -export function createWarningLedger(opts = {}) { - const windowMs = opts.windowMs ?? DEFAULT_WINDOW_MS - const now = opts.now ?? Date.now - const emit = opts.emit ?? defaultEmit - /** @type {Map} */ - const lastEmittedAt = new Map() - - return { - /** - * @param {UncapturedTurn} record - * @returns {boolean} true if the warning was emitted, false if it was suppressed by the rate limit - */ - warn(record) { - const key = `${record.provider}:${record.cause}` - const at = now() - const last = lastEmittedAt.get(key) - if (last !== undefined && at - last < windowMs) { - return false - } - lastEmittedAt.set(key, at) - emit({ - component: 'openclaw-steering-plugin', - operation: record.operation ?? 'before_model_resolve', - status: record.status ?? 'uncaptured', - provider: record.provider, - cause: record.cause, - session: record.session, - ...(record.detail === undefined ? {} : { detail: record.detail }), - }) - return true - }, - } -} - -/** - * @param {UncapturedTurn & { component: string, operation: string, status: string }} record - */ -function defaultEmit(record) { - console.warn('[hypaware-openclaw-steering] uncaptured provider turn', record) -} diff --git a/openclaw-steering-plugin/src/wire_parity.js b/openclaw-steering-plugin/src/wire_parity.js deleted file mode 100644 index fb322388..00000000 --- a/openclaw-steering-plugin/src/wire_parity.js +++ /dev/null @@ -1,504 +0,0 @@ -// Wire parity for the `hypaware-anthropic` shadow provider. -// -// OpenClaw's own Anthropic request shaping lives in -// `extensions/anthropic/stream-wrappers.ts` and is owner-scoped: it runs only -// for the provider literally named `anthropic`. A steered turn runs on -// `hypaware-anthropic`, so none of it runs, and a capture layer that changes -// the wire it captures undermines its own record. This module is the mirror -// LLP 0148 requires, kept pure (no `openclaw/plugin-sdk/*`, no `pi-ai`) so it -// is testable without an OpenClaw host. -// -// @ref LLP 0148#decision [implements]: the shadow provider's own -// `wrapStreamFn` mirrors OpenClaw's Anthropic request shaping, rather than the -// gateway growing an injection seam. -// -// Verified against the shipped OpenClaw bundle -// (`dist/extensions/anthropic/stream-wrappers.js`, openclaw 2026-07-30) and -// against `@mariozechner/pi-ai@0.73.1` (`dist/providers/anthropic.js`), which -// is the library that actually issues the request underneath. Three facts from -// that reading shape this file, and each is load-bearing: -// -// 1. **pi-ai adds the betas itself.** `createClient` pushes -// `fine-grained-tool-streaming-2025-05-14` / `interleaved-thinking-2025-05-14` -// from its own flags, and prepends `claude-code-20250219` / -// `oauth-2025-04-20` whenever the key looks like an OAuth token -// (`apiKey.includes('sk-ant-oat')`). This resolves LLP 0148's open question -// and LLP 0161 Section 10's first item: the mirror shrinks. OpenClaw only -// installs its own beta wrapper when the user configured `anthropicBeta` or -// opted into `context1m`, so `shouldMirrorAnthropicBetas` mirrors that -// installation condition too. Mirroring unconditionally would be a parity -// change in the other direction: pi-ai deliberately omits the interleaved -// beta on adaptive-thinking models, and an unconditional merge would put it -// back on the wire for a turn that would not have carried it unsteered. -// 2. **The merge must be a union, not an append.** pi-ai merges -// `options.headers` over its own defaults with `Object.assign`, so any -// `anthropic-beta` this mirror sets *replaces* pi-ai's computed value -// outright. The union below therefore has to carry the full set, and being -// a `Set` keyed by header name it stays idempotent: merging the same inputs -// twice is a no-op, so a future OpenClaw or pi-ai release that starts -// setting one of these itself cannot produce a duplicate (LLP 0157 R4). -// 3. **`service_tier` is a payload field, not a header**, and OpenClaw gates it -// on `provider === 'anthropic'` *and* a public Anthropic endpoint. Both -// gates fail for a shadow provider pointed at a loopback gateway, which is -// exactly the "refuses to act unless the base URL is public" loss LLP 0148 -// names. See `createAnthropicServiceTierWrapper` below for why this mirror -// inverts the endpoint gate rather than copying it. - -/** - * @import { ProviderWrapStreamFnContext, StreamFn, StreamOptions, WireModel } from './types.js' - */ - -/** - * OpenClaw's `OPENCLAW_DEFAULT_ANTHROPIC_BETAS`. - * @type {readonly string[]} - */ -export const ANTHROPIC_DEFAULT_BETAS = Object.freeze([ - 'fine-grained-tool-streaming-2025-05-14', - 'interleaved-thinking-2025-05-14', -]) - -/** - * OpenClaw's `OPENCLAW_OAUTH_ANTHROPIC_BETAS`: the OAuth-only additions come - * first and the defaults still apply, so this is a superset, never a swap. - * @type {readonly string[]} - */ -export const ANTHROPIC_OAUTH_BETAS = Object.freeze([ - 'claude-code-20250219', - 'oauth-2025-04-20', - ...ANTHROPIC_DEFAULT_BETAS, -]) - -/** - * The `context-1m` opt-in beta. LLP 0157's verified header list names it as - * "per the user's opt-in, excluded under OAuth", but the shipped OpenClaw - * calls it `ANTHROPIC_CONTEXT_1M_BETA_LEGACY` and strips it from every emitted - * set unconditionally (both from configured betas and from the merged list) - - * 1M context is GA on the model families listed below, so the beta header is - * no longer the opt-in switch. Parity means matching what OpenClaw actually - * puts on the wire, so this mirror strips it too. The opt-in itself is not - * ignored: `extraParams.context1m` still decides whether the mirror runs at - * all, exactly as it decides whether OpenClaw's own wrapper is installed. - */ -export const ANTHROPIC_CONTEXT_1M_BETA = 'context-1m-2025-08-07' - -/** OpenClaw's `ANTHROPIC_GA_1M_MODEL_PREFIXES`. */ -const ANTHROPIC_GA_1M_MODEL_PREFIXES = Object.freeze([ - 'claude-opus-4-8', - 'claude-opus-4.8', - 'claude-opus-4-6', - 'claude-opus-4.6', - 'claude-opus-4-7', - 'claude-opus-4.7', - 'claude-sonnet-4-6', - 'claude-sonnet-4.6', -]) - -/** - * OpenClaw resolves these two families through `resolveClaudeFable5ModelIdentity` - * / `resolveClaudeSonnet5ModelIdentity`, which normalize cloud ids and - * deployment metadata before matching. Those helpers are OpenClaw internals - * with no plugin-SDK export, so the mirror carries their id regexes. - */ -const CLAUDE_FABLE_5_MODEL_RE = /(?:^|-)claude-fable-5(?=$|[^a-z0-9])/ -const CLAUDE_SONNET_5_MODEL_RE = /(?:^|-)claude-sonnet-5(?=$|[^a-z0-9])/ - -/** - * @param {unknown} value - * @returns {string} - */ -function normalizeLower(value) { - return typeof value === 'string' ? value.trim().toLowerCase() : '' -} - -/** - * OpenClaw's `isAnthropicOAuthApiKey`, and the same predicate pi-ai uses to - * decide Bearer auth and the OAuth beta set. The borrowed credential - * (LLP 0145) is what reaches this, so a subscription-auth OpenClaw user is - * recognized as OAuth on the steered path exactly as on the unsteered one. - * - * @param {unknown} apiKey - * @returns {boolean} - */ -export function isAnthropicOAuthApiKey(apiKey) { - return typeof apiKey === 'string' && apiKey.includes('sk-ant-oat') -} - -/** - * @param {unknown} value - * @returns {string[]} - */ -export function parseHeaderList(value) { - if (typeof value !== 'string') return [] - return value - .split(',') - .map((item) => item.trim()) - .filter(Boolean) -} - -/** - * The idempotent merge (LLP 0157 R4): the header *name* is matched - * case-insensitively so an existing spelling is extended rather than - * shadowed by a second key, and the values are unioned through a `Set` so - * merging the same inputs twice changes nothing. - * - * @param {Record | undefined} headers - * @param {string} name - * @param {readonly string[]} values - * @returns {Record} - */ -export function mergeHeaderList(headers, name, values) { - const merged = { ...headers } - const wanted = normalizeLower(name) - const existingKey = Object.keys(merged).find((key) => normalizeLower(key) === wanted) - const existing = existingKey ? parseHeaderList(merged[existingKey]) : [] - merged[existingKey ?? name] = [...new Set([...existing, ...values])].join(',') - return merged -} - -/** - * OpenClaw's `mergeAnthropicBetaHeader`. - * - * @param {Record | undefined} headers - * @param {readonly string[]} betas - * @returns {Record} - */ -export function mergeAnthropicBetaHeader(headers, betas) { - return mergeHeaderList(headers, 'anthropic-beta', betas) -} - -/** - * True when the model's `extraParams` carry a non-blank `anthropicBeta`, - * mirroring OpenClaw's `hasConfiguredAnthropicBeta`. Distinct from - * `resolveConfiguredAnthropicBetas` returning a value: a user who configured - * only `context-1m` has a configured beta whose resolved list is empty, and - * OpenClaw still installs its wrapper for them. - * - * @param {Record | undefined} extraParams - * @returns {boolean} - */ -export function hasConfiguredAnthropicBeta(extraParams) { - const configured = extraParams?.anthropicBeta - if (typeof configured === 'string') return configured.trim().length > 0 - if (!Array.isArray(configured)) return false - return configured.some((beta) => typeof beta === 'string' && beta.trim().length > 0) -} - -/** - * OpenClaw's `resolveAnthropicBetas`: read `extraParams.anthropicBeta` as - * either a comma list or an array of comma lists, dropping the legacy - * `context-1m` entry. - * - * @param {Record | undefined} extraParams - * @returns {string[] | undefined} - */ -export function resolveConfiguredAnthropicBetas(extraParams) { - /** @type {Set} */ - const betas = new Set() - const configured = extraParams?.anthropicBeta - if (typeof configured === 'string' && configured.trim()) { - for (const beta of parseHeaderList(configured)) betas.add(beta) - } else if (Array.isArray(configured)) { - for (const entry of configured) { - if (typeof entry !== 'string' || !entry.trim()) continue - for (const beta of parseHeaderList(entry)) betas.add(beta) - } - } - betas.delete(ANTHROPIC_CONTEXT_1M_BETA) - return betas.size > 0 ? [...betas] : undefined -} - -/** - * OpenClaw's `isAnthropic1MModel`. - * - * @param {string | undefined} modelId - * @returns {boolean} - */ -export function isAnthropic1MModel(modelId) { - const normalized = normalizeLower(modelId) - if (CLAUDE_FABLE_5_MODEL_RE.test(normalized) || CLAUDE_SONNET_5_MODEL_RE.test(normalized)) return true - return ANTHROPIC_GA_1M_MODEL_PREFIXES.some((prefix) => normalized.startsWith(prefix)) -} - -/** - * OpenClaw's `needsAnthropicBetaWrapper`. Mirroring the *installation* - * condition, not just the merged set, is what keeps parity honest in both - * directions: when it is false, pi-ai's own beta computation is exactly what - * an unsteered turn would have produced, and touching the header would change - * the wire rather than preserve it. - * - * @param {Record | undefined} extraParams - * @param {string | undefined} modelId - * @returns {boolean} - */ -export function shouldMirrorAnthropicBetas(extraParams, modelId) { - if (resolveConfiguredAnthropicBetas(extraParams) !== undefined) return true - if (hasConfiguredAnthropicBeta(extraParams)) return true - return extraParams?.context1m === true && isAnthropic1MModel(modelId) -} - -/** - * OpenClaw's `createAnthropicBetaHeadersWrapper`. - * - * @param {StreamFn} baseStreamFn - * @param {readonly string[]} betas - * @returns {StreamFn} - */ -export function createAnthropicBetaHeadersWrapper(baseStreamFn, betas) { - return (model, context, options) => { - const effective = betas.filter((beta) => beta !== ANTHROPIC_CONTEXT_1M_BETA) - const base = isAnthropicOAuthApiKey(options?.apiKey) ? ANTHROPIC_OAUTH_BETAS : ANTHROPIC_DEFAULT_BETAS - const all = [...new Set([...base, ...effective])] - return baseStreamFn(model, context, { - ...options, - headers: mergeAnthropicBetaHeader(options?.headers, all), - }) - } -} - -/** - * OpenClaw's `streamWithPayloadPatch`, built on `pi-ai`'s public `onPayload` - * hook ("inspecting or replacing provider payloads before sending") rather - * than any OpenClaw internal, so the mirror needs no private import. - * - * @param {StreamFn} underlying - * @param {WireModel} model - * @param {unknown} context - * @param {StreamOptions | undefined} options - * @param {(payload: Record) => void} patchPayload - */ -function streamWithPayloadPatch(underlying, model, context, options, patchPayload) { - const originalOnPayload = options?.onPayload - return underlying(model, context, { - ...options, - onPayload: (payload, forModel) => { - if (payload && typeof payload === 'object') patchPayload(/** @type {Record} */ (payload)) - return originalOnPayload?.(payload, forModel) - }, - }) -} - -/** - * OpenClaw's `normalizeAnthropicServiceTier`. - * - * @param {unknown} value - * @returns {'auto' | 'standard_only' | undefined} - */ -export function normalizeAnthropicServiceTier(value) { - const normalized = normalizeLower(value) - if (normalized === 'auto' || normalized === 'standard_only') return normalized - return undefined -} - -/** - * OpenClaw's `resolveAnthropicServiceTier`, minus the invalid-value warning - * (this package has no OpenClaw subsystem logger; the pass-through ledger is - * the plugin's only emission surface and an invalid tier is not an uncaptured - * turn). - * - * @param {Record | undefined} extraParams - * @returns {'auto' | 'standard_only' | undefined} - */ -export function resolveAnthropicServiceTier(extraParams) { - return normalizeAnthropicServiceTier(extraParams?.serviceTier ?? extraParams?.service_tier) -} - -const FAST_MODE_FALSE = Object.freeze(['off', 'false', 'no', '0', 'disable', 'disabled', 'normal']) -const FAST_MODE_TRUE = Object.freeze(['on', 'true', 'yes', '1', 'enable', 'enabled', 'fast']) - -/** - * OpenClaw's `normalizeFastMode`. - * - * @param {unknown} raw - * @returns {boolean | 'auto' | undefined} - */ -export function normalizeFastMode(raw) { - if (typeof raw === 'boolean') return raw - if (!raw) return undefined - const key = normalizeLower(raw) - if (FAST_MODE_FALSE.includes(key)) return false - if (FAST_MODE_TRUE.includes(key)) return true - if (key === 'auto' || key === 'automatic') return 'auto' - return undefined -} - -/** - * OpenClaw's `resolveAnthropicFastMode`: `auto` means "leave it to the - * server", which is why it collapses to `undefined` rather than to a tier. - * - * @param {Record | undefined} extraParams - * @returns {boolean | undefined} - */ -export function resolveAnthropicFastMode(extraParams) { - const raw = extraParams?.fastMode ?? extraParams?.fast_mode - const normalized = normalizeFastMode(typeof raw === 'function' ? raw() : raw) - return normalized === 'auto' ? undefined : normalized -} - -/** - * OpenClaw's `resolveAnthropicFastServiceTier`. - * - * @param {boolean} enabled - * @returns {'auto' | 'standard_only'} - */ -export function resolveAnthropicFastServiceTier(enabled) { - return enabled ? 'auto' : 'standard_only' -} - -/** - * OpenClaw's `createAnthropicServiceTierWrapper`, with one deliberate - * inversion. OpenClaw gates the payload patch on - * `allowsAnthropicServiceTier`, which is - * `provider === 'anthropic' && api === 'anthropic-messages' && endpointClass - * is default|anthropic-public`. Copying that gate verbatim would make this - * mirror a no-op forever: the provider is `hypaware-anthropic` and the base - * URL is a loopback gateway, so both halves fail by construction. The gate - * exists to keep `service_tier` off requests bound for an endpoint that does - * not understand it, and here the true forward target *is* the public - * Anthropic API - the gateway's `anthropic` upstream preset carries a static - * `base_url` of `https://api.anthropic.com` and no per-request retarget - * (LLP 0161#steering-precedence), which is the same fact that makes steering - * safe at all. So the mirror keeps the two conditions it can still evaluate - * honestly (the shape, and the OAuth / Sonnet 5 carve-outs) and treats the - * endpoint as public. - * - * @param {StreamFn} baseStreamFn - * @param {'auto' | 'standard_only'} serviceTier - * @returns {StreamFn} - */ -export function createAnthropicServiceTierWrapper(baseStreamFn, serviceTier) { - return (model, context, options) => { - if (isAnthropicOAuthApiKey(options?.apiKey)) return baseStreamFn(model, context, options) - if (CLAUDE_SONNET_5_MODEL_RE.test(normalizeLower(model?.id))) return baseStreamFn(model, context, options) - if (normalizeLower(model?.api) !== 'anthropic-messages') return baseStreamFn(model, context, options) - return streamWithPayloadPatch(baseStreamFn, model, context, options, (payload) => { - if (payload.service_tier === undefined) payload.service_tier = serviceTier - }) - } -} - -/** - * OpenClaw's `createAnthropicFastModeWrapper`: the setting is read per call - * (it can be toggled mid-session), and an unresolvable value leaves the - * payload alone rather than picking a tier. - * - * @param {StreamFn} baseStreamFn - * @param {() => boolean | undefined} enabled - * @returns {StreamFn} - */ -export function createAnthropicFastModeWrapper(baseStreamFn, enabled) { - return (model, context, options) => { - const resolved = enabled() - if (resolved === undefined) return baseStreamFn(model, context, options) - return createAnthropicServiceTierWrapper(baseStreamFn, resolveAnthropicFastServiceTier(resolved))( - model, - context, - options, - ) - } -} - -/** - * OpenClaw's `stripTrailingAssistantPrefillMessages` / - * `stripTrailingAnthropicAssistantPrefillWhenThinking`: extended thinking - * requires the conversation to end on a user turn, so a trailing assistant - * prefill is an API error rather than a degraded response. A tool-use block - * is not a prefill and stops the walk. - * - * @param {Record} payload - * @returns {number} how many trailing prefill messages were removed - */ -export function stripTrailingAnthropicAssistantPrefillWhenThinking(payload) { - const thinking = payload.thinking - if (!thinking || typeof thinking !== 'object') return 0 - if (/** @type {Record} */ (thinking).type === 'disabled') return 0 - const messages = payload.messages - if (!Array.isArray(messages)) return 0 - let stripped = 0 - while (messages.length > 0) { - const last = messages[messages.length - 1] - if (!last || typeof last !== 'object') break - const message = /** @type {Record} */ (last) - if (message.role !== 'assistant' || hasAnthropicToolUse(message)) break - messages.pop() - stripped += 1 - } - return stripped -} - -/** - * @param {Record} message - * @returns {boolean} - */ -function hasAnthropicToolUse(message) { - if (Array.isArray(message.tool_calls) && message.tool_calls.length > 0) return true - const content = message.content - if (!Array.isArray(content)) return false - return content.some((block) => { - if (!block || typeof block !== 'object') return false - const type = /** @type {Record} */ (block).type - return type === 'tool_use' || type === 'toolCall' - }) -} - -/** - * OpenClaw's `createAnthropicThinkingPrefillWrapper`. Installed - * unconditionally there, so unconditionally here. - * - * @param {StreamFn} baseStreamFn - * @param {(stripped: number) => void} [onStripped] - * @returns {StreamFn} - */ -export function createAnthropicThinkingPrefillWrapper(baseStreamFn, onStripped) { - return (model, context, options) => - streamWithPayloadPatch(baseStreamFn, model, context, options, (payload) => { - const stripped = stripTrailingAnthropicAssistantPrefillWhenThinking(payload) - if (stripped > 0) onStripped?.(stripped) - }) -} - -/** - * The `wrapStreamFn` entry point for `hypaware-anthropic`, composing the same - * wrappers in the same order as OpenClaw's `wrapAnthropicProviderStream`. - * - * Returns `undefined` when there is no base stream function to wrap. OpenClaw's - * own wrappers fall back to `pi-ai`'s `streamSimple` here; this package refuses - * to take a dependency on `pi-ai` to reproduce a default it would then have to - * keep in step, so it declines to wrap instead. `onSkipped` makes that an - * observable event rather than a silent parity gap, which is the only way - * LLP 0148's "parity gaps are bugs, never accepted losses" rule can be - * enforced after ship. - * - * @ref LLP 0157#steering-plugin [implements]: the concrete mirror (default - * betas, OAuth additions, the context-1m opt-in, service_tier) verified - * against `extensions/anthropic/stream-wrappers.ts`. - * - * @param {ProviderWrapStreamFnContext} ctx - * @param {{ onSkipped?: (reason: string) => void, onPrefillStripped?: (stripped: number) => void }} [hooks] - * @returns {StreamFn | undefined} - */ -export function wrapAnthropicShadowStream(ctx, hooks = {}) { - const baseStreamFn = ctx.streamFn - if (typeof baseStreamFn !== 'function') { - hooks.onSkipped?.('no_base_stream_fn') - return undefined - } - - const extraParams = ctx.extraParams - const modelId = ctx.modelId ?? ctx.model?.id - - let streamFn = baseStreamFn - if (shouldMirrorAnthropicBetas(extraParams, modelId)) { - streamFn = createAnthropicBetaHeadersWrapper(streamFn, resolveConfiguredAnthropicBetas(extraParams) ?? []) - } - - const serviceTier = resolveAnthropicServiceTier(extraParams) - if (serviceTier) { - streamFn = createAnthropicServiceTierWrapper(streamFn, serviceTier) - } - - if (extraParams !== undefined && (Object.hasOwn(extraParams, 'fastMode') || Object.hasOwn(extraParams, 'fast_mode'))) { - streamFn = createAnthropicFastModeWrapper(streamFn, () => resolveAnthropicFastMode(extraParams)) - } - - return createAnthropicThinkingPrefillWrapper(streamFn, hooks.onPrefillStripped) -} diff --git a/openclaw-steering-plugin/test/gateway_endpoint.test.js b/openclaw-steering-plugin/test/gateway_endpoint.test.js deleted file mode 100644 index 841be9ec..00000000 --- a/openclaw-steering-plugin/test/gateway_endpoint.test.js +++ /dev/null @@ -1,21 +0,0 @@ -import assert from 'node:assert/strict' -import { test } from 'node:test' - -import { - DEFAULT_GATEWAY_ENDPOINT, - GATEWAY_ENDPOINT_ENV_VAR, - resolveGatewayEndpoint, -} from '../src/gateway_endpoint.js' - -test('resolveGatewayEndpoint: falls back to the fixed default when unset', () => { - assert.equal(resolveGatewayEndpoint({}), DEFAULT_GATEWAY_ENDPOINT) -}) - -test('resolveGatewayEndpoint: falls back when the env var is blank', () => { - assert.equal(resolveGatewayEndpoint({ [GATEWAY_ENDPOINT_ENV_VAR]: ' ' }), DEFAULT_GATEWAY_ENDPOINT) -}) - -test('resolveGatewayEndpoint: uses the configured value, trimmed', () => { - const endpoint = resolveGatewayEndpoint({ [GATEWAY_ENDPOINT_ENV_VAR]: ' http://127.0.0.1:19999 ' }) - assert.equal(endpoint, 'http://127.0.0.1:19999') -}) diff --git a/openclaw-steering-plugin/test/runtime_auth.test.js b/openclaw-steering-plugin/test/runtime_auth.test.js deleted file mode 100644 index 1d8b5ff6..00000000 --- a/openclaw-steering-plugin/test/runtime_auth.test.js +++ /dev/null @@ -1,196 +0,0 @@ -import assert from 'node:assert/strict' -import { test } from 'node:test' - -import { - BORROWED_OAUTH_REVALIDATE_MS, - REAL_PROVIDER_FOR_SHADOW, - SYNTHETIC_AUTH_MARKER, - createPrepareRuntimeAuth, - normalizeBorrowedCredential, - resolveShadowSyntheticAuth, -} from '../src/runtime_auth.js' - -const GATEWAY = 'http://127.0.0.1:18521' - -/** - * @param {{ provider: string }} ctx - */ -function contextFor(ctx) { - return { provider: ctx.provider, modelId: 'claude-sonnet-4-6', authMode: 'api-key' } -} - -test('REAL_PROVIDER_FOR_SHADOW is derived from the steering maps, not restated', () => { - assert.deepEqual(REAL_PROVIDER_FOR_SHADOW, { - 'hypaware-anthropic': 'anthropic', - 'hypaware-openai': 'openai', - }) -}) - -test('normalizeBorrowedCredential reads OpenClaw ResolvedProviderAuth records', () => { - assert.deepEqual(normalizeBorrowedCredential({ apiKey: 'sk-ant-abc', mode: 'api-key', source: 'env' }), { - apiKey: 'sk-ant-abc', - mode: 'api-key', - profileId: undefined, - source: 'env', - }) -}) - -test('normalizeBorrowedCredential treats a keyless resolution as no credential', () => { - // The SDK always resolves to an object, so "no credential" is a missing or - // blank `apiKey`, never a falsy return. - assert.equal(normalizeBorrowedCredential({ mode: 'api-key', source: 'none' }), undefined) - assert.equal(normalizeBorrowedCredential({ apiKey: ' ', mode: 'api-key' }), undefined) - assert.equal(normalizeBorrowedCredential(undefined), undefined) - assert.equal(normalizeBorrowedCredential(null), undefined) -}) - -test('normalizeBorrowedCredential accepts a bare key string', () => { - assert.deepEqual(normalizeBorrowedCredential('sk-ant-abc'), { apiKey: 'sk-ant-abc' }) - assert.equal(normalizeBorrowedCredential(' '), undefined) -}) - -test('resolveShadowSyntheticAuth only answers for providers this plugin owns', () => { - assert.deepEqual(resolveShadowSyntheticAuth({ provider: 'hypaware-anthropic' }), { - apiKey: SYNTHETIC_AUTH_MARKER, - source: 'hypaware-openclaw-steering (borrowed at request time)', - mode: 'api-key', - }) - assert.equal(resolveShadowSyntheticAuth({ provider: 'anthropic' }), undefined) - assert.equal(resolveShadowSyntheticAuth({ provider: 'openrouter' }), undefined) -}) - -test('prepareRuntimeAuth borrows the shadowed provider credential and pins the gateway baseUrl', async () => { - /** @type {string[]} */ - const asked = [] - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: ({ provider }) => { - asked.push(provider) - return { apiKey: 'sk-ant-real', mode: 'api-key', source: 'auth-profile' } - }, - }) - - const prepared = await prepare(contextFor({ provider: 'hypaware-anthropic' })) - - assert.deepEqual(asked, ['anthropic'], 'borrows for the real provider, not the shadow') - assert.deepEqual(prepared, { apiKey: 'sk-ant-real', baseUrl: GATEWAY }) -}) - -test('prepareRuntimeAuth borrows openai for the openai-completions shadow', async () => { - /** @type {string[]} */ - const asked = [] - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: ({ provider }) => { - asked.push(provider) - return { apiKey: 'sk-openai', mode: 'api-key' } - }, - }) - - await prepare(contextFor({ provider: 'hypaware-openai' })) - - assert.deepEqual(asked, ['openai']) -}) - -test('prepareRuntimeAuth re-resolves on every call and caches nothing', async () => { - let calls = 0 - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: () => { - calls += 1 - return { apiKey: `sk-ant-${calls}`, mode: 'api-key' } - }, - }) - - const first = await prepare(contextFor({ provider: 'hypaware-anthropic' })) - const second = await prepare(contextFor({ provider: 'hypaware-anthropic' })) - - assert.equal(calls, 2) - assert.equal(first?.apiKey, 'sk-ant-1') - assert.equal(second?.apiKey, 'sk-ant-2', 'a rotated credential is picked up, so nothing was cached') -}) - -test('prepareRuntimeAuth declares a re-resolution deadline for OAuth borrows only', async () => { - const now = () => 1_000_000 - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - now, - resolveCredential: () => ({ apiKey: 'sk-ant-oat01-live', mode: 'oauth' }), - }) - - const prepared = await prepare(contextFor({ provider: 'hypaware-anthropic' })) - assert.equal(prepared?.expiresAt, now() + BORROWED_OAUTH_REVALIDATE_MS) - - const apiKeyPrepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - now, - resolveCredential: () => ({ apiKey: 'sk-ant-api03', mode: 'api-key' }), - }) - assert.equal((await apiKeyPrepare(contextFor({ provider: 'hypaware-anthropic' })))?.expiresAt, undefined) -}) - -test('the OAuth re-resolution deadline stays clear of OpenClaw refresh margin', () => { - // OpenClaw schedules the re-preparation at `expiresAt - RUNTIME_AUTH_REFRESH_MARGIN_MS` - // (5 minutes) and clamps to a 5-second floor, so a TTL at or below the - // margin would re-prepare in a tight loop. - const OPENCLAW_REFRESH_MARGIN_MS = 5 * 60 * 1000 - assert.ok(BORROWED_OAUTH_REVALIDATE_MS > 2 * OPENCLAW_REFRESH_MARGIN_MS) -}) - -test('prepareRuntimeAuth declines for a provider this plugin does not shadow', async () => { - let called = false - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: () => { - called = true - return { apiKey: 'sk-nope', mode: 'api-key' } - }, - }) - - assert.equal(await prepare(contextFor({ provider: 'anthropic' })), undefined) - assert.equal(called, false, 'never borrows on behalf of a provider it does not own') -}) - -test('prepareRuntimeAuth returns undefined when nothing can be borrowed', async () => { - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: () => ({ mode: 'api-key', source: 'none' }), - }) - - assert.equal(await prepare(contextFor({ provider: 'hypaware-anthropic' })), undefined) -}) - -test('prepareRuntimeAuth reports a throwing resolver instead of failing the turn', async () => { - /** @type {Array<{ provider: string, error: unknown }>} */ - const reported = [] - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: () => { - throw new Error('auth profile store unavailable') - }, - onError: (info) => reported.push(info), - }) - - assert.equal(await prepare(contextFor({ provider: 'hypaware-anthropic' })), undefined) - assert.equal(reported.length, 1) - assert.equal(reported[0].provider, 'anthropic') -}) - -test('prepareRuntimeAuth never returns anything but the request-scoped triple', async () => { - const prepare = createPrepareRuntimeAuth({ - baseUrl: GATEWAY, - resolveCredential: () => ({ - apiKey: 'sk-ant-oat01-live', - mode: 'oauth', - profileId: 'anthropic-oauth', - source: 'auth-profiles.json', - }), - }) - - const prepared = await prepare(contextFor({ provider: 'hypaware-anthropic' })) - - // Profile id and source are resolution provenance, not request material: - // leaking them into the prepared auth would put credential-adjacent - // metadata somewhere OpenClaw persists (LLP 0145: never persist a borrow). - assert.deepEqual(Object.keys(prepared ?? {}).sort(), ['apiKey', 'baseUrl', 'expiresAt']) -}) diff --git a/openclaw-steering-plugin/test/steering.test.js b/openclaw-steering-plugin/test/steering.test.js deleted file mode 100644 index bcc9b134..00000000 --- a/openclaw-steering-plugin/test/steering.test.js +++ /dev/null @@ -1,150 +0,0 @@ -import assert from 'node:assert/strict' -import { test } from 'node:test' - -import { - CANONICAL_PROVIDER_FOR_SHAPE, - DEFERRED_SET, - SHADOW_FOR_SHAPE, - WARNING_CAUSES, - resolveSteering, - tryResolveApiKeyForProvider, -} from '../src/steering.js' - -/** - * @param {string|undefined} credential - */ -function ctxReturning(credential) { - return { resolveCredential: async () => credential } -} - -function ctxThrowing() { - return { - resolveCredential: async () => { - throw new Error('boom') - }, - } -} - -test('resolveSteering: steers a canonical anthropic candidate with a resolvable credential', async () => { - const result = await resolveSteering( - { provider: 'anthropic', api: 'anthropic-messages' }, - ctxReturning('sk-ant-abc123'), - ) - - assert.deepEqual(result, { - steer: true, - providerOverride: SHADOW_FOR_SHAPE['anthropic-messages'], - requestMeta: { 'x-hypaware-upstream': 'anthropic' }, - }) -}) - -test('resolveSteering: steers a canonical openai candidate with a resolvable credential', async () => { - const result = await resolveSteering( - { provider: 'openai', api: 'openai-completions' }, - ctxReturning('sk-openai-abc123'), - ) - - assert.deepEqual(result, { - steer: true, - providerOverride: SHADOW_FOR_SHAPE['openai-completions'], - requestMeta: { 'x-hypaware-upstream': 'openai' }, - }) -}) - -test('resolveSteering: warns no_preset when no shadow covers the api shape at all', async () => { - const result = await resolveSteering( - { provider: 'anthropic', api: 'some-future-shape' }, - ctxReturning('sk-ant-abc123'), - ) - - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.NO_PRESET, provider: 'anthropic' }) -}) - -test('resolveSteering: warns deferred for every DEFERRED_SET member sharing a canonical shape', async () => { - const deferredAnthropicShaped = ['anthropic-vertex', 'cloudflare-ai-gateway', 'vercel-ai-gateway'] - - for (const provider of deferredAnthropicShaped) { - assert.ok(DEFERRED_SET.has(provider), `expected ${provider} in DEFERRED_SET`) - const result = await resolveSteering({ provider, api: 'anthropic-messages' }, ctxReturning('irrelevant')) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.DEFERRED, provider }) - } -}) - -test('resolveSteering: warns deferred for the Google family sharing the openai-completions shape', async () => { - for (const provider of ['google', 'google-vertex', 'google-gemini-cli']) { - assert.ok(DEFERRED_SET.has(provider), `expected ${provider} in DEFERRED_SET`) - const result = await resolveSteering({ provider, api: 'openai-completions' }, ctxReturning('irrelevant')) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.DEFERRED, provider }) - } -}) - -test('resolveSteering: warns deferred for amazon-bedrock even though its shape is unspecified in LLP 0146, using the anthropic-messages shape it is verified to declare', async () => { - assert.ok(DEFERRED_SET.has('amazon-bedrock')) - const result = await resolveSteering( - { provider: 'amazon-bedrock', api: 'anthropic-messages' }, - ctxReturning('irrelevant'), - ) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.DEFERRED, provider: 'amazon-bedrock' }) -}) - -test('resolveSteering: warns no_preset for a shape-matching, non-canonical, non-deferred vendor (e.g. minimax, synthetic, kimi-coding)', async () => { - for (const provider of ['minimax', 'synthetic', 'kimi-coding']) { - assert.ok(!DEFERRED_SET.has(provider), `expected ${provider} to NOT be in DEFERRED_SET`) - const result = await resolveSteering({ provider, api: 'anthropic-messages' }, ctxReturning('irrelevant')) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.NO_PRESET, provider }) - } -}) - -test('resolveSteering: warns no_preset for a shape-matching, non-canonical, non-deferred openai-family vendor', async () => { - const result = await resolveSteering( - { provider: 'openrouter', api: 'openai-completions' }, - ctxReturning('irrelevant'), - ) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.NO_PRESET, provider: 'openrouter' }) -}) - -test('resolveSteering: warns no_credential when the canonical provider has no resolvable credential', async () => { - const result = await resolveSteering({ provider: 'anthropic', api: 'anthropic-messages' }, ctxReturning(undefined)) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.NO_CREDENTIAL, provider: 'anthropic' }) -}) - -test('resolveSteering: warns no_credential (not a throw) when credential resolution itself throws', async () => { - const result = await resolveSteering({ provider: 'openai', api: 'openai-completions' }, ctxThrowing()) - assert.deepEqual(result, { steer: false, cause: WARNING_CAUSES.NO_CREDENTIAL, provider: 'openai' }) -}) - -test('resolveSteering: credential check never runs for a candidate that already failed an earlier branch', async () => { - let called = false - const ctx = { - resolveCredential: async () => { - called = true - return 'sk-should-not-be-reached' - }, - } - - await resolveSteering({ provider: 'anthropic', api: 'some-future-shape' }, ctx) - assert.equal(called, false, 'no_preset (unknown shape) must not resolve a credential') - - await resolveSteering({ provider: 'minimax', api: 'anthropic-messages' }, ctx) - assert.equal(called, false, 'no_preset (wrong vendor) must not resolve a credential') - - await resolveSteering({ provider: 'anthropic-vertex', api: 'anthropic-messages' }, ctx) - assert.equal(called, false, 'deferred must not resolve a credential') -}) - -test('SHADOW_FOR_SHAPE and CANONICAL_PROVIDER_FOR_SHAPE cover exactly the two known shapes', () => { - assert.deepEqual(Object.keys(SHADOW_FOR_SHAPE).sort(), ['anthropic-messages', 'openai-completions']) - assert.deepEqual(Object.keys(CANONICAL_PROVIDER_FOR_SHAPE).sort(), ['anthropic-messages', 'openai-completions']) - assert.equal(SHADOW_FOR_SHAPE['anthropic-messages'], 'hypaware-anthropic') - assert.equal(SHADOW_FOR_SHAPE['openai-completions'], 'hypaware-openai') -}) - -test('tryResolveApiKeyForProvider: returns undefined for an empty-string credential, not the empty string', async () => { - const credential = await tryResolveApiKeyForProvider('anthropic', ctxReturning('')) - assert.equal(credential, undefined) -}) - -test('tryResolveApiKeyForProvider: passes a real credential through unchanged', async () => { - const credential = await tryResolveApiKeyForProvider('anthropic', ctxReturning('sk-ant-abc123')) - assert.equal(credential, 'sk-ant-abc123') -}) diff --git a/openclaw-steering-plugin/test/warning_ledger.test.js b/openclaw-steering-plugin/test/warning_ledger.test.js deleted file mode 100644 index 52ef7d7d..00000000 --- a/openclaw-steering-plugin/test/warning_ledger.test.js +++ /dev/null @@ -1,78 +0,0 @@ -import assert from 'node:assert/strict' -import { test } from 'node:test' - -import { createWarningLedger } from '../src/warning_ledger.js' - -test('createWarningLedger: emits the first warning for a provider+cause pair', () => { - const emitted = [] - const ledger = createWarningLedger({ emit: (record) => emitted.push(record) }) - - const wasEmitted = ledger.warn({ provider: 'anthropic-vertex', cause: 'deferred', session: 'sess-1' }) - - assert.equal(wasEmitted, true) - assert.equal(emitted.length, 1) - assert.deepEqual(emitted[0], { - component: 'openclaw-steering-plugin', - operation: 'before_model_resolve', - status: 'uncaptured', - provider: 'anthropic-vertex', - cause: 'deferred', - session: 'sess-1', - }) -}) - -test('createWarningLedger: suppresses a repeat within the window, keyed by provider+cause', () => { - const emitted = [] - let now = 0 - const ledger = createWarningLedger({ emit: (r) => emitted.push(r), now: () => now, windowMs: 1000 }) - - assert.equal(ledger.warn({ provider: 'anthropic', cause: 'no_credential' }), true) - now = 500 - assert.equal(ledger.warn({ provider: 'anthropic', cause: 'no_credential' }), false) - assert.equal(emitted.length, 1) -}) - -test('createWarningLedger: re-emits once the window elapses', () => { - const emitted = [] - let now = 0 - const ledger = createWarningLedger({ emit: (r) => emitted.push(r), now: () => now, windowMs: 1000 }) - - assert.equal(ledger.warn({ provider: 'anthropic', cause: 'no_credential' }), true) - now = 1000 - assert.equal(ledger.warn({ provider: 'anthropic', cause: 'no_credential' }), true) - assert.equal(emitted.length, 2) -}) - -test('createWarningLedger: a different cause for the same provider is a distinct key', () => { - const emitted = [] - const ledger = createWarningLedger({ emit: (r) => emitted.push(r) }) - - ledger.warn({ provider: 'anthropic', cause: 'no_credential' }) - ledger.warn({ provider: 'anthropic', cause: 'no_preset' }) - - assert.equal(emitted.length, 2) -}) - -test('createWarningLedger: a different provider for the same cause is a distinct key', () => { - const emitted = [] - const ledger = createWarningLedger({ emit: (r) => emitted.push(r) }) - - ledger.warn({ provider: 'anthropic', cause: 'no_preset' }) - ledger.warn({ provider: 'openai', cause: 'no_preset' }) - - assert.equal(emitted.length, 2) -}) - -test('createWarningLedger: defaults to console.warn when no emit is supplied', () => { - const original = console.warn - const calls = [] - console.warn = (...args) => calls.push(args) - try { - const ledger = createWarningLedger() - ledger.warn({ provider: 'anthropic', cause: 'no_preset' }) - assert.equal(calls.length, 1) - assert.equal(calls[0][0], '[hypaware-openclaw-steering] uncaptured provider turn') - } finally { - console.warn = original - } -}) diff --git a/openclaw-steering-plugin/test/wire_parity.test.js b/openclaw-steering-plugin/test/wire_parity.test.js deleted file mode 100644 index dbc67e4a..00000000 --- a/openclaw-steering-plugin/test/wire_parity.test.js +++ /dev/null @@ -1,292 +0,0 @@ -import assert from 'node:assert/strict' -import { test } from 'node:test' - -import { - ANTHROPIC_CONTEXT_1M_BETA, - ANTHROPIC_DEFAULT_BETAS, - ANTHROPIC_OAUTH_BETAS, - createAnthropicBetaHeadersWrapper, - createAnthropicServiceTierWrapper, - isAnthropic1MModel, - isAnthropicOAuthApiKey, - mergeAnthropicBetaHeader, - mergeHeaderList, - normalizeFastMode, - parseHeaderList, - resolveAnthropicFastMode, - resolveAnthropicServiceTier, - resolveConfiguredAnthropicBetas, - shouldMirrorAnthropicBetas, - stripTrailingAnthropicAssistantPrefillWhenThinking, - wrapAnthropicShadowStream, -} from '../src/wire_parity.js' - -const OAUTH_KEY = 'sk-ant-oat01-abc' -const API_KEY = 'sk-ant-api03-abc' - -/** - * Records what the underlying stream fn was finally called with. - */ -function recordingStreamFn() { - /** @type {Array<{ model: unknown, context: unknown, options: any }>} */ - const calls = [] - /** @type {any} */ - const streamFn = (model, context, options) => { - calls.push({ model, context, options }) - return 'stream' - } - return { streamFn, calls } -} - -function anthropicModel(id = 'claude-opus-4-6') { - return { id, api: 'anthropic-messages', provider: 'hypaware-anthropic', baseUrl: 'http://127.0.0.1:18521' } -} - -test('parseHeaderList splits, trims, and drops blanks', () => { - assert.deepEqual(parseHeaderList(' a , b ,, c '), ['a', 'b', 'c']) - assert.deepEqual(parseHeaderList(undefined), []) -}) - -test('mergeHeaderList unions values under the existing header name spelling', () => { - const merged = mergeHeaderList({ 'Anthropic-Beta': 'already-there' }, 'anthropic-beta', ['added']) - assert.deepEqual(merged, { 'Anthropic-Beta': 'already-there,added' }) -}) - -test('mergeHeaderList is idempotent', () => { - const once = mergeAnthropicBetaHeader({}, ANTHROPIC_DEFAULT_BETAS) - const twice = mergeAnthropicBetaHeader(once, ANTHROPIC_DEFAULT_BETAS) - assert.deepEqual(twice, once) - // R4: a future OpenClaw release that starts setting one of these itself - // must not produce a duplicate. - const withUpstream = mergeAnthropicBetaHeader( - { 'anthropic-beta': ANTHROPIC_DEFAULT_BETAS[0] }, - ANTHROPIC_DEFAULT_BETAS, - ) - assert.deepEqual(withUpstream, { 'anthropic-beta': ANTHROPIC_DEFAULT_BETAS.join(',') }) -}) - -test('mergeHeaderList leaves unrelated headers untouched', () => { - const merged = mergeHeaderList({ 'x-app': 'cli' }, 'anthropic-beta', ['b1']) - assert.deepEqual(merged, { 'x-app': 'cli', 'anthropic-beta': 'b1' }) -}) - -test('isAnthropicOAuthApiKey matches the predicate pi-ai and OpenClaw both use', () => { - assert.equal(isAnthropicOAuthApiKey(OAUTH_KEY), true) - assert.equal(isAnthropicOAuthApiKey(API_KEY), false) - assert.equal(isAnthropicOAuthApiKey(undefined), false) -}) - -test('the beta wrapper emits the default set under an api key', () => { - const { streamFn, calls } = recordingStreamFn() - createAnthropicBetaHeadersWrapper(streamFn, [])(anthropicModel(), {}, { apiKey: API_KEY }) - assert.equal(calls[0].options.headers['anthropic-beta'], ANTHROPIC_DEFAULT_BETAS.join(',')) -}) - -test('the beta wrapper adds the OAuth betas on top of the defaults, never instead of them', () => { - const { streamFn, calls } = recordingStreamFn() - createAnthropicBetaHeadersWrapper(streamFn, [])(anthropicModel(), {}, { apiKey: OAUTH_KEY }) - const emitted = parseHeaderList(calls[0].options.headers['anthropic-beta']) - assert.deepEqual(emitted, [...ANTHROPIC_OAUTH_BETAS]) - for (const beta of ANTHROPIC_DEFAULT_BETAS) assert.ok(emitted.includes(beta)) -}) - -test('the beta wrapper unions configured betas without duplicating the defaults', () => { - const { streamFn, calls } = recordingStreamFn() - createAnthropicBetaHeadersWrapper(streamFn, ['custom-beta', ANTHROPIC_DEFAULT_BETAS[0]])( - anthropicModel(), - {}, - { apiKey: API_KEY }, - ) - assert.deepEqual(parseHeaderList(calls[0].options.headers['anthropic-beta']), [ - ...ANTHROPIC_DEFAULT_BETAS, - 'custom-beta', - ]) -}) - -test('the legacy context-1m beta never reaches the wire, matching shipped OpenClaw', () => { - assert.equal(resolveConfiguredAnthropicBetas({ anthropicBeta: ANTHROPIC_CONTEXT_1M_BETA }), undefined) - const { streamFn, calls } = recordingStreamFn() - createAnthropicBetaHeadersWrapper(streamFn, [ANTHROPIC_CONTEXT_1M_BETA])(anthropicModel(), {}, { apiKey: API_KEY }) - assert.ok(!calls[0].options.headers['anthropic-beta'].includes(ANTHROPIC_CONTEXT_1M_BETA)) -}) - -test('resolveConfiguredAnthropicBetas reads a string or an array of comma lists', () => { - assert.deepEqual(resolveConfiguredAnthropicBetas({ anthropicBeta: 'a, b' }), ['a', 'b']) - assert.deepEqual(resolveConfiguredAnthropicBetas({ anthropicBeta: ['a,b', ' c '] }), ['a', 'b', 'c']) - assert.equal(resolveConfiguredAnthropicBetas({}), undefined) -}) - -test('isAnthropic1MModel covers the GA prefixes and the identity-resolved families', () => { - assert.equal(isAnthropic1MModel('claude-opus-4-6-20260101'), true) - assert.equal(isAnthropic1MModel('claude-sonnet-4.6'), true) - assert.equal(isAnthropic1MModel('vertex-claude-sonnet-5'), true) - assert.equal(isAnthropic1MModel('claude-fable-5-preview'), true) - assert.equal(isAnthropic1MModel('claude-3-5-haiku'), false) -}) - -test('the mirror installs only where OpenClaw installs its own beta wrapper', () => { - // Nothing configured: pi-ai already produces exactly the unsteered wire, so - // touching the header would change it rather than preserve it. - assert.equal(shouldMirrorAnthropicBetas(undefined, 'claude-opus-4-6'), false) - assert.equal(shouldMirrorAnthropicBetas({}, 'claude-opus-4-6'), false) - assert.equal(shouldMirrorAnthropicBetas({ anthropicBeta: 'custom' }, 'claude-opus-4-6'), true) - // Configured with only the legacy beta: resolves to no betas, but OpenClaw - // still installs, so the mirror does too. - assert.equal(shouldMirrorAnthropicBetas({ anthropicBeta: ANTHROPIC_CONTEXT_1M_BETA }, 'claude-opus-4-6'), true) - assert.equal(shouldMirrorAnthropicBetas({ context1m: true }, 'claude-opus-4-6'), true) - assert.equal(shouldMirrorAnthropicBetas({ context1m: true }, 'claude-3-5-haiku'), false) -}) - -test('wrapAnthropicShadowStream leaves headers alone when nothing is configured', () => { - const { streamFn, calls } = recordingStreamFn() - const wrapped = wrapAnthropicShadowStream({ provider: 'hypaware-anthropic', modelId: 'claude-opus-4-6', streamFn }) - wrapped?.(anthropicModel(), {}, { apiKey: OAUTH_KEY }) - assert.equal(calls[0].options.headers, undefined) -}) - -test('wrapAnthropicShadowStream merges betas once the user configured any', () => { - const { streamFn, calls } = recordingStreamFn() - const wrapped = wrapAnthropicShadowStream({ - provider: 'hypaware-anthropic', - modelId: 'claude-opus-4-6', - extraParams: { anthropicBeta: 'custom-beta' }, - streamFn, - }) - wrapped?.(anthropicModel(), {}, { apiKey: OAUTH_KEY }) - assert.deepEqual(parseHeaderList(calls[0].options.headers['anthropic-beta']), [ - ...ANTHROPIC_OAUTH_BETAS, - 'custom-beta', - ]) -}) - -test('wrapAnthropicShadowStream reports rather than silently skipping with no base stream fn', () => { - /** @type {string[]} */ - const skipped = [] - const wrapped = wrapAnthropicShadowStream( - { provider: 'hypaware-anthropic', modelId: 'claude-opus-4-6' }, - { onSkipped: (reason) => skipped.push(reason) }, - ) - assert.equal(wrapped, undefined) - assert.deepEqual(skipped, ['no_base_stream_fn']) -}) - -test('resolveAnthropicServiceTier reads both spellings and rejects unknown values', () => { - assert.equal(resolveAnthropicServiceTier({ serviceTier: 'auto' }), 'auto') - assert.equal(resolveAnthropicServiceTier({ service_tier: 'STANDARD_ONLY' }), 'standard_only') - assert.equal(resolveAnthropicServiceTier({ serviceTier: 'priority' }), undefined) - assert.equal(resolveAnthropicServiceTier(undefined), undefined) -}) - -test('normalizeFastMode and resolveAnthropicFastMode mirror OpenClaw coercion', () => { - assert.equal(normalizeFastMode('on'), true) - assert.equal(normalizeFastMode('disabled'), false) - assert.equal(normalizeFastMode('automatic'), 'auto') - assert.equal(resolveAnthropicFastMode({ fastMode: 'auto' }), undefined) - assert.equal(resolveAnthropicFastMode({ fast_mode: 'fast' }), true) - assert.equal(resolveAnthropicFastMode({ fastMode: () => false }), false) -}) - -/** - * Drives the payload patch the same way pi-ai does: call the wrapped stream - * fn, then invoke the `onPayload` hook it installed. - * - * @param {any} wrapped - * @param {any} payload - * @param {any} options - * @param {any} [model] - */ -function runPayload(wrapped, payload, options, model = anthropicModel()) { - const { streamFn, calls } = recordingStreamFn() - wrapped(streamFn)(model, {}, options) - calls[0].options.onPayload?.(payload, model) - return calls[0] -} - -test('the service tier wrapper patches the payload for an api-key turn', () => { - const payload = { messages: [] } - runPayload((base) => createAnthropicServiceTierWrapper(base, 'auto'), payload, { apiKey: API_KEY }) - assert.equal(payload.service_tier, 'auto') -}) - -test('the service tier wrapper never overwrites a tier already on the payload', () => { - const payload = { service_tier: 'standard_only' } - runPayload((base) => createAnthropicServiceTierWrapper(base, 'auto'), payload, { apiKey: API_KEY }) - assert.equal(payload.service_tier, 'standard_only') -}) - -test('the service tier wrapper keeps OpenClaw carve-outs: OAuth and Sonnet 5', () => { - const oauthPayload = {} - runPayload((base) => createAnthropicServiceTierWrapper(base, 'auto'), oauthPayload, { apiKey: OAUTH_KEY }) - assert.equal(oauthPayload.service_tier, undefined) - - const sonnet5Payload = {} - runPayload( - (base) => createAnthropicServiceTierWrapper(base, 'auto'), - sonnet5Payload, - { apiKey: API_KEY }, - anthropicModel('claude-sonnet-5-20260101'), - ) - assert.equal(sonnet5Payload.service_tier, undefined) -}) - -test('the service tier wrapper preserves a caller-supplied onPayload hook', () => { - const seen = [] - const payload = {} - runPayload((base) => createAnthropicServiceTierWrapper(base, 'auto'), payload, { - apiKey: API_KEY, - onPayload: (value) => seen.push(value), - }) - assert.equal(payload.service_tier, 'auto') - assert.deepEqual(seen, [payload]) -}) - -test('wrapAnthropicShadowStream applies fast mode as a service tier', () => { - const payload = {} - const { streamFn, calls } = recordingStreamFn() - const wrapped = wrapAnthropicShadowStream({ - provider: 'hypaware-anthropic', - modelId: 'claude-opus-4-6', - extraParams: { fastMode: true }, - streamFn, - }) - wrapped?.(anthropicModel(), {}, { apiKey: API_KEY }) - calls[0].options.onPayload?.(payload, anthropicModel()) - assert.equal(payload.service_tier, 'auto') -}) - -test('trailing assistant prefill is stripped only when thinking is enabled', () => { - const enabled = { - thinking: { type: 'enabled' }, - messages: [{ role: 'user' }, { role: 'assistant', content: [{ type: 'text' }] }], - } - assert.equal(stripTrailingAnthropicAssistantPrefillWhenThinking(enabled), 1) - assert.equal(enabled.messages.length, 1) - - const disabled = { - thinking: { type: 'disabled' }, - messages: [{ role: 'user' }, { role: 'assistant', content: [] }], - } - assert.equal(stripTrailingAnthropicAssistantPrefillWhenThinking(disabled), 0) - assert.equal(disabled.messages.length, 2) -}) - -test('a trailing assistant tool_use is not a prefill', () => { - const payload = { - thinking: { type: 'enabled' }, - messages: [{ role: 'user' }, { role: 'assistant', content: [{ type: 'tool_use' }] }], - } - assert.equal(stripTrailingAnthropicAssistantPrefillWhenThinking(payload), 0) - assert.equal(payload.messages.length, 2) -}) - -test('the prefill wrapper is always installed, even with nothing configured', () => { - const payload = { - thinking: { type: 'enabled' }, - messages: [{ role: 'user' }, { role: 'assistant', content: [] }], - } - const { streamFn, calls } = recordingStreamFn() - const wrapped = wrapAnthropicShadowStream({ provider: 'hypaware-anthropic', modelId: 'claude-opus-4-6', streamFn }) - wrapped?.(anthropicModel(), {}, { apiKey: API_KEY }) - calls[0].options.onPayload?.(payload, anthropicModel()) - assert.equal(payload.messages.length, 1) -}) diff --git a/src/core/commands/backfill.js b/src/core/commands/backfill.js index 1396ce49..e44b20fc 100644 --- a/src/core/commands/backfill.js +++ b/src/core/commands/backfill.js @@ -23,7 +23,7 @@ const BACKFILL_PARTITION_SEGMENT = 'backfill' /** * @import { BackfillContribution, BackfillItem, BackfillEvent, BackfillMaterializerContribution, BackfillPlan, BackfillPlanContext, BackfillRunContext, CommandRunContext, PluginLogger, PluginName } from '../../../hypaware-plugin-kernel-types.js' - * @import { BackfillProviderResult } from '../../../src/core/commands/types.js' + * @import { BackfillProviderResult, BackfillRunnerContext } from '../../../src/core/commands/types.js' * @import { EntrypointOwners } from '../../../src/core/backfill/types.js' */ @@ -299,7 +299,7 @@ export async function runBackfillPlan(argv, ctx) { * status line without a try/catch. * * @param {{ - * ctx: CommandRunContext, + * ctx: BackfillRunnerContext, * provider: string, * dryRun: boolean, * retentionDays?: number, @@ -361,7 +361,7 @@ function markProviderFailed(result, error) { * * @param {{ * provider: BackfillContribution, - * ctx: CommandRunContext, + * ctx: BackfillRunnerContext, * devRunId: string, * retentionDays: number | undefined, * since: string | undefined, @@ -502,7 +502,13 @@ async function runProvider(args) { if (!dryRun) { for (const dataset of datasetsTouched) { - await flushDataset({ dataset, provider: provider.name, devRunId, ctx, log }) + await flushDataset({ + dataset, + provider: provider.name, + devRunId, + ctx, + log, + }) } } } catch (err) { @@ -546,7 +552,7 @@ async function runProvider(args) { * @param {{ * materializer: BackfillMaterializerContribution, * item: BackfillItem, - * ctx: CommandRunContext, + * ctx: BackfillRunnerContext, * devRunId: string, * provider: string, * log: PluginLogger, @@ -591,7 +597,7 @@ async function materializeItem(args) { * dataset: string, * provider: string, * devRunId: string, - * ctx: CommandRunContext, + * ctx: BackfillRunnerContext, * log: PluginLogger, * }} args * @returns {Promise<{ rowsWritten: number, status: 'ok' | 'failed', error?: string }>} @@ -646,7 +652,7 @@ async function writeRows(args) { * dataset: string, * provider: string, * devRunId: string, - * ctx: CommandRunContext, + * ctx: BackfillRunnerContext, * log: PluginLogger, * }} args */ @@ -761,7 +767,7 @@ function buildRunContext(args) { * container was scanned at all. Degrading never captures MORE than * intended. * - * @param {CommandRunContext} ctx + * @param {BackfillRunnerContext} ctx * @param {PluginLogger} log * @returns {Promise<{ entrypointOwners: EntrypointOwners, isPluginConfigured?: (plugin: PluginName) => boolean }>} */ @@ -814,7 +820,13 @@ async function resolveOwnersForRun(ctx, log) { * state for it, not the `config.load_failed` error row that helper emits. * * @ref LLP 0140#manifest-declares-ownership [implements]: "configured" is membership of the effective config, read fresh, not of the boot profile's activation set - * @param {CommandRunContext} ctx + * @ref LLP 0172#lane-b-sweep [constrained-by]: `ctx.plugins` stays optional + * here (rather than `CommandRunContext`'s required array) so this helper + * keeps working unchanged under `resolveOwnersForRun`'s narrowed + * `BackfillRunnerContext`, which carries no activation set; the union + * already treats an absent source as "answers nothing," so a caller with + * no `plugins` field degrades to the other two sources, not a type error. + * @param {Pick & { plugins?: CommandRunContext['plugins'] }} ctx * @param {string} hypHome * @returns {Promise>} */ diff --git a/src/core/commands/clients.js b/src/core/commands/clients.js index b705ea95..0899e5d8 100644 --- a/src/core/commands/clients.js +++ b/src/core/commands/clients.js @@ -376,6 +376,57 @@ async function materializeAttachAssets({ name, descriptorMap, ctx, dryRun, json }) } +/** + * The gateway's own base origin, for the `json_path` undo's ownership check: + * that format's undo record is the entry attach wrote, so telling our entry + * from a user's is a comparison against this URL and nothing else + * (LLP 0172 §2.1). + * + * Same three rungs the manual attach path already walks, in the same order, + * so a detach can never decide ownership against a different origin than the + * attach it is reversing wrote: the live `localEndpoint()` first, the + * configured `listen` next, the running daemon's persisted bound port last. + * + * Every rung is optional, deliberately. Detach must keep working with the + * `@hypaware/ai-gateway` capability absent or unloaded (that is why the + * detach branch of `runClientLifecycle` runs ahead of the capability gate), + * so this resolves what it can and hands back `undefined` otherwise; the + * formats that carry their own marker do not need it, and the one that does + * refuses loudly rather than guessing. + * + * @param {CommandRunContext} ctx + * @returns {string | undefined} + * @ref LLP 0172#lane-a-detach [implements]: detachClientViaCore resolves the gateway base URL through the existing AiGatewayCapability lookup plus the manual path's fallbacks, and threads it into the one core undo + */ +function resolveExpectedGatewayBaseUrl(ctx) { + // Every rung is wrapped, and the whole walk is optional. Detach is reached + // with the capability registry, the config, or the state dir missing - that + // is the point of resolving the undo ahead of the capability gate - so a rung + // that cannot answer must yield to the next one rather than turn a detach + // into an internal error. + try { + if (ctx.capabilities?.has('hypaware.ai-gateway') === true) { + /** @type {AiGatewayCapability} */ + const gateway = ctx.capabilities.require('hyp-core', 'hypaware.ai-gateway', '^2.0.0') + const live = gateway.localEndpoint() + if (typeof live === 'string' && live.length > 0) return live + } + } catch { + // Registered but not bound in this process (the usual CLI case). + } + try { + const configured = ctx.config ? configuredGatewayEndpoint(ctx.config) : undefined + if (configured !== undefined) return configured + } catch { + // A config shape this helper cannot read is not a detach failure. + } + try { + return resolveLiveGatewayEndpointFromStatus({ stateRoot: readObservabilityEnv(ctx.env).stateDir }) + } catch { + return undefined + } +} + /** * Reverse a client's attach from disk: the single core undo * (`detachClientFromDisk`). The manual `hyp detach` command and the @@ -442,7 +493,12 @@ export async function detachClientViaCore({ name, descriptor, dryRun, json, ctx return } try { - const result = await detachClientFromDisk({ descriptor, homeDir, env: ctx.env }) + const result = await detachClientFromDisk({ + descriptor, + homeDir, + env: ctx.env, + expectedBaseUrl: resolveExpectedGatewayBaseUrl(ctx), + }) const restored = result.changed === true span.setAttribute('status', 'ok') span.setAttribute('restored', restored) diff --git a/src/core/commands/types.d.ts b/src/core/commands/types.d.ts index 6cbd6926..454dcc9a 100644 --- a/src/core/commands/types.d.ts +++ b/src/core/commands/types.d.ts @@ -1,4 +1,11 @@ import type { UsageClass } from '../usage-policy/types.d.ts' +import type { + BackfillMaterializerRegistry, + BackfillRegistry, + HypAwareV2Config, + QueryRegistry, + QueryStorageService, +} from '../../../hypaware-plugin-kernel-types.js' // How the shared machine-local marking writers (`runMarkMachineLocal`, // `runUnmarkMachineLocal`, `runIgnoreCheck`) render internal values for a @@ -26,6 +33,28 @@ export interface PolicyHumanVocabulary { implicitSuffix?(): string } +// A structural subset of `CommandRunContext`: exactly the fields +// `runBackfillProvider`, `runProvider`, `resolveOwnersForRun`, and the +// materialize/write/flush helpers they call +// (`src/core/commands/backfill.js`) read off `ctx`. Every existing +// `CommandRunContext` already satisfies it, so `hyp backfill`'s CLI path +// and the onboarding finale's call keep typechecking unchanged; the +// daemon sweep driver (LLP 0173 T9) can build one directly out of +// `boot.runtime` fields without assembling a full, mostly-unused +// `CommandRunContext`. `query` was missing from this list until LLP 0173 +// T12's smoke (the first caller to drive a real, non-mocked write through +// the sweep driver) found `writeRows`/`flushDataset` crash on +// `ctx.query.getDataset` when a sweep-built `ctx` reached them. +// @ref LLP 0172#lane-b-sweep [implements]: the narrowed context type `runBackfillProvider`, `runProvider`, `resolveOwnersForRun`, and the materialize/write/flush helpers declare, so the daemon sweep driver can build one without a full `CommandRunContext` +export interface BackfillRunnerContext { + env: NodeJS.ProcessEnv + config: HypAwareV2Config + storage: QueryStorageService + query: QueryRegistry + backfills: BackfillRegistry + backfillMaterializers: BackfillMaterializerRegistry +} + export interface BackfillProviderResult { provider: string plugin: string diff --git a/src/core/config/action_attach.js b/src/core/config/action_attach.js index 1a6171ec..6826d992 100644 --- a/src/core/config/action_attach.js +++ b/src/core/config/action_attach.js @@ -330,7 +330,12 @@ export function createAttachHandler(opts = {}) { let result try { - result = await detach({ descriptor, env: ctx.env }) + // `ctx.endpoint` is the proven-bound gateway URL `perform()` already + // attaches with, so reverse decides ownership against the very origin + // the forward action wrote. The `json_path` format needs it (its undo + // record is the entry it wrote); the marker-carrying formats ignore it. + // @ref LLP 0172#lane-a-detach [implements]: reverse threads the gateway's own base URL into the one core undo, from the ActionContext perform() already uses + result = await detach({ descriptor, env: ctx.env, expectedBaseUrl: ctx.endpoint }) } catch (err) { return { status: 'failed', reason: err instanceof Error ? err.message : String(err) } } diff --git a/src/core/config/client_detach_disk.js b/src/core/config/client_detach_disk.js index b5846778..44658013 100644 --- a/src/core/config/client_detach_disk.js +++ b/src/core/config/client_detach_disk.js @@ -2,12 +2,16 @@ import fsp from 'node:fs/promises' import os from 'node:os' +import path from 'node:path' import { resolveClientSettingsPath } from '../daemon/client_settings_path.js' +import { Attr, getLogger } from '../observability/index.js' import { ConcurrentEditError, atomicWriteFile } from '../util/fs_atomic.js' import { errCode, getAtDottedPath, isPlainObject } from '../util/json_util.js' +import { isOwnedProviderEntry, ownedBaseUrls } from './provider_entry_ownership.js' /** + * @import { Dirent } from 'node:fs' * @import { ClientDescriptor } from '../../../src/core/types.js' * @import { DetachFromDiskResult } from '../../../src/core/config/types.js' */ @@ -96,15 +100,30 @@ export class ClientDetachError extends Error { * `attachProbe` and the settings-file marker. No-op (`{ changed: false }`) when * the descriptor has no probe, the file is absent, or it carries no marker. * + * `expectedBaseUrl` is the one fact the dispatcher cannot read off disk: the + * gateway's own currently-resolved base origin. Only the `json_path` format + * needs it, and it needs it for a reason no marker can supply - that format's + * undo record IS the entry it wrote, so "is this entry ours?" can only be + * answered by comparing the URL it points at against the URL we would have + * written. The `json`/`toml` formats carry a HypAware-owned marker key or + * managed block, which answers ownership on its own, so they ignore it. + * * @param {{ * descriptor: ClientDescriptor, * homeDir?: string, * env?: NodeJS.ProcessEnv, + * expectedBaseUrl?: string, * fs?: typeof fsp, * }} args * @returns {Promise} */ -export async function detachClientFromDisk({ descriptor, homeDir = os.homedir(), env, fs = fsp }) { +export async function detachClientFromDisk({ + descriptor, + homeDir = os.homedir(), + env, + expectedBaseUrl, + fs = fsp, +}) { const probe = descriptor.attachProbe if (!probe) return { changed: false } @@ -116,6 +135,19 @@ export async function detachClientFromDisk({ descriptor, homeDir = os.homedir(), if (probe.format === 'toml') { return await detachTomlManagedBlock({ settingsPath, fs }) } + // @ref LLP 0172#lane-a-detach [implements]: the json_path branch LLP 0143 removed returns, reshaped for two provider entries plus a cache purge + if (probe.format === 'json_path') { + return await detachJsonPathProviders({ + settingsPath, + settingsFile: probe.settings_file, + containerPath: probe.container_path, + providerKeys: probe.provider_keys, + markerHeader: probe.marker_header, + cacheGlob: probe.cache_glob, + expectedBaseUrl, + fs, + }) + } // Unknown/incomplete probe: nothing this core routine knows how to reverse. return { changed: false, settingsPath } } @@ -664,6 +696,397 @@ function restoreAtDottedPath(root, dottedPath, newValue) { return true } +/* ----------------------------- json_path format ---------------------------- */ + +/** + * The container-relative key the backup of a present-but-not-ours provider + * entry lands under, so `models.providers.anthropic` moves to + * `models.providers._hypaware_detach_backup.anthropic`. + * + * A **sibling** of the provider key, not a top-level marker: LLP 0163 ruled a + * top-level HypAware key out for this client because its own config schema + * rejects one, which is exactly why that LLP left OpenClaw refusing where the + * `json`/`toml` formats backed up. Keeping the backup inside the container the + * undo already navigates converges the *outcome* (never discard a value + * HypAware did not write) without reintroducing the mechanism that was ruled + * out. + * + * @ref LLP 0163#open-questions [implements]: json_path converges on backup-not-discard without adopting the top-level marker key LLP 0163 ruled out for this client + */ +const JSON_PATH_BACKUP_KEY = '_hypaware_detach_backup' + +/** + * Reverse a `json_path` attach: the format whose undo record is the entries it + * wrote. There is no marker to replay, so each provider key is judged on what + * it points at. + * + * 1. Absent settings file: `{ changed: false }`, like every other format. + * 2. For each `providerKeys` entry under `containerPath` that is present: + * **ours** (its `baseUrl` is the gateway's, its `markerHeader` names the + * key) is deleted; anything else is **backed up, never discarded**. + * 3. The same provider keys are then best-effort purged from the client's + * derived caches (`cacheGlob`). Those caches do not self-heal, so a partial + * purge is strictly better than none, and one unreadable cache file must + * not fail a detach whose settings half already landed. + * + * @param {{ + * settingsPath: string, + * settingsFile: string, + * containerPath: string | undefined, + * providerKeys: string[] | undefined, + * markerHeader: string | undefined, + * cacheGlob: string | undefined, + * expectedBaseUrl: string | undefined, + * fs: typeof fsp, + * }} args + * @returns {Promise} + * @ref LLP 0169#decision [implements]: delete an entry only when its baseUrl is the gateway's, back up a present-but-not-ours one instead of discarding it, and purge the written provider keys from the derived caches + */ +async function detachJsonPathProviders({ + settingsPath, + settingsFile, + containerPath, + providerKeys, + markerHeader, + cacheGlob, + expectedBaseUrl, + fs, +}) { + // `contributes.client` is unvalidated manifest input (the same reason + // `resolveClientSettingsPath` guards its own field), so a probe missing any + // of the three fields this undo navigates by, or naming a path segment the + // restore helper already refuses, reverses nothing rather than guessing. + const keys = Array.isArray(providerKeys) + ? providerKeys.filter((key) => typeof key === 'string' && key.length > 0 && !UNWRITABLE_PATH_SEGMENTS.has(key)) + : [] + const container = typeof containerPath === 'string' && containerPath.length > 0 && !hasUnwritableSegment(containerPath) + ? containerPath + : undefined + if (container === undefined || keys.length === 0 || typeof markerHeader !== 'string' || markerHeader.length === 0) { + return { changed: false, settingsPath } + } + + const read = await readJson(settingsPath, fs) + if (!read.existed) return { changed: false, settingsPath } + + const value = read.value + const providers = getAtDottedPath(value, container) + const present = isPlainObject(providers) + ? keys.filter((key) => Object.hasOwn(providers, key)) + : [] + + // The two spellings attach writes: the bare origin for the vendor whose + // client appends its own path, `+ '/v1'` for the one that does not. Both are + // ours; anything else at the key is not. + const ours = ownedBaseUrls(expectedBaseUrl) + if (present.length > 0 && ours === undefined) { + // Without the gateway's own base URL there is no way to tell our entry from + // the user's, and both wrong answers are destructive (delete a value we + // never wrote, or leave the client routed at a dead port and report the + // undo done). Fail loudly: the reconciler's reverse() keeps the marker and + // retries, `hyp detach` prints the reason. + throw new ClientDetachError( + `cannot reverse ${settingsPath}: the gateway's base URL is unknown, ` + + 'so a provider entry HypAware wrote cannot be told from one it did not', + { code: 'EXPECTED_BASE_URL_UNKNOWN' } + ) + } + + /** @type {Record} */ + const containerObj = /** @type {Record} */ (providers) + /** @type {string[]} */ + const warnings = [] + /** @type {string | undefined} */ + let removed + let changed = false + + for (const key of present) { + const entry = containerObj[key] + // `ours` is always a set here, never the shared predicate's + // accept-any-baseUrl `undefined`: the guard above already threw if the + // gateway origin was unknown while an entry was present. Deleting is + // destructive, so this side never relaxes the origin check. + if (isOwnedProviderEntry(entry, key, markerHeader, ours)) { + if (removed === undefined) removed = providerBaseUrl(entry) + delete containerObj[key] + changed = true + continue + } + + const backups = isPlainObject(containerObj[JSON_PATH_BACKUP_KEY]) + ? /** @type {Record} */ (containerObj[JSON_PATH_BACKUP_KEY]) + : {} + if (Object.hasOwn(backups, key)) { + // An earlier detach already parked a value here. Overwriting it would + // destroy the older backup to save the newer one, which is the exact + // destruction this branch exists to prevent, so the live key stays put + // and the user is told which two values are now in play. + warnings.push( + `${container}.${key} was not written by this gateway and ` + + `${container}.${JSON_PATH_BACKUP_KEY}.${key} already holds an earlier backup; ` + + 'leaving it in place rather than overwriting that backup' + ) + continue + } + backups[key] = entry + containerObj[JSON_PATH_BACKUP_KEY] = backups + delete containerObj[key] + changed = true + // Paths, never values: a provider entry carries headers, and this string is + // printed to the terminal and echoed into `hyp detach --json` (LLP 0163). + warnings.push( + `${container}.${key} was not written by this gateway; ` + + `backed up to ${container}.${JSON_PATH_BACKUP_KEY}.${key} rather than discarded` + ) + } + + if (changed) await writeJsonAtomic(settingsPath, value, read.mtimeMs, fs) + + warnings.push(...await purgeProviderCaches({ + cacheGlob, + configHome: clientConfigHome(settingsPath, settingsFile), + containerPath: container, + providerKeys: keys, + fs, + })) + + const warning = joinWarnings(warnings) + + /** @type {DetachFromDiskResult} */ + const result = { changed, settingsPath } + if (removed !== undefined) result.removed = removed + if (warning !== undefined) result.warning = warning + return result +} + +/** @param {unknown} entry @returns {string | undefined} */ +function providerBaseUrl(entry) { + if (!isPlainObject(entry)) return undefined + return typeof entry.baseUrl === 'string' ? entry.baseUrl : undefined +} + +/** + * The client's config home: the already-resolved `settingsPath` with the + * manifest's own `settings_file` tail stripped back off, which is what + * `cache_glob` is declared relative to. It is the exact inverse of what + * `resolveClientSettingsPath` joined on, whose two branches both append + * `settings_file`'s segments *after the first* to a base (`$HOME/` + * normally, `$_HOME` under the relocation), so stripping that many + * segments recovers the base either way. Derived from the resolved path rather + * than re-resolved, so the purge and the settings write can never disagree + * about which home they are working in. + * + * Measuring against `homeDir` instead is what this does *not* do, and the bug + * it fixes: taking the first segment of `path.relative(homeDir, settingsPath)` + * is only the config home when `$_HOME` is outside `$HOME` or one level + * inside it. A nested relocation (`OPENCLAW_HOME=$HOME/.config/openclaw`) stays + * relative to `homeDir`, so the fallback never fires and the first segment is + * `.config`: the glob then matches nothing, an unmatched glob is not an error, + * and the cache purge silently no-ops while the settings half reports success. + * + * @param {string} settingsPath + * @param {string} settingsFile the manifest value, home-relative + * @returns {string} + */ +function clientConfigHome(settingsPath, settingsFile) { + const tail = settingsFile.split('/').slice(1) + const depth = tail.length === 0 ? 0 : path.join(...tail).split(path.sep).filter((s) => s !== '.').length + return path.resolve(settingsPath, ...new Array(depth).fill('..')) +} + +/** + * Best-effort removal of the same provider keys from the client's derived + * caches. These are files the client regenerates from its config and does not + * re-derive on its own after the config changes, so leaving them keeps a + * detached client pointed at a dead gateway. + * + * Every failure here is a warning, never a throw: the settings undo has + * already landed by the time this runs, and failing the whole detach over one + * unreadable cache file would leave the caller unable to finish an operation + * that is already most of the way done. A file that will not parse is one the + * client itself will have to rebuild. + * + * @param {{ + * cacheGlob: string | undefined, + * configHome: string, + * containerPath: string, + * providerKeys: string[], + * fs: typeof fsp, + * }} args + * @returns {Promise} the per-file notices, for the caller's `warning` + */ +async function purgeProviderCaches({ cacheGlob, configHome, containerPath, providerKeys, fs }) { + if (typeof cacheGlob !== 'string' || cacheGlob.length === 0) return [] + const log = getLogger('client-detach') + + /** @type {string[]} */ + const warnings = [] + /** @type {string[]} */ + let files + try { + files = await expandCacheGlob(configHome, cacheGlob, fs) + } catch (err) { + // A glob the manifest declares that this expander refuses (an absolute + // pattern, or one that climbs out of the config home) purges nothing. + log.warn('client.detach.cache_glob_refused', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.cache_purge', + [Attr.ERROR_KIND]: 'glob_refused', + cache_glob: cacheGlob, + detail: errMsg(err), + }) + return [`cache purge skipped: ${errMsg(err)}`] + } + + for (const file of files) { + /** @type {string} */ + let raw + try { + raw = await fs.readFile(file, 'utf8') + } catch (err) { + if (errCode(err) === 'ENOENT') continue + log.warn('client.detach.cache_purge_skipped', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.cache_purge', + [Attr.ERROR_KIND]: 'read_failed', + cache_path: file, + detail: errMsg(err), + }) + warnings.push(`${file} could not be read; its cached provider entries were left in place`) + continue + } + + /** @type {unknown} */ + let parsed + try { + parsed = JSON.parse(raw) + } catch (err) { + // Logged and skipped, not fatal (LLP 0172 §2.2 step 6). + log.warn('client.detach.cache_purge_skipped', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.cache_purge', + [Attr.ERROR_KIND]: 'malformed_json', + cache_path: file, + detail: errMsg(err), + }) + warnings.push(`${file} is not valid JSON; its cached provider entries were left in place`) + continue + } + if (!isPlainObject(parsed)) continue + + // The cache may mirror the settings file's container or hold the provider + // keys at its root; both spellings are the same keys, so purge whichever + // one this file uses rather than pinning a shape core cannot validate. + const targets = [parsed, getAtDottedPath(parsed, containerPath)] + let purged = false + for (const target of targets) { + if (!isPlainObject(target)) continue + for (const key of providerKeys) { + if (!Object.hasOwn(target, key)) continue + delete target[key] + purged = true + } + } + if (!purged) continue + + try { + await atomicWriteFile(file, JSON.stringify(parsed, null, 2) + '\n', { fsync: true, fs }) + log.info('client.detach.cache_purged', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.cache_purge', + [Attr.STATUS]: 'ok', + cache_path: file, + }) + } catch (err) { + log.warn('client.detach.cache_purge_skipped', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.cache_purge', + [Attr.ERROR_KIND]: 'write_failed', + cache_path: file, + detail: errMsg(err), + }) + warnings.push(`${file} could not be rewritten; its cached provider entries were left in place`) + } + } + return warnings +} + +/** + * Expand a `cache_glob` under the client's config home. `*` matches within one + * path segment only, and `..`/absolute patterns are refused outright, so an + * expansion can never leave the config home: containment is a property of the + * expander rather than a check bolted on after it. A directory that cannot be + * listed contributes no matches (the cache simply is not there). + * + * @param {string} configHome + * @param {string} pattern + * @param {typeof fsp} fs + * @returns {Promise} + */ +async function expandCacheGlob(configHome, pattern, fs) { + if (path.isAbsolute(pattern)) { + throw new Error(`cache_glob '${pattern}' must be relative to the client's config home`) + } + const segments = pattern.split('/').filter((segment) => segment.length > 0 && segment !== '.') + if (segments.length === 0) throw new Error('cache_glob names no file') + if (segments.some((segment) => segment === '..')) { + throw new Error(`cache_glob '${pattern}' must stay under the client's config home`) + } + + /** @type {string[]} */ + let dirs = [configHome] + /** @type {string[]} */ + const matches = [] + for (const [index, segment] of segments.entries()) { + const last = index === segments.length - 1 + /** @type {string[]} */ + const next = [] + for (const dir of dirs) { + if (!segment.includes('*')) { + const candidate = path.join(dir, segment) + if (last) matches.push(candidate) + else next.push(candidate) + continue + } + const matcher = segmentMatcher(segment) + /** @type {Dirent[]} */ + let entries + try { + entries = await fs.readdir(dir, { withFileTypes: true }) + } catch { + continue + } + for (const entry of entries) { + if (!matcher.test(entry.name)) continue + const candidate = path.join(dir, entry.name) + if (last) { + if (entry.isFile()) matches.push(candidate) + } else if (entry.isDirectory()) { + next.push(candidate) + } + } + } + dirs = next + } + return matches +} + +/** + * One glob segment as a whole-segment regex. `*` is the only metacharacter; + * everything else is literal, so a cache path with a `.` or `+` in it matches + * itself rather than acting as a pattern. + * + * @param {string} segment + * @returns {RegExp} + */ +function segmentMatcher(segment) { + const source = segment + .split('*') + .map((part) => part.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')) + .join('[^/]*') + return new RegExp(`^${source}$`) +} + /* ------------------------------- TOML format ------------------------------ */ /** diff --git a/src/core/config/provider_entry_ownership.js b/src/core/config/provider_entry_ownership.js new file mode 100644 index 00000000..05384762 --- /dev/null +++ b/src/core/config/provider_entry_ownership.js @@ -0,0 +1,70 @@ +// @ts-check + +import { isPlainObject } from '../util/json_util.js' + +/** + * The ownership test for a `json_path` provider entry, shared by the two sides + * that have to agree about it: the plugin's `attach()` (which overwrites its + * own entry and refuses over anyone else's) and core's `detachClientFromDisk` + * (which deletes its own entry and backs up anyone else's). Kept in one module + * rather than copied, because the failure mode of the two drifting apart is + * asymmetric and silent: attach would refuse to rewrite an entry detach is + * happy to delete, or delete-on-detach an entry attach believes is the user's. + * + * The entry attach writes is self-identifying by construction: the gateway + * origin in `baseUrl`, a marker header naming the provider key it sits at, and + * the empty `models` array OpenClaw's schema requires. Nothing else writes that + * triple - the marker header is HypAware's own name. + * + * @ref LLP 0167#attach-detach [implements]: one ownership predicate behind both + * halves, so "is this entry ours" has a single answer + */ + +/** + * The two `baseUrl` spellings a `json_path` attach writes for one gateway + * origin, or `undefined` when the origin is unknown. Trailing slashes are + * trimmed on the way in for the same reason attach trims them: `+ '/v1'` on a + * slash-terminated origin is a different string that names the same URL, and + * the comparison here is textual. + * + * @param {string | undefined} expectedBaseUrl + * @returns {Set | undefined} + */ +export function ownedBaseUrls(expectedBaseUrl) { + if (typeof expectedBaseUrl !== 'string') return undefined + const origin = expectedBaseUrl.trim().replace(/\/+$/, '') + if (origin.length === 0) return undefined + return new Set([origin, `${origin}/v1`]) +} + +/** + * Whether a provider entry is one this gateway wrote: its marker header names + * its own key, its shape is the one attach produces (a `baseUrl` string and an + * empty `models` array), and - when `ours` is given - it points at one of the + * gateway's own origins. Every other outcome (a missing or renamed marker + * header, a hand-edited `models` list, no `baseUrl` at all) is somebody else's + * entry that merely sits at our key. + * + * `ours` is `undefined` only on the **attach** side, and deliberately so: attach + * re-runs precisely when the endpoint has moved (an ephemeral-port rebind, + * LLP 0086), so the entry it is about to overwrite carries the *old* origin by + * construction and pinning the check to the live one would make every + * re-attach-on-drift refuse. Overwriting our own entry with a fresh URL is what + * that pass is for, and it destroys nothing the user authored. **Detach** always + * passes the set, because there the wrong answer deletes a value HypAware never + * wrote (`detachJsonPathProviders` refuses outright rather than run without it). + * + * @param {unknown} entry + * @param {string} key + * @param {string} markerHeader + * @param {Set | undefined} ours gateway origins, or `undefined` to accept any `baseUrl` + * @returns {boolean} + */ +export function isOwnedProviderEntry(entry, key, markerHeader, ours) { + if (!isPlainObject(entry)) return false + if (typeof entry.baseUrl !== 'string') return false + if (ours !== undefined && !ours.has(entry.baseUrl)) return false + const headers = entry.headers + if (!isPlainObject(headers) || headers[markerHeader] !== key) return false + return Array.isArray(entry.models) && entry.models.length === 0 +} diff --git a/src/core/config/types.d.ts b/src/core/config/types.d.ts index 98951586..d39b8d5d 100644 --- a/src/core/config/types.d.ts +++ b/src/core/config/types.d.ts @@ -706,6 +706,14 @@ export type ClientDetachFromDisk = (args: { descriptor: ClientDescriptor homeDir?: string env?: NodeJS.ProcessEnv + /** + * The gateway's own currently-resolved base origin, for the `json_path` + * format whose undo record is the entry it wrote: ownership can only be + * decided by comparing that entry's `baseUrl` against the URL attach would + * have written (LLP 0172 §2.1). The `json`/`toml` formats carry a + * HypAware-owned marker and ignore it, so it stays optional. + */ + expectedBaseUrl?: string }) => Promise export interface CreateAttachHandlerOptions { diff --git a/src/core/daemon/backfill_sweep.js b/src/core/daemon/backfill_sweep.js new file mode 100644 index 00000000..cbd9239f --- /dev/null +++ b/src/core/daemon/backfill_sweep.js @@ -0,0 +1,204 @@ +// @ts-check + +import { Attr, getLogger } from '../observability/index.js' +import { runBackfillProvider } from '../commands/backfill.js' +import { cronMatches } from '../sinks/driver.js' + +// The sweep's telemetry identity: one pair on every record this driver emits, +// so a failing run is greppable by the same `component`/`operation` everywhere +// it is logged. `component` names the emitting module (matching the +// `getLogger('backfill-sweep')` below), never the plugin that happens to have +// opted in: this driver fires any contribution carrying a `sweep` field, and +// OpenClaw is only the first. Plugin identity is already on every record as +// `hyp_plugin` and `provider`, which is where an operator filtering by client +// should look. +const SWEEP_COMPONENT = 'backfill-sweep' +const SWEEP_OPERATION = 'backfill.sweep' + +/** + * @import { BackfillContribution } from '../../../hypaware-plugin-kernel-types.js' + * @import { + * BackfillSweepDriver, + * BackfillSweepDriverOptions, + * BackfillSweepTickOptions, + * BackfillSweepTickReport, + * } from '../../../src/core/daemon/types.js' + */ + +/** + * Build the daemon's backfill sweep driver: the periodic, in-process re-run of + * every registered backfill provider that opted into a schedule. + * + * The driver owns no timer. `tick({ now })` is called from the daemon's + * existing 60-second sink tick, evaluates each contribution's `sweep.cron` + * against `now` with the same `cronMatches` the sink driver uses, and fires a + * run for each due provider. A contribution with no `sweep` field is never + * ticked, which is why adding this driver is zero behavior change for Claude's + * and Codex's contributions. + * + * `tick()` resolves once every due provider's run has been *started*, not once + * any of them finishes. Runs are fired unblocked: `runProvider`'s scan, materialize, + * write and flush pass is unbounded in the size of a user's transcript tree, and + * the tick it rides also refreshes source details and persists `status.json`. + * Blocking on a sweep would stall those behind a provider's disk walk. The + * fired promise is still handled, so a failing run is a logged + * `backfill.sweep_failed` record rather than an unhandled rejection that takes + * the daemon process down. + * + * Not blocking is what makes the re-entrancy guard necessary: a provider whose + * run outlives its own cron interval is due again while the first pass is still + * running, so the driver tracks which providers are in flight and skips a due + * one that already is. + * + * @ref LLP 0172#lane-b-sweep [implements]: the sweep rides the existing sink-tick cadence with `cronMatches` as its due-check, fires each due provider without blocking the tick, and never overlaps two runs of the same provider + * @ref LLP 0170#decision [implements]: scheduling an existing job (the backfill provider) on the daemon's existing cron-matched loop, not building a new scheduling primitive + * @param {BackfillSweepDriverOptions} opts + * @returns {BackfillSweepDriver} + */ +export function createBackfillSweepDriver(opts) { + const { backfills, backfillMaterializers, env, config, storage, query } = opts + if (!backfills) throw new Error('createBackfillSweepDriver: backfills required') + if (!backfillMaterializers) throw new Error('createBackfillSweepDriver: backfillMaterializers required') + if (!storage) throw new Error('createBackfillSweepDriver: storage required') + if (!query) throw new Error('createBackfillSweepDriver: query required') + const runBackfill = opts.runBackfill ?? runBackfillProvider + const log = getLogger('backfill-sweep') + + /** + * The providers whose fired run has not settled yet. Because `tick()` does + * not block on the run it fires, a provider whose pass outlives its own cron + * interval is due again while the previous one is still walking the + * transcript tree, and firing again would put two runs on the same datasets + * and the same mid-flush spool. Neither `runBackfillProvider` nor + * `runProvider` carries a lock of its own, so the guard belongs here, in the + * only place that knows a run was started. Same shape as the daemon's + * `maintenanceInFlight` (`src/core/daemon/runtime.js`), a set rather than a + * single handle because this driver fires one run per provider. + * + * @type {Set} + */ + const inFlight = new Set() + + /** + * @param {BackfillSweepTickOptions} [tickOpts] + * @returns {Promise} + */ + async function tick(tickOpts = {}) { + const now = tickOpts.now ?? new Date() + /** @type {string[]} */ + const fired = [] + for (const provider of backfills.list()) { + if (!provider.sweep) continue + if (!isDue(provider, now, tickOpts.force === true)) continue + // A due provider whose previous run is still going is skipped, not + // queued: the sweep is level-triggered, so the next tick that finds it + // due and idle picks up whatever this one would have. + if (inFlight.has(provider.name)) { + log.warn('backfill.sweep_skipped', { + [Attr.COMPONENT]: SWEEP_COMPONENT, + [Attr.OPERATION]: SWEEP_OPERATION, + [Attr.ERROR_KIND]: 'already_running', + [Attr.PLUGIN]: provider.plugin, + provider: provider.name, + hyp_sweep_schedule: provider.sweep.cron, + status: 'ok', + }) + continue + } + const devRunId = `sweep-${provider.name}-${now.getTime()}` + fired.push(provider.name) + inFlight.add(provider.name) + log.info('backfill.sweep_due', { + [Attr.COMPONENT]: SWEEP_COMPONENT, + [Attr.OPERATION]: SWEEP_OPERATION, + [Attr.PLUGIN]: provider.plugin, + [Attr.DEV_RUN_ID]: devRunId, + provider: provider.name, + hyp_sweep_schedule: provider.sweep.cron, + status: 'ok', + }) + // Fire-and-forget, with both settlements handled: `void` here means "not + // awaited", never "not observed". + void runBackfill({ + ctx: { env, config: config ?? { version: 2 }, storage, query, backfills, backfillMaterializers }, + provider: provider.name, + dryRun: false, + devRunId, + }).then( + (result) => { inFlight.delete(provider.name); logSettled(provider, devRunId, result) }, + (err) => { inFlight.delete(provider.name); logFailed(provider, devRunId, err) } + ) + } + return { fired } + } + + /** + * Whether a contribution's schedule is due at `now`. A malformed cron + * expression throws out of `cronMatches`; here that is one provider's + * scheduling metadata being wrong, not a reason to skip every later + * provider in the list or to fail the daemon tick this runs inside, so it + * is logged and treated as not due. + * + * @param {BackfillContribution} provider + * @param {Date} now + * @param {boolean} force + * @returns {boolean} + */ + function isDue(provider, now, force) { + if (force) return true + try { + return cronMatches(provider.sweep?.cron ?? '', now) + } catch (err) { + log.warn('backfill.sweep_schedule_invalid', { + [Attr.COMPONENT]: SWEEP_COMPONENT, + [Attr.OPERATION]: SWEEP_OPERATION, + [Attr.ERROR_KIND]: 'invalid_cron', + [Attr.PLUGIN]: provider.plugin, + provider: provider.name, + hyp_sweep_schedule: provider.sweep?.cron, + status: 'failed', + }) + return false + } + } + + /** + * @param {BackfillContribution} provider + * @param {string} devRunId + * @param {{ ok: boolean, scanned: number, rowsWritten: number, skipped: number }} result + */ + function logSettled(provider, devRunId, result) { + log.info('backfill.sweep_finished', { + [Attr.COMPONENT]: SWEEP_COMPONENT, + [Attr.OPERATION]: SWEEP_OPERATION, + [Attr.PLUGIN]: provider.plugin, + [Attr.DEV_RUN_ID]: devRunId, + provider: provider.name, + status: result.ok ? 'ok' : 'failed', + ...(result.ok ? {} : { [Attr.ERROR_KIND]: 'provider_run_failed' }), + items_seen: result.scanned, + rows_written: result.rowsWritten, + rows_skipped: result.skipped, + }) + } + + /** + * @param {BackfillContribution} provider + * @param {string} devRunId + * @param {unknown} err + */ + function logFailed(provider, devRunId, err) { + log.error('backfill.sweep_failed', { + [Attr.COMPONENT]: SWEEP_COMPONENT, + [Attr.OPERATION]: SWEEP_OPERATION, + [Attr.ERROR_KIND]: 'sweep_run_rejected', + [Attr.PLUGIN]: provider.plugin, + [Attr.DEV_RUN_ID]: devRunId, + provider: provider.name, + status: 'failed', + error: err instanceof Error ? err.message : String(err), + }) + } + + return { tick } +} diff --git a/src/core/daemon/runtime.js b/src/core/daemon/runtime.js index c5f96ba8..7ba0e756 100644 --- a/src/core/daemon/runtime.js +++ b/src/core/daemon/runtime.js @@ -19,6 +19,7 @@ import { backfillHandler } from '../config/action_backfill.js' import { bootKernel, resolveLayeredConfigForDaemon } from '../runtime/boot.js' import { createSinkDriver } from '../sinks/driver.js' import { materializeSinks } from '../sinks/materialize.js' +import { createBackfillSweepDriver } from './backfill_sweep.js' import { clearPidFile, pidFilePath, @@ -442,6 +443,19 @@ export async function runDaemon(opts = {}) { config: boot.config ?? undefined, }) + // ----- Backfill sweep driver ----- + // Rides the sink tick below rather than owning a timer of its own: a + // contribution's coarsest useful schedule still only needs a due-check once + // a minute, which is exactly this loop's cadence. + const sweepDriver = createBackfillSweepDriver({ + backfills: boot.runtime.backfills, + backfillMaterializers: boot.runtime.backfillMaterializers, + env, + storage: boot.runtime.storage, + query: boot.runtime.query, + config: boot.config ?? undefined, + }) + status.sinks = collectSinkSnapshots({ runtime: boot.runtime, sinkSnapshots }) persist() // Derive the boot health event from the SAME aggregate written to @@ -599,6 +613,13 @@ export async function runDaemon(opts = {}) { }, async () => { const report = await driver.tick({ now, source: 'daemon' }) + // The scheduled backfill sweep (LLP 0170) rides this same tick. The + // await covers only the cron due-check and the fire: each due + // provider's run is started unblocked inside `tick`, so a slow + // transcript scan never stalls the sink snapshots, the source-detail + // refresh, or `persist()` below. + // @ref LLP 0172#lane-b-sweep [implements]: one sibling call on the existing 60-second loop, not a second timer + await sweepDriver.tick({ now }) for (const sinkReport of report.sinks) { const snap = sinkSnapshots.get(sinkReport.instance) ?? { instance: sinkReport.instance, diff --git a/src/core/daemon/status.js b/src/core/daemon/status.js index 3e7e9d91..44774c67 100644 --- a/src/core/daemon/status.js +++ b/src/core/daemon/status.js @@ -19,7 +19,7 @@ import { discoverBundledPlugins } from '../runtime/bundled.js' import { buildPluginCatalog } from '../plugin_catalog.js' import { classifyClientProvenance } from '../cli/wizard/provenance.js' import { atomicWriteJsonSync, readFileIfExistsSync } from '../util/fs_atomic.js' -import { isPlainObject, sanitizeLabel } from '../util/json_util.js' +import { getAtDottedPath, isPlainObject, sanitizeLabel } from '../util/json_util.js' import { localOnlyListPath, LocalOnlyListUnreadableError, readLocalOnlyDirs } from '../usage-policy/index.js' import { readFirstSyncDeadline } from '../usage-policy/first_sync_hold.js' import { resolveClientSettingsPath } from './client_settings_path.js' @@ -1084,6 +1084,26 @@ export async function probeClientAttachFromDescriptor({ descriptor, homeDir, env return { attached: raw.includes(probe.marker_header), settingsPath } } + // @ref LLP 0172#lane-a-detach [implements]: the json_path read branch removed by + // LLP 0143 / PR #510, restored parallel to the json/toml branches above; pure + // read, attached when any configured provider key's marker header matches. + if (probe.format === 'json_path' && probe.container_path && probe.provider_keys && probe.marker_header) { + /** @type {unknown} */ + const parsed = JSON.parse(raw) + if (!parsed || typeof parsed !== 'object') { + return { attached: false, settingsPath } + } + const container = getAtDottedPath(parsed, probe.container_path) + if (!isPlainObject(container)) return { attached: false, settingsPath } + const markerHeader = probe.marker_header + const attached = probe.provider_keys.some((key) => { + const entry = container[key] + if (!isPlainObject(entry) || !isPlainObject(entry.headers)) return false + return entry.headers[markerHeader] === key + }) + return { attached, settingsPath } + } + return { attached: false, settingsPath } } catch (err) { const code = err && /** @type {NodeJS.ErrnoException} */ (err).code diff --git a/src/core/daemon/types.d.ts b/src/core/daemon/types.d.ts index a5f6d65e..f354ce9e 100644 --- a/src/core/daemon/types.d.ts +++ b/src/core/daemon/types.d.ts @@ -1,7 +1,12 @@ import type { + BackfillMaterializerRegistry, + BackfillRegistry, CapabilityRegistry, + HypAwareV2Config, QueryRegistry, + QueryStorageService, } from '../../../hypaware-plugin-kernel-types.d.ts' +import type { BackfillRunnerContext } from '../commands/types.d.ts' import type { ActionReconciler, ConfigControlStatus, ConfigLayerDrop, V1Diagnostic } from '../config/types.d.ts' import type { ExtendedSinkRegistry, @@ -561,3 +566,58 @@ export interface PidFileEntry { /** `foreground` (Phase 3) or `detached` (Phase 4 installers). */ mode: string } + +/** + * The runner the sweep driver fires per due contribution: exactly + * `runBackfillProvider`'s (`src/core/commands/backfill.js`) shape, narrowed to + * the arguments a sweep passes. Declared as a type rather than taken from the + * import so the driver can accept an injected fake in a unit test without the + * test having to stand up a real cache, storage service, and materializer set. + */ +export interface BackfillSweepRunner { + (args: { + ctx: BackfillRunnerContext + provider: string + dryRun: boolean + devRunId?: string + }): Promise<{ ok: boolean, scanned: number, rowsWritten: number, skipped: number }> +} + +export interface BackfillSweepDriverOptions { + backfills: BackfillRegistry + backfillMaterializers: BackfillMaterializerRegistry + env: NodeJS.ProcessEnv + storage: QueryStorageService + /** + * Dataset registry. `runBackfillProvider`'s write/flush path + * (`writeRows`/`flushDataset` in `src/core/commands/backfill.js`) resolves + * a dataset's registered table path through this before it can commit a + * row, so a fired sweep run needs it on `BackfillRunnerContext` exactly + * like `hyp backfill`'s CLI path already gets it from `CommandRunContext`. + */ + query: QueryRegistry + /** The daemon's effective config; absent on a host with no readable document. */ + config?: HypAwareV2Config + /** Test seam: defaults to `runBackfillProvider`. */ + runBackfill?: BackfillSweepRunner +} + +export interface BackfillSweepTickOptions { + /** Tick instant the cron due-check evaluates against. Defaults to `new Date()`. */ + now?: Date + /** Ignore the cron due-check and fire every sweep-bearing provider (test use). */ + force?: boolean +} + +/** + * What one `tick()` decided, for the caller's telemetry and for tests. Runs are + * fired unblocked, so `fired` names the providers a run was *started* for, never + * the ones that finished. + */ +export interface BackfillSweepTickReport { + fired: string[] +} + +export interface BackfillSweepDriver { + tick(opts?: BackfillSweepTickOptions): Promise +} diff --git a/test/core/client-detach-json-path.test.js b/test/core/client-detach-json-path.test.js new file mode 100644 index 00000000..079595a9 --- /dev/null +++ b/test/core/client-detach-json-path.test.js @@ -0,0 +1,302 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { ClientDetachError, detachClientFromDisk } from '../../src/core/config/client_detach_disk.js' +// Fixture setup only. The core undo under test imports no plugin code; building +// the "this entry is ours" case with the real attach is what proves the two +// halves agree on the shape rather than on a shape this file invented. +import { createOpenclawAttach } from '../../hypaware-core/plugins-workspace/openclaw/src/attach.js' + +/** + * LLP 0173 T2: the `json_path` undo (`detachJsonPathProviders`). Unlike the + * `json`/`toml` formats there is no HypAware-owned marker to replay: the + * entries attach wrote *are* the record, so every outcome here turns on the + * ownership check (`baseUrl` is the gateway's, `marker_header` names the key). + * + * @import { ClientDescriptor } from '../../src/core/types.js' + */ + +/** @type {ClientDescriptor} */ +const OPENCLAW_DESCRIPTOR = { + plugin: /** @type {any} */ ('@hypaware/openclaw'), + name: 'openclaw', + skillDir: 'skills/openclaw', + attachProbe: { + format: 'json_path', + settings_file: '.openclaw/openclaw.json', + container_path: 'models.providers', + provider_keys: ['anthropic', 'openai'], + marker_header: 'x-hypaware-upstream', + cache_glob: 'agents/*/agent/models.json', + }, +} + +const ENDPOINT = 'http://127.0.0.1:18521' + +/** @returns {Promise} */ +async function stageHome() { + return await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-detach-json-path-')) +} + +/** + * @param {string} home + * @param {unknown} value + * @returns {Promise} + */ +async function writeOpenclawConfig(home, value) { + const p = path.join(home, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(p), { recursive: true }) + await fs.writeFile(p, JSON.stringify(value, null, 2) + '\n') + return p +} + +/** + * @param {string} home + * @param {string} agentId + * @param {string} content + * @returns {Promise} + */ +async function writeAgentCache(home, agentId, content) { + const p = path.join(home, '.openclaw', 'agents', agentId, 'agent', 'models.json') + await fs.mkdir(path.dirname(p), { recursive: true }) + await fs.writeFile(p, content) + return p +} + +/** @param {string} settingsPath */ +async function readJsonFile(settingsPath) { + return JSON.parse(await fs.readFile(settingsPath, 'utf8')) +} + +/** + * Attach for real, with its output swallowed: these tests assert the file the + * write produced, not the prose around it. + * + * @param {string} home + */ +async function attachForReal(home) { + const attach = createOpenclawAttach({ homeDir: home, env: {} }) + return await attach.attach(/** @type {any} */ ({ + endpoint: ENDPOINT, + stdout: { write() { return true } }, + })) +} + +/** @param {string} upstream @param {string} baseUrl */ +function ourEntry(upstream, baseUrl) { + return { baseUrl, headers: { 'x-hypaware-upstream': upstream }, models: [] } +} + +/* ------------------------------ ours: deleted ----------------------------- */ + +test('json_path undo deletes the two entries the gateway wrote', async () => { + const home = await stageHome() + try { + const settingsPath = await writeOpenclawConfig(home, { theme: 'dark', models: { default: 'sonnet' } }) + assert.deepEqual(await attachForReal(home), { status: 'done' }) + + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: {}, + expectedBaseUrl: ENDPOINT, + }) + + assert.equal(result.changed, true) + assert.equal(result.settingsPath, settingsPath) + // Both spellings attach writes are ours: the bare origin (anthropic) and + // the `+ /v1` one (openai). A check that only knew one of them would leave + // the other entry behind. + const after = await readJsonFile(settingsPath) + assert.deepEqual(after.models.providers, {}) + // Nothing outside the two keys is touched, and nothing is backed up: + // there was no foreign value to preserve. + assert.equal(after.theme, 'dark') + assert.equal(after.models.default, 'sonnet') + assert.equal(result.warning, undefined) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +/* ------------------- present but not ours: backed up, kept ------------------ */ + +test('json_path undo backs a present-but-not-ours entry up instead of discarding it', async () => { + const home = await stageHome() + try { + const foreign = { + baseUrl: 'https://foreign.example/anthropic', + headers: { authorization: 'Bearer user-secret' }, + models: ['claude-x'], + } + const settingsPath = await writeOpenclawConfig(home, { + models: { providers: { anthropic: foreign, openai: ourEntry('openai', `${ENDPOINT}/v1`) } }, + }) + + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: {}, + expectedBaseUrl: ENDPOINT, + }) + + assert.equal(result.changed, true) + const providers = (await readJsonFile(settingsPath)).models.providers + // Ours went; theirs did not merely survive, it is still readable in the + // same file a human opens after the detach. + assert.equal('openai' in providers, false) + assert.equal('anthropic' in providers, false) + assert.deepEqual(providers._hypaware_detach_backup.anthropic, foreign) + // Reported by path, never by value: a provider entry is exactly where a + // credential header ends up, and this string is printed and echoed into + // `hyp detach --json`. + assert.match(String(result.warning), /models\.providers\._hypaware_detach_backup\.anthropic/) + assert.equal(String(result.warning).includes('user-secret'), false) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('json_path undo treats a right-URL wrong-marker entry as not ours', async () => { + const home = await stageHome() + try { + // Points at the gateway, but the marker header names the other key: not a + // shape attach produces, so it is preserved rather than deleted. + const impostor = { baseUrl: ENDPOINT, headers: { 'x-hypaware-upstream': 'openai' }, models: [] } + const settingsPath = await writeOpenclawConfig(home, { models: { providers: { anthropic: impostor } } }) + + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: {}, + expectedBaseUrl: ENDPOINT, + }) + + assert.equal(result.changed, true) + const providers = (await readJsonFile(settingsPath)).models.providers + assert.deepEqual(providers._hypaware_detach_backup.anthropic, impostor) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('json_path undo refuses rather than guessing when the gateway base URL is unknown', async () => { + const home = await stageHome() + try { + await writeOpenclawConfig(home, { + models: { providers: { anthropic: ourEntry('anthropic', ENDPOINT) } }, + }) + + await assert.rejects( + detachClientFromDisk({ descriptor: OPENCLAW_DESCRIPTOR, homeDir: home, env: {} }), + (err) => err instanceof ClientDetachError && err.code === 'EXPECTED_BASE_URL_UNKNOWN' + ) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +/* ------------------------------ absent file ------------------------------- */ + +test('json_path undo is a no-op when the settings file is absent', async () => { + const home = await stageHome() + try { + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: {}, + expectedBaseUrl: ENDPOINT, + }) + + assert.equal(result.changed, false) + assert.equal(result.settingsPath, path.join(home, '.openclaw', 'openclaw.json')) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +/* --------------------------- best-effort cache purge ---------------------- */ + +test('json_path undo purges the derived caches, skipping one that will not parse', async () => { + const home = await stageHome() + try { + const settingsPath = await writeOpenclawConfig(home, { + models: { providers: { anthropic: ourEntry('anthropic', ENDPOINT) } }, + }) + const good = await writeAgentCache(home, 'main', JSON.stringify({ + anthropic: { baseUrl: ENDPOINT }, + openai: { baseUrl: `${ENDPOINT}/v1` }, + google: { baseUrl: 'https://vendor.example' }, + }, null, 2) + '\n') + const brokenText = '{ this is not json' + const broken = await writeAgentCache(home, 'sidecar', brokenText) + + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: {}, + expectedBaseUrl: ENDPOINT, + }) + + // The settings half landed and the malformed sibling did not fail it. + assert.equal(result.changed, true) + assert.deepEqual((await readJsonFile(settingsPath)).models.providers, {}) + + // The purge is provider-key scoped: another vendor's cached entry stays. + assert.deepEqual(await readJsonFile(good), { google: { baseUrl: 'https://vendor.example' } }) + // Skipped, not rewritten and not truncated. + assert.equal(await fs.readFile(broken, 'utf8'), brokenText) + assert.match(String(result.warning), /sidecar/) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +// `$OPENCLAW_HOME` may sit NESTED inside `$HOME`, which is the case that broke: +// deriving the config home from the first segment of the settings path's +// home-relative form answered `$HOME/.config` here, the `agents/*/...` glob +// matched nothing, and an unmatched glob is not an error - so the settings half +// reported success while the caches kept routing at the dead gateway. Two +// segments is what makes it a regression test; a one-segment relocation +// (`OPENCLAW_HOME=$HOME/elsewhere`) passed either way. +// @ref LLP 0169#decision [tests]: the cache purge follows the relocated config +// home, since the caches are what do not self-heal +test('json_path undo purges the caches under a nested $OPENCLAW_HOME', async () => { + const home = await stageHome() + try { + const openclawHome = path.join(home, '.config', 'openclaw') + const settingsPath = path.join(openclawHome, 'openclaw.json') + await fs.mkdir(openclawHome, { recursive: true }) + await fs.writeFile(settingsPath, JSON.stringify({ + models: { providers: { anthropic: ourEntry('anthropic', ENDPOINT) } }, + }, null, 2) + '\n') + + const cachePath = path.join(openclawHome, 'agents', 'main', 'agent', 'models.json') + await fs.mkdir(path.dirname(cachePath), { recursive: true }) + await fs.writeFile(cachePath, JSON.stringify({ + anthropic: { baseUrl: ENDPOINT }, + google: { baseUrl: 'https://vendor.example' }, + }, null, 2) + '\n') + + const result = await detachClientFromDisk({ + descriptor: OPENCLAW_DESCRIPTOR, + homeDir: home, + env: { OPENCLAW_HOME: openclawHome }, + expectedBaseUrl: ENDPOINT, + }) + + assert.equal(result.changed, true) + assert.equal(result.settingsPath, settingsPath) + assert.equal(result.removed, ENDPOINT) + assert.deepEqual((await readJsonFile(settingsPath)).models.providers, {}) + // The half that silently no-opped before. + assert.deepEqual(await readJsonFile(cachePath), { google: { baseUrl: 'https://vendor.example' } }) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) diff --git a/test/core/daemon-backfill-sweep-wiring.test.js b/test/core/daemon-backfill-sweep-wiring.test.js new file mode 100644 index 00000000..f56b2c60 --- /dev/null +++ b/test/core/daemon-backfill-sweep-wiring.test.js @@ -0,0 +1,138 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { runDaemon } from '../../src/core/daemon/runtime.js' +import { defaultConfigPath } from '../../src/core/config/schema.js' +import { writeLock } from '../../src/core/plugin_install/lock.js' + +// The unit tests next door prove the driver's due-check and containment. This +// one proves the part no unit test can: that the daemon's own tick actually +// calls it. The sweep rides the sink tick rather than owning a timer, so a +// wiring regression here is silent - the driver still passes every test it has +// and simply never runs. +// @ref LLP 0172#lane-b-sweep [tests]: the sweep is called from `runTick()`, on the existing tick interval, with no timer of its own + +const PLUGIN = '@third-party/sweeping-fixture' + +/** + * Stage a plugin whose backfill contribution opts into a once-a-minute sweep + * and records each run by appending to a file. It yields nothing, so the run + * exercises the runner's full lifecycle without needing a materializer or a + * writable dataset. + * + * @param {string} hypHome + * @param {string} marker + * @returns {Promise} + */ +async function stageSweepingPlugin(hypHome, marker) { + const installDir = path.join(hypHome, 'hypaware', 'plugins', PLUGIN) + await fs.mkdir(installDir, { recursive: true }) + await fs.writeFile(path.join(installDir, 'hypaware.plugin.json'), JSON.stringify({ + schema_version: 1, + name: PLUGIN, + version: '0.1.0', + hypaware_api: '^1.0.0', + runtime: 'node', + entrypoint: './index.js', + })) + await fs.writeFile( + path.join(installDir, 'index.js'), + ` +import fs from 'node:fs' + +export async function activate(ctx) { + ctx.backfills.register({ + name: 'sweeping-fixture', + plugin: '${PLUGIN}', + datasets: ['ai_gateway_messages'], + sweep: { cron: '* * * * *' }, + async *run() { + fs.appendFileSync(${JSON.stringify(marker)}, 'swept\\n') + }, + }) + ctx.sources.register({ + name: 'sweeping-fixture', + plugin: '${PLUGIN}', + async start() { + return { + async status() { return { state: 'ready', details: {} } }, + async stop() {}, + } + }, + }) +} +` + ) + return installDir +} + +/** + * @param {string} hypHome + * @param {string} installDir + */ +async function writeInstall(hypHome, installDir) { + await writeLock(path.join(hypHome, 'hypaware'), { + schema_version: 1, + plugins: { + [PLUGIN]: { + name: PLUGIN, + version: '0.1.0', + source: { kind: 'local-dir', raw: installDir, path: installDir }, + install_dir: installDir, + content_hash: 'a'.repeat(64), + manifest_hash: 'b'.repeat(64), + installed_at: '2026-07-30T00:00:00.000Z', + }, + }, + }) + const configPath = defaultConfigPath(hypHome) + await fs.mkdir(path.dirname(configPath), { recursive: true }) + await fs.writeFile(configPath, JSON.stringify({ + version: 2, + plugins: [{ name: PLUGIN, config: {} }], + })) + return configPath +} + +test('the daemon tick runs a sweep-bearing backfill contribution', async () => { + const hypHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hypaware-sweep-tick-')) + const marker = path.join(hypHome, 'sweeps.log') + let handle + try { + const configPath = await writeInstall(hypHome, await stageSweepingPlugin(hypHome, marker)) + handle = await runDaemon({ + hypHome, + configPath, + env: { ...process.env, HYP_HOME: hypHome }, + runId: 'sweep-tick', + // Fast enough that a tick lands inside the wait below, slow enough that + // the poll sees the first sweep rather than a dozen piled-up ones. + tickIntervalMs: 200, + installSignalHandlers: false, + }) + + const deadline = Date.now() + 20_000 + let swept = false + while (Date.now() < deadline) { + await new Promise((resolve) => setTimeout(resolve, 50)) + swept = await fs.readFile(marker, 'utf8').then((t) => t.includes('swept'), () => false) + if (swept) break + } + assert.ok(swept, 'the daemon tick never ran the sweep-bearing provider') + } finally { + if (handle) { + await handle.stop() + await handle.done + } + // Sweeps are fired unblocked, so shutdown does not drain them: a run + // started by the last tick can still be touching the state tree here. + // Retry the teardown rather than racing it. + await new Promise((resolve) => setTimeout(resolve, 100)) + await fs.rm(hypHome, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 }) + } +}) diff --git a/test/core/daemon-backfill-sweep.test.js b/test/core/daemon-backfill-sweep.test.js new file mode 100644 index 00000000..5d80bb4e --- /dev/null +++ b/test/core/daemon-backfill-sweep.test.js @@ -0,0 +1,293 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' + +import { createBackfillSweepDriver } from '../../src/core/daemon/backfill_sweep.js' + +// Lane B's scheduling seam. The sweep is the only reason a transcript that +// never crossed the live gateway lands at all, and it runs inside the daemon's +// own tick loop, so the two things worth pinning are *which* contributions it +// fires (opt-in only, cron-due only) and that a failing run stays contained: +// an unhandled rejection here would take the daemon process down with it. +// @ref LLP 0172#lane-b-sweep [tests]: the due-check is `sweep`-gated and cron-gated, and the fired run never blocks or breaks the tick it rides +// @ref LLP 0171#requirements [tests]: R7's periodic sweep fires on the contribution's own configured schedule + +/** + * @param {Record} [overrides] + * @returns {any} + */ +function contribution(overrides = {}) { + return { + name: 'openclaw', + plugin: '@hypaware/openclaw', + datasets: ['ai_gateway_messages'], + async *run() {}, + ...overrides, + } +} + +/** + * A `BackfillRegistry` over a fixed contribution list: `list()` is the only + * method the sweep driver calls, and the runner is faked, so nothing here + * needs a real kernel. + * + * @param {any[]} contributions + * @returns {any} + */ +function registry(contributions) { + return { + register() {}, + get: (name) => contributions.find((c) => c.name === name), + list: () => contributions.slice(), + } +} + +/** + * @param {{ contributions: any[], runBackfill: any, config?: any }} args + */ +function driverFor(args) { + return createBackfillSweepDriver({ + backfills: registry(args.contributions), + backfillMaterializers: /** @type {any} */ ({ register() {}, get: () => undefined, list: () => [] }), + env: /** @type {any} */ ({ HYP_HOME: '/nonexistent-home' }), + storage: /** @type {any} */ ({ cacheRoot: '/nonexistent-cache' }), + query: /** @type {any} */ ({ getDataset: () => undefined }), + config: args.config, + runBackfill: args.runBackfill, + }) +} + +/** @param {string} iso */ +function at(iso) { + return new Date(iso) +} + +const OK = { ok: true, scanned: 0, rowsWritten: 0, skipped: 0 } + +test('tick fires only the sweep-bearing contributions that are cron-due', async () => { + /** @type {any[]} */ + const calls = [] + const driver = driverFor({ + contributions: [ + // Opted in, due every five minutes. + contribution({ name: 'openclaw', sweep: { cron: '*/5 * * * *' } }), + // Opted in, but only on the hour: not due at :05. + contribution({ name: 'hourly', plugin: '@hypaware/hourly', sweep: { cron: '0 * * * *' } }), + // Never opted in: the absent-by-default case every provider is in today. + contribution({ name: 'claude', plugin: '@hypaware/claude' }), + ], + runBackfill: async (args) => { calls.push(args); return OK }, + }) + + const report = await driver.tick({ now: at('2026-08-01T10:05:00.000Z') }) + + assert.deepEqual(report.fired, ['openclaw']) + assert.equal(calls.length, 1) + assert.equal(calls[0].provider, 'openclaw') + assert.equal(calls[0].dryRun, false) + assert.equal(calls[0].devRunId, `sweep-openclaw-${at('2026-08-01T10:05:00.000Z').getTime()}`) +}) + +test('tick fires nothing when no contribution is due, and both when both are', async () => { + /** @type {string[]} */ + const fired = [] + const driver = driverFor({ + contributions: [ + contribution({ name: 'openclaw', sweep: { cron: '*/5 * * * *' } }), + contribution({ name: 'hourly', plugin: '@hypaware/hourly', sweep: { cron: '0 * * * *' } }), + ], + runBackfill: async (args) => { fired.push(args.provider); return OK }, + }) + + // :07 is neither a five-minute boundary nor the top of the hour. + assert.deepEqual((await driver.tick({ now: at('2026-08-01T10:07:00.000Z') })).fired, []) + assert.deepEqual(fired, []) + + // :00 satisfies both schedules. + assert.deepEqual((await driver.tick({ now: at('2026-08-01T11:00:00.000Z') })).fired, ['openclaw', 'hourly']) + assert.deepEqual(fired, ['openclaw', 'hourly']) +}) + +test('the fired run gets the narrowed runner context, built from the daemon runtime fields', async () => { + /** @type {any} */ + let seen = null + const contributions = [contribution({ sweep: { cron: '* * * * *' } })] + const config = { version: 2, plugins: [{ name: '@hypaware/openclaw', config: {} }] } + const backfills = registry(contributions) + const backfillMaterializers = /** @type {any} */ ({ register() {}, get: () => undefined, list: () => [] }) + const env = /** @type {any} */ ({ HYP_HOME: '/nonexistent-home' }) + const storage = /** @type {any} */ ({ cacheRoot: '/nonexistent-cache' }) + const query = /** @type {any} */ ({ getDataset: () => undefined }) + const driver = createBackfillSweepDriver({ + backfills, + backfillMaterializers, + env, + storage, + query, + config: /** @type {any} */ (config), + runBackfill: async (args) => { seen = args.ctx; return OK }, + }) + + await driver.tick({ now: at('2026-08-01T10:00:00.000Z') }) + + assert.equal(seen.env, env) + assert.equal(seen.storage, storage) + assert.equal(seen.query, query) + assert.equal(seen.config, config) + assert.equal(seen.backfills, backfills) + assert.equal(seen.backfillMaterializers, backfillMaterializers) +}) + +test('a rejected sweep run neither throws out of tick nor becomes an unhandled rejection', async () => { + /** @type {unknown[]} */ + const unhandled = [] + /** @param {unknown} reason */ + const onUnhandled = (reason) => { unhandled.push(reason) } + process.on('unhandledRejection', onUnhandled) + try { + const driver = driverFor({ + contributions: [ + contribution({ name: 'openclaw', sweep: { cron: '* * * * *' } }), + contribution({ name: 'codex', plugin: '@hypaware/codex', sweep: { cron: '* * * * *' } }), + ], + runBackfill: async (args) => { + if (args.provider === 'openclaw') throw new Error('cache is unwritable') + return OK + }, + }) + + // The rejection is raised by the fired run, not by the due-check, so the + // tick itself resolves normally and the *later* provider still fires: one + // broken run does not cancel the rest of the sweep. + const report = await driver.tick({ now: at('2026-08-01T10:00:00.000Z') }) + assert.deepEqual(report.fired, ['openclaw', 'codex']) + + // Two macrotask turns: enough for the rejected promise's handler to run, + // and for Node to have reported it had there been none. + await new Promise((resolve) => setTimeout(resolve, 0)) + await new Promise((resolve) => setTimeout(resolve, 0)) + assert.deepEqual(unhandled, [], 'the fired run left an unhandled rejection') + } finally { + process.off('unhandledRejection', onUnhandled) + } +}) + +test('tick does not block on a run that never settles', async () => { + let settle = () => {} + const pending = new Promise((resolve) => { settle = () => resolve(OK) }) + const driver = driverFor({ + contributions: [contribution({ sweep: { cron: '* * * * *' } })], + runBackfill: () => /** @type {any} */ (pending), + }) + + // If `tick` awaited the run, this would hang until the test timed out. + const report = await driver.tick({ now: at('2026-08-01T10:00:00.000Z') }) + assert.deepEqual(report.fired, ['openclaw']) + settle() + await pending +}) + +// The companion to the test above: not blocking on a run is exactly what lets a +// pass that outruns its own cron interval be due again while it is still going, +// and a second concurrent run would land on the same datasets and the same +// mid-flush spool. Neither `runBackfillProvider` nor `runProvider` locks, so the +// driver has to. +// @ref LLP 0172#lane-b-sweep [tests]: two runs of one provider never overlap; +// a due-but-running provider is skipped, and the next idle tick picks it up +test('a second tick fires nothing while the first run is still in flight', async () => { + /** @type {string[]} */ + const started = [] + let settle = () => {} + const pending = new Promise((resolve) => { settle = () => resolve(OK) }) + const driver = driverFor({ + contributions: [contribution({ sweep: { cron: '* * * * *' } })], + runBackfill: (args) => { + started.push(args.provider) + return /** @type {any} */ (pending) + }, + }) + + const first = await driver.tick({ now: at('2026-08-01T10:00:00.000Z') }) + assert.deepEqual(first.fired, ['openclaw']) + + const second = await driver.tick({ now: at('2026-08-01T10:01:00.000Z') }) + assert.deepEqual(second.fired, []) + assert.deepEqual(started, ['openclaw']) + + // Skipped for that tick only: once the run settles, the next due tick fires. + settle() + await pending + const third = await driver.tick({ now: at('2026-08-01T10:02:00.000Z') }) + assert.deepEqual(third.fired, ['openclaw']) + assert.deepEqual(started, ['openclaw', 'openclaw']) +}) + +// A run that rejects has to clear the guard too, or one failure wedges the +// provider's sweep for the life of the daemon. +// @ref LLP 0172#lane-b-sweep [tests]: the in-flight entry clears on both +// settlements, not just the resolving one +test('a rejected run clears the in-flight guard so the next due tick still fires', async () => { + /** @type {string[]} */ + const started = [] + let reject = () => {} + const pending = new Promise((_resolve, rej) => { reject = () => rej(new Error('boom')) }) + const driver = driverFor({ + contributions: [contribution({ sweep: { cron: '* * * * *' } })], + runBackfill: (args) => { + started.push(args.provider) + return /** @type {any} */ (pending) + }, + }) + + assert.deepEqual((await driver.tick({ now: at('2026-08-01T10:00:00.000Z') })).fired, ['openclaw']) + reject() + await pending.catch(() => {}) + // One turn for the driver's own rejection handler to run before the retick. + await new Promise((resolve) => { setImmediate(resolve) }) + + assert.deepEqual((await driver.tick({ now: at('2026-08-01T10:01:00.000Z') })).fired, ['openclaw']) + assert.deepEqual(started, ['openclaw', 'openclaw']) +}) + +test('a malformed sweep cron is skipped, not thrown, and later providers still fire', async () => { + /** @type {string[]} */ + const fired = [] + const driver = driverFor({ + contributions: [ + contribution({ name: 'broken', plugin: '@hypaware/broken', sweep: { cron: 'not a cron' } }), + contribution({ name: 'openclaw', sweep: { cron: '* * * * *' } }), + ], + runBackfill: async (args) => { fired.push(args.provider); return OK }, + }) + + const report = await driver.tick({ now: at('2026-08-01T10:00:00.000Z') }) + assert.deepEqual(report.fired, ['openclaw']) + assert.deepEqual(fired, ['openclaw']) +}) + +test('createBackfillSweepDriver refuses to build without the registries it fires through', () => { + const ok = { + backfills: registry([]), + backfillMaterializers: /** @type {any} */ ({ register() {}, get: () => undefined, list: () => [] }), + env: /** @type {any} */ ({}), + storage: /** @type {any} */ ({ cacheRoot: '/nonexistent-cache' }), + query: /** @type {any} */ ({ getDataset: () => undefined }), + } + assert.throws( + () => createBackfillSweepDriver(/** @type {any} */ ({ ...ok, backfills: undefined })), + /backfills required/ + ) + assert.throws( + () => createBackfillSweepDriver(/** @type {any} */ ({ ...ok, backfillMaterializers: undefined })), + /backfillMaterializers required/ + ) + assert.throws( + () => createBackfillSweepDriver(/** @type {any} */ ({ ...ok, storage: undefined })), + /storage required/ + ) + assert.throws( + () => createBackfillSweepDriver(/** @type {any} */ ({ ...ok, query: undefined })), + /query required/ + ) +}) diff --git a/test/core/daemon.test.js b/test/core/daemon.test.js index ec371a72..752db7b8 100644 --- a/test/core/daemon.test.js +++ b/test/core/daemon.test.js @@ -269,6 +269,76 @@ test('probeClientAttachFromDescriptor honors sanitized TOML home overrides', asy ) }) +/** @type {ClientDescriptor} */ +const OPENCLAW_JSON_PATH_DESCRIPTOR = /** @type {ClientDescriptor} */ ({ + plugin: '@hypaware/openclaw', + name: 'openclaw', + skillDir: '.openclaw/skills', + attachProbe: { + format: 'json_path', + settings_file: '.openclaw/openclaw.json', + container_path: 'models.providers', + provider_keys: ['anthropic', 'openai'], + marker_header: 'x-hypaware-upstream', + }, +}) + +test('probeClientAttachFromDescriptor reads json_path attach markers when the entry is present', async () => { + const tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'hypaware-attach-json-path-present-')) + const settingsPath = path.join(tmp, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(settingsPath), { recursive: true }) + await fs.writeFile(settingsPath, JSON.stringify({ + models: { + providers: { + anthropic: { + baseUrl: 'http://127.0.0.1:4388', + headers: { 'x-hypaware-upstream': 'anthropic' }, + models: [], + }, + }, + }, + })) + + assert.deepEqual( + await probeClientAttachFromDescriptor({ descriptor: OPENCLAW_JSON_PATH_DESCRIPTOR, homeDir: tmp }), + { attached: true, settingsPath } + ) +}) + +test('probeClientAttachFromDescriptor reports json_path as not attached when the entry is absent', async () => { + const tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'hypaware-attach-json-path-absent-')) + const settingsPath = path.join(tmp, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(settingsPath), { recursive: true }) + await fs.writeFile(settingsPath, JSON.stringify({ models: { providers: {} } })) + + assert.deepEqual( + await probeClientAttachFromDescriptor({ descriptor: OPENCLAW_JSON_PATH_DESCRIPTOR, homeDir: tmp }), + { attached: false, settingsPath } + ) +}) + +test('probeClientAttachFromDescriptor reports json_path as not attached when the marker header is wrong', async () => { + const tmp = await fs.mkdtemp(path.join(os.tmpdir(), 'hypaware-attach-json-path-wrong-header-')) + const settingsPath = path.join(tmp, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(settingsPath), { recursive: true }) + await fs.writeFile(settingsPath, JSON.stringify({ + models: { + providers: { + anthropic: { + baseUrl: 'http://somewhere-else:9999', + headers: { 'x-hypaware-upstream': 'not-anthropic' }, + models: [], + }, + }, + }, + })) + + assert.deepEqual( + await probeClientAttachFromDescriptor({ descriptor: OPENCLAW_JSON_PATH_DESCRIPTOR, homeDir: tmp }), + { attached: false, settingsPath } + ) +}) + test('renderDaemonInstall renders a deterministic systemd dry-run payload', () => { const plan = renderDaemonInstall({ platform: 'linux', diff --git a/test/plugins/openclaw-attach.test.js b/test/plugins/openclaw-attach.test.js new file mode 100644 index 00000000..d21db7ae --- /dev/null +++ b/test/plugins/openclaw-attach.test.js @@ -0,0 +1,397 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { createOpenclawAttach } from '../../hypaware-core/plugins-workspace/openclaw/src/attach.js' + +/** + * LLP 0172 §1.2 (design) / LLP 0171 R1, R2, R4: the OpenClaw attach surface + * writes the two `models.providers` entries of LLP 0167#override-entries and + * nothing else, refuses instead of merging when either key is already there, + * and ends by telling the user to restart the gateway. + * + * Three of the four cases here are the ones the design and plan singled out as + * "worth a dedicated unit test rather than trusting the acceptance run": + * + * - the bare-origin (`anthropic`) vs `+/v1` (`openai`) asymmetry, because both + * spellings are schema-valid, so the wrong one produces a config OpenClaw + * accepts and silently does not route through the gateway; + * - the refusal, because it must be a *pure read-then-decide* with no partial + * write to roll back (R2); + * - and the refusal not throwing, because that is the whole mechanism by which + * a refuse during attach-on-join warns instead of failing the join + * (LLP 0169#decision). + * + * @ref LLP 0167#attach-detach [tests] + * @ref LLP 0169#decision [tests] + */ + +const ENDPOINT = 'http://127.0.0.1:18521' + +/** @returns {{ write(chunk: unknown): boolean, text(): string }} */ +function makeBuf() { + let value = '' + return { + write(chunk) { + value += String(chunk) + return true + }, + text() { + return value + }, + } +} + +/** + * Stage an OpenClaw config home with `openclaw.json` holding `config`. + * + * @param {Record} config + * @returns {Promise<{ homeDir: string, settingsPath: string }>} + */ +async function stage(config) { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-openclaw-attach-')) + const settingsPath = path.join(homeDir, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(settingsPath), { recursive: true }) + await fs.writeFile(settingsPath, JSON.stringify(config, null, 2)) + return { homeDir, settingsPath } +} + +/** + * @param {{ homeDir: string }} staged + * @param {{ json?: boolean, dryRun?: boolean }} [opts] + */ +async function runAttach(staged, opts = {}) { + const stdout = makeBuf() + const stderr = makeBuf() + const attacher = createOpenclawAttach({ homeDir: staged.homeDir, env: {} }) + const outcome = await attacher.attach( + /** @type {any} */ ({ + endpoint: ENDPOINT, + config: {}, + stdout, + stderr, + dryRun: opts.dryRun === true, + json: opts.json === true, + }) + ) + return { outcome, stdout: stdout.text(), stderr: stderr.text() } +} + +/** @param {string} settingsPath */ +async function readConfig(settingsPath) { + return JSON.parse(await fs.readFile(settingsPath, 'utf8')) +} + +test('attach writes exactly the two provider entries, bare origin vs +/v1', async () => { + const staged = await stage({ models: { providers: {} } }) + try { + const { outcome } = await runAttach(staged) + assert.deepEqual(outcome, { status: 'done' }) + + const written = await readConfig(staged.settingsPath) + // The exact two-entry shape, asserted whole rather than field by field: a + // stray extra key under `models.providers.` is as wrong as a missing + // one, since this is the shape LLP 0167 verified live. + assert.deepEqual(written.models.providers, { + anthropic: { + baseUrl: 'http://127.0.0.1:18521', + headers: { 'x-hypaware-upstream': 'anthropic' }, + models: [], + }, + openai: { + // The asymmetry: OpenClaw's Anthropic client appends `/v1/messages` + // itself and wants the bare origin, its OpenAI client appends only + // `/responses` or `/chat/completions` and needs the `/v1` baked in. + baseUrl: 'http://127.0.0.1:18521/v1', + headers: { 'x-hypaware-upstream': 'openai' }, + models: [], + }, + }) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach preserves every other key in openclaw.json (R1)', async () => { + const staged = await stage({ + $schema: 'https://openclaw.dev/schema.json', + theme: 'dark', + models: { + default: 'anthropic/claude-opus-4', + providers: { azure: { baseUrl: 'https://azure.example', models: [] } }, + }, + }) + try { + await runAttach(staged) + + const written = await readConfig(staged.settingsPath) + assert.equal(written.$schema, 'https://openclaw.dev/schema.json') + assert.equal(written.theme, 'dark') + // Both the sibling `models` key and the unrelated provider survive: attach + // owns two keys under `models.providers` and touches nothing else. + assert.equal(written.models.default, 'anthropic/claude-opus-4') + assert.deepEqual(written.models.providers.azure, { + baseUrl: 'https://azure.example', + models: [], + }) + assert.deepEqual(Object.keys(written.models.providers).sort(), ['anthropic', 'azure', 'openai']) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach refuses without writing when a provider key already exists (R2)', async () => { + for (const existingKey of ['anthropic', 'openai']) { + const before = { + models: { providers: { [existingKey]: { baseUrl: 'https://mine.example', models: [] } } }, + } + const staged = await stage(before) + try { + const { outcome, stdout } = await runAttach(staged) + + assert.equal(outcome.status, 'failed') + assert.match( + outcome.status === 'failed' ? outcome.reason : '', + new RegExp(`models\\.providers\\.${existingKey} already exists`) + ) + // The reason has to be actionable, not just a diagnosis. + assert.match(outcome.status === 'failed' ? outcome.reason : '', /hyp detach --client openclaw/) + assert.match(stdout, /did not apply/) + + // Pure read-then-decide: the file is byte-identical to what was staged. + // A partial write here is the failure mode the ordering exists to + // prevent, and it would not show up in the returned status. + assert.deepEqual(await readConfig(staged.settingsPath), before) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } + } +}) + +// The other half of R2, and the one the presence-only refusal got wrong: the +// entry HypAware itself wrote is not a user override, so re-attaching over it +// must succeed. `action_attach.js`'s `isCurrent()` re-performs attach whenever +// the daemon rebound to a new ephemeral port (LLP 0086) or the contributed +// asset set changed (LLP 0107), and its own contract says `perform()` is +// idempotent. Refusing there churned the marker to `failed` and left +// `openclaw.json` pointing at the dead port while the marker-header probe still +// reported `attached: true`. +// @ref LLP 0086#re-attach-on-drift [tests]: a second attach at a moved endpoint +// rewrites the entries the first one wrote rather than refusing over them +test('a second attach at a moved endpoint rewrites both baseUrls (re-attach on drift)', async () => { + const staged = await stage({ theme: 'dark', models: { providers: {} } }) + try { + const first = await runAttach(staged) + assert.deepEqual(first.outcome, { status: 'done' }) + + // The ephemeral-port rebind `isCurrent()` exists to catch. + const moved = 'http://127.0.0.1:4111' + const stdout = makeBuf() + const stderr = makeBuf() + const outcome = await createOpenclawAttach({ homeDir: staged.homeDir, env: {} }).attach( + /** @type {any} */ ({ endpoint: moved, config: {}, stdout, stderr, json: false }) + ) + assert.deepEqual(outcome, { status: 'done' }) + + const written = await readConfig(staged.settingsPath) + assert.equal(written.models.providers.anthropic.baseUrl, moved) + assert.equal(written.models.providers.openai.baseUrl, `${moved}/v1`) + // Still exactly the two entries, still the marker headers, still the rest + // of the file: a rewrite is not a merge. + assert.deepEqual(written.models.providers.anthropic.headers, { 'x-hypaware-upstream': 'anthropic' }) + assert.deepEqual(written.models.providers.openai.models, []) + assert.equal(written.theme, 'dark') + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +// R2 must survive the ownership rule: everything that is not *exactly* the +// entry attach writes is still somebody else's, and still refuses. These are +// the near misses, not the obvious cases the test above covers. +test('an entry that is not ours still refuses, however close it looks (R2)', async () => { + /** @type {Array<[string, unknown]>} */ + const notOurs = [ + // A deliberate "route nothing here" override: bare presence, no shape. + ['null', null], + // The marker header, but a hand-edited model list: not the shape attach + // produces, so not an entry attach may silently replace. + ['marker header but a hand-edited models list', { + baseUrl: 'http://127.0.0.1:4000', + headers: { 'x-hypaware-upstream': 'anthropic' }, + models: ['claude-opus-4'], + }], + // The marker header naming a *different* upstream: whatever wrote this, it + // is not the anthropic entry attach writes at this key. + ['marker header naming another key', { + baseUrl: 'http://127.0.0.1:4000', + headers: { 'x-hypaware-upstream': 'openai' }, + models: [], + }], + // Our exact shape minus the marker: the marker is the whole ownership + // claim, so without it this is a user pointing at a local proxy of theirs. + ['no marker header', { baseUrl: 'http://127.0.0.1:4000', models: [] }], + ] + for (const [label, entry] of notOurs) { + const before = { models: { providers: { anthropic: entry } } } + const staged = await stage(before) + try { + const { outcome } = await runAttach(staged) + assert.equal(outcome.status, 'failed', label) + assert.match(outcome.status === 'failed' ? outcome.reason : '', /models\.providers\.anthropic already exists/) + // Pure read-then-decide still: nothing partially written over a refusal. + assert.deepEqual(await readConfig(staged.settingsPath), before, label) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } + } +}) + +test('attach never throws on refusal, so attach-on-join warns instead of failing', async () => { + const staged = await stage({ + models: { providers: { openai: { baseUrl: 'https://mine.example', models: [] } } }, + }) + try { + // Deliberately unguarded by assert.rejects/doesNotReject wrappers: any + // throw fails the test outright, which is the assertion. The reconciler's + // `perform()` turns a throw into a `failed` outcome for the *whole join* + // action, so the refusal has to come back as a value. + const { outcome } = await runAttach(staged, { json: true }) + assert.equal(outcome.status, 'failed') + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach prints the openclaw gateway restart instruction on the human path (R4)', async () => { + const staged = await stage({ models: { providers: {} } }) + try { + const { stdout } = await runAttach(staged) + assert.match(stdout, /openclaw gateway restart/) + assert.match(stdout, /OpenClaw attached/) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach prints the restart instruction on the --json path too (R4)', async () => { + const staged = await stage({ models: { providers: {} } }) + try { + const { stdout } = await runAttach(staged, { json: true }) + const payload = JSON.parse(stdout.trim()) + assert.equal(payload.status, 'ok') + assert.equal(payload.action, 'attach') + assert.equal(payload.client, 'openclaw') + assert.equal(payload.changed, true) + assert.equal(payload.settings_path, staged.settingsPath) + // A scripted caller is as blocked on the restart as a human is, so the + // instruction is a field, not prose it would have to scrape. + assert.equal(payload.restart_required, true) + assert.equal(payload.restart_command, 'openclaw gateway restart') + assert.match(payload.message, /openclaw gateway restart/) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach --dry-run reports the write without touching the file', async () => { + const before = { models: { providers: {} } } + const staged = await stage(before) + try { + const { outcome, stdout } = await runAttach(staged, { dryRun: true }) + assert.deepEqual(outcome, { status: 'done' }) + assert.match(stdout, /\(dry-run\) Would attach OpenClaw/) + assert.match(stdout, /openclaw gateway restart/) + assert.deepEqual(await readConfig(staged.settingsPath), before) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach --dry-run reports the refusal it would hit, not a write it would not do', async () => { + const staged = await stage({ + models: { providers: { anthropic: { baseUrl: 'https://mine.example', models: [] } } }, + }) + try { + const { outcome } = await runAttach(staged, { dryRun: true }) + assert.equal(outcome.status, 'failed') + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('attach resolves openclaw.json through $OPENCLAW_HOME when set', async () => { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-openclaw-attach-home-')) + const openclawHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-openclaw-attach-oc-')) + try { + const settingsPath = path.join(openclawHome, 'openclaw.json') + await fs.writeFile(settingsPath, JSON.stringify({ models: { providers: {} } }, null, 2)) + + const stdout = makeBuf() + const stderr = makeBuf() + const attacher = createOpenclawAttach({ homeDir, env: { OPENCLAW_HOME: openclawHome } }) + const outcome = await attacher.attach( + /** @type {any} */ ({ endpoint: ENDPOINT, config: {}, stdout, stderr, json: true }) + ) + + assert.deepEqual(outcome, { status: 'done' }) + assert.equal(JSON.parse(stdout.text().trim()).settings_path, settingsPath) + const written = JSON.parse(await fs.readFile(settingsPath, 'utf8')) + assert.equal(written.models.providers.anthropic.baseUrl, ENDPOINT) + } finally { + await fs.rm(homeDir, { recursive: true, force: true }) + await fs.rm(openclawHome, { recursive: true, force: true }) + } +}) + +test('a missing openclaw.json is a hard failure, not a config attach invents', async () => { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-openclaw-attach-missing-')) + try { + const { outcome } = await runAttach({ homeDir }) + assert.equal(outcome.status, 'failed') + assert.match(outcome.status === 'failed' ? outcome.reason : '', /does not exist/) + // Attach cannot reason about a config it cannot read, and creating one + // would hand OpenClaw a file it never had. + await assert.rejects(fs.stat(path.join(homeDir, '.openclaw', 'openclaw.json'))) + } finally { + await fs.rm(homeDir, { recursive: true, force: true }) + } +}) + +test('a malformed openclaw.json is a hard failure, and is left alone', async () => { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-openclaw-attach-bad-')) + try { + const settingsPath = path.join(homeDir, '.openclaw', 'openclaw.json') + await fs.mkdir(path.dirname(settingsPath), { recursive: true }) + await fs.writeFile(settingsPath, '{ not json') + + const { outcome } = await runAttach({ homeDir }) + assert.equal(outcome.status, 'failed') + assert.match(outcome.status === 'failed' ? outcome.reason : '', /malformed JSON/) + assert.equal(await fs.readFile(settingsPath, 'utf8'), '{ not json') + } finally { + await fs.rm(homeDir, { recursive: true, force: true }) + } +}) + +test('a trailing slash on the endpoint does not double the openai /v1 separator', async () => { + const staged = await stage({ models: { providers: {} } }) + try { + const stdout = makeBuf() + const stderr = makeBuf() + const attacher = createOpenclawAttach({ homeDir: staged.homeDir, env: {} }) + await attacher.attach( + /** @type {any} */ ({ endpoint: `${ENDPOINT}/`, config: {}, stdout, stderr }) + ) + + const written = await readConfig(staged.settingsPath) + assert.equal(written.models.providers.anthropic.baseUrl, ENDPOINT) + assert.equal(written.models.providers.openai.baseUrl, `${ENDPOINT}/v1`) + } finally { + await fs.rm(staged.homeDir, { recursive: true, force: true }) + } +}) diff --git a/test/plugins/openclaw-backfill.test.js b/test/plugins/openclaw-backfill.test.js index 5320eda8..32fc4ea3 100644 --- a/test/plugins/openclaw-backfill.test.js +++ b/test/plugins/openclaw-backfill.test.js @@ -64,6 +64,24 @@ function messageLine(fields) { } } +/** + * How far `writeSession` backdates a fixture's mtime below "now" (LLP + * 0170#decision, LLP 0172#45-the-quiesce-window). Most tests below disable + * the quiesce gate with `config.backfill.quiesce_ms: 0` (the `provider()` + * default), which only means "no margin required," not "no comparison at + * all": `listSessionFiles` still checks `stat.mtimeMs <= Date.now()`, and a + * file written moments earlier can race that later `Date.now()` call across + * two different clocks (the filesystem's mtime clock and V8's), occasionally + * losing (#570: several of these tests flaked to 0 projected items on a CI + * runner where that race went the wrong way, though never locally). A small + * backdate removes the race with a wide margin while staying far below the + * real 180000ms default quiesce window, so the "fresh vs. three-minutes-old" + * tests below (which never call `provider()`, and rely on genuine freshness + * against that default) are unaffected, and any test that wants a specific + * age still gets the last word by calling `ageFile` itself afterward. + */ +const FIXTURE_MTIME_MARGIN_MS = 2_000 + /** * Write one `~/.openclaw/agents//sessions/.jsonl`. * @@ -95,6 +113,7 @@ async function writeSession(env, doc) { } for (const record of doc.records ?? []) lines.push(JSON.stringify(messageLine(record))) await fs.writeFile(filePath, lines.join('\n') + '\n', 'utf8') + await ageFile(filePath, FIXTURE_MTIME_MARGIN_MS) return filePath } @@ -211,11 +230,36 @@ const ASSISTANT_RECORD = { } /** + * `config.backfill.quiesce_ms: 0` by default: every test below writes a + * session file and runs the provider against it moments later, well inside + * the real 180000ms default quiesce window (LLP 0172#45-the-quiesce-window). Without this + * override every test in this file would scan to zero files, for a reason + * that has nothing to do with what each test actually checks. Tests that + * exercise the quiesce window itself pass their own `config` (or none, to + * exercise the real default), which fully replaces this one rather than + * merging with it. + * * @param {{ homeDir: string }} env - * @param {{ resolver?: any, env?: NodeJS.ProcessEnv }} [opts] + * @param {{ resolver?: any, env?: NodeJS.ProcessEnv, config?: any }} [opts] */ function provider(env, opts = {}) { - return createOpenclawBackfillProvider({ homeDir: env.homeDir, ...opts }) + return createOpenclawBackfillProvider({ + homeDir: env.homeDir, + config: { backfill: { quiesce_ms: 0 } }, + ...opts, + }) +} + +/** + * Back-date `filePath`'s mtime by `msAgo` milliseconds, so a quiesce-window + * test can control file recency without waiting on the wall clock. + * + * @param {string} filePath + * @param {number} msAgo + */ +async function ageFile(filePath, msAgo) { + const past = new Date(Date.now() - msAgo) + await fs.utimes(filePath, past, past) } // --------------------------------------------------------------------------- @@ -676,14 +720,19 @@ test('a relocated install is found through OPENCLAW_HOME, the same way settlemen const openclawHome = path.join(env.homeDir, 'elsewhere') const dir = path.join(openclawHome, 'agents', 'main', 'sessions') await fs.mkdir(dir, { recursive: true }) + const filePath = path.join(dir, 'sess-relocated.jsonl') await fs.writeFile( - path.join(dir, 'sess-relocated.jsonl'), + filePath, [ JSON.stringify({ type: 'session', id: 'sess-relocated', cwd: '/work/repo', timestamp: '2026-07-30T10:00:00.000Z' }), JSON.stringify(messageLine(ASSISTANT_RECORD)), ].join('\n') + '\n', 'utf8' ) + // Written outside `writeSession`, so it needs its own backdate to clear + // the same mtime-vs-`Date.now()` race `FIXTURE_MTIME_MARGIN_MS` guards + // against there (#570). + await ageFile(filePath, FIXTURE_MTIME_MARGIN_MS) // Nothing at $HOME/.openclaw: only the override names a real install. const { items } = await collect( provider(env, { env: { OPENCLAW_HOME: openclawHome } }).run(runContext().ctx) @@ -749,3 +798,150 @@ test('reruns are deterministic: the same session yields byte-identical row ident await env.cleanup() } }) + +// --------------------------------------------------------------------------- +// Lane B: sweep scheduling metadata (LLP 0172#lane-b-sweep, LLP 0173 T7) +// --------------------------------------------------------------------------- + +test('sweep.cron reads the configured backfill.sweep_cron value', async () => { + const env = await stageEnv() + try { + const contribution = createOpenclawBackfillProvider({ + homeDir: env.homeDir, + config: { backfill: { sweep_cron: '*/10 * * * *' } }, + }) + assert.deepEqual(contribution.sweep, { cron: '*/10 * * * *' }) + } finally { + await env.cleanup() + } +}) + +test('sweep.cron falls back to the every-5-minutes default when config is absent', async () => { + const env = await stageEnv() + try { + assert.deepEqual(createOpenclawBackfillProvider({ homeDir: env.homeDir }).sweep, { + cron: '*/5 * * * *', + }) + // Also absent when `config` is present but carries no `backfill` section, + // and when `backfill` is present but carries no `sweep_cron` key. + assert.deepEqual(createOpenclawBackfillProvider({ homeDir: env.homeDir, config: {} }).sweep, { + cron: '*/5 * * * *', + }) + assert.deepEqual( + createOpenclawBackfillProvider({ homeDir: env.homeDir, config: { backfill: {} } }).sweep, + { cron: '*/5 * * * *' } + ) + } finally { + await env.cleanup() + } +}) + +// --------------------------------------------------------------------------- +// LLP 0172#45-the-quiesce-window / LLP 0170#decision: the quiesce window +// --------------------------------------------------------------------------- + +// @ref LLP 0170#decision [tests]: a sweep (or any run) must not import a +// session file still inside the quiesce window, so it never races a file +// OpenClaw is still mid-write on. +test('a session file with mtime inside the quiesce window is excluded from the run', async () => { + const env = await stageEnv() + try { + const filePath = await writeSession(env, { header: { cwd: '/work/repo' }, records: [USER_RECORD, ASSISTANT_RECORD] }) + await ageFile(filePath, 1_000) // 1s old, well inside a 5s window + const { items } = await collect( + provider(env, { config: { backfill: { quiesce_ms: 5_000 } } }).run(runContext().ctx) + ) + assert.equal(items.length, 0) + } finally { + await env.cleanup() + } +}) + +test('a session file with mtime outside the quiesce window is included', async () => { + const env = await stageEnv() + try { + const filePath = await writeSession(env, { header: { cwd: '/work/repo' }, records: [USER_RECORD, ASSISTANT_RECORD] }) + await ageFile(filePath, 60_000) // 60s old, outside a 5s window + const { items } = await collect( + provider(env, { config: { backfill: { quiesce_ms: 5_000 } } }).run(runContext().ctx) + ) + assert.equal(items.length, 1) + } finally { + await env.cleanup() + } +}) + +// @ref LLP 0172#45-the-quiesce-window [tests]: the default is the cited constant +// (QUERY_FLUSH_DEBOUNCE_MS + one minute margin), not a re-guessed number. +test('the quiesce window defaults to exactly 180000ms when config.backfill.quiesce_ms is absent', async () => { + const env = await stageEnv() + try { + await writeSession(env, { header: { cwd: '/work/repo' }, records: [ASSISTANT_RECORD] }) + const { ctx, entries } = runContext() + // No config at all: exercises the real default, not the harness's + // quiesce-disabling override. + await collect(createOpenclawBackfillProvider({ homeDir: env.homeDir }).run(ctx)) + const started = entries.find((e) => e.message === 'openclaw.backfill.scan_started') + assert.ok(started, 'scan_started must be logged') + assert.equal(started.fields.quiesce_ms, 180_000) + } finally { + await env.cleanup() + } +}) + +// The same default, proven behaviorally rather than through the log field: a +// freshly-written file (mtime "now") is inside the real default window and a +// file backdated well past 180s is outside it. +test('the default quiesce window excludes a fresh file and includes one older than three minutes', async () => { + const env = await stageEnv() + try { + await writeSession(env, { agentId: 'fresh', header: { cwd: '/work/repo' }, records: [ASSISTANT_RECORD] }) + const oldFilePath = await writeSession(env, { + agentId: 'old', + sessionId: 'sess-old', + header: { cwd: '/work/repo' }, + records: [ASSISTANT_RECORD], + }) + await ageFile(oldFilePath, 4 * 60 * 1000) // 4 minutes old, outside 180000ms + const { items } = await collect(createOpenclawBackfillProvider({ homeDir: env.homeDir }).run(runContext().ctx)) + assert.deepEqual(items.map((item) => value(item).session_id), ['sess-old']) + } finally { + await env.cleanup() + } +}) + +// R10 composition: the quiesce filter operates on file recency only, and +// must not disturb the existing CLI-backend forward/backward-fill logic once +// a file clears the window. +test('a file outside the quiesce window still goes through the CLI-backend allowlist unchanged', async () => { + const env = await stageEnv() + try { + const filePath = await writeSession(env, { + header: { cwd: '/work/repo' }, + records: [ + USER_RECORD, + ASSISTANT_RECORD, + { + id: 'msg-asst-2', + timestamp: '2026-07-30T10:01:05.000Z', + role: 'assistant', + content: [{ type: 'text', text: 'done' }], + model: 'claude-cli/sonnet', + provider: 'claude-cli', + api: 'anthropic-messages', + }, + ], + }) + await ageFile(filePath, 60_000) + const { items, events } = await collect( + provider(env, { config: { backfill: { quiesce_ms: 5_000 } } }).run(runContext().ctx) + ) + const exchange = value(items[0]) + assert.deepEqual(exchange.messages.map((/** @type {any} */ m) => m.message_id), ['msg-user-1', 'msg-asst-1']) + const excluded = events.filter((e) => e.event === 'excluded_backend') + assert.equal(excluded.length, 1) + assert.equal(excluded[0].attributes?.provider, 'claude-cli') + } finally { + await env.cleanup() + } +}) diff --git a/test/plugins/openclaw-client-registration.test.js b/test/plugins/openclaw-client-registration.test.js index 7d3bd755..943b864f 100644 --- a/test/plugins/openclaw-client-registration.test.js +++ b/test/plugins/openclaw-client-registration.test.js @@ -1,23 +1,28 @@ // @ts-check import assert from 'node:assert/strict' -import { mkdtempSync, rmSync } from 'node:fs' +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import path from 'node:path' import test from 'node:test' import { activate } from '../../hypaware-core/plugins-workspace/openclaw/src/index.js' +// Fixture setup only for the real-attach detach case below: the write this +// stages is the exact real-world undo record detachClientFromDisk reverses, +// mirroring client-detach-json-path.test.js's own use of the real effect. +import { createOpenclawAttach } from '../../hypaware-core/plugins-workspace/openclaw/src/attach.js' import { buildClientDescriptorMap, runAttach, runDetach } from '../../src/core/commands/clients.js' +import { createAttachHandler } from '../../src/core/config/action_attach.js' /** - * LLP 0161 3.3: retiring `src/settings.js`'s settings-file write means - * `attach()` becomes an honest no-op, but `gateway.registerClient({ name: - * 'openclaw', ... })` stays registered so the manual `hyp attach openclaw` - * / `hyp detach openclaw` / `hyp clients openclaw` commands keep resolving - * the client instead of erroring `unknown client 'openclaw'` (a real - * discoverability regression the manifest's `attach_probe` removal alone - * would not cause, since `hyp clients`/`hyp attach` resolve `getClient()` - * directly and do not gate on `attachProbe`). + * `gateway.registerClient({ name: 'openclaw', ... })` is what makes the + * manual `hyp attach openclaw` / `hyp detach openclaw` / `hyp clients + * openclaw` commands resolve the client instead of erroring `unknown client + * 'openclaw'`; those commands resolve `getClient()` directly and do not gate + * on `attachProbe`. These tests cover the registration itself and the wiring + * from `activate()` to the real effect in `attach.js` (LLP 0169 restored the + * settings write LLP 0161 had retired); `openclaw-attach.test.js` covers the + * effect's own contract. * * @import { CommandRunContext } from '../../hypaware-plugin-kernel-types.js' */ @@ -36,11 +41,24 @@ function makeBuf() { } } -test('activate() registers the openclaw client with an honest no-op attach()', async () => { +/** + * Stage a HOME with an OpenClaw config in it, and the activation context + * pointed at that HOME so `activate()`'s attach writes into the temp tree + * rather than the developer's own `~/.openclaw`. + * + * @param {Record} config + * @param {Record} [pluginConfig] the plugin's own validated + * `config` slice, i.e. `ctx.config` + * @returns {{ homeDir: string, settingsPath: string, ctx: any, client(): any }} + */ +function stageActivation(config, pluginConfig) { + const homeDir = mkdtempSync(path.join(tmpdir(), 'hyp-openclaw-activate-')) + const settingsPath = path.join(homeDir, '.openclaw', 'openclaw.json') + mkdirSync(path.dirname(settingsPath), { recursive: true }) + writeFileSync(settingsPath, JSON.stringify(config, null, 2)) + /** @type {any} */ let registeredClient - /** @type {any} */ - let registeredBackfill const gateway = /** @type {any} */ ({ registerUpstreamPreset() {}, registerExchangeProjector() {}, @@ -50,67 +68,201 @@ test('activate() registers the openclaw client with an honest no-op attach()', a }, }) const ctx = /** @type {any} */ ({ - env: {}, + env: { HOME: homeDir }, plugin: { version: '0.0.0-test' }, + // `ctx.config` is the kernel's already-validated slice of this plugin's own + // `config` block (LLP 0037), the same shape `config.js` checks. + config: pluginConfig ?? {}, configRegistry: { registerSection() {} }, - backfills: { register(contribution) { registeredBackfill = contribution } }, + backfills: { register() {} }, requireCapability: () => gateway, }) + return { homeDir, settingsPath, ctx, client: () => registeredClient } +} - await activate(ctx) +test('activate() registers the openclaw client and wires attach() to the real write', async () => { + /** @type {any} */ + let registeredBackfill + const staged = stageActivation({ models: { providers: {} } }) + staged.ctx.backfills = { register(/** @type {any} */ contribution) { registeredBackfill = contribution } } + try { + await activate(staged.ctx) - assert.ok(registeredClient, 'activate() registered a client') - assert.equal(registeredClient.name, 'openclaw') - assert.equal(registeredClient.defaultUpstream, 'anthropic') + const registeredClient = staged.client() + assert.ok(registeredClient, 'activate() registered a client') + assert.equal(registeredClient.name, 'openclaw') + assert.equal(registeredClient.defaultUpstream, 'anthropic') - // The session-transcript backfill provider rides the same activation, the - // imperative `ctx.backfills.register(...)` house pattern @hypaware/codex - // already follows. @ref LLP 0161#backfill-provider [tests] - assert.ok(registeredBackfill, 'activate() registered a backfill provider') - assert.equal(registeredBackfill.name, 'openclaw') - assert.equal(registeredBackfill.plugin, '@hypaware/openclaw') - assert.deepEqual(registeredBackfill.datasets, ['ai_gateway_messages']) + // The session-transcript backfill provider rides the same activation, the + // imperative `ctx.backfills.register(...)` house pattern @hypaware/codex + // already follows. @ref LLP 0161#backfill-provider [tests] + assert.ok(registeredBackfill, 'activate() registered a backfill provider') + assert.equal(registeredBackfill.name, 'openclaw') + assert.equal(registeredBackfill.plugin, '@hypaware/openclaw') + assert.deepEqual(registeredBackfill.datasets, ['ai_gateway_messages']) - const stdout = makeBuf() - const stderr = makeBuf() - await registeredClient.attach({ endpoint: 'http://127.0.0.1:4317', stdout, stderr, dryRun: false, json: false }) + const stdout = makeBuf() + const stderr = makeBuf() + await registeredClient.attach({ endpoint: 'http://127.0.0.1:4317', stdout, stderr, dryRun: false, json: false }) - // Writes nothing to the "settings file" (there is none any more); it only - // reports that routing is owned by the steering plugin. - assert.match(stdout.text(), /openclaw-steering-plugin/) - assert.match(stdout.text(), /openclaw plugins install/) + // Registration is wired to `attach.js`'s effect, not to the retired no-op: + // the two provider entries land on disk and the user is told to restart. + // The entries' exact shape is `openclaw-attach.test.js`'s business. + // @ref LLP 0169#decision [tests] + const written = JSON.parse(readFileSync(staged.settingsPath, 'utf8')) + assert.deepEqual(Object.keys(written.models.providers).sort(), ['anthropic', 'openai']) + assert.match(stdout.text(), /openclaw gateway restart/) + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } }) -test('activate() attach() emits the same report under --json, still writing nothing', async () => { +// The wiring, not the resolver. `backfill.js` has always read `sweep_cron` and +// `quiesce_ms` off the `config` it is handed, and its own unit tests hand that +// config straight to the factory, so they could not see that `activate()` never +// passed one: both keys were validated by `config.js` on the way in and then +// silently discarded, and every install got the hardcoded defaults no matter +// what the operator configured. `ctx.config` is the seam, so the assertion has +// to start from an activation. +// @ref LLP 0172#lane-b-sweep [tests]: the registered contribution's `sweep` is +// populated from the plugin's own validated config, not from the default +test('activate() threads ctx.config into the backfill provider (sweep_cron and quiesce_ms)', async () => { /** @type {any} */ - let registeredClient - const gateway = /** @type {any} */ ({ - registerUpstreamPreset() {}, - registerExchangeProjector() {}, - registerSettlementEnricher() {}, - registerClient(client) { - registeredClient = client - }, + let registeredBackfill + const staged = stageActivation( + { models: { providers: {} } }, + { backfill: { sweep_cron: '*/30 * * * *', quiesce_ms: 777 } } + ) + staged.ctx.backfills = { register(/** @type {any} */ contribution) { registeredBackfill = contribution } } + try { + await activate(staged.ctx) + + assert.ok(registeredBackfill, 'activate() registered a backfill provider') + // The configured cadence is the one the daemon's sweep driver will match + // against, not `*/5 * * * *`. + assert.deepEqual(registeredBackfill.sweep, { cron: '*/30 * * * *' }) + + // `quiesce_ms` has no contribution-level surface, so read it off the run's + // own `scan_started` record (the same field openclaw-backfill.test.js + // asserts the 180000ms default on). An empty HOME is enough: the record is + // emitted before any file is read. + /** @type {Array<{ message: string, fields: any }>} */ + const entries = [] + /** @param {string} message @param {any} fields */ + const record = (message, fields) => { entries.push({ message, fields }) } + const runCtx = /** @type {any} */ ({ + env: {}, + cacheRoot: path.join(staged.homeDir, 'cache-unused'), + dryRun: true, + storage: {}, + log: { debug: record, info: record, warn: record, error: record }, + }) + for await (const _yielded of registeredBackfill.run(runCtx)) { /* drain */ } + + const started = entries.find((entry) => entry.message === 'openclaw.backfill.scan_started') + assert.ok(started, 'the run logs scan_started') + assert.equal(started.fields.quiesce_ms, 777) + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } +}) + +test('activate() attach() reports the same write under --json', async () => { + const staged = stageActivation({ models: { providers: {} } }) + try { + await activate(staged.ctx) + + const stdout = makeBuf() + const stderr = makeBuf() + await staged.client().attach({ endpoint: 'http://127.0.0.1:4317', stdout, stderr, dryRun: false, json: true }) + + const payload = JSON.parse(stdout.text().trim()) + assert.equal(payload.status, 'ok') + assert.equal(payload.action, 'attach') + assert.equal(payload.client, 'openclaw') + assert.equal(payload.changed, true) + assert.equal(payload.settings_path, staged.settingsPath) + assert.equal(payload.restart_command, 'openclaw gateway restart') + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } +}) + +// @ref LLP 0172#lane-a-attach [tests]: a refusal reaches the reconciler as a +// retryable failure. The kernel types the registered `attach()` as +// `Promise`, so the effect's returned outcome reaches no caller and the +// wrapper has to rethrow it. Returning quietly recorded a `done` marker that +// `isCurrent()` then matched forever, so the join never re-attached after the +// user cleared the conflicting entry, and `hyp attach` exited 0 on a refusal. +test('activate() attach() rethrows a refusal so the join records a retryable failure', async () => { + const staged = stageActivation({ + models: { providers: { anthropic: { baseUrl: 'https://mine.example', models: [] } } }, }) - const ctx = /** @type {any} */ ({ - env: {}, - plugin: { version: '0.0.0-test' }, - configRegistry: { registerSection() {} }, - backfills: { register() {} }, - requireCapability: () => gateway, + try { + await activate(staged.ctx) + + const stdout = makeBuf() + const stderr = makeBuf() + await assert.rejects( + () => staged.client().attach({ endpoint: 'http://127.0.0.1:4317', stdout, stderr, json: true }), + /already exists/ + ) + + // The reported line is still written before the throw: the throw is what + // the void-typed callers can see, the payload is what a `--json` reader + // parses. + const payload = JSON.parse(stdout.text().trim()) + assert.equal(payload.status, 'failed') + assert.match(payload.reason, /already exists/) + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } +}) + +// The other half of the same clause: the throw must reach the reconciler as a +// recorded `failed` outcome and must NOT abort the join's other actions. +// `perform()`'s catch is the seam that converts it, so drive the real handler +// rather than asserting on the wrapper alone. +// @ref LLP 0172#lane-a-attach [tests]: refusal is recorded and retried, and the +// sibling client attached in the same pass still lands +test('a refused openclaw attach is a failed reconciler outcome, not a done marker, and the join continues', async () => { + const staged = stageActivation({ + models: { providers: { openai: { baseUrl: 'https://mine.example', models: [] } } }, }) - await activate(ctx) + try { + await activate(staged.ctx) + const openclaw = staged.client() - const stdout = makeBuf() - const stderr = makeBuf() - await registeredClient.attach({ endpoint: 'http://127.0.0.1:4317', stdout, stderr, dryRun: false, json: true }) - - const payload = JSON.parse(stdout.text().trim()) - assert.equal(payload.status, 'ok') - assert.equal(payload.action, 'attach') - assert.equal(payload.client, 'openclaw') - assert.equal(payload.changed, false) - assert.match(payload.routing_owned_by, /openclaw-steering-plugin/) + /** @type {string[]} */ + const attached = [] + const handler = createAttachHandler() + const ctx = /** @type {any} */ ({ + env: { HOME: staged.homeDir }, + endpoint: 'http://127.0.0.1:4317', + log: { info() {}, warn() {}, error() {} }, + clients: { + getClient(name) { + if (name === 'openclaw') return openclaw + if (name === 'sibling') return { name, async attach() { attached.push(name) } } + return undefined + }, + }, + }) + + const refused = await handler.perform({ requestKey: 'openclaw', params: { client: 'openclaw' } }, ctx) + assert.equal(refused.status, 'failed') + assert.match(String(refused.reason), /already exists/) + + // A `failed` outcome carries no marker detail to go `done` on, so the next + // pass re-performs rather than short-circuiting on `isCurrent`. + assert.equal(refused.detail, undefined) + + const sibling = await handler.perform({ requestKey: 'sibling', params: { client: 'sibling' } }, ctx) + assert.equal(sibling.status, 'done') + assert.deepEqual(attached, ['sibling']) + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } }) // @ref LLP 0161#settlement-enricher [tests]: the settlement enricher is @@ -194,10 +346,48 @@ test('hyp attach openclaw resolves the client and does not error unknown client' assert.doesNotMatch(stderr.text(), /unknown client/) }) +// The user-facing half of the same clause: a refusal must be distinguishable +// from a success by a script, which under exit 0 it was not. +// @ref LLP 0172#lane-a-attach [tests]: `hyp attach --client openclaw` exits +// nonzero when attach refuses +test('hyp attach openclaw exits nonzero when the attach refuses', async () => { + const staged = stageActivation({ + models: { providers: { anthropic: { baseUrl: 'https://mine.example', models: [] } } }, + }) + try { + await activate(staged.ctx) + const openclaw = staged.client() + + const stdout = makeBuf() + const stderr = makeBuf() + const gateway = /** @type {any} */ ({ + localEndpoint: () => 'http://127.0.0.1:4388', + getClient: (name) => (name === 'openclaw' ? openclaw : null), + listClients: () => [{ name: 'openclaw' }], + }) + const ctx = /** @type {CommandRunContext} */ (/** @type {any} */ ({ + stdout, + stderr, + env: { HOME: staged.homeDir, HYP_HOME: path.join(staged.homeDir, '.hyp') }, + config: { version: 2 }, + capabilities: { has: () => true, require: () => gateway }, + })) + + const code = await runAttach(['openclaw'], ctx) + + assert.equal(code, 1, stdout.text()) + assert.match(stderr.text(), /already exists/) + } finally { + rmSync(staged.homeDir, { recursive: true, force: true }) + } +}) + test('hyp detach openclaw resolves the client from the real manifest as an honest no-op', async () => { - // No attach_probe (R7) means detachClientFromDisk's no-probe guard fires: - // { changed: false }. The point here is resolution, not restoration - the - // command must not error `unknown client 'openclaw'`. + // The manifest declares an attach_probe again (LLP 0173 T5 reversed R7), so + // the no-op observed here on a fresh temp HOME is detachClientFromDisk's + // absent-settings-file guard ({ changed: false }, no .openclaw/openclaw.json + // to reverse), not a no-probe guard. The point here is resolution, not + // restoration - the command must not error `unknown client 'openclaw'`. const home = mkdtempSync(path.join(tmpdir(), 'hyp-openclaw-detach-')) try { const stdout = makeBuf() @@ -219,6 +409,57 @@ test('hyp detach openclaw resolves the client from the real manifest as an hones } }) +test('hyp detach openclaw reverses a real attach: the ownership-based json_path undo fires end to end', async () => { + // Companion to the honest-no-op case above: stage a real openclaw.json + // written by the actual attach() effect, then drive the same CLI entry + // point (buildClientDescriptorMap's real manifest descriptor -> + // detachClientFromDisk's json_path branch) and prove it is not a no-op + // once there is something on disk to reverse. + // @ref LLP 0172#lane-a-detach [tests]: the ownership-based json_path undo, + // exercised through the real manifest descriptor rather than a hand-built + // one, so the wiring this file is about (not just the core undo, which + // client-detach-json-path.test.js already covers directly) is proven too. + const home = mkdtempSync(path.join(tmpdir(), 'hyp-openclaw-detach-real-')) + try { + const settingsPath = path.join(home, '.openclaw', 'openclaw.json') + mkdirSync(path.dirname(settingsPath), { recursive: true }) + writeFileSync(settingsPath, JSON.stringify({ theme: 'dark', models: { providers: {} } }, null, 2)) + + const endpoint = 'http://127.0.0.1:18521' + const attachOutcome = await createOpenclawAttach({ homeDir: home, env: {} }).attach( + /** @type {any} */ ({ endpoint, stdout: makeBuf(), stderr: makeBuf(), dryRun: false, json: true }) + ) + assert.equal(attachOutcome.status, 'done') + + const stdout = makeBuf() + const stderr = makeBuf() + const ctx = /** @type {CommandRunContext} */ (/** @type {any} */ ({ + stdout, + stderr, + env: { HOME: home, HYP_HOME: path.join(home, '.hyp') }, + config: { version: 2, plugins: [{ name: '@hypaware/ai-gateway', config: { listen: '127.0.0.1:18521' } }] }, + })) + + const code = await runDetach(['openclaw', '--json'], ctx) + + assert.equal(code, 0, stderr.text()) + assert.doesNotMatch(stderr.text(), /unknown client/) + const payload = JSON.parse(stdout.text().trim()) + assert.equal(payload.status, 'ok') + assert.equal(payload.changed, true) + assert.equal(payload.settings_path, settingsPath) + assert.equal(payload.removed, endpoint) + + // The two entries attach wrote are gone; everything else in the file + // (the theme, the container itself) is untouched. + const written = JSON.parse(readFileSync(settingsPath, 'utf8')) + assert.deepEqual(written.models.providers, {}) + assert.equal(written.theme, 'dark') + } finally { + rmSync(home, { recursive: true, force: true }) + } +}) + test('hyp clients: the descriptor map behind client listing/status resolves openclaw', async () => { const home = mkdtempSync(path.join(tmpdir(), 'hyp-openclaw-clients-')) try { @@ -232,8 +473,16 @@ test('hyp clients: the descriptor map behind client listing/status resolves open assert.ok(descriptors.has('openclaw'), 'openclaw client descriptor resolves') const descriptor = descriptors.get('openclaw') assert.equal(descriptor?.plugin, '@hypaware/openclaw') - // R7: the manifest declares no attach_probe any more. - assert.equal(descriptor?.attachProbe, undefined) + // LLP 0173 T5: R7 is reversed (LLP 0167#deletion-inventory), and the + // manifest again declares a json_path attach_probe (design 1.4). + assert.deepEqual(descriptor?.attachProbe, { + format: 'json_path', + settings_file: '.openclaw/openclaw.json', + container_path: 'models.providers', + provider_keys: ['anthropic', 'openai'], + marker_header: 'x-hypaware-upstream', + cache_glob: 'agents/*/agent/models.json', + }) } finally { rmSync(home, { recursive: true, force: true }) } diff --git a/test/plugins/openclaw-config.test.js b/test/plugins/openclaw-config.test.js index e46f8a1c..6492f832 100644 --- a/test/plugins/openclaw-config.test.js +++ b/test/plugins/openclaw-config.test.js @@ -99,3 +99,61 @@ test('validateBackfillSection mounts errors at the caller-supplied pointer', () assert.equal(errors.length, 1) assert.equal(errors[0].pointer, '/plugins/0/config/backfill/window_days') }) + +// Lane B's scheduled-sweep tunables (LLP 0170#decision, LLP 0172#4.2): +// `sweep_cron` and `quiesce_ms` land in the same change as the existing +// `on_join`/`window_days` keys so the unknown-key rejection loop never +// treats whichever key's task merged second as unrecognized. +// +// @ref LLP 0170#decision [tests]: sweep_cron/quiesce_ms are validated +// together in the plugin's own backfill config section. +test('validateOpenclawConfig accepts sweep_cron and quiesce_ms', () => { + assert.deepEqual(validateOpenclawConfig({ backfill: { sweep_cron: '*/5 * * * *' } }), { ok: true }) + assert.deepEqual(validateOpenclawConfig({ backfill: { quiesce_ms: 180000 } }), { ok: true }) + assert.deepEqual(validateOpenclawConfig({ backfill: { quiesce_ms: 0 } }), { ok: true }) + assert.deepEqual( + validateOpenclawConfig({ + backfill: { on_join: true, window_days: 30, sweep_cron: '0 * * * *', quiesce_ms: 60000 }, + }), + { ok: true }, + ) +}) + +test('validateOpenclawConfig rejects an invalid sweep_cron', () => { + /** @type {unknown[]} */ + const cases = ['not-a-cron', '@hourly', '* * * *', '', 7, null, { every: 5 }] + for (const sweep_cron of cases) { + const result = validateOpenclawConfig({ backfill: { sweep_cron } }) + assert.equal(result.ok, false, `expected failure for sweep_cron=${JSON.stringify(sweep_cron)}`) + if (result.ok) continue + assert.equal(result.errors[0].pointer, '/backfill/sweep_cron') + } +}) + +test('validateOpenclawConfig rejects a negative quiesce_ms', () => { + const result = validateOpenclawConfig({ backfill: { quiesce_ms: -1 } }) + assert.equal(result.ok, false) + if (result.ok) return + assert.equal(result.errors[0].pointer, '/backfill/quiesce_ms') +}) + +test('validateOpenclawConfig rejects a non-integer quiesce_ms', () => { + /** @type {unknown[]} */ + const cases = [1.5, '180000', null, true] + for (const quiesce_ms of cases) { + const result = validateOpenclawConfig({ backfill: { quiesce_ms } }) + assert.equal(result.ok, false, `expected failure for quiesce_ms=${JSON.stringify(quiesce_ms)}`) + if (result.ok) continue + assert.equal(result.errors[0].pointer, '/backfill/quiesce_ms') + } +}) + +test('validateOpenclawConfig still rejects a genuinely unknown backfill key', () => { + const result = validateOpenclawConfig({ + backfill: { sweep_cron: '*/5 * * * *', quiesce_ms: 180000, bogus: true }, + }) + assert.equal(result.ok, false) + if (result.ok) return + assert.equal(result.errors.length, 1) + assert.equal(result.errors[0].pointer, '/backfill/bogus') +}) diff --git a/test/plugins/openclaw-manifest.test.js b/test/plugins/openclaw-manifest.test.js new file mode 100644 index 00000000..f0d83922 --- /dev/null +++ b/test/plugins/openclaw-manifest.test.js @@ -0,0 +1,84 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' +import path from 'node:path' +import { fileURLToPath } from 'node:url' + +import { loadManifests } from '../../src/core/manifest.js' + +/** + * Manifest-shape tests for `@hypaware/openclaw` (LLP 0173 T5): the + * restored `json_path` `attach_probe` block (design 1.4) parses to the + * exact fields Lane A's detach/read sides (T2, T3) and `attach()` (T4) + * already agree on, and the steering-plugin package this change set + * retires (LLP 0167#deletion-inventory) is named nowhere in the + * onboarding copy any more. + * + * @ref LLP 0172#lane-a-attach [tests]: attach_probe's exact field shape, the one this format needs closed by construction (#212) + */ + +const WORKSPACE = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + '../../hypaware-core/plugins-workspace' +) + +const STEERING_PLUGIN_RE = /openclaw-steering-plugin/ + +test('openclaw manifest loads and validates', async () => { + const { loaded, failed } = await loadManifests([path.join(WORKSPACE, 'openclaw')]) + assert.equal(failed.length, 0, failed.map((f) => f.message).join('; ')) + assert.equal(loaded.length, 1) + + const manifest = loaded[0].manifest + assert.equal(manifest.name, '@hypaware/openclaw') +}) + +test('openclaw contributes.client.attach_probe parses to the exact json_path shape', async () => { + const { loaded } = await loadManifests([path.join(WORKSPACE, 'openclaw')]) + const manifest = /** @type {any} */ (loaded[0].manifest) + const probe = manifest.contributes?.client?.attach_probe + + assert.deepEqual(probe, { + format: 'json_path', + settings_file: '.openclaw/openclaw.json', + container_path: 'models.providers', + provider_keys: ['anthropic', 'openai'], + marker_header: 'x-hypaware-upstream', + cache_glob: 'agents/*/agent/models.json', + }) +}) + +test('openclaw description and picker summary no longer reference the steering plugin', async () => { + const { loaded } = await loadManifests([path.join(WORKSPACE, 'openclaw')]) + const manifest = /** @type {any} */ (loaded[0].manifest) + + assert.doesNotMatch(manifest.description, STEERING_PLUGIN_RE) + + const picker = manifest.contributes?.picker ?? [] + assert.ok(picker.length > 0) + for (const row of picker) { + assert.doesNotMatch(row.summary ?? '', STEERING_PLUGIN_RE) + } +}) + +test('openclaw description and picker summary state the two capture tiers directly', async () => { + const { loaded } = await loadManifests([path.join(WORKSPACE, 'openclaw')]) + const manifest = /** @type {any} */ (loaded[0].manifest) + + assert.match(manifest.description, /live/i) + assert.match(manifest.description, /sweep|transcript/i) + + const summary = manifest.contributes?.picker?.[0]?.summary ?? '' + assert.match(summary, /live/i) + assert.match(summary, /sweep/i) +}) + +test('claude manifest onboarding copy names the claude-cli/ OpenClaw case', async () => { + const { loaded } = await loadManifests([path.join(WORKSPACE, 'claude')]) + const manifest = /** @type {any} */ (loaded[0].manifest) + + const summary = manifest.contributes?.picker?.[0]?.summary ?? '' + assert.match(summary, /claude-cli\//) + assert.match(summary, /OpenClaw/) +}) diff --git a/test/plugins/openclaw-steering-plugin.test.js b/test/plugins/openclaw-steering-plugin.test.js deleted file mode 100644 index fbc1e62e..00000000 --- a/test/plugins/openclaw-steering-plugin.test.js +++ /dev/null @@ -1,24 +0,0 @@ -// @ts-check - -// The `@hypaware/openclaw-steering-plugin` package keeps its unit tests beside -// its own source, because it is an npm package OpenClaw installs rather than a -// relative-import HypAware kernel plugin (LLP 0161#package-layout). But the -// repo's `npm test` is deliberately scoped to root `test/**/*.test.js` -// (CLAUDE.md), so nothing under `openclaw-steering-plugin/test/` was reaching -// the gate: the four-branch `resolveSteering` precedence (LLP 0162's single -// highest-complexity task, where a wrong branch is a wrong answer to "is this -// provider captured") and the credential-borrow contract (R3: never persisted, -// re-resolved every call) were unit-tested and then never run in CI. -// -// Importing the package's test modules registers their `test()` calls in this -// runner, so they run under `npm test` without either moving them out of the -// package or widening the runner's root. -// -// @ref LLP 0157#requirements [tests]: R2, R3, R4, R5 - the steering plugin's -// own unit tests, gated by the repo's test command. - -import '../../openclaw-steering-plugin/test/gateway_endpoint.test.js' -import '../../openclaw-steering-plugin/test/runtime_auth.test.js' -import '../../openclaw-steering-plugin/test/steering.test.js' -import '../../openclaw-steering-plugin/test/warning_ledger.test.js' -import '../../openclaw-steering-plugin/test/wire_parity.test.js' diff --git a/tsconfig.json b/tsconfig.json index 36f3b87d..9ca58d7e 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -16,7 +16,6 @@ "bin", "hypaware-plugin-kernel-types.d.ts", "hypaware-core", - "openclaw-steering-plugin", "scripts", "src", "test"