A hermetic smoke writes a real-shape OpenClaw session file (#555 item 1)

[philcunliffe]

Follow-up to PR #552. Actions item 1 only of issue #555; item 2 is deliberately not actioned (see below).

What landed

hypaware-core/smoke/flows/backfill_openclaw_fixture.js, the missing sibling of
backfill_codex_fixture.js / backfill_claude_fixture.js. It:

• boots @hypaware/ai-gateway + @hypaware/openclaw against a tmp HYP_HOME
• stages a session file under the fake HOME's .openclaw/agents/<agentId>/sessions/<sessionId>.jsonl
  with a type: "session" v3 header and two type: "message" records in the verified two-level
  shape: the record line states only id/message/parentId/timestamp/type, and
  role, content, model, provider, api, stopReason, usage are nested under message
• asserts the staged shape itself before asserting anything downstream of it, so a future edit
  cannot quietly flatten the fixture back into something the pre-OpenClaw backfill reads message fields one level too high, projecting 0 rows #552 reader would also pass
• drives hyp backfill openclaw through dispatch / createCommandRegistry, mirroring the Codex flow
• asserts the user-visible result: non-zero rows in ai_gateway_messages (the literal OpenClaw backfill projects 0 rows from real session files: message fields are read at the top level but OpenClaw nests them under 'message' #543
  regression, which reported status: ok with rows_written: 0), carrying the record lines'
  native message_ids, the envelope's content_text and model, and
  provider = anthropic / conversation_source = openclaw / client_name = openclaw
• asserts the internal telemetry the Codex/Claude flows assert: a backfill.provider_finish span
  and a backfill.finish log under the run's dev_run_id with matching row counts, a
  backfill.write span for ai_gateway_messages, plus the provider's own
  openclaw.backfill.session_projected log with identity_source = native
• asserts idempotency: a second run under a fresh run id writes zero new rows and the query
  still returns exactly two

hypaware-core/smoke/lib/openclaw_session_fixture.js holds the fixture writer
(openclawMessageLine / writeOpenclawSessionFixture), mirroring
test/plugins/openclaw-backfill.test.js's messageLine() and
test/plugins/openclaw-settlement.test.js's sessionFileLine() rather than inventing a shape.

Flows are resolved by filename by hypaware-core/smoke/lib/harness.js, so registering the flow is
the file itself; there is no separate index list to edit. No LLP changed: this is a test-coverage
gap, not a design change (no LLP in the 0157-0162 set states a smoke inventory).

Mutation-test evidence

The fix here is the test, so it has to be shown to have teeth. The two-level handling in
hypaware-core/plugins-workspace/openclaw/src/session_file.js was temporarily reverted to the
pre-#552 behavior (read every message field off the record line):

 function openclawMessageEnvelope(row) {
-  return isPlainObject(row.message) ? row.message : row
+  return row
 }

Under mutation, the new smoke FAILS with #543's exact signature (status: ok, rows_written: 0):

$ npm run smoke -- backfill_openclaw_fixture
> node ./hypaware-core/smoke/index.js backfill_openclaw_fixture

smoke backfill_openclaw_fixture: FAIL
backfill run 1: openclaw provider ok and wrote NON-ZERO rows (#543 reported ok with 0)
  value={
  "provider": "openclaw",
  "plugin": "@hypaware/openclaw",
  "datasets": [
    "ai_gateway_messages"
  ],
  "status": "ok",
  "items_seen": 0,
  "sessions_seen": 0,
  "rows_written": 0,
  "rows_skipped": 0
}
  smoke=backfill_openclaw_fixture dev_run_id=smoke-backfill_openclaw_fixture-2026-07-31T23-29-42-811Z-20779

Restored (git checkout -- hypaware-core/plugins-workspace/openclaw/src/session_file.js;
git status clean of the mutation), and it passes again:

$ git status --short
?? hypaware-core/smoke/flows/backfill_openclaw_fixture.js
?? hypaware-core/smoke/lib/openclaw_session_fixture.js

$ npm run smoke -- backfill_openclaw_fixture
> node ./hypaware-core/smoke/index.js backfill_openclaw_fixture

smoke backfill_openclaw_fixture: ok (dev_run_id=smoke-backfill_openclaw_fixture-2026-07-31T23-29-48-481Z-20919)

So the smoke would have caught #543.

Other checks in the worktree: npm test 3281 pass / 0 fail / 1 skipped; npm run typecheck clean;
npm run build:types clean.

Item 2 was deliberately NOT actioned

Issue #555 item 2 (the timestamp / id precedence asymmetry in session_file.js, where
timestamp resolves envelope-first via messageField while id resolves line-first) is left
exactly as merged. Quoting the issue's own reasoning:

Round 2 review pinned this with a dedicated test
(test/plugins/openclaw-session-file.test.js: 'the envelope wins for timestamp, the record line wins for id') and could construct no failure from the asymmetry itself: both consumers
(backfill.js, settle.js) read through the one shared reader, so they cannot disagree with each
other about which value wins. It was left pinned rather than changed, specifically to avoid a
behavior change with no evidence behind it in a final review round.

The issue asks to revisit it only "if a future change ever gives backfill.js and settle.js (or a
third consumer) independent access to the raw record for timestamp or id rather than going
through the shared reader." This PR adds no such consumer. The issue's "Non-issues (do not action)"
section was likewise left untouched.

Base branch note

This PR is stacked on fix/issue-543 (PR #552), not on master. PR #552 is still open and
unmerged, so on master the OpenClaw reader still takes message fields off the record line and this
smoke cannot pass there by construction. LLP 0173 records the same finding for its own T12: the
fixture "genuinely cannot be built correctly against the current tree." GitHub will retarget this
PR's base to master automatically when #552 merges. Merge #552 first.

Coordination with integration/openclaw-two-lane-capture (LLP 0173, task T12)

T12 on that branch also plans a backfill_openclaw_fixture, extended with a quiesce-window
dimension (mtime-based skipping of recently-touched session files) and a two-lane dedupe
assertion. T12 is currently blocked and explicitly held until #552 merges, which is the same
blocker this PR has.

This flow is written so T12 extends rather than replaces it:

• the fixture writer is factored out into smoke/lib/openclaw_session_fixture.js, importable by a
  second flow rather than private to this one
• writeOpenclawSessionFixture takes an optional caller-chosen mtimeMs (applied via fs.utimes),
  which is precisely the "controllable-mtime agents/<id>/sessions/ tree" T12 calls for. Nothing in
  this flow uses it; it is there so T12 does not have to fork the helper
• header, records, agentId, and sessionId are all parameterizable, so a quiesce case can
  stage a second session file under the same agents root
• naming is left open for a sibling flow (e.g. sweep_openclaw_quiesce) rather than claiming the
  generic name for this one case

Backlink for whoever merges: LLP 0173 task T12 ("the hermetic smoke gap", Section 9),
llp/0173-openclaw-two-lane-capture.plan.md on integration/openclaw-two-lane-capture.

Fixes #555

[philcunliffe]

Review round 1 (head 98ff30c)

Verdict: findings (2 minor, both fixed at af73aa2). No blockers. codex is not
available on this host, so this was a single-family review (code-review), not a dual review.

The PR adds the missing hermetic smoke that would have caught #543, plus a factored-out
fixture writer. Both findings are the same class of problem: an assertion that reads as a
gate but could not actually fail.

Finding 1 (minor) - hypaware-core/smoke/lib/openclaw_session_fixture.js:35

JSDoc claimed the fixture makes the envelope-first timestamp read observable; it could not.
The doc block said writing timestamp at both levels is "what makes the envelope-first read of
it observable", but the implementation copied the same destructured timestamp binding to
both the record line and the nested message. The two levels were byte-identical, so no
assertion could distinguish envelope-first from line-first precedence. The flow also asserted
nothing about timestamps at all. The one field LLP 0158 and issue #555 single out as
non-obvious was the one field the fixture provably did not exercise, while the comment
asserted the opposite.

Fixed (af73aa2): openclawMessageLine now takes an optional messageTimestamp that
overrides the nested message.timestamp only. The flow stages an assistant record with a
distinct line timestamp (...10:00:02Z) and envelope timestamp (...10:00:09Z), asserts the
two differ in the fixture, and asserts the projected row's message_created_at carries the
envelope value. Default behavior with no messageTimestamp is unchanged, since a real
session file does carry the same value at both levels.

Finding 2 (minor) - hypaware-core/smoke/flows/backfill_openclaw_fixture.js:260

The idempotency assertion would have gone silently vacuous. Run 2 checked only
status === 'ok' && rows_written === 0 under a comment claiming "all part_ids already
present". It does dedupe for real today (measured run-2 result: items_seen: 1, sessions_seen: 0, rows_written: 0, rows_skipped: 1, i.e. the provider re-yielded the exchange
and materializeItem returned zero rows). But a run that skipped the file entirely would
yield items_seen: 0, rows_written: 0 and pass identically. That is not hypothetical here:
the mtimeMs hook this PR deliberately adds exists so LLP 0173 task T12 can add
mtime-based quiesce-window skipping to this same fixture, which is precisely the change that
would hollow out this gate without failing it.

Fixed (af73aa2): the run-2 predicate now also requires items_seen >= 1 && rows_skipped >= 1,
pinning the part_id-dedupe mechanism the comment claims. Field names were verified against
src/core/commands/backfill.js.

Teeth check (re-run after the fixes)

The smoke is a regression gate, so the decisive question is whether it still fails when the
bug returns. Mutating openclawMessageEnvelope in
hypaware-core/plugins-workspace/openclaw/src/session_file.js to return row (the pre-#552
flat read) makes the smoke fail on its first assertion, reproducing #543's exact signature:

backfill run 1: openclaw provider ok and wrote NON-ZERO rows (#543 reported ok with 0)
value={"provider":"openclaw", ..., "status":"ok","items_seen":0,"sessions_seen":0,"rows_written":0,"rows_skipped":0}

Restored, the smoke passes. Checks at af73aa2: npm test 3281 pass / 0 fail (1 pre-existing
skip), npm run typecheck clean, target smoke green. CI at the new head is the authority.

Deliberately not raised

• The base branch. This PR targets fix/issue-543 rather than master on purpose: the reader
  fix it depends on is unmerged, so the smoke cannot pass on master by construction.
  OpenClaw backfill reads message fields one level too high, projecting 0 rows #552 must merge first, and GitHub will retarget this PR to master when it does.
• The timestamp/id precedence asymmetry in session_file.js, which issue Follow-up: deferred review findings from PR #552 #555 explicitly
  defers and pins with an existing test.
• The items in issue Follow-up: deferred review findings from PR #552 #555's "Non-issues (do not action)" section.

The head has moved to af73aa2, so the next tick reviews that head as round 2.

[philcunliffe]

Review round 2 (head af73aa2)

Verdict: findings (2 nits, both fixed at 3f8dc01). No blockers, no majors.
Single-family review (code-review); codex is not available on this host.

This round re-reviewed the whole head with particular attention to whether round 1's two
fixes were correct and complete. They are. Both new findings are about assertion precision
in the same telemetry block, which matters here because this file exists purely as a
regression gate.

Teeth check: PASS (two independent mutations)

The decisive question for this PR is whether the smoke actually fails when the bug returns.
Two separate mutations were applied to
hypaware-core/plugins-workspace/openclaw/src/session_file.js, each reverted with
git checkout -- and re-verified green afterwards:

(A) Revert the two-level read (openclawMessageEnvelope to return row). The smoke fails
fast on its first behavioral assertion, reproducing #543's exact signature:

backfill run 1: openclaw provider ok and wrote NON-ZERO rows (#543 reported ok with 0)
value={"provider":"openclaw","status":"ok","items_seen":0,"sessions_seen":0,"rows_written":0,"rows_skipped":0}

(B) Isolate round 1's fix (flip messageField(envelope, row, 'timestamp', ...) to
messageField(row, envelope, ...), line-first for that one field only). Every earlier
assertion still passes and the smoke fails precisely on the new assertion, with
message_created_at showing the record-line value 2026-05-20T10:00:02.000Z instead of the
envelope value ...10:00:09.000Z. This confirms the envelope-precedence assertion pins what
it claims and is not passing for an unrelated reason.

expect.that throws on first failure, so the flow is fail-fast and no later assertion can
mask an earlier one.

Finding 1 (nit) - hypaware-core/smoke/flows/backfill_openclaw_fixture.js:347

The session_projected filter depended on emission order rather than identity. It matched
on l.body alone and then read projected[0], while its three sibling telemetry assertions
all pin dev_run_id. Because the idempotency rerun deliberately sets a fresh DEV_RUN_ID,
two matching records exist (run 1, and run 2 with a -rerun suffix). Not vacuous today,
since emission order puts run 1 first, but it was the one place a run-2 record could satisfy a
run-1 claim.

Fixed (3f8dc01): the filter now also requires
l.attributes?.[Attr.DEV_RUN_ID] === harness.devRunId && l.attributes?.session_id === sessionId.
Both attributes were empirically confirmed present on the log before being relied on.

Finding 2 (nit) - hypaware-core/smoke/flows/backfill_openclaw_fixture.js:353

identity_source === 'native' was a tautological conjunct. The value is a hardcoded string
literal at the single emission site (plugins-workspace/openclaw/src/backfill.js:317), so it can
never be false when the log exists, yet the surrounding comment read as though the assertion
distinguished a native-identity path from a fallback one.

Fixed (3f8dc01): the conjunct is dropped and the comment rewritten to state what is
actually pinned. The identity claim moved onto the filter as session_id, where it genuinely
can be false. The row assertions already pin native message_ids, so no change was needed there.

Checks at 3f8dc01

npm run smoke -- backfill_openclaw_fixture ok; npm test 3281 pass / 0 fail (1 pre-existing
skip); npm run typecheck clean. Both mutations re-run after the fixes and still fail as above,
so the gate survived the edit. CI at the new head is the authority.

Caveat for the human, not a finding and not introduced by this PR

.github/workflows/ci.yml runs only npm i, npm run typecheck, and npm test. No smoke
flow runs in CI, including this one and its backfill_codex_fixture / backfill_claude_fixture
siblings. Issue #555 frames the missing smoke as the reason #543 shipped green, but a smoke CI
never executes would not have stopped it either. This PR still closes the stated coverage gap and
the flow is correct; whether the hermetic tier should gate CI is a repo-level tiering decision,
deliberately left to the maintainer.

Deliberately not raised

• The base branch: targeting fix/issue-543 rather than master is intentional, because the
  reader fix is unmerged and the smoke cannot pass on master by construction. OpenClaw backfill reads message fields one level too high, projecting 0 rows #552 must merge
  first; GitHub will retarget this PR when it does.
• The timestamp/id precedence asymmetry in session_file.js, explicitly deferred by Follow-up: deferred review findings from PR #552 #555.
• Issue Follow-up: deferred review findings from PR #552 #555's "Non-issues (do not action)" items.

This is the second and final review round under the 2-round cap. The head has moved to
3f8dc01; the next tick triages rather than reviewing again.

[philcunliffe]

The review fix-loop reached its 2-round cap, so this landed on the triage rung (LLP 0017).

I independently re-verified all four findings raised across both review rounds against the
committed tree at 3f8dc01 (the current head), reading the code rather than trusting the
review comments' prose:

• R1-1 (hypaware-core/smoke/lib/openclaw_session_fixture.js): openclawMessageLine now
  takes an optional messageTimestamp that overrides only the nested message.timestamp
  (lines 45-51). The flow (hypaware-core/smoke/flows/backfill_openclaw_fixture.js) stages a
  record with distinct line (10:00:02Z) and envelope (10:00:09Z) timestamps, asserts they
  differ in the fixture (line 149), and asserts the projected message_created_at carries the
  envelope value (line 263). Fixed.
• R1-2 (same flow file, idempotency assertion): the run-2 predicate now requires
  v.status === 'ok' && v.rows_written === 0 && v.items_seen >= 1 && v.rows_skipped >= 1
  (lines 292-297), no longer vacuous under a whole-file skip. Fixed.
• R2-1 (session_projected log filter): now pins both
  l.attributes?.[Attr.DEV_RUN_ID] === harness.devRunId and l.attributes?.session_id === sessionId
  (lines 349-356), rather than relying on emission order between the two dev_run_ids in play.
  Fixed.
• R2-2 (identity_source === 'native' tautology): the conjunct is gone; identity_source
  no longer appears in any assertion, only in the explanatory comment noting why it was dropped.
  The identity claim moved onto the filter's session_id check. Fixed.

I also re-ran npm run smoke -- backfill_openclaw_fixture in an isolated worktree at this head:
it passes.

Zero unresolved findings remain. Per the task's guidance, an issue enumerating nothing would be
noise, so no follow-up issue was opened. Out-of-scope items (the deliberate fix/issue-543 base
branch, the timestamp/id asymmetry #555 explicitly defers, and #555's "Non-issues" list) were
left untouched, as instructed by both review rounds and issue #555 itself.

Judgement: this PR can ship safely.

[philcunliffe]

⚠️ Merge-order conflict: #558 and #570 both create the same file

Both open PRs add hypaware-core/smoke/flows/backfill_openclaw_fixture.js, and neither is
on master yet. Whichever merges first, the second will conflict on that path.

PR	Why it adds the file	What its version does
#558	Fixes issue #555 (no hermetic smoke ever wrote a real-shape OpenClaw session file, which is why #543 shipped green)	Plain fixture smoke: real two-level envelope shape, non-zero rows, telemetry assertions, idempotency. Mutation-tested: reverting openclawMessageEnvelope to a flat read reproduces #543's signature and fails the smoke. Also adds hypaware-core/smoke/lib/openclaw_session_fixture.js.
#570	Task T12 of LLP 0173, the openclaw-two-lane-capture change set	Adds the Lane B dimensions the change set needs: quiesce-window behaviour (a file inside the window is skipped, outside it is captured) and a zero-duplicate assertion across both capture lanes.

They were written to different specs and neither is a superset of the other, so I have not
picked a winner or merged them. That is a design call about what the smoke tier should
assert, and it is yours.

How this happened

When #558 was written I asked its author to factor the fixture writer out and make mtimeMs
caller-controllable specifically so T12 could extend it rather than replace it. T12 was then
implemented from LLP 0173's own task spec, independently, and produced its own version. The
two efforts never saw each other because they ran on branch-disjoint workers, which is the
same isolation that lets them run in parallel.

Options

1. Merge OpenClaw two-lane capture (LLP 0172/0173): Lane A attach, Lane B sweep, json_path revival #570 first, then reconcile A hermetic smoke writes a real-shape OpenClaw session file (#555 item 1) #558 onto it — keeps the richer two-lane version, and
   A hermetic smoke writes a real-shape OpenClaw session file (#555 item 1) #558's distinctive value (the mutation-tested regression gate for OpenClaw backfill projects 0 rows from real session files: message fields are read at the top level but OpenClaw nests them under 'message' #543) gets folded in.
2. Merge A hermetic smoke writes a real-shape OpenClaw session file (#555 item 1) #558 first, then reconcile OpenClaw two-lane capture (LLP 0172/0173): Lane A attach, Lane B sweep, json_path revival #570 onto it — keeps the mutation-tested gate as the
   base, and T12's quiesce/dedupe cases get added on top.
3. Tell me which to keep and I will reconcile the other onto it and re-verify from CI.

Say which on either thread and I will carry it out on the next tick. Until then both PRs stay
held.

Related: #552 was squash-merged, so #558's branch still carries the original pre-squash
commits. GitHub still reports it MERGEABLE against master, so this is informational only,
but it is why #558's diff looks wider than its own changes.