From a9833ff6f27248e0edea08a0345bdca1ac62549b Mon Sep 17 00:00:00 2001 From: Siddhant Sinha Date: Mon, 20 Jul 2026 14:59:07 +0530 Subject: [PATCH 1/5] =?UTF-8?q?docs(merge):=200046=20=E2=80=94=20collision?= =?UTF-8?q?=20identity=20grouping=20(nest=20supersessions,=20split=20disti?= =?UTF-8?q?nct=20tests)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An optional tests.identity block in merge-rules.yaml turns a test-id collision from "pick one winner" into "group by identity". Same declared identity keeps the latest run in the canonical tests// slot and nests superseded runs beneath it as 1/, 2/ …; different identity splits into a suffixed sibling folder. Realises the cross-environment matrix identity slice 0045 deferred. Purely additive — absent the block, merge behaves exactly as 0045 specifies. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../0046-collision-identity-grouping.md | 193 ++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 design/decisions/0046-collision-identity-grouping.md diff --git a/design/decisions/0046-collision-identity-grouping.md b/design/decisions/0046-collision-identity-grouping.md new file mode 100644 index 0000000..417072f --- /dev/null +++ b/design/decisions/0046-collision-identity-grouping.md @@ -0,0 +1,193 @@ +--- +id: 46 +slug: collision-identity-grouping +title: Collision identity grouping — nest supersessions, split distinct tests +status: accepted +date: 2026-07-20 +proposition: > + 0045 resolves a test-id collision to exactly ONE surviving copy and lists + "cross-environment matrix identity (keeping both copies of a colliding test + disambiguated by environment)" as deferred. Two copies of `tests//` can + mean two different things: the SAME logical test run twice, or two GENUINELY + DIFFERENT tests that happen to share a derived id. How does a caller declare + which is which, and what shape does the merged pack take in each case? +options: + - id: identity-block + summary: > + An optional `tests.identity: {keys, on_same, on_different}` block names + the key paths that constitute test identity and the outcome for each + case. Collisions become GROUPS keyed by identity: same identity nests + superseded copies under the canonical folder, different identity splits + into a suffixed sibling folder. Absent the block, 0045's behaviour is + unchanged. + chosen: true + - id: split-action-in-rules + summary: > + No new block — add a `split` action to the existing result.yaml + `on_violation` enum and a `nest` action to `tests.on_collision`, so + identity is whatever set of rules happens to carry `split`. + chosen: false + - id: caller-side-pregrouping + summary: > + Leave merge alone. The caller inspects packs, computes identity groups + itself, pre-renames colliding test directories in staging copies, and + merges a set that no longer collides. + chosen: false +decision: > + `tests.identity` is an OPTIONAL block in merge-rules.yaml: + `{keys: [...], on_same: , on_different: }`. When + it is ABSENT, merge behaves EXACTLY as 0045 specifies — this decision is + purely additive and changes no default. `keys` is a non-empty array of + dot-paths into the parsed `result.yaml`, resolved by the same `getKey` + walker and compared by the same canonical `deepEqual` as 0045's generic + rules, inheriting its absent-key semantics VERBATIM: absent == absent + counts as SAME, absent-vs-present counts as DIFFERENT. Two copies have the + SAME IDENTITY when every listed key compares equal. `keys` are the + CALLER'S FACT — evidence-cli never interprets them and ships no default + set, so a producer's identity vocabulary (a TMS test uuid, a commit sha, a + browser name) stays in the producer's policy file and the tool stays + vendor-neutral. + RESOLUTION ORDER on a collision is fixed and does NOT change 0045's first + stage: the `result.yaml` rules run FIRST, in file order, pairwise against + the FIRST GROUP's representative, and the first violated rule applies its + `on_violation` exactly as today; only when no rule is violated does the + identity block run. A guard rule (`must: same` + `on_violation: error`) + therefore still ABORTS ahead of any grouping and can never be quietly + downgraded into a split folder — the property that lets a caller keep + tenancy guards and identity keys in one file without their precedence + being ambiguous. With no identity block and no rule violated, the default + `tests.on_collision` applies, unchanged. A GROUP'S REPRESENTATIVE is its + FIRST MEMBER IN CLI ORDER — the group's original claimant — so stage one + preserves 0045's incumbent semantics exactly, and stage two's comparison is + well-defined because every member of a group shares an identity by + construction. `discard` TOMBSTONES THE BASE ID, not a folder: every group of + that base id is dropped, INCLUDING split siblings already allocated, and the + id stays tombstoned against later packs — a split folder is not a way around + a tombstone. + The union walk generalises from ONE INCUMBENT per test id to an ORDERED + LIST OF GROUPS per test id. A group is `{baseId, folder, members[]}`: + `baseId` is the original `tests/` name, `folder` is the output + directory name, `members` are the eligible packs contributing a copy, in + CLI order. A challenger is compared against each existing group of its + baseId IN ALLOCATION ORDER; the FIRST group whose representative has the + same identity ABSORBS it (`on_same`); if no group matches, `on_different` + applies. This is what makes 3+-way collisions well-defined — pack C's copy + is tested against the `` group AND the `-1` group before a third + folder is minted — and it is why identity is a DECLARED SET of keys rather + than an emergent property of the rule list: the walk needs the set as an + up-front fact. The single-member group where `folder == baseId` is + precisely 0045's behaviour, so there is ONE code path, not two. + `on_same` takes `nest | prefer_latest | prefer_first | error`; + `on_different` takes `split | error`. `nest` KEEPS EVERY MEMBER: members + sort ASCENDING by the source pack's `run.yaml` `ended` (fallback + `started`, ties broken by CLI order), the LAST — the latest run — occupies + the canonical `tests//` exactly as an uncontested test does, and + each earlier member is written WHOLE to `tests///`, 1-based, + `1/` being the OLDEST. WHOLE-TREE ATOMICITY (0045) is preserved per + member: a member's `result.yaml`, definition, `logs/`, `steps/` and video + travel together and nothing is mixed between copies. `split` allocates a + NEW SIBLING folder: the first group of a baseId keeps the UNSUFFIXED name, + later groups take `-1`, `-2`, and so on. Suffixes are + allocated against a RESERVATION SET — the union of every eligible pack's + test ids, computed once before the walk (the eligible set is already fixed + by then) — and a candidate name already in that set or already allocated + is SKIPPED, so a split never steals a name some pack legitimately owns. + A split folder's `result.yaml` `test` field is REWRITTEN to equal its new + directory name, because 0031's validator cross-check requires that + equality; the rewrite goes through the comment-preserving + `parseDoc`/`setIn` path 0045 already uses for the environment push-down, so + the definition file is never touched and hash checks stay green. A NESTED + member's `result.yaml` `test` field is NOT rewritten: nothing validates a + nested copy, and preserving the original id keeps the archive truthful + about what it was. 0043's environment push-down runs on EVERY copy + written, canonical and nested alike, each against ITS OWN source pack's + environment — the divergent keys are what let an archived copy be read + standalone, which is the entire reason for keeping it. + NESTED COPIES ARE INERT to every derived artifact, by construction rather + than by special-casing: finalize's totals walk only top-level `tests/*` + directories, its failure index reads only `tests//steps/`, and the L1 + step checks enumerate only `/steps/`. A numbered subdirectory is + therefore invisible to totals, to the root failure index, to + `listTestIds`, and to validation — it rides along as an archive and + changes no count. `MergeReport.tests.merged` counts TOP-LEVEL test + folders, which is now the number of GROUPS. `MergeReport.tests.collisions` + gains two OPTIONAL fields, `action` (`nest` | `split`) and `folder` (the + output directory), leaving `{test, winner, rule}` intact so existing + `--json` consumers keep parsing; `winner` carries the canonical member's + run_id for `nest` and the absorbed challenger's for `split`. Merge remains + DETERMINISTIC — every ordering derives from pack metadata and CLI order, + no clock and no randomness are read. +--- + +## Reasoning + +**Why an identity block rather than a `split` action in the rule list.** The +rule list is a sequence of INDEPENDENT pairwise predicates: each one fires on +its own and the first violation decides. That shape is exactly right for +"which copy wins", where the answer is a choice between two things already in +hand. It is wrong for grouping, because grouping needs to ask a question the +rule list cannot express — *does this challenger belong to group 0, group 1, +or neither?* — and answering it requires knowing the WHOLE SET of keys that +constitute identity before any comparison happens. Scraping that set back out +of a flat list as "every rule whose action is `split`" would make identity an +emergent property of an unrelated ordering: adding a tenancy guard would +silently redefine what "the same test" means. Declaring the set once, in a +block that exists to declare it, keeps the fact where a reader looks for it. + +**Why the new actions are not enum values on `on_collision`.** `error`, +`prefer_first`, `prefer_latest` and `discard` all answer one question — which +of two copies survives — and none of them changes the SHAPE of the merged +pack; the output is always `tests//`, one directory per colliding id. +`nest` and `split` answer a different question: what topology does the output +take. Keeping them in the identity block, where the condition that selects +them also lives, states that difference in the format instead of hiding it +behind a wider enum whose members no longer share a meaning. + +**Why the caller cannot do this outside merge.** Merge DISCARDS the losing +tree, so by the time a caller could post-process, the superseded artifacts are +gone. The only caller-side alternative is to pre-group: read every pack, apply +identity comparison, rename directories in staging copies, then merge a +non-colliding set. That reimplements pack reading, dot-path resolution and +canonical equality in a second place, and it puts the rename — including the +0031 `test`-field rewrite — outside the tool that owns the contract those +rules come from. + +**Why the latest run keeps the canonical slot.** The alternative — nesting +every member uniformly under `tests//1..n/` — is more symmetric, but it +would move `result.yaml` out of `tests//result.yaml` and break the L0 +layout for every consumer, requiring contract changes to validate, finalize +and every reader. Keeping the latest run exactly where an uncontested test +sits means a consumer that does not care about history reads a nested pack +identically to a flat one, and the archive is strictly additive. + +## Consequences + +- `src/merge/collide.ts` is reshaped: `claims` becomes + `Map`, the exported `UnionEntry` is replaced by + `TestGroup {baseId, folder, members[]}`, and the reservation set is computed + from the eligible packs before the walk. +- `src/merge/assemble.ts` takes `groups: TestGroup[]` in place of + `union: UnionEntry[]`, sorts each group's members chronologically, writes the + canonical and nested copies, and rewrites `test` on split folders only. +- `src/merge/rules.ts` gains the `IdentityPolicy` type; `DEFAULT_RULES` is + UNCHANGED (`tests.identity` stays undefined), so the strict defaults of 0045 + still hold when `--rules` is omitted. +- `src/schemas/merge-rules.schema.json` gains `tests.identity` with + `additionalProperties: false`, `keys` as a `minItems: 1` array of non-empty + strings, and the two action enums. +- `contract.ts`'s `MergeReport.tests.collisions` gains optional `action` and + `folder`; `src/report/reporter.ts` prints the nest/split outcome. +- `design/contract/03-commands.md` gains the identity block in the `merge` + section — the rules-file shape, the resolution order, and the two output + layouts. +- A `nest` merge produces a LARGER pack than the same merge under + `prefer_latest`: artifacts that are discarded today are retained. This is + the intent of the feature, not a regression, but it is a real change to + output size for callers who opt in. +- Deferred, additively: identity keys addressing `run.yaml` (today's `keys` + are result-scoped, matching where per-test identity lives); a `fold_attempts` + action folding nested members into 0033's `attempts[]` (still deferred from + 0045, and now expressible as a third `on_same` action); an + auto-derived identity from the environment block, which 0045 sketched as + "cross-environment matrix identity" — the explicit `keys` list subsumes it + for now, since a caller can name the environment paths directly. From 3488b1a8a2b4d5cb70caf0f951dce4c941922221 Mon Sep 17 00:00:00 2001 From: Siddhant Sinha Date: Mon, 20 Jul 2026 15:17:36 +0530 Subject: [PATCH 2/5] =?UTF-8?q?feat(merge):=20identity=20grouping=20?= =?UTF-8?q?=E2=80=94=20nest=20supersessions,=20split=20distinct=20tests=20?= =?UTF-8?q?(0046)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An optional tests.identity block turns a test-id collision from "pick one winner" into "group by identity". Copies whose declared keys all match keep the latest run in the canonical tests// slot and archive superseded runs beneath it as 1/, 2/ … oldest first; copies that differ split into a suffixed sibling tests/-1/. - collide: the union walk generalises from one incumbent per test id to an ordered list of groups, so a 3+-way collision is matched against every existing group before a new folder is minted. Single-member groups with folder === baseId are exactly 0045's behaviour — one code path, not two. - assemble: writes canonical + nested copies, rewrites `test` on split folders only (0031's directory equality), and pushes divergent environment down into every copy from its own source pack. - Guard rules still run first, so a must:same + error rule can never be downgraded into a split; discard tombstones the base id, siblings included. - Suffixes are reserved against every eligible pack's test ids, so a split never steals a name a pack legitimately owns. Nested copies are inert to totals, the failure index and validation, all of which scope themselves one level. Purely additive: absent the block, output is unchanged. 192 tests passing, build clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- design/contract/03-commands.md | 49 +++++++++- src/contract.ts | 5 +- src/merge/assemble.test.ts | 108 +++++++++++++++++++--- src/merge/assemble.ts | 73 +++++++++++---- src/merge/collide.test.ts | 120 +++++++++++++++++++++++-- src/merge/collide.ts | 135 +++++++++++++++++++++++----- src/merge/index.test.ts | 40 +++++++++ src/merge/index.ts | 6 +- src/merge/rules.test.ts | 33 +++++++ src/merge/rules.ts | 16 +++- src/merge/testkit.ts | 2 + src/report/reporter.test.ts | 22 ++++- src/report/reporter.ts | 5 +- src/schemas/merge-rules.schema.json | 16 ++++ 14 files changed, 562 insertions(+), 68 deletions(-) diff --git a/design/contract/03-commands.md b/design/contract/03-commands.md index 3b04a24..52b54b1 100644 --- a/design/contract/03-commands.md +++ b/design/contract/03-commands.md @@ -186,6 +186,12 @@ packs: on_ineligible: abort # abort | skip tests: on_collision: error # error | prefer_first | prefer_latest | discard + identity: # OPTIONAL (0046); absent → 0045 behaviour, unchanged + keys: # dot-paths into result.yaml — the caller's vocabulary + - external_id.commit_id + - external_id.test_id + on_same: nest # nest | prefer_latest | prefer_first | error + on_different: split # split | error rules: - file: run.yaml # scope: compared ACROSS all eligible packs key: environment.producer.name @@ -275,6 +281,36 @@ copies. Folding collisions into `attempts[]` (retry semantics) is deliberately out of scope — see decision [0033 — attempts is a per-attempt outcome list](#/decisions). +### Identity grouping (optional, 0046) + +When `tests.identity` is configured, a collision that violates **no** +`result.yaml` rule is resolved by **grouping** instead of by picking a winner — +guard rules still abort first, so a `must: same` + `error` rule can never be +downgraded into a split. Two copies have the same identity when **every** key +in `keys` compares equal under the canonical deep equality above (absent == +absent counts as same). The challenger is matched against each existing group +of that id **in allocation order**; the first match absorbs it (`on_same`), and +no match allocates a new folder (`on_different`). + +| Action | Meaning | +| --- | --- | +| `on_same: nest` | keep **every** copy: the latest run takes the canonical `tests//`, each superseded run is archived whole beneath it as `1/`, `2/` … oldest first (by `ended`, fallback `started`; ties → CLI order) | +| `on_different: split` | the copy lands in a **new sibling** `tests/-1/`, `-2/` … the first group keeps the unsuffixed name | + +Suffixes are allocated against the union of every eligible pack's test ids, so +a split never steals a name some pack legitimately owns — if `-1` exists as +a real test, the split takes `-2`. A split folder's `result.yaml` `test` +field is **rewritten** to equal its directory (the L0 cross-check requires the +equality); a nested copy keeps its original id, since nothing validates it and +the archive stays truthful. `discard` tombstones the **base id** — every group +of it goes, split siblings included. + +Nested copies are **inert**: totals walk only top-level `tests/*`, the failure +index reads only `tests//steps/`, and the L1 step checks enumerate only +`/steps/`. An archive therefore changes no count and fails no check. Note +that `nest` retains artifacts `prefer_latest` would discard, so a merged pack +grows with the number of runs kept. + ### `run.yaml` — per-key disposition | Key | Across the N inputs | Value in merged `run.yaml` | @@ -313,12 +349,21 @@ packs: eligible: [shard-a, shard-b] # run_ids, CLI order skipped: [{ run_id: shard-a2, rule: "run.yaml run_id must different", reason: duplicate of shard-a }] tests: - merged: 214 - collisions: [{ test: checkout, winner: shard-b, rule: tests.on_collision=prefer_latest }] + merged: 214 # TOP-LEVEL test folders (= groups) + collisions: + - { test: checkout, winner: shard-b, rule: tests.on_collision=prefer_latest } + # 0046 outcomes carry the shape they took and where the copy landed: + - { test: login, winner: shard-c, rule: tests.identity.on_same=nest, action: nest, folder: login } + - { test: login, winner: shard-d, rule: tests.identity.on_different=split, action: split, folder: login-1 } discarded: [flaky-login] output: { path: merged.evidence, run_id: nightly-2026-07-08, finalized: true } ``` +`action` and `folder` appear only on 0046's shape-changing outcomes; the other +three fields are unconditional, so existing `--json` consumers keep parsing +unchanged. `winner` is the canonical member for `nest` and the absorbed +challenger for `split`. + Policy-sanctioned skips/discards exit `0` — the report carries the story. The pack itself stays clean: `merged_from` is the only in-pack trace of the merge. diff --git a/src/contract.ts b/src/contract.ts index 6c520ce..975f283 100644 --- a/src/contract.ts +++ b/src/contract.ts @@ -44,7 +44,10 @@ export interface MergeReport { }; tests: { merged: number; - collisions: { test: string; winner: string; rule: string }[]; + // `action`/`folder` are present only for 0046's shape-changing outcomes; + // the three original fields are unconditional, so existing --json + // consumers keep parsing unchanged. + collisions: { test: string; winner: string; rule: string; action?: "nest" | "split"; folder?: string }[]; discarded: string[]; }; output: { path: string; runId: string; finalized: boolean }; diff --git a/src/merge/assemble.test.ts b/src/merge/assemble.test.ts index 1f62c7c..2b123d1 100644 --- a/src/merge/assemble.test.ts +++ b/src/merge/assemble.test.ts @@ -10,7 +10,7 @@ import { stagePack, sealCopy } from "./testkit"; import { parseYaml } from "../yaml"; import { validate } from "../validate"; -async function stagePair(): Promise<{ out: string; eligible: any[]; union: any[] }> { +async function stagePair(): Promise<{ out: string; eligible: any[]; groups: any[] }> { const a = await sealCopy( await stagePack({ runId: "a", @@ -33,15 +33,101 @@ async function stagePair(): Promise<{ out: string; eligible: any[]; union: any[] }), ); const { eligible } = await gatePacks([a, b], DEFAULT_RULES); - const { union } = await resolveCollisions(eligible, DEFAULT_RULES); + const { groups } = await resolveCollisions(eligible, DEFAULT_RULES); const out = path.join(await fs.mkdtemp(path.join(os.tmpdir(), "evi-asm-")), "m.evidence"); - return { out, eligible, union }; + return { out, eligible, groups }; } +const IDENTITY_RULES: any = { + ...DEFAULT_RULES, + tests: { + on_collision: "error", + identity: { keys: ["external_id.commit_id"], on_same: "nest", on_different: "split" }, + }, +}; + +/** Stage N packs that all claim `login`, resolve them under the identity policy. */ +async function stageGrouped(specs: any[]): Promise<{ out: string; eligible: any[]; groups: any[] }> { + const paths: string[] = []; + for (const s of specs) paths.push(await sealCopy(await stagePack({ l1: true, ...s }))); + const { eligible } = await gatePacks(paths, IDENTITY_RULES); + const { groups } = await resolveCollisions(eligible, IDENTITY_RULES); + const out = path.join(await fs.mkdtemp(path.join(os.tmpdir(), "evi-asm-")), "m.evidence"); + return { out, eligible, groups }; +} + +const readResult = async (out: string, rel: string): Promise => + parseYaml(await fs.readFile(path.join(out, "tests", rel, "result.yaml"), "utf8")); + +describe("assemble — identity grouping (0046)", () => { + it("keeps the latest member canonical and nests superseded ones oldest-first", async () => { + // CLI order deliberately differs from chronological order + const { out, eligible, groups } = await stageGrouped([ + { runId: "b", ended: "2026-07-08T10:00:00Z", tests: { login: { externalId: { commit_id: "abc", session_id: "s-b" } } } }, + { runId: "a", ended: "2026-07-08T09:00:00Z", tests: { login: { externalId: { commit_id: "abc", session_id: "s-a" } } } }, + { runId: "c", ended: "2026-07-08T11:00:00Z", tests: { login: { externalId: { commit_id: "abc", session_id: "s-c" } } } }, + ]); + await assemble(out, { runId: "nightly" }, eligible, groups); + + expect((await readResult(out, "login")).external_id.session_id).toBe("s-c"); // latest + expect((await readResult(out, "login/1")).external_id.session_id).toBe("s-a"); // oldest + expect((await readResult(out, "login/2")).external_id.session_id).toBe("s-b"); + // nested copies carry their whole tree + await fs.access(path.join(out, "tests/login/1/test.md")); + await fs.access(path.join(out, "tests/login/1/logs/console.ndjson")); + await fs.access(path.join(out, "tests/login/1/steps/2-pay/screenshot.png")); + }); + + it("rewrites test: in a split folder, leaves it untouched in a nested copy", async () => { + const { out, eligible, groups } = await stageGrouped([ + { runId: "a", ended: "2026-07-08T09:00:00Z", tests: { login: { externalId: { commit_id: "abc" } } } }, + { runId: "b", ended: "2026-07-08T10:00:00Z", tests: { login: { externalId: { commit_id: "abc" } } } }, + { runId: "c", ended: "2026-07-08T11:00:00Z", tests: { login: { externalId: { commit_id: "zzz" } } } }, + ]); + await assemble(out, { runId: "nightly" }, eligible, groups); + + expect((await readResult(out, "login")).test).toBe("login"); + expect((await readResult(out, "login/1")).test).toBe("login"); // archive stays truthful + expect((await readResult(out, "login-1")).test).toBe("login-1"); // must equal its directory + }); + + it("pushes divergent environment down into nested copies from their own source", async () => { + const { out, eligible, groups } = await stageGrouped([ + { + runId: "a", + ended: "2026-07-08T09:00:00Z", + environment: { producer: { name: "kane" }, ci: { shard: "1" } }, + tests: { login: { externalId: { commit_id: "abc" } } }, + }, + { + runId: "b", + ended: "2026-07-08T10:00:00Z", + environment: { producer: { name: "kane" }, ci: { shard: "2" } }, + tests: { login: { externalId: { commit_id: "abc" } } }, + }, + ]); + await assemble(out, { runId: "nightly" }, eligible, groups); + + expect((await readResult(out, "login")).environment.ci).toEqual({ shard: "2" }); // canonical = pack b + expect((await readResult(out, "login/1")).environment.ci).toEqual({ shard: "1" }); // nested = pack a + }); + + it("a pack with nested and split folders validates clean at L0", async () => { + const { out, eligible, groups } = await stageGrouped([ + { runId: "a", ended: "2026-07-08T09:00:00Z", tests: { login: { externalId: { commit_id: "abc" } } } }, + { runId: "b", ended: "2026-07-08T10:00:00Z", tests: { login: { externalId: { commit_id: "abc" } } } }, + { runId: "c", ended: "2026-07-08T11:00:00Z", tests: { login: { externalId: { commit_id: "zzz" } } } }, + ]); + await assemble(out, { runId: "nightly" }, eligible, groups); + const report = await validate(out, { profile: "L0" }); + expect(report.valid).toBe(true); + }); +}); + describe("assemble", () => { it("synthesizes run.yaml, pushes divergent env down, namespaces coverage/metrics, copies whole trees", async () => { - const { out, eligible, union } = await stagePair(); - await assemble(out, { runId: "nightly" }, eligible, union); + const { out, eligible, groups } = await stagePair(); + await assemble(out, { runId: "nightly" }, eligible, groups); const run = parseYaml(await fs.readFile(path.join(out, "run.yaml"), "utf8")) as any; expect(run).toMatchObject({ evidence: "0.1", run_id: "nightly", status: "running", title: "t-a" }); @@ -70,22 +156,22 @@ describe("assemble", () => { }); it("the assembled pack validates clean at L0 while running", async () => { - const { out, eligible, union } = await stagePair(); - await assemble(out, { runId: "nightly" }, eligible, union); + const { out, eligible, groups } = await stagePair(); + await assemble(out, { runId: "nightly" }, eligible, groups); const report = await validate(out, { profile: "L0" }); expect(report.valid).toBe(true); expect(report.status).toBe("running"); }); it("refuses to overwrite an existing output path (USAGE)", async () => { - const { out, eligible, union } = await stagePair(); + const { out, eligible, groups } = await stagePair(); await fs.mkdir(out, { recursive: true }); - await expect(assemble(out, { runId: "nightly" }, eligible, union)).rejects.toMatchObject({ code: "USAGE" }); + await expect(assemble(out, { runId: "nightly" }, eligible, groups)).rejects.toMatchObject({ code: "USAGE" }); }); it("--title overrides the first pack's title", async () => { - const { out, eligible, union } = await stagePair(); - await assemble(out, { runId: "nightly", title: "Nightly regression" }, eligible, union); + const { out, eligible, groups } = await stagePair(); + await assemble(out, { runId: "nightly", title: "Nightly regression" }, eligible, groups); const run = parseYaml(await fs.readFile(path.join(out, "run.yaml"), "utf8")) as any; expect(run.title).toBe("Nightly regression"); }); diff --git a/src/merge/assemble.ts b/src/merge/assemble.ts index 9c5ce1f..6fd06ec 100644 --- a/src/merge/assemble.ts +++ b/src/merge/assemble.ts @@ -4,7 +4,8 @@ import { CONTRACT_VERSION } from "../contract"; import type { PackContainer } from "../pack/container"; import { parseDoc, parseYaml, stringifyDoc, stringifyYaml } from "../yaml"; import type { EligiblePack } from "./gates"; -import type { UnionEntry } from "./collide"; +import { orderMembers } from "./collide"; +import type { TestGroup } from "./collide"; import { deepEqual, getKey } from "./rules"; export interface AssembleOptions { @@ -22,7 +23,7 @@ export async function assemble( outDir: string, opts: AssembleOptions, eligible: EligiblePack[], - union: UnionEntry[], + groups: TestGroup[], ): Promise { try { await fs.access(outDir); @@ -34,9 +35,33 @@ export async function assemble( } await fs.mkdir(outDir, { recursive: true }); - // Winners' whole trees + per-source coverage nesting. - for (const entry of union) { - await copyTree(entry.source.container, `tests/${entry.testId}`, path.join(outDir, "tests", entry.testId)); + // Each group's members, whole-tree: the latest run takes the canonical + // tests// exactly as an uncontested test does, and every superseded + // member is archived beneath it as 1/, 2/ … oldest first (decision 0046). + // `written` is the per-copy record the environment push-down works from. + const written: { pack: EligiblePack; resultPath: string }[] = []; + for (const group of groups) { + const ordered = orderMembers(group.members); + const canonical = ordered[ordered.length - 1]; + const groupDir = path.join(outDir, "tests", group.folder); + await copyTree(canonical.container, `tests/${group.baseId}`, groupDir); + written.push({ pack: canonical, resultPath: path.join(groupDir, "result.yaml") }); + + for (let i = 0; i < ordered.length - 1; i++) { + const nestedDir = path.join(groupDir, String(i + 1)); + await copyTree(ordered[i].container, `tests/${group.baseId}`, nestedDir); + written.push({ pack: ordered[i], resultPath: path.join(nestedDir, "result.yaml") }); + } + + // A split folder must satisfy 0031's test-id/directory equality. A NESTED + // copy is not validated and keeps its original id — the archive stays + // truthful about what it was. + if (group.folder !== group.baseId) { + await editResult(path.join(groupDir, "result.yaml"), (doc) => { + doc.set("test", group.folder); + return true; + }); + } } for (const pack of eligible) { if (await pack.container.isDir("coverage")) { @@ -54,21 +79,22 @@ export async function assemble( if (envs.every((e) => key in e && deepEqual(e[key], first))) commonEnv[key] = first; else divergent.add(key); } - for (const entry of union) { - const sourceEnv = entry.source.run?.environment ?? {}; + // Applied to EVERY copy written, canonical and nested alike, each against + // its own source pack — the divergent keys are what let an archived copy be + // read standalone, which is the reason for keeping it at all. + for (const { pack, resultPath } of written) { + const sourceEnv = pack.run?.environment ?? {}; const pushable = [...divergent].filter((k) => k in sourceEnv); if (pushable.length === 0) continue; - const resultPath = path.join(outDir, "tests", entry.testId, "result.yaml"); - const raw = await fs.readFile(resultPath, "utf8"); - const parsed = parseYaml(raw); - const doc = parseDoc(raw); - let dirty = false; - for (const key of pushable) { - if (getKey(parsed, `environment.${key}`) !== undefined) continue; // per-test value wins - doc.setIn(["environment", key], sourceEnv[key]); - dirty = true; - } - if (dirty) await fs.writeFile(resultPath, stringifyDoc(doc), "utf8"); + await editResult(resultPath, (doc, parsed) => { + let dirty = false; + for (const key of pushable) { + if (getKey(parsed, `environment.${key}`) !== undefined) continue; // per-test value wins + doc.setIn(["environment", key], sourceEnv[key]); + dirty = true; + } + return dirty; + }); } // Metrics: namespaced by flattening into the name —