diff --git a/CHANGELOG.md b/CHANGELOG.md index c47ab45..59d4e4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Batched staging-workspace page retention** - + `workspace.pages.putBatch()` writes an explicitly count-and-byte-bounded + ordered page group and anchors every returned page in one workspace + generation, avoiding a growing root-set rewrite for each page. + ## [6.5.3] — 2026-07-19 ### Performance diff --git a/README.md b/README.md index 757d574..f9cd228 100644 --- a/README.md +++ b/README.md @@ -149,9 +149,10 @@ Core capabilities: complete-validation member reads, reachability roots, compare-and-swap refs, and immutable lifecycle evidence. - **Scoped staging workspaces**: `workspaces.open()` mirrors application writes - behind one renewable temporary RootSet, returns only after each handle is - anchored, promotes destination-first, and exposes bounded age, expiry, - logical-content, and direct-root diagnostics with opaque cleanup pagination. + behind one renewable temporary RootSet, supports one-generation bounded page + batches, returns only after each handle is anchored, promotes + destination-first, and exposes bounded age, expiry, logical-content, and + direct-root diagnostics with opaque cleanup pagination. - **Envelope recipients**: multi-recipient key wrapping and recipient rotation avoid re-encrypting data blobs. - **Operational diagnostics**: `cas.diagnostics.doctor()` streams repository diff --git a/docs/API.md b/docs/API.md index 1cfc9a5..abaf823 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1342,8 +1342,12 @@ const workspace = await cas.workspaces.open({ }); try { - const first = await workspace.pages.put({ source: firstShard }); - const second = await workspace.pages.put({ source: secondShard }); + const [first, second] = await workspace.pages.putBatch({ + pages: [ + { source: firstShard }, + { source: secondShard }, + ], + }); const bundle = await workspace.bundles.putOrdered({ members: [ ['shards/first.cbor', first.handle], @@ -1374,6 +1378,7 @@ The workspace mirrors only application-storage writes: await workspace.assets.put(options); await workspace.assets.adopt(options); await workspace.pages.put(options); +await workspace.pages.putBatch(options); await workspace.bundles.put(options); await workspace.bundles.putOrdered(options); ``` @@ -1383,6 +1388,12 @@ returned typed handle. The result is otherwise the ordinary staged result plus a workspace `RetentionWitness`. Calls on one workspace serialize their ref mutations so concurrent staging cannot silently lose an accumulated root. +`workspace.pages.putBatch()` accepts the same `pages`, `maxBatchBytes`, and +`maxBatchPages` bounds as `cas.pages.putBatch()`. It returns one retained result +per input in input order, while all results from the batch name the same exact +workspace generation. Duplicate page content preserves ordered results but is +retained only once in that generation. + This guarantee starts when the method returns. Like all Git object composition, the object-write-to-ref-update interval still relies on Git's ordinary unreachable-object grace period. Running immediate-expiry prune concurrently diff --git a/docs/design/0054-batched-page-retention/batched-page-retention.md b/docs/design/0054-batched-page-retention/batched-page-retention.md new file mode 100644 index 0000000..49708da --- /dev/null +++ b/docs/design/0054-batched-page-retention/batched-page-retention.md @@ -0,0 +1,300 @@ +--- +title: 'PERF-0054 - Batched Page Retention' +cycle: '0054' +task_id: 'batched-page-retention' +legend: 'PERF' +release_home: 'v6.5.4' +issue: 'https://github.com/git-stunts/git-cas/issues/99' +goalpost_issue: 'https://github.com/git-stunts/git-cas/issues/99' +tracker_source: 'github' +status: 'active' +base_commit: 'd446d97e0436afccdfc0398091ba18888375d8b0' +owners: + - '@git-stunts' +sponsors: + human: 'James' + agent: 'Codex' +blocking_issues: [] +supersedes: [] +superseded_by: null +created: '2026-07-26' +updated: '2026-07-26' +--- + +# PERF-0054 - Batched Page Retention + +## Linked Issue + +- https://github.com/git-stunts/git-cas/issues/99 + +## Linked Tracker + +- Milestone: `v6.5.4` +- Goalpost issue: https://github.com/git-stunts/git-cas/issues/99 +- Slice issue: https://github.com/git-stunts/git-cas/issues/99 + +## Design Type + +- [x] Runtime/API +- [x] Storage/substrate +- [x] Migration/release +- [ ] CLI/operator +- [x] Docs/public guidance +- [ ] TUI/visual surface +- [x] Test/tooling + +## Decision Summary + +Staging workspaces will expose `pages.putBatch()`. It will delegate bounded +page creation to the existing page-batch service, resolve every returned page, +and install the complete batch in one compare-and-swap workspace generation. +Results remain ordered and individually witnessed. + +## Sponsored Human + +A storage operator wants large bounded page sets retained without a growing +root-set rewrite for every page, so one-time migrations finish in practical +time without weakening reachability. + +## Sponsored Agent + +An agent needs an explicit bounded batch contract and per-page witnesses so it +can prove retention and cost without inferring safety from loose Git objects. + +## Hill + +By the end of this cycle, git-warp can stage thousands of bounded index pages +through page batches, and git-cas proves that each batch creates one workspace +generation while returning exact retained witnesses. + +## Current Truth + +`PageService.putBatch()` already validates and writes a bounded ordered page +batch. `StagingWorkspace.pages` exposes only `put()`, and each call invokes +`#install()` with the growing set of workspace targets. A git-warp v19 +rehearsal measured 8,188 required pages; after 75 minutes, 5,213 pages had +produced about 1.16 GiB of loose scratch objects because every page rewrote the +flat root set. Issue #99 owns the release blocker. + +## Problem + +The workspace API forces page-oriented callers into quadratic root-set write +amplification even though git-cas already has a bounded bulk-write primitive. + +## Scope + +This cycle includes: + +- `workspace.pages.putBatch()` with the `PageService.putBatch()` options +- one serialized workspace installation for all unique batch handles +- ordered retained results and exact per-page witnesses +- limit, failure, duplicate, and generation-count tests +- public API and release documentation + +## Non-Goals + +This cycle does not include: + +- changing page, handle, root-set, or workspace serialization +- retaining an unbounded batch +- changing single-page `put()` behavior +- changing bundle layout or garbage-collection policy +- migrating existing refs or objects + +## Runtime / API Contract + +`workspace.pages.putBatch(options)` accepts the same bounded `pages`, +`maxBatchBytes`, and `maxBatchPages` fields as `ContentAddressableStore.pages`. +It returns a frozen array in input order. Every result is a retained staged +page whose witness names the same workspace generation. Duplicate page content +may share one target entry while preserving one ordered result per input. + +## User Experience / Product Shape + +There is no rendered interface. The public JavaScript API and its inspectable +retention witnesses are the user-visible surface. + +## Data / State Model + +| State | Source of truth | Derived state | Invalid states | Reset behavior | Serialization | Determinism assumptions | +| ----------------- | --------------------- | ------------- | ------------------------------- | -------------------------------- | ------------------------- | -------------------------- | +| staged pages | Git blobs | page handles | unresolved or wrong-type handle | failed objects stay unanchored | unchanged blob format | content-addressed | +| workspace targets | current workspace ref | witnesses | missing batch target | release deletes exact generation | unchanged root-set format | targets sorted canonically | + +## Architecture / Anti-SLUDGE Posture + +| Concern | Decision | +| ------------------------------- | ----------------------------------------------------------- | +| Domain changes | Compose existing page-batch and workspace-install contracts | +| Port changes | None | +| Adapter changes | None | +| Boundary validation | Reuse `PageService.putBatch()` limits and handle resolution | +| Runtime-backed nouns introduced | None | +| Expected failure representation | Existing `CasError` codes and retention wrapping | +| Banned shortcuts avoided | No unretained success result and no legacy path | + +## Cost / Residency Posture + +| Surface | Current cost | Target cost | Limit/budget | Failure mode | +| ---------------- | ------------------------------ | ------------------------ | ------------------------ | ------------------------ | +| stage N pages | N growing root-set generations | one generation per batch | explicit pages and bytes | batch or retention error | +| result witnesses | N exact witnesses | N exact witnesses | workspace target maximum | workspace state error | + +## Git Substrate Impact + +| Substrate area | Impact | +| ----------------------- | -------------------------------------------- | +| refs | one CAS workspace-ref update per batch | +| commits | one parentless root-set generation per batch | +| trees/blobs | unchanged page blobs; one current root tree | +| object ids | unchanged content addressing | +| tag/release behavior | patch release `v6.5.4` | +| migration compatibility | no stored-format migration | + +## Compatibility / Migration Posture + +| Concern | Decision | +| -------------------------- | ---------------------- | +| Public API compatibility | additive | +| Package export changes | none | +| Storage/read compatibility | unchanged | +| Legacy behavior retained | `pages.put()` remains | +| Deprecation behavior | none | +| Migration path | none | +| Release note impact | document batch staging | + +## Error Contract + +| Failure | Error/result | Caller recovery | Test | +| -------------------------- | --------------------------------- | -------------------------- | ----------------- | +| page/byte limit | existing page-batch limit error | reduce batch | focused unit test | +| page write failure | original page-service error | retry a new batch | failure test | +| resolution/install failure | workspace retention failure | release or retry workspace | failure test | +| concurrent mutation | serialized operation or CAS error | retry | concurrency test | + +## Security / Trust / Redaction Posture + +- trust boundary: application handles become workspace roots only after type resolution +- authority or capability checked: existing workspace and root-set capabilities +- secret-bearing values: none added +- redaction behavior: unchanged +- log/report behavior: no payload logging +- abuse or replay concern: explicit byte/page limits prevent unbounded batches + +## Lower Modes + +The lower mode is the ordered array of handles and machine-readable retention +witnesses; no visual context is required. + +## Accessibility Posture + +Documentation and witness evidence use a linear text reading order. No color, +layout, motion, or pointer interaction carries meaning. + +## Agent Inspectability / Explainability Posture + +An agent can compare result order, handle identities, witness generations, ref +update count, and reachable tree entries directly through the API and tests. + +## Linked Invariants + +- staged success always includes an anchored retention witness +- workspace mutations are serialized and compare-and-swap +- root sets remain parentless current-generation authority +- page batches remain explicitly bounded + +## Design Alternatives Considered + +### Option A: Keep staging one page at a time + +Rejected because it preserves the measured quadratic root-set amplification. + +### Option B: Write unretained pages and retain only a final bundle + +Rejected because it weakens the explicit workspace retention contract between +page creation and bundle publication. + +### Option C: Batch page creation and install the batch once + +Selected because it composes existing bounded primitives, preserves individual +witnesses, and changes no stored format. + +## Decision + +Add `pages.putBatch()` to `StagingWorkspace`, backed by one page-service batch +and one workspace installation. + +## Proof Surface + +- actual surface under test: `StagingWorkspace.pages.putBatch()` +- first RED test: three pages share one witness generation and one ref update +- required witness command: focused unit and Git-backed integration tests +- non-acceptable proof: timing claims without generation/object evidence + +## Implementation Slices + +- add RED unit coverage for ordering, witness identity, and one generation +- implement batch staging and dependency validation +- add Git-backed integration and limit/failure coverage +- update public docs, changelog, and witness + +## Tests To Write First + +- [x] one batch returns ordered retained pages under one generation +- [x] duplicate content preserves ordered results without duplicate targets +- [x] page-batch limits fail before workspace mutation +- [x] a batch retention failure returns no retained result + +## Acceptance Criteria + +The work is done when: + +- [x] behavior tests prove one installation per batch +- [x] every result has an exact anchored witness +- [x] limits and failure containment remain explicit +- [x] public docs and changelog describe the additive API +- [x] issue and PR are linked correctly +- [x] local validation is green; CI is pending the pull request + +## Validation Plan + +```bash +npx vitest run test/unit/domain/services/StagingWorkspace.test.js +npx vitest run test/integration/staging-workspace.test.js +npx eslint . +npm test +npm run release:verify -- --skip-jsr +``` + +## Playback / Witness + +The witness records the focused test output, the root-set update count, and the +git-warp migration benchmark before and after bounded batches. + +## Risks + +- a duplicate handle could lose one ordered result; preserve results separately + from the deduplicated target map +- a partial page-service failure could leave unreachable blobs; never return + retained success and rely on normal object cleanup +- a large caller-selected target set could hit the workspace maximum; retain + the existing explicit failure + +## Follow-On Debt + +None currently. Any new deferred work must be opened as a GitHub issue. + +## Tracker Disposition + +| Issue | Role | Expected disposition | +| ----------------------------------------------- | ------- | -------------------- | +| https://github.com/git-stunts/git-cas/issues/99 | primary | close after merge | + +## Done Does Not Mean + +When this lands, it does not prove every downstream caller uses batches, nor +does it authorize git-warp v19 release without its full migration rehearsal. + +## Retrospective + +Fill this section after implementation and review. diff --git a/docs/design/0054-batched-page-retention/witness/verification.md b/docs/design/0054-batched-page-retention/witness/verification.md new file mode 100644 index 0000000..de1baf3 --- /dev/null +++ b/docs/design/0054-batched-page-retention/witness/verification.md @@ -0,0 +1,61 @@ +# PERF-0054 verification + +Date: 2026-07-26 + +Issue: https://github.com/git-stunts/git-cas/issues/99 + +## Playback answers + +### Does one page batch install one exact workspace generation? + +Yes. Unit coverage observes one `updateRef()` call for a three-input batch, +including duplicate content. All ordered results carry retained witnesses for +the same generation. The Git-backed integration confirms every page remains +reachable through that generation across immediate prune and becomes +unreachable after exact workspace release. + +### Does batching remove the measured scale failure? + +Yes. A disposable bare repository staged 8,188 deterministic tiny pages in 32 +bounded batches of at most 256 pages: + +```json +{ + "pageCount": 8188, + "batches": 32, + "elapsedMs": 15546, + "generationMatchesHead": true, + "reachableEntries": 8190, + "looseObjectCount": 128, + "looseObjectSize": "8.27 MiB", + "packedObjectCount": 8188, + "packCount": 32, + "garbage": 0 +} +``` + +The pre-change git-warp rehearsal retained only 5,213 of the same 8,188-page +cardinality after 75 minutes and had produced about 1.16 GiB of loose scratch +objects. The witness uses synthetic tiny pages and contains no Think data. + +## Commands + +```text +pnpm vitest run test/unit/domain/services/StagingWorkspace.test.js + 22 passed + +docker compose run --build --rm test-node \ + npx vitest run test/integration/staging-workspace.test.js \ + --no-file-parallelism + 7 passed + +pnpm lint + passed + +pnpm test + 2,085 passed, 2 skipped +``` + +The final `release:verify -- --skip-jsr` run passed all 13 executed gates: +6,841 tests across Node, Bun, and Deno unit/integration suites, public type +compatibility, examples, lint, build stamping, and the npm pack dry-run. diff --git a/docs/design/README.md b/docs/design/README.md index 169e52e..c962d54 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -11,6 +11,7 @@ process in [docs/method/process.md](../method/process.md). ## Active METHOD Cycles +- [0054-batched-page-retention - batched-page-retention](./0054-batched-page-retention/batched-page-retention.md) - [0050-lazy-bundle-reference-reads - lazy-bundle-reference-reads](./0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md) - [0049-scoped-staging-workspaces — scoped-staging-workspaces](./0049-scoped-staging-workspaces/scoped-staging-workspaces.md) - [0048-scoped-cache-acquisitions — scoped-cache-acquisitions](./0048-scoped-cache-acquisitions/scoped-cache-acquisitions.md) diff --git a/index.d.ts b/index.d.ts index 19daf80..1133307 100644 --- a/index.d.ts +++ b/index.d.ts @@ -1637,6 +1637,9 @@ export declare class StagingWorkspace { }; readonly pages: { put(options: { source: PageSource; maxBytes?: number }): Promise; + putBatch( + options: Parameters[0], + ): Promise>; }; readonly bundles: { put(options: Parameters[0]): Promise; diff --git a/src/domain/services/StagingWorkspace.js b/src/domain/services/StagingWorkspace.js index 0aadc8e..0566334 100644 --- a/src/domain/services/StagingWorkspace.js +++ b/src/domain/services/StagingWorkspace.js @@ -61,6 +61,7 @@ export default class StagingWorkspace { }); this.pages = Object.freeze({ put: (options) => this.#enqueue(() => this.#stage(this.#pages, 'put', options)), + putBatch: (options) => this.#stagePageBatch(options), }); this.bundles = Object.freeze({ put: (options) => this.#enqueue(() => this.#stage(this.#bundles, 'put', options)), @@ -205,6 +206,68 @@ export default class StagingWorkspace { } } + async #stageBatch(service, method, options) { + this.#assertActive(); + const staged = await service[method](options); + if (!Array.isArray(staged) || staged.some((page) => !page?.handle)) { + throw createCasError( + `Workspace ${method}() did not return staged handles`, + ErrorCodes.WORKSPACE_STATE_INVALID, + { method }, + ); + } + if (staged.length === 0) { + return Object.freeze([]); + } + try { + return await this.#retainBatch(staged); + } catch (error) { + if (error?.code === ErrorCodes.WORKSPACE_TTL_INVALID) { + throw error; + } + throw createCasError( + 'Workspace staged a batch but could not establish retention', + ErrorCodes.WORKSPACE_RETENTION_FAILED, + { + method, + workspaceId: this.id, + stagedCount: staged.length, + originalError: error, + }, + ); + } + } + + #stagePageBatch(options) { + return this.#enqueue(() => this.#stageBatch(this.#pages, 'putBatch', options)); + } + + async #retainBatch(staged) { + const resolved = []; + const targets = new Map(this.#targets); + for (const page of staged) { + const target = await this.#resolveTarget(page.handle); + resolved.push(target); + targets.set(target.handle.toString(), target); + } + const installation = await this.#install([...targets.values()]); + const witnesses = new Map( + installation.witnesses.map((witness) => [witness.handle.toString(), witness]), + ); + return Object.freeze(staged.map((page, index) => { + const handle = resolved[index].handle.toString(); + const witness = witnesses.get(handle); + if (!witness) { + throw createCasError( + 'Workspace generation omitted a newly staged batch handle', + ErrorCodes.WORKSPACE_STATE_INVALID, + { handle, generation: installation.generation }, + ); + } + return StagingWorkspace.#retainedStage(page, witness); + })); + } + async #install(targets) { if (targets.length > MAX_WORKSPACE_TARGETS) { throw createCasError( @@ -509,7 +572,7 @@ export default class StagingWorkspace { ['rootSet', StagingWorkspace.#hasMethods(rootSet, ['replace'])], ['refs', StagingWorkspace.#hasMethods(refs, ['deleteRef'])], ['assets', StagingWorkspace.#hasMethods(assets, ['put', 'adopt'])], - ['pages', StagingWorkspace.#hasMethods(pages, ['put'])], + ['pages', StagingWorkspace.#hasMethods(pages, ['put', 'putBatch'])], ['bundles', StagingWorkspace.#hasMethods(bundles, ['put', 'putOrdered'])], ['resolveHandle', typeof resolveHandle === 'function'], ['descriptorCodec', StagingWorkspace.#hasMethods(descriptorCodec, ['encode'])], diff --git a/src/domain/services/StagingWorkspaceRegistry.js b/src/domain/services/StagingWorkspaceRegistry.js index 97a60d9..fe8ba0b 100644 --- a/src/domain/services/StagingWorkspaceRegistry.js +++ b/src/domain/services/StagingWorkspaceRegistry.js @@ -24,6 +24,13 @@ const TARGET_ROOT_TYPES = Object.freeze({ page: 'blob', }); const DEFAULT_CLOCK = Object.freeze({ now: () => new Date() }); +const PERSISTENCE_METHODS = Object.freeze([ + 'writeBlob', 'writeTree', 'readBlob', 'readTree', 'readObjectType', 'readObjectSize', +]); +const REF_METHODS = Object.freeze([ + 'resolveRef', 'resolveTree', 'resolveParents', 'createCommit', 'updateRef', 'deleteRef', + 'iterateRefs', +]); /** Creates staging workspaces and provides bounded abandoned-workspace operations. */ export default class StagingWorkspaceRegistry { @@ -365,15 +372,10 @@ export default class StagingWorkspaceRegistry { static #assertDependencies({ persistence, ref, assets, pages, bundles, resolveHandle, crypto, clock, descriptorCodec }) { const dependencies = [ - ['persistence', StagingWorkspaceRegistry.#hasMethods(persistence, [ - 'writeBlob', 'writeTree', 'readBlob', 'readTree', 'readObjectType', 'readObjectSize', - ])], - ['ref', StagingWorkspaceRegistry.#hasMethods(ref, [ - 'resolveRef', 'resolveTree', 'resolveParents', 'createCommit', 'updateRef', 'deleteRef', - 'iterateRefs', - ])], + ['persistence', StagingWorkspaceRegistry.#hasMethods(persistence, PERSISTENCE_METHODS)], + ['ref', StagingWorkspaceRegistry.#hasMethods(ref, REF_METHODS)], ['assets', StagingWorkspaceRegistry.#hasMethods(assets, ['put', 'adopt'])], - ['pages', StagingWorkspaceRegistry.#hasMethods(pages, ['put', 'get'])], + ['pages', StagingWorkspaceRegistry.#hasMethods(pages, ['put', 'putBatch', 'get'])], ['bundles', StagingWorkspaceRegistry.#hasMethods(bundles, ['put', 'putOrdered'])], ['resolveHandle', typeof resolveHandle === 'function'], ['crypto', StagingWorkspaceRegistry.#hasMethods(crypto, ['randomBytes'])], diff --git a/test/integration/staging-workspace.test.js b/test/integration/staging-workspace.test.js index 25e3f42..352a0ef 100644 --- a/test/integration/staging-workspace.test.js +++ b/test/integration/staging-workspace.test.js @@ -67,6 +67,13 @@ async function collect(source) { return Buffer.concat(buffers); } +async function expectBundleMember(handle, memberPath, expected) { + await expect(cas.bundles.getMember({ + handle, + path: memberPath, + })).resolves.toMatchObject(expected); +} + beforeAll(async () => { repoDir = mkdtempSync(path.join(os.tmpdir(), 'cas-staging-workspace-')); tempDirs.push(repoDir); @@ -107,6 +114,45 @@ describe('scoped staging workspace direct reachability', () => { }); }); +describe('scoped staging workspace page batches', () => { + it('retains a bounded page batch under one exact workspace generation', async () => { + const workspace = await cas.workspaces.open({ + namespace: 'integration/direct-page-batch', + ttlMs: 60_000, + }); + + const staged = await workspace.pages.putBatch({ + pages: Array.from({ length: 32 }, (_, index) => ({ + source: Buffer.from(`retained batch page ${index}`), + })), + maxBatchBytes: 64 * 1024, + maxBatchPages: 32, + }); + + expect(staged).toHaveLength(32); + expect(new Set(staged.map((page) => page.witness.root.generation))).toEqual( + new Set([staged[0].witness.root.generation]), + ); + expect(git(['rev-parse', staged[0].witness.root.ref])).toBe( + staged[0].witness.root.generation, + ); + const reachable = reachableOids(staged[0].witness.root.ref); + for (const page of staged) { + expect(reachable).toContain(page.handle.oid); + } + pruneNow(); + for (const page of staged) { + expect(objectExists(page.handle.oid)).toBe(true); + } + + await workspace.release(); + pruneNow(); + for (const page of staged) { + expect(objectExists(page.handle.oid)).toBe(false); + } + }); +}); + describe('scoped staging workspace composition', () => { it('survives prune through bundle construction, checkpoint, and cache promotion', async () => { const cache = await cas.caches.open({ namespace: 'integration/promoted-materializations' }); @@ -178,10 +224,9 @@ describe('scoped staging workspace mirrored capabilities', () => { pruneNow(); expect(await collect(cas.assets.open({ handle: adopted.handle }))).toEqual(secondPayload); - await expect(cas.bundles.getMember({ - handle: bundle.handle, + await expectBundleMember(bundle.handle, 'inline/value.txt', { path: 'inline/value.txt', - })).resolves.toMatchObject({ path: 'inline/value.txt' }); + }); expect(bundle.toJSON().witness.root.kind).toBe('root-set'); const inspection = await cas.workspaces.inspect({ namespace: 'integration/mirrored-capabilities', diff --git a/test/unit/domain/services/StagingWorkspace.test.js b/test/unit/domain/services/StagingWorkspace.test.js index ad241c5..bf93cc4 100644 --- a/test/unit/domain/services/StagingWorkspace.test.js +++ b/test/unit/domain/services/StagingWorkspace.test.js @@ -84,6 +84,27 @@ function inventoryPage(records, after, limit) { return records.filter((record) => after === null || record.ref > after).slice(0, limit); } +async function expectRetainedPageBatch({ + fixture, + staged, + expectedHandles, + updateRef, +}) { + expect(staged).toHaveLength(3); + expect(staged.map((page) => page.handle.toString())).toEqual(expectedHandles); + expect(staged.map((page) => page.state)).toEqual(['retained', 'retained', 'retained']); + expect(new Set(staged.map((page) => page.witness.root.generation))).toEqual( + new Set([staged[0].witness.root.generation]), + ); + expect(updateRef).toHaveBeenCalledTimes(1); + + const roots = await reachableOids(fixture, staged[0].witness.root.ref); + expect(roots.headOid).toBe(staged[0].witness.root.generation); + expect(roots.oids).toContain(staged[0].handle.oid); + expect(roots.oids).toContain(staged[1].handle.oid); + expect(roots.oids).toHaveLength(4); +} + describe('StagingWorkspace staging', () => { it('returns from page staging only after the exact workspace generation reaches it', async () => { const fixture = makeFixture(); @@ -115,6 +136,75 @@ describe('StagingWorkspace staging', () => { }); }); +describe('StagingWorkspace page batches', () => { + it('retains one ordered page batch in one workspace generation', async () => { + const fixture = makeFixture(); + const updateRef = vi.spyOn(fixture.ref, 'updateRef'); + const sources = [ + Buffer.from('first retained page'), + Buffer.from('second retained page'), + Buffer.from('first retained page'), + ]; + const expectedHandles = await Promise.all(sources.map(async (source) => ( + (await fixture.pages.put({ source })).handle.toString() + ))); + const workspace = await fixture.registry.open({ + namespace: 'git-warp/materializations', + ttlMs: TTL_MS, + }); + + const staged = await workspace.pages.putBatch({ + pages: sources.map((source) => ({ source })), + }); + + await expectRetainedPageBatch({ + fixture, + staged, + expectedHandles, + updateRef, + }); + }); + + it('rejects an oversized page batch before mutating the workspace ref', async () => { + const fixture = makeFixture(); + const updateRef = vi.spyOn(fixture.ref, 'updateRef'); + const workspace = await fixture.registry.open({ + namespace: 'git-warp/materializations', + ttlMs: TTL_MS, + }); + + await expect(workspace.pages.putBatch({ + pages: [ + { source: Buffer.from('first') }, + { source: Buffer.from('second') }, + ], + maxBatchPages: 1, + })).rejects.toMatchObject({ + code: 'PAGE_BATCH_LIMIT', + }); + expect(updateRef).not.toHaveBeenCalled(); + }); +}); + +describe('StagingWorkspace page batch failure containment', () => { + it('returns no retained batch when the workspace generation cannot install', async () => { + const fixture = makeFixture(); + vi.spyOn(fixture.ref, 'updateRef').mockRejectedValueOnce( + new Error('simulated ref failure'), + ); + const workspace = await fixture.registry.open({ + namespace: 'git-warp/materializations', + ttlMs: TTL_MS, + }); + + await expect(workspace.pages.putBatch({ + pages: [{ source: Buffer.from('unretained batch page') }], + })).rejects.toMatchObject({ + code: 'WORKSPACE_RETENTION_FAILED', + }); + }); +}); + describe('StagingWorkspace expiry bounds', () => { it('maps an overflowing expiry to a stable workspace TTL error', async () => { const fixture = makeFixture();