Skip to content

Commit 623b686

Browse files
feat(api-extractor): versioned API model, Effect-aware diff and semver verdict (PR1) (#1)
* docs(plans): requirements for the Effect-aware public-API tool Requirements-only unified plan (ce-brainstorm, round-3 contract): premise table with sources, a survey of what already solves the problem, approach B (model-first), and R1-R23 mapped to Done 1-9. * docs(plans): planning contract and units for the Effect-aware public-API tool * feat(api-extractor): versioned API model, Effect-aware diff and semver verdict Repository foundation (MIT with Microsoft's notice, packs pinned to a systemfsoftware main SHA) and the model core: an Effect Schema model with schemaVersion 1, one canonical serialiser, a polarity-aware diff that decomposes Effect/Stream/Layer channels and expands local aliases, and a declarative rule corpus whose unmatched changes fail closed as major. Property tests cover AE1-AE5; the locale test runs the serialiser in child processes under tr_TR, de_DE and C. Plan dispositions recorded in the doc. * docs(plans): supersede the plan with the public-family-package ruling The 0205 plan was frozen once PR1 units ran, so the root ruling of 2026-10-10 lands as a new dated plan declaring supersedes: public systemfsoftware packages come from this repository's flake now (KTD2), the verdict is a cell (KTD14), .sfs-deps is built in CI (KTD12), and R8 keeps the oracle's compiler line on the fallback (KTD9). Only tsdown-config, vitest-config and stryker-config wait on P1. * feat(api-extractor): verdict on effect-cell-types, packages from the flake flake.nix pins systemfsoftware ba6664c1; its sfs-deps output and the consumer-store check (lib.mkConsumerStore) supply effect-cell-types, effect-schema-law, vitest and tsconfig as .sfs-deps tarballs. The verdict is a Workflow decision (judgeSurfaces) run by a Sandwich cell that writes canonical JSON or fails with SurfaceRejected / RuleCorpusRejected. Codec laws via ruleOfSchemas sit next to each schema. CI builds .sfs-deps once and compares x86_64 and arm64 output bytes. * refactor(api-extractor): simplify the PR1 core without changing its bytes ce-simplify-code pass: compare normalised types by canonical key instead of re-normalising at every level; key lattice members and overload signatures once; derive bump order and keyword names from their schemas; share the corpus-ambiguity fields between the decision error and the cell error; one outputsOf fixture for both determinism probes. host-digest over 200 fixed-seed samples is byte-identical before and after (e02a7331). * fix(api-extractor): resolve the code-review findings on the PR1 core Channel types read defaulted trailing arguments (Effect<A, E>, Layer<S, E>): corpus 1.1.0 declares each optional channel's default. Aliases reached through type-parameter defaults and constraints expand. Exports sharing a name (a value beside a type) pair by kind, and serialise in kind order. Class heritage keeps extends and implements apart. The corpus gate refuses equal-specificity ties and the reserved unclassified-change id. Schema laws come from effect-schema-vite (flake tarball): TypeExpr, Member and ApiItem declare recursion budgets at their recursion points, and the hand-written ruleOfSchemas registrations it supersedes are removed. The model barrel publishes the union members and KeywordName as a value. Every behavioural fix has a property test that fails on 8da33d7. * test(api-extractor): move PR1's suite onto the family test idiom Root ruling 2026-10-10: migrate inside PR1. - Decision and serialiser behaviour are it.prop laws over the production schemas, with the constant-impostor gate on. - Known-answer tables are generator-body example tests through the public surface, and every corpus reason code now has a row (review finding #12). - The locale test is an e2e it.prop in apps/api-extractor-e2e. It spawns processes, so the test-layer gate puts it outside the package. It drives only the public surface, and host-digest moved there with it. Drawing from the ApiModel schema turned up models that repeat an enum member name or an entry-point path. The diff of such a model against itself was not empty. The schema now refuses them (uniqueBy, with a generation hint), so API_MODEL_SCHEMA_VERSION is 2. The hand-mirrored arbitraries and fast-check are gone (DEL1). Plan 0517 supersedes plan 0314. * ci: upload the hidden .sfs-deps directory, and fail the job when nothing uploads upload-artifact v4+ skips hidden paths unless include-hidden-files is set, and if-no-files-found: warn let the sfs-deps job pass with no artifact, so every downstream job failed on download instead. * docs(solutions): keyed uniqueness is a schema invariant only for identities ce-compound learning from PR1: repeated export names are a diff pairing rule (declaration merging), entry-point paths and enum member names are schema refinements carrying the uniqueBy generation hint. * docs(agents): read docs/solutions before planning or changing code Sub-conductor ruling 2026-10-10: the learnings store is the first read. * fix(api-extractor): pair same-name same-kind exports independent of storage order Resolves the ce-code-review findings on 8da33d7..1f31aaf: - compareNamed paired same-kind items under one name by stored position, so two byte-identical models storing them in opposite orders diffed as changed. Equal items now pair first and the rest pair in canonical-key order. Two it.prop laws under "verdict on exports sharing a name and a kind" fail on the old pairing and pass now. - The cross-kind guard in compareItem was unreachable (groups are per kind) and is gone. - The cell refuses a repeated entry-point path and a repeated enum member name with SurfaceRejected; example rows pin both refusals. - The change-kind table takes model pairs and has a row for each of the 11 corpus reason codes that lacked one, so every code has a row as the plan ruling requires. - The keyed-uniqueness solution doc records the storage-order defect.
1 parent 11e027b commit 623b686

48 files changed

Lines changed: 5557 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.compound-engineering/config.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
packs:
2+
- source: https://github.com/systemfsoftware/systemfsoftware.git
3+
ref: df908ddfe497e53c1c03b92519a96c913c441a58
4+
path: compound-packs
5+
pack: [cell-architecture, boundary-testing, schema-laws, package-topology]

‎.github/workflows/ci.yml‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
name: CI
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: [main]
7+
8+
permissions:
9+
contents: read
10+
11+
concurrency:
12+
group: ci-${{ github.ref }}
13+
cancel-in-progress: true
14+
15+
jobs:
16+
# The @systemfsoftware tarballs the lockfile installs from, built by this
17+
# repository's flake from the pinned systemfsoftware commit. The tarballs are
18+
# architecture-independent, so one build serves every job below.
19+
sfs-deps:
20+
name: systemfsoftware tarballs from the flake
21+
runs-on: ubuntu-latest
22+
timeout-minutes: 60
23+
steps:
24+
- uses: actions/checkout@v7
25+
- uses: cachix/install-nix-action@v31
26+
- name: Build the tarballs the lockfile names
27+
run: cp -rL "$(nix build --no-link --print-out-paths .#sfs-deps)" .sfs-deps
28+
- name: The lockfile's recorded integrities match the flake's tarballs
29+
run: nix build --no-link .#checks.x86_64-linux.consumer-store
30+
- uses: actions/upload-artifact@v7
31+
with:
32+
name: sfs-deps
33+
path: .sfs-deps/
34+
retention-days: 1
35+
include-hidden-files: true
36+
if-no-files-found: error
37+
38+
test:
39+
name: Product tests
40+
needs: sfs-deps
41+
runs-on: ubuntu-latest
42+
timeout-minutes: 15
43+
steps:
44+
- uses: actions/checkout@v7
45+
- uses: actions/download-artifact@v8
46+
with:
47+
name: sfs-deps
48+
path: .sfs-deps
49+
- uses: pnpm/action-setup@v6
50+
- uses: actions/setup-node@v7
51+
with:
52+
node-version: 24
53+
cache: pnpm
54+
- run: pnpm install --frozen-lockfile
55+
- run: pnpm test
56+
57+
host-bytes:
58+
name: Output bytes on ${{ matrix.arch }}
59+
needs: sfs-deps
60+
strategy:
61+
matrix:
62+
include:
63+
- arch: x86_64
64+
runner: ubuntu-latest
65+
- arch: arm64
66+
runner: ubuntu-24.04-arm
67+
runs-on: ${{ matrix.runner }}
68+
timeout-minutes: 15
69+
steps:
70+
- uses: actions/checkout@v7
71+
- uses: actions/download-artifact@v8
72+
with:
73+
name: sfs-deps
74+
path: .sfs-deps
75+
- uses: pnpm/action-setup@v6
76+
- uses: actions/setup-node@v7
77+
with:
78+
node-version: 24
79+
cache: pnpm
80+
- run: pnpm install --frozen-lockfile
81+
- name: Serialise a fixed-seed sample of models and verdicts
82+
run: node apps/api-extractor-e2e/host-digest.ts > host-output.bin
83+
- run: sha256sum host-output.bin && uname -m
84+
- uses: actions/upload-artifact@v7
85+
with:
86+
name: host-output-${{ matrix.arch }}
87+
path: host-output.bin
88+
89+
host-compare:
90+
name: Same bytes on every host
91+
needs: host-bytes
92+
runs-on: ubuntu-latest
93+
timeout-minutes: 5
94+
steps:
95+
- uses: actions/download-artifact@v8
96+
with:
97+
pattern: host-output-*
98+
- run: sha256sum host-output-*/host-output.bin
99+
- name: Compare x86_64 and arm64 outputs byte for byte
100+
run: cmp host-output-x86_64/host-output.bin host-output-arm64/host-output.bin

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
node_modules/
2+
dist/
3+
coverage/
4+
*.tsbuildinfo
5+
# systemfsoftware tarballs the lockfile installs from: `nix build .#sfs-deps`, copied here
6+
.sfs-deps/

‎AGENTS.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# AGENTS.md
2+
3+
`@systemfsoftware/api-extractor`: versioned API model, diff and Effect-aware semver
4+
verdict. Requirements and decisions: `docs/plans/2026-10-10-0517-feat-effect-api-extractor-plan.md`.
5+
6+
## Rules
7+
8+
- Read `docs/solutions/` before planning or changing code here.
9+
- Every output goes through `canonicalJson` (`packages/api-extractor/src/canonical-json.ts`).
10+
Order strings with `compareCodeUnits`; never `localeCompare`, `toLocale*` or `Intl` in `src`.
11+
- A change to the model schema bumps `API_MODEL_SCHEMA_VERSION`.
12+
- Bump semantics live in the rule corpus (`src/rules/corpus.v1.json`), not in code. A
13+
corpus change bumps its `corpusVersion`.
14+
- A file derived from `microsoft/rushstack` or systemfsoftware PR #552 carries a header
15+
naming the repository, path and commit.
16+
- Shared tooling (tsconfig bases, lint presets, release, mutation) comes from
17+
systemfsoftware family flakes and reusable workflows. Never copy it in.
18+
- Mutation testing runs on `main` only.
19+
20+
## Commands
21+
22+
```sh
23+
pnpm install
24+
pnpm test # property tests, including the cross-locale process test
25+
pnpm typecheck
26+
```

‎LICENSE‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,34 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1919
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2020
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2121
SOFTWARE.
22+
23+
------------------------------------------------------------------------------
24+
25+
Portions of this software are derived from @microsoft/api-extractor 7.59.1
26+
(https://github.com/microsoft/rushstack, apps/api-extractor, commit
27+
bc4cd29cc4984cef267437565bf681582948285e), distributed under the following
28+
license:
29+
30+
@microsoft/api-extractor
31+
32+
Copyright (c) Microsoft Corporation. All rights reserved.
33+
34+
MIT License
35+
36+
Permission is hereby granted, free of charge, to any person obtaining a copy of
37+
this software and associated documentation files (the "Software"), to deal in
38+
the Software without restriction, including without limitation the rights to
39+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
40+
of the Software, and to permit persons to whom the Software is furnished to do
41+
so, subject to the following conditions:
42+
43+
The above copyright notice and this permission notice shall be included in all
44+
copies or substantial portions of the Software.
45+
46+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
47+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
48+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
49+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
50+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
51+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
52+
SOFTWARE.

‎README.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# api-extractor-effect
2+
3+
`@systemfsoftware/api-extractor` is a public-API tool for Effect-heavy TypeScript
4+
libraries. It reduces a package to one versioned JSON API model, diffs two models,
5+
and gives a semver verdict that understands Effect's `A`/`E`/`R` channels and dual
6+
APIs. Every output is byte-identical across locales, time zones and hosts.
7+
8+
The design and its requirements live in
9+
[`docs/plans/2026-10-10-0517-feat-effect-api-extractor-plan.md`](docs/plans/2026-10-10-0517-feat-effect-api-extractor-plan.md).
10+
11+
## Status
12+
13+
| Part | State |
14+
|---|---|
15+
| Versioned model, canonical serialiser, diff, rule corpus and verdict | in this repository |
16+
| Front-ends (isolated-declarations fast path, compiler fallback) | next |
17+
| `.api.md` report with a differential test against `@microsoft/api-extractor` 7.59.1 | planned |
18+
| Benchmark, CLI, changeset-bump check | planned |
19+
| systemfsoftware packages from the flake (`.sfs-deps`) | in this repository |
20+
| Release, mutation, shared lint/build jobs, tsdown and vitest presets | waits on shared family tooling |
21+
22+
## The verdict
23+
24+
```ts
25+
import { Effect } from 'effect'
26+
import { Verdict } from '@systemfsoftware/api-extractor'
27+
28+
// `before` and `after` are encoded API models (the JSON this tool writes).
29+
const document = await Effect.runPromise(Verdict.cell.run({ before, after }))
30+
process.stdout.write(document) // canonical JSON: { _tag, schemaVersion, engine, corpusVersion, declarations, bump, findings }
31+
```
32+
33+
`bump` is `none`, `patch`, `minor` or `major`; each finding is
34+
`{ rule_id, reason_code, bump, next_action, locations }`. A model that does not decode
35+
fails the cell with `SurfaceRejected`; a rule corpus in which two rules match the same
36+
change fails it with `RuleCorpusRejected`. The decision itself, `Verdict.judge`, is a
37+
pure `effect-cell-types` workflow over decoded values.
38+
39+
Rules are data: [`src/rules/corpus.v1.json`](packages/api-extractor/src/rules/corpus.v1.json)
40+
maps each kind of change, its position (`in`, `out`, `both`), its Effect channel and
41+
its direction (`widened`, `narrowed`, `changed`) to a bump. The most specific matching
42+
rule wins. A change no rule covers is reported as major `UNCLASSIFIED_CHANGE`.
43+
44+
For example, a function returning `Effect<User, NotFound>` that now returns
45+
`Effect<User, NotFound | Timeout>` is major `E_WIDENED`; the same change to a parameter
46+
is minor `E_WIDENED_IN_INPUT`. Removing a tagged error from a returned error channel is
47+
major `E_TAG_REMOVED`, because a caller's `catchTag` for that tag stops compiling.
48+
49+
## Development
50+
51+
The `@systemfsoftware/*` packages install from tarballs that this repository's flake
52+
builds from a pinned systemfsoftware commit, never from a registry:
53+
54+
```sh
55+
cp -rL "$(nix build --no-link --print-out-paths .#sfs-deps)" .sfs-deps
56+
pnpm install
57+
pnpm test
58+
pnpm typecheck
59+
```
60+
61+
## Provenance
62+
63+
This repository is MIT-licensed. The behaviour oracle is `@microsoft/api-extractor`
64+
7.59.1 (`microsoft/rushstack` `apps/api-extractor` at
65+
`bc4cd29cc4984cef267437565bf681582948285e`), whose MIT notice is reproduced in
66+
[`LICENSE`](LICENSE). A file derived from upstream, or from systemfsoftware PR #552,
67+
names its origin (repository, path and commit) in a header comment. No file in the
68+
current tree is derived from either.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
import { ApiModel } from '@systemfsoftware/api-extractor'
2+
import { Arbitrary, Effect, Schema } from 'effect'
3+
import { outputsOf } from './outputs.ts'
4+
5+
const Models = Schema.Array(ApiModel.ApiModel).check(Schema.isBetweenLength(2, 4))
6+
const samples = Effect.runSync(Arbitrary.sampleEffect(Arbitrary.schema(Models), { count: 200, seed: 20261010 }))
7+
process.stdout.write(samples.flatMap(outputsOf).join(''))
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
import { ApiModel } from '@systemfsoftware/api-extractor'
2+
import { describe } from '@systemfsoftware/vitest'
3+
import { Schema } from 'effect'
4+
import { spawnSync } from 'node:child_process'
5+
import { mkdtempSync } from 'node:fs'
6+
import { tmpdir } from 'node:os'
7+
import { join } from 'node:path'
8+
import { fileURLToPath } from 'node:url'
9+
import { outputsOf } from './outputs.ts'
10+
11+
const probe = fileURLToPath(new URL('./probe.ts', import.meta.url))
12+
const elsewhere = mkdtempSync(join(tmpdir(), 'api-extractor-locale-'))
13+
14+
const environments = [
15+
{ LC_ALL: 'C', TZ: 'UTC', cwd: process.cwd(), expectLocale: undefined },
16+
{ LC_ALL: 'tr_TR.UTF-8', TZ: 'Europe/Istanbul', cwd: process.cwd(), expectLocale: 'tr' },
17+
{ LC_ALL: 'de_DE.UTF-8', TZ: 'America/Los_Angeles', cwd: elsewhere, expectLocale: 'de' },
18+
] as const
19+
type Environment = (typeof environments)[number]
20+
21+
const inherited = Object.fromEntries(Object.entries(process.env).filter(([key]) => !/^(LC_|LANG|LANGUAGE$|TZ$)/.test(key)))
22+
23+
interface ProbeResult {
24+
readonly locale: string
25+
readonly timeZone: string
26+
readonly cwd: string
27+
readonly outputs: ReadonlyArray<string>
28+
}
29+
30+
const run = (environment: Environment, input: string): ProbeResult => {
31+
const child = spawnSync(process.execPath, [probe], {
32+
input,
33+
cwd: environment.cwd,
34+
env: { ...inherited, LC_ALL: environment.LC_ALL, TZ: environment.TZ },
35+
encoding: 'utf8',
36+
})
37+
if (child.status !== 0) throw new Error(`probe failed under ${environment.LC_ALL}: ${child.stderr}`)
38+
return JSON.parse(child.stdout) as ProbeResult
39+
}
40+
41+
const runEverywhere = (models: ReadonlyArray<ApiModel.ApiModel>): ReadonlyArray<ProbeResult> => {
42+
const input = JSON.stringify(models.map(ApiModel.encode))
43+
return environments.map((environment) => run(environment, input))
44+
}
45+
46+
const resolved = (environment: Environment, result: ProbeResult): boolean =>
47+
(environment.expectLocale === undefined || result.locale.startsWith(environment.expectLocale)) &&
48+
result.timeZone === environment.TZ &&
49+
result.cwd === environment.cwd
50+
51+
describe('output bytes across locales, time zones and working directories', (it) => {
52+
it.prop(
53+
'AE5: every model and verdict serialises, in every environment, to the bytes this process writes',
54+
{ of: [Schema.Array(ApiModel.ApiModel).check(Schema.isBetweenLength(2, 4))], subject: runEverywhere, runs: 4 },
55+
(runEverywhere, [models]) => {
56+
const expected = JSON.stringify(outputsOf(models))
57+
return runEverywhere(models).every((result, index) => resolved(environments[index]!, result) && JSON.stringify(result.outputs) === expected)
58+
},
59+
)
60+
})

‎apps/api-extractor-e2e/outputs.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
import { ApiModel, Verdict } from '@systemfsoftware/api-extractor'
2+
import { Effect } from 'effect'
3+
4+
/** The bytes the determinism checks compare: every model, then the verdict between each consecutive pair. */
5+
export const outputsOf = (models: ReadonlyArray<ApiModel.ApiModel>): ReadonlyArray<string> => [
6+
...models.map(ApiModel.serialise),
7+
...models.slice(1).map((after, index) => Effect.runSync(Verdict.cell.run({ before: models[index]!, after }))),
8+
]
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"name": "api-extractor-e2e",
3+
"private": true,
4+
"type": "module",
5+
"engines": {
6+
"node": ">=24"
7+
},
8+
"scripts": {
9+
"test": "vitest run",
10+
"typecheck": "tsc --noEmit"
11+
},
12+
"devDependencies": {
13+
"@systemfsoftware/api-extractor": "workspace:*",
14+
"@systemfsoftware/tsconfig": "^2.0.1",
15+
"@systemfsoftware/vitest": "^2.0.0",
16+
"@types/node": "catalog:",
17+
"effect": "catalog:",
18+
"typescript": "catalog:",
19+
"vitest": "catalog:"
20+
}
21+
}

0 commit comments

Comments
 (0)