Tested against v0.4.0 (darwin_arm64 release tarball; checksums.txt sha256 matched, and
its sigstore bundle verifies with cosign against
https://github.com/okfcli/okf/.github/workflows/release.yml@refs/tags/v0.4.0).
What is missing
Both validate and lint report warnings and exit 0, and neither accepts a flag:
$ okf validate --help
error[usage]: unknown validate flag: --help
$ okf schema | grep -ci 'strict\|severity\|promote'
0
So there is no supported way to make a CI job or a commit hook fail on a warning. A warning that
nobody is obliged to act on is not a gate — it is a log line, and in a repository that anyone
commits to it will eventually be one nobody reads.
I am not arguing these findings should be errors. For the ones I care about — the footnote rules
from #20 — the spec treats footnotes only as the sources join key and requires neither direction,
so warning is the correct upstream default. The judgement is local; the mechanism is not. Today
each project that wants to enforce one has to build the mechanism itself.
What I had to build, and what its shape suggests
I maintain a small wrapper that shells out to okf validate, parses the JSON, and exits non-zero if
a finding's rule ID is in a promoted set. Three things about it are, I think, requirements rather
than preferences — each was learned by getting it wrong first.
1 — Per-rule promotion, not only a global --strict.
Different contexts need different subsets, and one of mine cannot be expressed by a single switch.
The three footnote rules are rendering faults caused by the diff in front of you, so I promote them
on every commit. okf/lifecycle/stale is not: it arrives because a date passed, on a repository
nobody has touched. Failing a commit hook on it blocks whoever commits next for something they did
not do — and still misses the case that matters, the untouched repository where no commit ever
comes. So it is promoted only in a weekly scheduled run. A global strict mode makes those two
contexts the same.
2 — An unknown rule ID must be an error, not a no-op.
This is the one that decides whether the feature is safe. If a promoted ID is silently ignored, then
a typo — or a rule renamed upstream — turns the gate into a pass while enforcing nothing, and it
reports success while doing it. My wrapper has to refuse an okf older than the release that
introduced the rules for exactly this reason: on an older binary they are never emitted, so the gate
would pass while checking nothing. If the flag validates its arguments against the known rule set,
no consumer needs that guard.
3 — Additive and replacing forms are not interchangeable.
If a default promoted set ever ships, a caller wanting one more rule will reach for the replacing
form and thereby stop enforcing anything added to the defaults later — silently, and reporting a
pass. I ended up with both spellings and a note explaining which to prefer. Worth deciding
deliberately upstream rather than discovering it downstream.
Possible shapes
No preference between these; the requirement is only that (1) and (2) above survive.
--fail-on <severity> / --min-severity <severity> — simplest, but cannot express the
per-context split in (1) on its own.--promote <rule-id> (repeatable) — expresses everything I need; must reject unknown IDs.- A distinct exit code for "no errors, but warnings present" — leaves the policy to the caller and
needs no new flags, though it is coarser than per-rule.
The raw material is already there: findings carry stable rule IDs, and okf schema publishes CLI
metadata, so the set of valid IDs is machine-readable.
Relation to #26
#26 asks for broken links to be reclassified as ERROR. Reclassifying a rule moves the line for
everyone, and each such request has to be argued on its merits. A promotion mechanism lets each
project draw its own line and settles the whole class of request rather than one instance — so I
think this is the more general fix, and would leave #26 as a question about the default rather
than about enforceability.
Why I care
The wrapper is about 400 lines of Python and its README says, in as many words, that it exists to be deleted
the day okf can be made to exit non-zero on a warning. I would rather delete it than maintain it,
and I would rather other projects did not each write their own.
Happy to send a PR if you would like one, and happy to be told this belongs in a caller instead.
Tested against v0.4.0 (
darwin_arm64release tarball;checksums.txtsha256 matched, andits sigstore bundle verifies with
cosignagainsthttps://github.com/okfcli/okf/.github/workflows/release.yml@refs/tags/v0.4.0).What is missing
Both
validateandlintreport warnings and exit 0, and neither accepts a flag:So there is no supported way to make a CI job or a commit hook fail on a warning. A warning that
nobody is obliged to act on is not a gate — it is a log line, and in a repository that anyone
commits to it will eventually be one nobody reads.
I am not arguing these findings should be errors. For the ones I care about — the footnote rules
from #20 — the spec treats footnotes only as the
sourcesjoin key and requires neither direction,so warning is the correct upstream default. The judgement is local; the mechanism is not. Today
each project that wants to enforce one has to build the mechanism itself.
What I had to build, and what its shape suggests
I maintain a small wrapper that shells out to
okf validate, parses the JSON, and exits non-zero ifa finding's rule ID is in a promoted set. Three things about it are, I think, requirements rather
than preferences — each was learned by getting it wrong first.
1 — Per-rule promotion, not only a global
--strict.Different contexts need different subsets, and one of mine cannot be expressed by a single switch.
The three footnote rules are rendering faults caused by the diff in front of you, so I promote them
on every commit.
okf/lifecycle/staleis not: it arrives because a date passed, on a repositorynobody has touched. Failing a commit hook on it blocks whoever commits next for something they did
not do — and still misses the case that matters, the untouched repository where no commit ever
comes. So it is promoted only in a weekly scheduled run. A global strict mode makes those two
contexts the same.
2 — An unknown rule ID must be an error, not a no-op.
This is the one that decides whether the feature is safe. If a promoted ID is silently ignored, then
a typo — or a rule renamed upstream — turns the gate into a pass while enforcing nothing, and it
reports success while doing it. My wrapper has to refuse an
okfolder than the release thatintroduced the rules for exactly this reason: on an older binary they are never emitted, so the gate
would pass while checking nothing. If the flag validates its arguments against the known rule set,
no consumer needs that guard.
3 — Additive and replacing forms are not interchangeable.
If a default promoted set ever ships, a caller wanting one more rule will reach for the replacing
form and thereby stop enforcing anything added to the defaults later — silently, and reporting a
pass. I ended up with both spellings and a note explaining which to prefer. Worth deciding
deliberately upstream rather than discovering it downstream.
Possible shapes
No preference between these; the requirement is only that (1) and (2) above survive.
--fail-on <severity>/--min-severity <severity>— simplest, but cannot express theper-context split in (1) on its own.
--promote <rule-id>(repeatable) — expresses everything I need; must reject unknown IDs.needs no new flags, though it is coarser than per-rule.
The raw material is already there: findings carry stable rule IDs, and
okf schemapublishes CLImetadata, so the set of valid IDs is machine-readable.
Relation to #26
#26 asks for broken links to be reclassified as
ERROR. Reclassifying a rule moves the line foreveryone, and each such request has to be argued on its merits. A promotion mechanism lets each
project draw its own line and settles the whole class of request rather than one instance — so I
think this is the more general fix, and would leave #26 as a question about the default rather
than about enforceability.
Why I care
The wrapper is about 400 lines of Python and its README says, in as many words, that it exists to be deleted
the day
okfcan be made to exit non-zero on a warning. I would rather delete it than maintain it,and I would rather other projects did not each write their own.
Happy to send a PR if you would like one, and happy to be told this belongs in a caller instead.