The CODECHECK register management bot.
If the research paper has code, it must run.
@chekhovbot assists the repeated tasks around a CODECHECK: welcoming
codecheckers, assigning them, validating codecheck.yml early, and opening the
pull request against register.csv. It is controlled by mentions in the checks
issue in codecheckers/register,
following the convention established by the Open Journals bots: the command
goes on the first line of the comment, one command per comment.
@chekhovbot commands
@chekhovbot assign @user as codechecker
@chekhovbot codecheckers suggest
@chekhovbot check codecheck.yml
@chekhovbot check ready
@chekhovbot check start
@chekhovbot status
@chekhovbot remind me in 2 weeksTracking issue: codecheckers/register#209
Anton Chekhov (1860-1904): a pun on check, and on Chekhov's gun — if the gun
is on the wall in Act I it must fire in Act III; if the paper has code, it must
run. (Star Trek's Chekov has one h.)
The bot answers commands, hello and an unknown command on the testing
register, and validates a codecheck.yml when a comment names the repository
to read it from. Editors can announce a published certificate on Mastodon with
@chekhovbot announce <certificate>, which previews the toot, and
@chekhovbot announce <certificate> confirm, which posts it; in development
the toot is a direct message with every mention defused. @chekhovbot codecheckers says where the codechecker lists are, and an editor can ask
@chekhovbot codecheckers suggest for candidates, named without notifying
them. @chekhovbot certificate says which certificate identifier the check
holds, @chekhovbot certificate next which one comes next, and
@chekhovbot certificate set reserves it for the check. A command names its
subject first and then what to do with it, so the commands about one thing sit
together; the forms written the other way round before 0.22 -
set certificate, suggest codecheckers and the rest - still work.
@chekhovbot check ready says whether a check can start - a README, a
codecheck.yml with a manifest, a licence and a paper - for the author as much
as the editor. @chekhovbot check zenodo audits the certificate's Zenodo
record against the CODECHECK community's curation policy and the
codecheck.yml, and reads a draft through a share link, so that the record can
be fixed before it is published; it deletes the comment that held the link,
which anyone who can read the issue could otherwise use. @chekhovbot check start, @chekhovbot status and
@chekhovbot check finish say where a check stands and move the register's labels with it, running the
checks that already exist rather than repeating them: because those take time,
the reply is posted at once and rewritten in place as each answers. Once the
certificate is in register.csv, check finish also reads its page in the
register and links it when it says what codecheck.yml says.
On the codechecker lists,
codecheckers/codecheckers,
the bot answers a registration as a codechecker: when the issue is opened, and
again on @chekhovbot include check, it says what the row still needs - the
columns, the ORCID, a way to reach the person besides GitHub, and whether they
are listed already. @chekhovbot include, for editors, then opens the pull
request adding the row to codecheckers.csv and checks the organisation's side:
it asks the owners to invite somebody it sees no invitation for, says when one
is waiting to be accepted, and puts a member in the codecheckers team. The
nightly sweep follows it up from there - and opens a registration by itself for a
codechecker who joins after an assign asked the owners for them, as
@chekhovbot include open @handle does for an editor - and once they are in the team and
their row is merged, posts the welcome and closes the registration. It
answers nothing else there.
@chekhovbot check register builds the row the check would get in
register.csv - from the issue's Repository line and labels, or what an editor
set with repository set and venue set - and runs the rules on it before
anything is written; @chekhovbot register then opens the pull request adding
it, for an editor to merge, and @chekhovbot check certificate <certificate>
runs the rules on a row already there. It runs
against
codecheckers/testing-dev-register
only; the production register is not configured anywhere. See
docs/deployment.md.
go run ./cmd/chekhov serve # the bot: /dispatch, /healthz, /nudges
echo '@chekhovbot commands' | go run ./cmd/chekhov comment - # the same answer, offline
echo '@chekhovbot announce 1970-001' | go run ./cmd/chekhov comment --as nuest --online - # an editor's previewgo build ./...
go run ./cmd/chekhov check path/to/codecheck.yml # the report, non-zero on failure
go run ./cmd/chekhov check --online path/to/codecheck.yml # also asks OpenAlex, ORCID, Zenodo
go run ./cmd/chekhov check github::codecheckers/Piccolo-2020 # read it from the repository
go run ./cmd/chekhov check codecheckers/Piccolo-2020 # the same, platform inferred
go run ./cmd/chekhov check 2020-001 # the certificate, looked up in the register
go run ./cmd/chekhov check bundle github::codecheckers/demo # one part of the catalogue
go run ./cmd/chekhov check repository codecheckers/demo # what the repository is, not a verdict
go run ./cmd/chekhov check links codecheckers/Piccolo-2020 # which links no longer answer, advice only
go run ./cmd/chekhov check --markdown path/to/codecheck.yml # the reply the bot posts
go run ./cmd/chekhov rules # the rules, and which are checked
go run ./cmd/chekhov rules --check # are they the register's current ones?The bot reports one version, internal/build.Version, and it is bumped by
hand in the commit that earns it - beside the CHANGELOG.md entry, never saved
up for a release. /healthz, @chekhovbot version and the development footer
all read that constant, so a deployment and a local build answer the same way.
It is semantic versioning applied to what somebody talking to the bot can see. A new command or a new thing a reply reports is a minor bump; a fix, a wording change or a rule refresh is a patch; a command removed or renamed, or a reply whose meaning changes under somebody already relying on it, is major.
Two things hold it together. A test compares the constant with the newest
heading in CHANGELOG.md, checks the headings are newest-first and dated, and
fails when anything is parked under Unreleased. And because that test reads
one tree rather than a diff - so a feature written under the existing heading
would pass - CI additionally fails a pull request that changes non-test Go
without moving the version, unless it carries the no-version-bump label.
The commit the build was made from is a separate question, answered only by the
stamp go build makes from the checkout since Go 1.24 - the one source nothing
can hand over wrongly. A development deployment, and a locally built binary
previewing a reply, name it and say when the tree had uncommitted changes; a
go run names nothing rather than guessing. The version itself is not taken
from git, because the deployment's builder may have no .git, and a version
that is sometimes semantic and sometimes a pseudo-version is worse than one
that is always the same shape.
The rules are maintained in the register, not here, one file per version of the
configuration file specification:
https://github.com/codecheckers/register/blob/master/RULES.md. Chekhov embeds
a copy, refreshed with scripts/update-rules.sh.
The same identifiers are used by the
codecheck R package, so the same
rule is CC-CFG-016 in both implementations, and the two can be compared by
grepping for an identifier. Which rules apply to a file follows from the
specification version it declares, and a rule's severity comes from the rule
file rather than from either codebase.
Everything is exercised against the testing register, never the real one:
https://github.com/codecheckers/testing-dev-register. See
docs/development.md for the full setup.
go test ./... # fast, offline
CHEKHOV_INTEGRATION=1 go test -run Integration ./internal/check/ # external servicesThe fast suite needs no network and covers every rule, including the ones that ask an external service: recorded cassettes replay what OpenAlex, ORCID, Zenodo, GitHub and the register really answered, and stubbed servers cover the answers a published CODECHECK never gives. CI runs it on every push and pull request.
The integration suite asks the live services the same questions, to catch the day one of them changes its answer. It reads only, and CI runs it weekly.
Code MIT (see LICENSE). Graphics in logo/ are CC BY 4.0, matching the rest
of the CODECHECK branding. The Chekhov quotes in
internal/command/data/quotes.yml come from
Wikiquote and are CC BY-SA 4.0;
every reply that uses one links back to the page.