Skip to content

fix(ops): the drift checker was boolean-shaped, and the risk never was - #242

Merged
sshlg merged 1 commit into
mainfrom
fix/config-drift-beyond-booleans
Aug 28, 2026
Merged

fix(ops): the drift checker was boolean-shaped, and the risk never was#242
sshlg merged 1 commit into
mainfrom
fix/config-drift-beyond-booleans

Conversation

@sshlg

@sshlg sshlg commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

What it missed

make config-drift read only name: bool = True|False.

VECTOR_STORE_BACKEND decides whether the entire knowledge layer reads from Postgres or from a dyno's local disk. It is a str. It was set in production against a code default of chroma, and the checker printed "No drift".

The tool was written after ten boolean divergences were found on 2026-08-23, and inherited the shape of its first evidence as a claim about the whole problem.

What widening it found

DEFAULT_LLM_PROVIDER is openrouter in production against openai in code, and was never recorded. 6 338 of 6 479 logged LLM calls go through OpenRouter. The boolean-only version could not have seen it. Now in DELIBERATE with its reason, alongside VECTOR_STORE_BACKEND.

What widening it broke, and what that taught

The first run printed MASTER_ENCRYPTION_KEY, OPENROUTER_API_KEY, the Redis password and RESEND_API_KEY to stdout — which is where CI logs live. Two fixes, and the second is the one that matters:

  • mask() hides any value whose name looks like a credential. The check is on the name because the value is exactly what must not be examined to make the decision.
  • An empty code default is not a default. It is "the environment supplies this" — how every credential in config.py is declared. Comparing a deployed secret against "" asks a question that does not apply to it, reports all of them as drift, and prints them. Skipping those is what stopped most of the printing; masking is the belt to that brace.

A third category the boolean tool never needed

ENVIRONMENT_SHAPED. A container path differs from a relative dev path by construction and always will. DEFAULT_LLM_PROVIDER changes which vendor answers a question.

Location versus behaviour — and only behaviour needs a written reason. Listing both is how a tool teaches people to skim it, which is the failure this script exists to prevent.

Contract change

code_defaults() returns strings so one comparison path serves bool, str and int; booleans normalise to "true"/"false" so 1, on and TRUE still match a True default. Two existing tests asserted the old bool-typed contract and are updated rather than worked around.

Verification

  • pytest tests/7000 passed, 4 skipped, 1 xfailed.
  • make config-drift — exits clean, with VECTOR_STORE_BACKEND and DEFAULT_LLM_PROVIDER recorded and no secret printed.
  • New tests: string/int parsing, the empty-default skip, the location/behaviour split, and masking over seven credential-shaped names — plus one asserting an ordinary setting is still shown, because masking everything would make the report unreadable.

`make config-drift` read only `name: bool = True|False`. `VECTOR_STORE_BACKEND`
decides whether the entire knowledge layer reads from Postgres or from a dyno's local
disk; it is a `str`; it was set in production against a code default of "chroma"; and
the checker printed "No drift". The tool was written after ten *boolean* divergences
were found and inherited the shape of its first evidence as a claim about the whole
problem.

Widened to `str` and `int`, and it immediately found one real thing the boolean-only
version could never have seen: DEFAULT_LLM_PROVIDER is `openrouter` in production
against `openai` in code, unrecorded — 6 338 of 6 479 logged calls go through
OpenRouter. Now in DELIBERATE with its reason, alongside VECTOR_STORE_BACKEND.

Two things the widening broke, both fixed here rather than later:

* It printed MASTER_ENCRYPTION_KEY, OPENROUTER_API_KEY, the Redis password and
  RESEND_API_KEY to stdout, which is where CI logs live. `mask()` now hides any value
  whose NAME looks like a credential — the name, because the value is exactly what
  must not be examined to make the decision.
* More fundamentally, an empty code default is not a default. It is "the environment
  supplies this", which is how every credential in `config.py` is declared, so
  comparing a deployed secret against "" asks a question that does not apply to it.
  Those are skipped now, which is also what stopped most of the printing.

And a third category the boolean tool never needed: ENVIRONMENT_SHAPED. A container
path differs from a relative dev path by construction and always will;
DEFAULT_LLM_PROVIDER changes which vendor answers a question. Location versus
behaviour — and only behaviour needs a written reason. Listing both is how a tool
teaches people to skim it.

`code_defaults()` now returns strings so one comparison path serves all three types;
booleans normalise to "true"/"false" so `1`, `on` and `TRUE` still match a `True`
default. Two existing tests asserted the old bool-typed contract and are updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sshlg
sshlg merged commit 4002c34 into main Aug 28, 2026
2 checks passed
@sshlg
sshlg deleted the fix/config-drift-beyond-booleans branch August 28, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant