fix(ops): the drift checker was boolean-shaped, and the risk never was - #242
Merged
Conversation
`make config-drift` read only `name: bool = True|False`. `VECTOR_STORE_BACKEND` decides whether the entire knowledge layer reads from Postgres or from a dyno's local disk; it is a `str`; it was set in production against a code default of "chroma"; and the checker printed "No drift". The tool was written after ten *boolean* divergences were found and inherited the shape of its first evidence as a claim about the whole problem. Widened to `str` and `int`, and it immediately found one real thing the boolean-only version could never have seen: DEFAULT_LLM_PROVIDER is `openrouter` in production against `openai` in code, unrecorded — 6 338 of 6 479 logged calls go through OpenRouter. Now in DELIBERATE with its reason, alongside VECTOR_STORE_BACKEND. Two things the widening broke, both fixed here rather than later: * It printed MASTER_ENCRYPTION_KEY, OPENROUTER_API_KEY, the Redis password and RESEND_API_KEY to stdout, which is where CI logs live. `mask()` now hides any value whose NAME looks like a credential — the name, because the value is exactly what must not be examined to make the decision. * More fundamentally, an empty code default is not a default. It is "the environment supplies this", which is how every credential in `config.py` is declared, so comparing a deployed secret against "" asks a question that does not apply to it. Those are skipped now, which is also what stopped most of the printing. And a third category the boolean tool never needed: ENVIRONMENT_SHAPED. A container path differs from a relative dev path by construction and always will; DEFAULT_LLM_PROVIDER changes which vendor answers a question. Location versus behaviour — and only behaviour needs a written reason. Listing both is how a tool teaches people to skim it. `code_defaults()` now returns strings so one comparison path serves all three types; booleans normalise to "true"/"false" so `1`, `on` and `TRUE` still match a `True` default. Two existing tests asserted the old bool-typed contract and are updated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it missed
make config-driftread onlyname: bool = True|False.VECTOR_STORE_BACKENDdecides whether the entire knowledge layer reads from Postgres or from a dyno's local disk. It is astr. It was set in production against a code default ofchroma, and the checker printed "No drift".The tool was written after ten boolean divergences were found on 2026-08-23, and inherited the shape of its first evidence as a claim about the whole problem.
What widening it found
DEFAULT_LLM_PROVIDERisopenrouterin production againstopenaiin code, and was never recorded. 6 338 of 6 479 logged LLM calls go through OpenRouter. The boolean-only version could not have seen it. Now inDELIBERATEwith its reason, alongsideVECTOR_STORE_BACKEND.What widening it broke, and what that taught
The first run printed
MASTER_ENCRYPTION_KEY,OPENROUTER_API_KEY, the Redis password andRESEND_API_KEYto stdout — which is where CI logs live. Two fixes, and the second is the one that matters:mask()hides any value whose name looks like a credential. The check is on the name because the value is exactly what must not be examined to make the decision.config.pyis declared. Comparing a deployed secret against""asks a question that does not apply to it, reports all of them as drift, and prints them. Skipping those is what stopped most of the printing; masking is the belt to that brace.A third category the boolean tool never needed
ENVIRONMENT_SHAPED. A container path differs from a relative dev path by construction and always will.DEFAULT_LLM_PROVIDERchanges which vendor answers a question.Location versus behaviour — and only behaviour needs a written reason. Listing both is how a tool teaches people to skim it, which is the failure this script exists to prevent.
Contract change
code_defaults()returns strings so one comparison path servesbool,strandint; booleans normalise to"true"/"false"so1,onandTRUEstill match aTruedefault. Two existing tests asserted the old bool-typed contract and are updated rather than worked around.Verification
pytest tests/— 7000 passed, 4 skipped, 1 xfailed.make config-drift— exits clean, withVECTOR_STORE_BACKENDandDEFAULT_LLM_PROVIDERrecorded and no secret printed.