Summary
No single cached/checkpointed artifact in the US release builder records the solver/calibration configuration that produced a given calibrated result. During the Build F campaign (populace#299) this did not cause a wrong result, but it nearly misled a diagnosis. Filing so the identity gap is closed before it bites a run where config genuinely changes the cached values.
What's already correct (so this issue is narrow)
Two things I initially suspected turned out to be fine, and I want to record that so nobody "fixes" them:
target_registry_version is already a content hash. TargetRegistry.version hashes the canonical spec JSON (every spec's name/value/source/sign/period). A feed edit that changes the target registry (e.g. v5→v6→v7 removing M-CHIP CHIP rows) therefore does invalidate the target-frame checkpoint. No feed-collision risk there.
- The target-frame checkpoint is legitimately solver-independent.
_target_frame_checkpoint_identity (keys: base_dataset_sha256, policyengine_us_version, seed, target_period, target_registry_version, congressional_district_vintage_crosswalk_sha256) caches pre-calibration PE-US engine outputs over the base pool. Those don't depend on λ / max_weight_ratio / epochs, so omitting solver config from this checkpoint's key is correct.
The real gap
The calibrated result (weights, loss, diagnostics, exported H5) depends on the full solver config — l2_lambda, l0_refit_lambda_share, refit_l2_lambda, max_weight_ratio, epochs, learning_rate, l2_anchor/refit_l2_anchor, warm-start identity, mass policy — but no artifact's identity records that config as a hash. The release manifest records the values in prose (default_dataset, options), but there is no solver_config_sha256 you can compare across runs to answer "did the calibration config change?" from identity alone.
Why it matters (Build F evidence, framed accurately)
The campaign ran back-to-back invocations differing only in --l2-lambda (0.08 vs default 0.0) and --max-weight-ratio (5 vs 50). An l2_lambda=0.08 run that produced +258% income tax immediately preceded a default-λ dense run. When the default-λ run then failed a downstream gate, checkpoint bleed was a natural first hypothesis.
It was NOT the cause — attempt 5 was run in a fresh --checkpoint-root (clean room), and its dense result was byte-identical on the flagged masses to the prior attempt despite the clean room, proving the behavior was deterministic pipeline logic, not stale cache. But had the clean-room control not been run, the identical numbers could have been misattributed to a stale checkpoint. A solver_config_sha256 on the calibrated artifacts (and any calibration-stage cache added later) would make that question answerable from identity, without a control run.
Proposed fix
- Add
solver_config_sha256 (canonical-JSON hash of the full calibration option set) to the identity/manifest of the calibrated artifacts and any future calibration-stage checkpoint whose values depend on solver config.
- Add a test pinning the invariant that the target-frame checkpoint is solver- and (given
target_registry_version) feed-independent, so that independence is a stated contract rather than an accident.
Context
Found during the Build F certified-lineage rerun (populace#299). Not a blocker for that campaign's conclusions.
Summary
No single cached/checkpointed artifact in the US release builder records the solver/calibration configuration that produced a given calibrated result. During the Build F campaign (populace#299) this did not cause a wrong result, but it nearly misled a diagnosis. Filing so the identity gap is closed before it bites a run where config genuinely changes the cached values.
What's already correct (so this issue is narrow)
Two things I initially suspected turned out to be fine, and I want to record that so nobody "fixes" them:
target_registry_versionis already a content hash.TargetRegistry.versionhashes the canonical spec JSON (every spec's name/value/source/sign/period). A feed edit that changes the target registry (e.g. v5→v6→v7 removing M-CHIP CHIP rows) therefore does invalidate the target-frame checkpoint. No feed-collision risk there._target_frame_checkpoint_identity(keys:base_dataset_sha256,policyengine_us_version,seed,target_period,target_registry_version,congressional_district_vintage_crosswalk_sha256) caches pre-calibration PE-US engine outputs over the base pool. Those don't depend on λ / max_weight_ratio / epochs, so omitting solver config from this checkpoint's key is correct.The real gap
The calibrated result (weights, loss, diagnostics, exported H5) depends on the full solver config —
l2_lambda,l0_refit_lambda_share,refit_l2_lambda,max_weight_ratio,epochs,learning_rate,l2_anchor/refit_l2_anchor, warm-start identity,masspolicy — but no artifact's identity records that config as a hash. The release manifest records the values in prose (default_dataset, options), but there is nosolver_config_sha256you can compare across runs to answer "did the calibration config change?" from identity alone.Why it matters (Build F evidence, framed accurately)
The campaign ran back-to-back invocations differing only in
--l2-lambda(0.08 vs default 0.0) and--max-weight-ratio(5 vs 50). Anl2_lambda=0.08run that produced +258% income tax immediately preceded a default-λ dense run. When the default-λ run then failed a downstream gate, checkpoint bleed was a natural first hypothesis.It was NOT the cause — attempt 5 was run in a fresh
--checkpoint-root(clean room), and its dense result was byte-identical on the flagged masses to the prior attempt despite the clean room, proving the behavior was deterministic pipeline logic, not stale cache. But had the clean-room control not been run, the identical numbers could have been misattributed to a stale checkpoint. Asolver_config_sha256on the calibrated artifacts (and any calibration-stage cache added later) would make that question answerable from identity, without a control run.Proposed fix
solver_config_sha256(canonical-JSON hash of the full calibration option set) to the identity/manifest of the calibrated artifacts and any future calibration-stage checkpoint whose values depend on solver config.target_registry_version) feed-independent, so that independence is a stated contract rather than an accident.Context
Found during the Build F certified-lineage rerun (populace#299). Not a blocker for that campaign's conclusions.