Skip to content

fix scorecard verdict when sealed-generation census is invalid #795

Description

@ScriptedAlchemy

Defect

efficiency-scorecard can report a successful run and overall ok verdict when the run's cold-index graph census is explicitly unavailable because the sealed generation census is invalid.

Confirmed artifact:

  • target/efficiency-scorecard-20260902T093325Z-final-certification/scorecard.json
  • source commit: 5749e4fcfe268e17bd19a0e6ef90c646f7b37289
  • run 3: cold_index.graph_statistics = { \"state\": \"unavailable\", \"reason\": \"sealed_generation_census_invalid\" }
  • the same run still has status: \"ok\"
  • the top-level scorecard still has verdict: \"ok\"

Defect location

scripts/efficiency-scorecard.py:1058-1070 derives the overall verdict and exit code only from run[\"status\"] != \"ok\". The cold-index census state never contributes to run status, overall verdict, or process exit status.

Required behavior

A run with cold_index.graph_statistics.state != \"observed\" due to sealed_generation_census_invalid must not report status: \"ok\"; consequently the scorecard verdict must not be ok and the command must exit non-zero. Add a regression fixture covering this exact contradictory shape.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions