Problem
When a licensed Unity leg aborts at its first step, it produces two failures: the honest one, and a second from Require confirmed Unity cleanup reporting a cleanup that was never needed.
Evidence
Run 30642224168, job Unity 6000.3.16f1 playmode, on PR #326. The leg was scheduled for a superseded head, so Require current PR head before setup failed first:
failure Require current PR head before setup
skipped ... every step through Require acquired Unity lock ...
skipped Return Unity license
skipped Classify Unity cleanup evidence
success Release organization Unity lock
failure Require confirmed Unity cleanup
Five of the nine legs in that run show the identical shape.
Why the second failure happens
Return Unity license and Classify Unity cleanup evidence both carry
if: ${{ always() && steps.acquire_lock.outputs.acquired == 'true' }}, so both correctly skip when the lock was never acquired. Require confirmed Unity cleanup carries a bare if: always() and reads classification-complete from the step that just skipped, so it sees an empty value and fails.
In this shape there is nothing to clean: Acquire organization Unity lock never ran, so no seat was consumed and no lock was held. The gate is reporting the absence of a classification for work that never started.
Why it is not simply "add the same condition"
The bare always() is load-bearing. If acquire_lock itself fails part-way through, acquired can read empty while the lock is genuinely held, and that is exactly the leak the gate exists to catch. Gating the gate on acquired == 'true' would make the one case it must never miss the case it skips.
So the fix has to distinguish "acquisition never started" from "acquisition failed after taking the lock", rather than collapse both into a skip. Candidates:
- Have the gate accept an explicit
not-attempted classification, emitted when acquire_lock is skipped (as opposed to failure or empty output). steps.acquire_lock.outcome distinguishes the two, and the aggregate can require outcome == 'skipped' before accepting the empty classification.
- Have
Classify Unity cleanup evidence run on always() and emit not-attempted itself, so the gate always receives a value and keeps its bare always().
Option 2 keeps the gate untouched, which is the safer of the two.
Impact
Cosmetic today: the second failure only ever appears on a leg that is already red, so no verdict changes. It is worth fixing because a spurious cleanup failure is precisely the signal that must stay trustworthy -- the whole four-layer license-return guarantee depends on a reader believing the cleanup gate when it goes red.
After #311 lands (in #326) superseded runs will not schedule these legs at all, which removes the common source. The residual case is a push that lands after a leg has started, where the per-leg head guard fires mid-flight.
Acceptance criteria
- A leg that aborts before
Acquire organization Unity lock runs reports exactly one failure.
- A leg where
acquire_lock fails after taking the lock still fails Require confirmed Unity cleanup.
Require confirmed Unity cleanup keeps a bare if: always(), or the change that removes it is covered by a case in scripts/validate-unity-pr-policy.py's executed truth table.
- Applies to all five licensed lock windows, not just
unity-tests.
Related
Problem
When a licensed Unity leg aborts at its first step, it produces two failures: the honest one, and a second from
Require confirmed Unity cleanupreporting a cleanup that was never needed.Evidence
Run 30642224168, job
Unity 6000.3.16f1 playmode, on PR #326. The leg was scheduled for a superseded head, soRequire current PR head before setupfailed first:Five of the nine legs in that run show the identical shape.
Why the second failure happens
Return Unity licenseandClassify Unity cleanup evidenceboth carryif: ${{ always() && steps.acquire_lock.outputs.acquired == 'true' }}, so both correctly skip when the lock was never acquired.Require confirmed Unity cleanupcarries a bareif: always()and readsclassification-completefrom the step that just skipped, so it sees an empty value and fails.In this shape there is nothing to clean:
Acquire organization Unity locknever ran, so no seat was consumed and no lock was held. The gate is reporting the absence of a classification for work that never started.Why it is not simply "add the same condition"
The bare
always()is load-bearing. Ifacquire_lockitself fails part-way through,acquiredcan read empty while the lock is genuinely held, and that is exactly the leak the gate exists to catch. Gating the gate onacquired == 'true'would make the one case it must never miss the case it skips.So the fix has to distinguish "acquisition never started" from "acquisition failed after taking the lock", rather than collapse both into a skip. Candidates:
not-attemptedclassification, emitted whenacquire_lockisskipped(as opposed tofailureor empty output).steps.acquire_lock.outcomedistinguishes the two, and the aggregate can requireoutcome == 'skipped'before accepting the empty classification.Classify Unity cleanup evidencerun onalways()and emitnot-attempteditself, so the gate always receives a value and keeps its barealways().Option 2 keeps the gate untouched, which is the safer of the two.
Impact
Cosmetic today: the second failure only ever appears on a leg that is already red, so no verdict changes. It is worth fixing because a spurious cleanup failure is precisely the signal that must stay trustworthy -- the whole four-layer license-return guarantee depends on a reader believing the cleanup gate when it goes red.
After #311 lands (in #326) superseded runs will not schedule these legs at all, which removes the common source. The residual case is a push that lands after a leg has started, where the per-leg head guard fires mid-flight.
Acceptance criteria
Acquire organization Unity lockruns reports exactly one failure.acquire_lockfails after taking the lock still failsRequire confirmed Unity cleanup.Require confirmed Unity cleanupkeeps a bareif: always(), or the change that removes it is covered by a case inscripts/validate-unity-pr-policy.py's executed truth table.unity-tests.Related