[SPECIAL BUILD — DO NOT MERGE] checkpointing: §4.7.1 gap breach warns instead of invalidating#834
Draft
FileSystemGuy wants to merge 2 commits into
Draft
[SPECIAL BUILD — DO NOT MERGE] checkpointing: §4.7.1 gap breach warns instead of invalidating#834FileSystemGuy wants to merge 2 commits into
FileSystemGuy wants to merge 2 commits into
Conversation
Special build for one submitter: a >30s write→read inter-phase gap in a split-mode checkpointing submission must be reported as a warning rather than a hard error, in both the submission checker (cache_flush_validation) and the reportgen-side rules checker (check_invocation_structure).
…of invalidating Downgrade the >30s write→read inter-phase gap breach from a hard error to a warning in both enforcement points: - submission checker cache_flush_validation: warn_violation instead of log_violation; the check no longer returns False for a gap breach - reportgen-side check_invocation_structure: warning-severity CLOSED issue instead of PARAM_VALIDATION.INVALID Missing/unparseable timestamps and overlapping phases remain hard errors. One-off build for a single submitter; not intended for merge.
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
Contributor
|
This does not work. Open still check the timestamp gap |
FileSystemGuy
added a commit
that referenced
this pull request
Jul 25, 2026
Downgrade the >30s write→read inter-phase gap breach from a hard error to a warning in both enforcement points: - submission checker cache_flush_validation: warn_violation instead of log_violation; the check no longer returns False for a gap breach - reportgen-side check_invocation_structure: warning-severity CLOSED issue instead of PARAM_VALIDATION.INVALID Missing/unparseable timestamps and overlapping phases remain hard errors. Originally authored on the checkpoint-split-gap-warning special build (PR #834) for one submitter; Curtis decided 2026-07-24 the relaxation applies to the whole v3.0 review tree, so it lands on the integration branch for all submitters (worklist A7). Cherry-picked from c45278c.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge
One-off special build for a single submitter. This PR exists so the change is reviewable and the branch is buildable; it is not intended to land on main.
What it changes
Downgrades the >30 s write→read inter-phase gap breach in split-mode checkpointing submissions (Rules.md §4.7.1) from a hard error to a warning, in both enforcement points:
mlpstorage_py/submission_checker/checks/checkpointing_checks.py,cache_flush_validation): emitswarn_violationinstead oflog_violationand no longer returnsFalseon a gap breach.mlpstorage_py/rules/submission_checkers/checkpointing.py,check_invocation_structure): emits a warning-severityCLOSEDissue instead ofPARAM_VALIDATION.INVALID, so the verifier no longer marks the submission INVALID.Unchanged (still hard errors): missing/unparseable timestamps, overlapping phases (negative gap remains the clock-skew warning it already was), wrong invocation counts/shapes.
Testing
TDD: first commit updates/adds the tests RED (gap breach expected to warn), second commit turns them GREEN. All four CI suites pass locally:
tests(2939),mlpstorage_py/tests(890),kv_cache_benchmark/tests(238),vdb_benchmark/tests(228).