fix(vm): tolerate processes vanishing mid /proc scan#129
Merged
Conversation
A process exiting between the /proc readdir and its cmdline read made ScanProcsByBinary fail the whole scan and vm rm refuse the delete: the scan skipped ENOENT (gone before open) but not ESRCH (reaped between open and read). Gate the fail-closed path on IsProcessAlive instead of errno matching: a gone process cannot hold VM resources, whatever errno its disappearance surfaced. Fixes #128
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Under a 16-way concurrent clone+release burst,
vm rmintermittently failed with:ScanProcsByBinarywalks /proc reading every numeric entry's cmdline before the binary-name filter, so any short-lived process on the host can race the scan. The scan already skipped ENOENT (process gone before open), but a process reaped between open and read surfaces ESRCH from read(2) (fs/proc/base.c returns -ESRCH once the task is gone), which fell through to the fail-closed path and refused the whole batch delete. The same scan backsWithRunningVM's fallback and GC'sensureOrphanVMMDead, so those paths could spuriously fail the same way.Fix
Gate the fail-closed latch on
IsProcessAlive(pid)instead of errno matching: a vanished process cannot be holding the VM's resources, whatever errno its disappearance surfaced; read errors on live processes (EACCES/EIO) still fail closed. Thefmt.Errorfwrap is dropped —os.ReadFile's*PathErroralready carriesread /proc/<pid>/cmdline. Hot-path cost: zero — the aliveness probe runs only on the error branch.ScanProcsByBinarynow delegates to an injectable seam so the classification is pinned by deterministic tests: read error + live pid fails closed, read error + vanished pid is skipped, and skipped entries don't drop valid matches.Verification
fs.ErrNotExist), a fresh open returns ENOENT, and kill(pid, 0) returns ESRCH — the aliveness gate covers both raced errnos.go test ./utils/green on Linux (container) and darwin; newTestScanProcsByBinaryReadErrorspasses on Linux.make lint0 issues on both GOOS=linux and GOOS=darwin.Also bumps the install doc download link to v0.5.1 (release to be published separately).
Fixes #128