Harden hypervisor process liveness checks

[yummybomb]

Summary

Layer 1 of the vendor VFIO vGPU stack (generalize-vgpu-device ← this ← vendor-vfio-backend ← vendor-vfio-vgpu). Pure hypervisor-process hardening with no vGPU-specific code; reviewable in isolation.

The upper layers guard vGPU release decisions on "is this instance's hypervisor still alive", so the liveness answer has to be trustworthy first:

• Unify liveness checks on ProcessExists — one exported, EPERM-aware, zombie-filtering definition instead of scattered bare kill(pid, 0) probes. EPERM means the process exists but cannot be signaled; treating it as dead would be wrong.
• Wait for non-child hypervisor exit before finishing kill — after a hypeman restart the hypervisor is not our child, so Wait4 returns ECHILD immediately and the kill loop finished before the process had exited. Poll for actual exit in that case.
• Verify process identity before teardown — a bare PID probe treats any process that reused a stored hypervisor PID as the owning VMM. On Linux, require confirmed socket ownership or a matching PID/start-time token scoped to the current host boot, including the graceful stop/delete wait path.
• Treat provable death as death — two disproofs unwedge stop/delete instead of failing closed forever: an identity token recorded under a different host boot cannot be the recorded VMM, and a socket with no owning listener proves the recorded VMM is gone even when a recycled PID is live. The recycled PID is never signaled.
• Delete what the hardened resolver made redundant — the startup identity backfill (correctness never depended on persisted tokens: token-less metadata confirms through the expected-owner fast path, and an empty listener scan is provable death) and stop's unconditional force-kill after a shutdown that already confirmed exit.
• Drop the command-line fallback from socket resolution — a process matching the socket path by argv only could never authorize teardown, so its sole effect was turning provable death into a wedge: a debug client (ch-remote, socat) holding the path in its argv blocked stop/delete with a 500 until it exited. The fd scan runs with root/CAP_SYS_PTRACE (documented on ResolveProcessPID), so a missing listener is proof of death on its own. If hypeman is ever de-privileged below that, this needs revisiting.

Failure contract

When the hypervisor can be confirmed dead or killed, delete completes synchronously (worst case ~2s graceful + SIGKILL waits). When ownership is genuinely unprovable — the /proc socket scan itself fails, or a process survives SIGKILL — stop/delete return an error (API 500) with metadata retained and the restart policy blocked, so a retried delete is safe and converges once the ambiguity clears. An earlier revision handled this with a background delete finalizer; it was removed in favor of the synchronous error after review — the ambiguous cases are rare, self-resolving, and better surfaced loudly than retried silently.

Testing

• go build ./..., go vet clean
• Identity/kill/shutdown suites additionally cross-compiled and run as root on a dev host against real /proc: all pass. Validated against the host's production firecracker binary: a live VMM resolves with no hint, the correct owner hint, and a wrong hint; a SIGKILLed VMM whose socket file remains on disk classifies as provably dead; a dead socket path held in a live process's argv resolves as provably dead instead of wedging
• go test -race ./lib/instances/ targeted suites pass (TestCreateInstanceWithNetwork requires image pulls + iptables and fails in this environment on the unmodified base as well; TestStandbyAndRestore/TestForkCloudHypervisorFromRunningNetwork boot real VMs and cannot run in this sandbox — CI covers them)

---

Note

High Risk
Changes authentication-adjacent lifecycle behavior: wrong ownership resolution could SIGKILL the wrong process or leave a live VMM while releasing network/GPU; ambiguous cases now error instead of best-effort cleanup.

Overview
Hardens how hypeman decides which process owns an instance’s hypervisor control socket before stop, standby, or delete tear down network, vGPU, and disks.

On Linux, socket resolution now maps only listening Unix sockets (via __SO_ACCEPTCON), scans /proc fds (with ResolveProcessPIDForOwner for expected owners), drops cmdline matching so debug clients can’t block teardown, and surfaces ErrNoOwningProcess when the listener is gone. Instance metadata gains HypervisorProcessIdentity (PID + start time + boot ID, flat JSON) with Set / SetUnconfirmed / Clear; resolveLiveHypervisorPID trusts matching tokens or confirmed socket ownership and treats provable death (wrong boot, no listener) as gone without signaling a recycled PID.

Stop/delete/standby wait on the resolved owner, use shared killProcessAndWait / exported ProcessExists (EPERM-aware, zombie-aware), and abort resource teardown when kill or ownership confirmation fails (delete retains metadata; standby resumes the VM on shutdown failure). Boot/restore paths record identity through resolveRuntimeHypervisorPID.

^{Reviewed by Cursor Bugbot for commit 6cb7738. Bugbot is set up for automated code reviews on this repo. Configure here.}