fix(e2e): stabilize split-process census acquisition - #8644
Conversation
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe split-process security report advances to version 2. The probe records stabilized final supervisors and observed supervisors, detects process identity changes, bounds retained diagnostics, and validates census consistency. Tests use controlled ChangesSplit-process security posture
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant ControlledProcHarness
participant EmbeddedPythonProbe
participant ReportValidator
ControlledProcHarness->>EmbeddedPythonProbe: provide controlled process censuses
EmbeddedPythonProbe->>ControlledProcHarness: inspect and recheck process identities
EmbeddedPythonProbe->>ReportValidator: submit final and observed supervisors
ReportValidator-->>EmbeddedPythonProbe: validate version-2 census data
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (4)
test/e2e/fixtures/security-posture.ts (2)
311-319: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueClarify the
first/seconddiagnostic labels.
first_processesis reassigned at Line 318 on each attempt. When the loop fails at Line 311,firstandsecondare the last two censuses, not the initial census and the last one. The labels suggest otherwise to a responder who reads the failure message.Rename the labels to reflect the compared pair, and include the attempt number.
♻️ Proposed label change
raise RuntimeError( "nemoclaw-start child supervisor census did not stabilize " f"after {MAX_CENSUS_STABILITY_ATTEMPTS} attempts: " - f"first={json.dumps(diagnostic_census(first_processes), sort_keys=True)} " - f"second={json.dumps(diagnostic_census(second_processes), sort_keys=True)}" + f"attempt={attempt} " + f"previous={json.dumps(diagnostic_census(first_processes), sort_keys=True)} " + f"latest={json.dumps(diagnostic_census(second_processes), sort_keys=True)}" )Note: the test at
test/e2e/support/security-posture.test.tsLines 470-472 matchesfirst=andsecond=. Update that assertion together with this change.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/fixtures/security-posture.ts` around lines 311 - 319, Update the stabilization failure diagnostics in the census loop to label the compared censuses accurately as the last pair and include the current attempt number. Adjust the corresponding assertion in the security posture test to match the new diagnostic labels instead of first= and second=.
671-680: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse a
Set<number>for the observed PID check.
observedByPidstores each process, but the value is never read. Onlyhasis used. ASet<number>states the intent and drops the unused value.♻️ Proposed simplification
- const observedByPid = new Map<number, ProcessSecurityIdentity>(); + const observedPids = new Set<number>(); const observedIdentityKeys = new Set<string>(); for (const process of observedChildSupervisors) { validateNemoclawStartProcess(process, sandboxUid, sandboxGid); - if (observedByPid.has(process.pid)) { + if (observedPids.has(process.pid)) { throw new Error(`observed nemoclaw-start process PID ${process.pid} appeared more than once`); } - observedByPid.set(process.pid, process); + observedPids.add(process.pid); observedIdentityKeys.add(stableProcessIdentityKey(process)); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/fixtures/security-posture.ts` around lines 671 - 680, Replace observedByPid in the observedChildSupervisors loop with a Set<number>, preserving the duplicate-PID check via has(process.pid) and recording each PID with add(process.pid). Leave observedIdentityKeys and the surrounding validation unchanged.test/e2e/support/security-posture.test.ts (2)
90-95: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the patch-before-exec ordering dependency.
The harness works because the probe runs
from pathlib import Pathinsideexec, after Line 90 replacespathlib.Path. If the probe ever moved that import or capturedPathearlier, the redirection would silently stop applying and the tests would exercise the real/proc.Add a short comment that records this contract.
♻️ Proposed comment
+# The probe imports pathlib, os, pwd and grp inside exec below, so these +# module attributes must be replaced before exec runs. pathlib.Path = controlled_path os.scandir = controlled_scandir os.stat = controlled_stat🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/support/security-posture.test.ts` around lines 90 - 95, Document the ordering contract immediately before the probe’s exec flow: the pathlib.Path, os.scandir, os.stat, pwd.getpwnam, and grp.getgrnam replacements must be installed before the probe imports or captures those symbols. Keep the comment short and explicitly note that moving the probe’s imports earlier would bypass the controlled fakes and access real /proc.
468-474: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winScope the redaction assertion to the error message.
Line 473 matches
/argv|executable|status/uagainst the wholestderr, which includes the Python traceback. The assertion passes today only because no traceback source line or symbol name in the raising frame contains those substrings. A rename in the probe, or a change in traceback rendering between Python versions, can fail this test without any redaction regression.Extract the
RuntimeErrormessage line and assert on it. Keep the whole-stderrscan for concrete secret values, as the test at Lines 420-432 does.♻️ Proposed narrowing
expect(result.stderr).toMatch( /did not stabilize after 4 attempts: first=.*"pid": 43.*second=.*"pid": 44/su, ); - expect(result.stderr).not.toMatch(/argv|executable|status/u); + const message = result.stderr + .split("\n") + .find((line) => line.includes("did not stabilize after 4 attempts")); + expect(message).toBeDefined(); + expect(message).not.toMatch(/argv|executable|status/u);As per path instructions, tests must "prefer observable outcomes through the public boundary over source-text ... assertions"; matching the traceback text couples this test to probe source text.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/support/security-posture.test.ts` around lines 468 - 474, Update the assertions in the embedded probe test around result.stderr to extract the RuntimeError message line and apply the redaction check for argv, executable, and status only to that message. Preserve the existing whole-stderr assertion for concrete secret values, and keep the stabilization-message and status assertions unchanged.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/e2e/fixtures/security-posture.ts`:
- Around line 311-319: Update the stabilization failure diagnostics in the
census loop to label the compared censuses accurately as the last pair and
include the current attempt number. Adjust the corresponding assertion in the
security posture test to match the new diagnostic labels instead of first= and
second=.
- Around line 671-680: Replace observedByPid in the observedChildSupervisors
loop with a Set<number>, preserving the duplicate-PID check via has(process.pid)
and recording each PID with add(process.pid). Leave observedIdentityKeys and the
surrounding validation unchanged.
In `@test/e2e/support/security-posture.test.ts`:
- Around line 90-95: Document the ordering contract immediately before the
probe’s exec flow: the pathlib.Path, os.scandir, os.stat, pwd.getpwnam, and
grp.getgrnam replacements must be installed before the probe imports or captures
those symbols. Keep the comment short and explicitly note that moving the
probe’s imports earlier would bypass the controlled fakes and access real /proc.
- Around line 468-474: Update the assertions in the embedded probe test around
result.stderr to extract the RuntimeError message line and apply the redaction
check for argv, executable, and status only to that message. Preserve the
existing whole-stderr assertion for concrete secret values, and keep the
stabilization-message and status assertions unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 90626c40-5e7a-4d22-8898-11cdc0dd793c
📒 Files selected for processing (2)
test/e2e/fixtures/security-posture.tstest/e2e/support/security-posture.test.ts
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
2 terminology differences from the second opinionAdvisory only. These are normalized differences from the primary terminology receipt.
1 additional E2E selection from the second opinionAdvisory only. The primary lane did not select these E2E jobs or targets.
Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests. 3 semantic terminology decisionsTerminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.
E2E guidanceAdvisory only. A maintainer can dispatch the default E2E suite against this exact revision. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
wscurran
added
chore
3 participants
Summary
Fix a latent security-posture probe defect that compared two
/proccensuses in enumeration order and could reject an otherwise stable process topology. The probe now obtains a bounded consecutive-stable census while preserving fail-closed checks for every process observed during acquisition.Changes
NoNewPrivs./procfixtures covering ordering, process churn, stale history, privilege retention, same-command PID reuse, selection/capture mutation, and bounded failures.The separate final and observed arrays are required because live topology and historical privilege evidence are different security claims; one combined array can let a stale process satisfy the final topology check.
test/e2e/support/security-posture.test.tsprotects both report consumers and the acquisition behavior.Type of Change
Quality Gates
91b721f7eed2a93e9af203a6730790cadb6129b2; all found no blocking findings.Documentation Writer Review
no-docs-needed91b721f7echanges only the internal live-E2E split-process security probe, report, and semantic support tests. The probe binds a selected/procentry's PID/start time andnemoclaw-startcommand to the same stable sample. Report v2 keeps final live child-supervisor topology separate from the bounded observednemoclaw-startset used for privilege validation. No public command, configuration, user workflow, default, API, product error, documentation surface, or runtime behavior changed.DGX Station Hardware Evidence
scripts/prepare-dgx-station-host.shis unchanged.Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailablenpx vitest run --project e2e-support test/e2e/support/security-posture.test.tspassed 72/72; repository, test-size, source-shape-zero, test-conditionals, title, project, import, formatting, typecheck, and semantic E2E phase checks passed.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Signed-off-by: Apurv Kumaria akumaria@nvidia.com