Skip to content

fix(e2e): prevent false failure signals - #7217

Merged
ericksoa merged 5 commits into
mainfrom
fix/e2e-false-negative-contracts/ae
Jul 20, 2026
Merged

fix(e2e): prevent false failure signals#7217
ericksoa merged 5 commits into
mainfrom
fix/e2e-false-negative-contracts/ae

Conversation

@ericksoa

@ericksoa ericksoa commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Prevents repeated scheduled E2E false negatives caused by the Node/tsx module boundary, attached-provider cleanup ordering, and the Bedrock snapshot probe observing its own injected environment-variable name. The leak check remains strict for real token values and for matches outside the trusted probe environment.

Changes

  • Treat the private-file controller helper as ESM and exercise outcome classification through the same tsx path used by the live workflow.
  • Detach the inference provider before resetting its credential, then assert both the sandbox attachment and gateway credential are gone.
  • Frame sandbox and host snapshot records so scanned content cannot forge a trusted location, then exempt only the probe's injected adapter environment-variable name; record evidence when that exemption is used and document the upstream condition for removing it, while continuing to reject token values, other processes, command lines, and persisted files.
  • Bound private controller-file reads at the I/O boundary and strengthen regressions for loader errors, post-stat growth, stdout-only provider state, host-log leaks, and forged snapshot markers.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: changes are confined to internal E2E controllers and test contracts; no user command, flag, configuration, API, protocol, or policy behavior changes.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: pending independent review; no waiver claimed.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npx vitest run --project e2e-support --project integration test/e2e/support/live-test-outcome-invocation.test.ts test/e2e/support/bedrock-runtime-compatible-anthropic-leaks.test.ts test/e2e-private-file.test.ts test/pr-risk-plan.test.ts (75 passed); Node 22.23.1 live runner-pressure.mts snapshot reproduction passed; npm run typecheck:cli passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — npm test was attempted after installing the lockfile-pinned nested dependencies, but unrelated base-image, watcher, workflow-discovery, and Oclif environment failures appeared before an installer test stalled for more than 13 minutes; the run was stopped. Focused controller tests, the exact workflow-runtime reproduction, CLI typecheck, normal hooks, ordinary exact-head PR CI, and all seven controller-selected protected E2E shards passed after the current-main refresh (controller, child).
  • Quality Gates section completed with required justifications or waivers — pending independent sensitive-path review.
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@ericksoa ericksoa added area: e2e End-to-end tests, nightly failures, or validation infrastructure bug-fix PR fixes a bug or regression labels Jul 20, 2026
@ericksoa ericksoa self-assigned this Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The change updates private-file reading and E2E module references, extracts Bedrock leak framing and scanning utilities with dedicated tests, strengthens diagnostics credential-reset assertions, and runs the live outcome classifier through npx tsx.

Changes

Private file helper migration

Layer / File(s) Summary
Private file operations
tools/e2e/private-file.mts
Reads up to maxBytes + 1 bytes with fs.readSync, detects post-stat growth, and preserves private-file validation and cleanup behavior.
MTS entrypoint wiring
tools/e2e/*.mts, test/e2e/risk-signal-reporter.ts
Updates private-file helper imports to use private-file.mts.
Private file path coverage
test/e2e-private-file.test.ts, test/pr-risk-plan.test.ts
Updates helper paths and adds FIFO stderr checks plus coverage for files growing after fstat.

Bedrock leak detection

Layer / File(s) Summary
Leak scanner and probe context
test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts
Adds snapshot framing, structured forbidden patterns, probe-PID handling, and sorted leak detection.
Snapshot scanner integration
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
Uses shared framing and scanning utilities, emits probe PID markers, and configures pattern allowances.
Leak detection test cases
test/e2e/support/bedrock-runtime-compatible-anthropic-leaks.test.ts
Covers probe-environment allowances, token leaks, forged markers, framed logs, and attribution boundaries.

Diagnostics credential reset

Layer / File(s) Summary
Provider detach and reset assertions
test/e2e/live/diagnostics.test.ts
Detaches an existing provider, verifies removal, checks reset results, and records the detachment flag in completion assertions.

Classifier invocation

Layer / File(s) Summary
Classifier subprocess command
test/e2e/support/live-test-outcome-invocation.test.ts
Runs the classifier through npx tsx while preserving its working directory and environment variables.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SnapshotScript
  participant SnapshotFraming
  participant LeakScanner
  participant HostLogs
  SnapshotScript->>SnapshotFraming: emit probe PID and framed snapshot files
  HostLogs->>SnapshotFraming: frame host log sections
  SnapshotFraming->>LeakScanner: provide framed snapshot and log text
  LeakScanner-->>SnapshotScript: return deduplicated leak locations
Loading

Possibly related PRs

  • NVIDIA/NemoClaw#6232: Updates sandbox-name propagation in a related sandbox creation and identity flow.
  • NVIDIA/NemoClaw#7168: Updates the runner-pressure flow that reads and writes private-file artifacts.

Suggested labels: area: security

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main E2E bug fix: reducing false failure signals.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/e2e-false-negative-contracts/ae

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 5e69e52 in the fix/e2e-false-negati... branch remains at 96%, unchanged from commit 821aa06 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 5e69e52 in the fix/e2e-false-negati... branch remains at 80%, unchanged from commit 821aa06 in the main branch.

Show a code coverage summary of the most impacted files.
File main 821aa06 fix/e2e-false-negati... 5e69e52 +/-
src/lib/inferen...lama/process.ts 100% 50% -50%
src/lib/domain/.../connect-env.ts 97% 89% -8%
src/lib/inferen...er-lifecycle.ts 71% 65% -6%
src/lib/onboard/preflight.ts 81% 81% 0%
src/lib/sandbox...rce-identity.ts 91% 91% 0%
src/lib/state/m...ock-identity.ts 95% 95% 0%
src/lib/state/m...-acquisition.ts 84% 89% +5%

Updated July 20, 2026 13:43 UTC

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized E2E selections match; severity counts match.

Nemotron output stays in workflow artifacts and does not change the assessment above.

Since last review: 0 prior items resolved · 0 still apply · 0 new items found

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, bedrock-runtime-compatible-anthropic, diagnostics

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e-private-file.test.ts`:
- Line 53: Strengthen the FIFO subprocess assertions around moduleUrl so they
verify the private-file helper actually ran by checking its expected
error/output, rather than only asserting a nonzero exit status. Ensure loader or
--experimental-strip-types failures are rejected as unexpected output and cause
the test to fail.

In `@test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts`:
- Around line 46-58: Update the line-scanning logic around the
SNAPSHOT_FILE_PREFIX handling so markers embedded in emitted file content cannot
change the trusted current-file location or grant probe-environment exemptions.
Use unambiguous framing/escaping or structured per-file records to distinguish
protocol metadata from raw content, and add a regression test covering a forged
marker followed by an adapter environment name.

In `@test/e2e/live/diagnostics.test.ts`:
- Around line 347-373: Update the provider-state checks around credentialsText
and providersAfterDetach to use each command result’s stdout exclusively for
includes/not.toContain assertions, while retaining resultText(detach) and
resultText(providersAfterDetach) only as assertion failure messages. This
prevents stderr warnings from affecting the detach/reset control flow or
provider-list validation.

In `@tools/e2e/private-file.mts`:
- Around line 39-42: Update readPrivateRegularFile to read at most maxBytes plus
one byte, then reject results exceeding options.maxBytes so growth after stat
cannot bypass the limit. Apply equivalent synchronization in
appendPrivateRegularFile by locking the size-check and O_APPEND write as one
operation, or explicitly enforce and document its single-writer contract.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a3416e1f-c03f-4a30-84d7-bbc75708f1db

📥 Commits

Reviewing files that changed from the base of the PR and between 3c4ea5a and 5528d1a.

📒 Files selected for processing (12)
  • test/e2e-private-file.test.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
  • test/e2e/live/diagnostics.test.ts
  • test/e2e/risk-signal-reporter.ts
  • test/e2e/support/bedrock-runtime-compatible-anthropic-leaks.test.ts
  • test/e2e/support/live-test-outcome-invocation.test.ts
  • test/pr-risk-plan.test.ts
  • tools/e2e/live-test-outcome.mts
  • tools/e2e/pr-e2e-gate.mts
  • tools/e2e/private-file.mts
  • tools/e2e/runner-pressure.mts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e-private-file.test.ts`:
- Line 53: Strengthen the FIFO subprocess assertions around moduleUrl so they
verify the private-file helper actually ran by checking its expected
error/output, rather than only asserting a nonzero exit status. Ensure loader or
--experimental-strip-types failures are rejected as unexpected output and cause
the test to fail.

In `@test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts`:
- Around line 46-58: Update the line-scanning logic around the
SNAPSHOT_FILE_PREFIX handling so markers embedded in emitted file content cannot
change the trusted current-file location or grant probe-environment exemptions.
Use unambiguous framing/escaping or structured per-file records to distinguish
protocol metadata from raw content, and add a regression test covering a forged
marker followed by an adapter environment name.

In `@test/e2e/live/diagnostics.test.ts`:
- Around line 347-373: Update the provider-state checks around credentialsText
and providersAfterDetach to use each command result’s stdout exclusively for
includes/not.toContain assertions, while retaining resultText(detach) and
resultText(providersAfterDetach) only as assertion failure messages. This
prevents stderr warnings from affecting the detach/reset control flow or
provider-list validation.

In `@tools/e2e/private-file.mts`:
- Around line 39-42: Update readPrivateRegularFile to read at most maxBytes plus
one byte, then reject results exceeding options.maxBytes so growth after stat
cannot bypass the limit. Apply equivalent synchronization in
appendPrivateRegularFile by locking the size-check and O_APPEND write as one
operation, or explicitly enforce and document its single-writer contract.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a3416e1f-c03f-4a30-84d7-bbc75708f1db

📥 Commits

Reviewing files that changed from the base of the PR and between 3c4ea5a and 5528d1a.

📒 Files selected for processing (12)
  • test/e2e-private-file.test.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
  • test/e2e/live/diagnostics.test.ts
  • test/e2e/risk-signal-reporter.ts
  • test/e2e/support/bedrock-runtime-compatible-anthropic-leaks.test.ts
  • test/e2e/support/live-test-outcome-invocation.test.ts
  • test/pr-risk-plan.test.ts
  • tools/e2e/live-test-outcome.mts
  • tools/e2e/pr-e2e-gate.mts
  • tools/e2e/private-file.mts
  • tools/e2e/runner-pressure.mts
🛑 Comments failed to post (4)
test/e2e-private-file.test.ts (1)

53-53: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Files:\n'
git ls-files 'test/e2e-private-file.test.ts' 'tools/e2e/private-file.mts' || true

printf '\nOutline test file:\n'
ast-grep outline test/e2e-private-file.test.ts --view expanded || true

printf '\nOutline helper file:\n'
ast-grep outline tools/e2e/private-file.mts --view expanded || true

printf '\nRelevant lines from test:\n'
sed -n '1,220p' test/e2e-private-file.test.ts

printf '\nRelevant lines from helper:\n'
sed -n '1,240p' tools/e2e/private-file.mts

Repository: NVIDIA/NemoClaw

Length of output: 7006


Make the FIFO subprocess assertions prove the helper ran.

test/e2e-private-file.test.ts:53 — A module-load or --experimental-strip-types failure can still satisfy the nonzero-exit check. Assert the expected helper error/output and fail on loader errors.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-private-file.test.ts` at line 53, Strengthen the FIFO subprocess
assertions around moduleUrl so they verify the private-file helper actually ran
by checking its expected error/output, rather than only asserting a nonzero exit
status. Ensure loader or --experimental-strip-types failures are rejected as
unexpected output and cause the test to fail.

Source: Path instructions

test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts (1)

46-58: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Do not let scanned content forge file-location records.

emit_file appends raw file content to this same line-based protocol. A persisted file can contain @@NEMOCLAW_E2E_FILE@@ /proc/<probePid>/environ, after which its later adapter-env-name occurrence is exempted as if it came from the trusted probe. The probe PID is already disclosed by the leading marker. Use unambiguous framing/escaping (or scan structured per-file records), and add a regression test with a forged marker in file content.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts` around lines 46
- 58, Update the line-scanning logic around the SNAPSHOT_FILE_PREFIX handling so
markers embedded in emitted file content cannot change the trusted current-file
location or grant probe-environment exemptions. Use unambiguous framing/escaping
or structured per-file records to distinguish protocol metadata from raw
content, and add a regression test covering a forged marker followed by an
adapter environment name.
test/e2e/live/diagnostics.test.ts (1)

347-373: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the target test and helper definitions.
git ls-files test/e2e/live/diagnostics.test.ts
wc -l test/e2e/live/diagnostics.test.ts
sed -n '300,410p' test/e2e/live/diagnostics.test.ts

echo '--- resultText references ---'
rg -n "function resultText|const resultText|resultText\\(" test/e2e -S

echo '--- provider list usage ---'
rg -n 'provider.*list|credentialsText.includes|not.toContain\(hosted\.providerName\)' test/e2e/live/diagnostics.test.ts -S

Repository: NVIDIA/NemoClaw

Length of output: 50372


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '--- diagnostics helpers around resultText/rawResultText ---'
rg -n "function resultText|const resultText|function rawResultText|const rawResultText|runRawNodeCliForLeakAssertion|redactForAssertion" test/e2e/live/diagnostics.test.ts test/e2e -S

echo '--- top of diagnostics.test.ts ---'
sed -n '1,120p' test/e2e/live/diagnostics.test.ts

echo '--- helper definitions if in adjacent files ---'
rg -n "runRawNodeCliForLeakAssertion|rawResultText|redactForAssertion" test/e2e/live test/e2e/fixtures test/e2e/support -S

Repository: NVIDIA/NemoClaw

Length of output: 7488


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '--- provider list and result helpers ---'
rg -n 'provider", "list"|provider list|function resultText|export function resultText|type .*ShellProbeResult|interface .*ShellProbeResult|stdout.*stderr' test/e2e/fixtures test/e2e/live -S

echo '--- sandbox openshell result shape references ---'
rg -n 'openshell\(' test/e2e/live test/e2e/fixtures -S | head -n 40

echo '--- inspect command result helper used by resultText ---'
sed -n '1,140p' test/e2e/fixtures/clients/command.ts

Repository: NVIDIA/NemoClaw

Length of output: 29999


Use stdout for provider-state assertions
rawResultText() folds stderr into the same string, so warnings can make credentialsText.includes(hosted.providerName) take the detach/reset path or make not.toContain(hosted.providerName) fail even when provider output is correct. Use stdout for these checks and keep combined text only in the assertion messages.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/diagnostics.test.ts` around lines 347 - 373, Update the
provider-state checks around credentialsText and providersAfterDetach to use
each command result’s stdout exclusively for includes/not.toContain assertions,
while retaining resultText(detach) and resultText(providersAfterDetach) only as
assertion failure messages. This prevents stderr warnings from affecting the
detach/reset control flow or provider-list validation.

Source: Path instructions

tools/e2e/private-file.mts (1)

39-42: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Repo root: '; pwd
printf '\nFiles:\n'
git ls-files tools/e2e/private-file.mts tools/e2e | sed -n '1,120p'

printf '\nCONTRIBUTING.md (if present):\n'
git ls-files CONTRIBUTING.md && cat -n CONTRIBUTING.md | sed -n '1,220p' || true

printf '\nInspect target file:\n'
cat -n tools/e2e/private-file.mts | sed -n '1,220p'

printf '\nSearch for maxBytes and descriptor usage:\n'
rg -n "maxBytes|descriptor|append|stat\.size|readFileSync" tools/e2e -S

Repository: NVIDIA/NemoClaw

Length of output: 29620


🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '1,220p' tools/e2e/private-file.mts | cat -n
printf '\n---\n'
rg -n "private-file|maxBytes|O_APPEND|appendFile|readFileSync|statSync|openSync" tools/e2e -S

Repository: NVIDIA/NemoClaw

Length of output: 11572


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '--- lines 1-120 ---'
sed -n '1,120p' tools/e2e/private-file.mts | cat -n

echo '--- lines 121-240 ---'
sed -n '121,240p' tools/e2e/private-file.mts | cat -n

Repository: NVIDIA/NemoClaw

Length of output: 3598


🌐 Web query:

Node.js fs.readFileSync file descriptor reads until EOF and concurrent append race stat size read behavior

💡 Result:

In Node.js, the behavior of reading from a file descriptor depends on the specific method used and the state of the file descriptor's internal position pointer. 1. fs.readFileSync and File Descriptors: When using fs.readFileSync(path), Node.js typically opens the file, reads the entire content from the beginning to the end, and then closes it [1][2]. However, when passing an open file descriptor (fd) to fs.readFileSync(fd), it does not necessarily reset the file position [2][3]. If prior read operations have already moved the file's internal position pointer, fs.readFileSync(fd) will read from that current position until EOF [2][3]. 2. Concurrent Append and Read Race Conditions: Node.js does not provide built-in atomicity guarantees for concurrent reads and writes to the same file [4][5][6]. If a process is appending to a file while another is reading it, the reader may or may not observe the newly appended data depending on the timing of the operating system's I/O scheduling [4]. Because file operations are asynchronous (in the case of fs.readFile) or interleaved with the event loop, there is no guarantee that a read operation will capture a "snapshot" of the file state before or after a concurrent append [5][6]. 3. File System Stat and Read Behavior: Reading a file based on a pre-fetched size (e.g., using fs.statSync to get the size and then reading that number of bytes) is a common pattern but is inherently susceptible to race conditions [7]. If the file size changes due to an append operation between the time of the stat call and the completion of the read operation, the read may result in partial data, inconsistency, or require additional handling to detect the new EOF [4][7]. To ensure data integrity when dealing with concurrent file access, you must implement application-level locking (such as using file locks), maintain queues, or utilize database systems designed to handle concurrency, as the Node.js fs module's standard synchronous and asynchronous methods do not manage cross-operation synchronization for you [5][6][7].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Callers of private-file helpers:\n'
rg -n "readPrivateRegularFile|writePrivateRegularFile|appendPrivateRegularFile" tools/e2e -S

printf '\nRelevant surrounding code in runner-pressure.mts:\n'
sed -n '1,340p' tools/e2e/runner-pressure.mts | cat -n | sed -n '1,340p'

printf '\nRelevant surrounding code in live-test-outcome.mts:\n'
sed -n '1,180p' tools/e2e/live-test-outcome.mts | cat -n | sed -n '1,180p'

Repository: NVIDIA/NemoClaw

Length of output: 22381


🌐 Web query:

Node.js documentation fs.readFileSync file descriptor semantics current position EOF concurrent append

💡 Result:

When using fs.readFileSync with a file descriptor in Node.js, the function does not seek to the beginning of the file; it reads from the current file position [1]. If the file descriptor's position has been advanced by previous operations, fs.readFileSync will start reading from that current position until the end of the file (EOF) [2][3]. Key semantic points regarding this behavior include: 1. Current Position Persistence: The underlying file descriptor maintains its own offset (the current file position). Unlike some other high-level APIs that may implicitly reset the position, fs.readFileSync respects the existing state of the file descriptor [1]. 2. Concurrent Appends: If another process or thread appends data to the file while you are reading, fs.readFileSync will read the newly appended data if the read operation reaches the end of the original file and continues into the new data [2][4]. Node.js file system operations are not synchronized or thread-safe; therefore, performing concurrent modifications on the same file can lead to data corruption or unpredictable behavior [2][5]. 3. EOF Handling: If the file is not modified concurrently, the end-of-file is reached when the operation consumes the available data [2][6]. 4. Alternative Approaches: If you require consistent behavior—such as always reading from the beginning of a file—you should manage the file position explicitly using fs.readSync with a specific position offset [1], or open a new file descriptor for the read operation to ensure it starts at offset 0. Because fs.readFileSync and fs.readFile are helper functions that simply consume data until EOF based on the current descriptor position, they are often considered unsuitable for scenarios requiring complex seek patterns [1]. For granular control over file offsets and concurrent access, use fs.read or fs.readSync with an explicit position argument [2][1].

Citations:


🏁 Script executed:

node - <<'JS'
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');

const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'nemo-private-file-'));
const file = path.join(dir, 'sample.txt');

fs.writeFileSync(file, '12345');

const fd = fs.openSync(file, fs.constants.O_RDONLY);
try {
  const before = fs.fstatSync(fd).size;
  fs.appendFileSync(file, '67890');
  const after = fs.fstatSync(fd).size;
  const data = fs.readFileSync(fd, 'utf8');
  console.log(JSON.stringify({ before, after, data, dataLength: Buffer.byteLength(data) }));
} finally {
  fs.closeSync(fd);
  fs.rmSync(dir, { recursive: true, force: true });
}
JS

Repository: NVIDIA/NemoClaw

Length of output: 213


Enforce maxBytes at the I/O boundary. readPrivateRegularFile() can still return more than maxBytes if the file grows after fstatSync(), and appendPrivateRegularFile() has the same size-check race before O_APPEND writes. Read at most maxBytes + 1 bytes, and either lock appends or make the single-writer contract explicit.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/private-file.mts` around lines 39 - 42, Update
readPrivateRegularFile to read at most maxBytes plus one byte, then reject
results exceeding options.maxBytes so growth after stat cannot bypass the limit.
Apply equivalent synchronization in appendPrivateRegularFile by locking the
size-check and O_APPEND write as one operation, or explicitly enforce and
document its single-writer contract.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/e2e-private-file.test.ts (1)

78-83: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid locking this test to stack-frame names.

at readPrivateRegularFile and at openPrivateFileForWrite are implementation details, not part of the FIFO rejection contract. Keep the stable error/path and module-loading assertions, but remove these frame checks to avoid false failures after harmless renames or stack-format changes. As per path instructions, review tests for behavioral confidence rather than implementation lock-in.

Suggested adjustment
-      expect(read.stderr).toContain("at readPrivateRegularFile");
...
-      expect(write.stderr).toContain("at openPrivateFileForWrite");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-private-file.test.ts` around lines 78 - 83, Remove the stack-frame
assertions for “at readPrivateRegularFile” and “at openPrivateFileForWrite” from
this FIFO rejection test. Preserve the stable stderr error, FIFO path, status,
and module-loading assertions so the test continues validating behavior rather
than implementation-specific function names.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e-private-file.test.ts`:
- Around line 78-83: Remove the stack-frame assertions for “at
readPrivateRegularFile” and “at openPrivateFileForWrite” from this FIFO
rejection test. Preserve the stable stderr error, FIFO path, status, and
module-loading assertions so the test continues validating behavior rather than
implementation-specific function names.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e17088bc-c20a-47ae-828f-26a6a9957afb

📥 Commits

Reviewing files that changed from the base of the PR and between 5528d1a and 447a97e.

📒 Files selected for processing (6)
  • test/e2e-private-file.test.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts
  • test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
  • test/e2e/live/diagnostics.test.ts
  • test/e2e/support/bedrock-runtime-compatible-anthropic-leaks.test.ts
  • tools/e2e/private-file.mts
🚧 Files skipped from review as they are similar to previous changes (4)
  • test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
  • tools/e2e/private-file.mts
  • test/e2e/live/bedrock-runtime-compatible-anthropic-leaks.ts
  • test/e2e/live/diagnostics.test.ts

ericksoa added 2 commits July 19, 2026 23:36
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…e-contracts/ae

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@ericksoa
ericksoa requested a review from jyaunches July 20, 2026 14:25

@cjagwani cjagwani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review complete: exact-head gates pass, security-sensitive E2E boundaries were reviewed, and no unresolved correctness or security findings remain.

@ericksoa
ericksoa merged commit fdbb41f into main Jul 20, 2026
100 checks passed
@ericksoa
ericksoa deleted the fix/e2e-false-negative-contracts/ae branch July 20, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants