diff --git a/.agent/notes/kernel-security-audit-2026-04-05.md b/.agent/notes/kernel-security-audit-2026-04-05.md new file mode 100644 index 000000000..0f8ea91f3 --- /dev/null +++ b/.agent/notes/kernel-security-audit-2026-04-05.md @@ -0,0 +1,549 @@ +# Agent OS Kernel Security Audit Report + +**Date:** 2026-04-05 +**Scope:** Full adversarial review of kernel, execution engines, VFS, networking, permissions, and POSIX compliance +**Method:** 12 parallel adversarial review agents examining each subsystem independently + +--- + +## Executive Summary + +This audit examined the Agent OS kernel across 12 dimensions: VFS/overlay filesystem, Node.js isolation, WASM execution, process table/signals, network stack, permission system, Python/Pyodide isolation, sidecar RPC, POSIX edge cases, host information leakage, resource limits/DoS, and control channel security. + +**Key findings:** +- **58 CRITICAL/HIGH issues** across all subsystems +- **Node.js isolation is the weakest link** -- many builtins fall through to real host modules +- **Network stack has zero permission enforcement** -- guest code can connect anywhere +- **Control channels are in-band** -- guest can inject fake control messages via stderr +- **POSIX compliance has major gaps** -- no fork(), no file locking, no signal handlers, no mmap +- **Python/Pyodide is the most secure subsystem** -- proper WASM sandboxing with defense-in-depth + +--- + +## 1. Linux Kernel Compatibility Matrix + +### 1.1 Syscall / Feature Implementation Status + +| Feature | Linux | Agent OS | Status | Severity | +|---------|-------|----------|--------|----------| +| **Filesystem** | | | | | +| open/close/read/write | Full POSIX | Implemented | OK | - | +| pread/pwrite | Full POSIX | Implemented | OK | - | +| stat/lstat/fstat | Full POSIX | Implemented | Partial (missing blocks, dev fields) | LOW | +| readdir | Full POSIX | Implemented | OK (filters `.`/`..`) | - | +| mkdir/rmdir | Full POSIX | Implemented | OK | - | +| rename | Atomic | Non-atomic multi-step | BROKEN | HIGH | +| link/unlink | Full POSIX | Implemented | OK | - | +| symlink/readlink | Full POSIX | Implemented | OK | - | +| chmod/chown | Full POSIX | Implemented | Missing permission enforcement | MEDIUM | +| truncate/ftruncate | Full POSIX | Implemented | OK | - | +| O_APPEND | Atomic seek+write | Non-atomic (race condition) | BROKEN | CRITICAL | +| O_CREAT\|O_EXCL | Atomic create-if-not-exists | TOCTOU race (check then create) | BROKEN | CRITICAL | +| O_NONBLOCK | Per-FD flag | Not implemented | MISSING | HIGH | +| O_DIRECTORY | opendir validation | Not implemented | MISSING | LOW | +| O_NOFOLLOW | Symlink rejection | Not implemented | MISSING | MEDIUM | +| O_CLOEXEC / FD_CLOEXEC | Per-FD flag | Not implemented in kernel | MISSING | MEDIUM | +| flock / fcntl locking | Advisory/mandatory locks | Not implemented | MISSING | CRITICAL | +| mmap / munmap | Memory-mapped files | Not implemented | MISSING | HIGH | +| sendfile / splice | Zero-copy transfer | Not implemented | MISSING | LOW | +| sparse files | Hole-aware storage | Materialized as zeros | BROKEN | MEDIUM | +| xattr | Extended attributes | Not implemented | MISSING | LOW | +| umask | Default creation mask | Not implemented | MISSING | MEDIUM | +| sticky bit | /tmp protection | Not enforced | MISSING | MEDIUM | +| setgid on dirs | Group inheritance | Not implemented | MISSING | LOW | +| atime/mtime/ctime | Full tracking | Partial (atime only on pread) | BROKEN | LOW | +| inotify / fanotify | FS event monitoring | Not implemented | MISSING | LOW | +| **Process Management** | | | | | +| fork() | Full COW semantics | Not implemented (spawn only) | MISSING | CRITICAL | +| exec() | Replaces process image | Partial (no shebang parsing) | BROKEN | HIGH | +| waitpid() | Full flags (WNOHANG, etc.) | Blocking only, single PID | BROKEN | HIGH | +| kill() | Full signal delivery | Only SIGTERM/SIGKILL work | BROKEN | HIGH | +| getpid/getppid | Full | Virtualized (correct) | OK | - | +| setpgid/getpgid | Full | Implemented | OK | - | +| setsid/getsid | Full | Implemented (no orphan handling) | PARTIAL | MEDIUM | +| setuid/setgid/seteuid | Full | Not implemented | MISSING | LOW | +| process groups | Full signal delivery | Kill doesn't reach stopped processes | BROKEN | HIGH | +| sessions | Full with controlling TTY | Partial (no orphan group handling) | BROKEN | MEDIUM | +| reparenting to init | Automatic on parent death | Not implemented | MISSING | HIGH | +| zombie reaping | Via waitpid() | 60s TTL auto-reap (non-standard) | DIFFERENT | MEDIUM | +| **Signals** | | | | | +| SIGCHLD | On child exit | Not implemented | MISSING | CRITICAL | +| SIGPIPE | On broken pipe write | Not implemented (EPIPE only) | MISSING | HIGH | +| SIGWINCH | On terminal resize | Not implemented | MISSING | MEDIUM | +| SIGSTOP/SIGCONT | Job control | Not implemented | MISSING | HIGH | +| SIGINT/SIGQUIT/SIGTSTP | Terminal signals | PTY-only (correct) | OK | - | +| SIGTERM | Termination | Implemented | OK | - | +| SIGKILL | Forced kill | Implemented | OK | - | +| sigprocmask | Signal blocking | Not implemented | MISSING | HIGH | +| sigaction | Handler registration | Not implemented | MISSING | HIGH | +| SA_RESTART | Syscall restart | Not implemented | MISSING | MEDIUM | +| EINTR | Interrupted syscall | Not implemented | MISSING | HIGH | +| Real-time signals | SIGRTMIN-SIGRTMAX | Not implemented | MISSING | LOW | +| **Pipes & IPC** | | | | | +| pipe/pipe2 | 64KB buffer | 65KB buffer (close enough) | OK | - | +| PIPE_BUF atomicity | Writes <= 4096 atomic | Not atomic at any size | BROKEN | HIGH | +| Non-blocking pipes | O_NONBLOCK + EAGAIN | Not implemented | MISSING | HIGH | +| select/poll/epoll | FD multiplexing | Not implemented | MISSING | CRITICAL | +| Unix domain sockets | AF_UNIX | Not implemented | MISSING | MEDIUM | +| SCM_RIGHTS | FD passing | Not implemented | MISSING | LOW | +| **Networking** | | | | | +| TCP sockets | Full | Sidecar-managed (no kernel mediation) | BROKEN | CRITICAL | +| UDP sockets | Full | Sidecar-managed (no kernel mediation) | BROKEN | CRITICAL | +| DNS resolution | Full | Falls through to host resolver | BROKEN | CRITICAL | +| SO_REUSEADDR | Socket option | Not implemented in kernel | MISSING | MEDIUM | +| Non-blocking connect | O_NONBLOCK + EINPROGRESS | Not implemented in kernel | MISSING | MEDIUM | +| **TTY/PTY** | | | | | +| PTY pairs | Full | Implemented | OK | - | +| Canonical mode | Line editing | Partial | PARTIAL | LOW | +| Raw mode | Character-at-a-time | Partial (no full termios) | PARTIAL | MEDIUM | +| VMIN/VTIME | Read timing | Not implemented | MISSING | LOW | +| Echo control | Per-character | Basic flag only | PARTIAL | LOW | +| ^C/^D/^Z/^\ | Special chars | ^C/^Z/^\ work, ^D missing | PARTIAL | LOW | +| **Device Files** | | | | | +| /dev/null | Full | Implemented | OK | - | +| /dev/zero | Configurable read size | Fixed 4096 bytes always | BROKEN | LOW | +| /dev/urandom | Configurable read size | Fixed 4096 bytes always | BROKEN | LOW | +| /dev/full | ENOSPC on write | Not implemented | MISSING | LOW | +| /dev/random | Blocking entropy | Not implemented | MISSING | LOW | +| /dev/fd/N | FD directory | Stub (empty listing) | BROKEN | MEDIUM | +| /dev/tty | Controlling terminal | Not implemented | MISSING | MEDIUM | +| /dev/pts/* | PTY devices | Stub | PARTIAL | LOW | +| **/proc Filesystem** | | | | | +| /proc/self | Symlink to PID | Not implemented | MISSING | MEDIUM | +| /proc/[pid]/stat | Process status | Not implemented | MISSING | MEDIUM | +| /proc/[pid]/status | Process info | Not implemented | MISSING | MEDIUM | +| /proc/[pid]/fd/ | Open FDs | Not implemented | MISSING | MEDIUM | +| /proc/[pid]/cmdline | Command line | Not implemented | MISSING | LOW | +| /proc/[pid]/environ | Environment | Not implemented | MISSING | LOW | +| /proc/[pid]/cwd | Working dir link | Not implemented | MISSING | LOW | +| /proc/[pid]/exe | Executable link | Not implemented | MISSING | LOW | +| /proc/cpuinfo | CPU info | Not implemented | MISSING | LOW | +| /proc/meminfo | Memory info | Not implemented | MISSING | LOW | +| /proc/mounts | Mount table | Not implemented | MISSING | MEDIUM | +| /proc/sys/* | Sysctl | Not implemented | MISSING | LOW | + +### 1.2 Error Code Coverage + +| errno | Linux | Agent OS | Status | +|-------|-------|----------|--------| +| EACCES | Permission denied | Implemented | OK | +| EAGAIN | Try again | Implemented | OK | +| EBADF | Bad FD | Implemented | OK | +| EEXIST | File exists | Implemented | OK | +| EINTR | Interrupted syscall | Not implemented | MISSING | +| EINVAL | Invalid argument | Implemented | OK | +| EIO | I/O error | Implemented | OK | +| EISDIR | Is a directory | Partial (not on write) | BROKEN | +| ELOOP | Symlink loop | Implemented (40 depth) | OK | +| EMFILE | Too many FDs | Implemented | OK | +| ENAMETOOLONG | Path too long | Not implemented | MISSING | +| ENOENT | No such file | Implemented | OK | +| ENOSPC | No space | Implemented | OK | +| ENOSYS | Not implemented | Implemented | OK | +| ENOTDIR | Not a directory | Partial | BROKEN | +| ENOTEMPTY | Dir not empty | Implemented | OK | +| EPERM | Not permitted | Implemented | OK | +| EPIPE | Broken pipe | Implemented (no signal) | PARTIAL | +| EROFS | Read-only FS | Not implemented | MISSING | +| ESRCH | No such process | Implemented | OK | +| EXDEV | Cross-device link | Implemented in mount_table | OK | +| EBUSY | Resource busy | Not implemented | MISSING | +| EWOULDBLOCK | Would block | Not implemented | MISSING | + +--- + +## 2. Security & Sandboxing Gaps + +### 2.1 CRITICAL: Node.js Builtin Fallthrough to Host + +**Severity: CRITICAL** +**Location:** `crates/execution/src/node_import_cache.rs` + +The ESM loader only explicitly handles ~15 Node.js builtins. All others fall through to `nextResolve()`, which returns the real host module. Critical uncovered builtins include: + +- `node:crypto` -- Host cryptography, random sources +- `node:wasi` -- WebAssembly System Interface (host system access) +- `node:sqlite` -- Direct host database access +- `node:perf_hooks` -- Timing attacks, host uptime measurement +- `node:tty` -- Host terminal I/O +- `node:async_hooks` -- Internal state introspection +- `node:stream`, `node:buffer`, `node:zlib` -- No hardening + +**Impact:** Guest code can `import crypto from 'node:crypto'` and get the REAL host module. + +### 2.2 CRITICAL: Network Operations Bypass Permission System + +**Severity: CRITICAL** +**Location:** `crates/sidecar/src/service.rs` lines 6027-6245 + +The kernel has `check_network_access()` in `permissions.rs` but it is NEVER called for socket/DNS operations in the sidecar RPC handlers. Guest code can: + +- Connect to ANY host/port (including cloud metadata at 169.254.169.254) +- Bind to ANY interface including 0.0.0.0 (exposing to all VMs) +- Perform DNS lookups against host resolver +- Send UDP datagrams anywhere +- Bypass `fetch()` hardening via `http.request()`, `net.connect()`, etc. + +### 2.3 CRITICAL: Control Channel Message Injection + +**Severity: CRITICAL** +**Location:** `crates/execution/src/javascript.rs`, `crates/execution/src/node_process.rs` + +Guest code can write magic-prefixed lines to stderr to: +- Inject fake warmup metrics (`__AGENT_OS_NODE_WARMUP_METRICS__:`) +- Inject fake exit codes (`__AGENT_OS_PYTHON_EXIT__:`) +- Inject fake signal state (`__AGENT_OS_SIGNAL_STATE__:`) +- Suppress arbitrary stderr output + +```javascript +// Guest can write: +console.error('__AGENT_OS_PYTHON_EXIT__:{"exitCode":0}'); +``` + +### 2.4 CRITICAL: WASM Memory Limits Not Enforced at Runtime + +**Severity: CRITICAL** +**Location:** `crates/execution/src/wasm.rs` lines 840-843, 876-916 + +`WASM_MAX_MEMORY_BYTES_ENV` is only used for compile-time validation at module load. It is NOT passed to the Node.js runtime. Guest WASM code can grow memory beyond any configured limit at runtime, causing host OOM. + +### 2.5 CRITICAL: WASI Unconditionally Enabled + +**Severity: CRITICAL** +**Location:** `crates/execution/src/wasm.rs` line 612 + +`allow_wasi = true` is hardcoded for all WASM execution regardless of permission tier. Even "Isolated" tier gets WASI access. + +### 2.6 HIGH: Unvalidated FD Access for RPC Channels + +**Severity: HIGH** +**Location:** `crates/execution/src/javascript.rs` lines 725-730, 953-960 + +RPC channel FD numbers are passed via environment variables with `FD_CLOEXEC` explicitly cleared. Guest code can: +- Close RPC FDs to break sidecar communication +- Read/write to manipulate RPC messages +- Redirect them with dup2() to other FDs + +### 2.7 HIGH: Unmount Has No Permission Check + +**Severity: HIGH** +**Location:** `crates/kernel/src/kernel.rs` lines 1425-1432 + +`unmount_filesystem()` bypasses all permission checks. Guest can unmount any filesystem including `/`, `/etc`, `/proc`. + +### 2.8 HIGH: Symlink Resolution Bypass in Permission System + +**Severity: HIGH** +**Location:** `crates/kernel/src/permissions.rs` lines 484-491 + +`read_link()` and `lstat()` use `normalize_path()` instead of `check_subject()`, skipping symlink resolution before permission checks. Guest can create symlinks to forbidden paths and read targets. + +### 2.9 HIGH: Host Information Leakage via Path Fallbacks + +**Severity: HIGH** +**Location:** `crates/execution/src/node_import_cache.rs` + +- `guestVisiblePathFromHostPath()` falls back to raw host path when mapping fails +- `INITIAL_GUEST_CWD` falls back to `HOST_CWD` if not in path mappings +- `os.homedir()`, `os.userInfo()`, `os.tmpdir()` fall back to host values +- `process.config`, `process.versions` expose host build info +- `AGENT_OS_*` variables passed through to child processes + +### 2.10 HIGH: process Object Properties Leak Host Info + +**Severity: HIGH** +**Location:** `crates/execution/src/node_import_cache.rs` lines 6176-6223 + +The guest process proxy only overrides 5 properties. All others pass through via `Reflect.get()`: +- `process.version` -- Host Node version +- `process.config` -- Complete host build configuration +- `process.versions` -- Host module versions (openssl, v8, zlib) +- `process.memoryUsage()` -- Host memory usage +- `process.uptime()` -- Host uptime + +### 2.11 HIGH: CJS require() Loads from Host node_modules + +**Severity: HIGH** +**Location:** `crates/execution/src/node_import_cache.rs` lines 6225-6271 + +`createGuestRequire()` uses `Module.createRequire()` + `baseRequire()` which resolves packages from HOST `node_modules`. Guest code can load arbitrary host packages. + +### 2.12 HIGH: Default Permissions Are allow_all + +**Severity: HIGH (footgun)** +**Location:** `crates/kernel/src/kernel.rs` line 101 + +`KernelVmConfig::new()` defaults to `Permissions::allow_all()` instead of deny-by-default. Any code creating a VM without explicit permissions gets unrestricted access. + +--- + +## 3. Node.js / WASM Bridge Issues + +### 3.1 Node.js Builtin Coverage + +| Builtin | Has Polyfill? | Routes Through Kernel? | Security Status | +|---------|--------------|----------------------|----------------| +| `fs` / `fs/promises` | Yes (sync RPC) | Yes (VFS) | PARTIAL -- path-translating, not full polyfill | +| `child_process` | Yes (sync RPC) | Yes (process table) | PARTIAL -- wraps real spawn | +| `net` | Yes (sidecar RPC) | NO -- direct host sockets | BROKEN | +| `dgram` | Yes (sidecar RPC) | NO -- direct host sockets | BROKEN | +| `dns` | Yes (sidecar RPC) | NO -- direct host resolver | BROKEN | +| `http` / `https` | Yes (layered on net) | NO -- inherits net bypass | BROKEN | +| `http2` | Yes (layered on net) | NO -- inherits net bypass | BROKEN | +| `tls` | Yes (layered on net) | NO -- inherits net bypass | BROKEN | +| `os` | Yes (full polyfill) | Yes (virtualized values) | OK | +| `path` | Passthrough | N/A (pure computation) | OK | +| `url` | Passthrough | N/A (pure computation) | OK | +| `crypto` | NO | Falls through to host | CRITICAL | +| `wasi` | NO | Falls through to host | CRITICAL | +| `sqlite` | NO | Falls through to host | CRITICAL | +| `perf_hooks` | NO | Falls through to host | HIGH | +| `tty` | NO | Falls through to host | HIGH | +| `async_hooks` | NO | Falls through to host | MEDIUM | +| `stream` | NO | Falls through to host | MEDIUM | +| `buffer` | NO | Falls through to host | LOW | +| `zlib` | NO | Falls through to host | LOW | +| `vm` | Should be denied | Falls through if in ALLOWED | CRITICAL | +| `worker_threads` | Should be denied | Falls through if in ALLOWED | CRITICAL | +| `inspector` | Should be denied | Falls through if in ALLOWED | CRITICAL | +| `v8` | Should be denied | Falls through if in ALLOWED | CRITICAL | + +### 3.2 WASM Execution Gaps + +| Issue | Severity | Details | +|-------|----------|---------| +| Memory limits not runtime-enforced | CRITICAL | Only compile-time validation, no runtime cap | +| Fuel limits are coarse timeouts | CRITICAL | Fuel = millisecond timeout, not per-instruction | +| WASI always enabled | CRITICAL | Hardcoded `allow_wasi = true` regardless of tier | +| Module parser DoS | HIGH | Unbounded section iteration, no module size limit | +| Symlink TOCTOU in module path | HIGH | Different resolution at validation vs execution | +| Stack limit overflow | MEDIUM | No upper bound on `--stack-size` parameter | +| Prewarm phase no timeout | MEDIUM | `ensure_materialized()` can hang forever | +| File fingerprint TOCTOU | MEDIUM | size+mtime race for warmup cache | + +### 3.3 Sync RPC Bridge Vulnerabilities + +| Issue | Severity | Details | +|-------|----------|---------| +| No RPC authentication | MEDIUM | Simple integer IDs, no HMAC | +| Guest can forge RPC requests | MEDIUM | Write arbitrary JSON to request FD | +| Response writer can deadlock | MEDIUM | Guest slow-read causes sidecar hang | +| FD reservation race window | HIGH | Reservation dropped before clear_cloexec | + +--- + +## 4. POSIX Edge Cases That Will Break + +### 4.1 Things That Work on Linux But Break Here + +| Scenario | What Linux Does | What Agent OS Does | Impact | +|----------|----------------|-------------------|--------| +| `git commit` | Atomic O_CREAT\|O_EXCL for refs | TOCTOU race, can corrupt refs | git broken | +| `npm install` | fcntl locking for package-lock | No locking, concurrent installs corrupt | npm broken | +| `python -c "import mmap"` | Memory maps files | No mmap, ImportError | Python broken | +| Concurrent log writes | O_APPEND atomic | Race condition, interleaved data | Data corruption | +| Shell job control (^Z, bg, fg) | SIGTSTP/SIGCONT | Not implemented | Shell broken | +| `make -j4` | fork() for parallel jobs | No fork, must use spawn | make broken | +| `#!/bin/sh` scripts | Kernel parses shebang | Not parsed | Scripts fail | +| Pipe write <= 4KB | Atomic (PIPE_BUF guarantee) | Not atomic, interleaved | IPC corruption | +| `select()` on multiple FDs | Multiplexed I/O | Not implemented | Event loops broken | +| Parent gets SIGCHLD | Signal on child exit | Not delivered | Cannot async-reap children | +| `flock /tmp/lockfile` | Advisory file lock | Not implemented | Lock files useless | +| Non-blocking I/O | O_NONBLOCK + EAGAIN | Not implemented | Async I/O broken | + +### 4.2 Standard Tool Compatibility + +| Tool | Will It Work? | Why Not | +|------|--------------|---------| +| git | NO | No atomic O_CREAT\|O_EXCL, no flock | +| npm/yarn/pnpm | NO | No fcntl locking | +| python | PARTIAL | No mmap, no fork, no fcntl | +| curl/wget | NO | Network bypasses kernel | +| tar | PARTIAL | Sparse files materialized, timestamps incomplete | +| grep | YES | Basic file I/O works | +| sed/awk | YES | Basic file I/O works | +| make | NO | No fork() for parallel jobs | +| docker | NO | No fork, no namespace, no cgroups | +| ssh | NO | Network bypasses kernel | +| vim/nano | PARTIAL | PTY works, but missing VMIN/VTIME | + +--- + +## 5. Filesystem Deep Dive + +### 5.1 Overlay FS Issues + +| Issue | Severity | Details | +|-------|----------|---------| +| No opaque directory markers | HIGH | Lower layer entries leak through after copy-up | +| Whiteouts are in-memory only | HIGH | Lost on snapshot/persistence | +| No whiteout character devices | MEDIUM | Incompatible with standard OverlayFS tools | +| Copy-up TOCTOU race | MEDIUM | Symlink target can change between read and create | +| removeDir doesn't check lower children | HIGH | Can remove non-empty dir if children only in lower | +| Hardlink copy-up path resolution broken | HIGH | link() after copy-up references wrong path | +| Rename not atomic | HIGH | Read+write+delete pattern, crash-unsafe | + +### 5.2 VFS Issues + +| Issue | Severity | Details | +|-------|----------|---------| +| Hardlink across mounts not checked | HIGH | Should return EXDEV, currently allowed | +| Stat missing blocks/dev fields | LOW | Tools expecting `st_blocks` will get 0 | +| Time precision milliseconds only | LOW | Linux uses nanoseconds | +| No S_IFCHR/S_IFBLK/S_IFIFO/S_IFSOCK | MEDIUM | Missing file type bits in mode | +| /dev/zero returns fixed 4096 bytes | LOW | Should return requested length | +| /dev/urandom returns fixed 4096 bytes | LOW | Should return requested length | + +### 5.3 Remote Filesystem / Mount Issues + +| Issue | Severity | Details | +|-------|----------|---------| +| Mount permissions checked, unmount not | HIGH | Guest can unmount anything | +| TypeScript overlay has no resource limits | MEDIUM | Unlimited files/size in upper layer | +| Copy-up not counted against limits | MEDIUM | Large lower files can exhaust memory | +| S3 mount doesn't persist whiteouts | HIGH | Deleted files reappear | + +--- + +## 6. Resource Limits & DoS Vectors + +### 6.1 Properly Protected Resources + +| Resource | Limit | Default | Status | +|----------|-------|---------|--------| +| Filesystem total size | max_filesystem_bytes | 64 MB | OK | +| Inode count | max_inode_count | 16,384 | OK | +| FDs per process | MAX_FDS_PER_PROCESS | 256 | OK | +| Pipe buffer | MAX_PIPE_BUFFER_BYTES | 65,536 | OK | +| PTY buffer | MAX_PTY_BUFFER_BYTES | 65,536 | OK | +| Symlink depth | MAX_SYMLINK_DEPTH | 40 | OK | +| Zombie TTL | ZOMBIE_TTL | 60s | OK | +| Python output buffer | max_bytes | 1 MB | OK | + +### 6.2 Unbounded / Missing Limits + +| Resource | Status | Attack | +|----------|--------|--------| +| pread() length | NO LIMIT | `pread(fd, 0, usize::MAX)` -- host OOM | +| fd_write() data size | NO PER-OP LIMIT | Single huge write can OOM before FS limit check | +| Environment variable size | NO LIMIT | Gigabyte env vars in spawn | +| Command argument size | NO LIMIT | Gigabyte argv lists | +| readdir result size | NO LIMIT | 16K entry directory returns all at once | +| Filesystem snapshot | NO LIMIT | Clones entire FS state to memory | +| File truncate | NO LIMIT | `truncate("/f", 1TB)` allocates and zeros 1TB | +| WASM runtime memory | NOT ENFORCED | Compile-time only, runtime unbounded | +| Socket count | FIELD EXISTS, NOT ENFORCED | No enforcement code found | +| Connection count | FIELD EXISTS, NOT ENFORCED | No enforcement code found | +| Network bandwidth | NOT IMPLEMENTED | Guest can flood network | +| Process spawn as zombies | ONLY RUNNING COUNTED | Create+exit loop bypasses max_processes | +| Path length | NOT CHECKED | Unbounded path strings | +| Symlink target length | NOT CHECKED | Huge symlink targets | +| Single file size | ONLY TOTAL FS CHECKED | One file can be entire 64MB | + +--- + +## 7. Python/Pyodide Assessment (Best Secured) + +The Pyodide engine is the most well-secured subsystem: + +- Runs Python in WASM (not native), providing architectural isolation +- VFS RPC properly scoped to `/workspace` with path validation +- `js` and `pyodide_js` modules blocked (prevents WASM-JS interop escape) +- `os.system()` and `subprocess.*` monkey-patched to route through kernel +- `process.binding()` and `process.dlopen()` throw access denied +- `fetch()` restricted to `data:` URLs only +- Output buffers properly bounded (1MB default) +- ctypes neutered by WASM architecture (no native library loading) + +**Remaining concerns:** +- No memory limit on Pyodide process +- No execution timeout at Python level +- Recursion depth only limited by Python's default ~1000 frames + +--- + +## 8. Control Channel Security Summary + +| Channel | Mechanism | In-Band? | Guest Can Forge? | +|---------|-----------|----------|-----------------| +| Node.js warmup metrics | stderr prefix `__AGENT_OS_NODE_WARMUP_METRICS__:` | YES | YES | +| Python exit code | stderr prefix `__AGENT_OS_PYTHON_EXIT__:` | YES | YES | +| WASM warmup metrics | stderr prefix `__AGENT_OS_WASM_WARMUP_METRICS__:` | YES | YES | +| Signal state | stderr prefix `__AGENT_OS_SIGNAL_STATE__:` | YES | YES | +| Node sync RPC | Dedicated FD pipes | No | YES (FD accessible) | +| Python VFS RPC | Dedicated FD pipes | No | YES (FD accessible) | +| Node control channel | Dedicated FD pipe | No | YES (FD accessible) | +| Sidecar stdio protocol | stdin/stdout framed | Parent-controlled | No (proper isolation) | + +--- + +## 9. Priority Remediation Recommendations + +### P0 -- Immediate (Security-Critical) + +1. **Block all uncovered Node.js builtins** -- Every builtin not in BUILTIN_ASSETS must be in DENIED_BUILTINS. No fallthrough to `nextResolve()`. +2. **Add permission checks to network operations** -- All socket connect/bind/DNS operations must call `check_network_access()`. +3. **Move control messages out-of-band** -- Use dedicated FDs for all control signaling instead of stderr magic prefixes. +4. **Enforce WASM memory limits at runtime** -- Pass `WASM_MAX_MEMORY_BYTES_ENV` to Node.js runtime, not just compile-time validation. +5. **Make WASI conditional** -- Disable WASI for Isolated permission tier. +6. **Add permission check to unmount** -- `unmount_filesystem()` must check permissions. +7. **Fix symlink bypass in read_link/lstat** -- Use `check_subject()` not `check()`. + +### P1 -- High Priority (Correctness/Isolation) + +8. **Implement O_CREAT|O_EXCL atomicity** -- Single atomic create-if-not-exists operation. +9. **Implement O_APPEND atomicity** -- Atomic seek-to-end + write. +10. **Bound pread/fd_write per-operation size** -- Add max_read_length, max_write_length limits. +11. **Fix host info leakage** -- Never fall back to host paths; use safe defaults. +12. **Proxy all process properties** -- Block `process.config`, `process.versions`, `process.memoryUsage()`. +13. **Filter AGENT_OS_* from child processes** -- Strip internal vars before spawn. +14. **Fix overlay whiteout persistence** -- Store in durable layer, not in-memory Set. +15. **Add opaque directory support** -- Implement OverlayFS opaque markers. +16. **Fix hardlink across mounts** -- Return EXDEV. +17. **Default permissions to deny-all** -- Change `KernelVmConfig::new()` default. + +### P2 -- Medium Priority (POSIX Compliance) + +18. **Implement SIGCHLD** -- Deliver to parent on child exit. +19. **Implement SIGPIPE** -- Deliver on write to broken pipe. +20. **Implement waitpid flags** -- WNOHANG, WUNTRACED, WCONTINUED, negative PID. +21. **Implement file locking** -- At least advisory flock(). +22. **Implement select/poll** -- FD multiplexing for event loops. +23. **Implement O_NONBLOCK** -- Non-blocking I/O with EAGAIN. +24. **Implement PIPE_BUF atomicity** -- Writes <= 4096 bytes must be atomic. +25. **Count zombies against process limits** -- Prevent zombie storms. +26. **Implement reparenting** -- Orphaned children go to init. +27. **Implement /proc filesystem** -- At least /proc/self, /proc/[pid]/fd, /proc/mounts. +28. **Fix /dev/zero and /dev/urandom** -- Return requested byte count, not fixed 4096. + +### P3 -- Low Priority (Polish) + +29. Implement shebang parsing for exec() +30. Add EISDIR for write-to-directory +31. Add ENOTDIR for path component checks +32. Add ENAMETOOLONG +33. Implement umask +34. Implement sticky bit enforcement +35. Add stat blocks/dev fields +36. Implement /dev/full, /dev/tty +37. Add nanosecond time precision +38. Implement SIGWINCH for PTY resize + +--- + +## 10. Subsystem Security Scorecard + +| Subsystem | Score | Assessment | +|-----------|-------|-----------| +| Python/Pyodide | A- | Strong WASM boundary, proper import blocking, VFS scoping | +| Permission System | C+ | Good design, but bypasses in read_link, lstat, unmount, network | +| Process Table | C | Basic functionality, missing signals/fork/reparenting | +| VFS Core | C+ | Correct for basic ops, missing atomicity guarantees | +| Overlay FS | C- | Missing opaque dirs, in-memory whiteouts, broken hardlink copy-up | +| Sidecar RPC | B- | Good auth/ownership checks, but info leaks and missing timeouts | +| WASM Engine | D+ | Limits not enforced at runtime, WASI always on | +| Node.js Isolation | D | Many builtins fall through, host info leaks everywhere | +| Network Stack | F | Zero permission enforcement, no address validation, full SSRF | +| Control Channels | D | All in-band via stderr, guest can forge messages | +| Resource Limits | C- | Some limits exist but many unbounded vectors | +| Host Info Protection | D+ | Good intent, but fallback-to-host pattern leaks everywhere | diff --git a/.agent/todo/node-isolation-gaps.md b/.agent/todo/node-isolation-gaps.md new file mode 100644 index 000000000..859f9d58e --- /dev/null +++ b/.agent/todo/node-isolation-gaps.md @@ -0,0 +1,310 @@ +# Runtime Isolation Gaps + +Agent OS is a fully virtualized operating system. Every guest syscall must go through the kernel — no guest operation may fall through to a real host syscall. The Node.js execution model currently spawns real host OS child processes (`std::process::Command::new("node")`) and most builtins either fall through to real host modules or are thin path-translating wrappers over real host APIs. This violates the virtualization model. + +The original JS kernel (`@secure-exec/core` + `@secure-exec/nodejs`, deleted in commit `5a43882`) had full kernel-backed polyfills for `fs`, `net`, `http`, `dns`, `dgram`, `child_process`, and `os` using SharedArrayBuffer RPC and a kernel socket table. The work here is **porting those proven patterns** to the Rust sidecar, not designing from scratch. + +## P0: Remove dangerous builtins from DEFAULT_ALLOWED_NODE_BUILTINS + +**This is the single highest-value change. Zero effort, immediate security fix.** + +`packages/core/src/sidecar/native-kernel-proxy.ts` sets `DEFAULT_ALLOWED_NODE_BUILTINS` to include everything. Every builtin without a kernel polyfill falls through to the real host module. + +- [ ] Remove `dgram`, `dns`, `http`, `http2`, `https`, `net`, `tls`, `vm`, `worker_threads`, `inspector`, `v8` from `DEFAULT_ALLOWED_NODE_BUILTINS`. Only keep builtins that have kernel-backed polyfills. +- [ ] Add `os`, `cluster`, `diagnostics_channel` to `DENIED_BUILTINS`. `node:os` leaks host info (hostname, CPUs, memory, network interfaces). +- [ ] Make `ALLOWED_NODE_BUILTINS` configurable from `AgentOsOptions` (currently hardcoded). +- [ ] Fix `--allow-worker` inconsistency: currently always passed at `--permission` level even when `worker_threads` is denied at the loader level. + +## P0: Pyodide sandbox escapes + +### `import js` exposes all JS globals to Python (CRITICAL) +Python code can `import js` and access `js.process.env`, `js.process.kill()`, `js.require`, and any other JS global. Full sandbox escape. + +- [ ] Block or proxy `js` and `pyodide_js` FFI modules so Python code cannot reach raw JS globals. + +### Node.js `--permission` disabled for Python (CRITICAL) +`python.rs:622` sets `enable_permissions=false`. The `--permission` flag is not applied to the Pyodide host process. + +- [ ] Enable `--permission` for the Python runtime's host Node.js process. + +## P0: Port kernel-backed polyfills from original JS kernel + +These builtins need kernel-backed polyfills ported from the original `@secure-exec/nodejs` patterns. The Rust kernel already has the VFS, process table, and pipe manager. The missing piece is the JS polyfill layer + RPC bridge (SharedArrayBuffer for sync calls, same pattern the Pyodide VFS bridge already uses). + +### `fs` / `fs/promises` — port kernel VFS polyfill +Currently: `wrapFsModule()` translates paths then calls real `node:fs` (real host syscalls). Must route through kernel VFS via RPC instead. + +- [ ] Replace `wrapFsModule` with kernel VFS polyfill using SharedArrayBuffer RPC for sync methods +- [ ] Async `fs.promises.*` methods: IPC message to sidecar kernel (straightforward, ~20 methods with direct kernel counterparts) +- [ ] Sync methods (`readFileSync`, etc.): SharedArrayBuffer + `Atomics.wait` bridge (proven pattern from Pyodide VFS bridge) +- [ ] Fd-based operations (`fs.open` → `kernel.fd_open`, `fs.read(fd)` → `kernel.fd_read`, etc.) +- [ ] Streams (`createReadStream`/`createWriteStream`): reimplement using polyfilled fd operations +- [ ] `fs.watch`/`fs.watchFile`: kernel has no file-watching API — stub or add kernel-side support + +### `child_process` — port kernel process table polyfill +Currently: `wrapChildProcessModule()` translates paths but spawns real host processes. Must route through kernel process table. + +- [ ] Replace with polyfill that routes `spawn`/`exec`/`execFile` through `kernel.spawn_process()` +- [ ] Build synthetic `ChildProcess` EventEmitter backed by kernel pipe fds for stdio +- [ ] Wire `waitpid` for exit/close events, `kill_process` for `.kill()` +- [ ] **Fix `exec`/`execSync` bypass**: currently passed through with zero interception — no path translation, no `--permission` injection. Guest can run `execSync('cat /etc/passwd')` on the host unmodified. + +### `net` — port kernel socket table polyfill +Currently: no wrapper, falls through to real `node:net`. The kernel has a socket table and `HostNetworkAdapter` for external connections. The original JS kernel had `kernel.socketTable.create/connect/send/recv`. + +- [ ] Polyfill `net.Socket` as a Duplex stream backed by kernel socket table operations via RPC +- [ ] Polyfill `net.createServer` backed by kernel socket `listen`/`accept` +- [ ] Loopback connections stay in-kernel; external connections go through `HostNetworkAdapter` + +### `dgram` — port kernel UDP polyfill +- [ ] Polyfill `dgram.createSocket()` routed through kernel socket table + +### `dns` — port kernel DNS resolver polyfill +- [ ] Polyfill `dns.resolve*()` and `dns.lookup()` routed through kernel DNS resolver +- [ ] Note: `dns.lookup()` uses libuv's `getaddrinfo` internally, not `net` — needs its own interception regardless of `net` polyfill + +### `http` / `https` / `http2` — builds on `net` + `tls` polyfills +- [ ] Investigate: can real `node:http` use the polyfilled `net` module (loader hooks intercept `require('net')` inside `http` internals)? If yes, these may work automatically once `net` is polyfilled. +- [ ] If not: polyfill `http.request`/`http.get` directly as kernel-level fetch-style RPC calls (covers 95% of use cases without full streaming) + +### `tls` — port kernel TLS polyfill +- [ ] Polyfill TLS socket creation routed through kernel networking + +### `os` — polyfill with kernel-provided values (easy, ~100 lines) +- [ ] Return kernel hostname, configured CPU/memory values, etc. instead of real host info + +### Builtins that must stay permanently denied +- [ ] **`vm`** — Creates V8 contexts without loader hooks. Must stay denied. +- [ ] **`worker_threads`** — Workers may not inherit loader hooks. Must stay denied. +- [ ] **`inspector`** — V8 debugger access. Must stay permanently denied. +- [ ] **`v8`** — Exposes heap internals. Must stay permanently denied. + +### Safe builtins (no polyfill needed) +These are pure computation with no host I/O — safe to leave as real Node.js modules: +`stream`, `events`, `buffer`, `crypto`, `path`, `util`, `zlib`, `string_decoder`, `querystring`, `url`, `assert`, `timers`, `console` + +### Native addons (.node files) +Native addons are shared objects loaded via `process.dlopen()` — arbitrary native code on the host. Cannot be sandboxed. +- [ ] Deny native addon loading by intercepting `process.dlopen` and `Module._extensions['.node']`. + +### `process` global leaks host state +The `process` global is not virtualized. Multiple properties expose real host information: +- [ ] **`process.env`** — leaks all `AGENT_OS_*` internal env vars to guest. `AGENT_OS_GUEST_PATH_MAPPINGS` reveals real host paths where guest dirs are mapped. `AGENT_OS_NODE_IMPORT_CACHE_PATH` reveals host temp directory paths. Scrub `AGENT_OS_*` keys from guest-visible `process.env`. +- [ ] **`process.cwd()`** — returns real host path (e.g., `/tmp/agent-os-xxx/workspace`), not the guest's virtual path (e.g., `/root`). Must be virtualized to return the kernel CWD. +- [ ] **`process.execPath` / `process.argv[0]`** — exposes real host Node.js binary path (e.g., `/usr/local/bin/node`). Must be replaced with a virtual value. +- [ ] **`process.pid` / `process.ppid`** — returns real host OS PIDs. `process.ppid` leaks the sidecar's PID. Must be virtualized to return kernel PIDs. +- [ ] **`process.on('SIGINT'/'SIGTERM'/...)`** — guest can register signal handlers that prevent the sidecar from cleanly terminating the process. Must intercept `process.on()`/`process.once()` for signal events. +- [ ] **`process.chdir()`** — changes the real host CWD. Must be intercepted and routed through kernel. +- [ ] **`process.getuid()` / `process.getgid()`** — returns real host user IDs. Must be virtualized. + +### `node:module` not denied — module resolution manipulation +`node:module` is not in DENIED_BUILTINS. Guest can `import { createRequire, Module } from 'node:module'` and access `Module._cache`, `Module._resolveFilename`, `Module._extensions` directly — bypassing the `_load` hook, probing host filesystem via `_resolveFilename`, and poisoning the module cache. +- [ ] Add `module` to DENIED_BUILTINS, or wrap it to remove dangerous APIs. + +### `node:trace_events` not denied +Provides V8 tracing access. Not in DENIED_BUILTINS. +- [ ] Add `trace_events` to DENIED_BUILTINS. + +### Host paths leak through errors and `require.resolve()` +- [ ] **`require.resolve()`** — returns real host filesystem paths (e.g., `/tmp/agent-os-node-import-cache-1/...`). Must translate resolved paths back to guest-visible paths. +- [ ] **Error messages / stack traces** — module-not-found errors, loader errors, etc. contain real host paths. Must scrub or translate host paths in error messages before they reach guest code. + +### Loader metrics prefix injectable via guest stderr +Guest code can write `__AGENT_OS_NODE_IMPORT_CACHE_METRICS__:` to stderr to confuse the sidecar's metrics parsing (same class of issue as Pyodide exit code injection). +- [ ] Include in the side-channel fix for control messages. + +## P1: Pyodide runtime gaps + +### No `Drop` impl on `PythonExecution` +Orphaned Node+Pyodide processes (~200MB+ each) leak if caller drops without calling `wait()`. +- [ ] Implement `Drop` for `PythonExecution` that kills the child process. + +### `wait()` has no timeout +Infinite hang on runaway Python code. No cancel mechanism. +- [ ] Add timeout parameter to Python `wait()`. +- [ ] Add a `cancel()`/`kill()` method for in-flight Python executions. + +### No VFS RPC path validation +Python code can read/write any kernel VFS path. `service.rs:2394-2470` passes `request.path` directly to kernel. +- [ ] Scope VFS RPC operations to the guest's cwd or apply kernel permission checks. + +### No `spawn_waiter` thread +Exit detection relies on fragile stderr parsing + `try_wait()` polling. Ungraceful deaths detected late. +- [ ] Add dedicated `spawn_waiter` thread matching JS/WASM pattern. + +### Unbounded stdout/stderr buffering in `wait()` +All output accumulated in memory with no cap. OOM on large output. +- [ ] Cap buffer sizes or stream instead of accumulating. Use bounded mpsc channels. + +### VFS RPC sync bridge can deadlock +`readSync()` blocks forever if Rust side never responds. +- [ ] Add timeout to synchronous VFS RPC bridge calls. + +## P1: `options.permissions` not wired through + +The TypeScript `AgentOsOptions.permissions` field is accepted but never consumed. The `LocalBridge` allows everything. The protocol has `PermissionDescriptor` on the Rust side but TS always sends an empty array. + +- [ ] Wire `options.permissions` through to the sidecar bridge. +- [ ] Stop defaulting to `allowAll` in `LocalBridge`. + +## P1: CWD passed directly as host filesystem path + +`service.rs:2195-2206` uses the `Execute` request's `cwd` as the real host `current_dir()` AND adds it to `--allow-fs-read`/`--allow-fs-write`. No validation. Setting `cwd=/` grants host-wide access. + +- [ ] Validate that the execution CWD is within the configured sandbox root. + +## P1: `exec`/`execSync` bypass all child_process wrapping + +`wrapChildProcessModule` passes `exec`/`execSync` through as bare `.bind()` calls — no path translation, no `--permission` injection. Guest code calling `child_process.execSync('cat /etc/passwd')` executes on the host unmodified. + +- [ ] Wrap `exec`/`execSync` with the same interception as `spawn`/`execFile`. + +## P1: Shared import cache enables cross-VM cache poisoning + +`flushCacheState()` reads/merges/writes a shared on-disk cache. If two VMs share the same cache root, VM-A can write a poisoned resolution entry that VM-B picks up. `validateResolutionEntry` only checks file existence, not trust. + +- [ ] Use per-VM cache paths, or validate that resolved files are within trusted locations. + +## P1: `prependNodePermissionArgs` unconditionally passes `--allow-child-process` + +When spawning child Node processes, the wrapper injects `--allow-child-process` and `--allow-worker` unconditionally. Every child of a guest process gets full child_process/worker permissions, enabling recursive escalation. + +- [ ] Only pass `--allow-child-process` and `--allow-worker` if the parent was explicitly granted those permissions. + +## P2: Kernel permission model gaps + +### Permission bypass via symlinks (HIGH) +`PermissionedFileSystem` checks on caller-supplied path, then inner filesystem resolves symlinks independently. Only exploitable if mounts expose host paths. +- [ ] Resolve symlinks before permission checks, or check both raw and resolved paths. + +### `link()` only checks destination permission (MEDIUM) +- [ ] Check permissions on both source and destination for `link()`. + +### Symlinks can cross mount boundaries (HIGH) +`MountTable` enforces `EXDEV` for rename/link but not symlink. +- [ ] Enforce mount boundary checks for symlink targets. + +### `exists()` bypasses EACCES (LOW) +When permission check returns EACCES, `exists()` falls through — leaks file existence. +- [ ] Return `false` on EACCES instead of falling through. + +## P2: Process isolation gaps + +### Host PID reuse in `signal_runtime_process` (HIGH) +Sidecar sends real `kill(2)` to host PIDs. PID reuse could kill wrong host process. +- [ ] Check child liveness before signaling. +- [ ] Whitelist allowed signals to `SIGTERM`/`SIGKILL`/`SIGINT`/`SIGCONT`/signal-0. + +### PTY foreground PGID manipulation (MEDIUM) +Guest with PTY master FD can redirect signals to arbitrary process groups (guest-to-guest within same VM). +- [ ] Validate target PGID belongs to same session. + +### `dup2` skips FD bounds check (MEDIUM) +- [ ] Validate `new_fd < MAX_FDS_PER_PROCESS` in `dup2` and `open_with`. + +## P2: Resource exhaustion / DoS + +### No filesystem total size limit (HIGH — guest-exploitable) +All file data in-memory with no cap. Guest writes to OOM. +- [ ] Add `max_filesystem_bytes` and `max_inode_count` to `ResourceLimits`. + +### `truncate` / `pwrite` with large values cause OOM (MEDIUM) +- [ ] Validate against filesystem size limits before resizing. + +### `read_frame` pre-validation OOM (MEDIUM) +`stdio.rs` allocates from 4-byte prefix before checking `max_frame_bytes`. Reachable only from local socket (trusted caller), but trivial fix. +- [ ] Check `declared_len` against `max_frame_bytes` before allocation. + +### No WASM fuel/memory/stack limits (MEDIUM) +- [ ] Add execution fuel limits and memory growth caps. + +### `pipe.read()` / `pty.read()` block forever if write end leaks (MEDIUM) +- [ ] Add timeout to pipe/PTY read operations. + +### No socket/connection resource limits (MEDIUM) +- [ ] Add socket count and connection limits to `ResourceLimits`. + +## P2: Pyodide-specific + +### Exit code injection via stderr magic prefix (MEDIUM) +Guest can write `__AGENT_OS_PYTHON_EXIT__:0` to fake exit. +- [ ] Use side channel for control messages instead of in-band stderr parsing. + +### Hardening runs AFTER `loadPyodide()` (MEDIUM) +Pyodide may cache references to dangerous APIs before hardening runs. +- [ ] Run hardening before `loadPyodide()`. + +### Unbounded VFS RPC request queue (MEDIUM) +- [ ] Add bounded queue or rate limiting. + +### Missing Pyodide tests +- [ ] Test frozen time — Phase 1 AC 1.4 +- [ ] Test `node:child_process`/`node:vm` inaccessibility — Phase 1 AC 1.5 +- [ ] Test zero network requests during init — Phase 1 AC 1.6 +- [ ] Test kill (SIGTERM) — Phase 1 AC 1.7 +- [ ] Test concurrent executions — Phase 1 AC 1.8 +- [ ] Test cross-runtime file visibility — Phase 3 AC 3.5 + +## P2: Missing security infrastructure + +### No security audit logging +Auth failures, permission denials, mount operations, kill-process calls — none are logged. +- [ ] Add structured security event logging for auth failures, permission denials, mount/unmount, process kills. + +### Google Drive plugin SSRF via `token_url` and `api_base_url` +Mount config accepts arbitrary URLs. Can point `token_url` at internal services to exfiltrate JWTs. +- [ ] Validate URLs against expected hosts. + +### S3 plugin SSRF via `endpoint` +S3 mount config accepts arbitrary endpoint URL. Can reach cloud metadata. +- [ ] Validate endpoint against private IP ranges. + +### `mount_filesystem` has no permission checks +`kernel.rs` mount functions only check `assert_not_terminated()`. No path or caller validation. +- [ ] Add permission checks on mount operations. + +## P3: Kernel correctness + +### `host_dir` mount TOCTOU in path resolution (MEDIUM) +`fs::canonicalize()` + `ensure_within_root()` has race window for symlink swap. +- [ ] Use `O_NOFOLLOW`/`openat`-style resolution. + +### `setpgid` allows cross-driver group joining (MEDIUM) +- [ ] Validate target PGID's owning driver matches requester. + +### Poisoned mutex / `.expect()` inconsistency (MEDIUM) +`lock_or_recover()` in some modules, `.expect()` in others. +- [ ] Decide on single poison policy and apply consistently. + +### `hardenProperty` falls back to mutable assignment (LOW) +- [ ] Throw instead of falling back. + +### Signal/exit control messages via stderr (LOW) +Guest can emit magic prefixes on stderr to influence sidecar state. +- [ ] Use side channel for control messages. + +### Zombie reaper loses exit codes (LOW) +- [ ] Don't reap zombies with living parent that hasn't called `waitpid`. + +## P3: WASM permission tiers not enforced + +- [ ] Restrict WASI preopens based on declared permission tier. +- [ ] Only provide `host_process` imports to `full` tier commands. + +## P3: Pyodide code quality + +- [ ] ~870 lines embedded JS — extract to `.js` file loaded at build time. +- [ ] ~300 lines duplicated across `python.rs`/`wasm.rs`/`javascript.rs` — extract shared code. +- [ ] `@rivet-dev/agent-os-python-packages` registry package not created. +- [ ] Cold/warm start times not documented. +- [ ] `NodeImportCache` temp directories never cleaned up on crash. + +## P3: Low-priority robustness + +- [ ] `read_dir` linear scan — use tree structure for directory children lookup. +- [ ] `collect_snapshot_entries` unbounded recursion — add depth limit or iterate. +- [ ] `nlink` underflow — use `saturating_sub`. +- [ ] `allocate_fd` potential infinite loop — bounded scan. +- [ ] SQLite WASM VFS deterministic randomness — wire to `random_get`. +- [ ] WASM FFI `poll()` buffer validation, `getpwuid` buffer trust, `usize`→`u32` truncation. +- [ ] SQL buffer overflow in `sqlite3_cli.c` (WASM-contained). diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9d635d47a..d6816564f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -21,3 +21,5 @@ jobs: - run: pnpm check-types - run: pnpm build - run: pnpm test + env: + AGENTOS_E2E_NETWORK: '1' diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 000000000..681311eb9 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index f5564cc35..cf7caf986 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,21 +4,97 @@ A high-level wrapper around the Agent OS runtime that provides a clean API for r ## Agent OS Runtime -Agent OS uses a native kernel sidecar written in Rust. All guest code runs inside the sidecar's isolation boundary — nothing executes as an unsandboxed host process. The kernel orchestrates three execution environments: +Agent OS is a **fully virtualized operating system**. The kernel, written as a Rust sidecar, provides a complete POSIX-like environment — virtual filesystem, process table, socket table, pipe/PTY management, and permission system. Guest code sees a self-contained OS and must never interact with the host directly. Every system call (file I/O, networking, process spawning, DNS resolution) must be mediated by the kernel. No guest operation may fall through to a real host syscall. + +The kernel orchestrates three execution environments: - **WASM processes** — A custom libc and Rust toolchain compile a full suite of POSIX utilities (coreutils, sh, grep, etc.) to WebAssembly. All WASM execution happens within the sidecar's managed runtime. -- **Node.js (V8 isolates)** — A sandboxed reimplementation of Node.js APIs (`child_process`, `fs`, `net`, etc.) runs JS/TS inside isolated V8 contexts. Module loading is hijacked to route through the kernel VFS. This is how agent code runs. +- **Node.js** — JS/TS runs inside Node.js child processes with ESM loader hooks that intercept `require()`/`import` for builtins. Every Node.js builtin module that guest code can import must be a **kernel-backed polyfill** — never the real Node.js module. See "Node.js Isolation Model" below. - **Python (Pyodide)** — CPython compiled to WASM via Pyodide, running within the sidecar with kernel-backed file/network I/O. All runtimes are managed by the sidecar's execution engines and kernel process table. Processes can spawn children across runtimes (e.g., a Node process can spawn a WASM shell). Guest code must never escape the sidecar's isolation boundary to run on the host. +### Virtualization invariants + +These are hard rules with no exceptions: + +1. **Every guest syscall goes through the kernel.** File reads go through the kernel VFS, not real `node:fs`. Network connections go through the kernel socket table, not real `node:net`. Process spawning goes through the kernel process table, not real `node:child_process`. DNS resolution goes through the kernel's DNS resolver, not real `node:dns`. There are no shortcuts where guest code touches host resources directly. +2. **No real host builtins.** When a guest does `require('fs')` or `import net from 'node:net'`, the module loader must return a kernel-backed polyfill. If no polyfill exists yet, the builtin must be denied (`ERR_ACCESS_DENIED`). The loader must never fall through to Node.js's default resolution and hand the guest the real host module. +3. **The host is an implementation detail.** Guest code should not be able to observe that it is running on a host Node.js process. `process.pid` should be the kernel PID, `os.hostname()` should be the kernel hostname, `fs.readdirSync('/')` should show the kernel VFS root. `process.cwd()` should return the kernel CWD, not a host path. `process.env` must not contain internal `AGENT_OS_*` control variables. Error messages and stack traces must not reveal host filesystem paths. `require.resolve()` must return guest-visible paths, not host paths. Any host state leaking through to the guest is a bug. +4. **Polyfills are ports, not wrappers.** A path-translating shim over real `node:fs` is not a polyfill — it is a wrapper around a host API. A real polyfill implements the API semantics using only kernel primitives (VFS, socket table, process table, pipe manager). The original JS kernel (`@secure-exec/core` + `@secure-exec/nodejs`, deleted in commit `5a43882`) had full kernel-backed polyfills for `fs`, `net`, `http`, `dns`, `dgram`, `child_process`, and `os`. The Rust sidecar must reach the same level of isolation. +5. **Control channels must be out-of-band.** The sidecar must not use in-band magic prefixes on stdout/stderr for control signaling (exit codes, metrics, signal registration). Guest code can write these prefixes to inject fake control messages. Use dedicated file descriptors, separate pipes, or a side-channel protocol for all sidecar-internal communication. +6. **Resource consumption must be bounded.** Every guest-allocatable resource must have a configurable limit enforced by the kernel: filesystem total size, inode count, process count, open FDs, pipes, PTYs, sockets, connections. Unbounded allocation from guest input is a DoS vector. The kernel's `ResourceLimits` must cover all resource types, not just processes and FDs. + Sidecar metadata parsing should start from `ResourceLimits::default()` and only override keys that are actually present; rebuilding the struct from sparse metadata drops default filesystem byte/inode caps. + Per-operation memory guards also live in `ResourceLimits`: bound `pread`, `fd_write`/`fd_pwrite`, merged spawn `argv`/`env`, and `readdir` batches in `crates/kernel/src/kernel.rs`, and keep the matching `resource.max_*` metadata keys in `crates/sidecar/src/service.rs` in sync so the limits remain configurable. + WASM runtime caps are also carried through `ResourceLimits`: `crates/sidecar/src/service.rs` maps the configured `max_wasm_*` fields into reserved `AGENT_OS_WASM_*` env keys, and `crates/execution/src/wasm.rs` is responsible for enforcing the resulting fuel/memory/stack limits before guest code runs. + WebAssembly parser hardening in `crates/execution/src/wasm.rs` must stat module files before `fs::read()`, cap import/memory section entry counts before iterating them, and bound varuint encodings by byte length so malformed or oversized modules fail closed without parser DoS. +7. **Permission checks must use resolved paths.** Whenever the kernel checks permissions on a path, it must resolve symlinks first and check the resolved path. Checking the caller-supplied path and then operating on a symlink-resolved target is a TOCTOU bypass. Similarly, `link()` must check permissions on both source and destination. +8. **The VM must behave like a standard Linux environment.** Agents are written to target Linux. The kernel should implement POSIX semantics faithfully — correct `errno` values, proper signal delivery, standard `/proc` layout, expected filesystem behavior. Deviations from standard Linux behavior cause agent failures and must be documented in the friction log (`.agent/notes/vm-friction.md`). When in doubt, match Linux kernel behavior, not a simplified model. + ### Key subsystems - **Virtual filesystem (VFS)** — Layered chunked architecture: `ChunkedVFS` composes `FsMetadataStore` (directory tree, inodes, chunk mapping) + `FsBlockStore` (key-value blob store) into a `VirtualFileSystem`. Tiered storage keeps small files inline in metadata; larger files are split into chunks in the block store. The device layer (`/dev/null`, `/dev/urandom`, `/dev/pts/*`, etc.), proc layer (`/proc/[pid]/*`), and permission wrapper sit on top. All layers implement the `VirtualFileSystem` interface with full POSIX semantics. - **Process management** — Kernel-wide process table tracks PIDs across all runtimes. Full POSIX process model: parent/child relationships, process groups, sessions, signals (SIGCHLD, SIGTERM, SIGWINCH), zombie cleanup, and `waitpid`. Each process gets its own FD table (0-255) with refcounted file descriptions supporting dup/dup2. + Advisory `flock` state should stay kernel-global but be owned by the shared open-file-description (`FileDescription.id()`), keyed by the opened file identity, and released only when the last refcounted FD closes; dup/fork inheritance must see the same lock while separate opens still conflict. + Per-FD status bits such as `O_NONBLOCK` belong on `FdEntry` / `ProcessFdTable`, while shared `FileDescription.flags()` should stay limited to open-file-description semantics such as access mode and `O_APPEND`; `/dev/fd/N` duplication can layer new per-FD flags without mutating the shared description. + Host-side liveness probes that must not reap runtime children should use `waitid(..., WNOWAIT | WNOHANG | WEXITED | WSTOPPED | WCONTINUED)` rather than `waitpid`; the sidecar uses that non-reaping check before signaling host child PIDs to avoid PID-reuse races. + Parent-aware `waitpid` state tracking belongs in `crates/kernel/src/process_table.rs`: queue stop/continue notifications there, and only let `crates/kernel/src/kernel.rs` clean up process resources after an exited child is actually reaped. + Process exit handling in `crates/kernel/src/process_table.rs` has to keep child reparenting, orphaned stopped-process-group `SIGHUP`/`SIGCONT` delivery, and zombie-aware `max_processes` accounting aligned; changing only one of those paths breaks Linux-style lifecycle semantics. + POSIX signal side effects that depend on the calling PID should stay at `KernelVm` syscall entrypoints instead of low-level primitives: `PipeManager` only reports broken-pipe `EPIPE`, while `crates/kernel/src/kernel.rs` `fd_write` is responsible for turning that into guest-visible `SIGPIPE` delivery. + Job-control signal state transitions should stay aligned across `crates/kernel/src/process_table.rs` and `crates/kernel/src/kernel.rs`: `ProcessTable::kill(...)` owns `SIGSTOP`/`SIGTSTP`/`SIGCONT` status changes and `waitpid` notifications, while PTY resize should emit `SIGWINCH` from the `KernelVm` entrypoint after the PTY layer reports the foreground process group. - **Pipes & PTYs** — Kernel-managed pipes (64KB buffers) enable cross-runtime IPC. PTY master/slave pairs with line discipline support interactive shells. `openShell()` allocates a PTY and spawns sh/bash. - **Networking** — Socket table manages TCP/UDP/Unix domain sockets. Loopback connections stay entirely in-kernel. External connections delegate to a `HostNetworkAdapter` (implemented via `node:net`/`node:dgram` on the host). DNS resolution also goes through the adapter. -- **Permissions** — Deny-by-default access control. Four permission domains: `fs`, `network`, `childProcess`, `env`. Each is a function that returns `{allow, reason}`. The `allowAll` preset grants everything (used in agentOS). +- **Permissions** — Deny-by-default access control. Four permission domains: `fs`, `network`, `childProcess`, `env`. Each is a function that returns `{allow, reason}`. The `allowAll` preset grants everything (used in agentOS). See "Node.js Builtin Permission Model" for how these interact with the Node.js builtin interception layer. +- **Kernel VM configs must opt into broad access explicitly.** `KernelVmConfig::new()` should stay deny-all by default; tests, browser scaffolds, or other callers that need unrestricted behavior must set `config.permissions = Permissions::allow_all()` themselves. +- **Sensitive mount policy is a separate filesystem capability.** Kernel mount APIs check normal `fs.write` permission on the mount path, and mounts targeting `/`, `/etc`, or `/proc` also require `fs.mount_sensitive`. In the Rust sidecar, `configure_vm` reconciles mounts before it applies `payload.permissions`, so mount-time policy must already be present on the VM (or be injected directly in tests) before `ConfigureVm` runs. + +### Node.js Isolation Model + +**Current state (KNOWN DEFICIENT — see `.agent/todo/node-isolation-gaps.md`):** + +Guest Node.js code currently runs as **real host Node.js child processes** spawned via `std::process::Command::new("node")` in the Rust sidecar (`crates/execution/src/javascript.rs`). The ESM loader hooks intercept `require()`/`import` but most builtins either fall through to the real host module or are thin wrappers that call real host APIs. This violates the virtualization invariants above. + +**Prior art — the original JS kernel had full polyfills:** + +Before the Rust sidecar (commit `5a43882`), the JS kernel (`@secure-exec/core` + `@secure-exec/nodejs` + `packages/posix/`) had complete kernel-backed polyfills for all builtins. The pattern was: +- **Kernel socket table** — `kernel.socketTable.create/connect/send/recv` managed all TCP/UDP. Loopback stayed in-kernel; external connections went through a `HostNetworkAdapter`. +- **Kernel VFS** — All `fs` operations routed through the kernel VFS via syscall RPC. +- **Kernel process table** — `child_process.spawn` routed through `kernel.spawn()`. +- **SharedArrayBuffer RPC** — Synchronous syscalls from worker threads used `Atomics.wait` + shared memory buffers (same pattern the Pyodide VFS bridge uses today). +- **Module hijacking** — `require('net')` returned the kernel-backed socket implementation, not real `node:net`. + +The Rust sidecar kernel already has the VFS, process table, pipe manager, PTY manager, and permission system. What's missing is porting the **polyfill layer** — the code that makes `require('fs')` return a kernel-backed implementation instead of real `node:fs`. This is a port of proven patterns, not a greenfield design. + +**Current reality vs required state:** + +| Builtin | Required | Current | Gap | +|---------|----------|---------|-----| +| `fs` / `fs/promises` | Kernel VFS polyfill | Path-translating wrapper over real `node:fs` | Port: route through kernel VFS via RPC | +| `child_process` | Kernel process table polyfill | Path-translating wrapper over real `node:child_process` | Port: route through kernel process table | +| `net` | Kernel socket table polyfill | **No wrapper — falls through to real `node:net`** | Port: kernel socket table polyfill | +| `dgram` | Kernel socket table polyfill | **No wrapper — falls through to real `node:dgram`** | Port: kernel socket table polyfill | +| `dns` | Kernel DNS resolver polyfill | **No wrapper — falls through to real `node:dns`** | Port: kernel DNS resolver polyfill | +| `http` / `https` / `http2` | Built on kernel `net` polyfill | **No wrapper — falls through to real module** | Port: builds on `net` polyfill | +| `tls` | Kernel TLS polyfill | Guest-owned polyfill in `node_import_cache.rs` wraps the existing guest `net` transport with host TLS state (`tls.connect({ socket })`, `new TLSSocket(socket, { isServer: true, ... })`) | Keep client/server entrypoints on guest sockets and avoid direct host `node:tls` listeners/connections | +| `os` | Kernel-provided values | Guest-owned polyfill in `node_import_cache.rs` virtualizes hostname, CPU, memory, loopback networking, home, and user info | Keep future `os` additions aligned with VM defaults and kernel-backed resource config | +| `vm` | Must be denied | **No wrapper — falls through to real `node:vm`** | Must stay denied | +| `worker_threads` | Must be denied | **No wrapper — falls through to real module** | Must stay denied | +| `inspector` | Must be denied | **No wrapper — falls through to real module** | Must stay denied | +| `v8` | Must be denied | **No wrapper — falls through to real module** | Must stay denied | + +**How the loader interception works** (`crates/execution/src/node_import_cache.rs`): + +ESM loader hooks (`loader.mjs`) and CJS `Module._load` patches (`runner.mjs`) are generated from Rust string templates. Every `import`/`require` is intercepted: +1. `resolveBuiltinAsset()` — checks `BUILTIN_ASSETS` list. Redirects to a kernel-backed polyfill file. +2. `resolveDeniedBuiltin()` — checks `DENIED_BUILTINS` set. Redirects to a stub that throws `ERR_ACCESS_DENIED`. A builtin is in `DENIED_BUILTINS` only if it is NOT in `ALLOWED_BUILTINS`. +3. **Fall through to `nextResolve()`** — Node.js default resolution. Returns the real host module. **This must never happen for any builtin that guest code can import.** + +`AGENT_OS_ALLOWED_NODE_BUILTINS` (JSON string array env var) controls which builtins are removed from the deny list. `DEFAULT_ALLOWED_NODE_BUILTINS` in `packages/core/src/sidecar/native-kernel-proxy.ts` currently includes all builtins — this must be reduced to only builtins that have kernel-backed polyfills. + +**Additional hardening layers (defense-in-depth, NOT primary isolation):** +1. **`globalThis.fetch` hardening** — Replaced with `restrictedFetch` (loopback-only on exempt ports). Does NOT cover `http.request()`, `net.connect()`, or `dgram.createSocket()`. +2. **Node.js `--permission` flag** — OS-level backstop for filesystem and child_process only. No network restrictions. This is a safety net, not the isolation boundary. +3. **Guest env stripping** — `NODE_OPTIONS`, `LD_PRELOAD`, `DYLD_INSERT_LIBRARIES`, `LD_LIBRARY_PATH` stripped before spawn. ### What agentOS adds on top @@ -67,13 +143,51 @@ The registry software packages depend on `@rivet-dev/agent-os-registry-types` (i - **The default VM filesystem model should be Docker-like.** The root filesystem should be a layered overlay view with one writable upper layer on top of one or more immutable lower snapshot layers. The base filesystem artifact is the initial lower layer; additional frozen lower layers may be stacked beneath the writable upper if needed. Do not design the default VM root as a pile of ad hoc post-boot mutations. - **Everything runs inside the VM.** Agent processes, servers, network requests -- all spawned inside the Agent OS kernel, never on the host. This is a hard rule with no exceptions. - **All guest code must execute within the kernel's isolation boundary (WASM or in-kernel isolate).** No runtime may escape to a host-native process. If a language runtime requires a JavaScript host (e.g., Emscripten-compiled WASM like Pyodide), the JS host must itself run inside the kernel — not as a host-side Node.js subprocess. Spawning an unsandboxed host process to run guest code is never acceptable, even as a convenience shortcut. New runtimes must either compile to WASI (so they run in the kernel's WASM engine directly) or run inside an already-sandboxed in-kernel isolate. +- **Guest code must never touch real host APIs.** Every `require('fs')`, `require('net')`, `require('child_process')`, `require('dns')`, `require('dgram')`, `require('http')`, etc. must return a kernel-backed polyfill that routes operations through the kernel's VFS, socket table, process table, and DNS resolver respectively. Path-translating wrappers over real `node:fs` or real `node:child_process` are NOT acceptable — they call real host syscalls. The original JS kernel had full polyfills for all of these; the Rust sidecar must match that level of isolation. If a polyfill does not exist yet for a builtin, that builtin must be denied at the loader level until one is built. +- **Native sidecar permission policy has to be available during `create_vm`, not just `configure_vm`.** Guest env filtering and kernel bootstrap driver registration happen while the VM is being constructed, so `AgentOsOptions.permissions` must be serialized into the `CreateVmRequest`; `configure_vm` can only mirror or refine that policy after the fact. +- **Permissioned Pyodide host launches still need `--allow-worker`.** `crates/execution/src/python.rs` bootstraps through Node's internal ESM loader worker, so the host process must keep `--allow-worker` enabled even while guest `worker_threads` stays denied. +- **WASM permission tiers must gate host Node WASI access as well as guest-side preopens.** In `crates/execution/src/wasm.rs`, keep `Isolated` executions off `--allow-wasi` entirely, and let `ReadOnly` / `ReadWrite` / `Full` differentiate the read/write scope through the guest WASI layer rather than a blanket host flag. - **`sandbox_agent` mounts on `sandbox-agent@0.4.2` only get basic file endpoints (`entries`, `file`, `mkdir`, `move`, `stat`) from the HTTP fs API.** When the sidecar needs symlink/readlink/realpath/link/chmod/chown/utimes semantics, it must use the remote process API as a fallback and return `ENOSYS` when that helper path is unavailable. - The `AgentOs` class wraps the kernel and proxies its API directly - **All public methods on AgentOs must accept and return JSON-serializable data.** No object references (Session, ManagedProcess, ShellHandle) in the public API. Reference resources by ID (session ID, PID, shell ID). This keeps the API flat and portable across serialization boundaries (HTTP, RPC, IPC). - Filesystem methods mirror the kernel API 1:1 (readFile, writeFile, mkdir, readdir, stat, exists, move, delete) +- **Per-process filesystem state such as `umask` belongs in `ProcessContext` / `ProcessTable`.** Kernel create/write entrypoints should read it there, and any guest Node exposure must be threaded through the JavaScript sync-RPC bridge (`crates/sidecar/src/service.rs` and `crates/execution/src/node_import_cache.rs`) instead of inheriting host `process` behavior. +- **`VirtualStat` additions must be propagated end-to-end.** When stat grows new fields, update kernel-backed storage stats, synthetic `/proc` and `/dev` stats, sidecar mount/plugin conversions, sidecar protocol serialization, and the TypeScript `VirtualStat` / `GuestFilesystemStat` adapters together or some callers will silently keep incomplete metadata. - **readdir returns `.` and `..` entries** — always filter them when iterating children to avoid infinite recursion +- Guest Node `fs` and `fs/promises` polyfills share the JavaScript sync-RPC transport between `crates/execution/src/node_import_cache.rs` and `crates/sidecar/src/service.rs`; Node-facing `readdir` results must filter `.`/`..`, async methods should dispatch under `fs.promises.*`, fd-based APIs (`open`, `read`, `write`, `close`, `fstat`) plus `createReadStream`/`createWriteStream` should ride the same bridge, and runner-internal pipe/control writes must keep snapped host `node:fs` bindings because `syncBuiltinModuleExports(...)` mutates the builtin module for guests. +- JavaScript sync RPC timeouts and slow-reader backpressure should be enforced in `crates/execution/src/javascript.rs`, not in the generated runner: track the pending request ID on the host, auto-emit `ERR_AGENT_OS_NODE_SYNC_RPC_TIMEOUT` after the configured wait, queue replies through a bounded async writer so slow guest reads cannot block the sidecar thread, and have `crates/sidecar/src/service.rs` ignore stale `sync RPC request ... is no longer pending` races after the timeout fires. +- Execution-host runner scripts that are materialized by `NodeImportCache` should live as checked-in assets under `crates/execution/assets/runners/` and be loaded via `include_str!`; when testing import-cache temp-root cleanup, use a dedicated `NodeImportCache::new_in(...)` base dir so the one-time sweep stays isolated to that root. +- Active JavaScript/Python/WASM executions must hold a `NodeImportCache` cleanup guard until the child exits; otherwise dropping the engine can delete `timing-bootstrap.mjs` and related assets while the host runtime is still importing them. +- Guest path scrubbing in `crates/execution/src/node_import_cache.rs` should treat the real `HOST_CWD` as an implicit runtime-only mapping to the virtual guest cwd (for example `/root`) so entrypoint imports and stack traces stay usable without leaking the host path, and reserve `/unknown` for absolute host paths outside visible mappings or the internal cache roots. +- CommonJS module isolation in `crates/execution/src/node_import_cache.rs` has to patch `Module._resolveFilename` and the guest-facing `Module._cache` / `require.cache` view together; wrapping only `createGuestRequire()` does not constrain local `require()` inside already-loaded `.cjs` modules. +- Guest-visible `process` hardening in `crates/execution/src/node_import_cache.rs` should harden properties on the real host `process` before swapping in the guest proxy, and the proxy fallback must resolve via the proxy receiver (`Reflect.get(..., proxy)`) so accessors inherit the virtualized surface instead of the raw host object. +- Guest `child_process` launches should keep public child env and Node bootstrap internals separate: strip all `AGENT_OS_*` keys from the RPC `options.env` payload in `crates/execution/src/node_import_cache.rs`, carry only the Node runtime bootstrap allowlist in `options.internalBootstrapEnv`, and re-inject that allowlisted map only when `crates/sidecar/src/service.rs` starts a nested JavaScript runtime. +- Guest Node `net` Unix-socket support follows the same split as TCP: resolve guest socket paths against `host_dir` mounts when possible, otherwise map them under the VM sandbox root on the host, keep active Unix listeners/sockets in `crates/sidecar/src/service.rs`, and mirror non-mounted listener paths into the kernel VFS so guest `fs` APIs can see the socket file. +- When a guest Node networking port stops using real host listeners, mirror that state in `crates/sidecar/src/service.rs` `ActiveProcess` tracking and consult it from `find_listener`/socket snapshot queries before falling back to `/proc/[pid]/net/*`; procfs only sees host-owned sockets, not sidecar-managed polyfill listeners. +- Sidecar-managed loopback `net.listen` / `dgram.bind` listeners now use guest-port to host-port translation in `crates/sidecar/src/service.rs`: preserve guest-visible loopback addresses/ports in RPC responses and socket snapshots, but use the hidden host-bound port for external host-side probes and test clients. +- Sidecar JavaScript networking policy should read internal bootstrap env like `AGENT_OS_LOOPBACK_EXEMPT_PORTS` from `VmState.metadata` / `env.*`, not `vm.guest_env`; `guest_env` is permission-filtered and may be empty even when sidecar-only policy still needs the value. +- Guest Node `tls` should stay layered on the guest `net` polyfill rather than importing host `node:tls` directly: client connections must pass a preconnected guest socket into `tls.connect({ socket })`, and server handshakes should wrap accepted guest sockets with `new TLSSocket(..., { isServer: true })` and emit `secureConnection` from the wrapped socket's `secure` event. +- When a newly allowed Node builtin still has bypass-capable host-owned helpers or constructors (for example `dns.Resolver` / `dns.promises.Resolver`), replace those entrypoints with guest-owned shims or explicit unsupported stubs before adding the builtin to `DEFAULT_ALLOWED_NODE_BUILTINS`; inheriting the host module is only safe for exports that cannot escape the kernel-backed port. - Command execution mirrors the kernel API (exec, spawn) - `fetch(port, request)` reaches services running inside the VM using the kernel network adapter pattern (`proc.network.fetch`) +- Python execution in `crates/execution/src/python.rs` should keep `poll_event()` blocked until a real guest-visible event arrives or the caller timeout expires; filtered stderr/control messages are internal noise, `wait(None)` should still enforce the per-run `AGENT_OS_PYTHON_EXECUTION_TIMEOUT_MS` cap, `wait()` should bound accumulated stdout/stderr via the hidden `AGENT_OS_PYTHON_OUTPUT_BUFFER_MAX_BYTES` env knob rather than growing buffers without limit, and Node heap caps from `AGENT_OS_PYTHON_MAX_OLD_SPACE_MB` need to apply to both prewarm and execution launches without leaking those control vars into guest `process.env`. +- Pyodide bootstrap hardening in `crates/execution/src/node_import_cache.rs` must stay staged: `globalThis` guards can go in before `loadPyodide()`, but mutating `process` before `loadPyodide()` breaks the bundled Pyodide runtime under Node `--permission`. + +## Linux Compatibility + +The VM must behave like a standard Linux environment. Agents are written to target Linux and will break on non-standard behavior. + +- **Target: Linux userspace compatibility.** The kernel is not reimplementing the Linux kernel — it is providing a POSIX-like userspace environment. The goal is that a program written for Linux should run inside the VM without modification, subject to the execution runtimes available (Node.js, WASM, Python). +- **Correct errno values.** Every kernel operation that fails must return the correct POSIX errno (`ENOENT`, `EACCES`, `EEXIST`, `EISDIR`, `ENOTDIR`, `EXDEV`, `EBADF`, `EPERM`, `ENOSYS`, etc.). Agents check errno values to decide control flow — wrong errnos cause cascading failures. +- **Standard `/proc` layout.** `/proc/self/`, `/proc/[pid]/`, `/proc/[pid]/fd/`, `/proc/[pid]/environ`, `/proc/[pid]/cwd`, `/proc/[pid]/cmdline` should contain the expected content. Many tools and runtimes read `/proc` to discover their own state. +- **Synthetic procfs paths use guest-visible permission subjects.** Kernel-owned `/proc/...` entries are virtual, so permission checks for procfs access should authorize the guest-visible proc path directly rather than resolving through the backing VFS realpath. Otherwise procfs availability silently depends on whether the mounted root happens to contain a physical `/proc` directory. +- **Standard `/dev` devices.** `/dev/null`, `/dev/zero`, `/dev/urandom`, `/dev/stdin`, `/dev/stdout`, `/dev/stderr`, `/dev/fd/*`, `/dev/pts/*` must exist and behave correctly. `/dev/urandom` must return cryptographically random bytes, not deterministic values. +- **Stream-device byte counts belong on length-aware read paths.** For unbounded devices such as `/dev/zero` and `/dev/urandom`, exact Linux-style byte-count assertions should target `pread` / `fd_read` in `crates/kernel/src/device_layer.rs` and kernel FD tests; `read_file()` has no byte-count parameter and is only a bounded helper for whole-file-style callers. +- **Correct signal semantics.** `SIGCHLD` must be delivered to parent on child exit. `SIGPIPE` must be generated on write to broken pipe. `SIGWINCH` must be delivered on terminal resize. Signal delivery must respect process groups and sessions. +- **Standard filesystem paths.** `/tmp` must be writable. `/etc/hostname`, `/etc/resolv.conf`, `/etc/passwd`, `/etc/group` should contain valid content. `/usr/bin/env` should exist for shebangs. Shell (`/bin/sh`, `/bin/bash`) must be available. +- **Direct script exec should resolve registered stubs before reparsing files.** When the kernel executes a path under `/bin/` or `/usr/bin/` that corresponds to a registered command driver, dispatch that driver directly before falling back to shebang parsing; otherwise command stubs like `/bin/sh` recurse into their own `#!` wrapper instead of behaving like the real executable. +- **Environment variable conventions.** `HOME`, `USER`, `PATH`, `SHELL`, `TERM`, `HOSTNAME`, `PWD`, `LANG` must be set to reasonable values. `PATH` must include standard directories where commands are found. +- **Document deviations in the friction log.** Any behavior that differs from standard Linux must be documented in `.agent/notes/vm-friction.md` with the deviation, root cause, and whether a fix exists or is planned. ## Virtual Filesystem Design Reference @@ -91,6 +205,8 @@ The registry software packages depend on `@rivet-dev/agent-os-registry-types` (i - **OS-level content uses mounts, not post-boot writes.** If agentOS needs custom directories in the VM (e.g., `/etc/agentos/`), mount a pre-populated filesystem at boot — don't create the kernel and then write files into it afterward. This keeps the root filesystem clean and makes OS-provided paths read-only so agents can't tamper with them. - **Filesystem semantics must be durable.** Any state that changes filesystem behavior — including overlay deletes, whiteouts, tombstones, copy-up state, directory entries, inode metadata, or file contents — must be represented in durable filesystem or metadata storage. Do not implement correctness-critical filesystem behavior with in-memory side tables, in-memory whiteout sets, or other transient hacks. +- **Overlay metadata must stay out-of-band from the merged tree.** If an overlay implementation persists whiteouts or opaque-directory markers in the writable upper, store them under a reserved hidden metadata root and make every merged overlay read/snapshot path filter that root back out of user-visible results. +- **Overlay mutating ops need raw-layer checks plus upper-layer moves.** Once copy-up marks directories opaque, merged `read_dir()` no longer tells you whether lower layers still hold children, so `rmdir`-style emptiness checks must inspect raw upper and lower entries directly. For identity-preserving ops like `rename`, stage the source into the writable upper first and then call the upper filesystem's native `rename` so hardlinks and inode identity survive the move. - **Overlay filesystem behavior must match Linux OverlayFS as closely as possible, including mount-boundary semantics.** Treat the kernel OverlayFS docs as normative. OverlayFS overlays directory trees, not the mount table: the merged hierarchy is its own standalone mount, not a bind mount over underlying mounts. Do not design root overlay logic that "sees through" or absorbs unrelated mounted filesystems. Mounted filesystems remain separate mount boundaries, and cross-mount operations must keep normal mount semantics (`EXDEV`, separate identity, separate read-only rules). If we want overlay behavior inside a mounted filesystem such as an S3-backed or host-backed mount, that mounted filesystem must implement the layered metadata semantics itself rather than relying on the parent/root overlay to compose across the mount boundary. - **User-facing filesystem APIs should distinguish mounts from layers.** Mounts are separate mounted filesystems presented to the kernel VFS. Layers are overlay-building blocks used to construct a layered filesystem. Do not collapse those into one generic concept. A plain mounted `VirtualFileSystem` is not automatically a valid overlay layer. Overlay construction should consume explicit layer handles: one writable upper layer plus zero or more immutable lower snapshot layers. - **Middle layers in a Docker-like stack should be frozen layers, not extra writable uppers.** Linux OverlayFS supports one writable upper per overlay mount. Additional stacked layers should be represented as immutable snapshot/materialized lower layers. They may share the same layer-handle interface as the upper layer, but their state must mark them frozen/read-only. Any live whiteouts, opaque markers, or copy-up bookkeeping belong only to the active writable upper; once a layer is sealed into a reusable lower snapshot, it must be materialized into an ordinary read-only tree. @@ -159,6 +275,7 @@ Each agent type needs: ### Debugging Policy - **Never guess without concrete logs.** Every assertion about what's happening at runtime must be backed by log output. If you don't have logs proving something, add them before making claims. Use logging liberally when debugging -- add logs at every decision point and trace the full execution path before drawing conclusions. Never assume something is a timeout issue unless there are logs proving the system was actively busy for the entire duration. An idle hang and a slow operation look the same from the outside -- only logs can distinguish them. +- **Native sidecar security/audit telemetry should use structured bridge events, not ad hoc strings.** In `crates/sidecar/src/service.rs`, emit security-relevant records with `bridge.emit_structured_event(...)` and include a `timestamp` field plus stable keys such as `policy`, `path`, `source_pid`, `target_pid`, or `reason` so tests and downstream aggregation can assert on them directly. - **Never use CJS transpilation as a workaround** for ESM module loading issues. The VM must use V8's native ESM module system and Node.js native imports. Fix root causes in the ESM resolver, module access overlay, or V8 runtime instead of transforming ESM to CJS. The correct approach is to implement proper CJS/ESM interop in the V8 module resolver (wrapping CJS modules in ESM shims with named exports). - **Maintain a friction log** at `.agent/notes/vm-friction.md` for anything that behaves differently from a standard POSIX/Node.js system. Document the deviation, the root cause, and whether a fix exists. @@ -182,6 +299,10 @@ All agent working files live in `.agent/` at the repo root. When the user asks to track something in a note, store it in `.agent/notes/` by default. When something is identified as "do later", add it to `.agent/todo/`. Design documents and interface specs go in `.agent/specs/`. +## CLAUDE.md Convention + +- Every directory that has a `CLAUDE.md` must also have an `AGENTS.md` symlink pointing to it (`ln -s CLAUDE.md AGENTS.md`). This ensures other AI agents that look for `AGENTS.md` find the same instructions. + ## Git - **Commit messages**: Single-line conventional commits (e.g., `feat: add host tools RPC server`). No body, no co-author trailers. diff --git a/Cargo.lock b/Cargo.lock index becd2d882..1bf1526c9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -47,12 +47,14 @@ dependencies = [ "aws-sdk-s3", "base64 0.22.1", "filetime", + "hickory-resolver", "jsonwebtoken", "nix", "serde", "serde_json", "tokio", "ureq", + "url", "wat", ] @@ -76,6 +78,17 @@ version = "1.0.102" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" +[[package]] +name = "async-trait" +version = "0.1.89" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + [[package]] name = "atomic-waker" version = "1.1.2" @@ -631,6 +644,17 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" +[[package]] +name = "chacha20" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "rand_core 0.10.0", +] + [[package]] name = "cmake" version = "0.1.58" @@ -640,12 +664,32 @@ dependencies = [ "cc", ] +[[package]] +name = "combine" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba5a308b75df32fe02788e748662718f03fde005016435c444eea572398219fd" +dependencies = [ + "bytes", + "memchr", +] + [[package]] name = "const-oid" version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" +[[package]] +name = "core-foundation" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +dependencies = [ + "core-foundation-sys", + "libc", +] + [[package]] name = "core-foundation" version = "0.10.1" @@ -671,6 +715,15 @@ dependencies = [ "libc", ] +[[package]] +name = "cpufeatures" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" +dependencies = [ + "libc", +] + [[package]] name = "crc" version = "3.3.0" @@ -707,6 +760,36 @@ dependencies = [ "cfg-if", ] +[[package]] +name = "critical-section" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b" + +[[package]] +name = "crossbeam-channel" +version = "0.5.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28" + [[package]] name = "crypto-bigint" version = "0.4.9" @@ -714,7 +797,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ef2b4b23cddf68b89b8f8069890e8c270d54e2d5fe1b143820234805e4cb17ef" dependencies = [ "generic-array", - "rand_core", + "rand_core 0.6.4", "subtle", "zeroize", ] @@ -725,7 +808,7 @@ version = "0.5.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0dc92fb57ca44df6db8059111ab3af99a63d5d0f8375d9972e319a379c6bab76" dependencies = [ - "rand_core", + "rand_core 0.6.4", "subtle", ] @@ -739,6 +822,12 @@ dependencies = [ "typenum", ] +[[package]] +name = "data-encoding" +version = "2.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7a1e2f27636f116493b8b860f5546edb47c8d8f8ea73e1d2a20be88e28d1fea" + [[package]] name = "der" version = "0.6.1" @@ -818,7 +907,7 @@ dependencies = [ "generic-array", "group", "pkcs8", - "rand_core", + "rand_core 0.6.4", "sec1", "subtle", "zeroize", @@ -852,7 +941,7 @@ version = "0.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d013fc25338cc558c5c2cfbad646908fb23591e2404481826742b651c9af7160" dependencies = [ - "rand_core", + "rand_core 0.6.4", "subtle", ] @@ -931,6 +1020,23 @@ version = "0.3.32" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" +[[package]] +name = "futures-io" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718" + +[[package]] +name = "futures-macro" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + [[package]] name = "futures-sink" version = "0.3.32" @@ -950,8 +1056,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" dependencies = [ "futures-core", + "futures-macro", "futures-task", "pin-project-lite", + "slab", ] [[package]] @@ -996,6 +1104,7 @@ dependencies = [ "cfg-if", "libc", "r-efi 6.0.0", + "rand_core 0.10.0", "wasip2", "wasip3", ] @@ -1007,7 +1116,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5dfbfb3a6cfbd390d5c9564ab283a0349b9b9fcd46a706c1eb10e0db70bfbac7" dependencies = [ "ff", - "rand_core", + "rand_core 0.6.4", "subtle", ] @@ -1081,6 +1190,76 @@ version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" +[[package]] +name = "hickory-net" +version = "0.26.0-beta.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "acbafe58dd6a1bfa058c9c3dd3372c54665a1935e504a25783cdcf9bf14b21d6" +dependencies = [ + "async-trait", + "cfg-if", + "data-encoding", + "futures-channel", + "futures-io", + "futures-util", + "hickory-proto", + "idna", + "ipnet", + "jni", + "rand", + "thiserror", + "tinyvec", + "tokio", + "tracing", + "url", +] + +[[package]] +name = "hickory-proto" +version = "0.26.0-beta.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7ddac4552e5be0deead6df196824a5964b0797302569ef4686b75d32efad052" +dependencies = [ + "data-encoding", + "idna", + "ipnet", + "jni", + "once_cell", + "prefix-trie", + "rand", + "ring 0.17.14", + "thiserror", + "tinyvec", + "tracing", + "url", +] + +[[package]] +name = "hickory-resolver" +version = "0.26.0-beta.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a751e330e7cdf445892d6ce47cb4666a8b127834d2e42cee4db15713b9a27780" +dependencies = [ + "cfg-if", + "futures-util", + "hickory-net", + "hickory-proto", + "ipconfig", + "ipnet", + "jni", + "moka", + "ndk-context", + "once_cell", + "parking_lot", + "rand", + "resolv-conf", + "smallvec", + "system-configuration", + "thiserror", + "tokio", + "tracing", +] + [[package]] name = "hmac" version = "0.12.1" @@ -1378,11 +1557,27 @@ dependencies = [ "serde_core", ] +[[package]] +name = "ipconfig" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4d40460c0ce33d6ce4b0630ad68ff63d6661961c48b6dba35e5a4d81cfb48222" +dependencies = [ + "socket2 0.6.3", + "widestring", + "windows-registry", + "windows-result", + "windows-sys 0.61.2", +] + [[package]] name = "ipnet" version = "2.12.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2" +dependencies = [ + "serde", +] [[package]] name = "itoa" @@ -1390,6 +1585,55 @@ version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" +[[package]] +name = "jni" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5efd9a482cf3a427f00d6b35f14332adc7902ce91efb778580e180ff90fa3498" +dependencies = [ + "cfg-if", + "combine", + "jni-macros", + "jni-sys", + "log", + "simd_cesu8", + "thiserror", + "walkdir", + "windows-link", +] + +[[package]] +name = "jni-macros" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a00109accc170f0bdb141fed3e393c565b6f5e072365c3bd58f5b062591560a3" +dependencies = [ + "proc-macro2", + "quote", + "rustc_version", + "simd_cesu8", + "syn", +] + +[[package]] +name = "jni-sys" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6377a88cb3910bee9b0fa88d4f42e1d2da8e79915598f65fb0c7ee14c878af2" +dependencies = [ + "jni-sys-macros", +] + +[[package]] +name = "jni-sys-macros" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38c0b942f458fe50cdac086d2f946512305e5631e720728f2a61aabcd47a6264" +dependencies = [ + "quote", + "syn", +] + [[package]] name = "jobserver" version = "0.1.34" @@ -1445,7 +1689,7 @@ dependencies = [ "bitflags", "libc", "plain", - "redox_syscall", + "redox_syscall 0.7.3", ] [[package]] @@ -1460,6 +1704,15 @@ version = "0.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0" +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + [[package]] name = "log" version = "0.4.29" @@ -1512,6 +1765,29 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "moka" +version = "0.12.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "957228ad12042ee839f93c8f257b62b4c0ab5eaae1d4fa60de53b27c9d7c5046" +dependencies = [ + "crossbeam-channel", + "crossbeam-epoch", + "crossbeam-utils", + "equivalent", + "parking_lot", + "portable-atomic", + "smallvec", + "tagptr", + "uuid", +] + +[[package]] +name = "ndk-context" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27b02d87554356db9e9a873add8782d4ea6e3e58ea071a9adb9a2e8ddb884a8b" + [[package]] name = "nix" version = "0.29.0" @@ -1563,6 +1839,10 @@ name = "once_cell" version = "1.21.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" +dependencies = [ + "critical-section", + "portable-atomic", +] [[package]] name = "openssl-probe" @@ -1587,6 +1867,29 @@ dependencies = [ "sha2", ] +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall 0.5.18", + "smallvec", + "windows-link", +] + [[package]] name = "pem" version = "1.1.1" @@ -1630,6 +1933,12 @@ version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6" +[[package]] +name = "portable-atomic" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49" + [[package]] name = "potential_utf" version = "0.1.5" @@ -1645,6 +1954,17 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" +[[package]] +name = "prefix-trie" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23370be78b7e5bcbb0cab4a02047eb040279a693c78daad04c2c5f1c24a83503" +dependencies = [ + "either", + "ipnet", + "num-traits", +] + [[package]] name = "prettyplease" version = "0.2.37" @@ -1685,6 +2005,17 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" +[[package]] +name = "rand" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc266eb313df6c5c09c1c7b1fbe2510961e5bcd3add930c1e31f7ed9da0feff8" +dependencies = [ + "chacha20", + "getrandom 0.4.2", + "rand_core 0.10.0", +] + [[package]] name = "rand_core" version = "0.6.4" @@ -1694,6 +2025,21 @@ dependencies = [ "getrandom 0.2.17", ] +[[package]] +name = "rand_core" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c8d0fd677905edcbeedbf2edb6494d676f0e98d54d5cf9bda0b061cb8fb8aba" + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags", +] + [[package]] name = "redox_syscall" version = "0.7.3" @@ -1709,6 +2055,12 @@ version = "0.1.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973" +[[package]] +name = "resolv-conf" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e061d1b48cb8d38042de4ae0a7a6401009d6143dc80d2e2d6f31f0bdd6470c7" + [[package]] name = "rfc6979" version = "0.3.1" @@ -1854,6 +2206,15 @@ version = "1.0.23" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + [[package]] name = "schannel" version = "0.1.29" @@ -1863,6 +2224,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + [[package]] name = "sct" version = "0.7.1" @@ -1894,7 +2261,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" dependencies = [ "bitflags", - "core-foundation", + "core-foundation 0.10.1", "core-foundation-sys", "libc", "security-framework-sys", @@ -1966,7 +2333,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" dependencies = [ "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "digest", ] @@ -1977,7 +2344,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", - "cpufeatures", + "cpufeatures 0.2.17", "digest", ] @@ -2004,7 +2371,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "74233d3b3b2f6d4b006dc19dee745e73e2a6bfb6f93607cd3b02bd5b00797d7c" dependencies = [ "digest", - "rand_core", + "rand_core 0.6.4", ] [[package]] @@ -2013,6 +2380,22 @@ version = "0.3.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214" +[[package]] +name = "simd_cesu8" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94f90157bb87cddf702797c5dadfa0be7d266cdf49e22da2fcaa32eff75b2c33" +dependencies = [ + "rustc_version", + "simdutf8", +] + +[[package]] +name = "simdutf8" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e" + [[package]] name = "simple_asn1" version = "0.6.4" @@ -2113,6 +2496,33 @@ dependencies = [ "syn", ] +[[package]] +name = "system-configuration" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b" +dependencies = [ + "bitflags", + "core-foundation 0.9.4", + "system-configuration-sys", +] + +[[package]] +name = "system-configuration-sys" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "tagptr" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b2093cf4c8eb1e67749a6762251bc9cd836b6fc171623bd0a9d324d37af2417" + [[package]] name = "tempfile" version = "3.27.0" @@ -2187,6 +2597,21 @@ dependencies = [ "zerovec", ] +[[package]] +name = "tinyvec" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e61e67053d25a4e82c844e8424039d9745781b3fc4f32b8d55ed50f5f667ef3" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" + [[package]] name = "tokio" version = "1.51.0" @@ -2199,9 +2624,21 @@ dependencies = [ "pin-project-lite", "signal-hook-registry", "socket2 0.6.3", + "tokio-macros", "windows-sys 0.61.2", ] +[[package]] +name = "tokio-macros" +version = "2.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + [[package]] name = "tokio-rustls" version = "0.24.1" @@ -2378,6 +2815,7 @@ version = "1.23.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5ac8b6f42ead25368cf5b098aeb3dc8a1a2c05a3eee8a9a1a68c640edbfc79d9" dependencies = [ + "getrandom 0.4.2", "js-sys", "wasm-bindgen", ] @@ -2394,6 +2832,16 @@ version = "0.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64" +[[package]] +name = "walkdir" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b" +dependencies = [ + "same-file", + "winapi-util", +] + [[package]] name = "want" version = "0.3.1" @@ -2577,6 +3025,12 @@ dependencies = [ "rustls-pki-types", ] +[[package]] +name = "widestring" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471" + [[package]] name = "winapi" version = "0.3.9" @@ -2593,6 +3047,15 @@ version = "0.4.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys 0.61.2", +] + [[package]] name = "winapi-x86_64-pc-windows-gnu" version = "0.4.0" @@ -2605,6 +3068,35 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" +[[package]] +name = "windows-registry" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720" +dependencies = [ + "windows-link", + "windows-result", + "windows-strings", +] + +[[package]] +name = "windows-result" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-strings" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" +dependencies = [ + "windows-link", +] + [[package]] name = "windows-sys" version = "0.52.0" diff --git a/crates/execution/assets/runners/python-runner.mjs b/crates/execution/assets/runners/python-runner.mjs new file mode 100644 index 000000000..79e140850 --- /dev/null +++ b/crates/execution/assets/runners/python-runner.mjs @@ -0,0 +1,936 @@ +import { closeSync, createReadStream, readSync, writeSync } from 'node:fs'; +import { readFile } from 'node:fs/promises'; +import { register } from 'node:module'; +import { performance as realPerformance } from 'node:perf_hooks'; +import path from 'node:path'; +import readline from 'node:readline'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const ACCESS_DENIED_CODE = 'ERR_ACCESS_DENIED'; +const ASSET_ROOT_ENV = 'AGENT_OS_NODE_IMPORT_CACHE_ASSET_ROOT'; +const PYODIDE_INDEX_URL_ENV = 'AGENT_OS_PYODIDE_INDEX_URL'; +const PYTHON_CODE_ENV = 'AGENT_OS_PYTHON_CODE'; +const PYTHON_FILE_ENV = 'AGENT_OS_PYTHON_FILE'; +const PYTHON_PREWARM_ONLY_ENV = 'AGENT_OS_PYTHON_PREWARM_ONLY'; +const PYTHON_WARMUP_DEBUG_ENV = 'AGENT_OS_PYTHON_WARMUP_DEBUG'; +const PYTHON_WARMUP_METRICS_PREFIX = '__AGENT_OS_PYTHON_WARMUP_METRICS__:'; +const PYTHON_PRELOAD_PACKAGES_ENV = 'AGENT_OS_PYTHON_PRELOAD_PACKAGES'; +const PYTHON_VFS_RPC_REQUEST_FD_ENV = 'AGENT_OS_PYTHON_VFS_RPC_REQUEST_FD'; +const PYTHON_VFS_RPC_RESPONSE_FD_ENV = 'AGENT_OS_PYTHON_VFS_RPC_RESPONSE_FD'; +const STDIN_FD = 0; +const SUPPORTED_PRELOAD_PACKAGES = ['numpy', 'pandas']; +const SUPPORTED_PRELOAD_PACKAGE_SET = new Set(SUPPORTED_PRELOAD_PACKAGES); +const DENIED_BUILTINS = new Set([ + 'child_process', + 'cluster', + 'dgram', + 'diagnostics_channel', + 'dns', + 'http', + 'http2', + 'https', + 'inspector', + 'module', + 'net', + 'tls', + 'trace_events', + 'v8', + 'vm', + 'worker_threads', +]); +const originalFetch = + typeof globalThis.fetch === 'function' + ? globalThis.fetch.bind(globalThis) + : null; +const originalRequire = + typeof globalThis.require === 'function' + ? globalThis.require.bind(globalThis) + : null; +const originalGetBuiltinModule = + typeof process.getBuiltinModule === 'function' + ? process.getBuiltinModule.bind(process) + : null; +const CONTROL_PIPE_FD = parseControlPipeFd(process.env.AGENT_OS_CONTROL_PIPE_FD); + +function requiredEnv(name) { + const value = process.env[name]; + if (value == null) { + throw new Error(`${name} is required`); + } + return value; +} + +function parseControlPipeFd(value) { + if (typeof value !== 'string' || value.trim() === '') { + return null; + } + + const parsed = Number.parseInt(value, 10); + return Number.isInteger(parsed) && parsed >= 0 ? parsed : null; +} + +function emitControlMessage(message) { + if (CONTROL_PIPE_FD == null) { + return; + } + + try { + writeSync(CONTROL_PIPE_FD, `${JSON.stringify(message)}\n`); + } catch { + // Ignore control-channel write failures during teardown. + } +} + +function normalizeDirectoryPath(value) { + return value.endsWith(path.sep) ? value : `${value}${path.sep}`; +} + +function resolveIndexLocation(value) { + if (/^[A-Za-z][A-Za-z0-9+.-]*:/.test(value)) { + const normalizedUrl = value.endsWith('/') ? value : `${value}/`; + if (!normalizedUrl.startsWith('file:')) { + return { + indexPath: normalizedUrl, + indexUrl: normalizedUrl, + }; + } + + const indexPath = normalizeDirectoryPath(fileURLToPath(normalizedUrl)); + return { + indexPath, + indexUrl: pathToFileURL(indexPath).href, + }; + } + + const indexPath = normalizeDirectoryPath(path.resolve(value)); + return { + indexPath, + indexUrl: pathToFileURL(indexPath).href, + }; +} + +function writeStream(stream, message) { + if (message == null) { + return; + } + + const value = typeof message === 'string' ? message : String(message); + stream.write(value.endsWith('\n') ? value : `${value}\n`); +} + +function formatError(error) { + if (error instanceof Error) { + return error.stack || error.message || String(error); + } + + return String(error); +} + +function emitPythonStartupMetrics({ + prewarmOnly, + startupMs, + loadPyodideMs, + packageLoadMs, + packageCount, + source, +}) { + if (process.env[PYTHON_WARMUP_DEBUG_ENV] !== '1') { + return; + } + + writeStream( + process.stderr, + `${PYTHON_WARMUP_METRICS_PREFIX}${JSON.stringify({ + phase: 'startup', + prewarmOnly, + startupMs, + loadPyodideMs, + packageLoadMs, + packageCount, + source, + })}`, + ); +} + +function parsePreloadPackages(value) { + if (value == null || value.trim() === '') { + return []; + } + + let parsed; + try { + parsed = JSON.parse(value); + } catch (error) { + throw new Error( + `${PYTHON_PRELOAD_PACKAGES_ENV} must be a JSON array of package names: ${formatError(error)}`, + ); + } + + if (!Array.isArray(parsed)) { + throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} must be a JSON array of package names`); + } + + const packages = []; + const seen = new Set(); + + for (const entry of parsed) { + if (typeof entry !== 'string') { + throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} entries must be strings`); + } + + const name = entry.trim(); + if (name.length === 0) { + throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} entries must not be empty`); + } + + if (!SUPPORTED_PRELOAD_PACKAGE_SET.has(name)) { + throw new Error( + `Unsupported bundled Python package "${name}". Available packages: ${SUPPORTED_PRELOAD_PACKAGES.join(', ')}`, + ); + } + + if (!seen.has(name)) { + seen.add(name); + packages.push(name); + } + } + + return packages; +} + +function parseOptionalFd(name) { + const value = process.env[name]; + if (value == null || value.trim() === '') { + return null; + } + + const fd = Number.parseInt(value, 10); + if (!Number.isInteger(fd) || fd < 0) { + throw new Error(`${name} must be a non-negative integer file descriptor`); + } + + return fd; +} + +function rejectPendingRpcRequests(pending, error) { + for (const { reject } of pending.values()) { + reject(error); + } + pending.clear(); +} + +function createPythonVfsRpcBridge() { + const requestFd = parseOptionalFd(PYTHON_VFS_RPC_REQUEST_FD_ENV); + const responseFd = parseOptionalFd(PYTHON_VFS_RPC_RESPONSE_FD_ENV); + + if (requestFd == null && responseFd == null) { + return null; + } + + if (requestFd == null || responseFd == null) { + throw new Error( + `both ${PYTHON_VFS_RPC_REQUEST_FD_ENV} and ${PYTHON_VFS_RPC_RESPONSE_FD_ENV} are required`, + ); + } + + let nextRequestId = 1; + const queuedResponses = new Map(); + let responseBuffer = ''; + + function readResponseLineSync() { + while (true) { + const newlineIndex = responseBuffer.indexOf('\n'); + if (newlineIndex >= 0) { + const line = responseBuffer.slice(0, newlineIndex); + responseBuffer = responseBuffer.slice(newlineIndex + 1); + return line; + } + + const chunk = Buffer.alloc(4096); + const bytesRead = readSync(responseFd, chunk, 0, chunk.length, null); + if (bytesRead === 0) { + throw new Error('Agent OS Python VFS RPC response channel closed unexpectedly'); + } + responseBuffer += chunk.subarray(0, bytesRead).toString('utf8'); + } + } + + function parseResponseLine(line) { + try { + return JSON.parse(line); + } catch (error) { + throw new Error(`invalid Agent OS Python VFS RPC response: ${formatError(error)}`); + } + } + + function waitForResponseSync(id) { + const queued = queuedResponses.get(id); + if (queued) { + queuedResponses.delete(id); + return queued; + } + + while (true) { + const line = readResponseLineSync(); + if (line.trim() === '') { + continue; + } + + const message = parseResponseLine(line); + if (message?.id === id) { + return message; + } + queuedResponses.set(message?.id, message); + } + } + + function requestSync(method, payload = {}) { + const id = nextRequestId++; + writeSync( + requestFd, + `${JSON.stringify({ + id, + method, + ...payload, + })}\n`, + ); + + const message = waitForResponseSync(id); + if (message?.ok) { + return message.result ?? {}; + } + + const error = new Error(message?.error?.message || `Agent OS Python VFS RPC request ${id} failed`); + error.code = message?.error?.code || 'ERR_AGENT_OS_PYTHON_VFS_RPC'; + throw error; + } + + function request(method, payload = {}) { + return Promise.resolve().then(() => requestSync(method, payload)); + } + + function normalizeWriteContent(content) { + if (typeof content === 'string') { + return content; + } + if (ArrayBuffer.isView(content)) { + return Buffer.from(content.buffer, content.byteOffset, content.byteLength).toString('base64'); + } + if (content instanceof ArrayBuffer) { + return Buffer.from(content).toString('base64'); + } + throw new Error('fsWrite requires a base64 string or Uint8Array'); + } + + return { + fsReadSync(path) { + const result = requestSync('fsRead', { path }); + return result.contentBase64 ?? ''; + }, + async fsRead(path) { + return this.fsReadSync(path); + }, + fsWriteSync(path, content) { + requestSync('fsWrite', { + path, + contentBase64: normalizeWriteContent(content), + }); + }, + async fsWrite(path, content) { + this.fsWriteSync(path, content); + }, + fsStatSync(path) { + const result = requestSync('fsStat', { path }); + return result.stat ?? null; + }, + async fsStat(path) { + return this.fsStatSync(path); + }, + fsReaddirSync(path) { + const result = requestSync('fsReaddir', { path }); + return result.entries ?? []; + }, + async fsReaddir(path) { + return this.fsReaddirSync(path); + }, + fsMkdirSync(path, options = {}) { + requestSync('fsMkdir', { + path, + recursive: options?.recursive === true, + }); + }, + async fsMkdir(path, options = {}) { + this.fsMkdirSync(path, options); + }, + dispose() { + try { + closeSync(requestFd); + } catch { + // Ignore repeated-close shutdown races. + } + try { + closeSync(responseFd); + } catch { + // Ignore repeated-close shutdown races. + } + }, + }; +} + +function accessDenied(subject) { + const error = new Error(`${subject} is not available in the Agent OS guest Python runtime`); + error.code = ACCESS_DENIED_CODE; + return error; +} + +const PYTHON_GUEST_IMPORT_BLOCKLIST_SOURCE = String.raw` +import builtins as _agent_os_builtins +import sys as _agent_os_sys +import types as _agent_os_types + +def _agent_os_raise_access_denied(module_name): + raise RuntimeError(f"{module_name} is not available in the Agent OS guest Python runtime") + +class _AgentOsBlockedModule(_agent_os_types.ModuleType): + def __init__(self, name): + super().__init__(name) + self.__dict__['__all__'] = () + + def __getattr__(self, _name): + _agent_os_raise_access_denied(self.__name__) + + def __dir__(self): + return [] + +_agent_os_blocked_modules = { + _agent_os_module_name: _AgentOsBlockedModule(_agent_os_module_name) + for _agent_os_module_name in ('js', 'pyodide_js') +} + +_agent_os_original_import = _agent_os_builtins.__import__ + +def _agent_os_import(name, globals=None, locals=None, fromlist=(), level=0): + if name in _agent_os_blocked_modules: + return _agent_os_blocked_modules[name] + return _agent_os_original_import(name, globals, locals, fromlist, level) + +_agent_os_builtins.__import__ = _agent_os_import +_agent_os_sys.modules.update(_agent_os_blocked_modules) +`; + +function hardenProperty(target, key, value) { + try { + Object.defineProperty(target, key, { + value, + writable: false, + configurable: false, + }); + } catch (error) { + throw new Error(`Failed to harden property ${String(key)}`, { cause: error }); + } +} + +function normalizeBuiltin(specifier) { + if (typeof specifier !== 'string') { + return null; + } + + return specifier.startsWith('node:') ? specifier.slice('node:'.length) : specifier; +} + +function installPythonGuestImportBlocklist(pyodide) { + if (typeof pyodide?.runPython !== 'function') { + return; + } + + pyodide.runPython(PYTHON_GUEST_IMPORT_BLOCKLIST_SOURCE); +} + +function installPythonGuestPreloadHardening() { + if (originalRequire) { + hardenProperty(globalThis, 'require', () => { + throw accessDenied('require'); + }); + } + + if (originalFetch) { + const restrictedFetch = (resource, init) => { + const candidate = + typeof resource === 'string' + ? resource + : resource instanceof URL + ? resource.href + : resource?.url; + + let url; + try { + url = new URL(String(candidate ?? '')); + } catch { + throw accessDenied('network access'); + } + + if (url.protocol !== 'data:') { + throw accessDenied(`network access to ${url.protocol}`); + } + + return originalFetch(resource, init); + }; + + hardenProperty(globalThis, 'fetch', restrictedFetch); + } +} + +function installPythonGuestProcessHardening() { + hardenProperty(process, 'binding', () => { + throw accessDenied('process.binding'); + }); + hardenProperty(process, '_linkedBinding', () => { + throw accessDenied('process._linkedBinding'); + }); + hardenProperty(process, 'dlopen', () => { + throw accessDenied('process.dlopen'); + }); + + if (originalGetBuiltinModule) { + hardenProperty(process, 'getBuiltinModule', (specifier) => { + const normalized = normalizeBuiltin(specifier); + if (normalized && DENIED_BUILTINS.has(normalized)) { + throw accessDenied(`node:${normalized}`); + } + return originalGetBuiltinModule(specifier); + }); + } +} + +function installPythonGuestLoaderHooks() { + const assetRoot = process.env[ASSET_ROOT_ENV]; + if (!assetRoot) { + return; + } + + register(new URL('./loader.mjs', import.meta.url), import.meta.url); +} + +function installPythonVfsRpcBridge() { + const bridge = createPythonVfsRpcBridge(); + if (!bridge) { + return null; + } + + hardenProperty(globalThis, '__agentOsPythonVfsRpc', bridge); + return bridge; +} + +function installPythonWorkspaceFs(pyodide, bridge) { + if (!bridge) { + return; + } + + const { FS, ERRNO_CODES } = pyodide; + if (!FS?.mount || !FS?.filesystems?.MEMFS || !ERRNO_CODES) { + return; + } + + const MEMFS = FS.filesystems.MEMFS; + const memfsDirNodeOps = MEMFS.ops_table.dir.node; + const memfsDirStreamOps = MEMFS.ops_table.dir.stream; + const memfsFileNodeOps = MEMFS.ops_table.file.node; + const memfsFileStreamOps = MEMFS.ops_table.file.stream; + const workspaceDirStreamOps = memfsDirStreamOps; + + function joinGuestPath(parentPath, name) { + return parentPath === '/' ? `/${name}` : `${parentPath}/${name}`; + } + + function nodeGuestPath(node) { + return node.agentOsGuestPath || node.mount?.mountpoint || '/workspace'; + } + + function createFsError(error) { + if (error instanceof FS.ErrnoError) { + return error; + } + + const message = String(error?.message || error); + let errno = ERRNO_CODES.EIO; + if (/permission denied|access denied|denied/i.test(message)) { + errno = ERRNO_CODES.EACCES; + } else if (/read-only|erofs/i.test(message)) { + errno = ERRNO_CODES.EROFS; + } else if (/not a directory|enotdir/i.test(message)) { + errno = ERRNO_CODES.ENOTDIR; + } else if (/is a directory|eisdir/i.test(message)) { + errno = ERRNO_CODES.EISDIR; + } else if (/exists|already exists|eexist/i.test(message)) { + errno = ERRNO_CODES.EEXIST; + } else if (/not found|no such file|enoent/i.test(message)) { + errno = ERRNO_CODES.ENOENT; + } + + return new FS.ErrnoError(errno); + } + + function withFsErrors(operation) { + try { + return operation(); + } catch (error) { + throw createFsError(error); + } + } + + function updateNodeFromRemoteStat(node, stat) { + if (!stat) { + throw new FS.ErrnoError(ERRNO_CODES.ENOENT); + } + + node.mode = stat.mode; + node.timestamp = Date.now(); + if (FS.isFile(stat.mode) && !node.agentOsDirty) { + node.agentOsRemoteSize = stat.size; + } + } + + function createWorkspaceNode(parent, name, mode, dev, guestPath) { + const node = MEMFS.createNode(parent, name, mode, dev); + node.agentOsGuestPath = guestPath; + node.agentOsDirty = false; + node.agentOsLoaded = FS.isDir(mode); + node.agentOsRemoteSize = 0; + if (FS.isDir(mode)) { + node.node_ops = workspaceDirNodeOps; + node.stream_ops = workspaceDirStreamOps; + } else if (FS.isFile(mode)) { + node.node_ops = workspaceFileNodeOps; + node.stream_ops = workspaceFileStreamOps; + } + return node; + } + + function syncDirectory(node) { + const guestPath = nodeGuestPath(node); + const entries = withFsErrors(() => bridge.fsReaddirSync(guestPath)); + const remoteNames = new Set(entries); + + for (const name of Object.keys(node.contents || {})) { + if (remoteNames.has(name)) { + continue; + } + + const child = node.contents[name]; + if (FS.isDir(child.mode)) { + memfsDirNodeOps.rmdir(node, name); + } else { + memfsDirNodeOps.unlink(node, name); + } + } + + for (const name of entries) { + const childPath = joinGuestPath(guestPath, name); + const stat = withFsErrors(() => bridge.fsStatSync(childPath)); + const existing = node.contents[name]; + + if (existing) { + const existingIsDir = FS.isDir(existing.mode); + const remoteIsDir = Boolean(stat?.isDirectory); + if (existingIsDir !== remoteIsDir) { + if (existingIsDir) { + memfsDirNodeOps.rmdir(node, name); + } else { + memfsDirNodeOps.unlink(node, name); + } + } else { + existing.agentOsGuestPath = childPath; + updateNodeFromRemoteStat(existing, stat); + if (FS.isFile(existing.mode) && !existing.agentOsDirty) { + existing.agentOsLoaded = false; + } + continue; + } + } + + const mode = stat?.mode ?? (stat?.isDirectory ? 0o040755 : 0o100644); + const child = createWorkspaceNode(node, name, mode, 0, childPath); + updateNodeFromRemoteStat(child, stat); + } + } + + function loadFileContents(node) { + if (node.agentOsDirty) { + return; + } + + const stat = withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); + updateNodeFromRemoteStat(node, stat); + const contentBase64 = withFsErrors(() => bridge.fsReadSync(nodeGuestPath(node))); + const bytes = Uint8Array.from(Buffer.from(contentBase64, 'base64')); + node.contents = bytes; + node.usedBytes = bytes.length; + node.agentOsLoaded = true; + node.agentOsRemoteSize = bytes.length; + } + + function persistFile(node) { + const contents = node.contents ? MEMFS.getFileDataAsTypedArray(node) : new Uint8Array(0); + withFsErrors(() => bridge.fsWriteSync(nodeGuestPath(node), contents)); + node.agentOsDirty = false; + node.agentOsLoaded = true; + node.agentOsRemoteSize = contents.length; + node.timestamp = Date.now(); + } + + function makeStat(node, stat) { + const mode = stat?.mode ?? node.mode; + const size = FS.isDir(mode) ? 4096 : (node.agentOsDirty ? node.usedBytes : (stat?.size ?? node.usedBytes ?? 0)); + const timestamp = new Date(node.timestamp || Date.now()); + + return { + dev: 1, + ino: node.id, + mode, + nlink: FS.isDir(mode) ? 2 : 1, + uid: 0, + gid: 0, + rdev: 0, + size, + atime: timestamp, + mtime: timestamp, + ctime: timestamp, + blksize: 4096, + blocks: Math.max(1, Math.ceil(size / 4096)), + }; + } + + const workspaceFileNodeOps = { + getattr(node) { + const stat = node.agentOsDirty + ? null + : withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); + if (stat) { + updateNodeFromRemoteStat(node, stat); + } + return makeStat(node, stat); + }, + setattr(node, attr) { + memfsFileNodeOps.setattr(node, attr); + if (attr?.size != null) { + node.agentOsDirty = true; + node.agentOsLoaded = true; + } + }, + }; + + const workspaceFileStreamOps = { + llseek(stream, offset, whence) { + return memfsFileStreamOps.llseek(stream, offset, whence); + }, + read(stream, buffer, offset, length, position) { + if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { + loadFileContents(stream.node); + } + return memfsFileStreamOps.read(stream, buffer, offset, length, position); + }, + write(stream, buffer, offset, length, position, canOwn) { + if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { + loadFileContents(stream.node); + } + const written = memfsFileStreamOps.write(stream, buffer, offset, length, position, canOwn); + stream.node.agentOsDirty = true; + persistFile(stream.node); + return written; + }, + mmap(stream, length, position, prot, flags) { + if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { + loadFileContents(stream.node); + } + return memfsFileStreamOps.mmap(stream, length, position, prot, flags); + }, + msync(stream, buffer, offset, length, mmapFlags) { + const result = memfsFileStreamOps.msync(stream, buffer, offset, length, mmapFlags); + stream.node.agentOsDirty = true; + persistFile(stream.node); + return result; + }, + }; + + const workspaceDirNodeOps = { + getattr(node) { + const stat = withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); + updateNodeFromRemoteStat(node, stat); + return makeStat(node, stat); + }, + setattr(node, attr) { + memfsDirNodeOps.setattr(node, attr); + }, + lookup(parent, name) { + syncDirectory(parent); + try { + return memfsDirNodeOps.lookup(parent, name); + } catch (error) { + if (!(error instanceof FS.ErrnoError) || error.errno !== ERRNO_CODES.ENOENT) { + throw error; + } + + const guestPath = joinGuestPath(nodeGuestPath(parent), name); + const stat = withFsErrors(() => bridge.fsStatSync(guestPath)); + const child = createWorkspaceNode(parent, name, stat.mode, 0, guestPath); + updateNodeFromRemoteStat(child, stat); + return child; + } + }, + mknod(parent, name, mode, dev) { + const guestPath = joinGuestPath(nodeGuestPath(parent), name); + const node = createWorkspaceNode(parent, name, mode, dev, guestPath); + if (FS.isDir(mode)) { + withFsErrors(() => bridge.fsMkdirSync(guestPath, { recursive: false })); + } else if (FS.isFile(mode)) { + node.contents = new Uint8Array(0); + node.usedBytes = 0; + node.agentOsDirty = true; + persistFile(node); + } + return node; + }, + rename() { + throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); + }, + unlink() { + throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); + }, + rmdir() { + throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); + }, + readdir(node) { + syncDirectory(node); + return memfsDirNodeOps.readdir(node); + }, + symlink() { + throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); + }, + }; + + try { + FS.mkdir('/workspace'); + } catch (error) { + if (!(error instanceof FS.ErrnoError) || error.errno !== ERRNO_CODES.EEXIST) { + throw error; + } + } + + FS.mount( + { + mount(mount) { + const root = MEMFS.mount(mount); + root.agentOsGuestPath = mount.mountpoint; + root.agentOsDirty = false; + root.agentOsLoaded = true; + root.agentOsRemoteSize = 0; + root.node_ops = workspaceDirNodeOps; + root.stream_ops = workspaceDirStreamOps; + return root; + }, + }, + {}, + '/workspace', + ); +} + +async function readLockFileContents(indexURL) { + const lockFileUrl = new URL('pyodide-lock.json', indexURL); + return readFile(lockFileUrl, 'utf8'); +} + +function installPythonStdin(pyodide) { + if (typeof pyodide?.setStdin !== 'function') { + return; + } + + pyodide.setStdin({ + isatty: false, + read(buffer) { + return readSync(STDIN_FD, buffer, 0, buffer.length, null); + }, + }); +} + +function resolvePythonSource(pyodide) { + const filePath = process.env[PYTHON_FILE_ENV]; + if (filePath != null) { + if (typeof pyodide?.FS?.readFile !== 'function') { + throw new Error(`Pyodide FS.readFile() is required to execute ${filePath}`); + } + + return pyodide.FS.readFile(filePath, { encoding: 'utf8' }); + } + + return requiredEnv(PYTHON_CODE_ENV); +} + +let pythonVfsRpcBridge = null; + +try { + const startupStarted = realPerformance.now(); + const { indexPath, indexUrl } = resolveIndexLocation(requiredEnv(PYODIDE_INDEX_URL_ENV)); + const prewarmOnly = process.env[PYTHON_PREWARM_ONLY_ENV] === '1'; + const preloadPackages = parsePreloadPackages(process.env[PYTHON_PRELOAD_PACKAGES_ENV]); + const lockFileContents = await readLockFileContents(indexUrl); + const pyodideModuleUrl = new URL('pyodide.mjs', indexUrl).href; + const { loadPyodide } = await import(pyodideModuleUrl); + + if (typeof loadPyodide !== 'function') { + throw new Error(`pyodide.mjs at ${indexUrl} does not export loadPyodide()`); + } + + installPythonGuestPreloadHardening(); + const loadPyodideStarted = realPerformance.now(); + const pyodide = await loadPyodide({ + indexURL: indexPath, + lockFileContents, + packageBaseUrl: indexPath, + stdout: (message) => writeStream(process.stdout, message), + stderr: (message) => writeStream(process.stderr, message), + }); + const loadPyodideMs = realPerformance.now() - loadPyodideStarted; + let packageLoadMs = 0; + + if (prewarmOnly) { + emitPythonStartupMetrics({ + prewarmOnly: true, + startupMs: realPerformance.now() - startupStarted, + loadPyodideMs, + packageLoadMs, + packageCount: 0, + source: 'prewarm', + }); + process.exitCode = 0; + } else { + installPythonStdin(pyodide); + pythonVfsRpcBridge = installPythonVfsRpcBridge(); + installPythonWorkspaceFs(pyodide, pythonVfsRpcBridge); + installPythonGuestLoaderHooks(); + if (preloadPackages.length > 0) { + const packageLoadStarted = realPerformance.now(); + await pyodide.loadPackage(preloadPackages); + packageLoadMs = realPerformance.now() - packageLoadStarted; + } + installPythonGuestProcessHardening(); + installPythonGuestImportBlocklist(pyodide); + const source = process.env[PYTHON_FILE_ENV] != null ? 'file' : 'inline'; + emitPythonStartupMetrics({ + prewarmOnly: false, + startupMs: realPerformance.now() - startupStarted, + loadPyodideMs, + packageLoadMs, + packageCount: preloadPackages.length, + source, + }); + const code = resolvePythonSource(pyodide); + await pyodide.runPythonAsync(code); + } +} catch (error) { + writeStream(process.stderr, formatError(error)); + process.exitCode = 1; +} finally { + pythonVfsRpcBridge?.dispose(); + emitControlMessage({ type: 'python_exit', exitCode: process.exitCode ?? 0 }); +} +process.exit(process.exitCode ?? 0); diff --git a/crates/execution/src/benchmark.rs b/crates/execution/src/benchmark.rs index f51f68a86..873ec96ad 100644 --- a/crates/execution/src/benchmark.rs +++ b/crates/execution/src/benchmark.rs @@ -2750,6 +2750,12 @@ fn measure_transport_roundtrip( Some(crate::JavascriptExecutionEvent::Stderr(chunk)) => { stderr_buffer.push_str(&String::from_utf8(chunk)?); } + Some(crate::JavascriptExecutionEvent::SyncRpcRequest(request)) => { + return Err(JavascriptBenchmarkError::Execution( + JavascriptExecutionError::PendingSyncRpcRequest(request.id), + )); + } + Some(crate::JavascriptExecutionEvent::SignalState { .. }) => {} Some(crate::JavascriptExecutionEvent::Exited(exit_code)) => { return Err(JavascriptBenchmarkError::TransportProbeExited { exit_code, diff --git a/crates/execution/src/javascript.rs b/crates/execution/src/javascript.rs index 6c1dedbd8..b96e1caa7 100644 --- a/crates/execution/src/javascript.rs +++ b/crates/execution/src/javascript.rs @@ -1,18 +1,36 @@ use crate::common::{encode_json_string, frozen_time_ms, stable_hash64}; -use crate::node_import_cache::{NodeImportCache, NODE_IMPORT_CACHE_ASSET_ROOT_ENV}; +use crate::node_import_cache::{ + NodeImportCache, NodeImportCacheCleanup, NODE_IMPORT_CACHE_ASSET_ROOT_ENV, +}; use crate::node_process::{ - apply_guest_env, encode_json_string_array, harden_node_command, node_binary, - node_resolution_read_paths, resolve_path_like_specifier, spawn_stream_reader, spawn_waiter, + apply_guest_env, configure_node_control_channel, create_node_control_channel, + encode_json_string_array, env_builtin_enabled, harden_node_command, node_binary, + node_resolution_read_paths, resolve_path_like_specifier, spawn_node_control_reader, + spawn_stream_reader, spawn_waiter, ExportedChildFds, LinePrefixFilter, NodeControlMessage, + NodeSignalHandlerRegistration, +}; +use crate::runtime_support::{ + configure_compile_cache, env_flag_enabled, import_cache_root, sandbox_root, warmup_marker_path, + NODE_COMPILE_CACHE_ENV, NODE_DISABLE_COMPILE_CACHE_ENV, NODE_FROZEN_TIME_ENV, + NODE_SANDBOX_ROOT_ENV, }; -use serde_json::from_str; +use nix::fcntl::OFlag; +use nix::unistd::pipe2; +use serde::Deserialize; +use serde_json::{from_str, json, Value}; use std::collections::BTreeMap; use std::fmt; -use std::fs; -use std::io::Write; +use std::fs::{self, File}; +use std::io::{BufRead, BufReader, BufWriter, Write}; +use std::os::fd::OwnedFd; use std::path::PathBuf; use std::process::{ChildStdin, Command, Stdio}; -use std::sync::mpsc::{self, Receiver, RecvTimeoutError}; -use std::time::Duration; +use std::sync::{ + mpsc::{self, Receiver, RecvTimeoutError, SyncSender, TrySendError}, + Arc, Mutex, +}; +use std::thread; +use std::time::{Duration, Instant}; const NODE_ENTRYPOINT_ENV: &str = "AGENT_OS_ENTRYPOINT"; const NODE_BOOTSTRAP_ENV: &str = "AGENT_OS_BOOTSTRAP_MODULE"; @@ -20,19 +38,31 @@ const NODE_GUEST_ARGV_ENV: &str = "AGENT_OS_GUEST_ARGV"; const NODE_PREWARM_IMPORTS_ENV: &str = "AGENT_OS_NODE_PREWARM_IMPORTS"; const NODE_WARMUP_DEBUG_ENV: &str = "AGENT_OS_NODE_WARMUP_DEBUG"; const NODE_WARMUP_METRICS_PREFIX: &str = "__AGENT_OS_NODE_WARMUP_METRICS__:"; -const NODE_COMPILE_CACHE_ENV: &str = "NODE_COMPILE_CACHE"; -const NODE_DISABLE_COMPILE_CACHE_ENV: &str = "NODE_DISABLE_COMPILE_CACHE"; const NODE_IMPORT_COMPILE_CACHE_NAMESPACE_VERSION: &str = "3"; const NODE_IMPORT_CACHE_LOADER_PATH_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_LOADER_PATH"; const NODE_IMPORT_CACHE_PATH_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_PATH"; -const NODE_FROZEN_TIME_ENV: &str = "AGENT_OS_FROZEN_TIME_MS"; const NODE_KEEP_STDIN_OPEN_ENV: &str = "AGENT_OS_KEEP_STDIN_OPEN"; const NODE_GUEST_ENTRYPOINT_ENV: &str = "AGENT_OS_GUEST_ENTRYPOINT"; const NODE_GUEST_PATH_MAPPINGS_ENV: &str = "AGENT_OS_GUEST_PATH_MAPPINGS"; +const NODE_VIRTUAL_PROCESS_EXEC_PATH_ENV: &str = "AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH"; +const NODE_VIRTUAL_PROCESS_PID_ENV: &str = "AGENT_OS_VIRTUAL_PROCESS_PID"; +const NODE_VIRTUAL_PROCESS_PPID_ENV: &str = "AGENT_OS_VIRTUAL_PROCESS_PPID"; +const NODE_VIRTUAL_PROCESS_UID_ENV: &str = "AGENT_OS_VIRTUAL_PROCESS_UID"; +const NODE_VIRTUAL_PROCESS_GID_ENV: &str = "AGENT_OS_VIRTUAL_PROCESS_GID"; +const NODE_PARENT_ALLOW_CHILD_PROCESS_ENV: &str = "AGENT_OS_PARENT_NODE_ALLOW_CHILD_PROCESS"; +const NODE_PARENT_ALLOW_WORKER_ENV: &str = "AGENT_OS_PARENT_NODE_ALLOW_WORKER"; const NODE_EXTRA_FS_READ_PATHS_ENV: &str = "AGENT_OS_EXTRA_FS_READ_PATHS"; const NODE_EXTRA_FS_WRITE_PATHS_ENV: &str = "AGENT_OS_EXTRA_FS_WRITE_PATHS"; const NODE_ALLOWED_BUILTINS_ENV: &str = "AGENT_OS_ALLOWED_NODE_BUILTINS"; const NODE_LOOPBACK_EXEMPT_PORTS_ENV: &str = "AGENT_OS_LOOPBACK_EXEMPT_PORTS"; +const NODE_SYNC_RPC_ENABLE_ENV: &str = "AGENT_OS_NODE_SYNC_RPC_ENABLE"; +const NODE_SYNC_RPC_REQUEST_FD_ENV: &str = "AGENT_OS_NODE_SYNC_RPC_REQUEST_FD"; +const NODE_SYNC_RPC_RESPONSE_FD_ENV: &str = "AGENT_OS_NODE_SYNC_RPC_RESPONSE_FD"; +const NODE_SYNC_RPC_DATA_BYTES_ENV: &str = "AGENT_OS_NODE_SYNC_RPC_DATA_BYTES"; +const NODE_SYNC_RPC_WAIT_TIMEOUT_MS_ENV: &str = "AGENT_OS_NODE_SYNC_RPC_WAIT_TIMEOUT_MS"; +const NODE_SYNC_RPC_DEFAULT_DATA_BYTES: usize = 4 * 1024 * 1024; +const NODE_SYNC_RPC_DEFAULT_WAIT_TIMEOUT_MS: u64 = 30_000; +const NODE_SYNC_RPC_RESPONSE_QUEUE_CAPACITY: usize = 1; const NODE_WARMUP_MARKER_VERSION: &str = "1"; const NODE_WARMUP_SPECIFIERS: &[&str] = &[ "agent-os:builtin/path", @@ -40,6 +70,8 @@ const NODE_WARMUP_SPECIFIERS: &[&str] = &[ "agent-os:builtin/fs-promises", "agent-os:polyfill/path", ]; +const CONTROLLED_STDERR_PREFIXES: &[&str] = + &[crate::node_import_cache::NODE_IMPORT_CACHE_METRICS_PREFIX]; const RESERVED_NODE_ENV_KEYS: &[&str] = &[ NODE_BOOTSTRAP_ENV, NODE_COMPILE_CACHE_ENV, @@ -47,18 +79,118 @@ const RESERVED_NODE_ENV_KEYS: &[&str] = &[ NODE_ENTRYPOINT_ENV, NODE_EXTRA_FS_READ_PATHS_ENV, NODE_EXTRA_FS_WRITE_PATHS_ENV, + NODE_SANDBOX_ROOT_ENV, NODE_FROZEN_TIME_ENV, NODE_GUEST_ENTRYPOINT_ENV, NODE_GUEST_ARGV_ENV, NODE_GUEST_PATH_MAPPINGS_ENV, + NODE_VIRTUAL_PROCESS_EXEC_PATH_ENV, + NODE_VIRTUAL_PROCESS_PID_ENV, + NODE_VIRTUAL_PROCESS_PPID_ENV, + NODE_VIRTUAL_PROCESS_UID_ENV, + NODE_VIRTUAL_PROCESS_GID_ENV, + NODE_PARENT_ALLOW_CHILD_PROCESS_ENV, + NODE_PARENT_ALLOW_WORKER_ENV, NODE_IMPORT_CACHE_ASSET_ROOT_ENV, NODE_IMPORT_CACHE_LOADER_PATH_ENV, NODE_IMPORT_CACHE_PATH_ENV, NODE_KEEP_STDIN_OPEN_ENV, NODE_ALLOWED_BUILTINS_ENV, NODE_LOOPBACK_EXEMPT_PORTS_ENV, + NODE_SYNC_RPC_ENABLE_ENV, + NODE_SYNC_RPC_REQUEST_FD_ENV, + NODE_SYNC_RPC_RESPONSE_FD_ENV, + NODE_SYNC_RPC_DATA_BYTES_ENV, + NODE_SYNC_RPC_WAIT_TIMEOUT_MS_ENV, ]; +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct JavascriptSyncRpcRequest { + pub id: u64, + pub method: String, + pub args: Vec, +} + +#[derive(Debug, Deserialize)] +struct JavascriptSyncRpcRequestWire { + id: u64, + method: String, + #[serde(default)] + args: Vec, +} + +struct JavascriptSyncRpcChannels { + parent_request_reader: File, + parent_response_writer: File, + child_request_writer: OwnedFd, + child_response_reader: OwnedFd, +} + +#[derive(Debug)] +struct JavascriptSyncRpcResponseWriter { + sender: SyncSender>, + timeout: Duration, +} + +impl JavascriptSyncRpcResponseWriter { + fn new(writer: File, timeout: Duration) -> Self { + let (sender, receiver) = mpsc::sync_channel(NODE_SYNC_RPC_RESPONSE_QUEUE_CAPACITY); + spawn_javascript_sync_rpc_response_writer(writer, receiver); + Self { sender, timeout } + } + + fn send(&self, payload: Vec) -> Result<(), JavascriptExecutionError> { + let started = Instant::now(); + let mut payload = Some(payload); + + loop { + match self + .sender + .try_send(payload.take().expect("payload should be present")) + { + Ok(()) => return Ok(()), + Err(TrySendError::Disconnected(_)) => { + return Err(JavascriptExecutionError::RpcResponse(String::from( + "JavaScript sync RPC response channel closed unexpectedly", + ))); + } + Err(TrySendError::Full(returned_payload)) => { + if started.elapsed() >= self.timeout { + return Err(JavascriptExecutionError::RpcResponse(format!( + "timed out after {}ms while queueing JavaScript sync RPC response", + self.timeout.as_millis() + ))); + } + payload = Some(returned_payload); + thread::sleep(Duration::from_millis(5)); + } + } + } + } +} + +impl Clone for JavascriptSyncRpcResponseWriter { + fn clone(&self) -> Self { + Self { + sender: self.sender.clone(), + timeout: self.timeout, + } + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum PendingSyncRpcState { + Pending(u64), + TimedOut(u64), +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum PendingSyncRpcResolution { + Pending, + TimedOut, + Missing, +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct CreateJavascriptContextRequest { pub vm_id: String, @@ -87,6 +219,20 @@ pub struct StartJavascriptExecutionRequest { pub enum JavascriptExecutionEvent { Stdout(Vec), Stderr(Vec), + SyncRpcRequest(JavascriptSyncRpcRequest), + SignalState { + signal: u32, + registration: NodeSignalHandlerRegistration, + }, + Exited(i32), +} + +#[derive(Debug, Clone, PartialEq, Eq)] +enum JavascriptProcessEvent { + Stdout(Vec), + RawStderr(Vec), + SyncRpcRequest(JavascriptSyncRpcRequest), + Control(NodeControlMessage), Exited(i32), } @@ -108,6 +254,10 @@ pub enum JavascriptExecutionError { WarmupSpawn(std::io::Error), WarmupFailed { exit_code: i32, stderr: String }, Spawn(std::io::Error), + PendingSyncRpcRequest(u64), + ExpiredSyncRpcRequest(u64), + RpcChannel(String), + RpcResponse(String), StdinClosed, Stdin(std::io::Error), EventChannelClosed, @@ -148,6 +298,27 @@ impl fmt::Display for JavascriptExecutionError { } } Self::Spawn(err) => write!(f, "failed to start guest JavaScript runtime: {err}"), + Self::PendingSyncRpcRequest(id) => { + write!( + f, + "guest JavaScript execution requires servicing pending sync RPC request {id}" + ) + } + Self::ExpiredSyncRpcRequest(id) => { + write!(f, "sync RPC request {id} is no longer pending") + } + Self::RpcChannel(message) => { + write!( + f, + "failed to configure guest JavaScript sync RPC channel: {message}" + ) + } + Self::RpcResponse(message) => { + write!( + f, + "failed to reply to guest JavaScript sync RPC request: {message}" + ) + } Self::StdinClosed => f.write_str("guest JavaScript stdin is already closed"), Self::Stdin(err) => write!(f, "failed to write guest stdin: {err}"), Self::EventChannelClosed => { @@ -164,7 +335,12 @@ pub struct JavascriptExecution { execution_id: String, child_pid: u32, stdin: Option, - events: Receiver, + events: Receiver, + stderr_filter: Arc>, + pending_sync_rpc: Arc>>, + sync_rpc_responses: Option, + sync_rpc_timeout: Duration, + _import_cache_guard: Arc, } impl JavascriptExecution { @@ -194,12 +370,118 @@ impl JavascriptExecution { Ok(()) } + pub fn respond_sync_rpc_success( + &mut self, + id: u64, + result: Value, + ) -> Result<(), JavascriptExecutionError> { + let Some(writer) = &self.sync_rpc_responses else { + return Err(JavascriptExecutionError::RpcResponse(String::from( + "no sync RPC channel is active for this JavaScript execution", + ))); + }; + + match self.clear_pending_sync_rpc(id)? { + PendingSyncRpcResolution::Pending => {} + PendingSyncRpcResolution::TimedOut => { + return Err(JavascriptExecutionError::ExpiredSyncRpcRequest(id)); + } + PendingSyncRpcResolution::Missing => {} + } + + write_javascript_sync_rpc_response( + writer, + json!({ + "id": id, + "ok": true, + "result": result, + }), + ) + } + + pub fn respond_sync_rpc_error( + &mut self, + id: u64, + code: impl Into, + message: impl Into, + ) -> Result<(), JavascriptExecutionError> { + let Some(writer) = &self.sync_rpc_responses else { + return Err(JavascriptExecutionError::RpcResponse(String::from( + "no sync RPC channel is active for this JavaScript execution", + ))); + }; + + match self.clear_pending_sync_rpc(id)? { + PendingSyncRpcResolution::Pending => {} + PendingSyncRpcResolution::TimedOut => { + return Err(JavascriptExecutionError::ExpiredSyncRpcRequest(id)); + } + PendingSyncRpcResolution::Missing => {} + } + + write_javascript_sync_rpc_response( + writer, + json!({ + "id": id, + "ok": false, + "error": { + "code": code.into(), + "message": message.into(), + }, + }), + ) + } + pub fn poll_event( &self, timeout: Duration, ) -> Result, JavascriptExecutionError> { match self.events.recv_timeout(timeout) { - Ok(event) => Ok(Some(event)), + Ok(JavascriptProcessEvent::Stdout(chunk)) => { + Ok(Some(JavascriptExecutionEvent::Stdout(chunk))) + } + Ok(JavascriptProcessEvent::RawStderr(chunk)) => { + let mut filter = self + .stderr_filter + .lock() + .map_err(|_| JavascriptExecutionError::EventChannelClosed)?; + let filtered = filter.filter_chunk(&chunk, CONTROLLED_STDERR_PREFIXES); + if filtered.is_empty() { + return Ok(None); + } + Ok(Some(JavascriptExecutionEvent::Stderr(filtered))) + } + Ok(JavascriptProcessEvent::SyncRpcRequest(request)) => { + self.set_pending_sync_rpc(request.id)?; + spawn_javascript_sync_rpc_timeout( + request.id, + self.sync_rpc_timeout, + self.pending_sync_rpc.clone(), + self.sync_rpc_responses.clone(), + ); + Ok(Some(JavascriptExecutionEvent::SyncRpcRequest(request))) + } + Ok(JavascriptProcessEvent::Control(NodeControlMessage::NodeImportCacheMetrics { + metrics, + })) => Ok(Some(JavascriptExecutionEvent::Stderr( + format!( + "{}{}\n", + crate::node_import_cache::NODE_IMPORT_CACHE_METRICS_PREFIX, + serde_json::to_string(&metrics).unwrap_or_else(|_| String::from("{}")) + ) + .into_bytes(), + ))), + Ok(JavascriptProcessEvent::Control(NodeControlMessage::SignalState { + signal, + registration, + })) => Ok(Some(JavascriptExecutionEvent::SignalState { + signal, + registration, + })), + Ok(JavascriptProcessEvent::Control(_)) => Ok(None), + Ok(JavascriptProcessEvent::Exited(code)) => { + Ok(Some(JavascriptExecutionEvent::Exited(code))) + } Err(RecvTimeoutError::Timeout) => Ok(None), Err(RecvTimeoutError::Disconnected) => { Err(JavascriptExecutionError::EventChannelClosed) @@ -215,9 +497,30 @@ impl JavascriptExecution { loop { match self.events.recv() { - Ok(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), - Ok(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), - Ok(JavascriptExecutionEvent::Exited(exit_code)) => { + Ok(JavascriptProcessEvent::Stdout(chunk)) => stdout.extend(chunk), + Ok(JavascriptProcessEvent::RawStderr(chunk)) => { + let mut filter = self + .stderr_filter + .lock() + .map_err(|_| JavascriptExecutionError::EventChannelClosed)?; + stderr.extend(filter.filter_chunk(&chunk, CONTROLLED_STDERR_PREFIXES)); + } + Ok(JavascriptProcessEvent::SyncRpcRequest(request)) => { + return Err(JavascriptExecutionError::PendingSyncRpcRequest(request.id)); + } + Ok(JavascriptProcessEvent::Control( + NodeControlMessage::NodeImportCacheMetrics { metrics }, + )) => stderr.extend( + format!( + "{}{}\n", + crate::node_import_cache::NODE_IMPORT_CACHE_METRICS_PREFIX, + serde_json::to_string(&metrics).unwrap_or_else(|_| String::from("{}")) + ) + .into_bytes(), + ), + Ok(JavascriptProcessEvent::Control(NodeControlMessage::SignalState { .. })) => {} + Ok(JavascriptProcessEvent::Control(_)) => {} + Ok(JavascriptProcessEvent::Exited(exit_code)) => { return Ok(JavascriptExecutionResult { execution_id: self.execution_id, exit_code, @@ -229,6 +532,37 @@ impl JavascriptExecution { } } } + + fn set_pending_sync_rpc(&self, id: u64) -> Result<(), JavascriptExecutionError> { + let mut pending = self.pending_sync_rpc.lock().map_err(|_| { + JavascriptExecutionError::RpcResponse(String::from( + "sync RPC pending-request state lock poisoned", + )) + })?; + *pending = Some(PendingSyncRpcState::Pending(id)); + Ok(()) + } + + fn clear_pending_sync_rpc( + &self, + id: u64, + ) -> Result { + let mut pending = self.pending_sync_rpc.lock().map_err(|_| { + JavascriptExecutionError::RpcResponse(String::from( + "sync RPC pending-request state lock poisoned", + )) + })?; + match *pending { + Some(PendingSyncRpcState::Pending(current)) if current == id => { + *pending = None; + Ok(PendingSyncRpcResolution::Pending) + } + Some(PendingSyncRpcState::TimedOut(current)) if current == id => { + Ok(PendingSyncRpcResolution::TimedOut) + } + _ => Ok(PendingSyncRpcResolution::Missing), + } + } } #[derive(Debug, Default)] @@ -236,12 +570,13 @@ pub struct JavascriptExecutionEngine { next_context_id: usize, next_execution_id: usize, contexts: BTreeMap, - import_cache: NodeImportCache, + import_caches: BTreeMap, } impl JavascriptExecutionEngine { pub fn create_context(&mut self, request: CreateJavascriptContextRequest) -> JavascriptContext { self.next_context_id += 1; + self.import_caches.entry(request.vm_id.clone()).or_default(); let context = JavascriptContext { context_id: format!("js-ctx-{}", self.next_context_id), @@ -277,16 +612,34 @@ impl JavascriptExecutionEngine { return Err(JavascriptExecutionError::EmptyArgv); } - self.import_cache - .ensure_materialized() - .map_err(JavascriptExecutionError::PrepareImportCache)?; let frozen_time_ms = frozen_time_ms(); - let warmup_metrics = - prewarm_node_import_path(&self.import_cache, &context, &request, frozen_time_ms)?; + let warmup_metrics = { + let import_cache = self.import_caches.entry(context.vm_id.clone()).or_default(); + import_cache + .ensure_materialized() + .map_err(JavascriptExecutionError::PrepareImportCache)?; + prewarm_node_import_path(import_cache, &context, &request, frozen_time_ms)? + }; self.next_execution_id += 1; let execution_id = format!("exec-{}", self.next_execution_id); - let mut child = create_node_child(&self.import_cache, &context, &request, frozen_time_ms)?; + let control_channel = + create_node_control_channel().map_err(JavascriptExecutionError::Spawn)?; + let sync_rpc_channels = Some(create_javascript_sync_rpc_channels()?); + let import_cache = self + .import_caches + .get(&context.vm_id) + .expect("vm import cache should exist after materialization"); + let import_cache_guard = import_cache.cleanup_guard(); + let sync_rpc_timeout = javascript_sync_rpc_timeout(&request); + let (mut child, sync_rpc_request_reader, sync_rpc_response_writer) = create_node_child( + import_cache, + &context, + &request, + frozen_time_ms, + &control_channel.child_writer, + sync_rpc_channels, + )?; let child_pid = child.id(); let stdin = child.stdin.take(); @@ -301,21 +654,30 @@ impl JavascriptExecutionEngine { let (sender, receiver) = mpsc::channel(); if let Some(metrics) = warmup_metrics { - let _ = sender.send(JavascriptExecutionEvent::Stderr(metrics)); + let _ = sender.send(JavascriptProcessEvent::RawStderr(metrics)); } let stdout_reader = - spawn_stream_reader(stdout, sender.clone(), JavascriptExecutionEvent::Stdout); + spawn_stream_reader(stdout, sender.clone(), JavascriptProcessEvent::Stdout); let stderr_reader = - spawn_stream_reader(stderr, sender.clone(), JavascriptExecutionEvent::Stderr); + spawn_stream_reader(stderr, sender.clone(), JavascriptProcessEvent::RawStderr); + if let Some(reader) = sync_rpc_request_reader { + let _sync_rpc_reader = spawn_javascript_sync_rpc_reader(reader, sender.clone()); + } + let _control_reader = spawn_node_control_reader( + control_channel.parent_reader, + sender.clone(), + JavascriptProcessEvent::Control, + |message| JavascriptProcessEvent::RawStderr(message.into_bytes()), + ); spawn_waiter( child, stdout_reader, stderr_reader, true, sender, - JavascriptExecutionEvent::Exited, - |message| JavascriptExecutionEvent::Stderr(message.into_bytes()), + JavascriptProcessEvent::Exited, + |message| JavascriptProcessEvent::RawStderr(message.into_bytes()), ); Ok(JavascriptExecution { @@ -323,8 +685,37 @@ impl JavascriptExecutionEngine { child_pid, stdin, events: receiver, + stderr_filter: Arc::new(Mutex::new(LinePrefixFilter::default())), + pending_sync_rpc: Arc::new(Mutex::new(None)), + sync_rpc_responses: sync_rpc_response_writer, + sync_rpc_timeout, + _import_cache_guard: import_cache_guard, }) } + + pub fn dispose_vm(&mut self, vm_id: &str) { + self.contexts.retain(|_, context| context.vm_id != vm_id); + self.import_caches.remove(vm_id); + } + + #[doc(hidden)] + #[allow(dead_code)] + pub fn materialize_import_cache_for_vm( + &mut self, + vm_id: &str, + ) -> Result<&std::path::Path, std::io::Error> { + let import_cache = self.import_caches.entry(vm_id.to_owned()).or_default(); + import_cache.ensure_materialized()?; + Ok(import_cache.cache_path()) + } + + #[doc(hidden)] + #[allow(dead_code)] + pub fn import_cache_path_for_vm(&self, vm_id: &str) -> Option<&std::path::Path> { + self.import_caches + .get(vm_id) + .map(NodeImportCache::cache_path) + } } fn prewarm_node_import_path( @@ -333,10 +724,7 @@ fn prewarm_node_import_path( request: &StartJavascriptExecutionRequest, frozen_time_ms: u128, ) -> Result>, JavascriptExecutionError> { - let debug_enabled = request - .env - .get(NODE_WARMUP_DEBUG_ENV) - .is_some_and(|value| value == "1"); + let debug_enabled = env_flag_enabled(&request.env, NODE_WARMUP_DEBUG_ENV); let Some(_compile_cache_dir) = &context.compile_cache_dir else { return Ok(warmup_metrics_line( @@ -347,7 +735,12 @@ fn prewarm_node_import_path( )); }; - let marker_path = warmup_marker_path(import_cache); + let marker_path = warmup_marker_path( + import_cache.prewarm_marker_dir(), + "node-import-prewarm", + NODE_WARMUP_MARKER_VERSION, + &warmup_marker_contents(), + ); if marker_path.exists() { return Ok(warmup_metrics_line( debug_enabled, @@ -406,7 +799,16 @@ fn create_node_child( context: &JavascriptContext, request: &StartJavascriptExecutionRequest, frozen_time_ms: u128, -) -> Result { + control_fd: &std::os::fd::OwnedFd, + sync_rpc_channels: Option, +) -> Result< + ( + std::process::Child, + Option, + Option, + ), + JavascriptExecutionError, +> { let guest_argv = encode_json_string_array(&request.argv[1..]); let mut command = Command::new(node_binary()); configure_node_sandbox(&mut command, import_cache, context, request)?; @@ -432,19 +834,81 @@ fn create_node_child( NODE_GUEST_PATH_MAPPINGS_ENV, NODE_KEEP_STDIN_OPEN_ENV, NODE_LOOPBACK_EXEMPT_PORTS_ENV, + NODE_VIRTUAL_PROCESS_EXEC_PATH_ENV, + NODE_VIRTUAL_PROCESS_PID_ENV, + NODE_VIRTUAL_PROCESS_PPID_ENV, + NODE_VIRTUAL_PROCESS_UID_ENV, + NODE_VIRTUAL_PROCESS_GID_ENV, ] { if let Some(value) = request.env.get(key) { command.env(key, value); } } + command.env( + NODE_PARENT_ALLOW_CHILD_PROCESS_ENV, + if inherited_node_permission_enabled(&request.env, NODE_PARENT_ALLOW_CHILD_PROCESS_ENV) + .unwrap_or_else(|| env_builtin_enabled(&request.env, "child_process")) + { + "1" + } else { + "0" + }, + ); + command.env( + NODE_PARENT_ALLOW_WORKER_ENV, + if inherited_node_permission_enabled(&request.env, NODE_PARENT_ALLOW_WORKER_ENV) + .unwrap_or_else(|| env_builtin_enabled(&request.env, "worker_threads")) + { + "1" + } else { + "0" + }, + ); if let Some(bootstrap_module) = &context.bootstrap_module { command.env(NODE_BOOTSTRAP_ENV, bootstrap_module); } + let channels = sync_rpc_channels.expect("JavaScript sync RPC channels should be configured"); + let mut exported_fds = ExportedChildFds::default(); + command + .env(NODE_SYNC_RPC_ENABLE_ENV, "1") + .env( + NODE_SYNC_RPC_DATA_BYTES_ENV, + NODE_SYNC_RPC_DEFAULT_DATA_BYTES.to_string(), + ) + .env( + NODE_SYNC_RPC_WAIT_TIMEOUT_MS_ENV, + javascript_sync_rpc_timeout(request).as_millis().to_string(), + ); + exported_fds + .export( + &mut command, + NODE_SYNC_RPC_REQUEST_FD_ENV, + &channels.child_request_writer, + ) + .map_err(|error| JavascriptExecutionError::RpcChannel(error.to_string()))?; + exported_fds + .export( + &mut command, + NODE_SYNC_RPC_RESPONSE_FD_ENV, + &channels.child_response_reader, + ) + .map_err(|error| JavascriptExecutionError::RpcChannel(error.to_string()))?; + let (sync_rpc_request_reader, sync_rpc_response_writer) = ( + Some(channels.parent_request_reader), + Some(JavascriptSyncRpcResponseWriter::new( + channels.parent_response_writer, + javascript_sync_rpc_timeout(request), + )), + ); + + configure_node_control_channel(&mut command, control_fd, &mut exported_fds) + .map_err(JavascriptExecutionError::Spawn)?; configure_node_command(&mut command, import_cache, context, frozen_time_ms)?; - command.spawn().map_err(JavascriptExecutionError::Spawn) + let child = command.spawn().map_err(JavascriptExecutionError::Spawn)?; + Ok((child, sync_rpc_request_reader, sync_rpc_response_writer)) } fn configure_node_sandbox( @@ -453,13 +917,10 @@ fn configure_node_sandbox( context: &JavascriptContext, request: &StartJavascriptExecutionRequest, ) -> Result<(), JavascriptExecutionError> { - let cache_root = import_cache - .cache_path() - .parent() - .unwrap_or(import_cache.asset_root()) - .to_path_buf(); + let sandbox_root = sandbox_root(&request.env, &request.cwd); + let cache_root = import_cache_root(import_cache, import_cache.asset_root()); let mut read_paths = vec![cache_root.clone()]; - let mut write_paths = vec![cache_root, request.cwd.clone()]; + let mut write_paths = vec![cache_root, sandbox_root.clone()]; if let Some(entrypoint_path) = resolve_path_like_specifier(&request.cwd, &request.argv[0]) { read_paths.push(entrypoint_path.clone()); @@ -505,16 +966,27 @@ fn configure_node_sandbox( harden_node_command( command, - &request.cwd, + &sandbox_root, &read_paths, &write_paths, true, false, - env_builtin_enabled(&request.env, "child_process"), + inherited_node_permission_enabled(&request.env, NODE_PARENT_ALLOW_WORKER_ENV) + .unwrap_or(true), + inherited_node_permission_enabled(&request.env, NODE_PARENT_ALLOW_CHILD_PROCESS_ENV) + .unwrap_or_else(|| env_builtin_enabled(&request.env, "child_process")), ); Ok(()) } +fn inherited_node_permission_enabled(env: &BTreeMap, key: &str) -> Option { + env.get(key).and_then(|value| match value.as_str() { + "1" | "true" => Some(true), + "0" | "false" => Some(false), + _ => None, + }) +} + fn parse_env_path_list(env: &BTreeMap, key: &str) -> Vec { env.get(key) .and_then(|value| from_str::>(value).ok()) @@ -524,12 +996,6 @@ fn parse_env_path_list(env: &BTreeMap, key: &str) -> Vec, builtin: &str) -> bool { - env.get(NODE_ALLOWED_BUILTINS_ENV) - .and_then(|value| from_str::>(value).ok()) - .is_some_and(|builtins| builtins.iter().any(|entry| entry == builtin)) -} - fn configure_node_command( command: &mut Command, import_cache: &NodeImportCache, @@ -546,22 +1012,13 @@ fn configure_node_command( .env(NODE_FROZEN_TIME_ENV, frozen_time_ms.to_string()); if let Some(compile_cache_dir) = &context.compile_cache_dir { - fs::create_dir_all(compile_cache_dir) + configure_compile_cache(command, compile_cache_dir) .map_err(JavascriptExecutionError::PrepareImportCache)?; - command.env_remove(NODE_DISABLE_COMPILE_CACHE_ENV); - command.env(NODE_COMPILE_CACHE_ENV, compile_cache_dir); } Ok(()) } -fn warmup_marker_path(import_cache: &NodeImportCache) -> PathBuf { - import_cache.prewarm_marker_dir().join(format!( - "node-import-prewarm-v{NODE_WARMUP_MARKER_VERSION}-{:016x}.stamp", - stable_hash64(warmup_marker_contents().as_bytes()), - )) -} - fn warmup_marker_contents() -> String { [ env!("CARGO_PKG_NAME"), @@ -622,3 +1079,235 @@ fn stable_compile_cache_namespace_hash() -> u64 { .as_bytes(), ) } + +fn create_javascript_sync_rpc_channels( +) -> Result { + let fd_reservations = (0..64) + .map(|_| File::open("/dev/null")) + .collect::, _>>() + .map_err(JavascriptExecutionError::PrepareImportCache)?; + let (parent_request_reader, child_request_writer) = pipe2(OFlag::O_CLOEXEC) + .map_err(|error| JavascriptExecutionError::RpcChannel(error.to_string()))?; + let (child_response_reader, parent_response_writer) = pipe2(OFlag::O_CLOEXEC) + .map_err(|error| JavascriptExecutionError::RpcChannel(error.to_string()))?; + drop(fd_reservations); + + Ok(JavascriptSyncRpcChannels { + parent_request_reader: File::from(parent_request_reader), + parent_response_writer: File::from(parent_response_writer), + child_request_writer, + child_response_reader, + }) +} + +fn javascript_sync_rpc_timeout(request: &StartJavascriptExecutionRequest) -> Duration { + let timeout_ms = request + .env + .get(NODE_SYNC_RPC_WAIT_TIMEOUT_MS_ENV) + .and_then(|value| value.parse::().ok()) + .unwrap_or(NODE_SYNC_RPC_DEFAULT_WAIT_TIMEOUT_MS); + Duration::from_millis(timeout_ms) +} + +fn spawn_javascript_sync_rpc_timeout( + id: u64, + timeout: Duration, + pending_state: Arc>>, + responses: Option, +) { + let Some(responses) = responses else { + return; + }; + + thread::spawn(move || { + thread::sleep(timeout); + + let should_timeout = match pending_state.lock() { + Ok(mut guard) if *guard == Some(PendingSyncRpcState::Pending(id)) => { + *guard = Some(PendingSyncRpcState::TimedOut(id)); + true + } + Ok(_) => false, + Err(_) => false, + }; + + if !should_timeout { + return; + } + + let _ = write_javascript_sync_rpc_response( + &responses, + json!({ + "id": id, + "ok": false, + "error": { + "code": "ERR_AGENT_OS_NODE_SYNC_RPC_TIMEOUT", + "message": format!( + "guest JavaScript sync RPC request {id} timed out after {}ms", + timeout.as_millis() + ), + }, + }), + ); + }); +} + +fn spawn_javascript_sync_rpc_reader( + reader: File, + sender: mpsc::Sender, +) -> std::thread::JoinHandle<()> { + std::thread::spawn(move || { + let mut reader = BufReader::new(reader); + let mut line = String::new(); + + loop { + line.clear(); + match reader.read_line(&mut line) { + Ok(0) => return, + Ok(_) => { + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + + match parse_javascript_sync_rpc_request(trimmed) { + Ok(request) => { + if sender + .send(JavascriptProcessEvent::SyncRpcRequest(request)) + .is_err() + { + return; + } + } + Err(message) => { + if sender + .send(JavascriptProcessEvent::RawStderr( + format!("{message}\n").into_bytes(), + )) + .is_err() + { + return; + } + } + } + } + Err(error) => { + let _ = sender.send(JavascriptProcessEvent::RawStderr( + format!("failed to read JavaScript sync RPC request: {error}\n") + .into_bytes(), + )); + return; + } + } + } + }) +} + +fn parse_javascript_sync_rpc_request(line: &str) -> Result { + let wire: JavascriptSyncRpcRequestWire = + serde_json::from_str(line).map_err(|error| error.to_string())?; + Ok(JavascriptSyncRpcRequest { + id: wire.id, + method: wire.method, + args: wire.args, + }) +} + +fn write_javascript_sync_rpc_response( + writer: &JavascriptSyncRpcResponseWriter, + response: Value, +) -> Result<(), JavascriptExecutionError> { + let mut payload = serde_json::to_vec(&response) + .map_err(|error| JavascriptExecutionError::RpcResponse(error.to_string()))?; + payload.push(b'\n'); + writer.send(payload) +} + +fn spawn_javascript_sync_rpc_response_writer( + writer: File, + receiver: Receiver>, +) -> thread::JoinHandle<()> { + thread::spawn(move || { + let mut writer = BufWriter::new(writer); + while let Ok(payload) = receiver.recv() { + if writer + .write_all(&payload) + .and_then(|()| writer.flush()) + .is_err() + { + return; + } + } + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use nix::fcntl::OFlag; + use nix::unistd::pipe2; + use serde_json::Value; + use std::io::BufRead; + + #[test] + fn javascript_sync_rpc_timeout_writes_clear_error_response() { + let (reader_fd, writer_fd) = pipe2(OFlag::O_CLOEXEC).expect("create pipe"); + let reader = File::from(reader_fd); + let writer = File::from(writer_fd); + let response_writer = + JavascriptSyncRpcResponseWriter::new(writer, Duration::from_millis(50)); + let pending = Arc::new(Mutex::new(Some(PendingSyncRpcState::Pending(7)))); + + spawn_javascript_sync_rpc_timeout( + 7, + Duration::from_millis(20), + pending.clone(), + Some(response_writer), + ); + + let mut line = String::new(); + let mut reader = BufReader::new(reader); + reader.read_line(&mut line).expect("read timeout response"); + + let response: Value = serde_json::from_str(line.trim()).expect("parse timeout response"); + assert_eq!(response["id"], Value::from(7)); + assert_eq!(response["ok"], Value::from(false)); + assert_eq!( + response["error"]["code"], + Value::String(String::from("ERR_AGENT_OS_NODE_SYNC_RPC_TIMEOUT")) + ); + assert!(response["error"]["message"] + .as_str() + .expect("timeout message") + .contains("timed out after 20ms")); + assert_eq!( + *pending.lock().expect("pending state lock"), + Some(PendingSyncRpcState::TimedOut(7)) + ); + } + + #[test] + fn javascript_sync_rpc_response_writer_times_out_when_queue_is_full() { + let (sender, _receiver) = mpsc::sync_channel(1); + let writer = JavascriptSyncRpcResponseWriter { + sender, + timeout: Duration::from_millis(30), + }; + + writer + .send(b"first\n".to_vec()) + .expect("queue first response"); + + let started = Instant::now(); + let error = writer + .send(b"second\n".to_vec()) + .expect_err("full queue should time out"); + assert!( + started.elapsed() >= Duration::from_millis(30), + "send should wait for the configured timeout" + ); + assert!(error + .to_string() + .contains("timed out after 30ms while queueing JavaScript sync RPC response")); + } +} diff --git a/crates/execution/src/lib.rs b/crates/execution/src/lib.rs index c149b25e0..8918441f8 100644 --- a/crates/execution/src/lib.rs +++ b/crates/execution/src/lib.rs @@ -5,6 +5,7 @@ mod common; mod node_import_cache; mod node_process; +mod runtime_support; pub mod benchmark; pub mod javascript; @@ -15,8 +16,9 @@ pub use agent_os_bridge::GuestRuntime; pub use javascript::{ CreateJavascriptContextRequest, JavascriptContext, JavascriptExecution, JavascriptExecutionEngine, JavascriptExecutionError, JavascriptExecutionEvent, - JavascriptExecutionResult, StartJavascriptExecutionRequest, + JavascriptExecutionResult, JavascriptSyncRpcRequest, StartJavascriptExecutionRequest, }; +pub use node_process::{NodeSignalDispositionAction, NodeSignalHandlerRegistration}; pub use python::{ CreatePythonContextRequest, PythonContext, PythonExecution, PythonExecutionEngine, PythonExecutionError, PythonExecutionEvent, PythonExecutionResult, PythonVfsRpcMethod, @@ -26,6 +28,7 @@ pub use python::{ pub use wasm::{ CreateWasmContextRequest, StartWasmExecutionRequest, WasmContext, WasmExecution, WasmExecutionEngine, WasmExecutionError, WasmExecutionEvent, WasmExecutionResult, + WasmPermissionTier, }; pub trait NativeExecutionBridge: agent_os_bridge::ExecutionBridge {} diff --git a/crates/execution/src/node_import_cache.rs b/crates/execution/src/node_import_cache.rs index f33745463..705eab196 100644 --- a/crates/execution/src/node_import_cache.rs +++ b/crates/execution/src/node_import_cache.rs @@ -1,8 +1,11 @@ +use std::collections::BTreeSet; use std::env; use std::fs; use std::io; use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Mutex, OnceLock}; +use std::time::Duration; pub(crate) const NODE_IMPORT_CACHE_DEBUG_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_DEBUG"; pub(crate) const NODE_IMPORT_CACHE_METRICS_PREFIX: &str = "__AGENT_OS_NODE_IMPORT_CACHE_METRICS__:"; @@ -11,8 +14,10 @@ pub(crate) const NODE_IMPORT_CACHE_ASSET_ROOT_ENV: &str = "AGENT_OS_NODE_IMPORT_ const NODE_IMPORT_CACHE_PATH_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_PATH"; const NODE_IMPORT_CACHE_LOADER_PATH_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_LOADER_PATH"; const NODE_IMPORT_CACHE_SCHEMA_VERSION: &str = "1"; -const NODE_IMPORT_CACHE_LOADER_VERSION: &str = "5"; -const NODE_IMPORT_CACHE_ASSET_VERSION: &str = "2"; +const NODE_IMPORT_CACHE_LOADER_VERSION: &str = "7"; +const NODE_IMPORT_CACHE_ASSET_VERSION: &str = "4"; +const NODE_IMPORT_CACHE_DIR_PREFIX: &str = "agent-os-node-import-cache"; +const DEFAULT_NODE_IMPORT_CACHE_MATERIALIZE_TIMEOUT: Duration = Duration::from_secs(30); const PYODIDE_DIST_DIR: &str = "pyodide-dist"; const AGENT_OS_BUILTIN_SPECIFIER_PREFIX: &str = "agent-os:builtin/"; const AGENT_OS_POLYFILL_SPECIFIER_PREFIX: &str = "agent-os:polyfill/"; @@ -30,6 +35,11 @@ const BUNDLED_PYTHON_DATEUTIL_WHL: &[u8] = const BUNDLED_PYTZ_WHL: &[u8] = include_bytes!("../assets/pyodide/pytz-2025.2-py2.py3-none-any.whl"); const BUNDLED_SIX_WHL: &[u8] = include_bytes!("../assets/pyodide/six-1.17.0-py2.py3-none-any.whl"); +const NODE_PYTHON_RUNNER_SOURCE: &str = include_str!("../assets/runners/python-runner.mjs"); + +static CLEANED_NODE_IMPORT_CACHE_ROOTS: OnceLock>> = OnceLock::new(); +#[cfg(test)] +static NODE_IMPORT_CACHE_TEST_MATERIALIZE_DELAY_MS: AtomicU64 = AtomicU64::new(0); #[derive(Clone, Copy)] struct BundledPyodidePackageAsset { @@ -68,12 +78,21 @@ import { fileURLToPath, pathToFileURL } from 'node:url'; const GUEST_PATH_MAPPINGS = parseGuestPathMappings(process.env.AGENT_OS_GUEST_PATH_MAPPINGS); const ALLOWED_BUILTINS = new Set(parseJsonArray(process.env.AGENT_OS_ALLOWED_NODE_BUILTINS)); const CACHE_PATH = process.env.__NODE_IMPORT_CACHE_PATH_ENV__; +const CACHE_ROOT = CACHE_PATH ? path.dirname(CACHE_PATH) : null; +const GUEST_INTERNAL_CACHE_ROOT = '/.agent-os/node-import-cache'; +const HOST_CWD = process.cwd(); +const DEFAULT_GUEST_CWD = + typeof process.env.AGENT_OS_VIRTUAL_OS_HOMEDIR === 'string' && + process.env.AGENT_OS_VIRTUAL_OS_HOMEDIR.startsWith('/') + ? path.posix.normalize(process.env.AGENT_OS_VIRTUAL_OS_HOMEDIR) + : '/root'; +const UNMAPPED_GUEST_PATH = '/unknown'; const PROJECTED_SOURCE_CACHE_ROOT = CACHE_PATH ? path.join(path.dirname(CACHE_PATH), 'projected-sources') : null; const ASSET_ROOT = process.env.__NODE_IMPORT_CACHE_ASSET_ROOT_ENV__; const DEBUG_ENABLED = process.env.__NODE_IMPORT_CACHE_DEBUG_ENV__ === '1'; -const METRICS_PREFIX = '__NODE_IMPORT_CACHE_METRICS_PREFIX__'; +const CONTROL_PIPE_FD = parseControlPipeFd(process.env.AGENT_OS_CONTROL_PIPE_FD); const SCHEMA_VERSION = '__NODE_IMPORT_CACHE_SCHEMA_VERSION__'; const LOADER_VERSION = '__NODE_IMPORT_CACHE_LOADER_VERSION__'; const ASSET_VERSION = '__NODE_IMPORT_CACHE_ASSET_VERSION__'; @@ -82,16 +101,28 @@ const POLYFILL_PREFIX = '__AGENT_OS_POLYFILL_SPECIFIER_PREFIX__'; const FS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}fs`; const FS_PROMISES_ASSET_SPECIFIER = `${BUILTIN_PREFIX}fs-promises`; const CHILD_PROCESS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}child-process`; +const NET_ASSET_SPECIFIER = `${BUILTIN_PREFIX}net`; +const DGRAM_ASSET_SPECIFIER = `${BUILTIN_PREFIX}dgram`; +const DNS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}dns`; +const HTTP_ASSET_SPECIFIER = `${BUILTIN_PREFIX}http`; +const HTTP2_ASSET_SPECIFIER = `${BUILTIN_PREFIX}http2`; +const HTTPS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}https`; +const TLS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}tls`; +const OS_ASSET_SPECIFIER = `${BUILTIN_PREFIX}os`; const DENIED_BUILTINS = new Set([ 'child_process', + 'cluster', 'dgram', + 'diagnostics_channel', 'dns', 'http', 'http2', 'https', 'inspector', + 'module', 'net', 'tls', + 'trace_events', 'v8', 'vm', 'worker_threads', @@ -201,7 +232,14 @@ export async function resolve(specifier, context, nextResolve) { } const translatedContext = translateContextParentUrl(context); - const resolved = await nextResolve(specifier, translatedContext); + let resolved; + try { + resolved = await nextResolve(specifier, translatedContext); + } catch (error) { + flushCacheState(); + emitMetrics(); + throw translateErrorToGuest(error); + } const translatedUrl = translateResolvedUrlToGuest(resolved.url); const translatedResolved = translatedUrl === resolved.url ? resolved : { ...resolved, url: translatedUrl }; @@ -226,34 +264,40 @@ export async function resolve(specifier, context, nextResolve) { } export async function load(url, context, nextLoad) { - const filePath = filePathFromUrl(url); - const format = lookupModuleFormat(url) ?? context.format; + try { + const filePath = filePathFromUrl(url); + const format = lookupModuleFormat(url) ?? context.format; - if (!filePath || !format || format === 'builtin') { - return nextLoad(url, context); - } + if (!filePath || !format || format === 'builtin') { + return await nextLoad(url, context); + } + + const projectedPackageSource = loadProjectedPackageSource(url, filePath, format); + if (projectedPackageSource != null) { + flushCacheState(); + emitMetrics(); + return { + shortCircuit: true, + format, + source: projectedPackageSource, + }; + } + + const source = + format === 'wasm' + ? fs.readFileSync(filePath) + : rewriteBuiltinImports(fs.readFileSync(filePath, 'utf8'), filePath); - const projectedPackageSource = loadProjectedPackageSource(url, filePath, format); - if (projectedPackageSource != null) { - flushCacheState(); - emitMetrics(); return { shortCircuit: true, format, - source: projectedPackageSource, + source, }; + } catch (error) { + flushCacheState(); + emitMetrics(); + throw translateErrorToGuest(error); } - - const source = - format === 'wasm' - ? fs.readFileSync(filePath) - : rewriteBuiltinImports(fs.readFileSync(filePath, 'utf8'), filePath); - - return { - shortCircuit: true, - format, - source, - }; } function loadCacheState() { @@ -310,10 +354,27 @@ function emitMetrics() { ? { ...metrics, cacheWriteError } : metrics; + emitControlMessage({ type: 'node_import_cache_metrics', metrics: payload }); +} + +function parseControlPipeFd(value) { + if (typeof value !== 'string' || value.trim() === '') { + return null; + } + + const parsed = Number.parseInt(value, 10); + return Number.isInteger(parsed) && parsed >= 0 ? parsed : null; +} + +function emitControlMessage(message) { + if (CONTROL_PIPE_FD == null) { + return; + } + try { - process.stderr.write(`${METRICS_PREFIX}${JSON.stringify(payload)}\n`); + fs.writeSync(CONTROL_PIPE_FD, `${JSON.stringify(message)}\n`); } catch { - // Ignore stderr write failures during teardown. + // Ignore control-channel write failures during teardown. } } @@ -500,6 +561,126 @@ function rewriteBuiltinImports(source, filePath) { } } + if (ALLOWED_BUILTINS.has('net')) { + for (const specifier of ['node:net', 'net']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + NET_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + NET_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('dgram')) { + for (const specifier of ['node:dgram', 'dgram']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + DGRAM_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + DGRAM_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('dns')) { + for (const specifier of ['node:dns', 'dns']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + DNS_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + DNS_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('http')) { + for (const specifier of ['node:http', 'http']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + HTTP_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + HTTP_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('http2')) { + for (const specifier of ['node:http2', 'http2']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + HTTP2_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + HTTP2_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('https')) { + for (const specifier of ['node:https', 'https']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + HTTPS_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + HTTPS_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('tls')) { + for (const specifier of ['node:tls', 'tls']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + TLS_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + TLS_ASSET_SPECIFIER, + ); + } + } + + if (ALLOWED_BUILTINS.has('os')) { + for (const specifier of ['node:os', 'os']) { + rewritten = replaceBuiltinImportSpecifier( + rewritten, + specifier, + OS_ASSET_SPECIFIER, + ); + rewritten = replaceBuiltinDynamicImportSpecifier( + rewritten, + specifier, + OS_ASSET_SPECIFIER, + ); + } + } + return rewritten; } @@ -577,6 +758,38 @@ function resolveBuiltinAsset(specifier, context) { return ALLOWED_BUILTINS.has('child_process') ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'child-process.mjs')) : null; + case 'net': + return ALLOWED_BUILTINS.has('net') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'net.mjs')) + : null; + case 'dgram': + return ALLOWED_BUILTINS.has('dgram') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'dgram.mjs')) + : null; + case 'dns': + return ALLOWED_BUILTINS.has('dns') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'dns.mjs')) + : null; + case 'http': + return ALLOWED_BUILTINS.has('http') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'http.mjs')) + : null; + case 'http2': + return ALLOWED_BUILTINS.has('http2') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'http2.mjs')) + : null; + case 'https': + return ALLOWED_BUILTINS.has('https') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'https.mjs')) + : null; + case 'tls': + return ALLOWED_BUILTINS.has('tls') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'tls.mjs')) + : null; + case 'os': + return ALLOWED_BUILTINS.has('os') + ? assetModuleDescriptor(path.join(ASSET_ROOT, 'builtins', 'os.mjs')) + : null; default: return null; } @@ -1148,8 +1361,7 @@ function translateResolvedUrlToGuest(url) { return url; } - const guestPath = guestPathFromHostPath(hostPath); - return guestPath ? pathToFileURL(guestPath).href : url; + return pathToFileURL(guestVisiblePathFromHostPath(hostPath)).href; } function translateResolvedUrlToHost(url) { @@ -1197,6 +1409,18 @@ function hostPathFromGuestPath(guestPath) { } const normalized = path.posix.normalize(guestPath); + if ( + CACHE_ROOT && + (normalized === GUEST_INTERNAL_CACHE_ROOT || + normalized.startsWith(`${GUEST_INTERNAL_CACHE_ROOT}/`)) + ) { + const suffix = + normalized === GUEST_INTERNAL_CACHE_ROOT + ? '' + : normalized.slice(GUEST_INTERNAL_CACHE_ROOT.length + 1); + return suffix ? path.join(CACHE_ROOT, ...suffix.split('/')) : CACHE_ROOT; + } + for (const mapping of GUEST_PATH_MAPPINGS) { if (mapping.guestPath === '/') { const suffix = normalized.replace(/^\/+/, ''); @@ -1217,6 +1441,17 @@ function hostPathFromGuestPath(guestPath) { return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath; } + if ( + normalized === DEFAULT_GUEST_CWD || + normalized.startsWith(`${DEFAULT_GUEST_CWD}/`) + ) { + const suffix = + normalized === DEFAULT_GUEST_CWD + ? '' + : normalized.slice(DEFAULT_GUEST_CWD.length + 1); + return suffix ? path.join(HOST_CWD, ...suffix.split('/')) : HOST_CWD; + } + return null; } @@ -1250,125 +1485,493 @@ function guestPathFromHostPath(hostPath) { return null; } -function pathExists(targetPath) { - try { - return fs.existsSync(targetPath); - } catch { - return false; +function guestCwdPathFromHostPath(hostPath) { + if (typeof hostPath !== 'string') { + return null; } -} -function safeRealpath(targetPath) { - try { - return fs.realpathSync.native(targetPath); - } catch { + const normalized = path.resolve(hostPath); + const hostRoot = path.resolve(HOST_CWD); + if ( + normalized !== hostRoot && + !normalized.startsWith(`${hostRoot}${path.sep}`) + ) { return null; } + + const suffix = + normalized === hostRoot + ? '' + : normalized.slice(hostRoot.length + path.sep.length); + return suffix + ? path.posix.join(DEFAULT_GUEST_CWD, suffix.split(path.sep).join('/')) + : DEFAULT_GUEST_CWD; } -function parseJsonArray(value) { - if (!value) { - return []; +function guestInternalPathFromHostPath(hostPath) { + if (typeof hostPath !== 'string' || !CACHE_ROOT) { + return null; } - try { - const parsed = JSON.parse(value); - return Array.isArray(parsed) ? parsed.filter((entry) => typeof entry === 'string') : []; - } catch { - return []; + const normalized = path.resolve(hostPath); + const hostRoot = path.resolve(CACHE_ROOT); + if ( + normalized !== hostRoot && + !normalized.startsWith(`${hostRoot}${path.sep}`) + ) { + return null; } + + const suffix = + normalized === hostRoot + ? '' + : normalized.slice(hostRoot.length + path.sep.length); + return suffix + ? path.posix.join(GUEST_INTERNAL_CACHE_ROOT, suffix.split(path.sep).join('/')) + : GUEST_INTERNAL_CACHE_ROOT; } -function isInternalImportCachePath(filePath) { - return typeof filePath === 'string' && filePath.includes(`${path.sep}agent-os-node-import-cache-`); +function guestVisiblePathFromHostPath(hostPath) { + return ( + guestPathFromHostPath(hostPath) ?? + guestInternalPathFromHostPath(hostPath) ?? + guestCwdPathFromHostPath(hostPath) ?? + UNMAPPED_GUEST_PATH + ); } -function parseGuestPathMappings(value) { - const parsed = parseJsonArrayLikeObjects(value); - return parsed - .map((entry) => { - const guestPath = - typeof entry.guestPath === 'string' - ? path.posix.normalize(entry.guestPath) - : null; - const hostPath = - typeof entry.hostPath === 'string' ? path.resolve(entry.hostPath) : null; - return guestPath && hostPath ? { guestPath, hostPath } : null; - }) - .filter(Boolean) - .sort((left, right) => { - if (right.guestPath.length !== left.guestPath.length) { - return right.guestPath.length - left.guestPath.length; - } - return right.hostPath.length - left.hostPath.length; - }); +function isGuestVisiblePath(value) { + if (typeof value !== 'string' || !path.posix.isAbsolute(value)) { + return false; + } + + const normalized = path.posix.normalize(value); + return ( + normalized === UNMAPPED_GUEST_PATH || + normalized === GUEST_INTERNAL_CACHE_ROOT || + normalized.startsWith(`${GUEST_INTERNAL_CACHE_ROOT}/`) || + normalized === DEFAULT_GUEST_CWD || + normalized.startsWith(`${DEFAULT_GUEST_CWD}/`) || + hostPathFromGuestPath(normalized) != null + ); } -function parseJsonArrayLikeObjects(value) { - if (!value) { - return []; +function translatePathStringToGuest(value) { + if (typeof value !== 'string') { + return value; } - try { - const parsed = JSON.parse(value); - return Array.isArray(parsed) ? parsed.filter(isRecord) : []; - } catch { - return []; + if (value.startsWith('file:')) { + const hostPath = guestFilePathFromUrl(value); + if (!hostPath) { + return value; + } + + const guestPath = isGuestVisiblePath(hostPath) + ? path.posix.normalize(hostPath) + : guestVisiblePathFromHostPath(hostPath); + return pathToFileURL(guestPath).href; } -} -function hashString(contents) { - return crypto.createHash('sha256').update(contents).digest('hex'); -} + if (!path.isAbsolute(value)) { + return value; + } -function isRecord(value) { - return value != null && typeof value === 'object' && !Array.isArray(value); + return isGuestVisiblePath(value) + ? path.posix.normalize(value) + : guestVisiblePathFromHostPath(value); } -"#; -const NODE_IMPORT_CACHE_REGISTER_SOURCE: &str = r#" -import { register } from 'node:module'; +function buildHostToGuestTextReplacements() { + const replacements = new Map(); + const addReplacement = (hostValue, guestValue) => { + if ( + typeof hostValue !== 'string' || + hostValue.length === 0 || + typeof guestValue !== 'string' || + guestValue.length === 0 + ) { + return; + } -const loaderPath = process.env.__NODE_IMPORT_CACHE_LOADER_PATH_ENV__; + replacements.set(hostValue, guestValue); + }; -if (!loaderPath) { - throw new Error('__NODE_IMPORT_CACHE_LOADER_PATH_ENV__ is required'); + for (const mapping of GUEST_PATH_MAPPINGS) { + const hostRoot = path.resolve(mapping.hostPath); + addReplacement(hostRoot, mapping.guestPath); + addReplacement(pathToFileURL(hostRoot).href, pathToFileURL(mapping.guestPath).href); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, mapping.guestPath); + } + } + + if (CACHE_ROOT) { + const hostRoot = path.resolve(CACHE_ROOT); + addReplacement(hostRoot, GUEST_INTERNAL_CACHE_ROOT); + addReplacement( + pathToFileURL(hostRoot).href, + pathToFileURL(GUEST_INTERNAL_CACHE_ROOT).href, + ); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, GUEST_INTERNAL_CACHE_ROOT); + } + } + + if (!guestPathFromHostPath(HOST_CWD)) { + const hostRoot = path.resolve(HOST_CWD); + addReplacement(hostRoot, DEFAULT_GUEST_CWD); + addReplacement(pathToFileURL(hostRoot).href, pathToFileURL(DEFAULT_GUEST_CWD).href); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, DEFAULT_GUEST_CWD); + } + } + + return [...replacements.entries()].sort((left, right) => right[0].length - left[0].length); } -register(loaderPath, import.meta.url); -"#; +function splitPathLocationSuffix(value) { + if (typeof value !== 'string') { + return { pathLike: value, suffix: '' }; + } -const NODE_EXECUTION_RUNNER_SOURCE: &str = r#" -import fs from 'node:fs'; -import Module, { syncBuiltinESMExports } from 'node:module'; -import path from 'node:path'; -import { pathToFileURL } from 'node:url'; + const match = /^(.*?)(:\d+(?::\d+)?)$/.exec(value); + return match + ? { pathLike: match[1], suffix: match[2] } + : { pathLike: value, suffix: '' }; +} -const GUEST_PATH_MAPPINGS = parseGuestPathMappings(process.env.AGENT_OS_GUEST_PATH_MAPPINGS); -const ALLOWED_BUILTINS = new Set(parseJsonArray(process.env.AGENT_OS_ALLOWED_NODE_BUILTINS)); -const LOOPBACK_EXEMPT_PORTS = new Set(parseJsonArray(process.env.AGENT_OS_LOOPBACK_EXEMPT_PORTS)); +function translateTextTokenToGuest(token) { + if (typeof token !== 'string' || token.length === 0) { + return token; + } + + const leading = token.match(/^[("'`[{<]+/)?.[0] ?? ''; + const trailing = token.match(/[)"'`\]}>.,;!?]+$/)?.[0] ?? ''; + const coreEnd = token.length - trailing.length; + const core = token.slice(leading.length, coreEnd); + if (core.length === 0) { + return token; + } + + const { pathLike, suffix } = splitPathLocationSuffix(core); + if ( + typeof pathLike !== 'string' || + (!pathLike.startsWith('file:') && !path.isAbsolute(pathLike)) + ) { + return token; + } + + return `${leading}${translatePathStringToGuest(pathLike)}${suffix}${trailing}`; +} + +function translateTextToGuest(value) { + if (typeof value !== 'string' || value.length === 0) { + return value; + } + + let translated = value; + for (const [hostValue, guestValue] of buildHostToGuestTextReplacements()) { + translated = translated.split(hostValue).join(guestValue); + } + + return translated + .split(/(\s+)/) + .map((token) => (/^\s+$/.test(token) ? token : translateTextTokenToGuest(token))) + .join(''); +} + +function translateErrorToGuest(error) { + if (error == null || typeof error !== 'object') { + return error; + } + + if (typeof error.message === 'string') { + try { + error.message = translateTextToGuest(error.message); + } catch { + // Ignore readonly message bindings. + } + } + + if (typeof error.stack === 'string') { + try { + error.stack = translateTextToGuest(error.stack); + } catch { + // Ignore readonly stack bindings. + } + } + + if (typeof error.path === 'string') { + try { + error.path = translatePathStringToGuest(error.path); + } catch { + // Ignore readonly path bindings. + } + } + + if (typeof error.filename === 'string') { + try { + error.filename = translatePathStringToGuest(error.filename); + } catch { + // Ignore readonly filename bindings. + } + } + + if (typeof error.url === 'string') { + try { + error.url = translatePathStringToGuest(error.url); + } catch { + // Ignore readonly url bindings. + } + } + + if (Array.isArray(error.requireStack)) { + try { + error.requireStack = error.requireStack.map((entry) => translatePathStringToGuest(entry)); + } catch { + // Ignore readonly requireStack bindings. + } + } + + return error; +} + +function pathExists(targetPath) { + try { + return fs.existsSync(targetPath); + } catch { + return false; + } +} + +function safeRealpath(targetPath) { + try { + return fs.realpathSync.native(targetPath); + } catch { + return null; + } +} + +function parseJsonArray(value) { + if (!value) { + return []; + } + + try { + const parsed = JSON.parse(value); + return Array.isArray(parsed) ? parsed.filter((entry) => typeof entry === 'string') : []; + } catch { + return []; + } +} + +function isInternalImportCachePath(filePath) { + return typeof filePath === 'string' && filePath.includes(`${path.sep}agent-os-node-import-cache-`); +} + +function parseGuestPathMappings(value) { + const parsed = parseJsonArrayLikeObjects(value); + return parsed + .map((entry) => { + const guestPath = + typeof entry.guestPath === 'string' + ? path.posix.normalize(entry.guestPath) + : null; + const hostPath = + typeof entry.hostPath === 'string' ? path.resolve(entry.hostPath) : null; + return guestPath && hostPath ? { guestPath, hostPath } : null; + }) + .filter(Boolean) + .sort((left, right) => { + if (right.guestPath.length !== left.guestPath.length) { + return right.guestPath.length - left.guestPath.length; + } + return right.hostPath.length - left.hostPath.length; + }); +} + +function parseJsonArrayLikeObjects(value) { + if (!value) { + return []; + } + + try { + const parsed = JSON.parse(value); + return Array.isArray(parsed) ? parsed.filter(isRecord) : []; + } catch { + return []; + } +} + +function hashString(contents) { + return crypto.createHash('sha256').update(contents).digest('hex'); +} + +function isRecord(value) { + return value != null && typeof value === 'object' && !Array.isArray(value); +} +"#; + +const NODE_IMPORT_CACHE_REGISTER_SOURCE: &str = r#" +import { register } from 'node:module'; + +const loaderPath = process.env.__NODE_IMPORT_CACHE_LOADER_PATH_ENV__; + +if (!loaderPath) { + throw new Error('__NODE_IMPORT_CACHE_LOADER_PATH_ENV__ is required'); +} + +register(loaderPath, import.meta.url); +"#; + +const NODE_EXECUTION_RUNNER_SOURCE: &str = r#" +const fs = process.getBuiltinModule?.('node:fs'); +const path = process.getBuiltinModule?.('node:path'); +const { pathToFileURL } = process.getBuiltinModule?.('node:url') ?? {}; + +if (!fs || !path || typeof pathToFileURL !== 'function') { + throw new Error('node builtin access is required for the Agent OS guest runtime'); +} + +const HOST_PROCESS_ENV = { ...process.env }; +const Module = + typeof process.getBuiltinModule === 'function' + ? process.getBuiltinModule('node:module') + : null; +const syncBuiltinESMExports = + typeof Module?.syncBuiltinESMExports === 'function' + ? Module.syncBuiltinESMExports.bind(Module) + : () => {}; +const GUEST_PATH_MAPPINGS = parseGuestPathMappings(HOST_PROCESS_ENV.AGENT_OS_GUEST_PATH_MAPPINGS); +const ALLOWED_BUILTINS = new Set(parseJsonArray(HOST_PROCESS_ENV.AGENT_OS_ALLOWED_NODE_BUILTINS)); +const LOOPBACK_EXEMPT_PORTS = new Set(parseJsonArray(HOST_PROCESS_ENV.AGENT_OS_LOOPBACK_EXEMPT_PORTS)); const DENIED_BUILTINS = new Set([ 'child_process', + 'cluster', 'dgram', + 'diagnostics_channel', 'dns', 'http', 'http2', 'https', 'inspector', + 'module', 'net', 'tls', + 'trace_events', 'v8', 'vm', 'worker_threads', ].filter((name) => !ALLOWED_BUILTINS.has(name))); +const originalGetBuiltinModule = + typeof process.getBuiltinModule === 'function' + ? process.getBuiltinModule.bind(process) + : null; +const originalModuleResolveFilename = + typeof Module?._resolveFilename === 'function' + ? Module._resolveFilename.bind(Module) + : null; const originalModuleLoad = - typeof Module._load === 'function' ? Module._load.bind(Module) : null; + typeof Module?._load === 'function' ? Module._load.bind(Module) : null; +const originalModuleCache = + Module?._cache && typeof Module._cache === 'object' ? Module._cache : null; const originalFetch = typeof globalThis.fetch === 'function' ? globalThis.fetch.bind(globalThis) : null; +const HOST_CWD = process.cwd(); +const HOST_EXEC_PATH = process.execPath; +const HOST_EXEC_DIR = path.dirname(HOST_EXEC_PATH); +if (!Module || typeof Module.createRequire !== 'function') { + throw new Error('node:module builtin access is required for the Agent OS guest runtime'); +} const hostRequire = Module.createRequire(import.meta.url); -const guestEntryPoint = process.env.AGENT_OS_GUEST_ENTRYPOINT ?? process.env.AGENT_OS_ENTRYPOINT; +const hostOs = hostRequire('node:os'); +const hostNet = hostRequire('node:net'); +const hostDgram = hostRequire('node:dgram'); +const hostDns = hostRequire('node:dns'); +const hostHttp = hostRequire('node:http'); +const hostHttp2 = hostRequire('node:http2'); +const hostHttps = hostRequire('node:https'); +const hostTls = hostRequire('node:tls'); +const { EventEmitter } = hostRequire('node:events'); +const { Duplex, Readable, Writable } = hostRequire('node:stream'); +const NODE_SYNC_RPC_ENABLE = HOST_PROCESS_ENV.AGENT_OS_NODE_SYNC_RPC_ENABLE === '1'; +const hostWorkerThreads = NODE_SYNC_RPC_ENABLE ? hostRequire('node:worker_threads') : null; +const SIGNAL_EVENTS = new Set( + Object.keys(hostOs.constants?.signals ?? {}).filter((name) => + name.startsWith('SIG'), + ), +); +const TRACKED_PROCESS_SIGNAL_EVENTS = new Set(['SIGCHLD']); +const guestEntryPoint = + HOST_PROCESS_ENV.AGENT_OS_GUEST_ENTRYPOINT ?? HOST_PROCESS_ENV.AGENT_OS_ENTRYPOINT; +const DEFAULT_VIRTUAL_EXEC_PATH = '/usr/bin/node'; +const DEFAULT_VIRTUAL_PID = 1; +const DEFAULT_VIRTUAL_PPID = 0; +const DEFAULT_VIRTUAL_UID = 0; +const DEFAULT_VIRTUAL_GID = 0; +const DEFAULT_VIRTUAL_OS_HOSTNAME = 'agent-os'; +const DEFAULT_VIRTUAL_OS_TYPE = 'Linux'; +const DEFAULT_VIRTUAL_OS_PLATFORM = 'linux'; +const DEFAULT_VIRTUAL_OS_RELEASE = '6.8.0-agent-os'; +const DEFAULT_VIRTUAL_OS_VERSION = '#1 SMP PREEMPT_DYNAMIC Agent OS'; +const DEFAULT_VIRTUAL_OS_ARCH = 'x64'; +const DEFAULT_VIRTUAL_OS_MACHINE = 'x86_64'; +const DEFAULT_VIRTUAL_OS_CPU_MODEL = 'Agent OS Virtual CPU'; +const DEFAULT_VIRTUAL_OS_CPU_COUNT = 1; +const DEFAULT_VIRTUAL_OS_TOTALMEM = 1024 * 1024 * 1024; +const DEFAULT_VIRTUAL_OS_FREEMEM = 768 * 1024 * 1024; +const DEFAULT_VIRTUAL_OS_USER = 'root'; +const DEFAULT_VIRTUAL_OS_HOMEDIR = '/root'; +const DEFAULT_VIRTUAL_OS_SHELL = '/bin/sh'; +const DEFAULT_VIRTUAL_OS_TMPDIR = '/tmp'; +const NODE_SYNC_RPC_REQUEST_FD = parseOptionalFd(HOST_PROCESS_ENV.AGENT_OS_NODE_SYNC_RPC_REQUEST_FD); +const NODE_SYNC_RPC_RESPONSE_FD = parseOptionalFd(HOST_PROCESS_ENV.AGENT_OS_NODE_SYNC_RPC_RESPONSE_FD); +const NODE_SYNC_RPC_DATA_BYTES = parsePositiveInt( + HOST_PROCESS_ENV.AGENT_OS_NODE_SYNC_RPC_DATA_BYTES, + 4 * 1024 * 1024, +); +const NODE_SYNC_RPC_WAIT_TIMEOUT_MS = parsePositiveInt( + HOST_PROCESS_ENV.AGENT_OS_NODE_SYNC_RPC_WAIT_TIMEOUT_MS, + 30_000, +); +const NODE_IMPORT_CACHE_PATH = HOST_PROCESS_ENV.AGENT_OS_NODE_IMPORT_CACHE_PATH ?? null; +const NODE_IMPORT_CACHE_ROOT = + typeof NODE_IMPORT_CACHE_PATH === 'string' && NODE_IMPORT_CACHE_PATH.length > 0 + ? path.dirname(NODE_IMPORT_CACHE_PATH) + : null; +const CONTROL_PIPE_FD = parseOptionalFd(HOST_PROCESS_ENV.AGENT_OS_CONTROL_PIPE_FD); +const GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT = '/.agent-os/node-import-cache'; +const UNMAPPED_GUEST_PATH = '/unknown'; +const VIRTUAL_EXEC_PATH = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH, + DEFAULT_VIRTUAL_EXEC_PATH, +); +const VIRTUAL_PID = parseVirtualProcessNumber( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_PID, + DEFAULT_VIRTUAL_PID, +); +const VIRTUAL_PPID = parseVirtualProcessNumber( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_PPID, + DEFAULT_VIRTUAL_PPID, +); +const VIRTUAL_UID = parseVirtualProcessNumber( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_UID, + DEFAULT_VIRTUAL_UID, +); +const VIRTUAL_GID = parseVirtualProcessNumber( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_GID, + DEFAULT_VIRTUAL_GID, +); +const DEFAULT_GUEST_CWD = resolveVirtualPath( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_HOMEDIR, + DEFAULT_VIRTUAL_OS_HOMEDIR, +); function isPathLike(specifier) { return specifier.startsWith('.') || specifier.startsWith('/') || specifier.startsWith('file:'); @@ -1376,15 +1979,17 @@ function isPathLike(specifier) { function toImportSpecifier(specifier) { if (specifier.startsWith('file:')) { - return specifier; + return translatePathStringToGuest(specifier); } if (isPathLike(specifier)) { if (specifier.startsWith('/')) { return pathToFileURL( - pathExists(specifier) ? path.resolve(specifier) : path.posix.normalize(specifier), + translatePathStringToGuest( + pathExists(specifier) ? path.resolve(specifier) : path.posix.normalize(specifier), + ), ).href; } - return pathToFileURL(path.resolve(process.cwd(), specifier)).href; + return pathToFileURL(translatePathStringToGuest(path.resolve(HOST_CWD, specifier))).href; } return specifier; } @@ -1438,6 +2043,101 @@ function parseJsonArray(value) { } } +function parseOptionalFd(value) { + if (value == null || value === '') { + return null; + } + + const parsed = Number.parseInt(value, 10); + return Number.isInteger(parsed) && parsed >= 0 ? parsed : null; +} + +function parsePositiveInt(value, fallback) { + if (value == null || value === '') { + return fallback; + } + + const parsed = Number(value); + return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback; +} + +function parseVirtualProcessNumber(value, fallback) { + if (value == null || value === '') { + return fallback; + } + + const parsed = Number(value); + return Number.isInteger(parsed) && parsed >= 0 ? parsed : fallback; +} + +function parseVirtualProcessString(value, fallback) { + return typeof value === 'string' && value.length > 0 ? value : fallback; +} + +function isInternalProcessEnvKey(key) { + return typeof key === 'string' && key.startsWith('AGENT_OS_'); +} + +function createGuestProcessEnv(env) { + const guestEnv = {}; + + for (const [key, value] of Object.entries(env ?? {})) { + if (typeof value !== 'string' || isInternalProcessEnvKey(key)) { + continue; + } + guestEnv[key] = value; + } + + return new Proxy(guestEnv, { + defineProperty(target, key, descriptor) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return true; + } + + const normalized = { ...descriptor }; + if ('value' in normalized) { + normalized.value = String(normalized.value); + } + return Reflect.defineProperty(target, key, normalized); + }, + deleteProperty(target, key) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return true; + } + return Reflect.deleteProperty(target, key); + }, + get(target, key, receiver) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return undefined; + } + return Reflect.get(target, key, receiver); + }, + getOwnPropertyDescriptor(target, key) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return undefined; + } + return Reflect.getOwnPropertyDescriptor(target, key); + }, + has(target, key) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return false; + } + return Reflect.has(target, key); + }, + ownKeys(target) { + return Reflect.ownKeys(target).filter( + (key) => typeof key !== 'string' || !isInternalProcessEnvKey(key), + ); + }, + set(target, key, value, receiver) { + if (typeof key === 'string' && isInternalProcessEnvKey(key)) { + return true; + } + return Reflect.set(target, key, String(value), receiver); + }, + }); +} + function parseGuestPathMappings(value) { if (!value) { return []; @@ -1474,6 +2174,20 @@ function hostPathFromGuestPath(guestPath) { } const normalized = path.posix.normalize(guestPath); + if ( + NODE_IMPORT_CACHE_ROOT && + (normalized === GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT || + normalized.startsWith(`${GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT}/`)) + ) { + const suffix = + normalized === GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT + ? '' + : normalized.slice(GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT.length + 1); + return suffix + ? path.join(NODE_IMPORT_CACHE_ROOT, ...suffix.split('/')) + : NODE_IMPORT_CACHE_ROOT; + } + for (const mapping of GUEST_PATH_MAPPINGS) { if (mapping.guestPath === '/') { const suffix = normalized.replace(/^\/+/, ''); @@ -1494,1800 +2208,5920 @@ function hostPathFromGuestPath(guestPath) { return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath; } + if ( + normalized === DEFAULT_GUEST_CWD || + normalized.startsWith(`${DEFAULT_GUEST_CWD}/`) + ) { + const suffix = + normalized === DEFAULT_GUEST_CWD + ? '' + : normalized.slice(DEFAULT_GUEST_CWD.length + 1); + return suffix ? path.join(HOST_CWD, ...suffix.split('/')) : HOST_CWD; + } + return null; } -function hostPathForSpecifier(specifier, fromGuestDir) { - if (typeof specifier !== 'string') { +function guestPathFromHostPath(hostPath) { + if (typeof hostPath !== 'string') { return null; } - if (specifier.startsWith('file:')) { - try { - return hostPathFromGuestPath(new URL(specifier).pathname); - } catch { - return null; + const normalized = path.resolve(hostPath); + for (const mapping of GUEST_PATH_MAPPINGS) { + const hostRoot = path.resolve(mapping.hostPath); + if ( + normalized !== hostRoot && + !normalized.startsWith(`${hostRoot}${path.sep}`) + ) { + continue; } - } - - if (specifier.startsWith('/')) { - return hostPathFromGuestPath(specifier); - } - if (specifier.startsWith('./') || specifier.startsWith('../')) { - return hostPathFromGuestPath( - path.posix.normalize(path.posix.join(fromGuestDir, specifier)), - ); + const suffix = + normalized === hostRoot + ? '' + : normalized.slice(hostRoot.length + path.sep.length); + return suffix + ? path.posix.join(mapping.guestPath, suffix.split(path.sep).join('/')) + : mapping.guestPath; } return null; } -function translateGuestPath(value, fromGuestDir = '/') { - if (typeof value !== 'string') { - return value; +function guestCwdPathFromHostPath(hostPath) { + if (typeof hostPath !== 'string') { + return null; } - const translated = hostPathForSpecifier(value, fromGuestDir); - return translated ?? value; + const normalized = path.resolve(hostPath); + const hostRoot = path.resolve(HOST_CWD); + if ( + normalized !== hostRoot && + !normalized.startsWith(`${hostRoot}${path.sep}`) + ) { + return null; + } + + const suffix = + normalized === hostRoot + ? '' + : normalized.slice(hostRoot.length + path.sep.length); + return suffix + ? path.posix.join(INITIAL_GUEST_CWD, suffix.split(path.sep).join('/')) + : INITIAL_GUEST_CWD; } -function guestMappedChildNames(guestDir) { - if (typeof guestDir !== 'string') { - return []; +function guestInternalPathFromHostPath(hostPath) { + if (typeof hostPath !== 'string' || !NODE_IMPORT_CACHE_ROOT) { + return null; } - const normalized = path.posix.normalize(guestDir); - const prefix = normalized === '/' ? '/' : `${normalized}/`; - const children = new Set(); - - for (const mapping of GUEST_PATH_MAPPINGS) { - if (!mapping.guestPath.startsWith(prefix)) { - continue; - } - const remainder = mapping.guestPath.slice(prefix.length); - const childName = remainder.split('/')[0]; - if (childName) { - children.add(childName); - } + const normalized = path.resolve(hostPath); + const hostRoot = path.resolve(NODE_IMPORT_CACHE_ROOT); + if ( + normalized !== hostRoot && + !normalized.startsWith(`${hostRoot}${path.sep}`) + ) { + return null; } - return [...children].sort(); + const suffix = + normalized === hostRoot + ? '' + : normalized.slice(hostRoot.length + path.sep.length); + return suffix + ? path.posix.join( + GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT, + suffix.split(path.sep).join('/'), + ) + : GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT; } -function createSyntheticDirent(name) { - return { - name, - isBlockDevice: () => false, - isCharacterDevice: () => false, - isDirectory: () => true, - isFIFO: () => false, - isFile: () => false, - isSocket: () => false, - isSymbolicLink: () => false, - }; +function guestVisiblePathFromHostPath(hostPath) { + return ( + guestPathFromHostPath(hostPath) ?? + guestInternalPathFromHostPath(hostPath) ?? + guestCwdPathFromHostPath(hostPath) ?? + UNMAPPED_GUEST_PATH + ); } -function wrapFsModule(fsModule, fromGuestDir = '/') { - const wrapPathFirst = (methodName) => { - const fn = fsModule[methodName]; - return (...args) => - fn(translateGuestPath(args[0], fromGuestDir), ...args.slice(1)); - }; - const wrapRenameLike = (methodName) => { - const fn = fsModule[methodName]; - return (...args) => - fn( - translateGuestPath(args[0], fromGuestDir), - translateGuestPath(args[1], fromGuestDir), - ...args.slice(2), - ); - }; - const existsSync = fsModule.existsSync.bind(fsModule); - const readdirSync = fsModule.readdirSync.bind(fsModule); - - const wrapped = { - ...fsModule, - accessSync: wrapPathFirst('accessSync'), - appendFileSync: wrapPathFirst('appendFileSync'), - chmodSync: wrapPathFirst('chmodSync'), - chownSync: wrapPathFirst('chownSync'), - createReadStream: wrapPathFirst('createReadStream'), - createWriteStream: wrapPathFirst('createWriteStream'), - existsSync: (target) => { - const translated = translateGuestPath(target, fromGuestDir); - return existsSync(translated) || guestMappedChildNames(target).length > 0; - }, - lstatSync: wrapPathFirst('lstatSync'), - mkdirSync: wrapPathFirst('mkdirSync'), - openSync: wrapPathFirst('openSync'), - readFileSync: wrapPathFirst('readFileSync'), - readdirSync: (target, options) => { - const translated = translateGuestPath(target, fromGuestDir); - if (existsSync(translated)) { - return readdirSync(translated, options); - } - - const synthetic = guestMappedChildNames(target); - if (synthetic.length > 0) { - return options && typeof options === 'object' && options.withFileTypes - ? synthetic.map((name) => createSyntheticDirent(name)) - : synthetic; - } - - return readdirSync(translated, options); - }, - readlinkSync: wrapPathFirst('readlinkSync'), - realpathSync: wrapPathFirst('realpathSync'), - renameSync: wrapRenameLike('renameSync'), - rmSync: wrapPathFirst('rmSync'), - rmdirSync: wrapPathFirst('rmdirSync'), - statSync: wrapPathFirst('statSync'), - symlinkSync: wrapRenameLike('symlinkSync'), - unlinkSync: wrapPathFirst('unlinkSync'), - utimesSync: wrapPathFirst('utimesSync'), - writeFileSync: wrapPathFirst('writeFileSync'), - }; - - if (fsModule.promises) { - wrapped.promises = { - ...fsModule.promises, - access: wrapPathFirstAsync(fsModule.promises.access, fromGuestDir), - appendFile: wrapPathFirstAsync(fsModule.promises.appendFile, fromGuestDir), - chmod: wrapPathFirstAsync(fsModule.promises.chmod, fromGuestDir), - chown: wrapPathFirstAsync(fsModule.promises.chown, fromGuestDir), - lstat: wrapPathFirstAsync(fsModule.promises.lstat, fromGuestDir), - mkdir: wrapPathFirstAsync(fsModule.promises.mkdir, fromGuestDir), - open: wrapPathFirstAsync(fsModule.promises.open, fromGuestDir), - readFile: wrapPathFirstAsync(fsModule.promises.readFile, fromGuestDir), - readdir: wrapPathFirstAsync(fsModule.promises.readdir, fromGuestDir), - readlink: wrapPathFirstAsync(fsModule.promises.readlink, fromGuestDir), - realpath: wrapPathFirstAsync(fsModule.promises.realpath, fromGuestDir), - rename: wrapRenameLikeAsync(fsModule.promises.rename, fromGuestDir), - rm: wrapPathFirstAsync(fsModule.promises.rm, fromGuestDir), - rmdir: wrapPathFirstAsync(fsModule.promises.rmdir, fromGuestDir), - stat: wrapPathFirstAsync(fsModule.promises.stat, fromGuestDir), - symlink: wrapRenameLikeAsync(fsModule.promises.symlink, fromGuestDir), - unlink: wrapPathFirstAsync(fsModule.promises.unlink, fromGuestDir), - utimes: wrapPathFirstAsync(fsModule.promises.utimes, fromGuestDir), - writeFile: wrapPathFirstAsync(fsModule.promises.writeFile, fromGuestDir), - }; +function isGuestVisiblePath(value) { + if (typeof value !== 'string' || !path.posix.isAbsolute(value)) { + return false; } - return wrapped; -} - -function wrapPathFirstAsync(fn, fromGuestDir) { - return (...args) => - fn(translateGuestPath(args[0], fromGuestDir), ...args.slice(1)); + const normalized = path.posix.normalize(value); + return ( + normalized === UNMAPPED_GUEST_PATH || + normalized === GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT || + normalized.startsWith(`${GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT}/`) || + normalized === INITIAL_GUEST_CWD || + normalized.startsWith(`${INITIAL_GUEST_CWD}/`) || + hostPathFromGuestPath(normalized) != null + ); } -function wrapRenameLikeAsync(fn, fromGuestDir) { - return (...args) => - fn( - translateGuestPath(args[0], fromGuestDir), - translateGuestPath(args[1], fromGuestDir), - ...args.slice(2), - ); -} +function translatePathStringToGuest(value) { + if (typeof value !== 'string') { + return value; + } -function wrapChildProcessModule(childProcessModule, fromGuestDir = '/') { - const isNodeCommand = (command) => - command === 'node' || String(command).endsWith('/node'); - const isNodeScriptCommand = (command) => - typeof command === 'string' && - (command.startsWith('./') || - command.startsWith('../') || - command.startsWith('/') || - command.startsWith('file:')) && - /\.(?:[cm]?js)$/i.test(command); - const usesNodeRuntime = (command) => - isNodeCommand(command) || isNodeScriptCommand(command); - const translateCommand = (command) => - usesNodeRuntime(command) - ? process.execPath - : translateGuestPath(command, fromGuestDir); - const isGuestCommandPath = (command) => - typeof command === 'string' && - (command.startsWith('/') || command.startsWith('file:')); - const ensureRuntimeEnv = (env) => { - const sourceEnv = - env && typeof env === 'object' ? env : process.env; - const { NODE_OPTIONS: _nodeOptions, ...safeEnv } = sourceEnv; - for (const key of ['HOME', 'PWD', 'TMPDIR', 'TEMP', 'TMP', 'PI_CODING_AGENT_DIR']) { - if (typeof safeEnv[key] === 'string') { - safeEnv[key] = translateGuestPath(safeEnv[key], fromGuestDir); - } - } - const nodeDir = path.dirname(process.execPath); - const existingPath = - typeof safeEnv.PATH === 'string' - ? safeEnv.PATH - : typeof process.env.PATH === 'string' - ? process.env.PATH - : ''; - const segments = existingPath - .split(path.delimiter) - .filter(Boolean); - - if (!segments.includes(nodeDir)) { - segments.unshift(nodeDir); + if (value.startsWith('file:')) { + try { + const hostPath = new URL(value).pathname; + const guestPath = isGuestVisiblePath(hostPath) + ? path.posix.normalize(hostPath) + : guestVisiblePathFromHostPath(hostPath); + return pathToFileURL(guestPath).href; + } catch { + return value; } + } - return { - ...safeEnv, - PATH: segments.join(path.delimiter), - }; - }; - const translateProcessOptions = (options) => { - if (options == null) { - return { - env: ensureRuntimeEnv(process.env), - }; - } + if (!path.isAbsolute(value)) { + return value; + } - if (typeof options !== 'object') { - return options; - } + return isGuestVisiblePath(value) + ? path.posix.normalize(value) + : guestVisiblePathFromHostPath(value); +} - return { - ...options, - cwd: - typeof options.cwd === 'string' - ? translateGuestPath(options.cwd, fromGuestDir) - : options.cwd, - env: ensureRuntimeEnv(options.env), - }; - }; - const translateArgs = (command, args) => { - if (isNodeScriptCommand(command)) { - const translatedScript = translateGuestPath(command, fromGuestDir); - const translatedArgs = Array.isArray(args) - ? args.map((arg) => translateGuestPath(arg, fromGuestDir)) - : []; - return [translatedScript, ...translatedArgs]; +function buildHostToGuestTextReplacements() { + const replacements = new Map(); + const addReplacement = (hostValue, guestValue) => { + if ( + typeof hostValue !== 'string' || + hostValue.length === 0 || + typeof guestValue !== 'string' || + guestValue.length === 0 + ) { + return; } - if (!Array.isArray(args)) { - return args; - } - if (!isNodeCommand(command)) { - return args.map((arg) => translateGuestPath(arg, fromGuestDir)); - } - return args.map((arg, index) => - index === 0 ? translateGuestPath(arg, fromGuestDir) : arg, - ); + replacements.set(hostValue, guestValue); }; - const prependNodePermissionArgs = (command, args, options) => { - if (!usesNodeRuntime(command)) { - return args; - } - - const translatedArgs = Array.isArray(args) ? args : []; - const readPaths = new Set(); - const writePaths = new Set(); - const addReadPathChain = (value) => { - if (typeof value !== 'string' || value.length === 0) { - return; - } - let current = value; - while (true) { - readPaths.add(current); - const parent = path.dirname(current); - if (parent === current) { - break; - } - current = parent; - } - }; - const addWritePath = (value) => { - if (typeof value !== 'string' || value.length === 0) { - return; - } - writePaths.add(value); - }; - if (typeof options?.cwd === 'string') { - addReadPathChain(options.cwd); - addWritePath(options.cwd); + for (const mapping of GUEST_PATH_MAPPINGS) { + const hostRoot = path.resolve(mapping.hostPath); + addReplacement(hostRoot, mapping.guestPath); + addReplacement(pathToFileURL(hostRoot).href, pathToFileURL(mapping.guestPath).href); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, mapping.guestPath); } + } - const homePath = - typeof options?.env?.HOME === 'string' - ? translateGuestPath(options.env.HOME, fromGuestDir) - : typeof process.env.HOME === 'string' - ? translateGuestPath(process.env.HOME, fromGuestDir) - : null; - if (homePath) { - addReadPathChain(homePath); - addWritePath(homePath); + if (NODE_IMPORT_CACHE_ROOT) { + const hostRoot = path.resolve(NODE_IMPORT_CACHE_ROOT); + addReplacement(hostRoot, GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT); + addReplacement( + pathToFileURL(hostRoot).href, + pathToFileURL(GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT).href, + ); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, GUEST_INTERNAL_NODE_IMPORT_CACHE_ROOT); } + } - if (translatedArgs.length > 0 && typeof translatedArgs[0] === 'string') { - addReadPathChain(translatedArgs[0]); + if (!guestPathFromHostPath(HOST_CWD)) { + const hostRoot = path.resolve(HOST_CWD); + addReplacement(hostRoot, INITIAL_GUEST_CWD); + addReplacement(pathToFileURL(hostRoot).href, pathToFileURL(INITIAL_GUEST_CWD).href); + const forwardSlashHostRoot = hostRoot.split(path.sep).join('/'); + if (forwardSlashHostRoot !== hostRoot) { + addReplacement(forwardSlashHostRoot, INITIAL_GUEST_CWD); } + } - const permissionArgs = [ - '--allow-child-process', - '--allow-worker', - '--disable-warning=SecurityWarning', - ]; - - for (const allowedPath of readPaths) { - permissionArgs.push(`--allow-fs-read=${allowedPath}`); - } - for (const allowedPath of writePaths) { - permissionArgs.push(`--allow-fs-write=${allowedPath}`); - } + return [...replacements.entries()].sort((left, right) => right[0].length - left[0].length); +} - return [...permissionArgs, ...translatedArgs]; - }; +function splitPathLocationSuffix(value) { + if (typeof value !== 'string') { + return { pathLike: value, suffix: '' }; + } - return { - ...childProcessModule, - exec: childProcessModule.exec.bind(childProcessModule), - execFile: (file, args, options, callback) => { - const translatedOptions = translateProcessOptions(options); - return childProcessModule.execFile( - translateCommand(file), - prependNodePermissionArgs( - file, - translateArgs(file, args), - translatedOptions, - ), - translatedOptions, - callback, - ); - }, - execFileSync: (file, args, options) => { - const translatedOptions = translateProcessOptions(options); - return childProcessModule.execFileSync( - translateCommand(file), - prependNodePermissionArgs( - file, - translateArgs(file, args), - translatedOptions, - ), - translatedOptions, - ); - }, - execSync: childProcessModule.execSync.bind(childProcessModule), - fork: (modulePath, args, options) => { - const translatedOptions = translateProcessOptions(options); - return childProcessModule.fork( - translateGuestPath(modulePath, fromGuestDir), - prependNodePermissionArgs( - 'node', - translateArgs('node', args), - translatedOptions, - ), - translatedOptions, - ); - }, - spawn: (command, args, options) => { - const translatedOptions = translateProcessOptions(options); - return childProcessModule.spawn( - translateCommand(command), - prependNodePermissionArgs( - command, - translateArgs(command, args), - translatedOptions, - ), - translatedOptions, - ); - }, - spawnSync: (command, args, options) => - { - const translatedOptions = translateProcessOptions(options); - const result = childProcessModule.spawnSync( - translateCommand(command), - prependNodePermissionArgs( - command, - translateArgs(command, args), - translatedOptions, - ), - translatedOptions, - ); - if ( - isGuestCommandPath(command) && - result?.status == null && - (result.error?.code === 'ENOENT' || result.error?.code === 'EACCES') - ) { - return { - ...result, - status: 1, - stderr: Buffer.from(result.error.message), - }; - } - return result; - }, - }; + const match = /^(.*?)(:\d+(?::\d+)?)$/.exec(value); + return match + ? { pathLike: match[1], suffix: match[2] } + : { pathLike: value, suffix: '' }; } -const guestRequireCache = new Map(); -let rootGuestRequire = null; -const hostFs = fs; -const hostFsPromises = fs.promises; -const hostChildProcess = hostRequire('child_process'); -const guestFs = wrapFsModule(hostFs); -const guestChildProcess = wrapChildProcessModule(hostChildProcess); +function translateTextTokenToGuest(token) { + if (typeof token !== 'string' || token.length === 0) { + return token; + } -function syncBuiltinModuleExports(hostModule, wrappedModule) { + const leading = token.match(/^[("'`[{<]+/)?.[0] ?? ''; + const trailing = token.match(/[)"'`\]}>.,;!?]+$/)?.[0] ?? ''; + const coreEnd = token.length - trailing.length; + const core = token.slice(leading.length, coreEnd); + if (core.length === 0) { + return token; + } + + const { pathLike, suffix } = splitPathLocationSuffix(core); if ( - hostModule == null || - wrappedModule == null || - typeof hostModule !== 'object' || - typeof wrappedModule !== 'object' + typeof pathLike !== 'string' || + (!pathLike.startsWith('file:') && !path.isAbsolute(pathLike)) ) { - return; + return token; } - for (const [key, value] of Object.entries(wrappedModule)) { - try { - hostModule[key] = value; - } catch { - // Ignore immutable bindings and keep the original builtin export. - } - } + return `${leading}${translatePathStringToGuest(pathLike)}${suffix}${trailing}`; } -function cloneFsModule(fsModule) { - if (fsModule == null || typeof fsModule !== 'object') { - return fsModule; +function translateTextToGuest(value) { + if (typeof value !== 'string' || value.length === 0) { + return value; } - const cloned = { ...fsModule }; - if (fsModule.promises && typeof fsModule.promises === 'object') { - cloned.promises = { ...fsModule.promises }; + let translated = value; + for (const [hostValue, guestValue] of buildHostToGuestTextReplacements()) { + translated = translated.split(hostValue).join(guestValue); } - return cloned; + + return translated + .split(/(\s+)/) + .map((token) => (/^\s+$/.test(token) ? token : translateTextTokenToGuest(token))) + .join(''); } -function createGuestRequire(fromGuestDir) { - const normalizedGuestDir = path.posix.normalize(fromGuestDir || '/'); - const cached = guestRequireCache.get(normalizedGuestDir); - if (cached) { - return cached; +function translateErrorToGuest(error) { + if (error == null || typeof error !== 'object') { + return error; } - const hostDir = hostPathFromGuestPath(normalizedGuestDir) ?? process.cwd(); - const baseRequire = Module.createRequire( - pathToFileURL(path.join(hostDir, '__agent_os_require__.cjs')), - ); + if (typeof error.message === 'string') { + try { + error.message = translateTextToGuest(error.message); + } catch { + // Ignore readonly message bindings. + } + } - const guestRequire = function(specifier) { - const translated = hostPathForSpecifier(specifier, normalizedGuestDir); - if (translated) { - return baseRequire(translated); + if (typeof error.stack === 'string') { + try { + error.stack = translateTextToGuest(error.stack); + } catch { + // Ignore readonly stack bindings. } + } + if (typeof error.path === 'string') { try { - return baseRequire(specifier); - } catch (error) { - if (rootGuestRequire && rootGuestRequire !== guestRequire && isBareSpecifier(specifier)) { - return rootGuestRequire(specifier); - } - throw error; + error.path = translatePathStringToGuest(error.path); + } catch { + // Ignore readonly path bindings. } - }; + } - guestRequire.resolve = (specifier) => { - const translated = hostPathForSpecifier(specifier, normalizedGuestDir); - if (translated) { - return baseRequire.resolve(translated); + if (typeof error.filename === 'string') { + try { + error.filename = translatePathStringToGuest(error.filename); + } catch { + // Ignore readonly filename bindings. } + } + if (typeof error.url === 'string') { try { - return baseRequire.resolve(specifier); - } catch (error) { - if (rootGuestRequire && rootGuestRequire !== guestRequire && isBareSpecifier(specifier)) { - return rootGuestRequire.resolve(specifier); - } - throw error; + error.url = translatePathStringToGuest(error.url); + } catch { + // Ignore readonly url bindings. } - }; + } - guestRequireCache.set(normalizedGuestDir, guestRequire); - return guestRequire; + if (Array.isArray(error.requireStack)) { + try { + error.requireStack = error.requireStack.map((entry) => translatePathStringToGuest(entry)); + } catch { + // Ignore readonly requireStack bindings. + } + } + + return error; } -function hardenProperty(target, key, value) { - try { - Object.defineProperty(target, key, { - value, - writable: false, - configurable: false, - }); - return; - } catch { - // Fall back to assignment below. +function hostPathForSpecifier(specifier, fromGuestDir) { + if (typeof specifier !== 'string') { + return null; } - try { - target[key] = value; - } catch { - // Ignore immutable properties; the Node permission model still applies. + if (specifier.startsWith('file:')) { + try { + return hostPathFromGuestPath(new URL(specifier).pathname); + } catch { + return null; + } } -} -function installGuestHardening() { - syncBuiltinModuleExports(hostFs, guestFs); - syncBuiltinModuleExports(hostFsPromises, guestFs.promises); - try { - syncBuiltinESMExports(); - } catch { - // Ignore runtimes that reject syncing builtin ESM exports. + if (specifier.startsWith('/')) { + return hostPathFromGuestPath(specifier); } - hardenProperty(process, 'binding', () => { - throw accessDenied('process.binding'); - }); - hardenProperty(process, '_linkedBinding', () => { - throw accessDenied('process._linkedBinding'); - }); - hardenProperty(process, 'dlopen', () => { - throw accessDenied('process.dlopen'); - }); + if (specifier.startsWith('./') || specifier.startsWith('../')) { + return hostPathFromGuestPath( + path.posix.normalize(path.posix.join(fromGuestDir, specifier)), + ); + } - if (originalModuleLoad) { - Module._load = function(request, parent, isMain) { - const normalized = - typeof request === 'string' ? normalizeBuiltin(request) : null; - if (normalized === 'fs') { - return cloneFsModule(guestFs); - } - if (normalized === 'child_process' && ALLOWED_BUILTINS.has('child_process')) { - return guestChildProcess; - } - if (normalized && DENIED_BUILTINS.has(normalized)) { - throw accessDenied(`node:${normalized}`); - } + return null; +} - return originalModuleLoad(request, parent, isMain); - }; +function translateGuestPath(value, fromGuestDir = '/') { + if (typeof value !== 'string') { + return value; } - if (originalFetch) { - const restrictedFetch = async (resource, init) => { - const candidate = - typeof resource === 'string' - ? resource - : resource instanceof URL - ? resource.href - : resource?.url; - - let url; - try { - url = new URL(String(candidate ?? '')); - } catch { - throw accessDenied('network access'); - } + const translated = hostPathForSpecifier(value, fromGuestDir); + return translated ?? value; +} - if (url.protocol !== 'data:') { - const normalizedPort = - url.port || (url.protocol === 'https:' ? '443' : url.protocol === 'http:' ? '80' : ''); - const loopbackHost = - url.hostname === '127.0.0.1' || - url.hostname === 'localhost' || - url.hostname === '::1' || - url.hostname === '[::1]'; - const loopbackAllowed = - loopbackHost && - (url.protocol === 'http:' || url.protocol === 'https:') && - LOOPBACK_EXEMPT_PORTS.has(normalizedPort); +function resolveGuestFsPath(value, fromGuestDir = '/') { + if (typeof value !== 'string') { + return value; + } - if (!loopbackAllowed) { - throw accessDenied(`network access to ${url.protocol}`); - } - } + if (value.startsWith('file:')) { + try { + return path.posix.normalize(new URL(value).pathname); + } catch { + return value; + } + } - return originalFetch(resource, init); - }; + if (value.startsWith('/')) { + return path.posix.normalize(value); + } - hardenProperty(globalThis, 'fetch', restrictedFetch); + if (value.startsWith('./') || value.startsWith('../')) { + return path.posix.normalize(path.posix.join(fromGuestDir, value)); } -} -const entrypoint = process.env.AGENT_OS_ENTRYPOINT; -if (!entrypoint) { - throw new Error('AGENT_OS_ENTRYPOINT is required'); + return value; } -installGuestHardening(); -rootGuestRequire = createGuestRequire('/root/node_modules'); -if (ALLOWED_BUILTINS.has('child_process')) { - hardenProperty(globalThis, '__agentOsBuiltinChildProcess', guestChildProcess); +function normalizeFsReadOptions(options) { + return typeof options === 'string' ? { encoding: options } : options; } -hardenProperty(globalThis, '__agentOsBuiltinFs', guestFs); -hardenProperty(globalThis, '_requireFrom', (specifier, fromDir = '/') => - createGuestRequire(fromDir)(specifier), -); -hardenProperty( - globalThis, - 'require', - createGuestRequire(path.posix.dirname(guestEntryPoint ?? entrypoint)), -); -if (process.env.AGENT_OS_KEEP_STDIN_OPEN === '1') { - let stdinKeepalive = setInterval(() => {}, 1_000_000); - const releaseStdinKeepalive = () => { - if (stdinKeepalive !== null) { - clearInterval(stdinKeepalive); - stdinKeepalive = null; - } - }; +function normalizeFsWriteContents(contents, options) { + if (typeof contents !== 'string') { + return contents; + } - process.stdin.resume(); - process.stdin.once('end', releaseStdinKeepalive); - process.stdin.once('close', releaseStdinKeepalive); - process.stdin.once('error', releaseStdinKeepalive); -} + const encoding = + typeof options === 'string' + ? options + : options && typeof options === 'object' + ? options.encoding + : undefined; + if (typeof encoding === 'string' && encoding !== 'utf8' && encoding !== 'utf-8') { + return Buffer.from(contents, encoding); + } -const guestArgv = JSON.parse(process.env.AGENT_OS_GUEST_ARGV ?? '[]'); -const bootstrapModule = process.env.AGENT_OS_BOOTSTRAP_MODULE; -const entrypointPath = isPathLike(entrypoint) - ? path.resolve(process.cwd(), entrypoint) - : entrypoint; + return contents; +} -process.argv = [process.execPath, guestEntryPoint ?? entrypointPath, ...guestArgv]; +function normalizeFsTimeValue(value) { + if (value instanceof Date) { + return value.getTime(); + } -if (bootstrapModule) { - await import(toImportSpecifier(bootstrapModule)); + return value; } -await import(toImportSpecifier(entrypoint)); -"#; +function createGuestFsStats(stat) { + if (stat == null || typeof stat !== 'object') { + return stat; + } -const NODE_TIMING_BOOTSTRAP_SOURCE: &str = r#" -const frozenTimeValue = Number(process.env.AGENT_OS_FROZEN_TIME_MS); -const frozenTimeMs = Number.isFinite(frozenTimeValue) ? Math.trunc(frozenTimeValue) : Date.now(); -const frozenDateNow = () => frozenTimeMs; -const OriginalDate = Date; + const flags = { + isDirectory: Boolean(stat.isDirectory), + isSymbolicLink: Boolean(stat.isSymbolicLink), + }; + const target = { ...stat }; + + return new Proxy(target, { + get(source, key, receiver) { + switch (key) { + case 'isBlockDevice': + case 'isCharacterDevice': + case 'isFIFO': + case 'isSocket': + return () => false; + case 'isDirectory': + return () => flags.isDirectory; + case 'isFile': + return () => !flags.isDirectory && !flags.isSymbolicLink; + case 'isSymbolicLink': + return () => flags.isSymbolicLink; + case 'toJSON': + return () => ({ ...source, ...flags }); + default: + return Reflect.get(source, key, receiver); + } + }, + }); +} -function FrozenDate(...args) { - if (new.target) { - if (args.length === 0) { - return new OriginalDate(frozenTimeMs); - } - return new OriginalDate(...args); +function requireAgentOsSyncRpcBridge() { + const bridge = globalThis.__agentOsSyncRpc; + if ( + bridge && + typeof bridge.call === 'function' && + typeof bridge.callSync === 'function' + ) { + return bridge; } - return new OriginalDate(frozenTimeMs).toString(); -} -Object.setPrototypeOf(FrozenDate, OriginalDate); -Object.defineProperty(FrozenDate, 'prototype', { - value: OriginalDate.prototype, - writable: false, - configurable: false, -}); -FrozenDate.parse = OriginalDate.parse; -FrozenDate.UTC = OriginalDate.UTC; -Object.defineProperty(FrozenDate, 'now', { - value: frozenDateNow, - writable: false, - configurable: false, -}); + const error = new Error('Agent OS sync RPC bridge is unavailable'); + error.code = 'ERR_AGENT_OS_NODE_SYNC_RPC_UNAVAILABLE'; + throw error; +} -try { - Object.defineProperty(globalThis, 'Date', { - value: FrozenDate, - writable: false, - configurable: false, - }); -} catch { - globalThis.Date = FrozenDate; +function requireFsSyncRpcBridge() { + return requireAgentOsSyncRpcBridge(); } -const originalPerformance = globalThis.performance; -const frozenPerformance = Object.create(null); -if (typeof originalPerformance !== 'undefined' && originalPerformance !== null) { - const performanceSource = - Object.getPrototypeOf(originalPerformance) ?? originalPerformance; - for (const key of Object.getOwnPropertyNames(performanceSource)) { - if (key === 'now') { - continue; - } - try { - const value = originalPerformance[key]; - frozenPerformance[key] = - typeof value === 'function' ? value.bind(originalPerformance) : value; - } catch { - // Ignore properties that throw during access. - } +function guestProcessUmask(mask) { + const bridge = requireAgentOsSyncRpcBridge(); + if (mask == null) { + return bridge.callSync('process.umask', []); } + return bridge.callSync('process.umask', [normalizeFsMode(mask) ?? 0]); } -Object.defineProperty(frozenPerformance, 'now', { - value: () => 0, - writable: false, - configurable: false, -}); -Object.freeze(frozenPerformance); -try { - Object.defineProperty(globalThis, 'performance', { - value: frozenPerformance, - writable: false, - configurable: false, - }); -} catch { - globalThis.performance = frozenPerformance; -} +function createRpcBackedFsPromises(fromGuestDir = '/') { + const call = (method, args = []) => requireFsSyncRpcBridge().call(method, args); -const frozenHrtimeBigint = BigInt(frozenTimeMs) * 1000000n; -const frozenHrtime = (previous) => { - const seconds = Math.trunc(frozenTimeMs / 1000); - const nanoseconds = Math.trunc((frozenTimeMs % 1000) * 1000000); + return { + access: async (target, mode) => { + await call('fs.promises.access', [ + resolveGuestFsPath(target, fromGuestDir), + mode, + ]); + }, + chmod: async (target, mode) => + call('fs.promises.chmod', [ + resolveGuestFsPath(target, fromGuestDir), + mode, + ]), + chown: async (target, uid, gid) => + call('fs.promises.chown', [ + resolveGuestFsPath(target, fromGuestDir), + uid, + gid, + ]), + copyFile: async (source, destination, mode) => + call('fs.promises.copyFile', [ + resolveGuestFsPath(source, fromGuestDir), + resolveGuestFsPath(destination, fromGuestDir), + mode, + ]), + lstat: async (target) => + createGuestFsStats( + await call('fs.promises.lstat', [resolveGuestFsPath(target, fromGuestDir)]), + ), + mkdir: async (target, options) => + call('fs.promises.mkdir', [ + resolveGuestFsPath(target, fromGuestDir), + options, + ]), + readFile: async (target, options) => + call('fs.promises.readFile', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsReadOptions(options), + ]), + readdir: async (target, options) => + call('fs.promises.readdir', [ + resolveGuestFsPath(target, fromGuestDir), + options, + ]), + rename: async (source, destination) => + call('fs.promises.rename', [ + resolveGuestFsPath(source, fromGuestDir), + resolveGuestFsPath(destination, fromGuestDir), + ]), + rmdir: async (target, options) => + call('fs.promises.rmdir', [ + resolveGuestFsPath(target, fromGuestDir), + options, + ]), + stat: async (target) => + createGuestFsStats( + await call('fs.promises.stat', [resolveGuestFsPath(target, fromGuestDir)]), + ), + unlink: async (target) => + call('fs.promises.unlink', [resolveGuestFsPath(target, fromGuestDir)]), + utimes: async (target, atime, mtime) => + call('fs.promises.utimes', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsTimeValue(atime), + normalizeFsTimeValue(mtime), + ]), + writeFile: async (target, contents, options) => + call('fs.promises.writeFile', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsWriteContents(contents, options), + normalizeFsReadOptions(options), + ]), + }; +} - if (!Array.isArray(previous) || previous.length < 2) { - return [seconds, nanoseconds]; +function resolveGuestSymlinkTarget(value, fromGuestDir = '/') { + if (typeof value !== 'string') { + return value; } - let deltaSeconds = seconds - Number(previous[0]); - let deltaNanoseconds = nanoseconds - Number(previous[1]); - if (deltaNanoseconds < 0) { - deltaSeconds -= 1; - deltaNanoseconds += 1000000000; + if (value.startsWith('file:') || value.startsWith('/')) { + return resolveGuestFsPath(value, fromGuestDir); } - return [deltaSeconds, deltaNanoseconds]; -}; -frozenHrtime.bigint = () => frozenHrtimeBigint; -try { - process.hrtime = frozenHrtime; -} catch { - // Ignore runtimes that expose a non-writable process.hrtime binding. + return value; } -"#; - -const NODE_PREWARM_SOURCE: &str = r#" -import path from 'node:path'; -import { pathToFileURL } from 'node:url'; -function isPathLike(specifier) { - return specifier.startsWith('.') || specifier.startsWith('/') || specifier.startsWith('file:'); -} +const INITIAL_GUEST_CWD = guestPathFromHostPath(HOST_CWD) ?? DEFAULT_GUEST_CWD; -function toImportSpecifier(specifier) { - if (specifier.startsWith('file:')) { - return specifier; - } - if (isPathLike(specifier)) { - return pathToFileURL(path.resolve(process.cwd(), specifier)).href; +function guestMappedChildNames(guestDir) { + if (typeof guestDir !== 'string') { + return []; } - return specifier; -} - -const imports = JSON.parse(process.env.AGENT_OS_NODE_PREWARM_IMPORTS ?? '[]'); -for (const specifier of imports) { - await import(toImportSpecifier(specifier)); -} -"#; -const NODE_WASM_RUNNER_SOURCE: &str = r#" -import fs from 'node:fs/promises'; -import path from 'node:path'; -import { WASI } from 'node:wasi'; + const normalized = path.posix.normalize(guestDir); + const prefix = normalized === '/' ? '/' : `${normalized}/`; + const children = new Set(); -const WASI_ERRNO_SUCCESS = 0; -const WASI_ERRNO_FAULT = 21; + for (const mapping of GUEST_PATH_MAPPINGS) { + if (!mapping.guestPath.startsWith(prefix)) { + continue; + } + const remainder = mapping.guestPath.slice(prefix.length); + const childName = remainder.split('/')[0]; + if (childName) { + children.add(childName); + } + } -function isPathLike(specifier) { - return specifier.startsWith('.') || specifier.startsWith('/') || specifier.startsWith('file:'); + return [...children].sort(); } -function resolveModulePath(specifier) { - if (specifier.startsWith('file:')) { - return new URL(specifier); - } - if (isPathLike(specifier)) { - return path.resolve(process.cwd(), specifier); - } - return specifier; +function createSyntheticDirent(name) { + return { + name, + isBlockDevice: () => false, + isCharacterDevice: () => false, + isDirectory: () => true, + isFIFO: () => false, + isFile: () => false, + isSocket: () => false, + isSymbolicLink: () => false, + }; } -const modulePath = process.env.AGENT_OS_WASM_MODULE_PATH; -if (!modulePath) { - throw new Error('AGENT_OS_WASM_MODULE_PATH is required'); +function createGuestDirent(name, stat) { + return { + name, + isBlockDevice: stat.isBlockDevice, + isCharacterDevice: stat.isCharacterDevice, + isDirectory: stat.isDirectory, + isFIFO: stat.isFIFO, + isFile: stat.isFile, + isSocket: stat.isSocket, + isSymbolicLink: stat.isSymbolicLink, + }; } -const guestArgv = JSON.parse(process.env.AGENT_OS_GUEST_ARGV ?? '[]'); -const guestEnv = JSON.parse(process.env.AGENT_OS_GUEST_ENV ?? '{}'); -const prewarmOnly = process.env.AGENT_OS_WASM_PREWARM_ONLY === '1'; -const frozenTimeValue = Number(process.env.AGENT_OS_FROZEN_TIME_MS); -const frozenTimeMs = Number.isFinite(frozenTimeValue) ? Math.trunc(frozenTimeValue) : Date.now(); -const frozenTimeNs = BigInt(frozenTimeMs) * 1000000n; -const SIGNAL_STATE_CONTROL_PREFIX = '__AGENT_OS_SIGNAL_STATE__:'; - -const moduleBytes = await fs.readFile(resolveModulePath(modulePath)); -const module = await WebAssembly.compile(moduleBytes); +const GUEST_FS_O_RDONLY = 0; +const GUEST_FS_O_WRONLY = 1; +const GUEST_FS_O_RDWR = 2; +const GUEST_FS_O_CREAT = 0o100; +const GUEST_FS_O_EXCL = 0o200; +const GUEST_FS_O_TRUNC = 0o1000; +const GUEST_FS_O_APPEND = 0o2000; +const GUEST_FS_DEFAULT_STREAM_HWM = 64 * 1024; -if (prewarmOnly) { - process.exit(0); +function normalizeFsInteger(value, label) { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'bigint' + ? Number(value) + : Number.NaN; + if (!Number.isFinite(numeric) || !Number.isInteger(numeric) || numeric < 0) { + throw new TypeError(`Agent OS ${label} must be a non-negative integer`); + } + return numeric; } -const wasi = new WASI({ - version: 'preview1', - args: guestArgv, - env: guestEnv, - preopens: { - '/workspace': process.cwd(), - }, - returnOnExit: true, -}); - -let instanceMemory = null; -const wasiImport = { ...wasi.wasiImport }; -const delegateClockTimeGet = - typeof wasi.wasiImport.clock_time_get === 'function' - ? wasi.wasiImport.clock_time_get.bind(wasi.wasiImport) - : null; -const delegateClockResGet = - typeof wasi.wasiImport.clock_res_get === 'function' - ? wasi.wasiImport.clock_res_get.bind(wasi.wasiImport) - : null; +function normalizeFsFd(value) { + return normalizeFsInteger(value, 'fd'); +} -function decodeSignalMask(maskLo, maskHi) { - const values = []; - const lo = Number(maskLo) >>> 0; - const hi = Number(maskHi) >>> 0; - for (let bit = 0; bit < 32; bit += 1) { - if (((lo >>> bit) & 1) === 1) { - values.push(bit + 1); - } +function normalizeFsMode(mode) { + if (mode == null) { + return null; } - for (let bit = 0; bit < 32; bit += 1) { - if (((hi >>> bit) & 1) === 1) { - values.push(bit + 33); + if (typeof mode === 'string') { + const parsed = Number.parseInt(mode, 8); + if (!Number.isNaN(parsed)) { + return parsed; } } - return values; + return normalizeFsInteger(mode, 'mode'); } -const hostProcessImport = { - proc_sigaction(signal, action, maskLo, maskHi, flags) { - try { - const registration = { - action: action === 0 ? 'default' : action === 1 ? 'ignore' : 'user', - mask: decodeSignalMask(maskLo, maskHi), - flags: Number(flags) >>> 0, - }; - process.stderr.write( - `${SIGNAL_STATE_CONTROL_PREFIX}${JSON.stringify({ - signal: Number(signal) >>> 0, - registration, - })}\n`, - ); - return WASI_ERRNO_SUCCESS; - } catch { - return WASI_ERRNO_FAULT; - } - }, -}; - -wasiImport.clock_time_get = (clockId, precision, resultPtr) => { - if (!(instanceMemory instanceof WebAssembly.Memory)) { - return delegateClockTimeGet - ? delegateClockTimeGet(clockId, precision, resultPtr) - : WASI_ERRNO_FAULT; +function normalizeFsPosition(position) { + if (position == null) { + return null; } - - try { - const view = new DataView(instanceMemory.buffer); - view.setBigUint64(Number(resultPtr), frozenTimeNs, true); - return WASI_ERRNO_SUCCESS; - } catch { - return WASI_ERRNO_FAULT; + return normalizeFsInteger(position, 'position'); +} + +function normalizeFsOpenFlags(flags = 'r') { + if (typeof flags === 'number') { + return flags; + } + + switch (flags) { + case 'r': + case 'rs': + case 'sr': + return GUEST_FS_O_RDONLY; + case 'r+': + case 'rs+': + case 'sr+': + return GUEST_FS_O_RDWR; + case 'w': + return GUEST_FS_O_WRONLY | GUEST_FS_O_CREAT | GUEST_FS_O_TRUNC; + case 'wx': + case 'xw': + return GUEST_FS_O_WRONLY | GUEST_FS_O_CREAT | GUEST_FS_O_TRUNC | GUEST_FS_O_EXCL; + case 'w+': + return GUEST_FS_O_RDWR | GUEST_FS_O_CREAT | GUEST_FS_O_TRUNC; + case 'wx+': + case 'xw+': + return GUEST_FS_O_RDWR | GUEST_FS_O_CREAT | GUEST_FS_O_TRUNC | GUEST_FS_O_EXCL; + case 'a': + return GUEST_FS_O_WRONLY | GUEST_FS_O_CREAT | GUEST_FS_O_APPEND; + case 'ax': + case 'xa': + return GUEST_FS_O_WRONLY | GUEST_FS_O_CREAT | GUEST_FS_O_APPEND | GUEST_FS_O_EXCL; + case 'a+': + return GUEST_FS_O_RDWR | GUEST_FS_O_CREAT | GUEST_FS_O_APPEND; + case 'ax+': + case 'xa+': + return GUEST_FS_O_RDWR | GUEST_FS_O_CREAT | GUEST_FS_O_APPEND | GUEST_FS_O_EXCL; + default: + throw new TypeError(`Agent OS does not support fs open flag ${String(flags)}`); } -}; +} -wasiImport.clock_res_get = (clockId, resultPtr) => { - if (!(instanceMemory instanceof WebAssembly.Memory)) { - return delegateClockResGet - ? delegateClockResGet(clockId, resultPtr) - : WASI_ERRNO_FAULT; +function toGuestBufferView(value, label) { + if (Buffer.isBuffer(value)) { + return value; } - - try { - const view = new DataView(instanceMemory.buffer); - view.setBigUint64(Number(resultPtr), 1000000n, true); - return WASI_ERRNO_SUCCESS; - } catch { - return WASI_ERRNO_FAULT; + if (ArrayBuffer.isView(value)) { + return Buffer.from(value.buffer, value.byteOffset, value.byteLength); } -}; - -const instance = await WebAssembly.instantiate(module, { - wasi_snapshot_preview1: wasiImport, - wasi_unstable: wasiImport, - host_process: hostProcessImport, -}); - -if (instance.exports.memory instanceof WebAssembly.Memory) { - instanceMemory = instance.exports.memory; + throw new TypeError(`Agent OS ${label} must be a Buffer, TypedArray, or DataView`); } -if (typeof instance.exports._start === 'function') { - const exitCode = wasi.start(instance); - if (typeof exitCode === 'number' && exitCode !== 0) { - process.exitCode = exitCode; +function decodeFsBytesPayload(value, label) { + if (Buffer.isBuffer(value)) { + return value; } -} else if (typeof instance.exports.run === 'function') { - const result = await instance.exports.run(); - if (typeof result !== 'undefined') { - console.log(String(result)); + if (ArrayBuffer.isView(value)) { + return Buffer.from(value.buffer, value.byteOffset, value.byteLength); + } + if (typeof value === 'string') { + return Buffer.from(value); } -} else { - throw new Error('WebAssembly module must export _start or run'); -} -"#; - -const NODE_PYTHON_RUNNER_SOURCE: &str = r#" -import { closeSync, createReadStream, readSync, writeSync } from 'node:fs'; -import { readFile } from 'node:fs/promises'; -import { register } from 'node:module'; -import { performance as realPerformance } from 'node:perf_hooks'; -import path from 'node:path'; -import readline from 'node:readline'; -import { fileURLToPath, pathToFileURL } from 'node:url'; - -const ACCESS_DENIED_CODE = 'ERR_ACCESS_DENIED'; -const ASSET_ROOT_ENV = 'AGENT_OS_NODE_IMPORT_CACHE_ASSET_ROOT'; -const PYODIDE_INDEX_URL_ENV = 'AGENT_OS_PYODIDE_INDEX_URL'; -const PYTHON_CODE_ENV = 'AGENT_OS_PYTHON_CODE'; -const PYTHON_FILE_ENV = 'AGENT_OS_PYTHON_FILE'; -const PYTHON_PREWARM_ONLY_ENV = 'AGENT_OS_PYTHON_PREWARM_ONLY'; -const PYTHON_WARMUP_DEBUG_ENV = 'AGENT_OS_PYTHON_WARMUP_DEBUG'; -const PYTHON_WARMUP_METRICS_PREFIX = '__AGENT_OS_PYTHON_WARMUP_METRICS__:'; -const PYTHON_EXIT_CONTROL_PREFIX = '__AGENT_OS_PYTHON_EXIT__:'; -const PYTHON_PRELOAD_PACKAGES_ENV = 'AGENT_OS_PYTHON_PRELOAD_PACKAGES'; -const PYTHON_VFS_RPC_REQUEST_FD_ENV = 'AGENT_OS_PYTHON_VFS_RPC_REQUEST_FD'; -const PYTHON_VFS_RPC_RESPONSE_FD_ENV = 'AGENT_OS_PYTHON_VFS_RPC_RESPONSE_FD'; -const STDIN_FD = 0; -const SUPPORTED_PRELOAD_PACKAGES = ['numpy', 'pandas']; -const SUPPORTED_PRELOAD_PACKAGE_SET = new Set(SUPPORTED_PRELOAD_PACKAGES); -const DENIED_BUILTINS = new Set([ - 'child_process', - 'dgram', - 'dns', - 'http', - 'http2', - 'https', - 'inspector', - 'net', - 'tls', - 'v8', - 'vm', - 'worker_threads', -]); -const originalFetch = - typeof globalThis.fetch === 'function' - ? globalThis.fetch.bind(globalThis) - : null; -const originalRequire = - typeof globalThis.require === 'function' - ? globalThis.require.bind(globalThis) - : null; -const originalGetBuiltinModule = - typeof process.getBuiltinModule === 'function' - ? process.getBuiltinModule.bind(process) - : null; -function requiredEnv(name) { - const value = process.env[name]; - if (value == null) { - throw new Error(`${name} is required`); + const base64Value = + value && + typeof value === 'object' && + value.__agentOsType === 'bytes' && + typeof value.base64 === 'string' + ? value.base64 + : null; + if (base64Value == null) { + throw new TypeError(`Agent OS ${label} must be an encoded bytes payload`); } - return value; + return Buffer.from(base64Value, 'base64'); } -function normalizeDirectoryPath(value) { - return value.endsWith(path.sep) ? value : `${value}${path.sep}`; +function normalizeFsReadTarget(buffer, offset, length) { + const target = toGuestBufferView(buffer, 'read buffer'); + const normalizedOffset = offset == null ? 0 : normalizeFsInteger(offset, 'read offset'); + const available = target.byteLength - normalizedOffset; + if (normalizedOffset > target.byteLength) { + throw new RangeError('Agent OS read offset is out of range'); + } + const normalizedLength = + length == null ? available : normalizeFsInteger(length, 'read length'); + if (normalizedLength > available) { + throw new RangeError('Agent OS read length is out of range'); + } + return { target, offset: normalizedOffset, length: normalizedLength }; } -function resolveIndexLocation(value) { - if (/^[A-Za-z][A-Za-z0-9+.-]*:/.test(value)) { - const normalizedUrl = value.endsWith('/') ? value : `${value}/`; - if (!normalizedUrl.startsWith('file:')) { - return { - indexPath: normalizedUrl, - indexUrl: normalizedUrl, - }; - } - - const indexPath = normalizeDirectoryPath(fileURLToPath(normalizedUrl)); +function normalizeFsWriteOperation(value, offsetOrPosition, lengthOrEncoding, position) { + if (typeof value === 'string') { + const normalizedPosition = normalizeFsPosition(offsetOrPosition); + const encoding = + typeof lengthOrEncoding === 'string' ? lengthOrEncoding : 'utf8'; return { - indexPath, - indexUrl: pathToFileURL(indexPath).href, + payload: normalizeFsWriteContents(value, { encoding }), + position: normalizedPosition, + result: value, }; } - const indexPath = normalizeDirectoryPath(path.resolve(value)); + const source = toGuestBufferView(value, 'write buffer'); + const normalizedOffset = + offsetOrPosition == null ? 0 : normalizeFsInteger(offsetOrPosition, 'write offset'); + const available = source.byteLength - normalizedOffset; + if (normalizedOffset > source.byteLength) { + throw new RangeError('Agent OS write offset is out of range'); + } + const normalizedLength = + lengthOrEncoding == null + ? available + : normalizeFsInteger(lengthOrEncoding, 'write length'); + if (normalizedLength > available) { + throw new RangeError('Agent OS write length is out of range'); + } + return { - indexPath, - indexUrl: pathToFileURL(indexPath).href, + payload: source.subarray(normalizedOffset, normalizedOffset + normalizedLength), + position: normalizeFsPosition(position), + result: value, }; } -function writeStream(stream, message) { - if (message == null) { - return; +function normalizeFsBytesResult(value, label) { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'bigint' + ? Number(value) + : Number.NaN; + if (!Number.isFinite(numeric) || numeric < 0) { + throw new TypeError(`Agent OS ${label} must be numeric`); } - - const value = typeof message === 'string' ? message : String(message); - stream.write(value.endsWith('\n') ? value : `${value}\n`); + return Math.trunc(numeric); } -function formatError(error) { - if (error instanceof Error) { - return error.stack || error.message || String(error); +function requireFsCallback(callback, methodName) { + if (typeof callback !== 'function') { + throw new TypeError(`Agent OS ${methodName} requires a callback`); } - - return String(error); + return callback; } -function emitPythonStartupMetrics({ - prewarmOnly, - startupMs, - loadPyodideMs, - packageLoadMs, - packageCount, - source, -}) { - if (process.env[PYTHON_WARMUP_DEBUG_ENV] !== '1') { - return; - } +function invokeFsCallback(callback, error, ...results) { + queueMicrotask(() => callback(error, ...results)); +} - writeStream( - process.stderr, - `${PYTHON_WARMUP_METRICS_PREFIX}${JSON.stringify({ - phase: 'startup', - prewarmOnly, - startupMs, - loadPyodideMs, - packageLoadMs, - packageCount, - source, - })}`, +function createFsWatchUnavailableError(methodName) { + const error = new Error( + `Agent OS ${methodName} is unavailable because the kernel has no file-watching API`, ); + error.code = 'ERR_AGENT_OS_FS_WATCH_UNAVAILABLE'; + return error; } -function parsePreloadPackages(value) { - if (value == null || value.trim() === '') { - return []; - } - - let parsed; - try { - parsed = JSON.parse(value); - } catch (error) { - throw new Error( - `${PYTHON_PRELOAD_PACKAGES_ENV} must be a JSON array of package names: ${formatError(error)}`, - ); - } - - if (!Array.isArray(parsed)) { - throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} must be a JSON array of package names`); - } - - const packages = []; - const seen = new Set(); - - for (const entry of parsed) { - if (typeof entry !== 'string') { - throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} entries must be strings`); - } +function createRpcBackedFsCallbacks(fromGuestDir = '/') { + const call = (method, args = []) => requireFsSyncRpcBridge().call(method, args); - const name = entry.trim(); - if (name.length === 0) { - throw new Error(`${PYTHON_PRELOAD_PACKAGES_ENV} entries must not be empty`); - } + return { + close: (fd, callback) => { + const done = requireFsCallback(callback, 'fs.close'); + call('fs.close', [normalizeFsFd(fd)]).then( + () => invokeFsCallback(done, null), + (error) => invokeFsCallback(done, error), + ); + }, + fstat: (fd, options, callback) => { + const done = requireFsCallback( + typeof options === 'function' ? options : callback, + 'fs.fstat', + ); + call('fs.fstat', [normalizeFsFd(fd)]).then( + (stat) => invokeFsCallback(done, null, createGuestFsStats(stat)), + (error) => invokeFsCallback(done, error), + ); + }, + open: (target, flags, mode, callback) => { + if (typeof flags === 'function') { + callback = flags; + flags = undefined; + mode = undefined; + } else if (typeof mode === 'function') { + callback = mode; + mode = undefined; + } - if (!SUPPORTED_PRELOAD_PACKAGE_SET.has(name)) { - throw new Error( - `Unsupported bundled Python package "${name}". Available packages: ${SUPPORTED_PRELOAD_PACKAGES.join(', ')}`, + const done = requireFsCallback(callback, 'fs.open'); + call('fs.open', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsOpenFlags(flags ?? 'r'), + normalizeFsMode(mode), + ]).then( + (fd) => invokeFsCallback(done, null, normalizeFsFd(fd)), + (error) => invokeFsCallback(done, error), ); - } + }, + read: (fd, buffer, offset, length, position, callback) => { + if (typeof offset === 'function') { + callback = offset; + offset = undefined; + length = undefined; + position = undefined; + } else if (typeof length === 'function') { + callback = length; + length = undefined; + position = undefined; + } else if (typeof position === 'function') { + callback = position; + position = undefined; + } - if (!seen.has(name)) { - seen.add(name); - packages.push(name); - } - } + const done = requireFsCallback(callback, 'fs.read'); + const target = normalizeFsReadTarget(buffer, offset, length); + call('fs.read', [ + normalizeFsFd(fd), + target.length, + normalizeFsPosition(position), + ]).then( + (payload) => { + const chunk = decodeFsBytesPayload(payload, 'fs.read result'); + const bytesRead = Math.min(target.length, chunk.byteLength); + chunk.copy(target.target, target.offset, 0, bytesRead); + invokeFsCallback(done, null, bytesRead, buffer); + }, + (error) => invokeFsCallback(done, error), + ); + }, + write: (fd, value, offsetOrPosition, lengthOrEncoding, position, callback) => { + if (typeof offsetOrPosition === 'function') { + callback = offsetOrPosition; + offsetOrPosition = undefined; + lengthOrEncoding = undefined; + position = undefined; + } else if (typeof lengthOrEncoding === 'function') { + callback = lengthOrEncoding; + lengthOrEncoding = undefined; + position = undefined; + } else if (typeof position === 'function') { + callback = position; + position = undefined; + } - return packages; + const done = requireFsCallback(callback, 'fs.write'); + const write = normalizeFsWriteOperation( + value, + offsetOrPosition, + lengthOrEncoding, + position, + ); + call('fs.write', [normalizeFsFd(fd), write.payload, write.position]).then( + (bytesWritten) => + invokeFsCallback( + done, + null, + normalizeFsBytesResult(bytesWritten, 'fs.write result'), + write.result, + ), + (error) => invokeFsCallback(done, error), + ); + }, + }; } -function parseOptionalFd(name) { - const value = process.env[name]; - if (value == null || value.trim() === '') { - return null; - } - - const fd = Number.parseInt(value, 10); - if (!Number.isInteger(fd) || fd < 0) { - throw new Error(`${name} must be a non-negative integer file descriptor`); - } +function createRpcBackedFsSync(fromGuestDir = '/') { + const callSync = (method, args = []) => requireFsSyncRpcBridge().callSync(method, args); - return fd; -} + return { + accessSync: (target, mode) => + callSync('fs.accessSync', [resolveGuestFsPath(target, fromGuestDir), mode]), + chmodSync: (target, mode) => + callSync('fs.chmodSync', [resolveGuestFsPath(target, fromGuestDir), mode]), + chownSync: (target, uid, gid) => + callSync('fs.chownSync', [resolveGuestFsPath(target, fromGuestDir), uid, gid]), + closeSync: (fd) => callSync('fs.closeSync', [normalizeFsFd(fd)]), + copyFileSync: (source, destination, mode) => + callSync('fs.copyFileSync', [ + resolveGuestFsPath(source, fromGuestDir), + resolveGuestFsPath(destination, fromGuestDir), + mode, + ]), + existsSync: (target) => { + try { + return Boolean(callSync('fs.existsSync', [resolveGuestFsPath(target, fromGuestDir)])); + } catch { + return false; + } + }, + fstatSync: (fd) => + createGuestFsStats(callSync('fs.fstatSync', [normalizeFsFd(fd)])), + linkSync: (existingPath, newPath) => + callSync('fs.linkSync', [ + resolveGuestFsPath(existingPath, fromGuestDir), + resolveGuestFsPath(newPath, fromGuestDir), + ]), + lstatSync: (target) => + createGuestFsStats(callSync('fs.lstatSync', [resolveGuestFsPath(target, fromGuestDir)])), + mkdirSync: (target, options) => + callSync('fs.mkdirSync', [resolveGuestFsPath(target, fromGuestDir), options]), + openSync: (target, flags, mode) => + normalizeFsFd( + callSync('fs.openSync', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsOpenFlags(flags ?? 'r'), + normalizeFsMode(mode), + ]), + ), + readFileSync: (target, options) => + callSync('fs.readFileSync', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsReadOptions(options), + ]), + readSync: (fd, buffer, offset, length, position) => { + const target = normalizeFsReadTarget(buffer, offset, length); + const chunk = decodeFsBytesPayload( + callSync('fs.readSync', [ + normalizeFsFd(fd), + target.length, + normalizeFsPosition(position), + ]), + 'fs.readSync result', + ); + const bytesRead = Math.min(target.length, chunk.byteLength); + chunk.copy(target.target, target.offset, 0, bytesRead); + return bytesRead; + }, + readdirSync: (target, options) => { + const guestPath = resolveGuestFsPath(target, fromGuestDir); + const entries = callSync('fs.readdirSync', [guestPath, options]); + if (!options || typeof options !== 'object' || !options.withFileTypes) { + return entries; + } -function rejectPendingRpcRequests(pending, error) { - for (const { reject } of pending.values()) { - reject(error); - } - pending.clear(); + return entries.map((name) => + createGuestDirent( + name, + createGuestFsStats(callSync('fs.lstatSync', [path.posix.join(guestPath, name)])), + ), + ); + }, + readlinkSync: (target) => + callSync('fs.readlinkSync', [resolveGuestFsPath(target, fromGuestDir)]), + renameSync: (source, destination) => + callSync('fs.renameSync', [ + resolveGuestFsPath(source, fromGuestDir), + resolveGuestFsPath(destination, fromGuestDir), + ]), + rmdirSync: (target, options) => + callSync('fs.rmdirSync', [resolveGuestFsPath(target, fromGuestDir), options]), + statSync: (target) => + createGuestFsStats(callSync('fs.statSync', [resolveGuestFsPath(target, fromGuestDir)])), + symlinkSync: (target, linkPath, type) => + callSync('fs.symlinkSync', [ + resolveGuestSymlinkTarget(target, fromGuestDir), + resolveGuestFsPath(linkPath, fromGuestDir), + type, + ]), + unlinkSync: (target) => + callSync('fs.unlinkSync', [resolveGuestFsPath(target, fromGuestDir)]), + utimesSync: (target, atime, mtime) => + callSync('fs.utimesSync', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsTimeValue(atime), + normalizeFsTimeValue(mtime), + ]), + writeSync: (fd, value, offsetOrPosition, lengthOrEncoding, position) => { + const write = normalizeFsWriteOperation( + value, + offsetOrPosition, + lengthOrEncoding, + position, + ); + return normalizeFsBytesResult( + callSync('fs.writeSync', [normalizeFsFd(fd), write.payload, write.position]), + 'fs.writeSync result', + ); + }, + writeFileSync: (target, contents, options) => + callSync('fs.writeFileSync', [ + resolveGuestFsPath(target, fromGuestDir), + normalizeFsWriteContents(contents, options), + normalizeFsReadOptions(options), + ]), + }; } -function createPythonVfsRpcBridge() { - const requestFd = parseOptionalFd(PYTHON_VFS_RPC_REQUEST_FD_ENV); - const responseFd = parseOptionalFd(PYTHON_VFS_RPC_RESPONSE_FD_ENV); - - if (requestFd == null && responseFd == null) { - return null; - } - - if (requestFd == null || responseFd == null) { - throw new Error( - `both ${PYTHON_VFS_RPC_REQUEST_FD_ENV} and ${PYTHON_VFS_RPC_RESPONSE_FD_ENV} are required`, - ); - } +function createGuestReadStreamClass(fromGuestDir = '/') { + const call = (method, args = []) => requireFsSyncRpcBridge().call(method, args); - let nextRequestId = 1; - let responseBuffer = ''; - const queuedResponses = new Map(); + return class AgentOsReadStream extends Readable { + constructor(target, options = {}) { + super({ + autoDestroy: options.autoClose !== false, + emitClose: options.emitClose !== false, + highWaterMark: options.highWaterMark, + }); - function readResponseLineSync() { - while (true) { - const newlineIndex = responseBuffer.indexOf('\n'); - if (newlineIndex >= 0) { - const line = responseBuffer.slice(0, newlineIndex); - responseBuffer = responseBuffer.slice(newlineIndex + 1); - return line; + this.path = target; + this.fd = typeof options.fd === 'number' ? options.fd : null; + this.flags = options.flags ?? 'r'; + this.mode = options.mode; + this.autoClose = options.autoClose !== false; + this.start = options.start; + this.end = options.end; + this.bytesRead = 0; + this.pending = false; + this.position = + options.start == null ? null : normalizeFsInteger(options.start, 'stream start'); + this.guestDir = fromGuestDir; + + if (options.end != null) { + this.end = normalizeFsInteger(options.end, 'stream end'); + if (this.position != null && this.end < this.position) { + throw new RangeError('Agent OS read stream end must be >= start'); + } } - const chunk = Buffer.alloc(4096); - const bytesRead = readSync(responseFd, chunk, 0, chunk.length, null); - if (bytesRead === 0) { - throw new Error('Agent OS Python VFS RPC response channel closed unexpectedly'); + if (options.encoding) { + this.setEncoding(options.encoding); } - responseBuffer += chunk.subarray(0, bytesRead).toString('utf8'); } - } - function parseResponseLine(line) { - try { - return JSON.parse(line); - } catch (error) { - throw new Error(`invalid Agent OS Python VFS RPC response: ${formatError(error)}`); - } - } + _construct(callback) { + if (typeof this.fd === 'number') { + this.emit('open', this.fd); + this.emit('ready'); + callback(); + return; + } - function waitForResponseSync(id) { - const queued = queuedResponses.get(id); - if (queued) { - queuedResponses.delete(id); - return queued; + call('fs.open', [ + resolveGuestFsPath(this.path, this.guestDir), + normalizeFsOpenFlags(this.flags), + normalizeFsMode(this.mode), + ]).then( + (fd) => { + this.fd = normalizeFsFd(fd); + this.emit('open', this.fd); + this.emit('ready'); + callback(); + }, + (error) => callback(error), + ); } - while (true) { - const line = readResponseLineSync(); - if (line.trim() === '') { - continue; + _read(size) { + if (this.pending || typeof this.fd !== 'number') { + return; } - const message = parseResponseLine(line); - if (message?.id === id) { - return message; + let length = size > 0 ? size : this.readableHighWaterMark ?? GUEST_FS_DEFAULT_STREAM_HWM; + if (this.position != null && this.end != null) { + const remaining = this.end - this.position + 1; + if (remaining <= 0) { + this.push(null); + return; + } + length = Math.min(length, remaining); } - queuedResponses.set(message?.id, message); + + this.pending = true; + call('fs.read', [this.fd, length, this.position]).then( + (payload) => { + this.pending = false; + const chunk = decodeFsBytesPayload(payload, 'fs.createReadStream chunk'); + if (this.position != null) { + this.position += chunk.byteLength; + } + this.bytesRead += chunk.byteLength; + if (chunk.byteLength === 0) { + this.push(null); + return; + } + this.push(chunk); + }, + (error) => { + this.pending = false; + this.destroy(error); + }, + ); } - } - function requestSync(method, payload = {}) { - const id = nextRequestId++; - writeSync( - requestFd, - `${JSON.stringify({ - id, - method, - ...payload, - })}\n`, - ); + _destroy(error, callback) { + if (!this.autoClose || typeof this.fd !== 'number') { + callback(error); + return; + } - const message = waitForResponseSync(id); - if (message?.ok) { - return message.result ?? {}; + const fd = this.fd; + this.fd = null; + call('fs.close', [fd]).then( + () => callback(error), + (closeError) => callback(error ?? closeError), + ); } + }; +} - const error = new Error(message?.error?.message || `Agent OS Python VFS RPC request ${id} failed`); - error.code = message?.error?.code || 'ERR_AGENT_OS_PYTHON_VFS_RPC'; - throw error; - } +function createGuestWriteStreamClass(fromGuestDir = '/') { + const call = (method, args = []) => requireFsSyncRpcBridge().call(method, args); - function request(method, payload = {}) { - return Promise.resolve().then(() => requestSync(method, payload)); - } + return class AgentOsWriteStream extends Writable { + constructor(target, options = {}) { + super({ + autoDestroy: options.autoClose !== false, + defaultEncoding: options.defaultEncoding, + decodeStrings: options.decodeStrings !== false, + emitClose: options.emitClose !== false, + highWaterMark: options.highWaterMark, + }); - function normalizeWriteContent(content) { - if (typeof content === 'string') { - return content; + this.path = target; + this.fd = typeof options.fd === 'number' ? options.fd : null; + this.flags = options.flags ?? 'w'; + this.mode = options.mode; + this.autoClose = options.autoClose !== false; + this.bytesWritten = 0; + this.position = + options.start == null ? null : normalizeFsInteger(options.start, 'stream start'); + this.guestDir = fromGuestDir; } - if (ArrayBuffer.isView(content)) { - return Buffer.from(content.buffer, content.byteOffset, content.byteLength).toString('base64'); + + _construct(callback) { + if (typeof this.fd === 'number') { + this.emit('open', this.fd); + this.emit('ready'); + callback(); + return; + } + + call('fs.open', [ + resolveGuestFsPath(this.path, this.guestDir), + normalizeFsOpenFlags(this.flags), + normalizeFsMode(this.mode), + ]).then( + (fd) => { + this.fd = normalizeFsFd(fd); + this.emit('open', this.fd); + this.emit('ready'); + callback(); + }, + (error) => callback(error), + ); } - if (content instanceof ArrayBuffer) { - return Buffer.from(content).toString('base64'); + + _write(chunk, encoding, callback) { + const write = normalizeFsWriteOperation(chunk, 0, chunk.length, this.position); + call('fs.write', [normalizeFsFd(this.fd), write.payload, write.position]).then( + (bytesWritten) => { + const normalized = normalizeFsBytesResult( + bytesWritten, + 'fs.createWriteStream result', + ); + this.bytesWritten += normalized; + if (this.position != null) { + this.position += normalized; + } + callback(); + }, + (error) => callback(error), + ); } - throw new Error('fsWrite requires a base64 string or Uint8Array'); - } - return { - fsReadSync(path) { - const result = requestSync('fsRead', { path }); - return result.contentBase64 ?? ''; - }, - async fsRead(path) { - return this.fsReadSync(path); - }, - fsWriteSync(path, content) { - requestSync('fsWrite', { - path, - contentBase64: normalizeWriteContent(content), - }); - }, - async fsWrite(path, content) { - this.fsWriteSync(path, content); - }, - fsStatSync(path) { - const result = requestSync('fsStat', { path }); - return result.stat ?? null; - }, - async fsStat(path) { - return this.fsStatSync(path); - }, - fsReaddirSync(path) { - const result = requestSync('fsReaddir', { path }); - return result.entries ?? []; - }, - async fsReaddir(path) { - return this.fsReaddirSync(path); - }, - fsMkdirSync(path, options = {}) { - requestSync('fsMkdir', { - path, - recursive: options?.recursive === true, - }); - }, - async fsMkdir(path, options = {}) { - this.fsMkdirSync(path, options); + _destroy(error, callback) { + if (!this.autoClose || typeof this.fd !== 'number') { + callback(error); + return; + } + + const fd = this.fd; + this.fd = null; + call('fs.close', [fd]).then( + () => callback(error), + (closeError) => callback(error ?? closeError), + ); + } + }; +} + +function wrapFsModule(fsModule, fromGuestDir = '/') { + const wrapPathFirst = (methodName) => { + const fn = fsModule[methodName]; + return (...args) => + fn(translateGuestPath(args[0], fromGuestDir), ...args.slice(1)); + }; + const wrapRenameLike = (methodName) => { + const fn = fsModule[methodName]; + return (...args) => + fn( + translateGuestPath(args[0], fromGuestDir), + translateGuestPath(args[1], fromGuestDir), + ...args.slice(2), + ); + }; + const existsSync = fsModule.existsSync.bind(fsModule); + const readdirSync = fsModule.readdirSync.bind(fsModule); + const ReadStream = createGuestReadStreamClass(fromGuestDir); + const WriteStream = createGuestWriteStreamClass(fromGuestDir); + + const wrapped = { + ...fsModule, + ReadStream, + WriteStream, + accessSync: wrapPathFirst('accessSync'), + appendFileSync: wrapPathFirst('appendFileSync'), + chmodSync: wrapPathFirst('chmodSync'), + chownSync: wrapPathFirst('chownSync'), + createReadStream: (target, options) => new ReadStream(target, options), + createWriteStream: (target, options) => new WriteStream(target, options), + existsSync: (target) => { + const translated = translateGuestPath(target, fromGuestDir); + return existsSync(translated) || guestMappedChildNames(target).length > 0; }, - dispose() { - try { - closeSync(requestFd); - } catch { - // Ignore repeated-close shutdown races. + lstatSync: wrapPathFirst('lstatSync'), + mkdirSync: wrapPathFirst('mkdirSync'), + readFileSync: wrapPathFirst('readFileSync'), + readdirSync: (target, options) => { + const translated = translateGuestPath(target, fromGuestDir); + if (existsSync(translated)) { + return readdirSync(translated, options); } - try { - closeSync(responseFd); - } catch { - // Ignore repeated-close shutdown races. + + const synthetic = guestMappedChildNames(target); + if (synthetic.length > 0) { + return options && typeof options === 'object' && options.withFileTypes + ? synthetic.map((name) => createSyntheticDirent(name)) + : synthetic; } + + return readdirSync(translated, options); + }, + readlinkSync: wrapPathFirst('readlinkSync'), + realpathSync: wrapPathFirst('realpathSync'), + renameSync: wrapRenameLike('renameSync'), + rmSync: wrapPathFirst('rmSync'), + rmdirSync: wrapPathFirst('rmdirSync'), + statSync: wrapPathFirst('statSync'), + symlinkSync: wrapRenameLike('symlinkSync'), + unlinkSync: wrapPathFirst('unlinkSync'), + unwatchFile: () => {}, + utimesSync: wrapPathFirst('utimesSync'), + watch: () => { + throw createFsWatchUnavailableError('fs.watch'); + }, + watchFile: () => { + throw createFsWatchUnavailableError('fs.watchFile'); }, + writeFileSync: wrapPathFirst('writeFileSync'), }; -} -function accessDenied(subject) { - const error = new Error(`${subject} is not available in the Agent OS guest Python runtime`); - error.code = ACCESS_DENIED_CODE; - return error; + if (fsModule.promises) { + wrapped.promises = { + ...fsModule.promises, + access: wrapPathFirstAsync(fsModule.promises.access, fromGuestDir), + appendFile: wrapPathFirstAsync(fsModule.promises.appendFile, fromGuestDir), + chmod: wrapPathFirstAsync(fsModule.promises.chmod, fromGuestDir), + chown: wrapPathFirstAsync(fsModule.promises.chown, fromGuestDir), + lstat: wrapPathFirstAsync(fsModule.promises.lstat, fromGuestDir), + mkdir: wrapPathFirstAsync(fsModule.promises.mkdir, fromGuestDir), + open: wrapPathFirstAsync(fsModule.promises.open, fromGuestDir), + readFile: wrapPathFirstAsync(fsModule.promises.readFile, fromGuestDir), + readdir: wrapPathFirstAsync(fsModule.promises.readdir, fromGuestDir), + readlink: wrapPathFirstAsync(fsModule.promises.readlink, fromGuestDir), + realpath: wrapPathFirstAsync(fsModule.promises.realpath, fromGuestDir), + rename: wrapRenameLikeAsync(fsModule.promises.rename, fromGuestDir), + rm: wrapPathFirstAsync(fsModule.promises.rm, fromGuestDir), + rmdir: wrapPathFirstAsync(fsModule.promises.rmdir, fromGuestDir), + stat: wrapPathFirstAsync(fsModule.promises.stat, fromGuestDir), + symlink: wrapRenameLikeAsync(fsModule.promises.symlink, fromGuestDir), + unlink: wrapPathFirstAsync(fsModule.promises.unlink, fromGuestDir), + utimes: wrapPathFirstAsync(fsModule.promises.utimes, fromGuestDir), + writeFile: wrapPathFirstAsync(fsModule.promises.writeFile, fromGuestDir), + }; + Object.assign(wrapped.promises, createRpcBackedFsPromises(fromGuestDir)); + } + + Object.assign(wrapped, createRpcBackedFsCallbacks(fromGuestDir)); + Object.assign(wrapped, createRpcBackedFsSync(fromGuestDir)); + + return wrapped; } -function hardenProperty(target, key, value) { - try { - Object.defineProperty(target, key, { - value, - writable: false, - configurable: false, - }); - return; - } catch { - // Fall back to assignment below. - } +function wrapPathFirstAsync(fn, fromGuestDir) { + return (...args) => + fn(translateGuestPath(args[0], fromGuestDir), ...args.slice(1)); +} - try { - target[key] = value; - } catch { - // Ignore immutable properties. - } +function wrapRenameLikeAsync(fn, fromGuestDir) { + return (...args) => + fn( + translateGuestPath(args[0], fromGuestDir), + translateGuestPath(args[1], fromGuestDir), + ...args.slice(2), + ); } -function normalizeBuiltin(specifier) { - if (typeof specifier !== 'string') { - return null; +function createRpcBackedChildProcessModule(fromGuestDir = '/') { + const RPC_POLL_WAIT_MS = 50; + const RPC_IDLE_POLL_DELAY_MS = 10; + const INTERNAL_BOOTSTRAP_ENV_KEYS = [ + 'AGENT_OS_ALLOWED_NODE_BUILTINS', + 'AGENT_OS_GUEST_PATH_MAPPINGS', + 'AGENT_OS_LOOPBACK_EXEMPT_PORTS', + 'AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH', + 'AGENT_OS_VIRTUAL_PROCESS_UID', + 'AGENT_OS_VIRTUAL_PROCESS_GID', + 'AGENT_OS_VIRTUAL_PROCESS_VERSION', + ]; + + const bridge = () => requireAgentOsSyncRpcBridge(); + const createUnsupportedChildProcessError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS child_process polyfill`); + error.code = 'ERR_AGENT_OS_CHILD_PROCESS_UNSUPPORTED'; + return error; + }; + const normalizeSpawnInvocation = (args, options) => { + if (!Array.isArray(args)) { + return { + args: [], + options: args && typeof args === 'object' ? args : options, + }; + } + + return { + args, + options, + }; + }; + const normalizeExecInvocation = (options, callback) => + typeof options === 'function' + ? { options: undefined, callback: options } + : { options, callback }; + const normalizeExecFileInvocation = (args, options, callback) => { + if (typeof args === 'function') { + return { args: [], options: undefined, callback: args }; + } + if (!Array.isArray(args)) { + return { + args: [], + options: args, + callback: typeof options === 'function' ? options : callback, + }; + } + if (typeof options === 'function') { + return { args, options: undefined, callback: options }; + } + return { args, options, callback }; + }; + const normalizeChildProcessSignal = (value) => + typeof value === 'string' && value.length > 0 ? value : 'SIGTERM'; + const normalizeChildProcessEncoding = (options) => + typeof options?.encoding === 'string' ? options.encoding : null; + const normalizeChildProcessTimeout = (options) => + Number.isInteger(options?.timeout) && options.timeout > 0 ? options.timeout : null; + const normalizeChildProcessEnv = (env) => { + const source = env && typeof env === 'object' ? env : {}; + const merged = { + ...Object.fromEntries( + Object.entries(process.env).filter( + ([key, value]) => typeof value === 'string' && !isInternalProcessEnvKey(key), + ), + ), + ...Object.fromEntries( + Object.entries(source).filter( + ([key, value]) => value != null && !isInternalProcessEnvKey(key), + ), + ), + }; + delete merged.NODE_OPTIONS; + + return Object.fromEntries( + Object.entries(merged).map(([key, value]) => [key, String(value)]), + ); + }; + const createChildProcessInternalBootstrapEnv = () => { + const bootstrapEnv = {}; + + for (const key of INTERNAL_BOOTSTRAP_ENV_KEYS) { + if (typeof HOST_PROCESS_ENV[key] === 'string') { + bootstrapEnv[key] = HOST_PROCESS_ENV[key]; + } + } + for (const [key, value] of Object.entries(HOST_PROCESS_ENV)) { + if (key.startsWith('AGENT_OS_VIRTUAL_OS_') && typeof value === 'string') { + bootstrapEnv[key] = value; + } + } + + return bootstrapEnv; + }; + const normalizeChildProcessStdioEntry = (value, index) => { + if (value == null) { + return 'pipe'; + } + if (value === 'pipe' || value === 'ignore' || value === 'inherit') { + return value; + } + if (value === 'ipc') { + throw createUnsupportedChildProcessError('child_process IPC stdio'); + } + if (value === null && index === 0) { + return 'pipe'; + } + throw createUnsupportedChildProcessError(`child_process stdio=${String(value)}`); + }; + const normalizeChildProcessStdio = (stdio) => { + if (stdio == null) { + return ['pipe', 'pipe', 'pipe']; + } + if (typeof stdio === 'string') { + return [ + normalizeChildProcessStdioEntry(stdio, 0), + normalizeChildProcessStdioEntry(stdio, 1), + normalizeChildProcessStdioEntry(stdio, 2), + ]; + } + if (!Array.isArray(stdio)) { + throw createUnsupportedChildProcessError('child_process stdio configuration'); + } + return [0, 1, 2].map((index) => + normalizeChildProcessStdioEntry(stdio[index], index), + ); + }; + const normalizeChildProcessOptions = (options, shell = false) => { + if (options != null && typeof options !== 'object') { + throw new TypeError('child_process options must be an object'); + } + if (options?.detached) { + throw createUnsupportedChildProcessError('child_process detached'); + } + + return { + cwd: + typeof options?.cwd === 'string' + ? resolveGuestFsPath(options.cwd, fromGuestDir) + : fromGuestDir, + env: normalizeChildProcessEnv(options?.env), + internalBootstrapEnv: createChildProcessInternalBootstrapEnv(), + shell: shell || options?.shell === true, + stdio: normalizeChildProcessStdio(options?.stdio), + timeout: normalizeChildProcessTimeout(options), + killSignal: normalizeChildProcessSignal(options?.killSignal), + }; + }; + const createRpcSpawnRequest = (command, args, options, shell = false) => ({ + command: String(command), + args: Array.isArray(args) ? args.map((arg) => String(arg)) : [], + options: normalizeChildProcessOptions(options, shell), + }); + const callSpawn = (command, args, options, shell = false) => + bridge().callSync('child_process.spawn', [ + createRpcSpawnRequest(command, args, options, shell), + ]); + const callPoll = (childId, waitMs = 0) => + bridge().callSync('child_process.poll', [childId, waitMs]); + const callKill = (childId, signal) => + bridge().callSync('child_process.kill', [childId, normalizeChildProcessSignal(signal)]); + const callWriteStdin = (childId, chunk) => + bridge().call('child_process.write_stdin', [childId, toGuestBufferView(chunk, 'stdin chunk')]); + const callCloseStdin = (childId) => + bridge().call('child_process.close_stdin', [childId]); + const encodeChildProcessOutput = (buffer, encoding) => + encoding ? buffer.toString(encoding) : buffer; + const createChildProcessExecError = (subject, exitCode, signal, stdout, stderr) => { + const error = new Error( + signal == null + ? `${subject} exited with code ${exitCode ?? 'unknown'}` + : `${subject} terminated by signal ${signal}`, + ); + error.code = signal == null ? 'ERR_AGENT_OS_CHILD_PROCESS_EXIT' : signal; + error.killed = signal != null; + error.signal = signal; + error.stdout = stdout; + error.stderr = stderr; + if (typeof exitCode === 'number') { + error.status = exitCode; + } + return error; + }; + const createSpawnSyncResult = (pid, stdout, stderr, exitCode, signal, error, encoding) => { + const encodedStdout = encodeChildProcessOutput(stdout, encoding); + const encodedStderr = encodeChildProcessOutput(stderr, encoding); + return { + pid, + output: [null, encodedStdout, encodedStderr], + stdout: encodedStdout, + stderr: encodedStderr, + status: typeof exitCode === 'number' ? exitCode : null, + signal: signal ?? null, + error, + }; + }; + const runChildProcessSync = (command, args, options, shell = false) => { + const normalizedOptions = normalizeChildProcessOptions(options, shell); + const encoding = normalizeChildProcessEncoding(options); + const stdout = []; + const stderr = []; + let child; + try { + child = callSpawn(command, args, options, shell); + } catch (error) { + return createSpawnSyncResult( + 0, + Buffer.alloc(0), + Buffer.from(error instanceof Error ? error.message : String(error)), + null, + null, + error, + encoding, + ); + } + + const startedAt = Date.now(); + let exitCode = null; + let signal = null; + while (exitCode == null && signal == null) { + if ( + normalizedOptions.timeout != null && + Date.now() - startedAt > normalizedOptions.timeout + ) { + callKill(child.childId, normalizedOptions.killSignal); + } + + const event = callPoll(child.childId, RPC_POLL_WAIT_MS); + if (!event) { + continue; + } + + if (event.type === 'stdout') { + stdout.push(decodeFsBytesPayload(event.data, 'child_process.spawnSync stdout')); + } else if (event.type === 'stderr') { + stderr.push(decodeFsBytesPayload(event.data, 'child_process.spawnSync stderr')); + } else if (event.type === 'exit') { + exitCode = + typeof event.exitCode === 'number' ? Math.trunc(event.exitCode) : null; + signal = typeof event.signal === 'string' ? event.signal : null; + } + } + + const stdoutBuffer = Buffer.concat(stdout); + const stderrBuffer = Buffer.concat(stderr); + return createSpawnSyncResult( + Number(child.pid) || 0, + stdoutBuffer, + stderrBuffer, + exitCode, + signal, + null, + encoding, + ); + }; + + class AgentOsChildReadable extends Readable { + _read() {} + } + + class AgentOsChildWritable extends Writable { + constructor(childId) { + super(); + this.childId = childId; + } + + _write(chunk, encoding, callback) { + callWriteStdin(this.childId, chunk).then( + () => callback(), + (error) => callback(error), + ); + } + + _final(callback) { + callCloseStdin(this.childId).then( + () => callback(), + (error) => callback(error), + ); + } + } + + const finalizeChildStream = (stream) => { + if (!stream || stream.destroyed) { + return; + } + stream.push(null); + }; + const emitChildLifecycleEvents = (child) => { + queueMicrotask(() => { + child.emit('exit', child.exitCode, child.signalCode); + child.emit('close', child.exitCode, child.signalCode); + }); + }; + const deliverChildOutput = (child, channel, payload) => { + const chunk = decodeFsBytesPayload(payload, `child_process.${channel}`); + const mode = channel === 'stdout' ? child._stdio[1] : child._stdio[2]; + if (mode === 'ignore') { + return; + } + if (mode === 'inherit') { + (channel === 'stdout' ? process.stdout : process.stderr).write(chunk); + return; + } + + const stream = channel === 'stdout' ? child.stdout : child.stderr; + stream?.push(chunk); + }; + const closeSyntheticChild = (child, exitCode, signalCode) => { + if (child._closed) { + return; + } + child._closed = true; + child.exitCode = exitCode; + child.signalCode = signalCode; + finalizeChildStream(child.stdout); + finalizeChildStream(child.stderr); + if (child.stdin && !child.stdin.destroyed) { + child.stdin.destroy(); + } + emitChildLifecycleEvents(child); + }; + const scheduleSyntheticChildPoll = (child, delayMs) => { + if (child._closed || child._pollTimer != null) { + return; + } + child._pollTimer = setTimeout(() => { + child._pollTimer = null; + if (child._closed) { + return; + } + + let event; + try { + event = callPoll(child._childId, RPC_POLL_WAIT_MS); + } catch (error) { + child._closed = true; + finalizeChildStream(child.stdout); + finalizeChildStream(child.stderr); + queueMicrotask(() => child.emit('error', error)); + return; + } + + if (!event) { + scheduleSyntheticChildPoll(child, RPC_IDLE_POLL_DELAY_MS); + return; + } + + if (event.type === 'stdout' || event.type === 'stderr') { + deliverChildOutput(child, event.type, event.data); + scheduleSyntheticChildPoll(child, 0); + return; + } + + if (event.type === 'exit') { + closeSyntheticChild( + child, + typeof event.exitCode === 'number' ? Math.trunc(event.exitCode) : null, + typeof event.signal === 'string' ? event.signal : null, + ); + return; + } + + scheduleSyntheticChildPoll(child, 0); + }, delayMs); + if (!child._refed) { + child._pollTimer.unref?.(); + } + }; + const createSyntheticChildProcess = (spawnResult, options) => { + const child = Object.create(EventEmitter.prototype); + EventEmitter.call(child); + child._childId = spawnResult.childId; + child._closed = false; + child._pollTimer = null; + child._refed = true; + child._stdio = options.stdio; + child.pid = Math.trunc(Number(spawnResult.pid) || 0); + child.exitCode = null; + child.signalCode = null; + child.spawnfile = String(spawnResult.command ?? ''); + child.spawnargs = [ + child.spawnfile, + ...(Array.isArray(spawnResult.args) ? spawnResult.args.map(String) : []), + ]; + child.stdin = options.stdio[0] === 'pipe' ? new AgentOsChildWritable(child._childId) : null; + child.stdout = options.stdio[1] === 'pipe' ? new AgentOsChildReadable() : null; + child.stderr = options.stdio[2] === 'pipe' ? new AgentOsChildReadable() : null; + child.killed = false; + child.connected = false; + child.kill = (signal = 'SIGTERM') => { + try { + callKill(child._childId, signal); + child.killed = true; + return true; + } catch (error) { + if (error && typeof error === 'object' && error.code === 'ESRCH') { + return false; + } + throw error; + } + }; + child.ref = () => { + child._refed = true; + child._pollTimer?.ref?.(); + return child; + }; + child.unref = () => { + child._refed = false; + child._pollTimer?.unref?.(); + return child; + }; + child.disconnect = () => { + throw createUnsupportedChildProcessError('child_process.disconnect'); + }; + child.send = () => { + throw createUnsupportedChildProcessError('child_process.send'); + }; + queueMicrotask(() => child.emit('spawn')); + scheduleSyntheticChildPoll(child, 0); + return child; + }; + const collectSyntheticChildOutput = (child, options, callback) => { + const encoding = normalizeChildProcessEncoding(options) ?? 'utf8'; + const stdoutChunks = []; + const stderrChunks = []; + const timeout = normalizeChildProcessTimeout(options); + const killSignal = normalizeChildProcessSignal(options?.killSignal); + let timer = null; + + if (child.stdout) { + child.stdout.on('data', (chunk) => { + stdoutChunks.push(Buffer.from(chunk)); + }); + } + if (child.stderr) { + child.stderr.on('data', (chunk) => { + stderrChunks.push(Buffer.from(chunk)); + }); + } + + const promise = new Promise((resolve, reject) => { + if (timeout != null) { + timer = setTimeout(() => { + try { + child.kill(killSignal); + } catch {} + }, timeout); + timer.unref?.(); + } + + child.once('error', reject); + child.once('close', (exitCode, signalCode) => { + if (timer) { + clearTimeout(timer); + } + const stdout = encodeChildProcessOutput(Buffer.concat(stdoutChunks), encoding); + const stderr = encodeChildProcessOutput(Buffer.concat(stderrChunks), encoding); + if (exitCode === 0 && signalCode == null) { + resolve({ stdout, stderr, exitCode, signalCode }); + return; + } + reject(createChildProcessExecError('child_process', exitCode, signalCode, stdout, stderr)); + }); + }); + + if (typeof callback === 'function') { + promise.then( + ({ stdout, stderr }) => callback(null, stdout, stderr), + (error) => callback(error, error.stdout, error.stderr), + ); + } + + return promise; + }; + + const module = { + ChildProcess: EventEmitter, + spawn(command, args, options) { + const invocation = normalizeSpawnInvocation(args, options); + const normalizedOptions = normalizeChildProcessOptions(invocation.options); + const child = createSyntheticChildProcess( + callSpawn(command, invocation.args, invocation.options), + normalizedOptions, + ); + return child; + }, + spawnSync(command, args, options) { + const invocation = normalizeSpawnInvocation(args, options); + return runChildProcessSync(command, invocation.args, invocation.options); + }, + exec(command, options, callback) { + const invocation = normalizeExecInvocation(options, callback); + const child = module.spawn(command, [], { + ...invocation.options, + stdio: ['pipe', 'pipe', 'pipe'], + shell: true, + }); + collectSyntheticChildOutput(child, invocation.options, invocation.callback); + return child; + }, + execSync(command, options) { + const result = runChildProcessSync(command, [], { + ...options, + stdio: ['pipe', 'pipe', 'pipe'], + }, true); + if (result.error) { + throw result.error; + } + if (result.status !== 0 || result.signal != null) { + throw createChildProcessExecError( + 'child_process.execSync', + result.status, + result.signal, + result.stdout, + result.stderr, + ); + } + return result.stdout; + }, + execFile(file, args, options, callback) { + const invocation = normalizeExecFileInvocation(args, options, callback); + const child = module.spawn(file, invocation.args, { + ...invocation.options, + stdio: ['pipe', 'pipe', 'pipe'], + }); + collectSyntheticChildOutput(child, invocation.options, invocation.callback); + return child; + }, + execFileSync(file, args, options) { + const invocation = normalizeExecFileInvocation(args, options); + const result = runChildProcessSync(file, invocation.args, { + ...invocation.options, + stdio: ['pipe', 'pipe', 'pipe'], + }); + if (result.error) { + throw result.error; + } + if (result.status !== 0 || result.signal != null) { + throw createChildProcessExecError( + 'child_process.execFileSync', + result.status, + result.signal, + result.stdout, + result.stderr, + ); + } + return result.stdout; + }, + fork(modulePath, args, options) { + const invocation = normalizeSpawnInvocation(args, options); + return module.spawn('node', [modulePath, ...invocation.args], { + ...invocation.options, + stdio: invocation.options?.stdio ?? ['pipe', 'pipe', 'pipe'], + }); + }, + }; + + return module; +} + +function createRpcBackedNetModule(netModule, fromGuestDir = '/') { + const RPC_POLL_WAIT_MS = 50; + const RPC_IDLE_POLL_DELAY_MS = 10; + const bridge = () => requireAgentOsSyncRpcBridge(); + const createUnsupportedNetError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS net polyfill yet`); + error.code = 'ERR_AGENT_OS_NET_UNSUPPORTED'; + return error; + }; + const normalizeNetPort = (value) => { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if (!Number.isInteger(numeric) || numeric < 0 || numeric > 65535) { + throw new RangeError(`Agent OS net port must be an integer between 0 and 65535`); + } + return numeric; + }; + const normalizeNetBacklog = (value) => { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if (!Number.isInteger(numeric) || numeric < 0) { + throw new RangeError(`Agent OS net backlog must be a non-negative integer`); + } + return numeric; + }; + const normalizeNetConnectInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options; + if (values[0] != null && typeof values[0] === 'object') { + options = { ...values[0] }; + } else { + options = { port: values[0] }; + if (typeof values[1] === 'string') { + options.host = values[1]; + } + } + + if (options?.lookup != null) { + throw createUnsupportedNetError('net.connect({ lookup })'); + } + + if (typeof options?.path === 'string' && options.path.length > 0) { + return { + callback, + options: { + allowHalfOpen: options?.allowHalfOpen === true, + path: resolveGuestFsPath(options.path, fromGuestDir), + }, + }; + } + + return { + callback, + options: { + allowHalfOpen: options?.allowHalfOpen === true, + host: + typeof options?.host === 'string' && options.host.length > 0 + ? options.host + : 'localhost', + port: normalizeNetPort(options?.port), + }, + }; + }; + const normalizeNetServerCreation = (args) => { + let options = {}; + let connectionListener; + + if (typeof args[0] === 'function') { + connectionListener = args[0]; + } else { + if (args[0] != null) { + if (typeof args[0] !== 'object') { + throw new TypeError('net.createServer options must be an object'); + } + options = { ...args[0] }; + } + if (typeof args[1] === 'function') { + connectionListener = args[1]; + } + } + + return { + connectionListener, + options: { + allowHalfOpen: options.allowHalfOpen === true, + pauseOnConnect: options.pauseOnConnect === true, + }, + }; + }; + const normalizeNetListenInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let backlog; + if (typeof values[values.length - 1] === 'number') { + backlog = normalizeNetBacklog(values.pop()); + } + + let options; + if (values[0] != null && typeof values[0] === 'object') { + options = { ...values[0] }; + } else { + options = { port: values[0] }; + if (typeof values[1] === 'string') { + options.host = values[1]; + } + } + + if (options?.signal != null) { + throw createUnsupportedNetError('net.Server.listen({ signal })'); + } + + if (typeof options?.path === 'string' && options.path.length > 0) { + return { + callback, + options: { + backlog: + options?.backlog != null + ? normalizeNetBacklog(options.backlog) + : backlog, + path: resolveGuestFsPath(options.path, fromGuestDir), + }, + }; + } + + return { + callback, + options: { + backlog: + options?.backlog != null + ? normalizeNetBacklog(options.backlog) + : backlog, + host: + typeof options?.host === 'string' && options.host.length > 0 + ? options.host + : '127.0.0.1', + port: normalizeNetPort(options?.port ?? 0), + }, + }; + }; + const socketFamilyForAddress = (value) => { + if (typeof value !== 'string') { + return undefined; + } + return value.includes(':') ? 'IPv6' : 'IPv4'; + }; + const callConnect = (options) => bridge().callSync('net.connect', [options]); + const callListen = (options) => bridge().callSync('net.listen', [options]); + const callPoll = (socketId, waitMs = 0) => bridge().callSync('net.poll', [socketId, waitMs]); + const callServerPoll = (serverId, waitMs = 0) => + bridge().callSync('net.server_poll', [serverId, waitMs]); + const callServerConnections = (serverId) => + bridge().callSync('net.server_connections', [serverId]); + const callWrite = (socketId, chunk) => + bridge().call('net.write', [socketId, toGuestBufferView(chunk, 'net.write chunk')]); + const callShutdown = (socketId) => bridge().call('net.shutdown', [socketId]); + const callDestroy = (socketId) => bridge().call('net.destroy', [socketId]); + const callServerClose = (serverId) => bridge().call('net.server_close', [serverId]); + + const finalizeSocketClose = (socket, hadError = false) => { + if (socket._agentOsClosed) { + return; + } + socket._agentOsClosed = true; + socket._agentOsCloseHadError = hadError === true; + socket._agentOsSocketId = null; + socket.connecting = false; + socket.pending = false; + socket._pollTimer && clearTimeout(socket._pollTimer); + socket._pollTimer = null; + if (!socket.readableEnded) { + socket.push(null); + } + queueMicrotask(() => socket.emit('close', hadError)); + }; + + const scheduleSocketPoll = (socket, delayMs) => { + if (socket._agentOsClosed || socket._agentOsSocketId == null || socket._pollTimer != null) { + return; + } + + socket._pollTimer = setTimeout(() => { + socket._pollTimer = null; + if (socket._agentOsClosed || socket._agentOsSocketId == null) { + return; + } + + let event; + try { + event = callPoll(socket._agentOsSocketId, RPC_POLL_WAIT_MS); + } catch (error) { + socket.destroy(error); + return; + } + + if (!event) { + scheduleSocketPoll(socket, RPC_IDLE_POLL_DELAY_MS); + return; + } + + if (event.type === 'data') { + const chunk = decodeFsBytesPayload(event.data, 'net.data'); + socket.bytesRead += chunk.length; + socket.push(chunk); + scheduleSocketPoll(socket, 0); + return; + } + + if (event.type === 'end') { + socket.push(null); + if (!socket._agentOsAllowHalfOpen && !socket.writableEnded) { + socket.end(); + } + scheduleSocketPoll(socket, 0); + return; + } + + if (event.type === 'error') { + const error = new Error( + typeof event.message === 'string' ? event.message : 'Agent OS net socket error', + ); + if (typeof event.code === 'string' && event.code.length > 0) { + error.code = event.code; + } + socket.emit('error', error); + scheduleSocketPoll(socket, 0); + return; + } + + if (event.type === 'close') { + finalizeSocketClose(socket, event.hadError === true); + return; + } + + scheduleSocketPoll(socket, 0); + }, delayMs); + + if (!socket._agentOsRefed) { + socket._pollTimer.unref?.(); + } + }; + const attachSocketState = (socket, result, options = {}, emitConnect = false) => { + socket._agentOsAllowHalfOpen = options.allowHalfOpen === true; + socket._agentOsSocketId = String(result.socketId); + socket.localPath = + typeof result.localPath === 'string' + ? result.localPath + : typeof result.path === 'string' + ? result.path + : undefined; + socket.remotePath = + typeof result.remotePath === 'string' + ? result.remotePath + : typeof result.path === 'string' + ? result.path + : undefined; + socket.localAddress = + socket.localPath ?? result.localAddress; + socket.localPort = result.localPort; + socket.remoteAddress = + socket.remotePath ?? result.remoteAddress; + socket.remotePort = result.remotePort; + socket.remoteFamily = + socket.remotePath != null + ? undefined + : result.remoteFamily ?? socketFamilyForAddress(socket.remoteAddress); + socket.connecting = false; + socket.pending = false; + socket._agentOsClosed = false; + if (emitConnect) { + queueMicrotask(() => { + if (socket._agentOsClosed) { + return; + } + socket.emit('connect'); + socket.emit('ready'); + }); + } + scheduleSocketPoll(socket, 0); + }; + + class AgentOsSocket extends Duplex { + constructor(options = undefined) { + super(options); + this._agentOsAllowHalfOpen = options?.allowHalfOpen === true; + this._agentOsClosed = false; + this._agentOsCloseHadError = false; + this._agentOsExplicitDestroy = false; + this._agentOsRefed = true; + this._agentOsSocketId = null; + this._pollTimer = null; + this.bytesRead = 0; + this.bytesWritten = 0; + this.connecting = false; + this.pending = false; + this.localAddress = undefined; + this.localPort = undefined; + this.localPath = undefined; + this.remoteAddress = undefined; + this.remoteFamily = undefined; + this.remotePort = undefined; + this.remotePath = undefined; + this.emit = (eventName, ...eventArgs) => { + if (eventName === 'close' && eventArgs.length === 0 && this._agentOsClosed) { + eventArgs = [this._agentOsCloseHadError === true]; + } + return Duplex.prototype.emit.call(this, eventName, ...eventArgs); + }; + this.destroy = (error) => { + this._agentOsExplicitDestroy = true; + return Duplex.prototype.destroy.call(this, error); + }; + } + + _read() {} + + _write(chunk, encoding, callback) { + if (this._agentOsSocketId == null) { + callback(new Error('Agent OS net socket is not connected')); + return; + } + const payload = + typeof chunk === 'string' ? Buffer.from(chunk, encoding) : Buffer.from(chunk); + callWrite(this._agentOsSocketId, payload).then( + (written) => { + if (typeof written === 'number') { + this.bytesWritten += written; + } else { + this.bytesWritten += payload.length; + } + callback(); + }, + (error) => callback(error), + ); + } + + _final(callback) { + if (this._agentOsSocketId == null || this._agentOsClosed) { + callback(); + return; + } + callShutdown(this._agentOsSocketId).then( + () => callback(), + (error) => callback(error), + ); + } + + _destroy(error, callback) { + const socketId = this._agentOsSocketId; + this._agentOsSocketId = null; + const finishDestroy = () => { + finalizeSocketClose(this, Boolean(error)); + callback(error); + }; + if ( + socketId == null || + this._agentOsClosed || + (error == null && !this._agentOsExplicitDestroy) + ) { + finishDestroy(); + return; + } + callDestroy(socketId).then(finishDestroy, () => finishDestroy()); + } + + address() { + if (typeof this.localPath === 'string') { + return this.localPath; + } + if (typeof this.localAddress !== 'string' || typeof this.localPort !== 'number') { + return null; + } + return { + address: this.localAddress, + family: socketFamilyForAddress(this.localAddress), + port: this.localPort, + }; + } + + connect(...args) { + const { callback, options } = normalizeNetConnectInvocation(args); + if (typeof callback === 'function') { + this.once('connect', callback); + } + if (this._agentOsSocketId != null || this.connecting) { + throw new Error('Agent OS net socket is already connected'); + } + + this._agentOsAllowHalfOpen = options.allowHalfOpen; + this.connecting = true; + this.pending = true; + + try { + const result = callConnect(options); + attachSocketState( + this, + { + ...result, + remotePath: result.remotePath ?? options.path, + remoteAddress: result.remoteAddress ?? options.host, + remotePort: result.remotePort ?? options.port, + }, + options, + true, + ); + } catch (error) { + this.connecting = false; + this.pending = false; + this.destroy(error); + } + + return this; + } + + ref() { + this._agentOsRefed = true; + this._pollTimer?.ref?.(); + return this; + } + + unref() { + this._agentOsRefed = false; + this._pollTimer?.unref?.(); + return this; + } + + setKeepAlive() { + return this; + } + + setNoDelay() { + return this; + } + + setTimeout(timeout, callback) { + if (typeof callback === 'function') { + if (Number(timeout) > 0) { + setTimeout(() => { + if (!this._agentOsClosed) { + this.emit('timeout'); + callback(); + } + }, Number(timeout)).unref?.(); + } else { + queueMicrotask(() => callback()); + } + } + return this; + } + } + + const finalizeServerClose = (server) => { + if (server._agentOsClosed) { + return; + } + server._agentOsClosed = true; + server.listening = false; + server._agentOsServerId = null; + server._pollTimer && clearTimeout(server._pollTimer); + server._pollTimer = null; + queueMicrotask(() => server.emit('close')); + }; + const scheduleServerPoll = (server, delayMs) => { + if (server._agentOsClosed || server._agentOsServerId == null || server._pollTimer != null) { + return; + } + + server._pollTimer = setTimeout(() => { + server._pollTimer = null; + if (server._agentOsClosed || server._agentOsServerId == null) { + return; + } + + let event; + try { + event = callServerPoll(server._agentOsServerId, RPC_POLL_WAIT_MS); + } catch (error) { + server.emit('error', error); + finalizeServerClose(server); + return; + } + + if (!event) { + scheduleServerPoll(server, RPC_IDLE_POLL_DELAY_MS); + return; + } + + if (event.type === 'connection') { + const socket = new AgentOsSocket({ allowHalfOpen: server.allowHalfOpen }); + attachSocketState(socket, event, { allowHalfOpen: server.allowHalfOpen }); + if (server.pauseOnConnect) { + socket.pause(); + } + server.emit('connection', socket); + scheduleServerPoll(server, 0); + return; + } + + if (event.type === 'error') { + const error = new Error( + typeof event.message === 'string' ? event.message : 'Agent OS net server error', + ); + if (typeof event.code === 'string' && event.code.length > 0) { + error.code = event.code; + } + server.emit('error', error); + scheduleServerPoll(server, 0); + return; + } + + if (event.type === 'close') { + finalizeServerClose(server); + return; + } + + scheduleServerPoll(server, 0); + }, delayMs); + + if (!server._agentOsRefed) { + server._pollTimer.unref?.(); + } + }; + + class AgentOsServer extends EventEmitter { + constructor(options = {}, connectionListener = undefined) { + super(); + this.allowHalfOpen = options.allowHalfOpen === true; + this.pauseOnConnect = options.pauseOnConnect === true; + this.listening = false; + this.maxConnections = undefined; + this._agentOsClosed = false; + this._agentOsRefed = true; + this._agentOsServerId = null; + this._pollTimer = null; + this._address = null; + if (typeof connectionListener === 'function') { + this.on('connection', connectionListener); + } + } + + address() { + return this._address; + } + + close(callback) { + if (this._agentOsServerId == null || this._agentOsClosed) { + const error = new Error('Agent OS net server is not running'); + error.code = 'ERR_SERVER_NOT_RUNNING'; + if (typeof callback === 'function') { + queueMicrotask(() => callback(error)); + return this; + } + throw error; + } + + if (typeof callback === 'function') { + this.once('close', callback); + } + const serverId = this._agentOsServerId; + callServerClose(serverId).then( + () => finalizeServerClose(this), + (error) => this.emit('error', error), + ); + return this; + } + + getConnections(callback) { + if (this._agentOsServerId == null || this._agentOsClosed) { + const error = new Error('Agent OS net server is not running'); + error.code = 'ERR_SERVER_NOT_RUNNING'; + if (typeof callback === 'function') { + queueMicrotask(() => callback(error)); + return this; + } + throw error; + } + + try { + const count = callServerConnections(this._agentOsServerId); + if (typeof callback === 'function') { + queueMicrotask(() => callback(null, count)); + } + } catch (error) { + if (typeof callback === 'function') { + queueMicrotask(() => callback(error)); + return this; + } + throw error; + } + + return this; + } + + listen(...args) { + const { callback, options } = normalizeNetListenInvocation(args); + if (typeof callback === 'function') { + this.once('listening', callback); + } + if (this._agentOsServerId != null || this.listening) { + throw new Error('Agent OS net server is already listening'); + } + + this._agentOsClosed = false; + try { + const result = callListen(options); + this._agentOsServerId = String(result.serverId); + this._address = + typeof result.path === 'string' + ? result.path + : { + address: result.localAddress, + family: result.family ?? socketFamilyForAddress(result.localAddress), + port: result.localPort, + }; + this.listening = true; + queueMicrotask(() => { + if (this._agentOsClosed) { + return; + } + this.emit('listening'); + }); + scheduleServerPoll(this, 0); + } catch (error) { + this._agentOsServerId = null; + this._address = null; + this.listening = false; + throw error; + } + + return this; + } + + ref() { + this._agentOsRefed = true; + this._pollTimer?.ref?.(); + return this; + } + + unref() { + this._agentOsRefed = false; + this._pollTimer?.unref?.(); + return this; + } + } + + const connect = (...args) => new AgentOsSocket().connect(...args); + const createServer = (...args) => { + const { connectionListener, options } = normalizeNetServerCreation(args); + return new AgentOsServer(options, connectionListener); + }; + const module = Object.assign(Object.create(netModule ?? null), { + Server: AgentOsServer, + Socket: AgentOsSocket, + Stream: AgentOsSocket, + connect, + createConnection: connect, + createServer, + }); + + return module; +} + +function createRpcBackedTlsModule(tlsModule, netModule) { + const createUnsupportedTlsError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS tls polyfill yet`); + error.code = 'ERR_AGENT_OS_TLS_UNSUPPORTED'; + return error; + }; + const defineSocketMetadataPassthrough = (tlsSocket, rawSocket) => { + for (const key of ['localAddress', 'localPort', 'remoteAddress', 'remotePort', 'remoteFamily']) { + try { + Object.defineProperty(tlsSocket, key, { + configurable: true, + enumerable: true, + get() { + return rawSocket[key]; + }, + set(value) { + rawSocket[key] = value; + }, + }); + } catch { + // Ignore non-configurable host properties. + } + } + }; + const normalizeTlsPort = (value) => { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if (!Number.isInteger(numeric) || numeric < 0 || numeric > 65535) { + throw new RangeError('Agent OS tls port must be between 0 and 65535'); + } + return numeric; + }; + const normalizeTlsConnectInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options; + if (values[0] != null && typeof values[0] === 'object') { + options = { ...values[0] }; + } else { + const positional = {}; + if (values.length > 0) { + positional.port = values.shift(); + } + if (typeof values[0] === 'string') { + positional.host = values.shift(); + } + const providedOptions = + values[0] != null && typeof values[0] === 'object' ? { ...values[0] } : {}; + options = { ...providedOptions, ...positional }; + } + + if (typeof options?.path === 'string') { + throw createUnsupportedTlsError('tls.connect({ path })'); + } + if (options?.lookup != null) { + throw createUnsupportedTlsError('tls.connect({ lookup })'); + } + + const transportSocket = options?.socket ?? null; + const host = + typeof options?.host === 'string' && options.host.length > 0 + ? options.host + : 'localhost'; + const tlsOptions = { ...options }; + delete tlsOptions.allowHalfOpen; + delete tlsOptions.host; + delete tlsOptions.lookup; + delete tlsOptions.path; + delete tlsOptions.port; + delete tlsOptions.socket; + if ( + typeof tlsOptions.servername !== 'string' && + typeof host === 'string' && + host.length > 0 && + hostNet.isIP(host) === 0 + ) { + tlsOptions.servername = host; + } + + return { + callback, + transportOptions: + transportSocket == null + ? { + allowHalfOpen: options?.allowHalfOpen === true, + host, + port: normalizeTlsPort(options?.port), + } + : null, + transportSocket, + tlsOptions, + }; + }; + const normalizeTlsServerCreation = (args) => { + let options = {}; + let secureConnectionListener; + + if (typeof args[0] === 'function') { + secureConnectionListener = args[0]; + } else { + if (args[0] != null) { + if (typeof args[0] !== 'object') { + throw new TypeError('tls.createServer options must be an object'); + } + options = { ...args[0] }; + } + if (typeof args[1] === 'function') { + secureConnectionListener = args[1]; + } + } + + return { + secureConnectionListener, + options, + }; + }; + const createServerSecureContext = (options) => + options?.secureContext ?? tlsModule.createSecureContext(options ?? {}); + const createClientTlsSocket = (rawSocket, tlsOptions) => { + const tlsSocket = tlsModule.connect({ + ...tlsOptions, + socket: rawSocket, + }); + defineSocketMetadataPassthrough(tlsSocket, rawSocket); + return tlsSocket; + }; + const createServerTlsSocket = (rawSocket, options, secureContext) => { + const tlsSocket = new tlsModule.TLSSocket(rawSocket, { + ...options, + isServer: true, + secureContext, + }); + defineSocketMetadataPassthrough(tlsSocket, rawSocket); + return tlsSocket; + }; + + class AgentOsTlsServer extends EventEmitter { + constructor(options = {}, secureConnectionListener = undefined) { + super(); + this._tlsOptions = { ...options }; + this._secureContext = createServerSecureContext(this._tlsOptions); + this._netServer = netModule.createServer( + { + allowHalfOpen: options.allowHalfOpen === true, + pauseOnConnect: options.pauseOnConnect === true, + }, + (socket) => { + const tlsSocket = createServerTlsSocket(socket, this._tlsOptions, this._secureContext); + tlsSocket.on('secure', () => { + this.emit('secureConnection', tlsSocket); + }); + tlsSocket.on('error', (error) => { + this.emit('tlsClientError', error, tlsSocket); + }); + }, + ); + if (typeof secureConnectionListener === 'function') { + this.on('secureConnection', secureConnectionListener); + } + this._netServer.on('close', () => this.emit('close')); + this._netServer.on('error', (error) => this.emit('error', error)); + this._netServer.on('listening', () => this.emit('listening')); + + Object.defineProperties(this, { + listening: { + enumerable: true, + get: () => this._netServer.listening, + }, + maxConnections: { + enumerable: true, + get: () => this._netServer.maxConnections, + set: (value) => { + this._netServer.maxConnections = value; + }, + }, + }); + } + + address() { + return this._netServer.address(); + } + + close(callback) { + this._netServer.close(callback); + return this; + } + + getConnections(callback) { + return this._netServer.getConnections(callback); + } + + listen(...args) { + this._netServer.listen(...args); + return this; + } + + ref() { + this._netServer.ref(); + return this; + } + + setSecureContext(options) { + if (options == null || typeof options !== 'object') { + throw new TypeError('tls.Server.setSecureContext options must be an object'); + } + this._tlsOptions = { ...options }; + this._secureContext = createServerSecureContext(this._tlsOptions); + return this; + } + + unref() { + this._netServer.unref(); + return this; + } + } + + const connect = (...args) => { + const { callback, transportOptions, transportSocket, tlsOptions } = + normalizeTlsConnectInvocation(args); + const rawSocket = + transportSocket ?? + netModule.connect({ + allowHalfOpen: transportOptions.allowHalfOpen, + host: transportOptions.host, + port: transportOptions.port, + }); + const tlsSocket = createClientTlsSocket(rawSocket, tlsOptions); + if (typeof callback === 'function') { + tlsSocket.once('secureConnect', callback); + } + return tlsSocket; + }; + const createServer = (...args) => { + const { options, secureConnectionListener } = normalizeTlsServerCreation(args); + return new AgentOsTlsServer(options, secureConnectionListener); + }; + const module = Object.assign(Object.create(tlsModule ?? null), { + Server: AgentOsTlsServer, + TLSSocket: tlsModule.TLSSocket, + connect, + createConnection: connect, + createServer, + }); + + return module; +} + +function createTransportBackedServer( + hostServer, + transportServer, + connectionEventName, + forwardedEvents = [], +) { + const forward = (sourceEvent, targetEvent = sourceEvent) => { + transportServer.on(sourceEvent, (...args) => { + hostServer.emit(targetEvent, ...args); + }); + }; + + forward(connectionEventName); + forward('close'); + forward('error'); + forward('listening'); + for (const entry of forwardedEvents) { + if (Array.isArray(entry)) { + forward(entry[0], entry[1] ?? entry[0]); + } else { + forward(entry); + } + } + + const definePassthroughProperty = (property, getter, setter = undefined) => { + try { + Object.defineProperty(hostServer, property, { + configurable: true, + enumerable: true, + get: getter, + set: setter, + }); + } catch { + // Ignore host properties that reject redefinition. + } + }; + + hostServer.address = () => transportServer.address(); + hostServer.close = (callback) => { + transportServer.close(callback); + return hostServer; + }; + hostServer.getConnections = (callback) => transportServer.getConnections(callback); + hostServer.listen = (...args) => { + transportServer.listen(...args); + return hostServer; + }; + hostServer.ref = () => { + transportServer.ref(); + return hostServer; + }; + hostServer.unref = () => { + transportServer.unref(); + return hostServer; + }; + + definePassthroughProperty('listening', () => transportServer.listening); + definePassthroughProperty( + 'maxConnections', + () => transportServer.maxConnections, + (value) => { + transportServer.maxConnections = value; + }, + ); + + return hostServer; +} + +function normalizeHttpPort(value, subject = 'Agent OS http port') { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if (!Number.isInteger(numeric) || numeric < 0 || numeric > 65535) { + throw new RangeError(`${subject} must be an integer between 0 and 65535`); + } + return numeric; +} + +function defaultPortForProtocol(protocol) { + switch (protocol) { + case 'https:': + return 443; + case 'http2:': + case 'http:': + default: + return 80; + } +} + +function parseRequestTargetFromHostOption(value, protocol) { + if (typeof value !== 'string' || value.length === 0) { + return null; + } + if (hostNet.isIP(value) !== 0) { + return { + hostname: value, + port: null, + }; + } + + const looksLikeHostPort = + value.startsWith('[') || /^[^:]+:\d+$/.test(value); + if (!looksLikeHostPort) { + return { + hostname: value, + port: null, + }; + } + + try { + const parsed = new URL(`${protocol}//${value}`); + return { + hostname: parsed.hostname || 'localhost', + port: + parsed.port.length > 0 ? normalizeHttpPort(parsed.port) : null, + }; + } catch { + return { + hostname: value, + port: null, + }; + } +} + +function parseRequestTargetFromUrl(value, defaultProtocol) { + if (!(value instanceof URL) && typeof value !== 'string') { + return null; + } + + const parsed = value instanceof URL ? value : new URL(String(value)); + const protocol = + typeof parsed.protocol === 'string' && parsed.protocol.length > 0 + ? parsed.protocol + : defaultProtocol; + const auth = + parsed.username.length > 0 || parsed.password.length > 0 + ? `${decodeURIComponent(parsed.username)}:${decodeURIComponent(parsed.password)}` + : undefined; + return { + protocol, + hostname: parsed.hostname || 'localhost', + port: + parsed.port.length > 0 + ? normalizeHttpPort(parsed.port) + : defaultPortForProtocol(protocol), + path: `${parsed.pathname || '/'}${parsed.search || ''}`, + auth, + }; +} + +function createRpcBackedHttpModule(httpModule, transportModule, defaultProtocol = 'http:') { + const createUnsupportedHttpError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS http polyfill yet`); + error.code = 'ERR_AGENT_OS_HTTP_UNSUPPORTED'; + return error; + }; + const normalizeRequestInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options = {}; + if (values[0] instanceof URL || typeof values[0] === 'string') { + options = { + ...options, + ...parseRequestTargetFromUrl(values.shift(), defaultProtocol), + }; + } + if (values[0] != null) { + if (typeof values[0] !== 'object') { + throw new TypeError('Agent OS http request options must be an object'); + } + options = { + ...options, + ...values[0], + }; + } + + if (typeof options.socketPath === 'string') { + throw createUnsupportedHttpError('http request socketPath'); + } + if (options.lookup != null) { + throw createUnsupportedHttpError('http request lookup'); + } + + const protocol = + typeof options.protocol === 'string' && options.protocol.length > 0 + ? options.protocol + : defaultProtocol; + const hostTarget = parseRequestTargetFromHostOption(options.host, protocol); + const hostname = + typeof options.hostname === 'string' && options.hostname.length > 0 + ? options.hostname + : hostTarget?.hostname ?? 'localhost'; + const port = + options.port != null + ? normalizeHttpPort(options.port) + : hostTarget?.port ?? defaultPortForProtocol(protocol); + const path = + typeof options.path === 'string' && options.path.length > 0 + ? options.path + : '/'; + const requestOptions = { + ...options, + protocol, + hostname, + port, + path, + agent: false, + }; + delete requestOptions.agent; + delete requestOptions.createConnection; + delete requestOptions.host; + delete requestOptions.lookup; + delete requestOptions.socketPath; + + return { + callback, + requestOptions, + connectionOptions: { + allowHalfOpen: options.allowHalfOpen === true, + family: options.family, + host: hostname, + localAddress: options.localAddress, + port, + }, + }; + }; + const createRequest = (options, callback) => { + const request = httpModule.request( + { + ...options.requestOptions, + agent: false, + createConnection: () => transportModule.connect(options.connectionOptions), + }, + callback, + ); + return request; + }; + const normalizeServerCreation = (args) => { + let options = {}; + let requestListener; + + if (typeof args[0] === 'function') { + requestListener = args[0]; + } else { + if (args[0] != null) { + if (typeof args[0] !== 'object') { + throw new TypeError('http.createServer options must be an object'); + } + options = { ...args[0] }; + } + if (typeof args[1] === 'function') { + requestListener = args[1]; + } + } + + return { + options, + requestListener, + transportOptions: { + allowHalfOpen: options.allowHalfOpen === true, + pauseOnConnect: options.pauseOnConnect === true, + }, + }; + }; + + const request = (...args) => { + const normalized = normalizeRequestInvocation(args); + return createRequest(normalized, normalized.callback); + }; + const get = (...args) => { + const req = request(...args); + req.end(); + return req; + }; + const createServer = (...args) => { + const { options, requestListener, transportOptions } = + normalizeServerCreation(args); + const server = httpModule.createServer(options, requestListener); + const transportServer = transportModule.createServer(transportOptions); + return createTransportBackedServer(server, transportServer, 'connection'); + }; + const module = Object.assign(Object.create(httpModule ?? null), { + Agent: httpModule.Agent, + globalAgent: httpModule.globalAgent, + get, + request, + createServer, + }); + + return module; +} + +function createRpcBackedHttpsModule(httpsModule, tlsModule) { + const createUnsupportedHttpsError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS https polyfill yet`); + error.code = 'ERR_AGENT_OS_HTTPS_UNSUPPORTED'; + return error; + }; + const normalizeRequestInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options = {}; + if (values[0] instanceof URL || typeof values[0] === 'string') { + options = { + ...options, + ...parseRequestTargetFromUrl(values.shift(), 'https:'), + }; + } + if (values[0] != null) { + if (typeof values[0] !== 'object') { + throw new TypeError('Agent OS https request options must be an object'); + } + options = { + ...options, + ...values[0], + }; + } + + if (typeof options.socketPath === 'string') { + throw createUnsupportedHttpsError('https request socketPath'); + } + if (options.lookup != null) { + throw createUnsupportedHttpsError('https request lookup'); + } + + const hostTarget = parseRequestTargetFromHostOption(options.host, 'https:'); + const hostname = + typeof options.hostname === 'string' && options.hostname.length > 0 + ? options.hostname + : hostTarget?.hostname ?? 'localhost'; + const port = + options.port != null + ? normalizeHttpPort(options.port) + : hostTarget?.port ?? 443; + const path = + typeof options.path === 'string' && options.path.length > 0 + ? options.path + : '/'; + const requestOptions = { + ...options, + protocol: 'https:', + hostname, + port, + path, + agent: false, + }; + delete requestOptions.agent; + delete requestOptions.createConnection; + delete requestOptions.host; + delete requestOptions.lookup; + delete requestOptions.socketPath; + + const tlsConnectOptions = { + allowHalfOpen: options.allowHalfOpen === true, + ALPNProtocols: options.ALPNProtocols, + ca: options.ca, + cert: options.cert, + ciphers: options.ciphers, + crl: options.crl, + ecdhCurve: options.ecdhCurve, + family: options.family, + host: hostname, + key: options.key, + localAddress: options.localAddress, + maxVersion: options.maxVersion, + minVersion: options.minVersion, + passphrase: options.passphrase, + pfx: options.pfx, + port, + rejectUnauthorized: options.rejectUnauthorized, + secureContext: options.secureContext, + servername: options.servername, + session: options.session, + sigalgs: options.sigalgs, + }; + + return { + callback, + requestOptions, + tlsConnectOptions, + }; + }; + const normalizeServerCreation = (args) => { + let options = {}; + let requestListener; + + if (typeof args[0] === 'function') { + requestListener = args[0]; + } else { + if (args[0] != null) { + if (typeof args[0] !== 'object') { + throw new TypeError('https.createServer options must be an object'); + } + options = { ...args[0] }; + } + if (typeof args[1] === 'function') { + requestListener = args[1]; + } + } + + return { + options, + requestListener, + }; + }; + + const request = (...args) => { + const normalized = normalizeRequestInvocation(args); + return httpsModule.request( + { + ...normalized.requestOptions, + agent: false, + createConnection: () => tlsModule.connect(normalized.tlsConnectOptions), + }, + normalized.callback, + ); + }; + const get = (...args) => { + const req = request(...args); + req.end(); + return req; + }; + const createServer = (...args) => { + const { options, requestListener } = normalizeServerCreation(args); + const server = httpsModule.createServer(options, requestListener); + const transportServer = tlsModule.createServer(options); + return createTransportBackedServer(server, transportServer, 'secureConnection', [ + 'tlsClientError', + ]); + }; + const module = Object.assign(Object.create(httpsModule ?? null), { + Agent: httpsModule.Agent, + globalAgent: httpsModule.globalAgent, + get, + request, + createServer, + }); + + return module; +} + +function createRpcBackedHttp2Module(http2Module, netModule, tlsModule) { + const createUnsupportedHttp2Error = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS http2 polyfill yet`); + error.code = 'ERR_AGENT_OS_HTTP2_UNSUPPORTED'; + return error; + }; + const normalizeConnectInvocation = (args) => { + const values = [...args]; + const authority = + values[0] instanceof URL || typeof values[0] === 'string' + ? values.shift() + : 'http://localhost'; + const authorityTarget = parseRequestTargetFromUrl(authority, 'http:'); + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + const options = + values[0] != null && typeof values[0] === 'object' ? { ...values[0] } : {}; + + if (typeof options.socketPath === 'string') { + throw createUnsupportedHttp2Error('http2.connect socketPath'); + } + if (options.lookup != null) { + throw createUnsupportedHttp2Error('http2.connect lookup'); + } + + const connectOptions = { ...options }; + delete connectOptions.createConnection; + delete connectOptions.host; + delete connectOptions.hostname; + delete connectOptions.lookup; + delete connectOptions.port; + delete connectOptions.socketPath; + + const isSecure = authorityTarget.protocol === 'https:'; + return { + authority, + callback, + connectOptions, + createConnection: () => + isSecure + ? tlsModule.connect({ + ALPNProtocols: options.ALPNProtocols ?? ['h2'], + ca: options.ca, + cert: options.cert, + ciphers: options.ciphers, + family: options.family, + host: authorityTarget.hostname, + key: options.key, + localAddress: options.localAddress, + passphrase: options.passphrase, + pfx: options.pfx, + port: authorityTarget.port, + rejectUnauthorized: options.rejectUnauthorized, + secureContext: options.secureContext, + servername: options.servername, + session: options.session, + }) + : netModule.connect({ + allowHalfOpen: options.allowHalfOpen === true, + family: options.family, + host: authorityTarget.hostname, + localAddress: options.localAddress, + port: authorityTarget.port, + }), + }; + }; + const normalizeServerCreation = (args, secure) => { + let options = {}; + let onStream; + + if (typeof args[0] === 'function') { + onStream = args[0]; + } else { + if (args[0] != null) { + if (typeof args[0] !== 'object') { + throw new TypeError( + `http2.${secure ? 'createSecureServer' : 'createServer'} options must be an object`, + ); + } + options = { ...args[0] }; + } + if (typeof args[1] === 'function') { + onStream = args[1]; + } + } + + return { + onStream, + options, + }; + }; + + const connect = (...args) => { + const normalized = normalizeConnectInvocation(args); + return http2Module.connect( + normalized.authority, + { + ...normalized.connectOptions, + createConnection: normalized.createConnection, + }, + normalized.callback, + ); + }; + const createServer = (...args) => { + const { onStream, options } = normalizeServerCreation(args, false); + const server = http2Module.createServer(options, onStream); + const transportServer = netModule.createServer({ + allowHalfOpen: options.allowHalfOpen === true, + pauseOnConnect: options.pauseOnConnect === true, + }); + return createTransportBackedServer(server, transportServer, 'connection'); + }; + const createSecureServer = (...args) => { + const { onStream, options } = normalizeServerCreation(args, true); + const server = http2Module.createSecureServer(options, onStream); + const transportServer = tlsModule.createServer( + { + ...options, + ALPNProtocols: options.ALPNProtocols ?? ['h2'], + }, + ); + return createTransportBackedServer(server, transportServer, 'secureConnection', [ + 'tlsClientError', + ]); + }; + const module = Object.assign(Object.create(http2Module ?? null), { + connect, + createServer, + createSecureServer, + }); + + return module; +} + +function createRpcBackedDgramModule(dgramModule, fromGuestDir = '/') { + const RPC_POLL_WAIT_MS = 50; + const RPC_IDLE_POLL_DELAY_MS = 10; + const bridge = () => requireAgentOsSyncRpcBridge(); + const createUnsupportedDgramError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS dgram polyfill yet`); + error.code = 'ERR_AGENT_OS_DGRAM_UNSUPPORTED'; + return error; + }; + const normalizeDgramInteger = (value, label) => { + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if (!Number.isInteger(numeric) || numeric < 0) { + throw new RangeError(`Agent OS ${label} must be a non-negative integer`); + } + return numeric; + }; + const normalizeDgramPort = (value) => { + const numeric = normalizeDgramInteger(value, 'dgram port'); + if (numeric > 65535) { + throw new RangeError(`Agent OS dgram port must be between 0 and 65535`); + } + return numeric; + }; + const socketFamilyForAddress = (value) => { + if (typeof value !== 'string') { + return undefined; + } + return value.includes(':') ? 'IPv6' : 'IPv4'; + }; + const normalizeDgramType = (value) => { + if (value === 'udp4' || value === 'udp6') { + return value; + } + throw new TypeError(`Agent OS dgram socket type must be udp4 or udp6`); + }; + const normalizeDgramCreateSocketInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options; + if (typeof values[0] === 'string') { + options = { type: values[0] }; + } else if (values[0] != null && typeof values[0] === 'object') { + options = { ...values[0] }; + } else { + throw new TypeError('dgram.createSocket requires a socket type or options object'); + } + + if (options?.recvBufferSize != null || options?.sendBufferSize != null) { + throw createUnsupportedDgramError('dgram.createSocket({ recvBufferSize/sendBufferSize })'); + } + + return { + callback, + options: { + type: normalizeDgramType(options.type), + }, + }; + }; + const normalizeDgramBindInvocation = (args, socketType) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + + let options; + if (values[0] != null && typeof values[0] === 'object') { + options = { ...values[0] }; + } else { + options = { port: values[0] }; + if (typeof values[1] === 'string') { + options.address = values[1]; + } + } + + if (options?.exclusive != null || options?.fd != null || options?.signal != null) { + throw createUnsupportedDgramError('dgram.Socket.bind advanced options'); + } + + return { + callback, + options: { + port: normalizeDgramPort(options?.port ?? 0), + address: + typeof options?.address === 'string' && options.address.length > 0 + ? options.address + : socketType === 'udp6' + ? '::1' + : '127.0.0.1', + }, + }; + }; + const normalizeDgramMessageBuffer = (value) => { + if (typeof value === 'string') { + return Buffer.from(value); + } + if (Array.isArray(value)) { + return Buffer.concat(value.map((entry) => normalizeDgramMessageBuffer(entry))); + } + return Buffer.from(toGuestBufferView(value, 'dgram payload')); + }; + const normalizeDgramSendInvocation = (args) => { + const values = [...args]; + const callback = + typeof values[values.length - 1] === 'function' ? values.pop() : undefined; + if (values.length === 0) { + throw new TypeError('dgram.Socket.send requires a payload'); + } + + let payload = normalizeDgramMessageBuffer(values.shift()); + let port; + let address; + + if ( + values.length >= 3 && + typeof values[0] === 'number' && + typeof values[1] === 'number' + ) { + const offset = normalizeDgramInteger(values.shift(), 'dgram send offset'); + const length = normalizeDgramInteger(values.shift(), 'dgram send length'); + if (offset > payload.length || offset + length > payload.length) { + throw new RangeError('Agent OS dgram send offset/length is out of range'); + } + payload = payload.subarray(offset, offset + length); + port = normalizeDgramPort(values.shift()); + if (typeof values[0] === 'string') { + address = values.shift(); + } + } else if (values[0] != null && typeof values[0] === 'object') { + const options = { ...values.shift() }; + port = normalizeDgramPort(options.port); + address = options.address; + } else { + port = normalizeDgramPort(values.shift()); + if (typeof values[0] === 'string') { + address = values.shift(); + } + } + + return { + callback, + options: { + port, + address: typeof address === 'string' && address.length > 0 ? address : 'localhost', + }, + payload, + }; + }; + const callCreateSocket = (options) => bridge().callSync('dgram.createSocket', [options]); + const callBind = (socketId, options) => bridge().callSync('dgram.bind', [socketId, options]); + const callSend = (socketId, payload, options) => + bridge().call('dgram.send', [socketId, toGuestBufferView(payload, 'dgram.send payload'), options]); + const callPoll = (socketId, waitMs = 0) => bridge().callSync('dgram.poll', [socketId, waitMs]); + const callClose = (socketId) => bridge().call('dgram.close', [socketId]); + + const finalizeDatagramClose = (socket) => { + if (socket._agentOsClosed) { + return; + } + socket._agentOsClosed = true; + socket._agentOsBound = false; + socket._agentOsPollTimer && clearTimeout(socket._agentOsPollTimer); + socket._agentOsPollTimer = null; + queueMicrotask(() => socket.emit('close')); + }; + const attachDatagramBindState = (socket, result, emitListening = false) => { + const alreadyBound = socket._agentOsBound; + socket._agentOsBound = true; + socket._address = { + address: result.localAddress, + family: result.family ?? socketFamilyForAddress(result.localAddress), + port: result.localPort, + }; + if (emitListening && !alreadyBound) { + queueMicrotask(() => { + if (!socket._agentOsClosed) { + socket.emit('listening'); + } + }); + } + scheduleDatagramPoll(socket, 0); + }; + const scheduleDatagramPoll = (socket, delayMs) => { + if ( + socket._agentOsClosed || + socket._agentOsSocketId == null || + !socket._agentOsBound || + socket._agentOsPollTimer != null + ) { + return; + } + + socket._agentOsPollTimer = setTimeout(() => { + socket._agentOsPollTimer = null; + if ( + socket._agentOsClosed || + socket._agentOsSocketId == null || + !socket._agentOsBound + ) { + return; + } + + let event; + try { + event = callPoll(socket._agentOsSocketId, RPC_POLL_WAIT_MS); + } catch (error) { + socket.emit('error', error); + scheduleDatagramPoll(socket, 0); + return; + } + + if (!event) { + scheduleDatagramPoll(socket, RPC_IDLE_POLL_DELAY_MS); + return; + } + + if (event.type === 'message') { + socket.emit( + 'message', + decodeFsBytesPayload(event.data, 'dgram.message'), + { + address: event.remoteAddress, + family: event.remoteFamily ?? socketFamilyForAddress(event.remoteAddress), + port: event.remotePort, + size: decodeFsBytesPayload(event.data, 'dgram.message').length, + }, + ); + scheduleDatagramPoll(socket, 0); + return; + } + + if (event.type === 'error') { + const error = new Error( + typeof event.message === 'string' ? event.message : 'Agent OS dgram socket error', + ); + if (typeof event.code === 'string' && event.code.length > 0) { + error.code = event.code; + } + socket.emit('error', error); + scheduleDatagramPoll(socket, 0); + return; + } + + scheduleDatagramPoll(socket, 0); + }, delayMs); + + if (!socket._agentOsRefed) { + socket._agentOsPollTimer.unref?.(); + } + }; + + class AgentOsDatagramSocket extends EventEmitter { + constructor(options = {}, messageListener = undefined) { + super(); + this.type = options.type; + this._agentOsClosed = false; + this._agentOsRefed = true; + this._agentOsBound = false; + this._agentOsSocketId = null; + this._agentOsPollTimer = null; + this._address = null; + if (typeof messageListener === 'function') { + this.on('message', messageListener); + } + const result = callCreateSocket(options); + this._agentOsSocketId = String(result.socketId); + } + + address() { + return this._address; + } + + bind(...args) { + const { callback, options } = normalizeDgramBindInvocation(args, this.type); + if (typeof callback === 'function') { + this.once('listening', callback); + } + if (this._agentOsClosed) { + throw new Error('Agent OS dgram socket is closed'); + } + attachDatagramBindState(this, callBind(this._agentOsSocketId, options), true); + return this; + } + + close(callback) { + if (typeof callback === 'function') { + this.once('close', callback); + } + if (this._agentOsClosed || this._agentOsSocketId == null) { + queueMicrotask(() => finalizeDatagramClose(this)); + return this; + } + this._agentOsBound = false; + this._agentOsPollTimer && clearTimeout(this._agentOsPollTimer); + this._agentOsPollTimer = null; + const socketId = this._agentOsSocketId; + this._agentOsSocketId = null; + callClose(socketId).then( + () => finalizeDatagramClose(this), + (error) => this.emit('error', error), + ); + return this; + } + + send(...args) { + if (this._agentOsClosed || this._agentOsSocketId == null) { + const error = new Error('Agent OS dgram socket is closed'); + const callback = + typeof args[args.length - 1] === 'function' ? args[args.length - 1] : null; + if (callback) { + queueMicrotask(() => callback(error)); + return; + } + throw error; + } + + const { callback, options, payload } = normalizeDgramSendInvocation(args); + callSend(this._agentOsSocketId, payload, options).then( + (result) => { + attachDatagramBindState(this, result, true); + if (typeof callback === 'function') { + callback(null, typeof result?.bytes === 'number' ? result.bytes : payload.length); + } + }, + (error) => { + if (typeof callback === 'function') { + callback(error); + return; + } + this.emit('error', error); + }, + ); + } + + ref() { + this._agentOsRefed = true; + this._agentOsPollTimer?.ref?.(); + return this; + } + + unref() { + this._agentOsRefed = false; + this._agentOsPollTimer?.unref?.(); + return this; + } + + setBroadcast() { + return this; + } + + setMulticastInterface() { + return this; + } + + setMulticastLoopback() { + return this; + } + + setMulticastTTL() { + return this; + } + + setRecvBufferSize() { + return this; + } + + setSendBufferSize() { + return this; + } + + setTTL() { + return this; + } + + addMembership() { + throw createUnsupportedDgramError('dgram.Socket.addMembership'); + } + + connect() { + throw createUnsupportedDgramError('dgram.Socket.connect'); + } + + disconnect() { + throw createUnsupportedDgramError('dgram.Socket.disconnect'); + } + + dropMembership() { + throw createUnsupportedDgramError('dgram.Socket.dropMembership'); + } + + getRecvBufferSize() { + return 0; + } + + getSendBufferSize() { + return 0; + } + + remoteAddress() { + throw createUnsupportedDgramError('dgram.Socket.remoteAddress'); + } + } + + const createSocket = (...args) => { + const { callback, options } = normalizeDgramCreateSocketInvocation(args); + return new AgentOsDatagramSocket(options, callback); + }; + const module = Object.assign(Object.create(dgramModule ?? null), { + Socket: AgentOsDatagramSocket, + createSocket, + }); + + return module; +} + +function createRpcBackedDnsModule(dnsModule) { + const bridge = () => requireAgentOsSyncRpcBridge(); + const dnsConstants = Object.freeze({ ...(dnsModule?.constants ?? {}) }); + let defaultResultOrder = 'verbatim'; + + const createUnsupportedDnsError = (subject) => { + const error = new Error(`${subject} is not supported by the Agent OS dns polyfill yet`); + error.code = 'ERR_AGENT_OS_DNS_UNSUPPORTED'; + return error; + }; + + const normalizeDnsHostname = (hostname, methodName) => { + if (typeof hostname !== 'string' || hostname.length === 0) { + throw new TypeError(`Agent OS ${methodName} hostname must be a non-empty string`); + } + return hostname; + }; + + const normalizeDnsFamily = (value, label, allowAny = true) => { + if (value == null) { + return allowAny ? 0 : 4; + } + const numeric = + typeof value === 'number' + ? value + : typeof value === 'string' && value.length > 0 + ? Number(value) + : Number.NaN; + if ( + !Number.isInteger(numeric) || + (!allowAny && numeric !== 4 && numeric !== 6) || + (allowAny && numeric !== 0 && numeric !== 4 && numeric !== 6) + ) { + throw new TypeError( + `Agent OS ${label} must be ${allowAny ? '0, 4, or 6' : '4 or 6'}`, + ); + } + return numeric; + }; + + const normalizeDnsResultOrder = (value) => { + const normalized = value == null ? defaultResultOrder : String(value); + if ( + normalized !== 'verbatim' && + normalized !== 'ipv4first' && + normalized !== 'ipv6first' + ) { + throw new TypeError( + 'Agent OS dns result order must be one of verbatim, ipv4first, or ipv6first', + ); + } + return normalized; + }; + + const sortLookupAddresses = (records, order) => { + if (!Array.isArray(records) || order === 'verbatim') { + return [...records]; + } + const rankFamily = (family) => { + if (order === 'ipv4first') { + return family === 4 ? 0 : family === 6 ? 1 : 2; + } + return family === 6 ? 0 : family === 4 ? 1 : 2; + }; + return [...records].sort((left, right) => rankFamily(left.family) - rankFamily(right.family)); + }; + + const normalizeLookupInvocation = (hostname, options, callback) => { + let normalizedOptions = {}; + let done = callback; + + if (typeof options === 'function') { + done = options; + } else if (typeof options === 'number') { + normalizedOptions = { family: options }; + } else if (options == null) { + normalizedOptions = {}; + } else if (typeof options === 'object') { + normalizedOptions = { ...options }; + } else { + throw new TypeError('Agent OS dns.lookup options must be a number, object, or callback'); + } + + return { + callback: done, + options: { + hostname: normalizeDnsHostname(hostname, 'dns.lookup'), + family: normalizeDnsFamily(normalizedOptions.family, 'dns.lookup family'), + all: normalizedOptions.all === true, + order: normalizeDnsResultOrder( + normalizedOptions.order ?? + (normalizedOptions.verbatim === false ? 'ipv4first' : undefined), + ), + }, + }; + }; + + const normalizeResolveInvocation = (methodName, hostname, rrtype, callback) => { + let type = rrtype; + let done = callback; + if (typeof rrtype === 'function') { + done = rrtype; + type = undefined; + } + if (type == null) { + type = 'A'; + } + const normalizedType = String(type).toUpperCase(); + if (normalizedType !== 'A' && normalizedType !== 'AAAA') { + throw createUnsupportedDnsError(`${methodName}(${normalizedType})`); + } + return { + callback: done, + options: { + hostname: normalizeDnsHostname(hostname, methodName), + rrtype: normalizedType, + }, + }; + }; + + const resolveRecords = (method, options) => bridge().callSync(method, [options]); + const lookupRecords = (options) => bridge().callSync('dns.lookup', [options]); + + const lookup = (hostname, options, callback) => { + const invocation = normalizeLookupInvocation(hostname, options, callback); + const records = sortLookupAddresses(lookupRecords(invocation.options), invocation.options.order); + if (typeof invocation.callback === 'function') { + queueMicrotask(() => { + if (invocation.options.all) { + invocation.callback(null, records); + } else { + const first = records[0] ?? { address: null, family: invocation.options.family || 0 }; + invocation.callback(null, first.address, first.family); + } + }); + } + return invocation.options.all + ? records + : { + address: records[0]?.address ?? null, + family: records[0]?.family ?? (invocation.options.family || 0), + }; + }; + + const resolve = (hostname, rrtype, callback) => { + const invocation = normalizeResolveInvocation('dns.resolve', hostname, rrtype, callback); + const records = resolveRecords('dns.resolve', invocation.options); + if (typeof invocation.callback === 'function') { + queueMicrotask(() => invocation.callback(null, records)); + } + return records; + }; + + const resolve4 = (hostname, callback) => { + const invocation = normalizeResolveInvocation('dns.resolve4', hostname, 'A', callback); + const records = resolveRecords('dns.resolve4', invocation.options); + if (typeof invocation.callback === 'function') { + queueMicrotask(() => invocation.callback(null, records)); + } + return records; + }; + + const resolve6 = (hostname, callback) => { + const invocation = normalizeResolveInvocation('dns.resolve6', hostname, 'AAAA', callback); + const records = resolveRecords('dns.resolve6', invocation.options); + if (typeof invocation.callback === 'function') { + queueMicrotask(() => invocation.callback(null, records)); + } + return records; + }; + + class AgentOsResolver { + cancel() {} + + getServers() { + return []; + } + + lookup(hostname, options, callback) { + return lookup(hostname, options, callback); + } + + resolve(hostname, rrtype, callback) { + return resolve(hostname, rrtype, callback); + } + + resolve4(hostname, callback) { + return resolve4(hostname, callback); + } + + resolve6(hostname, callback) { + return resolve6(hostname, callback); + } + + setServers() { + throw createUnsupportedDnsError('dns.Resolver.setServers'); + } + } + + class AgentOsPromisesResolver { + cancel() {} + + getServers() { + return []; + } + + lookup(hostname, options) { + return Promise.resolve(lookup(hostname, options)); + } + + resolve(hostname, rrtype) { + return Promise.resolve(resolve(hostname, rrtype)); + } + + resolve4(hostname) { + return Promise.resolve(resolve4(hostname)); + } + + resolve6(hostname) { + return Promise.resolve(resolve6(hostname)); + } + + setServers() { + throw createUnsupportedDnsError('dns.promises.Resolver.setServers'); + } + } + + const promises = Object.freeze({ + Resolver: AgentOsPromisesResolver, + lookup(hostname, options) { + return Promise.resolve(lookup(hostname, options)); + }, + resolve(hostname, rrtype) { + return Promise.resolve(resolve(hostname, rrtype)); + }, + resolve4(hostname) { + return Promise.resolve(resolve4(hostname)); + }, + resolve6(hostname) { + return Promise.resolve(resolve6(hostname)); + }, + }); + + const module = { + ADDRCONFIG: dnsConstants.ADDRCONFIG, + ALL: dnsConstants.ALL, + V4MAPPED: dnsConstants.V4MAPPED, + Resolver: AgentOsResolver, + constants: dnsConstants, + getDefaultResultOrder() { + return defaultResultOrder; + }, + getServers() { + return []; + }, + lookup, + lookupService() { + throw createUnsupportedDnsError('dns.lookupService'); + }, + promises, + resolve, + resolve4, + resolve6, + reverse() { + throw createUnsupportedDnsError('dns.reverse'); + }, + setDefaultResultOrder(order) { + defaultResultOrder = normalizeDnsResultOrder(order); + }, + setServers() { + throw createUnsupportedDnsError('dns.setServers'); + }, + }; + + return module; +} + +const guestRequireCache = new Map(); +let rootGuestRequire = null; +const hostFs = fs; +const hostFsPromises = fs.promises; +const hostFsWriteSync = fs.writeSync.bind(fs); +const hostFsCloseSync = fs.closeSync.bind(fs); +const guestFs = wrapFsModule(hostFs); +globalThis.__agentOsGuestFs = guestFs; +const guestChildProcess = createRpcBackedChildProcessModule(INITIAL_GUEST_CWD); +const guestNet = createRpcBackedNetModule(hostNet, INITIAL_GUEST_CWD); +const guestDgram = createRpcBackedDgramModule(hostDgram, INITIAL_GUEST_CWD); +const guestDns = createRpcBackedDnsModule(hostDns); +const guestTls = createRpcBackedTlsModule(hostTls, guestNet); +const guestHttp = createRpcBackedHttpModule(hostHttp, guestNet); +const guestHttps = createRpcBackedHttpsModule(hostHttps, guestTls); +const guestHttp2 = createRpcBackedHttp2Module(hostHttp2, guestNet, guestTls); +const guestGetUid = () => VIRTUAL_UID; +const guestGetGid = () => VIRTUAL_GID; +const guestMonotonicNow = + globalThis.performance && typeof globalThis.performance.now === 'function' + ? globalThis.performance.now.bind(globalThis.performance) + : Date.now; +const VIRTUAL_OS_HOSTNAME = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_HOSTNAME, + DEFAULT_VIRTUAL_OS_HOSTNAME, +); +const VIRTUAL_OS_TYPE = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_TYPE, + DEFAULT_VIRTUAL_OS_TYPE, +); +const VIRTUAL_OS_PLATFORM = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_PLATFORM, + DEFAULT_VIRTUAL_OS_PLATFORM, +); +const VIRTUAL_OS_RELEASE = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_RELEASE, + DEFAULT_VIRTUAL_OS_RELEASE, +); +const VIRTUAL_OS_VERSION = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_VERSION, + DEFAULT_VIRTUAL_OS_VERSION, +); +const VIRTUAL_OS_ARCH = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_ARCH, + DEFAULT_VIRTUAL_OS_ARCH, +); +const VIRTUAL_OS_MACHINE = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_MACHINE, + DEFAULT_VIRTUAL_OS_MACHINE, +); +const VIRTUAL_OS_CPU_MODEL = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_CPU_MODEL, + DEFAULT_VIRTUAL_OS_CPU_MODEL, +); +const VIRTUAL_OS_CPU_COUNT = parsePositiveInt( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_CPU_COUNT, + DEFAULT_VIRTUAL_OS_CPU_COUNT, +); +const VIRTUAL_OS_TOTALMEM = parsePositiveInt( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_TOTALMEM, + DEFAULT_VIRTUAL_OS_TOTALMEM, +); +const VIRTUAL_OS_FREEMEM = Math.min( + parsePositiveInt( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_FREEMEM, + DEFAULT_VIRTUAL_OS_FREEMEM, + ), + VIRTUAL_OS_TOTALMEM, +); +const DEFAULT_VIRTUAL_PROCESS_VERSION = 'v24.0.0'; +const VIRTUAL_PROCESS_VERSION = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_PROCESS_VERSION, + DEFAULT_VIRTUAL_PROCESS_VERSION, +); +const VIRTUAL_PROCESS_RELEASE = deepFreezeObject({ + name: 'node', + lts: 'Agent OS', +}); +const VIRTUAL_PROCESS_CONFIG = deepFreezeObject({ + target_defaults: {}, + variables: { + host_arch: VIRTUAL_OS_ARCH, + node_shared: false, + node_use_openssl: false, + }, +}); +const VIRTUAL_PROCESS_VERSIONS = deepFreezeObject({ + node: VIRTUAL_PROCESS_VERSION.replace(/^v/, ''), + modules: '0', + napi: '0', + uv: '0.0.0', + zlib: '0.0.0', + openssl: '0.0.0', + v8: '0.0', +}); +const VIRTUAL_PROCESS_START_TIME_MS = guestMonotonicNow(); +let guestProcess = process; + +function syncBuiltinModuleExports(hostModule, wrappedModule) { + if ( + hostModule == null || + wrappedModule == null || + typeof hostModule !== 'object' || + typeof wrappedModule !== 'object' + ) { + return; + } + + for (const [key, value] of Object.entries(wrappedModule)) { + try { + hostModule[key] = value; + } catch { + // Ignore immutable bindings and keep the original builtin export. + } + } +} + +function cloneFsModule(fsModule) { + if (fsModule == null || typeof fsModule !== 'object') { + return fsModule; + } + + const cloned = { ...fsModule }; + if (fsModule.promises && typeof fsModule.promises === 'object') { + cloned.promises = { ...fsModule.promises }; + } + return cloned; +} + +function resolveVirtualPath(value, fallback) { + if (typeof value !== 'string' || value.length === 0) { + return fallback; + } + + if (path.posix.isAbsolute(value)) { + return path.posix.normalize(value); + } + + return translatePathStringToGuest(value); +} + +function cloneVirtualCpuInfo(cpu) { + return { + ...cpu, + times: { ...cpu.times }, + }; +} + +function cloneVirtualNetworkInterfaces(networkInterfaces) { + return Object.fromEntries( + Object.entries(networkInterfaces).map(([name, entries]) => [ + name, + entries.map((entry) => ({ ...entry })), + ]), + ); +} + +function encodeUserInfoValue(value, encoding) { + return encoding === 'buffer' ? Buffer.from(String(value)) : String(value); +} + +function deepFreezeObject(value) { + if ( + value == null || + (typeof value !== 'object' && typeof value !== 'function') || + Object.isFrozen(value) + ) { + return value; + } + + for (const nestedValue of Object.values(value)) { + deepFreezeObject(nestedValue); + } + + return Object.freeze(value); +} + +function createVirtualProcessMemoryUsageSnapshot() { + const rss = Math.max( + 1, + Math.min( + VIRTUAL_OS_TOTALMEM, + Math.max(VIRTUAL_OS_TOTALMEM - VIRTUAL_OS_FREEMEM, Math.floor(VIRTUAL_OS_TOTALMEM / 4)), + ), + ); + const heapTotal = Math.max(1, Math.min(rss, Math.floor(rss / 2))); + const heapUsed = Math.max(1, Math.min(heapTotal, Math.floor(heapTotal / 2))); + const external = Math.max(0, Math.min(rss - heapUsed, Math.floor(rss / 8))); + const arrayBuffers = Math.max(0, Math.min(external, Math.floor(external / 2))); + + return { + rss, + heapTotal, + heapUsed, + external, + arrayBuffers, + }; +} + +function createGuestMemoryUsage() { + const memoryUsage = () => createVirtualProcessMemoryUsageSnapshot(); + hardenProperty(memoryUsage, 'rss', () => createVirtualProcessMemoryUsageSnapshot().rss); + return memoryUsage; +} + +function createGuestProcessUptime() { + return () => Math.max(0, (guestMonotonicNow() - VIRTUAL_PROCESS_START_TIME_MS) / 1000); +} + +function createGuestOsModule(osModule) { + const virtualHomeDir = resolveVirtualPath( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_HOMEDIR, + DEFAULT_VIRTUAL_OS_HOMEDIR, + ); + const virtualTmpDir = resolveVirtualPath( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_TMPDIR, + DEFAULT_VIRTUAL_OS_TMPDIR, + ); + const virtualUserName = parseVirtualProcessString( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_USER, + DEFAULT_VIRTUAL_OS_USER, + ); + const virtualShell = resolveVirtualPath( + HOST_PROCESS_ENV.AGENT_OS_VIRTUAL_OS_SHELL, + DEFAULT_VIRTUAL_OS_SHELL, + ); + const virtualCpuInfo = Object.freeze( + Array.from({ length: VIRTUAL_OS_CPU_COUNT }, () => + Object.freeze({ + model: VIRTUAL_OS_CPU_MODEL, + speed: 0, + times: Object.freeze({ + user: 0, + nice: 0, + sys: 0, + idle: 0, + irq: 0, + }), + }), + ), + ); + const virtualNetworkInterfaces = Object.freeze({ + lo: Object.freeze([ + Object.freeze({ + address: '127.0.0.1', + netmask: '255.0.0.0', + family: 'IPv4', + mac: '00:00:00:00:00:00', + internal: true, + cidr: '127.0.0.1/8', + }), + Object.freeze({ + address: '::1', + netmask: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff', + family: 'IPv6', + mac: '00:00:00:00:00:00', + internal: true, + cidr: '::1/128', + scopeid: 0, + }), + ]), + }); + + return Object.assign(Object.create(osModule ?? null), { + arch: () => VIRTUAL_OS_ARCH, + availableParallelism: () => VIRTUAL_OS_CPU_COUNT, + cpus: () => virtualCpuInfo.map((cpu) => cloneVirtualCpuInfo(cpu)), + freemem: () => VIRTUAL_OS_FREEMEM, + getPriority: () => 0, + homedir: () => virtualHomeDir, + hostname: () => VIRTUAL_OS_HOSTNAME, + loadavg: () => [0, 0, 0], + machine: () => VIRTUAL_OS_MACHINE, + networkInterfaces: () => cloneVirtualNetworkInterfaces(virtualNetworkInterfaces), + platform: () => VIRTUAL_OS_PLATFORM, + release: () => VIRTUAL_OS_RELEASE, + setPriority: () => { + throw accessDenied('os.setPriority'); + }, + tmpdir: () => virtualTmpDir, + totalmem: () => VIRTUAL_OS_TOTALMEM, + type: () => VIRTUAL_OS_TYPE, + uptime: () => 0, + userInfo: (options = undefined) => { + const encoding = + options && typeof options === 'object' ? options.encoding : undefined; + return { + username: encodeUserInfoValue(virtualUserName, encoding), + uid: VIRTUAL_UID, + gid: VIRTUAL_GID, + shell: encodeUserInfoValue(virtualShell, encoding), + homedir: encodeUserInfoValue(virtualHomeDir, encoding), + }; + }, + version: () => VIRTUAL_OS_VERSION, + }); +} + +const guestOs = createGuestOsModule(hostOs); +const guestMemoryUsage = createGuestMemoryUsage(); +const guestProcessUptime = createGuestProcessUptime(); + +function isProcessSignalEventName(eventName) { + return typeof eventName === 'string' && SIGNAL_EVENTS.has(eventName); +} + +function emitControlMessage(message) { + if (CONTROL_PIPE_FD == null) { + return; + } + + try { + hostFsWriteSync(CONTROL_PIPE_FD, `${JSON.stringify(message)}\n`); + } catch { + // Ignore control-channel write failures during teardown. + } +} + +function isTrackedProcessSignalEventName(eventName) { + return typeof eventName === 'string' && TRACKED_PROCESS_SIGNAL_EVENTS.has(eventName); +} + +function signalEventsAffectedByProcessMethod(methodName, eventName) { + if (methodName === 'removeAllListeners' && eventName == null) { + return [...TRACKED_PROCESS_SIGNAL_EVENTS]; + } + + return isTrackedProcessSignalEventName(eventName) ? [eventName] : []; +} + +function emitGuestProcessSignalState(eventName) { + if (!isTrackedProcessSignalEventName(eventName)) { + return; + } + + const signal = hostOs.constants?.signals?.[eventName]; + if (typeof signal !== 'number') { + return; + } + + const listenerCount = + typeof process.listenerCount === 'function' ? process.listenerCount(eventName) : 0; + emitControlMessage({ + type: 'signal_state', + signal: Number(signal) >>> 0, + registration: { + action: listenerCount > 0 ? 'user' : 'default', + mask: [], + flags: 0, + }, + }); +} + +function createBlockedProcessSignalMethod(methodName) { + const target = process; + const method = + typeof target[methodName] === 'function' ? target[methodName].bind(target) : null; + if (!method) { + return null; + } + + return (...args) => { + const [eventName] = args; + const affectedSignals = signalEventsAffectedByProcessMethod(methodName, eventName); + if (isProcessSignalEventName(eventName) && affectedSignals.length === 0) { + throw accessDenied(`process.${methodName}(${eventName})`); + } + + const result = method(...args); + for (const signalName of affectedSignals) { + emitGuestProcessSignalState(signalName); + } + return result === target ? guestProcess : result; + }; +} + +function createGuestProcessProxy(target) { + let proxy = null; + proxy = new Proxy(target, { + get(source, key) { + return Reflect.get(source, key, proxy); + }, + }); + return proxy; +} + +function normalizeGuestRequireDir(fromGuestDir) { + if (typeof fromGuestDir !== 'string' || fromGuestDir.length === 0) { + return INITIAL_GUEST_CWD; + } + + if (fromGuestDir.startsWith('file:')) { + try { + return path.posix.normalize(new URL(fromGuestDir).pathname); + } catch { + return INITIAL_GUEST_CWD; + } + } + + if (path.posix.isAbsolute(fromGuestDir)) { + return path.posix.normalize(fromGuestDir); + } + + return path.posix.normalize(path.posix.join(INITIAL_GUEST_CWD, fromGuestDir)); +} + +function isPathWithinRoot(candidatePath, rootPath) { + if (typeof candidatePath !== 'string' || typeof rootPath !== 'string') { + return false; + } + + const normalizedCandidate = path.resolve(candidatePath); + const normalizedRoot = path.resolve(rootPath); + return ( + normalizedCandidate === normalizedRoot || + normalizedCandidate.startsWith(`${normalizedRoot}${path.sep}`) + ); +} + +function runtimeHostPathFromGuestPath(guestPath) { + if (typeof guestPath !== 'string') { + return null; + } + + const translated = hostPathFromGuestPath(guestPath); + if (translated) { + return translated; + } + + const cwdGuestPath = guestPathFromHostPath(HOST_CWD); + if ( + typeof cwdGuestPath !== 'string' || + !path.posix.isAbsolute(guestPath) || + !path.posix.isAbsolute(cwdGuestPath) + ) { + return null; + } + + const relative = path.posix.relative(cwdGuestPath, path.posix.normalize(guestPath)); + if ( + relative.startsWith('..') || + relative === '..' || + path.posix.isAbsolute(relative) + ) { + return null; + } + + return relative ? path.join(HOST_CWD, ...relative.split('/')) : HOST_CWD; +} + +function translateModuleResolutionPath(value) { + if (typeof value !== 'string') { + return value; + } + + if (value.startsWith('file:')) { + try { + const guestPath = path.posix.normalize(new URL(value).pathname); + const hostPath = runtimeHostPathFromGuestPath(guestPath); + return hostPath ? pathToFileURL(hostPath).href : value; + } catch { + return value; + } + } + + if (path.posix.isAbsolute(value)) { + return runtimeHostPathFromGuestPath(value) ?? value; + } + + return value; +} + +function translateModuleResolutionParent(parent) { + if (!parent || typeof parent !== 'object') { + return parent; + } + + let nextParent = parent; + let changed = false; + + if (typeof parent.filename === 'string') { + const translatedFilename = translateModuleResolutionPath(parent.filename); + if (translatedFilename !== parent.filename) { + nextParent = { ...nextParent, filename: translatedFilename }; + changed = true; + } + } + + if (Array.isArray(parent.paths)) { + const translatedPaths = parent.paths.map((entry) => + translateModuleResolutionPath(entry), + ); + if (translatedPaths.some((entry, index) => entry !== parent.paths[index])) { + nextParent = { ...nextParent, paths: translatedPaths }; + changed = true; + } + } + + return changed ? nextParent : parent; +} + +function translateModuleResolutionOptions(options) { + if (Array.isArray(options)) { + return options.map((entry) => translateModuleResolutionPath(entry)); + } + + if (!options || typeof options !== 'object' || !Array.isArray(options.paths)) { + return options; + } + + const translatedPaths = options.paths.map((entry) => + translateModuleResolutionPath(entry), + ); + if (translatedPaths.every((entry, index) => entry === options.paths[index])) { + return options; + } + + return { + ...options, + paths: translatedPaths, + }; +} + +function ensureGuestVisibleModuleResolution(specifier, resolved, parent) { + if (typeof resolved !== 'string' || !path.isAbsolute(resolved)) { + return resolved; + } + + if ( + guestVisiblePathFromHostPath(resolved) || + isPathWithinRoot(resolved, HOST_CWD) + ) { + return resolved; + } + + const error = new Error(`Cannot find module '${specifier}'`); + error.code = 'MODULE_NOT_FOUND'; + if (typeof parent?.filename === 'string') { + error.requireStack = [translatePathStringToGuest(parent.filename)]; + } + throw translateErrorToGuest(error); +} + +function createGuestModuleCacheProxy(moduleCache) { + if (!moduleCache || typeof moduleCache !== 'object') { + return moduleCache; + } + + const toHostKey = (key) => + typeof key === 'string' ? translateModuleResolutionPath(key) : key; + const toGuestKey = (key) => + typeof key === 'string' ? translatePathStringToGuest(key) : key; + + return new Proxy(moduleCache, { + defineProperty(target, key, descriptor) { + return Reflect.defineProperty(target, toHostKey(key), descriptor); + }, + deleteProperty(target, key) { + return Reflect.deleteProperty(target, toHostKey(key)); + }, + get(target, key, receiver) { + return Reflect.get(target, toHostKey(key), receiver); + }, + getOwnPropertyDescriptor(target, key) { + const descriptor = Reflect.getOwnPropertyDescriptor(target, toHostKey(key)); + if (!descriptor) { + return descriptor; + } + return { + ...descriptor, + configurable: true, + }; + }, + has(target, key) { + return Reflect.has(target, toHostKey(key)); + }, + ownKeys(target) { + return Reflect.ownKeys(target).map((key) => toGuestKey(key)); + }, + set(target, key, value, receiver) { + return Reflect.set(target, toHostKey(key), value, receiver); + }, + }); +} + +const guestModuleCache = createGuestModuleCacheProxy(originalModuleCache); + +function createGuestRequire(fromGuestDir) { + const normalizedGuestDir = normalizeGuestRequireDir(fromGuestDir); + const cached = guestRequireCache.get(normalizedGuestDir); + if (cached) { + return cached; + } + + const baseRequire = Module.createRequire( + pathToFileURL(path.posix.join(normalizedGuestDir, '__agent_os_require__.cjs')), + ); + + const guestRequire = function(specifier) { + const translated = hostPathForSpecifier(specifier, normalizedGuestDir); + try { + if (translated) { + return baseRequire(translated); + } + + return baseRequire(specifier); + } catch (error) { + if (rootGuestRequire && rootGuestRequire !== guestRequire && isBareSpecifier(specifier)) { + return rootGuestRequire(specifier); + } + throw translateErrorToGuest(error); + } + }; + + guestRequire.resolve = (specifier, options) => { + const translated = hostPathForSpecifier(specifier, normalizedGuestDir); + try { + if (translated) { + return translatePathStringToGuest(baseRequire.resolve(translated, options)); + } + + return translatePathStringToGuest(baseRequire.resolve(specifier, options)); + } catch (error) { + if (rootGuestRequire && rootGuestRequire !== guestRequire && isBareSpecifier(specifier)) { + return rootGuestRequire.resolve(specifier, options); + } + throw translateErrorToGuest(error); + } + }; + + guestRequire.cache = guestModuleCache; + + guestRequireCache.set(normalizedGuestDir, guestRequire); + return guestRequire; +} + +function hardenProperty(target, key, value) { + try { + Object.defineProperty(target, key, { + value, + writable: false, + configurable: false, + }); + } catch (error) { + throw new Error(`Failed to harden property ${String(key)}`, { cause: error }); + } +} + +function encodeSyncRpcValue(value) { + if (value == null || typeof value === 'string' || typeof value === 'number' || typeof value === 'boolean') { + return value; + } + + if (typeof Buffer === 'function' && Buffer.isBuffer(value)) { + return { + __agentOsType: 'bytes', + base64: value.toString('base64'), + }; + } + + if (ArrayBuffer.isView(value)) { + return { + __agentOsType: 'bytes', + base64: Buffer.from(value.buffer, value.byteOffset, value.byteLength).toString('base64'), + }; + } + + if (value instanceof ArrayBuffer) { + return { + __agentOsType: 'bytes', + base64: Buffer.from(value).toString('base64'), + }; + } + + if (Array.isArray(value)) { + return value.map((entry) => encodeSyncRpcValue(entry)); + } + + if (typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, entry]) => [key, encodeSyncRpcValue(entry)]), + ); + } + + return String(value); +} + +function decodeSyncRpcValue(value) { + if (Array.isArray(value)) { + return value.map((entry) => decodeSyncRpcValue(entry)); + } + + if (value && typeof value === 'object') { + if (value.__agentOsType === 'bytes' && typeof value.base64 === 'string') { + return Buffer.from(value.base64, 'base64'); + } + + return Object.fromEntries( + Object.entries(value).map(([key, entry]) => [key, decodeSyncRpcValue(entry)]), + ); + } + + return value; +} + +function formatSyncRpcError(error) { + if (error instanceof Error) { + return { + message: error.message, + code: typeof error.code === 'string' ? error.code : undefined, + }; + } + + return { + message: String(error), + }; +} + +function createNodeSyncRpcBridge() { + if (!NODE_SYNC_RPC_ENABLE) { + return null; + } + + if (NODE_SYNC_RPC_REQUEST_FD == null || NODE_SYNC_RPC_RESPONSE_FD == null) { + throw new Error('Agent OS Node sync RPC requires request and response file descriptors'); + } + + const Worker = hostWorkerThreads?.Worker; + if (typeof Worker !== 'function') { + throw new Error('Agent OS Node sync RPC requires node:worker_threads support'); + } + + const STATE_INDEX = 0; + const STATUS_INDEX = 1; + const KIND_INDEX = 2; + const REQUEST_LENGTH_INDEX = 3; + const RESPONSE_LENGTH_INDEX = 4; + const STATE_IDLE = 0; + const STATE_REQUEST_READY = 1; + const STATE_RESPONSE_READY = 2; + const STATE_SHUTDOWN = 3; + const STATUS_OK = 0; + const STATUS_ERROR = 1; + const KIND_JSON = 3; + const signalBuffer = new SharedArrayBuffer(5 * Int32Array.BYTES_PER_ELEMENT); + const dataBuffer = new SharedArrayBuffer(NODE_SYNC_RPC_DATA_BYTES); + const signal = new Int32Array(signalBuffer); + const data = new Uint8Array(dataBuffer); + const encoder = new TextEncoder(); + const decoder = new TextDecoder(); + let nextRequestId = 1; + let disposed = false; + + const workerSource = ` + const { parentPort, workerData } = require('node:worker_threads'); + const { readSync, writeSync, closeSync } = require('node:fs'); + const STATE_INDEX = 0; + const STATUS_INDEX = 1; + const KIND_INDEX = 2; + const REQUEST_LENGTH_INDEX = 3; + const RESPONSE_LENGTH_INDEX = 4; + const STATE_IDLE = 0; + const STATE_REQUEST_READY = 1; + const STATE_RESPONSE_READY = 2; + const STATE_SHUTDOWN = 3; + const STATUS_OK = 0; + const STATUS_ERROR = 1; + const KIND_JSON = 3; + const signal = new Int32Array(workerData.signalBuffer); + const data = new Uint8Array(workerData.dataBuffer); + const responseFd = workerData.responseFd; + const encoder = new TextEncoder(); + const decoder = new TextDecoder(); + let responseBuffer = ''; + + function setResponse(status, bytes) { + let payload = bytes; + let nextStatus = status; + if (payload.byteLength > data.byteLength) { + payload = encoder.encode(JSON.stringify({ + message: 'Agent OS Node sync RPC payload exceeded shared buffer capacity', + code: 'ERR_AGENT_OS_NODE_SYNC_RPC_PAYLOAD_TOO_LARGE', + })); + nextStatus = STATUS_ERROR; + } + + data.fill(0); + data.set(payload, 0); + Atomics.store(signal, STATUS_INDEX, nextStatus); + Atomics.store(signal, KIND_INDEX, KIND_JSON); + Atomics.store(signal, RESPONSE_LENGTH_INDEX, payload.byteLength); + Atomics.store(signal, STATE_INDEX, STATE_RESPONSE_READY); + Atomics.notify(signal, STATE_INDEX, 1); + } + + function readResponseLineSync() { + while (true) { + const newlineIndex = responseBuffer.indexOf('\\n'); + if (newlineIndex >= 0) { + const line = responseBuffer.slice(0, newlineIndex); + responseBuffer = responseBuffer.slice(newlineIndex + 1); + return line; + } + + const chunk = Buffer.alloc(4096); + const bytesRead = readSync(responseFd, chunk, 0, chunk.length, null); + if (bytesRead === 0) { + throw new Error('Agent OS Node sync RPC response channel closed unexpectedly'); + } + responseBuffer += chunk.subarray(0, bytesRead).toString('utf8'); + } + } + + function waitForRequest() { + while (true) { + const state = Atomics.load(signal, STATE_INDEX); + if (state === STATE_REQUEST_READY || state === STATE_SHUTDOWN) { + return state; + } + + Atomics.wait(signal, STATE_INDEX, state); + } + } + + try { + while (true) { + const state = waitForRequest(); + if (state === STATE_SHUTDOWN) { + break; + } + + try { + const responseLine = readResponseLineSync(); + setResponse(STATUS_OK, encoder.encode(responseLine)); + } catch (error) { + setResponse( + STATUS_ERROR, + encoder.encode(JSON.stringify({ + message: error instanceof Error ? error.message : String(error), + code: typeof error?.code === 'string' ? error.code : 'ERR_AGENT_OS_NODE_SYNC_RPC', + })), + ); + } + } + } finally { + try { + closeSync(responseFd); + } catch {} + } + `; + + const worker = new Worker(workerSource, { + eval: true, + workerData: { + signalBuffer, + dataBuffer, + responseFd: NODE_SYNC_RPC_RESPONSE_FD, + }, + }); + worker.unref?.(); + + const readBytes = (length) => { + if (length <= 0) { + return new Uint8Array(0); + } + return data.slice(0, length); + }; + + const resetSignal = () => { + Atomics.store(signal, STATUS_INDEX, STATUS_OK); + Atomics.store(signal, KIND_INDEX, KIND_JSON); + Atomics.store(signal, REQUEST_LENGTH_INDEX, 0); + Atomics.store(signal, RESPONSE_LENGTH_INDEX, 0); + Atomics.store(signal, STATE_INDEX, STATE_IDLE); + Atomics.notify(signal, STATE_INDEX, 1); + }; + + const requestRaw = (method, args = []) => { + if (disposed) { + throw new Error('Agent OS Node sync RPC bridge is already disposed'); + } + + const payload = encoder.encode( + JSON.stringify({ + id: nextRequestId++, + method, + args: encodeSyncRpcValue(args), + }), + ); + if (payload.byteLength > data.byteLength) { + const error = new Error('Agent OS Node sync RPC request exceeded shared buffer capacity'); + error.code = 'ERR_AGENT_OS_NODE_SYNC_RPC_PAYLOAD_TOO_LARGE'; + throw error; + } + + data.fill(0); + data.set(payload, 0); + hostFsWriteSync( + NODE_SYNC_RPC_REQUEST_FD, + `${decoder.decode(data.subarray(0, payload.byteLength))}\n`, + ); + Atomics.store(signal, STATUS_INDEX, STATUS_OK); + Atomics.store(signal, KIND_INDEX, KIND_JSON); + Atomics.store(signal, REQUEST_LENGTH_INDEX, payload.byteLength); + Atomics.store(signal, RESPONSE_LENGTH_INDEX, 0); + Atomics.store(signal, STATE_INDEX, STATE_REQUEST_READY); + Atomics.notify(signal, STATE_INDEX, 1); + + while (true) { + const result = Atomics.wait( + signal, + STATE_INDEX, + STATE_REQUEST_READY, + NODE_SYNC_RPC_WAIT_TIMEOUT_MS, + ); + if (result !== 'timed-out') { + break; + } + throw new Error(`Agent OS Node sync RPC timed out while handling ${method}`); + } + + const status = Atomics.load(signal, STATUS_INDEX); + const kind = Atomics.load(signal, KIND_INDEX); + const length = Atomics.load(signal, RESPONSE_LENGTH_INDEX); + const bytes = readBytes(length); + resetSignal(); + + if (kind !== KIND_JSON) { + throw new Error(`Agent OS Node sync RPC returned unsupported payload kind ${kind}`); + } + + if (status === STATUS_ERROR) { + const payload = JSON.parse(decoder.decode(bytes)); + const error = new Error(payload?.message || `Agent OS Node sync RPC ${method} failed`); + if (typeof payload?.code === 'string') { + error.code = payload.code; + } + throw error; + } + + return JSON.parse(decoder.decode(bytes)); + }; + + return { + callSync(method, args = []) { + const response = requestRaw(method, args); + if (response?.ok) { + return decodeSyncRpcValue(response.result); + } + + const error = new Error( + response?.error?.message || `Agent OS Node sync RPC ${method} failed`, + ); + if (typeof response?.error?.code === 'string') { + error.code = response.error.code; + } + throw error; + }, + async call(method, args = []) { + return this.callSync(method, args); + }, + dispose() { + if (disposed) { + return; + } + disposed = true; + Atomics.store(signal, STATE_INDEX, STATE_SHUTDOWN); + Atomics.notify(signal, STATE_INDEX, 1); + try { + hostFsCloseSync(NODE_SYNC_RPC_REQUEST_FD); + } catch {} + worker.terminate().catch(() => {}); + }, + }; +} + +function installGuestHardening() { + hardenProperty(process, 'env', createGuestProcessEnv(HOST_PROCESS_ENV)); + hardenProperty(process, 'cwd', () => INITIAL_GUEST_CWD); + hardenProperty(process, 'chdir', () => { + throw accessDenied('process.chdir'); + }); + syncBuiltinModuleExports(hostFs, guestFs); + syncBuiltinModuleExports(hostFsPromises, guestFs.promises); + try { + syncBuiltinESMExports(); + } catch { + // Ignore runtimes that reject syncing builtin ESM exports. + } + + hardenProperty(process, 'execPath', VIRTUAL_EXEC_PATH); + hardenProperty(process, 'pid', VIRTUAL_PID); + hardenProperty(process, 'ppid', VIRTUAL_PPID); + hardenProperty(process, 'version', VIRTUAL_PROCESS_VERSION); + hardenProperty(process, 'versions', VIRTUAL_PROCESS_VERSIONS); + hardenProperty(process, 'release', VIRTUAL_PROCESS_RELEASE); + hardenProperty(process, 'config', VIRTUAL_PROCESS_CONFIG); + hardenProperty(process, 'platform', VIRTUAL_OS_PLATFORM); + hardenProperty(process, 'arch', VIRTUAL_OS_ARCH); + hardenProperty(process, 'memoryUsage', guestMemoryUsage); + hardenProperty(process, 'uptime', guestProcessUptime); + hardenProperty(process, 'getuid', guestGetUid); + hardenProperty(process, 'getgid', guestGetGid); + hardenProperty(process, 'umask', guestProcessUmask); + + hardenProperty(process, 'binding', () => { + throw accessDenied('process.binding'); + }); + hardenProperty(process, '_linkedBinding', () => { + throw accessDenied('process._linkedBinding'); + }); + hardenProperty(process, 'dlopen', () => { + throw accessDenied('process.dlopen'); + }); + for (const methodName of [ + 'addListener', + 'on', + 'once', + 'removeAllListeners', + 'removeListener', + 'off', + 'prependListener', + 'prependOnceListener', + ]) { + const blockedMethod = createBlockedProcessSignalMethod(methodName); + if (blockedMethod) { + hardenProperty(process, methodName, blockedMethod); + } + } + if (Module?._extensions && typeof Module._extensions === 'object') { + hardenProperty(Module._extensions, '.node', () => { + throw accessDenied('native addon loading'); + }); + } + if (originalGetBuiltinModule) { + hardenProperty(process, 'getBuiltinModule', (specifier) => { + const normalized = + typeof specifier === 'string' ? normalizeBuiltin(specifier) : null; + if (normalized === 'process') { + return guestProcess; + } + if (normalized === 'fs') { + return cloneFsModule(guestFs); + } + if (normalized === 'os' && ALLOWED_BUILTINS.has('os')) { + return guestOs; + } + if (normalized === 'net' && ALLOWED_BUILTINS.has('net')) { + return guestNet; + } + if (normalized === 'dgram' && ALLOWED_BUILTINS.has('dgram')) { + return guestDgram; + } + if (normalized === 'dns' && ALLOWED_BUILTINS.has('dns')) { + return guestDns; + } + if (normalized === 'http' && ALLOWED_BUILTINS.has('http')) { + return guestHttp; + } + if (normalized === 'http2' && ALLOWED_BUILTINS.has('http2')) { + return guestHttp2; + } + if (normalized === 'https' && ALLOWED_BUILTINS.has('https')) { + return guestHttps; + } + if (normalized === 'tls' && ALLOWED_BUILTINS.has('tls')) { + return guestTls; + } + if (normalized === 'child_process' && ALLOWED_BUILTINS.has('child_process')) { + return guestChildProcess; + } + if (normalized && DENIED_BUILTINS.has(normalized)) { + throw accessDenied(`node:${normalized}`); + } + return originalGetBuiltinModule(specifier); + }); + } + + if (originalModuleLoad) { + Module._load = function(request, parent, isMain) { + const normalized = + typeof request === 'string' ? normalizeBuiltin(request) : null; + if (normalized === 'process') { + return guestProcess; + } + if (normalized === 'fs') { + return cloneFsModule(guestFs); + } + if (normalized === 'os' && ALLOWED_BUILTINS.has('os')) { + return guestOs; + } + if (normalized === 'net' && ALLOWED_BUILTINS.has('net')) { + return guestNet; + } + if (normalized === 'dgram' && ALLOWED_BUILTINS.has('dgram')) { + return guestDgram; + } + if (normalized === 'dns' && ALLOWED_BUILTINS.has('dns')) { + return guestDns; + } + if (normalized === 'http' && ALLOWED_BUILTINS.has('http')) { + return guestHttp; + } + if (normalized === 'http2' && ALLOWED_BUILTINS.has('http2')) { + return guestHttp2; + } + if (normalized === 'https' && ALLOWED_BUILTINS.has('https')) { + return guestHttps; + } + if (normalized === 'tls' && ALLOWED_BUILTINS.has('tls')) { + return guestTls; + } + if (normalized === 'child_process' && ALLOWED_BUILTINS.has('child_process')) { + return guestChildProcess; + } + if (normalized && DENIED_BUILTINS.has(normalized)) { + throw accessDenied(`node:${normalized}`); + } + + return originalModuleLoad(request, parent, isMain); + }; + } + + if (originalModuleResolveFilename) { + Module._resolveFilename = function(request, parent, isMain, options) { + const translatedRequest = translateModuleResolutionPath(request); + const translatedParent = translateModuleResolutionParent(parent); + const translatedOptions = translateModuleResolutionOptions(options); + const resolved = originalModuleResolveFilename( + translatedRequest, + translatedParent, + isMain, + translatedOptions, + ); + return ensureGuestVisibleModuleResolution( + request, + resolved, + translatedParent, + ); + }; + } + + if (guestModuleCache) { + hardenProperty(Module, '_cache', guestModuleCache); + } + + if (originalFetch) { + const restrictedFetch = async (resource, init) => { + const candidate = + typeof resource === 'string' + ? resource + : resource instanceof URL + ? resource.href + : resource?.url; + + let url; + try { + url = new URL(String(candidate ?? '')); + } catch { + throw accessDenied('network access'); + } + + if (url.protocol !== 'data:') { + const normalizedPort = + url.port || (url.protocol === 'https:' ? '443' : url.protocol === 'http:' ? '80' : ''); + const loopbackHost = + url.hostname === '127.0.0.1' || + url.hostname === 'localhost' || + url.hostname === '::1' || + url.hostname === '[::1]'; + const loopbackAllowed = + loopbackHost && + (url.protocol === 'http:' || url.protocol === 'https:') && + LOOPBACK_EXEMPT_PORTS.has(normalizedPort); + + if (!loopbackAllowed) { + throw accessDenied(`network access to ${url.protocol}`); + } + } + + return originalFetch(resource, init); + }; + + hardenProperty(globalThis, 'fetch', restrictedFetch); + } +} + +const entrypoint = HOST_PROCESS_ENV.AGENT_OS_ENTRYPOINT; +if (!entrypoint) { + throw new Error('AGENT_OS_ENTRYPOINT is required'); +} + +const guestSyncRpc = createNodeSyncRpcBridge(); +installGuestHardening(); +rootGuestRequire = createGuestRequire('/root/node_modules'); +if (ALLOWED_BUILTINS.has('child_process')) { + hardenProperty(globalThis, '__agentOsBuiltinChildProcess', guestChildProcess); +} +hardenProperty(globalThis, '__agentOsBuiltinFs', guestFs); +if (ALLOWED_BUILTINS.has('net')) { + hardenProperty(globalThis, '__agentOsBuiltinNet', guestNet); +} +if (ALLOWED_BUILTINS.has('dgram')) { + hardenProperty(globalThis, '__agentOsBuiltinDgram', guestDgram); +} +if (ALLOWED_BUILTINS.has('dns')) { + hardenProperty(globalThis, '__agentOsBuiltinDns', guestDns); +} +if (ALLOWED_BUILTINS.has('http')) { + hardenProperty(globalThis, '__agentOsBuiltinHttp', guestHttp); +} +if (ALLOWED_BUILTINS.has('http2')) { + hardenProperty(globalThis, '__agentOsBuiltinHttp2', guestHttp2); +} +if (ALLOWED_BUILTINS.has('https')) { + hardenProperty(globalThis, '__agentOsBuiltinHttps', guestHttps); +} +if (ALLOWED_BUILTINS.has('tls')) { + hardenProperty(globalThis, '__agentOsBuiltinTls', guestTls); +} +if (ALLOWED_BUILTINS.has('os')) { + hardenProperty(globalThis, '__agentOsBuiltinOs', guestOs); +} +if (guestSyncRpc) { + hardenProperty(globalThis, '__agentOsSyncRpc', guestSyncRpc); +} +hardenProperty(globalThis, '_requireFrom', (specifier, fromDir = '/') => + createGuestRequire(fromDir)(specifier), +); +hardenProperty( + globalThis, + 'require', + createGuestRequire(path.posix.dirname(guestEntryPoint ?? entrypoint)), +); + +if (HOST_PROCESS_ENV.AGENT_OS_KEEP_STDIN_OPEN === '1') { + let stdinKeepalive = setInterval(() => {}, 1_000_000); + const releaseStdinKeepalive = () => { + if (stdinKeepalive !== null) { + clearInterval(stdinKeepalive); + stdinKeepalive = null; + } + }; + + process.stdin.resume(); + process.stdin.once('end', releaseStdinKeepalive); + process.stdin.once('close', releaseStdinKeepalive); + process.stdin.once('error', releaseStdinKeepalive); +} + +const guestArgv = JSON.parse(HOST_PROCESS_ENV.AGENT_OS_GUEST_ARGV ?? '[]'); +const bootstrapModule = HOST_PROCESS_ENV.AGENT_OS_BOOTSTRAP_MODULE; +const entrypointPath = isPathLike(entrypoint) + ? path.resolve(process.cwd(), entrypoint) + : entrypoint; + +process.argv = [VIRTUAL_EXEC_PATH, guestEntryPoint ?? entrypointPath, ...guestArgv]; +guestProcess = createGuestProcessProxy(process); +hardenProperty(globalThis, 'process', guestProcess); + +try { + if (bootstrapModule) { + await import(toImportSpecifier(bootstrapModule)); + } + + await import(toImportSpecifier(entrypoint)); +} catch (error) { + throw translateErrorToGuest(error); +} finally { + guestSyncRpc?.dispose?.(); +} +"#; + +const NODE_TIMING_BOOTSTRAP_SOURCE: &str = r#" +const frozenTimeValue = Number(process.env.AGENT_OS_FROZEN_TIME_MS); +const frozenTimeMs = Number.isFinite(frozenTimeValue) ? Math.trunc(frozenTimeValue) : Date.now(); +const frozenDateNow = () => frozenTimeMs; +const OriginalDate = Date; + +function FrozenDate(...args) { + if (new.target) { + if (args.length === 0) { + return new OriginalDate(frozenTimeMs); + } + return new OriginalDate(...args); + } + return new OriginalDate(frozenTimeMs).toString(); +} + +Object.setPrototypeOf(FrozenDate, OriginalDate); +Object.defineProperty(FrozenDate, 'prototype', { + value: OriginalDate.prototype, + writable: false, + configurable: false, +}); +FrozenDate.parse = OriginalDate.parse; +FrozenDate.UTC = OriginalDate.UTC; +Object.defineProperty(FrozenDate, 'now', { + value: frozenDateNow, + writable: false, + configurable: false, +}); + +try { + Object.defineProperty(globalThis, 'Date', { + value: FrozenDate, + writable: false, + configurable: false, + }); +} catch { + globalThis.Date = FrozenDate; +} + +const originalPerformance = globalThis.performance; +const frozenPerformance = Object.create(null); +if (typeof originalPerformance !== 'undefined' && originalPerformance !== null) { + const performanceSource = + Object.getPrototypeOf(originalPerformance) ?? originalPerformance; + for (const key of Object.getOwnPropertyNames(performanceSource)) { + if (key === 'now') { + continue; + } + try { + const value = originalPerformance[key]; + frozenPerformance[key] = + typeof value === 'function' ? value.bind(originalPerformance) : value; + } catch { + // Ignore properties that throw during access. + } + } +} +Object.defineProperty(frozenPerformance, 'now', { + value: () => 0, + writable: false, + configurable: false, +}); +Object.freeze(frozenPerformance); + +try { + Object.defineProperty(globalThis, 'performance', { + value: frozenPerformance, + writable: false, + configurable: false, + }); +} catch { + globalThis.performance = frozenPerformance; +} + +const frozenHrtimeBigint = BigInt(frozenTimeMs) * 1000000n; +const frozenHrtime = (previous) => { + const seconds = Math.trunc(frozenTimeMs / 1000); + const nanoseconds = Math.trunc((frozenTimeMs % 1000) * 1000000); + + if (!Array.isArray(previous) || previous.length < 2) { + return [seconds, nanoseconds]; + } + + let deltaSeconds = seconds - Number(previous[0]); + let deltaNanoseconds = nanoseconds - Number(previous[1]); + if (deltaNanoseconds < 0) { + deltaSeconds -= 1; + deltaNanoseconds += 1000000000; } + return [deltaSeconds, deltaNanoseconds]; +}; +frozenHrtime.bigint = () => frozenHrtimeBigint; - return specifier.startsWith('node:') ? specifier.slice('node:'.length) : specifier; +try { + process.hrtime = frozenHrtime; +} catch { + // Ignore runtimes that expose a non-writable process.hrtime binding. } +"#; -function installPythonGuestHardening() { - const assetRoot = process.env[ASSET_ROOT_ENV]; - if (assetRoot) { - register(new URL('./loader.mjs', import.meta.url), import.meta.url); - } +const NODE_PREWARM_SOURCE: &str = r#" +import path from 'node:path'; +import { pathToFileURL } from 'node:url'; - hardenProperty(process, 'binding', () => { - throw accessDenied('process.binding'); - }); - hardenProperty(process, '_linkedBinding', () => { - throw accessDenied('process._linkedBinding'); - }); - hardenProperty(process, 'dlopen', () => { - throw accessDenied('process.dlopen'); - }); +function isPathLike(specifier) { + return specifier.startsWith('.') || specifier.startsWith('/') || specifier.startsWith('file:'); +} - if (originalGetBuiltinModule) { - hardenProperty(process, 'getBuiltinModule', (specifier) => { - const normalized = normalizeBuiltin(specifier); - if (normalized && DENIED_BUILTINS.has(normalized)) { - throw accessDenied(`node:${normalized}`); - } - return originalGetBuiltinModule(specifier); - }); +function toImportSpecifier(specifier) { + if (specifier.startsWith('file:')) { + return specifier; } - - if (originalRequire) { - hardenProperty(globalThis, 'require', () => { - throw accessDenied('require'); - }); + if (isPathLike(specifier)) { + return pathToFileURL(path.resolve(process.cwd(), specifier)).href; } + return specifier; +} - if (originalFetch) { - const restrictedFetch = (resource, init) => { - const candidate = - typeof resource === 'string' - ? resource - : resource instanceof URL - ? resource.href - : resource?.url; - - let url; - try { - url = new URL(String(candidate ?? '')); - } catch { - throw accessDenied('network access'); - } +const imports = JSON.parse(process.env.AGENT_OS_NODE_PREWARM_IMPORTS ?? '[]'); +for (const specifier of imports) { + await import(toImportSpecifier(specifier)); +} +"#; - if (url.protocol !== 'data:') { - throw accessDenied(`network access to ${url.protocol}`); - } +const NODE_WASM_RUNNER_SOURCE: &str = r#" +import fs from 'node:fs/promises'; +import { writeSync } from 'node:fs'; +import path from 'node:path'; +import { WASI } from 'node:wasi'; - return originalFetch(resource, init); - }; +const WASI_ERRNO_SUCCESS = 0; +const WASI_ERRNO_ROFS = 69; +const WASI_ERRNO_FAULT = 21; +const WASI_RIGHT_FD_WRITE = 64n; +const WASM_PAGE_BYTES = 65536; - hardenProperty(globalThis, 'fetch', restrictedFetch); - } +function isPathLike(specifier) { + return specifier.startsWith('.') || specifier.startsWith('/') || specifier.startsWith('file:'); } -function installPythonVfsRpcBridge() { - const bridge = createPythonVfsRpcBridge(); - if (!bridge) { - return null; +function resolveModulePath(specifier) { + if (specifier.startsWith('file:')) { + return new URL(specifier); + } + if (isPathLike(specifier)) { + return path.resolve(process.cwd(), specifier); } + return specifier; +} - hardenProperty(globalThis, '__agentOsPythonVfsRpc', bridge); - return bridge; +const modulePath = process.env.AGENT_OS_WASM_MODULE_PATH; +if (!modulePath) { + throw new Error('AGENT_OS_WASM_MODULE_PATH is required'); } -function installPythonWorkspaceFs(pyodide, bridge) { - if (!bridge) { - return; +const guestArgv = JSON.parse(process.env.AGENT_OS_GUEST_ARGV ?? '[]'); +const guestEnv = JSON.parse(process.env.AGENT_OS_GUEST_ENV ?? '{}'); +const permissionTier = process.env.AGENT_OS_WASM_PERMISSION_TIER ?? 'full'; +const prewarmOnly = process.env.AGENT_OS_WASM_PREWARM_ONLY === '1'; +const maxMemoryBytesValue = Number(process.env.AGENT_OS_WASM_MAX_MEMORY_BYTES); +const maxMemoryPages = Number.isFinite(maxMemoryBytesValue) + ? Math.max(0, Math.floor(maxMemoryBytesValue / WASM_PAGE_BYTES)) + : null; +const frozenTimeValue = Number(process.env.AGENT_OS_FROZEN_TIME_MS); +const frozenTimeMs = Number.isFinite(frozenTimeValue) ? Math.trunc(frozenTimeValue) : Date.now(); +const frozenTimeNs = BigInt(frozenTimeMs) * 1000000n; +const CONTROL_PIPE_FD = parseControlPipeFd(process.env.AGENT_OS_CONTROL_PIPE_FD); + +function buildPreopens() { + switch (permissionTier) { + case 'isolated': + return {}; + case 'read-only': + case 'read-write': + case 'full': + default: + return { + '/workspace': process.cwd(), + }; } +} - const { FS, ERRNO_CODES } = pyodide; - if (!FS?.mount || !FS?.filesystems?.MEMFS || !ERRNO_CODES) { - return; +function readVarUint(bytes, offset, label) { + let value = 0; + let shift = 0; + let cursor = offset; + for (let count = 0; count < 10; count += 1) { + if (cursor >= bytes.length) { + throw new Error(`WebAssembly ${label} truncated`); + } + const byte = bytes[cursor]; + cursor += 1; + value += (byte & 0x7f) * 2 ** shift; + if ((byte & 0x80) === 0) { + return { value, offset: cursor }; + } + shift += 7; } + throw new Error(`WebAssembly ${label} exceeds varuint limit`); +} + +function encodeVarUint(value) { + const encoded = []; + let remaining = Math.trunc(value); + do { + let byte = remaining & 0x7f; + remaining = Math.floor(remaining / 128); + if (remaining > 0) { + byte |= 0x80; + } + encoded.push(byte); + } while (remaining > 0); + return encoded; +} - const MEMFS = FS.filesystems.MEMFS; - const memfsDirNodeOps = MEMFS.ops_table.dir.node; - const memfsDirStreamOps = MEMFS.ops_table.dir.stream; - const memfsFileNodeOps = MEMFS.ops_table.file.node; - const memfsFileStreamOps = MEMFS.ops_table.file.stream; - const workspaceDirStreamOps = memfsDirStreamOps; +function rewriteMemorySection(sectionBytes, limitPages) { + let offset = 0; + const countResult = readVarUint(sectionBytes, offset, 'memory count'); + const count = countResult.value; + offset = countResult.offset; + const rewritten = [...encodeVarUint(count)]; - function joinGuestPath(parentPath, name) { - return parentPath === '/' ? `/${name}` : `${parentPath}/${name}`; - } + for (let index = 0; index < count; index += 1) { + const flagsResult = readVarUint(sectionBytes, offset, 'memory flags'); + const flags = flagsResult.value; + offset = flagsResult.offset; - function nodeGuestPath(node) { - return node.agentOsGuestPath || node.mount?.mountpoint || '/workspace'; - } + if ((flags & ~1) !== 0) { + throw new Error( + `configured WebAssembly memory limit does not support memory flags ${flags}`, + ); + } - function createFsError(error) { - if (error instanceof FS.ErrnoError) { - return error; + const initialResult = readVarUint(sectionBytes, offset, 'memory minimum'); + const initialPages = initialResult.value; + offset = initialResult.offset; + + let maximumPages = null; + if ((flags & 1) !== 0) { + const maximumResult = readVarUint(sectionBytes, offset, 'memory maximum'); + maximumPages = maximumResult.value; + offset = maximumResult.offset; } - const message = String(error?.message || error); - let errno = ERRNO_CODES.EIO; - if (/permission denied|access denied|denied/i.test(message)) { - errno = ERRNO_CODES.EACCES; - } else if (/read-only|erofs/i.test(message)) { - errno = ERRNO_CODES.EROFS; - } else if (/not a directory|enotdir/i.test(message)) { - errno = ERRNO_CODES.ENOTDIR; - } else if (/is a directory|eisdir/i.test(message)) { - errno = ERRNO_CODES.EISDIR; - } else if (/exists|already exists|eexist/i.test(message)) { - errno = ERRNO_CODES.EEXIST; - } else if (/not found|no such file|enoent/i.test(message)) { - errno = ERRNO_CODES.ENOENT; + if (initialPages > limitPages) { + throw new Error( + `initial WebAssembly memory of ${initialPages * WASM_PAGE_BYTES} bytes exceeds the configured limit of ${limitPages * WASM_PAGE_BYTES} bytes`, + ); } - return new FS.ErrnoError(errno); + const cappedMaximumPages = + maximumPages == null ? limitPages : Math.min(maximumPages, limitPages); + rewritten.push(...encodeVarUint(1)); + rewritten.push(...encodeVarUint(initialPages)); + rewritten.push(...encodeVarUint(cappedMaximumPages)); } - function withFsErrors(operation) { - try { - return operation(); - } catch (error) { - throw createFsError(error); - } + if (offset !== sectionBytes.length) { + throw new Error('memory section parsing did not consume the full section'); } - function updateNodeFromRemoteStat(node, stat) { - if (!stat) { - throw new FS.ErrnoError(ERRNO_CODES.ENOENT); - } + return rewritten; +} - node.mode = stat.mode; - node.timestamp = Date.now(); - if (FS.isFile(stat.mode) && !node.agentOsDirty) { - node.agentOsRemoteSize = stat.size; - } +function enforceMemoryLimit(moduleBytes, limitPages) { + if (!Number.isInteger(limitPages)) { + return moduleBytes; } - function createWorkspaceNode(parent, name, mode, dev, guestPath) { - const node = MEMFS.createNode(parent, name, mode, dev); - node.agentOsGuestPath = guestPath; - node.agentOsDirty = false; - node.agentOsLoaded = FS.isDir(mode); - node.agentOsRemoteSize = 0; - if (FS.isDir(mode)) { - node.node_ops = workspaceDirNodeOps; - node.stream_ops = workspaceDirStreamOps; - } else if (FS.isFile(mode)) { - node.node_ops = workspaceFileNodeOps; - node.stream_ops = workspaceFileStreamOps; - } - return node; + const bytes = moduleBytes instanceof Uint8Array ? moduleBytes : new Uint8Array(moduleBytes); + if (bytes.length < 8 || bytes[0] !== 0 || bytes[1] !== 0x61 || bytes[2] !== 0x73 || bytes[3] !== 0x6d) { + throw new Error('module is not a valid WebAssembly binary'); } - function syncDirectory(node) { - const guestPath = nodeGuestPath(node); - const entries = withFsErrors(() => bridge.fsReaddirSync(guestPath)); - const remoteNames = new Set(entries); + const rewritten = Array.from(bytes.slice(0, 8)); + let offset = 8; - for (const name of Object.keys(node.contents || {})) { - if (remoteNames.has(name)) { - continue; - } + while (offset < bytes.length) { + const sectionStart = offset; + const sectionId = bytes[offset]; + offset += 1; + const sectionSizeResult = readVarUint(bytes, offset, 'section size'); + const sectionSize = sectionSizeResult.value; + offset = sectionSizeResult.offset; + const sectionEnd = offset + sectionSize; + if (sectionEnd > bytes.length) { + throw new Error('section extends past end of module'); + } - const child = node.contents[name]; - if (FS.isDir(child.mode)) { - memfsDirNodeOps.rmdir(node, name); - } else { - memfsDirNodeOps.unlink(node, name); - } + if (sectionId !== 5) { + rewritten.push(...bytes.slice(sectionStart, sectionEnd)); + offset = sectionEnd; + continue; } - for (const name of entries) { - const childPath = joinGuestPath(guestPath, name); - const stat = withFsErrors(() => bridge.fsStatSync(childPath)); - const existing = node.contents[name]; + const rewrittenSection = rewriteMemorySection(bytes.slice(offset, sectionEnd), limitPages); + rewritten.push(sectionId); + rewritten.push(...encodeVarUint(rewrittenSection.length)); + rewritten.push(...rewrittenSection); + offset = sectionEnd; + } - if (existing) { - const existingIsDir = FS.isDir(existing.mode); - const remoteIsDir = Boolean(stat?.isDirectory); - if (existingIsDir !== remoteIsDir) { - if (existingIsDir) { - memfsDirNodeOps.rmdir(node, name); - } else { - memfsDirNodeOps.unlink(node, name); - } - } else { - existing.agentOsGuestPath = childPath; - updateNodeFromRemoteStat(existing, stat); - if (FS.isFile(existing.mode) && !existing.agentOsDirty) { - existing.agentOsLoaded = false; - } - continue; - } - } + return Buffer.from(rewritten); +} + +const moduleBytes = enforceMemoryLimit( + await fs.readFile(resolveModulePath(modulePath)), + maxMemoryPages, +); +const module = await WebAssembly.compile(moduleBytes); + +if (prewarmOnly) { + process.exit(0); +} + +const wasi = new WASI({ + version: 'preview1', + args: guestArgv, + env: guestEnv, + preopens: buildPreopens(), + returnOnExit: true, +}); + +let instanceMemory = null; +const wasiImport = { ...wasi.wasiImport }; +const delegateClockTimeGet = + typeof wasi.wasiImport.clock_time_get === 'function' + ? wasi.wasiImport.clock_time_get.bind(wasi.wasiImport) + : null; +const delegateClockResGet = + typeof wasi.wasiImport.clock_res_get === 'function' + ? wasi.wasiImport.clock_res_get.bind(wasi.wasiImport) + : null; +const delegatePathOpen = + typeof wasi.wasiImport.path_open === 'function' + ? wasi.wasiImport.path_open.bind(wasi.wasiImport) + : null; +const delegateFdWrite = + typeof wasi.wasiImport.fd_write === 'function' + ? wasi.wasiImport.fd_write.bind(wasi.wasiImport) + : null; +const delegateFdPwrite = + typeof wasi.wasiImport.fd_pwrite === 'function' + ? wasi.wasiImport.fd_pwrite.bind(wasi.wasiImport) + : null; - const mode = stat?.mode ?? (stat?.isDirectory ? 0o040755 : 0o100644); - const child = createWorkspaceNode(node, name, mode, 0, childPath); - updateNodeFromRemoteStat(child, stat); +function decodeSignalMask(maskLo, maskHi) { + const values = []; + const lo = Number(maskLo) >>> 0; + const hi = Number(maskHi) >>> 0; + for (let bit = 0; bit < 32; bit += 1) { + if (((lo >>> bit) & 1) === 1) { + values.push(bit + 1); } } - - function loadFileContents(node) { - if (node.agentOsDirty) { - return; + for (let bit = 0; bit < 32; bit += 1) { + if (((hi >>> bit) & 1) === 1) { + values.push(bit + 33); } + } + return values; +} + +function parseControlPipeFd(value) { + if (typeof value !== 'string' || value.trim() === '') { + return null; + } - const stat = withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); - updateNodeFromRemoteStat(node, stat); - const contentBase64 = withFsErrors(() => bridge.fsReadSync(nodeGuestPath(node))); - const bytes = Uint8Array.from(Buffer.from(contentBase64, 'base64')); - node.contents = bytes; - node.usedBytes = bytes.length; - node.agentOsLoaded = true; - node.agentOsRemoteSize = bytes.length; + const parsed = Number.parseInt(value, 10); + return Number.isInteger(parsed) && parsed >= 0 ? parsed : null; +} + +function emitControlMessage(message) { + if (CONTROL_PIPE_FD == null) { + return; } - function persistFile(node) { - const contents = node.contents ? MEMFS.getFileDataAsTypedArray(node) : new Uint8Array(0); - withFsErrors(() => bridge.fsWriteSync(nodeGuestPath(node), contents)); - node.agentOsDirty = false; - node.agentOsLoaded = true; - node.agentOsRemoteSize = contents.length; - node.timestamp = Date.now(); + try { + writeSync(CONTROL_PIPE_FD, `${JSON.stringify(message)}\n`); + } catch { + // Ignore control-channel write failures during teardown. } +} - function makeStat(node, stat) { - const mode = stat?.mode ?? node.mode; - const size = FS.isDir(mode) ? 4096 : (node.agentOsDirty ? node.usedBytes : (stat?.size ?? node.usedBytes ?? 0)); - const timestamp = new Date(node.timestamp || Date.now()); +function isWorkspaceReadOnly() { + return permissionTier === 'read-only' || permissionTier === 'isolated'; +} - return { - dev: 1, - ino: node.id, - mode, - nlink: FS.isDir(mode) ? 2 : 1, - uid: 0, - gid: 0, - rdev: 0, - size, - atime: timestamp, - mtime: timestamp, - ctime: timestamp, - blksize: 4096, - blocks: Math.max(1, Math.ceil(size / 4096)), - }; +function hasWriteRights(rights) { + try { + return (BigInt(rights) & WASI_RIGHT_FD_WRITE) !== 0n; + } catch { + return true; } +} - const workspaceFileNodeOps = { - getattr(node) { - const stat = node.agentOsDirty - ? null - : withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); - if (stat) { - updateNodeFromRemoteStat(node, stat); - } - return makeStat(node, stat); - }, - setattr(node, attr) { - memfsFileNodeOps.setattr(node, attr); - if (attr?.size != null) { - node.agentOsDirty = true; - node.agentOsLoaded = true; +function denyReadOnlyMutation() { + return WASI_ERRNO_ROFS; +} + +const hostProcessImport = + permissionTier === 'full' + ? { + proc_sigaction(signal, action, maskLo, maskHi, flags) { + try { + const registration = { + action: action === 0 ? 'default' : action === 1 ? 'ignore' : 'user', + mask: decodeSignalMask(maskLo, maskHi), + flags: Number(flags) >>> 0, + }; + emitControlMessage({ + type: 'signal_state', + signal: Number(signal) >>> 0, + registration, + }); + return WASI_ERRNO_SUCCESS; + } catch { + return WASI_ERRNO_FAULT; + } + }, } - }, - }; + : {}; - const workspaceFileStreamOps = { - llseek(stream, offset, whence) { - return memfsFileStreamOps.llseek(stream, offset, whence); - }, - read(stream, buffer, offset, length, position) { - if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { - loadFileContents(stream.node); - } - return memfsFileStreamOps.read(stream, buffer, offset, length, position); - }, - write(stream, buffer, offset, length, position, canOwn) { - if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { - loadFileContents(stream.node); - } - const written = memfsFileStreamOps.write(stream, buffer, offset, length, position, canOwn); - stream.node.agentOsDirty = true; - persistFile(stream.node); - return written; - }, - mmap(stream, length, position, prot, flags) { - if (!stream.node.agentOsLoaded && !stream.node.agentOsDirty) { - loadFileContents(stream.node); - } - return memfsFileStreamOps.mmap(stream, length, position, prot, flags); - }, - msync(stream, buffer, offset, length, mmapFlags) { - const result = memfsFileStreamOps.msync(stream, buffer, offset, length, mmapFlags); - stream.node.agentOsDirty = true; - persistFile(stream.node); - return result; - }, - }; +wasiImport.clock_time_get = (clockId, precision, resultPtr) => { + if (!(instanceMemory instanceof WebAssembly.Memory)) { + return delegateClockTimeGet + ? delegateClockTimeGet(clockId, precision, resultPtr) + : WASI_ERRNO_FAULT; + } - const workspaceDirNodeOps = { - getattr(node) { - const stat = withFsErrors(() => bridge.fsStatSync(nodeGuestPath(node))); - updateNodeFromRemoteStat(node, stat); - return makeStat(node, stat); - }, - setattr(node, attr) { - memfsDirNodeOps.setattr(node, attr); - }, - lookup(parent, name) { - syncDirectory(parent); - try { - return memfsDirNodeOps.lookup(parent, name); - } catch (error) { - if (!(error instanceof FS.ErrnoError) || error.errno !== ERRNO_CODES.ENOENT) { - throw error; - } + try { + const view = new DataView(instanceMemory.buffer); + view.setBigUint64(Number(resultPtr), frozenTimeNs, true); + return WASI_ERRNO_SUCCESS; + } catch { + return WASI_ERRNO_FAULT; + } +}; - const guestPath = joinGuestPath(nodeGuestPath(parent), name); - const stat = withFsErrors(() => bridge.fsStatSync(guestPath)); - const child = createWorkspaceNode(parent, name, stat.mode, 0, guestPath); - updateNodeFromRemoteStat(child, stat); - return child; - } - }, - mknod(parent, name, mode, dev) { - const guestPath = joinGuestPath(nodeGuestPath(parent), name); - const node = createWorkspaceNode(parent, name, mode, dev, guestPath); - if (FS.isDir(mode)) { - withFsErrors(() => bridge.fsMkdirSync(guestPath, { recursive: false })); - } else if (FS.isFile(mode)) { - node.contents = new Uint8Array(0); - node.usedBytes = 0; - node.agentOsDirty = true; - persistFile(node); - } - return node; - }, - rename() { - throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); - }, - unlink() { - throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); - }, - rmdir() { - throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); - }, - readdir(node) { - syncDirectory(node); - return memfsDirNodeOps.readdir(node); - }, - symlink() { - throw new FS.ErrnoError(ERRNO_CODES.ENOSYS); - }, - }; +wasiImport.clock_res_get = (clockId, resultPtr) => { + if (!(instanceMemory instanceof WebAssembly.Memory)) { + return delegateClockResGet + ? delegateClockResGet(clockId, resultPtr) + : WASI_ERRNO_FAULT; + } try { - FS.mkdir('/workspace'); - } catch (error) { - if (!(error instanceof FS.ErrnoError) || error.errno !== ERRNO_CODES.EEXIST) { - throw error; - } + const view = new DataView(instanceMemory.buffer); + view.setBigUint64(Number(resultPtr), 1000000n, true); + return WASI_ERRNO_SUCCESS; + } catch { + return WASI_ERRNO_FAULT; } +}; - FS.mount( - { - mount(mount) { - const root = MEMFS.mount(mount); - root.agentOsGuestPath = mount.mountpoint; - root.agentOsDirty = false; - root.agentOsLoaded = true; - root.agentOsRemoteSize = 0; - root.node_ops = workspaceDirNodeOps; - root.stream_ops = workspaceDirStreamOps; - return root; - }, - }, - {}, - '/workspace', - ); -} +if (isWorkspaceReadOnly()) { + wasiImport.path_open = ( + fd, + dirflags, + pathPtr, + pathLen, + oflags, + rightsBase, + rightsInheriting, + fdflags, + openedFdPtr, + ) => { + if (Number(oflags) !== 0 || hasWriteRights(rightsBase) || hasWriteRights(rightsInheriting)) { + return denyReadOnlyMutation(); + } -async function readLockFileContents(indexURL) { - const lockFileUrl = new URL('pyodide-lock.json', indexURL); - return readFile(lockFileUrl, 'utf8'); -} + return delegatePathOpen + ? delegatePathOpen( + fd, + dirflags, + pathPtr, + pathLen, + oflags, + rightsBase, + rightsInheriting, + fdflags, + openedFdPtr, + ) + : WASI_ERRNO_FAULT; + }; -function installPythonStdin(pyodide) { - if (typeof pyodide?.setStdin !== 'function') { - return; - } + wasiImport.fd_write = (fd, iovs, iovsLen, nwrittenPtr) => { + if (Number(fd) > 2) { + return denyReadOnlyMutation(); + } - pyodide.setStdin({ - isatty: false, - read(buffer) { - return readSync(STDIN_FD, buffer, 0, buffer.length, null); - }, - }); -} + return delegateFdWrite ? delegateFdWrite(fd, iovs, iovsLen, nwrittenPtr) : WASI_ERRNO_FAULT; + }; -function resolvePythonSource(pyodide) { - const filePath = process.env[PYTHON_FILE_ENV]; - if (filePath != null) { - if (typeof pyodide?.FS?.readFile !== 'function') { - throw new Error(`Pyodide FS.readFile() is required to execute ${filePath}`); + wasiImport.fd_pwrite = (fd, iovs, iovsLen, offset, nwrittenPtr) => { + if (Number(fd) > 2) { + return denyReadOnlyMutation(); } - return pyodide.FS.readFile(filePath, { encoding: 'utf8' }); - } + return delegateFdPwrite + ? delegateFdPwrite(fd, iovs, iovsLen, offset, nwrittenPtr) + : WASI_ERRNO_FAULT; + }; - return requiredEnv(PYTHON_CODE_ENV); + for (const name of [ + 'fd_allocate', + 'fd_filestat_set_size', + 'fd_filestat_set_times', + 'path_create_directory', + 'path_filestat_set_times', + 'path_link', + 'path_remove_directory', + 'path_rename', + 'path_symlink', + 'path_unlink_file', + ]) { + if (typeof wasiImport[name] === 'function') { + wasiImport[name] = () => denyReadOnlyMutation(); + } + } } -let pythonVfsRpcBridge = null; +const instance = await WebAssembly.instantiate(module, { + wasi_snapshot_preview1: wasiImport, + wasi_unstable: wasiImport, + host_process: hostProcessImport, +}); -try { - const startupStarted = realPerformance.now(); - const { indexPath, indexUrl } = resolveIndexLocation(requiredEnv(PYODIDE_INDEX_URL_ENV)); - const prewarmOnly = process.env[PYTHON_PREWARM_ONLY_ENV] === '1'; - const preloadPackages = parsePreloadPackages(process.env[PYTHON_PRELOAD_PACKAGES_ENV]); - const lockFileContents = await readLockFileContents(indexUrl); - const pyodideModuleUrl = new URL('pyodide.mjs', indexUrl).href; - const { loadPyodide } = await import(pyodideModuleUrl); - - if (typeof loadPyodide !== 'function') { - throw new Error(`pyodide.mjs at ${indexUrl} does not export loadPyodide()`); - } - - const loadPyodideStarted = realPerformance.now(); - const pyodide = await loadPyodide({ - indexURL: indexPath, - lockFileContents, - packageBaseUrl: indexPath, - stdout: (message) => writeStream(process.stdout, message), - stderr: (message) => writeStream(process.stderr, message), - }); - const loadPyodideMs = realPerformance.now() - loadPyodideStarted; - let packageLoadMs = 0; - - if (prewarmOnly) { - emitPythonStartupMetrics({ - prewarmOnly: true, - startupMs: realPerformance.now() - startupStarted, - loadPyodideMs, - packageLoadMs, - packageCount: 0, - source: 'prewarm', - }); - process.exitCode = 0; - } else { - installPythonStdin(pyodide); - pythonVfsRpcBridge = installPythonVfsRpcBridge(); - installPythonWorkspaceFs(pyodide, pythonVfsRpcBridge); - installPythonGuestHardening(); - if (preloadPackages.length > 0) { - const packageLoadStarted = realPerformance.now(); - await pyodide.loadPackage(preloadPackages); - packageLoadMs = realPerformance.now() - packageLoadStarted; - } - const source = process.env[PYTHON_FILE_ENV] != null ? 'file' : 'inline'; - emitPythonStartupMetrics({ - prewarmOnly: false, - startupMs: realPerformance.now() - startupStarted, - loadPyodideMs, - packageLoadMs, - packageCount: preloadPackages.length, - source, - }); - const code = resolvePythonSource(pyodide); - await pyodide.runPythonAsync(code); +if (instance.exports.memory instanceof WebAssembly.Memory) { + instanceMemory = instance.exports.memory; +} + +if (typeof instance.exports._start === 'function') { + const exitCode = wasi.start(instance); + if (typeof exitCode === 'number' && exitCode !== 0) { + process.exitCode = exitCode; } -} catch (error) { - writeStream(process.stderr, formatError(error)); - process.exitCode = 1; -} finally { - pythonVfsRpcBridge?.dispose(); - writeStream(process.stderr, `${PYTHON_EXIT_CONTROL_PREFIX}${process.exitCode ?? 0}`); +} else if (typeof instance.exports.run === 'function') { + const result = await instance.exports.run(); + if (typeof result !== 'undefined') { + console.log(String(result)); + } +} else { + throw new Error('WebAssembly module must export _start or run'); } -process.exit(process.exitCode ?? 0); "#; static NEXT_NODE_IMPORT_CACHE_ID: AtomicU64 = AtomicU64::new(1); @@ -3331,6 +8165,46 @@ const BUILTIN_ASSETS: &[BuiltinAsset] = &[ module_specifier: "node:child_process", init_counter_key: "__agentOsBuiltinChildProcessInitCount", }, + BuiltinAsset { + name: "net", + module_specifier: "node:net", + init_counter_key: "__agentOsBuiltinNetInitCount", + }, + BuiltinAsset { + name: "dgram", + module_specifier: "node:dgram", + init_counter_key: "__agentOsBuiltinDgramInitCount", + }, + BuiltinAsset { + name: "dns", + module_specifier: "node:dns", + init_counter_key: "__agentOsBuiltinDnsInitCount", + }, + BuiltinAsset { + name: "http", + module_specifier: "node:http", + init_counter_key: "__agentOsBuiltinHttpInitCount", + }, + BuiltinAsset { + name: "http2", + module_specifier: "node:http2", + init_counter_key: "__agentOsBuiltinHttp2InitCount", + }, + BuiltinAsset { + name: "https", + module_specifier: "node:https", + init_counter_key: "__agentOsBuiltinHttpsInitCount", + }, + BuiltinAsset { + name: "tls", + module_specifier: "node:tls", + init_counter_key: "__agentOsBuiltinTlsInitCount", + }, + BuiltinAsset { + name: "os", + module_specifier: "node:os", + init_counter_key: "__agentOsBuiltinOsInitCount", + }, ]; const DENIED_BUILTIN_ASSETS: &[DeniedBuiltinAsset] = &[ @@ -3338,13 +8212,17 @@ const DENIED_BUILTIN_ASSETS: &[DeniedBuiltinAsset] = &[ name: "child_process", module_specifier: "node:child_process", }, + DeniedBuiltinAsset { + name: "cluster", + module_specifier: "node:cluster", + }, DeniedBuiltinAsset { name: "dgram", module_specifier: "node:dgram", }, DeniedBuiltinAsset { - name: "dns", - module_specifier: "node:dns", + name: "diagnostics_channel", + module_specifier: "node:diagnostics_channel", }, DeniedBuiltinAsset { name: "http", @@ -3362,13 +8240,17 @@ const DENIED_BUILTIN_ASSETS: &[DeniedBuiltinAsset] = &[ name: "inspector", module_specifier: "node:inspector", }, + DeniedBuiltinAsset { + name: "module", + module_specifier: "node:module", + }, DeniedBuiltinAsset { name: "net", module_specifier: "node:net", }, DeniedBuiltinAsset { - name: "tls", - module_specifier: "node:tls", + name: "trace_events", + module_specifier: "node:trace_events", }, DeniedBuiltinAsset { name: "v8", @@ -3387,9 +8269,10 @@ const DENIED_BUILTIN_ASSETS: &[DeniedBuiltinAsset] = &[ const PATH_POLYFILL_ASSET_NAME: &str = "path"; const PATH_POLYFILL_INIT_COUNTER_KEY: &str = "__agentOsPolyfillPathInitCount"; -#[derive(Debug, Clone)] +#[derive(Debug)] pub(crate) struct NodeImportCache { root_dir: PathBuf, + cleanup: Arc, cache_path: PathBuf, loader_path: PathBuf, register_path: PathBuf, @@ -3403,16 +8286,100 @@ pub(crate) struct NodeImportCache { prewarm_marker_dir: PathBuf, } +#[derive(Debug)] +pub(crate) struct NodeImportCacheCleanup { + root_dir: PathBuf, +} + +#[derive(Debug, Clone)] +struct NodeImportCacheMaterialization { + root_dir: PathBuf, + loader_path: PathBuf, + register_path: PathBuf, + runner_path: PathBuf, + python_runner_path: PathBuf, + timing_bootstrap_path: PathBuf, + prewarm_path: PathBuf, + wasm_runner_path: PathBuf, + asset_root: PathBuf, + pyodide_dist_path: PathBuf, + prewarm_marker_dir: PathBuf, +} + impl Default for NodeImportCache { fn default() -> Self { + Self::new_in(env::temp_dir()) + } +} + +fn cleanup_stale_node_import_caches_once(base_dir: &Path) { + let cleaned_roots = CLEANED_NODE_IMPORT_CACHE_ROOTS.get_or_init(|| Mutex::new(BTreeSet::new())); + let should_cleanup = cleaned_roots + .lock() + .map(|mut roots| roots.insert(base_dir.to_path_buf())) + .unwrap_or(true); + + if should_cleanup { + cleanup_stale_node_import_caches(base_dir); + } +} + +fn cleanup_stale_node_import_caches(base_dir: &Path) { + let entries = match fs::read_dir(base_dir) { + Ok(entries) => entries, + Err(error) if error.kind() == io::ErrorKind::NotFound => return, + Err(error) => { + eprintln!( + "agent-os: failed to scan node import cache root {}: {error}", + base_dir.display() + ); + return; + } + }; + + for entry in entries.flatten() { + let file_type = match entry.file_type() { + Ok(file_type) => file_type, + Err(_) => continue, + }; + if !file_type.is_dir() { + continue; + } + + let name = entry.file_name(); + if !name + .to_str() + .is_some_and(|name| name.starts_with(NODE_IMPORT_CACHE_DIR_PREFIX)) + { + continue; + } + + let path = entry.path(); + if let Err(error) = fs::remove_dir_all(&path) { + if error.kind() != io::ErrorKind::NotFound { + eprintln!( + "agent-os: failed to clean up stale node import cache {}: {error}", + path.display() + ); + } + } + } +} + +impl NodeImportCache { + fn new_in(base_dir: PathBuf) -> Self { + cleanup_stale_node_import_caches_once(&base_dir); let cache_id = NEXT_NODE_IMPORT_CACHE_ID.fetch_add(1, Ordering::Relaxed); - let root_dir = env::temp_dir().join(format!( - "agent-os-node-import-cache-{}-{cache_id}", + let root_dir = base_dir.join(format!( + "{NODE_IMPORT_CACHE_DIR_PREFIX}-{}-{cache_id}", std::process::id() )); Self { root_dir: root_dir.clone(), + cleanup: Arc::new(NodeImportCacheCleanup { + root_dir: root_dir.clone(), + }), cache_path: root_dir.join("state.json"), loader_path: root_dir.join("loader.mjs"), register_path: root_dir.join("register.mjs"), @@ -3428,11 +8395,28 @@ impl Default for NodeImportCache { } } +impl Drop for NodeImportCacheCleanup { + fn drop(&mut self) { + if let Err(error) = fs::remove_dir_all(&self.root_dir) { + if error.kind() != io::ErrorKind::NotFound { + eprintln!( + "agent-os: failed to clean up node import cache {}: {error}", + self.root_dir.display() + ); + } + } + } +} + impl NodeImportCache { pub(crate) fn cache_path(&self) -> &Path { &self.cache_path } + pub(crate) fn cleanup_guard(&self) -> Arc { + Arc::clone(&self.cleanup) + } + pub(crate) fn loader_path(&self) -> &Path { &self.loader_path } @@ -3479,6 +8463,63 @@ impl NodeImportCache { } pub(crate) fn ensure_materialized(&self) -> Result<(), io::Error> { + self.ensure_materialized_with_timeout(DEFAULT_NODE_IMPORT_CACHE_MATERIALIZE_TIMEOUT) + } + + pub(crate) fn ensure_materialized_with_timeout( + &self, + timeout: Duration, + ) -> Result<(), io::Error> { + let materialization = NodeImportCacheMaterialization::from(self); + let (sender, receiver) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + let _ = sender.send(materialization.materialize()); + }); + + match receiver.recv_timeout(timeout) { + Ok(result) => result, + Err(std::sync::mpsc::RecvTimeoutError::Timeout) => Err(io::Error::new( + io::ErrorKind::TimedOut, + format!( + "timed out materializing node import cache after {} ms", + timeout.as_millis() + ), + )), + Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => Err(io::Error::other( + "node import cache materialization thread exited unexpectedly", + )), + } + } +} + +impl From<&NodeImportCache> for NodeImportCacheMaterialization { + fn from(cache: &NodeImportCache) -> Self { + Self { + root_dir: cache.root_dir.clone(), + loader_path: cache.loader_path.clone(), + register_path: cache.register_path.clone(), + runner_path: cache.runner_path.clone(), + python_runner_path: cache.python_runner_path.clone(), + timing_bootstrap_path: cache.timing_bootstrap_path.clone(), + prewarm_path: cache.prewarm_path.clone(), + wasm_runner_path: cache.wasm_runner_path.clone(), + asset_root: cache.asset_root.clone(), + pyodide_dist_path: cache.pyodide_dist_path.clone(), + prewarm_marker_dir: cache.prewarm_marker_dir.clone(), + } + } +} + +impl NodeImportCacheMaterialization { + fn materialize(self) -> Result<(), io::Error> { + #[cfg(test)] + { + let delay_ms = NODE_IMPORT_CACHE_TEST_MATERIALIZE_DELAY_MS.load(Ordering::Relaxed); + if delay_ms > 0 { + std::thread::sleep(Duration::from_millis(delay_ms)); + } + } + fs::create_dir_all(&self.root_dir)?; fs::create_dir_all(self.asset_root.join("builtins"))?; fs::create_dir_all(self.asset_root.join("denied"))?; @@ -3597,6 +8638,14 @@ fn render_builtin_asset_source(asset: &BuiltinAsset) -> String { "fs" => render_fs_builtin_asset_source(asset.init_counter_key), "fs-promises" => render_fs_promises_builtin_asset_source(asset.init_counter_key), "child-process" => render_child_process_builtin_asset_source(asset.init_counter_key), + "net" => render_net_builtin_asset_source(asset.init_counter_key), + "dgram" => render_dgram_builtin_asset_source(asset.init_counter_key), + "dns" => render_dns_builtin_asset_source(asset.init_counter_key), + "http" => render_http_builtin_asset_source(asset.init_counter_key), + "http2" => render_http2_builtin_asset_source(asset.init_counter_key), + "https" => render_https_builtin_asset_source(asset.init_counter_key), + "tls" => render_tls_builtin_asset_source(asset.init_counter_key), + "os" => render_os_builtin_asset_source(asset.init_counter_key), _ => { render_passthrough_builtin_asset_source(asset.module_specifier, asset.init_counter_key) } @@ -3625,12 +8674,12 @@ fn render_fs_builtin_asset_source(init_counter_key: &str) -> String { let init_counter_key = format!("{init_counter_key:?}"); format!( - "import fs from \"node:fs\";\n\ -import path from \"node:path\";\n\n\ -const GUEST_PATH_MAPPINGS = parseGuestPathMappings(process.env.AGENT_OS_GUEST_PATH_MAPPINGS);\n\ -const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ + "const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ globalThis[{init_counter_key}] = initCount;\n\ -const mod = wrapFsModule(fs);\n\n\ +const mod = globalThis.__agentOsBuiltinFs ?? globalThis.__agentOsGuestFs ?? process.getBuiltinModule?.(\"node:fs\");\n\ +if (!mod) {{\n\ + throw new Error('Agent OS guest fs polyfill was not initialized');\n\ +}}\n\n\ export const __agentOsInitCount = initCount;\n\ export default mod;\n\ export const Dir = mod.Dir;\n\ @@ -3711,628 +8760,331 @@ export const write = mod.write;\n\ export const writeFile = mod.writeFile;\n\ export const writeFileSync = mod.writeFileSync;\n\ export const writeSync = mod.writeSync;\n\ -export * from \"node:fs\";\n\n\ -function parseGuestPathMappings(value) {{\n\ - if (!value) {{\n\ - return [];\n\ - }}\n\n\ - try {{\n\ - const parsed = JSON.parse(value);\n\ - if (!Array.isArray(parsed)) {{\n\ - return [];\n\ - }}\n\n\ - return parsed\n\ - .map((entry) => {{\n\ - const guestPath =\n\ - entry && typeof entry.guestPath === \"string\"\n\ - ? path.posix.normalize(entry.guestPath)\n\ - : null;\n\ - const hostPath =\n\ - entry && typeof entry.hostPath === \"string\"\n\ - ? path.resolve(entry.hostPath)\n\ - : null;\n\ - return guestPath && hostPath ? {{ guestPath, hostPath }} : null;\n\ - }})\n\ - .filter(Boolean)\n\ - .sort((left, right) => right.guestPath.length - left.guestPath.length);\n\ - }} catch {{\n\ - return [];\n\ - }}\n\ -}}\n\n\ -function hostPathFromGuestPath(guestPath) {{\n\ - if (typeof guestPath !== \"string\") {{\n\ - return null;\n\ - }}\n\n\ - const normalized = path.posix.normalize(guestPath);\n\ - for (const mapping of GUEST_PATH_MAPPINGS) {{\n\ - if (mapping.guestPath === \"/\") {{\n\ - const suffix = normalized.replace(/^\\/+/, \"\");\n\ - return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath;\n\ - }}\n\n\ - if (\n\ - normalized !== mapping.guestPath &&\n\ - !normalized.startsWith(`${{mapping.guestPath}}/`)\n\ - ) {{\n\ - continue;\n\ - }}\n\n\ - const suffix =\n\ - normalized === mapping.guestPath\n\ - ? \"\"\n\ - : normalized.slice(mapping.guestPath.length + 1);\n\ - return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath;\n\ - }}\n\n\ - return null;\n\ -}}\n\n\ -function safeRealpath(targetPath) {{\n\ - try {{\n\ - return fs.realpathSync.native(targetPath);\n\ - }} catch {{\n\ - return null;\n\ - }}\n\ -}}\n\n\ -function isKnownHostPath(hostPath) {{\n\ - if (typeof hostPath !== \"string\") {{\n\ - return false;\n\ - }}\n\n\ - const normalized = path.resolve(hostPath);\n\ - const hasPrefix = (hostRoot) =>\n\ - !!hostRoot &&\n\ - (normalized === hostRoot || normalized.startsWith(`${{hostRoot}}${{path.sep}}`));\n\ - for (const mapping of GUEST_PATH_MAPPINGS) {{\n\ - for (const hostRoot of [path.resolve(mapping.hostPath), safeRealpath(mapping.hostPath)]) {{\n\ - if (hasPrefix(hostRoot)) {{\n\ - return true;\n\ - }}\n\ - }}\n\n\ - let current = path.dirname(mapping.hostPath);\n\ - while (true) {{\n\ - const candidate = path.join(current, \"node_modules\");\n\ - if (pathExists(candidate)) {{\n\ - for (const hostRoot of [path.resolve(candidate), safeRealpath(candidate)]) {{\n\ - if (hasPrefix(hostRoot)) {{\n\ - return true;\n\ - }}\n\ - }}\n\ - }}\n\n\ - const parent = path.dirname(current);\n\ - if (parent === current) {{\n\ - break;\n\ - }}\n\ - current = parent;\n\ - }}\n\n\ - }}\n\n\ - return false;\n\ -}}\n\n\ -function pathExists(targetPath) {{\n\ - try {{\n\ - return fs.existsSync(targetPath);\n\ - }} catch {{\n\ - return false;\n\ - }}\n\ -}}\n\n\ -function translateGuestPath(value, fromGuestDir = \"/\") {{\n\ - if (typeof value !== \"string\") {{\n\ - return value;\n\ - }}\n\n\ - if (value.startsWith(\"file:\")) {{\n\ - try {{\n\ - const pathname = new URL(value).pathname;\n\ - if (pathExists(pathname) && isKnownHostPath(pathname)) {{\n\ - return value;\n\ - }}\n\ - const hostPath = hostPathFromGuestPath(pathname);\n\ - return hostPath ?? value;\n\ - }} catch {{\n\ - return value;\n\ - }}\n\ - }}\n\n\ - if (value.startsWith(\"/\")) {{\n\ - if (pathExists(value) && isKnownHostPath(value)) {{\n\ - return value;\n\ - }}\n\ - return hostPathFromGuestPath(value) ?? value;\n\ - }}\n\n\ - if (value.startsWith(\"./\") || value.startsWith(\"../\")) {{\n\ - const guestPath = path.posix.normalize(path.posix.join(fromGuestDir, value));\n\ - return hostPathFromGuestPath(guestPath) ?? value;\n\ - }}\n\n\ - return value;\n\ -}}\n\n\ -function guestMappedChildNames(guestDir) {{\n\ - if (typeof guestDir !== \"string\") {{\n\ - return [];\n\ - }}\n\n\ - const normalized = path.posix.normalize(guestDir);\n\ - const prefix = normalized === \"/\" ? \"/\" : `${{normalized}}/`;\n\ - const children = new Set();\n\n\ - for (const mapping of GUEST_PATH_MAPPINGS) {{\n\ - if (!mapping.guestPath.startsWith(prefix)) {{\n\ - continue;\n\ - }}\n\ - const remainder = mapping.guestPath.slice(prefix.length);\n\ - const childName = remainder.split(\"/\")[0];\n\ - if (childName) {{\n\ - children.add(childName);\n\ - }}\n\ - }}\n\n\ - return [...children].sort();\n\ +export * from \"node:fs\";\n" + ) +} + +fn render_fs_promises_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "import fsModule from \"agent-os:builtin/fs\";\n\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +const mod = fsModule.promises;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const constants = fsModule.constants;\n\ +export const FileHandle = mod.FileHandle;\n\ +export const access = mod.access;\n\ +export const appendFile = mod.appendFile;\n\ +export const chmod = mod.chmod;\n\ +export const chown = mod.chown;\n\ +export const copyFile = mod.copyFile;\n\ +export const cp = mod.cp;\n\ +export const lchmod = mod.lchmod;\n\ +export const lchown = mod.lchown;\n\ +export const link = mod.link;\n\ +export const lstat = mod.lstat;\n\ +export const lutimes = mod.lutimes;\n\ +export const mkdir = mod.mkdir;\n\ +export const mkdtemp = mod.mkdtemp;\n\ +export const open = mod.open;\n\ +export const opendir = mod.opendir;\n\ +export const readFile = mod.readFile;\n\ +export const readdir = mod.readdir;\n\ +export const readlink = mod.readlink;\n\ +export const realpath = mod.realpath;\n\ +export const rename = mod.rename;\n\ +export const rm = mod.rm;\n\ +export const rmdir = mod.rmdir;\n\ +export const stat = mod.stat;\n\ +export const statfs = mod.statfs;\n\ +export const symlink = mod.symlink;\n\ +export const truncate = mod.truncate;\n\ +export const unlink = mod.unlink;\n\ +export const utimes = mod.utimes;\n\ +export const watch = mod.watch;\n\ +export const writeFile = mod.writeFile;\n\ +export * from \"node:fs/promises\";\n" + ) +} + +fn render_child_process_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinChildProcess) {{\n\ + const error = new Error(\"node:child_process is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function createSyntheticDirent(name) {{\n\ - return {{\n\ - name,\n\ - isBlockDevice: () => false,\n\ - isCharacterDevice: () => false,\n\ - isDirectory: () => true,\n\ - isFIFO: () => false,\n\ - isFile: () => false,\n\ - isSocket: () => false,\n\ - isSymbolicLink: () => false,\n\ - }};\n\ +const mod = globalThis.__agentOsBuiltinChildProcess;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const ChildProcess = mod.ChildProcess;\n\ +export const _forkChild = mod._forkChild;\n\ +export const exec = mod.exec;\n\ +export const execFile = mod.execFile;\n\ +export const execFileSync = mod.execFileSync;\n\ +export const execSync = mod.execSync;\n\ +export const fork = mod.fork;\n\ +export const spawn = mod.spawn;\n\ +export const spawnSync = mod.spawnSync;\n" + ) +} + +fn render_net_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinNet) {{\n\ + const error = new Error(\"node:net is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function wrapFsModule(fsModule, fromGuestDir = \"/\") {{\n\ - const wrapPathFirst = (methodName) => (...args) =>\n\ - fsModule[methodName](translateGuestPath(args[0], fromGuestDir), ...args.slice(1));\n\ - const wrapRenameLike = (methodName) => (...args) =>\n\ - fsModule[methodName](\n\ - translateGuestPath(args[0], fromGuestDir),\n\ - translateGuestPath(args[1], fromGuestDir),\n\ - ...args.slice(2),\n\ - );\n\n\ - const wrapped = {{\n\ - ...fsModule,\n\ - accessSync: wrapPathFirst(\"accessSync\"),\n\ - appendFileSync: wrapPathFirst(\"appendFileSync\"),\n\ - chmodSync: wrapPathFirst(\"chmodSync\"),\n\ - chownSync: wrapPathFirst(\"chownSync\"),\n\ - createReadStream: wrapPathFirst(\"createReadStream\"),\n\ - createWriteStream: wrapPathFirst(\"createWriteStream\"),\n\ - existsSync: (target) => {{\n\ - const translated = translateGuestPath(target, fromGuestDir);\n\ - return fsModule.existsSync(translated) || guestMappedChildNames(target).length > 0;\n\ - }},\n\ - lstatSync: wrapPathFirst(\"lstatSync\"),\n\ - mkdirSync: wrapPathFirst(\"mkdirSync\"),\n\ - openSync: wrapPathFirst(\"openSync\"),\n\ - readFileSync: wrapPathFirst(\"readFileSync\"),\n\ - readdirSync: (target, options) => {{\n\ - const translated = translateGuestPath(target, fromGuestDir);\n\ - if (fsModule.existsSync(translated)) {{\n\ - return fsModule.readdirSync(translated, options);\n\ - }}\n\n\ - const synthetic = guestMappedChildNames(target);\n\ - if (synthetic.length > 0) {{\n\ - return options && typeof options === \"object\" && options.withFileTypes\n\ - ? synthetic.map((name) => createSyntheticDirent(name))\n\ - : synthetic;\n\ - }}\n\n\ - return fsModule.readdirSync(translated, options);\n\ - }},\n\ - readlinkSync: wrapPathFirst(\"readlinkSync\"),\n\ - realpathSync: wrapPathFirst(\"realpathSync\"),\n\ - renameSync: wrapRenameLike(\"renameSync\"),\n\ - rmSync: wrapPathFirst(\"rmSync\"),\n\ - rmdirSync: wrapPathFirst(\"rmdirSync\"),\n\ - statSync: wrapPathFirst(\"statSync\"),\n\ - symlinkSync: wrapRenameLike(\"symlinkSync\"),\n\ - unlinkSync: wrapPathFirst(\"unlinkSync\"),\n\ - utimesSync: wrapPathFirst(\"utimesSync\"),\n\ - writeFileSync: wrapPathFirst(\"writeFileSync\"),\n\ - }};\n\n\ - if (fsModule.promises) {{\n\ - wrapped.promises = {{\n\ - ...fsModule.promises,\n\ - access: wrapPathFirstAsync(fsModule.promises.access, fromGuestDir),\n\ - appendFile: wrapPathFirstAsync(fsModule.promises.appendFile, fromGuestDir),\n\ - chmod: wrapPathFirstAsync(fsModule.promises.chmod, fromGuestDir),\n\ - chown: wrapPathFirstAsync(fsModule.promises.chown, fromGuestDir),\n\ - lstat: wrapPathFirstAsync(fsModule.promises.lstat, fromGuestDir),\n\ - mkdir: wrapPathFirstAsync(fsModule.promises.mkdir, fromGuestDir),\n\ - open: wrapPathFirstAsync(fsModule.promises.open, fromGuestDir),\n\ - readFile: wrapPathFirstAsync(fsModule.promises.readFile, fromGuestDir),\n\ - readdir: wrapPathFirstAsync(fsModule.promises.readdir, fromGuestDir),\n\ - readlink: wrapPathFirstAsync(fsModule.promises.readlink, fromGuestDir),\n\ - realpath: wrapPathFirstAsync(fsModule.promises.realpath, fromGuestDir),\n\ - rename: wrapRenameLikeAsync(fsModule.promises.rename, fromGuestDir),\n\ - rm: wrapPathFirstAsync(fsModule.promises.rm, fromGuestDir),\n\ - rmdir: wrapPathFirstAsync(fsModule.promises.rmdir, fromGuestDir),\n\ - stat: wrapPathFirstAsync(fsModule.promises.stat, fromGuestDir),\n\ - symlink: wrapRenameLikeAsync(fsModule.promises.symlink, fromGuestDir),\n\ - unlink: wrapPathFirstAsync(fsModule.promises.unlink, fromGuestDir),\n\ - utimes: wrapPathFirstAsync(fsModule.promises.utimes, fromGuestDir),\n\ - writeFile: wrapPathFirstAsync(fsModule.promises.writeFile, fromGuestDir),\n\ - }};\n\ - }}\n\n\ - return wrapped;\n\ +const mod = globalThis.__agentOsBuiltinNet;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const BlockList = mod.BlockList;\n\ +export const Server = mod.Server;\n\ +export const Socket = mod.Socket;\n\ +export const SocketAddress = mod.SocketAddress;\n\ +export const Stream = mod.Stream;\n\ +export const connect = mod.connect;\n\ +export const createConnection = mod.createConnection;\n\ +export const createServer = mod.createServer;\n\ +export const getDefaultAutoSelectFamily = mod.getDefaultAutoSelectFamily;\n\ +export const getDefaultAutoSelectFamilyAttemptTimeout = mod.getDefaultAutoSelectFamilyAttemptTimeout;\n\ +export const isIP = mod.isIP;\n\ +export const isIPv4 = mod.isIPv4;\n\ +export const isIPv6 = mod.isIPv6;\n\ +export const setDefaultAutoSelectFamily = mod.setDefaultAutoSelectFamily;\n\ +export const setDefaultAutoSelectFamilyAttemptTimeout = mod.setDefaultAutoSelectFamilyAttemptTimeout;\n" + ) +} + +fn render_dgram_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinDgram) {{\n\ + const error = new Error(\"node:dgram is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function wrapPathFirstAsync(fn, fromGuestDir) {{\n\ - return (...args) =>\n\ - fn(translateGuestPath(args[0], fromGuestDir), ...args.slice(1));\n\ +const mod = globalThis.__agentOsBuiltinDgram;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const Socket = mod.Socket;\n\ +export const createSocket = mod.createSocket;\n" + ) +} + +fn render_dns_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinDns) {{\n\ + const error = new Error(\"node:dns is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function wrapRenameLikeAsync(fn, fromGuestDir) {{\n\ - return (...args) =>\n\ - fn(\n\ - translateGuestPath(args[0], fromGuestDir),\n\ - translateGuestPath(args[1], fromGuestDir),\n\ - ...args.slice(2),\n\ - );\n\ -}}\n" +const mod = globalThis.__agentOsBuiltinDns;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const ADDRCONFIG = mod.ADDRCONFIG;\n\ +export const ALL = mod.ALL;\n\ +export const Resolver = mod.Resolver;\n\ +export const V4MAPPED = mod.V4MAPPED;\n\ +export const constants = mod.constants;\n\ +export const getDefaultResultOrder = mod.getDefaultResultOrder;\n\ +export const getServers = mod.getServers;\n\ +export const lookup = mod.lookup;\n\ +export const lookupService = mod.lookupService;\n\ +export const promises = mod.promises;\n\ +export const resolve = mod.resolve;\n\ +export const resolve4 = mod.resolve4;\n\ +export const resolve6 = mod.resolve6;\n\ +export const reverse = mod.reverse;\n\ +export const setDefaultResultOrder = mod.setDefaultResultOrder;\n\ +export const setServers = mod.setServers;\n" ) } -fn render_fs_promises_builtin_asset_source(init_counter_key: &str) -> String { +fn render_http_builtin_asset_source(init_counter_key: &str) -> String { let init_counter_key = format!("{init_counter_key:?}"); format!( - "import fsModule from \"agent-os:builtin/fs\";\n\n\ + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ globalThis[{init_counter_key}] = initCount;\n\ -const mod = fsModule.promises;\n\n\ +if (!globalThis.__agentOsBuiltinHttp) {{\n\ + const error = new Error(\"node:http is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ +}}\n\n\ +const mod = globalThis.__agentOsBuiltinHttp;\n\n\ export const __agentOsInitCount = initCount;\n\ export default mod;\n\ -export const constants = fsModule.constants;\n\ -export const FileHandle = mod.FileHandle;\n\ -export const access = mod.access;\n\ -export const appendFile = mod.appendFile;\n\ -export const chmod = mod.chmod;\n\ -export const chown = mod.chown;\n\ -export const copyFile = mod.copyFile;\n\ -export const cp = mod.cp;\n\ -export const lchmod = mod.lchmod;\n\ -export const lchown = mod.lchown;\n\ -export const link = mod.link;\n\ -export const lstat = mod.lstat;\n\ -export const lutimes = mod.lutimes;\n\ -export const mkdir = mod.mkdir;\n\ -export const mkdtemp = mod.mkdtemp;\n\ -export const open = mod.open;\n\ -export const opendir = mod.opendir;\n\ -export const readFile = mod.readFile;\n\ -export const readdir = mod.readdir;\n\ -export const readlink = mod.readlink;\n\ -export const realpath = mod.realpath;\n\ -export const rename = mod.rename;\n\ -export const rm = mod.rm;\n\ -export const rmdir = mod.rmdir;\n\ -export const stat = mod.stat;\n\ -export const statfs = mod.statfs;\n\ -export const symlink = mod.symlink;\n\ -export const truncate = mod.truncate;\n\ -export const unlink = mod.unlink;\n\ -export const utimes = mod.utimes;\n\ -export const watch = mod.watch;\n\ -export const writeFile = mod.writeFile;\n\ -export * from \"node:fs/promises\";\n" +export const Agent = mod.Agent;\n\ +export const ClientRequest = mod.ClientRequest;\n\ +export const IncomingMessage = mod.IncomingMessage;\n\ +export const METHODS = mod.METHODS;\n\ +export const OutgoingMessage = mod.OutgoingMessage;\n\ +export const STATUS_CODES = mod.STATUS_CODES;\n\ +export const Server = mod.Server;\n\ +export const ServerResponse = mod.ServerResponse;\n\ +export const createServer = mod.createServer;\n\ +export const get = mod.get;\n\ +export const globalAgent = mod.globalAgent;\n\ +export const maxHeaderSize = mod.maxHeaderSize;\n\ +export const request = mod.request;\n\ +export const setMaxIdleHTTPParsers = mod.setMaxIdleHTTPParsers;\n\ +export const validateHeaderName = mod.validateHeaderName;\n\ +export const validateHeaderValue = mod.validateHeaderValue;\n" ) } -fn render_child_process_builtin_asset_source(init_counter_key: &str) -> String { +fn render_http2_builtin_asset_source(init_counter_key: &str) -> String { let init_counter_key = format!("{init_counter_key:?}"); format!( - "import childProcess from \"node:child_process\";\n\ -import path from \"node:path\";\n\n\ -const GUEST_PATH_MAPPINGS = parseGuestPathMappings(process.env.AGENT_OS_GUEST_PATH_MAPPINGS);\n\ -const ALLOWED_BUILTINS = new Set(parseJsonArray(process.env.AGENT_OS_ALLOWED_NODE_BUILTINS));\n\ + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ globalThis[{init_counter_key}] = initCount;\n\ -if (!ALLOWED_BUILTINS.has(\"child_process\")) {{\n\ - const error = new Error(\"node:child_process is not available in the Agent OS guest runtime\");\n\ - error.code = \"ERR_ACCESS_DENIED\";\n\ +if (!globalThis.__agentOsBuiltinHttp2) {{\n\ + const error = new Error(\"node:http2 is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ throw error;\n\ }}\n\n\ -const mod = wrapChildProcessModule(childProcess);\n\n\ +const mod = globalThis.__agentOsBuiltinHttp2;\n\n\ export const __agentOsInitCount = initCount;\n\ export default mod;\n\ -export const ChildProcess = mod.ChildProcess;\n\ -export const _forkChild = mod._forkChild;\n\ -export const exec = mod.exec;\n\ -export const execFile = mod.execFile;\n\ -export const execFileSync = mod.execFileSync;\n\ -export const execSync = mod.execSync;\n\ -export const fork = mod.fork;\n\ -export const spawn = mod.spawn;\n\ -export const spawnSync = mod.spawnSync;\n\n\ -function parseJsonArray(value) {{\n\ - if (!value) {{\n\ - return [];\n\ - }}\n\n\ - try {{\n\ - const parsed = JSON.parse(value);\n\ - return Array.isArray(parsed) ? parsed.filter((entry) => typeof entry === \"string\") : [];\n\ - }} catch {{\n\ - return [];\n\ - }}\n\ -}}\n\n\ -function parseGuestPathMappings(value) {{\n\ - if (!value) {{\n\ - return [];\n\ - }}\n\n\ - try {{\n\ - const parsed = JSON.parse(value);\n\ - if (!Array.isArray(parsed)) {{\n\ - return [];\n\ - }}\n\n\ - return parsed\n\ - .map((entry) => {{\n\ - const guestPath =\n\ - entry && typeof entry.guestPath === \"string\"\n\ - ? path.posix.normalize(entry.guestPath)\n\ - : null;\n\ - const hostPath =\n\ - entry && typeof entry.hostPath === \"string\"\n\ - ? path.resolve(entry.hostPath)\n\ - : null;\n\ - return guestPath && hostPath ? {{ guestPath, hostPath }} : null;\n\ - }})\n\ - .filter(Boolean)\n\ - .sort((left, right) => right.guestPath.length - left.guestPath.length);\n\ - }} catch {{\n\ - return [];\n\ - }}\n\ +export const Http2ServerRequest = mod.Http2ServerRequest;\n\ +export const Http2ServerResponse = mod.Http2ServerResponse;\n\ +export const Http2Session = mod.Http2Session;\n\ +export const Http2Stream = mod.Http2Stream;\n\ +export const constants = mod.constants;\n\ +export const connect = mod.connect;\n\ +export const createServer = mod.createServer;\n\ +export const createSecureServer = mod.createSecureServer;\n\ +export const getDefaultSettings = mod.getDefaultSettings;\n\ +export const getPackedSettings = mod.getPackedSettings;\n\ +export const getUnpackedSettings = mod.getUnpackedSettings;\n\ +export const sensitiveHeaders = mod.sensitiveHeaders;\n" + ) +} + +fn render_https_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinHttps) {{\n\ + const error = new Error(\"node:https is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function hostPathFromGuestPath(guestPath) {{\n\ - if (typeof guestPath !== \"string\") {{\n\ - return null;\n\ - }}\n\n\ - const normalized = path.posix.normalize(guestPath);\n\ - for (const mapping of GUEST_PATH_MAPPINGS) {{\n\ - if (mapping.guestPath === \"/\") {{\n\ - const suffix = normalized.replace(/^\\/+/, \"\");\n\ - return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath;\n\ - }}\n\n\ - if (\n\ - normalized !== mapping.guestPath &&\n\ - !normalized.startsWith(`${{mapping.guestPath}}/`)\n\ - ) {{\n\ - continue;\n\ - }}\n\n\ - const suffix =\n\ - normalized === mapping.guestPath\n\ - ? \"\"\n\ - : normalized.slice(mapping.guestPath.length + 1);\n\ - return suffix ? path.join(mapping.hostPath, suffix) : mapping.hostPath;\n\ - }}\n\n\ - return null;\n\ +const mod = globalThis.__agentOsBuiltinHttps;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const Agent = mod.Agent;\n\ +export const Server = mod.Server;\n\ +export const createServer = mod.createServer;\n\ +export const get = mod.get;\n\ +export const globalAgent = mod.globalAgent;\n\ +export const request = mod.request;\n" + ) +} + +fn render_tls_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinTls) {{\n\ + const error = new Error(\"node:tls is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function translateGuestPath(value, fromGuestDir = \"/\") {{\n\ - if (typeof value !== \"string\") {{\n\ - return value;\n\ - }}\n\n\ - if (value.startsWith(\"file:\")) {{\n\ - try {{\n\ - const hostPath = hostPathFromGuestPath(new URL(value).pathname);\n\ - return hostPath ?? value;\n\ - }} catch {{\n\ - return value;\n\ - }}\n\ - }}\n\n\ - if (value.startsWith(\"/\")) {{\n\ - return hostPathFromGuestPath(value) ?? value;\n\ - }}\n\n\ - if (value.startsWith(\"./\") || value.startsWith(\"../\")) {{\n\ - const guestPath = path.posix.normalize(path.posix.join(fromGuestDir, value));\n\ - return hostPathFromGuestPath(guestPath) ?? value;\n\ - }}\n\n\ - return value;\n\ +const mod = globalThis.__agentOsBuiltinTls;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const CLIENT_RENEG_LIMIT = mod.CLIENT_RENEG_LIMIT;\n\ +export const CLIENT_RENEG_WINDOW = mod.CLIENT_RENEG_WINDOW;\n\ +export const DEFAULT_CIPHERS = mod.DEFAULT_CIPHERS;\n\ +export const DEFAULT_ECDH_CURVE = mod.DEFAULT_ECDH_CURVE;\n\ +export const DEFAULT_MAX_VERSION = mod.DEFAULT_MAX_VERSION;\n\ +export const DEFAULT_MIN_VERSION = mod.DEFAULT_MIN_VERSION;\n\ +export const SecureContext = mod.SecureContext;\n\ +export const Server = mod.Server;\n\ +export const TLSSocket = mod.TLSSocket;\n\ +export const checkServerIdentity = mod.checkServerIdentity;\n\ +export const connect = mod.connect;\n\ +export const createConnection = mod.createConnection;\n\ +export const createSecureContext = mod.createSecureContext;\n\ +export const createSecurePair = mod.createSecurePair;\n\ +export const createServer = mod.createServer;\n\ +export const getCiphers = mod.getCiphers;\n\ +export const rootCertificates = mod.rootCertificates;\n" + ) +} + +fn render_os_builtin_asset_source(init_counter_key: &str) -> String { + let init_counter_key = format!("{init_counter_key:?}"); + + format!( + "const ACCESS_DENIED_CODE = \"ERR_ACCESS_DENIED\";\n\ +const initCount = (globalThis[{init_counter_key}] ?? 0) + 1;\n\ +globalThis[{init_counter_key}] = initCount;\n\ +if (!globalThis.__agentOsBuiltinOs) {{\n\ + const error = new Error(\"node:os is not available in the Agent OS guest runtime\");\n\ + error.code = ACCESS_DENIED_CODE;\n\ + throw error;\n\ }}\n\n\ -function wrapChildProcessModule(childProcessModule, fromGuestDir = \"/\") {{\n\ - const isNodeCommand = (command) =>\n\ - command === \"node\" || String(command).endsWith(\"/node\");\n\ - const isNodeScriptCommand = (command) =>\n\ - typeof command === \"string\" &&\n\ - (command.startsWith(\"./\") ||\n\ - command.startsWith(\"../\") ||\n\ - command.startsWith(\"/\") ||\n\ - command.startsWith(\"file:\")) &&\n\ - /\\.(?:[cm]?js)$/i.test(command);\n\ - const usesNodeRuntime = (command) =>\n\ - isNodeCommand(command) || isNodeScriptCommand(command);\n\ - const translateCommand = (command) =>\n\ - usesNodeRuntime(command)\n\ - ? process.execPath\n\ - : translateGuestPath(command, fromGuestDir);\n\ - const isGuestCommandPath = (command) =>\n\ - typeof command === \"string\" &&\n\ - (command.startsWith(\"/\") || command.startsWith(\"file:\"));\n\ - const ensureRuntimeEnv = (env) => {{\n\ - const sourceEnv =\n\ - env && typeof env === \"object\" ? env : process.env;\n\ - const {{ NODE_OPTIONS: _nodeOptions, ...safeEnv }} = sourceEnv;\n\ - for (const key of [\"HOME\", \"PWD\", \"TMPDIR\", \"TEMP\", \"TMP\", \"PI_CODING_AGENT_DIR\"]) {{\n\ - if (typeof safeEnv[key] === \"string\") {{\n\ - safeEnv[key] = translateGuestPath(safeEnv[key], fromGuestDir);\n\ - }}\n\ - }}\n\ - const nodeDir = path.dirname(process.execPath);\n\ - const existingPath =\n\ - typeof safeEnv.PATH === \"string\"\n\ - ? safeEnv.PATH\n\ - : typeof process.env.PATH === \"string\"\n\ - ? process.env.PATH\n\ - : \"\";\n\ - const segments = existingPath\n\ - .split(path.delimiter)\n\ - .filter(Boolean);\n\n\ - if (!segments.includes(nodeDir)) {{\n\ - segments.unshift(nodeDir);\n\ - }}\n\n\ - return {{\n\ - ...safeEnv,\n\ - PATH: segments.join(path.delimiter),\n\ - }};\n\ - }};\n\ - const translateProcessOptions = (options) => {{\n\ - if (options == null) {{\n\ - return {{\n\ - env: ensureRuntimeEnv(process.env),\n\ - }};\n\ - }}\n\n\ - if (typeof options !== \"object\") {{\n\ - return options;\n\ - }}\n\n\ - return {{\n\ - ...options,\n\ - cwd:\n\ - typeof options.cwd === \"string\"\n\ - ? translateGuestPath(options.cwd, fromGuestDir)\n\ - : options.cwd,\n\ - env: ensureRuntimeEnv(options.env),\n\ - }};\n\ - }};\n\ - const translateArgs = (command, args) => {{\n\ - if (isNodeScriptCommand(command)) {{\n\ - const translatedScript = translateGuestPath(command, fromGuestDir);\n\ - const translatedArgs = Array.isArray(args)\n\ - ? args.map((arg) => translateGuestPath(arg, fromGuestDir))\n\ - : [];\n\ - return [translatedScript, ...translatedArgs];\n\ - }}\n\n\ - if (!Array.isArray(args)) {{\n\ - return args;\n\ - }}\n\ - if (!isNodeCommand(command)) {{\n\ - return args.map((arg) => translateGuestPath(arg, fromGuestDir));\n\ - }}\n\ - return args.map((arg, index) =>\n\ - index === 0 ? translateGuestPath(arg, fromGuestDir) : arg,\n\ - );\n\ - }};\n\n\ - const prependNodePermissionArgs = (command, args, options) => {{\n\ - if (!usesNodeRuntime(command)) {{\n\ - return args;\n\ - }}\n\n\ - const translatedArgs = Array.isArray(args) ? args : [];\n\ - const readPaths = new Set();\n\ - const writePaths = new Set();\n\ - const addReadPathChain = (value) => {{\n\ - if (typeof value !== \"string\" || value.length === 0) {{\n\ - return;\n\ - }}\n\ - let current = value;\n\ - while (true) {{\n\ - readPaths.add(current);\n\ - const parent = path.dirname(current);\n\ - if (parent === current) {{\n\ - break;\n\ - }}\n\ - current = parent;\n\ - }}\n\ - }};\n\ - const addWritePath = (value) => {{\n\ - if (typeof value !== \"string\" || value.length === 0) {{\n\ - return;\n\ - }}\n\ - writePaths.add(value);\n\ - }};\n\n\ - if (typeof options?.cwd === \"string\") {{\n\ - addReadPathChain(options.cwd);\n\ - addWritePath(options.cwd);\n\ - }}\n\n\ - const homePath =\n\ - typeof options?.env?.HOME === \"string\"\n\ - ? translateGuestPath(options.env.HOME, fromGuestDir)\n\ - : typeof process.env.HOME === \"string\"\n\ - ? translateGuestPath(process.env.HOME, fromGuestDir)\n\ - : null;\n\ - if (homePath) {{\n\ - addReadPathChain(homePath);\n\ - addWritePath(homePath);\n\ - }}\n\n\ - if (translatedArgs.length > 0 && typeof translatedArgs[0] === \"string\") {{\n\ - addReadPathChain(translatedArgs[0]);\n\ - }}\n\n\ - const permissionArgs = [\n\ - \"--allow-child-process\",\n\ - \"--allow-worker\",\n\ - \"--disable-warning=SecurityWarning\",\n\ - ];\n\n\ - for (const allowedPath of readPaths) {{\n\ - permissionArgs.push(`--allow-fs-read=${{allowedPath}}`);\n\ - }}\n\ - for (const allowedPath of writePaths) {{\n\ - permissionArgs.push(`--allow-fs-write=${{allowedPath}}`);\n\ - }}\n\n\ - return [...permissionArgs, ...translatedArgs];\n\ - }};\n\n\ - return {{\n\ - ...childProcessModule,\n\ - exec: childProcessModule.exec.bind(childProcessModule),\n\ - execFile: (file, args, options, callback) => {{\n\ - const translatedOptions = translateProcessOptions(options);\n\ - return childProcessModule.execFile(\n\ - translateCommand(file),\n\ - prependNodePermissionArgs(\n\ - file,\n\ - translateArgs(file, args),\n\ - translatedOptions,\n\ - ),\n\ - translatedOptions,\n\ - callback,\n\ - );\n\ - }},\n\ - execFileSync: (file, args, options) => {{\n\ - const translatedOptions = translateProcessOptions(options);\n\ - return childProcessModule.execFileSync(\n\ - translateCommand(file),\n\ - prependNodePermissionArgs(\n\ - file,\n\ - translateArgs(file, args),\n\ - translatedOptions,\n\ - ),\n\ - translatedOptions,\n\ - );\n\ - }},\n\ - execSync: childProcessModule.execSync.bind(childProcessModule),\n\ - fork: (modulePath, args, options) => {{\n\ - const translatedOptions = translateProcessOptions(options);\n\ - return childProcessModule.fork(\n\ - translateGuestPath(modulePath, fromGuestDir),\n\ - prependNodePermissionArgs(\n\ - \"node\",\n\ - translateArgs(\"node\", args),\n\ - translatedOptions,\n\ - ),\n\ - translatedOptions,\n\ - );\n\ - }},\n\ - spawn: (command, args, options) => {{\n\ - const translatedOptions = translateProcessOptions(options);\n\ - return childProcessModule.spawn(\n\ - translateCommand(command),\n\ - prependNodePermissionArgs(\n\ - command,\n\ - translateArgs(command, args),\n\ - translatedOptions,\n\ - ),\n\ - translatedOptions,\n\ - );\n\ - }},\n\ - spawnSync: (command, args, options) => {{\n\ - const translatedOptions = translateProcessOptions(options);\n\ - const result = childProcessModule.spawnSync(\n\ - translateCommand(command),\n\ - prependNodePermissionArgs(\n\ - command,\n\ - translateArgs(command, args),\n\ - translatedOptions,\n\ - ),\n\ - translatedOptions,\n\ - );\n\ - if (\n\ - isGuestCommandPath(command) &&\n\ - result?.status == null &&\n\ - (result.error?.code === \"ENOENT\" || result.error?.code === \"EACCES\")\n\ - ) {{\n\ - return {{\n\ - ...result,\n\ - status: 1,\n\ - stderr: Buffer.from(result.error.message),\n\ - }};\n\ - }}\n\ - return result;\n\ - }},\n\ - }};\n\ -}}\n" +const mod = globalThis.__agentOsBuiltinOs;\n\n\ +export const __agentOsInitCount = initCount;\n\ +export default mod;\n\ +export const EOL = mod.EOL;\n\ +export const arch = mod.arch;\n\ +export const availableParallelism = mod.availableParallelism;\n\ +export const constants = mod.constants;\n\ +export const cpus = mod.cpus;\n\ +export const devNull = mod.devNull;\n\ +export const endianness = mod.endianness;\n\ +export const freemem = mod.freemem;\n\ +export const getPriority = mod.getPriority;\n\ +export const homedir = mod.homedir;\n\ +export const hostname = mod.hostname;\n\ +export const loadavg = mod.loadavg;\n\ +export const machine = mod.machine;\n\ +export const networkInterfaces = mod.networkInterfaces;\n\ +export const platform = mod.platform;\n\ +export const release = mod.release;\n\ +export const setPriority = mod.setPriority;\n\ +export const tmpdir = mod.tmpdir;\n\ +export const totalmem = mod.totalmem;\n\ +export const type = mod.type;\n\ +export const uptime = mod.uptime;\n\ +export const userInfo = mod.userInfo;\n\ +export const version = mod.version;\n" ) } @@ -4375,12 +9127,16 @@ fn write_file_if_changed(path: &Path, contents: &str) -> Result<(), io::Error> { #[cfg(test)] mod tests { - use super::NodeImportCache; + use super::{NodeImportCache, NODE_IMPORT_CACHE_TEST_MATERIALIZE_DELAY_MS}; use crate::node_process::node_binary; + use serde_json::Value; + use std::collections::BTreeSet; use std::fs; use std::io::Write; use std::path::Path; use std::process::{Command, Output, Stdio}; + use std::sync::atomic::Ordering; + use std::time::Duration; use tempfile::tempdir; fn assert_node_available() { @@ -4411,6 +9167,8 @@ mod tests { ) -> Output { let mut command = Command::new(node_binary()); command + .arg("--import") + .arg(import_cache.timing_bootstrap_path()) .arg(import_cache.python_runner_path()) .env("AGENT_OS_PYODIDE_INDEX_URL", pyodide_index_url) .env("AGENT_OS_PYTHON_CODE", code); @@ -4429,6 +9187,8 @@ mod tests { ) -> Output { let mut command = Command::new(node_binary()); command + .arg("--import") + .arg(import_cache.timing_bootstrap_path()) .arg(import_cache.python_runner_path()) .env("AGENT_OS_PYODIDE_INDEX_URL", pyodide_index_url) .env("AGENT_OS_PYTHON_PREWARM_ONLY", "1"); @@ -4449,6 +9209,8 @@ mod tests { ) -> Output { let mut command = Command::new(node_binary()); command + .arg("--import") + .arg(import_cache.timing_bootstrap_path()) .arg(import_cache.python_runner_path()) .env("AGENT_OS_PYODIDE_INDEX_URL", pyodide_index_url) .env("AGENT_OS_PYTHON_CODE", code) @@ -4473,6 +9235,62 @@ mod tests { child.wait_with_output().expect("wait for python runner") } + #[test] + fn materialized_python_runner_hardens_builtin_access_before_load_pyodide() { + assert_node_available(); + + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let pyodide_dir = tempdir().expect("create pyodide fixture dir"); + write_fixture( + &pyodide_dir.path().join("pyodide.mjs"), + r#" +export async function loadPyodide(options) { + const capturedFetch = globalThis.fetch; + return { + setStdin(_stdin) {}, + async runPythonAsync() { + try { + await capturedFetch('http://127.0.0.1:1/'); + options.stdout('unexpected'); + } catch (error) { + options.stdout(JSON.stringify({ + code: error.code ?? null, + message: error.message, + })); + } + }, + }; +} +"#, + ); + write_fixture( + &pyodide_dir.path().join("pyodide-lock.json"), + "{\"packages\":[]}\n", + ); + + let output = run_python_runner(&import_cache, pyodide_dir.path(), "print('hello')"); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse hardening JSON"); + + assert_eq!(output.status.code(), Some(0), "stderr: {stderr}"); + assert_eq!( + parsed["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!( + parsed["message"] + .as_str() + .expect("fetch denial message") + .contains("network access"), + "unexpected stdout: {stdout}" + ); + } + #[test] fn materialized_python_runner_executes_python_code_via_pyodide_callbacks() { assert_node_available(); @@ -4667,6 +9485,145 @@ export async function loadPyodide() { ); } + #[test] + fn materialized_python_runner_blocks_pyodide_js_escape_modules() { + assert_node_available(); + + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let output = run_python_runner( + &import_cache, + import_cache.pyodide_dist_path(), + r#" +import json +import js +import pyodide_js + +def capture(action): + try: + action() + return {"ok": True} + except Exception as error: + return { + "ok": False, + "type": type(error).__name__, + "message": str(error), + } + +print(json.dumps({ + "js_process_env": capture(lambda: js.process.env), + "js_require": capture(lambda: js.require), + "js_process_exit": capture(lambda: js.process.exit), + "js_process_kill": capture(lambda: js.process.kill), + "js_child_process_builtin": capture( + lambda: js.process.getBuiltinModule("node:child_process") + ), + "js_vm_builtin": capture( + lambda: js.process.getBuiltinModule("node:vm") + ), + "pyodide_js_eval_code": capture(lambda: pyodide_js.eval_code), +})) +"#, + ); + + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + let parsed: Value = + serde_json::from_str(stdout.trim()).expect("parse Python hardening JSON"); + + assert_eq!(output.status.code(), Some(0), "stderr: {stderr}"); + + for key in [ + "js_process_env", + "js_require", + "js_process_exit", + "js_process_kill", + "js_child_process_builtin", + "js_vm_builtin", + ] { + assert_eq!(parsed[key]["ok"], Value::Bool(false), "stdout: {stdout}"); + assert_eq!( + parsed[key]["type"], + Value::String(String::from("RuntimeError")) + ); + assert!( + parsed[key]["message"] + .as_str() + .expect("js hardening message") + .contains("js is not available"), + "stdout: {stdout}" + ); + } + + assert_eq!( + parsed["pyodide_js_eval_code"]["ok"], + Value::Bool(false), + "stdout: {stdout}" + ); + assert_eq!( + parsed["pyodide_js_eval_code"]["type"], + Value::String(String::from("RuntimeError")) + ); + assert!( + parsed["pyodide_js_eval_code"]["message"] + .as_str() + .expect("pyodide_js hardening message") + .contains("pyodide_js is not available"), + "stdout: {stdout}" + ); + } + + #[test] + fn materialized_python_runner_exposes_frozen_time_to_python() { + assert_node_available(); + + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let frozen_time_ms = 1_704_067_200_123_u64; + let output = run_python_runner_with_env( + &import_cache, + import_cache.pyodide_dist_path(), + r#" +import datetime +import json +import time + +first_ns = time.time_ns() +second_ns = time.time_ns() +utc_now = datetime.datetime.now(datetime.timezone.utc) + +print(json.dumps({ + "first_ns": first_ns, + "second_ns": second_ns, + "iso": utc_now.isoformat(timespec="milliseconds"), +})) +"#, + &[("AGENT_OS_FROZEN_TIME_MS", "1704067200123")], + ); + + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse frozen-time JSON"); + + assert_eq!(output.status.code(), Some(0), "stderr: {stderr}"); + assert_eq!(parsed["first_ns"], parsed["second_ns"], "stdout: {stdout}"); + let first_ns = parsed["first_ns"] + .as_u64() + .expect("frozen time.time_ns() value"); + assert_eq!(first_ns / 1_000_000, frozen_time_ms, "stdout: {stdout}"); + assert_eq!( + parsed["iso"], + Value::String(String::from("2024-01-01T00:00:00.123+00:00")), + "stdout: {stdout}" + ); + } + #[test] fn materialized_python_runner_preloads_bundled_packages_from_local_disk() { assert_node_available(); @@ -4887,4 +9844,201 @@ export async function loadPyodide(options) { ); } } + + #[test] + fn ensure_materialized_honors_configured_timeout() { + let temp_root = tempdir().expect("create node import cache temp root"); + let import_cache = NodeImportCache::new_in(temp_root.path().to_path_buf()); + + NODE_IMPORT_CACHE_TEST_MATERIALIZE_DELAY_MS.store(50, Ordering::Relaxed); + let error = import_cache + .ensure_materialized_with_timeout(Duration::from_millis(5)) + .expect_err("materialization should time out"); + NODE_IMPORT_CACHE_TEST_MATERIALIZE_DELAY_MS.store(0, Ordering::Relaxed); + + assert_eq!(error.kind(), std::io::ErrorKind::TimedOut); + assert!( + error + .to_string() + .contains("timed out materializing node import cache"), + "unexpected error: {error}" + ); + + std::thread::sleep(Duration::from_millis(75)); + } + + #[test] + fn new_in_cleans_stale_temp_roots_without_touching_unrelated_entries() { + let temp_root = tempdir().expect("create node import cache temp root"); + let stale_cache_dir = temp_root + .path() + .join("agent-os-node-import-cache-stale-test"); + let unrelated_dir = temp_root.path().join("keep-me"); + fs::create_dir_all(&stale_cache_dir).expect("create stale cache dir"); + fs::create_dir_all(&unrelated_dir).expect("create unrelated dir"); + fs::write(stale_cache_dir.join("state.json"), b"stale").expect("seed stale cache"); + + let import_cache = NodeImportCache::new_in(temp_root.path().to_path_buf()); + + assert!( + !stale_cache_dir.exists(), + "expected stale cache dir to be removed" + ); + assert!(unrelated_dir.exists(), "expected unrelated dir to remain"); + assert!( + import_cache.root_dir.starts_with(temp_root.path()), + "expected import cache root to stay inside the configured temp root" + ); + } + + #[test] + fn ensure_materialized_writes_denied_builtin_assets_for_hardened_modules() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let denied_root = import_cache.asset_root().join("denied"); + let actual = fs::read_dir(&denied_root) + .expect("read denied builtin assets") + .map(|entry| { + entry + .expect("denied builtin asset entry") + .path() + .file_stem() + .expect("denied builtin asset file stem") + .to_string_lossy() + .into_owned() + }) + .collect::>(); + let expected = BTreeSet::from([ + String::from("child_process"), + String::from("cluster"), + String::from("dgram"), + String::from("diagnostics_channel"), + String::from("http"), + String::from("http2"), + String::from("https"), + String::from("inspector"), + String::from("module"), + String::from("net"), + String::from("trace_events"), + String::from("v8"), + String::from("vm"), + String::from("worker_threads"), + ]); + + assert_eq!(actual, expected); + + let module_asset = + fs::read_to_string(denied_root.join("module.mjs")).expect("read module denied asset"); + let trace_events_asset = fs::read_to_string(denied_root.join("trace_events.mjs")) + .expect("read trace_events denied asset"); + + assert!(module_asset.contains("node:module is not available")); + assert!(trace_events_asset.contains("ERR_ACCESS_DENIED")); + } + + #[test] + fn ensure_materialized_writes_os_builtin_asset() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let os_asset = + fs::read_to_string(import_cache.asset_root().join("builtins").join("os.mjs")) + .expect("read os builtin asset"); + + assert!(os_asset.contains("__agentOsBuiltinOs")); + assert!(os_asset.contains("export const hostname = mod.hostname")); + assert!(os_asset.contains("export const userInfo = mod.userInfo")); + } + + #[test] + fn ensure_materialized_writes_http_builtin_assets() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let builtins_root = import_cache.asset_root().join("builtins"); + let http_asset = + fs::read_to_string(builtins_root.join("http.mjs")).expect("read http builtin asset"); + let http2_asset = + fs::read_to_string(builtins_root.join("http2.mjs")).expect("read http2 builtin asset"); + let https_asset = + fs::read_to_string(builtins_root.join("https.mjs")).expect("read https builtin asset"); + + assert!(http_asset.contains("__agentOsBuiltinHttp")); + assert!(http_asset.contains("export const request = mod.request")); + assert!(http2_asset.contains("__agentOsBuiltinHttp2")); + assert!(http2_asset.contains("export const connect = mod.connect")); + assert!(https_asset.contains("__agentOsBuiltinHttps")); + assert!(https_asset.contains("export const createServer = mod.createServer")); + } + + #[test] + fn ensure_materialized_writes_net_builtin_asset() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let net_asset = + fs::read_to_string(import_cache.asset_root().join("builtins").join("net.mjs")) + .expect("read net builtin asset"); + + assert!(net_asset.contains("__agentOsBuiltinNet")); + assert!(net_asset.contains("export const connect = mod.connect")); + assert!(net_asset.contains("export const createServer = mod.createServer")); + } + + #[test] + fn ensure_materialized_writes_dgram_builtin_asset() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let dgram_asset = + fs::read_to_string(import_cache.asset_root().join("builtins").join("dgram.mjs")) + .expect("read dgram builtin asset"); + + assert!(dgram_asset.contains("__agentOsBuiltinDgram")); + assert!(dgram_asset.contains("export const Socket = mod.Socket")); + assert!(dgram_asset.contains("export const createSocket = mod.createSocket")); + } + + #[test] + fn ensure_materialized_writes_dns_builtin_asset() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let dns_asset = + fs::read_to_string(import_cache.asset_root().join("builtins").join("dns.mjs")) + .expect("read dns builtin asset"); + + assert!(dns_asset.contains("__agentOsBuiltinDns")); + assert!(dns_asset.contains("export const lookup = mod.lookup")); + assert!(dns_asset.contains("export const resolve4 = mod.resolve4")); + } + + #[test] + fn ensure_materialized_writes_tls_builtin_asset() { + let import_cache = NodeImportCache::default(); + import_cache + .ensure_materialized() + .expect("materialize node import cache"); + + let tls_asset = + fs::read_to_string(import_cache.asset_root().join("builtins").join("tls.mjs")) + .expect("read tls builtin asset"); + + assert!(tls_asset.contains("__agentOsBuiltinTls")); + assert!(tls_asset.contains("export const connect = mod.connect")); + assert!(tls_asset.contains("export const createServer = mod.createServer")); + } } diff --git a/crates/execution/src/node_process.rs b/crates/execution/src/node_process.rs index f924ae309..2844c0e29 100644 --- a/crates/execution/src/node_process.rs +++ b/crates/execution/src/node_process.rs @@ -1,6 +1,12 @@ pub(crate) use crate::common::{encode_json_string_array, encode_json_string_map}; +use nix::fcntl::{fcntl, FcntlArg, OFlag}; +use nix::unistd::{close, pipe2}; +use serde::{Deserialize, Serialize}; +use serde_json::from_str; use std::collections::{BTreeMap, BTreeSet}; -use std::io::Read; +use std::fs::File; +use std::io::{BufRead, BufReader, Read}; +use std::os::fd::{AsRawFd, OwnedFd, RawFd}; use std::path::{Path, PathBuf}; use std::process::{Child, Command}; use std::sync::mpsc::Sender; @@ -15,12 +21,57 @@ const NODE_ALLOW_CHILD_PROCESS_FLAG: &str = "--allow-child-process"; const NODE_DISABLE_SECURITY_WARNING_FLAG: &str = "--disable-warning=SecurityWarning"; const NODE_ALLOW_FS_READ_FLAG: &str = "--allow-fs-read="; const NODE_ALLOW_FS_WRITE_FLAG: &str = "--allow-fs-write="; +const NODE_ALLOWED_BUILTINS_ENV: &str = "AGENT_OS_ALLOWED_NODE_BUILTINS"; const DANGEROUS_GUEST_ENV_KEYS: &[&str] = &[ "DYLD_INSERT_LIBRARIES", "LD_LIBRARY_PATH", "LD_PRELOAD", "NODE_OPTIONS", ]; +pub const NODE_CONTROL_PIPE_FD_ENV: &str = "AGENT_OS_CONTROL_PIPE_FD"; +const RESERVED_CHILD_FD_MIN: RawFd = 1000; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum NodeSignalDispositionAction { + Default, + Ignore, + User, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct NodeSignalHandlerRegistration { + pub action: NodeSignalDispositionAction, + pub mask: Vec, + pub flags: u32, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "type", rename_all = "snake_case")] +pub enum NodeControlMessage { + NodeImportCacheMetrics { + metrics: serde_json::Value, + }, + PythonExit { + #[serde(rename = "exitCode")] + exit_code: i32, + }, + SignalState { + signal: u32, + registration: NodeSignalHandlerRegistration, + }, +} + +pub struct NodeControlChannel { + pub parent_reader: File, + pub child_writer: OwnedFd, +} + +#[derive(Debug, Default)] +pub struct LinePrefixFilter { + pending: Vec, +} pub fn node_binary() -> String { let configured = @@ -28,6 +79,55 @@ pub fn node_binary() -> String { resolve_executable_path(&configured).unwrap_or(configured) } +pub fn create_node_control_channel() -> std::io::Result { + let (parent_reader, child_writer) = pipe2(OFlag::O_CLOEXEC).map_err(std::io::Error::other)?; + + Ok(NodeControlChannel { + parent_reader: File::from(parent_reader), + child_writer, + }) +} + +#[derive(Debug, Default)] +pub(crate) struct ExportedChildFds { + fds: Vec, +} + +impl ExportedChildFds { + pub(crate) fn export( + &mut self, + command: &mut Command, + env_key: &str, + source_fd: &OwnedFd, + ) -> std::io::Result { + let exported_fd = fcntl( + source_fd.as_raw_fd(), + FcntlArg::F_DUPFD(RESERVED_CHILD_FD_MIN), + ) + .map_err(std::io::Error::other)?; + command.env(env_key, exported_fd.to_string()); + self.fds.push(exported_fd); + Ok(exported_fd) + } +} + +impl Drop for ExportedChildFds { + fn drop(&mut self) { + for fd in self.fds.drain(..) { + let _ = close(fd); + } + } +} + +pub fn configure_node_control_channel( + command: &mut Command, + child_writer: &OwnedFd, + exported_fds: &mut ExportedChildFds, +) -> std::io::Result<()> { + exported_fds.export(command, NODE_CONTROL_PIPE_FD_ENV, child_writer)?; + Ok(()) +} + pub fn harden_node_command( command: &mut Command, cwd: &Path, @@ -35,11 +135,14 @@ pub fn harden_node_command( write_paths: &[PathBuf], enable_permissions: bool, allow_wasi: bool, + allow_worker: bool, allow_child_process: bool, ) { if enable_permissions { command.arg(NODE_PERMISSION_FLAG); - command.arg(NODE_ALLOW_WORKER_FLAG); + if allow_worker { + command.arg(NODE_ALLOW_WORKER_FLAG); + } command.arg(NODE_DISABLE_SECURITY_WARNING_FLAG); if allow_wasi { command.arg(NODE_ALLOW_WASI_FLAG); @@ -64,6 +167,12 @@ pub fn harden_node_command( command.env_clear(); } +pub fn env_builtin_enabled(env: &BTreeMap, builtin: &str) -> bool { + env.get(NODE_ALLOWED_BUILTINS_ENV) + .and_then(|value| from_str::>(value).ok()) + .is_some_and(|builtins| builtins.iter().any(|entry| entry == builtin)) +} + pub fn node_resolution_read_paths(roots: impl IntoIterator) -> Vec { let mut paths = Vec::new(); @@ -150,6 +259,77 @@ where } }) } + +pub fn spawn_node_control_reader( + reader: File, + sender: Sender, + map_message: FM, + map_error: FE, +) -> JoinHandle<()> +where + E: Send + 'static, + FM: Fn(NodeControlMessage) -> E + Send + 'static, + FE: Fn(String) -> E + Send + 'static, +{ + thread::spawn(move || { + let mut reader = BufReader::new(reader); + let mut line = String::new(); + + loop { + line.clear(); + match reader.read_line(&mut line) { + Ok(0) => return, + Ok(_) => { + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + + match serde_json::from_str::(trimmed) { + Ok(message) => { + if sender.send(map_message(message)).is_err() { + return; + } + } + Err(error) => { + if sender + .send(map_error(format!( + "invalid agent-os node control message: {error}\n" + ))) + .is_err() + { + return; + } + } + } + } + Err(error) => { + let _ = sender.send(map_error(format!( + "agent-os node control read error: {error}\n" + ))); + return; + } + } + } + }) +} + +impl LinePrefixFilter { + pub fn filter_chunk(&mut self, chunk: &[u8], prefixes: &[&str]) -> Vec { + self.pending.extend_from_slice(chunk); + let mut filtered = Vec::new(); + + while let Some(newline_index) = self.pending.iter().position(|byte| *byte == b'\n') { + let line = self.pending.drain(..=newline_index).collect::>(); + if !has_control_prefix(&line, prefixes) { + filtered.extend_from_slice(&line); + } + } + + filtered + } +} + fn allowed_paths(paths: impl IntoIterator) -> Vec { let mut unique = Vec::new(); let mut seen = BTreeSet::new(); @@ -177,6 +357,50 @@ fn normalize_path(path: PathBuf) -> PathBuf { absolute.canonicalize().unwrap_or(absolute) } +fn has_control_prefix(line: &[u8], prefixes: &[&str]) -> bool { + let text = String::from_utf8_lossy(line); + let trimmed = text.trim_end_matches(['\r', '\n']); + prefixes.iter().any(|prefix| trimmed.starts_with(prefix)) +} + +#[cfg(test)] +mod tests { + use super::*; + use nix::fcntl::FdFlag; + use std::process::Command; + + #[test] + fn exported_child_fds_use_reserved_high_numbers_while_sources_stay_cloexec() { + let channel = create_node_control_channel().expect("create control channel"); + let source_fd = channel.child_writer.as_raw_fd(); + let source_flags = fcntl(channel.child_writer.as_raw_fd(), FcntlArg::F_GETFD) + .expect("read source fd flags"); + + assert!( + FdFlag::from_bits_retain(source_flags).contains(FdFlag::FD_CLOEXEC), + "child-side source fd should remain close-on-exec until it is remapped" + ); + + let mut command = Command::new("true"); + let mut exported_fds = ExportedChildFds::default(); + configure_node_control_channel(&mut command, &channel.child_writer, &mut exported_fds) + .expect("export control fd"); + + let exported_fd = command + .get_envs() + .find_map(|(key, value)| { + (key == NODE_CONTROL_PIPE_FD_ENV) + .then(|| value.expect("exported fd env value")) + .and_then(|value| value.to_str()) + .and_then(|value| value.parse::().ok()) + }) + .expect("control fd env"); + + assert!(exported_fd >= RESERVED_CHILD_FD_MIN); + assert_ne!(exported_fd, source_fd); + } +} + fn resolve_executable_path(binary: &str) -> Option { let path = Path::new(binary); if path.is_absolute() || binary.contains(std::path::MAIN_SEPARATOR) { diff --git a/crates/execution/src/python.rs b/crates/execution/src/python.rs index 1ef5c7e3f..7459d4ac6 100644 --- a/crates/execution/src/python.rs +++ b/crates/execution/src/python.rs @@ -1,7 +1,19 @@ -use crate::common::{encode_json_string, frozen_time_ms, stable_hash64}; -use crate::node_import_cache::{NodeImportCache, NODE_IMPORT_CACHE_ASSET_ROOT_ENV}; -use crate::node_process::{apply_guest_env, harden_node_command, node_binary, spawn_stream_reader}; -use nix::fcntl::{fcntl, FcntlArg, FdFlag, OFlag}; +use crate::common::{encode_json_string, frozen_time_ms}; +use crate::node_import_cache::{ + NodeImportCache, NodeImportCacheCleanup, NODE_IMPORT_CACHE_ASSET_ROOT_ENV, +}; +use crate::node_process::{ + apply_guest_env, configure_node_control_channel, create_node_control_channel, + harden_node_command, node_binary, spawn_node_control_reader, spawn_stream_reader, + ExportedChildFds, LinePrefixFilter, NodeControlMessage, +}; +use crate::runtime_support::{ + compile_cache_ready, configure_compile_cache, env_flag_enabled, file_fingerprint, + import_cache_root, resolve_execution_path, sandbox_root, warmup_marker_path, + NODE_COMPILE_CACHE_ENV, NODE_DISABLE_COMPILE_CACHE_ENV, NODE_FROZEN_TIME_ENV, + NODE_SANDBOX_ROOT_ENV, +}; +use nix::fcntl::OFlag; use nix::unistd::pipe2; use serde::Deserialize; use serde_json::json; @@ -10,17 +22,14 @@ use std::fmt; use std::fs; use std::fs::File; use std::io::{BufRead, BufReader, BufWriter, Write}; -use std::os::fd::{AsRawFd, OwnedFd}; +use std::os::fd::OwnedFd; use std::path::{Path, PathBuf}; use std::process::{Child, ChildStdin, Command, Stdio}; +use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender}; use std::sync::{Arc, Mutex}; use std::thread::{self, JoinHandle}; -use std::time::{Duration, Instant, UNIX_EPOCH}; - -const NODE_COMPILE_CACHE_ENV: &str = "NODE_COMPILE_CACHE"; -const NODE_DISABLE_COMPILE_CACHE_ENV: &str = "NODE_DISABLE_COMPILE_CACHE"; -const NODE_FROZEN_TIME_ENV: &str = "AGENT_OS_FROZEN_TIME_MS"; +use std::time::{Duration, Instant}; const NODE_ALLOWED_BUILTINS_ENV: &str = "AGENT_OS_ALLOWED_NODE_BUILTINS"; const NODE_IMPORT_CACHE_PATH_ENV: &str = "AGENT_OS_NODE_IMPORT_CACHE_PATH"; const PYODIDE_INDEX_URL_ENV: &str = "AGENT_OS_PYODIDE_INDEX_URL"; @@ -29,23 +38,41 @@ const PYTHON_FILE_ENV: &str = "AGENT_OS_PYTHON_FILE"; const PYTHON_PREWARM_ONLY_ENV: &str = "AGENT_OS_PYTHON_PREWARM_ONLY"; const PYTHON_WARMUP_DEBUG_ENV: &str = "AGENT_OS_PYTHON_WARMUP_DEBUG"; const PYTHON_WARMUP_METRICS_PREFIX: &str = "__AGENT_OS_PYTHON_WARMUP_METRICS__:"; +const PYTHON_OUTPUT_BUFFER_MAX_BYTES_ENV: &str = "AGENT_OS_PYTHON_OUTPUT_BUFFER_MAX_BYTES"; +const PYTHON_EXECUTION_TIMEOUT_MS_ENV: &str = "AGENT_OS_PYTHON_EXECUTION_TIMEOUT_MS"; +const PYTHON_MAX_OLD_SPACE_MB_ENV: &str = "AGENT_OS_PYTHON_MAX_OLD_SPACE_MB"; const PYTHON_VFS_RPC_REQUEST_FD_ENV: &str = "AGENT_OS_PYTHON_VFS_RPC_REQUEST_FD"; const PYTHON_VFS_RPC_RESPONSE_FD_ENV: &str = "AGENT_OS_PYTHON_VFS_RPC_RESPONSE_FD"; +const PYTHON_VFS_RPC_TIMEOUT_MS_ENV: &str = "AGENT_OS_PYTHON_VFS_RPC_TIMEOUT_MS"; +const PYTHON_VFS_RPC_MAX_PENDING_REQUESTS_ENV: &str = + "AGENT_OS_PYTHON_VFS_RPC_MAX_PENDING_REQUESTS"; const PYTHON_EXIT_CONTROL_PREFIX: &str = "__AGENT_OS_PYTHON_EXIT__:"; const PYTHON_WARMUP_MARKER_VERSION: &str = "1"; +const DEFAULT_PYTHON_OUTPUT_BUFFER_MAX_BYTES: usize = 1024 * 1024; +const DEFAULT_PYTHON_EXECUTION_TIMEOUT_MS: u64 = 5 * 60 * 1000; +const DEFAULT_PYTHON_MAX_OLD_SPACE_MB: usize = 1024; +const DEFAULT_PYTHON_VFS_RPC_TIMEOUT_MS: u64 = 30_000; +const DEFAULT_PYTHON_VFS_RPC_MAX_PENDING_REQUESTS: usize = 1000; +const CONTROLLED_STDERR_PREFIXES: &[&str] = &[PYTHON_EXIT_CONTROL_PREFIX]; const RESERVED_PYTHON_ENV_KEYS: &[&str] = &[ NODE_COMPILE_CACHE_ENV, NODE_DISABLE_COMPILE_CACHE_ENV, NODE_ALLOWED_BUILTINS_ENV, + NODE_SANDBOX_ROOT_ENV, NODE_FROZEN_TIME_ENV, NODE_IMPORT_CACHE_ASSET_ROOT_ENV, NODE_IMPORT_CACHE_PATH_ENV, PYODIDE_INDEX_URL_ENV, PYTHON_CODE_ENV, + PYTHON_EXECUTION_TIMEOUT_MS_ENV, PYTHON_FILE_ENV, + PYTHON_MAX_OLD_SPACE_MB_ENV, + PYTHON_OUTPUT_BUFFER_MAX_BYTES_ENV, PYTHON_PREWARM_ONLY_ENV, PYTHON_VFS_RPC_REQUEST_FD_ENV, PYTHON_VFS_RPC_RESPONSE_FD_ENV, + PYTHON_VFS_RPC_MAX_PENDING_REQUESTS_ENV, + PYTHON_VFS_RPC_TIMEOUT_MS_ENV, ]; #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -145,6 +172,15 @@ pub enum PythonExecutionEvent { Exited(i32), } +#[derive(Debug, Clone, PartialEq, Eq)] +enum PythonProcessEvent { + Stdout(Vec), + RawStderr(Vec), + VfsRpcRequest(PythonVfsRpcRequest), + Control(NodeControlMessage), + Exited(i32), +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct PythonExecutionResult { pub execution_id: String, @@ -165,7 +201,9 @@ pub enum PythonExecutionError { Spawn(std::io::Error), StdinClosed, Stdin(std::io::Error), + Kill(std::io::Error), Wait(std::io::Error), + TimedOut(Duration), PendingVfsRpcRequest(u64), RpcChannel(String), RpcResponse(String), @@ -208,7 +246,13 @@ impl fmt::Display for PythonExecutionError { Self::Spawn(err) => write!(f, "failed to start guest Python runtime: {err}"), Self::StdinClosed => f.write_str("guest Python stdin is already closed"), Self::Stdin(err) => write!(f, "failed to write guest stdin: {err}"), + Self::Kill(err) => write!(f, "failed to kill guest Python runtime: {err}"), Self::Wait(err) => write!(f, "failed to wait for guest Python runtime: {err}"), + Self::TimedOut(timeout) => write!( + f, + "guest Python runtime timed out after {}ms", + timeout.as_millis() + ), Self::PendingVfsRpcRequest(id) => { write!( f, @@ -242,9 +286,29 @@ pub struct PythonExecution { child_pid: u32, child: Arc>>, stdin: Option, - events: Receiver, + events: Receiver, pending_exit_code: Arc>>, + pending_vfs_rpc: Arc>>, + pending_vfs_rpc_count: Arc, vfs_rpc_responses: Arc>>, + stderr_filter: Arc>, + output_buffer_max_bytes: usize, + execution_timeout: Option, + vfs_rpc_timeout: Duration, + _import_cache_guard: Arc, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum PendingVfsRpcState { + Pending(u64), + TimedOut(u64), +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum PendingVfsRpcResolution { + Pending, + TimedOut, + Missing, } impl PythonExecution { @@ -274,11 +338,35 @@ impl PythonExecution { Ok(()) } + pub fn cancel(&mut self) -> Result<(), PythonExecutionError> { + self.kill() + } + + pub fn kill(&mut self) -> Result<(), PythonExecutionError> { + self.close_stdin()?; + if let Some(exit_code) = self.terminate_child()? { + self.store_pending_exit_code(exit_code)?; + } + Ok(()) + } + pub fn respond_vfs_rpc_success( &mut self, id: u64, payload: PythonVfsRpcResponsePayload, ) -> Result<(), PythonExecutionError> { + match self.clear_pending_vfs_rpc(id)? { + PendingVfsRpcResolution::Pending => { + release_python_vfs_rpc_slot(self.pending_vfs_rpc_count.as_ref()); + } + PendingVfsRpcResolution::TimedOut => { + return Err(PythonExecutionError::RpcResponse(format!( + "VFS RPC request {id} is no longer pending" + ))); + } + PendingVfsRpcResolution::Missing => {} + } + let result = match payload { PythonVfsRpcResponsePayload::Empty => json!({}), PythonVfsRpcResponsePayload::Read { content_base64 } => { @@ -313,6 +401,18 @@ impl PythonExecution { code: impl Into, message: impl Into, ) -> Result<(), PythonExecutionError> { + match self.clear_pending_vfs_rpc(id)? { + PendingVfsRpcResolution::Pending => { + release_python_vfs_rpc_slot(self.pending_vfs_rpc_count.as_ref()); + } + PendingVfsRpcResolution::TimedOut => { + return Err(PythonExecutionError::RpcResponse(format!( + "VFS RPC request {id} is no longer pending" + ))); + } + PendingVfsRpcResolution::Missing => {} + } + write_python_vfs_rpc_response( &self.vfs_rpc_responses, json!({ @@ -330,66 +430,124 @@ impl PythonExecution { &self, timeout: Duration, ) -> Result, PythonExecutionError> { - match self.events.recv_timeout(timeout) { - Ok(PythonExecutionEvent::Stderr(chunk)) => { - let (exit_code, filtered_chunk) = extract_python_exit_control(&chunk); - if let Some(exit_code) = exit_code { - self.store_pending_exit_code(exit_code)?; - if filtered_chunk.is_empty() { - return self.poll_event(Duration::from_millis(10)); + let started = Instant::now(); + + loop { + let remaining = timeout.saturating_sub(started.elapsed()); + match self.events.recv_timeout(remaining) { + Ok(PythonProcessEvent::Stdout(chunk)) => { + return Ok(Some(PythonExecutionEvent::Stdout(chunk))); + } + Ok(PythonProcessEvent::RawStderr(chunk)) => { + let mut filter = self + .stderr_filter + .lock() + .map_err(|_| PythonExecutionError::EventChannelClosed)?; + let filtered = filter.filter_chunk(&chunk, CONTROLLED_STDERR_PREFIXES); + if filtered.is_empty() { + if started.elapsed() >= timeout { + if let Some(exit_code) = self.take_pending_exit_code()? { + return Ok(Some(PythonExecutionEvent::Exited(exit_code))); + } + return Ok(None); + } + continue; } - return Ok(Some(PythonExecutionEvent::Stderr(filtered_chunk))); + return Ok(Some(PythonExecutionEvent::Stderr(filtered))); } - Ok(Some(PythonExecutionEvent::Stderr(chunk))) - } - Ok(event) => Ok(Some(event)), - Err(RecvTimeoutError::Timeout) => { - if let Some(exit_code) = self.take_pending_exit_code()? { - self.finalize_child_exit(exit_code)?; - return Ok(Some(PythonExecutionEvent::Exited(exit_code))); + Ok(PythonProcessEvent::VfsRpcRequest(request)) => { + self.set_pending_vfs_rpc(request.id)?; + spawn_python_vfs_rpc_timeout( + request.id, + self.vfs_rpc_timeout, + self.pending_vfs_rpc.clone(), + self.pending_vfs_rpc_count.clone(), + self.vfs_rpc_responses.clone(), + ); + return Ok(Some(PythonExecutionEvent::VfsRpcRequest(request))); } - self.poll_child_exit() - } - Err(RecvTimeoutError::Disconnected) => { - if let Some(exit_code) = self.take_pending_exit_code()? { - self.finalize_child_exit(exit_code)?; + Ok(PythonProcessEvent::Exited(exit_code)) => { return Ok(Some(PythonExecutionEvent::Exited(exit_code))); } - if let Some(event) = self.poll_child_exit()? { - return Ok(Some(event)); + Ok(PythonProcessEvent::Control(_)) => { + if started.elapsed() >= timeout { + if let Some(exit_code) = self.take_pending_exit_code()? { + return Ok(Some(PythonExecutionEvent::Exited(exit_code))); + } + return Ok(None); + } + } + Err(RecvTimeoutError::Timeout) => { + if let Some(exit_code) = self.take_pending_exit_code()? { + return Ok(Some(PythonExecutionEvent::Exited(exit_code))); + } + return Ok(None); + } + Err(RecvTimeoutError::Disconnected) => { + if let Some(exit_code) = self.take_pending_exit_code()? { + return Ok(Some(PythonExecutionEvent::Exited(exit_code))); + } + return Err(PythonExecutionError::EventChannelClosed); } - Err(PythonExecutionError::EventChannelClosed) } } } - pub fn wait(mut self) -> Result { + pub fn wait( + mut self, + timeout: Option, + ) -> Result { self.close_stdin()?; - let mut stdout = Vec::new(); - let mut stderr = Vec::new(); + let mut stdout = PythonOutputBuffer::new(self.output_buffer_max_bytes); + let mut stderr = PythonOutputBuffer::new(self.output_buffer_max_bytes); + let started = Instant::now(); + let timeout = match (timeout, self.execution_timeout) { + (Some(requested), Some(configured)) => Some(requested.min(configured)), + (Some(requested), None) => Some(requested), + (None, Some(configured)) => Some(configured), + (None, None) => None, + }; loop { - match self.poll_event(Duration::from_millis(50))? { - Some(PythonExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), - Some(PythonExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + let poll_timeout = timeout + .map(|limit| { + let elapsed = started.elapsed(); + if elapsed >= limit { + Duration::ZERO + } else { + limit.saturating_sub(elapsed).min(Duration::from_millis(50)) + } + }) + .unwrap_or_else(|| Duration::from_millis(50)); + + match self.poll_event(poll_timeout)? { + Some(PythonExecutionEvent::Stdout(chunk)) => stdout.extend(&chunk), + Some(PythonExecutionEvent::Stderr(chunk)) => stderr.extend(&chunk), Some(PythonExecutionEvent::VfsRpcRequest(request)) => { return Err(PythonExecutionError::PendingVfsRpcRequest(request.id)); } Some(PythonExecutionEvent::Exited(exit_code)) => { return Ok(PythonExecutionResult { - execution_id: self.execution_id, + execution_id: self.execution_id.clone(), exit_code, - stdout, - stderr, + stdout: stdout.into_inner(), + stderr: stderr.into_inner(), }); } None => {} } + + if let Some(limit) = timeout { + if started.elapsed() >= limit { + self.kill()?; + return Err(PythonExecutionError::TimedOut(limit)); + } + } } } - fn poll_child_exit(&self) -> Result, PythonExecutionError> { + fn terminate_child(&self) -> Result, PythonExecutionError> { let mut child_slot = self .child .lock() @@ -398,15 +556,20 @@ impl PythonExecution { return Ok(None); }; - match child.try_wait().map_err(PythonExecutionError::Wait)? { - Some(status) => { - *child_slot = None; - Ok(Some(PythonExecutionEvent::Exited( - status.code().unwrap_or(1), - ))) + let exit_code = match child.try_wait().map_err(PythonExecutionError::Wait)? { + Some(status) => status.code().unwrap_or(1), + None => { + child.kill().map_err(PythonExecutionError::Kill)?; + child + .wait() + .map_err(PythonExecutionError::Wait)? + .code() + .unwrap_or(1) } - None => Ok(None), - } + }; + + *child_slot = None; + Ok(Some(exit_code)) } fn store_pending_exit_code(&self, exit_code: i32) -> Result<(), PythonExecutionError> { @@ -426,24 +589,40 @@ impl PythonExecution { Ok(pending.take()) } - fn finalize_child_exit(&self, _exit_code: i32) -> Result<(), PythonExecutionError> { - let mut child_slot = self - .child + fn set_pending_vfs_rpc(&self, id: u64) -> Result<(), PythonExecutionError> { + let mut pending = self + .pending_vfs_rpc .lock() .map_err(|_| PythonExecutionError::EventChannelClosed)?; - if let Some(child) = child_slot.as_mut() { - match child.try_wait().map_err(PythonExecutionError::Wait)? { - Some(_) => { - *child_slot = None; - } - None => { - let _ = child.kill(); - let _ = child.wait(); - *child_slot = None; - } + *pending = Some(PendingVfsRpcState::Pending(id)); + Ok(()) + } + + fn clear_pending_vfs_rpc( + &self, + id: u64, + ) -> Result { + let mut pending = self + .pending_vfs_rpc + .lock() + .map_err(|_| PythonExecutionError::EventChannelClosed)?; + match *pending { + Some(PendingVfsRpcState::Pending(current)) if current == id => { + *pending = None; + Ok(PendingVfsRpcResolution::Pending) + } + Some(PendingVfsRpcState::TimedOut(current)) if current == id => { + Ok(PendingVfsRpcResolution::TimedOut) } + _ => Ok(PendingVfsRpcResolution::Missing), } - Ok(()) + } +} + +impl Drop for PythonExecution { + fn drop(&mut self) { + let _ = self.close_stdin(); + let _ = self.terminate_child(); } } @@ -452,16 +631,24 @@ pub struct PythonExecutionEngine { next_context_id: usize, next_execution_id: usize, contexts: BTreeMap, - import_cache: NodeImportCache, + import_caches: BTreeMap, } impl PythonExecutionEngine { - pub fn bundled_pyodide_dist_path(&self) -> &Path { - self.import_cache.pyodide_dist_path() + pub fn bundled_pyodide_dist_path_for_vm( + &mut self, + vm_id: &str, + ) -> Result { + let import_cache = self.import_caches.entry(vm_id.to_owned()).or_default(); + import_cache + .ensure_materialized() + .map_err(PythonExecutionError::PrepareRuntime)?; + Ok(import_cache.pyodide_dist_path().to_path_buf()) } pub fn create_context(&mut self, request: CreatePythonContextRequest) -> PythonContext { self.next_context_id += 1; + self.import_caches.entry(request.vm_id.clone()).or_default(); let context = PythonContext { context_id: format!("python-ctx-{}", self.next_context_id), @@ -490,21 +677,31 @@ impl PythonExecutionEngine { }); } - self.import_cache - .ensure_materialized() - .map_err(PythonExecutionError::PrepareRuntime)?; let frozen_time_ms = frozen_time_ms(); - let warmup_metrics = - prewarm_python_path(&self.import_cache, &context, &request, frozen_time_ms)?; + let warmup_metrics = { + let import_cache = self.import_caches.entry(context.vm_id.clone()).or_default(); + import_cache + .ensure_materialized() + .map_err(PythonExecutionError::PrepareRuntime)?; + prewarm_python_path(import_cache, &context, &request, frozen_time_ms)? + }; self.next_execution_id += 1; let execution_id = format!("exec-{}", self.next_execution_id); let rpc_channels = create_python_vfs_rpc_channels()?; + let control_channel = create_node_control_channel().map_err(PythonExecutionError::Spawn)?; + let import_cache = self + .import_caches + .get(&context.vm_id) + .expect("vm import cache should exist after materialization"); + let import_cache_guard = import_cache.cleanup_guard(); + let pending_vfs_rpc_count = Arc::new(AtomicUsize::new(0)); let (mut child, rpc_request_reader, rpc_response_writer) = create_node_child( - &self.import_cache, + import_cache, &context, &request, rpc_channels, + &control_channel.child_writer, frozen_time_ms, )?; let child_pid = child.id(); @@ -521,17 +718,33 @@ impl PythonExecutionEngine { let (sender, receiver) = mpsc::channel(); if let Some(metrics) = warmup_metrics { - let _ = sender.send(PythonExecutionEvent::Stderr(metrics)); + let _ = sender.send(PythonProcessEvent::RawStderr(metrics)); } - let stdout_reader = - spawn_stream_reader(stdout, sender.clone(), PythonExecutionEvent::Stdout); + let stdout_reader = spawn_stream_reader(stdout, sender.clone(), PythonProcessEvent::Stdout); let stderr_reader = - spawn_stream_reader(stderr, sender.clone(), PythonExecutionEvent::Stderr); - let _rpc_reader = spawn_python_vfs_rpc_reader(rpc_request_reader, sender.clone()); - let _stdout_reader = stdout_reader; - let _stderr_reader = stderr_reader; - let _sender = sender; + spawn_stream_reader(stderr, sender.clone(), PythonProcessEvent::RawStderr); + let _rpc_reader = spawn_python_vfs_rpc_reader( + rpc_request_reader, + sender.clone(), + rpc_response_writer.clone(), + pending_vfs_rpc_count.clone(), + python_vfs_rpc_max_pending_requests(&request), + ); + let _control_reader = spawn_node_control_reader( + control_channel.parent_reader, + sender.clone(), + PythonProcessEvent::Control, + |message| PythonProcessEvent::RawStderr(message.into_bytes()), + ); let child = Arc::new(Mutex::new(Some(child))); + spawn_python_waiter( + child.clone(), + stdout_reader, + stderr_reader, + sender, + PythonProcessEvent::Exited, + |message| PythonProcessEvent::RawStderr(message.into_bytes()), + ); Ok(PythonExecution { execution_id, @@ -540,9 +753,206 @@ impl PythonExecutionEngine { stdin, events: receiver, pending_exit_code: Arc::new(Mutex::new(None)), + pending_vfs_rpc: Arc::new(Mutex::new(None)), + pending_vfs_rpc_count, vfs_rpc_responses: rpc_response_writer, + stderr_filter: Arc::new(Mutex::new(LinePrefixFilter::default())), + output_buffer_max_bytes: python_output_buffer_max_bytes(&request), + execution_timeout: python_execution_timeout(&request), + vfs_rpc_timeout: python_vfs_rpc_timeout(&request), + _import_cache_guard: import_cache_guard, }) } + + pub fn dispose_vm(&mut self, vm_id: &str) { + self.contexts.retain(|_, context| context.vm_id != vm_id); + self.import_caches.remove(vm_id); + } +} + +#[derive(Debug)] +struct PythonOutputBuffer { + bytes: Vec, + max_bytes: usize, +} + +impl PythonOutputBuffer { + fn new(max_bytes: usize) -> Self { + Self { + bytes: Vec::new(), + max_bytes, + } + } + + fn extend(&mut self, chunk: &[u8]) { + if self.bytes.len() >= self.max_bytes { + return; + } + + let remaining = self.max_bytes - self.bytes.len(); + let take = remaining.min(chunk.len()); + self.bytes.extend_from_slice(&chunk[..take]); + } + + fn into_inner(self) -> Vec { + self.bytes + } +} + +fn python_output_buffer_max_bytes(request: &StartPythonExecutionRequest) -> usize { + request + .env + .get(PYTHON_OUTPUT_BUFFER_MAX_BYTES_ENV) + .and_then(|value| value.trim().parse::().ok()) + .unwrap_or(DEFAULT_PYTHON_OUTPUT_BUFFER_MAX_BYTES) +} + +fn python_execution_timeout(request: &StartPythonExecutionRequest) -> Option { + match request.env.get(PYTHON_EXECUTION_TIMEOUT_MS_ENV) { + Some(value) => { + let trimmed = value.trim(); + if trimmed == "0" { + None + } else { + Some(Duration::from_millis( + trimmed + .parse::() + .ok() + .filter(|value| *value > 0) + .unwrap_or(DEFAULT_PYTHON_EXECUTION_TIMEOUT_MS), + )) + } + } + None => Some(Duration::from_millis(DEFAULT_PYTHON_EXECUTION_TIMEOUT_MS)), + } +} + +fn python_max_old_space_mb(request: &StartPythonExecutionRequest) -> usize { + request + .env + .get(PYTHON_MAX_OLD_SPACE_MB_ENV) + .and_then(|value| value.trim().parse::().ok()) + .filter(|value| *value > 0) + .unwrap_or(DEFAULT_PYTHON_MAX_OLD_SPACE_MB) +} + +fn python_vfs_rpc_timeout(request: &StartPythonExecutionRequest) -> Duration { + Duration::from_millis( + request + .env + .get(PYTHON_VFS_RPC_TIMEOUT_MS_ENV) + .and_then(|value| value.trim().parse::().ok()) + .filter(|value| *value > 0) + .unwrap_or(DEFAULT_PYTHON_VFS_RPC_TIMEOUT_MS), + ) +} + +fn python_vfs_rpc_max_pending_requests(request: &StartPythonExecutionRequest) -> usize { + request + .env + .get(PYTHON_VFS_RPC_MAX_PENDING_REQUESTS_ENV) + .and_then(|value| value.trim().parse::().ok()) + .filter(|value| *value > 0) + .unwrap_or(DEFAULT_PYTHON_VFS_RPC_MAX_PENDING_REQUESTS) +} + +fn spawn_python_vfs_rpc_timeout( + id: u64, + timeout: Duration, + pending: Arc>>, + pending_count: Arc, + responses: Arc>>, +) { + thread::spawn(move || { + thread::sleep(timeout); + let should_timeout = match pending.lock() { + Ok(mut guard) if *guard == Some(PendingVfsRpcState::Pending(id)) => { + *guard = Some(PendingVfsRpcState::TimedOut(id)); + true + } + Ok(_) => false, + Err(_) => false, + }; + + if !should_timeout { + return; + } + + release_python_vfs_rpc_slot(pending_count.as_ref()); + let _ = write_python_vfs_rpc_response( + &responses, + json!({ + "id": id, + "ok": false, + "error": { + "code": "ERR_AGENT_OS_PYTHON_VFS_RPC_TIMEOUT", + "message": format!( + "guest Python VFS RPC request {id} timed out after {}ms", + timeout.as_millis() + ), + }, + }), + ); + }); +} + +fn spawn_python_waiter( + child: Arc>>, + stdout_reader: JoinHandle<()>, + stderr_reader: JoinHandle<()>, + sender: Sender, + exit_event: FE, + wait_error_event: FW, +) where + E: Send + 'static, + FE: Fn(i32) -> E + Send + 'static, + FW: Fn(String) -> E + Send + 'static, +{ + thread::spawn(move || loop { + let outcome = { + let mut child_slot = match child.lock() { + Ok(child_slot) => child_slot, + Err(_) => { + let _ = sender.send(wait_error_event(String::from( + "agent-os execution wait error: child lock poisoned\n", + ))); + return; + } + }; + let Some(child) = child_slot.as_mut() else { + return; + }; + + match child.try_wait() { + Ok(Some(status)) => { + let exit_code = status.code().unwrap_or(1); + *child_slot = None; + Some(Ok(exit_code)) + } + Ok(None) => None, + Err(err) => { + *child_slot = None; + Some(Err(err)) + } + } + }; + + match outcome { + Some(Ok(exit_code)) => { + let _ = stdout_reader.join(); + let _ = stderr_reader.join(); + let _ = sender.send(exit_event(exit_code)); + return; + } + Some(Err(err)) => { + let _ = sender.send(wait_error_event(format!( + "agent-os execution wait error: {err}\n" + ))); + return; + } + None => thread::sleep(Duration::from_millis(10)), + } + }); } fn create_node_child( @@ -550,11 +960,17 @@ fn create_node_child( context: &PythonContext, request: &StartPythonExecutionRequest, rpc_channels: PythonVfsRpcChannels, + control_fd: &OwnedFd, frozen_time_ms: u128, ) -> Result<(std::process::Child, File, Arc>>), PythonExecutionError> { let mut command = Command::new(node_binary()); + let mut exported_fds = ExportedChildFds::default(); configure_python_node_sandbox(&mut command, import_cache, context, request); command + .arg(format!( + "--max-old-space-size={}", + python_max_old_space_mb(request) + )) .arg("--no-warnings") .arg("--import") .arg(import_cache.timing_bootstrap_path()) @@ -571,12 +987,12 @@ fn create_node_child( .env(NODE_IMPORT_CACHE_PATH_ENV, import_cache.cache_path()) .env(PYTHON_CODE_ENV, &request.code) .env( - PYTHON_VFS_RPC_REQUEST_FD_ENV, - rpc_channels.child_request_writer.as_raw_fd().to_string(), - ) - .env( - PYTHON_VFS_RPC_RESPONSE_FD_ENV, - rpc_channels.child_response_reader.as_raw_fd().to_string(), + PYTHON_VFS_RPC_TIMEOUT_MS_ENV, + request + .env + .get(PYTHON_VFS_RPC_TIMEOUT_MS_ENV) + .cloned() + .unwrap_or_else(|| DEFAULT_PYTHON_VFS_RPC_TIMEOUT_MS.to_string()), ) .env(NODE_FROZEN_TIME_ENV, frozen_time_ms.to_string()); @@ -584,7 +1000,23 @@ fn create_node_child( command.env(PYTHON_FILE_ENV, file_path); } + exported_fds + .export( + &mut command, + PYTHON_VFS_RPC_REQUEST_FD_ENV, + &rpc_channels.child_request_writer, + ) + .map_err(|error| PythonExecutionError::RpcChannel(error.to_string()))?; + exported_fds + .export( + &mut command, + PYTHON_VFS_RPC_RESPONSE_FD_ENV, + &rpc_channels.child_response_reader, + ) + .map_err(|error| PythonExecutionError::RpcChannel(error.to_string()))?; apply_guest_env(&mut command, &request.env, RESERVED_PYTHON_ENV_KEYS); + configure_node_control_channel(&mut command, control_fd, &mut exported_fds) + .map_err(PythonExecutionError::Spawn)?; configure_node_command(&mut command, import_cache)?; let child = command.spawn().map_err(PythonExecutionError::Spawn)?; Ok(( @@ -600,11 +1032,8 @@ fn configure_python_node_sandbox( context: &PythonContext, request: &StartPythonExecutionRequest, ) { - let cache_root = import_cache - .cache_path() - .parent() - .unwrap_or(import_cache.asset_root()) - .to_path_buf(); + let sandbox_root = sandbox_root(&request.env, &request.cwd); + let cache_root = import_cache_root(import_cache, import_cache.asset_root()); let compile_cache_dir = import_cache.shared_compile_cache_dir(); let pyodide_dist_path = resolved_pyodide_dist_path(&context.pyodide_dist_path, &request.cwd); let read_paths = vec![ @@ -612,15 +1041,16 @@ fn configure_python_node_sandbox( compile_cache_dir.clone(), pyodide_dist_path, ]; - let write_paths = vec![cache_root, compile_cache_dir, request.cwd.clone()]; + let write_paths = vec![cache_root, compile_cache_dir, sandbox_root.clone()]; harden_node_command( command, - &request.cwd, + &sandbox_root, &read_paths, &write_paths, + true, false, - false, + true, false, ); } @@ -630,20 +1060,13 @@ fn configure_node_command( import_cache: &NodeImportCache, ) -> Result<(), PythonExecutionError> { let compile_cache_dir = import_cache.shared_compile_cache_dir(); - fs::create_dir_all(&compile_cache_dir).map_err(PythonExecutionError::PrepareWarmPath)?; - - command - .env_remove(NODE_DISABLE_COMPILE_CACHE_ENV) - .env(NODE_COMPILE_CACHE_ENV, compile_cache_dir); + configure_compile_cache(command, &compile_cache_dir) + .map_err(PythonExecutionError::PrepareWarmPath)?; Ok(()) } fn resolved_pyodide_dist_path(path: &Path, cwd: &Path) -> PathBuf { - if path.is_absolute() { - path.to_path_buf() - } else { - cwd.join(path) - } + resolve_execution_path(path, cwd) } fn prewarm_python_path( @@ -653,8 +1076,14 @@ fn prewarm_python_path( frozen_time_ms: u128, ) -> Result>, PythonExecutionError> { let debug_enabled = python_warmup_metrics_enabled(request); - let marker_path = warmup_marker_path(import_cache, context, request); - if marker_path.exists() && compile_cache_ready(import_cache) { + let marker_contents = warmup_marker_contents(import_cache, context, request); + let marker_path = warmup_marker_path( + import_cache.prewarm_marker_dir(), + "python-runner-prewarm", + PYTHON_WARMUP_MARKER_VERSION, + &marker_contents, + ); + if marker_path.exists() && compile_cache_ready(&import_cache.shared_compile_cache_dir()) { return Ok(warmup_metrics_line( debug_enabled, false, @@ -670,6 +1099,10 @@ fn prewarm_python_path( let mut command = Command::new(node_binary()); configure_python_node_sandbox(&mut command, import_cache, context, request); command + .arg(format!( + "--max-old-space-size={}", + python_max_old_space_mb(request) + )) .arg("--no-warnings") .arg("--import") .arg(import_cache.timing_bootstrap_path()) @@ -699,11 +1132,7 @@ fn prewarm_python_path( }); } - fs::write( - &marker_path, - warmup_marker_contents(import_cache, context, request), - ) - .map_err(PythonExecutionError::PrepareWarmPath)?; + fs::write(&marker_path, marker_contents).map_err(PythonExecutionError::PrepareWarmPath)?; Ok(warmup_metrics_line( debug_enabled, true, @@ -715,17 +1144,6 @@ fn prewarm_python_path( )) } -fn warmup_marker_path( - import_cache: &NodeImportCache, - context: &PythonContext, - request: &StartPythonExecutionRequest, -) -> PathBuf { - import_cache.prewarm_marker_dir().join(format!( - "python-runner-prewarm-v{PYTHON_WARMUP_MARKER_VERSION}-{:016x}.stamp", - stable_hash64(warmup_marker_contents(import_cache, context, request).as_bytes()), - )) -} - fn warmup_marker_contents( import_cache: &NodeImportCache, context: &PythonContext, @@ -750,19 +1168,8 @@ fn warmup_marker_contents( .join("\n") } -fn compile_cache_ready(import_cache: &NodeImportCache) -> bool { - let compile_cache_dir = import_cache.shared_compile_cache_dir(); - fs::read_dir(compile_cache_dir) - .ok() - .and_then(|mut entries| entries.next()) - .is_some() -} - fn python_warmup_metrics_enabled(request: &StartPythonExecutionRequest) -> bool { - request - .env - .get(PYTHON_WARMUP_DEBUG_ENV) - .is_some_and(|value| value == "1") + env_flag_enabled(&request.env, PYTHON_WARMUP_DEBUG_ENV) } fn warmup_metrics_line( @@ -793,31 +1200,12 @@ fn warmup_metrics_line( ) } -fn file_fingerprint(path: &Path) -> String { - match fs::metadata(path) { - Ok(metadata) => format!( - "{}:{}", - metadata.len(), - metadata - .modified() - .ok() - .and_then(|modified| modified.duration_since(UNIX_EPOCH).ok()) - .map(|duration| duration.as_millis().to_string()) - .unwrap_or_else(|| String::from("unknown")) - ), - Err(_) => String::from("missing"), - } -} - fn create_python_vfs_rpc_channels() -> Result { let (parent_request_reader, child_request_writer) = pipe2(OFlag::O_CLOEXEC) .map_err(|error| PythonExecutionError::RpcChannel(error.to_string()))?; let (child_response_reader, parent_response_writer) = pipe2(OFlag::O_CLOEXEC) .map_err(|error| PythonExecutionError::RpcChannel(error.to_string()))?; - clear_cloexec(&child_request_writer)?; - clear_cloexec(&child_response_reader)?; - Ok(PythonVfsRpcChannels { parent_request_reader: File::from(parent_request_reader), parent_response_writer: Arc::new(Mutex::new(BufWriter::new(File::from( @@ -828,15 +1216,41 @@ fn create_python_vfs_rpc_channels() -> Result Result<(), PythonExecutionError> { - fcntl(fd.as_raw_fd(), FcntlArg::F_SETFD(FdFlag::empty())) - .map_err(|error| PythonExecutionError::RpcChannel(error.to_string()))?; - Ok(()) +fn try_reserve_python_vfs_rpc_slot( + pending_count: &AtomicUsize, + max_pending_requests: usize, +) -> bool { + let mut current = pending_count.load(Ordering::Acquire); + + loop { + if current >= max_pending_requests { + return false; + } + + match pending_count.compare_exchange( + current, + current + 1, + Ordering::AcqRel, + Ordering::Acquire, + ) { + Ok(_) => return true, + Err(observed) => current = observed, + } + } +} + +fn release_python_vfs_rpc_slot(pending_count: &AtomicUsize) { + let _ = pending_count.fetch_update(Ordering::AcqRel, Ordering::Acquire, |current| { + current.checked_sub(1) + }); } fn spawn_python_vfs_rpc_reader( reader: File, - sender: Sender, + sender: Sender, + responses: Arc>>, + pending_count: Arc, + max_pending_requests: usize, ) -> JoinHandle<()> { thread::spawn(move || { let mut reader = BufReader::new(reader); @@ -854,16 +1268,36 @@ fn spawn_python_vfs_rpc_reader( match parse_python_vfs_rpc_request(trimmed) { Ok(request) => { + if !try_reserve_python_vfs_rpc_slot( + pending_count.as_ref(), + max_pending_requests, + ) { + let _ = write_python_vfs_rpc_response( + &responses, + json!({ + "id": request.id, + "ok": false, + "error": { + "code": "ERR_AGENT_OS_PYTHON_VFS_RPC_QUEUE_FULL", + "message": format!( + "guest Python VFS RPC queue exceeded configured limit of {max_pending_requests} pending requests" + ), + }, + }), + ); + continue; + } if sender - .send(PythonExecutionEvent::VfsRpcRequest(request)) + .send(PythonProcessEvent::VfsRpcRequest(request)) .is_err() { + release_python_vfs_rpc_slot(pending_count.as_ref()); return; } } Err(message) => { if sender - .send(PythonExecutionEvent::Stderr(message.into_bytes())) + .send(PythonProcessEvent::RawStderr(message.into_bytes())) .is_err() { return; @@ -872,7 +1306,7 @@ fn spawn_python_vfs_rpc_reader( } } Err(error) => { - let _ = sender.send(PythonExecutionEvent::Stderr( + let _ = sender.send(PythonProcessEvent::RawStderr( format!("agent-os python vfs rpc read error: {error}\n").into_bytes(), )); return; @@ -915,25 +1349,3 @@ fn write_python_vfs_rpc_response( .and_then(|()| writer.flush()) .map_err(|error| PythonExecutionError::RpcResponse(error.to_string())) } - -fn extract_python_exit_control(chunk: &[u8]) -> (Option, Vec) { - let text = String::from_utf8_lossy(chunk); - let mut filtered_lines = Vec::new(); - let mut exit_code = None; - - for line in text.lines() { - if let Some(value) = line.strip_prefix(PYTHON_EXIT_CONTROL_PREFIX) { - exit_code = value.trim().parse::().ok(); - continue; - } - filtered_lines.push(line); - } - - if filtered_lines.is_empty() { - return (exit_code, Vec::new()); - } - - let mut filtered = filtered_lines.join("\n").into_bytes(); - filtered.push(b'\n'); - (exit_code, filtered) -} diff --git a/crates/execution/src/runtime_support.rs b/crates/execution/src/runtime_support.rs new file mode 100644 index 000000000..06a1e3aef --- /dev/null +++ b/crates/execution/src/runtime_support.rs @@ -0,0 +1,98 @@ +use crate::common::stable_hash64; +use crate::node_import_cache::NodeImportCache; +use std::collections::BTreeMap; +use std::fs; +use std::io; +use std::os::unix::fs::MetadataExt; +use std::path::{Path, PathBuf}; +use std::process::Command; + +pub(crate) const NODE_COMPILE_CACHE_ENV: &str = "NODE_COMPILE_CACHE"; +pub(crate) const NODE_DISABLE_COMPILE_CACHE_ENV: &str = "NODE_DISABLE_COMPILE_CACHE"; +pub(crate) const NODE_FROZEN_TIME_ENV: &str = "AGENT_OS_FROZEN_TIME_MS"; +pub(crate) const NODE_SANDBOX_ROOT_ENV: &str = "AGENT_OS_SANDBOX_ROOT"; + +pub(crate) fn env_flag_enabled(env: &BTreeMap, key: &str) -> bool { + env.get(key).is_some_and(|value| value == "1") +} + +pub(crate) fn sandbox_root(env: &BTreeMap, cwd: &Path) -> PathBuf { + env.get(NODE_SANDBOX_ROOT_ENV) + .map(PathBuf::from) + .unwrap_or_else(|| cwd.to_path_buf()) +} + +pub(crate) fn import_cache_root(import_cache: &NodeImportCache, fallback: &Path) -> PathBuf { + import_cache + .cache_path() + .parent() + .unwrap_or(fallback) + .to_path_buf() +} + +pub(crate) fn configure_compile_cache( + command: &mut Command, + compile_cache_dir: &Path, +) -> Result<(), io::Error> { + fs::create_dir_all(compile_cache_dir)?; + command + .env_remove(NODE_DISABLE_COMPILE_CACHE_ENV) + .env(NODE_COMPILE_CACHE_ENV, compile_cache_dir); + Ok(()) +} + +pub(crate) fn compile_cache_ready(compile_cache_dir: &Path) -> bool { + fs::read_dir(compile_cache_dir) + .ok() + .and_then(|mut entries| entries.next()) + .is_some() +} + +pub(crate) fn resolve_execution_path(path: &Path, cwd: &Path) -> PathBuf { + if path.is_absolute() { + path.to_path_buf() + } else { + cwd.join(path) + } +} + +pub(crate) fn warmup_marker_path( + marker_dir: &Path, + prefix: &str, + version: &str, + contents: &str, +) -> PathBuf { + marker_dir.join(format!( + "{prefix}-v{version}-{:016x}.stamp", + stable_hash64(contents.as_bytes()) + )) +} + +pub(crate) fn file_fingerprint(path: &Path) -> String { + match fs::metadata(path) { + Ok(metadata) => format!("{}:{}", metadata.dev(), metadata.ino()), + Err(_) => String::from("missing"), + } +} + +#[cfg(test)] +mod tests { + use super::file_fingerprint; + use std::fs; + use std::os::unix::fs::MetadataExt; + use tempfile::tempdir; + + #[test] + fn file_fingerprint_uses_inode_identity() { + let temp = tempdir().expect("create temp dir"); + let path = temp.path().join("module.wasm"); + + fs::write(&path, b"first").expect("write wasm file"); + let metadata = fs::metadata(&path).expect("stat wasm file"); + + assert_eq!( + file_fingerprint(&path), + format!("{}:{}", metadata.dev(), metadata.ino()) + ); + } +} diff --git a/crates/execution/src/wasm.rs b/crates/execution/src/wasm.rs index 80d29d578..689f0e214 100644 --- a/crates/execution/src/wasm.rs +++ b/crates/execution/src/wasm.rs @@ -1,38 +1,109 @@ -use crate::common::{encode_json_string, frozen_time_ms, stable_hash64}; -use crate::node_import_cache::NodeImportCache; +use crate::common::{encode_json_string, frozen_time_ms}; +use crate::node_import_cache::{NodeImportCache, NodeImportCacheCleanup}; use crate::node_process::{ - apply_guest_env, encode_json_string_array, encode_json_string_map, harden_node_command, - node_binary, node_resolution_read_paths, resolve_path_like_specifier, spawn_stream_reader, - spawn_waiter, + apply_guest_env, configure_node_control_channel, create_node_control_channel, + encode_json_string_array, encode_json_string_map, env_builtin_enabled, harden_node_command, + node_binary, node_resolution_read_paths, resolve_path_like_specifier, + spawn_node_control_reader, spawn_stream_reader, ExportedChildFds, LinePrefixFilter, + NodeControlMessage, NodeSignalDispositionAction, NodeSignalHandlerRegistration, +}; +use crate::runtime_support::{ + configure_compile_cache, env_flag_enabled, file_fingerprint, import_cache_root, sandbox_root, + warmup_marker_path, NODE_COMPILE_CACHE_ENV, NODE_DISABLE_COMPILE_CACHE_ENV, + NODE_FROZEN_TIME_ENV, NODE_SANDBOX_ROOT_ENV, }; use std::collections::BTreeMap; use std::fmt; use std::fs; -use std::io::Write; +use std::io::{Read, Write}; use std::path::{Path, PathBuf}; -use std::process::{ChildStdin, Command, Stdio}; -use std::sync::mpsc::{self, Receiver, RecvTimeoutError}; -use std::time::{Duration, UNIX_EPOCH}; +use std::process::{Child, ChildStdin, Command, Stdio}; +use std::sync::{ + mpsc::{self, Receiver, RecvTimeoutError}, + Arc, Mutex, +}; +use std::thread::JoinHandle; +use std::time::Duration; const WASM_MODULE_PATH_ENV: &str = "AGENT_OS_WASM_MODULE_PATH"; const WASM_GUEST_ARGV_ENV: &str = "AGENT_OS_GUEST_ARGV"; const WASM_GUEST_ENV_ENV: &str = "AGENT_OS_GUEST_ENV"; +const WASM_PERMISSION_TIER_ENV: &str = "AGENT_OS_WASM_PERMISSION_TIER"; const WASM_PREWARM_ONLY_ENV: &str = "AGENT_OS_WASM_PREWARM_ONLY"; const WASM_WARMUP_DEBUG_ENV: &str = "AGENT_OS_WASM_WARMUP_DEBUG"; +pub const WASM_PREWARM_TIMEOUT_MS_ENV: &str = "AGENT_OS_WASM_PREWARM_TIMEOUT_MS"; +pub const WASM_MAX_FUEL_ENV: &str = "AGENT_OS_WASM_MAX_FUEL"; +pub const WASM_MAX_MEMORY_BYTES_ENV: &str = "AGENT_OS_WASM_MAX_MEMORY_BYTES"; +pub const WASM_MAX_STACK_BYTES_ENV: &str = "AGENT_OS_WASM_MAX_STACK_BYTES"; const WASM_WARMUP_METRICS_PREFIX: &str = "__AGENT_OS_WASM_WARMUP_METRICS__:"; -const NODE_COMPILE_CACHE_ENV: &str = "NODE_COMPILE_CACHE"; -const NODE_DISABLE_COMPILE_CACHE_ENV: &str = "NODE_DISABLE_COMPILE_CACHE"; -const NODE_FROZEN_TIME_ENV: &str = "AGENT_OS_FROZEN_TIME_MS"; const WASM_WARMUP_MARKER_VERSION: &str = "1"; +const SIGNAL_STATE_CONTROL_PREFIX: &str = "__AGENT_OS_SIGNAL_STATE__:"; +const CONTROLLED_STDERR_PREFIXES: &[&str] = &[SIGNAL_STATE_CONTROL_PREFIX]; const RESERVED_WASM_ENV_KEYS: &[&str] = &[ NODE_COMPILE_CACHE_ENV, NODE_DISABLE_COMPILE_CACHE_ENV, NODE_FROZEN_TIME_ENV, + NODE_SANDBOX_ROOT_ENV, + WASM_PERMISSION_TIER_ENV, WASM_GUEST_ARGV_ENV, WASM_GUEST_ENV_ENV, WASM_MODULE_PATH_ENV, + WASM_MAX_FUEL_ENV, + WASM_MAX_MEMORY_BYTES_ENV, + WASM_MAX_STACK_BYTES_ENV, + WASM_PREWARM_TIMEOUT_MS_ENV, WASM_PREWARM_ONLY_ENV, ]; +const WASM_PAGE_BYTES: u64 = 65_536; +const WASM_TIMEOUT_EXIT_CODE: i32 = 124; +const MAX_WASM_MODULE_FILE_BYTES: u64 = 256 * 1024 * 1024; +const MAX_WASM_IMPORT_SECTION_ENTRIES: usize = 16_384; +const MAX_WASM_MEMORY_SECTION_ENTRIES: usize = 1_024; +const MAX_WASM_VARUINT_BYTES: usize = 10; +const DEFAULT_WASM_PREWARM_TIMEOUT_MS: u64 = 30_000; +const WASM_MAX_MEM_PAGES_FLAG: &str = "--wasm-max-mem-pages="; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum WasmSignalDispositionAction { + Default, + Ignore, + User, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum WasmPermissionTier { + Full, + ReadWrite, + ReadOnly, + Isolated, +} + +impl WasmPermissionTier { + fn as_env_value(self) -> &'static str { + match self { + Self::Full => "full", + Self::ReadWrite => "read-write", + Self::ReadOnly => "read-only", + Self::Isolated => "isolated", + } + } + + fn workspace_write_enabled(self) -> bool { + matches!(self, Self::Full | Self::ReadWrite) + } + + fn wasi_enabled(self) -> bool { + !matches!(self, Self::Isolated) + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct WasmSignalHandlerRegistration { + pub action: WasmSignalDispositionAction, + pub mask: Vec, + pub flags: u32, +} #[derive(Debug, Clone, PartialEq, Eq)] pub struct CreateWasmContextRequest { @@ -54,12 +125,25 @@ pub struct StartWasmExecutionRequest { pub argv: Vec, pub env: BTreeMap, pub cwd: PathBuf, + pub permission_tier: WasmPermissionTier, } #[derive(Debug, Clone, PartialEq, Eq)] pub enum WasmExecutionEvent { Stdout(Vec), Stderr(Vec), + SignalState { + signal: u32, + registration: WasmSignalHandlerRegistration, + }, + Exited(i32), +} + +#[derive(Debug, Clone, PartialEq, Eq)] +enum WasmProcessEvent { + Stdout(Vec), + RawStderr(Vec), + Control(NodeControlMessage), Exited(i32), } @@ -71,14 +155,23 @@ pub struct WasmExecutionResult { pub stderr: Vec, } +#[derive(Debug, Clone, PartialEq, Eq)] +struct ResolvedWasmModule { + specifier: String, + resolved_path: PathBuf, +} + #[derive(Debug)] pub enum WasmExecutionError { MissingContext(String), VmMismatch { expected: String, found: String }, MissingModulePath, + InvalidLimit(String), + InvalidModule(String), MissingChildStream(&'static str), PrepareWarmPath(std::io::Error), WarmupSpawn(std::io::Error), + WarmupTimeout(Duration), WarmupFailed { exit_code: i32, stderr: String }, Spawn(std::io::Error), StdinClosed, @@ -101,6 +194,8 @@ impl fmt::Display for WasmExecutionError { Self::MissingModulePath => { f.write_str("guest WebAssembly execution requires a module path") } + Self::InvalidLimit(message) => write!(f, "invalid WebAssembly limit: {message}"), + Self::InvalidModule(message) => write!(f, "invalid WebAssembly module: {message}"), Self::MissingChildStream(name) => write!(f, "node child missing {name} pipe"), Self::PrepareWarmPath(err) => { write!(f, "failed to prepare shared WebAssembly warm path: {err}") @@ -108,6 +203,13 @@ impl fmt::Display for WasmExecutionError { Self::WarmupSpawn(err) => { write!(f, "failed to start WebAssembly warmup process: {err}") } + Self::WarmupTimeout(timeout) => { + write!( + f, + "WebAssembly warmup exceeded the configured timeout after {} ms", + timeout.as_millis() + ) + } Self::WarmupFailed { exit_code, stderr } => { if stderr.trim().is_empty() { write!(f, "WebAssembly warmup exited with status {exit_code}") @@ -136,7 +238,9 @@ pub struct WasmExecution { execution_id: String, child_pid: u32, stdin: Option, - events: Receiver, + events: Receiver, + stderr_filter: Arc>, + _import_cache_guard: Arc, } impl WasmExecution { @@ -168,7 +272,27 @@ impl WasmExecution { timeout: Duration, ) -> Result, WasmExecutionError> { match self.events.recv_timeout(timeout) { - Ok(event) => Ok(Some(event)), + Ok(WasmProcessEvent::Stdout(chunk)) => Ok(Some(WasmExecutionEvent::Stdout(chunk))), + Ok(WasmProcessEvent::RawStderr(chunk)) => { + let mut filter = self + .stderr_filter + .lock() + .map_err(|_| WasmExecutionError::EventChannelClosed)?; + let filtered = filter.filter_chunk(&chunk, CONTROLLED_STDERR_PREFIXES); + if filtered.is_empty() { + return Ok(None); + } + Ok(Some(WasmExecutionEvent::Stderr(filtered))) + } + Ok(WasmProcessEvent::Control(NodeControlMessage::SignalState { + signal, + registration, + })) => Ok(Some(WasmExecutionEvent::SignalState { + signal, + registration: registration.into(), + })), + Ok(WasmProcessEvent::Control(_)) => Ok(None), + Ok(WasmProcessEvent::Exited(code)) => Ok(Some(WasmExecutionEvent::Exited(code))), Err(RecvTimeoutError::Timeout) => Ok(None), Err(RecvTimeoutError::Disconnected) => Err(WasmExecutionError::EventChannelClosed), } @@ -182,9 +306,16 @@ impl WasmExecution { loop { match self.events.recv() { - Ok(WasmExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), - Ok(WasmExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), - Ok(WasmExecutionEvent::Exited(exit_code)) => { + Ok(WasmProcessEvent::Stdout(chunk)) => stdout.extend(chunk), + Ok(WasmProcessEvent::RawStderr(chunk)) => { + let mut filter = self + .stderr_filter + .lock() + .map_err(|_| WasmExecutionError::EventChannelClosed)?; + stderr.extend(filter.filter_chunk(&chunk, CONTROLLED_STDERR_PREFIXES)); + } + Ok(WasmProcessEvent::Control(_)) => {} + Ok(WasmProcessEvent::Exited(exit_code)) => { return Ok(WasmExecutionResult { execution_id: self.execution_id, exit_code, @@ -203,12 +334,13 @@ pub struct WasmExecutionEngine { next_context_id: usize, next_execution_id: usize, contexts: BTreeMap, - import_cache: NodeImportCache, + import_caches: BTreeMap, } impl WasmExecutionEngine { pub fn create_context(&mut self, request: CreateWasmContextRequest) -> WasmContext { self.next_context_id += 1; + self.import_caches.entry(request.vm_id.clone()).or_default(); let context = WasmContext { context_id: format!("wasm-ctx-{}", self.next_context_id), @@ -237,22 +369,41 @@ impl WasmExecutionEngine { }); } - self.import_cache - .ensure_materialized() - .map_err(WasmExecutionError::PrepareWarmPath)?; + let resolved_module = resolve_wasm_module(&context, &request)?; + let prewarm_timeout = resolve_wasm_prewarm_timeout(&request)?; + { + let import_cache = self.import_caches.entry(context.vm_id.clone()).or_default(); + import_cache + .ensure_materialized_with_timeout(prewarm_timeout) + .map_err(WasmExecutionError::PrepareWarmPath)?; + } let frozen_time_ms = frozen_time_ms(); - let warmup_metrics = - prewarm_wasm_path(&self.import_cache, &context, &request, frozen_time_ms)?; + validate_module_limits(&resolved_module, &request)?; + let execution_timeout = resolve_wasm_execution_timeout(&request)?; + let import_cache = self + .import_caches + .get(&context.vm_id) + .expect("vm import cache should exist after materialization"); + let import_cache_guard = import_cache.cleanup_guard(); + let warmup_metrics = prewarm_wasm_path( + import_cache, + &resolved_module, + &request, + frozen_time_ms, + prewarm_timeout, + )?; self.next_execution_id += 1; let execution_id = format!("exec-{}", self.next_execution_id); let guest_argv = guest_argv(&context, &request)?; + let control_channel = create_node_control_channel().map_err(WasmExecutionError::Spawn)?; let mut child = create_node_child( - &self.import_cache, - &context, + import_cache, + &resolved_module, &request, &guest_argv, frozen_time_ms, + &control_channel.child_writer, )?; let child_pid = child.id(); @@ -268,19 +419,24 @@ impl WasmExecutionEngine { let (sender, receiver) = mpsc::channel(); if let Some(metrics) = warmup_metrics { - let _ = sender.send(WasmExecutionEvent::Stderr(metrics)); + let _ = sender.send(WasmProcessEvent::RawStderr(metrics)); } - let stdout_reader = spawn_stream_reader(stdout, sender.clone(), WasmExecutionEvent::Stdout); - let stderr_reader = spawn_stream_reader(stderr, sender.clone(), WasmExecutionEvent::Stderr); - spawn_waiter( + let stdout_reader = spawn_stream_reader(stdout, sender.clone(), WasmProcessEvent::Stdout); + let stderr_reader = + spawn_stream_reader(stderr, sender.clone(), WasmProcessEvent::RawStderr); + let _control_reader = spawn_node_control_reader( + control_channel.parent_reader, + sender.clone(), + WasmProcessEvent::Control, + |message| WasmProcessEvent::RawStderr(message.into_bytes()), + ); + spawn_wasm_waiter( child, stdout_reader, stderr_reader, - true, + execution_timeout, sender, - WasmExecutionEvent::Exited, - |message| WasmExecutionEvent::Stderr(message.into_bytes()), ); Ok(WasmExecution { @@ -288,8 +444,15 @@ impl WasmExecutionEngine { child_pid, stdin, events: receiver, + stderr_filter: Arc::new(Mutex::new(LinePrefixFilter::default())), + _import_cache_guard: import_cache_guard, }) } + + pub fn dispose_vm(&mut self, vm_id: &str) { + self.contexts.retain(|_, context| context.vm_id != vm_id); + self.import_caches.remove(vm_id); + } } fn guest_argv( @@ -322,13 +485,15 @@ fn module_path( fn create_node_child( import_cache: &NodeImportCache, - context: &WasmContext, + resolved_module: &ResolvedWasmModule, request: &StartWasmExecutionRequest, guest_argv: &[String], frozen_time_ms: u128, + control_fd: &std::os::fd::OwnedFd, ) -> Result { let mut command = Command::new(node_binary()); - configure_wasm_node_sandbox(&mut command, import_cache, context, request)?; + let mut exported_fds = ExportedChildFds::default(); + configure_wasm_node_sandbox(&mut command, import_cache, resolved_module, request)?; command .arg("--no-warnings") .arg("--import") @@ -338,29 +503,42 @@ fn create_node_child( .stdin(Stdio::piped()) .stdout(Stdio::piped()) .stderr(Stdio::piped()) - .env(WASM_MODULE_PATH_ENV, module_path(context, request)?); + .env( + WASM_MODULE_PATH_ENV, + resolved_module.resolved_path.as_os_str(), + ); apply_guest_env(&mut command, &request.env, RESERVED_WASM_ENV_KEYS); command .env(WASM_GUEST_ARGV_ENV, encode_json_string_array(guest_argv)) - .env(WASM_GUEST_ENV_ENV, encode_json_string_map(&request.env)); + .env(WASM_GUEST_ENV_ENV, encode_json_string_map(&request.env)) + .env( + WASM_PERMISSION_TIER_ENV, + request.permission_tier.as_env_value(), + ); - configure_node_command(&mut command, import_cache, frozen_time_ms)?; + configure_node_control_channel(&mut command, control_fd, &mut exported_fds) + .map_err(WasmExecutionError::Spawn)?; + configure_node_command(&mut command, import_cache, frozen_time_ms, request)?; command.spawn().map_err(WasmExecutionError::Spawn) } fn prewarm_wasm_path( import_cache: &NodeImportCache, - context: &WasmContext, + resolved_module: &ResolvedWasmModule, request: &StartWasmExecutionRequest, frozen_time_ms: u128, + prewarm_timeout: Duration, ) -> Result>, WasmExecutionError> { - let debug_enabled = request - .env - .get(WASM_WARMUP_DEBUG_ENV) - .is_some_and(|value| value == "1"); - let marker_path = warmup_marker_path(import_cache, context, request); + let debug_enabled = env_flag_enabled(&request.env, WASM_WARMUP_DEBUG_ENV); + let marker_contents = warmup_marker_contents(resolved_module); + let marker_path = warmup_marker_path( + import_cache.prewarm_marker_dir(), + "wasm-runner-prewarm", + WASM_WARMUP_MARKER_VERSION, + &marker_contents, + ); if marker_path.exists() { return Ok(warmup_metrics_line( @@ -368,14 +546,13 @@ fn prewarm_wasm_path( false, "cached", import_cache, - context, - request, + &resolved_module.specifier, )); } - let guest_argv = guest_argv(context, request)?; + let guest_argv = warmup_guest_argv(resolved_module, request); let mut command = Command::new(node_binary()); - configure_wasm_node_sandbox(&mut command, import_cache, context, request)?; + configure_wasm_node_sandbox(&mut command, import_cache, resolved_module, request)?; command .arg("--no-warnings") .arg("--import") @@ -386,13 +563,20 @@ fn prewarm_wasm_path( .stdout(Stdio::null()) .stderr(Stdio::piped()) .env(WASM_PREWARM_ONLY_ENV, "1") - .env(WASM_MODULE_PATH_ENV, module_path(context, request)?) + .env( + WASM_MODULE_PATH_ENV, + resolved_module.resolved_path.as_os_str(), + ) .env(WASM_GUEST_ARGV_ENV, encode_json_string_array(&guest_argv)) - .env(WASM_GUEST_ENV_ENV, encode_json_string_map(&request.env)); + .env(WASM_GUEST_ENV_ENV, encode_json_string_map(&request.env)) + .env( + WASM_PERMISSION_TIER_ENV, + request.permission_tier.as_env_value(), + ); - configure_node_command(&mut command, import_cache, frozen_time_ms)?; + configure_node_command(&mut command, import_cache, frozen_time_ms, request)?; - let output = command.output().map_err(WasmExecutionError::WarmupSpawn)?; + let output = run_warmup_command(command, Some(prewarm_timeout))?; if !output.status.success() { return Err(WasmExecutionError::WarmupFailed { exit_code: output.status.code().unwrap_or(1), @@ -400,57 +584,55 @@ fn prewarm_wasm_path( }); } - fs::write(&marker_path, warmup_marker_contents(context, request)) - .map_err(WasmExecutionError::PrepareWarmPath)?; + fs::write(&marker_path, marker_contents).map_err(WasmExecutionError::PrepareWarmPath)?; Ok(warmup_metrics_line( debug_enabled, true, "executed", import_cache, - context, - request, + &resolved_module.specifier, )) } fn configure_wasm_node_sandbox( command: &mut Command, import_cache: &NodeImportCache, - context: &WasmContext, + resolved_module: &ResolvedWasmModule, request: &StartWasmExecutionRequest, ) -> Result<(), WasmExecutionError> { - let cache_root = import_cache - .cache_path() - .parent() - .unwrap_or(import_cache.prewarm_marker_dir()) - .to_path_buf(); + let sandbox_root = sandbox_root(&request.env, &request.cwd); + let cache_root = import_cache_root(import_cache, import_cache.prewarm_marker_dir()); let compile_cache_dir = import_cache.shared_compile_cache_dir(); let mut read_paths = vec![cache_root.clone(), compile_cache_dir.clone()]; - let write_paths = vec![cache_root, compile_cache_dir, request.cwd.clone()]; - - if let Some(module_path) = - resolve_path_like_specifier(&request.cwd, &module_path(context, request)?) - { - read_paths.push(module_path.clone()); - if let Some(parent) = module_path.parent() { - read_paths.push(parent.to_path_buf()); - } + let mut write_paths = vec![cache_root, compile_cache_dir]; + + if request.permission_tier.workspace_write_enabled() { + write_paths.push(sandbox_root.clone()); + } + + read_paths.push(resolved_module.resolved_path.clone()); + if let Some(parent) = resolved_module.resolved_path.parent() { + read_paths.push(parent.to_path_buf()); } read_paths.extend(node_resolution_read_paths( std::iter::once(request.cwd.clone()).chain( - resolve_path_like_specifier(&request.cwd, &module_path(context, request)?) - .and_then(|path| path.parent().map(Path::to_path_buf)), + resolved_module + .resolved_path + .parent() + .map(Path::to_path_buf), ), )); harden_node_command( command, - &request.cwd, + &sandbox_root, &read_paths, &write_paths, true, - true, + request.permission_tier.wasi_enabled(), + env_builtin_enabled(&request.env, "worker_threads"), false, ); Ok(()) @@ -460,39 +642,41 @@ fn configure_node_command( command: &mut Command, import_cache: &NodeImportCache, frozen_time_ms: u128, + request: &StartWasmExecutionRequest, ) -> Result<(), WasmExecutionError> { let compile_cache_dir = import_cache.shared_compile_cache_dir(); - fs::create_dir_all(&compile_cache_dir).map_err(WasmExecutionError::PrepareWarmPath)?; + configure_compile_cache(command, &compile_cache_dir) + .map_err(WasmExecutionError::PrepareWarmPath)?; - command - .env_remove(NODE_DISABLE_COMPILE_CACHE_ENV) - .env(NODE_COMPILE_CACHE_ENV, &compile_cache_dir) - .env(NODE_FROZEN_TIME_ENV, frozen_time_ms.to_string()); - Ok(()) -} + if let Some(stack_bytes) = wasm_stack_limit_bytes(request)? { + let stack_kib = (stack_bytes.saturating_add(1023) / 1024).max(64); + command.arg(format!("--stack-size={stack_kib}")); + } -fn warmup_marker_path( - import_cache: &NodeImportCache, - context: &WasmContext, - request: &StartWasmExecutionRequest, -) -> PathBuf { - import_cache.prewarm_marker_dir().join(format!( - "wasm-runner-prewarm-v{WASM_WARMUP_MARKER_VERSION}-{:016x}.stamp", - stable_hash64(warmup_marker_contents(context, request).as_bytes()), - )) + if let Some(memory_limit_bytes) = wasm_memory_limit_bytes(request)? { + let memory_limit_pages = wasm_memory_limit_pages(memory_limit_bytes)?; + command + .arg(format!("{WASM_MAX_MEM_PAGES_FLAG}{memory_limit_pages}")) + .env(WASM_MAX_MEMORY_BYTES_ENV, memory_limit_bytes.to_string()); + } + + if let Some(fuel_limit) = wasm_limit_u64(&request.env, WASM_MAX_FUEL_ENV)? { + command.env(WASM_MAX_FUEL_ENV, fuel_limit.to_string()); + } + + command.env(NODE_FROZEN_TIME_ENV, frozen_time_ms.to_string()); + Ok(()) } -fn warmup_marker_contents(context: &WasmContext, request: &StartWasmExecutionRequest) -> String { - let module_specifier = module_path(context, request).unwrap_or_default(); - let resolved_path = resolved_module_path(&module_specifier, &request.cwd); - let module_fingerprint = file_fingerprint(&resolved_path); +fn warmup_marker_contents(resolved_module: &ResolvedWasmModule) -> String { + let module_fingerprint = file_fingerprint(&resolved_module.resolved_path); [ env!("CARGO_PKG_NAME").to_string(), env!("CARGO_PKG_VERSION").to_string(), WASM_WARMUP_MARKER_VERSION.to_string(), - module_specifier, - resolved_path.display().to_string(), + resolved_module.specifier.clone(), + resolved_module.resolved_path.display().to_string(), module_fingerprint, ] .join("\n") @@ -503,52 +687,597 @@ fn warmup_metrics_line( executed: bool, reason: &str, import_cache: &NodeImportCache, - context: &WasmContext, - request: &StartWasmExecutionRequest, + module_specifier: &str, ) -> Option> { if !debug_enabled { return None; } - let module_specifier = module_path(context, request).ok()?; Some( format!( "{WASM_WARMUP_METRICS_PREFIX}{{\"executed\":{},\"reason\":{},\"modulePath\":{},\"compileCacheDir\":{}}}\n", if executed { "true" } else { "false" }, encode_json_string(reason), - encode_json_string(&module_specifier), + encode_json_string(module_specifier), encode_json_string(&import_cache.shared_compile_cache_dir().display().to_string()), ) .into_bytes(), ) } -fn resolved_module_path(specifier: &str, cwd: &Path) -> PathBuf { - if specifier.starts_with("file:") { - return PathBuf::from(specifier); +#[derive(Debug)] +struct WarmupOutput { + status: std::process::ExitStatus, + stderr: Vec, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ChildWaitError { + TimedOut, + WaitFailed, +} + +fn run_warmup_command( + mut command: Command, + timeout: Option, +) -> Result { + let mut child = command.spawn().map_err(WasmExecutionError::WarmupSpawn)?; + let Some(mut stderr) = child.stderr.take() else { + return Err(WasmExecutionError::MissingChildStream("stderr")); + }; + + let status = + wait_for_child_with_optional_timeout(&mut child, timeout).map_err(|timed_out| { + if timed_out == ChildWaitError::TimedOut { + WasmExecutionError::WarmupTimeout(timeout.expect("timeout should be present")) + } else { + WasmExecutionError::WarmupSpawn(std::io::Error::other( + "failed to wait for WebAssembly warmup child", + )) + } + })?; + + let mut stderr_bytes = Vec::new(); + let _ = stderr.read_to_end(&mut stderr_bytes); + Ok(WarmupOutput { + status, + stderr: stderr_bytes, + }) +} + +fn spawn_wasm_waiter( + mut child: Child, + stdout_reader: JoinHandle<()>, + stderr_reader: JoinHandle<()>, + timeout: Option, + sender: mpsc::Sender, +) { + std::thread::spawn(move || { + let wait_result = wait_for_child_with_optional_timeout(&mut child, timeout); + match wait_result { + Ok(status) => { + let exit_code = status.code().unwrap_or(1); + let _ = sender.send(WasmProcessEvent::Exited(exit_code)); + let _ = stdout_reader.join(); + let _ = stderr_reader.join(); + return; + } + Err(ChildWaitError::TimedOut) => { + let _ = sender.send(WasmProcessEvent::RawStderr( + b"WebAssembly fuel budget exhausted\n".to_vec(), + )); + let _ = sender.send(WasmProcessEvent::Exited(WASM_TIMEOUT_EXIT_CODE)); + let _ = stdout_reader.join(); + let _ = stderr_reader.join(); + return; + } + Err(ChildWaitError::WaitFailed) => { + let _ = sender.send(WasmProcessEvent::RawStderr( + b"agent-os execution wait error: failed to wait for WebAssembly child\n" + .to_vec(), + )); + let _ = sender.send(WasmProcessEvent::Exited(1)); + let _ = stdout_reader.join(); + let _ = stderr_reader.join(); + return; + } + } + }); +} + +fn wait_for_child_with_optional_timeout( + child: &mut Child, + timeout: Option, +) -> Result { + if timeout.is_none() { + return child.wait().map_err(|_| ChildWaitError::WaitFailed); + } + + let timeout = timeout.expect("timeout should be present"); + let deadline = std::time::Instant::now() + timeout; + loop { + match child.try_wait() { + Ok(Some(status)) => return Ok(status), + Ok(None) => { + if std::time::Instant::now() >= deadline { + let _ = child.kill(); + let _ = child.wait(); + return Err(ChildWaitError::TimedOut); + } + std::thread::sleep(Duration::from_millis(1)); + } + Err(_) => return Err(ChildWaitError::WaitFailed), + } } - if is_path_like(specifier) { - return cwd.join(specifier); +} + +fn resolve_wasm_execution_timeout( + request: &StartWasmExecutionRequest, +) -> Result, WasmExecutionError> { + // Node's WASI runtime does not expose per-instruction fuel metering, so the + // configured "fuel" budget is currently enforced as a tight wall-clock + // timeout while still being passed through to the child process for + // observability and future in-runtime enforcement. + Ok(wasm_limit_u64(&request.env, WASM_MAX_FUEL_ENV)?.map(Duration::from_millis)) +} + +fn resolve_wasm_prewarm_timeout( + request: &StartWasmExecutionRequest, +) -> Result { + Ok(Duration::from_millis( + wasm_limit_u64(&request.env, WASM_PREWARM_TIMEOUT_MS_ENV)? + .unwrap_or(DEFAULT_WASM_PREWARM_TIMEOUT_MS), + )) +} + +fn resolve_wasm_module( + context: &WasmContext, + request: &StartWasmExecutionRequest, +) -> Result { + let specifier = module_path(context, request)?; + let resolved_path = resolved_module_path(&specifier, &request.cwd); + Ok(ResolvedWasmModule { + specifier, + resolved_path, + }) +} + +fn resolved_module_path(specifier: &str, cwd: &Path) -> PathBuf { + resolve_path_like_specifier(cwd, specifier) + .map(|path| path.canonicalize().unwrap_or(path)) + .unwrap_or_else(|| PathBuf::from(specifier)) +} + +fn warmup_guest_argv( + resolved_module: &ResolvedWasmModule, + request: &StartWasmExecutionRequest, +) -> Vec { + if !request.argv.is_empty() { + return request.argv.clone(); } - PathBuf::from(specifier) + + vec![resolved_module.specifier.clone()] } -fn is_path_like(specifier: &str) -> bool { - specifier.starts_with('.') || specifier.starts_with('/') || specifier.starts_with("file:") +fn wasm_stack_limit_bytes( + request: &StartWasmExecutionRequest, +) -> Result, WasmExecutionError> { + wasm_limit_usize(&request.env, WASM_MAX_STACK_BYTES_ENV) } -fn file_fingerprint(path: &Path) -> String { - match fs::metadata(path) { - Ok(metadata) => format!( - "{}:{}", +fn wasm_memory_limit_bytes( + request: &StartWasmExecutionRequest, +) -> Result, WasmExecutionError> { + wasm_limit_u64(&request.env, WASM_MAX_MEMORY_BYTES_ENV) +} + +fn wasm_memory_limit_pages(memory_limit_bytes: u64) -> Result { + let pages = memory_limit_bytes / WASM_PAGE_BYTES; + u32::try_from(pages).map_err(|_| { + WasmExecutionError::InvalidLimit(format!( + "{WASM_MAX_MEMORY_BYTES_ENV}={memory_limit_bytes}: exceeds V8's wasm page limit range" + )) + }) +} + +fn wasm_limit_u64( + env: &BTreeMap, + key: &str, +) -> Result, WasmExecutionError> { + let Some(value) = env.get(key) else { + return Ok(None); + }; + value + .parse::() + .map(Some) + .map_err(|error| WasmExecutionError::InvalidLimit(format!("{key}={value}: {error}"))) +} + +fn wasm_limit_usize( + env: &BTreeMap, + key: &str, +) -> Result, WasmExecutionError> { + let Some(value) = env.get(key) else { + return Ok(None); + }; + value + .parse::() + .map(Some) + .map_err(|error| WasmExecutionError::InvalidLimit(format!("{key}={value}: {error}"))) +} + +fn validate_module_limits( + resolved_module: &ResolvedWasmModule, + request: &StartWasmExecutionRequest, +) -> Result<(), WasmExecutionError> { + let Some(memory_limit) = wasm_memory_limit_bytes(request)? else { + return Ok(()); + }; + + let resolved_path = &resolved_module.resolved_path; + let metadata = fs::metadata(&resolved_path).map_err(|error| { + WasmExecutionError::InvalidModule(format!( + "failed to stat {}: {error}", + resolved_path.display() + )) + })?; + if metadata.len() > MAX_WASM_MODULE_FILE_BYTES { + return Err(WasmExecutionError::InvalidModule(format!( + "module file size of {} bytes exceeds the configured parser cap of {} bytes", metadata.len(), - metadata - .modified() - .ok() - .and_then(|modified| modified.duration_since(UNIX_EPOCH).ok()) - .map(|duration| duration.as_millis().to_string()) - .unwrap_or_else(|| String::from("unknown")) - ), - Err(_) => String::from("missing"), + MAX_WASM_MODULE_FILE_BYTES + ))); + } + let bytes = fs::read(&resolved_path).map_err(|error| { + WasmExecutionError::InvalidModule(format!( + "failed to read {}: {error}", + resolved_path.display() + )) + })?; + let module_limits = extract_wasm_module_limits(&bytes)?; + + if module_limits.imports_memory { + return Err(WasmExecutionError::InvalidModule(String::from( + "configured WebAssembly memory limit does not support imported memories yet", + ))); + } + + if let Some(initial_bytes) = module_limits.initial_memory_bytes { + if initial_bytes > memory_limit { + return Err(WasmExecutionError::InvalidModule(format!( + "initial WebAssembly memory of {initial_bytes} bytes exceeds the configured limit of {memory_limit} bytes" + ))); + } + } + + match module_limits.maximum_memory_bytes { + Some(maximum_bytes) if maximum_bytes > memory_limit => Err(WasmExecutionError::InvalidModule( + format!( + "WebAssembly memory maximum of {maximum_bytes} bytes exceeds the configured limit of {memory_limit} bytes" + ), + )), + Some(_) => Ok(()), + None => Ok(()), + } +} + +#[derive(Debug, Default)] +struct WasmModuleLimits { + imports_memory: bool, + initial_memory_bytes: Option, + maximum_memory_bytes: Option, +} + +fn extract_wasm_module_limits(bytes: &[u8]) -> Result { + if bytes.len() < 8 || &bytes[..4] != b"\0asm" { + return Err(WasmExecutionError::InvalidModule(String::from( + "module is not a valid WebAssembly binary", + ))); + } + + let mut offset = 8; + let mut limits = WasmModuleLimits::default(); + + while offset < bytes.len() { + let section_id = bytes[offset]; + offset += 1; + let section_size = read_varuint_usize(bytes, &mut offset, "section size")?; + let section_end = offset.checked_add(section_size).ok_or_else(|| { + WasmExecutionError::InvalidModule(String::from("section size overflow")) + })?; + if section_end > bytes.len() { + return Err(WasmExecutionError::InvalidModule(String::from( + "section extends past end of module", + ))); + } + + match section_id { + 2 => { + let mut cursor = offset; + let import_count = read_varuint_usize(bytes, &mut cursor, "import count")?; + if import_count > MAX_WASM_IMPORT_SECTION_ENTRIES { + return Err(WasmExecutionError::InvalidModule(format!( + "import section contains {import_count} entries, which exceeds the parser cap of {MAX_WASM_IMPORT_SECTION_ENTRIES}" + ))); + } + for _ in 0..import_count { + skip_name(bytes, &mut cursor)?; + skip_name(bytes, &mut cursor)?; + let kind = read_byte(bytes, &mut cursor)?; + match kind { + 0x02 => { + let _ = read_memory_limits(bytes, &mut cursor)?; + limits.imports_memory = true; + } + 0x00 => { + let _ = read_varuint(bytes, &mut cursor)?; + } + 0x01 => { + skip_table_type(bytes, &mut cursor)?; + } + 0x03 => { + let _ = read_byte(bytes, &mut cursor)?; + let _ = read_byte(bytes, &mut cursor)?; + } + other => { + return Err(WasmExecutionError::InvalidModule(format!( + "unsupported import kind {other}" + ))); + } + } + } + } + 5 => { + let mut cursor = offset; + let memory_count = read_varuint_usize(bytes, &mut cursor, "memory count")?; + if memory_count > MAX_WASM_MEMORY_SECTION_ENTRIES { + return Err(WasmExecutionError::InvalidModule(format!( + "memory section contains {memory_count} entries, which exceeds the parser cap of {MAX_WASM_MEMORY_SECTION_ENTRIES}" + ))); + } + if memory_count > 0 { + let (initial_pages, maximum_pages) = read_memory_limits(bytes, &mut cursor)?; + limits.initial_memory_bytes = + Some(initial_pages.saturating_mul(WASM_PAGE_BYTES)); + limits.maximum_memory_bytes = + maximum_pages.map(|pages| pages.saturating_mul(WASM_PAGE_BYTES)); + } + } + _ => {} + } + + offset = section_end; + } + + Ok(limits) +} + +fn read_memory_limits( + bytes: &[u8], + offset: &mut usize, +) -> Result<(u64, Option), WasmExecutionError> { + let flags = read_varuint(bytes, offset)?; + let initial = read_varuint(bytes, offset)?; + let maximum = if flags & 0x01 != 0 { + Some(read_varuint(bytes, offset)?) + } else { + None + }; + Ok((initial, maximum)) +} + +fn skip_name(bytes: &[u8], offset: &mut usize) -> Result<(), WasmExecutionError> { + let length = read_varuint_usize(bytes, offset, "name length")?; + let end = offset + .checked_add(length) + .ok_or_else(|| WasmExecutionError::InvalidModule(String::from("name length overflow")))?; + if end > bytes.len() { + return Err(WasmExecutionError::InvalidModule(String::from( + "name extends past end of module", + ))); + } + *offset = end; + Ok(()) +} + +fn skip_table_type(bytes: &[u8], offset: &mut usize) -> Result<(), WasmExecutionError> { + let _ = read_byte(bytes, offset)?; + let flags = read_varuint(bytes, offset)?; + let _ = read_varuint(bytes, offset)?; + if flags & 0x01 != 0 { + let _ = read_varuint(bytes, offset)?; + } + Ok(()) +} + +fn read_byte(bytes: &[u8], offset: &mut usize) -> Result { + let Some(byte) = bytes.get(*offset).copied() else { + return Err(WasmExecutionError::InvalidModule(String::from( + "unexpected end of module", + ))); + }; + *offset += 1; + Ok(byte) +} + +fn read_varuint(bytes: &[u8], offset: &mut usize) -> Result { + let mut shift = 0_u32; + let mut value = 0_u64; + let mut encoded_bytes = 0_usize; + + loop { + let byte = read_byte(bytes, offset)?; + encoded_bytes += 1; + if encoded_bytes > MAX_WASM_VARUINT_BYTES { + return Err(WasmExecutionError::InvalidModule(format!( + "varuint exceeds the parser cap of {MAX_WASM_VARUINT_BYTES} bytes" + ))); + } + value |= u64::from(byte & 0x7f) << shift; + if byte & 0x80 == 0 { + return Ok(value); + } + if encoded_bytes == MAX_WASM_VARUINT_BYTES { + return Err(WasmExecutionError::InvalidModule(format!( + "varuint exceeds the parser cap of {MAX_WASM_VARUINT_BYTES} bytes" + ))); + } + shift = shift.saturating_add(7); + if shift >= 64 { + return Err(WasmExecutionError::InvalidModule(String::from( + "varuint is too large", + ))); + } + } +} + +fn read_varuint_usize( + bytes: &[u8], + offset: &mut usize, + label: &str, +) -> Result { + let value = read_varuint(bytes, offset)?; + usize::try_from(value).map_err(|_| { + WasmExecutionError::InvalidModule(format!( + "{label} of {value} exceeds platform usize range" + )) + }) +} + +impl From for WasmSignalDispositionAction { + fn from(value: NodeSignalDispositionAction) -> Self { + match value { + NodeSignalDispositionAction::Default => Self::Default, + NodeSignalDispositionAction::Ignore => Self::Ignore, + NodeSignalDispositionAction::User => Self::User, + } + } +} + +impl From for WasmSignalHandlerRegistration { + fn from(value: NodeSignalHandlerRegistration) -> Self { + Self { + action: value.action.into(), + mask: value.mask, + flags: value.flags, + } + } +} + +#[cfg(test)] +mod tests { + use super::{ + resolve_wasm_execution_timeout, resolve_wasm_prewarm_timeout, resolved_module_path, + wasm_memory_limit_pages, StartWasmExecutionRequest, WasmPermissionTier, WASM_MAX_FUEL_ENV, + WASM_MAX_MEMORY_BYTES_ENV, WASM_PAGE_BYTES, WASM_PREWARM_TIMEOUT_MS_ENV, + }; + use std::collections::BTreeMap; + use std::fs; + use std::os::unix::fs::symlink; + use std::path::Path; + use std::time::Duration; + use tempfile::tempdir; + + fn request_with_env(cwd: &Path, env: BTreeMap) -> StartWasmExecutionRequest { + StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: String::from("ctx-wasm"), + argv: Vec::new(), + env, + cwd: cwd.to_path_buf(), + permission_tier: WasmPermissionTier::Full, + } + } + + #[test] + fn resolved_module_path_canonicalizes_path_like_specifiers() { + let temp = tempdir().expect("create temp dir"); + let real = temp.path().join("real.wasm"); + let alias = temp.path().join("alias.wasm"); + fs::write(&real, b"\0asm\x01\0\0\0").expect("write wasm file"); + symlink(&real, &alias).expect("create wasm symlink"); + + let resolved = resolved_module_path("./alias.wasm", temp.path()); + + assert_eq!( + resolved, + real.canonicalize().expect("canonicalize wasm target") + ); + } + + #[test] + fn wasm_prewarm_timeout_is_separate_from_execution_timeout() { + let temp = tempdir().expect("create temp dir"); + let request = request_with_env( + temp.path(), + BTreeMap::from([ + (String::from(WASM_MAX_FUEL_ENV), String::from("25")), + ( + String::from(WASM_PREWARM_TIMEOUT_MS_ENV), + String::from("750"), + ), + ]), + ); + + assert_eq!( + resolve_wasm_execution_timeout(&request).expect("execution timeout"), + Some(Duration::from_millis(25)) + ); + assert_eq!( + resolve_wasm_prewarm_timeout(&request).expect("prewarm timeout"), + Duration::from_millis(750) + ); + } + + #[test] + fn wasm_memory_limit_pages_floor_to_whole_wasm_pages() { + assert_eq!( + wasm_memory_limit_pages(WASM_PAGE_BYTES + 123).expect("page limit"), + 1 + ); + assert_eq!( + wasm_memory_limit_pages(2 * WASM_PAGE_BYTES).expect("page limit"), + 2 + ); + } + + #[test] + fn wasm_memory_limit_no_longer_requires_declared_module_maximum() { + let temp = tempdir().expect("create temp dir"); + let request = request_with_env( + temp.path(), + BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + (2 * WASM_PAGE_BYTES).to_string(), + )]), + ); + + assert!( + super::validate_module_limits( + &super::ResolvedWasmModule { + specifier: String::from("./guest.wasm"), + resolved_path: { + let path = temp.path().join("guest.wasm"); + fs::write( + &path, + wat::parse_str( + r#" +(module + (memory (export "memory") 1) + (func (export "_start")) +) +"#, + ) + .expect("compile wasm fixture"), + ) + .expect("write wasm fixture"); + path + }, + }, + &request, + ) + .is_ok(), + "runtime memory cap should allow modules without a declared maximum" + ); } } diff --git a/crates/execution/tests/javascript.rs b/crates/execution/tests/javascript.rs index bbc69d832..55cd744ba 100644 --- a/crates/execution/tests/javascript.rs +++ b/crates/execution/tests/javascript.rs @@ -2,6 +2,7 @@ use agent_os_execution::{ CreateJavascriptContextRequest, JavascriptExecutionEngine, JavascriptExecutionEvent, StartJavascriptExecutionRequest, }; +use serde_json::{json, Value}; use std::collections::BTreeMap; use std::fs; use std::path::{Path, PathBuf}; @@ -222,7 +223,7 @@ for await (const chunk of process.stdin) { input += chunk; } -console.log(`stdout:${process.env.AGENT_OS_TEST_ENV}:${input}`); +console.log(`stdout:${process.env.VISIBLE_TEST_ENV}:${input}`); console.error(`stderr:${process.argv.slice(2).join(",")}`); "#, ); @@ -243,7 +244,7 @@ console.error(`stderr:${process.argv.slice(2).join(",")}`); String::from("alpha"), String::from("beta"), ], - env: BTreeMap::from([(String::from("AGENT_OS_TEST_ENV"), String::from("ok"))]), + env: BTreeMap::from([(String::from("VISIBLE_TEST_ENV"), String::from("ok"))]), cwd: temp.path().to_path_buf(), }) .expect("start JavaScript execution"); @@ -266,6 +267,10 @@ console.error(`stderr:${process.argv.slice(2).join(",")}`); { Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + panic!("unexpected sync RPC request: {}", request.method); + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), None => panic!("timed out waiting for JavaScript execution event"), } @@ -339,6 +344,10 @@ process.stdin.on("end", () => { { Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), Some(JavascriptExecutionEvent::Stderr(_chunk)) => {} + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + panic!("unexpected sync RPC request: {}", request.method); + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), None => panic!("timed out waiting for JavaScript execution event"), } @@ -350,6 +359,736 @@ process.stdin.on("end", () => { .contains("stdin:still-open")); } +#[test] +fn javascript_execution_surfaces_shared_array_buffer_sync_rpc_requests() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import fs from "node:fs"; + +const stat = fs.statSync("/workspace/note.txt"); +const lstat = fs.lstatSync("/workspace/link.txt"); +const contents = fs.readFileSync("/workspace/note.txt", { encoding: "utf8" }); +const raw = Buffer.from(fs.readFileSync("/workspace/raw.bin")).toString("hex"); +const entries = fs.readdirSync("/workspace"); +const missing = fs.existsSync("/workspace/missing.txt"); + +fs.mkdirSync("/workspace/subdir", { recursive: true }); +fs.writeFileSync("/workspace/out.bin", Buffer.from([1, 2, 3, 4])); +fs.symlinkSync("/workspace/note.txt", "/workspace/link.txt"); +const linkTarget = fs.readlinkSync("/workspace/link.txt"); +fs.linkSync("/workspace/note.txt", "/workspace/hard.txt"); +fs.renameSync("/workspace/hard.txt", "/workspace/renamed.txt"); +fs.unlinkSync("/workspace/renamed.txt"); +fs.rmdirSync("/workspace/subdir"); + +console.log(JSON.stringify({ stat, lstat, contents, raw, entries, missing, linkTarget })); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_NODE_SYNC_RPC_ENABLE"), + String::from("1"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut exit_code = None; + let mut requests = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + requests.push((request.method.clone(), request.args.clone())); + match request.method.as_str() { + "fs.statSync" => execution + .respond_sync_rpc_success( + request.id, + json!({ + "mode": 0o100644, + "size": 14, + "isDirectory": false, + "isSymbolicLink": false, + }), + ) + .expect("respond to stat"), + "fs.lstatSync" => execution + .respond_sync_rpc_success( + request.id, + json!({ + "mode": 0o120777, + "size": 19, + "isDirectory": false, + "isSymbolicLink": true, + }), + ) + .expect("respond to lstat"), + "fs.readFileSync" => { + let path = request.args[0].as_str().expect("read path"); + let result = match path { + "/workspace/note.txt" => json!("hello from rpc"), + "/workspace/raw.bin" => json!({ + "__agentOsType": "bytes", + "base64": "q80=", + }), + other => panic!("unexpected read path: {other}"), + }; + execution + .respond_sync_rpc_success(request.id, result) + .expect("respond to read"); + } + "fs.existsSync" => execution + .respond_sync_rpc_success(request.id, json!(false)) + .expect("respond to exists"), + "fs.readdirSync" => execution + .respond_sync_rpc_success(request.id, json!(["note.txt", "raw.bin"])) + .expect("respond to readdir"), + "fs.mkdirSync" => execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to mkdir"), + "fs.writeFileSync" => { + assert_eq!(request.args[0], json!("/workspace/out.bin")); + assert_eq!( + request.args[1], + json!({ + "__agentOsType": "bytes", + "base64": "AQIDBA==", + }) + ); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to write"); + } + "fs.symlinkSync" => { + assert_eq!(request.args[0], json!("/workspace/note.txt")); + assert_eq!(request.args[1], json!("/workspace/link.txt")); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to symlink"); + } + "fs.readlinkSync" => execution + .respond_sync_rpc_success(request.id, json!("/workspace/note.txt")) + .expect("respond to readlink"), + "fs.linkSync" => { + assert_eq!(request.args[0], json!("/workspace/note.txt")); + assert_eq!(request.args[1], json!("/workspace/hard.txt")); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to link"); + } + "fs.renameSync" => { + assert_eq!(request.args[0], json!("/workspace/hard.txt")); + assert_eq!(request.args[1], json!("/workspace/renamed.txt")); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to rename"); + } + "fs.unlinkSync" => execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to unlink"), + "fs.rmdirSync" => execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to rmdir"), + other => panic!("unexpected sync RPC method: {other}"), + } + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + assert_eq!(exit_code, Some(0)); + assert_eq!( + requests + .iter() + .map(|(method, _)| method.as_str()) + .collect::>(), + vec![ + "fs.statSync", + "fs.lstatSync", + "fs.readFileSync", + "fs.readFileSync", + "fs.readdirSync", + "fs.existsSync", + "fs.mkdirSync", + "fs.writeFileSync", + "fs.symlinkSync", + "fs.readlinkSync", + "fs.linkSync", + "fs.renameSync", + "fs.unlinkSync", + "fs.rmdirSync", + ] + ); + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + assert!( + stdout.contains("\"contents\":\"hello from rpc\""), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"raw\":\"abcd\""), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"entries\":[\"note.txt\",\"raw.bin\"]"), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"size\":14"), + "unexpected stdout: {stdout}" + ); +} + +#[test] +fn javascript_execution_routes_fs_promises_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import fs from "node:fs/promises"; + +await fs.access("./note.txt"); +const contents = await fs.readFile("./note.txt", "utf8"); +const stat = await fs.stat("./note.txt"); +const lstat = await fs.lstat("./note.txt"); +const entries = await fs.readdir("."); +await fs.mkdir("./subdir", { recursive: true }); +await fs.writeFile("./out.bin", Buffer.from([1, 2, 3, 4])); +await fs.copyFile("./note.txt", "./copied.txt"); +await fs.rename("./copied.txt", "./renamed.txt"); +await fs.chmod("./renamed.txt", 0o600); +await fs.chown("./renamed.txt", 1000, 1001); +await fs.utimes("./renamed.txt", new Date(1000), new Date(2000)); +await fs.unlink("./out.bin"); +await fs.rmdir("./subdir"); + +console.log( + JSON.stringify({ + contents, + entries, + isDir: stat.isDirectory(), + isSymlink: lstat.isSymbolicLink(), + size: stat.size, + }), +); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::new(), + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut exit_code = None; + let mut requests = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + requests.push((request.method.clone(), request.args.clone())); + match request.method.as_str() { + "fs.promises.access" => execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to access"), + "fs.promises.readFile" => execution + .respond_sync_rpc_success(request.id, json!("hello from promises rpc")) + .expect("respond to readFile"), + "fs.promises.stat" => execution + .respond_sync_rpc_success( + request.id, + json!({ + "mode": 0o100644, + "size": 23, + "isDirectory": false, + "isSymbolicLink": false, + }), + ) + .expect("respond to stat"), + "fs.promises.lstat" => execution + .respond_sync_rpc_success( + request.id, + json!({ + "mode": 0o100644, + "size": 23, + "isDirectory": false, + "isSymbolicLink": true, + }), + ) + .expect("respond to lstat"), + "fs.promises.readdir" => execution + .respond_sync_rpc_success(request.id, json!(["note.txt", "raw.bin"])) + .expect("respond to readdir"), + "fs.promises.mkdir" + | "fs.promises.copyFile" + | "fs.promises.rename" + | "fs.promises.chmod" + | "fs.promises.chown" + | "fs.promises.utimes" + | "fs.promises.unlink" + | "fs.promises.rmdir" => execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to async fs mutation"), + "fs.promises.writeFile" => { + assert_eq!(request.args[0], json!("/out.bin")); + assert_eq!( + request.args[1], + json!({ + "__agentOsType": "bytes", + "base64": "AQIDBA==", + }) + ); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to writeFile"); + } + other => panic!("unexpected async fs RPC method: {other}"), + } + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + assert_eq!(exit_code, Some(0)); + assert_eq!( + requests + .iter() + .map(|(method, _)| method.as_str()) + .collect::>(), + vec![ + "fs.promises.access", + "fs.promises.readFile", + "fs.promises.stat", + "fs.promises.lstat", + "fs.promises.readdir", + "fs.promises.mkdir", + "fs.promises.writeFile", + "fs.promises.copyFile", + "fs.promises.rename", + "fs.promises.chmod", + "fs.promises.chown", + "fs.promises.utimes", + "fs.promises.unlink", + "fs.promises.rmdir", + ] + ); + + assert_eq!(requests[0].1[0], json!("/note.txt")); + assert_eq!( + requests[1].1, + vec![json!("/note.txt"), json!({ "encoding": "utf8" })] + ); + assert_eq!( + requests[5].1, + vec![json!("/subdir"), json!({ "recursive": true })] + ); + assert_eq!( + requests[7].1, + vec![json!("/note.txt"), json!("/copied.txt"), Value::Null] + ); + assert_eq!( + requests[11].1, + vec![json!("/renamed.txt"), json!(1000), json!(2000)] + ); + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + assert!( + stdout.contains("\"contents\":\"hello from promises rpc\""), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"isDir\":false"), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"isSymlink\":true"), + "unexpected stdout: {stdout}" + ); + assert!( + stdout.contains("\"entries\":[\"note.txt\",\"raw.bin\"]"), + "unexpected stdout: {stdout}" + ); +} + +#[test] +fn javascript_execution_routes_fd_fs_and_streams_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import fs from "node:fs"; +import { once } from "node:events"; + +const fd = fs.openSync("/workspace/data.txt", "r"); +const stat = fs.fstatSync(fd); +const buffer = Buffer.alloc(5); +const bytesRead = fs.readSync(fd, buffer, 0, buffer.length, 1); +fs.closeSync(fd); + +const fdOut = fs.openSync("/workspace/out.txt", "w"); +const written = fs.writeSync(fdOut, Buffer.from("hello"), 0, 5, 0); +fs.closeSync(fdOut); + +const asyncSummary = await new Promise((resolve, reject) => { + fs.open("/workspace/async.txt", "r", (openError, asyncFd) => { + if (openError) { + reject(openError); + return; + } + + const target = Buffer.alloc(5); + fs.read(asyncFd, target, 0, 5, 0, (readError, asyncBytesRead) => { + if (readError) { + reject(readError); + return; + } + + fs.fstat(asyncFd, (statError, asyncStat) => { + if (statError) { + reject(statError); + return; + } + + fs.close(asyncFd, (closeError) => { + if (closeError) { + reject(closeError); + return; + } + + resolve({ + asyncBytesRead, + asyncText: target.toString("utf8"), + asyncSize: asyncStat.size, + }); + }); + }); + }); + }); +}); + +const callbackWrite = await new Promise((resolve, reject) => { + fs.open("/workspace/callback-out.txt", "w", (openError, callbackFd) => { + if (openError) { + reject(openError); + return; + } + + fs.write(callbackFd, "done", 0, "utf8", (writeError, callbackBytesWritten) => { + if (writeError) { + reject(writeError); + return; + } + + fs.close(callbackFd, (closeError) => { + if (closeError) { + reject(closeError); + return; + } + + resolve(callbackBytesWritten); + }); + }); + }); +}); + +const reader = fs.createReadStream("/workspace/stream.txt", { + encoding: "utf8", + start: 0, + end: 9, + highWaterMark: 4, +}); +const streamChunks = []; +reader.on("data", (chunk) => streamChunks.push(chunk)); +await once(reader, "close"); + +const writer = fs.createWriteStream("/workspace/stream-out.txt", { start: 0 }); +writer.write("ab"); +writer.end("cd"); +await once(writer, "close"); + +let watchMessage = ""; +let watchFileMessage = ""; +try { + fs.watch("/workspace/data.txt"); +} catch (error) { + watchMessage = `${error.code}:${error.message}`; +} +try { + fs.watchFile("/workspace/data.txt", () => {}); +} catch (error) { + watchFileMessage = `${error.code}:${error.message}`; +} + +console.log( + JSON.stringify({ + text: buffer.toString("utf8"), + bytesRead, + size: stat.size, + written, + asyncSummary, + callbackWrite, + streamChunks, + watchMessage, + watchFileMessage, + }), +); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::new(), + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let files = BTreeMap::from([ + (String::from("/workspace/async.txt"), b"async".to_vec()), + (String::from("/workspace/data.txt"), b"abcdef".to_vec()), + ( + String::from("/workspace/stream.txt"), + b"streamdata".to_vec(), + ), + ]); + let mut fd_paths = BTreeMap::::new(); + let mut next_fd = 40_u64; + let mut stdout = Vec::new(); + let mut exit_code = None; + let mut requests = Vec::new(); + let mut writes = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + requests.push(request.method.clone()); + match request.method.as_str() { + "fs.open" | "fs.openSync" => { + let fd = next_fd; + next_fd += 1; + fd_paths + .insert(fd, request.args[0].as_str().expect("open path").to_string()); + execution + .respond_sync_rpc_success(request.id, json!(fd)) + .expect("respond to open"); + } + "fs.fstat" | "fs.fstatSync" => { + let fd = request.args[0].as_u64().expect("fstat fd"); + let path = fd_paths.get(&fd).expect("tracked fd path"); + let size = files.get(path).map_or(0, |contents| contents.len()); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "mode": 0o100644, + "size": size, + "isDirectory": false, + "isSymbolicLink": false, + }), + ) + .expect("respond to fstat"); + } + "fs.read" | "fs.readSync" => { + let fd = request.args[0].as_u64().expect("read fd"); + let length = request.args[1].as_u64().expect("read length") as usize; + let position = request.args[2].as_u64().expect("read position") as usize; + let path = fd_paths.get(&fd).expect("tracked read fd"); + let contents = files.get(path).expect("read file contents"); + let end = (position + length).min(contents.len()); + let text = String::from_utf8_lossy(&contents[position..end]).to_string(); + execution + .respond_sync_rpc_success(request.id, json!(text)) + .expect("respond to read"); + } + "fs.write" | "fs.writeSync" => { + let fd = request.args[0].as_u64().expect("write fd"); + let path = fd_paths.get(&fd).expect("tracked write fd").clone(); + let payload = if let Some(text) = request.args[1].as_str() { + text.to_string() + } else { + request.args[1] + .get("base64") + .and_then(Value::as_str) + .expect("buffer write payload") + .to_string() + }; + let position = request.args.get(2).and_then(Value::as_u64); + writes.push((path, payload.clone(), position)); + let bytes_written = match payload.as_str() { + "done" => 4, + "aGVsbG8=" => 5, + "YWI=" => 2, + "Y2Q=" => 2, + other => panic!("unexpected write payload: {other}"), + }; + execution + .respond_sync_rpc_success(request.id, json!(bytes_written)) + .expect("respond to write"); + } + "fs.close" | "fs.closeSync" => { + let fd = request.args[0].as_u64().expect("close fd"); + fd_paths.remove(&fd); + execution + .respond_sync_rpc_success(request.id, json!(null)) + .expect("respond to close"); + } + other => panic!("unexpected fd RPC method: {other}"), + } + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + assert_eq!(exit_code, Some(0)); + assert_eq!( + requests, + vec![ + "fs.openSync", + "fs.fstatSync", + "fs.readSync", + "fs.closeSync", + "fs.openSync", + "fs.writeSync", + "fs.closeSync", + "fs.open", + "fs.read", + "fs.fstat", + "fs.close", + "fs.open", + "fs.write", + "fs.close", + "fs.open", + "fs.read", + "fs.read", + "fs.read", + "fs.close", + "fs.open", + "fs.write", + "fs.write", + "fs.close", + ] + ); + assert_eq!( + writes, + vec![ + ( + String::from("/workspace/out.txt"), + String::from("aGVsbG8="), + Some(0), + ), + ( + String::from("/workspace/callback-out.txt"), + String::from("done"), + Some(0), + ), + ( + String::from("/workspace/stream-out.txt"), + String::from("YWI="), + Some(0), + ), + ( + String::from("/workspace/stream-out.txt"), + String::from("Y2Q="), + Some(2), + ), + ] + ); + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + assert!(stdout.contains("\"text\":\"bcdef\""), "stdout: {stdout}"); + assert!(stdout.contains("\"bytesRead\":5"), "stdout: {stdout}"); + assert!(stdout.contains("\"size\":6"), "stdout: {stdout}"); + assert!(stdout.contains("\"written\":5"), "stdout: {stdout}"); + assert!(stdout.contains("\"asyncBytesRead\":5"), "stdout: {stdout}"); + assert!( + stdout.contains("\"asyncText\":\"async\""), + "stdout: {stdout}" + ); + assert!(stdout.contains("\"asyncSize\":5"), "stdout: {stdout}"); + assert!(stdout.contains("\"callbackWrite\":4"), "stdout: {stdout}"); + assert!( + stdout.contains("\"streamChunks\":[\"stre\",\"amda\",\"ta\"]"), + "stdout: {stdout}" + ); + assert!( + stdout.contains("ERR_AGENT_OS_FS_WATCH_UNAVAILABLE"), + "stdout: {stdout}" + ); + assert!( + stdout.contains("kernel has no file-watching API"), + "stdout: {stdout}" + ); +} + #[test] fn javascript_execution_ignores_guest_overrides_for_internal_node_env() { assert_node_available(); @@ -362,6 +1101,10 @@ console.log(`entrypoint:${process.argv[1]}`); console.log(`args:${process.argv.slice(2).join(",")}`); console.log(`node-options:${process.env.NODE_OPTIONS ?? "missing"}`); console.log(`loader-path:${process.env.AGENT_OS_NODE_IMPORT_CACHE_LOADER_PATH ?? "missing"}`); +console.log(`loader-visible:${'AGENT_OS_NODE_IMPORT_CACHE_LOADER_PATH' in process.env}`); +console.log( + `internal-keys:${Object.keys(process.env).filter((key) => key.startsWith("AGENT_OS_")).length}`, +); "#, ); write_fixture( @@ -405,6 +1148,9 @@ console.log("evil override executed"); ); assert!(stdout.contains("args:safe-arg"), "stdout: {stdout}"); assert!(stdout.contains("node-options:missing"), "stdout: {stdout}"); + assert!(stdout.contains("loader-path:missing"), "stdout: {stdout}"); + assert!(stdout.contains("loader-visible:false"), "stdout: {stdout}"); + assert!(stdout.contains("internal-keys:0"), "stdout: {stdout}"); assert!( !stdout.contains("evil override executed"), "stdout: {stdout}" @@ -1308,15 +2054,2810 @@ console.log(`missing:${missing}`); format!("[{{\"guestPath\":\"/guest\",\"hostPath\":\"{guest_mount_host_path}\"}}]"), )]); - let (stdout, _stderr, exit_code) = run_javascript_execution( - &mut engine, - context.context_id, - temp.path(), - vec![String::from("./entry.mjs")], - env, + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut exit_code = None; + let mut requests = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + requests.push((request.method.clone(), request.args.clone())); + match request.method.as_str() { + "fs.readFileSync" => execution + .respond_sync_rpc_success(request.id, json!("mapped\n")) + .expect("respond to readFileSync"), + "fs.existsSync" => execution + .respond_sync_rpc_success(request.id, json!(false)) + .expect("respond to existsSync"), + other => panic!("unexpected sync RPC method: {other}"), + } + } + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + assert_eq!(exit_code, Some(0)); + assert_eq!( + requests + .iter() + .map(|(method, _)| method.as_str()) + .collect::>(), + vec!["fs.readFileSync", "fs.existsSync"] + ); + assert_eq!( + requests[0].1, + vec![json!("/guest/flag.txt"), json!({"encoding":"utf8"})] ); + assert_eq!(requests[1].1, vec![json!("/guest/missing.txt")]); - assert_eq!(exit_code, 0); + let stdout = String::from_utf8(stdout).expect("stdout utf8"); assert!(stdout.contains("text:mapped")); assert!(stdout.contains("missing:false")); } + +#[test] +fn javascript_execution_virtualizes_process_cwd_and_denies_chdir() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const result = { + cwd: process.cwd(), +}; + +try { + process.chdir("/other"); + result.chdir = "unexpected"; +} catch (error) { + result.chdir = { + code: error.code ?? null, + message: error.message, + }; +} + +result.cwdAfter = process.cwd(); +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + )]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse cwd JSON"); + assert_eq!(parsed["cwd"], Value::String(String::from("/root"))); + assert_eq!(parsed["cwdAfter"], Value::String(String::from("/root"))); + assert_eq!( + parsed["chdir"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["chdir"]["message"] + .as_str() + .expect("chdir message") + .contains("process.chdir")); +} + +#[test] +fn javascript_execution_uses_virtual_root_when_no_guest_path_mapping_exists() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("dep.cjs"), + "module.exports = { answer: 42 };\n", + ); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const result = { + cwd: process.cwd(), + resolved: require.resolve('./dep.cjs'), +}; + +try { + require.resolve('./missing.cjs'); + result.resolveMissing = 'unexpected'; +} catch (error) { + result.resolveMissing = { + message: error.message, + stack: error.stack ?? null, + }; +} + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + BTreeMap::new(), + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse cwd fallback JSON"); + let host_path = temp.path().to_string_lossy(); + + assert_eq!(parsed["cwd"], Value::String(String::from("/root"))); + assert_eq!( + parsed["resolved"], + Value::String(String::from("/root/dep.cjs")) + ); + let message = parsed["resolveMissing"]["message"] + .as_str() + .expect("missing resolve message"); + let stack = parsed["resolveMissing"]["stack"] + .as_str() + .expect("missing resolve stack"); + assert!( + message.contains("/root/missing.cjs"), + "message should use virtual cwd fallback: {message}" + ); + assert!( + stack.contains("/root/entry.mjs"), + "stack should use virtual cwd fallback: {stack}" + ); + assert!( + !message.contains(host_path.as_ref()), + "message leaked host path: {message}" + ); + assert!( + !stack.contains(host_path.as_ref()), + "stack leaked host path: {stack}" + ); +} + +#[test] +fn javascript_execution_virtualizes_process_identity() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const result = { + execPath: process.execPath, + argv0: process.argv[0], + pid: process.pid, + ppid: process.ppid, + uid: typeof process.getuid === "function" ? process.getuid() : null, + gid: typeof process.getgid === "function" ? process.getgid() : null, +}; + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([ + ( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH"), + String::from("/usr/bin/node"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_PID"), + String::from("41"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_PPID"), + String::from("7"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_UID"), + String::from("0"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_GID"), + String::from("0"), + ), + ]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse process identity JSON"); + assert_eq!( + parsed["execPath"], + Value::String(String::from("/usr/bin/node")) + ); + assert_eq!( + parsed["argv0"], + Value::String(String::from("/usr/bin/node")) + ); + assert_eq!(parsed["pid"], Value::from(41)); + assert_eq!(parsed["ppid"], Value::from(7)); + assert_eq!(parsed["uid"], Value::from(0)); + assert_eq!(parsed["gid"], Value::from(0)); +} + +#[test] +fn javascript_execution_blocks_remaining_process_property_leaks() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +function summarize(mod) { + return { + platform: mod.platform, + arch: mod.arch, + version: mod.version, + release: mod.release, + config: mod.config, + versions: mod.versions, + memoryUsage: typeof mod.memoryUsage === "function" ? mod.memoryUsage() : null, + memoryUsageRss: + typeof mod.memoryUsage === "function" && typeof mod.memoryUsage.rss === "function" + ? mod.memoryUsage.rss() + : null, + uptime: typeof mod.uptime === "function" ? mod.uptime() : null, + }; +} + +const result = { + globalProcess: summarize(process), + requireProcess: summarize(require("node:process")), + builtinProcess: summarize(process.getBuiltinModule("node:process")), +}; + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let env = BTreeMap::from([ + ( + String::from("AGENT_OS_VIRTUAL_OS_ARCH"), + String::from("arm64"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_VERSION"), + String::from("v24.0.0"), + ), + ]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse process leak JSON"); + for key in ["globalProcess", "requireProcess", "builtinProcess"] { + let summary = &parsed[key]; + assert_eq!(summary["platform"], Value::String(String::from("linux"))); + assert_eq!(summary["arch"], Value::String(String::from("arm64"))); + assert_eq!(summary["version"], Value::String(String::from("v24.0.0"))); + assert_eq!( + summary["release"]["name"], + Value::String(String::from("node")) + ); + assert_eq!( + summary["release"]["lts"], + Value::String(String::from("Agent OS")) + ); + assert!(summary["release"]["sourceUrl"].is_null()); + assert!(summary["release"]["headersUrl"].is_null()); + assert_eq!( + summary["config"]["variables"]["host_arch"], + Value::String(String::from("arm64")) + ); + assert_eq!( + summary["config"]["variables"]["node_shared"], + Value::Bool(false) + ); + assert_eq!( + summary["config"]["variables"]["node_use_openssl"], + Value::Bool(false) + ); + assert_eq!( + summary["versions"]["node"], + Value::String(String::from("24.0.0")) + ); + assert_eq!( + summary["versions"]["openssl"], + Value::String(String::from("0.0.0")) + ); + assert_eq!( + summary["versions"]["v8"], + Value::String(String::from("0.0")) + ); + assert_eq!( + summary["versions"]["zlib"], + Value::String(String::from("0.0.0")) + ); + + let memory_usage = summary["memoryUsage"] + .as_object() + .expect("memory usage object"); + for field in ["rss", "heapTotal", "heapUsed", "external", "arrayBuffers"] { + assert!( + memory_usage[field].as_u64().unwrap_or_default() > 0 + || field == "external" + || field == "arrayBuffers" + ); + } + assert_eq!( + summary["memoryUsageRss"], summary["memoryUsage"]["rss"], + "memoryUsage.rss() should match memoryUsage().rss for {key}" + ); + let uptime = summary["uptime"].as_f64().expect("uptime number"); + assert!(uptime >= 0.0, "uptime should not be negative for {key}"); + assert!( + uptime < 5.0, + "uptime should be VM-scoped for {key}, got {uptime}" + ); + } +} + +#[test] +fn javascript_execution_virtualizes_os_module() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import os from "node:os"; + +function summarize(mod) { + return { + hostname: mod.hostname(), + cpus: mod.cpus(), + totalmem: mod.totalmem(), + freemem: mod.freemem(), + homedir: mod.homedir(), + tmpdir: mod.tmpdir(), + platform: mod.platform(), + type: mod.type(), + release: mod.release(), + version: typeof mod.version === "function" ? mod.version() : null, + arch: typeof mod.arch === "function" ? mod.arch() : null, + machine: typeof mod.machine === "function" ? mod.machine() : null, + availableParallelism: + typeof mod.availableParallelism === "function" + ? mod.availableParallelism() + : null, + loadavg: typeof mod.loadavg === "function" ? mod.loadavg() : null, + uptime: typeof mod.uptime === "function" ? mod.uptime() : null, + networkInterfaces: mod.networkInterfaces(), + userInfo: mod.userInfo(), + userInfoBuffer: mod.userInfo({ encoding: "buffer" }), + getPriority: typeof mod.getPriority === "function" ? mod.getPriority(0) : null, + }; +} + +const result = { + importOs: summarize(os), + requireOs: summarize(require("node:os")), + builtinOs: summarize(process.getBuiltinModule("node:os")), +}; + +try { + os.setPriority(0, 0); + result.setPriority = "unexpected"; +} catch (error) { + result.setPriority = { + code: error.code ?? null, + message: error.message, + }; +} + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([ + ( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + ), + (String::from("HOME"), String::from("/root")), + (String::from("SHELL"), String::from("/bin/bash")), + (String::from("AGENT_OS_VIRTUAL_PROCESS_UID"), String::from("0")), + (String::from("AGENT_OS_VIRTUAL_PROCESS_GID"), String::from("0")), + ( + String::from("AGENT_OS_VIRTUAL_OS_HOSTNAME"), + String::from("agent-os-test"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_CPU_COUNT"), + String::from("4"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_CPU_MODEL"), + String::from("Agent OS Test CPU"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_TOTALMEM"), + String::from("2147483648"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_FREEMEM"), + String::from("1073741824"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_RELEASE"), + String::from("6.8.0-agent-os-test"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_VERSION"), + String::from("#1 SMP PREEMPT_DYNAMIC Agent OS Test"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_ARCH"), + String::from("x64"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_MACHINE"), + String::from("x86_64"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_USER"), + String::from("agent"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_SHELL"), + String::from("/bin/bash"), + ), + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"os\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + ), + ]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse os JSON"); + + for surface in ["importOs", "requireOs", "builtinOs"] { + assert_eq!( + parsed[surface]["hostname"], + Value::String(String::from("agent-os-test")) + ); + assert_eq!( + parsed[surface]["homedir"], + Value::String(String::from("/root")) + ); + assert_eq!( + parsed[surface]["tmpdir"], + Value::String(String::from("/tmp")) + ); + assert_eq!( + parsed[surface]["platform"], + Value::String(String::from("linux")) + ); + assert_eq!( + parsed[surface]["type"], + Value::String(String::from("Linux")) + ); + assert_eq!( + parsed[surface]["release"], + Value::String(String::from("6.8.0-agent-os-test")) + ); + assert_eq!( + parsed[surface]["version"], + Value::String(String::from("#1 SMP PREEMPT_DYNAMIC Agent OS Test")) + ); + assert_eq!(parsed[surface]["arch"], Value::String(String::from("x64"))); + assert_eq!( + parsed[surface]["machine"], + Value::String(String::from("x86_64")) + ); + assert_eq!(parsed[surface]["availableParallelism"], Value::from(4)); + assert_eq!(parsed[surface]["totalmem"], Value::from(2_147_483_648_u64)); + assert_eq!(parsed[surface]["freemem"], Value::from(1_073_741_824_u64)); + assert_eq!(parsed[surface]["loadavg"], json!([0, 0, 0])); + assert_eq!(parsed[surface]["uptime"], Value::from(0)); + assert_eq!(parsed[surface]["getPriority"], Value::from(0)); + assert_eq!(parsed[surface]["cpus"].as_array().map(Vec::len), Some(4)); + assert_eq!( + parsed[surface]["cpus"][0]["model"], + Value::String(String::from("Agent OS Test CPU")) + ); + assert_eq!( + parsed[surface]["userInfo"]["username"], + Value::String(String::from("agent")) + ); + assert_eq!(parsed[surface]["userInfo"]["uid"], Value::from(0)); + assert_eq!(parsed[surface]["userInfo"]["gid"], Value::from(0)); + assert_eq!( + parsed[surface]["userInfo"]["shell"], + Value::String(String::from("/bin/bash")) + ); + assert_eq!( + parsed[surface]["userInfo"]["homedir"], + Value::String(String::from("/root")) + ); + assert_eq!( + parsed[surface]["userInfoBuffer"]["username"]["type"], + Value::String(String::from("Buffer")) + ); + assert_eq!( + parsed[surface]["userInfoBuffer"]["shell"]["type"], + Value::String(String::from("Buffer")) + ); + + let interfaces = parsed[surface]["networkInterfaces"] + .as_object() + .expect("network interfaces object"); + assert_eq!(interfaces.len(), 1); + assert!(interfaces.contains_key("lo")); + let loopback = interfaces["lo"].as_array().expect("loopback interfaces"); + assert_eq!(loopback.len(), 2); + assert_eq!( + loopback[0]["address"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(loopback[0]["internal"], Value::Bool(true)); + assert_eq!(loopback[1]["address"], Value::String(String::from("::1"))); + } + + assert_eq!( + parsed["setPriority"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["setPriority"]["message"] + .as_str() + .expect("setPriority message") + .contains("os.setPriority")); +} + +#[test] +fn javascript_execution_os_module_safe_defaults_ignore_host_env() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import os from "node:os"; + +console.log(JSON.stringify({ + hostname: os.hostname(), + homedir: os.homedir(), + tmpdir: os.tmpdir(), + userInfo: os.userInfo(), +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([ + ( + String::from("HOME"), + String::from("/Users/host-user/should-not-leak"), + ), + ( + String::from("USER"), + String::from("host-user-should-not-leak"), + ), + ( + String::from("LOGNAME"), + String::from("host-logname-should-not-leak"), + ), + ( + String::from("TMPDIR"), + String::from("/var/folders/host-tmp-should-not-leak"), + ), + ( + String::from("TEMP"), + String::from("/tmp/host-temp-should-not-leak"), + ), + ( + String::from("TMP"), + String::from("/tmp/host-tmp-should-not-leak"), + ), + ( + String::from("HOSTNAME"), + String::from("host-machine-should-not-leak"), + ), + (String::from("SHELL"), String::from("/bin/zsh")), + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"os\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + ), + ]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse os defaults JSON"); + + assert_eq!(parsed["hostname"], Value::String(String::from("agent-os"))); + assert_eq!(parsed["homedir"], Value::String(String::from("/root"))); + assert_eq!(parsed["tmpdir"], Value::String(String::from("/tmp"))); + assert_eq!( + parsed["userInfo"]["username"], + Value::String(String::from("root")) + ); + assert_eq!( + parsed["userInfo"]["shell"], + Value::String(String::from("/bin/sh")) + ); + assert_eq!( + parsed["userInfo"]["homedir"], + Value::String(String::from("/root")) + ); +} + +#[test] +fn javascript_execution_denies_process_signal_handlers_and_native_addons() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture(&temp.path().join("addon.node"), "not-a-real-native-addon\n"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import { fileURLToPath } from 'node:url'; + +const addonPath = fileURLToPath(new URL('./addon.node', import.meta.url)); +const result = {}; + +try { + const returned = process.on('beforeExit', () => {}); + result.nonSignalReturnedSelf = returned === process; + process.removeAllListeners('beforeExit'); +} catch (error) { + result.nonSignal = { code: error.code ?? null, message: error.message }; +} + +try { + process.on('SIGTERM', () => {}); + result.signalOn = 'unexpected'; +} catch (error) { + result.signalOn = { code: error.code ?? null, message: error.message }; +} + +try { + process.once('SIGINT', () => {}); + result.signalOnce = 'unexpected'; +} catch (error) { + result.signalOnce = { code: error.code ?? null, message: error.message }; +} + +try { + const returned = process.on('SIGCHLD', () => {}); + result.sigchldReturnedSelf = returned === process; + process.removeAllListeners('SIGCHLD'); +} catch (error) { + result.sigchld = { code: error.code ?? null, message: error.message }; +} + +try { + process.dlopen({}, addonPath); + result.dlopen = 'unexpected'; +} catch (error) { + result.dlopen = { code: error.code ?? null, message: error.message }; +} + +try { + require(addonPath); + result.nativeAddon = 'unexpected'; +} catch (error) { + result.nativeAddon = { code: error.code ?? null, message: error.message }; +} + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + BTreeMap::new(), + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse hardening JSON"); + assert_eq!(parsed["nonSignalReturnedSelf"], Value::Bool(true)); + assert_eq!( + parsed["signalOn"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["signalOn"]["message"] + .as_str() + .expect("signal on message") + .contains("process.on(SIGTERM)")); + assert_eq!( + parsed["signalOnce"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["signalOnce"]["message"] + .as_str() + .expect("signal once message") + .contains("process.once(SIGINT)")); + assert_eq!(parsed.get("sigchld"), None); + assert_eq!( + parsed["dlopen"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["dlopen"]["message"] + .as_str() + .expect("dlopen message") + .contains("process.dlopen")); + assert_eq!( + parsed["nativeAddon"]["code"], + Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!(parsed["nativeAddon"]["message"] + .as_str() + .expect("native addon message") + .contains("native addon loading")); +} + +#[test] +fn javascript_execution_still_starts_with_fail_closed_property_hardening() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +console.log(JSON.stringify({ + envType: typeof process.env, + cwdType: typeof process.cwd, + execPathType: typeof process.execPath, +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + BTreeMap::new(), + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse hardening JSON"); + assert_eq!(parsed["envType"], Value::String(String::from("object"))); + assert_eq!(parsed["cwdType"], Value::String(String::from("function"))); + assert_eq!( + parsed["execPathType"], + Value::String(String::from("string")) + ); +} + +#[test] +fn javascript_execution_hardens_exec_and_execsync_child_process_calls() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const { exec, execSync } = require('node:child_process'); +const execAsync = (command) => + new Promise((resolve, reject) => { + exec(command, (error, stdout, stderr) => { + if (error) { + error.stdout = stdout; + error.stderr = stderr; + reject(error); + return; + } + + resolve({ stdout, stderr }); + }); + }); + +console.log(JSON.stringify({ + execSync: JSON.parse(execSync('node ./child.mjs sync', { encoding: 'utf8' }).trim()), + exec: JSON.parse((await execAsync('node ./child.mjs async')).stdout.trim()), +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([ + ( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + ), + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + ), + ]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut next_child_pid = 40_u64; + let mut child_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "child_process.spawn" => { + let payload = request.args[0].as_object().expect("spawn payload"); + let command = payload["command"].as_str().expect("spawn command"); + let args = payload["args"] + .as_array() + .expect("spawn args") + .iter() + .filter_map(Value::as_str) + .map(str::to_owned) + .collect::>(); + let shell = payload["options"]["shell"].as_bool().unwrap_or(false); + let marker = if shell { + command + .split_whitespace() + .last() + .expect("shell marker") + .to_owned() + } else { + args.last().expect("spawn marker").clone() + }; + let child_id = format!("child-{next_child_pid}"); + let stdout_payload = format!("{{\"marker\":\"{marker}\"}}\n"); + child_events.insert( + child_id.clone(), + vec![ + json!({ + "type": "stdout", + "data": stdout_payload, + }), + json!({ + "type": "exit", + "exitCode": 0, + }), + ], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "childId": child_id, + "pid": next_child_pid, + "command": command, + "args": args, + }), + ) + .expect("respond to child_process.spawn"); + next_child_pid += 1; + } + "child_process.poll" => { + let child_id = request.args[0].as_str().expect("poll child id"); + let next = child_events + .get_mut(child_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to child_process.poll"); + } + other => panic!("unexpected child_process sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse child_process JSON"); + assert_eq!( + parsed["execSync"]["marker"], + Value::String(String::from("sync")) + ); + assert_eq!( + parsed["exec"]["marker"], + Value::String(String::from("async")) + ); + assert!(methods.iter().any(|method| method == "child_process.spawn")); + assert!(methods.iter().any(|method| method == "child_process.poll")); +} + +#[test] +fn javascript_execution_strips_internal_env_from_child_process_rpc_payloads() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const { spawnSync } = require('node:child_process'); + +spawnSync('node', ['./child.mjs'], { + env: { + VISIBLE_MARKER: 'child-visible', + AGENT_OS_GUEST_PATH_MAPPINGS: 'user-override', + AGENT_OS_VIRTUAL_PROCESS_UID: '999', + AGENT_OS_VIRTUAL_OS_HOSTNAME: 'leak-attempt', + }, +}); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([ + ( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + ), + (String::from("VISIBLE_MARKER"), String::from("parent-visible")), + (String::from("AGENT_OS_VIRTUAL_PROCESS_UID"), String::from("0")), + ( + String::from("AGENT_OS_VIRTUAL_OS_HOSTNAME"), + String::from("agent-os-test"), + ), + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + ), + ]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut observed_env = None; + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(_chunk)) => {} + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + match request.method.as_str() { + "child_process.spawn" => { + let payload = request.args[0].as_object().expect("spawn payload"); + observed_env = Some( + payload["options"]["env"] + .as_object() + .expect("spawn env") + .clone(), + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "childId": "child-1", + "pid": 41, + "command": payload["command"], + "args": payload["args"], + }), + ) + .expect("respond to child_process.spawn"); + } + "child_process.poll" => { + execution + .respond_sync_rpc_success( + request.id, + json!({ + "type": "exit", + "exitCode": 0, + }), + ) + .expect("respond to child_process.poll"); + } + other => panic!("unexpected child_process sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let env = observed_env.expect("observed child env"); + assert_eq!( + env.get("VISIBLE_MARKER"), + Some(&Value::String(String::from("child-visible"))) + ); + assert!(!env.contains_key("AGENT_OS_GUEST_PATH_MAPPINGS")); + assert!(!env.contains_key("AGENT_OS_VIRTUAL_PROCESS_UID")); + assert!(!env.contains_key("AGENT_OS_VIRTUAL_OS_HOSTNAME")); +} + +#[test] +fn javascript_execution_routes_net_connect_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import net from "node:net"; + +const summary = await new Promise((resolve, reject) => { + const socket = net.createConnection({ host: "127.0.0.1", port: 43199 }); + let data = ""; + let ended = false; + socket.setEncoding("utf8"); + socket.on("connect", () => { + socket.write("ping"); + }); + socket.on("data", (chunk) => { + data += chunk; + }); + socket.on("end", () => { + ended = true; + }); + socket.on("error", reject); + socket.on("close", (hadError) => { + resolve({ + data, + ended, + hadError, + localPort: socket.localPort, + remoteAddress: socket.remoteAddress, + remotePort: socket.remotePort, + }); + }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut socket_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "net.connect" => { + socket_events.insert( + String::from("socket-1"), + vec![ + json!({ + "type": "data", + "data": "pong", + }), + json!({ + "type": "end", + }), + json!({ + "type": "close", + "hadError": false, + }), + ], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "socketId": "socket-1", + "localAddress": "127.0.0.1", + "localPort": 42001, + "remoteAddress": "127.0.0.1", + "remotePort": 43199, + "remoteFamily": "IPv4", + }), + ) + .expect("respond to net.connect"); + } + "net.write" => { + assert_eq!( + request.args[0].as_str(), + Some("socket-1"), + "unexpected socket id for write", + ); + execution + .respond_sync_rpc_success(request.id, json!(4)) + .expect("respond to net.write"); + } + "net.shutdown" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.shutdown"); + } + "net.destroy" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.destroy"); + } + "net.poll" => { + let socket_id = request.args[0].as_str().expect("poll socket id"); + let next = socket_events + .get_mut(socket_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.poll"); + } + other => panic!("unexpected net sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse net JSON"); + assert_eq!(parsed["data"], Value::String(String::from("pong"))); + assert_eq!(parsed["ended"], Value::Bool(true)); + assert_eq!(parsed["hadError"], Value::Bool(false)); + assert_eq!( + parsed["remoteAddress"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["remotePort"], Value::from(43199)); + assert!(methods.iter().any(|method| method == "net.connect")); + assert!(methods.iter().any(|method| method == "net.write")); + assert!(methods.iter().any(|method| method == "net.shutdown")); + assert!(methods.iter().any(|method| method == "net.poll")); +} + +#[test] +fn javascript_execution_routes_net_create_server_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import net from "node:net"; + +const summary = await new Promise((resolve, reject) => { + const server = net.createServer({ allowHalfOpen: false }, (socket) => { + let data = ""; + let connections = -1; + socket.setEncoding("utf8"); + socket.on("data", (chunk) => { + data += chunk; + server.getConnections((error, count) => { + if (error) { + reject(error); + return; + } + connections = count; + socket.end("pong"); + }); + }); + socket.on("error", reject); + socket.on("close", () => { + server.close(() => { + resolve({ + address: server.address(), + connections, + data, + localPort: socket.localPort, + remoteAddress: socket.remoteAddress, + remotePort: socket.remotePort, + }); + }); + }); + }); + server.on("error", reject); + server.listen({ port: 43111, host: "127.0.0.1", backlog: 2 }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut listener_events = BTreeMap::>::new(); + let mut socket_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "net.listen" => { + assert_eq!(request.args[0]["backlog"], Value::from(2)); + listener_events.insert( + String::from("listener-1"), + vec![json!({ + "type": "connection", + "socketId": "socket-1", + "localAddress": "127.0.0.1", + "localPort": 43111, + "remoteAddress": "127.0.0.1", + "remotePort": 54000, + "remoteFamily": "IPv4", + })], + ); + socket_events.insert( + String::from("socket-1"), + vec![ + json!({ + "type": "data", + "data": "ping", + }), + json!({ + "type": "end", + }), + json!({ + "type": "close", + "hadError": false, + }), + ], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "serverId": "listener-1", + "localAddress": "127.0.0.1", + "localPort": 43111, + "family": "IPv4", + }), + ) + .expect("respond to net.listen"); + } + "net.server_poll" => { + let listener_id = request.args[0].as_str().expect("poll listener id"); + let next = listener_events + .get_mut(listener_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.server_poll"); + } + "net.server_connections" => { + execution + .respond_sync_rpc_success(request.id, json!(1)) + .expect("respond to net.server_connections"); + } + "net.poll" => { + let socket_id = request.args[0].as_str().expect("poll socket id"); + let next = socket_events + .get_mut(socket_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.poll"); + } + "net.write" => { + assert_eq!(request.args[0].as_str(), Some("socket-1")); + execution + .respond_sync_rpc_success(request.id, json!(4)) + .expect("respond to net.write"); + } + "net.shutdown" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.shutdown"); + } + "net.server_close" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.server_close"); + } + "net.destroy" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.destroy"); + } + other => panic!("unexpected net sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse net JSON"); + assert_eq!(parsed["connections"], Value::from(1)); + assert_eq!(parsed["data"], Value::String(String::from("ping"))); + assert_eq!( + parsed["address"]["address"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["address"]["port"], Value::from(43111)); + assert_eq!( + parsed["remoteAddress"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["remotePort"], Value::from(54000)); + assert!(methods.iter().any(|method| method == "net.listen")); + assert!(methods.iter().any(|method| method == "net.server_poll")); + assert!(methods + .iter() + .any(|method| method == "net.server_connections")); + assert!(methods.iter().any(|method| method == "net.poll")); + assert!(methods.iter().any(|method| method == "net.write")); + assert!(methods.iter().any(|method| method == "net.shutdown")); + assert!(methods.iter().any(|method| method == "net.server_close")); +} + +#[test] +fn javascript_execution_routes_net_connect_path_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import net from "node:net"; + +const summary = await new Promise((resolve, reject) => { + const socket = net.createConnection({ path: "/tmp/agent-os.sock" }); + socket.on("connect", () => { + socket.end(); + }); + socket.on("error", reject); + socket.on("close", (hadError) => { + resolve({ + hadError, + remoteAddress: socket.remoteAddress, + address: socket.address(), + }); + }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut socket_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "net.connect" => { + assert_eq!( + request.args[0]["path"], + Value::String(String::from("/tmp/agent-os.sock")) + ); + socket_events.insert( + String::from("unix-socket-1"), + vec![json!({ + "type": "close", + "hadError": false, + })], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "socketId": "unix-socket-1", + "remotePath": "/tmp/agent-os.sock", + }), + ) + .expect("respond to net.connect"); + } + "net.shutdown" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.shutdown"); + } + "net.destroy" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.destroy"); + } + "net.poll" => { + let socket_id = request.args[0].as_str().expect("poll socket id"); + let next = socket_events + .get_mut(socket_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.poll"); + } + other => panic!("unexpected net sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse net JSON"); + assert_eq!(parsed["hadError"], Value::Bool(false)); + assert_eq!( + parsed["remoteAddress"], + Value::String(String::from("/tmp/agent-os.sock")) + ); + assert_eq!(parsed["address"], Value::Null); + assert!(methods.iter().any(|method| method == "net.connect")); + assert!(methods.iter().any(|method| method == "net.shutdown")); + assert!(methods.iter().any(|method| method == "net.poll")); +} + +#[test] +fn javascript_execution_routes_net_listen_path_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import net from "node:net"; + +const summary = await new Promise((resolve, reject) => { + const server = net.createServer((socket) => { + socket.on("error", reject); + socket.on("close", () => { + server.close(() => { + resolve({ + address: server.address(), + localAddress: socket.localAddress, + }); + }); + }); + socket.end(); + }); + server.on("error", reject); + server.listen({ path: "/tmp/agent-os.sock", backlog: 2 }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut listener_events = BTreeMap::>::new(); + let mut socket_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "net.listen" => { + assert_eq!( + request.args[0]["path"], + Value::String(String::from("/tmp/agent-os.sock")) + ); + assert_eq!(request.args[0]["backlog"], Value::from(2)); + listener_events.insert( + String::from("unix-listener-1"), + vec![json!({ + "type": "connection", + "socketId": "unix-socket-1", + "localPath": "/tmp/agent-os.sock", + "remotePath": Value::Null, + })], + ); + socket_events.insert( + String::from("unix-socket-1"), + vec![json!({ + "type": "close", + "hadError": false, + })], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "serverId": "unix-listener-1", + "path": "/tmp/agent-os.sock", + }), + ) + .expect("respond to net.listen"); + } + "net.server_poll" => { + let listener_id = request.args[0].as_str().expect("poll listener id"); + let next = listener_events + .get_mut(listener_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.server_poll"); + } + "net.poll" => { + let socket_id = request.args[0].as_str().expect("poll socket id"); + let next = socket_events + .get_mut(socket_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to net.poll"); + } + "net.shutdown" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.shutdown"); + } + "net.server_close" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.server_close"); + } + "net.destroy" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to net.destroy"); + } + other => panic!("unexpected net sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse net JSON"); + assert_eq!( + parsed["address"], + Value::String(String::from("/tmp/agent-os.sock")) + ); + assert_eq!( + parsed["localAddress"], + Value::String(String::from("/tmp/agent-os.sock")) + ); + assert!(methods.iter().any(|method| method == "net.listen")); + assert!(methods.iter().any(|method| method == "net.server_poll")); + assert!(methods.iter().any(|method| method == "net.poll")); + assert!(methods.iter().any(|method| method == "net.shutdown")); + assert!(methods.iter().any(|method| method == "net.server_close")); +} + +#[test] +fn javascript_execution_routes_dgram_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import dgram from "node:dgram"; + +const socket = dgram.createSocket("udp4"); +socket.on("error", (error) => { + console.error(error.stack ?? error.message); + process.exit(1); +}); + +const summary = await new Promise((resolve) => { + socket.on("message", (message, rinfo) => { + const address = socket.address(); + socket.close(() => { + resolve({ + address, + message: message.toString("utf8"), + rinfo, + }); + }); + }); + + socket.bind(43112, "127.0.0.1", () => { + socket.send("ping", 43199, "127.0.0.1"); + }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dgram\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut socket_events = BTreeMap::>::new(); + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "dgram.createSocket" => { + socket_events.insert( + String::from("udp-socket-1"), + vec![json!({ + "type": "message", + "data": { + "__agentOsType": "bytes", + "base64": "cG9uZw==", + }, + "remoteAddress": "127.0.0.1", + "remotePort": 43199, + "remoteFamily": "IPv4", + })], + ); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "socketId": "udp-socket-1", + "type": "udp4", + }), + ) + .expect("respond to dgram.createSocket"); + } + "dgram.bind" => { + assert_eq!(request.args[0].as_str(), Some("udp-socket-1")); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "localAddress": "127.0.0.1", + "localPort": 43112, + "family": "IPv4", + }), + ) + .expect("respond to dgram.bind"); + } + "dgram.send" => { + assert_eq!(request.args[0].as_str(), Some("udp-socket-1")); + execution + .respond_sync_rpc_success( + request.id, + json!({ + "bytes": 4, + "localAddress": "127.0.0.1", + "localPort": 43112, + "family": "IPv4", + }), + ) + .expect("respond to dgram.send"); + } + "dgram.poll" => { + let socket_id = request.args[0].as_str().expect("poll socket id"); + let next = socket_events + .get_mut(socket_id) + .and_then(|events| { + if events.is_empty() { + None + } else { + Some(events.remove(0)) + } + }) + .unwrap_or(Value::Null); + execution + .respond_sync_rpc_success(request.id, next) + .expect("respond to dgram.poll"); + } + "dgram.close" => { + execution + .respond_sync_rpc_success(request.id, Value::Null) + .expect("respond to dgram.close"); + } + other => panic!("unexpected dgram sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse dgram JSON"); + assert_eq!(parsed["message"], Value::String(String::from("pong"))); + assert_eq!( + parsed["address"]["address"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["address"]["port"], Value::from(43112)); + assert_eq!( + parsed["rinfo"]["address"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["rinfo"]["port"], Value::from(43199)); + assert!(methods.iter().any(|method| method == "dgram.createSocket")); + assert!(methods.iter().any(|method| method == "dgram.bind")); + assert!(methods.iter().any(|method| method == "dgram.send")); + assert!(methods.iter().any(|method| method == "dgram.poll")); + assert!(methods.iter().any(|method| method == "dgram.close")); +} + +#[test] +fn javascript_execution_routes_dns_through_sync_rpc() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import dns from "node:dns"; + +const lookup = await new Promise((resolve, reject) => { + dns.lookup("example.test", { family: 4 }, (error, address, family) => { + if (error) { + reject(error); + return; + } + resolve({ address, family }); + }); +}); + +const lookupAll = await dns.promises.lookup("example.test", { all: true }); +const resolved = await new Promise((resolve, reject) => { + dns.resolve("example.test", "A", (error, records) => { + if (error) { + reject(error); + return; + } + resolve(records); + }); +}); +const resolved4 = await dns.promises.resolve4("example.test"); +const resolved6 = await new Promise((resolve, reject) => { + dns.resolve6("example.test", (error, records) => { + if (error) { + reject(error); + return; + } + resolve(records); + }); +}); +const resolvedViaPromises = await dns.promises.resolve("example.test", "AAAA"); + +console.log(JSON.stringify({ + lookup, + lookupAll, + resolved, + resolved4, + resolved6, + resolvedViaPromises, +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let mut execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + let mut methods = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + methods.push(request.method.clone()); + match request.method.as_str() { + "dns.lookup" => { + let family = request.args[0]["family"].as_u64().expect("lookup family"); + let result = if family == 4 { + json!([{ "address": "203.0.113.10", "family": 4 }]) + } else { + json!([ + { "address": "203.0.113.10", "family": 4 }, + { "address": "2001:db8::10", "family": 6 }, + ]) + }; + execution + .respond_sync_rpc_success(request.id, result) + .expect("respond to dns.lookup"); + } + "dns.resolve" => { + let rrtype = request.args[0]["rrtype"].as_str().expect("resolve rrtype"); + let result = if rrtype == "AAAA" { + json!(["2001:db8::10"]) + } else { + json!(["203.0.113.10"]) + }; + execution + .respond_sync_rpc_success(request.id, result) + .expect("respond to dns.resolve"); + } + "dns.resolve4" => { + execution + .respond_sync_rpc_success(request.id, json!(["203.0.113.10"])) + .expect("respond to dns.resolve4"); + } + "dns.resolve6" => { + execution + .respond_sync_rpc_success(request.id, json!(["2001:db8::10"])) + .expect("respond to dns.resolve6"); + } + other => panic!("unexpected dns sync RPC method: {other}"), + } + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse dns JSON"); + assert_eq!( + parsed["lookup"]["address"], + Value::String(String::from("203.0.113.10")) + ); + assert_eq!(parsed["lookup"]["family"], Value::from(4)); + assert_eq!( + parsed["lookupAll"][1]["address"], + Value::String(String::from("2001:db8::10")) + ); + assert_eq!( + parsed["resolvedViaPromises"][0], + Value::String(String::from("2001:db8::10")) + ); + assert!(methods.iter().any(|method| method == "dns.lookup")); + assert!(methods.iter().any(|method| method == "dns.resolve")); + assert!(methods.iter().any(|method| method == "dns.resolve4")); + assert!(methods.iter().any(|method| method == "dns.resolve6")); +} + +#[test] +fn javascript_execution_imports_tls_builtin_when_allowed() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import tls from "node:tls"; + +const server = tls.createServer(); + +console.log(JSON.stringify({ + hasConnect: typeof tls.connect, + hasCreateServer: typeof tls.createServer, + serverHasListen: typeof server.listen, + tlsSocketName: tls.TLSSocket?.name ?? null, +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"tls\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + panic!("unexpected tls sync RPC method: {}", request.method) + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse tls JSON"); + assert_eq!( + parsed["hasConnect"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["hasCreateServer"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["serverHasListen"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["tlsSocketName"], + Value::String(String::from("TLSSocket")) + ); +} + +#[test] +fn javascript_execution_imports_http_builtins_when_allowed() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import http from "node:http"; +import http2 from "node:http2"; +import https from "node:https"; + +const builtinHttp = process.getBuiltinModule("node:http"); +const builtinHttp2 = process.getBuiltinModule("node:http2"); +const builtinHttps = process.getBuiltinModule("node:https"); + +console.log(JSON.stringify({ + http: { + request: typeof http.request, + get: typeof http.get, + createServer: typeof http.createServer, + builtinRequest: typeof builtinHttp?.request, + }, + http2: { + connect: typeof http2.connect, + createServer: typeof http2.createServer, + createSecureServer: typeof http2.createSecureServer, + builtinConnect: typeof builtinHttp2?.connect, + }, + https: { + request: typeof https.request, + get: typeof https.get, + createServer: typeof https.createServer, + builtinRequest: typeof builtinHttps?.request, + }, +})); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let env = BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"http\",\"http2\",\"https\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]); + let execution = engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env, + cwd: temp.path().to_path_buf(), + }) + .expect("start JavaScript execution"); + + let mut stdout = Vec::new(); + let mut stderr = Vec::new(); + let mut exit_code = None; + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll execution event") + { + Some(JavascriptExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(JavascriptExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(JavascriptExecutionEvent::SignalState { .. }) => {} + Some(JavascriptExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(JavascriptExecutionEvent::SyncRpcRequest(request)) => { + panic!( + "unexpected http builtin sync RPC method: {}", + request.method + ) + } + None => panic!("timed out waiting for JavaScript execution event"), + } + } + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse http JSON"); + assert_eq!( + parsed["http"]["request"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["http"]["get"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["http"]["createServer"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["http2"]["connect"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["http2"]["createSecureServer"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["https"]["request"], + Value::String(String::from("function")) + ); + assert_eq!( + parsed["https"]["createServer"], + Value::String(String::from("function")) + ); +} + +#[test] +fn javascript_execution_translates_require_resolve_and_cjs_errors_to_guest_paths() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("dep.cjs"), + "module.exports = { answer: 42 };\n", + ); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +const result = { + resolved: require.resolve('./dep.cjs'), +}; + +try { + require.resolve('/root/missing.cjs'); + result.resolveMissing = 'unexpected'; +} catch (error) { + result.resolveMissing = { + code: error.code ?? null, + message: error.message, + stack: error.stack ?? null, + requireStack: error.requireStack ?? [], + }; +} + +try { + require('/root/missing.cjs'); + result.requireMissing = 'unexpected'; +} catch (error) { + result.requireMissing = { + code: error.code ?? null, + message: error.message, + stack: error.stack ?? null, + requireStack: error.requireStack ?? [], + }; +} + +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + )]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse require JSON"); + let host_path = temp.path().to_string_lossy(); + + assert_eq!( + parsed["resolved"], + Value::String(String::from("/root/dep.cjs")) + ); + + for field in ["resolveMissing", "requireMissing"] { + assert_eq!( + parsed[field]["code"], + Value::String(String::from("MODULE_NOT_FOUND")) + ); + let message = parsed[field]["message"].as_str().expect("missing message"); + let stack = parsed[field]["stack"].as_str().expect("missing stack"); + assert!(message.contains("/root/missing.cjs"), "message: {message}"); + assert!( + !message.contains(host_path.as_ref()), + "message leaked host path: {message}" + ); + assert!( + !stack.contains(host_path.as_ref()), + "stack leaked host path: {stack}" + ); + + let require_stack = parsed[field]["requireStack"] + .as_array() + .expect("require stack array"); + let mut saw_guest_path = false; + for entry in require_stack { + let entry = entry.as_str().expect("require stack entry"); + saw_guest_path |= entry.starts_with("/root/"); + assert!( + !entry.contains(host_path.as_ref()), + "requireStack leaked host path: {entry}" + ); + } + assert!( + saw_guest_path, + "requireStack should contain guest-visible paths" + ); + } +} + +#[test] +fn javascript_execution_blocks_cjs_require_from_hidden_parent_node_modules() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let guest_root = temp.path().join("guest-root"); + let guest_package_dir = guest_root.join("node_modules/visible-pkg"); + let hidden_parent_package_dir = temp.path().join("node_modules/host-only-pkg"); + fs::create_dir_all(&guest_package_dir).expect("create guest package dir"); + fs::create_dir_all(&hidden_parent_package_dir).expect("create hidden parent package dir"); + + write_fixture( + &guest_root.join("dep.cjs"), + "module.exports = { answer: 41 };\n", + ); + write_fixture( + &guest_package_dir.join("package.json"), + "{\n \"name\": \"visible-pkg\",\n \"main\": \"./index.js\"\n}\n", + ); + write_fixture( + &guest_package_dir.join("index.js"), + "module.exports = { answer: 42 };\n", + ); + write_fixture( + &hidden_parent_package_dir.join("package.json"), + "{\n \"name\": \"host-only-pkg\",\n \"main\": \"./index.js\"\n}\n", + ); + write_fixture( + &hidden_parent_package_dir.join("index.js"), + "module.exports = { compromised: true };\n", + ); + write_fixture( + &guest_root.join("consumer.cjs"), + r#" +const dep = require("./dep.cjs"); +const visible = require("visible-pkg"); + +let hidden; +try { + hidden = require("host-only-pkg"); +} catch (error) { + hidden = { + code: error.code ?? null, + message: error.message, + }; +} + +module.exports = { + dep: dep.answer, + visible: visible.answer, + hidden, +}; +"#, + ); + write_fixture( + &guest_root.join("entry.mjs"), + r#" +import result from "./consumer.cjs"; +result.cacheKeys = Object.keys(require.cache) + .filter((key) => + key.includes("consumer.cjs") || + key.includes("dep.cjs") || + key.includes("visible-pkg"), + ) + .sort(); +console.log(JSON.stringify(result)); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let guest_root_host_path = guest_root.to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{guest_root_host_path}\"}}]"), + )]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + &guest_root, + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse CJS JSON"); + + assert_eq!(parsed["dep"], Value::from(41)); + assert_eq!(parsed["visible"], Value::from(42)); + assert_eq!( + parsed["hidden"]["code"], + Value::String(String::from("MODULE_NOT_FOUND")) + ); + let hidden_message = parsed["hidden"]["message"] + .as_str() + .expect("hidden module missing message"); + assert!( + hidden_message.contains("host-only-pkg"), + "message should mention blocked package: {hidden_message}" + ); + + let cache_keys = parsed["cacheKeys"].as_array().expect("cache keys array"); + let cache_key_values: Vec<&str> = cache_keys + .iter() + .map(|entry| entry.as_str().expect("cache key")) + .collect(); + assert!( + cache_key_values.contains(&"/root/consumer.cjs"), + "consumer cache key should use guest path: {cache_key_values:?}" + ); + assert!( + cache_key_values.contains(&"/root/dep.cjs"), + "dep cache key should use guest path: {cache_key_values:?}" + ); + assert!( + cache_key_values + .iter() + .any(|entry| entry.starts_with("/root/node_modules/visible-pkg/")), + "package cache key should stay in guest path space: {cache_key_values:?}" + ); +} + +#[test] +fn javascript_execution_translates_top_level_loader_stacks_to_guest_paths() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +export const broken = ; +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + let cwd_host_path = temp.path().to_string_lossy().replace('\\', "\\\\"); + let env = BTreeMap::from([( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + format!("[{{\"guestPath\":\"/root\",\"hostPath\":\"{cwd_host_path}\"}}]"), + )]); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + env, + ); + + assert_eq!(stdout.trim(), ""); + assert_eq!(exit_code, 1, "stderr: {stderr}"); + let host_path = temp.path().to_string_lossy(); + assert!( + stderr.contains("/root/entry.mjs"), + "stderr should use guest path: {stderr}" + ); + assert!( + stderr.contains("SyntaxError"), + "stderr should contain the parse failure: {stderr}" + ); + assert!( + !stderr.contains(host_path.as_ref()), + "stderr leaked host path: {stderr}" + ); +} + +#[test] +fn javascript_execution_scrubs_unmapped_host_paths_to_unknown() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let outside = tempdir().expect("create outside temp dir"); + let outside_path = outside + .path() + .join("outside-only.mjs") + .to_string_lossy() + .replace('\\', "\\\\"); + write_fixture( + &temp.path().join("entry.mjs"), + &format!( + r#" +const hostOnlyPath = "{outside_path}"; +const error = new Error(`boom at ${{hostOnlyPath}}`); +error.path = hostOnlyPath; +error.filename = hostOnlyPath; +throw error; +"# + ), + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + BTreeMap::new(), + ); + + assert_eq!(stdout.trim(), ""); + assert_eq!(exit_code, 1, "stderr: {stderr}"); + assert!( + stderr.contains("/unknown"), + "stderr should redact unmapped host paths: {stderr}" + ); + assert!( + !stderr.contains(outside.path().to_string_lossy().as_ref()), + "stderr leaked unmapped host path: {stderr}" + ); +} + +#[test] +fn javascript_execution_ignores_forged_import_cache_metrics_written_to_stderr() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture(&temp.path().join("dep.mjs"), "export const value = 1;\n"); + write_fixture( + &temp.path().join("entry.mjs"), + r#" +import "./dep.mjs"; +process.stderr.write('__AGENT_OS_NODE_IMPORT_CACHE_METRICS__:{"resolveHits":999,"resolveMisses":999}\n'); +console.log("ready"); +"#, + ); + + let mut engine = JavascriptExecutionEngine::default(); + let context = engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: Some(temp.path().join("compile-cache")), + }); + + let (stdout, stderr, exit_code) = run_javascript_execution( + &mut engine, + context.context_id, + temp.path(), + vec![String::from("./entry.mjs")], + BTreeMap::from([( + String::from("AGENT_OS_NODE_IMPORT_CACHE_DEBUG"), + String::from("1"), + )]), + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + assert!(stdout.contains("ready")); + assert!( + !stderr.contains("\"resolveHits\":999"), + "forged metrics should not survive stderr filtering: {stderr}" + ); + + let metrics = parse_import_cache_metrics(&stderr); + assert!( + metrics.resolve_hits < 999, + "unexpected metrics: {metrics:?}" + ); + assert!( + metrics.resolve_misses > 0, + "unexpected metrics: {metrics:?}" + ); +} diff --git a/crates/execution/tests/permission_flags.rs b/crates/execution/tests/permission_flags.rs index e22c8651b..d0fba1865 100644 --- a/crates/execution/tests/permission_flags.rs +++ b/crates/execution/tests/permission_flags.rs @@ -1,9 +1,11 @@ #![cfg(unix)] +use agent_os_execution::wasm::WASM_MAX_STACK_BYTES_ENV; use agent_os_execution::{ CreateJavascriptContextRequest, CreatePythonContextRequest, CreateWasmContextRequest, JavascriptExecutionEngine, PythonExecutionEngine, StartJavascriptExecutionRequest, StartPythonExecutionRequest, StartWasmExecutionRequest, WasmExecutionEngine, + WasmPermissionTier, }; use std::collections::BTreeMap; use std::fs; @@ -12,9 +14,19 @@ use std::path::{Path, PathBuf}; use tempfile::tempdir; const ARG_PREFIX: &str = "ARG="; +const ENV_PREFIX: &str = "ENV="; const INVOCATION_BREAK: &str = "--END--"; +const NODE_ALLOW_CHILD_PROCESS_FLAG: &str = "--allow-child-process"; +const NODE_ALLOW_WORKER_FLAG: &str = "--allow-worker"; +const NODE_ALLOW_WASI_FLAG: &str = "--allow-wasi"; const NODE_ALLOW_FS_READ_FLAG: &str = "--allow-fs-read="; const NODE_ALLOW_FS_WRITE_FLAG: &str = "--allow-fs-write="; +const NODE_MAX_OLD_SPACE_SIZE_FLAG_PREFIX: &str = "--max-old-space-size="; +const NODE_STACK_SIZE_FLAG_PREFIX: &str = "--stack-size="; +const NODE_WASM_MAX_MEM_PAGES_FLAG_PREFIX: &str = "--wasm-max-mem-pages="; +const PYTHON_MAX_OLD_SPACE_MB_ENV: &str = "AGENT_OS_PYTHON_MAX_OLD_SPACE_MB"; +const WASM_MAX_FUEL_ENV: &str = "AGENT_OS_WASM_MAX_FUEL"; +const WASM_MAX_MEMORY_BYTES_ENV: &str = "AGENT_OS_WASM_MAX_MEMORY_BYTES"; struct EnvVarGuard { key: &'static str, @@ -69,8 +81,10 @@ fn canonical(path: &Path) -> PathBuf { fn write_fake_node_binary(path: &Path, log_path: &Path) { let script = format!( - "#!/bin/sh\nset -eu\nlog=\"{}\"\nfor arg in \"$@\"; do\n printf 'ARG=%s\\n' \"$arg\" >> \"$log\"\ndone\nprintf '%s\\n' '{}' >> \"$log\"\nexit 0\n", + "#!/bin/sh\nset -eu\nlog=\"{}\"\nfor arg in \"$@\"; do\n printf 'ARG=%s\\n' \"$arg\" >> \"$log\"\ndone\nfor key in {} {}; do\n value=$(printenv \"$key\" || true)\n if [ -n \"$value\" ]; then\n printf 'ENV=%s=%s\\n' \"$key\" \"$value\" >> \"$log\"\n fi\ndone\nprintf '%s\\n' '{}' >> \"$log\"\nexit 0\n", log_path.display(), + WASM_MAX_FUEL_ENV, + WASM_MAX_MEMORY_BYTES_ENV, INVOCATION_BREAK, ); fs::write(path, script).expect("write fake node binary"); @@ -97,6 +111,23 @@ fn parse_invocations(log_path: &Path) -> Vec> { .collect() } +fn parse_invocation_env(log_path: &Path) -> Vec> { + let contents = fs::read_to_string(log_path).expect("read invocation log"); + let separator = format!("{INVOCATION_BREAK}\n"); + contents + .split(&separator) + .filter(|block| !block.trim().is_empty()) + .map(|block| { + block + .lines() + .filter_map(|line| line.strip_prefix(ENV_PREFIX)) + .filter_map(|entry| entry.split_once('=')) + .map(|(key, value)| (key.to_owned(), value.to_owned())) + .collect::>() + }) + .collect() +} + fn read_flags(args: &[String]) -> Vec<&str> { args.iter() .filter_map(|arg| arg.strip_prefix(NODE_ALLOW_FS_READ_FLAG)) @@ -171,7 +202,7 @@ fn node_permission_flags_do_not_expose_workspace_root_or_entrypoint_parent_write cwd: temp.path().to_path_buf(), }) .expect("start python execution") - .wait() + .wait(None) .expect("wait for python execution"); assert_eq!(python_result.exit_code, 0); @@ -185,8 +216,12 @@ fn node_permission_flags_do_not_expose_workspace_root_or_entrypoint_parent_write vm_id: String::from("vm-wasm"), context_id: wasm_context.context_id, argv: vec![String::from("./modules/guest.wasm")], - env: BTreeMap::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_STACK_BYTES_ENV), + String::from("131072"), + )]), cwd: wasm_cwd.clone(), + permission_tier: WasmPermissionTier::Full, }) .expect("start wasm execution") .wait() @@ -202,6 +237,8 @@ fn node_permission_flags_do_not_expose_workspace_root_or_entrypoint_parent_write let workspace_root = canonical(&workspace_root()).display().to_string(); let js_entry_parent = canonical(&js_entry_dir).display().to_string(); + let python_cwd = canonical(temp.path()).display().to_string(); + let python_pyodide_dir = canonical(&pyodide_dir).display().to_string(); let wasm_module_parent = canonical(&wasm_module_dir).display().to_string(); let javascript_args = &invocations[0]; @@ -230,16 +267,42 @@ fn node_permission_flags_do_not_expose_workspace_root_or_entrypoint_parent_write let python_reads = read_flags(python_args); let python_writes = write_flags(python_args); assert!( - !python_args.iter().any(|arg| arg == "--permission"), - "python should not run under Node permission mode because Pyodide requires process.binding: {python_args:?}" + python_args.iter().any(|arg| arg == "--permission"), + "python should run under Node permission mode: {python_args:?}" ); assert!( - python_reads.is_empty(), - "python should not receive Node fs read flags without permission mode: {python_args:?}" + python_reads.iter().any(|path| *path == python_cwd.as_str()), + "python should receive fs read access for the sandbox cwd: {python_args:?}" ); assert!( - python_writes.is_empty(), - "python should not receive Node fs write flags without permission mode: {python_args:?}" + python_reads + .iter() + .any(|path| *path == python_pyodide_dir.as_str()), + "python should receive fs read access for the Pyodide bundle: {python_args:?}" + ); + assert!( + python_reads + .iter() + .any(|path| path.contains("agent-os-node-import-cache-")), + "python should receive fs read access for the shared import cache: {python_args:?}" + ); + assert!( + python_writes + .iter() + .any(|path| *path == python_cwd.as_str()), + "python should receive fs write access for the sandbox cwd: {python_args:?}" + ); + assert!( + python_writes + .iter() + .any(|path| path.contains("agent-os-node-import-cache-")), + "python should receive fs write access for the shared import cache: {python_args:?}" + ); + assert!( + !python_writes + .iter() + .any(|path| *path == python_pyodide_dir.as_str()), + "python should not receive fs write access for the readonly Pyodide bundle: {python_args:?}" ); } @@ -264,5 +327,373 @@ fn node_permission_flags_do_not_expose_workspace_root_or_entrypoint_parent_write .any(|path| *path == wasm_module_parent.as_str()), "wasm write flags should not include the module parent: {wasm_args:?}" ); + assert!( + wasm_args + .iter() + .any(|arg| arg.starts_with(NODE_STACK_SIZE_FLAG_PREFIX)), + "wasm execution should apply the configured Node stack-size flag: {wasm_args:?}" + ); + } +} + +#[test] +fn node_permission_flags_allow_workers_for_internal_javascript_loader_runtime() { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + let log_path = temp.path().join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let js_cwd = temp.path().join("js-project"); + fs::create_dir_all(&js_cwd).expect("create js cwd"); + fs::write(js_cwd.join("entry.mjs"), "console.log('ignored');").expect("write js entry"); + + let mut js_engine = JavascriptExecutionEngine::default(); + let context = js_engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let default_result = js_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id.clone(), + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::new(), + cwd: js_cwd.clone(), + }) + .expect("start javascript execution without workers") + .wait() + .expect("wait for javascript execution without workers"); + assert_eq!(default_result.exit_code, 0); + + let worker_result = js_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from("[\"worker_threads\"]"), + )]), + cwd: js_cwd, + }) + .expect("start javascript execution with workers") + .wait() + .expect("wait for javascript execution with workers"); + assert_eq!(worker_result.exit_code, 0); + + let invocations = parse_invocations(&log_path); + assert_eq!( + invocations.len(), + 2, + "expected one invocation per javascript execution" + ); + assert!( + invocations[0] + .iter() + .any(|arg| arg == NODE_ALLOW_WORKER_FLAG), + "javascript executions should allow internal loader workers even by default: {:?}", + invocations[0] + ); + assert!( + invocations[1] + .iter() + .any(|arg| arg == NODE_ALLOW_WORKER_FLAG), + "javascript executions should keep worker permission enabled when worker_threads is allowed: {:?}", + invocations[1] + ); +} + +#[test] +fn node_permission_flags_only_propagate_nested_child_capabilities_when_parent_explicitly_allows_them( +) { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + let log_path = temp.path().join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let js_cwd = temp.path().join("js-project"); + fs::create_dir_all(&js_cwd).expect("create js cwd"); + fs::write(js_cwd.join("entry.mjs"), "console.log('ignored');").expect("write js entry"); + + let mut js_engine = JavascriptExecutionEngine::default(); + let context = js_engine.create_context(CreateJavascriptContextRequest { + vm_id: String::from("vm-js"), + bootstrap_module: None, + compile_cache_root: None, + }); + + let nested_env = |allow_child_process: &str, allow_worker: &str| { + BTreeMap::from([ + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from("[\"child_process\",\"worker_threads\"]"), + ), + ( + String::from("AGENT_OS_PARENT_NODE_ALLOW_CHILD_PROCESS"), + allow_child_process.to_owned(), + ), + ( + String::from("AGENT_OS_PARENT_NODE_ALLOW_WORKER"), + allow_worker.to_owned(), + ), + ]) + }; + + let denied_result = js_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id.clone(), + argv: vec![String::from("./entry.mjs")], + env: nested_env("0", "0"), + cwd: js_cwd.clone(), + }) + .expect("start nested javascript execution without inherited permissions") + .wait() + .expect("wait for nested javascript execution without inherited permissions"); + assert_eq!(denied_result.exit_code, 0); + + let allowed_result = js_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: String::from("vm-js"), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: nested_env("1", "1"), + cwd: js_cwd, + }) + .expect("start nested javascript execution with inherited permissions") + .wait() + .expect("wait for nested javascript execution with inherited permissions"); + assert_eq!(allowed_result.exit_code, 0); + + let invocations = parse_invocations(&log_path); + assert_eq!( + invocations.len(), + 2, + "expected one invocation per nested javascript execution" + ); + assert!( + !invocations[0] + .iter() + .any(|arg| arg == NODE_ALLOW_CHILD_PROCESS_FLAG), + "nested child should not inherit --allow-child-process without explicit parent permission: {:?}", + invocations[0] + ); + assert!( + !invocations[0] + .iter() + .any(|arg| arg == NODE_ALLOW_WORKER_FLAG), + "nested child should not inherit --allow-worker without explicit parent permission: {:?}", + invocations[0] + ); + assert!( + invocations[1] + .iter() + .any(|arg| arg == NODE_ALLOW_CHILD_PROCESS_FLAG), + "nested child should preserve --allow-child-process when the parent explicitly had it: {:?}", + invocations[1] + ); + assert!( + invocations[1] + .iter() + .any(|arg| arg == NODE_ALLOW_WORKER_FLAG), + "nested child should preserve --allow-worker when the parent explicitly had it: {:?}", + invocations[1] + ); +} + +#[test] +fn python_execution_applies_configured_heap_limit_to_prewarm_and_exec_processes() { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + let log_path = temp.path().join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let pyodide_dir = temp.path().join("pyodide-dist"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dist dir"); + fs::write( + pyodide_dir.join("pyodide.mjs"), + "export async function loadPyodide() { return { async runPythonAsync() {} }; }\n", + ) + .expect("write pyodide fixture"); + fs::write(pyodide_dir.join("pyodide-lock.json"), "{\"packages\":[]}\n") + .expect("write pyodide lock fixture"); + + let mut python_engine = PythonExecutionEngine::default(); + let context = python_engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let result = python_engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('ignored')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_MAX_OLD_SPACE_MB_ENV), + String::from("256"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start python execution") + .wait(None) + .expect("wait for python execution"); + assert_eq!(result.exit_code, 0); + + let invocations = parse_invocations(&log_path); + assert_eq!( + invocations.len(), + 2, + "expected one prewarm invocation and one execution invocation" + ); + + for args in &invocations { + assert!( + args.iter() + .any(|arg| arg == &format!("{NODE_MAX_OLD_SPACE_SIZE_FLAG_PREFIX}256")), + "python invocations should apply the configured Node heap limit: {args:?}" + ); + } +} + +#[test] +fn wasm_execution_passes_runtime_memory_and_fuel_limits_to_node_process() { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + let log_path = temp.path().join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let wasm_cwd = temp.path().join("wasm-project"); + fs::create_dir_all(&wasm_cwd).expect("create wasm cwd"); + fs::write(wasm_cwd.join("guest.wasm"), b"\0asm\x01\0\0\0").expect("write wasm module"); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let result = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: vec![String::from("./guest.wasm")], + env: BTreeMap::from([ + (String::from(WASM_MAX_FUEL_ENV), String::from("25")), + ( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + String::from("131072"), + ), + ]), + cwd: wasm_cwd, + permission_tier: WasmPermissionTier::Full, + }) + .expect("start wasm execution") + .wait() + .expect("wait for wasm execution"); + assert_eq!(result.exit_code, 0); + + let invocations = parse_invocations(&log_path); + let envs = parse_invocation_env(&log_path); + assert_eq!( + invocations.len(), + 2, + "expected prewarm and execution invocations" + ); + assert_eq!( + envs.len(), + 2, + "expected one env capture per prewarm and execution invocation" + ); + + for (args, env) in invocations.iter().zip(envs.iter()) { + assert!( + args.iter() + .any(|arg| arg == &format!("{NODE_WASM_MAX_MEM_PAGES_FLAG_PREFIX}2")), + "wasm invocations should enforce the configured runtime page limit: {args:?}" + ); + assert_eq!( + env.get(WASM_MAX_MEMORY_BYTES_ENV).map(String::as_str), + Some("131072"), + "wasm invocations should receive the configured memory limit env: {env:?}" + ); + assert_eq!( + env.get(WASM_MAX_FUEL_ENV).map(String::as_str), + Some("25"), + "wasm invocations should receive the configured fuel limit env: {env:?}" + ); + } +} + +#[test] +fn wasm_permission_tiers_only_enable_wasi_outside_isolated_mode() { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + let log_path = temp.path().join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let mut engine = WasmExecutionEngine::default(); + let tiers = [ + WasmPermissionTier::Isolated, + WasmPermissionTier::ReadOnly, + WasmPermissionTier::ReadWrite, + WasmPermissionTier::Full, + ]; + + for tier in tiers { + let tier_name = match tier { + WasmPermissionTier::Isolated => "isolated", + WasmPermissionTier::ReadOnly => "read-only", + WasmPermissionTier::ReadWrite => "read-write", + WasmPermissionTier::Full => "full", + }; + let wasm_cwd = temp.path().join(format!("wasm-{tier_name}")); + fs::create_dir_all(&wasm_cwd).expect("create tier-specific wasm cwd"); + fs::write(wasm_cwd.join("guest.wasm"), b"\0asm\x01\0\0\0").expect("write wasm module"); + + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let result = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: vec![String::from("./guest.wasm")], + env: BTreeMap::new(), + cwd: wasm_cwd, + permission_tier: tier, + }) + .expect("start wasm execution") + .wait() + .expect("wait for wasm execution"); + assert_eq!(result.exit_code, 0); + } + + let invocations = parse_invocations(&log_path); + assert_eq!( + invocations.len(), + tiers.len() * 2, + "expected prewarm and execution invocations for each tier" + ); + + for (index, tier) in tiers.iter().enumerate() { + for args in &invocations[index * 2..index * 2 + 2] { + let has_wasi_flag = args.iter().any(|arg| arg == NODE_ALLOW_WASI_FLAG); + assert_eq!( + has_wasi_flag, + !matches!(tier, WasmPermissionTier::Isolated), + "unexpected --allow-wasi flag for tier {tier:?}: {args:?}" + ); + } } } diff --git a/crates/execution/tests/python.rs b/crates/execution/tests/python.rs index 621560064..e821b057f 100644 --- a/crates/execution/tests/python.rs +++ b/crates/execution/tests/python.rs @@ -4,12 +4,55 @@ use agent_os_execution::{ }; use std::collections::BTreeMap; use std::fs; +use std::os::unix::fs::PermissionsExt; use std::path::{Path, PathBuf}; -use std::process::Command; +use std::process::{Command, Stdio}; +use std::thread; use std::time::Duration; use tempfile::tempdir; const PYTHON_WARMUP_METRICS_PREFIX: &str = "__AGENT_OS_PYTHON_WARMUP_METRICS__:"; +const PYTHON_EXECUTION_TIMEOUT_MS_ENV: &str = "AGENT_OS_PYTHON_EXECUTION_TIMEOUT_MS"; +const PYTHON_MAX_OLD_SPACE_MB_ENV: &str = "AGENT_OS_PYTHON_MAX_OLD_SPACE_MB"; +const PYTHON_OUTPUT_BUFFER_MAX_BYTES_ENV: &str = "AGENT_OS_PYTHON_OUTPUT_BUFFER_MAX_BYTES"; +const PYTHON_VFS_RPC_MAX_PENDING_REQUESTS_ENV: &str = + "AGENT_OS_PYTHON_VFS_RPC_MAX_PENDING_REQUESTS"; +const PYTHON_VFS_RPC_TIMEOUT_MS_ENV: &str = "AGENT_OS_PYTHON_VFS_RPC_TIMEOUT_MS"; + +struct EnvVarGuard { + key: &'static str, + previous: Option, +} + +impl EnvVarGuard { + fn set_path(key: &'static str, value: &Path) -> Self { + let previous = std::env::var(key).ok(); + // SAFETY: These tests scope process-env mutation to a single-threaded test body. + unsafe { + std::env::set_var(key, value); + } + Self { key, previous } + } +} + +impl Drop for EnvVarGuard { + fn drop(&mut self) { + match &self.previous { + Some(value) => { + // SAFETY: See EnvVarGuard::set_path. + unsafe { + std::env::set_var(self.key, value); + } + } + None => { + // SAFETY: See EnvVarGuard::set_path. + unsafe { + std::env::remove_var(self.key); + } + } + } + } +} #[derive(Debug, Clone, PartialEq)] struct PythonPrewarmMetrics { @@ -47,6 +90,15 @@ fn write_pyodide_lock_fixture(path: &Path) { write_fixture(path, "{\"packages\":[]}\n"); } +fn write_fake_node_binary(path: &Path, contents: &str) { + fs::write(path, contents).expect("write fake node binary"); + let mut permissions = fs::metadata(path) + .expect("fake node metadata") + .permissions(); + permissions.set_mode(0o755); + fs::set_permissions(path, permissions).expect("chmod fake node binary"); +} + fn parse_metrics_line<'a>(stderr: &'a str, phase: &str) -> &'a str { stderr .lines() @@ -156,13 +208,31 @@ fn run_python_execution( }) .expect("start Python execution"); - let result = execution.wait().expect("wait for Python execution"); + let result = execution.wait(None).expect("wait for Python execution"); let stdout = String::from_utf8(result.stdout).expect("stdout utf8"); let stderr = String::from_utf8(result.stderr).expect("stderr utf8"); (stdout, stderr, result.exit_code) } +fn assert_process_exits(pid: u32) { + for _ in 0..20 { + let status = Command::new("kill") + .arg("-0") + .arg(pid.to_string()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .expect("probe process with kill -0"); + if !status.success() { + return; + } + thread::sleep(Duration::from_millis(25)); + } + + panic!("process {pid} was still alive after waiting for cleanup"); +} + #[test] fn python_contexts_preserve_vm_and_pyodide_configuration() { let pyodide_dist_path = PathBuf::from("/tmp/pyodide"); @@ -227,6 +297,106 @@ export async function loadPyodide(options) { ); } +#[test] +fn python_execution_wait_bounds_output_buffers() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide(options) { + return { + setStdin(_stdin) {}, + async runPythonAsync() { + options.stdout('x'.repeat(80)); + options.stderr('y'.repeat(80)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let result = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('ignored')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_OUTPUT_BUFFER_MAX_BYTES_ENV), + String::from("32"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution") + .wait(None) + .expect("wait for Python execution"); + + assert_eq!(result.exit_code, 0); + assert_eq!(result.stdout.len(), 32, "stdout should be capped"); + assert_eq!(result.stderr.len(), 32, "stderr should be capped"); + assert!(result.stdout.iter().all(|byte| *byte == b'x')); + assert!(result.stderr.iter().all(|byte| *byte == b'y')); +} + +#[test] +fn python_execution_ignores_forged_exit_control_written_to_stderr() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide(options) { + return { + setStdin(_stdin) {}, + async runPythonAsync() { + options.stderr("__AGENT_OS_PYTHON_EXIT__:0"); + throw new Error("python-control-forgery"); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let (_stdout, stderr, exit_code) = run_python_execution( + &mut engine, + context.context_id, + temp.path(), + "print('ignored')", + BTreeMap::new(), + ); + + assert_eq!(exit_code, 1); + assert!( + stderr.contains("python-control-forgery"), + "unexpected stderr: {stderr}" + ); + assert!( + !stderr.contains("__AGENT_OS_PYTHON_EXIT__:0"), + "unexpected control line in stderr: {stderr}" + ); +} + #[test] fn python_execution_emits_stdout_before_exit() { assert_node_available(); @@ -630,3 +800,553 @@ export async function loadPyodide(options) { "unexpected stdout: {stdout}" ); } + +#[test] +fn python_execution_rejects_vfs_rpc_requests_past_queue_limit() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +import { readSync, writeSync } from 'node:fs'; + +let responseBuffer = ''; + +function readResponse(fd) { + while (true) { + const newlineIndex = responseBuffer.indexOf('\n'); + if (newlineIndex >= 0) { + const line = responseBuffer.slice(0, newlineIndex); + responseBuffer = responseBuffer.slice(newlineIndex + 1); + return JSON.parse(line); + } + + const chunk = Buffer.alloc(4096); + const bytesRead = readSync(fd, chunk, 0, chunk.length, null); + if (bytesRead === 0) { + throw new Error('response pipe closed'); + } + responseBuffer += chunk.subarray(0, bytesRead).toString('utf8'); + } +} + +export async function loadPyodide(options) { + const requestFd = Number.parseInt(process.env.AGENT_OS_PYTHON_VFS_RPC_REQUEST_FD, 10); + const responseFd = Number.parseInt(process.env.AGENT_OS_PYTHON_VFS_RPC_RESPONSE_FD, 10); + + return { + setStdin(_stdin) {}, + async runPythonAsync() { + for (const id of [1, 2, 3]) { + writeSync( + requestFd, + `${JSON.stringify({ id, method: 'fsRead', path: `/workspace/${id}.txt` })}\n`, + ); + } + + const responses = [readResponse(responseFd), readResponse(responseFd), readResponse(responseFd)]; + options.stdout(JSON.stringify(responses)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let mut execution = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('rpc queue bound')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_VFS_RPC_MAX_PENDING_REQUESTS_ENV), + String::from("1"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution"); + + let mut stdout = Vec::new(); + let mut exit_code = None; + let mut requests = Vec::new(); + + while exit_code.is_none() { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll Python event") + { + Some(PythonExecutionEvent::Stdout(chunk)) => stdout.extend(chunk), + Some(PythonExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(PythonExecutionEvent::VfsRpcRequest(request)) => { + requests.push((request.id, request.method.clone(), request.path.clone())); + execution + .respond_vfs_rpc_success( + request.id, + PythonVfsRpcResponsePayload::Read { + content_base64: String::from("aGVsbG8="), + }, + ) + .expect("respond to read"); + } + Some(PythonExecutionEvent::Exited(code)) => exit_code = Some(code), + None => panic!("timed out waiting for Python execution event"), + } + } + + assert_eq!(exit_code, Some(0)); + assert_eq!( + requests, + vec![( + 1, + PythonVfsRpcMethod::Read, + String::from("/workspace/1.txt"), + )] + ); + + let stdout = String::from_utf8(stdout).expect("stdout utf8"); + let parsed: serde_json::Value = + serde_json::from_str(stdout.trim()).expect("parse rpc queue JSON"); + let responses = parsed.as_array().expect("responses array"); + assert_eq!(responses.len(), 3, "stdout: {stdout}"); + + let ok_count = responses + .iter() + .filter(|response| response["ok"] == serde_json::Value::Bool(true)) + .count(); + let queue_full_count = responses + .iter() + .filter(|response| { + response["ok"] == serde_json::Value::Bool(false) + && response["error"]["code"] + == serde_json::Value::String(String::from( + "ERR_AGENT_OS_PYTHON_VFS_RPC_QUEUE_FULL", + )) + }) + .count(); + + assert_eq!(ok_count, 1, "stdout: {stdout}"); + assert_eq!(queue_full_count, 2, "stdout: {stdout}"); +} + +#[test] +fn python_execution_wait_timeout_cleans_up_hanging_child() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + return { + setStdin(_stdin) {}, + async runPythonAsync() { + await new Promise(() => setInterval(() => {}, 1000)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let execution = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('hang')"), + file_path: None, + env: BTreeMap::new(), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution"); + let child_pid = execution.child_pid(); + + let error = execution + .wait(Some(Duration::from_millis(100))) + .expect_err("timed out wait"); + match error { + agent_os_execution::PythonExecutionError::TimedOut(timeout) => { + assert_eq!(timeout, Duration::from_millis(100)); + } + other => panic!("expected timeout error, got {other:?}"), + } + + assert_process_exits(child_pid); +} + +#[test] +fn python_execution_uses_configured_default_timeout_when_wait_timeout_not_provided() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + return { + setStdin(_stdin) {}, + async runPythonAsync() { + await new Promise(() => setInterval(() => {}, 1000)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let execution = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('hang')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_EXECUTION_TIMEOUT_MS_ENV), + String::from("75"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution"); + let child_pid = execution.child_pid(); + + let error = execution + .wait(None) + .expect_err("configured timeout should fire"); + match error { + agent_os_execution::PythonExecutionError::TimedOut(timeout) => { + assert_eq!(timeout, Duration::from_millis(75)); + } + other => panic!("expected timeout error, got {other:?}"), + } + + assert_process_exits(child_pid); +} + +#[test] +fn python_vfs_rpc_bridge_times_out_when_sidecar_never_responds() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + return { + setStdin(_stdin) {}, + async runPythonAsync() { + globalThis.__agentOsPythonVfsRpc.fsReadSync('/workspace/never.txt'); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let execution = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('rpc timeout')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_VFS_RPC_TIMEOUT_MS_ENV), + String::from("50"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution"); + let child_pid = execution.child_pid(); + + let mut saw_request = false; + let mut stderr = Vec::new(); + let mut exit_code = None; + + for _ in 0..40 { + match execution + .poll_event(Duration::from_millis(250)) + .expect("poll Python event") + { + Some(PythonExecutionEvent::VfsRpcRequest(request)) => { + saw_request = true; + assert_eq!(request.method, PythonVfsRpcMethod::Read); + assert_eq!(request.path, "/workspace/never.txt"); + } + Some(PythonExecutionEvent::Stderr(chunk)) => stderr.extend(chunk), + Some(PythonExecutionEvent::Exited(code)) => { + exit_code = Some(code); + break; + } + Some(PythonExecutionEvent::Stdout(chunk)) => { + panic!("unexpected stdout: {}", String::from_utf8_lossy(&chunk)); + } + None => {} + } + } + + assert!(saw_request, "expected a VFS RPC request before timeout"); + assert_eq!( + exit_code, + Some(1), + "stderr: {}", + String::from_utf8_lossy(&stderr) + ); + + let stderr = String::from_utf8(stderr).expect("stderr utf8"); + assert!( + stderr.contains("ERR_AGENT_OS_PYTHON_VFS_RPC_TIMEOUT") + || stderr.contains("timed out waiting for a response") + || stderr.contains("timed out after 50ms"), + "unexpected stderr: {stderr}" + ); + assert_process_exits(child_pid); +} + +#[test] +fn python_execution_surfaces_node_heap_oom_stderr() { + let temp = tempdir().expect("create temp dir"); + let fake_node_path = temp.path().join("fake-node.sh"); + write_fake_node_binary( + &fake_node_path, + r#"#!/bin/sh +set -eu +if [ "${AGENT_OS_PYTHON_PREWARM_ONLY:-0}" = "1" ]; then + exit 0 +fi +printf '%s\n' 'FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory' >&2 +exit 134 +"#, + ); + let _node_binary = EnvVarGuard::set_path("AGENT_OS_NODE_BINARY", &fake_node_path); + + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + return { + setStdin(_stdin) {}, + async runPythonAsync() {}, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let result = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('oom')"), + file_path: None, + env: BTreeMap::from([( + String::from(PYTHON_MAX_OLD_SPACE_MB_ENV), + String::from("64"), + )]), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution") + .wait(None) + .expect("wait for Python execution"); + + let stderr = String::from_utf8(result.stderr).expect("stderr utf8"); + assert_eq!(result.exit_code, 134, "stderr: {stderr}"); + assert!( + stderr.contains("heap out of memory"), + "unexpected stderr: {stderr}" + ); +} + +#[test] +fn python_execution_kill_stops_inflight_process_and_emits_exit() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide(options) { + options.stdout("ready\n"); + return { + setStdin(_stdin) {}, + async runPythonAsync() { + await new Promise(() => setInterval(() => {}, 1000)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let mut execution = engine + .start_execution(StartPythonExecutionRequest { + vm_id: String::from("vm-python"), + context_id: context.context_id, + code: String::from("print('hang')"), + file_path: None, + env: BTreeMap::new(), + cwd: temp.path().to_path_buf(), + }) + .expect("start Python execution"); + let child_pid = execution.child_pid(); + + let mut saw_ready = false; + while !saw_ready { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll Python event before kill") + { + Some(PythonExecutionEvent::Stdout(chunk)) => { + saw_ready = String::from_utf8(chunk) + .expect("stdout utf8") + .contains("ready"); + } + Some(PythonExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + Some(PythonExecutionEvent::VfsRpcRequest(request)) => { + panic!("unexpected VFS RPC request during kill test: {request:?}"); + } + Some(PythonExecutionEvent::Exited(code)) => { + panic!("execution exited unexpectedly before kill with code {code}"); + } + None => panic!("timed out waiting for Python execution readiness"), + } + } + + execution.kill().expect("kill hanging Python execution"); + + let mut exit_code = None; + while exit_code.is_none() { + match execution + .poll_event(Duration::from_millis(100)) + .expect("poll Python event after kill") + { + Some(PythonExecutionEvent::Exited(code)) => exit_code = Some(code), + Some(PythonExecutionEvent::Stdout(_)) | Some(PythonExecutionEvent::Stderr(_)) => {} + Some(PythonExecutionEvent::VfsRpcRequest(request)) => { + panic!("unexpected VFS RPC request after kill: {request:?}"); + } + None => {} + } + } + + assert_eq!(exit_code, Some(1)); + assert_process_exits(child_pid); +} + +#[test] +fn python_execution_blocks_network_requests_during_pyodide_init() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let pyodide_dir = temp.path().join("pyodide"); + fs::create_dir_all(&pyodide_dir).expect("create pyodide dir"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + let initResult; + try { + await fetch('https://example.com/pyodide-init-check'); + initResult = { ok: true }; + } catch (error) { + initResult = { + ok: false, + code: error.code ?? null, + message: error.message, + }; + } + + return { + setStdin(_stdin) {}, + async runPythonAsync() { + console.log(JSON.stringify(initResult)); + }, + }; +} +"#, + ); + write_pyodide_lock_fixture(&pyodide_dir.join("pyodide-lock.json")); + + let mut engine = PythonExecutionEngine::default(); + let context = engine.create_context(CreatePythonContextRequest { + vm_id: String::from("vm-python"), + pyodide_dist_path: pyodide_dir, + }); + + let (stdout, stderr, exit_code) = run_python_execution( + &mut engine, + context.context_id, + temp.path(), + "print('ignored')", + BTreeMap::new(), + ); + + assert_eq!(exit_code, 0, "stderr: {stderr}"); + assert!(stderr.is_empty(), "unexpected stderr: {stderr}"); + + let parsed: serde_json::Value = + serde_json::from_str(stdout.trim()).expect("parse init network JSON"); + assert_eq!(parsed["ok"], serde_json::Value::Bool(false)); + assert_eq!( + parsed["code"], + serde_json::Value::String(String::from("ERR_ACCESS_DENIED")) + ); + assert!( + parsed["message"] + .as_str() + .expect("network denial message") + .contains("network access"), + "unexpected stdout: {stdout}" + ); +} diff --git a/crates/execution/tests/python_prewarm.rs b/crates/execution/tests/python_prewarm.rs index 6d4b284cd..8d7bafc16 100644 --- a/crates/execution/tests/python_prewarm.rs +++ b/crates/execution/tests/python_prewarm.rs @@ -124,7 +124,7 @@ fn start_python_execution( cwd: cwd.to_path_buf(), }) .expect("start Python execution") - .wait() + .wait(None) .expect("wait for Python execution"); assert_eq!(result.exit_code, 0); } diff --git a/crates/execution/tests/wasm.rs b/crates/execution/tests/wasm.rs index e848a2784..c13462bab 100644 --- a/crates/execution/tests/wasm.rs +++ b/crates/execution/tests/wasm.rs @@ -1,8 +1,13 @@ +use agent_os_execution::wasm::{ + WASM_MAX_FUEL_ENV, WASM_MAX_MEMORY_BYTES_ENV, WASM_PREWARM_TIMEOUT_MS_ENV, +}; use agent_os_execution::{ CreateWasmContextRequest, StartWasmExecutionRequest, WasmExecutionEngine, WasmExecutionEvent, + WasmPermissionTier, }; use std::collections::BTreeMap; use std::fs; +use std::os::unix::fs::symlink; use std::path::Path; use std::process::Command; use std::time::Duration; @@ -114,6 +119,7 @@ fn run_wasm_execution( cwd: &Path, argv: Vec, env: BTreeMap, + permission_tier: WasmPermissionTier, ) -> (String, String, i32) { let execution = engine .start_execution(StartWasmExecutionRequest { @@ -122,6 +128,7 @@ fn run_wasm_execution( argv, env, cwd: cwd.to_path_buf(), + permission_tier, }) .expect("start wasm execution"); @@ -228,6 +235,188 @@ fn wasm_timing_module() -> Vec { .expect("compile timing wasm fixture") } +fn wasm_signal_state_module() -> Vec { + wat::parse_str( + r#" +(module + (type $fd_write_t (func (param i32 i32 i32 i32) (result i32))) + (type $proc_sigaction_t (func (param i32 i32 i32 i32 i32) (result i32))) + (import "wasi_snapshot_preview1" "fd_write" (func $fd_write (type $fd_write_t))) + (import "host_process" "proc_sigaction" (func $proc_sigaction (type $proc_sigaction_t))) + (memory (export "memory") 1) + (data (i32.const 32) "signal:ready\n") + (func $_start (export "_start") + (drop + (call $proc_sigaction + (i32.const 2) + (i32.const 2) + (i32.const 16384) + (i32.const 0) + (i32.const 4660) + ) + ) + (i32.store (i32.const 0) (i32.const 32)) + (i32.store (i32.const 4) (i32.const 13)) + (drop + (call $fd_write + (i32.const 1) + (i32.const 0) + (i32.const 1) + (i32.const 24) + ) + ) + ) +) +"#, + ) + .expect("compile signal wasm fixture") +} + +fn wasm_write_file_module() -> Vec { + wat::parse_str( + r#" +(module + (type $path_open_t (func (param i32 i32 i32 i32 i32 i64 i64 i32 i32) (result i32))) + (type $fd_write_t (func (param i32 i32 i32 i32) (result i32))) + (type $fd_close_t (func (param i32) (result i32))) + (import "wasi_snapshot_preview1" "path_open" (func $path_open (type $path_open_t))) + (import "wasi_snapshot_preview1" "fd_write" (func $fd_write (type $fd_write_t))) + (import "wasi_snapshot_preview1" "fd_close" (func $fd_close (type $fd_close_t))) + (memory (export "memory") 1) + (data (i32.const 64) "output.txt") + (data (i32.const 80) "tiered-write\n") + (func $_start (export "_start") + (if + (i32.ne + (call $path_open + (i32.const 3) + (i32.const 0) + (i32.const 64) + (i32.const 10) + (i32.const 9) + (i64.const 64) + (i64.const 64) + (i32.const 0) + (i32.const 8) + ) + (i32.const 0) + ) + (then unreachable) + ) + (i32.store (i32.const 0) (i32.const 80)) + (i32.store (i32.const 4) (i32.const 13)) + (if + (i32.ne + (call $fd_write + (i32.load (i32.const 8)) + (i32.const 0) + (i32.const 1) + (i32.const 12) + ) + (i32.const 0) + ) + (then unreachable) + ) + (drop (call $fd_close (i32.load (i32.const 8)))) + ) +) +"#, + ) + .expect("compile write-file wasm fixture") +} + +fn wasm_infinite_loop_module() -> Vec { + wat::parse_str( + r#" +(module + (memory (export "memory") 1) + (func $_start (export "_start") + (loop $spin + br $spin + ) + ) +) +"#, + ) + .expect("compile infinite-loop wasm fixture") +} + +fn wasm_memory_capped_module() -> Vec { + wat::parse_str( + r#" +(module + (memory (export "memory") 1 3) + (func $_start (export "_start")) +) +"#, + ) + .expect("compile memory-capped wasm fixture") +} + +fn wasm_memory_grow_until_runtime_limit_module() -> Vec { + wat::parse_str( + r#" +(module + (type $fd_write_t (func (param i32 i32 i32 i32) (result i32))) + (import "wasi_snapshot_preview1" "fd_write" (func $fd_write (type $fd_write_t))) + (memory (export "memory") 1) + (data (i32.const 32) "memory-grow-limited\n") + (func $_start (export "_start") + (if + (i32.ne + (memory.grow (i32.const 1)) + (i32.const 1) + ) + (then unreachable) + ) + (if + (i32.ne + (memory.grow (i32.const 1)) + (i32.const -1) + ) + (then unreachable) + ) + (i32.store (i32.const 0) (i32.const 32)) + (i32.store (i32.const 4) (i32.const 20)) + (drop + (call $fd_write + (i32.const 1) + (i32.const 0) + (i32.const 1) + (i32.const 24) + ) + ) + ) +) +"#, + ) + .expect("compile runtime memory-limit wasm fixture") +} + +fn raw_wasm_module(section_id: u8, section_contents: &[u8]) -> Vec { + let mut bytes = Vec::from(*b"\0asm"); + bytes.extend_from_slice(&[0x01, 0x00, 0x00, 0x00]); + bytes.push(section_id); + bytes.extend(encode_varuint(section_contents.len() as u64)); + bytes.extend_from_slice(section_contents); + bytes +} + +fn encode_varuint(mut value: u64) -> Vec { + let mut encoded = Vec::new(); + loop { + let mut byte = (value & 0x7f) as u8; + value >>= 7; + if value != 0 { + byte |= 0x80; + } + encoded.push(byte); + if value == 0 { + return encoded; + } + } +} + #[test] fn wasm_contexts_preserve_vm_and_module_configuration() { let mut engine = WasmExecutionEngine::default(); @@ -261,6 +450,7 @@ fn wasm_execution_runs_guest_module_through_v8() { argv: vec![String::from("guest.wasm")], env: BTreeMap::from([(String::from("IGNORED_FOR_NOW"), String::from("ok"))]), cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, }) .expect("start wasm execution"); @@ -308,6 +498,7 @@ fn wasm_execution_ignores_guest_overrides_for_internal_node_env() { ), (String::from("NODE_OPTIONS"), String::from("--no-warnings")), ]), + WasmPermissionTier::Full, ); assert_eq!(exit_code, 0, "stderr: {stderr}"); @@ -334,6 +525,7 @@ fn wasm_execution_freezes_wasi_clock_time() { temp.path(), Vec::new(), BTreeMap::new(), + WasmPermissionTier::Full, ); assert_eq!(exit_code, 0); @@ -356,6 +548,7 @@ fn wasm_execution_rejects_vm_mismatch() { argv: Vec::new(), env: BTreeMap::new(), cwd: Path::new("/tmp").to_path_buf(), + permission_tier: WasmPermissionTier::Full, }) .expect_err("vm mismatch should fail"); @@ -384,6 +577,7 @@ fn wasm_execution_streams_exit_event() { argv: Vec::new(), env: BTreeMap::new(), cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, }) .expect("start wasm execution"); @@ -407,6 +601,7 @@ fn wasm_execution_streams_exit_event() { Some(WasmExecutionEvent::Stderr(chunk)) => { panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); } + Some(WasmExecutionEvent::SignalState { .. }) => {} None => panic!("timed out waiting for wasm execution event"), } } @@ -414,6 +609,174 @@ fn wasm_execution_streams_exit_event() { assert!(saw_stdout, "expected stdout event before exit"); } +#[test] +fn wasm_execution_emits_signal_state_from_control_channel() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture(&temp.path().join("guest.wasm"), &wasm_signal_state_module()); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let execution = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::new(), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect("start wasm execution"); + + let mut saw_stdout = false; + let mut saw_signal = false; + let mut saw_exit = false; + + while !saw_exit { + match execution + .poll_event(Duration::from_secs(5)) + .expect("poll wasm event") + { + Some(WasmExecutionEvent::Stdout(chunk)) => { + saw_stdout = String::from_utf8(chunk) + .expect("stdout utf8") + .contains("signal:ready"); + } + Some(WasmExecutionEvent::SignalState { + signal, + registration, + }) => { + assert_eq!(signal, 2); + assert_eq!( + registration.action, + agent_os_execution::wasm::WasmSignalDispositionAction::User + ); + assert_eq!(registration.mask, vec![15]); + assert_eq!(registration.flags, 0x1234); + saw_signal = true; + } + Some(WasmExecutionEvent::Exited(code)) => { + assert_eq!(code, 0); + saw_exit = true; + } + Some(WasmExecutionEvent::Stderr(chunk)) => { + panic!("unexpected stderr: {}", String::from_utf8_lossy(&chunk)); + } + None => panic!("timed out waiting for wasm execution event"), + } + } + + assert!(saw_stdout, "expected stdout event before exit"); + assert!(saw_signal, "expected signal-state event before exit"); +} + +#[test] +fn wasm_read_only_tier_blocks_workspace_writes_but_read_write_allows_them() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture(&temp.path().join("guest.wasm"), &wasm_write_file_module()); + + let mut engine = WasmExecutionEngine::default(); + let read_only_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + let read_write_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let (read_only_stdout, read_only_stderr, read_only_exit) = run_wasm_execution( + &mut engine, + read_only_context.context_id, + temp.path(), + Vec::new(), + BTreeMap::new(), + WasmPermissionTier::ReadOnly, + ); + + assert_ne!( + read_only_exit, 0, + "read-only tier unexpectedly wrote to workspace: stdout={read_only_stdout} stderr={read_only_stderr}" + ); + assert!( + !temp.path().join("output.txt").exists(), + "read-only tier should not create workspace files" + ); + + let (read_write_stdout, read_write_stderr, read_write_exit) = run_wasm_execution( + &mut engine, + read_write_context.context_id, + temp.path(), + Vec::new(), + BTreeMap::new(), + WasmPermissionTier::ReadWrite, + ); + + assert_eq!( + read_write_exit, 0, + "read-write tier should allow workspace writes: stdout={read_write_stdout} stderr={read_write_stderr}" + ); + assert_eq!( + fs::read_to_string(temp.path().join("output.txt")).expect("read output"), + "tiered-write\n" + ); +} + +#[test] +fn wasm_full_tier_exposes_host_process_imports_but_read_write_does_not() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture(&temp.path().join("guest.wasm"), &wasm_signal_state_module()); + + let mut engine = WasmExecutionEngine::default(); + let full_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + let read_write_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let (full_stdout, full_stderr, full_exit) = run_wasm_execution( + &mut engine, + full_context.context_id, + temp.path(), + Vec::new(), + BTreeMap::new(), + WasmPermissionTier::Full, + ); + + assert_eq!(full_exit, 0, "stderr: {full_stderr}"); + assert!(full_stdout.contains("signal:ready")); + + let (_stdout, stderr, exit_code) = run_wasm_execution( + &mut engine, + read_write_context.context_id, + temp.path(), + Vec::new(), + BTreeMap::new(), + WasmPermissionTier::ReadWrite, + ); + + assert_ne!( + exit_code, 0, + "read-write tier should deny host_process imports" + ); + assert!( + stderr.contains("host_process") || stderr.contains("proc_sigaction"), + "unexpected stderr for denied host_process import: {stderr}" + ); +} + #[test] fn wasm_execution_reuses_shared_warmup_path_across_contexts() { assert_node_available(); @@ -441,6 +804,7 @@ fn wasm_execution_reuses_shared_warmup_path_across_contexts() { temp.path(), Vec::new(), debug_env.clone(), + WasmPermissionTier::Full, ); let first_warmup = parse_warmup_metrics(&first_stderr); @@ -460,6 +824,7 @@ fn wasm_execution_reuses_shared_warmup_path_across_contexts() { temp.path(), Vec::new(), debug_env, + WasmPermissionTier::Full, ); let second_warmup = parse_warmup_metrics(&second_stderr); @@ -473,6 +838,63 @@ fn wasm_execution_reuses_shared_warmup_path_across_contexts() { ); } +#[test] +fn wasm_execution_rewarms_when_symlink_target_changes_with_same_size_module() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + let stable_link = temp.path().join("guest.wasm"); + write_fixture(&temp.path().join("good.wasm"), &wasm_stdout_module()); + write_fixture(&temp.path().join("evil.wasm"), &wasm_override_module()); + symlink("./good.wasm", &stable_link).expect("create initial wasm symlink"); + + let mut engine = WasmExecutionEngine::default(); + let first_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + let second_context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + let debug_env = BTreeMap::from([( + String::from("AGENT_OS_WASM_WARMUP_DEBUG"), + String::from("1"), + )]); + + let (first_stdout, first_stderr, first_exit) = run_wasm_execution( + &mut engine, + first_context.context_id, + temp.path(), + Vec::new(), + debug_env.clone(), + WasmPermissionTier::Full, + ); + let first_warmup = parse_warmup_metrics(&first_stderr); + + assert_eq!(first_exit, 0, "stderr: {first_stderr}"); + assert!(first_stdout.contains("stdout:wasm-smoke")); + assert!(first_warmup.executed, "stderr: {first_stderr}"); + + fs::remove_file(&stable_link).expect("remove wasm symlink"); + symlink("./evil.wasm", &stable_link).expect("retarget wasm symlink"); + + let (second_stdout, second_stderr, second_exit) = run_wasm_execution( + &mut engine, + second_context.context_id, + temp.path(), + Vec::new(), + debug_env, + WasmPermissionTier::Full, + ); + let second_warmup = parse_warmup_metrics(&second_stderr); + + assert_eq!(second_exit, 0, "stderr: {second_stderr}"); + assert!(second_stdout.contains("stdout:evil-smoke")); + assert!(second_warmup.executed, "stderr: {second_stderr}"); + assert_eq!(second_warmup.reason, "executed"); +} + #[test] fn wasm_warmup_metrics_encode_emoji_module_paths_as_json() { assert_node_available(); @@ -496,6 +918,7 @@ fn wasm_warmup_metrics_encode_emoji_module_paths_as_json() { String::from("AGENT_OS_WASM_WARMUP_DEBUG"), String::from("1"), )]), + WasmPermissionTier::Full, ); let warmup = parse_warmup_metrics(&stderr); @@ -505,3 +928,295 @@ fn wasm_warmup_metrics_encode_emoji_module_paths_as_json() { assert_eq!(warmup.module_path, format!("./{module_name}")); assert!(stderr.contains("\\ud83d\\ude00"), "stderr: {stderr}"); } + +#[test] +fn wasm_execution_times_out_when_fuel_budget_is_exhausted() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("guest.wasm"), + &wasm_infinite_loop_module(), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let (stdout, stderr, exit_code) = run_wasm_execution( + &mut engine, + context.context_id, + temp.path(), + Vec::new(), + BTreeMap::from([(String::from(WASM_MAX_FUEL_ENV), String::from("25"))]), + WasmPermissionTier::Full, + ); + + assert_eq!(exit_code, 124, "stdout={stdout} stderr={stderr}"); + assert!(stdout.is_empty(), "stdout={stdout}"); + assert!( + stderr.contains("fuel budget exhausted"), + "stderr should mention the exhausted fuel budget: {stderr}" + ); +} + +#[test] +fn wasm_execution_allows_prewarm_timeout_to_differ_from_execution_timeout() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("guest.wasm"), + &wasm_infinite_loop_module(), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let (stdout, stderr, exit_code) = run_wasm_execution( + &mut engine, + context.context_id, + temp.path(), + Vec::new(), + BTreeMap::from([ + (String::from(WASM_MAX_FUEL_ENV), String::from("25")), + ( + String::from(WASM_PREWARM_TIMEOUT_MS_ENV), + String::from("1000"), + ), + ]), + WasmPermissionTier::Full, + ); + + assert_eq!(exit_code, 124, "stdout={stdout} stderr={stderr}"); + assert!(stdout.is_empty(), "stdout={stdout}"); + assert!( + stderr.contains("fuel budget exhausted"), + "stderr should mention the exhausted fuel budget: {stderr}" + ); +} + +#[test] +fn wasm_execution_rejects_modules_whose_memory_cap_exceeds_limit() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("guest.wasm"), + &wasm_memory_capped_module(), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let error = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + (2 * 65_536_u64).to_string(), + )]), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect_err("memory limit should reject oversized module maximum"); + + assert!( + error.to_string().contains("memory maximum"), + "unexpected error: {error}" + ); +} + +#[test] +fn wasm_execution_enforces_runtime_memory_growth_limit_for_modules_without_declared_maximum() { + assert_node_available(); + + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("guest.wasm"), + &wasm_memory_grow_until_runtime_limit_module(), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let (stdout, stderr, exit_code) = run_wasm_execution( + &mut engine, + context.context_id, + temp.path(), + Vec::new(), + BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + (2 * 65_536_u64).to_string(), + )]), + WasmPermissionTier::Full, + ); + + assert_eq!(exit_code, 0, "stdout={stdout} stderr={stderr}"); + assert!(stderr.is_empty(), "stderr={stderr}"); + assert!( + stdout.contains("memory-grow-limited"), + "stdout should confirm runtime memory.grow enforcement: {stdout}" + ); +} + +#[test] +fn wasm_execution_rejects_modules_that_exceed_parser_file_size_cap() { + let temp = tempdir().expect("create temp dir"); + let module_path = temp.path().join("guest.wasm"); + let file = fs::File::create(&module_path).expect("create oversize wasm file"); + file.set_len(256_u64 * 1024 * 1024 + 1) + .expect("sparsely size oversize wasm file"); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let error = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + String::from("65536"), + )]), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect_err("oversized module should be rejected before read"); + + assert!( + error + .to_string() + .contains("module file size of 268435457 bytes exceeds the configured parser cap"), + "unexpected error: {error}" + ); +} + +#[test] +fn wasm_execution_rejects_modules_with_too_many_import_entries() { + let temp = tempdir().expect("create temp dir"); + let mut import_section = encode_varuint(16_385); + import_section.extend_from_slice(&[0x00, 0x00]); + write_fixture( + &temp.path().join("guest.wasm"), + &raw_wasm_module(2, &import_section), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let error = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + String::from("65536"), + )]), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect_err("import cap should reject oversized import section"); + + assert!( + error + .to_string() + .contains("import section contains 16385 entries"), + "unexpected error: {error}" + ); +} + +#[test] +fn wasm_execution_rejects_modules_with_too_many_memory_entries() { + let temp = tempdir().expect("create temp dir"); + write_fixture( + &temp.path().join("guest.wasm"), + &raw_wasm_module(5, &encode_varuint(1_025)), + ); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let error = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + String::from("65536"), + )]), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect_err("memory cap should reject oversized memory section"); + + assert!( + error + .to_string() + .contains("memory section contains 1025 entries"), + "unexpected error: {error}" + ); +} + +#[test] +fn wasm_execution_rejects_varuints_that_exceed_parser_iteration_cap() { + let temp = tempdir().expect("create temp dir"); + let mut bytes = Vec::from(*b"\0asm"); + bytes.extend_from_slice(&[0x01, 0x00, 0x00, 0x00]); + bytes.push(5); + bytes.extend_from_slice(&[0x80; 11]); + bytes.push(0x00); + write_fixture(&temp.path().join("guest.wasm"), &bytes); + + let mut engine = WasmExecutionEngine::default(); + let context = engine.create_context(CreateWasmContextRequest { + vm_id: String::from("vm-wasm"), + module_path: Some(String::from("./guest.wasm")), + }); + + let error = engine + .start_execution(StartWasmExecutionRequest { + vm_id: String::from("vm-wasm"), + context_id: context.context_id, + argv: Vec::new(), + env: BTreeMap::from([( + String::from(WASM_MAX_MEMORY_BYTES_ENV), + String::from("65536"), + )]), + cwd: temp.path().to_path_buf(), + permission_tier: WasmPermissionTier::Full, + }) + .expect_err("varuint cap should reject oversized encodings"); + + assert!( + error + .to_string() + .contains("varuint exceeds the parser cap of 10 bytes"), + "unexpected error: {error}" + ); +} diff --git a/crates/kernel/src/device_layer.rs b/crates/kernel/src/device_layer.rs index f29530ae1..c7d508fda 100644 --- a/crates/kernel/src/device_layer.rs +++ b/crates/kernel/src/device_layer.rs @@ -12,6 +12,7 @@ const DEVICE_PATHS: &[&str] = &[ ]; const DEVICE_DIRS: &[&str] = &["/dev/fd", "/dev/pts"]; +const DEFAULT_STREAM_DEVICE_READ_BYTES: usize = 4096; const DEV_DIR_ENTRIES: &[(&str, bool)] = &[ ("null", false), ("zero", false), @@ -47,12 +48,11 @@ impl DeviceLayer { impl VirtualFileSystem for DeviceLayer { fn read_file(&mut self, path: &str) -> VfsResult> { - match path { - "/dev/null" => Ok(Vec::new()), - "/dev/zero" => Ok(vec![0; 4096]), - "/dev/urandom" => random_bytes(4096), - _ => self.inner.read_file(path), + if let Some(bytes) = read_stream_device(path, DEFAULT_STREAM_DEVICE_READ_BYTES) { + return bytes; } + + self.inner.read_file(path) } fn read_dir(&mut self, path: &str) -> VfsResult> { @@ -68,6 +68,28 @@ impl VirtualFileSystem for DeviceLayer { self.inner.read_dir(path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + if path == "/dev" { + let entries = DEV_DIR_ENTRIES + .iter() + .map(|(name, _)| String::from(*name)) + .collect::>(); + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + return Ok(entries); + } + if DEVICE_DIRS.contains(&path) { + return Ok(Vec::new()); + } + self.inner.read_dir_limited(path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { if path == "/dev" { return Ok(DEV_DIR_ENTRIES @@ -93,6 +115,25 @@ impl VirtualFileSystem for DeviceLayer { self.inner.write_file(path, content) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + if is_device_path(path) || is_device_dir(path) { + let _ = content.into(); + return Err(VfsError::new( + "EEXIST", + format!("file already exists, open '{path}'"), + )); + } + self.inner.create_file_exclusive(path, content) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + if matches!(path, "/dev/null" | "/dev/zero" | "/dev/urandom") { + let _ = content.into(); + return Ok(0); + } + self.inner.append_file(path, content) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { if is_device_dir(path) { return Ok(()); @@ -206,12 +247,11 @@ impl VirtualFileSystem for DeviceLayer { } fn pread(&mut self, path: &str, offset: u64, length: usize) -> VfsResult> { - match path { - "/dev/null" => Ok(Vec::new()), - "/dev/zero" => Ok(vec![0; length]), - "/dev/urandom" => random_bytes(length), - _ => self.inner.pread(path, offset, length), + if let Some(bytes) = read_stream_device(path, length) { + return bytes; } + + self.inner.pread(path, offset, length) } } @@ -228,6 +268,9 @@ fn device_stat(path: &str) -> VirtualStat { VirtualStat { mode: 0o666, size: 0, + blocks: 0, + dev: 2, + rdev: device_rdev(path), is_directory: false, is_symbolic_link: false, atime_ms: now, @@ -246,6 +289,9 @@ fn device_dir_stat(path: &str) -> VirtualStat { VirtualStat { mode: 0o755, size: 0, + blocks: 0, + dev: 2, + rdev: 0, is_directory: true, is_symbolic_link: false, atime_ms: now, @@ -271,6 +317,22 @@ fn device_ino(path: &str) -> u64 { } } +fn device_rdev(path: &str) -> u64 { + match path { + "/dev/null" => encode_device_id(1, 3), + "/dev/zero" => encode_device_id(1, 5), + "/dev/stdin" => encode_device_id(5, 0), + "/dev/stdout" => encode_device_id(5, 1), + "/dev/stderr" => encode_device_id(5, 2), + "/dev/urandom" => encode_device_id(1, 9), + _ => 0, + } +} + +fn encode_device_id(major: u64, minor: u64) -> u64 { + (major << 8) | minor +} + fn random_bytes(length: usize) -> VfsResult> { let mut buffer = vec![0; length]; getrandom(&mut buffer) @@ -278,6 +340,15 @@ fn random_bytes(length: usize) -> VfsResult> { Ok(buffer) } +fn read_stream_device(path: &str, length: usize) -> Option>> { + match path { + "/dev/null" => Some(Ok(Vec::new())), + "/dev/zero" => Some(Ok(vec![0; length])), + "/dev/urandom" => Some(random_bytes(length)), + _ => None, + } +} + fn now_ms() -> u64 { SystemTime::now() .duration_since(UNIX_EPOCH) diff --git a/crates/kernel/src/fd_table.rs b/crates/kernel/src/fd_table.rs index 3dac48ff8..a1273348c 100644 --- a/crates/kernel/src/fd_table.rs +++ b/crates/kernel/src/fd_table.rs @@ -1,8 +1,8 @@ -use std::collections::{btree_map::Values, BTreeMap}; +use std::collections::{btree_map::Values, BTreeMap, BTreeSet}; use std::error::Error; use std::fmt; use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; -use std::sync::Arc; +use std::sync::{Arc, Condvar, Mutex, MutexGuard}; pub const MAX_FDS_PER_PROCESS: usize = 256; @@ -13,6 +13,11 @@ pub const O_CREAT: u32 = 0o100; pub const O_EXCL: u32 = 0o200; pub const O_TRUNC: u32 = 0o1000; pub const O_APPEND: u32 = 0o2000; +pub const O_NONBLOCK: u32 = 0o4000; +pub const LOCK_SH: u32 = 1; +pub const LOCK_EX: u32 = 2; +pub const LOCK_NB: u32 = 4; +pub const LOCK_UN: u32 = 8; pub const FILETYPE_UNKNOWN: u8 = 0; pub const FILETYPE_CHARACTER_DEVICE: u8 = 2; @@ -48,6 +53,20 @@ impl FdTableError { message: String::from("too many open files"), } } + + fn invalid_argument(message: impl Into) -> Self { + Self { + code: "EINVAL", + message: message.into(), + } + } + + fn would_block(message: impl Into) -> Self { + Self { + code: "EWOULDBLOCK", + message: message.into(), + } + } } impl fmt::Display for FdTableError { @@ -62,6 +81,7 @@ impl Error for FdTableError {} pub struct FileDescription { id: u64, path: String, + lock_target: Option, cursor: AtomicU64, flags: u32, ref_count: AtomicUsize, @@ -69,13 +89,33 @@ pub struct FileDescription { impl FileDescription { pub fn new(id: u64, path: impl Into, flags: u32) -> Self { - Self::with_ref_count(id, path, flags, 1) + Self::with_ref_count_and_lock(id, path, flags, 1, None) + } + + pub fn new_with_lock( + id: u64, + path: impl Into, + flags: u32, + lock_target: Option, + ) -> Self { + Self::with_ref_count_and_lock(id, path, flags, 1, lock_target) } pub fn with_ref_count(id: u64, path: impl Into, flags: u32, ref_count: usize) -> Self { + Self::with_ref_count_and_lock(id, path, flags, ref_count, None) + } + + pub fn with_ref_count_and_lock( + id: u64, + path: impl Into, + flags: u32, + ref_count: usize, + lock_target: Option, + ) -> Self { Self { id, path: path.into(), + lock_target, cursor: AtomicU64::new(0), flags, ref_count: AtomicUsize::new(ref_count), @@ -90,6 +130,10 @@ impl FileDescription { &self.path } + pub fn lock_target(&self) -> Option { + self.lock_target + } + pub fn cursor(&self) -> u64 { self.cursor.load(Ordering::SeqCst) } @@ -129,6 +173,7 @@ impl FileDescription { pub struct FdEntry { pub fd: u32, pub description: SharedFileDescription, + pub status_flags: u32, pub rights: u64, pub filetype: u8, } @@ -159,8 +204,64 @@ impl DescriptionFactory { } fn allocate(&self, path: &str, flags: u32) -> SharedFileDescription { + self.allocate_with_lock(path, flags, None) + } + + fn allocate_with_lock( + &self, + path: &str, + flags: u32, + lock_target: Option, + ) -> SharedFileDescription { let next_id = self.next_description_id.fetch_add(1, Ordering::SeqCst); - Arc::new(FileDescription::new(next_id, path, flags)) + Arc::new(FileDescription::new_with_lock( + next_id, + path, + flags, + lock_target, + )) + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub struct FileLockTarget { + ino: u64, +} + +impl FileLockTarget { + pub const fn new(ino: u64) -> Self { + Self { ino } + } + + pub const fn ino(self) -> u64 { + self.ino + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum FileLockMode { + Shared, + Exclusive, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum FlockOperation { + Shared { nonblocking: bool }, + Exclusive { nonblocking: bool }, + Unlock, +} + +impl FlockOperation { + pub fn from_bits(operation: u32) -> FdResult { + let nonblocking = operation & LOCK_NB != 0; + match operation & !LOCK_NB { + LOCK_SH => Ok(Self::Shared { nonblocking }), + LOCK_EX => Ok(Self::Exclusive { nonblocking }), + LOCK_UN => Ok(Self::Unlock), + _ => Err(FdTableError::invalid_argument(format!( + "invalid flock operation {operation:#x}" + ))), + } } } @@ -191,6 +292,7 @@ impl ProcessFdTable { FdEntry { fd: 0, description: stdin_desc, + status_flags: 0, rights: 0, filetype: FILETYPE_CHARACTER_DEVICE, }, @@ -200,6 +302,7 @@ impl ProcessFdTable { FdEntry { fd: 1, description: stdout_desc, + status_flags: 0, rights: 0, filetype: FILETYPE_CHARACTER_DEVICE, }, @@ -209,6 +312,7 @@ impl ProcessFdTable { FdEntry { fd: 2, description: stderr_desc, + status_flags: 0, rights: 0, filetype: FILETYPE_CHARACTER_DEVICE, }, @@ -232,6 +336,7 @@ impl ProcessFdTable { FdEntry { fd: 0, description: stdin_desc, + status_flags: 0, rights: 0, filetype: stdin_type, }, @@ -241,6 +346,7 @@ impl ProcessFdTable { FdEntry { fd: 1, description: stdout_desc, + status_flags: 0, rights: 0, filetype: stdout_type, }, @@ -250,6 +356,7 @@ impl ProcessFdTable { FdEntry { fd: 2, description: stderr_desc, + status_flags: 0, rights: 0, filetype: stderr_type, }, @@ -257,17 +364,30 @@ impl ProcessFdTable { } pub fn open(&mut self, path: &str, flags: u32) -> FdResult { - self.open_with_filetype(path, flags, FILETYPE_REGULAR_FILE) + self.open_with_details(path, flags, FILETYPE_REGULAR_FILE, None) } pub fn open_with_filetype(&mut self, path: &str, flags: u32, filetype: u8) -> FdResult { + self.open_with_details(path, flags, filetype, None) + } + + pub fn open_with_details( + &mut self, + path: &str, + flags: u32, + filetype: u8, + lock_target: Option, + ) -> FdResult { let fd = self.allocate_fd()?; - let description = self.alloc_desc.allocate(path, flags); + let description = + self.alloc_desc + .allocate_with_lock(path, description_flags(flags), lock_target); self.entries.insert( fd, FdEntry { fd, description, + status_flags: status_flags(flags), rights: 0, filetype, }, @@ -282,7 +402,10 @@ impl ProcessFdTable { target_fd: Option, ) -> FdResult { let fd = match target_fd { - Some(fd) => fd, + Some(fd) => { + validate_fd_bounds(fd)?; + fd + } None => self.allocate_fd()?, }; description.increment_ref_count(); @@ -291,6 +414,7 @@ impl ProcessFdTable { FdEntry { fd, description, + status_flags: 0, rights: 0, filetype, }, @@ -311,6 +435,14 @@ impl ProcessFdTable { } pub fn dup(&mut self, fd: u32) -> FdResult { + self.dup_with_status_flags(fd, None) + } + + pub fn dup_with_status_flags( + &mut self, + fd: u32, + status_flags_override: Option, + ) -> FdResult { let entry = self .entries .get(&fd) @@ -323,6 +455,7 @@ impl ProcessFdTable { FdEntry { fd: new_fd, description: entry.description, + status_flags: status_flags_override.unwrap_or(entry.status_flags), rights: entry.rights, filetype: entry.filetype, }, @@ -336,6 +469,7 @@ impl ProcessFdTable { .get(&old_fd) .cloned() .ok_or_else(|| FdTableError::bad_file_descriptor(old_fd))?; + validate_fd_bounds(new_fd)?; if old_fd == new_fd { return Ok(()); } @@ -350,6 +484,7 @@ impl ProcessFdTable { FdEntry { fd: new_fd, description: entry.description, + status_flags: entry.status_flags, rights: entry.rights, filetype: entry.filetype, }, @@ -364,7 +499,7 @@ impl ProcessFdTable { .ok_or_else(|| FdTableError::bad_file_descriptor(fd))?; Ok(FdStat { filetype: entry.filetype, - flags: entry.description.flags(), + flags: entry.description.flags() | entry.status_flags, rights: entry.rights, }) } @@ -380,6 +515,7 @@ impl ProcessFdTable { FdEntry { fd: *fd, description: Arc::clone(&entry.description), + status_flags: entry.status_flags, rights: entry.rights, filetype: entry.filetype, }, @@ -413,14 +549,32 @@ impl ProcessFdTable { return Err(FdTableError::too_many_open_files()); } - while self.entries.contains_key(&self.next_fd) { - self.next_fd += 1; + let start = usize::try_from(self.next_fd).unwrap_or(0) % MAX_FDS_PER_PROCESS; + for offset in 0..MAX_FDS_PER_PROCESS { + let candidate = ((start + offset) % MAX_FDS_PER_PROCESS) as u32; + if !self.entries.contains_key(&candidate) { + self.next_fd = candidate.saturating_add(1); + return Ok(candidate); + } } - let fd = self.next_fd; - self.next_fd += 1; - Ok(fd) + Err(FdTableError::too_many_open_files()) + } +} + +fn validate_fd_bounds(fd: u32) -> FdResult<()> { + if fd as usize >= MAX_FDS_PER_PROCESS { + return Err(FdTableError::bad_file_descriptor(fd)); } + Ok(()) +} + +fn description_flags(flags: u32) -> u32 { + flags & !status_flags(flags) +} + +fn status_flags(flags: u32) -> u32 { + flags & O_NONBLOCK } impl<'a> IntoIterator for &'a ProcessFdTable { @@ -562,3 +716,135 @@ impl FdTableManager { } } } + +#[derive(Debug, Clone, Default)] +pub struct FileLockManager { + inner: Arc, +} + +#[derive(Debug, Default)] +struct FileLockManagerInner { + state: Mutex, + wake: Condvar, +} + +#[derive(Debug, Default)] +struct FileLockState { + entries: BTreeMap, +} + +#[derive(Debug, Default)] +struct FileLockEntry { + shared: BTreeSet, + exclusive: Option, +} + +impl FileLockManager { + pub fn new() -> Self { + Self::default() + } + + pub fn apply( + &self, + owner_id: u64, + target: FileLockTarget, + operation: FlockOperation, + ) -> FdResult<()> { + match operation { + FlockOperation::Shared { nonblocking } => { + self.acquire(owner_id, target, FileLockMode::Shared, nonblocking) + } + FlockOperation::Exclusive { nonblocking } => { + self.acquire(owner_id, target, FileLockMode::Exclusive, nonblocking) + } + FlockOperation::Unlock => { + self.release_owner(owner_id); + Ok(()) + } + } + } + + pub fn release_owner(&self, owner_id: u64) -> bool { + let mut state = lock_or_recover(&self.inner.state); + let mut released = false; + state.entries.retain(|_, entry| { + let entry_changed = entry.shared.remove(&owner_id) || entry.exclusive == Some(owner_id); + if entry.exclusive == Some(owner_id) { + entry.exclusive = None; + } + released |= entry_changed; + !entry.is_empty() + }); + drop(state); + if released { + self.inner.wake.notify_all(); + } + released + } + + fn acquire( + &self, + owner_id: u64, + target: FileLockTarget, + mode: FileLockMode, + nonblocking: bool, + ) -> FdResult<()> { + let mut state = lock_or_recover(&self.inner.state); + loop { + let entry = state.entries.entry(target).or_default(); + if entry.can_grant(owner_id, mode) { + entry.grant(owner_id, mode); + return Ok(()); + } + + if nonblocking { + return Err(FdTableError::would_block( + "advisory file lock is unavailable", + )); + } + + state = wait_or_recover(&self.inner.wake, state); + } + } +} + +impl FileLockEntry { + fn can_grant(&self, owner_id: u64, mode: FileLockMode) -> bool { + match mode { + FileLockMode::Shared => self.exclusive.is_none_or(|owner| owner == owner_id), + FileLockMode::Exclusive => { + self.exclusive.is_none_or(|owner| owner == owner_id) + && self.shared.iter().all(|owner| *owner == owner_id) + } + } + } + + fn grant(&mut self, owner_id: u64, mode: FileLockMode) { + match mode { + FileLockMode::Shared => { + self.exclusive = None; + self.shared.insert(owner_id); + } + FileLockMode::Exclusive => { + self.shared.retain(|owner| *owner != owner_id); + self.exclusive = Some(owner_id); + } + } + } + + fn is_empty(&self) -> bool { + self.exclusive.is_none() && self.shared.is_empty() + } +} + +fn lock_or_recover(mutex: &Mutex) -> MutexGuard<'_, T> { + mutex + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) +} + +fn wait_or_recover<'a, T>(condvar: &Condvar, guard: MutexGuard<'a, T>) -> MutexGuard<'a, T> { + condvar + .wait(guard) + .unwrap_or_else(|poisoned| poisoned.into_inner()) +} diff --git a/crates/kernel/src/kernel.rs b/crates/kernel/src/kernel.rs index bf82b0a9b..6d9f8954f 100644 --- a/crates/kernel/src/kernel.rs +++ b/crates/kernel/src/kernel.rs @@ -2,37 +2,47 @@ use crate::bridge::LifecycleState; use crate::command_registry::{CommandDriver, CommandRegistry}; use crate::device_layer::{create_device_layer, DeviceLayer}; use crate::fd_table::{ - FdStat, FdTableError, FdTableManager, FileDescription, ProcessFdTable, - FILETYPE_CHARACTER_DEVICE, FILETYPE_DIRECTORY, FILETYPE_PIPE, FILETYPE_REGULAR_FILE, - FILETYPE_SYMBOLIC_LINK, O_APPEND, O_CREAT, O_EXCL, O_TRUNC, + FdEntry, FdStat, FdTableError, FdTableManager, FileDescription, FileLockManager, + FileLockTarget, FlockOperation, ProcessFdTable, FILETYPE_CHARACTER_DEVICE, FILETYPE_DIRECTORY, + FILETYPE_PIPE, FILETYPE_REGULAR_FILE, FILETYPE_SYMBOLIC_LINK, O_APPEND, O_CREAT, O_EXCL, + O_NONBLOCK, O_TRUNC, }; use crate::mount_table::{MountEntry, MountOptions, MountTable, MountedFileSystem}; use crate::permissions::{ - check_command_execution, PermissionError, PermissionedFileSystem, Permissions, + check_command_execution, FsOperation, PermissionError, PermissionedFileSystem, Permissions, }; use crate::pipe_manager::{PipeError, PipeManager}; +use crate::poll::{ + PollEvents, PollFd, PollNotifier, PollResult, POLLERR, POLLHUP, POLLIN, POLLNVAL, POLLOUT, +}; use crate::process_table::{ - DriverProcess, ProcessContext, ProcessExitCallback, ProcessInfo, ProcessTable, - ProcessTableError, + DriverProcess, ProcessContext, ProcessExitCallback, ProcessInfo, ProcessStatus, ProcessTable, + ProcessTableError, ProcessWaitResult, DEFAULT_PROCESS_UMASK, SIGCONT, SIGPIPE, SIGSTOP, + SIGTSTP, SIGWINCH, }; use crate::pty::{LineDisciplineConfig, PartialTermios, PtyError, PtyManager, Termios}; use crate::resource_accounting::{ - ResourceAccountant, ResourceError, ResourceLimits, ResourceSnapshot, + measure_filesystem_usage, FileSystemUsage, ResourceAccountant, ResourceError, ResourceLimits, + ResourceSnapshot, }; use crate::root_fs::{RootFileSystem, RootFilesystemError, RootFilesystemSnapshot}; use crate::user::UserManager; -use crate::vfs::{VfsError, VirtualFileSystem, VirtualStat}; +use crate::vfs::{normalize_path, VfsError, VfsResult, VirtualFileSystem, VirtualStat}; +use std::any::Any; use std::collections::{BTreeMap, BTreeSet}; use std::error::Error; use std::fmt; -use std::sync::{Arc, Condvar, Mutex}; -use std::time::{Duration, SystemTime, UNIX_EPOCH}; +use std::sync::{Arc, Condvar, Mutex, MutexGuard, WaitTimeoutResult}; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; pub type KernelResult = Result; +pub use crate::process_table::{ProcessWaitEvent as WaitPidEvent, WaitPidFlags}; pub const SEEK_SET: u8 = 0; pub const SEEK_CUR: u8 = 1; pub const SEEK_END: u8 = 2; +const EXECUTABLE_PERMISSION_BITS: u32 = 0o111; +const SHEBANG_LINE_MAX_BYTES: usize = 256; #[derive(Debug, Clone, PartialEq, Eq)] pub struct KernelError { @@ -97,7 +107,7 @@ impl KernelVmConfig { vm_id: vm_id.into(), env: BTreeMap::new(), cwd: String::from("/home/user"), - permissions: Permissions::allow_all(), + permissions: Permissions::default(), resources: ResourceLimits::default(), zombie_ttl: Duration::from_secs(60), } @@ -135,6 +145,26 @@ pub struct WaitPidResult { pub status: i32, } +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct WaitPidEventResult { + pub pid: u32, + pub status: i32, + pub event: WaitPidEvent, +} + +#[derive(Debug, Clone)] +struct ResolvedSpawnCommand { + command: String, + args: Vec, + driver: CommandDriver, +} + +#[derive(Debug, Clone)] +struct ShebangCommand { + interpreter: String, + args: Vec, +} + #[derive(Clone)] pub struct KernelProcessHandle { pid: u32, @@ -218,21 +248,24 @@ pub struct KernelVm { processes: ProcessTable, pipes: PipeManager, ptys: PtyManager, + poll_notifier: PollNotifier, users: UserManager, resources: ResourceAccountant, + file_locks: FileLockManager, driver_pids: Arc>>>, terminated: bool, } fn cleanup_process_resources( fd_tables: &Mutex, + file_locks: &FileLockManager, pipes: &PipeManager, ptys: &PtyManager, driver_pids: &Mutex>>, pid: u32, ) { let descriptors = { - let tables = fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(fd_tables); tables .get(pid) .map(|table| { @@ -246,7 +279,7 @@ fn cleanup_process_resources( let mut cleanup = Vec::new(); { - let mut tables = fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(fd_tables); if let Some(table) = tables.get_mut(pid) { for (fd, description, filetype) in &descriptors { table.close(*fd); @@ -257,16 +290,17 @@ fn cleanup_process_resources( } for (description, filetype) in cleanup { - close_special_resource_if_needed(pipes, ptys, &description, filetype); + close_special_resource_if_needed(file_locks, pipes, ptys, &description, filetype); } - let mut owners = driver_pids.lock().expect("driver PID lock poisoned"); + let mut owners = lock_or_recover(driver_pids); for pids in owners.values_mut() { pids.remove(&pid); } } fn close_special_resource_if_needed( + file_locks: &FileLockManager, pipes: &PipeManager, ptys: &PtyManager, description: &Arc, @@ -276,6 +310,8 @@ fn close_special_resource_if_needed( return; } + file_locks.release_owner(description.id()); + if filetype == FILETYPE_PIPE && pipes.is_pipe(description.id()) { pipes.close(description.id()); } @@ -285,26 +321,47 @@ fn close_special_resource_if_needed( } } -impl KernelVm { +#[derive(Debug, Clone, PartialEq, Eq)] +enum ProcNode { + RootDir, + MountsFile, + SelfLink { pid: u32 }, + PidDir { pid: u32 }, + PidFdDir { pid: u32 }, + PidCmdline { pid: u32 }, + PidEnviron { pid: u32 }, + PidCwdLink { pid: u32 }, + PidStatFile { pid: u32 }, + PidFdLink { pid: u32, fd: u32 }, +} + +impl KernelVm { pub fn new(filesystem: F, config: KernelVmConfig) -> Self { let vm_id = config.vm_id; let permissions = config.permissions.clone(); let process_table = ProcessTable::with_zombie_ttl(config.zombie_ttl); let process_table_for_pty = process_table.clone(); let fd_tables = Arc::new(Mutex::new(FdTableManager::new())); + let file_locks = FileLockManager::new(); let driver_pids = Arc::new(Mutex::new(BTreeMap::new())); - let pipes = PipeManager::new(); - let ptys = PtyManager::with_signal_handler(Arc::new(move |pgid, signal| { - let _ = process_table_for_pty.kill(-(pgid as i32), signal); - })); + let poll_notifier = PollNotifier::default(); + let pipes = PipeManager::with_notifier(poll_notifier.clone()); + let ptys = PtyManager::with_signal_handler_and_notifier( + Arc::new(move |pgid, signal| { + let _ = process_table_for_pty.kill(-(pgid as i32), signal); + }), + poll_notifier.clone(), + ); let fd_tables_for_exit = Arc::clone(&fd_tables); + let file_locks_for_exit = file_locks.clone(); let driver_pids_for_exit = Arc::clone(&driver_pids); let pipes_for_exit = pipes.clone(); let ptys_for_exit = ptys.clone(); process_table.set_on_process_exit(Some(Arc::new(move |pid| { cleanup_process_resources( fd_tables_for_exit.as_ref(), + &file_locks_for_exit, &pipes_for_exit, &ptys_for_exit, driver_pids_for_exit.as_ref(), @@ -327,8 +384,10 @@ impl KernelVm { processes: process_table, pipes, ptys, + poll_notifier, users: UserManager::new(), resources: ResourceAccountant::new(config.resources), + file_locks, driver_pids, terminated: false, } @@ -365,16 +424,18 @@ impl KernelVm { } pub fn resource_snapshot(&self) -> ResourceSnapshot { - let fd_tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let fd_tables = lock_or_recover(&self.fd_tables); self.resources .snapshot(&self.processes, &fd_tables, &self.pipes, &self.ptys) } + pub fn resource_limits(&self) -> &ResourceLimits { + self.resources.limits() + } + pub fn register_driver(&mut self, driver: CommandDriver) -> KernelResult<()> { self.assert_not_terminated()?; - self.driver_pids - .lock() - .expect("driver PID lock poisoned") + lock_or_recover(&self.driver_pids) .entry(driver.name().to_owned()) .or_default(); self.commands.register(driver); @@ -426,96 +487,335 @@ impl KernelVm { pub fn read_file(&mut self, path: &str) -> KernelResult> { self.assert_not_terminated()?; - Ok(self.filesystem.read_file(path)?) + self.read_file_internal(None, path) + } + + pub fn read_file_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult> { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.read_file_internal(Some(pid), path) } pub fn write_file(&mut self, path: &str, content: impl Into>) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } + let content = content.into(); + self.check_write_file_limits(path, content.len() as u64)?; Ok(self.filesystem.write_file(path, content)?) } + pub fn write_file_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + content: impl Into>, + mode: Option, + ) -> KernelResult<()> { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + let existed = self.exists_internal(Some(pid), path)?; + let content = content.into(); + self.write_file(path, content)?; + if !existed { + let umask = self.processes.get_umask(pid)?; + self.apply_creation_mode(path, mode.unwrap_or(0o666), umask)?; + } + Ok(()) + } + pub fn create_dir(&mut self, path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } + self.check_create_dir_limits(path)?; Ok(self.filesystem.create_dir(path)?) } + pub fn create_dir_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + mode: Option, + ) -> KernelResult<()> { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + let existed = self.exists_internal(Some(pid), path)?; + self.create_dir(path)?; + if !existed { + let umask = self.processes.get_umask(pid)?; + self.apply_creation_mode(path, mode.unwrap_or(0o777), umask)?; + } + Ok(()) + } + pub fn mkdir(&mut self, path: &str, recursive: bool) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } + self.check_mkdir_limits(path, recursive)?; Ok(self.filesystem.mkdir(path, recursive)?) } + pub fn mkdir_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + recursive: bool, + mode: Option, + ) -> KernelResult<()> { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + let created_paths = self.missing_directory_paths(path, recursive)?; + self.mkdir(path, recursive)?; + if !created_paths.is_empty() { + let umask = self.processes.get_umask(pid)?; + let mode = mode.unwrap_or(0o777); + for created_path in created_paths { + self.apply_creation_mode(&created_path, mode, umask)?; + } + } + Ok(()) + } + + pub fn umask( + &self, + requester_driver: &str, + pid: u32, + new_mask: Option, + ) -> KernelResult { + self.assert_driver_owns(requester_driver, pid)?; + match new_mask { + Some(mask) => Ok(self.processes.set_umask(pid, mask)?), + None => Ok(self.processes.get_umask(pid)?), + } + } + pub fn exists(&self, path: &str) -> KernelResult { self.assert_not_terminated()?; - Ok(self.filesystem.exists(path)?) + self.exists_internal(None, path) + } + + pub fn exists_for_process( + &self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.exists_internal(Some(pid), path) } pub fn stat(&mut self, path: &str) -> KernelResult { self.assert_not_terminated()?; - Ok(self.filesystem.stat(path)?) + self.stat_internal(None, path) + } + + pub fn stat_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.stat_internal(Some(pid), path) } pub fn lstat(&self, path: &str) -> KernelResult { self.assert_not_terminated()?; - Ok(self.filesystem.lstat(path)?) + self.lstat_internal(None, path) + } + + pub fn lstat_for_process( + &self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.lstat_internal(Some(pid), path) } pub fn read_link(&self, path: &str) -> KernelResult { self.assert_not_terminated()?; - Ok(self.filesystem.read_link(path)?) + self.read_link_internal(None, path) + } + + pub fn read_link_for_process( + &self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.read_link_internal(Some(pid), path) } pub fn read_dir(&mut self, path: &str) -> KernelResult> { self.assert_not_terminated()?; - Ok(self.filesystem.read_dir(path)?) + let entries = self.read_dir_internal(None, path)?; + self.resources.check_readdir_entries(entries.len())?; + Ok(entries) + } + + pub fn read_dir_for_process( + &mut self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult> { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + let entries = self.read_dir_internal(Some(pid), path)?; + self.resources.check_readdir_entries(entries.len())?; + Ok(entries) } pub fn remove_file(&mut self, path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } Ok(self.filesystem.remove_file(path)?) } pub fn remove_dir(&mut self, path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } Ok(self.filesystem.remove_dir(path)?) } pub fn rename(&mut self, old_path: &str, new_path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(old_path) || is_proc_path(new_path) { + self.filesystem + .check_virtual_path(FsOperation::Write, old_path) + .map_err(KernelError::from)?; + self.filesystem + .check_virtual_path(FsOperation::Write, new_path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(if is_proc_path(new_path) { + new_path + } else { + old_path + })); + } Ok(self.filesystem.rename(old_path, new_path)?) } pub fn realpath(&self, path: &str) -> KernelResult { self.assert_not_terminated()?; - Ok(self.filesystem.realpath(path)?) + self.realpath_internal(None, path) + } + + pub fn realpath_for_process( + &self, + requester_driver: &str, + pid: u32, + path: &str, + ) -> KernelResult { + self.assert_not_terminated()?; + self.assert_driver_owns(requester_driver, pid)?; + self.realpath_internal(Some(pid), path) } pub fn symlink(&mut self, target: &str, link_path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(target) || is_proc_path(link_path) { + self.filesystem + .check_virtual_path(FsOperation::Write, link_path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(link_path)); + } + self.check_symlink_limits(target, link_path)?; Ok(self.filesystem.symlink(target, link_path)?) } pub fn chmod(&mut self, path: &str, mode: u32) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } Ok(self.filesystem.chmod(path, mode)?) } pub fn link(&mut self, old_path: &str, new_path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(old_path) || is_proc_path(new_path) { + self.filesystem + .check_virtual_path(FsOperation::Write, new_path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(new_path)); + } Ok(self.filesystem.link(old_path, new_path)?) } pub fn chown(&mut self, path: &str, uid: u32, gid: u32) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } Ok(self.filesystem.chown(path, uid, gid)?) } pub fn utimes(&mut self, path: &str, atime_ms: u64, mtime_ms: u64) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } Ok(self.filesystem.utimes(path, atime_ms, mtime_ms)?) } pub fn truncate(&mut self, path: &str, length: u64) -> KernelResult<()> { self.assert_not_terminated()?; + if is_proc_path(path) { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } + self.check_truncate_limits(path, length)?; Ok(self.filesystem.truncate(path, length)?) } @@ -534,32 +834,33 @@ impl KernelVm { options: SpawnOptions, ) -> KernelResult { self.assert_not_terminated()?; - let driver = self - .commands - .resolve(command) - .cloned() - .ok_or_else(|| KernelError::command_not_found(command))?; - if let (Some(requester), Some(parent_pid)) = (options.requester_driver.as_deref(), options.parent_pid) { self.assert_driver_owns(requester, parent_pid)?; } + let cwd = options.cwd.clone().unwrap_or_else(|| self.cwd.clone()); + let resolved = self.resolve_spawn_command(command, &args, &cwd)?; + + self.resources + .check_process_argv_bytes(&resolved.command, &resolved.args)?; + self.resources + .check_process_env_bytes(&self.env, &options.env)?; + let mut env = self.env.clone(); env.extend(options.env.clone()); - let cwd = options.cwd.clone().unwrap_or_else(|| self.cwd.clone()); check_command_execution( &self.vm_id, &self.permissions, - command, - &args, + &resolved.command, + &resolved.args, Some(&cwd), &env, )?; let inherited_fds = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); options .parent_pid .and_then(|pid| tables.get(pid).map(ProcessFdTable::len)) @@ -570,7 +871,7 @@ impl KernelVm { let pid = self.processes.allocate_pid(); { - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); if let Some(parent_pid) = options.parent_pid { tables.fork(parent_pid, pid); } else { @@ -579,23 +880,24 @@ impl KernelVm { } let process = Arc::new(StubDriverProcess::default()); - let driver_name = driver.name().to_owned(); + let driver_name = resolved.driver.name().to_owned(); self.processes.register( pid, driver_name.clone(), - command.to_owned(), - args, + resolved.command, + resolved.args, ProcessContext { pid, ppid: options.parent_pid.unwrap_or(0), env, cwd, + umask: DEFAULT_PROCESS_UMASK, fds: Default::default(), }, process.clone(), ); - let mut owners = self.driver_pids.lock().expect("driver PID lock poisoned"); + let mut owners = lock_or_recover(&self.driver_pids); owners.entry(driver_name.clone()).or_default().insert(pid); if let Some(requester) = options.requester_driver { owners.entry(requester).or_default().insert(pid); @@ -614,6 +916,18 @@ impl KernelVm { Ok(WaitPidResult { pid, status }) } + pub fn waitpid_with_options( + &mut self, + requester_driver: &str, + waiter_pid: u32, + pid: i32, + flags: WaitPidFlags, + ) -> KernelResult> { + self.assert_driver_owns(requester_driver, waiter_pid)?; + let result = self.processes.waitpid_for(waiter_pid, pid, flags)?; + Ok(result.map(|result| self.finish_waitpid_event(result))) + } + pub fn wait_and_reap(&mut self, pid: u32) -> KernelResult<(u32, i32)> { let result = self.waitpid(pid)?; Ok((result.pid, result.status)) @@ -624,7 +938,7 @@ impl KernelVm { self.assert_driver_owns(requester_driver, pid)?; self.resources .check_pipe_allocation(&self.resource_snapshot())?; - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; @@ -640,7 +954,7 @@ impl KernelVm { self.assert_driver_owns(requester_driver, pid)?; self.resources .check_pty_allocation(&self.resource_snapshot())?; - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; @@ -653,24 +967,75 @@ impl KernelVm { pid: u32, path: &str, flags: u32, - _mode: Option, + mode: Option, ) -> KernelResult { self.assert_not_terminated()?; self.assert_driver_owns(requester_driver, pid)?; if let Some(existing_fd) = parse_dev_fd_path(path)? { - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); + let table = tables + .get_mut(pid) + .ok_or_else(|| KernelError::no_such_process(pid))?; + let entry = table + .get(existing_fd) + .cloned() + .ok_or_else(|| KernelError::bad_file_descriptor(existing_fd))?; + return Ok(table.dup_with_status_flags( + existing_fd, + Some(entry.status_flags | (flags & O_NONBLOCK)), + )?); + } + + if let Some(proc_node) = self.resolve_proc_node(path, Some(pid))? { + if flags & (O_CREAT | O_EXCL | O_TRUNC) != 0 + || (flags & 0b11) != crate::fd_table::O_RDONLY + { + self.filesystem + .check_virtual_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + return Err(read_only_filesystem_error(path)); + } + + if matches!( + proc_node, + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } + ) { + let target = self.proc_symlink_target(&proc_node)?; + return self.fd_open(requester_driver, pid, &target, flags, mode); + } + + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; - return Ok(table.dup(existing_fd)?); + return Ok(table.open_with_details( + &self.proc_canonical_path(&proc_node), + flags, + proc_filetype(&proc_node), + None, + )?); } - let filetype = self.prepare_fd_open(path, flags)?; - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let existed = if flags & O_CREAT != 0 { + self.exists_internal(Some(pid), path)? + } else { + false + }; + let (filetype, lock_target) = self.prepare_fd_open(path, flags)?; + if flags & O_CREAT != 0 && !existed { + let umask = self.processes.get_umask(pid)?; + self.apply_creation_mode(path, mode.unwrap_or(0o666), umask)?; + } + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; - Ok(table.open_with_filetype(path, flags, filetype)?) + Ok(table.open_with_details(path, flags, filetype, lock_target)?) } pub fn fd_read( @@ -682,7 +1047,7 @@ impl KernelVm { ) -> KernelResult> { self.assert_driver_owns(requester_driver, pid)?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -693,17 +1058,51 @@ impl KernelVm { if self.pipes.is_pipe(entry.description.id()) { return Ok(self .pipes - .read(entry.description.id(), length)? + .read_with_timeout( + entry.description.id(), + length, + if entry.status_flags & O_NONBLOCK != 0 { + Some(Duration::ZERO) + } else { + self.blocking_read_timeout() + }, + )? .unwrap_or_default()); } if self.ptys.is_pty(entry.description.id()) { return Ok(self .ptys - .read(entry.description.id(), length)? + .read_with_timeout( + entry.description.id(), + length, + if entry.status_flags & O_NONBLOCK != 0 { + Some(Duration::ZERO) + } else { + self.blocking_read_timeout() + }, + )? .unwrap_or_default()); } + if is_proc_path(entry.description.path()) { + let bytes = self.proc_read_file_from_open_path(Some(pid), entry.description.path())?; + let start = entry.description.cursor() as usize; + let end = start.saturating_add(length).min(bytes.len()); + let chunk = if start >= bytes.len() { + Vec::new() + } else { + bytes[start..end].to_vec() + }; + entry.description.set_cursor( + entry + .description + .cursor() + .saturating_add(chunk.len() as u64), + ); + return Ok(chunk); + } + let cursor = entry.description.cursor(); let bytes = VirtualFileSystem::pread( &mut self.filesystem, @@ -725,8 +1124,9 @@ impl KernelVm { data: &[u8], ) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; + self.resources.check_fd_write_size(data.len())?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -735,23 +1135,48 @@ impl KernelVm { }; if self.pipes.is_pipe(entry.description.id()) { - return Ok(self.pipes.write(entry.description.id(), data)?); + return match self.pipes.write_with_mode( + entry.description.id(), + data, + entry.status_flags & O_NONBLOCK != 0, + ) { + Ok(bytes) => Ok(bytes), + Err(error) => { + if error.code() == "EPIPE" { + self.processes.kill(pid as i32, SIGPIPE)?; + } + Err(error.into()) + } + }; } if self.ptys.is_pty(entry.description.id()) { return Ok(self.ptys.write(entry.description.id(), data)?); } + if is_proc_path(entry.description.path()) { + return Err(read_only_filesystem_error(entry.description.path())); + } + let path = entry.description.path().to_owned(); + let current_size = self.current_storage_file_size(&path)?; + let cursor = entry.description.cursor() as usize; + if entry.description.flags() & O_APPEND != 0 { + let required_size = current_size.max(checked_write_end(current_size, data.len())?); + self.check_path_resize_limits(&path, required_size)?; + let new_len = VirtualFileSystem::append_file(&mut self.filesystem, &path, data)?; + entry.description.set_cursor(new_len); + return Ok(data.len()); + } + + let required_size = current_size.max(checked_write_end(cursor as u64, data.len())?); + self.check_path_resize_limits(&path, required_size)?; + let mut existing = if VirtualFileSystem::exists(&self.filesystem, &path) { VirtualFileSystem::read_file(&mut self.filesystem, &path)? } else { Vec::new() }; - let mut cursor = entry.description.cursor() as usize; - if entry.description.flags() & O_APPEND != 0 { - cursor = existing.len(); - } if cursor > existing.len() { existing.resize(cursor, 0); } @@ -766,6 +1191,49 @@ impl KernelVm { Ok(data.len()) } + pub fn poll_fds( + &self, + requester_driver: &str, + pid: u32, + mut fds: Vec, + timeout_ms: i32, + ) -> KernelResult { + self.assert_driver_owns(requester_driver, pid)?; + if timeout_ms < -1 { + return Err(KernelError::new( + "EINVAL", + format!("invalid poll timeout {timeout_ms}"), + )); + } + + let timeout = if timeout_ms < 0 { + None + } else { + Some(Duration::from_millis(timeout_ms as u64)) + }; + let deadline = timeout.map(|duration| Instant::now() + duration); + + loop { + let observed_generation = self.poll_notifier.snapshot(); + let ready_count = self.populate_poll_revents(pid, &mut fds)?; + if ready_count > 0 || matches!(timeout, Some(duration) if duration.is_zero()) { + return Ok(PollResult { ready_count, fds }); + } + + let remaining = deadline.map(|target| target.saturating_duration_since(Instant::now())); + if matches!(remaining, Some(duration) if duration.is_zero()) { + return Ok(PollResult { ready_count, fds }); + } + + if !self + .poll_notifier + .wait_for_change(observed_generation, remaining) + { + return Ok(PollResult { ready_count, fds }); + } + } + } + pub fn fd_seek( &mut self, requester_driver: &str, @@ -776,7 +1244,7 @@ impl KernelVm { ) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -791,7 +1259,15 @@ impl KernelVm { let base = match whence { SEEK_SET => 0_i128, SEEK_CUR => i128::from(entry.description.cursor()), - SEEK_END => i128::from(self.filesystem.stat(entry.description.path())?.size), + SEEK_END => { + let size = if is_proc_path(entry.description.path()) { + self.proc_stat_from_open_path(Some(pid), entry.description.path())? + .size + } else { + self.filesystem.stat(entry.description.path())?.size + }; + i128::from(size) + } _ => { return Err(KernelError::new( "EINVAL", @@ -818,8 +1294,9 @@ impl KernelVm { offset: u64, ) -> KernelResult> { self.assert_driver_owns(requester_driver, pid)?; + self.resources.check_pread_length(length)?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -831,6 +1308,18 @@ impl KernelVm { return Err(KernelError::new("ESPIPE", "illegal seek")); } + if is_proc_path(entry.description.path()) { + let bytes = self.proc_read_file_from_open_path(Some(pid), entry.description.path())?; + let start = usize::try_from(offset) + .map_err(|_| KernelError::new("EINVAL", "pread offset out of range"))?; + let end = start.saturating_add(length).min(bytes.len()); + return Ok(if start >= bytes.len() { + Vec::new() + } else { + bytes[start..end].to_vec() + }); + } + Ok(VirtualFileSystem::pread( &mut self.filesystem, entry.description.path(), @@ -848,8 +1337,9 @@ impl KernelVm { offset: u64, ) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; + self.resources.check_fd_write_size(data.len())?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -861,6 +1351,14 @@ impl KernelVm { return Err(KernelError::new("ESPIPE", "illegal seek")); } + if is_proc_path(entry.description.path()) { + return Err(read_only_filesystem_error(entry.description.path())); + } + + let required_size = self + .current_storage_file_size(entry.description.path())? + .max(checked_write_end(offset, data.len())?); + self.check_path_resize_limits(entry.description.path(), required_size)?; VirtualFileSystem::pwrite( &mut self.filesystem, entry.description.path(), @@ -872,7 +1370,7 @@ impl KernelVm { pub fn fd_dup(&mut self, requester_driver: &str, pid: u32, fd: u32) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; @@ -888,7 +1386,7 @@ impl KernelVm { ) -> KernelResult<()> { self.assert_driver_owns(requester_driver, pid)?; let replaced = { - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; @@ -910,7 +1408,7 @@ impl KernelVm { pub fn fd_close(&mut self, requester_driver: &str, pid: u32, fd: u32) -> KernelResult<()> { self.assert_driver_owns(requester_driver, pid)?; let (description, filetype) = { - let mut tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let mut tables = lock_or_recover(&self.fd_tables); let table = tables .get_mut(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; @@ -925,9 +1423,45 @@ impl KernelVm { Ok(()) } + pub fn fd_flock( + &self, + requester_driver: &str, + pid: u32, + fd: u32, + operation: u32, + ) -> KernelResult<()> { + self.assert_driver_owns(requester_driver, pid)?; + let entry = { + let tables = lock_or_recover(&self.fd_tables); + tables + .get(pid) + .and_then(|table| table.get(fd)) + .cloned() + .ok_or_else(|| KernelError::bad_file_descriptor(fd))? + }; + + if entry.filetype != FILETYPE_REGULAR_FILE { + return Err(KernelError::new( + "EBADF", + format!("file descriptor {fd} does not support advisory locking"), + )); + } + + let target = entry.description.lock_target().ok_or_else(|| { + KernelError::new( + "EBADF", + format!("file descriptor {fd} is missing advisory lock metadata"), + ) + })?; + let operation = FlockOperation::from_bits(operation)?; + self.file_locks + .apply(entry.description.id(), target, operation)?; + Ok(()) + } + pub fn fd_stat(&self, requester_driver: &str, pid: u32, fd: u32) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); Ok(tables .get(pid) .ok_or_else(|| KernelError::no_such_process(pid))? @@ -937,7 +1471,7 @@ impl KernelVm { pub fn isatty(&self, requester_driver: &str, pid: u32, fd: u32) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -967,6 +1501,18 @@ impl KernelVm { pgid: u32, ) -> KernelResult<()> { let description = self.description_for_fd(requester_driver, pid, fd)?; + let requester_sid = self.processes.getsid(pid)?; + let group = self + .processes + .list_processes() + .into_values() + .find(|process| process.pgid == pgid && process.status != ProcessStatus::Exited) + .ok_or_else(|| KernelError::new("ESRCH", format!("no such process group {pgid}")))?; + if group.sid != requester_sid { + return Err(KernelError::permission_denied( + "cannot set foreground process group in different session", + )); + } self.ptys.set_foreground_pgid(description.id(), pgid)?; Ok(()) } @@ -993,6 +1539,26 @@ impl KernelVm { Ok(self.ptys.get_foreground_pgid(description.id())?) } + pub fn pty_resize( + &self, + requester_driver: &str, + pid: u32, + fd: u32, + cols: u16, + rows: u16, + ) -> KernelResult<()> { + let description = self.description_for_fd(requester_driver, pid, fd)?; + let target_pgid = self.ptys.resize(description.id(), cols, rows)?; + if let Some(pgid) = target_pgid { + match self.processes.kill(-(pgid as i32), SIGWINCH) { + Ok(()) => {} + Err(error) if error.code() == "ESRCH" => {} + Err(error) => return Err(error.into()), + } + } + Ok(()) + } + pub fn kill_process(&self, requester_driver: &str, pid: u32, signal: i32) -> KernelResult<()> { self.assert_driver_owns(requester_driver, pid)?; self.processes.kill(pid as i32, signal)?; @@ -1001,6 +1567,24 @@ impl KernelVm { pub fn setpgid(&self, requester_driver: &str, pid: u32, pgid: u32) -> KernelResult<()> { self.assert_driver_owns(requester_driver, pid)?; + let target_pgid = if pgid == 0 { pid } else { pgid }; + if target_pgid != pid { + if let Some(group_owner) = + self.processes + .list_processes() + .into_values() + .find(|process| { + process.pgid == target_pgid && process.status == ProcessStatus::Running + }) + { + if group_owner.driver != requester_driver { + return Err(KernelError::permission_denied(format!( + "driver \"{requester_driver}\" cannot join process group {target_pgid} owned by \"{}\"", + group_owner.driver + ))); + } + } + } self.processes.setpgid(pid, pgid)?; Ok(()) } @@ -1032,10 +1616,12 @@ impl KernelVm { pub fn dev_fd_read_dir(&self, requester_driver: &str, pid: u32) -> KernelResult> { self.assert_driver_owns(requester_driver, pid)?; - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); let table = tables .get(pid) .ok_or_else(|| KernelError::no_such_process(pid))?; + let entry_count = table.len(); + self.resources.check_readdir_entries(entry_count)?; Ok(table.iter().map(|entry| entry.fd.to_string()).collect()) } @@ -1047,7 +1633,7 @@ impl KernelVm { ) -> KernelResult { self.assert_driver_owns(requester_driver, pid)?; let entry = { - let tables = self.fd_tables.lock().expect("FD table lock poisoned"); + let tables = lock_or_recover(&self.fd_tables); tables .get(pid) .and_then(|table| table.get(fd)) @@ -1059,6 +1645,10 @@ impl KernelVm { return Ok(synthetic_character_device_stat(entry.description.id())); } + if is_proc_path(entry.description.path()) { + return self.proc_stat_from_open_path(Some(pid), entry.description.path()); + } + Ok(self.filesystem.stat(entry.description.path())?) } @@ -1068,35 +1658,38 @@ impl KernelVm { } self.processes.terminate_all(); - let pids = self - .fd_tables - .lock() - .expect("FD table lock poisoned") - .pids(); + let pids = lock_or_recover(&self.fd_tables).pids(); for pid in pids { self.cleanup_process_resources(pid); } - self.driver_pids - .lock() - .expect("driver PID lock poisoned") - .clear(); + lock_or_recover(&self.driver_pids).clear(); self.terminated = true; Ok(()) } - fn prepare_fd_open(&mut self, path: &str, flags: u32) -> KernelResult { + fn prepare_fd_open( + &mut self, + path: &str, + flags: u32, + ) -> KernelResult<(u8, Option)> { + if flags & O_CREAT != 0 && flags & O_EXCL != 0 { + self.check_write_file_limits(path, 0)?; + VirtualFileSystem::create_file_exclusive(&mut self.filesystem, path, Vec::new())?; + let stat = VirtualFileSystem::stat(&mut self.filesystem, path)?; + return Ok(( + filetype_for_path(path, &stat), + Some(FileLockTarget::new(stat.ino)), + )); + } + let exists = self.filesystem.exists(path)?; if exists { - if flags & O_CREAT != 0 && flags & O_EXCL != 0 { - return Err(KernelError::new( - "EEXIST", - format!("file already exists: {path}"), - )); - } if flags & O_TRUNC != 0 { + self.check_truncate_limits(path, 0)?; VirtualFileSystem::truncate(&mut self.filesystem, path, 0)?; } } else if flags & O_CREAT != 0 { + self.check_write_file_limits(path, 0)?; VirtualFileSystem::write_file(&mut self.filesystem, path, Vec::new())?; } else { let _ = VirtualFileSystem::stat(&mut self.filesystem, path)?; @@ -1104,21 +1697,78 @@ impl KernelVm { } let stat = VirtualFileSystem::stat(&mut self.filesystem, path)?; - Ok(filetype_for_path(path, &stat)) + Ok(( + filetype_for_path(path, &stat), + Some(FileLockTarget::new(stat.ino)), + )) } - fn description_for_fd( - &self, - requester_driver: &str, - pid: u32, - fd: u32, - ) -> KernelResult> { - self.assert_driver_owns(requester_driver, pid)?; - self.fd_tables - .lock() - .expect("FD table lock poisoned") - .get(pid) - .and_then(|table| table.get(fd)) + fn populate_poll_revents(&self, pid: u32, fds: &mut [PollFd]) -> KernelResult { + let entries = { + let tables = lock_or_recover(&self.fd_tables); + let table = tables + .get(pid) + .ok_or_else(|| KernelError::no_such_process(pid))?; + fds.iter() + .map(|poll_fd| table.get(poll_fd.fd).cloned()) + .collect::>() + }; + + let mut ready_count = 0; + for (poll_fd, entry) in fds.iter_mut().zip(entries.into_iter()) { + poll_fd.revents = if let Some(entry) = entry { + self.poll_entry(&entry, poll_fd.events)? + } else { + POLLNVAL + }; + if !poll_fd.revents.is_empty() { + ready_count += 1; + } + } + + Ok(ready_count) + } + + fn poll_entry( + &self, + entry: &crate::fd_table::FdEntry, + requested: PollEvents, + ) -> KernelResult { + if self.pipes.is_pipe(entry.description.id()) { + return Ok(self.pipes.poll(entry.description.id(), requested)?); + } + + if self.ptys.is_pty(entry.description.id()) { + return Ok(self.ptys.poll(entry.description.id(), requested)?); + } + + let access_mode = entry.description.flags() & 0b11; + let mut events = PollEvents::empty(); + if requested.intersects(POLLIN) && access_mode != crate::fd_table::O_WRONLY { + events |= POLLIN; + } + if requested.intersects(POLLOUT) && access_mode != crate::fd_table::O_RDONLY { + events |= POLLOUT; + } + if entry.filetype == FILETYPE_DIRECTORY && requested.intersects(POLLOUT) { + events |= POLLERR; + } + if self.terminated { + events |= POLLHUP; + } + Ok(events) + } + + fn description_for_fd( + &self, + requester_driver: &str, + pid: u32, + fd: u32, + ) -> KernelResult> { + self.assert_driver_owns(requester_driver, pid)?; + lock_or_recover(&self.fd_tables) + .get(pid) + .and_then(|table| table.get(fd)) .map(|entry| Arc::clone(&entry.description)) .ok_or_else(|| KernelError::bad_file_descriptor(fd)) } @@ -1132,7 +1782,7 @@ impl KernelVm { } fn assert_driver_owns(&self, requester_driver: &str, pid: u32) -> KernelResult<()> { - let driver_pids = self.driver_pids.lock().expect("driver PID lock poisoned"); + let driver_pids = lock_or_recover(&self.driver_pids); if driver_pids .get(requester_driver) .map(|pids| pids.contains(&pid)) @@ -1153,6 +1803,7 @@ impl KernelVm { fn cleanup_process_resources(&self, pid: u32) { cleanup_process_resources( self.fd_tables.as_ref(), + &self.file_locks, &self.pipes, &self.ptys, self.driver_pids.as_ref(), @@ -1160,12 +1811,839 @@ impl KernelVm { ); } + fn resolve_spawn_command( + &mut self, + command: &str, + args: &[String], + cwd: &str, + ) -> KernelResult { + if let Some(driver) = self.commands.resolve(command).cloned() { + return Ok(ResolvedSpawnCommand { + command: command.to_owned(), + args: args.to_vec(), + driver, + }); + } + + let Some(path) = self.resolve_executable_path(command, cwd)? else { + return Err(KernelError::command_not_found(command)); + }; + + if let Some(registered_command) = self.resolve_registered_command_path(&path) { + let driver = self + .commands + .resolve(®istered_command) + .cloned() + .ok_or_else(|| KernelError::command_not_found(®istered_command))?; + return Ok(ResolvedSpawnCommand { + command: registered_command, + args: args.to_vec(), + driver, + }); + } + + let shebang = self + .parse_shebang_command(&path)? + .ok_or_else(|| KernelError::new("ENOEXEC", format!("exec format error: {path}")))?; + self.resolve_shebang_command(&path, args, shebang) + } + + fn resolve_executable_path( + &mut self, + command: &str, + cwd: &str, + ) -> KernelResult> { + if !command.contains('/') { + return Ok(None); + } + + let path = if command.starts_with('/') { + normalize_path(command) + } else { + normalize_path(&format!("{cwd}/{command}")) + }; + let stat = self.filesystem.stat(&path)?; + if stat.is_directory { + return Err(KernelError::new( + "EACCES", + format!("permission denied, execute '{path}'"), + )); + } + if stat.mode & EXECUTABLE_PERMISSION_BITS == 0 { + return Err(KernelError::new( + "EACCES", + format!("permission denied, execute '{path}'"), + )); + } + Ok(Some(path)) + } + + fn resolve_registered_command_path(&self, path: &str) -> Option { + let normalized = normalize_path(path); + for prefix in ["/bin/", "/usr/bin/"] { + let Some(name) = normalized.strip_prefix(prefix) else { + continue; + }; + if !name.is_empty() && !name.contains('/') && self.commands.resolve(name).is_some() { + return Some(name.to_owned()); + } + } + None + } + + fn parse_shebang_command(&mut self, path: &str) -> KernelResult> { + let header = self.filesystem.pread(path, 0, SHEBANG_LINE_MAX_BYTES + 1)?; + if !header.starts_with(b"#!") { + return Ok(None); + } + + let line_end = match header.iter().position(|byte| *byte == b'\n') { + Some(index) => index, + None if header.len() <= SHEBANG_LINE_MAX_BYTES => header.len(), + None => { + return Err(KernelError::new( + "ENOEXEC", + format!("shebang line exceeds {SHEBANG_LINE_MAX_BYTES} bytes: {path}"), + )) + } + }; + let line = header[2..line_end] + .strip_suffix(b"\r") + .unwrap_or(&header[2..line_end]); + let text = std::str::from_utf8(line) + .map_err(|_| KernelError::new("ENOEXEC", format!("invalid shebang line: {path}")))?; + let mut parts = text.split_ascii_whitespace(); + let interpreter = parts + .next() + .ok_or_else(|| KernelError::new("ENOEXEC", format!("invalid shebang line: {path}")))?; + Ok(Some(ShebangCommand { + interpreter: interpreter.to_owned(), + args: parts.map(ToOwned::to_owned).collect(), + })) + } + + fn resolve_shebang_command( + &self, + path: &str, + args: &[String], + shebang: ShebangCommand, + ) -> KernelResult { + let mut interpreter_args = shebang.args; + let interpreter = normalize_path(&shebang.interpreter); + let command = if interpreter == "/usr/bin/env" || interpreter == "/bin/env" { + if interpreter_args.is_empty() { + return Err(KernelError::new( + "ENOENT", + format!("missing interpreter after /usr/bin/env in shebang: {path}"), + )); + } + interpreter_args.remove(0) + } else if let Some(command) = self.resolve_registered_command_path(&interpreter) { + command + } else if self.commands.resolve(&shebang.interpreter).is_some() { + shebang.interpreter + } else { + return Err(KernelError::command_not_found(&shebang.interpreter)); + }; + + let driver = self + .commands + .resolve(&command) + .cloned() + .ok_or_else(|| KernelError::command_not_found(&command))?; + let mut resolved_args = interpreter_args; + resolved_args.push(path.to_owned()); + resolved_args.extend(args.iter().cloned()); + Ok(ResolvedSpawnCommand { + command, + args: resolved_args, + driver, + }) + } + + fn finish_waitpid_event(&mut self, result: ProcessWaitResult) -> WaitPidEventResult { + if result.event == WaitPidEvent::Exited { + self.cleanup_process_resources(result.pid); + } + WaitPidEventResult { + pid: result.pid, + status: result.status, + event: result.event, + } + } + + fn raw_filesystem_mut(&mut self) -> &mut F { + self.filesystem.inner_mut().inner_mut() + } + + fn read_file_internal( + &mut self, + current_pid: Option, + path: &str, + ) -> KernelResult> { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_read_file(current_pid, &proc_node); + } + + Ok(self.filesystem.read_file(path)?) + } + + fn exists_internal(&self, current_pid: Option, path: &str) -> KernelResult { + match self.resolve_proc_node(path, current_pid) { + Ok(Some(_)) => { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + Ok(true) + } + Ok(None) => Ok(self.filesystem.exists(path)?), + Err(error) if error.code() == "ENOENT" => Ok(false), + Err(error) => Err(error), + } + } + + fn stat_internal(&mut self, current_pid: Option, path: &str) -> KernelResult { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_stat(current_pid, &proc_node); + } + + Ok(self.filesystem.stat(path)?) + } + + fn lstat_internal(&self, current_pid: Option, path: &str) -> KernelResult { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_lstat(&proc_node); + } + + Ok(self.filesystem.lstat(path)?) + } + + fn read_link_internal(&self, current_pid: Option, path: &str) -> KernelResult { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_read_link(&proc_node); + } + + Ok(self.filesystem.read_link(path)?) + } + + fn read_dir_internal( + &mut self, + current_pid: Option, + path: &str, + ) -> KernelResult> { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_read_dir(current_pid, &proc_node); + } + + if let Some(limit) = self.resources.max_readdir_entries() { + Ok(self.filesystem.read_dir_limited(path, limit)?) + } else { + Ok(self.filesystem.read_dir(path)?) + } + } + + fn realpath_internal(&self, current_pid: Option, path: &str) -> KernelResult { + if let Some(proc_node) = self.resolve_proc_node(path, current_pid)? { + self.filesystem + .check_virtual_path(FsOperation::Read, path) + .map_err(KernelError::from)?; + return self.proc_realpath(current_pid, &proc_node); + } + + Ok(self.filesystem.realpath(path)?) + } + + fn resolve_proc_node( + &self, + path: &str, + current_pid: Option, + ) -> KernelResult> { + let normalized = normalize_path(path); + if !is_proc_path(&normalized) { + return Ok(None); + } + + if normalized == "/proc" { + return Ok(Some(ProcNode::RootDir)); + } + + let suffix = normalized + .strip_prefix("/proc/") + .expect("proc path should have /proc prefix"); + let parts = suffix.split('/').collect::>(); + if parts.is_empty() { + return Ok(Some(ProcNode::RootDir)); + } + + if parts == ["mounts"] { + return Ok(Some(ProcNode::MountsFile)); + } + + let pid = match parts[0] { + "self" => current_pid.ok_or_else(|| proc_not_found_error(&normalized))?, + raw => raw + .parse::() + .map_err(|_| proc_not_found_error(&normalized))?, + }; + self.proc_entry(pid)?; + + let node = match parts.as_slice() { + ["self"] => ProcNode::SelfLink { pid }, + [_pid] => ProcNode::PidDir { pid }, + [_pid, "fd"] => ProcNode::PidFdDir { pid }, + [_pid, "cmdline"] => ProcNode::PidCmdline { pid }, + [_pid, "environ"] => ProcNode::PidEnviron { pid }, + [_pid, "cwd"] => ProcNode::PidCwdLink { pid }, + [_pid, "stat"] => ProcNode::PidStatFile { pid }, + [_pid, "fd", fd] => { + let fd = fd + .parse::() + .map_err(|_| proc_not_found_error(&normalized))?; + self.proc_fd_entry(pid, fd)?; + ProcNode::PidFdLink { pid, fd } + } + _ => return Err(proc_not_found_error(&normalized)), + }; + + Ok(Some(node)) + } + + fn proc_entry(&self, pid: u32) -> KernelResult { + self.processes + .get(pid) + .ok_or_else(|| proc_not_found_error(&format!("/proc/{pid}"))) + } + + fn proc_fd_entry(&self, pid: u32, fd: u32) -> KernelResult { + lock_or_recover(&self.fd_tables) + .get(pid) + .and_then(|table| table.get(fd)) + .cloned() + .ok_or_else(|| proc_not_found_error(&format!("/proc/{pid}/fd/{fd}"))) + } + + fn proc_read_file( + &mut self, + current_pid: Option, + node: &ProcNode, + ) -> KernelResult> { + match node { + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => { + let target = self.proc_symlink_target(node)?; + self.read_file_internal(current_pid, &target) + } + ProcNode::MountsFile => Ok(self.proc_mounts_bytes()), + ProcNode::PidCmdline { pid } => Ok(self.proc_cmdline_bytes(*pid)), + ProcNode::PidEnviron { pid } => Ok(self.proc_environ_bytes(*pid)), + ProcNode::PidStatFile { pid } => Ok(self.proc_stat_bytes(*pid)), + ProcNode::RootDir | ProcNode::PidDir { .. } | ProcNode::PidFdDir { .. } => { + Err(KernelError::new( + "EISDIR", + format!( + "illegal operation on a directory, read '{}'", + self.proc_canonical_path(node) + ), + )) + } + } + } + + fn proc_stat( + &mut self, + current_pid: Option, + node: &ProcNode, + ) -> KernelResult { + match node { + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => { + let target = self.proc_symlink_target(node)?; + self.stat_internal(current_pid, &target) + } + _ => self.proc_lstat(node), + } + } + + fn proc_lstat(&self, node: &ProcNode) -> KernelResult { + match node { + ProcNode::RootDir | ProcNode::PidDir { .. } | ProcNode::PidFdDir { .. } => { + Ok(proc_dir_stat(proc_inode(node))) + } + ProcNode::MountsFile => Ok(proc_file_stat( + proc_inode(node), + self.proc_mounts_bytes().len() as u64, + )), + ProcNode::PidCmdline { pid } => Ok(proc_file_stat( + proc_inode(node), + self.proc_cmdline_bytes(*pid).len() as u64, + )), + ProcNode::PidEnviron { pid } => Ok(proc_file_stat( + proc_inode(node), + self.proc_environ_bytes(*pid).len() as u64, + )), + ProcNode::PidStatFile { pid } => Ok(proc_file_stat( + proc_inode(node), + self.proc_stat_bytes(*pid).len() as u64, + )), + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => Ok(proc_symlink_stat( + proc_inode(node), + self.proc_read_link(node)?.len() as u64, + )), + } + } + + fn proc_read_link(&self, node: &ProcNode) -> KernelResult { + match node { + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => self.proc_symlink_target(node), + _ => Err(KernelError::new( + "EINVAL", + format!( + "invalid argument, readlink '{}'", + self.proc_canonical_path(node) + ), + )), + } + } + + fn proc_read_dir( + &mut self, + current_pid: Option, + node: &ProcNode, + ) -> KernelResult> { + match node { + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => { + let target = self.proc_symlink_target(node)?; + self.read_dir_internal(current_pid, &target) + } + ProcNode::RootDir => { + let mut entries = self + .processes + .list_processes() + .keys() + .map(|pid| pid.to_string()) + .collect::>(); + entries.push(String::from("mounts")); + entries.push(String::from("self")); + entries.sort(); + Ok(entries) + } + ProcNode::PidDir { .. } => Ok(vec![ + String::from("cmdline"), + String::from("cwd"), + String::from("environ"), + String::from("fd"), + String::from("stat"), + ]), + ProcNode::PidFdDir { pid } => { + let tables = lock_or_recover(&self.fd_tables); + let table = tables + .get(*pid) + .ok_or_else(|| proc_not_found_error(&format!("/proc/{pid}/fd")))?; + Ok(table.iter().map(|entry| entry.fd.to_string()).collect()) + } + _ => Err(KernelError::new( + "ENOTDIR", + format!( + "not a directory, scandir '{}'", + self.proc_canonical_path(node) + ), + )), + } + } + + fn proc_realpath(&self, current_pid: Option, node: &ProcNode) -> KernelResult { + match node { + ProcNode::SelfLink { .. } + | ProcNode::PidCwdLink { .. } + | ProcNode::PidFdLink { .. } => { + let target = self.proc_symlink_target(node)?; + self.realpath_internal(current_pid, &target) + } + _ => Ok(self.proc_canonical_path(node)), + } + } + + fn proc_symlink_target(&self, node: &ProcNode) -> KernelResult { + match node { + ProcNode::SelfLink { pid } => Ok(format!("/proc/{pid}")), + ProcNode::PidCwdLink { pid } => Ok(self.proc_entry(*pid)?.cwd), + ProcNode::PidFdLink { pid, fd } => { + Ok(self.proc_fd_entry(*pid, *fd)?.description.path().to_owned()) + } + _ => Err(KernelError::new( + "EINVAL", + format!( + "'{}' is not a symbolic link", + self.proc_canonical_path(node) + ), + )), + } + } + + fn proc_canonical_path(&self, node: &ProcNode) -> String { + match node { + ProcNode::RootDir => String::from("/proc"), + ProcNode::MountsFile => String::from("/proc/mounts"), + ProcNode::SelfLink { pid } => format!("/proc/{pid}"), + ProcNode::PidDir { pid } => format!("/proc/{pid}"), + ProcNode::PidFdDir { pid } => format!("/proc/{pid}/fd"), + ProcNode::PidCmdline { pid } => format!("/proc/{pid}/cmdline"), + ProcNode::PidEnviron { pid } => format!("/proc/{pid}/environ"), + ProcNode::PidCwdLink { pid } => format!("/proc/{pid}/cwd"), + ProcNode::PidStatFile { pid } => format!("/proc/{pid}/stat"), + ProcNode::PidFdLink { pid, fd } => format!("/proc/{pid}/fd/{fd}"), + } + } + + fn proc_cmdline_bytes(&self, pid: u32) -> Vec { + let entry = self + .processes + .get(pid) + .expect("process must exist while procfs path is resolved"); + let mut argv = vec![entry.command]; + argv.extend(entry.args); + null_separated_bytes(argv) + } + + fn proc_environ_bytes(&self, pid: u32) -> Vec { + let entry = self + .processes + .get(pid) + .expect("process must exist while procfs path is resolved"); + null_separated_bytes( + entry + .env + .into_iter() + .map(|(key, value)| format!("{key}={value}")) + .collect(), + ) + } + + fn proc_stat_bytes(&self, pid: u32) -> Vec { + let entry = self + .processes + .get(pid) + .expect("process must exist while procfs path is resolved"); + let command = entry.command.replace(')', "]"); + let state = match entry.status { + ProcessStatus::Running => 'R', + ProcessStatus::Stopped => 'T', + ProcessStatus::Exited => 'Z', + }; + format!( + "{pid} ({command}) {state} {ppid} {pgid} {sid} 0 0 0 0 0 0 0 0 0 0 20 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0", + ppid = entry.ppid, + pgid = entry.pgid, + sid = entry.sid, + ) + .into_bytes() + } + + fn proc_mounts_bytes(&self) -> Vec { + let mounts = if let Some(table) = + (self.filesystem.inner().inner() as &dyn Any).downcast_ref::() + { + table.get_mounts() + } else { + vec![MountEntry { + path: String::from("/"), + plugin_id: String::from("root"), + read_only: false, + }] + }; + + mounts + .into_iter() + .map(|mount| { + let options = if mount.read_only { "ro" } else { "rw" }; + format!( + "{source} {target} {fstype} {options} 0 0\n", + source = mount.plugin_id, + target = mount.path, + fstype = mount.plugin_id, + ) + }) + .collect::() + .into_bytes() + } + + fn proc_read_file_from_open_path( + &mut self, + current_pid: Option, + path: &str, + ) -> KernelResult> { + let node = self + .resolve_proc_node(path, current_pid)? + .ok_or_else(|| proc_not_found_error(path))?; + self.proc_read_file(current_pid, &node) + } + + fn proc_stat_from_open_path( + &mut self, + current_pid: Option, + path: &str, + ) -> KernelResult { + let node = self + .resolve_proc_node(path, current_pid)? + .ok_or_else(|| proc_not_found_error(path))?; + self.proc_stat(current_pid, &node) + } + + fn filesystem_usage(&mut self) -> KernelResult { + Ok(measure_filesystem_usage(self.raw_filesystem_mut())?) + } + + fn storage_stat(&mut self, path: &str) -> KernelResult> { + if is_virtual_device_storage_path(path) { + return Ok(None); + } + + match self.raw_filesystem_mut().stat(path) { + Ok(stat) => Ok(Some(stat)), + Err(error) if error.code() == "ENOENT" => Ok(None), + Err(error) => Err(error.into()), + } + } + + fn storage_lstat(&mut self, path: &str) -> KernelResult> { + if is_virtual_device_storage_path(path) { + return Ok(None); + } + + match self.raw_filesystem_mut().lstat(path) { + Ok(stat) => Ok(Some(stat)), + Err(error) if error.code() == "ENOENT" => Ok(None), + Err(error) => Err(error.into()), + } + } + + fn current_storage_file_size(&mut self, path: &str) -> KernelResult { + Ok(self + .storage_stat(path)? + .filter(|stat| !stat.is_directory) + .map(|stat| stat.size) + .unwrap_or(0)) + } + + fn apply_creation_mode(&mut self, path: &str, mode: u32, umask: u32) -> KernelResult<()> { + let masked_mode = (mode & !0o777) | ((mode & 0o777) & !(umask & 0o777)); + Ok(self.filesystem.chmod(path, masked_mode)?) + } + + fn missing_directory_paths( + &mut self, + path: &str, + recursive: bool, + ) -> KernelResult> { + let normalized = normalize_path(path); + if normalized == "/" { + return Ok(Vec::new()); + } + + if !recursive { + return Ok(if self.storage_lstat(&normalized)?.is_none() { + vec![normalized] + } else { + Vec::new() + }); + } + + let mut created = Vec::new(); + let mut current = String::from("/"); + for component in normalized + .split('/') + .filter(|component| !component.is_empty()) + { + current = if current == "/" { + format!("/{component}") + } else { + format!("{current}/{component}") + }; + if self.storage_lstat(¤t)?.is_none() { + created.push(current.clone()); + } + } + Ok(created) + } + + fn check_write_file_limits(&mut self, path: &str, new_size: u64) -> KernelResult<()> { + if is_virtual_device_storage_path(path) { + return Ok(()); + } + + let usage = self.filesystem_usage()?; + if let Some(existing) = self.storage_stat(path)? { + if existing.is_directory { + return Ok(()); + } + + self.resources.check_filesystem_usage( + &usage, + usage + .total_bytes + .saturating_sub(existing.size) + .saturating_add(new_size), + usage.inode_count, + )?; + return Ok(()); + } + + let new_inodes = + count_missing_directory_components(self.raw_filesystem_mut(), path, false)? + .saturating_add(1); + self.resources.check_filesystem_usage( + &usage, + usage.total_bytes.saturating_add(new_size), + usage.inode_count.saturating_add(new_inodes), + )?; + Ok(()) + } + + fn check_create_dir_limits(&mut self, path: &str) -> KernelResult<()> { + if is_virtual_device_storage_path(path) || self.storage_lstat(path)?.is_some() { + return Ok(()); + } + + let parent = parent_path(path); + let Some(parent_stat) = self.storage_stat(&parent)? else { + return Ok(()); + }; + if !parent_stat.is_directory { + return Ok(()); + } + + let usage = self.filesystem_usage()?; + self.resources.check_filesystem_usage( + &usage, + usage.total_bytes, + usage.inode_count.saturating_add(1), + )?; + Ok(()) + } + + fn check_mkdir_limits(&mut self, path: &str, recursive: bool) -> KernelResult<()> { + if is_virtual_device_storage_path(path) { + return Ok(()); + } + + if !recursive { + return self.check_create_dir_limits(path); + } + + let usage = self.filesystem_usage()?; + let new_inodes = count_missing_directory_components(self.raw_filesystem_mut(), path, true)?; + self.resources.check_filesystem_usage( + &usage, + usage.total_bytes, + usage.inode_count.saturating_add(new_inodes), + )?; + Ok(()) + } + + fn check_symlink_limits(&mut self, target: &str, link_path: &str) -> KernelResult<()> { + if is_virtual_device_storage_path(link_path) || self.storage_lstat(link_path)?.is_some() { + return Ok(()); + } + + let parent = parent_path(link_path); + let Some(parent_stat) = self.storage_stat(&parent)? else { + return Ok(()); + }; + if !parent_stat.is_directory { + return Ok(()); + } + + let usage = self.filesystem_usage()?; + self.resources.check_filesystem_usage( + &usage, + usage.total_bytes.saturating_add(target.len() as u64), + usage.inode_count.saturating_add(1), + )?; + Ok(()) + } + + fn check_truncate_limits(&mut self, path: &str, length: u64) -> KernelResult<()> { + self.check_path_resize_limits(path, length) + } + + fn check_path_resize_limits(&mut self, path: &str, new_size: u64) -> KernelResult<()> { + if is_virtual_device_storage_path(path) { + return Ok(()); + } + + let Some(existing) = self.storage_stat(path)? else { + return Ok(()); + }; + if existing.is_directory { + return Ok(()); + } + + let usage = self.filesystem_usage()?; + self.resources.check_filesystem_usage( + &usage, + usage + .total_bytes + .saturating_sub(existing.size) + .saturating_add(new_size), + usage.inode_count, + )?; + Ok(()) + } + + fn blocking_read_timeout(&self) -> Option { + self.resources + .limits() + .max_blocking_read_ms + .map(Duration::from_millis) + } + fn close_special_resource_if_needed(&self, description: &Arc, filetype: u8) { - close_special_resource_if_needed(&self.pipes, &self.ptys, description, filetype); + close_special_resource_if_needed( + &self.file_locks, + &self.pipes, + &self.ptys, + description, + filetype, + ); } } impl KernelVm { + fn check_mount_permissions(&self, path: &str) -> KernelResult<()> { + self.filesystem + .check_path(FsOperation::Write, path) + .map_err(KernelError::from)?; + if is_sensitive_mount_path(path) { + self.filesystem + .check_path(FsOperation::MountSensitive, path) + .map_err(KernelError::from)?; + } + Ok(()) + } + pub fn mount_filesystem( &mut self, path: &str, @@ -1173,6 +2651,7 @@ impl KernelVm { options: MountOptions, ) -> KernelResult<()> { self.assert_not_terminated()?; + self.check_mount_permissions(path)?; self.filesystem .inner_mut() .inner_mut() @@ -1187,6 +2666,7 @@ impl KernelVm { options: MountOptions, ) -> KernelResult<()> { self.assert_not_terminated()?; + self.check_mount_permissions(path)?; self.filesystem .inner_mut() .inner_mut() @@ -1196,6 +2676,7 @@ impl KernelVm { pub fn unmount_filesystem(&mut self, path: &str) -> KernelResult<()> { self.assert_not_terminated()?; + self.check_mount_permissions(path)?; self.filesystem .inner_mut() .inner_mut() @@ -1238,7 +2719,7 @@ struct StubDriverProcess { impl StubDriverProcess { fn finish(&self, exit_code: i32) { let callback = { - let mut state = self.state.lock().expect("stub process lock poisoned"); + let mut state = lock_or_recover(&self.state); if state.exit_code.is_some() { return; } @@ -1253,39 +2734,38 @@ impl StubDriverProcess { } fn kill_signals(&self) -> Vec { - self.state - .lock() - .expect("stub process lock poisoned") - .kill_signals - .clone() + lock_or_recover(&self.state).kill_signals.clone() } } impl DriverProcess for StubDriverProcess { fn kill(&self, signal: i32) { { - let mut state = self.state.lock().expect("stub process lock poisoned"); + let mut state = lock_or_recover(&self.state); state.kill_signals.push(signal); } + if matches!( + signal, + crate::process_table::SIGCHLD | SIGCONT | SIGSTOP | SIGTSTP | SIGWINCH + ) { + return; + } self.finish(128 + signal); } fn wait(&self, timeout: Duration) -> Option { - let state = self.state.lock().expect("stub process lock poisoned"); + let state = lock_or_recover(&self.state); if let Some(code) = state.exit_code { return Some(code); } - let (state, _) = self - .waiters - .wait_timeout(state, timeout) - .expect("stub process wait lock poisoned"); + let (state, _) = wait_timeout_or_recover(&self.waiters, state, timeout); state.exit_code } fn set_on_exit(&self, callback: ProcessExitCallback) { let maybe_exit = { - let mut state = self.state.lock().expect("stub process lock poisoned"); + let mut state = lock_or_recover(&self.state); state.on_exit = Some(callback.clone()); state.exit_code }; @@ -1302,6 +2782,33 @@ impl From for KernelError { } } +fn lock_or_recover<'a, T>(mutex: &'a Mutex) -> MutexGuard<'a, T> { + match mutex.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + } +} + +fn wait_timeout_or_recover<'a, T>( + condvar: &Condvar, + guard: MutexGuard<'a, T>, + timeout: Duration, +) -> (MutexGuard<'a, T>, WaitTimeoutResult) { + match condvar.wait_timeout(guard, timeout) { + Ok(result) => result, + Err(poisoned) => poisoned.into_inner(), + } +} + +fn is_sensitive_mount_path(path: &str) -> bool { + let normalized = crate::vfs::normalize_path(path); + normalized == "/" + || normalized == "/etc" + || normalized.starts_with("/etc/") + || normalized == "/proc" + || normalized.starts_with("/proc/") +} + impl From for KernelError { fn from(error: FdTableError) -> Self { map_error(error.code(), error.to_string()) @@ -1372,6 +2879,85 @@ fn parse_dev_fd_path(path: &str) -> KernelResult> { Ok(Some(fd)) } +fn count_missing_directory_components( + filesystem: &mut F, + path: &str, + include_final: bool, +) -> VfsResult { + let normalized = normalize_path(path); + let parts = normalized + .split('/') + .filter(|part| !part.is_empty()) + .collect::>(); + let limit = if include_final { + parts.len() + } else { + parts.len().saturating_sub(1) + }; + + let mut current = String::from("/"); + for (index, part) in parts.iter().take(limit).enumerate() { + let candidate = if current == "/" { + format!("/{}", part) + } else { + format!("{current}/{}", part) + }; + + match filesystem.stat(&candidate) { + Ok(stat) => { + if !stat.is_directory { + return Err(VfsError::new( + "ENOTDIR", + format!("not a directory, mkdir '{candidate}'"), + )); + } + current = candidate; + } + Err(error) if error.code() == "ENOENT" => { + return Ok(limit.saturating_sub(index)); + } + Err(error) => return Err(error), + } + } + + Ok(0) +} + +fn parent_path(path: &str) -> String { + let normalized = normalize_path(path); + let Some((head, _)) = normalized.rsplit_once('/') else { + return String::from("/"); + }; + + if head.is_empty() { + String::from("/") + } else { + String::from(head) + } +} + +fn is_virtual_device_storage_path(path: &str) -> bool { + matches!( + path, + "/dev/null" | "/dev/zero" | "/dev/stdin" | "/dev/stdout" | "/dev/stderr" | "/dev/urandom" + ) || path == "/dev" + || path == "/dev/fd" + || path == "/dev/pts" + || path.starts_with("/dev/fd/") + || path.starts_with("/dev/pts/") +} + +fn is_proc_path(path: &str) -> bool { + let normalized = normalize_path(path); + normalized == "/proc" || normalized.starts_with("/proc/") +} + +fn checked_write_end(offset: u64, len: usize) -> KernelResult { + offset + .checked_add(len as u64) + .ok_or_else(|| KernelError::new("EINVAL", "write offset out of range")) +} + fn filetype_for_path(path: &str, stat: &VirtualStat) -> u8 { if stat.is_directory { FILETYPE_DIRECTORY @@ -1389,6 +2975,51 @@ fn synthetic_character_device_stat(ino: u64) -> VirtualStat { VirtualStat { mode: 0o666, size: 0, + blocks: 0, + dev: 2, + rdev: 0, + is_directory: false, + is_symbolic_link: false, + atime_ms: now, + mtime_ms: now, + ctime_ms: now, + birthtime_ms: now, + ino, + nlink: 1, + uid: 0, + gid: 0, + } +} + +fn proc_dir_stat(ino: u64) -> VirtualStat { + let now = now_ms(); + VirtualStat { + mode: 0o555, + size: 0, + blocks: 0, + dev: 3, + rdev: 0, + is_directory: true, + is_symbolic_link: false, + atime_ms: now, + mtime_ms: now, + ctime_ms: now, + birthtime_ms: now, + ino, + nlink: 2, + uid: 0, + gid: 0, + } +} + +fn proc_file_stat(ino: u64, size: u64) -> VirtualStat { + let now = now_ms(); + VirtualStat { + mode: 0o444, + size, + blocks: if size == 0 { 0 } else { size.div_ceil(512) }, + dev: 3, + rdev: 0, is_directory: false, is_symbolic_link: false, atime_ms: now, @@ -1402,9 +3033,140 @@ fn synthetic_character_device_stat(ino: u64) -> VirtualStat { } } +fn proc_symlink_stat(ino: u64, size: u64) -> VirtualStat { + let now = now_ms(); + VirtualStat { + mode: 0o777, + size, + blocks: if size == 0 { 0 } else { size.div_ceil(512) }, + dev: 3, + rdev: 0, + is_directory: false, + is_symbolic_link: true, + atime_ms: now, + mtime_ms: now, + ctime_ms: now, + birthtime_ms: now, + ino, + nlink: 1, + uid: 0, + gid: 0, + } +} + +fn proc_filetype(node: &ProcNode) -> u8 { + match node { + ProcNode::RootDir | ProcNode::PidDir { .. } | ProcNode::PidFdDir { .. } => { + FILETYPE_DIRECTORY + } + ProcNode::SelfLink { .. } | ProcNode::PidCwdLink { .. } | ProcNode::PidFdLink { .. } => { + FILETYPE_SYMBOLIC_LINK + } + ProcNode::MountsFile + | ProcNode::PidCmdline { .. } + | ProcNode::PidEnviron { .. } + | ProcNode::PidStatFile { .. } => FILETYPE_REGULAR_FILE, + } +} + +fn proc_inode(node: &ProcNode) -> u64 { + match node { + ProcNode::RootDir => 0xfffe_0001, + ProcNode::MountsFile => 0xfffe_0002, + ProcNode::SelfLink { pid } => 0xfffe_1000 + u64::from(*pid), + ProcNode::PidDir { pid } => 0xfffe_2000 + u64::from(*pid), + ProcNode::PidFdDir { pid } => 0xfffe_3000 + u64::from(*pid), + ProcNode::PidCmdline { pid } => 0xfffe_4000 + u64::from(*pid), + ProcNode::PidEnviron { pid } => 0xfffe_5000 + u64::from(*pid), + ProcNode::PidCwdLink { pid } => 0xfffe_6000 + u64::from(*pid), + ProcNode::PidStatFile { pid } => 0xfffe_7000 + u64::from(*pid), + ProcNode::PidFdLink { pid, fd } => 0xffff_0000 + ((u64::from(*pid)) << 8) + u64::from(*fd), + } +} + +fn null_separated_bytes(parts: Vec) -> Vec { + if parts.is_empty() { + return Vec::new(); + } + + let mut bytes = parts.join("\0").into_bytes(); + bytes.push(0); + bytes +} + +fn proc_not_found_error(path: &str) -> KernelError { + KernelError::new( + "ENOENT", + format!("no such file or directory, stat '{path}'"), + ) +} + +fn read_only_filesystem_error(path: &str) -> KernelError { + KernelError::new("EROFS", format!("read-only filesystem: {path}")) +} + fn now_ms() -> u64 { SystemTime::now() .duration_since(UNIX_EPOCH) .unwrap_or_default() .as_millis() as u64 } + +#[cfg(test)] +mod tests { + use super::*; + use crate::vfs::MemoryFileSystem; + + #[test] + fn setpgid_rejects_joining_a_process_group_owned_by_another_driver() { + let kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-setpgid")); + + let leader_pid = kernel.processes.allocate_pid(); + kernel.processes.register( + leader_pid, + String::from("driver-a"), + String::from("sh"), + Vec::new(), + ProcessContext { + pid: leader_pid, + ppid: 0, + env: BTreeMap::new(), + cwd: String::from("/"), + umask: DEFAULT_PROCESS_UMASK, + fds: Default::default(), + }, + Arc::new(StubDriverProcess::default()), + ); + + let peer_pid = kernel.processes.allocate_pid(); + kernel.processes.register( + peer_pid, + String::from("driver-b"), + String::from("sh"), + Vec::new(), + ProcessContext { + pid: peer_pid, + ppid: leader_pid, + env: BTreeMap::new(), + cwd: String::from("/"), + umask: DEFAULT_PROCESS_UMASK, + fds: Default::default(), + }, + Arc::new(StubDriverProcess::default()), + ); + + lock_or_recover(&kernel.driver_pids) + .entry(String::from("driver-a")) + .or_default() + .insert(leader_pid); + lock_or_recover(&kernel.driver_pids) + .entry(String::from("driver-b")) + .or_default() + .insert(peer_pid); + + let error = kernel + .setpgid("driver-b", peer_pid, leader_pid) + .expect_err("cross-driver process-group join should be denied"); + assert_eq!(error.code(), "EPERM"); + } +} diff --git a/crates/kernel/src/lib.rs b/crates/kernel/src/lib.rs index c4b1b1063..eac75a98b 100644 --- a/crates/kernel/src/lib.rs +++ b/crates/kernel/src/lib.rs @@ -12,6 +12,7 @@ pub mod mount_table; pub mod overlay_fs; pub mod permissions; pub mod pipe_manager; +pub mod poll; pub mod process_table; pub mod pty; pub mod resource_accounting; diff --git a/crates/kernel/src/mount_table.rs b/crates/kernel/src/mount_table.rs index 5e1d3b4f1..8cecfb7f7 100644 --- a/crates/kernel/src/mount_table.rs +++ b/crates/kernel/src/mount_table.rs @@ -8,8 +8,36 @@ pub trait MountedFileSystem: Any { fn as_any_mut(&mut self) -> &mut dyn Any; fn read_file(&mut self, path: &str) -> VfsResult>; fn read_dir(&mut self, path: &str) -> VfsResult>; + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + let entries = self.read_dir(path)?; + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + Ok(entries) + } fn read_dir_with_types(&mut self, path: &str) -> VfsResult>; fn write_file(&mut self, path: &str, content: Vec) -> VfsResult<()>; + fn create_file_exclusive(&mut self, path: &str, content: Vec) -> VfsResult<()> { + if self.exists(path) { + return Err(VfsError::new( + "EEXIST", + format!("file already exists, open '{path}'"), + )); + } + self.write_file(path, content) + } + fn append_file(&mut self, path: &str, content: Vec) -> VfsResult { + let mut existing = self.read_file(path)?; + existing.extend_from_slice(&content); + let new_len = existing.len() as u64; + self.write_file(path, existing)?; + Ok(new_len) + } fn create_dir(&mut self, path: &str) -> VfsResult<()>; fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()>; fn exists(&self, path: &str) -> bool; @@ -70,6 +98,10 @@ where VirtualFileSystem::read_dir(&mut self.inner, path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + VirtualFileSystem::read_dir_limited(&mut self.inner, path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { VirtualFileSystem::read_dir_with_types(&mut self.inner, path) } @@ -78,6 +110,14 @@ where VirtualFileSystem::write_file(&mut self.inner, path, content) } + fn create_file_exclusive(&mut self, path: &str, content: Vec) -> VfsResult<()> { + VirtualFileSystem::create_file_exclusive(&mut self.inner, path, content) + } + + fn append_file(&mut self, path: &str, content: Vec) -> VfsResult { + VirtualFileSystem::append_file(&mut self.inner, path, content) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { VirtualFileSystem::create_dir(&mut self.inner, path) } @@ -167,6 +207,10 @@ where (**self).read_dir(path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + (**self).read_dir_limited(path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { (**self).read_dir_with_types(path) } @@ -278,6 +322,10 @@ where self.inner.read_dir(path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + self.inner.read_dir_limited(path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { self.inner.read_dir_with_types(path) } @@ -609,6 +657,31 @@ impl VirtualFileSystem for MountTable { Ok(merged.into_iter().collect()) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + let normalized = normalize_path(path); + let (index, relative_path) = self.resolve_index(&normalized)?; + let mut entries = self.mounts[index] + .filesystem + .read_dir_limited(&relative_path, max_entries)?; + let child_mounts = self.child_mount_basenames(&normalized); + if child_mounts.is_empty() { + return Ok(entries); + } + + let mut merged = BTreeSet::new(); + merged.extend(entries.drain(..)); + merged.extend(child_mounts); + if merged.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + Ok(merged.into_iter().collect()) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { let normalized = normalize_path(path); let (index, relative_path) = self.resolve_index(&normalized)?; @@ -644,6 +717,20 @@ impl VirtualFileSystem for MountTable { .write_file(&relative_path, content.into()) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + let (index, relative_path) = self.resolve_index(path)?; + self.mounts[index] + .filesystem + .create_file_exclusive(&relative_path, content.into()) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + let (index, relative_path) = self.resolve_index(path)?; + self.mounts[index] + .filesystem + .append_file(&relative_path, content.into()) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { let (index, relative_path) = self.resolve_index(path)?; self.mounts[index].filesystem.create_dir(&relative_path) @@ -710,7 +797,23 @@ impl VirtualFileSystem for MountTable { } fn symlink(&mut self, target: &str, link_path: &str) -> VfsResult<()> { - let (index, relative_path) = self.resolve_index(link_path)?; + let normalized_link_path = normalize_path(link_path); + let link_parent = parent_path(&normalized_link_path); + let absolute_target = if target.starts_with('/') { + normalize_path(target) + } else { + normalize_path(&format!("{link_parent}/{target}")) + }; + + let (index, relative_path) = self.resolve_index(&normalized_link_path)?; + let (target_index, _) = self.resolve_index(&absolute_target)?; + if index != target_index { + return Err(VfsError::new( + "EXDEV", + format!("symlink across mounts: {link_path} -> {target}"), + )); + } + self.mounts[index] .filesystem .symlink(target, &relative_path) diff --git a/crates/kernel/src/overlay_fs.rs b/crates/kernel/src/overlay_fs.rs index e10ea098a..e3b5f2ffb 100644 --- a/crates/kernel/src/overlay_fs.rs +++ b/crates/kernel/src/overlay_fs.rs @@ -2,8 +2,14 @@ use crate::vfs::{ normalize_path, MemoryFileSystem, VfsError, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat, }; +use base64::Engine; use std::collections::BTreeSet; +const MAX_SNAPSHOT_DEPTH: usize = 1024; +const OVERLAY_METADATA_ROOT: &str = "/.agent-os-overlay"; +const OVERLAY_WHITEOUT_DIR: &str = "/.agent-os-overlay/whiteouts"; +const OVERLAY_OPAQUE_DIR: &str = "/.agent-os-overlay/opaque"; + #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum OverlayMode { Ephemeral, @@ -14,10 +20,15 @@ pub enum OverlayMode { pub struct OverlayFileSystem { lowers: Vec, upper: Option, - whiteouts: BTreeSet, writes_locked: bool, } +#[derive(Debug, Clone, Copy)] +enum OverlayMarkerKind { + Whiteout, + Opaque, +} + #[derive(Debug)] enum OverlaySnapshotKind { Directory, @@ -50,7 +61,6 @@ impl OverlayFileSystem { Self { lowers: effective_lowers, upper, - whiteouts: BTreeSet::new(), writes_locked: matches!(mode, OverlayMode::ReadOnly), } } @@ -64,7 +74,6 @@ impl OverlayFileSystem { Self { lowers: effective_lowers, upper: Some(upper), - whiteouts: BTreeSet::new(), writes_locked: false, } } @@ -77,16 +86,102 @@ impl OverlayFileSystem { normalize_path(path) } + fn encode_marker_path(path: &str) -> String { + base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(path) + } + + fn marker_directory(kind: OverlayMarkerKind) -> &'static str { + match kind { + OverlayMarkerKind::Whiteout => OVERLAY_WHITEOUT_DIR, + OverlayMarkerKind::Opaque => OVERLAY_OPAQUE_DIR, + } + } + + fn marker_path(kind: OverlayMarkerKind, path: &str) -> String { + format!( + "{}/{}", + Self::marker_directory(kind), + Self::encode_marker_path(&Self::normalized(path)) + ) + } + + fn is_internal_metadata_path(path: &str) -> bool { + let normalized = Self::normalized(path); + normalized == OVERLAY_METADATA_ROOT + || normalized.starts_with(&(String::from(OVERLAY_METADATA_ROOT) + "/")) + } + + fn hidden_root_entry_name() -> &'static str { + ".agent-os-overlay" + } + + fn should_hide_directory_entry(path: &str, entry: &str) -> bool { + let normalized = Self::normalized(path); + normalized == "/" && entry == Self::hidden_root_entry_name() + } + + fn marker_exists(&self, kind: OverlayMarkerKind, path: &str) -> bool { + Self::marker_exists_in_upper(self.upper.as_ref(), kind, path) + } + + fn marker_exists_in_upper( + upper: Option<&MemoryFileSystem>, + kind: OverlayMarkerKind, + path: &str, + ) -> bool { + upper.is_some_and(|filesystem| filesystem.exists(&Self::marker_path(kind, path))) + } + fn is_whited_out(&self, path: &str) -> bool { - self.whiteouts.contains(&Self::normalized(path)) + self.marker_exists(OverlayMarkerKind::Whiteout, path) } - fn add_whiteout(&mut self, path: &str) { - self.whiteouts.insert(Self::normalized(path)); + fn ensure_metadata_directories_in_upper(&mut self, path: &str) -> VfsResult<()> { + let upper = self.writable_upper(path)?; + upper.mkdir(OVERLAY_METADATA_ROOT, true)?; + upper.mkdir(OVERLAY_WHITEOUT_DIR, true)?; + upper.mkdir(OVERLAY_OPAQUE_DIR, true)?; + Ok(()) } - fn remove_whiteout(&mut self, path: &str) { - self.whiteouts.remove(&Self::normalized(path)); + fn set_marker(&mut self, kind: OverlayMarkerKind, path: &str, present: bool) -> VfsResult<()> { + let marker_path = Self::marker_path(kind, path); + if present { + self.ensure_metadata_directories_in_upper(path)?; + self.writable_upper(path)? + .write_file(&marker_path, Self::normalized(path).into_bytes())?; + return Ok(()); + } + + if self + .upper + .as_ref() + .is_some_and(|upper| upper.exists(&marker_path)) + { + self.writable_upper(path)?.remove_file(&marker_path)?; + } + Ok(()) + } + + fn add_whiteout(&mut self, path: &str) -> VfsResult<()> { + self.set_marker(OverlayMarkerKind::Whiteout, path, true) + } + + fn remove_whiteout(&mut self, path: &str) -> VfsResult<()> { + self.set_marker(OverlayMarkerKind::Whiteout, path, false) + } + + fn mark_opaque_directory(&mut self, path: &str) -> VfsResult<()> { + self.set_marker(OverlayMarkerKind::Opaque, path, true) + } + + fn clear_opaque_directory(&mut self, path: &str) -> VfsResult<()> { + self.set_marker(OverlayMarkerKind::Opaque, path, false) + } + + fn clear_path_metadata(&mut self, path: &str) -> VfsResult<()> { + self.remove_whiteout(path)?; + self.clear_opaque_directory(path) } fn join_path(base: &str, name: &str) -> String { @@ -168,6 +263,9 @@ impl OverlayFileSystem { } fn merged_lstat(&self, path: &str) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -184,6 +282,9 @@ impl OverlayFileSystem { } fn ensure_ancestor_directories_in_upper(&mut self, path: &str) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("mkdir", path)); + } let normalized = Self::normalized(path); let parts = normalized .split('/') @@ -242,6 +343,7 @@ impl OverlayFileSystem { upper.mkdir(path, false)?; upper.chmod(path, stat.mode)?; upper.chown(path, stat.uid, stat.gid)?; + self.mark_opaque_directory(path)?; return Ok(()); } @@ -267,92 +369,51 @@ impl OverlayFileSystem { VfsError::new("ENOTEMPTY", format!("directory not empty, rmdir '{path}'")) } - fn remove_existing_destination(&mut self, path: &str) -> VfsResult<()> { - let stat = match self.merged_lstat(path) { - Ok(stat) => stat, - Err(error) if error.code() == "ENOENT" => return Ok(()), - Err(error) => return Err(error), - }; - - if stat.is_directory && !stat.is_symbolic_link { - if !self.read_dir(path)?.is_empty() { - return Err(Self::not_empty(path)); - } - self.remove_dir(path) - } else { - self.remove_file(path) - } - } - fn collect_snapshot_entries( &mut self, path: &str, entries: &mut Vec, ) -> VfsResult<()> { - let normalized = Self::normalized(path); - let stat = self.lstat(&normalized)?; - - if stat.is_symbolic_link { - entries.push(OverlaySnapshotEntry { - path: normalized, - stat, - kind: OverlaySnapshotKind::Symlink(self.read_link(path)?), - }); - return Ok(()); - } - - if stat.is_directory { - entries.push(OverlaySnapshotEntry { - path: normalized.clone(), - stat, - kind: OverlaySnapshotKind::Directory, - }); - - for entry in self.read_dir_with_types(&normalized)? { - let child_path = Self::join_path(&normalized, &entry.name); - self.collect_snapshot_entries(&child_path, entries)?; + let mut pending = vec![(Self::normalized(path), 0usize)]; + while let Some((current_path, depth)) = pending.pop() { + if depth > MAX_SNAPSHOT_DEPTH { + return Err(VfsError::new( + "EINVAL", + format!("overlay snapshot depth limit exceeded at '{current_path}'"), + )); } - return Ok(()); - } + let stat = self.lstat(¤t_path)?; - entries.push(OverlaySnapshotEntry { - path: normalized, - stat, - kind: OverlaySnapshotKind::File(self.read_file(path)?), - }); - Ok(()) - } + if stat.is_symbolic_link { + entries.push(OverlaySnapshotEntry { + path: current_path.clone(), + stat, + kind: OverlaySnapshotKind::Symlink(self.read_link(¤t_path)?), + }); + continue; + } - fn materialize_snapshot_entries( - &mut self, - old_root: &str, - new_root: &str, - entries: &[OverlaySnapshotEntry], - ) -> VfsResult<()> { - for entry in entries { - let destination = Self::rebase_path(&entry.path, old_root, new_root); + if stat.is_directory { + entries.push(OverlaySnapshotEntry { + path: current_path.clone(), + stat, + kind: OverlaySnapshotKind::Directory, + }); - match &entry.kind { - OverlaySnapshotKind::Directory => { - self.create_dir(&destination)?; - self.chmod(&destination, entry.stat.mode)?; - self.chown(&destination, entry.stat.uid, entry.stat.gid)?; - } - OverlaySnapshotKind::File(data) => { - self.write_file(&destination, data.clone())?; - self.chmod(&destination, entry.stat.mode)?; - self.chown(&destination, entry.stat.uid, entry.stat.gid)?; - } - OverlaySnapshotKind::Symlink(target) => { - self.remove_whiteout(&destination); - self.ensure_ancestor_directories_in_upper(&destination)?; - self.writable_upper(&destination)? - .symlink(target, &destination)?; + let children = self.read_dir_with_types(¤t_path)?; + for entry in children.into_iter().rev() { + pending.push((Self::join_path(¤t_path, &entry.name), depth + 1)); } + continue; } - } + entries.push(OverlaySnapshotEntry { + path: current_path.clone(), + stat, + kind: OverlaySnapshotKind::File(self.read_file(¤t_path)?), + }); + } Ok(()) } @@ -370,9 +431,159 @@ impl OverlayFileSystem { } if self.find_lower_by_entry(&entry.path).is_some() { - self.add_whiteout(&entry.path); + self.clear_opaque_directory(&entry.path)?; + self.add_whiteout(&entry.path)?; } else { - self.remove_whiteout(&entry.path); + self.clear_path_metadata(&entry.path)?; + } + } + + Ok(()) + } + + fn directory_has_raw_children(&mut self, path: &str) -> VfsResult { + let normalized = Self::normalized(path); + let mut directory_exists = false; + + if let Some(upper) = self.upper.as_mut() { + if let Ok(entries) = upper.read_dir(&normalized) { + directory_exists = true; + if entries.into_iter().any(|entry| { + entry != "." + && entry != ".." + && !Self::should_hide_directory_entry(&normalized, &entry) + }) { + return Ok(true); + } + } + } + + for lower in self.lowers.iter_mut().rev() { + if let Ok(entries) = lower.read_dir(&normalized) { + directory_exists = true; + if entries.into_iter().any(|entry| { + entry != "." + && entry != ".." + && !Self::should_hide_directory_entry(&normalized, &entry) + }) { + return Ok(true); + } + } + } + + if !directory_exists { + return Err(Self::directory_not_found(path)); + } + + Ok(false) + } + + fn marker_paths_in_upper(&mut self, kind: OverlayMarkerKind) -> VfsResult> { + let Some(upper) = self.upper.as_mut() else { + return Ok(Vec::new()); + }; + + let marker_dir = Self::marker_directory(kind); + let entries = match upper.read_dir(marker_dir) { + Ok(entries) => entries, + Err(error) if error.code() == "ENOENT" => return Ok(Vec::new()), + Err(error) => return Err(error), + }; + + let mut marker_paths = Vec::new(); + for entry in entries { + if entry == "." || entry == ".." { + continue; + } + + let marker_file = Self::join_path(marker_dir, &entry); + let marker_path = + String::from_utf8(upper.read_file(&marker_file).map_err(|_| { + VfsError::io(format!("invalid overlay marker '{marker_file}'")) + })?) + .map_err(|_| VfsError::io(format!("invalid overlay marker '{marker_file}'")))?; + marker_paths.push(Self::normalized(&marker_path)); + } + + Ok(marker_paths) + } + + fn path_in_subtree(path: &str, root: &str) -> bool { + path == root || path.starts_with(&(String::from(root) + "/")) + } + + fn clear_subtree_metadata(&mut self, path: &str) -> VfsResult<()> { + let normalized = Self::normalized(path); + for kind in [OverlayMarkerKind::Whiteout, OverlayMarkerKind::Opaque] { + for marker_path in self.marker_paths_in_upper(kind)? { + if Self::path_in_subtree(&marker_path, &normalized) { + self.set_marker(kind, &marker_path, false)?; + } + } + } + Ok(()) + } + + fn copy_subtree_metadata(&mut self, old_root: &str, new_root: &str) -> VfsResult<()> { + let old_normalized = Self::normalized(old_root); + let new_normalized = Self::normalized(new_root); + + for kind in [OverlayMarkerKind::Whiteout, OverlayMarkerKind::Opaque] { + for marker_path in self.marker_paths_in_upper(kind)? { + if Self::path_in_subtree(&marker_path, &old_normalized) { + let destination = + Self::rebase_path(&marker_path, &old_normalized, &new_normalized); + self.set_marker(kind, &destination, true)?; + } + } + } + + Ok(()) + } + + fn stage_snapshot_entries_in_upper( + &mut self, + entries: &[OverlaySnapshotEntry], + ) -> VfsResult<()> { + for entry in entries { + match &entry.kind { + OverlaySnapshotKind::Directory => { + if !self.has_entry_in_upper(&entry.path) { + self.ensure_ancestor_directories_in_upper(&entry.path)?; + self.writable_upper(&entry.path)?.create_dir(&entry.path)?; + } + self.writable_upper(&entry.path)? + .chmod(&entry.path, entry.stat.mode)?; + self.writable_upper(&entry.path)?.chown( + &entry.path, + entry.stat.uid, + entry.stat.gid, + )?; + self.mark_opaque_directory(&entry.path)?; + } + OverlaySnapshotKind::File(data) => { + if self.has_entry_in_upper(&entry.path) { + continue; + } + self.ensure_ancestor_directories_in_upper(&entry.path)?; + self.writable_upper(&entry.path)? + .write_file(&entry.path, data.clone())?; + self.writable_upper(&entry.path)? + .chmod(&entry.path, entry.stat.mode)?; + self.writable_upper(&entry.path)?.chown( + &entry.path, + entry.stat.uid, + entry.stat.gid, + )?; + } + OverlaySnapshotKind::Symlink(target) => { + if self.has_entry_in_upper(&entry.path) { + continue; + } + self.ensure_ancestor_directories_in_upper(&entry.path)?; + self.writable_upper(&entry.path)? + .symlink(target, &entry.path)?; + } } } @@ -395,6 +606,9 @@ fn sync_upper_root_metadata(upper: &mut MemoryFileSystem, lowers: &[MemoryFileSy impl VirtualFileSystem for OverlayFileSystem { fn read_file(&mut self, path: &str) -> VfsResult> { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -412,6 +626,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn read_dir(&mut self, path: &str) -> VfsResult> { + if Self::is_internal_metadata_path(path) { + return Err(Self::directory_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::directory_not_found(path)); } @@ -419,22 +636,104 @@ impl VirtualFileSystem for OverlayFileSystem { let normalized = Self::normalized(path); let mut directory_exists = false; let mut entries = BTreeSet::new(); - let whiteouts = self.whiteouts.clone(); + let upper = self.upper.as_ref(); + let include_lowers = !Self::marker_exists_in_upper(upper, OverlayMarkerKind::Opaque, path); + + if include_lowers { + for lower in self.lowers.iter_mut().rev() { + if let Ok(lower_entries) = lower.read_dir(path) { + directory_exists = true; + for entry in lower_entries { + if entry == "." + || entry == ".." + || Self::should_hide_directory_entry(path, &entry) + { + continue; + } + let child_path = if normalized == "/" { + format!("/{entry}") + } else { + format!("{normalized}/{entry}") + }; + if !Self::marker_exists_in_upper( + upper, + OverlayMarkerKind::Whiteout, + &child_path, + ) { + entries.insert(entry); + } + } + } + } + } - for lower in self.lowers.iter_mut().rev() { - if let Ok(lower_entries) = lower.read_dir(path) { + if let Some(upper) = self.upper.as_mut() { + if let Ok(upper_entries) = upper.read_dir(path) { directory_exists = true; - for entry in lower_entries { - if entry == "." || entry == ".." { + for entry in upper_entries { + if entry == "." + || entry == ".." + || Self::should_hide_directory_entry(path, &entry) + { continue; } - let child_path = if normalized == "/" { - format!("/{entry}") - } else { - format!("{normalized}/{entry}") - }; - if !whiteouts.contains(&Self::normalized(&child_path)) { - entries.insert(entry); + entries.insert(entry); + } + } + } + + if !directory_exists { + return Err(Self::directory_not_found(path)); + } + + Ok(entries.into_iter().collect()) + } + + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + if Self::is_internal_metadata_path(path) { + return Err(Self::directory_not_found(path)); + } + if self.is_whited_out(path) { + return Err(Self::directory_not_found(path)); + } + + let normalized = Self::normalized(path); + let mut directory_exists = false; + let mut entries = BTreeSet::new(); + let upper = self.upper.as_ref(); + let include_lowers = !Self::marker_exists_in_upper(upper, OverlayMarkerKind::Opaque, path); + + if include_lowers { + for lower in self.lowers.iter_mut().rev() { + if let Ok(lower_entries) = lower.read_dir(path) { + directory_exists = true; + for entry in lower_entries { + if entry == "." + || entry == ".." + || Self::should_hide_directory_entry(path, &entry) + { + continue; + } + let child_path = if normalized == "/" { + format!("/{entry}") + } else { + format!("{normalized}/{entry}") + }; + if !Self::marker_exists_in_upper( + upper, + OverlayMarkerKind::Whiteout, + &child_path, + ) { + entries.insert(entry); + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + } } } } @@ -444,10 +743,21 @@ impl VirtualFileSystem for OverlayFileSystem { if let Ok(upper_entries) = upper.read_dir(path) { directory_exists = true; for entry in upper_entries { - if entry == "." || entry == ".." { + if entry == "." + || entry == ".." + || Self::should_hide_directory_entry(path, &entry) + { continue; } entries.insert(entry); + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } } } } @@ -460,6 +770,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { + if Self::is_internal_metadata_path(path) { + return Err(Self::directory_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::directory_not_found(path)); } @@ -468,27 +781,36 @@ impl VirtualFileSystem for OverlayFileSystem { let mut directory_exists = false; let mut entries = Vec::::new(); let mut seen = BTreeSet::::new(); - let whiteouts = self.whiteouts.clone(); - - for lower in self.lowers.iter_mut().rev() { - if let Ok(lower_entries) = lower.read_dir_with_types(path) { - directory_exists = true; - for entry in lower_entries { - if entry.name == "." || entry.name == ".." { - continue; - } - let child_path = if normalized == "/" { - format!("/{}", entry.name) - } else { - format!("{normalized}/{}", entry.name) - }; - if whiteouts.contains(&Self::normalized(&child_path)) - || seen.contains(&entry.name) - { - continue; + let upper = self.upper.as_ref(); + let include_lowers = !Self::marker_exists_in_upper(upper, OverlayMarkerKind::Opaque, path); + + if include_lowers { + for lower in self.lowers.iter_mut().rev() { + if let Ok(lower_entries) = lower.read_dir_with_types(path) { + directory_exists = true; + for entry in lower_entries { + if entry.name == "." + || entry.name == ".." + || Self::should_hide_directory_entry(path, &entry.name) + { + continue; + } + let child_path = if normalized == "/" { + format!("/{}", entry.name) + } else { + format!("{normalized}/{}", entry.name) + }; + if Self::marker_exists_in_upper( + upper, + OverlayMarkerKind::Whiteout, + &child_path, + ) || seen.contains(&entry.name) + { + continue; + } + seen.insert(entry.name.clone()); + entries.push(entry); } - seen.insert(entry.name.clone()); - entries.push(entry); } } } @@ -497,7 +819,10 @@ impl VirtualFileSystem for OverlayFileSystem { if let Ok(upper_entries) = upper.read_dir_with_types(path) { directory_exists = true; for entry in upper_entries { - if entry.name == "." || entry.name == ".." { + if entry.name == "." + || entry.name == ".." + || Self::should_hide_directory_entry(path, &entry.name) + { continue; } if let Some(index) = entries @@ -521,7 +846,10 @@ impl VirtualFileSystem for OverlayFileSystem { } fn write_file(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { - self.remove_whiteout(path); + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("open", path)); + } + self.clear_path_metadata(path)?; if self.find_lower_by_entry(path).is_some() { self.copy_up_path(path)?; } else { @@ -530,8 +858,37 @@ impl VirtualFileSystem for OverlayFileSystem { self.writable_upper(path)?.write_file(path, content.into()) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("open", path)); + } + self.clear_path_metadata(path)?; + if self.path_exists_in_merged_view(path) { + return Err(Self::already_exists(path)); + } + self.ensure_ancestor_directories_in_upper(path)?; + self.writable_upper(path)? + .create_file_exclusive(path, content.into()) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("open", path)); + } + self.clear_path_metadata(path)?; + if self.find_lower_by_entry(path).is_some() { + self.copy_up_path(path)?; + } else { + self.ensure_ancestor_directories_in_upper(path)?; + } + self.writable_upper(path)?.append_file(path, content.into()) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { - self.remove_whiteout(path); + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("mkdir", path)); + } + self.clear_path_metadata(path)?; if self.path_exists_in_merged_view(path) { return Err(Self::already_exists(path)); } @@ -540,7 +897,10 @@ impl VirtualFileSystem for OverlayFileSystem { } fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()> { - self.remove_whiteout(path); + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("mkdir", path)); + } + self.clear_path_metadata(path)?; if self.path_exists_in_merged_view(path) { let stat = self.merged_lstat(path)?; if recursive && stat.is_directory && !stat.is_symbolic_link { @@ -553,10 +913,16 @@ impl VirtualFileSystem for OverlayFileSystem { } fn exists(&self, path: &str) -> bool { + if Self::is_internal_metadata_path(path) { + return false; + } self.path_exists_in_merged_view(path) } fn stat(&mut self, path: &str) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -574,6 +940,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn remove_file(&mut self, path: &str) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("unlink", path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -587,12 +956,16 @@ impl VirtualFileSystem for OverlayFileSystem { } else { self.writable_upper(path)?; } - self.add_whiteout(path); + self.clear_opaque_directory(path)?; + self.add_whiteout(path)?; Ok(()) } fn remove_dir(&mut self, path: &str) -> VfsResult<()> { let normalized = Self::normalized(path); + if Self::is_internal_metadata_path(&normalized) { + return Err(VfsError::permission_denied("rmdir", path)); + } if normalized == "/" { return Err(VfsError::permission_denied("rmdir", path)); } @@ -607,7 +980,7 @@ impl VirtualFileSystem for OverlayFileSystem { return Err(Self::not_directory(path)); } - if !self.read_dir(path)?.is_empty() { + if self.directory_has_raw_children(path)? { return Err(Self::not_empty(path)); } @@ -619,9 +992,10 @@ impl VirtualFileSystem for OverlayFileSystem { self.writable_upper(path)?; } if lower_exists { - self.add_whiteout(path); + self.clear_opaque_directory(path)?; + self.add_whiteout(path)?; } else { - self.remove_whiteout(path); + self.clear_path_metadata(path)?; } Ok(()) } @@ -629,6 +1003,11 @@ impl VirtualFileSystem for OverlayFileSystem { fn rename(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { let old_normalized = Self::normalized(old_path); let new_normalized = Self::normalized(new_path); + if Self::is_internal_metadata_path(&old_normalized) + || Self::is_internal_metadata_path(&new_normalized) + { + return Err(VfsError::permission_denied("rename", old_path)); + } if old_normalized == "/" { return Err(VfsError::permission_denied("rename", old_path)); @@ -651,12 +1030,38 @@ impl VirtualFileSystem for OverlayFileSystem { let mut snapshot_entries = Vec::new(); self.collect_snapshot_entries(&old_normalized, &mut snapshot_entries)?; - self.remove_existing_destination(&new_normalized)?; - self.materialize_snapshot_entries(&old_normalized, &new_normalized, &snapshot_entries)?; + + if let Ok(destination_stat) = self.merged_lstat(&new_normalized) { + if destination_stat.is_directory + && !destination_stat.is_symbolic_link + && !self.read_dir(&new_normalized)?.is_empty() + { + return Err(Self::not_empty(&new_normalized)); + } + + if self.has_entry_in_upper(&new_normalized) { + if destination_stat.is_directory && !destination_stat.is_symbolic_link { + self.writable_upper(&new_normalized)? + .remove_dir(&new_normalized)?; + } else { + self.writable_upper(&new_normalized)? + .remove_file(&new_normalized)?; + } + } + self.clear_subtree_metadata(&new_normalized)?; + } + + self.stage_snapshot_entries_in_upper(&snapshot_entries)?; + self.copy_subtree_metadata(&old_normalized, &new_normalized)?; + self.writable_upper(&old_normalized)? + .rename(&old_normalized, &new_normalized)?; self.remove_snapshot_entries(&snapshot_entries) } fn realpath(&self, path: &str) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -674,12 +1079,18 @@ impl VirtualFileSystem for OverlayFileSystem { } fn symlink(&mut self, target: &str, link_path: &str) -> VfsResult<()> { - self.remove_whiteout(link_path); + if Self::is_internal_metadata_path(link_path) { + return Err(VfsError::permission_denied("symlink", link_path)); + } + self.clear_path_metadata(link_path)?; self.ensure_ancestor_directories_in_upper(link_path)?; self.writable_upper(link_path)?.symlink(target, link_path) } fn read_link(&self, path: &str) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -697,6 +1108,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn lstat(&self, path: &str) -> VfsResult { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -713,13 +1127,19 @@ impl VirtualFileSystem for OverlayFileSystem { } fn link(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { - self.remove_whiteout(new_path); + if Self::is_internal_metadata_path(old_path) || Self::is_internal_metadata_path(new_path) { + return Err(VfsError::permission_denied("link", new_path)); + } + self.clear_path_metadata(new_path)?; self.copy_up_path(old_path)?; self.ensure_ancestor_directories_in_upper(new_path)?; self.writable_upper(new_path)?.link(old_path, new_path) } fn chmod(&mut self, path: &str, mode: u32) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("chmod", path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -730,6 +1150,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn chown(&mut self, path: &str, uid: u32, gid: u32) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("chown", path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -740,6 +1163,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn utimes(&mut self, path: &str, atime_ms: u64, mtime_ms: u64) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("utime", path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -750,6 +1176,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn truncate(&mut self, path: &str, length: u64) -> VfsResult<()> { + if Self::is_internal_metadata_path(path) { + return Err(VfsError::permission_denied("truncate", path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -760,6 +1189,9 @@ impl VirtualFileSystem for OverlayFileSystem { } fn pread(&mut self, path: &str, offset: u64, length: usize) -> VfsResult> { + if Self::is_internal_metadata_path(path) { + return Err(Self::entry_not_found(path)); + } if self.is_whited_out(path) { return Err(Self::entry_not_found(path)); } @@ -776,3 +1208,60 @@ impl VirtualFileSystem for OverlayFileSystem { self.lowers[index].pread(path, offset, length) } } + +#[cfg(test)] +mod tests { + use super::{OverlayFileSystem, OverlayMode}; + use crate::vfs::{MemoryFileSystem, VirtualFileSystem}; + + #[test] + fn whiteouts_persist_when_overlay_reopens_with_same_upper() { + let mut lower = MemoryFileSystem::new(); + lower.mkdir("/data", true).expect("create lower directory"); + lower + .write_file("/data/base.txt", b"base".to_vec()) + .expect("seed lower file"); + let lower_snapshot = lower.snapshot(); + + let mut overlay = OverlayFileSystem::with_upper( + vec![MemoryFileSystem::from_snapshot(lower_snapshot.clone())], + MemoryFileSystem::new(), + ); + overlay + .remove_file("/data/base.txt") + .expect("whiteout lower file"); + + let upper = overlay.upper.take().expect("overlay upper"); + let restored_lower = MemoryFileSystem::from_snapshot(lower_snapshot); + let mut restored = OverlayFileSystem::with_upper(vec![restored_lower], upper); + + assert!(!restored.exists("/data/base.txt")); + assert_eq!( + restored.read_dir("/data").expect("read merged directory"), + Vec::::new() + ); + } + + #[test] + fn copied_up_directories_become_opaque_and_hide_overlay_metadata() { + let mut lower = MemoryFileSystem::new(); + lower.mkdir("/data", true).expect("create lower directory"); + lower + .write_file("/data/base.txt", b"base".to_vec()) + .expect("seed lower file"); + + let mut overlay = OverlayFileSystem::new(vec![lower], OverlayMode::Ephemeral); + overlay + .chmod("/data", 0o700) + .expect("copy up lower directory"); + + assert_eq!( + overlay.read_dir("/data").expect("read opaque directory"), + Vec::::new() + ); + let root_entries = overlay.read_dir("/").expect("read root"); + assert!(!root_entries + .iter() + .any(|entry| entry == ".agent-os-overlay")); + } +} diff --git a/crates/kernel/src/permissions.rs b/crates/kernel/src/permissions.rs index 49e129289..b2fcfc9a0 100644 --- a/crates/kernel/src/permissions.rs +++ b/crates/kernel/src/permissions.rs @@ -1,7 +1,10 @@ -use crate::vfs::{VfsError, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat}; +use crate::vfs::{ + validate_path, VfsError, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat, +}; use std::collections::BTreeMap; use std::error::Error; use std::fmt; +use std::path::Path; use std::sync::Arc; pub type FsPermissionCheck = Arc PermissionDecision + Send + Sync>; @@ -85,6 +88,7 @@ pub enum FsOperation { Chown, Utimes, Truncate, + MountSensitive, } impl FsOperation { @@ -106,6 +110,7 @@ impl FsOperation { Self::Chown => "chown", Self::Utimes => "utimes", Self::Truncate => "truncate", + Self::MountSensitive => "mount", } } } @@ -286,6 +291,7 @@ impl PermissionedFileSystem { } fn check(&self, op: FsOperation, path: &str) -> VfsResult<()> { + validate_path(path)?; let Some(check) = self.permissions.filesystem.as_ref() else { return Err(VfsError::access_denied(op.as_str(), path, None)); }; @@ -309,119 +315,233 @@ impl PermissionedFileSystem { } impl PermissionedFileSystem { + fn resolved_existing_path(&self, path: &str) -> VfsResult { + self.inner.realpath(path) + } + + fn resolved_destination_path(&self, path: &str) -> VfsResult { + let normalized = crate::vfs::normalize_path(path); + if normalized == "/" { + return Ok(normalized); + } + + let parent = Path::new(&normalized) + .parent() + .unwrap_or_else(|| Path::new("/")) + .to_string_lossy() + .into_owned(); + let basename = Path::new(&normalized) + .file_name() + .map(|value| value.to_string_lossy().into_owned()) + .unwrap_or_default(); + + let mut candidate = parent; + let mut unresolved_segments = Vec::new(); + + let resolved_parent = loop { + match self.inner.realpath(&candidate) { + Ok(resolved) => break resolved, + Err(error) if matches!(error.code(), "ENOENT" | "ENOTDIR") => { + if candidate == "/" { + break String::from("/"); + } + let candidate_path = Path::new(&candidate); + if let Some(segment) = candidate_path.file_name() { + unresolved_segments.push(segment.to_string_lossy().into_owned()); + } + candidate = candidate_path + .parent() + .unwrap_or_else(|| Path::new("/")) + .to_string_lossy() + .into_owned(); + } + Err(error) => return Err(error), + } + }; + + let mut resolved = resolved_parent; + for segment in unresolved_segments.iter().rev() { + if resolved == "/" { + resolved = format!("/{segment}"); + } else { + resolved = format!("{resolved}/{segment}"); + } + } + + if resolved == "/" { + Ok(format!("/{basename}")) + } else { + Ok(format!("{resolved}/{basename}")) + } + } + + fn permission_subject(&self, op: FsOperation, path: &str) -> VfsResult { + validate_path(path)?; + match op { + FsOperation::Read + | FsOperation::ReadDir + | FsOperation::Stat + | FsOperation::ReadLink + | FsOperation::Chmod + | FsOperation::Chown + | FsOperation::Utimes + | FsOperation::Truncate => self.resolved_existing_path(path), + FsOperation::Exists | FsOperation::Write => self + .resolved_existing_path(path) + .or_else(|_| self.resolved_destination_path(path)), + FsOperation::Mkdir + | FsOperation::CreateDir + | FsOperation::Rename + | FsOperation::Symlink + | FsOperation::Link + | FsOperation::MountSensitive => self.resolved_destination_path(path), + FsOperation::Remove => Ok(crate::vfs::normalize_path(path)), + } + } + + fn check_subject(&self, op: FsOperation, path: &str) -> VfsResult<()> { + let subject = self.permission_subject(op, path)?; + self.check(op, &subject) + } + + pub fn check_path(&self, op: FsOperation, path: &str) -> VfsResult<()> { + self.check_subject(op, path) + } + + pub fn check_virtual_path(&self, op: FsOperation, path: &str) -> VfsResult<()> { + self.check(op, path) + } + pub fn exists(&self, path: &str) -> VfsResult { - self.check(FsOperation::Exists, path)?; + if let Err(error) = self.check_subject(FsOperation::Exists, path) { + if matches!(error.code(), "EACCES" | "ENOENT" | "ENOTDIR" | "ELOOP") { + return Ok(false); + } + return Err(error); + } Ok(self.inner.exists(path)) } } impl VirtualFileSystem for PermissionedFileSystem { fn read_file(&mut self, path: &str) -> VfsResult> { - self.check(FsOperation::Read, path)?; + self.check_subject(FsOperation::Read, path)?; self.inner.read_file(path) } fn read_dir(&mut self, path: &str) -> VfsResult> { - self.check(FsOperation::ReadDir, path)?; + self.check_subject(FsOperation::ReadDir, path)?; self.inner.read_dir(path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + self.check_subject(FsOperation::ReadDir, path)?; + self.inner.read_dir_limited(path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { - self.check(FsOperation::ReadDir, path)?; + self.check_subject(FsOperation::ReadDir, path)?; self.inner.read_dir_with_types(path) } fn write_file(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { - self.check(FsOperation::Write, path)?; + self.check_subject(FsOperation::Write, path)?; self.inner.write_file(path, content) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + self.check_subject(FsOperation::Write, path)?; + self.inner.create_file_exclusive(path, content) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + self.check_subject(FsOperation::Write, path)?; + self.inner.append_file(path, content) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { - self.check(FsOperation::CreateDir, path)?; + self.check_subject(FsOperation::CreateDir, path)?; self.inner.create_dir(path) } fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()> { - self.check(FsOperation::Mkdir, path)?; + self.check_subject(FsOperation::Mkdir, path)?; self.inner.mkdir(path, recursive) } fn exists(&self, path: &str) -> bool { - match PermissionedFileSystem::exists(self, path) { - Ok(exists) => exists, - Err(error) if error.code() == "EACCES" => self.inner.exists(path), - Err(_) => false, - } + PermissionedFileSystem::exists(self, path).unwrap_or(false) } fn stat(&mut self, path: &str) -> VfsResult { - self.check(FsOperation::Stat, path)?; + self.check_subject(FsOperation::Stat, path)?; self.inner.stat(path) } fn remove_file(&mut self, path: &str) -> VfsResult<()> { - self.check(FsOperation::Remove, path)?; + self.check_subject(FsOperation::Remove, path)?; self.inner.remove_file(path) } fn remove_dir(&mut self, path: &str) -> VfsResult<()> { - self.check(FsOperation::Remove, path)?; + self.check_subject(FsOperation::Remove, path)?; self.inner.remove_dir(path) } fn rename(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { - self.check(FsOperation::Rename, old_path)?; - self.check(FsOperation::Rename, new_path)?; + self.check_subject(FsOperation::Rename, old_path)?; + self.check_subject(FsOperation::Rename, new_path)?; self.inner.rename(old_path, new_path) } fn realpath(&self, path: &str) -> VfsResult { - self.check(FsOperation::Read, path)?; + self.check_subject(FsOperation::Read, path)?; self.inner.realpath(path) } fn symlink(&mut self, target: &str, link_path: &str) -> VfsResult<()> { - self.check(FsOperation::Symlink, link_path)?; + self.check_subject(FsOperation::Symlink, link_path)?; self.inner.symlink(target, link_path) } fn read_link(&self, path: &str) -> VfsResult { - self.check(FsOperation::ReadLink, path)?; + self.check(FsOperation::ReadLink, &crate::vfs::normalize_path(path))?; self.inner.read_link(path) } fn lstat(&self, path: &str) -> VfsResult { - self.check(FsOperation::Stat, path)?; + self.check(FsOperation::Stat, &crate::vfs::normalize_path(path))?; self.inner.lstat(path) } fn link(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { - self.check(FsOperation::Link, new_path)?; + self.check_subject(FsOperation::Link, old_path)?; + self.check_subject(FsOperation::Link, new_path)?; self.inner.link(old_path, new_path) } fn chmod(&mut self, path: &str, mode: u32) -> VfsResult<()> { - self.check(FsOperation::Chmod, path)?; + self.check_subject(FsOperation::Chmod, path)?; self.inner.chmod(path, mode) } fn chown(&mut self, path: &str, uid: u32, gid: u32) -> VfsResult<()> { - self.check(FsOperation::Chown, path)?; + self.check_subject(FsOperation::Chown, path)?; self.inner.chown(path, uid, gid) } fn utimes(&mut self, path: &str, atime_ms: u64, mtime_ms: u64) -> VfsResult<()> { - self.check(FsOperation::Utimes, path)?; + self.check_subject(FsOperation::Utimes, path)?; self.inner.utimes(path, atime_ms, mtime_ms) } fn truncate(&mut self, path: &str, length: u64) -> VfsResult<()> { - self.check(FsOperation::Truncate, path)?; + self.check_subject(FsOperation::Truncate, path)?; self.inner.truncate(path, length) } fn pread(&mut self, path: &str, offset: u64, length: usize) -> VfsResult> { - self.check(FsOperation::Read, path)?; + self.check_subject(FsOperation::Read, path)?; self.inner.pread(path, offset, length) } } diff --git a/crates/kernel/src/pipe_manager.rs b/crates/kernel/src/pipe_manager.rs index 7c1f15c31..33ba08df1 100644 --- a/crates/kernel/src/pipe_manager.rs +++ b/crates/kernel/src/pipe_manager.rs @@ -2,12 +2,15 @@ use crate::fd_table::{ FdResult, FileDescription, ProcessFdTable, SharedFileDescription, FILETYPE_PIPE, O_RDONLY, O_WRONLY, }; +use crate::poll::{PollEvents, PollNotifier, POLLERR, POLLHUP, POLLIN, POLLOUT}; use std::collections::{BTreeMap, VecDeque}; use std::error::Error; use std::fmt; use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::{Duration, Instant}; pub const MAX_PIPE_BUFFER_BYTES: usize = 65_536; +pub const PIPE_BUF_BYTES: usize = 4_096; pub type PipeResult = Result; @@ -121,6 +124,7 @@ struct PipeManagerInner { #[derive(Debug, Clone)] pub struct PipeManager { inner: Arc, + notifier: Option, } impl Default for PipeManager { @@ -130,6 +134,7 @@ impl Default for PipeManager { state: Mutex::new(PipeManagerState::default()), waiters: Condvar::new(), }), + notifier: None, } } } @@ -139,6 +144,13 @@ impl PipeManager { Self::default() } + pub(crate) fn with_notifier(notifier: PollNotifier) -> Self { + Self { + notifier: Some(notifier), + ..Self::default() + } + } + pub fn create_pipe(&self) -> PipePair { let mut state = lock_or_recover(&self.inner.state); let pipe_id = state.next_pipe_id; @@ -188,7 +200,56 @@ impl PipeManager { } } + pub fn poll(&self, description_id: u64, requested: PollEvents) -> PipeResult { + let state = lock_or_recover(&self.inner.state); + let pipe_ref = state + .desc_to_pipe + .get(&description_id) + .copied() + .ok_or_else(|| PipeError::bad_file_descriptor("not a pipe end"))?; + let pipe = state + .pipes + .get(&pipe_ref.pipe_id) + .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; + + let mut events = PollEvents::empty(); + match pipe_ref.end { + PipeSide::Read => { + if requested.intersects(POLLIN) && !pipe.buffer.is_empty() { + events |= POLLIN; + } + if pipe.closed_write { + events |= POLLHUP; + } + } + PipeSide::Write => { + if pipe.closed_read { + events |= POLLERR; + } else if requested.intersects(POLLOUT) + && (available_capacity(pipe) > 0 || !pipe.waiting_reads.is_empty()) + { + events |= POLLOUT; + } + } + } + + Ok(events) + } + pub fn write(&self, description_id: u64, data: impl AsRef<[u8]>) -> PipeResult { + self.write_with_mode(description_id, data, true) + } + + pub fn write_blocking(&self, description_id: u64, data: impl AsRef<[u8]>) -> PipeResult { + self.write_with_mode(description_id, data, false) + } + + pub fn write_with_mode( + &self, + description_id: u64, + data: impl AsRef<[u8]>, + nonblocking: bool, + ) -> PipeResult { let payload = data.as_ref(); let mut state = lock_or_recover(&self.inner.state); let pipe_ref = state @@ -200,50 +261,78 @@ impl PipeManager { return Err(PipeError::bad_file_descriptor("not a pipe write end")); } - let waiter_id = { - let pipe = state - .pipes - .get_mut(&pipe_ref.pipe_id) - .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; - if pipe.closed_write { - return Err(PipeError::broken_pipe("write end closed")); - } - if pipe.closed_read { - return Err(PipeError::broken_pipe("read end closed")); + loop { + let waiter_id = { + let pipe = state + .pipes + .get_mut(&pipe_ref.pipe_id) + .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; + if pipe.closed_write { + return Err(PipeError::broken_pipe("write end closed")); + } + if pipe.closed_read { + return Err(PipeError::broken_pipe("read end closed")); + } + pipe.waiting_reads.pop_front() + }; + + if let Some(waiter_id) = waiter_id { + if let Some(waiter) = state.waiters.get_mut(&waiter_id) { + waiter.result = Some(Some(payload.to_vec())); + self.notify_waiters_and_pollers(); + return Ok(payload.len()); + } + continue; } - pipe.waiting_reads.pop_front() - }; - if let Some(waiter_id) = waiter_id { - if let Some(waiter) = state.waiters.get_mut(&waiter_id) { - waiter.result = Some(Some(payload.to_vec())); - self.inner.waiters.notify_all(); - return Ok(payload.len()); + let current_buffer_size = { + let pipe = state + .pipes + .get(&pipe_ref.pipe_id) + .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; + buffer_size(&pipe.buffer) + }; + let available = MAX_PIPE_BUFFER_BYTES.saturating_sub(current_buffer_size); + + if payload.len() <= PIPE_BUF_BYTES { + if available >= payload.len() { + let pipe = state + .pipes + .get_mut(&pipe_ref.pipe_id) + .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; + pipe.buffer.push_back(payload.to_vec()); + self.notify_waiters_and_pollers(); + return Ok(payload.len()); + } + } else if available > 0 { + let chunk_len = available.min(payload.len()); + let pipe = state + .pipes + .get_mut(&pipe_ref.pipe_id) + .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; + pipe.buffer.push_back(payload[..chunk_len].to_vec()); + self.notify_waiters_and_pollers(); + return Ok(chunk_len); } - } - let current_buffer_size = { - let pipe = state - .pipes - .get(&pipe_ref.pipe_id) - .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; - buffer_size(&pipe.buffer) - }; + if nonblocking { + return Err(PipeError::would_block("pipe buffer full")); + } - if current_buffer_size.saturating_add(payload.len()) > MAX_PIPE_BUFFER_BYTES { - return Err(PipeError::would_block("pipe buffer full")); + state = wait_or_recover(&self.inner.waiters, state); } - - let pipe = state - .pipes - .get_mut(&pipe_ref.pipe_id) - .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; - pipe.buffer.push_back(payload.to_vec()); - self.inner.waiters.notify_all(); - Ok(payload.len()) } pub fn read(&self, description_id: u64, length: usize) -> PipeResult>> { + self.read_with_timeout(description_id, length, None) + } + + pub fn read_with_timeout( + &self, + description_id: u64, + length: usize, + timeout: Option, + ) -> PipeResult>> { let mut state = lock_or_recover(&self.inner.state); let pipe_ref = state .desc_to_pipe @@ -255,6 +344,7 @@ impl PipeManager { } let mut waiter_id = None; + let deadline = timeout.map(|duration| Instant::now() + duration); loop { if let Some(id) = waiter_id { @@ -273,7 +363,9 @@ impl PipeManager { .ok_or_else(|| PipeError::bad_file_descriptor("pipe not found"))?; if !pipe.buffer.is_empty() { - return Ok(Some(drain_buffer(&mut pipe.buffer, length))); + let result = drain_buffer(&mut pipe.buffer, length); + self.notify_waiters_and_pollers(); + return Ok(Some(result)); } if pipe.closed_write { @@ -295,15 +387,48 @@ impl PipeManager { return Err(PipeError::bad_file_descriptor("pipe not found")); }; pipe.waiting_reads.push_back(next); + self.notify_waiters_and_pollers(); waiter_id = Some(next); next }; - state = wait_or_recover(&self.inner.waiters, state); + let Some(deadline) = deadline else { + state = wait_or_recover(&self.inner.waiters, state); + if !state.waiters.contains_key(&id) { + waiter_id = None; + } + continue; + }; + + let now = Instant::now(); + if now >= deadline { + if let Some(id) = waiter_id.take() { + state.waiters.remove(&id); + if let Some(pipe) = state.pipes.get_mut(&pipe_ref.pipe_id) { + pipe.waiting_reads.retain(|queued| *queued != id); + } + self.notify_waiters_and_pollers(); + } + return Err(PipeError::would_block("pipe read timed out")); + } + let remaining = deadline.saturating_duration_since(now); + let (next_state, wait_result) = + wait_timeout_or_recover(&self.inner.waiters, state, remaining); + state = next_state; if !state.waiters.contains_key(&id) { waiter_id = None; } + if wait_result.timed_out() { + if let Some(id) = waiter_id.take() { + state.waiters.remove(&id); + if let Some(pipe) = state.pipes.get_mut(&pipe_ref.pipe_id) { + pipe.waiting_reads.retain(|queued| *queued != id); + } + self.notify_waiters_and_pollers(); + } + return Err(PipeError::would_block("pipe read timed out")); + } } } @@ -318,7 +443,7 @@ impl PipeManager { match pipe_ref.end { PipeSide::Read => { pipe.closed_read = true; - (Vec::new(), pipe.closed_read && pipe.closed_write, false) + (Vec::new(), pipe.closed_read && pipe.closed_write, true) } PipeSide::Write => { pipe.closed_write = true; @@ -340,7 +465,7 @@ impl PipeManager { state.pipes.remove(&pipe_ref.pipe_id); } if should_notify { - self.inner.waiters.notify_all(); + self.notify_waiters_and_pollers(); } } @@ -383,12 +508,23 @@ impl PipeManager { } } } + + fn notify_waiters_and_pollers(&self) { + self.inner.waiters.notify_all(); + if let Some(notifier) = &self.notifier { + notifier.notify(); + } + } } fn buffer_size(buffer: &VecDeque>) -> usize { buffer.iter().map(Vec::len).sum() } +fn available_capacity(pipe: &PipeState) -> usize { + MAX_PIPE_BUFFER_BYTES.saturating_sub(buffer_size(&pipe.buffer)) +} + fn drain_buffer(buffer: &mut VecDeque>, length: usize) -> Vec { let mut chunks = Vec::new(); let mut remaining = length; @@ -433,3 +569,14 @@ fn wait_or_recover<'a, T>(condvar: &Condvar, guard: MutexGuard<'a, T>) -> MutexG Err(poisoned) => poisoned.into_inner(), } } + +fn wait_timeout_or_recover<'a, T>( + condvar: &Condvar, + guard: MutexGuard<'a, T>, + timeout: Duration, +) -> (MutexGuard<'a, T>, std::sync::WaitTimeoutResult) { + match condvar.wait_timeout(guard, timeout) { + Ok(result) => result, + Err(poisoned) => poisoned.into_inner(), + } +} diff --git a/crates/kernel/src/poll.rs b/crates/kernel/src/poll.rs new file mode 100644 index 000000000..ce2b6d082 --- /dev/null +++ b/crates/kernel/src/poll.rs @@ -0,0 +1,156 @@ +use std::ops::{BitOr, BitOrAssign}; +use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::{Duration, Instant}; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct PollEvents(u16); + +impl PollEvents { + pub const fn empty() -> Self { + Self(0) + } + + pub const fn bits(self) -> u16 { + self.0 + } + + pub const fn is_empty(self) -> bool { + self.0 == 0 + } + + pub const fn contains(self, other: Self) -> bool { + self.0 & other.0 == other.0 + } + + pub const fn intersects(self, other: Self) -> bool { + self.0 & other.0 != 0 + } +} + +impl BitOr for PollEvents { + type Output = Self; + + fn bitor(self, rhs: Self) -> Self::Output { + Self(self.0 | rhs.0) + } +} + +impl BitOrAssign for PollEvents { + fn bitor_assign(&mut self, rhs: Self) { + self.0 |= rhs.0; + } +} + +pub const POLLIN: PollEvents = PollEvents(0x0001); +pub const POLLOUT: PollEvents = PollEvents(0x0004); +pub const POLLERR: PollEvents = PollEvents(0x0008); +pub const POLLHUP: PollEvents = PollEvents(0x0010); +pub const POLLNVAL: PollEvents = PollEvents(0x0020); + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct PollFd { + pub fd: u32, + pub events: PollEvents, + pub revents: PollEvents, +} + +impl PollFd { + pub const fn new(fd: u32, events: PollEvents) -> Self { + Self { + fd, + events, + revents: PollEvents::empty(), + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PollResult { + pub ready_count: usize, + pub fds: Vec, +} + +#[derive(Debug, Clone, Default)] +pub(crate) struct PollNotifier { + inner: Arc, +} + +#[derive(Debug, Default)] +struct PollNotifierInner { + generation: Mutex, + waiters: Condvar, +} + +impl PollNotifier { + pub(crate) fn notify(&self) { + let mut generation = lock_or_recover(&self.inner.generation); + *generation = generation.saturating_add(1); + self.inner.waiters.notify_all(); + } + + pub(crate) fn snapshot(&self) -> u64 { + *lock_or_recover(&self.inner.generation) + } + + pub(crate) fn wait_for_change(&self, observed: u64, timeout: Option) -> bool { + let mut generation = lock_or_recover(&self.inner.generation); + if *generation != observed { + return true; + } + + let Some(timeout) = timeout else { + while *generation == observed { + generation = wait_or_recover(&self.inner.waiters, generation); + } + return true; + }; + + if timeout.is_zero() { + return *generation != observed; + } + + let deadline = Instant::now() + timeout; + loop { + let now = Instant::now(); + if now >= deadline { + return *generation != observed; + } + + let remaining = deadline.saturating_duration_since(now); + let (next_generation, wait_result) = + wait_timeout_or_recover(&self.inner.waiters, generation, remaining); + generation = next_generation; + if *generation != observed { + return true; + } + if wait_result.timed_out() { + return false; + } + } + } +} + +fn lock_or_recover<'a, T>(mutex: &'a Mutex) -> MutexGuard<'a, T> { + match mutex.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + } +} + +fn wait_or_recover<'a, T>(condvar: &Condvar, guard: MutexGuard<'a, T>) -> MutexGuard<'a, T> { + match condvar.wait(guard) { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + } +} + +fn wait_timeout_or_recover<'a, T>( + condvar: &Condvar, + guard: MutexGuard<'a, T>, + timeout: Duration, +) -> (MutexGuard<'a, T>, std::sync::WaitTimeoutResult) { + match condvar.wait_timeout(guard, timeout) { + Ok(result) => result, + Err(poisoned) => poisoned.into_inner(), + } +} diff --git a/crates/kernel/src/process_table.rs b/crates/kernel/src/process_table.rs index 76c5e7093..21f10ab85 100644 --- a/crates/kernel/src/process_table.rs +++ b/crates/kernel/src/process_table.rs @@ -1,14 +1,24 @@ -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet, VecDeque}; use std::error::Error; use std::fmt; +use std::ops::{BitOr, BitOrAssign}; use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::{Arc, Condvar, Mutex, MutexGuard, WaitTimeoutResult, Weak}; use std::thread; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; const ZOMBIE_TTL: Duration = Duration::from_secs(60); +const INIT_PID: u32 = 1; +pub const DEFAULT_PROCESS_UMASK: u32 = 0o022; +pub const SIGHUP: i32 = 1; +pub const SIGCHLD: i32 = 17; +pub const SIGCONT: i32 = 18; +pub const SIGSTOP: i32 = 19; +pub const SIGTSTP: i32 = 20; pub const SIGTERM: i32 = 15; pub const SIGKILL: i32 = 9; +pub const SIGPIPE: i32 = 13; +pub const SIGWINCH: i32 = 28; pub type ProcessResult = Result; pub type ProcessExitCallback = Arc; @@ -51,6 +61,13 @@ impl ProcessTableError { } } + fn no_matching_child(waiter_pid: u32, pid: i32) -> Self { + Self { + code: "ECHILD", + message: format!("process {waiter_pid} has no matching child for waitpid({pid})"), + } + } + fn permission_denied(message: impl Into) -> Self { Self { code: "EPERM", @@ -74,6 +91,61 @@ pub enum ProcessStatus { Exited, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct WaitPidFlags { + bits: u32, +} + +impl WaitPidFlags { + pub const WNOHANG: Self = Self { bits: 1 << 0 }; + pub const WUNTRACED: Self = Self { bits: 1 << 1 }; + pub const WCONTINUED: Self = Self { bits: 1 << 2 }; + + pub const fn empty() -> Self { + Self { bits: 0 } + } + + pub const fn contains(self, other: Self) -> bool { + (self.bits & other.bits) == other.bits + } +} + +impl Default for WaitPidFlags { + fn default() -> Self { + Self::empty() + } +} + +impl BitOr for WaitPidFlags { + type Output = Self; + + fn bitor(self, rhs: Self) -> Self::Output { + Self { + bits: self.bits | rhs.bits, + } + } +} + +impl BitOrAssign for WaitPidFlags { + fn bitor_assign(&mut self, rhs: Self) { + self.bits |= rhs.bits; + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ProcessWaitEvent { + Exited, + Stopped, + Continued, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ProcessWaitResult { + pub pid: u32, + pub status: i32, + pub event: ProcessWaitEvent, +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct ProcessFileDescriptors { pub stdin: u32, @@ -97,6 +169,7 @@ pub struct ProcessContext { pub ppid: u32, pub env: BTreeMap, pub cwd: String, + pub umask: u32, pub fds: ProcessFileDescriptors, } @@ -107,6 +180,7 @@ impl Default for ProcessContext { ppid: 0, env: BTreeMap::new(), cwd: String::from("/"), + umask: DEFAULT_PROCESS_UMASK, fds: ProcessFileDescriptors::default(), } } @@ -126,6 +200,7 @@ pub struct ProcessEntry { pub exit_time_ms: Option, pub env: BTreeMap, pub cwd: String, + pub umask: u32, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -154,6 +229,20 @@ struct ProcessTableInner { struct ProcessRecord { entry: ProcessEntry, driver_process: Arc, + pending_wait_events: VecDeque, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct PendingWaitEvent { + status: i32, + event: ProcessWaitEvent, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum WaitSelector { + AnyChild, + ChildPid(u32), + ProcessGroup(u32), } struct ZombieReaper { @@ -257,6 +346,7 @@ impl ProcessTable { exit_time_ms: None, env: ctx.env, cwd: ctx.cwd, + umask: ctx.umask & 0o777, }; let weak = Arc::downgrade(&self.inner); @@ -271,6 +361,7 @@ impl ProcessTable { ProcessRecord { entry: entry.clone(), driver_process, + pending_wait_events: VecDeque::new(), }, ); @@ -306,6 +397,30 @@ impl ProcessTable { mark_exited_inner(&self.inner, pid, exit_code); } + pub fn mark_stopped(&self, pid: u32, signal: i32) { + mark_wait_event_inner( + &self.inner, + pid, + ProcessStatus::Stopped, + PendingWaitEvent { + status: signal, + event: ProcessWaitEvent::Stopped, + }, + ); + } + + pub fn mark_continued(&self, pid: u32) { + mark_wait_event_inner( + &self.inner, + pid, + ProcessStatus::Running, + PendingWaitEvent { + status: SIGCONT, + event: ProcessWaitEvent::Continued, + }, + ); + } + pub fn waitpid(&self, pid: u32) -> ProcessResult<(u32, i32)> { let mut state = self.inner.lock_state(); loop { @@ -326,6 +441,38 @@ impl ProcessTable { } } + pub fn waitpid_for( + &self, + waiter_pid: u32, + pid: i32, + flags: WaitPidFlags, + ) -> ProcessResult> { + let mut state = self.inner.lock_state(); + loop { + let selector = resolve_wait_selector(&state, waiter_pid, pid)?; + let matching_children = matching_child_pids(&state, waiter_pid, selector); + if matching_children.is_empty() { + return Err(ProcessTableError::no_matching_child(waiter_pid, pid)); + } + + if let Some(result) = take_waitable_event(&mut state, &matching_children, flags) { + let should_reap = result.event == ProcessWaitEvent::Exited; + drop(state); + if should_reap { + self.inner.reaper.cancel(result.pid); + self.inner.waiters.notify_all(); + } + return Ok(Some(result)); + } + + if flags.contains(WaitPidFlags::WNOHANG) { + return Ok(None); + } + + state = self.inner.wait_for_state(state); + } + } + pub fn kill(&self, pid: i32, signal: i32) -> ProcessResult<()> { if !(0..=64).contains(&signal) { return Err(ProcessTableError::invalid_signal(signal)); @@ -338,10 +485,14 @@ impl ProcessTable { let grouped: Vec<_> = state .entries .values() - .filter(|record| { - record.entry.pgid == pgid && record.entry.status == ProcessStatus::Running + .filter(|record| record.entry.pgid == pgid) + .map(|record| { + ( + record.entry.pid, + record.entry.status, + Arc::clone(&record.driver_process), + ) }) - .map(|record| Arc::clone(&record.driver_process)) .collect(); if grouped.is_empty() { return Err(ProcessTableError::no_such_process_group(pgid)); @@ -355,7 +506,11 @@ impl ProcessTable { if record.entry.status == ProcessStatus::Exited || signal == 0 { return Ok(()); } - vec![Arc::clone(&record.driver_process)] + vec![( + record.entry.pid, + record.entry.status, + Arc::clone(&record.driver_process), + )] } }; @@ -363,9 +518,23 @@ impl ProcessTable { return Ok(()); } - for driver in targets { + let mut stopped = Vec::new(); + let mut continued = Vec::new(); + for (target_pid, status, driver) in &targets { + match signal { + SIGSTOP | SIGTSTP if *status == ProcessStatus::Running => stopped.push(*target_pid), + SIGCONT if *status == ProcessStatus::Stopped => continued.push(*target_pid), + _ => {} + } driver.kill(signal); } + + for pid in stopped { + self.mark_stopped(pid, signal); + } + for pid in continued { + self.mark_continued(pid); + } Ok(()) } @@ -441,6 +610,23 @@ impl ProcessTable { .ok_or_else(|| ProcessTableError::no_such_process(pid)) } + pub fn get_umask(&self, pid: u32) -> ProcessResult { + self.get(pid) + .map(|entry| entry.umask) + .ok_or_else(|| ProcessTableError::no_such_process(pid)) + } + + pub fn set_umask(&self, pid: u32, umask: u32) -> ProcessResult { + let mut state = self.inner.lock_state(); + let record = state + .entries + .get_mut(&pid) + .ok_or_else(|| ProcessTableError::no_such_process(pid))?; + let previous = record.entry.umask; + record.entry.umask = umask & 0o777; + Ok(previous) + } + pub fn has_process_group(&self, pgid: u32) -> bool { self.inner .lock_state() @@ -522,26 +708,46 @@ fn to_process_info(entry: &ProcessEntry) -> ProcessInfo { } fn mark_exited_inner(inner: &Arc, pid: u32, exit_code: i32) { - let (callback, zombie_ttl, should_schedule) = { + let (callback, zombie_ttl, should_schedule, parent_driver, orphaned_group_targets) = { let mut state = inner.lock_state(); - let Some(record) = state.entries.get_mut(&pid) else { - return; - }; + let (ppid, pgid) = { + let Some(record) = state.entries.get_mut(&pid) else { + return; + }; - if record.entry.status == ProcessStatus::Exited { - return; - } + if record.entry.status == ProcessStatus::Exited { + return; + } + + record.entry.status = ProcessStatus::Exited; + record.entry.exit_code = Some(exit_code); + record.entry.exit_time_ms = Some(now_ms()); + let ppid = record.entry.ppid; + let pgid = record.entry.pgid; + (ppid, pgid) + }; + let mut affected_pgids = BTreeSet::from([pgid]); + reparent_children_to_init(&mut state, pid, &mut affected_pgids); - record.entry.status = ProcessStatus::Exited; - record.entry.exit_code = Some(exit_code); - record.entry.exit_time_ms = Some(now_ms()); + let orphaned_group_targets = collect_orphaned_group_signal_targets(&state, &affected_pgids); let should_schedule = !state.terminating_all; + let parent_driver = if should_schedule { + state + .entries + .get(&ppid) + .filter(|parent| parent.entry.status == ProcessStatus::Running) + .map(|parent| Arc::clone(&parent.driver_process)) + } else { + None + }; ( state.on_process_exit.clone(), state.zombie_ttl, should_schedule, + parent_driver, + orphaned_group_targets, ) }; @@ -551,6 +757,17 @@ fn mark_exited_inner(inner: &Arc, pid: u32, exit_code: i32) { inner.reaper.cancel(pid); } + if let Some(parent_driver) = parent_driver { + parent_driver.kill(SIGCHLD); + } + + for driver in &orphaned_group_targets { + driver.kill(SIGHUP); + } + for driver in &orphaned_group_targets { + driver.kill(SIGCONT); + } + if let Some(on_process_exit) = callback { on_process_exit(pid); } @@ -558,6 +775,224 @@ fn mark_exited_inner(inner: &Arc, pid: u32, exit_code: i32) { inner.waiters.notify_all(); } +fn reparent_children_to_init( + state: &mut ProcessTableState, + exiting_pid: u32, + affected_pgids: &mut BTreeSet, +) { + let new_parent = reparent_target_pid(state, exiting_pid); + for record in state.entries.values_mut() { + if record.entry.ppid != exiting_pid { + continue; + } + record.entry.ppid = new_parent; + affected_pgids.insert(record.entry.pgid); + } +} + +fn reparent_target_pid(state: &ProcessTableState, exiting_pid: u32) -> u32 { + if exiting_pid != INIT_PID + && state + .entries + .get(&INIT_PID) + .map(|record| record.entry.status != ProcessStatus::Exited) + .unwrap_or(false) + { + INIT_PID + } else { + 0 + } +} + +fn collect_orphaned_group_signal_targets( + state: &ProcessTableState, + candidate_pgids: &BTreeSet, +) -> Vec> { + let mut targets = Vec::new(); + for &pgid in candidate_pgids { + if !process_group_is_orphaned(state, pgid) || !process_group_has_stopped_member(state, pgid) + { + continue; + } + + for record in state.entries.values() { + if record.entry.pgid == pgid && record.entry.status != ProcessStatus::Exited { + targets.push(Arc::clone(&record.driver_process)); + } + } + } + targets +} + +fn process_group_is_orphaned(state: &ProcessTableState, pgid: u32) -> bool { + let mut has_member = false; + for record in state.entries.values() { + if record.entry.pgid != pgid || record.entry.status == ProcessStatus::Exited { + continue; + } + has_member = true; + if has_parent_outside_group_in_same_session(state, &record.entry) { + return false; + } + } + + has_member +} + +fn has_parent_outside_group_in_same_session( + state: &ProcessTableState, + entry: &ProcessEntry, +) -> bool { + match entry.ppid { + 0 | INIT_PID => false, + ppid => state + .entries + .get(&ppid) + .map(|parent| { + parent.entry.status != ProcessStatus::Exited + && parent.entry.sid == entry.sid + && parent.entry.pgid != entry.pgid + }) + .unwrap_or(false), + } +} + +fn process_group_has_stopped_member(state: &ProcessTableState, pgid: u32) -> bool { + state + .entries + .values() + .any(|record| record.entry.pgid == pgid && record.entry.status == ProcessStatus::Stopped) +} + +fn mark_wait_event_inner( + inner: &Arc, + pid: u32, + next_status: ProcessStatus, + event: PendingWaitEvent, +) { + let parent_driver = { + let mut state = inner.lock_state(); + let ppid = { + let Some(record) = state.entries.get_mut(&pid) else { + return; + }; + + if record.entry.status == ProcessStatus::Exited || record.entry.status == next_status { + return; + } + + record.entry.status = next_status; + record.pending_wait_events.push_back(event); + record.entry.ppid + }; + + state + .entries + .get(&ppid) + .filter(|parent| parent.entry.status == ProcessStatus::Running) + .map(|parent| Arc::clone(&parent.driver_process)) + }; + + if let Some(parent_driver) = parent_driver { + parent_driver.kill(SIGCHLD); + } + + inner.waiters.notify_all(); +} + +fn resolve_wait_selector( + state: &ProcessTableState, + waiter_pid: u32, + pid: i32, +) -> ProcessResult { + let waiter = state + .entries + .get(&waiter_pid) + .ok_or_else(|| ProcessTableError::no_such_process(waiter_pid))?; + + Ok(match pid { + -1 => WaitSelector::AnyChild, + 0 => WaitSelector::ProcessGroup(waiter.entry.pgid), + p if p < -1 => WaitSelector::ProcessGroup(p.unsigned_abs()), + p => WaitSelector::ChildPid(p as u32), + }) +} + +fn matching_child_pids( + state: &ProcessTableState, + waiter_pid: u32, + selector: WaitSelector, +) -> Vec { + state + .entries + .values() + .filter(|record| record.entry.ppid == waiter_pid) + .filter(|record| match selector { + WaitSelector::AnyChild => true, + WaitSelector::ChildPid(pid) => record.entry.pid == pid, + WaitSelector::ProcessGroup(pgid) => record.entry.pgid == pgid, + }) + .map(|record| record.entry.pid) + .collect() +} + +fn take_waitable_event( + state: &mut ProcessTableState, + matching_children: &[u32], + flags: WaitPidFlags, +) -> Option { + for child_pid in matching_children { + let mut non_exit_result = None; + let mut should_reap = false; + { + let record = state.entries.get_mut(child_pid)?; + if let Some(index) = record + .pending_wait_events + .iter() + .position(|event| is_waitable_event(event.event, flags)) + { + let event = record + .pending_wait_events + .remove(index) + .expect("pending wait event should exist"); + non_exit_result = Some(ProcessWaitResult { + pid: *child_pid, + status: event.status, + event: event.event, + }); + } else if record.entry.status == ProcessStatus::Exited { + should_reap = true; + } + } + + if let Some(result) = non_exit_result { + return Some(result); + } + + if should_reap { + let record = state + .entries + .remove(child_pid) + .expect("exited child should still exist"); + return Some(ProcessWaitResult { + pid: *child_pid, + status: record.entry.exit_code.unwrap_or_default(), + event: ProcessWaitEvent::Exited, + }); + } + } + + None +} + +fn is_waitable_event(event: ProcessWaitEvent, flags: WaitPidFlags) -> bool { + match event { + ProcessWaitEvent::Exited => true, + ProcessWaitEvent::Stopped => flags.contains(WaitPidFlags::WUNTRACED), + ProcessWaitEvent::Continued => flags.contains(WaitPidFlags::WCONTINUED), + } +} + fn start_zombie_reaper(inner: Weak, reaper: Arc) { reaper.thread_spawns.fetch_add(1, Ordering::SeqCst); thread::spawn(move || loop { @@ -570,19 +1005,38 @@ fn start_zombie_reaper(inner: Weak, reaper: Arc }; let mut state = inner.lock_state(); - if state + let should_reap = state + .entries + .get(&pid) + .map(|record| { + record.entry.status == ProcessStatus::Exited + && !has_living_parent(&state, record.entry.ppid) + }) + .unwrap_or(false); + if should_reap { + state.entries.remove(&pid); + } else if state .entries .get(&pid) .map(|record| record.entry.status == ProcessStatus::Exited) .unwrap_or(false) { - state.entries.remove(&pid); + reaper.schedule(pid, state.zombie_ttl); } drop(state); inner.waiters.notify_all(); }); } +fn has_living_parent(state: &ProcessTableState, ppid: u32) -> bool { + ppid != 0 + && state + .entries + .get(&ppid) + .map(|record| record.entry.status != ProcessStatus::Exited) + .unwrap_or(false) +} + impl ProcessTableInner { fn lock_state(&self) -> MutexGuard<'_, ProcessTableState> { lock_or_recover(&self.state) diff --git a/crates/kernel/src/pty.rs b/crates/kernel/src/pty.rs index 90db19df7..450b6bb6c 100644 --- a/crates/kernel/src/pty.rs +++ b/crates/kernel/src/pty.rs @@ -2,16 +2,20 @@ use crate::fd_table::{ FdResult, FileDescription, ProcessFdTable, SharedFileDescription, FILETYPE_CHARACTER_DEVICE, O_RDWR, }; +use crate::poll::{PollEvents, PollNotifier, POLLHUP, POLLIN, POLLOUT}; use std::collections::{BTreeMap, VecDeque}; use std::error::Error; use std::fmt; use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::{Duration, Instant}; pub const MAX_PTY_BUFFER_BYTES: usize = 65_536; pub const MAX_CANON: usize = 4_096; pub const SIGINT: i32 = 2; pub const SIGQUIT: i32 = 3; pub const SIGTSTP: i32 = 20; +const DEFAULT_PTY_COLUMNS: u16 = 80; +const DEFAULT_PTY_ROWS: u16 = 24; pub type PtyResult = Result; pub type SignalHandler = Arc; @@ -104,6 +108,21 @@ pub struct PartialTermiosControlChars { pub verase: Option, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct PtyWindowSize { + pub cols: u16, + pub rows: u16, +} + +impl Default for PtyWindowSize { + fn default() -> Self { + Self { + cols: DEFAULT_PTY_COLUMNS, + rows: DEFAULT_PTY_ROWS, + } + } +} + impl Default for Termios { fn default() -> Self { Self { @@ -212,6 +231,7 @@ struct PtyState { termios: Termios, line_buffer: Vec, foreground_pgid: u32, + window_size: PtyWindowSize, } #[derive(Debug)] @@ -247,6 +267,7 @@ struct PtyManagerInner { pub struct PtyManager { inner: Arc, on_signal: Option, + notifier: Option, } impl Default for PtyManager { @@ -257,6 +278,7 @@ impl Default for PtyManager { waiters: Condvar::new(), }), on_signal: None, + notifier: None, } } } @@ -272,6 +294,22 @@ impl PtyManager { manager } + pub(crate) fn with_signal_handler_and_notifier( + on_signal: SignalHandler, + notifier: PollNotifier, + ) -> Self { + let mut manager = Self::with_notifier(notifier); + manager.on_signal = Some(on_signal); + manager + } + + pub(crate) fn with_notifier(notifier: PollNotifier) -> Self { + Self { + notifier: Some(notifier), + ..Self::default() + } + } + pub fn create_pty(&self) -> PtyPair { let mut state = lock_or_recover(&self.inner.state); let pty_id = state.next_pty_id; @@ -288,6 +326,7 @@ impl PtyManager { PtyState { path: path.clone(), termios: Termios::default(), + window_size: PtyWindowSize::default(), ..PtyState::default() }, ); @@ -352,6 +391,51 @@ impl PtyManager { } } + pub fn poll(&self, description_id: u64, requested: PollEvents) -> PtyResult { + let state = lock_or_recover(&self.inner.state); + let pty_ref = state + .desc_to_pty + .get(&description_id) + .copied() + .ok_or_else(|| PtyError::bad_file_descriptor("not a PTY end"))?; + let pty = state + .ptys + .get(&pty_ref.pty_id) + .ok_or_else(|| PtyError::bad_file_descriptor("PTY not found"))?; + + let mut events = PollEvents::empty(); + match pty_ref.end { + PtyEndKind::Master => { + if requested.intersects(POLLIN) && !pty.output_buffer.is_empty() { + events |= POLLIN; + } + if pty.closed_slave { + events |= POLLHUP; + } else if requested.intersects(POLLOUT) + && (available_capacity(&pty.input_buffer) > 0 + || !pty.waiting_input_reads.is_empty()) + { + events |= POLLOUT; + } + } + PtyEndKind::Slave => { + if requested.intersects(POLLIN) && !pty.input_buffer.is_empty() { + events |= POLLIN; + } + if pty.closed_master { + events |= POLLHUP; + } else if requested.intersects(POLLOUT) + && (available_capacity(&pty.output_buffer) > 0 + || !pty.waiting_output_reads.is_empty()) + { + events |= POLLOUT; + } + } + } + + Ok(events) + } + pub fn write(&self, description_id: u64, data: impl AsRef<[u8]>) -> PtyResult { let payload = data.as_ref(); let mut signals = Vec::new(); @@ -392,7 +476,7 @@ impl PtyManager { } } - self.inner.waiters.notify_all(); + self.notify_waiters_and_pollers(); if let Some(on_signal) = &self.on_signal { for (pgid, signal) in signals { if pgid > 0 { @@ -405,6 +489,15 @@ impl PtyManager { } pub fn read(&self, description_id: u64, length: usize) -> PtyResult>> { + self.read_with_timeout(description_id, length, None) + } + + pub fn read_with_timeout( + &self, + description_id: u64, + length: usize, + timeout: Option, + ) -> PtyResult>> { let mut state = lock_or_recover(&self.inner.state); let pty_ref = state .desc_to_pty @@ -412,6 +505,7 @@ impl PtyManager { .copied() .ok_or_else(|| PtyError::bad_file_descriptor("not a PTY end"))?; let mut waiter_id = None; + let deadline = timeout.map(|duration| Instant::now() + duration); loop { if let Some(id) = waiter_id { @@ -439,7 +533,9 @@ impl PtyManager { } if !pty.output_buffer.is_empty() { - return Ok(Some(drain_buffer(&mut pty.output_buffer, length))); + let result = drain_buffer(&mut pty.output_buffer, length); + self.notify_waiters_and_pollers(); + return Ok(Some(result)); } if pty.closed_slave { @@ -458,7 +554,9 @@ impl PtyManager { } if !pty.input_buffer.is_empty() { - return Ok(Some(drain_buffer(&mut pty.input_buffer, length))); + let result = drain_buffer(&mut pty.input_buffer, length); + self.notify_waiters_and_pollers(); + return Ok(Some(result)); } if pty.closed_master { @@ -485,15 +583,50 @@ impl PtyManager { PtyEndKind::Master => pty.waiting_output_reads.push_back(next), PtyEndKind::Slave => pty.waiting_input_reads.push_back(next), } + self.notify_waiters_and_pollers(); waiter_id = Some(next); next }; - state = wait_or_recover(&self.inner.waiters, state); + let Some(deadline) = deadline else { + state = wait_or_recover(&self.inner.waiters, state); + if !state.waiters.contains_key(&id) { + waiter_id = None; + } + continue; + }; + let now = Instant::now(); + if now >= deadline { + if let Some(id) = waiter_id.take() { + state.waiters.remove(&id); + if let Some(pty) = state.ptys.get_mut(&pty_ref.pty_id) { + pty.waiting_input_reads.retain(|queued| *queued != id); + pty.waiting_output_reads.retain(|queued| *queued != id); + } + self.notify_waiters_and_pollers(); + } + return Err(PtyError::would_block("PTY read timed out")); + } + + let remaining = deadline.saturating_duration_since(now); + let (next_state, wait_result) = + wait_timeout_or_recover(&self.inner.waiters, state, remaining); + state = next_state; if !state.waiters.contains_key(&id) { waiter_id = None; } + if wait_result.timed_out() { + if let Some(id) = waiter_id.take() { + state.waiters.remove(&id); + if let Some(pty) = state.ptys.get_mut(&pty_ref.pty_id) { + pty.waiting_input_reads.retain(|queued| *queued != id); + pty.waiting_output_reads.retain(|queued| *queued != id); + } + self.notify_waiters_and_pollers(); + } + return Err(PtyError::would_block("PTY read timed out")); + } } } @@ -531,7 +664,7 @@ impl PtyManager { if remove_pty { state.ptys.remove(&pty_ref.pty_id); } - self.inner.waiters.notify_all(); + self.notify_waiters_and_pollers(); } pub fn is_pty(&self, description_id: u64) -> bool { @@ -634,6 +767,25 @@ impl PtyManager { .ok_or_else(|| PtyError::bad_file_descriptor("PTY not found")) } + pub fn resize(&self, description_id: u64, cols: u16, rows: u16) -> PtyResult> { + let mut state = lock_or_recover(&self.inner.state); + let pty_ref = state + .desc_to_pty + .get(&description_id) + .copied() + .ok_or_else(|| PtyError::bad_file_descriptor("not a PTY end"))?; + let pty = state + .ptys + .get_mut(&pty_ref.pty_id) + .ok_or_else(|| PtyError::bad_file_descriptor("PTY not found"))?; + let next_size = PtyWindowSize { cols, rows }; + if pty.window_size == next_size { + return Ok(None); + } + pty.window_size = next_size; + Ok((pty.foreground_pgid > 0).then_some(pty.foreground_pgid)) + } + pub fn pty_count(&self) -> usize { lock_or_recover(&self.inner.state).ptys.len() } @@ -659,6 +811,13 @@ impl PtyManager { let pty_ref = state.desc_to_pty.get(&description_id)?; state.ptys.get(&pty_ref.pty_id).map(|pty| pty.path.clone()) } + + fn notify_waiters_and_pollers(&self) { + self.inner.waiters.notify_all(); + if let Some(notifier) = &self.notifier { + notifier.notify(); + } + } } fn process_output(termios: &Termios, data: &[u8]) -> Vec { @@ -838,6 +997,10 @@ fn buffer_size(buffer: &VecDeque>) -> usize { buffer.iter().map(Vec::len).sum() } +fn available_capacity(buffer: &VecDeque>) -> usize { + MAX_PTY_BUFFER_BYTES.saturating_sub(buffer_size(buffer)) +} + fn drain_buffer(buffer: &mut VecDeque>, length: usize) -> Vec { let mut chunks = Vec::new(); let mut remaining = length; @@ -882,3 +1045,14 @@ fn wait_or_recover<'a, T>(condvar: &Condvar, guard: MutexGuard<'a, T>) -> MutexG Err(poisoned) => poisoned.into_inner(), } } + +fn wait_timeout_or_recover<'a, T>( + condvar: &Condvar, + guard: MutexGuard<'a, T>, + timeout: Duration, +) -> (MutexGuard<'a, T>, std::sync::WaitTimeoutResult) { + match condvar.wait_timeout(guard, timeout) { + Ok(result) => result, + Err(poisoned) => poisoned.into_inner(), + } +} diff --git a/crates/kernel/src/resource_accounting.rs b/crates/kernel/src/resource_accounting.rs index 4da12d72b..a3a1e1b56 100644 --- a/crates/kernel/src/resource_accounting.rs +++ b/crates/kernel/src/resource_accounting.rs @@ -2,9 +2,20 @@ use crate::fd_table::FdTableManager; use crate::pipe_manager::PipeManager; use crate::process_table::{ProcessStatus, ProcessTable}; use crate::pty::PtyManager; +use crate::vfs::{VfsResult, VirtualFileSystem}; +use std::collections::{BTreeMap, BTreeSet}; use std::error::Error; use std::fmt; +pub const DEFAULT_MAX_FILESYSTEM_BYTES: u64 = 64 * 1024 * 1024; +pub const DEFAULT_MAX_INODE_COUNT: usize = 16_384; +pub const DEFAULT_BLOCKING_READ_TIMEOUT_MS: u64 = 5_000; +pub const DEFAULT_MAX_PREAD_BYTES: usize = 64 * 1024 * 1024; +pub const DEFAULT_MAX_FD_WRITE_BYTES: usize = 64 * 1024 * 1024; +pub const DEFAULT_MAX_PROCESS_ARGV_BYTES: usize = 1024 * 1024; +pub const DEFAULT_MAX_PROCESS_ENV_BYTES: usize = 1024 * 1024; +pub const DEFAULT_MAX_READDIR_ENTRIES: usize = 4_096; + #[derive(Debug, Clone, PartialEq, Eq, Default)] pub struct ResourceSnapshot { pub running_processes: usize, @@ -18,12 +29,55 @@ pub struct ResourceSnapshot { pub pty_buffered_output_bytes: usize, } -#[derive(Debug, Clone, PartialEq, Eq, Default)] +#[derive(Debug, Clone, PartialEq, Eq)] pub struct ResourceLimits { pub max_processes: Option, pub max_open_fds: Option, pub max_pipes: Option, pub max_ptys: Option, + pub max_sockets: Option, + pub max_connections: Option, + pub max_filesystem_bytes: Option, + pub max_inode_count: Option, + pub max_blocking_read_ms: Option, + pub max_pread_bytes: Option, + pub max_fd_write_bytes: Option, + pub max_process_argv_bytes: Option, + pub max_process_env_bytes: Option, + pub max_readdir_entries: Option, + pub max_wasm_fuel: Option, + pub max_wasm_memory_bytes: Option, + pub max_wasm_stack_bytes: Option, +} + +impl Default for ResourceLimits { + fn default() -> Self { + Self { + max_processes: None, + max_open_fds: None, + max_pipes: None, + max_ptys: None, + max_sockets: None, + max_connections: None, + max_filesystem_bytes: Some(DEFAULT_MAX_FILESYSTEM_BYTES), + max_inode_count: Some(DEFAULT_MAX_INODE_COUNT), + max_blocking_read_ms: Some(DEFAULT_BLOCKING_READ_TIMEOUT_MS), + max_pread_bytes: Some(DEFAULT_MAX_PREAD_BYTES), + max_fd_write_bytes: Some(DEFAULT_MAX_FD_WRITE_BYTES), + max_process_argv_bytes: Some(DEFAULT_MAX_PROCESS_ARGV_BYTES), + max_process_env_bytes: Some(DEFAULT_MAX_PROCESS_ENV_BYTES), + max_readdir_entries: Some(DEFAULT_MAX_READDIR_ENTRIES), + max_wasm_fuel: None, + max_wasm_memory_bytes: None, + max_wasm_stack_bytes: None, + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct FileSystemUsage { + pub total_bytes: u64, + pub inode_count: usize, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -43,6 +97,27 @@ impl ResourceError { message: message.into(), } } + + fn filesystem_full(message: impl Into) -> Self { + Self { + code: "ENOSPC", + message: message.into(), + } + } + + fn invalid_input(message: impl Into) -> Self { + Self { + code: "EINVAL", + message: message.into(), + } + } + + fn out_of_memory(message: impl Into) -> Self { + Self { + code: "ENOMEM", + message: message.into(), + } + } } impl fmt::Display for ResourceError { @@ -103,7 +178,7 @@ impl ResourceAccountant { additional_fds: usize, ) -> Result<(), ResourceError> { if let Some(limit) = self.limits.max_processes { - if snapshot.running_processes >= limit { + if snapshot.running_processes + snapshot.exited_processes >= limit { return Err(ResourceError::exhausted("maximum process limit reached")); } } @@ -111,6 +186,40 @@ impl ResourceAccountant { self.check_open_fds(snapshot, additional_fds) } + pub fn check_process_argv_bytes( + &self, + command: &str, + args: &[String], + ) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_process_argv_bytes { + let total = argv_payload_bytes(command, args); + if total > limit { + return Err(ResourceError::invalid_input(format!( + "process argv payload {total} bytes exceeds configured limit {limit}" + ))); + } + } + + Ok(()) + } + + pub fn check_process_env_bytes( + &self, + inherited_env: &BTreeMap, + overrides: &BTreeMap, + ) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_process_env_bytes { + let total = merged_env_payload_bytes(inherited_env, overrides); + if total > limit { + return Err(ResourceError::invalid_input(format!( + "process environment payload {total} bytes exceeds configured limit {limit}" + ))); + } + } + + Ok(()) + } + pub fn check_pipe_allocation(&self, snapshot: &ResourceSnapshot) -> Result<(), ResourceError> { if let Some(limit) = self.limits.max_pipes { if snapshot.pipes >= limit { @@ -131,6 +240,46 @@ impl ResourceAccountant { self.check_open_fds(snapshot, 2) } + pub fn check_pread_length(&self, length: usize) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_pread_bytes { + if length > limit { + return Err(ResourceError::invalid_input(format!( + "pread length {length} exceeds configured limit {limit}" + ))); + } + } + + Ok(()) + } + + pub fn check_fd_write_size(&self, size: usize) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_fd_write_bytes { + if size > limit { + return Err(ResourceError::invalid_input(format!( + "write size {size} exceeds configured limit {limit}" + ))); + } + } + + Ok(()) + } + + pub fn max_readdir_entries(&self) -> Option { + self.limits.max_readdir_entries + } + + pub fn check_readdir_entries(&self, entries: usize) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_readdir_entries { + if entries > limit { + return Err(ResourceError::out_of_memory(format!( + "directory listing with {entries} entries exceeds configured limit {limit}" + ))); + } + } + + Ok(()) + } + fn check_open_fds( &self, snapshot: &ResourceSnapshot, @@ -146,4 +295,107 @@ impl ResourceAccountant { Ok(()) } + + pub fn check_filesystem_usage( + &self, + _usage: &FileSystemUsage, + resulting_bytes: u64, + resulting_inodes: usize, + ) -> Result<(), ResourceError> { + if let Some(limit) = self.limits.max_filesystem_bytes { + if resulting_bytes > limit { + return Err(ResourceError::filesystem_full( + "maximum filesystem size limit reached", + )); + } + } + + if let Some(limit) = self.limits.max_inode_count { + if resulting_inodes > limit { + return Err(ResourceError::filesystem_full( + "maximum inode count limit reached", + )); + } + } + Ok(()) + } +} + +fn argv_payload_bytes(command: &str, args: &[String]) -> usize { + let command_bytes = command.len().saturating_add(1); + command_bytes.saturating_add( + args.iter() + .map(|arg| arg.len().saturating_add(1)) + .sum::(), + ) +} + +fn env_entry_payload_bytes(key: &str, value: &str) -> usize { + key.len() + .saturating_add(1) + .saturating_add(value.len()) + .saturating_add(1) +} + +fn merged_env_payload_bytes( + inherited_env: &BTreeMap, + overrides: &BTreeMap, +) -> usize { + let mut total = inherited_env + .iter() + .map(|(key, value)| env_entry_payload_bytes(key, value)) + .sum::(); + + for (key, value) in overrides { + if let Some(previous) = inherited_env.get(key) { + total = total.saturating_sub(env_entry_payload_bytes(key, previous)); + } + total = total.saturating_add(env_entry_payload_bytes(key, value)); + } + + total +} + +pub fn measure_filesystem_usage( + filesystem: &mut F, +) -> VfsResult { + let mut visited = BTreeSet::new(); + measure_path_usage(filesystem, "/", &mut visited) +} + +fn measure_path_usage( + filesystem: &mut F, + path: &str, + visited: &mut BTreeSet, +) -> VfsResult { + let stat = filesystem.lstat(path)?; + let mut usage = FileSystemUsage::default(); + + if visited.insert(stat.ino) { + usage.inode_count += 1; + if !stat.is_directory { + usage.total_bytes = usage.total_bytes.saturating_add(stat.size); + } + } + + if !stat.is_directory || stat.is_symbolic_link { + return Ok(usage); + } + + for entry in filesystem.read_dir_with_types(path)? { + if matches!(entry.name.as_str(), "." | "..") { + continue; + } + + let child_path = if path == "/" { + format!("/{}", entry.name) + } else { + format!("{path}/{}", entry.name) + }; + let child_usage = measure_path_usage(filesystem, &child_path, visited)?; + usage.total_bytes = usage.total_bytes.saturating_add(child_usage.total_bytes); + usage.inode_count = usage.inode_count.saturating_add(child_usage.inode_count); + } + + Ok(usage) } diff --git a/crates/kernel/src/root_fs.rs b/crates/kernel/src/root_fs.rs index b688f5797..281edbc0c 100644 --- a/crates/kernel/src/root_fs.rs +++ b/crates/kernel/src/root_fs.rs @@ -235,6 +235,10 @@ impl VirtualFileSystem for RootFileSystem { self.overlay.read_dir(path) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + self.overlay.read_dir_limited(path, max_entries) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { self.overlay.read_dir_with_types(path) } @@ -243,6 +247,14 @@ impl VirtualFileSystem for RootFileSystem { self.overlay.write_file(path, content.into()) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + self.overlay.create_file_exclusive(path, content.into()) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + self.overlay.append_file(path, content.into()) + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { self.overlay.create_dir(path) } diff --git a/crates/kernel/src/vfs.rs b/crates/kernel/src/vfs.rs index 1a59f3e1a..9aeffca87 100644 --- a/crates/kernel/src/vfs.rs +++ b/crates/kernel/src/vfs.rs @@ -7,10 +7,12 @@ use std::time::{SystemTime, UNIX_EPOCH}; pub const S_IFREG: u32 = 0o100000; pub const S_IFDIR: u32 = 0o040000; pub const S_IFLNK: u32 = 0o120000; +const MEMORY_FILESYSTEM_DEVICE_ID: u64 = 1; const DEFAULT_UID: u32 = 1000; const DEFAULT_GID: u32 = 1000; const DIRECTORY_SIZE: u64 = 4096; +pub const MAX_PATH_LENGTH: usize = 4096; const MAX_SYMLINK_DEPTH: usize = 40; pub type VfsResult = Result; @@ -67,6 +69,10 @@ impl VfsError { Self::new("ENOTDIR", format!("not a directory, {op} '{path}'")) } + fn path_too_long(path: &str) -> Self { + Self::new("ENAMETOOLONG", format!("file name too long: {path}")) + } + fn not_empty(path: &str) -> Self { Self::new("ENOTEMPTY", format!("directory not empty, rmdir '{path}'")) } @@ -126,6 +132,9 @@ pub struct VirtualDirEntry { pub struct VirtualStat { pub mode: u32, pub size: u64, + pub blocks: u64, + pub dev: u64, + pub rdev: u64, pub is_directory: bool, pub is_symbolic_link: bool, pub atime_ms: u64, @@ -144,8 +153,35 @@ pub trait VirtualFileSystem { String::from_utf8(self.read_file(path)?).map_err(|_| VfsError::invalid_utf8(path)) } fn read_dir(&mut self, path: &str) -> VfsResult>; + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + let entries = self.read_dir(path)?; + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + Ok(entries) + } fn read_dir_with_types(&mut self, path: &str) -> VfsResult>; fn write_file(&mut self, path: &str, content: impl Into>) -> VfsResult<()>; + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + let content = content.into(); + if self.exists(path) { + return Err(VfsError::already_exists("open", path)); + } + self.write_file(path, content) + } + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + let content = content.into(); + let mut existing = self.read_file(path)?; + existing.extend_from_slice(&content); + let new_len = existing.len() as u64; + self.write_file(path, existing)?; + Ok(new_len) + } fn create_dir(&mut self, path: &str) -> VfsResult<()>; fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()>; fn exists(&self, path: &str) -> bool; @@ -327,6 +363,7 @@ impl MemoryFileSystem { follow_final_symlink: bool, depth: usize, ) -> VfsResult { + validate_path(path)?; if depth > MAX_SYMLINK_DEPTH { return Err(VfsError::symlink_loop(path)); } @@ -351,13 +388,13 @@ impl MemoryFileSystem { let is_final = index + 1 == components.len(); let should_follow = !is_final || follow_final_symlink; - if should_follow { - if let Some(ino) = self.path_index.get(&candidate) { - let inode = self - .inodes - .get(ino) - .expect("path index should always point at a valid inode"); + if let Some(ino) = self.path_index.get(&candidate) { + let inode = self + .inodes + .get(ino) + .expect("path index should always point at a valid inode"); + if should_follow { if let InodeKind::SymbolicLink { target } = &inode.kind { let target_path = if target.starts_with('/') { target.clone() @@ -377,6 +414,10 @@ impl MemoryFileSystem { ); } } + + if !is_final && !matches!(inode.kind, InodeKind::Directory) { + return Err(VfsError::not_directory("stat", &candidate)); + } } current = candidate; @@ -462,6 +503,11 @@ impl MemoryFileSystem { return Err(VfsError::is_directory("unlink", path)); } + self.inodes + .get_mut(&ino) + .expect("inode should exist when unlinking") + .metadata + .ctime_ms = now_ms(); self.path_index.remove(&normalized); self.decrement_link_count(ino); Ok(()) @@ -489,6 +535,11 @@ impl MemoryFileSystem { } } + self.inodes + .get_mut(&ino) + .expect("inode should exist when removing destination") + .metadata + .ctime_ms = now_ms(); self.path_index.remove(&normalized); self.decrement_link_count(ino); Ok(()) @@ -500,7 +551,7 @@ impl MemoryFileSystem { .inodes .get_mut(&ino) .expect("inode should exist when decrementing link count"); - inode.metadata.nlink -= 1; + inode.metadata.nlink = inode.metadata.nlink.saturating_sub(1); inode.metadata.nlink == 0 }; @@ -519,6 +570,9 @@ impl MemoryFileSystem { VirtualStat { mode: inode.metadata.mode, size, + blocks: block_count_for_size(size), + dev: MEMORY_FILESYSTEM_DEVICE_ID, + rdev: 0, is_directory: matches!(inode.kind, InodeKind::Directory), is_symbolic_link: matches!(inode.kind, InodeKind::SymbolicLink { .. }), atime_ms: inode.metadata.atime_ms, @@ -636,9 +690,49 @@ impl VirtualFileSystem for MemoryFileSystem { .collect()) } + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + self.assert_directory_path(path, "scandir")?; + let resolved = self.resolve_path(path, 0)?; + self.inode_mut_for_existing_path(&resolved, "scandir", false)? + .metadata + .atime_ms = now_ms(); + let prefix = if resolved == "/" { + String::from("/") + } else { + format!("{resolved}/") + }; + + let mut entries = BTreeMap::::new(); + for (candidate_path, _) in self.path_index.range(prefix.clone()..) { + if !candidate_path.starts_with(&prefix) { + break; + } + + let rest = &candidate_path[prefix.len()..]; + if rest.is_empty() || rest.contains('/') { + continue; + } + + entries.insert(String::from(rest), String::from(rest)); + if entries.len() > max_entries { + return Err(VfsError::new( + "ENOMEM", + format!( + "directory listing for '{path}' exceeds configured limit of {max_entries} entries" + ), + )); + } + } + + Ok(entries.into_values().collect()) + } + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { self.assert_directory_path(path, "scandir")?; let resolved = self.resolve_path(path, 0)?; + self.inode_mut_for_existing_path(&resolved, "scandir", false)? + .metadata + .atime_ms = now_ms(); let prefix = if resolved == "/" { String::from("/") } else { @@ -646,9 +740,9 @@ impl VirtualFileSystem for MemoryFileSystem { }; let mut entries = BTreeMap::::new(); - for (candidate_path, ino) in &self.path_index { + for (candidate_path, ino) in self.path_index.range(prefix.clone()..) { if !candidate_path.starts_with(&prefix) { - continue; + break; } let rest = &candidate_path[prefix.len()..]; @@ -698,6 +792,40 @@ impl VirtualFileSystem for MemoryFileSystem { Ok(()) } + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + let normalized = self.resolve_path(path, 0)?; + self.mkdir(&dirname(&normalized), true)?; + if self.path_index.contains_key(&normalized) { + return Err(VfsError::already_exists("open", path)); + } + + let ino = self.allocate_inode( + InodeKind::File { + data: content.into(), + }, + S_IFREG | 0o644, + ); + self.path_index.insert(normalized, ino); + Ok(()) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + let normalized = self.resolve_path(path, 0)?; + let data = content.into(); + let inode = self.inode_mut_for_existing_path(&normalized, "open", false)?; + let now = now_ms(); + match &mut inode.kind { + InodeKind::File { data: existing } => { + existing.extend_from_slice(&data); + inode.metadata.mtime_ms = now; + inode.metadata.ctime_ms = now; + Ok(existing.len() as u64) + } + InodeKind::Directory => Err(VfsError::is_directory("open", path)), + InodeKind::SymbolicLink { .. } => Err(VfsError::not_found("open", path)), + } + } + fn create_dir(&mut self, path: &str) -> VfsResult<()> { let normalized = self.resolve_exact_path(path)?; if normalized == "/" { @@ -855,6 +983,11 @@ impl VirtualFileSystem for MemoryFileSystem { if !is_directory { self.path_index.remove(&old_normalized); self.path_index.insert(new_normalized, ino); + self.inodes + .get_mut(&ino) + .expect("renamed inode should exist") + .metadata + .ctime_ms = now_ms(); return Ok(()); } @@ -879,6 +1012,12 @@ impl VirtualFileSystem for MemoryFileSystem { self.path_index.insert(relocated_path, inode_id); } + self.inodes + .get_mut(&ino) + .expect("renamed directory inode should exist") + .metadata + .ctime_ms = now_ms(); + Ok(()) } @@ -926,11 +1065,12 @@ impl VirtualFileSystem for MemoryFileSystem { self.assert_directory_path(&dirname(&normalized), "link")?; self.path_index.insert(normalized, ino); - self.inodes + let inode = self + .inodes .get_mut(&ino) - .expect("path index should always point at a valid inode") - .metadata - .nlink += 1; + .expect("path index should always point at a valid inode"); + inode.metadata.nlink += 1; + inode.metadata.ctime_ms = now_ms(); Ok(()) } @@ -1001,6 +1141,14 @@ impl Default for MemoryFileSystem { } } +pub fn validate_path(path: &str) -> VfsResult<()> { + let normalized = normalize_path(path); + if normalized.len() > MAX_PATH_LENGTH { + return Err(VfsError::path_too_long(path)); + } + Ok(()) +} + pub fn normalize_path(path: &str) -> String { if path.is_empty() { return String::from("/"); @@ -1030,6 +1178,14 @@ pub fn normalize_path(path: &str) -> String { } } +fn block_count_for_size(size: u64) -> u64 { + if size == 0 { + 0 + } else { + size.div_ceil(512) + } +} + fn dirname(path: &str) -> String { let normalized = normalize_path(path); let Some((head, _)) = normalized.rsplit_once('/') else { diff --git a/crates/kernel/tests/api_surface.rs b/crates/kernel/tests/api_surface.rs index 13345b21c..e2cef0d9e 100644 --- a/crates/kernel/tests/api_surface.rs +++ b/crates/kernel/tests/api_surface.rs @@ -1,9 +1,27 @@ use agent_os_kernel::command_registry::CommandDriver; -use agent_os_kernel::fd_table::{O_CREAT, O_RDWR}; +use agent_os_kernel::fd_table::{ + LOCK_EX, LOCK_NB, LOCK_SH, LOCK_UN, O_APPEND, O_CREAT, O_EXCL, O_NONBLOCK, O_RDWR, +}; use agent_os_kernel::kernel::{ - ExecOptions, KernelVm, KernelVmConfig, OpenShellOptions, SpawnOptions, WaitPidResult, SEEK_SET, + ExecOptions, KernelVm, KernelVmConfig, OpenShellOptions, SpawnOptions, WaitPidFlags, + WaitPidResult, SEEK_SET, +}; +use agent_os_kernel::mount_table::{MountOptions, MountTable}; +use agent_os_kernel::permissions::Permissions; +use agent_os_kernel::pipe_manager::MAX_PIPE_BUFFER_BYTES; +use agent_os_kernel::process_table::{ProcessWaitEvent, SIGWINCH}; +use agent_os_kernel::vfs::{ + MemoryFileSystem, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat, MAX_PATH_LENGTH, }; -use agent_os_kernel::vfs::{MemoryFileSystem, VirtualFileSystem}; +use std::cell::{Cell, RefCell}; + +fn assert_kernel_error_code( + result: agent_os_kernel::kernel::KernelResult, + expected: &str, +) { + let error = result.expect_err("operation should fail"); + assert_eq!(error.code(), expected); +} fn spawn_shell( kernel: &mut KernelVm, @@ -20,9 +38,204 @@ fn spawn_shell( .expect("spawn shell") } +fn spawn_shell_in( + kernel: &mut KernelVm, +) -> agent_os_kernel::kernel::KernelProcessHandle { + kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn shell") +} + +fn assert_not_trivial_pattern(bytes: &[u8]) { + assert!(bytes.iter().any(|byte| *byte != 0)); + assert!( + bytes.windows(2).any(|window| window[0] != window[1]), + "random data should not collapse to a repeated byte" + ); +} + +struct AtomicityProbeFileSystem { + inner: RefCell, + exclusive_race_pending: Cell, + append_race_pending: Cell, + target_path: &'static str, +} + +impl AtomicityProbeFileSystem { + fn new(target_path: &'static str) -> Self { + let mut inner = MemoryFileSystem::new(); + inner + .write_file(target_path, Vec::new()) + .expect("seed append target"); + Self { + inner: RefCell::new(inner), + exclusive_race_pending: Cell::new(false), + append_race_pending: Cell::new(false), + target_path, + } + } + + fn trigger_exclusive_race(&self) { + self.inner + .borrow_mut() + .remove_file(self.target_path) + .expect("clear target before exclusive race"); + self.exclusive_race_pending.set(true); + } + + fn trigger_append_race(&self) { + self.inner + .borrow_mut() + .write_file(self.target_path, Vec::new()) + .expect("reset target before append race"); + self.append_race_pending.set(true); + } +} + +impl VirtualFileSystem for AtomicityProbeFileSystem { + fn read_file(&mut self, path: &str) -> VfsResult> { + self.inner.borrow_mut().read_file(path) + } + + fn read_dir(&mut self, path: &str) -> VfsResult> { + self.inner.borrow_mut().read_dir(path) + } + + fn read_dir_limited(&mut self, path: &str, max_entries: usize) -> VfsResult> { + self.inner.borrow_mut().read_dir_limited(path, max_entries) + } + + fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { + self.inner.borrow_mut().read_dir_with_types(path) + } + + fn write_file(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + let content = content.into(); + if path == self.target_path { + if self.exclusive_race_pending.replace(false) { + self.inner + .borrow_mut() + .write_file(path, b"winner".to_vec()) + .expect("inject competing exclusive creator"); + } + if self.append_race_pending.replace(false) { + self.inner + .borrow_mut() + .write_file(path, b"RACE".to_vec()) + .expect("inject competing append writer"); + } + } + self.inner.borrow_mut().write_file(path, content) + } + + fn create_file_exclusive(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { + if path == self.target_path && self.exclusive_race_pending.replace(false) { + self.inner + .borrow_mut() + .write_file(path, b"winner".to_vec()) + .expect("inject competing exclusive creator"); + return Err(agent_os_kernel::vfs::VfsError::new( + "EEXIST", + format!("file already exists, open '{path}'"), + )); + } + self.inner.borrow_mut().create_file_exclusive(path, content) + } + + fn append_file(&mut self, path: &str, content: impl Into>) -> VfsResult { + if path == self.target_path && self.append_race_pending.replace(false) { + self.inner + .borrow_mut() + .append_file(path, b"RACE".to_vec()) + .expect("inject competing append writer"); + } + self.inner.borrow_mut().append_file(path, content) + } + + fn create_dir(&mut self, path: &str) -> VfsResult<()> { + self.inner.borrow_mut().create_dir(path) + } + + fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()> { + self.inner.borrow_mut().mkdir(path, recursive) + } + + fn exists(&self, path: &str) -> bool { + if path == self.target_path && self.exclusive_race_pending.get() { + return false; + } + self.inner.borrow().exists(path) + } + + fn stat(&mut self, path: &str) -> VfsResult { + self.inner.borrow_mut().stat(path) + } + + fn remove_file(&mut self, path: &str) -> VfsResult<()> { + self.inner.borrow_mut().remove_file(path) + } + + fn remove_dir(&mut self, path: &str) -> VfsResult<()> { + self.inner.borrow_mut().remove_dir(path) + } + + fn rename(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { + self.inner.borrow_mut().rename(old_path, new_path) + } + + fn realpath(&self, path: &str) -> VfsResult { + self.inner.borrow().realpath(path) + } + + fn symlink(&mut self, target: &str, link_path: &str) -> VfsResult<()> { + self.inner.borrow_mut().symlink(target, link_path) + } + + fn read_link(&self, path: &str) -> VfsResult { + self.inner.borrow().read_link(path) + } + + fn lstat(&self, path: &str) -> VfsResult { + self.inner.borrow().lstat(path) + } + + fn link(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { + self.inner.borrow_mut().link(old_path, new_path) + } + + fn chmod(&mut self, path: &str, mode: u32) -> VfsResult<()> { + self.inner.borrow_mut().chmod(path, mode) + } + + fn chown(&mut self, path: &str, uid: u32, gid: u32) -> VfsResult<()> { + self.inner.borrow_mut().chown(path, uid, gid) + } + + fn utimes(&mut self, path: &str, atime_ms: u64, mtime_ms: u64) -> VfsResult<()> { + self.inner.borrow_mut().utimes(path, atime_ms, mtime_ms) + } + + fn truncate(&mut self, path: &str, length: u64) -> VfsResult<()> { + self.inner.borrow_mut().truncate(path, length) + } + + fn pread(&mut self, path: &str, offset: u64, length: usize) -> VfsResult> { + self.inner.borrow_mut().pread(path, offset, length) + } +} + #[test] fn kernel_fd_surface_supports_open_seek_positional_io_dup_and_dev_fd_views() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-api-fd")); + let mut config = KernelVmConfig::new("vm-api-fd"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -108,6 +321,9 @@ fn kernel_fd_surface_supports_open_seek_positional_io_dup_and_dev_fd_views() { .dev_fd_stat("shell", process.pid(), fd) .expect("stat regular file fd"); assert_eq!(file_stat.size, 5); + assert_eq!(file_stat.blocks, 1); + assert_eq!(file_stat.dev, 1); + assert_eq!(file_stat.rdev, 0); assert!(!file_stat.is_directory); let (read_fd, write_fd) = kernel.open_pipe("shell", process.pid()).expect("open pipe"); @@ -133,15 +349,413 @@ fn kernel_fd_surface_supports_open_seek_positional_io_dup_and_dev_fd_views() { .expect("stat pipe fd"); assert_eq!(pipe_stat.mode, 0o666); assert_eq!(pipe_stat.size, 0); + assert_eq!(pipe_stat.blocks, 0); + assert_eq!(pipe_stat.dev, 2); assert!(!pipe_stat.is_directory); process.finish(0); kernel.waitpid(process.pid()).expect("wait for shell"); } +#[test] +fn kernel_process_umask_applies_to_created_files_and_directories() { + let mut config = KernelVmConfig::new("vm-api-umask"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = spawn_shell(&mut kernel); + assert_eq!( + kernel + .umask("shell", process.pid(), None) + .expect("read default umask"), + 0o022 + ); + assert_eq!( + kernel + .umask("shell", process.pid(), Some(0o027)) + .expect("set umask"), + 0o022 + ); + + let created_fd = kernel + .fd_open( + "shell", + process.pid(), + "/tmp/umask-file.txt", + O_CREAT | O_RDWR, + Some(0o666), + ) + .expect("create file with umask"); + kernel + .fd_close("shell", process.pid(), created_fd) + .expect("close created fd"); + let file_stat = kernel.stat("/tmp/umask-file.txt").expect("stat umask file"); + assert_eq!(file_stat.mode & 0o777, 0o640); + + kernel + .mkdir_for_process( + "shell", + process.pid(), + "/tmp/private-dir", + false, + Some(0o777), + ) + .expect("create directory with umask"); + let dir_stat = kernel.stat("/tmp/private-dir").expect("stat private dir"); + assert_eq!(dir_stat.mode & 0o777, 0o750); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait for shell"); +} + +#[test] +fn kernel_fd_surface_reads_exact_byte_counts_from_device_nodes() { + let mut config = KernelVmConfig::new("vm-api-fd-devices"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = spawn_shell(&mut kernel); + + let zero_fd = kernel + .fd_open("shell", process.pid(), "/dev/zero", O_RDWR, None) + .expect("open /dev/zero"); + let zeroes = kernel + .fd_read("shell", process.pid(), zero_fd, 5) + .expect("read 5 bytes from /dev/zero"); + assert_eq!(zeroes.len(), 5); + assert!(zeroes.iter().all(|byte| *byte == 0)); + + let random_fd = kernel + .fd_open("shell", process.pid(), "/dev/urandom", O_RDWR, None) + .expect("open /dev/urandom"); + let random = kernel + .fd_read("shell", process.pid(), random_fd, 1024 * 1024) + .expect("read 1MiB from /dev/urandom"); + assert_eq!(random.len(), 1024 * 1024); + assert_not_trivial_pattern(&random[..1024]); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait for shell"); +} + +#[test] +fn kernel_fd_surface_supports_nonblocking_pipe_duplicates_via_dev_fd() { + let mut config = KernelVmConfig::new("vm-api-fd-nonblock"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = spawn_shell(&mut kernel); + let (read_fd, write_fd) = kernel.open_pipe("shell", process.pid()).expect("open pipe"); + let nonblocking_read_fd = kernel + .fd_open( + "shell", + process.pid(), + &format!("/dev/fd/{read_fd}"), + O_NONBLOCK, + None, + ) + .expect("duplicate read end with O_NONBLOCK"); + let nonblocking_write_fd = kernel + .fd_open( + "shell", + process.pid(), + &format!("/dev/fd/{write_fd}"), + O_NONBLOCK, + None, + ) + .expect("duplicate write end with O_NONBLOCK"); + + assert_eq!( + kernel + .fd_stat("shell", process.pid(), read_fd) + .expect("stat blocking read fd") + .flags + & O_NONBLOCK, + 0 + ); + assert_ne!( + kernel + .fd_stat("shell", process.pid(), nonblocking_read_fd) + .expect("stat nonblocking read fd") + .flags + & O_NONBLOCK, + 0 + ); + assert_ne!( + kernel + .fd_stat("shell", process.pid(), nonblocking_write_fd) + .expect("stat nonblocking write fd") + .flags + & O_NONBLOCK, + 0 + ); + + assert_kernel_error_code( + kernel.fd_read("shell", process.pid(), nonblocking_read_fd, 1), + "EAGAIN", + ); + + kernel + .fd_write( + "shell", + process.pid(), + write_fd, + &vec![7; MAX_PIPE_BUFFER_BYTES], + ) + .expect("fill pipe buffer"); + assert_kernel_error_code( + kernel.fd_write("shell", process.pid(), nonblocking_write_fd, &[8]), + "EAGAIN", + ); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait for shell"); +} + +#[test] +fn kernel_fd_surface_uses_atomic_exclusive_create() { + let target = "/tmp/race.txt"; + let filesystem = AtomicityProbeFileSystem::new(target); + filesystem.trigger_exclusive_race(); + + let mut config = KernelVmConfig::new("vm-api-exclusive-create"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(filesystem, config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = spawn_shell_in(&mut kernel); + assert_kernel_error_code( + kernel.fd_open( + "shell", + process.pid(), + target, + O_CREAT | O_EXCL | O_RDWR, + None, + ), + "EEXIST", + ); + assert_eq!( + kernel + .filesystem_mut() + .read_file(target) + .expect("winner should remain visible"), + b"winner".to_vec() + ); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait shell"); +} + +#[test] +fn kernel_fd_surface_uses_atomic_append_writes() { + let target = "/tmp/race.txt"; + let filesystem = AtomicityProbeFileSystem::new(target); + filesystem.trigger_append_race(); + + let mut config = KernelVmConfig::new("vm-api-append-write"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(filesystem, config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = spawn_shell_in(&mut kernel); + let fd = kernel + .fd_open("shell", process.pid(), target, O_APPEND | O_RDWR, None) + .expect("open append target"); + assert_eq!( + kernel + .fd_write("shell", process.pid(), fd, b"mine") + .expect("append write"), + 4 + ); + assert_eq!( + kernel + .filesystem_mut() + .read_file(target) + .expect("read appended file"), + b"RACEmine".to_vec() + ); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait shell"); +} + +#[test] +fn kernel_fd_surface_supports_advisory_locks_and_releases_on_last_close() { + let mut config = KernelVmConfig::new("vm-api-flock-close"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .filesystem_mut() + .write_file("/tmp/lock.txt", b"lock".to_vec()) + .expect("seed file"); + + let owner = spawn_shell(&mut kernel); + let contender = spawn_shell(&mut kernel); + let owner_fd = kernel + .fd_open("shell", owner.pid(), "/tmp/lock.txt", O_RDWR, None) + .expect("owner opens lock file"); + let owner_dup = kernel + .fd_dup("shell", owner.pid(), owner_fd) + .expect("duplicate owner fd"); + let contender_fd = kernel + .fd_open("shell", contender.pid(), "/tmp/lock.txt", O_RDWR, None) + .expect("contender opens lock file"); + + kernel + .fd_flock("shell", owner.pid(), owner_fd, LOCK_EX) + .expect("owner acquires exclusive lock"); + kernel + .fd_flock("shell", owner.pid(), owner_dup, LOCK_EX | LOCK_NB) + .expect("duplicate shares exclusive lock"); + assert_kernel_error_code( + kernel.fd_flock("shell", contender.pid(), contender_fd, LOCK_SH | LOCK_NB), + "EWOULDBLOCK", + ); + + kernel + .fd_close("shell", owner.pid(), owner_fd) + .expect("close original owner fd"); + assert_kernel_error_code( + kernel.fd_flock("shell", contender.pid(), contender_fd, LOCK_SH | LOCK_NB), + "EWOULDBLOCK", + ); + + kernel + .fd_close("shell", owner.pid(), owner_dup) + .expect("close duplicate owner fd"); + kernel + .fd_flock("shell", contender.pid(), contender_fd, LOCK_SH | LOCK_NB) + .expect("lock released on last close"); + kernel + .fd_flock("shell", contender.pid(), contender_fd, LOCK_UN) + .expect("unlock contender"); + + owner.finish(0); + contender.finish(0); + kernel.waitpid(owner.pid()).expect("wait owner"); + kernel.waitpid(contender.pid()).expect("wait contender"); +} + +#[test] +fn kernel_fd_surface_supports_shared_locks_and_nonblocking_upgrade_conflicts() { + let mut config = KernelVmConfig::new("vm-api-flock-shared"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .filesystem_mut() + .write_file("/tmp/shared-lock.txt", b"shared".to_vec()) + .expect("seed file"); + + let first = spawn_shell(&mut kernel); + let second = spawn_shell(&mut kernel); + let first_fd = kernel + .fd_open("shell", first.pid(), "/tmp/shared-lock.txt", O_RDWR, None) + .expect("first opens file"); + let second_fd = kernel + .fd_open("shell", second.pid(), "/tmp/shared-lock.txt", O_RDWR, None) + .expect("second opens file"); + + kernel + .fd_flock("shell", first.pid(), first_fd, LOCK_SH) + .expect("first shared lock"); + kernel + .fd_flock("shell", second.pid(), second_fd, LOCK_SH) + .expect("second shared lock"); + assert_kernel_error_code( + kernel.fd_flock("shell", first.pid(), first_fd, LOCK_EX | LOCK_NB), + "EWOULDBLOCK", + ); + + kernel + .fd_flock("shell", second.pid(), second_fd, LOCK_UN) + .expect("unlock second shared lock"); + kernel + .fd_flock("shell", first.pid(), first_fd, LOCK_EX | LOCK_NB) + .expect("first upgrades to exclusive once peer unlocks"); + + first.finish(0); + second.finish(0); + kernel.waitpid(first.pid()).expect("wait first"); + kernel.waitpid(second.pid()).expect("wait second"); +} + +#[test] +fn kernel_fd_surface_shares_advisory_locks_across_fork_inherited_fds() { + let mut config = KernelVmConfig::new("vm-api-flock-fork"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .filesystem_mut() + .write_file("/tmp/fork-lock.txt", b"fork".to_vec()) + .expect("seed file"); + + let parent = spawn_shell(&mut kernel); + let inherited_fd = kernel + .fd_open("shell", parent.pid(), "/tmp/fork-lock.txt", O_RDWR, None) + .expect("parent opens file"); + kernel + .fd_flock("shell", parent.pid(), inherited_fd, LOCK_EX) + .expect("parent acquires exclusive lock"); + + let child = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + parent_pid: Some(parent.pid()), + ..SpawnOptions::default() + }, + ) + .expect("spawn child with inherited fds"); + let contender = spawn_shell(&mut kernel); + let contender_fd = kernel + .fd_open("shell", contender.pid(), "/tmp/fork-lock.txt", O_RDWR, None) + .expect("contender opens file"); + + kernel + .fd_flock("shell", child.pid(), inherited_fd, LOCK_EX | LOCK_NB) + .expect("child sees the inherited open-file-description lock"); + assert_kernel_error_code( + kernel.fd_flock("shell", contender.pid(), contender_fd, LOCK_SH | LOCK_NB), + "EWOULDBLOCK", + ); + + parent.finish(0); + child.finish(0); + contender.finish(0); + kernel.waitpid(parent.pid()).expect("wait parent"); + kernel.waitpid(child.pid()).expect("wait child"); + kernel.waitpid(contender.pid()).expect("wait contender"); +} + #[test] fn waitpid_returns_structured_result_and_process_introspection_works() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-api-proc")); + let mut config = KernelVmConfig::new("vm-api-proc"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -203,9 +817,218 @@ fn waitpid_returns_structured_result_and_process_introspection_works() { kernel.waitpid(parent.pid()).expect("wait parent"); } +#[test] +fn waitpid_with_options_supports_wnohang_and_any_child_waits() { + let mut config = KernelVmConfig::new("vm-api-waitpid-flags"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let parent = spawn_shell(&mut kernel); + let child = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + parent_pid: Some(parent.pid()), + ..SpawnOptions::default() + }, + ) + .expect("spawn child"); + + assert_eq!( + kernel + .waitpid_with_options("shell", parent.pid(), -1, WaitPidFlags::WNOHANG) + .expect("wnohang wait should succeed"), + None + ); + + child.finish(9); + let waited = kernel + .waitpid_with_options("shell", parent.pid(), -1, WaitPidFlags::empty()) + .expect("wait for any child should succeed") + .expect("child exit should be reported"); + assert_eq!(waited.pid, child.pid()); + assert_eq!(waited.status, 9); + assert_eq!(waited.event, ProcessWaitEvent::Exited); + assert_eq!( + kernel.list_processes().get(&child.pid()), + None, + "exited child should be reaped after wait" + ); + + parent.finish(0); + kernel.waitpid(parent.pid()).expect("wait parent"); +} + +#[test] +fn proc_filesystem_exposes_live_process_metadata_and_fd_symlinks() { + let mut config = KernelVmConfig::new("vm-api-procfs"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .filesystem_mut() + .write_file("/tmp/data.txt", b"hello".to_vec()) + .expect("seed procfs data file"); + + let process = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + cwd: Some(String::from("/tmp")), + env: std::collections::BTreeMap::from([( + String::from("VISIBLE_MARKER"), + String::from("present"), + )]), + ..SpawnOptions::default() + }, + ) + .expect("spawn procfs shell"); + let fd = kernel + .fd_open("shell", process.pid(), "/tmp/data.txt", O_RDWR, None) + .expect("open procfs data file"); + + let proc_entries = kernel + .read_dir_for_process("shell", process.pid(), "/proc") + .expect("read /proc"); + assert!(proc_entries.contains(&String::from("self"))); + assert!(proc_entries.contains(&String::from("mounts"))); + assert!(proc_entries.contains(&process.pid().to_string())); + + assert_eq!( + kernel + .read_link_for_process("shell", process.pid(), "/proc/self") + .expect("read /proc/self link"), + format!("/proc/{}", process.pid()) + ); + assert_eq!( + kernel + .realpath_for_process("shell", process.pid(), "/proc/self") + .expect("realpath /proc/self"), + format!("/proc/{}", process.pid()) + ); + + let self_lstat = kernel + .lstat_for_process("shell", process.pid(), "/proc/self") + .expect("lstat /proc/self"); + assert!(self_lstat.is_symbolic_link); + let self_stat = kernel + .stat_for_process("shell", process.pid(), "/proc/self") + .expect("stat /proc/self"); + assert!(self_stat.is_directory); + + let fd_entries = kernel + .read_dir_for_process("shell", process.pid(), "/proc/self/fd") + .expect("read /proc/self/fd"); + assert!(fd_entries.contains(&String::from("0"))); + assert!(fd_entries.contains(&fd.to_string())); + assert_eq!( + kernel + .read_link_for_process("shell", process.pid(), &format!("/proc/self/fd/{fd}"),) + .expect("read proc fd link"), + String::from("/tmp/data.txt") + ); + + assert_eq!( + kernel + .read_link_for_process("shell", process.pid(), "/proc/self/cwd") + .expect("read cwd link"), + String::from("/tmp") + ); + assert_eq!( + kernel + .read_file_for_process("shell", process.pid(), "/proc/self/cmdline") + .expect("read cmdline"), + b"sh\0".to_vec() + ); + + let environ = String::from_utf8( + kernel + .read_file_for_process("shell", process.pid(), "/proc/self/environ") + .expect("read environ"), + ) + .expect("proc environ should be utf8"); + assert!(environ.contains("VISIBLE_MARKER=present")); + + let stat_text = String::from_utf8( + kernel + .read_file_for_process("shell", process.pid(), "/proc/self/stat") + .expect("read stat"), + ) + .expect("proc stat should be utf8"); + assert!(stat_text.starts_with(&format!("{} (sh) R ", process.pid()))); + + let error = kernel + .write_file("/proc/mounts", b"blocked".to_vec()) + .expect_err("procfs should be read-only"); + assert_eq!(error.code(), "EROFS"); + + process.finish(0); + kernel.waitpid(process.pid()).expect("wait procfs shell"); +} + +#[test] +fn proc_mounts_lists_root_and_active_mounts() { + let mut config = KernelVmConfig::new("vm-api-proc-mounts"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + kernel + .mount_filesystem( + "/data", + MemoryFileSystem::new(), + MountOptions::new("memory").read_only(true), + ) + .expect("mount memory filesystem"); + + let mounts = String::from_utf8(kernel.read_file("/proc/mounts").expect("read proc mounts")) + .expect("proc mounts should be utf8"); + assert!(mounts.contains("root / root rw 0 0")); + assert!(mounts.contains("memory /data memory ro 0 0")); +} + +#[test] +fn filesystem_operations_return_linux_errno_values_for_common_failures() { + let mut config = KernelVmConfig::new("vm-api-errno"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + + kernel.create_dir("/dir").expect("create dir"); + assert_kernel_error_code(kernel.write_file("/dir", b"blocked".to_vec()), "EISDIR"); + + kernel + .write_file("/file", b"parent".to_vec()) + .expect("write file parent"); + assert_kernel_error_code(kernel.stat("/file/child"), "ENOTDIR"); + + let long_path = format!("/{}", "a".repeat(MAX_PATH_LENGTH)); + assert_kernel_error_code(kernel.stat(&long_path), "ENAMETOOLONG"); + + kernel + .mount_filesystem( + "/readonly", + MemoryFileSystem::new(), + MountOptions::new("memory").read_only(true), + ) + .expect("mount readonly fs"); + assert_kernel_error_code( + kernel.write_file("/readonly/blocked.txt", b"blocked".to_vec()), + "EROFS", + ); +} + #[test] fn open_shell_configures_pty_and_exec_uses_shell_driver() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-api-shell")); + let mut config = KernelVmConfig::new("vm-api-shell"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -254,6 +1077,83 @@ fn open_shell_configures_pty_and_exec_uses_shell_driver() { kernel.waitpid(exec.pid()).expect("wait exec"); } +#[test] +fn pty_resize_delivers_sigwinch_to_the_foreground_process_group() { + let mut config = KernelVmConfig::new("vm-api-shell"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let shell = kernel + .open_shell(OpenShellOptions { + requester_driver: Some(String::from("shell")), + ..OpenShellOptions::default() + }) + .expect("open shell"); + + kernel + .pty_resize("shell", shell.pid(), shell.master_fd(), 120, 40) + .expect("resize shell pty"); + kernel + .pty_resize("shell", shell.pid(), shell.master_fd(), 120, 40) + .expect("repeat shell pty resize"); + + assert_eq!(shell.process().kill_signals(), vec![SIGWINCH]); + + shell.process().finish(0); + kernel.waitpid(shell.pid()).expect("wait shell"); +} + +#[test] +fn shell_foreground_process_group_must_stay_in_the_same_session() { + let mut config = KernelVmConfig::new("vm-api-shell"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let shell = kernel + .open_shell(OpenShellOptions { + requester_driver: Some(String::from("shell")), + ..OpenShellOptions::default() + }) + .expect("open shell"); + let peer = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + parent_pid: Some(shell.pid()), + ..SpawnOptions::default() + }, + ) + .expect("spawn peer"); + + assert_eq!( + kernel.getsid("shell", peer.pid()).expect("peer sid"), + shell.pid() + ); + assert_eq!( + kernel.setsid("shell", peer.pid()).expect("setsid"), + peer.pid() + ); + + let error = kernel + .pty_set_foreground_pgid("shell", shell.pid(), shell.master_fd(), peer.pid()) + .expect_err("different-session process group should be rejected"); + assert_eq!(error.code(), "EPERM"); + assert!(error.to_string().contains("different session")); + + peer.finish(0); + kernel.waitpid(peer.pid()).expect("wait peer"); + shell.process().finish(0); + kernel.waitpid(shell.pid()).expect("wait shell"); +} + #[test] fn virtual_filesystem_default_pwrite_zero_fills_missing_bytes() { let mut filesystem = MemoryFileSystem::new(); diff --git a/crates/kernel/tests/device_layer.rs b/crates/kernel/tests/device_layer.rs index 6e2429e3c..bc54a9895 100644 --- a/crates/kernel/tests/device_layer.rs +++ b/crates/kernel/tests/device_layer.rs @@ -42,19 +42,22 @@ fn special_devices_expose_expected_read_and_write_behavior() { filesystem .write_file("/dev/zero", "ignored") .expect("write /dev/zero"); - let zeroes = filesystem.read_file("/dev/zero").expect("read /dev/zero"); - assert_eq!(zeroes.len(), 4096); + let zeroes = filesystem + .pread("/dev/zero", 0, 5) + .expect("pread 5 bytes from /dev/zero"); + assert_eq!(zeroes.len(), 5); assert!(zeroes.iter().all(|byte| *byte == 0)); let first = filesystem .pread("/dev/urandom", 0, 1024) .expect("pread /dev/urandom"); let second = filesystem - .read_file("/dev/urandom") - .expect("read /dev/urandom twice"); + .pread("/dev/urandom", 0, 1024 * 1024) + .expect("pread 1MiB from /dev/urandom"); assert_eq!(first.len(), 1024); - assert_eq!(second.len(), 4096); + assert_eq!(second.len(), 1024 * 1024); assert_not_trivial_pattern(&first); + assert_not_trivial_pattern(&second[..1024]); assert_ne!(first, second); } diff --git a/crates/kernel/tests/fd_table.rs b/crates/kernel/tests/fd_table.rs index 1b54553fd..858509cea 100644 --- a/crates/kernel/tests/fd_table.rs +++ b/crates/kernel/tests/fd_table.rs @@ -1,5 +1,7 @@ use agent_os_kernel::fd_table::{ - FdResult, FdTableManager, FILETYPE_CHARACTER_DEVICE, FILETYPE_REGULAR_FILE, O_RDONLY, O_WRONLY, + FdResult, FdTableManager, FileDescription, FileLockManager, FileLockTarget, FlockOperation, + FILETYPE_CHARACTER_DEVICE, FILETYPE_REGULAR_FILE, LOCK_EX, LOCK_NB, LOCK_SH, LOCK_UN, + MAX_FDS_PER_PROCESS, O_NONBLOCK, O_RDONLY, O_WRONLY, }; use std::fmt::Debug; use std::sync::Arc; @@ -77,6 +79,36 @@ fn dup2_replaces_the_target_fd() { assert!(Arc::ptr_eq(&source, &target)); } +#[test] +fn dup2_rejects_target_fds_past_the_process_limit() { + let mut manager = FdTableManager::new(); + manager.create(1); + + let table = manager.get_mut(1).expect("FD table should exist"); + let fd = table + .open("/tmp/test.txt", O_RDONLY) + .expect("open source FD"); + let result = table.dup2(fd, MAX_FDS_PER_PROCESS as u32); + + assert_error_code(result, "EBADF"); +} + +#[test] +fn open_with_rejects_target_fds_past_the_process_limit() { + let mut manager = FdTableManager::new(); + manager.create(1); + + let table = manager.get_mut(1).expect("FD table should exist"); + let description = Arc::new(FileDescription::new(999, "/tmp/test.txt", O_RDONLY)); + let result = table.open_with( + description, + FILETYPE_REGULAR_FILE, + Some(MAX_FDS_PER_PROCESS as u32), + ); + + assert_error_code(result, "EBADF"); +} + #[test] fn close_decrements_refcount() { let mut manager = FdTableManager::new(); @@ -149,6 +181,42 @@ fn stat_returns_fd_metadata() { assert_eq!(stat.flags, O_WRONLY); } +#[test] +fn nonblocking_status_flags_are_tracked_per_fd_entry() { + let mut manager = FdTableManager::new(); + manager.create(1); + + let table = manager.get_mut(1).expect("FD table should exist"); + let fd = table + .open_with_filetype( + "/tmp/test.txt", + O_WRONLY | O_NONBLOCK, + FILETYPE_REGULAR_FILE, + ) + .expect("open regular file"); + let dup_fd = table + .dup_with_status_flags(fd, Some(0)) + .expect("duplicate regular file without nonblocking"); + + let original = table.stat(fd).expect("stat original FD"); + let duplicated = table.stat(dup_fd).expect("stat duplicate FD"); + + assert_eq!(original.flags, O_WRONLY | O_NONBLOCK); + assert_eq!(duplicated.flags, O_WRONLY); + assert_eq!( + table.get(fd).expect("original entry").description.flags(), + O_WRONLY + ); + assert_eq!( + table + .get(dup_fd) + .expect("duplicate entry") + .description + .flags(), + O_WRONLY + ); +} + #[test] fn stat_reports_ebadf_for_invalid_fd() { let mut manager = FdTableManager::new(); @@ -158,3 +226,87 @@ fn stat_reports_ebadf_for_invalid_fd() { assert_error_code(result, "EBADF"); } + +#[test] +fn open_reuses_a_freed_fd_after_next_fd_moves_past_the_limit() { + let mut manager = FdTableManager::new(); + manager.create(1); + + let table = manager.get_mut(1).expect("FD table should exist"); + let mut opened = Vec::new(); + for _ in 3..MAX_FDS_PER_PROCESS { + opened.push( + table + .open("/tmp/test.txt", O_RDONLY) + .expect("open should fill remaining slots"), + ); + } + + assert!(table.close(5), "fd 5 should be open before reuse"); + + let reused = table + .open("/tmp/reused.txt", O_RDONLY) + .expect("open should wrap and reuse a freed fd"); + assert_eq!(reused, 5); +} + +#[test] +fn flock_operation_parser_accepts_supported_modes() { + assert_eq!( + FlockOperation::from_bits(LOCK_SH).expect("shared operation"), + FlockOperation::Shared { nonblocking: false } + ); + assert_eq!( + FlockOperation::from_bits(LOCK_EX | LOCK_NB).expect("exclusive nonblocking operation"), + FlockOperation::Exclusive { nonblocking: true } + ); + assert_eq!( + FlockOperation::from_bits(LOCK_UN).expect("unlock operation"), + FlockOperation::Unlock + ); +} + +#[test] +fn flock_manager_enforces_shared_and_exclusive_conflicts() { + let locks = FileLockManager::new(); + let target = FileLockTarget::new(42); + + locks + .apply(1, target, FlockOperation::Shared { nonblocking: false }) + .expect("first shared lock"); + locks + .apply(2, target, FlockOperation::Shared { nonblocking: false }) + .expect("second shared lock"); + + let blocked = locks.apply(3, target, FlockOperation::Exclusive { nonblocking: true }); + assert_error_code(blocked, "EWOULDBLOCK"); + + locks + .apply(1, target, FlockOperation::Unlock) + .expect("unlock first shared lock"); + locks + .apply(2, target, FlockOperation::Unlock) + .expect("unlock second shared lock"); + locks + .apply(3, target, FlockOperation::Exclusive { nonblocking: true }) + .expect("exclusive lock becomes available"); +} + +#[test] +fn flock_manager_treats_reacquire_on_same_description_as_non_conflicting() { + let locks = FileLockManager::new(); + let target = FileLockTarget::new(7); + + locks + .apply(99, target, FlockOperation::Exclusive { nonblocking: false }) + .expect("initial exclusive lock"); + locks + .apply(99, target, FlockOperation::Exclusive { nonblocking: true }) + .expect("same description can reacquire exclusive lock"); + locks + .apply(99, target, FlockOperation::Shared { nonblocking: true }) + .expect("same description can downgrade to shared lock"); + + let shared = locks.apply(100, target, FlockOperation::Shared { nonblocking: true }); + shared.expect("downgrade should allow other shared holders"); +} diff --git a/crates/kernel/tests/kernel_integration.rs b/crates/kernel/tests/kernel_integration.rs index a924d5db7..791741353 100644 --- a/crates/kernel/tests/kernel_integration.rs +++ b/crates/kernel/tests/kernel_integration.rs @@ -1,13 +1,17 @@ use agent_os_kernel::bridge::LifecycleState; use agent_os_kernel::command_registry::CommandDriver; use agent_os_kernel::kernel::{KernelVm, KernelVmConfig, SpawnOptions}; +use agent_os_kernel::permissions::Permissions; +use agent_os_kernel::process_table::SIGPIPE; use agent_os_kernel::pty::LineDisciplineConfig; use agent_os_kernel::vfs::MemoryFileSystem; use std::time::Duration; #[test] fn minimal_vm_lifecycle_transitions_between_ready_busy_and_terminated() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-kernel")); + let mut config = KernelVmConfig::new("vm-kernel"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -62,7 +66,9 @@ fn minimal_vm_lifecycle_transitions_between_ready_busy_and_terminated() { #[test] fn dispose_kills_running_processes_and_cleans_special_resources() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-dispose")); + let mut config = KernelVmConfig::new("vm-dispose"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -94,10 +100,9 @@ fn dispose_kills_running_processes_and_cleans_special_resources() { #[test] fn process_exit_cleanup_closes_pipe_writers_and_returns_eof_to_readers() { - let mut kernel = KernelVm::new( - MemoryFileSystem::new(), - KernelVmConfig::new("vm-process-exit-pipe"), - ); + let mut config = KernelVmConfig::new("vm-process-exit-pipe"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -153,12 +158,46 @@ fn process_exit_cleanup_closes_pipe_writers_and_returns_eof_to_readers() { assert!(eof.is_empty()); } +#[test] +fn broken_pipe_writes_deliver_sigpipe_and_return_epipe() { + let mut config = KernelVmConfig::new("vm-broken-pipe-sigpipe"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let writer = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn writer"); + let (read_fd, write_fd) = kernel + .open_pipe("shell", writer.pid()) + .expect("open writer pipe"); + + kernel + .fd_close("shell", writer.pid(), read_fd) + .expect("close inherited read end"); + + let error = kernel + .fd_write("shell", writer.pid(), write_fd, b"fail") + .expect_err("broken pipe writes should fail"); + assert_eq!(error.code(), "EPIPE"); + assert_eq!(writer.kill_signals(), vec![SIGPIPE]); + assert_eq!(writer.wait(Duration::from_millis(50)), Some(128 + SIGPIPE)); +} + #[test] fn process_exit_cleanup_removes_fd_tables_before_and_after_reap() { - let mut kernel = KernelVm::new( - MemoryFileSystem::new(), - KernelVmConfig::new("vm-process-exit-fds"), - ); + let mut config = KernelVmConfig::new("vm-process-exit-fds"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -200,3 +239,88 @@ fn process_exit_cleanup_removes_fd_tables_before_and_after_reap() { "ESRCH" ); } + +#[test] +fn spawn_process_executes_shebang_scripts_with_registered_interpreters() { + let mut config = KernelVmConfig::new("vm-shebang"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .register_driver(CommandDriver::new("node", ["node"])) + .expect("register node"); + + kernel + .write_file("/tmp/script.sh", b"#!/bin/sh -eu\necho shell\n".to_vec()) + .expect("write shell script"); + kernel + .chmod("/tmp/script.sh", 0o755) + .expect("chmod shell script"); + let shell_process = kernel + .spawn_process( + "/tmp/script.sh", + vec![String::from("arg")], + SpawnOptions::default(), + ) + .expect("spawn shell script"); + assert_eq!( + kernel + .read_file(&format!("/proc/{}/cmdline", shell_process.pid())) + .expect("read shell cmdline"), + b"sh\0-eu\0/tmp/script.sh\0arg\0".to_vec() + ); + + kernel + .write_file( + "/tmp/script.mjs", + b"#!/usr/bin/env node --trace-warnings\nconsole.log('node');\n".to_vec(), + ) + .expect("write node script"); + kernel + .chmod("/tmp/script.mjs", 0o755) + .expect("chmod node script"); + let node_process = kernel + .spawn_process("/tmp/script.mjs", Vec::new(), SpawnOptions::default()) + .expect("spawn node script"); + assert_eq!( + kernel + .read_file(&format!("/proc/{}/cmdline", node_process.pid())) + .expect("read node cmdline"), + b"node\0--trace-warnings\0/tmp/script.mjs\0".to_vec() + ); +} + +#[test] +fn spawn_process_rejects_invalid_shebang_scripts() { + let mut config = KernelVmConfig::new("vm-shebang-errors"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + kernel + .write_file("/tmp/missing.sh", b"#!/missing/interpreter\n".to_vec()) + .expect("write missing-interpreter script"); + kernel + .chmod("/tmp/missing.sh", 0o755) + .expect("chmod missing-interpreter script"); + let missing = kernel + .spawn_process("/tmp/missing.sh", Vec::new(), SpawnOptions::default()) + .expect_err("missing interpreter should fail"); + assert_eq!(missing.code(), "ENOENT"); + + let long_shebang = format!("#!/{0}\n", "a".repeat(256)); + kernel + .write_file("/tmp/long.sh", long_shebang.into_bytes()) + .expect("write long-shebang script"); + kernel + .chmod("/tmp/long.sh", 0o755) + .expect("chmod long-shebang script"); + let long_error = kernel + .spawn_process("/tmp/long.sh", Vec::new(), SpawnOptions::default()) + .expect_err("overlong shebang should fail"); + assert_eq!(long_error.code(), "ENOEXEC"); +} diff --git a/crates/kernel/tests/mount_table.rs b/crates/kernel/tests/mount_table.rs index bd3f48a12..01fc5f8ee 100644 --- a/crates/kernel/tests/mount_table.rs +++ b/crates/kernel/tests/mount_table.rs @@ -60,3 +60,47 @@ fn mount_table_enforces_read_only_and_cross_mount_boundaries() { .expect_err("rename across mounts should fail"); assert_eq!(cross_mount_error.code(), "EXDEV"); } + +#[test] +fn mount_table_rejects_symlinks_that_cross_mount_boundaries() { + let mut root = MemoryFileSystem::new(); + root.write_file("/root.txt", b"root".to_vec()) + .expect("seed root file"); + + let mut mounted = MemoryFileSystem::new(); + mounted + .write_file("/inside.txt", b"inside".to_vec()) + .expect("seed mounted file"); + + let mut table = MountTable::new(root); + table + .mount("/mounted", mounted, MountOptions::new("memory")) + .expect("mount memory filesystem"); + + let error = table + .symlink("../root.txt", "/mounted/root-link") + .expect_err("cross-mount symlink should fail"); + assert_eq!(error.code(), "EXDEV"); +} + +#[test] +fn mount_table_rejects_hardlinks_that_cross_mount_boundaries() { + let mut root = MemoryFileSystem::new(); + root.write_file("/root.txt", b"root".to_vec()) + .expect("seed root file"); + + let mut mounted = MemoryFileSystem::new(); + mounted + .write_file("/inside.txt", b"inside".to_vec()) + .expect("seed mounted file"); + + let mut table = MountTable::new(root); + table + .mount("/mounted", mounted, MountOptions::new("memory")) + .expect("mount memory filesystem"); + + let error = table + .link("/root.txt", "/mounted/root-link") + .expect_err("cross-mount hardlink should fail"); + assert_eq!(error.code(), "EXDEV"); +} diff --git a/crates/kernel/tests/permissions.rs b/crates/kernel/tests/permissions.rs index c2ecaba39..5717b99cb 100644 --- a/crates/kernel/tests/permissions.rs +++ b/crates/kernel/tests/permissions.rs @@ -1,5 +1,6 @@ use agent_os_kernel::command_registry::CommandDriver; use agent_os_kernel::kernel::{KernelVm, KernelVmConfig, SpawnOptions}; +use agent_os_kernel::mount_table::{MountOptions, MountTable}; use agent_os_kernel::permissions::{ filter_env, EnvAccessRequest, FsAccessRequest, PermissionDecision, PermissionedFileSystem, Permissions, @@ -63,7 +64,10 @@ fn permission_wrapped_filesystem_denies_access_by_default() { assert_fs_access_denied(filesystem.read_file("/existing.txt")); assert_fs_access_denied(filesystem.write_file("/new.txt", b"hello".to_vec())); assert_fs_access_denied(filesystem.stat("/existing.txt")); - assert_fs_access_denied(filesystem.exists("/existing.txt")); + assert!( + !PermissionedFileSystem::exists(&filesystem, "/existing.txt") + .expect("permissioned exists should fail closed") + ); assert_fs_access_denied(filesystem.mkdir("/created-dir", false)); assert_fs_access_denied(filesystem.read_dir("/")); assert_fs_access_denied(filesystem.remove_file("/existing.txt")); @@ -104,6 +108,107 @@ fn permission_wrapped_filesystem_allows_access_with_explicit_allow_all_callback( assert!(!filesystem.inner().exists("/existing.txt")); } +#[test] +fn permission_wrapped_filesystem_resolves_symlinks_before_permission_checks() { + let mut inner = MemoryFileSystem::new(); + inner.mkdir("/allowed", true).expect("seed allowed dir"); + inner.mkdir("/private", true).expect("seed private dir"); + inner + .write_file("/private/secret.txt", b"secret".to_vec()) + .expect("seed secret file"); + inner + .symlink("/private/secret.txt", "/allowed/alias.txt") + .expect("seed symlink"); + + let checked_paths = Arc::new(Mutex::new(Vec::new())); + let checked_paths_for_permission = Arc::clone(&checked_paths); + let permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_paths_for_permission + .lock() + .expect("permission path lock poisoned") + .push(request.path.clone()); + if request.path.starts_with("/allowed") { + PermissionDecision::allow() + } else { + PermissionDecision::deny("allowed-only") + } + })), + ..Permissions::default() + }; + + let mut filesystem = PermissionedFileSystem::new(inner, "vm-permissions", permissions); + + let error = filesystem + .read_file("/allowed/alias.txt") + .expect_err("symlink read should use resolved target path"); + assert_eq!(error.code(), "EACCES"); + assert_eq!( + checked_paths + .lock() + .expect("permission path lock poisoned") + .as_slice(), + [String::from("/private/secret.txt")].as_slice() + ); +} + +#[test] +fn permission_wrapped_filesystem_link_checks_source_and_destination_permissions() { + let mut inner = MemoryFileSystem::new(); + inner.mkdir("/allowed", true).expect("seed allowed dir"); + inner.mkdir("/private", true).expect("seed private dir"); + inner + .write_file("/private/source.txt", b"source".to_vec()) + .expect("seed source file"); + + let checked_paths = Arc::new(Mutex::new(Vec::new())); + let checked_paths_for_permission = Arc::clone(&checked_paths); + let permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_paths_for_permission + .lock() + .expect("permission path lock poisoned") + .push(request.path.clone()); + PermissionDecision::allow() + })), + ..Permissions::default() + }; + + let mut filesystem = PermissionedFileSystem::new(inner, "vm-permissions", permissions); + filesystem + .link("/private/source.txt", "/allowed/linked.txt") + .expect("hardlink should succeed"); + + assert_eq!( + checked_paths + .lock() + .expect("permission path lock poisoned") + .as_slice(), + [ + String::from("/private/source.txt"), + String::from("/allowed/linked.txt"), + ] + .as_slice() + ); +} + +#[test] +fn permission_wrapped_filesystem_exists_fails_closed_on_permission_denied() { + let permissions = Permissions { + filesystem: Some(Arc::new(|_: &FsAccessRequest| { + PermissionDecision::deny("hidden") + })), + ..Permissions::default() + }; + let filesystem = wrap_filesystem(permissions); + + assert!( + !PermissionedFileSystem::exists(&filesystem, "/existing.txt") + .expect("permissioned exists should fail closed") + ); + assert!(!VirtualFileSystem::exists(&filesystem, "/existing.txt")); +} + #[test] fn filter_env_only_keeps_allowed_keys() { let permissions = Permissions { @@ -152,6 +257,16 @@ fn child_process_permissions_block_spawn() { assert!(error.to_string().contains("blocked by policy")); } +#[test] +fn kernel_vm_config_defaults_to_deny_all_permissions() { + let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-defaults")); + + let error = kernel + .write_file("/tmp/denied.txt", b"nope".to_vec()) + .expect_err("default config should deny filesystem writes"); + assert_eq!(error.code(), "EACCES"); +} + #[test] fn kernel_default_spawn_cwd_matches_home_user() { let captured_cwd = Arc::new(Mutex::new(None)); @@ -191,7 +306,9 @@ fn kernel_default_spawn_cwd_matches_home_user() { #[test] fn driver_pid_ownership_is_enforced_across_kernel_operations() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-auth")); + let mut config = KernelVmConfig::new("vm-auth"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("alpha", ["alpha-cmd"])) .expect("register alpha"); @@ -236,3 +353,193 @@ fn driver_pid_ownership_is_enforced_across_kernel_operations() { kernel.wait_and_reap(alpha.pid()).expect("reap alpha"); kernel.wait_and_reap(beta.pid()).expect("reap beta"); } + +#[test] +fn kernel_mounts_require_write_permission_on_the_mount_path() { + let checked = Arc::new(Mutex::new(Vec::new())); + let checked_for_permission = Arc::clone(&checked); + let mut config = KernelVmConfig::new("vm-mount-permissions"); + config.permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_for_permission + .lock() + .expect("checked mount paths lock poisoned") + .push((request.op, request.path.clone())); + PermissionDecision::deny("mounts disabled") + })), + ..Permissions::default() + }; + + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + let error = kernel + .mount_filesystem( + "/workspace", + MemoryFileSystem::new(), + MountOptions::new("memory"), + ) + .expect_err("mount should be denied"); + assert_eq!(error.code(), "EACCES"); + assert!(error.to_string().contains("mounts disabled")); + assert_eq!( + checked + .lock() + .expect("checked mount paths lock poisoned") + .as_slice(), + [( + agent_os_kernel::permissions::FsOperation::Write, + String::from("/workspace") + )] + .as_slice() + ); +} + +#[test] +fn kernel_sensitive_mounts_require_explicit_sensitive_permission() { + let checked = Arc::new(Mutex::new(Vec::new())); + let checked_for_permission = Arc::clone(&checked); + let mut config = KernelVmConfig::new("vm-sensitive-mounts"); + config.permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_for_permission + .lock() + .expect("checked mount paths lock poisoned") + .push((request.op, request.path.clone())); + match request.op { + agent_os_kernel::permissions::FsOperation::Write => PermissionDecision::allow(), + agent_os_kernel::permissions::FsOperation::MountSensitive => { + PermissionDecision::deny("sensitive mounts require elevation") + } + other => panic!("unexpected filesystem permission probe: {other:?}"), + } + })), + ..Permissions::default() + }; + + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + let error = kernel + .mount_filesystem("/etc", MemoryFileSystem::new(), MountOptions::new("memory")) + .expect_err("sensitive mount should be denied"); + assert_eq!(error.code(), "EACCES"); + assert!(error + .to_string() + .contains("sensitive mounts require elevation")); + assert_eq!( + checked + .lock() + .expect("checked mount paths lock poisoned") + .as_slice(), + [ + ( + agent_os_kernel::permissions::FsOperation::Write, + String::from("/etc"), + ), + ( + agent_os_kernel::permissions::FsOperation::MountSensitive, + String::from("/etc"), + ), + ] + .as_slice() + ); +} + +#[test] +fn kernel_unmounts_require_write_permission_on_the_mount_path() { + let checked = Arc::new(Mutex::new(Vec::new())); + let checked_for_permission = Arc::clone(&checked); + let mut config = KernelVmConfig::new("vm-unmount-permissions"); + config.permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_for_permission + .lock() + .expect("checked unmount paths lock poisoned") + .push((request.op, request.path.clone())); + PermissionDecision::deny("unmounts disabled") + })), + ..Permissions::default() + }; + + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + kernel + .filesystem_mut() + .inner_mut() + .inner_mut() + .mount( + "/workspace", + MemoryFileSystem::new(), + MountOptions::new("memory"), + ) + .expect("seed mount"); + + let error = kernel + .unmount_filesystem("/workspace") + .expect_err("unmount should be denied"); + assert_eq!(error.code(), "EACCES"); + assert!(error.to_string().contains("unmounts disabled")); + assert_eq!( + checked + .lock() + .expect("checked unmount paths lock poisoned") + .as_slice(), + [( + agent_os_kernel::permissions::FsOperation::Write, + String::from("/workspace") + )] + .as_slice() + ); +} + +#[test] +fn kernel_sensitive_unmounts_require_explicit_sensitive_permission() { + let checked = Arc::new(Mutex::new(Vec::new())); + let checked_for_permission = Arc::clone(&checked); + let mut config = KernelVmConfig::new("vm-sensitive-unmounts"); + config.permissions = Permissions { + filesystem: Some(Arc::new(move |request: &FsAccessRequest| { + checked_for_permission + .lock() + .expect("checked sensitive unmount paths lock poisoned") + .push((request.op, request.path.clone())); + match request.op { + agent_os_kernel::permissions::FsOperation::Write => PermissionDecision::allow(), + agent_os_kernel::permissions::FsOperation::MountSensitive => { + PermissionDecision::deny("sensitive mounts require elevation") + } + other => panic!("unexpected filesystem permission probe: {other:?}"), + } + })), + ..Permissions::default() + }; + + let mut kernel = KernelVm::new(MountTable::new(MemoryFileSystem::new()), config); + kernel + .filesystem_mut() + .inner_mut() + .inner_mut() + .mount("/etc", MemoryFileSystem::new(), MountOptions::new("memory")) + .expect("seed sensitive mount"); + + let error = kernel + .unmount_filesystem("/etc") + .expect_err("sensitive unmount should be denied"); + assert_eq!(error.code(), "EACCES"); + assert!(error + .to_string() + .contains("sensitive mounts require elevation")); + assert_eq!( + checked + .lock() + .expect("checked sensitive unmount paths lock poisoned") + .as_slice(), + [ + ( + agent_os_kernel::permissions::FsOperation::Write, + String::from("/etc"), + ), + ( + agent_os_kernel::permissions::FsOperation::MountSensitive, + String::from("/etc"), + ), + ] + .as_slice() + ); +} diff --git a/crates/kernel/tests/pipe_manager.rs b/crates/kernel/tests/pipe_manager.rs index bceea0920..186099ab1 100644 --- a/crates/kernel/tests/pipe_manager.rs +++ b/crates/kernel/tests/pipe_manager.rs @@ -1,5 +1,7 @@ use agent_os_kernel::fd_table::{FdResult, FdTableManager, FILETYPE_PIPE}; -use agent_os_kernel::pipe_manager::{PipeManager, PipeResult, MAX_PIPE_BUFFER_BYTES}; +use agent_os_kernel::pipe_manager::{ + PipeManager, PipeResult, MAX_PIPE_BUFFER_BYTES, PIPE_BUF_BYTES, +}; use std::fmt::Debug; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; @@ -135,6 +137,57 @@ fn buffer_limit_is_enforced_until_the_reader_drains_the_pipe() { .expect("write after draining"); } +#[test] +fn blocking_small_writes_wait_for_full_pipe_buf_capacity() { + let manager = PipeManager::new(); + let pipe = manager.create_pipe(); + let read_id = pipe.read.description.id(); + let write_id = pipe.write.description.id(); + let writer = manager.clone(); + + manager + .write( + write_id, + vec![0; MAX_PIPE_BUFFER_BYTES - (PIPE_BUF_BYTES - 1)], + ) + .expect("fill pipe to one byte below PIPE_BUF headroom"); + + let handle = thread::spawn(move || { + writer + .write_blocking(write_id, vec![1; PIPE_BUF_BYTES]) + .expect("small blocking write should eventually succeed") + }); + + thread::sleep(Duration::from_millis(25)); + assert!( + !handle.is_finished(), + "PIPE_BUF-sized write should wait until the full chunk fits" + ); + + let first = manager + .read(read_id, 1) + .expect("drain one byte") + .expect("byte should be present"); + assert_eq!(first, vec![0]); + + assert_eq!( + handle.join().expect("writer thread should finish"), + PIPE_BUF_BYTES + ); + + let drained = manager + .read(read_id, MAX_PIPE_BUFFER_BYTES) + .expect("drain remainder") + .expect("remainder should be present"); + assert_eq!(drained.len(), MAX_PIPE_BUFFER_BYTES); + assert!(drained[..drained.len() - PIPE_BUF_BYTES] + .iter() + .all(|byte| *byte == 0)); + assert!(drained[drained.len() - PIPE_BUF_BYTES..] + .iter() + .all(|byte| *byte == 1)); +} + #[test] fn waiting_reader_receives_large_writes_without_hitting_the_buffer_limit() { let manager = PipeManager::new(); diff --git a/crates/kernel/tests/poll.rs b/crates/kernel/tests/poll.rs new file mode 100644 index 000000000..f507345a4 --- /dev/null +++ b/crates/kernel/tests/poll.rs @@ -0,0 +1,128 @@ +use agent_os_kernel::command_registry::CommandDriver; +use agent_os_kernel::kernel::{KernelVm, KernelVmConfig, SpawnOptions}; +use agent_os_kernel::permissions::Permissions; +use agent_os_kernel::poll::{PollFd, POLLERR, POLLHUP, POLLIN, POLLOUT}; +use agent_os_kernel::vfs::MemoryFileSystem; +use std::time::{Duration, Instant}; + +fn kernel_vm(vm_id: &str) -> KernelVm { + let mut config = KernelVmConfig::new(vm_id); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell driver"); + kernel +} + +fn spawn_shell(kernel: &mut KernelVm) -> u32 { + kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn shell") + .pid() +} + +#[test] +fn poll_reports_pipe_readiness_and_hangup() { + let mut kernel = kernel_vm("vm-poll-pipe"); + let pid = spawn_shell(&mut kernel); + let (read_fd, write_fd) = kernel.open_pipe("shell", pid).expect("open pipe"); + + let initial = kernel + .poll_fds( + "shell", + pid, + vec![PollFd::new(read_fd, POLLIN), PollFd::new(write_fd, POLLOUT)], + 0, + ) + .expect("poll initial pipe state"); + assert_eq!(initial.ready_count, 1); + assert_eq!(initial.fds[0].revents.bits(), 0); + assert_eq!(initial.fds[1].revents, POLLOUT); + + kernel + .fd_write("shell", pid, write_fd, b"hello") + .expect("write pipe payload"); + kernel + .fd_close("shell", pid, write_fd) + .expect("close pipe writer"); + + let ready = kernel + .poll_fds("shell", pid, vec![PollFd::new(read_fd, POLLIN)], 0) + .expect("poll readable pipe"); + assert_eq!(ready.ready_count, 1); + assert!(ready.fds[0].revents.contains(POLLIN)); + assert!(ready.fds[0].revents.contains(POLLHUP)); +} + +#[test] +fn poll_reports_pipe_peer_close_as_pollerr_on_writer() { + let mut kernel = kernel_vm("vm-poll-pipe-err"); + let pid = spawn_shell(&mut kernel); + let (read_fd, write_fd) = kernel.open_pipe("shell", pid).expect("open pipe"); + + kernel + .fd_close("shell", pid, read_fd) + .expect("close pipe reader"); + + let ready = kernel + .poll_fds("shell", pid, vec![PollFd::new(write_fd, POLLOUT)], 0) + .expect("poll closed writer peer"); + assert_eq!(ready.ready_count, 1); + assert!(ready.fds[0].revents.contains(POLLERR)); + assert!(!ready.fds[0].revents.contains(POLLOUT)); +} + +#[test] +fn poll_supports_mixed_fd_sets_and_infinite_timeout_when_ready() { + let mut kernel = kernel_vm("vm-poll-mixed"); + let pid = spawn_shell(&mut kernel); + let (pipe_read_fd, _pipe_write_fd) = kernel.open_pipe("shell", pid).expect("open pipe"); + let (master_fd, slave_fd, _path) = kernel.open_pty("shell", pid).expect("open pty"); + + kernel + .fd_write("shell", pid, slave_fd, b"tty-ready") + .expect("write pty output"); + + let ready = kernel + .poll_fds( + "shell", + pid, + vec![ + PollFd::new(pipe_read_fd, POLLIN), + PollFd::new(master_fd, POLLIN), + ], + -1, + ) + .expect("poll mixed fd set"); + assert_eq!(ready.ready_count, 1); + assert_eq!(ready.fds[0].revents.bits(), 0); + assert_eq!(ready.fds[1].revents, POLLIN); +} + +#[test] +fn poll_respects_finite_timeouts() { + let mut kernel = kernel_vm("vm-poll-timeout"); + let pid = spawn_shell(&mut kernel); + let (read_fd, _write_fd) = kernel.open_pipe("shell", pid).expect("open pipe"); + + let start = Instant::now(); + let ready = kernel + .poll_fds("shell", pid, vec![PollFd::new(read_fd, POLLIN)], 30) + .expect("poll timeout"); + let elapsed = start.elapsed(); + + assert_eq!(ready.ready_count, 0); + assert_eq!(ready.fds[0].revents.bits(), 0); + assert!( + elapsed >= Duration::from_millis(20), + "expected poll to wait, observed {elapsed:?}" + ); +} diff --git a/crates/kernel/tests/process_table.rs b/crates/kernel/tests/process_table.rs index 6d8f850d9..e48bd6340 100644 --- a/crates/kernel/tests/process_table.rs +++ b/crates/kernel/tests/process_table.rs @@ -1,5 +1,6 @@ use agent_os_kernel::process_table::{ DriverProcess, ProcessContext, ProcessExitCallback, ProcessResult, ProcessStatus, ProcessTable, + ProcessWaitEvent, WaitPidFlags, SIGCHLD, SIGCONT, SIGHUP, SIGSTOP, SIGTSTP, }; use std::collections::BTreeMap; use std::fmt::Debug; @@ -79,7 +80,7 @@ impl DriverProcess for MockDriverProcess { let should_exit = { let mut state = self.state.lock().expect("mock process lock poisoned"); state.kills.push(signal); - signal == 9 || !state.ignore_sigterm + signal == 9 || (signal == 15 && !state.ignore_sigterm) }; if should_exit { @@ -185,7 +186,10 @@ fn waitpid_resolves_for_exiting_and_already_exited_processes() { (pid, 0) ); assert_eq!(table.zombie_timer_count(), 0); - assert!(table.get(pid).is_none(), "waitpid should reap exited processes"); + assert!( + table.get(pid).is_none(), + "waitpid should reap exited processes" + ); let exited_pid = table.allocate_pid(); table.register( @@ -211,6 +215,70 @@ fn waitpid_resolves_for_exiting_and_already_exited_processes() { ); } +#[test] +fn waitpid_for_supports_wnohang_and_waiting_for_any_child() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let child_a = MockDriverProcess::new(); + let child_b = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let child_a_pid = table.allocate_pid(); + let child_b_pid = table.allocate_pid(); + + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent, + ); + table.register( + child_a_pid, + "wasmvm", + "child-a", + Vec::new(), + create_context(parent_pid), + child_a, + ); + table.register( + child_b_pid, + "wasmvm", + "child-b", + Vec::new(), + create_context(parent_pid), + child_b.clone(), + ); + + assert_eq!( + table + .waitpid_for(parent_pid, -1, WaitPidFlags::WNOHANG) + .expect("wnohang wait should succeed"), + None + ); + + child_b.exit(27); + assert_eq!( + table + .waitpid_for(parent_pid, -1, WaitPidFlags::empty()) + .expect("wait for any child should succeed"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: child_b_pid, + status: 27, + event: ProcessWaitEvent::Exited, + }) + ); + assert!( + table.get(child_b_pid).is_none(), + "waited child should be reaped" + ); + assert!( + table.get(child_a_pid).is_some(), + "other matching children should remain" + ); +} + #[test] fn on_process_exit_runs_before_waitpid_waiters_are_notified() { let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); @@ -274,6 +342,85 @@ fn on_process_exit_runs_before_waitpid_waiters_are_notified() { waiter.join().expect("waiter thread should finish"); } +#[test] +fn waitpid_for_reports_stopped_and_continued_children_once() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child, + ); + + table.mark_stopped(child_pid, SIGSTOP); + assert_eq!( + table + .waitpid_for(parent_pid, child_pid as i32, WaitPidFlags::WNOHANG) + .expect("stopped child lookup should succeed"), + None + ); + assert_eq!( + table + .waitpid_for( + parent_pid, + child_pid as i32, + WaitPidFlags::WNOHANG | WaitPidFlags::WUNTRACED, + ) + .expect("wuntraced wait should succeed"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: child_pid, + status: SIGSTOP, + event: ProcessWaitEvent::Stopped, + }) + ); + assert_eq!( + table + .get(child_pid) + .expect("child remains registered") + .status, + ProcessStatus::Stopped + ); + + table.mark_continued(child_pid); + assert_eq!( + table + .waitpid_for( + parent_pid, + child_pid as i32, + WaitPidFlags::WNOHANG | WaitPidFlags::WCONTINUED, + ) + .expect("wcontinued wait should succeed"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: child_pid, + status: SIGCONT, + event: ProcessWaitEvent::Continued, + }) + ); + assert_eq!( + table + .get(child_pid) + .expect("child remains registered") + .status, + ProcessStatus::Running + ); + assert_eq!(parent.kills(), vec![SIGCHLD, SIGCHLD]); +} + #[test] fn kill_routes_signals_and_validates_process_existence() { let table = ProcessTable::new(); @@ -303,6 +450,161 @@ fn kill_routes_signals_and_validates_process_existence() { assert_error_code(table.kill(pid as i32, 100), "EINVAL"); } +#[test] +fn kill_updates_job_control_state_for_stop_and_continue_signals() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child.clone(), + ); + + table + .kill(child_pid as i32, SIGTSTP) + .expect("SIGTSTP should stop the child"); + assert_eq!(child.kills(), vec![SIGTSTP]); + assert_eq!( + table + .get(child_pid) + .expect("child remains registered") + .status, + ProcessStatus::Stopped + ); + assert_eq!( + table + .waitpid_for( + parent_pid, + child_pid as i32, + WaitPidFlags::WNOHANG | WaitPidFlags::WUNTRACED, + ) + .expect("stopped child wait should succeed"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: child_pid, + status: SIGTSTP, + event: ProcessWaitEvent::Stopped, + }) + ); + + table + .kill(child_pid as i32, SIGCONT) + .expect("SIGCONT should continue the child"); + assert_eq!(child.kills(), vec![SIGTSTP, SIGCONT]); + assert_eq!( + table + .get(child_pid) + .expect("child remains registered") + .status, + ProcessStatus::Running + ); + assert_eq!( + table + .waitpid_for( + parent_pid, + child_pid as i32, + WaitPidFlags::WNOHANG | WaitPidFlags::WCONTINUED, + ) + .expect("continued child wait should succeed"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: child_pid, + status: SIGCONT, + event: ProcessWaitEvent::Continued, + }) + ); + assert_eq!(parent.kills(), vec![SIGCHLD, SIGCHLD]); +} + +#[test] +fn exiting_child_delivers_sigchld_to_living_parent() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child.clone(), + ); + + child.exit(0); + + wait_for( + || parent.kills() == vec![SIGCHLD], + Duration::from_millis(100), + ); + assert_eq!( + table.waitpid(child_pid).expect("reap child"), + (child_pid, 0) + ); +} + +#[test] +fn killed_child_delivers_sigchld_to_living_parent() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child.clone(), + ); + + table + .kill(child_pid as i32, 15) + .expect("deliver SIGTERM to child"); + + wait_for( + || parent.kills() == vec![SIGCHLD], + Duration::from_millis(100), + ); + assert_eq!( + table.waitpid(child_pid).expect("reap killed child"), + (child_pid, 143) + ); +} + #[test] fn process_groups_and_sessions_follow_legacy_rules() { let table = ProcessTable::new(); @@ -394,6 +696,184 @@ fn negative_pid_kill_targets_entire_process_groups() { assert_eq!(peer.kills(), vec![15]); } +#[test] +fn negative_pid_kill_reaches_stopped_and_exited_group_members() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let init = MockDriverProcess::new(); + let parent = MockDriverProcess::new(); + let leader = MockDriverProcess::stubborn(); + let stopped = MockDriverProcess::stubborn(); + let zombie = MockDriverProcess::stubborn(); + let init_pid = table.allocate_pid(); + let parent_pid = table.allocate_pid(); + let leader_pid = table.allocate_pid(); + let stopped_pid = table.allocate_pid(); + let zombie_pid = table.allocate_pid(); + + table.register( + init_pid, + "wasmvm", + "init", + Vec::new(), + create_context(0), + init, + ); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(init_pid), + parent, + ); + table.register( + leader_pid, + "wasmvm", + "leader", + Vec::new(), + create_context(parent_pid), + leader.clone(), + ); + table.register( + stopped_pid, + "wasmvm", + "stopped", + Vec::new(), + create_context(parent_pid), + stopped.clone(), + ); + table.register( + zombie_pid, + "wasmvm", + "zombie", + Vec::new(), + create_context(parent_pid), + zombie.clone(), + ); + table + .setpgid(leader_pid, 0) + .expect("leader becomes process-group leader"); + table + .setpgid(stopped_pid, leader_pid) + .expect("stopped peer joins leader group"); + table + .setpgid(zombie_pid, leader_pid) + .expect("zombie peer joins leader group"); + table.mark_stopped(stopped_pid, SIGSTOP); + zombie.exit(23); + + table + .kill(-(leader_pid as i32), 15) + .expect("group kill should include stopped and zombie members"); + + assert_eq!(leader.kills(), vec![15]); + assert_eq!(stopped.kills(), vec![15]); + assert_eq!(zombie.kills(), vec![15]); +} + +#[test] +fn exiting_parent_reparents_children_to_pid_one_when_available() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let init = MockDriverProcess::new(); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + let init_pid = table.allocate_pid(); + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + + table.register( + init_pid, + "wasmvm", + "init", + Vec::new(), + create_context(0), + init, + ); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(init_pid), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child, + ); + + parent.exit(0); + + assert_eq!( + table + .getppid(child_pid) + .expect("child should be reparented"), + 1 + ); +} + +#[test] +fn orphaned_stopped_process_groups_receive_sighup_and_sigcont() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let init = MockDriverProcess::new(); + let parent = MockDriverProcess::new(); + let leader = MockDriverProcess::new(); + let stopped = MockDriverProcess::new(); + let init_pid = table.allocate_pid(); + let parent_pid = table.allocate_pid(); + let leader_pid = table.allocate_pid(); + let stopped_pid = table.allocate_pid(); + + table.register( + init_pid, + "wasmvm", + "init", + Vec::new(), + create_context(0), + init, + ); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(init_pid), + parent.clone(), + ); + table.register( + leader_pid, + "wasmvm", + "leader", + Vec::new(), + create_context(parent_pid), + leader.clone(), + ); + table.register( + stopped_pid, + "wasmvm", + "stopped", + Vec::new(), + create_context(parent_pid), + stopped.clone(), + ); + table + .setpgid(leader_pid, 0) + .expect("leader becomes process-group leader"); + table + .setpgid(stopped_pid, leader_pid) + .expect("stopped peer joins leader group"); + table.mark_stopped(stopped_pid, SIGSTOP); + + parent.exit(0); + + assert_eq!(leader.kills(), vec![SIGHUP, SIGCONT]); + assert_eq!(stopped.kills(), vec![SIGHUP, SIGCONT]); +} + #[test] fn terminate_all_escalates_from_sigterm_to_sigkill_for_survivors() { let table = ProcessTable::new(); @@ -475,6 +955,80 @@ fn waitpid_rejects_unknown_processes() { assert_error_code(table.waitpid(9999), "ESRCH"); } +#[test] +fn waitpid_for_supports_pid_zero_and_negative_process_group_selectors() { + let table = ProcessTable::with_zombie_ttl(Duration::from_secs(3600)); + let parent = MockDriverProcess::new(); + let same_group_child = MockDriverProcess::new(); + let other_group_child = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let same_group_child_pid = table.allocate_pid(); + let other_group_child_pid = table.allocate_pid(); + + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent, + ); + table.register( + same_group_child_pid, + "wasmvm", + "same-group", + Vec::new(), + create_context(parent_pid), + same_group_child.clone(), + ); + table.register( + other_group_child_pid, + "wasmvm", + "other-group", + Vec::new(), + create_context(parent_pid), + other_group_child.clone(), + ); + table + .setpgid(other_group_child_pid, 0) + .expect("child should become group leader"); + + other_group_child.exit(13); + assert_eq!( + table + .waitpid_for(parent_pid, 0, WaitPidFlags::WNOHANG) + .expect("pid=0 wait should succeed"), + None + ); + + same_group_child.exit(11); + assert_eq!( + table + .waitpid_for(parent_pid, 0, WaitPidFlags::empty()) + .expect("pid=0 wait should reap same-group child"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: same_group_child_pid, + status: 11, + event: ProcessWaitEvent::Exited, + }) + ); + assert_eq!( + table + .waitpid_for( + parent_pid, + -(other_group_child_pid as i32), + WaitPidFlags::empty(), + ) + .expect("negative pgid wait should reap matching child"), + Some(agent_os_kernel::process_table::ProcessWaitResult { + pid: other_group_child_pid, + status: 13, + event: ProcessWaitEvent::Exited, + }) + ); +} + #[test] fn zombie_reaper_uses_a_single_worker_for_many_exits() { let table = ProcessTable::with_zombie_ttl(Duration::from_millis(100)); @@ -504,3 +1058,74 @@ fn zombie_reaper_uses_a_single_worker_for_many_exits() { assert!(table.get(pid).is_none(), "process {pid} should be reaped"); } } + +#[test] +fn zombie_reaper_preserves_child_exit_code_while_parent_is_alive() { + let table = ProcessTable::with_zombie_ttl(Duration::from_millis(50)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent, + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child.clone(), + ); + + child.exit(41); + thread::sleep(Duration::from_millis(200)); + + assert_eq!( + table + .waitpid(child_pid) + .expect("child exit code should be preserved"), + (child_pid, 41) + ); +} + +#[test] +fn zombie_reaper_reaps_exited_children_after_their_parent_exits() { + let table = ProcessTable::with_zombie_ttl(Duration::from_millis(50)); + let parent = MockDriverProcess::new(); + let child = MockDriverProcess::new(); + + let parent_pid = table.allocate_pid(); + let child_pid = table.allocate_pid(); + table.register( + parent_pid, + "wasmvm", + "parent", + Vec::new(), + create_context(0), + parent.clone(), + ); + table.register( + child_pid, + "wasmvm", + "child", + Vec::new(), + create_context(parent_pid), + child.clone(), + ); + + child.exit(17); + thread::sleep(Duration::from_millis(120)); + parent.exit(0); + + wait_for( + || table.get(parent_pid).is_none() && table.get(child_pid).is_none(), + Duration::from_secs(1), + ); +} diff --git a/crates/kernel/tests/resource_accounting.rs b/crates/kernel/tests/resource_accounting.rs index 67a75a7c9..12d2d975e 100644 --- a/crates/kernel/tests/resource_accounting.rs +++ b/crates/kernel/tests/resource_accounting.rs @@ -1,12 +1,17 @@ use agent_os_kernel::command_registry::CommandDriver; use agent_os_kernel::kernel::{KernelVm, KernelVmConfig, SpawnOptions}; +use agent_os_kernel::permissions::Permissions; use agent_os_kernel::pty::LineDisciplineConfig; use agent_os_kernel::resource_accounting::ResourceLimits; -use agent_os_kernel::vfs::MemoryFileSystem; +use agent_os_kernel::vfs::{MemoryFileSystem, VirtualFileSystem}; +use std::collections::BTreeMap; +use std::time::{Duration, Instant}; #[test] fn resource_snapshot_counts_processes_fds_pipes_and_ptys() { - let mut kernel = KernelVm::new(MemoryFileSystem::new(), KernelVmConfig::new("vm-resources")); + let mut config = KernelVmConfig::new("vm-resources"); + config.permissions = Permissions::allow_all(); + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); kernel .register_driver(CommandDriver::new("shell", ["sh"])) .expect("register shell"); @@ -66,11 +71,13 @@ fn resource_snapshot_counts_processes_fds_pipes_and_ptys() { #[test] fn resource_limits_reject_extra_processes_pipes_and_ptys() { let mut config = KernelVmConfig::new("vm-limits"); + config.permissions = Permissions::allow_all(); config.resources = ResourceLimits { max_processes: Some(1), max_open_fds: Some(6), max_pipes: Some(1), max_ptys: Some(1), + ..ResourceLimits::default() }; let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); @@ -117,3 +124,328 @@ fn resource_limits_reject_extra_processes_pipes_and_ptys() { process.finish(0); kernel.wait_and_reap(process.pid()).expect("reap process"); } + +#[test] +fn zombie_processes_count_against_process_limits_until_reaped() { + let mut config = KernelVmConfig::new("vm-zombie-process-limit"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_processes: Some(1), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn initial process"); + process.finish(0); + + let error = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect_err("zombie should still count against process limit"); + assert_eq!(error.code(), "EAGAIN"); + + kernel.wait_and_reap(process.pid()).expect("reap zombie"); + kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn should succeed after zombie is reaped"); +} + +#[test] +fn filesystem_limits_reject_inode_growth_and_file_expansion() { + let mut config = KernelVmConfig::new("vm-filesystem-limits"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_filesystem_bytes: Some(5), + max_inode_count: Some(4), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .write_file("/tmp/a.txt", b"hello".to_vec()) + .expect("seed file within byte limit"); + kernel + .create_dir("/tmp/dir") + .expect("create directory within inode limit"); + + let write_error = kernel + .write_file("/tmp/b.txt", b"!".to_vec()) + .expect_err("additional file should exceed inode limit"); + assert_eq!(write_error.code(), "ENOSPC"); + + let truncate_error = kernel + .truncate("/tmp/a.txt", 6) + .expect_err("truncate should exceed filesystem byte limit"); + assert_eq!(truncate_error.code(), "ENOSPC"); + assert_eq!( + kernel + .read_file("/tmp/a.txt") + .expect("file should stay unchanged"), + b"hello".to_vec() + ); +} + +#[test] +fn filesystem_limits_reject_fd_pwrite_before_resizing_file() { + let mut config = KernelVmConfig::new("vm-fd-pwrite-limit"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_filesystem_bytes: Some(16), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .filesystem_mut() + .write_file("/tmp/data.txt", b"abc".to_vec()) + .expect("seed file"); + + let process = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn shell"); + let fd = kernel + .fd_open("shell", process.pid(), "/tmp/data.txt", 0, None) + .expect("open file"); + + let error = kernel + .fd_pwrite("shell", process.pid(), fd, b"z", 16) + .expect_err("pwrite should exceed filesystem byte limit"); + assert_eq!(error.code(), "ENOSPC"); + assert_eq!( + kernel + .read_file("/tmp/data.txt") + .expect("file should stay unchanged"), + b"abc".to_vec() + ); + + process.finish(0); + kernel.wait_and_reap(process.pid()).expect("reap shell"); +} + +#[test] +fn blocking_pipe_and_pty_reads_time_out_instead_of_hanging_forever() { + let mut config = KernelVmConfig::new("vm-read-timeouts"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_blocking_read_ms: Some(25), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let process = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn shell"); + + let (read_fd, _write_fd) = kernel.open_pipe("shell", process.pid()).expect("open pipe"); + let (master_fd, slave_fd, _) = kernel.open_pty("shell", process.pid()).expect("open pty"); + kernel + .pty_set_discipline( + "shell", + process.pid(), + master_fd, + LineDisciplineConfig { + canonical: Some(false), + echo: Some(false), + isig: Some(false), + }, + ) + .expect("set raw pty"); + + let started = Instant::now(); + let pipe_error = kernel + .fd_read("shell", process.pid(), read_fd, 16) + .expect_err("empty pipe read should time out"); + assert_eq!(pipe_error.code(), "EAGAIN"); + assert!( + started.elapsed() >= Duration::from_millis(20), + "pipe read timed out too early: {:?}", + started.elapsed() + ); + + let started = Instant::now(); + let pty_error = kernel + .fd_read("shell", process.pid(), slave_fd, 16) + .expect_err("empty PTY read should time out"); + assert_eq!(pty_error.code(), "EAGAIN"); + assert!( + started.elapsed() >= Duration::from_millis(20), + "PTY read timed out too early: {:?}", + started.elapsed() + ); + + process.finish(0); + kernel.wait_and_reap(process.pid()).expect("reap shell"); +} + +#[test] +fn resource_limits_reject_oversized_spawn_payloads() { + let mut config = KernelVmConfig::new("vm-spawn-payload-limits"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_process_argv_bytes: Some(13), + max_process_env_bytes: Some(15), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + + let argv_error = kernel + .spawn_process( + "sh", + vec![String::from("1234567890")], + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect_err("oversized argv should be rejected"); + assert_eq!(argv_error.code(), "EINVAL"); + + let env_error = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + env: BTreeMap::from([(String::from("LONG"), String::from("1234567890"))]), + ..SpawnOptions::default() + }, + ) + .expect_err("oversized environment should be rejected"); + assert_eq!(env_error.code(), "EINVAL"); +} + +#[test] +fn resource_limits_reject_oversized_pread_and_write_operations() { + let mut config = KernelVmConfig::new("vm-io-op-limits"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_pread_bytes: Some(4), + max_fd_write_bytes: Some(3), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel + .register_driver(CommandDriver::new("shell", ["sh"])) + .expect("register shell"); + kernel + .write_file("/tmp/data.txt", b"hello".to_vec()) + .expect("seed file"); + + let process = kernel + .spawn_process( + "sh", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("shell")), + ..SpawnOptions::default() + }, + ) + .expect("spawn shell"); + let fd = kernel + .fd_open("shell", process.pid(), "/tmp/data.txt", 0, None) + .expect("open file"); + + let pread_error = kernel + .fd_pread("shell", process.pid(), fd, 5, 0) + .expect_err("oversized pread should be rejected"); + assert_eq!(pread_error.code(), "EINVAL"); + + let write_error = kernel + .fd_write("shell", process.pid(), fd, b"four") + .expect_err("oversized fd_write should be rejected"); + assert_eq!(write_error.code(), "EINVAL"); + + let pwrite_error = kernel + .fd_pwrite("shell", process.pid(), fd, b"four", 0) + .expect_err("oversized fd_pwrite should be rejected"); + assert_eq!(pwrite_error.code(), "EINVAL"); + + assert_eq!( + kernel + .read_file("/tmp/data.txt") + .expect("file should remain unchanged"), + b"hello".to_vec() + ); + + process.finish(0); + kernel.wait_and_reap(process.pid()).expect("reap shell"); +} + +#[test] +fn resource_limits_reject_oversized_readdir_batches() { + let mut config = KernelVmConfig::new("vm-readdir-limit"); + config.permissions = Permissions::allow_all(); + config.resources = ResourceLimits { + max_readdir_entries: Some(2), + ..ResourceLimits::default() + }; + + let mut kernel = KernelVm::new(MemoryFileSystem::new(), config); + kernel.create_dir("/tmp").expect("create tmp"); + kernel + .write_file("/tmp/a.txt", b"a".to_vec()) + .expect("write first entry"); + kernel + .write_file("/tmp/b.txt", b"b".to_vec()) + .expect("write second entry"); + kernel + .write_file("/tmp/c.txt", b"c".to_vec()) + .expect("write third entry"); + + let error = kernel + .read_dir("/tmp") + .expect_err("oversized readdir batch should be rejected"); + assert_eq!(error.code(), "ENOMEM"); +} diff --git a/crates/kernel/tests/root_fs.rs b/crates/kernel/tests/root_fs.rs index 94b381796..9124c05b3 100644 --- a/crates/kernel/tests/root_fs.rs +++ b/crates/kernel/tests/root_fs.rs @@ -146,6 +146,79 @@ fn overlay_remove_dir_rejects_lower_only_children_in_merged_view() { assert!(overlay.exists("/tmp/nonempty/child.txt")); } +#[test] +fn overlay_remove_dir_rejects_lower_children_after_directory_copy_up() { + let mut lower = MemoryFileSystem::new(); + lower + .mkdir("/tmp/nonempty", true) + .expect("create lower directory"); + lower + .write_file("/tmp/nonempty/child.txt", b"child".to_vec()) + .expect("seed lower child"); + + let mut overlay = OverlayFileSystem::new(vec![lower], OverlayMode::Ephemeral); + overlay + .chmod("/tmp/nonempty", 0o700) + .expect("copy up lower directory"); + + assert_error_code(overlay.remove_dir("/tmp/nonempty"), "ENOTEMPTY"); + assert!(overlay.exists("/tmp/nonempty/child.txt")); +} + +#[test] +fn overlay_rename_rejects_directory_trees_that_exceed_snapshot_depth_limit() { + let mut lower = MemoryFileSystem::new(); + let mut path = String::from("/deep"); + lower.create_dir(&path).expect("create root of deep tree"); + for _ in 0..1025 { + path.push_str("/d"); + lower.create_dir(&path).expect("create nested directory"); + } + + let mut overlay = OverlayFileSystem::new(vec![lower], OverlayMode::Ephemeral); + assert_error_code(overlay.rename("/deep", "/renamed"), "EINVAL"); +} + +#[test] +fn overlay_link_and_rename_preserve_upper_hardlinks_after_copy_up() { + let mut lower = MemoryFileSystem::new(); + lower + .write_file("/src.txt", b"base".to_vec()) + .expect("seed lower file"); + + let mut overlay = OverlayFileSystem::new(vec![lower], OverlayMode::Ephemeral); + overlay + .link("/src.txt", "/alias.txt") + .expect("hardlink copied-up file"); + + overlay + .write_file("/alias.txt", b"mutated".to_vec()) + .expect("mutate linked file"); + assert_eq!( + overlay.read_file("/src.txt").expect("read linked source"), + b"mutated".to_vec() + ); + + overlay + .rename("/src.txt", "/renamed.txt") + .expect("rename hardlinked source"); + + let alias_stat = overlay.stat("/alias.txt").expect("stat alias"); + let renamed_stat = overlay.stat("/renamed.txt").expect("stat renamed"); + assert_eq!(alias_stat.ino, renamed_stat.ino); + assert_eq!(alias_stat.nlink, 2); + assert_eq!(renamed_stat.nlink, 2); + assert_eq!( + overlay.read_file("/alias.txt").expect("read alias"), + b"mutated".to_vec() + ); + assert_eq!( + overlay.read_file("/renamed.txt").expect("read renamed"), + b"mutated".to_vec() + ); + assert_error_code(overlay.read_file("/src.txt"), "ENOENT"); +} + #[test] fn root_filesystem_uses_bundled_base_and_round_trips_snapshots() { let mut root = RootFileSystem::from_descriptor(RootFilesystemDescriptor::default()) diff --git a/crates/kernel/tests/vfs.rs b/crates/kernel/tests/vfs.rs index 7ba8187a1..7bd06c210 100644 --- a/crates/kernel/tests/vfs.rs +++ b/crates/kernel/tests/vfs.rs @@ -1,5 +1,5 @@ use agent_os_kernel::vfs::{normalize_path, MemoryFileSystem, VirtualFileSystem, S_IFLNK, S_IFREG}; -use std::fmt::Debug; +use std::{fmt::Debug, thread::sleep, time::Duration}; fn assert_error_code(result: agent_os_kernel::vfs::VfsResult, expected: &str) { let error = result.expect_err("operation should fail"); @@ -273,6 +273,9 @@ fn chmod_chown_utimes_truncate_and_pread_update_metadata_and_contents() { assert_eq!(stat.atime_ms, 1_700_000_000_000); assert_eq!(stat.mtime_ms, 1_710_000_000_000); assert_eq!(stat.size, 8); + assert_eq!(stat.blocks, 1); + assert_eq!(stat.dev, 1); + assert_eq!(stat.rdev, 0); let bytes = filesystem .read_file("/meta.txt") @@ -292,6 +295,49 @@ fn chmod_chown_utimes_truncate_and_pread_update_metadata_and_contents() { .is_empty()); } +#[test] +fn directory_reads_and_metadata_updates_refresh_timestamps() { + let mut filesystem = MemoryFileSystem::new(); + filesystem + .write_file("/workspace/file.txt", "hello") + .expect("seed file"); + + let before_dir_read = filesystem.stat("/workspace").expect("stat workspace"); + sleep(Duration::from_millis(2)); + filesystem + .read_dir("/workspace") + .expect("read workspace directory"); + let after_dir_read = filesystem.stat("/workspace").expect("restat workspace"); + assert!( + after_dir_read.atime_ms > before_dir_read.atime_ms, + "directory atime should advance after read_dir" + ); + + let before_link = filesystem.stat("/workspace/file.txt").expect("stat file"); + sleep(Duration::from_millis(2)); + filesystem + .link("/workspace/file.txt", "/workspace/file-link.txt") + .expect("create hard link"); + let after_link = filesystem.stat("/workspace/file.txt").expect("restat file"); + assert!( + after_link.ctime_ms > before_link.ctime_ms, + "ctime should advance when link count changes" + ); + + let before_rename = after_link.ctime_ms; + sleep(Duration::from_millis(2)); + filesystem + .rename("/workspace/file-link.txt", "/workspace/file-renamed.txt") + .expect("rename linked path"); + let renamed = filesystem + .stat("/workspace/file-renamed.txt") + .expect("stat renamed path"); + assert!( + renamed.ctime_ms > before_rename, + "ctime should advance on rename" + ); +} + #[test] fn read_dir_with_types_reports_direct_children() { let mut filesystem = MemoryFileSystem::new(); diff --git a/crates/sidecar-browser/tests/service.rs b/crates/sidecar-browser/tests/service.rs index 5cad0f79d..a9194b3e7 100644 --- a/crates/sidecar-browser/tests/service.rs +++ b/crates/sidecar-browser/tests/service.rs @@ -7,6 +7,7 @@ use agent_os_bridge::{ StartExecutionRequest, }; use agent_os_kernel::kernel::KernelVmConfig; +use agent_os_kernel::permissions::Permissions; use agent_os_sidecar_browser::{ BrowserSidecar, BrowserSidecarConfig, BrowserWorkerBridge, BrowserWorkerEntrypoint, BrowserWorkerHandle, BrowserWorkerHandleRequest, BrowserWorkerSpawnRequest, @@ -42,9 +43,9 @@ impl BrowserWorkerBridge for RecordingBridge { fn browser_sidecar_runs_guest_javascript_from_main_thread_workers() { let mut sidecar = BrowserSidecar::new(RecordingBridge::default(), BrowserSidecarConfig::default()); - sidecar - .create_vm(KernelVmConfig::new("vm-browser")) - .expect("create vm"); + let mut config = KernelVmConfig::new("vm-browser"); + config.permissions = Permissions::allow_all(); + sidecar.create_vm(config).expect("create vm"); let context = sidecar .create_javascript_context(CreateJavascriptContextRequest { @@ -124,9 +125,9 @@ fn browser_sidecar_runs_guest_javascript_from_main_thread_workers() { fn browser_sidecar_runs_guest_wasm_from_main_thread_workers() { let mut sidecar = BrowserSidecar::new(RecordingBridge::default(), BrowserSidecarConfig::default()); - sidecar - .create_vm(KernelVmConfig::new("vm-browser")) - .expect("create vm"); + let mut config = KernelVmConfig::new("vm-browser"); + config.permissions = Permissions::allow_all(); + sidecar.create_vm(config).expect("create vm"); let context = sidecar .create_wasm_context(CreateWasmContextRequest { diff --git a/crates/sidecar/Cargo.toml b/crates/sidecar/Cargo.toml index 5d4e49052..fc7fd0847 100644 --- a/crates/sidecar/Cargo.toml +++ b/crates/sidecar/Cargo.toml @@ -18,12 +18,14 @@ aws-credential-types = "1" aws-sdk-s3 = "1" base64 = "0.22" filetime = "0.2" +hickory-resolver = "0.26.0-beta.3" jsonwebtoken = "8.3.0" nix = { version = "0.29", features = ["fs", "poll", "process", "signal", "user"] } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" tokio = { version = "1", features = ["rt-multi-thread"] } ureq = { version = "2.10", features = ["json"] } +url = "2" [dev-dependencies] wat = "1.0" diff --git a/crates/sidecar/src/google_drive_plugin.rs b/crates/sidecar/src/google_drive_plugin.rs index 78c66ea69..d3d60fb1d 100644 --- a/crates/sidecar/src/google_drive_plugin.rs +++ b/crates/sidecar/src/google_drive_plugin.rs @@ -15,6 +15,7 @@ use serde_json::json; use std::collections::{BTreeMap, BTreeSet}; use std::io::Read; use std::time::{SystemTime, UNIX_EPOCH}; +use url::Url; const DEFAULT_CHUNK_SIZE: usize = 4 * 1024 * 1024; const DEFAULT_INLINE_THRESHOLD: usize = 64 * 1024; @@ -22,6 +23,8 @@ const MANIFEST_FORMAT: &str = "agent_os_google_drive_filesystem_manifest_v1"; const DRIVE_SCOPE: &str = "https://www.googleapis.com/auth/drive.file"; const DEFAULT_TOKEN_URL: &str = "https://oauth2.googleapis.com/token"; const DEFAULT_API_BASE_URL: &str = "https://www.googleapis.com"; +const GOOGLE_TOKEN_HOSTS: &[&str] = &["oauth2.googleapis.com"]; +const GOOGLE_API_BASE_HOSTS: &[&str] = &["www.googleapis.com"]; const TOKEN_REFRESH_SKEW_SECONDS: u64 = 60; const MAX_PERSISTED_MANIFEST_FILE_BYTES: u64 = 1024 * 1024 * 1024; @@ -273,9 +276,8 @@ impl GoogleDriveObjectStore { token_url: String, api_base_url: String, ) -> Result { - let api_base_url = normalize_base_url(&api_base_url).ok_or_else(|| { - PluginError::invalid_input("google_drive mount requires a valid apiBaseUrl") - })?; + let api_base_url = + validate_google_drive_url(&api_base_url, "apiBaseUrl", GOOGLE_API_BASE_HOSTS, false)?; Ok(Self { auth: GoogleServiceAccountAuth::new(credentials, token_url)?, @@ -533,6 +535,8 @@ impl GoogleServiceAccountAuth { "google_drive mount credentials.privateKey is not valid PEM: {error}" )) })?; + let token_url = + validate_google_drive_url(&token_url, "tokenUrl", GOOGLE_TOKEN_HOSTS, true)?; Ok(Self { client_email: credentials.client_email, @@ -884,6 +888,83 @@ fn normalize_base_url(raw: &str) -> Option { } } +fn validate_google_drive_url( + raw: &str, + field_name: &str, + allowed_hosts: &[&str], + allow_path: bool, +) -> Result { + let normalized = normalize_base_url(raw).ok_or_else(|| { + PluginError::invalid_input(format!("google_drive mount requires a valid {field_name}")) + })?; + let url = Url::parse(&normalized).map_err(|error| { + PluginError::invalid_input(format!( + "google_drive mount {field_name} is not a valid URL: {error}" + )) + })?; + + if is_google_drive_test_url(&url) { + return Ok(normalized); + } + + if url.scheme() != "https" { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must use https" + ))); + } + if url.host_str().is_none() { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must include a host" + ))); + } + if url.port().is_some() { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must not override the default port" + ))); + } + if !url.username().is_empty() || url.password().is_some() { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must not include user credentials" + ))); + } + if url.query().is_some() || url.fragment().is_some() { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must not include query or fragment components" + ))); + } + if !allow_path && url.path() != "/" { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} must not include a path" + ))); + } + + let host = url.host_str().expect("host checked above"); + if !allowed_hosts.iter().any(|candidate| candidate == &host) { + return Err(PluginError::invalid_input(format!( + "google_drive mount {field_name} host must be one of: {}", + allowed_hosts.join(", ") + ))); + } + + Ok(normalized) +} + +fn is_google_drive_test_url(url: &Url) -> bool { + #[cfg(test)] + { + matches!(url.scheme(), "http" | "https") + && matches!( + url.host_str(), + Some("127.0.0.1") | Some("localhost") | Some("[::1]") + ) + } + #[cfg(not(test))] + { + let _ = url; + false + } +} + fn escape_query_literal(raw: &str) -> String { raw.replace('\'', "\\'") } @@ -1494,6 +1575,42 @@ oFnGY0OFksX/ye0/XGpy2SFxYRwGU98HPYeBvAQQrVjdkzfy7BmXQQ==\n\ } } + #[test] + fn google_drive_plugin_rejects_untrusted_token_hosts() { + let server = MockGoogleDriveServer::start(); + let mut config = test_config(&server, "reject-token-host"); + config.token_url = Some(String::from("https://evil.example/token")); + + let error = match GoogleDriveBackedFilesystem::from_config(config) { + Ok(_) => panic!("untrusted token host should be rejected"), + Err(error) => error, + }; + assert!( + error + .to_string() + .contains("google_drive mount tokenUrl host must be one of"), + "unexpected error: {error}" + ); + } + + #[test] + fn google_drive_plugin_rejects_untrusted_api_base_hosts() { + let server = MockGoogleDriveServer::start(); + let mut config = test_config(&server, "reject-api-host"); + config.api_base_url = Some(String::from("https://metadata.google.internal")); + + let error = match GoogleDriveBackedFilesystem::from_config(config) { + Ok(_) => panic!("untrusted api base host should be rejected"), + Err(error) => error, + }; + assert!( + error + .to_string() + .contains("google_drive mount apiBaseUrl host must be one of"), + "unexpected error: {error}" + ); + } + #[test] fn google_drive_plugin_persists_files_across_reopen_and_preserves_links() { let server = MockGoogleDriveServer::start(); diff --git a/crates/sidecar/src/host_dir_plugin.rs b/crates/sidecar/src/host_dir_plugin.rs index 07b5989cd..dc5dfee02 100644 --- a/crates/sidecar/src/host_dir_plugin.rs +++ b/crates/sidecar/src/host_dir_plugin.rs @@ -7,13 +7,43 @@ use agent_os_kernel::mount_table::{ use agent_os_kernel::vfs::{ normalize_path, VfsError, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat, }; -use filetime::{set_file_times, FileTime}; -use nix::unistd::{chown, Gid, Uid}; +use nix::errno::Errno; +use nix::fcntl::{openat2, readlinkat, renameat, AtFlags, OFlag, OpenHow, ResolveFlag}; +use nix::sys::stat::{ + fchmodat, fstatat, mkdirat, utimensat, FchmodatFlags, Mode, SFlag, UtimensatFlags, +}; +use nix::sys::time::{TimeSpec, TimeValLike}; +use nix::unistd::{fchownat, linkat, symlinkat, unlinkat, Gid, Uid, UnlinkatFlags}; use serde::Deserialize; use std::fs::{self, File}; -use std::io; -use std::os::unix::fs::{symlink as create_symlink, FileExt, MetadataExt, PermissionsExt}; +use std::io::{self, Read, Write}; +use std::os::fd::{AsRawFd, RawFd}; +use std::os::unix::fs::{FileExt, MetadataExt}; use std::path::{Component, Path, PathBuf}; +use std::sync::Arc; + +#[derive(Debug)] +struct AnchoredFd { + fd: RawFd, +} + +impl AnchoredFd { + fn proc_path(&self) -> PathBuf { + PathBuf::from(format!("/proc/self/fd/{}", self.fd)) + } +} + +impl AsRawFd for AnchoredFd { + fn as_raw_fd(&self) -> RawFd { + self.fd + } +} + +impl Drop for AnchoredFd { + fn drop(&mut self) { + let _ = nix::unistd::close(self.fd); + } +} #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] @@ -50,6 +80,7 @@ impl FileSystemPluginFactory for HostDirMountPlugin { #[derive(Debug, Clone)] pub(crate) struct HostDirFilesystem { host_root: PathBuf, + host_root_dir: Arc, } impl HostDirFilesystem { @@ -69,7 +100,10 @@ impl HostDirFilesystem { } Ok(Self { - host_root: canonical_root, + host_root: canonical_root.clone(), + host_root_dir: Arc::new( + File::open(&canonical_root).map_err(|error| io_error_to_vfs("open", "/", error))?, + ), }) } @@ -97,51 +131,124 @@ impl HostDirFilesystem { Ok(joined) } - fn resolve(&self, path: &str) -> VfsResult { - let joined = self.lexical_host_path(path)?; - match fs::canonicalize(&joined) { - Ok(real) => { - self.ensure_within_root(&real, path)?; - Ok(real) + fn relative_virtual_path(&self, path: &str) -> (String, PathBuf) { + let normalized = normalize_path(path); + let relative = normalized.trim_start_matches('/'); + let relative = if relative.is_empty() { + PathBuf::from(".") + } else { + PathBuf::from(relative) + }; + (normalized, relative) + } + + fn resolve_flags() -> ResolveFlag { + ResolveFlag::RESOLVE_BENEATH | ResolveFlag::RESOLVE_NO_MAGICLINKS + } + + fn open_beneath(&self, relative: &Path, flags: OFlag, mode: Mode) -> VfsResult { + let relative_display = relative.display().to_string(); + let fd = openat2( + self.host_root_dir.as_raw_fd(), + relative, + OpenHow::new() + .flags(flags | OFlag::O_CLOEXEC) + .mode(mode) + .resolve(Self::resolve_flags()), + ) + .map_err(|error| match error { + Errno::EXDEV => VfsError::access_denied( + "open", + &relative_display, + Some("path escapes host directory"), + ), + other => io_error_to_vfs("open", &relative_display, nix_to_io(other)), + })?; + Ok(AnchoredFd { fd }) + } + + fn open_directory_beneath(&self, relative: &Path) -> VfsResult { + self.open_beneath( + relative, + OFlag::O_DIRECTORY | OFlag::O_RDONLY, + Mode::empty(), + ) + } + + fn host_path_for_fd(&self, fd: &AnchoredFd, virtual_path: &str) -> VfsResult { + let host_path = fs::read_link(fd.proc_path()) + .map_err(|error| io_error_to_vfs("open", virtual_path, error))?; + self.ensure_within_root(&host_path, virtual_path)?; + Ok(host_path) + } + + fn ensure_directory_tree(&self, relative_dir: &Path, virtual_path: &str) -> VfsResult<()> { + if relative_dir == Path::new(".") { + return Ok(()); + } + + let mut prefix = PathBuf::new(); + for component in relative_dir.components() { + match component { + Component::Normal(segment) => prefix.push(segment), + Component::CurDir => continue, + _ => { + return Err(VfsError::new( + "EINVAL", + format!("invalid host_dir component in {virtual_path}"), + )); + } } - Err(error) if error.kind() == io::ErrorKind::NotFound => { - let parent = joined - .parent() - .map(Path::to_path_buf) - .unwrap_or_else(|| self.host_root.clone()); - match fs::canonicalize(&parent) { - Ok(real_parent) => { - self.ensure_within_root(&real_parent, path)?; - } - Err(parent_error) if parent_error.kind() == io::ErrorKind::NotFound => { - self.ensure_within_root(&joined, path)?; - } - Err(parent_error) => { - return Err(io_error_to_vfs("open", path, parent_error)); - } + + if self.open_directory_beneath(&prefix).is_ok() { + continue; + } + + let parent = match prefix.parent() { + Some(parent) if !parent.as_os_str().is_empty() => parent, + _ => Path::new("."), + }; + let parent_dir = self.open_directory_beneath(parent)?; + let name = prefix.file_name().ok_or_else(|| { + VfsError::new("EINVAL", format!("invalid directory path: {virtual_path}")) + })?; + match mkdirat( + Some(parent_dir.as_raw_fd()), + name, + Mode::from_bits_truncate(0o755), + ) { + Ok(()) => {} + Err(Errno::EEXIST) => {} + Err(error) => { + return Err(io_error_to_vfs("mkdir", virtual_path, nix_to_io(error))); } - Ok(joined) } - Err(error) => Err(io_error_to_vfs("open", path, error)), } + + Ok(()) } - fn resolve_no_follow(&self, path: &str) -> VfsResult { - let joined = self.lexical_host_path(path)?; - let parent = joined - .parent() - .map(Path::to_path_buf) - .unwrap_or_else(|| self.host_root.clone()); - match fs::canonicalize(&parent) { - Ok(real_parent) => { - self.ensure_within_root(&real_parent, path)?; - } - Err(error) if error.kind() == io::ErrorKind::NotFound => { - self.ensure_within_root(&joined, path)?; - } - Err(error) => return Err(io_error_to_vfs("open", path, error)), + fn split_parent( + &self, + path: &str, + create_parent_dirs: bool, + ) -> VfsResult<(AnchoredFd, PathBuf, std::ffi::OsString, String)> { + let (normalized, relative) = self.relative_virtual_path(path); + let name = relative.file_name().ok_or_else(|| { + VfsError::new( + "EINVAL", + format!("path does not reference an entry: {normalized}"), + ) + })?; + let parent = match relative.parent() { + Some(parent) if !parent.as_os_str().is_empty() => parent.to_path_buf(), + _ => PathBuf::from("."), + }; + if create_parent_dirs { + self.ensure_directory_tree(&parent, &normalized)?; } - Ok(joined) + let parent_dir = self.open_directory_beneath(&parent)?; + Ok((parent_dir, parent, name.to_os_string(), normalized)) } fn host_to_virtual_path(&self, host_path: &Path, virtual_path: &str) -> VfsResult { @@ -175,6 +282,9 @@ impl HostDirFilesystem { VirtualStat { mode: metadata.mode(), size: metadata.size(), + blocks: metadata.blocks(), + dev: metadata.dev(), + rdev: metadata.rdev(), is_directory: metadata.is_dir(), is_symbolic_link: metadata.file_type().is_symlink(), atime_ms, @@ -187,15 +297,52 @@ impl HostDirFilesystem { gid: metadata.gid(), } } + + fn stat_from_file_stat(stat: nix::sys::stat::FileStat) -> VirtualStat { + let file_type = SFlag::from_bits_truncate(stat.st_mode); + let atime_ms = + stat.st_atime.max(0) as u64 * 1_000 + (stat.st_atime_nsec.max(0) as u64 / 1_000_000); + let mtime_ms = + stat.st_mtime.max(0) as u64 * 1_000 + (stat.st_mtime_nsec.max(0) as u64 / 1_000_000); + let ctime_ms = + stat.st_ctime.max(0) as u64 * 1_000 + (stat.st_ctime_nsec.max(0) as u64 / 1_000_000); + + VirtualStat { + mode: stat.st_mode, + size: stat.st_size as u64, + blocks: stat.st_blocks as u64, + dev: stat.st_dev, + rdev: stat.st_rdev, + is_directory: file_type == SFlag::S_IFDIR, + is_symbolic_link: file_type == SFlag::S_IFLNK, + atime_ms, + mtime_ms, + ctime_ms, + birthtime_ms: ctime_ms, + ino: stat.st_ino, + nlink: stat.st_nlink, + uid: stat.st_uid, + gid: stat.st_gid, + } + } } impl VirtualFileSystem for HostDirFilesystem { fn read_file(&mut self, path: &str) -> VfsResult> { - fs::read(self.resolve(path)?).map_err(|error| io_error_to_vfs("open", path, error)) + let (_, relative) = self.relative_virtual_path(path); + let handle = self.open_beneath(&relative, OFlag::O_RDONLY, Mode::empty())?; + let mut file = + File::open(handle.proc_path()).map_err(|error| io_error_to_vfs("open", path, error))?; + let mut buffer = Vec::new(); + file.read_to_end(&mut buffer) + .map_err(|error| io_error_to_vfs("open", path, error))?; + Ok(buffer) } fn read_dir(&mut self, path: &str) -> VfsResult> { - let mut entries = fs::read_dir(self.resolve(path)?) + let (_, relative) = self.relative_virtual_path(path); + let directory = self.open_directory_beneath(&relative)?; + let mut entries = fs::read_dir(directory.proc_path()) .map_err(|error| io_error_to_vfs("readdir", path, error))? .map(|entry| { entry @@ -208,7 +355,9 @@ impl VirtualFileSystem for HostDirFilesystem { } fn read_dir_with_types(&mut self, path: &str) -> VfsResult> { - let mut entries = fs::read_dir(self.resolve(path)?) + let (_, relative) = self.relative_virtual_path(path); + let directory = self.open_directory_beneath(&relative)?; + let mut entries = fs::read_dir(directory.proc_path()) .map_err(|error| io_error_to_vfs("readdir", path, error))? .map(|entry| { let entry = entry.map_err(|error| io_error_to_vfs("readdir", path, error))?; @@ -227,71 +376,99 @@ impl VirtualFileSystem for HostDirFilesystem { } fn write_file(&mut self, path: &str, content: impl Into>) -> VfsResult<()> { - let host_path = self.resolve(path)?; - if let Some(parent) = host_path.parent() { - fs::create_dir_all(parent).map_err(|error| io_error_to_vfs("mkdir", path, error))?; + let (_, relative) = self.relative_virtual_path(path); + if let Some(parent) = relative.parent() { + self.ensure_directory_tree(parent, path)?; } - fs::write(host_path, content.into()).map_err(|error| io_error_to_vfs("write", path, error)) + let handle = self.open_beneath( + &relative, + OFlag::O_WRONLY | OFlag::O_CREAT | OFlag::O_TRUNC, + Mode::from_bits_truncate(0o644), + )?; + let mut file = File::options() + .write(true) + .open(handle.proc_path()) + .map_err(|error| io_error_to_vfs("write", path, error))?; + file.write_all(&content.into()) + .map_err(|error| io_error_to_vfs("write", path, error)) } fn create_dir(&mut self, path: &str) -> VfsResult<()> { - fs::create_dir(self.resolve(path)?).map_err(|error| io_error_to_vfs("mkdir", path, error)) + let (parent_dir, _, name, normalized) = self.split_parent(path, false)?; + mkdirat( + Some(parent_dir.as_raw_fd()), + name.as_os_str(), + Mode::from_bits_truncate(0o755), + ) + .map_err(|error| io_error_to_vfs("mkdir", &normalized, nix_to_io(error))) } fn mkdir(&mut self, path: &str, recursive: bool) -> VfsResult<()> { - let host_path = self.resolve(path)?; if recursive { - fs::create_dir_all(host_path) + let (normalized, relative) = self.relative_virtual_path(path); + self.ensure_directory_tree(&relative, &normalized) } else { - fs::create_dir(host_path) + self.create_dir(path) } - .map_err(|error| io_error_to_vfs("mkdir", path, error)) } fn exists(&self, path: &str) -> bool { - self.resolve(path) - .map(|resolved| resolved.exists()) - .unwrap_or(false) + let (_, relative) = self.relative_virtual_path(path); + self.open_beneath(&relative, OFlag::O_PATH, Mode::empty()) + .is_ok() } fn stat(&mut self, path: &str) -> VfsResult { - fs::metadata(self.resolve(path)?) + let (_, relative) = self.relative_virtual_path(path); + let handle = self.open_beneath(&relative, OFlag::O_PATH, Mode::empty())?; + fs::metadata(handle.proc_path()) .map(Self::stat_from_metadata) .map_err(|error| io_error_to_vfs("stat", path, error)) } fn remove_file(&mut self, path: &str) -> VfsResult<()> { - fs::remove_file(self.resolve_no_follow(path)?) - .map_err(|error| io_error_to_vfs("unlink", path, error)) + let (parent_dir, _, name, normalized) = self.split_parent(path, false)?; + unlinkat( + Some(parent_dir.as_raw_fd()), + name.as_os_str(), + UnlinkatFlags::NoRemoveDir, + ) + .map_err(|error| io_error_to_vfs("unlink", &normalized, nix_to_io(error))) } fn remove_dir(&mut self, path: &str) -> VfsResult<()> { - fs::remove_dir(self.resolve(path)?).map_err(|error| io_error_to_vfs("rmdir", path, error)) + let (parent_dir, _, name, normalized) = self.split_parent(path, false)?; + unlinkat( + Some(parent_dir.as_raw_fd()), + name.as_os_str(), + UnlinkatFlags::RemoveDir, + ) + .map_err(|error| io_error_to_vfs("rmdir", &normalized, nix_to_io(error))) } fn rename(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { - let old_host_path = self.resolve_no_follow(old_path)?; - let new_host_path = self.resolve_no_follow(new_path)?; - if let Some(parent) = new_host_path.parent() { - fs::create_dir_all(parent) - .map_err(|error| io_error_to_vfs("mkdir", new_path, error))?; - } - fs::rename(old_host_path, new_host_path) - .map_err(|error| io_error_to_vfs("rename", old_path, error)) + let (old_parent_dir, _, old_name, old_normalized) = self.split_parent(old_path, false)?; + let (new_parent_dir, _, new_name, _) = self.split_parent(new_path, true)?; + renameat( + Some(old_parent_dir.as_raw_fd()), + old_name.as_os_str(), + Some(new_parent_dir.as_raw_fd()), + new_name.as_os_str(), + ) + .map_err(|error| io_error_to_vfs("rename", &old_normalized, nix_to_io(error))) } fn realpath(&self, path: &str) -> VfsResult { - let resolved = fs::canonicalize(self.resolve_no_follow(path)?) - .map_err(|error| io_error_to_vfs("realpath", path, error))?; + let (_, relative) = self.relative_virtual_path(path); + let file = self.open_beneath(&relative, OFlag::O_PATH, Mode::empty())?; + let resolved = self.host_path_for_fd(&file, path)?; self.host_to_virtual_path(&resolved, path) } fn symlink(&mut self, target: &str, link_path: &str) -> VfsResult<()> { - let host_link_path = self.resolve_no_follow(link_path)?; - if let Some(parent) = host_link_path.parent() { - fs::create_dir_all(parent) - .map_err(|error| io_error_to_vfs("mkdir", link_path, error))?; - } + let (parent_dir, _, name, normalized) = self.split_parent(link_path, true)?; + let parent_host_path = self.host_path_for_fd(&parent_dir, &normalized)?; + let host_link_path = parent_host_path.join(&name); let link_virtual_path = normalize_path(link_path); let target_virtual_path = if target.starts_with('/') { @@ -308,84 +485,109 @@ impl VirtualFileSystem for HostDirFilesystem { host_link_path.parent().unwrap_or(self.host_root.as_path()), &host_target_path, ); - create_symlink(&relative_target, host_link_path) - .map_err(|error| io_error_to_vfs("symlink", link_path, error)) + symlinkat( + &relative_target, + Some(parent_dir.as_raw_fd()), + name.as_os_str(), + ) + .map_err(|error| io_error_to_vfs("symlink", link_path, nix_to_io(error))) } fn read_link(&self, path: &str) -> VfsResult { - let host_link_path = self.resolve_no_follow(path)?; - let link_target = fs::read_link(&host_link_path) - .map_err(|error| io_error_to_vfs("readlink", path, error))?; - let resolved_target = if link_target.is_absolute() { - lexical_normalize_path(&link_target) + let (parent_dir, _, name, normalized) = self.split_parent(path, false)?; + let parent_host_path = self.host_path_for_fd(&parent_dir, &normalized)?; + let host_link_path = parent_host_path.join(&name); + let link_target = readlinkat(Some(parent_dir.as_raw_fd()), name.as_os_str()) + .map_err(|error| io_error_to_vfs("readlink", path, nix_to_io(error)))?; + let link_target_path = PathBuf::from(&link_target); + let resolved_target = if link_target_path.is_absolute() { + lexical_normalize_path(&link_target_path) } else { lexical_normalize_path( &host_link_path .parent() .unwrap_or(self.host_root.as_path()) - .join(link_target), + .join(link_target_path), ) }; self.host_to_virtual_path(&resolved_target, path) } fn lstat(&self, path: &str) -> VfsResult { - fs::symlink_metadata(self.resolve_no_follow(path)?) - .map(Self::stat_from_metadata) - .map_err(|error| io_error_to_vfs("lstat", path, error)) + let (parent_dir, _, name, normalized) = self.split_parent(path, false)?; + fstatat( + Some(parent_dir.as_raw_fd()), + name.as_os_str(), + AtFlags::AT_SYMLINK_NOFOLLOW, + ) + .map(Self::stat_from_file_stat) + .map_err(|error| io_error_to_vfs("lstat", &normalized, nix_to_io(error))) } fn link(&mut self, old_path: &str, new_path: &str) -> VfsResult<()> { - let host_old_path = self.resolve_no_follow(old_path)?; - let host_new_path = self.resolve_no_follow(new_path)?; - if let Some(parent) = host_new_path.parent() { - fs::create_dir_all(parent) - .map_err(|error| io_error_to_vfs("mkdir", new_path, error))?; - } - fs::hard_link(host_old_path, host_new_path) - .map_err(|error| io_error_to_vfs("link", new_path, error)) + let (old_parent_dir, _, old_name, _) = self.split_parent(old_path, false)?; + let (new_parent_dir, _, new_name, new_normalized) = self.split_parent(new_path, true)?; + linkat( + Some(old_parent_dir.as_raw_fd()), + old_name.as_os_str(), + Some(new_parent_dir.as_raw_fd()), + new_name.as_os_str(), + AtFlags::empty(), + ) + .map_err(|error| io_error_to_vfs("link", &new_normalized, nix_to_io(error))) } fn chmod(&mut self, path: &str, mode: u32) -> VfsResult<()> { - fs::set_permissions(self.resolve(path)?, fs::Permissions::from_mode(mode)) - .map_err(|error| io_error_to_vfs("chmod", path, error)) + let (_, relative) = self.relative_virtual_path(path); + fchmodat( + Some(self.host_root_dir.as_raw_fd()), + &relative, + Mode::from_bits_truncate(mode), + FchmodatFlags::FollowSymlink, + ) + .map_err(|error| io_error_to_vfs("chmod", path, nix_to_io(error))) } fn chown(&mut self, path: &str, uid: u32, gid: u32) -> VfsResult<()> { - chown( - &self.resolve(path)?, + let (_, relative) = self.relative_virtual_path(path); + fchownat( + Some(self.host_root_dir.as_raw_fd()), + &relative, Some(Uid::from_raw(uid)), Some(Gid::from_raw(gid)), + AtFlags::empty(), ) .map_err(|error| VfsError::new(error_code(&error), error.to_string())) } fn utimes(&mut self, path: &str, atime_ms: u64, mtime_ms: u64) -> VfsResult<()> { - set_file_times( - self.resolve(path)?, - FileTime::from_unix_time( - (atime_ms / 1_000) as i64, - ((atime_ms % 1_000) * 1_000_000) as u32, - ), - FileTime::from_unix_time( - (mtime_ms / 1_000) as i64, - ((mtime_ms % 1_000) * 1_000_000) as u32, - ), + let (_, relative) = self.relative_virtual_path(path); + utimensat( + Some(self.host_root_dir.as_raw_fd()), + &relative, + &TimeSpec::nanoseconds((atime_ms as i64) * 1_000_000), + &TimeSpec::nanoseconds((mtime_ms as i64) * 1_000_000), + UtimensatFlags::FollowSymlink, ) - .map_err(|error| io_error_to_vfs("utimes", path, error)) + .map_err(|error| io_error_to_vfs("utimes", path, nix_to_io(error))) } fn truncate(&mut self, path: &str, length: u64) -> VfsResult<()> { - File::options() + let (_, relative) = self.relative_virtual_path(path); + let handle = self.open_beneath(&relative, OFlag::O_WRONLY, Mode::empty())?; + let file = File::options() .write(true) - .open(self.resolve(path)?) - .and_then(|file| file.set_len(length)) + .open(handle.proc_path()) + .map_err(|error| io_error_to_vfs("truncate", path, error))?; + file.set_len(length) .map_err(|error| io_error_to_vfs("truncate", path, error)) } fn pread(&mut self, path: &str, offset: u64, length: usize) -> VfsResult> { - let file = File::open(self.resolve(path)?) - .map_err(|error| io_error_to_vfs("open", path, error))?; + let (_, relative) = self.relative_virtual_path(path); + let handle = self.open_beneath(&relative, OFlag::O_RDONLY, Mode::empty())?; + let file = + File::open(handle.proc_path()).map_err(|error| io_error_to_vfs("open", path, error))?; let mut buffer = vec![0; length]; let bytes_read = file .read_at(&mut buffer, offset) @@ -395,6 +597,10 @@ impl VirtualFileSystem for HostDirFilesystem { } } +fn nix_to_io(error: Errno) -> io::Error { + io::Error::from_raw_os_error(error as i32) +} + fn io_error_to_vfs(op: &'static str, path: &str, error: io::Error) -> VfsError { let code = match error.raw_os_error() { Some(1) => "EPERM", @@ -514,8 +720,10 @@ mod tests { #[test] fn filesystem_rejects_symlink_escapes_and_round_trips_writes() { let host_dir = temp_dir("agent-os-host-dir-plugin"); + let outside_dir = temp_dir("agent-os-host-dir-plugin-outside"); fs::write(host_dir.join("hello.txt"), "hello from host").expect("seed host file"); - std::os::unix::fs::symlink("/etc", host_dir.join("escape")).expect("seed escape symlink"); + std::os::unix::fs::symlink(&outside_dir, host_dir.join("escape")) + .expect("seed escape symlink"); let mut filesystem = HostDirFilesystem::new(&host_dir).expect("create host dir fs"); assert_eq!( @@ -537,8 +745,22 @@ mod tests { .read_file("/escape/hostname") .expect_err("escape symlink should fail closed"); assert_eq!(error.code(), "EACCES"); + assert!( + !outside_dir.join("hostname").exists(), + "read should not materialize files outside the host mount" + ); + + let error = filesystem + .write_file("/escape/owned.txt", b"owned".to_vec()) + .expect_err("escape symlink write should fail closed"); + assert_eq!(error.code(), "EACCES"); + assert!( + !outside_dir.join("owned.txt").exists(), + "write should not escape the mounted host directory" + ); fs::remove_dir_all(host_dir).expect("remove temp dir"); + fs::remove_dir_all(outside_dir).expect("remove outside temp dir"); } #[test] diff --git a/crates/sidecar/src/protocol.rs b/crates/sidecar/src/protocol.rs index 1bab503e7..c53e63a32 100644 --- a/crates/sidecar/src/protocol.rs +++ b/crates/sidecar/src/protocol.rs @@ -322,6 +322,8 @@ pub struct CreateVmRequest { pub metadata: BTreeMap, #[serde(default)] pub root_filesystem: RootFilesystemDescriptor, + #[serde(default)] + pub permissions: Vec, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -380,6 +382,8 @@ pub struct ConfigureVmRequest { pub permissions: Vec, pub instructions: Vec, pub projected_modules: Vec, + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + pub command_permissions: BTreeMap, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -445,6 +449,15 @@ pub struct ProjectedModuleDescriptor { pub entrypoint: String, } +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum WasmPermissionTier { + Full, + ReadWrite, + ReadOnly, + Isolated, +} + #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct ExecuteRequest { pub process_id: String, @@ -455,6 +468,8 @@ pub struct ExecuteRequest { pub env: BTreeMap, #[serde(default, skip_serializing_if = "Option::is_none")] pub cwd: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub wasm_permission_tier: Option, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -562,6 +577,9 @@ pub struct VmConfiguredResponse { pub struct GuestFilesystemStat { pub mode: u32, pub size: u64, + pub blocks: u64, + pub dev: u64, + pub rdev: u64, pub is_directory: bool, pub is_symbolic_link: bool, pub atime_ms: u64, diff --git a/crates/sidecar/src/s3_plugin.rs b/crates/sidecar/src/s3_plugin.rs index 01e8d8832..d6dcee8aa 100644 --- a/crates/sidecar/src/s3_plugin.rs +++ b/crates/sidecar/src/s3_plugin.rs @@ -17,7 +17,9 @@ use base64::engine::general_purpose::STANDARD as BASE64; use base64::Engine; use serde::{Deserialize, Serialize}; use std::collections::{BTreeMap, BTreeSet}; +use std::net::IpAddr; use tokio::runtime::Runtime; +use url::Url; const DEFAULT_CHUNK_SIZE: usize = 4 * 1024 * 1024; const DEFAULT_INLINE_THRESHOLD: usize = 64 * 1024; @@ -449,7 +451,7 @@ impl S3ObjectStore { let mut builder = S3ConfigBuilder::from(&shared_config).force_path_style(true); if let Some(endpoint) = endpoint { - builder = builder.endpoint_url(endpoint); + builder = builder.endpoint_url(validate_s3_endpoint(&endpoint)?); } Ok(Self { @@ -542,6 +544,72 @@ impl S3ObjectStore { } } +fn validate_s3_endpoint(raw: &str) -> Result { + let normalized = raw.trim().trim_end_matches('/').to_owned(); + if normalized.is_empty() { + return Err(PluginError::invalid_input( + "s3 mount endpoint must be a valid URL", + )); + } + + let url = Url::parse(&normalized).map_err(|error| { + PluginError::invalid_input(format!("s3 mount endpoint is not a valid URL: {error}")) + })?; + let host = url + .host_str() + .ok_or_else(|| PluginError::invalid_input("s3 mount endpoint must include a host"))?; + + if is_allowed_test_endpoint_host(host) { + return Ok(normalized); + } + + if host.eq_ignore_ascii_case("localhost") { + return Err(PluginError::invalid_input( + "s3 mount endpoint must not target localhost", + )); + } + if let Ok(ip) = host.parse::() { + if is_disallowed_s3_endpoint_ip(ip) { + return Err(PluginError::invalid_input(format!( + "s3 mount endpoint must not target a private or local IP address ({host})" + ))); + } + } + + Ok(normalized) +} + +fn is_disallowed_s3_endpoint_ip(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(ip) => { + ip.is_private() + || ip.is_loopback() + || ip.is_link_local() + || ip.is_multicast() + || ip.is_unspecified() + } + IpAddr::V6(ip) => { + ip.is_loopback() + || ip.is_unique_local() + || ip.is_unicast_link_local() + || ip.is_multicast() + || ip.is_unspecified() + } + } +} + +fn is_allowed_test_endpoint_host(host: &str) -> bool { + #[cfg(test)] + { + matches!(host, "127.0.0.1" | "localhost" | "::1") + } + #[cfg(not(test))] + { + let _ = host; + false + } +} + #[derive(Debug, Clone)] struct StorageError { message: String, @@ -1130,6 +1198,24 @@ mod tests { } } + #[test] + fn s3_plugin_rejects_private_ip_endpoints() { + let server = MockS3Server::start(); + let mut config = test_config(&server, "reject-private-endpoint"); + config.endpoint = Some(String::from("http://169.254.169.254/latest")); + + let error = match S3BackedFilesystem::from_config(config) { + Ok(_) => panic!("private IP endpoint should fail"), + Err(error) => error, + }; + assert!( + error + .to_string() + .contains("s3 mount endpoint must not target a private or local IP address"), + "unexpected error: {error}" + ); + } + #[test] fn s3_plugin_persists_files_across_reopen_and_preserves_links() { let server = MockS3Server::start(); diff --git a/crates/sidecar/src/sandbox_agent_plugin.rs b/crates/sidecar/src/sandbox_agent_plugin.rs index 654f02f3d..631669d19 100644 --- a/crates/sidecar/src/sandbox_agent_plugin.rs +++ b/crates/sidecar/src/sandbox_agent_plugin.rs @@ -119,6 +119,13 @@ impl SandboxAgentFilesystem { S_IFREG | 0o644 }, size: stat.size, + blocks: if stat.size == 0 { + 0 + } else { + stat.size.div_ceil(512) + }, + dev: 1, + rdev: 0, is_directory, is_symbolic_link: false, atime_ms: modified_ms, diff --git a/crates/sidecar/src/service.rs b/crates/sidecar/src/service.rs index 14347945b..169dc6ea0 100644 --- a/crates/sidecar/src/service.rs +++ b/crates/sidecar/src/service.rs @@ -2,9 +2,8 @@ use crate::google_drive_plugin::GoogleDriveMountPlugin; use crate::host_dir_plugin::HostDirMountPlugin; use crate::protocol::{ AuthenticatedResponse, BoundUdpSnapshotResponse, CloseStdinRequest, ConfigureVmRequest, - DiagnosticsRequest, DiagnosticsSnapshotResponse, DisposeReason, DisposeVmRequest, EventFrame, - EventPayload, ExecuteRequest, FindBoundUdpRequest, FindListenerRequest, GetSignalStateRequest, - GetZombieTimerCountRequest, + DisposeReason, DisposeVmRequest, EventFrame, EventPayload, ExecuteRequest, FindBoundUdpRequest, + FindListenerRequest, GetSignalStateRequest, GetZombieTimerCountRequest, GuestFilesystemCallRequest, GuestFilesystemOperation, GuestFilesystemResultResponse, GuestFilesystemStat, GuestRuntimeKind, KillProcessRequest, ListenerSnapshotResponse, OpenSessionRequest, OwnershipScope, ProcessExitedEvent, ProcessKilledResponse, @@ -13,10 +12,11 @@ use crate::protocol::{ RootFilesystemDescriptor, RootFilesystemEntry, RootFilesystemEntryEncoding, RootFilesystemEntryKind, RootFilesystemLowerDescriptor, RootFilesystemMode, RootFilesystemSnapshotResponse, SessionOpenedResponse, SidecarPlacement, - SignalHandlerRegistration, SignalStateResponse, SnapshotRootFilesystemRequest, - SocketStateEntry, StdinClosedResponse, StdinWrittenResponse, StreamChannel, - VmConfiguredResponse, VmCreatedResponse, VmDisposedResponse, VmLifecycleEvent, - VmLifecycleState, WriteStdinRequest, ZombieTimerCountResponse, DEFAULT_MAX_FRAME_BYTES, + SignalDispositionAction, SignalHandlerRegistration, SignalStateResponse, + SnapshotRootFilesystemRequest, SocketStateEntry, StdinClosedResponse, StdinWrittenResponse, + StreamChannel, VmConfiguredResponse, VmCreatedResponse, VmDisposedResponse, VmLifecycleEvent, + VmLifecycleState, WasmPermissionTier, WriteStdinRequest, ZombieTimerCountResponse, + DEFAULT_MAX_FRAME_BYTES, }; use crate::s3_plugin::S3MountPlugin; use crate::sandbox_agent_plugin::SandboxAgentMountPlugin; @@ -27,16 +27,20 @@ use agent_os_bridge::{ FilesystemPermissionRequest, FilesystemSnapshot, FlushFilesystemStateRequest, LifecycleEventRecord, LifecycleState, LoadFilesystemStateRequest, LogLevel, LogRecord, NetworkAccess, NetworkPermissionRequest, PathRequest, ReadDirRequest, ReadFileRequest, - RenameRequest, SymlinkRequest, TruncateRequest, WriteFileRequest, + RenameRequest, StructuredEventRecord, SymlinkRequest, TruncateRequest, WriteFileRequest, +}; +use agent_os_execution::wasm::{ + WASM_MAX_FUEL_ENV, WASM_MAX_MEMORY_BYTES_ENV, WASM_MAX_STACK_BYTES_ENV, }; use agent_os_execution::{ CreateJavascriptContextRequest, CreatePythonContextRequest, CreateWasmContextRequest, JavascriptExecution, JavascriptExecutionEngine, JavascriptExecutionError, - JavascriptExecutionEvent, PythonExecution, PythonExecutionEngine, PythonExecutionError, + JavascriptExecutionEvent, JavascriptSyncRpcRequest, NodeSignalDispositionAction, + NodeSignalHandlerRegistration, PythonExecution, PythonExecutionEngine, PythonExecutionError, PythonExecutionEvent, PythonVfsRpcMethod, PythonVfsRpcRequest, PythonVfsRpcResponsePayload, PythonVfsRpcStat, StartJavascriptExecutionRequest, StartPythonExecutionRequest, StartWasmExecutionRequest, WasmExecution, WasmExecutionEngine, WasmExecutionError, - WasmExecutionEvent, + WasmExecutionEvent, WasmPermissionTier as ExecutionWasmPermissionTier, }; use agent_os_kernel::command_registry::CommandDriver; use agent_os_kernel::kernel::{ @@ -63,16 +67,29 @@ use agent_os_kernel::vfs::{ MemoryFileSystem, VfsError, VfsResult, VirtualDirEntry, VirtualFileSystem, VirtualStat, }; use base64::Engine; +use hickory_resolver::config::{NameServerConfig, ResolverConfig}; +use hickory_resolver::net::runtime::TokioRuntimeProvider; +use hickory_resolver::TokioResolver; use nix::libc; use nix::sys::signal::{kill as send_signal, Signal}; +use nix::sys::wait::{waitid as wait_on_child, Id as WaitId, WaitPidFlag, WaitStatus}; use nix::unistd::Pid; +use serde::Deserialize; +use serde_json::json; use serde_json::Value; use std::collections::{BTreeMap, BTreeSet}; use std::error::Error; use std::fmt; use std::fs; -use std::net::{Ipv4Addr, Ipv6Addr}; +use std::io::{Read, Write}; +use std::net::{ + IpAddr, Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr, TcpListener, TcpStream, ToSocketAddrs, + UdpSocket, +}; +use std::os::unix::net::{SocketAddr as UnixSocketAddr, UnixListener, UnixStream}; use std::path::{Component, Path, PathBuf}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender}; use std::sync::{Arc, Mutex}; use std::thread; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; @@ -81,10 +98,18 @@ const EXECUTION_DRIVER_NAME: &str = "agent-os-sidecar-execution"; const JAVASCRIPT_COMMAND: &str = "node"; const PYTHON_COMMAND: &str = "python"; const WASM_COMMAND: &str = "wasm"; +const PYTHON_VFS_RPC_GUEST_ROOT: &str = "/workspace"; +const EXECUTION_SANDBOX_ROOT_ENV: &str = "AGENT_OS_SANDBOX_ROOT"; const HOST_REALPATH_MAX_SYMLINK_DEPTH: usize = 40; const DISPOSE_VM_SIGTERM_GRACE: Duration = Duration::from_millis(100); const DISPOSE_VM_SIGKILL_GRACE: Duration = Duration::from_millis(100); -const SIGNAL_STATE_CONTROL_PREFIX: &str = "__AGENT_OS_SIGNAL_STATE__:"; +const VM_DNS_SERVERS_METADATA_KEY: &str = "network.dns.servers"; +const VM_DNS_OVERRIDE_METADATA_PREFIX: &str = "network.dns.override."; +const VM_LISTEN_PORT_MIN_METADATA_KEY: &str = "network.listen.port_min"; +const VM_LISTEN_PORT_MAX_METADATA_KEY: &str = "network.listen.port_max"; +const VM_LISTEN_ALLOW_PRIVILEGED_METADATA_KEY: &str = "network.listen.allow_privileged"; +const DEFAULT_JAVASCRIPT_NET_BACKLOG: u32 = 511; +const LOOPBACK_EXEMPT_PORTS_ENV: &str = "AGENT_OS_LOOPBACK_EXEMPT_PORTS"; type BridgeError = ::Error; type SidecarKernel = KernelVm; @@ -147,12 +172,14 @@ impl Error for SidecarError {} struct SharedBridge { inner: Arc>, + permissions: Arc>>>, } impl SharedBridge { fn new(bridge: B) -> Self { Self { inner: Arc::new(Mutex::new(bridge)), + permissions: Arc::new(Mutex::new(BTreeMap::new())), } } } @@ -161,6 +188,7 @@ impl Clone for SharedBridge { fn clone(&self) -> Self { Self { inner: Arc::clone(&self.inner), + permissions: Arc::clone(&self.permissions), } } } @@ -213,6 +241,11 @@ where path: &str, access: FilesystemAccess, ) -> PermissionDecision { + if let Some(decision) = + self.static_permission_decision(vm_id, filesystem_permission_capability(access), "fs") + { + return decision; + } match self.with_mut(|bridge| { bridge.check_filesystem_access(FilesystemPermissionRequest { vm_id: vm_id.to_owned(), @@ -226,6 +259,11 @@ where } fn command_decision(&self, vm_id: &str, request: &CommandAccessRequest) -> PermissionDecision { + if let Some(decision) = + self.static_permission_decision(vm_id, "child_process.spawn", "child_process") + { + return decision; + } match self.with_mut(|bridge| { bridge.check_command_execution(CommandPermissionRequest { vm_id: vm_id.to_owned(), @@ -241,6 +279,13 @@ where } fn environment_decision(&self, vm_id: &str, request: &EnvAccessRequest) -> PermissionDecision { + if let Some(decision) = self.static_permission_decision( + vm_id, + environment_permission_capability(request.op), + "env", + ) { + return decision; + } match self.with_mut(|bridge| { bridge.check_environment_access(EnvironmentPermissionRequest { vm_id: vm_id.to_owned(), @@ -258,6 +303,13 @@ where } fn network_decision(&self, vm_id: &str, request: &NetworkAccessRequest) -> PermissionDecision { + if let Some(decision) = self.static_permission_decision( + vm_id, + network_permission_capability(request.op), + "network", + ) { + return decision; + } match self.with_mut(|bridge| { bridge.check_network_access(NetworkPermissionRequest { vm_id: vm_id.to_owned(), @@ -274,6 +326,151 @@ where Err(error) => PermissionDecision::deny(error.to_string()), } } + + fn require_network_access( + &self, + vm_id: &str, + op: NetworkOperation, + resource: impl Into, + ) -> Result<(), SidecarError> { + let resource = resource.into(); + let decision = self.network_decision( + vm_id, + &NetworkAccessRequest { + vm_id: vm_id.to_owned(), + op, + resource: resource.clone(), + }, + ); + if decision.allow { + return Ok(()); + } + + let message = match decision.reason.as_deref() { + Some(reason) => format!("EACCES: permission denied, {resource}: {reason}"), + None => format!("EACCES: permission denied, {resource}"), + }; + Err(SidecarError::Execution(message)) + } + + fn set_vm_permissions( + &self, + vm_id: &str, + permissions: &[crate::protocol::PermissionDescriptor], + ) -> Result<(), SidecarError> { + let mut stored = self.permissions.lock().map_err(|_| { + SidecarError::Bridge(String::from( + "native sidecar permission policy lock poisoned", + )) + })?; + stored.insert( + vm_id.to_owned(), + normalize_permission_descriptors(permissions), + ); + Ok(()) + } + + fn clear_vm_permissions(&self, vm_id: &str) -> Result<(), SidecarError> { + let mut stored = self.permissions.lock().map_err(|_| { + SidecarError::Bridge(String::from( + "native sidecar permission policy lock poisoned", + )) + })?; + stored.remove(vm_id); + Ok(()) + } + + fn static_permission_decision( + &self, + vm_id: &str, + capability: &str, + domain: &str, + ) -> Option { + let stored = self.permissions.lock().ok()?; + let permissions = stored.get(vm_id)?; + let mode = permissions + .get(capability) + .or_else(|| permissions.get(domain)) + .cloned() + .unwrap_or(crate::protocol::PermissionMode::Deny); + Some(permission_mode_to_kernel_decision(mode, capability)) + } +} + +fn default_allow_all_permissions() -> BTreeMap { + BTreeMap::from([ + (String::from("fs"), crate::protocol::PermissionMode::Allow), + ( + String::from("network"), + crate::protocol::PermissionMode::Allow, + ), + ( + String::from("child_process"), + crate::protocol::PermissionMode::Allow, + ), + (String::from("env"), crate::protocol::PermissionMode::Allow), + ]) +} + +fn normalize_permission_descriptors( + permissions: &[crate::protocol::PermissionDescriptor], +) -> BTreeMap { + if permissions.is_empty() { + return default_allow_all_permissions(); + } + + let mut normalized = BTreeMap::new(); + for permission in permissions { + normalized.insert(permission.capability.clone(), permission.mode.clone()); + } + normalized +} + +fn permission_mode_to_kernel_decision( + mode: crate::protocol::PermissionMode, + capability: &str, +) -> PermissionDecision { + match mode { + crate::protocol::PermissionMode::Allow => PermissionDecision::allow(), + crate::protocol::PermissionMode::Ask => { + PermissionDecision::deny(format!("permission prompt required for {capability}")) + } + crate::protocol::PermissionMode::Deny => { + PermissionDecision::deny(format!("blocked by {capability} policy")) + } + } +} + +fn filesystem_permission_capability(access: FilesystemAccess) -> &'static str { + match access { + FilesystemAccess::Read => "fs.read", + FilesystemAccess::Write => "fs.write", + FilesystemAccess::Stat => "fs.stat", + FilesystemAccess::ReadDir => "fs.readdir", + FilesystemAccess::CreateDir => "fs.create_dir", + FilesystemAccess::Remove => "fs.rm", + FilesystemAccess::Rename => "fs.rename", + FilesystemAccess::Symlink => "fs.symlink", + FilesystemAccess::ReadLink => "fs.readlink", + FilesystemAccess::Chmod => "fs.chmod", + FilesystemAccess::Truncate => "fs.truncate", + } +} + +fn network_permission_capability(operation: NetworkOperation) -> &'static str { + match operation { + NetworkOperation::Fetch => "network.fetch", + NetworkOperation::Http => "network.http", + NetworkOperation::Dns => "network.dns", + NetworkOperation::Listen => "network.listen", + } +} + +fn environment_permission_capability(operation: EnvironmentOperation) -> &'static str { + match operation { + EnvironmentOperation::Read => "env.read", + EnvironmentOperation::Write => "env.write", + } } #[derive(Clone)] @@ -372,6 +569,13 @@ impl HostFilesystem { let mut stat = VirtualStat { mode: metadata.mode, size: metadata.size, + blocks: if metadata.size == 0 { + 0 + } else { + metadata.size.div_ceil(512) + }, + dev: 1, + rdev: 0, is_directory: metadata.kind == FileKind::Directory, is_symbolic_link: metadata.kind == FileKind::SymbolicLink, atime_ms: 0, @@ -1278,6 +1482,7 @@ struct VmConfiguration { permissions: Vec, instructions: Vec, projected_modules: Vec, + command_permissions: BTreeMap, } #[allow(dead_code)] @@ -1285,131 +1490,964 @@ struct VmState { connection_id: String, session_id: String, metadata: BTreeMap, + dns: VmDnsConfig, guest_env: BTreeMap, requested_runtime: GuestRuntimeKind, cwd: PathBuf, kernel: SidecarKernel, loaded_snapshot: Option, configuration: VmConfiguration, + command_guest_paths: BTreeMap, + command_permissions: BTreeMap, active_processes: BTreeMap, signal_states: BTreeMap>, } +#[derive(Debug, Clone)] +struct JavascriptSocketPathContext { + sandbox_root: PathBuf, + mounts: Vec, + listen_policy: VmListenPolicy, + loopback_exempt_ports: BTreeSet, + tcp_loopback_guest_to_host_ports: BTreeMap<(JavascriptSocketFamily, u16), u16>, + udp_loopback_guest_to_host_ports: BTreeMap<(JavascriptSocketFamily, u16), u16>, + udp_loopback_host_to_guest_ports: BTreeMap<(JavascriptSocketFamily, u16), u16>, + used_tcp_guest_ports: BTreeMap>, + used_udp_guest_ports: BTreeMap>, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +enum JavascriptSocketFamily { + Ipv4, + Ipv6, +} + +impl JavascriptSocketFamily { + fn from_ip(ip: IpAddr) -> Self { + match ip { + IpAddr::V4(_) => Self::Ipv4, + IpAddr::V6(_) => Self::Ipv6, + } + } +} + +impl From for JavascriptSocketFamily { + fn from(value: JavascriptUdpFamily) -> Self { + match value { + JavascriptUdpFamily::Ipv4 => Self::Ipv4, + JavascriptUdpFamily::Ipv6 => Self::Ipv6, + } + } +} + +#[derive(Debug, Clone, Copy)] +struct VmListenPolicy { + port_min: u16, + port_max: u16, + allow_privileged: bool, +} + +impl Default for VmListenPolicy { + fn default() -> Self { + Self { + port_min: 1, + port_max: u16::MAX, + allow_privileged: false, + } + } +} + +impl JavascriptSocketPathContext { + fn loopback_port_allowed(&self, port: u16) -> bool { + self.loopback_exempt_ports.contains(&port) + || self + .tcp_loopback_guest_to_host_ports + .keys() + .any(|(_, guest_port)| *guest_port == port) + } + + fn translate_tcp_loopback_port( + &self, + family: JavascriptSocketFamily, + port: u16, + ) -> Option { + self.tcp_loopback_guest_to_host_ports + .get(&(family, port)) + .copied() + } + + fn translate_udp_loopback_port( + &self, + family: JavascriptSocketFamily, + port: u16, + ) -> Option { + self.udp_loopback_guest_to_host_ports + .get(&(family, port)) + .copied() + } + + fn guest_udp_port_for_host_port( + &self, + family: JavascriptSocketFamily, + port: u16, + ) -> Option { + self.udp_loopback_host_to_guest_ports + .get(&(family, port)) + .copied() + } +} + #[allow(dead_code)] struct ActiveProcess { kernel_pid: u32, kernel_handle: KernelProcessHandle, runtime: GuestRuntimeKind, execution: ActiveExecution, + child_processes: BTreeMap, + next_child_process_id: usize, + tcp_listeners: BTreeMap, + next_tcp_listener_id: usize, + tcp_sockets: BTreeMap, + next_tcp_socket_id: usize, + unix_listeners: BTreeMap, + next_unix_listener_id: usize, + unix_sockets: BTreeMap, + next_unix_socket_id: usize, + udp_sockets: BTreeMap, + next_udp_socket_id: usize, +} + +#[derive(Debug, Clone, Copy, Default)] +struct NetworkResourceCounts { + sockets: usize, + connections: usize, +} + +impl ActiveProcess { + fn new( + kernel_pid: u32, + kernel_handle: KernelProcessHandle, + runtime: GuestRuntimeKind, + execution: ActiveExecution, + ) -> Self { + Self { + kernel_pid, + kernel_handle, + runtime, + execution, + child_processes: BTreeMap::new(), + next_child_process_id: 0, + tcp_listeners: BTreeMap::new(), + next_tcp_listener_id: 0, + tcp_sockets: BTreeMap::new(), + next_tcp_socket_id: 0, + unix_listeners: BTreeMap::new(), + next_unix_listener_id: 0, + unix_sockets: BTreeMap::new(), + next_unix_socket_id: 0, + udp_sockets: BTreeMap::new(), + next_udp_socket_id: 0, + } + } + + fn allocate_child_process_id(&mut self) -> String { + self.next_child_process_id += 1; + format!("child-{}", self.next_child_process_id) + } + + fn allocate_tcp_listener_id(&mut self) -> String { + self.next_tcp_listener_id += 1; + format!("listener-{}", self.next_tcp_listener_id) + } + + fn allocate_tcp_socket_id(&mut self) -> String { + self.next_tcp_socket_id += 1; + format!("socket-{}", self.next_tcp_socket_id) + } + + fn allocate_unix_listener_id(&mut self) -> String { + self.next_unix_listener_id += 1; + format!("unix-listener-{}", self.next_unix_listener_id) + } + + fn allocate_unix_socket_id(&mut self) -> String { + self.next_unix_socket_id += 1; + format!("unix-socket-{}", self.next_unix_socket_id) + } + + fn allocate_udp_socket_id(&mut self) -> String { + self.next_udp_socket_id += 1; + format!("udp-socket-{}", self.next_udp_socket_id) + } + + fn network_resource_counts(&self) -> NetworkResourceCounts { + let mut counts = NetworkResourceCounts { + sockets: self.tcp_listeners.len() + + self.tcp_sockets.len() + + self.unix_listeners.len() + + self.unix_sockets.len() + + self.udp_sockets.len(), + connections: self.tcp_sockets.len() + self.unix_sockets.len(), + }; + + for child in self.child_processes.values() { + let child_counts = child.network_resource_counts(); + counts.sockets += child_counts.sockets; + counts.connections += child_counts.connections; + } + + counts + } } #[derive(Debug)] -enum ActiveExecution { - Javascript(JavascriptExecution), - Python(PythonExecution), - Wasm(WasmExecution), +enum JavascriptTcpListenerEvent { + Connection(PendingTcpSocket), + Error { + code: Option, + message: String, + }, } #[derive(Debug)] -enum ActiveExecutionEvent { - Stdout(Vec), - Stderr(Vec), - PythonVfsRpcRequest(PythonVfsRpcRequest), - Exited(i32), +struct PendingTcpSocket { + stream: TcpStream, + guest_local_addr: SocketAddr, + guest_remote_addr: SocketAddr, } -#[derive(Debug, Clone, PartialEq, Eq, serde::Deserialize)] -struct SignalControlMessage { - signal: u32, - registration: SignalHandlerRegistration, +#[derive(Debug)] +enum JavascriptTcpSocketEvent { + Data(Vec), + End, + Close { + had_error: bool, + }, + Error { + code: Option, + message: String, + }, } -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -enum SocketQueryKind { - TcpListener, - UdpBound, +#[derive(Debug)] +struct ActiveTcpSocket { + stream: Arc>, + events: Receiver, + event_sender: Sender, + guest_local_addr: SocketAddr, + guest_remote_addr: SocketAddr, + listener_id: Option, + saw_local_shutdown: Arc, + saw_remote_end: Arc, + close_notified: Arc, } -impl ActiveExecution { - fn child_pid(&self) -> u32 { - match self { - Self::Javascript(execution) => execution.child_pid(), - Self::Python(execution) => execution.child_pid(), - Self::Wasm(execution) => execution.child_pid(), +#[derive(Debug, Clone, Copy)] +struct ResolvedTcpConnectAddr { + actual_addr: SocketAddr, + guest_remote_addr: SocketAddr, +} + +impl ActiveTcpSocket { + fn connect( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + host: &str, + port: u16, + context: &JavascriptSocketPathContext, + ) -> Result + where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, + { + let resolved = resolve_tcp_connect_addr(bridge, vm_id, dns, host, port, context)?; + let stream = TcpStream::connect_timeout(&resolved.actual_addr, Duration::from_secs(30)) + .map_err(sidecar_net_error)?; + let guest_local_addr = stream.local_addr().map_err(sidecar_net_error)?; + Self::from_stream(stream, None, guest_local_addr, resolved.guest_remote_addr) + } + + fn from_stream( + stream: TcpStream, + listener_id: Option, + guest_local_addr: SocketAddr, + guest_remote_addr: SocketAddr, + ) -> Result { + let read_stream = stream.try_clone().map_err(sidecar_net_error)?; + let stream = Arc::new(Mutex::new(stream)); + let (sender, events) = mpsc::channel(); + let saw_local_shutdown = Arc::new(AtomicBool::new(false)); + let saw_remote_end = Arc::new(AtomicBool::new(false)); + let close_notified = Arc::new(AtomicBool::new(false)); + spawn_tcp_socket_reader( + read_stream, + sender.clone(), + Arc::clone(&saw_local_shutdown), + Arc::clone(&saw_remote_end), + Arc::clone(&close_notified), + ); + + Ok(Self { + stream, + events, + event_sender: sender, + guest_local_addr, + guest_remote_addr, + listener_id, + saw_local_shutdown, + saw_remote_end, + close_notified, + }) + } + + fn poll(&mut self, wait: Duration) -> Result, SidecarError> { + match self.events.recv_timeout(wait) { + Ok(event) => Ok(Some(event)), + Err(RecvTimeoutError::Timeout) => Ok(None), + Err(RecvTimeoutError::Disconnected) => Ok(None), } } - fn write_stdin(&mut self, chunk: &[u8]) -> Result<(), SidecarError> { - match self { - Self::Javascript(execution) => execution - .write_stdin(chunk) - .map_err(|error| SidecarError::Execution(error.to_string())), - Self::Python(execution) => execution - .write_stdin(chunk) - .map_err(|error| SidecarError::Execution(error.to_string())), - Self::Wasm(execution) => execution - .write_stdin(chunk) - .map_err(|error| SidecarError::Execution(error.to_string())), + fn write_all(&self, contents: &[u8]) -> Result { + let mut stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("TCP socket lock poisoned")))?; + stream.write_all(contents).map_err(sidecar_net_error)?; + Ok(contents.len()) + } + + fn shutdown_write(&self) -> Result<(), SidecarError> { + let stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("TCP socket lock poisoned")))?; + self.saw_local_shutdown.store(true, Ordering::SeqCst); + stream + .shutdown(Shutdown::Write) + .map_err(sidecar_net_error)?; + if self.saw_remote_end.load(Ordering::SeqCst) + && !self.close_notified.swap(true, Ordering::SeqCst) + { + let _ = self + .event_sender + .send(JavascriptTcpSocketEvent::Close { had_error: false }); } + Ok(()) } - fn close_stdin(&mut self) -> Result<(), SidecarError> { - match self { - Self::Javascript(execution) => execution - .close_stdin() - .map_err(|error| SidecarError::Execution(error.to_string())), - Self::Python(execution) => execution - .close_stdin() - .map_err(|error| SidecarError::Execution(error.to_string())), - Self::Wasm(execution) => execution - .close_stdin() - .map_err(|error| SidecarError::Execution(error.to_string())), + fn close(&self) -> Result<(), SidecarError> { + let stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("TCP socket lock poisoned")))?; + stream.shutdown(Shutdown::Both).map_err(sidecar_net_error) + } +} + +#[derive(Debug)] +struct ActiveTcpListener { + listener: TcpListener, + local_addr: SocketAddr, + guest_local_addr: SocketAddr, + backlog: usize, + active_connection_ids: BTreeSet, +} + +#[derive(Debug)] +enum JavascriptUnixListenerEvent { + Connection(PendingUnixSocket), + Error { + code: Option, + message: String, + }, +} + +#[derive(Debug)] +struct PendingUnixSocket { + stream: UnixStream, + local_path: Option, + remote_path: Option, +} + +#[derive(Debug)] +struct ActiveUnixSocket { + stream: Arc>, + events: Receiver, + event_sender: Sender, + listener_id: Option, + saw_local_shutdown: Arc, + saw_remote_end: Arc, + close_notified: Arc, +} + +impl ActiveUnixSocket { + fn connect(host_path: &Path, guest_path: &str) -> Result { + let stream = UnixStream::connect(host_path).map_err(sidecar_net_error)?; + Self::from_stream(stream, None, None, Some(guest_path.to_owned())) + } + + fn from_stream( + stream: UnixStream, + listener_id: Option, + _local_path: Option, + _remote_path: Option, + ) -> Result { + let read_stream = stream.try_clone().map_err(sidecar_net_error)?; + let stream = Arc::new(Mutex::new(stream)); + let (sender, events) = mpsc::channel(); + let saw_local_shutdown = Arc::new(AtomicBool::new(false)); + let saw_remote_end = Arc::new(AtomicBool::new(false)); + let close_notified = Arc::new(AtomicBool::new(false)); + spawn_unix_socket_reader( + read_stream, + sender.clone(), + Arc::clone(&saw_local_shutdown), + Arc::clone(&saw_remote_end), + Arc::clone(&close_notified), + ); + + Ok(Self { + stream, + events, + event_sender: sender, + listener_id, + saw_local_shutdown, + saw_remote_end, + close_notified, + }) + } + + fn poll(&mut self, wait: Duration) -> Result, SidecarError> { + match self.events.recv_timeout(wait) { + Ok(event) => Ok(Some(event)), + Err(RecvTimeoutError::Timeout) => Ok(None), + Err(RecvTimeoutError::Disconnected) => Ok(None), } } - fn respond_python_vfs_rpc_success( - &mut self, - id: u64, - payload: PythonVfsRpcResponsePayload, - ) -> Result<(), SidecarError> { - match self { - Self::Python(execution) => execution - .respond_vfs_rpc_success(id, payload) - .map_err(|error| SidecarError::Execution(error.to_string())), - _ => Err(SidecarError::InvalidState(String::from( - "only Python executions can service Python VFS RPC responses", - ))), + fn write_all(&self, contents: &[u8]) -> Result { + let mut stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("Unix socket lock poisoned")))?; + stream.write_all(contents).map_err(sidecar_net_error)?; + Ok(contents.len()) + } + + fn shutdown_write(&self) -> Result<(), SidecarError> { + let stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("Unix socket lock poisoned")))?; + self.saw_local_shutdown.store(true, Ordering::SeqCst); + stream + .shutdown(Shutdown::Write) + .map_err(sidecar_net_error)?; + if self.saw_remote_end.load(Ordering::SeqCst) + && !self.close_notified.swap(true, Ordering::SeqCst) + { + let _ = self + .event_sender + .send(JavascriptTcpSocketEvent::Close { had_error: false }); } + Ok(()) } - fn respond_python_vfs_rpc_error( - &mut self, - id: u64, - code: impl Into, - message: impl Into, - ) -> Result<(), SidecarError> { - match self { - Self::Python(execution) => execution - .respond_vfs_rpc_error(id, code, message) - .map_err(|error| SidecarError::Execution(error.to_string())), - _ => Err(SidecarError::InvalidState(String::from( - "only Python executions can service Python VFS RPC responses", - ))), + fn close(&self) -> Result<(), SidecarError> { + let stream = self + .stream + .lock() + .map_err(|_| SidecarError::InvalidState(String::from("Unix socket lock poisoned")))?; + stream.shutdown(Shutdown::Both).map_err(sidecar_net_error) + } +} + +#[derive(Debug)] +struct ActiveUnixListener { + listener: UnixListener, + path: String, + backlog: usize, + active_connection_ids: BTreeSet, +} + +impl ActiveUnixListener { + fn bind( + host_path: &Path, + guest_path: &str, + backlog: Option, + ) -> Result { + if let Some(parent) = host_path.parent() { + fs::create_dir_all(parent).map_err(sidecar_net_error)?; } + let listener = UnixListener::bind(host_path).map_err(sidecar_net_error)?; + listener.set_nonblocking(true).map_err(sidecar_net_error)?; + Ok(Self { + listener, + path: guest_path.to_owned(), + backlog: usize::try_from(backlog.unwrap_or(DEFAULT_JAVASCRIPT_NET_BACKLOG)) + .expect("default backlog fits within usize"), + active_connection_ids: BTreeSet::new(), + }) } - fn poll_event(&self, timeout: Duration) -> Result, SidecarError> { - match self { - Self::Javascript(execution) => execution - .poll_event(timeout) - .map(|event| { - event.map(|event| match event { - JavascriptExecutionEvent::Stdout(chunk) => { - ActiveExecutionEvent::Stdout(chunk) - } - JavascriptExecutionEvent::Stderr(chunk) => { + fn path(&self) -> &str { + &self.path + } + + fn poll( + &mut self, + wait: Duration, + ) -> Result, SidecarError> { + let deadline = Instant::now() + wait; + loop { + match self.listener.accept() { + Ok((stream, remote_addr)) => { + if self.active_connection_ids.len() >= self.backlog { + let _ = stream.shutdown(Shutdown::Both); + if wait.is_zero() || Instant::now() >= deadline { + return Ok(None); + } + continue; + } + + let local_path = Some(self.path.clone()); + let remote_path = unix_socket_path(&remote_addr); + return Ok(Some(JavascriptUnixListenerEvent::Connection( + PendingUnixSocket { + stream, + local_path, + remote_path, + }, + ))); + } + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + if wait.is_zero() || Instant::now() >= deadline { + return Ok(None); + } + thread::sleep(Duration::from_millis(10)); + } + Err(error) => { + return Ok(Some(JavascriptUnixListenerEvent::Error { + code: io_error_code(&error), + message: error.to_string(), + })); + } + } + } + } + + fn close(&self) -> Result<(), SidecarError> { + Ok(()) + } + + fn active_connection_count(&self) -> usize { + self.active_connection_ids.len() + } + + fn register_connection(&mut self, socket_id: &str) { + self.active_connection_ids.insert(socket_id.to_string()); + } + + fn release_connection(&mut self, socket_id: &str) { + self.active_connection_ids.remove(socket_id); + } +} + +impl ActiveTcpListener { + fn bind(guest_host: &str, guest_port: u16, backlog: Option) -> Result { + let bind_addr = resolve_tcp_bind_addr(guest_host, 0)?; + let listener = TcpListener::bind(bind_addr).map_err(sidecar_net_error)?; + listener.set_nonblocking(true).map_err(sidecar_net_error)?; + let local_addr = listener.local_addr().map_err(sidecar_net_error)?; + Ok(Self { + listener, + local_addr, + guest_local_addr: SocketAddr::new(bind_addr.ip(), guest_port), + backlog: usize::try_from(backlog.unwrap_or(DEFAULT_JAVASCRIPT_NET_BACKLOG)) + .expect("default backlog fits within usize"), + active_connection_ids: BTreeSet::new(), + }) + } + + fn local_addr(&self) -> SocketAddr { + self.local_addr + } + + fn guest_local_addr(&self) -> SocketAddr { + self.guest_local_addr + } + + fn poll(&mut self, wait: Duration) -> Result, SidecarError> { + let deadline = Instant::now() + wait; + loop { + match self.listener.accept() { + Ok((stream, remote_addr)) => { + if self.active_connection_ids.len() >= self.backlog { + let _ = stream.shutdown(Shutdown::Both); + if wait.is_zero() || Instant::now() >= deadline { + return Ok(None); + } + continue; + } + return Ok(Some(JavascriptTcpListenerEvent::Connection( + PendingTcpSocket { + stream, + guest_local_addr: self.guest_local_addr, + guest_remote_addr: SocketAddr::new( + remote_addr.ip(), + remote_addr.port(), + ), + }, + ))); + } + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + if wait.is_zero() || Instant::now() >= deadline { + return Ok(None); + } + thread::sleep(Duration::from_millis(10)); + } + Err(error) => { + return Ok(Some(JavascriptTcpListenerEvent::Error { + code: io_error_code(&error), + message: error.to_string(), + })); + } + } + } + } + + fn close(&self) -> Result<(), SidecarError> { + Ok(()) + } + + fn active_connection_count(&self) -> usize { + self.active_connection_ids.len() + } + + fn register_connection(&mut self, socket_id: &str) { + self.active_connection_ids.insert(socket_id.to_string()); + } + + fn release_connection(&mut self, socket_id: &str) { + self.active_connection_ids.remove(socket_id); + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum JavascriptUdpFamily { + Ipv4, + Ipv6, +} + +impl JavascriptUdpFamily { + fn from_socket_type(value: &str) -> Result { + match value { + "udp4" => Ok(Self::Ipv4), + "udp6" => Ok(Self::Ipv6), + other => Err(SidecarError::InvalidState(format!( + "unsupported dgram socket type {other}" + ))), + } + } + + fn socket_type(self) -> &'static str { + match self { + Self::Ipv4 => "udp4", + Self::Ipv6 => "udp6", + } + } + + fn matches_addr(self, addr: &SocketAddr) -> bool { + match (self, addr) { + (Self::Ipv4, SocketAddr::V4(_)) | (Self::Ipv6, SocketAddr::V6(_)) => true, + _ => false, + } + } +} + +#[derive(Debug)] +enum JavascriptUdpSocketEvent { + Message { + data: Vec, + remote_addr: SocketAddr, + }, + Error { + code: Option, + message: String, + }, +} + +#[derive(Debug)] +struct ActiveUdpSocket { + family: JavascriptUdpFamily, + socket: Option, + guest_local_addr: Option, +} + +impl ActiveUdpSocket { + fn new(family: JavascriptUdpFamily) -> Self { + Self { + family, + socket: None, + guest_local_addr: None, + } + } + + fn local_addr(&self) -> Option { + self.guest_local_addr + } + + fn bind( + &mut self, + host: Option<&str>, + port: u16, + context: &JavascriptSocketPathContext, + ) -> Result { + if self.socket.is_some() { + return Err(SidecarError::Execution(String::from( + "EINVAL: Agent OS dgram socket is already bound", + ))); + } + + let (guest_host, guest_family) = normalize_udp_bind_host(host, self.family)?; + let guest_port = allocate_guest_listen_port( + port, + guest_family, + &context.used_udp_guest_ports, + context.listen_policy, + )?; + let bind_addr = resolve_udp_bind_addr(guest_host, 0, self.family)?; + let socket = UdpSocket::bind(bind_addr).map_err(sidecar_net_error)?; + socket.set_nonblocking(true).map_err(sidecar_net_error)?; + let local_addr = SocketAddr::new(bind_addr.ip(), guest_port); + self.socket = Some(socket); + self.guest_local_addr = Some(local_addr); + Ok(local_addr) + } + + fn ensure_bound_for_send( + &mut self, + context: &JavascriptSocketPathContext, + ) -> Result { + if let Some(local_addr) = self.local_addr() { + return Ok(local_addr); + } + + self.bind(None, 0, context) + } + + fn send_to( + &mut self, + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + host: &str, + port: u16, + context: &JavascriptSocketPathContext, + contents: &[u8], + ) -> Result<(usize, SocketAddr), SidecarError> + where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, + { + let remote_addr = resolve_udp_addr(bridge, vm_id, dns, host, port, self.family, context)?; + let local_addr = self.ensure_bound_for_send(context)?; + let socket = self.socket.as_ref().ok_or_else(|| { + SidecarError::InvalidState(String::from("UDP socket is not initialized")) + })?; + let written = socket + .send_to(contents, remote_addr) + .map_err(sidecar_net_error)?; + Ok((written, local_addr)) + } + + fn poll(&self, wait: Duration) -> Result, SidecarError> { + let socket = self + .socket + .as_ref() + .ok_or_else(|| SidecarError::InvalidState(String::from("UDP socket is not bound")))?; + let deadline = Instant::now() + wait; + let mut buffer = vec![0_u8; 64 * 1024]; + + loop { + match socket.recv_from(&mut buffer) { + Ok((bytes_read, remote_addr)) => { + return Ok(Some(JavascriptUdpSocketEvent::Message { + data: buffer[..bytes_read].to_vec(), + remote_addr, + })) + } + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + if wait.is_zero() || Instant::now() >= deadline { + return Ok(None); + } + thread::sleep(Duration::from_millis(10)); + } + Err(error) => { + return Ok(Some(JavascriptUdpSocketEvent::Error { + code: io_error_code(&error), + message: error.to_string(), + })) + } + } + } + } + + fn close(&mut self) { + self.socket.take(); + self.guest_local_addr = None; + } +} + +#[derive(Debug)] +enum ActiveExecution { + Javascript(JavascriptExecution), + Python(PythonExecution), + Wasm(WasmExecution), +} + +#[derive(Debug)] +enum ActiveExecutionEvent { + Stdout(Vec), + Stderr(Vec), + JavascriptSyncRpcRequest(JavascriptSyncRpcRequest), + PythonVfsRpcRequest(PythonVfsRpcRequest), + SignalState { + signal: u32, + registration: SignalHandlerRegistration, + }, + Exited(i32), +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum SocketQueryKind { + TcpListener, + UdpBound, +} + +impl ActiveExecution { + fn child_pid(&self) -> u32 { + match self { + Self::Javascript(execution) => execution.child_pid(), + Self::Python(execution) => execution.child_pid(), + Self::Wasm(execution) => execution.child_pid(), + } + } + + fn write_stdin(&mut self, chunk: &[u8]) -> Result<(), SidecarError> { + match self { + Self::Javascript(execution) => execution + .write_stdin(chunk) + .map_err(|error| SidecarError::Execution(error.to_string())), + Self::Python(execution) => execution + .write_stdin(chunk) + .map_err(|error| SidecarError::Execution(error.to_string())), + Self::Wasm(execution) => execution + .write_stdin(chunk) + .map_err(|error| SidecarError::Execution(error.to_string())), + } + } + + fn close_stdin(&mut self) -> Result<(), SidecarError> { + match self { + Self::Javascript(execution) => execution + .close_stdin() + .map_err(|error| SidecarError::Execution(error.to_string())), + Self::Python(execution) => execution + .close_stdin() + .map_err(|error| SidecarError::Execution(error.to_string())), + Self::Wasm(execution) => execution + .close_stdin() + .map_err(|error| SidecarError::Execution(error.to_string())), + } + } + + fn respond_python_vfs_rpc_success( + &mut self, + id: u64, + payload: PythonVfsRpcResponsePayload, + ) -> Result<(), SidecarError> { + match self { + Self::Python(execution) => execution + .respond_vfs_rpc_success(id, payload) + .map_err(|error| SidecarError::Execution(error.to_string())), + _ => Err(SidecarError::InvalidState(String::from( + "only Python executions can service Python VFS RPC responses", + ))), + } + } + + fn respond_python_vfs_rpc_error( + &mut self, + id: u64, + code: impl Into, + message: impl Into, + ) -> Result<(), SidecarError> { + match self { + Self::Python(execution) => execution + .respond_vfs_rpc_error(id, code, message) + .map_err(|error| SidecarError::Execution(error.to_string())), + _ => Err(SidecarError::InvalidState(String::from( + "only Python executions can service Python VFS RPC responses", + ))), + } + } + + fn respond_javascript_sync_rpc_success( + &mut self, + id: u64, + result: Value, + ) -> Result<(), SidecarError> { + match self { + Self::Javascript(execution) => execution + .respond_sync_rpc_success(id, result) + .map_err(|error| SidecarError::Execution(error.to_string())), + _ => Err(SidecarError::InvalidState(String::from( + "only JavaScript executions can service JavaScript sync RPC responses", + ))), + } + } + + fn respond_javascript_sync_rpc_error( + &mut self, + id: u64, + code: impl Into, + message: impl Into, + ) -> Result<(), SidecarError> { + match self { + Self::Javascript(execution) => execution + .respond_sync_rpc_error(id, code, message) + .map_err(|error| SidecarError::Execution(error.to_string())), + _ => Err(SidecarError::InvalidState(String::from( + "only JavaScript executions can service JavaScript sync RPC responses", + ))), + } + } + + fn poll_event(&self, timeout: Duration) -> Result, SidecarError> { + match self { + Self::Javascript(execution) => execution + .poll_event(timeout) + .map(|event| { + event.map(|event| match event { + JavascriptExecutionEvent::Stdout(chunk) => { + ActiveExecutionEvent::Stdout(chunk) + } + JavascriptExecutionEvent::Stderr(chunk) => { ActiveExecutionEvent::Stderr(chunk) } + JavascriptExecutionEvent::SyncRpcRequest(request) => { + ActiveExecutionEvent::JavascriptSyncRpcRequest(request) + } + JavascriptExecutionEvent::SignalState { + signal, + registration, + } => ActiveExecutionEvent::SignalState { + signal, + registration: map_node_signal_registration(registration), + }, JavascriptExecutionEvent::Exited(code) => { ActiveExecutionEvent::Exited(code) } @@ -1435,6 +2473,13 @@ impl ActiveExecution { event.map(|event| match event { WasmExecutionEvent::Stdout(chunk) => ActiveExecutionEvent::Stdout(chunk), WasmExecutionEvent::Stderr(chunk) => ActiveExecutionEvent::Stderr(chunk), + WasmExecutionEvent::SignalState { + signal, + registration, + } => ActiveExecutionEvent::SignalState { + signal, + registration: map_wasm_signal_registration(registration), + }, WasmExecutionEvent::Exited(code) => ActiveExecutionEvent::Exited(code), }) }) @@ -1655,7 +2700,22 @@ where payload: crate::protocol::AuthenticateRequest, ) -> Result { let _ = self.connection_id_for(&request.ownership)?; - self.validate_auth_token(&payload.auth_token)?; + if let Err(error) = self.validate_auth_token(&payload.auth_token) { + let mut fields = audit_fields([ + (String::from("source"), payload.client_name.clone()), + (String::from("reason"), error.to_string()), + ]); + if let OwnershipScope::Connection { connection_id } = &request.ownership { + fields.insert(String::from("connection_id"), connection_id.clone()); + } + emit_security_audit_event( + &self.bridge, + &self.config.sidecar_id, + "security.auth.failed", + fields, + ); + return Err(error); + } let connection_id = self.allocate_connection_id(); self.connections.insert( @@ -1728,10 +2788,13 @@ where self.next_vm_id += 1; let vm_id = format!("vm-{}", self.next_vm_id); - let permissions = bridge_permissions(self.bridge.clone(), &vm_id); let cwd = resolve_cwd(payload.metadata.get("cwd"))?; - let guest_env = filter_env(&vm_id, &extract_guest_env(&payload.metadata), &permissions); let resource_limits = parse_resource_limits(&payload.metadata)?; + let dns = parse_vm_dns_config(&payload.metadata)?; + self.bridge + .set_vm_permissions(&vm_id, &payload.permissions)?; + let permissions = bridge_permissions(self.bridge.clone(), &vm_id); + let guest_env = filter_env(&vm_id, &extract_guest_env(&payload.metadata), &permissions); let loaded_snapshot = self.bridge.with_mut(|bridge| { bridge.load_filesystem_state(LoadFilesystemStateRequest { vm_id: vm_id.clone(), @@ -1776,12 +2839,15 @@ where connection_id: connection_id.clone(), session_id: session_id.clone(), metadata: payload.metadata, + dns, guest_env, requested_runtime: payload.runtime, cwd, kernel, loaded_snapshot, configuration: VmConfiguration::default(), + command_guest_paths: BTreeMap::new(), + command_permissions: BTreeMap::new(), active_processes: BTreeMap::new(), signal_states: BTreeMap::new(), }, @@ -1870,13 +2936,32 @@ where vm_id: vm_id.clone(), }, )?; + vm.command_guest_paths = discover_command_guest_paths(&mut vm.kernel); + let mut execution_commands = vec![ + String::from(JAVASCRIPT_COMMAND), + String::from(PYTHON_COMMAND), + String::from(WASM_COMMAND), + ]; + execution_commands.extend(vm.command_guest_paths.keys().cloned()); + vm.kernel + .register_driver(CommandDriver::new( + EXECUTION_DRIVER_NAME, + execution_commands, + )) + .map_err(kernel_error)?; + vm.command_permissions = payload.command_permissions.clone(); vm.configuration = VmConfiguration { mounts: payload.mounts.clone(), software: payload.software.clone(), permissions: payload.permissions.clone(), instructions: payload.instructions.clone(), projected_modules: payload.projected_modules.clone(), + command_permissions: payload.command_permissions.clone(), }; + if !payload.permissions.is_empty() { + self.bridge + .set_vm_permissions(&vm_id, &payload.permissions)?; + } Ok(DispatchResult { response: self.respond( @@ -2253,18 +3338,12 @@ where }; let mut env = vm.guest_env.clone(); env.extend(payload.env.clone()); - let cwd = payload - .cwd - .as_ref() - .map(|cwd| { - let candidate = PathBuf::from(cwd); - if candidate.is_absolute() { - candidate - } else { - vm.cwd.join(candidate) - } - }) - .unwrap_or_else(|| vm.cwd.clone()); + let sandbox_root = normalize_host_path(&vm.cwd); + let cwd = resolve_execution_cwd(vm, payload.cwd.as_deref())?; + env.insert( + String::from(EXECUTION_SANDBOX_ROOT_ENV), + sandbox_root.to_string_lossy().into_owned(), + ); let argv = std::iter::once(payload.entrypoint.clone()) .chain(payload.args.iter().cloned()) .collect::>(); @@ -2306,8 +3385,10 @@ where } GuestRuntimeKind::Python => { let python_file_path = python_file_entrypoint(&payload.entrypoint); - let pyodide_dist_path = - self.python_engine.bundled_pyodide_dist_path().to_path_buf(); + let pyodide_dist_path = self + .python_engine + .bundled_pyodide_dist_path_for_vm(&vm_id) + .map_err(python_error)?; let context = self .python_engine .create_context(CreatePythonContextRequest { @@ -2328,6 +3409,13 @@ where ActiveExecution::Python(execution) } GuestRuntimeKind::WebAssembly => { + apply_wasm_limit_env(&mut env, vm.kernel.resource_limits()); + let wasm_permission_tier = resolve_wasm_permission_tier( + vm, + None, + payload.wasm_permission_tier, + &payload.entrypoint, + ); let context = self.wasm_engine.create_context(CreateWasmContextRequest { vm_id: vm_id.clone(), module_path: Some(payload.entrypoint.clone()), @@ -2340,6 +3428,7 @@ where argv: payload.args.clone(), env, cwd, + permission_tier: execution_wasm_permission_tier(wasm_permission_tier), }) .map_err(wasm_error)?; ActiveExecution::Wasm(execution) @@ -2349,12 +3438,12 @@ where vm.active_processes.insert( payload.process_id.clone(), - ActiveProcess { - kernel_pid: kernel_handle.pid(), + ActiveProcess::new( + kernel_handle.pid(), kernel_handle, - runtime: payload.runtime, + payload.runtime, execution, - }, + ), ); self.bridge.emit_lifecycle(&vm_id, LifecycleState::Busy)?; @@ -2623,6 +3712,10 @@ where snapshot, }) })?; + self.bridge.clear_vm_permissions(vm_id)?; + self.javascript_engine.dispose_vm(vm_id); + self.python_engine.dispose_vm(vm_id); + self.wasm_engine.dispose_vm(vm_id); if let Some(session) = self.sessions.get_mut(session_id) { session.vm_ids.remove(vm_id); @@ -2718,6 +3811,7 @@ where process_id: &str, signal: &str, ) -> Result<(), SidecarError> { + let signal_name = signal.to_owned(); let signal = parse_signal(signal)?; let vm = self .vms @@ -2728,6 +3822,22 @@ where })?; signal_runtime_process(process.execution.child_pid(), signal)?; + emit_security_audit_event( + &self.bridge, + vm_id, + "security.process.kill", + audit_fields([ + (String::from("source"), String::from("control_plane")), + (String::from("source_pid"), String::from("0")), + (String::from("target_pid"), process.kernel_pid.to_string()), + (String::from("process_id"), process_id.to_owned()), + (String::from("signal"), signal_name), + ( + String::from("host_pid"), + process.execution.child_pid().to_string(), + ), + ]), + ); Ok(()) } @@ -2785,32 +3895,41 @@ where chunk: String::from_utf8_lossy(&chunk).into_owned(), }), ))), - ActiveExecutionEvent::Stderr(chunk) => { - if self.record_signal_state_from_control(vm_id, process_id, &chunk)? { - return Ok(None); - } - - Ok(Some(EventFrame::new( - ownership, - EventPayload::ProcessOutput(ProcessOutputEvent { - process_id: process_id.to_owned(), - channel: StreamChannel::Stderr, - chunk: String::from_utf8_lossy(&chunk).into_owned(), - }), - ))) + ActiveExecutionEvent::Stderr(chunk) => Ok(Some(EventFrame::new( + ownership, + EventPayload::ProcessOutput(ProcessOutputEvent { + process_id: process_id.to_owned(), + channel: StreamChannel::Stderr, + chunk: String::from_utf8_lossy(&chunk).into_owned(), + }), + ))), + ActiveExecutionEvent::JavascriptSyncRpcRequest(request) => { + self.handle_javascript_sync_rpc_request(vm_id, process_id, request)?; + Ok(None) } ActiveExecutionEvent::PythonVfsRpcRequest(request) => { self.handle_python_vfs_rpc_request(vm_id, process_id, request)?; Ok(None) } + ActiveExecutionEvent::SignalState { + signal, + registration, + } => { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + vm.signal_states + .entry(process_id.to_owned()) + .or_default() + .insert(signal, registration); + Ok(None) + } ActiveExecutionEvent::Exited(exit_code) => { let became_idle = { let vm = self.vms.get_mut(vm_id).expect("VM should exist"); - let process = vm + let mut process = vm .active_processes .remove(process_id) .expect("process should still exist"); - vm.signal_states.remove(process_id); + terminate_child_process_tree(&mut vm.kernel, &mut process); process.kernel_handle.finish(exit_code); let _ = vm.kernel.wait_and_reap(process.kernel_pid); vm.active_processes.is_empty() @@ -2837,59 +3956,64 @@ where process_id: &str, request: PythonVfsRpcRequest, ) -> Result<(), SidecarError> { - let response = { - let vm = self.vms.get_mut(vm_id).expect("VM should exist"); - match request.method { - PythonVfsRpcMethod::Read => vm - .kernel - .read_file(&request.path) - .map(|content| PythonVfsRpcResponsePayload::Read { - content_base64: base64::engine::general_purpose::STANDARD.encode(content), - }) - .map_err(kernel_error), - PythonVfsRpcMethod::Write => { - let content_base64 = request.content_base64.as_deref().ok_or_else(|| { - SidecarError::InvalidState(format!( - "python VFS fsWrite for {} requires contentBase64", - request.path - )) - })?; - let bytes = base64::engine::general_purpose::STANDARD - .decode(content_base64) - .map_err(|error| { - SidecarError::InvalidState(format!( - "invalid base64 python VFS content for {}: {error}", - request.path - )) - })?; - vm.kernel - .write_file(&request.path, bytes) + let response = match normalize_python_vfs_rpc_path(&request.path) { + Ok(path) => { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + match request.method { + PythonVfsRpcMethod::Read => vm + .kernel + .read_file(&path) + .map(|content| PythonVfsRpcResponsePayload::Read { + content_base64: base64::engine::general_purpose::STANDARD + .encode(content), + }) + .map_err(kernel_error), + PythonVfsRpcMethod::Write => { + let content_base64 = + request.content_base64.as_deref().ok_or_else(|| { + SidecarError::InvalidState(format!( + "python VFS fsWrite for {} requires contentBase64", + path + )) + })?; + let bytes = base64::engine::general_purpose::STANDARD + .decode(content_base64) + .map_err(|error| { + SidecarError::InvalidState(format!( + "invalid base64 python VFS content for {}: {error}", + path + )) + })?; + vm.kernel + .write_file(&path, bytes) + .map(|()| PythonVfsRpcResponsePayload::Empty) + .map_err(kernel_error) + } + PythonVfsRpcMethod::Stat => vm + .kernel + .stat(&path) + .map(|stat| PythonVfsRpcResponsePayload::Stat { + stat: PythonVfsRpcStat { + mode: stat.mode, + size: stat.size, + is_directory: stat.is_directory, + is_symbolic_link: stat.is_symbolic_link, + }, + }) + .map_err(kernel_error), + PythonVfsRpcMethod::ReadDir => vm + .kernel + .read_dir(&path) + .map(|entries| PythonVfsRpcResponsePayload::ReadDir { entries }) + .map_err(kernel_error), + PythonVfsRpcMethod::Mkdir => vm + .kernel + .mkdir(&path, request.recursive) .map(|()| PythonVfsRpcResponsePayload::Empty) - .map_err(kernel_error) + .map_err(kernel_error), } - PythonVfsRpcMethod::Stat => vm - .kernel - .stat(&request.path) - .map(|stat| PythonVfsRpcResponsePayload::Stat { - stat: PythonVfsRpcStat { - mode: stat.mode, - size: stat.size, - is_directory: stat.is_directory, - is_symbolic_link: stat.is_symbolic_link, - }, - }) - .map_err(kernel_error), - PythonVfsRpcMethod::ReadDir => vm - .kernel - .read_dir(&request.path) - .map(|entries| PythonVfsRpcResponsePayload::ReadDir { entries }) - .map_err(kernel_error), - PythonVfsRpcMethod::Mkdir => vm - .kernel - .mkdir(&request.path, request.recursive) - .map(|()| PythonVfsRpcResponsePayload::Empty) - .map_err(kernel_error), } + Err(error) => Err(error), }; let vm = self.vms.get_mut(vm_id).expect("VM should exist"); @@ -2910,31 +4034,580 @@ where } } - fn record_signal_state_from_control( + fn resolve_javascript_child_process_execution( + &self, + vm: &VmState, + request: &JavascriptChildProcessSpawnRequest, + ) -> Result { + let guest_cwd = normalize_path(request.options.cwd.as_deref().unwrap_or("/")); + let host_cwd = host_mount_path_for_guest_path(vm, &guest_cwd).unwrap_or_else(|| { + let candidate = PathBuf::from(&guest_cwd); + if candidate.is_absolute() { + candidate + } else { + vm.cwd.clone() + } + }); + let mut env = vm.guest_env.clone(); + env.extend(request.options.env.clone()); + + let (command, process_args) = if request.options.shell { + if vm.command_guest_paths.contains_key("sh") { + ( + String::from("sh"), + vec![String::from("-c"), request.command.clone()], + ) + } else { + let tokens = tokenize_shell_free_command(&request.command); + let Some((command, args)) = tokens.split_first() else { + return Err(SidecarError::InvalidState(String::from( + "child_process shell command must not be empty", + ))); + }; + (command.clone(), args.to_vec()) + } + } else { + (request.command.clone(), request.args.clone()) + }; + + if matches!(command.as_str(), "node" | "npm" | "npx") { + let Some(entrypoint_specifier) = process_args.first() else { + return Err(SidecarError::InvalidState(format!( + "{command} child_process spawn requires an entrypoint" + ))); + }; + + let entrypoint = if is_path_like_specifier(entrypoint_specifier) { + let guest_entrypoint = if entrypoint_specifier.starts_with('/') { + normalize_path(entrypoint_specifier) + } else if entrypoint_specifier.starts_with("file:") { + normalize_path(entrypoint_specifier.trim_start_matches("file:")) + } else { + normalize_path(&format!("{guest_cwd}/{entrypoint_specifier}")) + }; + let host_entrypoint = if entrypoint_specifier.starts_with("./") + || entrypoint_specifier.starts_with("../") + { + host_cwd.join(entrypoint_specifier) + } else { + host_mount_path_for_guest_path(vm, &guest_entrypoint).unwrap_or_else(|| { + let candidate = PathBuf::from(&guest_entrypoint); + if candidate.is_absolute() { + candidate + } else { + host_cwd.join(&guest_entrypoint) + } + }) + }; + env.insert(String::from("AGENT_OS_GUEST_ENTRYPOINT"), guest_entrypoint); + host_entrypoint.to_string_lossy().into_owned() + } else { + entrypoint_specifier.clone() + }; + + return Ok(ResolvedChildProcessExecution { + command, + process_args: process_args.clone(), + runtime: GuestRuntimeKind::JavaScript, + entrypoint, + execution_args: process_args.iter().skip(1).cloned().collect(), + env, + guest_cwd, + host_cwd, + wasm_permission_tier: None, + }); + } + + if command == PYTHON_COMMAND { + return Err(SidecarError::InvalidState(String::from( + "nested python child_process execution is not supported yet", + ))); + } + + let guest_entrypoint = vm + .command_guest_paths + .get(&command) + .ok_or_else(|| SidecarError::InvalidState(format!("command not found: {command}")))?; + let host_entrypoint = + host_mount_path_for_guest_path(vm, guest_entrypoint).unwrap_or_else(|| { + let candidate = PathBuf::from(guest_entrypoint); + if candidate.is_absolute() { + candidate + } else { + host_cwd.join(guest_entrypoint) + } + }); + let wasm_permission_tier = vm.command_permissions.get(&command).copied(); + + Ok(ResolvedChildProcessExecution { + command, + process_args: process_args.clone(), + runtime: GuestRuntimeKind::WebAssembly, + entrypoint: host_entrypoint.to_string_lossy().into_owned(), + execution_args: process_args, + env, + guest_cwd, + host_cwd, + wasm_permission_tier, + }) + } + + fn spawn_javascript_child_process( &mut self, vm_id: &str, process_id: &str, - chunk: &[u8], - ) -> Result { - let text = String::from_utf8_lossy(chunk); - let trimmed = text.trim(); - let Some(payload) = trimmed.strip_prefix(SIGNAL_STATE_CONTROL_PREFIX) else { - return Ok(false); + request: JavascriptChildProcessSpawnRequest, + ) -> Result { + let resolved = { + let vm = self.vms.get(vm_id).expect("VM should exist"); + self.resolve_javascript_child_process_execution(vm, &request)? }; - let registration: SignalControlMessage = - serde_json::from_str(payload).map_err(|error| { - SidecarError::InvalidState(format!( - "invalid signal-state control payload for process {process_id}: {error}" - )) + let (parent_kernel_pid, child_process_id) = { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let process = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist"); + (process.kernel_pid, process.allocate_child_process_id()) + }; + + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let kernel_handle = vm + .kernel + .spawn_process( + &resolved.command, + resolved.process_args.clone(), + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + parent_pid: Some(parent_kernel_pid), + env: resolved.env.clone(), + cwd: Some(resolved.guest_cwd.clone()), + }, + ) + .map_err(kernel_error)?; + let kernel_pid = kernel_handle.pid(); + + let mut execution_env = resolved.env.clone(); + + let execution = match resolved.runtime { + GuestRuntimeKind::JavaScript => { + execution_env.extend(sanitize_javascript_child_process_internal_bootstrap_env( + &request.options.internal_bootstrap_env, + )); + execution_env.insert( + String::from("AGENT_OS_VIRTUAL_PROCESS_PID"), + kernel_pid.to_string(), + ); + execution_env.insert( + String::from("AGENT_OS_VIRTUAL_PROCESS_PPID"), + parent_kernel_pid.to_string(), + ); + let context = + self.javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.to_owned(), + bootstrap_module: None, + compile_cache_root: Some(self.cache_root.join("node-compile-cache")), + }); + let execution = self + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.to_owned(), + context_id: context.context_id, + argv: std::iter::once(resolved.entrypoint.clone()) + .chain(resolved.execution_args.clone()) + .collect(), + env: execution_env, + cwd: resolved.host_cwd.clone(), + }) + .map_err(javascript_error)?; + ActiveExecution::Javascript(execution) + } + GuestRuntimeKind::WebAssembly => { + apply_wasm_limit_env(&mut execution_env, vm.kernel.resource_limits()); + let context = self.wasm_engine.create_context(CreateWasmContextRequest { + vm_id: vm_id.to_owned(), + module_path: Some(resolved.entrypoint.clone()), + }); + let execution = self + .wasm_engine + .start_execution(StartWasmExecutionRequest { + vm_id: vm_id.to_owned(), + context_id: context.context_id, + argv: resolved.execution_args.clone(), + env: execution_env, + cwd: resolved.host_cwd.clone(), + permission_tier: execution_wasm_permission_tier( + resolved + .wasm_permission_tier + .unwrap_or(WasmPermissionTier::Full), + ), + }) + .map_err(wasm_error)?; + ActiveExecution::Wasm(execution) + } + GuestRuntimeKind::Python => unreachable!("python child_process execution is rejected"), + }; + + vm.active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .insert( + child_process_id.clone(), + ActiveProcess::new(kernel_pid, kernel_handle, resolved.runtime, execution), + ); + + Ok(json!({ + "childId": child_process_id, + "pid": kernel_pid, + "command": resolved.command, + "args": resolved.process_args, + })) + } + + fn poll_javascript_child_process( + &mut self, + vm_id: &str, + process_id: &str, + child_process_id: &str, + wait_ms: u64, + ) -> Result { + loop { + let event = { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .get_mut(child_process_id) + .ok_or_else(|| { + SidecarError::InvalidState(format!( + "unknown child process {child_process_id}" + )) + })?; + child + .execution + .poll_event(Duration::from_millis(wait_ms)) + .map_err(|error| SidecarError::Execution(error.to_string()))? + }; + + let Some(event) = event else { + return Ok(Value::Null); + }; + + match event { + ActiveExecutionEvent::Stdout(chunk) => { + return Ok(json!({ + "type": "stdout", + "data": javascript_sync_rpc_bytes_value(&chunk), + })); + } + ActiveExecutionEvent::Stderr(chunk) => { + return Ok(json!({ + "type": "stderr", + "data": javascript_sync_rpc_bytes_value(&chunk), + })); + } + ActiveExecutionEvent::Exited(exit_code) => { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let parent_runtime_pid = vm + .active_processes + .get(process_id) + .expect("process should still exist") + .execution + .child_pid(); + let should_signal_parent = vm + .signal_states + .get(process_id) + .and_then(|handlers| handlers.get(&(libc::SIGCHLD as u32))) + .is_some_and(|registration| { + registration.action != SignalDispositionAction::Default + }); + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .remove(child_process_id) + .expect("child process should still exist"); + child.kernel_handle.finish(exit_code); + let _ = vm.kernel.wait_and_reap(child.kernel_pid); + if should_signal_parent { + signal_runtime_process(parent_runtime_pid, libc::SIGCHLD)?; + } + return Ok(json!({ + "type": "exit", + "exitCode": exit_code, + })); + } + ActiveExecutionEvent::JavascriptSyncRpcRequest(request) => { + let response = { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + if request.method.starts_with("child_process.") { + Err(SidecarError::InvalidState(String::from( + "nested child_process calls from a child process are not supported yet", + ))) + } else { + let resource_limits = vm.kernel.resource_limits().clone(); + let network_counts = vm_network_resource_counts(vm); + let socket_paths = build_javascript_socket_path_context(vm)?; + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .get_mut(child_process_id) + .expect("child process should still exist"); + service_javascript_sync_rpc( + &self.bridge, + vm_id, + &vm.dns, + &socket_paths, + &mut vm.kernel, + child, + &request, + &resource_limits, + network_counts, + ) + } + }; + + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .get_mut(child_process_id) + .expect("child process should still exist"); + match response { + Ok(result) => child + .execution + .respond_javascript_sync_rpc_success(request.id, result) + .or_else(ignore_stale_javascript_sync_rpc_response)?, + Err(error) => child + .execution + .respond_javascript_sync_rpc_error( + request.id, + "ERR_AGENT_OS_NODE_SYNC_RPC", + error.to_string(), + ) + .or_else(ignore_stale_javascript_sync_rpc_response)?, + } + } + ActiveExecutionEvent::PythonVfsRpcRequest(_) => { + return Err(SidecarError::InvalidState(String::from( + "nested Python child_process execution is not supported yet", + ))); + } + ActiveExecutionEvent::SignalState { .. } => {} + } + } + } + + fn write_javascript_child_process_stdin( + &mut self, + vm_id: &str, + process_id: &str, + child_process_id: &str, + chunk: &[u8], + ) -> Result<(), SidecarError> { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .get_mut(child_process_id) + .ok_or_else(|| { + SidecarError::InvalidState(format!("unknown child process {child_process_id}")) })?; + child.execution.write_stdin(chunk) + } + fn close_javascript_child_process_stdin( + &mut self, + vm_id: &str, + process_id: &str, + child_process_id: &str, + ) -> Result<(), SidecarError> { let vm = self.vms.get_mut(vm_id).expect("VM should exist"); - vm.signal_states - .entry(process_id.to_owned()) - .or_default() - .insert(registration.signal, registration.registration); - Ok(true) + let child = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist") + .child_processes + .get_mut(child_process_id) + .ok_or_else(|| { + SidecarError::InvalidState(format!("unknown child process {child_process_id}")) + })?; + child.execution.close_stdin() + } + + fn kill_javascript_child_process( + &mut self, + vm_id: &str, + process_id: &str, + child_process_id: &str, + signal: &str, + ) -> Result<(), SidecarError> { + let signal_name = signal.to_owned(); + let signal = parse_signal(signal)?; + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let process = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist"); + let source_pid = process.kernel_pid; + let child = process + .child_processes + .get_mut(child_process_id) + .ok_or_else(|| { + SidecarError::InvalidState(format!("unknown child process {child_process_id}")) + })?; + vm.kernel + .kill_process(EXECUTION_DRIVER_NAME, child.kernel_pid, signal) + .map_err(kernel_error)?; + emit_security_audit_event( + &self.bridge, + vm_id, + "security.process.kill", + audit_fields([ + (String::from("source"), String::from("guest_child_process")), + (String::from("source_pid"), source_pid.to_string()), + (String::from("target_pid"), child.kernel_pid.to_string()), + (String::from("process_id"), process_id.to_owned()), + ( + String::from("child_process_id"), + child_process_id.to_owned(), + ), + (String::from("signal"), signal_name), + ]), + ); + Ok(()) + } + + fn handle_javascript_sync_rpc_request( + &mut self, + vm_id: &str, + process_id: &str, + request: JavascriptSyncRpcRequest, + ) -> Result<(), SidecarError> { + let response: Result = match request.method.as_str() { + "child_process.spawn" => { + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "child_process.spawn requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err( + |error| { + SidecarError::InvalidState(format!( + "invalid child_process.spawn payload: {error}" + )) + }, + ) + })?; + self.spawn_javascript_child_process(vm_id, process_id, payload) + } + "child_process.poll" => { + let child_process_id = + javascript_sync_rpc_arg_str(&request.args, 0, "child_process.poll child id")?; + let wait_ms = javascript_sync_rpc_arg_u64_optional( + &request.args, + 1, + "child_process.poll wait ms", + )? + .unwrap_or_default(); + self.poll_javascript_child_process(vm_id, process_id, child_process_id, wait_ms) + } + "child_process.write_stdin" => { + let child_process_id = javascript_sync_rpc_arg_str( + &request.args, + 0, + "child_process.write_stdin child id", + )?; + let chunk = javascript_sync_rpc_bytes_arg( + &request.args, + 1, + "child_process.write_stdin chunk", + )?; + self.write_javascript_child_process_stdin( + vm_id, + process_id, + child_process_id, + &chunk, + )?; + Ok(Value::Null) + } + "child_process.close_stdin" => { + let child_process_id = javascript_sync_rpc_arg_str( + &request.args, + 0, + "child_process.close_stdin child id", + )?; + self.close_javascript_child_process_stdin(vm_id, process_id, child_process_id)?; + Ok(Value::Null) + } + "child_process.kill" => { + let child_process_id = + javascript_sync_rpc_arg_str(&request.args, 0, "child_process.kill child id")?; + let signal = + javascript_sync_rpc_arg_str(&request.args, 1, "child_process.kill signal")?; + self.kill_javascript_child_process(vm_id, process_id, child_process_id, signal)?; + Ok(Value::Null) + } + _ => { + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let resource_limits = vm.kernel.resource_limits().clone(); + let network_counts = vm_network_resource_counts(vm); + let socket_paths = build_javascript_socket_path_context(vm)?; + let process = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist"); + service_javascript_sync_rpc( + &self.bridge, + vm_id, + &vm.dns, + &socket_paths, + &mut vm.kernel, + process, + &request, + &resource_limits, + network_counts, + ) + } + }; + + let vm = self.vms.get_mut(vm_id).expect("VM should exist"); + let process = vm + .active_processes + .get_mut(process_id) + .expect("process should still exist"); + + match response { + Ok(result) => process + .execution + .respond_javascript_sync_rpc_success(request.id, result) + .or_else(ignore_stale_javascript_sync_rpc_response), + Err(error) => process + .execution + .respond_javascript_sync_rpc_error( + request.id, + javascript_sync_rpc_error_code(&error), + error.to_string(), + ) + .or_else(ignore_stale_javascript_sync_rpc_response), + } } fn vm_ids_for_scope(&self, ownership: &OwnershipScope) -> Result, SidecarError> { @@ -3170,12 +4843,120 @@ fn map_bridge_permission(decision: agent_os_bridge::PermissionDecision) -> Permi } } -fn bridge_permissions(bridge: SharedBridge, vm_id: &str) -> Permissions +fn audit_timestamp() -> String { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("system time before unix epoch") + .as_millis() + .to_string() +} + +fn audit_fields(fields: I) -> BTreeMap +where + I: IntoIterator, + K: Into, + V: Into, +{ + let mut mapped = BTreeMap::from([(String::from("timestamp"), audit_timestamp())]); + for (key, value) in fields { + mapped.insert(key.into(), value.into()); + } + mapped +} + +fn emit_structured_event( + bridge: &SharedBridge, + vm_id: &str, + name: &str, + fields: BTreeMap, +) -> Result<(), SidecarError> where B: NativeSidecarBridge + Send + 'static, BridgeError: fmt::Debug + Send + Sync + 'static, { - let vm_id = vm_id.to_owned(); + bridge.with_mut(|bridge| { + bridge.emit_structured_event(StructuredEventRecord { + vm_id: vm_id.to_owned(), + name: name.to_owned(), + fields, + }) + }) +} + +fn emit_security_audit_event( + bridge: &SharedBridge, + vm_id: &str, + name: &str, + fields: BTreeMap, +) where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + let _ = emit_structured_event(bridge, vm_id, name, fields); +} + +fn filesystem_operation_label(operation: FsOperation) -> &'static str { + match operation { + FsOperation::Read => "read", + FsOperation::Write => "write", + FsOperation::Mkdir => "mkdir", + FsOperation::CreateDir => "createDir", + FsOperation::ReadDir => "readdir", + FsOperation::Stat => "stat", + FsOperation::Remove => "rm", + FsOperation::Rename => "rename", + FsOperation::Exists => "exists", + FsOperation::Symlink => "symlink", + FsOperation::ReadLink => "readlink", + FsOperation::Link => "link", + FsOperation::Chmod => "chmod", + FsOperation::Chown => "chown", + FsOperation::Utimes => "utimes", + FsOperation::Truncate => "truncate", + FsOperation::MountSensitive => "mount", + } +} + +fn map_wasm_signal_registration( + registration: agent_os_execution::wasm::WasmSignalHandlerRegistration, +) -> SignalHandlerRegistration { + SignalHandlerRegistration { + action: match registration.action { + agent_os_execution::wasm::WasmSignalDispositionAction::Default => { + crate::protocol::SignalDispositionAction::Default + } + agent_os_execution::wasm::WasmSignalDispositionAction::Ignore => { + crate::protocol::SignalDispositionAction::Ignore + } + agent_os_execution::wasm::WasmSignalDispositionAction::User => { + crate::protocol::SignalDispositionAction::User + } + }, + mask: registration.mask, + flags: registration.flags, + } +} + +fn map_node_signal_registration( + registration: NodeSignalHandlerRegistration, +) -> SignalHandlerRegistration { + SignalHandlerRegistration { + action: match registration.action { + NodeSignalDispositionAction::Default => SignalDispositionAction::Default, + NodeSignalDispositionAction::Ignore => SignalDispositionAction::Ignore, + NodeSignalDispositionAction::User => SignalDispositionAction::User, + }, + mask: registration.mask, + flags: registration.flags, + } +} + +fn bridge_permissions(bridge: SharedBridge, vm_id: &str) -> Permissions +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + let vm_id = vm_id.to_owned(); let filesystem_bridge = bridge.clone(); let filesystem_vm_id = vm_id.clone(); @@ -3187,26 +4968,60 @@ where Permissions { filesystem: Some(Arc::new(move |request: &FsAccessRequest| { - filesystem_bridge.filesystem_decision( - &filesystem_vm_id, - &request.path, - match request.op { - FsOperation::Read => FilesystemAccess::Read, - FsOperation::Write => FilesystemAccess::Write, - FsOperation::Mkdir | FsOperation::CreateDir => FilesystemAccess::CreateDir, - FsOperation::ReadDir => FilesystemAccess::ReadDir, - FsOperation::Stat | FsOperation::Exists => FilesystemAccess::Stat, - FsOperation::Remove => FilesystemAccess::Remove, - FsOperation::Rename => FilesystemAccess::Rename, - FsOperation::Symlink => FilesystemAccess::Symlink, - FsOperation::ReadLink => FilesystemAccess::Read, - FsOperation::Link => FilesystemAccess::Write, - FsOperation::Chmod => FilesystemAccess::Write, - FsOperation::Chown => FilesystemAccess::Write, - FsOperation::Utimes => FilesystemAccess::Write, - FsOperation::Truncate => FilesystemAccess::Write, - }, - ) + let access = match request.op { + FsOperation::Read => FilesystemAccess::Read, + FsOperation::Write => FilesystemAccess::Write, + FsOperation::Mkdir | FsOperation::CreateDir => FilesystemAccess::CreateDir, + FsOperation::ReadDir => FilesystemAccess::ReadDir, + FsOperation::Stat | FsOperation::Exists => FilesystemAccess::Stat, + FsOperation::Remove => FilesystemAccess::Remove, + FsOperation::Rename => FilesystemAccess::Rename, + FsOperation::Symlink => FilesystemAccess::Symlink, + FsOperation::ReadLink => FilesystemAccess::Read, + FsOperation::Link => FilesystemAccess::Write, + FsOperation::Chmod => FilesystemAccess::Write, + FsOperation::Chown => FilesystemAccess::Write, + FsOperation::Utimes => FilesystemAccess::Write, + FsOperation::Truncate => FilesystemAccess::Write, + FsOperation::MountSensitive => FilesystemAccess::Write, + }; + let policy = if request.op == FsOperation::MountSensitive { + "fs.mount_sensitive" + } else { + filesystem_permission_capability(access) + }; + let decision = if request.op == FsOperation::MountSensitive { + filesystem_bridge + .static_permission_decision(&filesystem_vm_id, policy, "fs") + .unwrap_or_else(PermissionDecision::allow) + } else { + filesystem_bridge.filesystem_decision(&filesystem_vm_id, &request.path, access) + }; + + if !decision.allow { + emit_security_audit_event( + &filesystem_bridge, + &filesystem_vm_id, + "security.permission.denied", + audit_fields([ + ( + String::from("operation"), + filesystem_operation_label(request.op).to_owned(), + ), + (String::from("path"), request.path.clone()), + (String::from("policy"), String::from(policy)), + ( + String::from("reason"), + decision + .reason + .clone() + .unwrap_or_else(|| String::from("permission denied")), + ), + ]), + ); + } + + decision })), network: Some(Arc::new(move |request: &NetworkAccessRequest| { network_bridge.network_decision(&network_vm_id, request) @@ -3255,10 +5070,19 @@ where BridgeError: fmt::Debug + Send + Sync + 'static, { for existing in &vm.configuration.mounts { - if let Err(error) = vm.kernel.unmount_filesystem(&existing.guest_path) { - if error.code() != "EINVAL" { - return Err(kernel_error(error)); - } + match vm.kernel.unmount_filesystem(&existing.guest_path) { + Ok(()) => emit_security_audit_event( + &context.bridge, + &context.vm_id, + "security.mount.unmounted", + audit_fields([ + (String::from("guest_path"), existing.guest_path.clone()), + (String::from("plugin_id"), existing.plugin.id.clone()), + (String::from("read_only"), existing.read_only.to_string()), + ]), + ), + Err(error) if error.code() == "EINVAL" => {} + Err(error) => return Err(kernel_error(error)), } } @@ -3283,6 +5107,16 @@ where MountOptions::new(mount.plugin.id.clone()).read_only(mount.read_only), ) .map_err(kernel_error)?; + emit_security_audit_event( + &context.bridge, + &context.vm_id, + "security.mount.mounted", + audit_fields([ + (String::from("guest_path"), mount.guest_path.clone()), + (String::from("plugin_id"), mount.plugin.id.clone()), + (String::from("read_only"), mount.read_only.to_string()), + ]), + ); } Ok(()) @@ -3309,6 +5143,32 @@ fn resolve_cwd(value: Option<&String>) -> Result { } } +fn resolve_execution_cwd(vm: &VmState, value: Option<&str>) -> Result { + let sandbox_root = normalize_host_path(&vm.cwd); + let candidate = match value { + Some(path) => { + let path = PathBuf::from(path); + if path.is_absolute() { + path + } else { + sandbox_root.join(path) + } + } + None => sandbox_root.clone(), + }; + let normalized = normalize_host_path(&candidate); + + if !path_is_within_root(&normalized, &sandbox_root) { + return Err(SidecarError::InvalidState(format!( + "execute cwd {} escapes VM sandbox root {}", + normalized.display(), + sandbox_root.display() + ))); + } + + Ok(normalized) +} + fn extract_guest_env(metadata: &BTreeMap) -> BTreeMap { metadata .iter() @@ -3319,15 +5179,81 @@ fn extract_guest_env(metadata: &BTreeMap) -> BTreeMap, limits: &ResourceLimits) { + if let Some(limit) = limits.max_wasm_fuel { + env.insert(String::from(WASM_MAX_FUEL_ENV), limit.to_string()); + } + if let Some(limit) = limits.max_wasm_memory_bytes { + env.insert(String::from(WASM_MAX_MEMORY_BYTES_ENV), limit.to_string()); + } + if let Some(limit) = limits.max_wasm_stack_bytes { + env.insert(String::from(WASM_MAX_STACK_BYTES_ENV), limit.to_string()); + } +} + fn parse_resource_limits( metadata: &BTreeMap, ) -> Result { - Ok(ResourceLimits { - max_processes: parse_resource_limit(metadata, "resource.max_processes")?, - max_open_fds: parse_resource_limit(metadata, "resource.max_open_fds")?, - max_pipes: parse_resource_limit(metadata, "resource.max_pipes")?, - max_ptys: parse_resource_limit(metadata, "resource.max_ptys")?, - }) + let mut limits = ResourceLimits::default(); + if metadata.contains_key("resource.max_processes") { + limits.max_processes = parse_resource_limit(metadata, "resource.max_processes")?; + } + if metadata.contains_key("resource.max_open_fds") { + limits.max_open_fds = parse_resource_limit(metadata, "resource.max_open_fds")?; + } + if metadata.contains_key("resource.max_pipes") { + limits.max_pipes = parse_resource_limit(metadata, "resource.max_pipes")?; + } + if metadata.contains_key("resource.max_ptys") { + limits.max_ptys = parse_resource_limit(metadata, "resource.max_ptys")?; + } + if metadata.contains_key("resource.max_sockets") { + limits.max_sockets = parse_resource_limit(metadata, "resource.max_sockets")?; + } + if metadata.contains_key("resource.max_connections") { + limits.max_connections = parse_resource_limit(metadata, "resource.max_connections")?; + } + if metadata.contains_key("resource.max_filesystem_bytes") { + limits.max_filesystem_bytes = + parse_resource_limit_u64(metadata, "resource.max_filesystem_bytes")?; + } + if metadata.contains_key("resource.max_inode_count") { + limits.max_inode_count = parse_resource_limit(metadata, "resource.max_inode_count")?; + } + if metadata.contains_key("resource.max_blocking_read_ms") { + limits.max_blocking_read_ms = + parse_resource_limit_u64(metadata, "resource.max_blocking_read_ms")?; + } + if metadata.contains_key("resource.max_pread_bytes") { + limits.max_pread_bytes = parse_resource_limit(metadata, "resource.max_pread_bytes")?; + } + if metadata.contains_key("resource.max_fd_write_bytes") { + limits.max_fd_write_bytes = parse_resource_limit(metadata, "resource.max_fd_write_bytes")?; + } + if metadata.contains_key("resource.max_process_argv_bytes") { + limits.max_process_argv_bytes = + parse_resource_limit(metadata, "resource.max_process_argv_bytes")?; + } + if metadata.contains_key("resource.max_process_env_bytes") { + limits.max_process_env_bytes = + parse_resource_limit(metadata, "resource.max_process_env_bytes")?; + } + if metadata.contains_key("resource.max_readdir_entries") { + limits.max_readdir_entries = + parse_resource_limit(metadata, "resource.max_readdir_entries")?; + } + if metadata.contains_key("resource.max_wasm_fuel") { + limits.max_wasm_fuel = parse_resource_limit_u64(metadata, "resource.max_wasm_fuel")?; + } + if metadata.contains_key("resource.max_wasm_memory_bytes") { + limits.max_wasm_memory_bytes = + parse_resource_limit_u64(metadata, "resource.max_wasm_memory_bytes")?; + } + if metadata.contains_key("resource.max_wasm_stack_bytes") { + limits.max_wasm_stack_bytes = + parse_resource_limit(metadata, "resource.max_wasm_stack_bytes")?; + } + Ok(limits) } fn parse_resource_limit( @@ -3344,6 +5270,309 @@ fn parse_resource_limit( Ok(Some(parsed)) } +fn parse_resource_limit_u64( + metadata: &BTreeMap, + key: &str, +) -> Result, SidecarError> { + let Some(value) = metadata.get(key) else { + return Ok(None); + }; + + let parsed = value.parse::().map_err(|error| { + SidecarError::InvalidState(format!("invalid resource limit {key}={value}: {error}")) + })?; + Ok(Some(parsed)) +} + +fn parse_vm_dns_config(metadata: &BTreeMap) -> Result { + let mut config = VmDnsConfig::default(); + + if let Some(value) = metadata.get(VM_DNS_SERVERS_METADATA_KEY) { + config.name_servers = value + .split(',') + .map(str::trim) + .filter(|entry| !entry.is_empty()) + .map(parse_vm_dns_nameserver) + .collect::, _>>()?; + } + + for (key, value) in metadata { + let Some(hostname) = key.strip_prefix(VM_DNS_OVERRIDE_METADATA_PREFIX) else { + continue; + }; + let normalized_hostname = normalize_dns_hostname(hostname)?; + let addresses = value + .split(',') + .map(str::trim) + .filter(|entry| !entry.is_empty()) + .map(|entry| { + entry.parse::().map_err(|error| { + SidecarError::InvalidState(format!( + "invalid DNS override {key}={value}: {error}" + )) + }) + }) + .collect::, _>>()?; + if addresses.is_empty() { + return Err(SidecarError::InvalidState(format!( + "DNS override {key} must contain at least one IP address" + ))); + } + config.overrides.insert(normalized_hostname, addresses); + } + + Ok(config) +} + +fn parse_vm_listen_policy( + metadata: &BTreeMap, +) -> Result { + let mut policy = VmListenPolicy::default(); + + if let Some(value) = metadata.get(VM_LISTEN_PORT_MIN_METADATA_KEY) { + policy.port_min = parse_listen_port_metadata(VM_LISTEN_PORT_MIN_METADATA_KEY, value)?; + } + if let Some(value) = metadata.get(VM_LISTEN_PORT_MAX_METADATA_KEY) { + policy.port_max = parse_listen_port_metadata(VM_LISTEN_PORT_MAX_METADATA_KEY, value)?; + } + if policy.port_min > policy.port_max { + return Err(SidecarError::InvalidState(format!( + "invalid listen port range {}={} exceeds {}={}", + VM_LISTEN_PORT_MIN_METADATA_KEY, + policy.port_min, + VM_LISTEN_PORT_MAX_METADATA_KEY, + policy.port_max + ))); + } + if let Some(value) = metadata.get(VM_LISTEN_ALLOW_PRIVILEGED_METADATA_KEY) { + policy.allow_privileged = value.parse::().map_err(|error| { + SidecarError::InvalidState(format!( + "invalid {}={value}: {error}", + VM_LISTEN_ALLOW_PRIVILEGED_METADATA_KEY + )) + })?; + } + + Ok(policy) +} + +fn parse_listen_port_metadata(key: &str, value: &str) -> Result { + let parsed = value + .parse::() + .map_err(|error| SidecarError::InvalidState(format!("invalid {key}={value}: {error}")))?; + if parsed == 0 { + return Err(SidecarError::InvalidState(format!( + "{key} must be between 1 and 65535" + ))); + } + Ok(parsed) +} + +fn parse_loopback_exempt_ports( + env: &BTreeMap, +) -> Result, SidecarError> { + let Some(value) = env.get(LOOPBACK_EXEMPT_PORTS_ENV) else { + return Ok(BTreeSet::new()); + }; + + let parsed = serde_json::from_str::>(value).map_err(|error| { + SidecarError::InvalidState(format!( + "invalid {LOOPBACK_EXEMPT_PORTS_ENV}={value}: {error}" + )) + })?; + + let mut ports = BTreeSet::new(); + for entry in parsed { + let port = match entry { + Value::String(raw) => raw.parse::().map_err(|error| { + SidecarError::InvalidState(format!( + "invalid {LOOPBACK_EXEMPT_PORTS_ENV} entry {raw:?}: {error}" + )) + })?, + Value::Number(raw) => raw + .as_u64() + .and_then(|port| u16::try_from(port).ok()) + .ok_or_else(|| { + SidecarError::InvalidState(format!( + "invalid {LOOPBACK_EXEMPT_PORTS_ENV} entry {raw}" + )) + })?, + other => { + return Err(SidecarError::InvalidState(format!( + "invalid {LOOPBACK_EXEMPT_PORTS_ENV} entry {other:?}" + ))) + } + }; + ports.insert(port); + } + + Ok(ports) +} + +fn parse_vm_dns_nameserver(value: &str) -> Result { + if let Ok(address) = value.parse::() { + return Ok(address); + } + if let Ok(ip) = value.parse::() { + return Ok(SocketAddr::new(ip, 53)); + } + Err(SidecarError::InvalidState(format!( + "invalid {} entry {value}; expected IP or IP:port", + VM_DNS_SERVERS_METADATA_KEY + ))) +} + +fn normalize_dns_hostname(hostname: &str) -> Result { + let normalized = hostname.trim().trim_end_matches('.').to_ascii_lowercase(); + if normalized.is_empty() { + return Err(SidecarError::InvalidState(String::from( + "DNS hostname must not be empty", + ))); + } + Ok(normalized) +} + +fn vm_dns_resolver_config(dns: &VmDnsConfig) -> Option { + if dns.name_servers.is_empty() { + return None; + } + + let name_servers = dns + .name_servers + .iter() + .map(|server| { + let mut config = NameServerConfig::udp_and_tcp(server.ip()); + for connection in &mut config.connections { + connection.port = server.port(); + connection.bind_addr = Some(SocketAddr::new( + if server.is_ipv6() { + IpAddr::V6(Ipv6Addr::UNSPECIFIED) + } else { + IpAddr::V4(Ipv4Addr::UNSPECIFIED) + }, + 0, + )); + } + config + }) + .collect(); + Some(ResolverConfig::from_parts(None, vec![], name_servers)) +} + +fn resolve_dns_with_sidecar_resolver( + dns: &VmDnsConfig, + hostname: &str, +) -> Result, SidecarError> { + let runtime = tokio::runtime::Runtime::new().map_err(|error| { + SidecarError::Execution(format!("failed to create DNS runtime: {error}")) + })?; + + runtime.block_on(async { + let builder = if let Some(config) = vm_dns_resolver_config(dns) { + TokioResolver::builder_with_config(config, TokioRuntimeProvider::default()) + } else { + TokioResolver::builder_tokio().map_err(|error| { + SidecarError::Execution(format!( + "failed to initialize DNS resolver from system configuration: {error}" + )) + })? + }; + + let resolver = builder.build().map_err(|error| { + SidecarError::Execution(format!("failed to build DNS resolver: {error}")) + })?; + let lookup = resolver.lookup_ip(hostname).await.map_err(|error| { + SidecarError::Execution(format!("failed to resolve DNS address {hostname}: {error}")) + })?; + + let mut addresses = Vec::new(); + let mut seen = BTreeSet::new(); + for ip in lookup.iter() { + if seen.insert(ip) { + addresses.push(ip); + } + } + + if addresses.is_empty() { + return Err(SidecarError::Execution(format!( + "failed to resolve DNS address {hostname}" + ))); + } + + Ok(addresses) + }) +} + +fn emit_dns_resolution_event( + bridge: &SharedBridge, + vm_id: &str, + hostname: &str, + source: DnsResolutionSource, + addresses: &[IpAddr], + dns: &VmDnsConfig, +) where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + let _ = emit_structured_event( + bridge, + vm_id, + "network.dns.resolved", + audit_fields([ + ("hostname", hostname.to_owned()), + ("source", source.as_str().to_owned()), + ( + "addresses", + addresses + .iter() + .map(ToString::to_string) + .collect::>() + .join(","), + ), + ("address_count", addresses.len().to_string()), + ("resolver_count", dns.name_servers.len().to_string()), + ( + "resolvers", + dns.name_servers + .iter() + .map(ToString::to_string) + .collect::>() + .join(","), + ), + ]), + ); +} + +fn emit_dns_resolution_failure_event( + bridge: &SharedBridge, + vm_id: &str, + hostname: &str, + dns: &VmDnsConfig, + error: &SidecarError, +) where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + let _ = emit_structured_event( + bridge, + vm_id, + "network.dns.resolve_failed", + audit_fields([ + ("hostname", hostname.to_owned()), + ("reason", error.to_string()), + ("resolver_count", dns.name_servers.len().to_string()), + ( + "resolvers", + dns.name_servers + .iter() + .map(ToString::to_string) + .collect::>() + .join(","), + ), + ]), + ); +} + fn build_root_filesystem( descriptor: &RootFilesystemDescriptor, loaded_snapshot: Option<&FilesystemSnapshot>, @@ -3473,6 +5702,9 @@ fn guest_filesystem_stat(stat: VirtualStat) -> GuestFilesystemStat { GuestFilesystemStat { mode: stat.mode, size: stat.size, + blocks: stat.blocks, + dev: stat.dev, + rdev: stat.rdev, is_directory: stat.is_directory, is_symbolic_link: stat.is_symbolic_link, atime_ms: stat.atime_ms, @@ -3588,6 +5820,69 @@ fn find_socket_state_entry( let vm = vm.ok_or_else(|| SidecarError::InvalidState(String::from("unknown sidecar VM")))?; for (process_id, process) in &vm.active_processes { + if let Some(path) = request.path.as_deref() { + if matches!(kind, SocketQueryKind::TcpListener) { + for listener in process.unix_listeners.values() { + if listener.path() != path { + continue; + } + return Ok(Some(SocketStateEntry { + process_id: process_id.to_owned(), + host: None, + port: None, + path: Some(path.to_owned()), + })); + } + } + } + + if request.path.is_none() { + match kind { + SocketQueryKind::TcpListener => { + for listener in process.tcp_listeners.values() { + let local_addr = listener.guest_local_addr(); + let local_host = local_addr.ip().to_string(); + if !socket_host_matches(request.host.as_deref(), &local_host) { + continue; + } + if let Some(port) = request.port { + if local_addr.port() != port { + continue; + } + } + return Ok(Some(SocketStateEntry { + process_id: process_id.to_owned(), + host: Some(local_host), + port: Some(local_addr.port()), + path: None, + })); + } + } + SocketQueryKind::UdpBound => { + for socket in process.udp_sockets.values() { + let Some(local_addr) = socket.local_addr() else { + continue; + }; + let local_host = local_addr.ip().to_string(); + if !socket_host_matches(request.host.as_deref(), &local_host) { + continue; + } + if let Some(port) = request.port { + if local_addr.port() != port { + continue; + } + } + return Ok(Some(SocketStateEntry { + process_id: process_id.to_owned(), + host: Some(local_host), + port: Some(local_addr.port()), + path: None, + })); + } + } + } + } + let child_pid = process.execution.child_pid(); let inodes = socket_inodes_for_pid(child_pid)?; if inodes.is_empty() { @@ -3666,6 +5961,11 @@ fn parse_socket_inode(target: &Path) -> Option { trimmed.parse().ok() } +fn unix_socket_path(addr: &UnixSocketAddr) -> Option { + addr.as_pathname() + .map(|path| path.to_string_lossy().into_owned()) +} + fn find_unix_socket_for_pid( pid: u32, inodes: &BTreeSet, @@ -3720,10 +6020,8 @@ fn find_inet_socket_for_pid( if matches!(kind, SocketQueryKind::TcpListener) && entry.state != "0A" { continue; } - if let Some(host) = requested_host { - if entry.local_host != host { - continue; - } + if !socket_host_matches(requested_host, &entry.local_host) { + continue; } if let Some(port) = requested_port { if entry.local_port != port { @@ -3741,42 +6039,270 @@ fn find_inet_socket_for_pid( Ok(None) } -fn parse_proc_net_entries(table_path: &str) -> Result, SidecarError> { - let contents = match fs::read_to_string(table_path) { - Ok(contents) => contents, - Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), - Err(error) => { - return Err(SidecarError::Io(format!( - "failed to inspect socket table {table_path}: {error}" - ))) - } - }; +fn is_unspecified_socket_host(host: &str) -> bool { + host == "0.0.0.0" || host == "::" +} - let mut entries = Vec::new(); - for line in contents.lines().skip(1) { - let columns = line.split_whitespace().collect::>(); - if columns.len() < 10 { - continue; +fn is_loopback_socket_host(host: &str) -> bool { + host == "127.0.0.1" || host == "::1" || host.eq_ignore_ascii_case("localhost") +} + +fn vm_network_resource_counts(vm: &VmState) -> NetworkResourceCounts { + let mut counts = NetworkResourceCounts::default(); + for process in vm.active_processes.values() { + let process_counts = process.network_resource_counts(); + counts.sockets += process_counts.sockets; + counts.connections += process_counts.connections; + } + counts +} + +fn collect_javascript_socket_port_state( + process: &ActiveProcess, + tcp_guest_to_host: &mut BTreeMap<(JavascriptSocketFamily, u16), u16>, + udp_guest_to_host: &mut BTreeMap<(JavascriptSocketFamily, u16), u16>, + udp_host_to_guest: &mut BTreeMap<(JavascriptSocketFamily, u16), u16>, + used_tcp_ports: &mut BTreeMap>, + used_udp_ports: &mut BTreeMap>, +) { + for listener in process.tcp_listeners.values() { + let guest_addr = listener.guest_local_addr(); + let family = JavascriptSocketFamily::from_ip(guest_addr.ip()); + used_tcp_ports + .entry(family) + .or_default() + .insert(guest_addr.port()); + if is_loopback_ip(guest_addr.ip()) { + tcp_guest_to_host.insert((family, guest_addr.port()), listener.local_addr().port()); } - let Some((host, port)) = parse_proc_ip_port(columns[1]) else { - continue; - }; - let Ok(inode) = columns[9].parse::() else { + } + + for socket in process.udp_sockets.values() { + let Some(guest_addr) = socket.local_addr() else { continue; }; - entries.push(ProcNetEntry { - local_host: host, - local_port: port, - state: columns[3].to_owned(), - inode, - }); + let family = JavascriptSocketFamily::from_ip(guest_addr.ip()); + used_udp_ports + .entry(family) + .or_default() + .insert(guest_addr.port()); + if let Some(host_addr) = socket + .socket + .as_ref() + .and_then(|socket| socket.local_addr().ok()) + { + if is_loopback_ip(guest_addr.ip()) { + udp_guest_to_host.insert((family, guest_addr.port()), host_addr.port()); + udp_host_to_guest.insert((family, host_addr.port()), guest_addr.port()); + } + } } - Ok(entries) + for child in process.child_processes.values() { + collect_javascript_socket_port_state( + child, + tcp_guest_to_host, + udp_guest_to_host, + udp_host_to_guest, + used_tcp_ports, + used_udp_ports, + ); + } } -fn parse_proc_ip_port(value: &str) -> Option<(String, u16)> { - let (raw_ip, raw_port) = value.split_once(':')?; +fn build_javascript_socket_path_context( + vm: &VmState, +) -> Result { + let internal_env = extract_guest_env(&vm.metadata); + let mut tcp_loopback_guest_to_host_ports = BTreeMap::new(); + let mut udp_loopback_guest_to_host_ports = BTreeMap::new(); + let mut udp_loopback_host_to_guest_ports = BTreeMap::new(); + let mut used_tcp_guest_ports = BTreeMap::new(); + let mut used_udp_guest_ports = BTreeMap::new(); + for process in vm.active_processes.values() { + collect_javascript_socket_port_state( + process, + &mut tcp_loopback_guest_to_host_ports, + &mut udp_loopback_guest_to_host_ports, + &mut udp_loopback_host_to_guest_ports, + &mut used_tcp_guest_ports, + &mut used_udp_guest_ports, + ); + } + Ok(JavascriptSocketPathContext { + sandbox_root: vm.cwd.clone(), + mounts: vm.configuration.mounts.clone(), + listen_policy: parse_vm_listen_policy(&vm.metadata)?, + loopback_exempt_ports: parse_loopback_exempt_ports(&internal_env)?, + tcp_loopback_guest_to_host_ports, + udp_loopback_guest_to_host_ports, + udp_loopback_host_to_guest_ports, + used_tcp_guest_ports, + used_udp_guest_ports, + }) +} + +fn check_network_resource_limit( + limit: Option, + current: usize, + additional: usize, + label: &str, +) -> Result<(), SidecarError> { + if let Some(limit) = limit { + if current.saturating_add(additional) > limit { + return Err(SidecarError::Execution(format!( + "EAGAIN: maximum {label} count reached" + ))); + } + } + Ok(()) +} + +fn normalize_tcp_listen_host( + host: Option<&str>, +) -> Result<(JavascriptSocketFamily, &'static str), SidecarError> { + match host.unwrap_or("127.0.0.1") { + "127.0.0.1" | "localhost" => Ok((JavascriptSocketFamily::Ipv4, "127.0.0.1")), + "::1" => Ok((JavascriptSocketFamily::Ipv6, "::1")), + "0.0.0.0" | "::" => Err(SidecarError::Execution(String::from( + "EACCES: TCP listeners must bind to loopback, not unspecified addresses", + ))), + other => Err(SidecarError::Execution(format!( + "EACCES: TCP listeners must bind to loopback, got {other}" + ))), + } +} + +fn normalize_udp_bind_host( + host: Option<&str>, + family: JavascriptUdpFamily, +) -> Result<(&'static str, JavascriptSocketFamily), SidecarError> { + match (family, host) { + (JavascriptUdpFamily::Ipv4, None) + | (JavascriptUdpFamily::Ipv4, Some("127.0.0.1")) + | (JavascriptUdpFamily::Ipv4, Some("localhost")) => { + Ok(("127.0.0.1", JavascriptSocketFamily::Ipv4)) + } + (JavascriptUdpFamily::Ipv6, None) + | (JavascriptUdpFamily::Ipv6, Some("::1")) + | (JavascriptUdpFamily::Ipv6, Some("localhost")) => { + Ok(("::1", JavascriptSocketFamily::Ipv6)) + } + (_, Some("0.0.0.0")) | (_, Some("::")) => Err(SidecarError::Execution(String::from( + "EACCES: UDP sockets must bind to loopback, not unspecified addresses", + ))), + (JavascriptUdpFamily::Ipv4, Some(other)) => Err(SidecarError::Execution(format!( + "EACCES: udp4 sockets must bind to 127.0.0.1, got {other}" + ))), + (JavascriptUdpFamily::Ipv6, Some(other)) => Err(SidecarError::Execution(format!( + "EACCES: udp6 sockets must bind to ::1, got {other}" + ))), + } +} + +fn allocate_guest_listen_port( + requested_port: u16, + family: JavascriptSocketFamily, + used_ports: &BTreeMap>, + policy: VmListenPolicy, +) -> Result { + let is_allowed = |port: u16| { + port >= policy.port_min + && port <= policy.port_max + && (policy.allow_privileged || port >= 1024) + }; + let used = used_ports.get(&family); + + if requested_port != 0 { + if !is_allowed(requested_port) { + let reason = if requested_port < 1024 && !policy.allow_privileged { + format!( + "EACCES: privileged listen port {requested_port} requires {}=true", + VM_LISTEN_ALLOW_PRIVILEGED_METADATA_KEY + ) + } else { + format!( + "EACCES: listen port {requested_port} is outside the allowed range {}-{}", + policy.port_min, policy.port_max + ) + }; + return Err(SidecarError::Execution(reason)); + } + if used.is_some_and(|ports| ports.contains(&requested_port)) { + return Err(sidecar_net_error(std::io::Error::from_raw_os_error( + libc::EADDRINUSE, + ))); + } + return Ok(requested_port); + } + + let allocation_start = policy + .port_min + .max(if policy.allow_privileged { 1 } else { 1024 }); + for candidate in allocation_start..=policy.port_max { + if used.is_some_and(|ports| ports.contains(&candidate)) { + continue; + } + return Ok(candidate); + } + + Err(sidecar_net_error(std::io::Error::from_raw_os_error( + libc::EADDRINUSE, + ))) +} + +fn socket_host_matches(requested: Option<&str>, actual: &str) -> bool { + match requested { + None => true, + Some(requested) if requested == actual => true, + Some(requested) + if is_unspecified_socket_host(requested) && is_unspecified_socket_host(actual) => + { + true + } + Some(requested) if requested.eq_ignore_ascii_case("localhost") => { + is_loopback_socket_host(actual) + } + _ => false, + } +} + +fn parse_proc_net_entries(table_path: &str) -> Result, SidecarError> { + let contents = match fs::read_to_string(table_path) { + Ok(contents) => contents, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(error) => { + return Err(SidecarError::Io(format!( + "failed to inspect socket table {table_path}: {error}" + ))) + } + }; + + let mut entries = Vec::new(); + for line in contents.lines().skip(1) { + let columns = line.split_whitespace().collect::>(); + if columns.len() < 10 { + continue; + } + let Some((host, port)) = parse_proc_ip_port(columns[1]) else { + continue; + }; + let Ok(inode) = columns[9].parse::() else { + continue; + }; + entries.push(ProcNetEntry { + local_host: host, + local_port: port, + state: columns[3].to_owned(), + inode, + }); + } + + Ok(entries) +} + +fn parse_proc_ip_port(value: &str) -> Option<(String, u16)> { + let (raw_ip, raw_port) = value.split_once(':')?; let port = u16::from_str_radix(raw_port, 16).ok()?; let host = match raw_ip.len() { 8 => { @@ -3824,6 +6350,57 @@ fn normalize_path(path: &str) -> String { } } +fn normalize_python_vfs_rpc_path(path: &str) -> Result { + if !path.starts_with('/') { + return Err(SidecarError::InvalidState(format!( + "python VFS RPC path {path} must be absolute within {PYTHON_VFS_RPC_GUEST_ROOT}" + ))); + } + + let normalized = normalize_path(path); + if normalized == PYTHON_VFS_RPC_GUEST_ROOT + || normalized.starts_with(&format!("{PYTHON_VFS_RPC_GUEST_ROOT}/")) + { + Ok(normalized) + } else { + Err(SidecarError::InvalidState(format!( + "python VFS RPC path {normalized} escapes guest workspace root {PYTHON_VFS_RPC_GUEST_ROOT}" + ))) + } +} + +fn normalize_host_path(path: &Path) -> PathBuf { + let mut normalized = PathBuf::new(); + + for component in path.components() { + match component { + Component::Prefix(prefix) => normalized.push(prefix.as_os_str()), + Component::RootDir => normalized.push(Path::new("/")), + Component::CurDir => {} + Component::ParentDir => { + if normalized != Path::new("/") { + normalized.pop(); + } + } + Component::Normal(part) => normalized.push(part), + } + } + + if normalized.as_os_str().is_empty() { + if path.is_absolute() { + PathBuf::from("/") + } else { + PathBuf::from(".") + } + } else { + normalized + } +} + +fn path_is_within_root(path: &Path, root: &Path) -> bool { + path == root || path.starts_with(root) +} + fn dirname(path: &str) -> String { let normalized = normalize_path(path); let parent = Path::new(&normalized) @@ -3843,1337 +6420,7828 @@ fn python_file_entrypoint(entrypoint: &str) -> Option { .then(|| path.to_path_buf()) } -fn kernel_error(error: KernelError) -> SidecarError { - SidecarError::Kernel(error.to_string()) -} +fn discover_command_guest_paths(kernel: &mut SidecarKernel) -> BTreeMap { + let mut command_guest_paths = BTreeMap::new(); + let Ok(command_roots) = kernel.read_dir("/__agentos/commands") else { + return command_guest_paths; + }; -fn plugin_error(error: PluginError) -> SidecarError { - SidecarError::Plugin(error.to_string()) + let mut ordered_roots = command_roots + .into_iter() + .filter(|entry| !entry.is_empty() && entry.chars().all(|ch| ch.is_ascii_digit())) + .collect::>(); + ordered_roots.sort(); + + for root in ordered_roots { + let guest_root = format!("/__agentos/commands/{root}"); + let Ok(entries) = kernel.read_dir(&guest_root) else { + continue; + }; + + for entry in entries { + if entry.starts_with('.') || command_guest_paths.contains_key(&entry) { + continue; + } + command_guest_paths.insert(entry.clone(), format!("{guest_root}/{entry}")); + } + } + + command_guest_paths } -fn javascript_error(error: JavascriptExecutionError) -> SidecarError { - SidecarError::Execution(error.to_string()) +fn is_path_like_specifier(specifier: &str) -> bool { + specifier.starts_with('/') + || specifier.starts_with("./") + || specifier.starts_with("../") + || specifier.starts_with("file:") } -fn wasm_error(error: WasmExecutionError) -> SidecarError { - SidecarError::Execution(error.to_string()) +fn execution_wasm_permission_tier(tier: WasmPermissionTier) -> ExecutionWasmPermissionTier { + match tier { + WasmPermissionTier::Full => ExecutionWasmPermissionTier::Full, + WasmPermissionTier::ReadWrite => ExecutionWasmPermissionTier::ReadWrite, + WasmPermissionTier::ReadOnly => ExecutionWasmPermissionTier::ReadOnly, + WasmPermissionTier::Isolated => ExecutionWasmPermissionTier::Isolated, + } } -fn python_error(error: PythonExecutionError) -> SidecarError { - SidecarError::Execution(error.to_string()) +fn resolve_wasm_permission_tier( + vm: &VmState, + command_name: Option<&str>, + explicit_tier: Option, + entrypoint: &str, +) -> WasmPermissionTier { + explicit_tier + .or_else(|| command_name.and_then(|command| vm.command_permissions.get(command).copied())) + .or_else(|| { + Path::new(entrypoint) + .file_name() + .and_then(|name| name.to_str()) + .and_then(|command| vm.command_permissions.get(command).copied()) + }) + .unwrap_or(WasmPermissionTier::Full) } -fn vfs_error(error: VfsError) -> SidecarError { - SidecarError::Kernel(error.to_string()) +fn tokenize_shell_free_command(command: &str) -> Vec { + command + .split_whitespace() + .filter(|segment| !segment.is_empty()) + .map(str::to_owned) + .collect() } -fn parse_signal(signal: &str) -> Result { - let trimmed = signal.trim(); - if trimmed.is_empty() { - return Err(SidecarError::InvalidState(String::from( - "kill_process requires a non-empty signal", - ))); - } +fn host_mount_path_for_guest_path(vm: &VmState, guest_path: &str) -> Option { + let normalized = normalize_path(guest_path); - if let Ok(value) = trimmed.parse::() { - return Ok(value); - } + let mut mounts = vm + .configuration + .mounts + .iter() + .filter_map(|mount| { + (mount.plugin.id == "host_dir") + .then(|| { + mount + .plugin + .config + .get("hostPath") + .and_then(Value::as_str) + .map(|host_path| (mount.guest_path.as_str(), host_path)) + }) + .flatten() + }) + .collect::>(); + mounts.sort_by(|left, right| right.0.len().cmp(&left.0.len())); - let upper = trimmed.to_ascii_uppercase(); - let normalized = upper.strip_prefix("SIG").unwrap_or(&upper); + for (guest_root, host_root) in mounts { + if normalized != guest_root && !normalized.starts_with(&format!("{guest_root}/")) { + continue; + } - signal_number_from_name(normalized).ok_or_else(|| { - SidecarError::InvalidState(format!("unsupported kill_process signal {signal}")) - }) + let suffix = normalized + .strip_prefix(guest_root) + .unwrap_or_default() + .trim_start_matches('/'); + let mut path = PathBuf::from(host_root); + if !suffix.is_empty() { + path.push(suffix); + } + return Some(path); + } + + None } -fn signal_number_from_name(signal: &str) -> Option { - match signal { - "HUP" => Some(libc::SIGHUP), - "INT" => Some(libc::SIGINT), - "QUIT" => Some(libc::SIGQUIT), - "ILL" => Some(libc::SIGILL), - "TRAP" => Some(libc::SIGTRAP), - "ABRT" | "IOT" => Some(libc::SIGABRT), - "BUS" => Some(libc::SIGBUS), - "FPE" => Some(libc::SIGFPE), - "KILL" => Some(SIGKILL), - "USR1" => Some(libc::SIGUSR1), - "SEGV" => Some(libc::SIGSEGV), - "USR2" => Some(libc::SIGUSR2), - "PIPE" => Some(libc::SIGPIPE), - "ALRM" => Some(libc::SIGALRM), - "TERM" => Some(SIGTERM), - "CHLD" | "CLD" => Some(libc::SIGCHLD), - "CONT" => Some(libc::SIGCONT), - "STOP" => Some(libc::SIGSTOP), - "TSTP" => Some(libc::SIGTSTP), - "TTIN" => Some(libc::SIGTTIN), - "TTOU" => Some(libc::SIGTTOU), - "URG" => Some(libc::SIGURG), - "XCPU" => Some(libc::SIGXCPU), - "XFSZ" => Some(libc::SIGXFSZ), - "VTALRM" => Some(libc::SIGVTALRM), - "PROF" => Some(libc::SIGPROF), - "WINCH" => Some(libc::SIGWINCH), - "IO" | "POLL" => Some(libc::SIGIO), - "SYS" => Some(libc::SIGSYS), - #[cfg(any(target_os = "linux", target_os = "android"))] - "STKFLT" => Some(libc::SIGSTKFLT), - #[cfg(any(target_os = "linux", target_os = "android"))] - "PWR" => Some(libc::SIGPWR), - #[cfg(any(target_os = "linux", target_os = "android"))] - "UNUSED" => Some(libc::SIGSYS), - #[cfg(any( - target_os = "macos", - target_os = "ios", - target_os = "freebsd", - target_os = "dragonfly", - target_os = "netbsd", - target_os = "openbsd", - ))] - "EMT" => Some(libc::SIGEMT), - #[cfg(any( - target_os = "macos", - target_os = "ios", - target_os = "freebsd", - target_os = "dragonfly", - target_os = "netbsd", - target_os = "openbsd", - ))] - "INFO" => Some(libc::SIGINFO), - _ => None, +fn host_mount_path_for_guest_path_from_mounts( + mounts: &[crate::protocol::MountDescriptor], + guest_path: &str, +) -> Option { + let normalized = normalize_path(guest_path); + + let mut host_mounts = mounts + .iter() + .filter_map(|mount| { + (mount.plugin.id == "host_dir") + .then(|| { + mount + .plugin + .config + .get("hostPath") + .and_then(Value::as_str) + .map(|host_path| (mount.guest_path.as_str(), host_path)) + }) + .flatten() + }) + .collect::>(); + host_mounts.sort_by(|left, right| right.0.len().cmp(&left.0.len())); + + for (guest_root, host_root) in host_mounts { + if normalized != guest_root && !normalized.starts_with(&format!("{guest_root}/")) { + continue; + } + + let suffix = normalized + .strip_prefix(guest_root) + .unwrap_or_default() + .trim_start_matches('/'); + let mut path = PathBuf::from(host_root); + if !suffix.is_empty() { + path.push(suffix); + } + return Some(path); } + + None } -fn signal_runtime_process(child_pid: u32, signal: i32) -> Result<(), SidecarError> { - let result = if signal == 0 { - send_signal(Pid::from_raw(child_pid as i32), None) - } else { - let parsed = Signal::try_from(signal).map_err(|_| { - SidecarError::InvalidState(format!("unsupported kill_process signal {signal}")) - })?; - send_signal(Pid::from_raw(child_pid as i32), Some(parsed)) - }; +fn resolve_guest_socket_host_path( + context: &JavascriptSocketPathContext, + guest_path: &str, +) -> PathBuf { + if let Some(path) = host_mount_path_for_guest_path_from_mounts(&context.mounts, guest_path) { + return path; + } - match result { - Ok(()) => Ok(()), - Err(nix::errno::Errno::ESRCH) => Ok(()), - Err(error) => Err(SidecarError::Execution(format!( - "failed to signal guest runtime process {child_pid}: {error}" - ))), + let normalized = normalize_path(guest_path); + let mut host_path = context.sandbox_root.clone(); + let suffix = normalized.trim_start_matches('/'); + if !suffix.is_empty() { + host_path.push(suffix); } + host_path } -fn error_code(error: &SidecarError) -> &'static str { - match error { - SidecarError::InvalidState(_) => "invalid_state", - SidecarError::Unauthorized(_) => "unauthorized", - SidecarError::Unsupported(_) => "unsupported", - SidecarError::FrameTooLarge(_) => "frame_too_large", - SidecarError::Kernel(_) => "kernel_error", - SidecarError::Plugin(_) => "plugin_error", - SidecarError::Execution(_) => "execution_error", - SidecarError::Bridge(_) => "bridge_error", - SidecarError::Io(_) => "io_error", +fn ensure_kernel_parent_directories( + kernel: &mut SidecarKernel, + path: &str, +) -> Result<(), SidecarError> { + let parent = dirname(path); + if parent != "/" && !kernel.exists(&parent).map_err(kernel_error)? { + kernel.mkdir(&parent, true).map_err(kernel_error)?; } + Ok(()) } -#[cfg(test)] -mod tests { - #[path = "/home/nathan/a5/crates/bridge/tests/support.rs"] - mod bridge_support; +#[derive(Debug, Deserialize, Default)] +struct JavascriptChildProcessSpawnOptions { + #[serde(default)] + cwd: Option, + #[serde(default)] + env: BTreeMap, + #[serde(rename = "internalBootstrapEnv", default)] + internal_bootstrap_env: BTreeMap, + #[serde(default)] + shell: bool, +} - use super::*; - use crate::protocol::{ - AuthenticateRequest, BootstrapRootFilesystemRequest, ConfigureVmRequest, CreateVmRequest, - GetZombieTimerCountRequest, GuestRuntimeKind, MountDescriptor, MountPluginDescriptor, - OpenSessionRequest, OwnershipScope, RequestFrame, RequestPayload, ResponsePayload, - RootFilesystemEntry, RootFilesystemEntryKind, SidecarPlacement, - }; - use crate::s3_plugin::test_support::MockS3Server; - use crate::sandbox_agent_plugin::test_support::MockSandboxAgentServer; - use agent_os_kernel::command_registry::CommandDriver; - use agent_os_kernel::kernel::SpawnOptions; - use agent_os_kernel::mount_table::MountEntry; - use bridge_support::RecordingBridge; - use serde_json::json; - use std::collections::BTreeMap; - use std::fs; - use std::path::{Path, PathBuf}; - use std::process::Command; - use std::time::{SystemTime, UNIX_EPOCH}; +#[derive(Debug, Deserialize)] +struct JavascriptChildProcessSpawnRequest { + command: String, + #[serde(default)] + args: Vec, + #[serde(default)] + options: JavascriptChildProcessSpawnOptions, +} - const TEST_AUTH_TOKEN: &str = "sidecar-test-token"; +#[derive(Debug)] +struct ResolvedChildProcessExecution { + command: String, + process_args: Vec, + runtime: GuestRuntimeKind, + entrypoint: String, + execution_args: Vec, + env: BTreeMap, + guest_cwd: String, + host_cwd: PathBuf, + wasm_permission_tier: Option, +} - fn request( - request_id: u64, - ownership: OwnershipScope, - payload: RequestPayload, - ) -> RequestFrame { - RequestFrame::new(request_id, ownership, payload) - } +fn sanitize_javascript_child_process_internal_bootstrap_env( + env: &BTreeMap, +) -> BTreeMap { + const ALLOWED_KEYS: &[&str] = &[ + "AGENT_OS_ALLOWED_NODE_BUILTINS", + "AGENT_OS_GUEST_PATH_MAPPINGS", + "AGENT_OS_LOOPBACK_EXEMPT_PORTS", + "AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH", + "AGENT_OS_VIRTUAL_PROCESS_UID", + "AGENT_OS_VIRTUAL_PROCESS_GID", + "AGENT_OS_VIRTUAL_PROCESS_VERSION", + ]; + + env.iter() + .filter(|(key, _)| { + ALLOWED_KEYS.contains(&key.as_str()) || key.starts_with("AGENT_OS_VIRTUAL_OS_") + }) + .map(|(key, value)| (key.clone(), value.clone())) + .collect() +} - fn create_test_sidecar() -> NativeSidecar { - NativeSidecar::with_config( - RecordingBridge::default(), - NativeSidecarConfig { - sidecar_id: String::from("sidecar-test"), - compile_cache_root: Some(std::env::temp_dir().join("agent-os-sidecar-test-cache")), - expected_auth_token: Some(String::from(TEST_AUTH_TOKEN)), - ..NativeSidecarConfig::default() - }, - ) - .expect("create sidecar") - } +#[derive(Debug, Deserialize)] +struct JavascriptNetConnectRequest { + #[serde(default)] + host: Option, + #[serde(default)] + port: Option, + #[serde(default)] + path: Option, +} - fn unexpected_response_error(expected: &str, other: ResponsePayload) -> SidecarError { - SidecarError::InvalidState(format!("expected {expected} response, got {other:?}")) - } +#[derive(Debug, Deserialize)] +struct JavascriptNetListenRequest { + #[serde(default)] + host: Option, + #[serde(default)] + port: Option, + #[serde(default)] + path: Option, + #[serde(default)] + backlog: Option, +} - fn authenticated_connection_id(auth: DispatchResult) -> Result { - match auth.response.payload { - ResponsePayload::Authenticated(response) => { - assert_eq!( - auth.response.ownership, - OwnershipScope::connection(&response.connection_id) - ); - Ok(response.connection_id) - } - other => Err(unexpected_response_error("authenticated", other)), +#[derive(Debug, Deserialize)] +struct JavascriptDgramCreateSocketRequest { + #[serde(rename = "type")] + socket_type: String, +} + +#[derive(Debug, Deserialize)] +struct JavascriptDgramBindRequest { + #[serde(default)] + address: Option, + #[serde(default)] + port: u16, +} + +#[derive(Debug, Deserialize)] +struct JavascriptDgramSendRequest { + #[serde(default)] + address: Option, + port: u16, +} + +#[derive(Debug, Deserialize)] +struct JavascriptDnsLookupRequest { + hostname: String, + #[serde(default)] + family: Option, +} + +#[derive(Debug, Deserialize)] +struct JavascriptDnsResolveRequest { + hostname: String, + #[serde(default)] + rrtype: Option, +} + +#[derive(Debug, Clone, Default)] +struct VmDnsConfig { + name_servers: Vec, + overrides: BTreeMap>, +} + +#[derive(Debug, Clone, Copy)] +enum DnsResolutionSource { + Literal, + Override, + Resolver, +} + +impl DnsResolutionSource { + fn as_str(self) -> &'static str { + match self { + Self::Literal => "literal", + Self::Override => "override", + Self::Resolver => "resolver", } } +} - fn opened_session_id(session: DispatchResult) -> Result { - match session.response.payload { - ResponsePayload::SessionOpened(response) => Ok(response.session_id), - other => Err(unexpected_response_error("session_opened", other)), +fn resolve_tcp_bind_addr(host: &str, port: u16) -> Result { + (host, port) + .to_socket_addrs() + .map_err(sidecar_net_error)? + .next() + .ok_or_else(|| { + SidecarError::Execution(format!("failed to resolve TCP bind address {host}:{port}")) + }) +} + +fn format_dns_resource(hostname: &str) -> String { + format!("dns://{hostname}") +} + +fn format_tcp_resource(host: &str, port: u16) -> String { + format!("tcp://{host}:{port}") +} + +fn is_loopback_ip(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(ip) => ip.is_loopback(), + IpAddr::V6(ip) => { + ip.is_loopback() + || ip + .to_ipv4_mapped() + .is_some_and(|mapped| mapped.is_loopback()) } } +} - fn created_vm_id(response: DispatchResult) -> Result { - match response.response.payload { - ResponsePayload::VmCreated(response) => Ok(response.vm_id), - other => Err(unexpected_response_error("vm_created", other)), +fn loopback_cidr(ip: IpAddr) -> &'static str { + match ip { + IpAddr::V4(ip) if ip.is_loopback() => "127.0.0.0/8", + IpAddr::V6(ip) + if ip + .to_ipv4_mapped() + .is_some_and(|mapped| mapped.is_loopback()) => + { + "127.0.0.0/8" } + IpAddr::V6(_) => "::1/128", + IpAddr::V4(_) => "127.0.0.0/8", } +} - fn authenticate_and_open_session( - sidecar: &mut NativeSidecar, - ) -> Result<(String, String), SidecarError> { - let auth = sidecar - .dispatch(request( - 1, - OwnershipScope::connection("conn-1"), - RequestPayload::Authenticate(AuthenticateRequest { - client_name: String::from("service-tests"), - auth_token: String::from(TEST_AUTH_TOKEN), - }), - )) - .expect("authenticate"); - let connection_id = authenticated_connection_id(auth)?; +fn restricted_non_loopback_ip_range(ip: IpAddr) -> Option<(&'static str, &'static str)> { + match ip { + IpAddr::V4(ip) => { + let [first, second, ..] = ip.octets(); + match (first, second) { + (10, _) => Some(("10.0.0.0/8", "private")), + (172, 16..=31) => Some(("172.16.0.0/12", "private")), + (192, 168) => Some(("192.168.0.0/16", "private")), + (169, 254) => Some(("169.254.0.0/16", "link-local")), + _ => None, + } + } + IpAddr::V6(ip) => { + if let Some(mapped) = ip.to_ipv4_mapped() { + return restricted_non_loopback_ip_range(IpAddr::V4(mapped)); + } - let session = sidecar - .dispatch(request( - 2, - OwnershipScope::connection(&connection_id), - RequestPayload::OpenSession(OpenSessionRequest { - placement: SidecarPlacement::Shared { pool: None }, - metadata: BTreeMap::new(), - }), - )) - .expect("open session"); - let session_id = opened_session_id(session)?; - Ok((connection_id, session_id)) + let segments = ip.segments(); + if (segments[0] & 0xfe00) == 0xfc00 { + return Some(("fc00::/7", "unique-local")); + } + if (segments[0] & 0xffc0) == 0xfe80 { + return Some(("fe80::/10", "link-local")); + } + None + } } +} - fn create_vm( - sidecar: &mut NativeSidecar, - connection_id: &str, - session_id: &str, - ) -> Result { - let response = sidecar - .dispatch(request( - 3, - OwnershipScope::session(connection_id, session_id), - RequestPayload::CreateVm(CreateVmRequest { - runtime: GuestRuntimeKind::JavaScript, - metadata: BTreeMap::new(), - root_filesystem: Default::default(), - }), - )) - .expect("create vm"); - - created_vm_id(response) - } +fn blocked_dns_resolution_error( + resource: &str, + ip: IpAddr, + cidr: &str, + label: &str, +) -> SidecarError { + SidecarError::Execution(format!( + "EACCES: blocked outbound network access to {resource}: {ip} is within restricted {label} range {cidr}" + )) +} - fn temp_dir(prefix: &str) -> PathBuf { - let suffix = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock should be monotonic enough for temp paths") - .as_nanos(); - let path = std::env::temp_dir().join(format!("{prefix}-{suffix}")); - fs::create_dir_all(&path).expect("create temp dir"); - path - } +fn blocked_loopback_connect_error(resource: &str, ip: IpAddr, port: u16) -> SidecarError { + SidecarError::Execution(format!( + "EACCES: blocked outbound network access to {resource}: {ip} is loopback ({}) and port {port} is not owned by this VM and is not listed in {LOOPBACK_EXEMPT_PORTS_ENV}", + loopback_cidr(ip) + )) +} - fn write_fixture(path: &Path, contents: &str) { - fs::write(path, contents).expect("write fixture"); +fn filter_dns_safe_ip_addrs( + addresses: Vec, + hostname: &str, +) -> Result, SidecarError> { + let resource = format_dns_resource(hostname); + let mut allowed = Vec::new(); + let mut blocked = None; + + for ip in addresses { + if let Some((cidr, label)) = restricted_non_loopback_ip_range(ip) { + blocked.get_or_insert((ip, cidr, label)); + continue; + } + allowed.push(ip); } - fn assert_node_available() { - let output = Command::new("node") - .arg("--version") - .output() - .expect("spawn node --version"); - assert!( - output.status.success(), - "node must be available for python dispatch tests" - ); + if allowed.is_empty() { + let (ip, cidr, label) = blocked.expect("blocked DNS results should capture a reason"); + return Err(blocked_dns_resolution_error(&resource, ip, cidr, label)); } - #[test] - fn get_zombie_timer_count_reports_kernel_state_before_and_after_waitpid() { - let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = - authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + Ok(allowed) +} - let zombie_pid = { - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - vm.kernel - .register_driver(CommandDriver::new("test-driver", ["test-zombie"])) - .expect("register test driver"); - let process = vm - .kernel - .spawn_process( - "test-zombie", - Vec::new(), - SpawnOptions { - requester_driver: Some(String::from("test-driver")), - ..SpawnOptions::default() - }, - ) - .expect("spawn test process"); - process.finish(17); - assert_eq!(vm.kernel.zombie_timer_count(), 1); - process.pid() - }; +fn loopback_connect_allowed(context: &JavascriptSocketPathContext, port: u16) -> bool { + context.loopback_port_allowed(port) +} - let zombie_count = sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::GetZombieTimerCount(GetZombieTimerCountRequest::default()), - )) - .expect("query zombie count"); - match zombie_count.response.payload { - ResponsePayload::ZombieTimerCount(response) => assert_eq!(response.count, 1), - other => panic!("unexpected zombie count response: {other:?}"), +fn filter_tcp_connect_ip_addrs( + addresses: Vec, + host: &str, + port: u16, + context: &JavascriptSocketPathContext, +) -> Result, SidecarError> { + let resource = format_tcp_resource(host, port); + let mut allowed = Vec::new(); + let mut blocked = None; + + for ip in addresses { + if let Some((cidr, label)) = restricted_non_loopback_ip_range(ip) { + blocked.get_or_insert_with(|| blocked_dns_resolution_error(&resource, ip, cidr, label)); + continue; } - - { - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let waited = vm.kernel.waitpid(zombie_pid).expect("waitpid"); - assert_eq!(waited.pid, zombie_pid); - assert_eq!(waited.status, 17); - assert_eq!(vm.kernel.zombie_timer_count(), 0); + if is_loopback_ip(ip) && !loopback_connect_allowed(context, port) { + blocked.get_or_insert_with(|| blocked_loopback_connect_error(&resource, ip, port)); + continue; } + allowed.push(ip); + } - let reaped_count = sidecar - .dispatch(request( - 5, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::GetZombieTimerCount(GetZombieTimerCountRequest::default()), - )) - .expect("query reaped zombie count"); - match reaped_count.response.payload { - ResponsePayload::ZombieTimerCount(response) => assert_eq!(response.count, 0), - other => panic!("unexpected zombie count response: {other:?}"), - } + if allowed.is_empty() { + return Err(blocked.expect("blocked TCP connect results should capture a reason")); } - #[test] - fn parse_signal_accepts_posix_names_and_aliases() { - assert_eq!( - parse_signal("SIGUSR1").expect("parse SIGUSR1"), - libc::SIGUSR1 - ); - assert_eq!(parse_signal("usr2").expect("parse SIGUSR2"), libc::SIGUSR2); - assert_eq!( - parse_signal("SIGSTOP").expect("parse SIGSTOP"), - libc::SIGSTOP + Ok(allowed) +} + +fn resolve_tcp_connect_addr( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + host: &str, + port: u16, + context: &JavascriptSocketPathContext, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + let allowed = filter_tcp_connect_ip_addrs( + resolve_dns_ip_addrs(bridge, vm_id, dns, host)?, + host, + port, + context, + )?; + let ip = allowed + .iter() + .copied() + .find(|candidate| { + let family = JavascriptSocketFamily::from_ip(*candidate); + context.translate_tcp_loopback_port(family, port).is_some() + }) + .or_else(|| allowed.first().copied()) + .ok_or_else(|| { + SidecarError::Execution(format!("failed to resolve TCP address {host}:{port}")) + })?; + let family = JavascriptSocketFamily::from_ip(ip); + let actual_port = if is_loopback_ip(ip) { + context + .translate_tcp_loopback_port(family, port) + .unwrap_or(port) + } else { + port + }; + Ok(ResolvedTcpConnectAddr { + actual_addr: SocketAddr::new(ip, actual_port), + guest_remote_addr: SocketAddr::new(ip, port), + }) +} + +fn resolve_dns_ip_addrs( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + hostname: &str, +) -> Result, SidecarError> +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + if let Ok(ip_addr) = hostname.parse::() { + let addresses = vec![ip_addr]; + emit_dns_resolution_event( + bridge, + vm_id, + hostname, + DnsResolutionSource::Literal, + &addresses, + dns, ); - assert_eq!( - parse_signal("SIGCONT").expect("parse SIGCONT"), - libc::SIGCONT + return Ok(addresses); + } + + let normalized_hostname = normalize_dns_hostname(hostname)?; + if let Some(addresses) = dns.overrides.get(&normalized_hostname) { + emit_dns_resolution_event( + bridge, + vm_id, + hostname, + DnsResolutionSource::Override, + addresses, + dns, ); - assert_eq!(parse_signal("SIGCLD").expect("parse SIGCLD"), libc::SIGCHLD); - assert_eq!(parse_signal("SIGIOT").expect("parse SIGIOT"), libc::SIGABRT); - assert_eq!(parse_signal("15").expect("parse numeric signal"), 15); + return Ok(addresses.clone()); } - #[test] - fn authenticated_connection_id_returns_error_for_unexpected_response() { - let error = authenticated_connection_id(DispatchResult { - response: ResponseFrame::new( - 1, - OwnershipScope::connection("conn-1"), - ResponsePayload::SessionOpened(SessionOpenedResponse { - session_id: String::from("session-1"), - owner_connection_id: String::from("conn-1"), - }), - ), - events: Vec::new(), - }) - .expect_err("unexpected auth payload should return an error"); + let addresses = match resolve_dns_with_sidecar_resolver(dns, &normalized_hostname) { + Ok(addresses) => addresses, + Err(error) => { + emit_dns_resolution_failure_event(bridge, vm_id, hostname, dns, &error); + return Err(error); + } + }; + emit_dns_resolution_event( + bridge, + vm_id, + hostname, + DnsResolutionSource::Resolver, + &addresses, + dns, + ); + Ok(addresses) +} - match error { - SidecarError::InvalidState(message) => { - assert!(message.contains("expected authenticated response")); - assert!(message.contains("SessionOpened")); - } - other => panic!("expected invalid_state error, got {other:?}"), +fn filter_dns_ip_addrs( + addresses: Vec, + family: Option, +) -> Result, SidecarError> { + let filtered: Vec<_> = match family.unwrap_or(0) { + 0 => addresses, + 4 => addresses + .into_iter() + .filter(|ip| matches!(ip, IpAddr::V4(_))) + .collect(), + 6 => addresses + .into_iter() + .filter(|ip| matches!(ip, IpAddr::V6(_))) + .collect(), + other => { + return Err(SidecarError::InvalidState(format!( + "unsupported dns family {other}" + ))) } + }; + + if filtered.is_empty() { + return Err(SidecarError::Execution(String::from( + "failed to resolve DNS address for requested family", + ))); } - #[test] - fn opened_session_id_returns_error_for_unexpected_response() { - let error = opened_session_id(DispatchResult { - response: ResponseFrame::new( - 2, - OwnershipScope::connection("conn-1"), - ResponsePayload::VmCreated(VmCreatedResponse { - vm_id: String::from("vm-1"), - }), - ), - events: Vec::new(), + Ok(filtered) +} + +fn resolve_udp_bind_addr( + host: &str, + port: u16, + family: JavascriptUdpFamily, +) -> Result { + (host, port) + .to_socket_addrs() + .map_err(sidecar_net_error)? + .find(|addr| family.matches_addr(addr)) + .ok_or_else(|| { + SidecarError::Execution(format!( + "failed to resolve {} UDP bind address {host}:{port}", + family.socket_type() + )) }) - .expect_err("unexpected session payload should return an error"); +} - match error { - SidecarError::InvalidState(message) => { - assert!(message.contains("expected session_opened response")); - assert!(message.contains("VmCreated")); +fn resolve_udp_addr( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + host: &str, + port: u16, + family: JavascriptUdpFamily, + context: &JavascriptSocketPathContext, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + resolve_dns_ip_addrs(bridge, vm_id, dns, host)? + .into_iter() + .map(|ip| { + let family_key = JavascriptSocketFamily::from_ip(ip); + let actual_port = if is_loopback_ip(ip) { + context + .translate_udp_loopback_port(family_key, port) + .unwrap_or(port) + } else { + port + }; + SocketAddr::new(ip, actual_port) + }) + .find(|addr| family.matches_addr(addr)) + .ok_or_else(|| { + SidecarError::Execution(format!( + "failed to resolve {} UDP address {host}:{port}", + family.socket_type() + )) + }) +} + +fn socket_addr_family(addr: &SocketAddr) -> &'static str { + match addr { + SocketAddr::V4(_) => "IPv4", + SocketAddr::V6(_) => "IPv6", + } +} + +fn io_error_code(error: &std::io::Error) -> Option { + match error.raw_os_error() { + Some(libc::EADDRINUSE) => Some(String::from("EADDRINUSE")), + Some(libc::EADDRNOTAVAIL) => Some(String::from("EADDRNOTAVAIL")), + Some(libc::ECONNREFUSED) => Some(String::from("ECONNREFUSED")), + Some(libc::ECONNRESET) => Some(String::from("ECONNRESET")), + Some(libc::EINVAL) => Some(String::from("EINVAL")), + Some(libc::EPIPE) => Some(String::from("EPIPE")), + Some(libc::ETIMEDOUT) => Some(String::from("ETIMEDOUT")), + Some(libc::EHOSTUNREACH) => Some(String::from("EHOSTUNREACH")), + Some(libc::ENETUNREACH) => Some(String::from("ENETUNREACH")), + _ => None, + } +} + +fn sidecar_net_error(error: std::io::Error) -> SidecarError { + let message = match io_error_code(&error) { + Some(code) => format!("{code}: {error}"), + None => error.to_string(), + }; + SidecarError::Execution(message) +} + +fn spawn_tcp_socket_reader( + stream: TcpStream, + sender: Sender, + saw_local_shutdown: Arc, + saw_remote_end: Arc, + close_notified: Arc, +) { + thread::spawn(move || { + let mut stream = stream; + let mut buffer = vec![0_u8; 64 * 1024]; + loop { + match stream.read(&mut buffer) { + Ok(0) => { + saw_remote_end.store(true, Ordering::SeqCst); + let _ = sender.send(JavascriptTcpSocketEvent::End); + if saw_local_shutdown.load(Ordering::SeqCst) + && !close_notified.swap(true, Ordering::SeqCst) + { + let _ = sender.send(JavascriptTcpSocketEvent::Close { had_error: false }); + } + break; + } + Ok(bytes_read) => { + if sender + .send(JavascriptTcpSocketEvent::Data( + buffer[..bytes_read].to_vec(), + )) + .is_err() + { + break; + } + } + Err(error) => { + let code = io_error_code(&error); + let _ = sender.send(JavascriptTcpSocketEvent::Error { + code, + message: error.to_string(), + }); + if !close_notified.swap(true, Ordering::SeqCst) { + let _ = sender.send(JavascriptTcpSocketEvent::Close { had_error: true }); + } + break; + } } - other => panic!("expected invalid_state error, got {other:?}"), + } + }); +} + +fn spawn_unix_socket_reader( + stream: UnixStream, + sender: Sender, + saw_local_shutdown: Arc, + saw_remote_end: Arc, + close_notified: Arc, +) { + thread::spawn(move || { + let mut stream = stream; + let mut buffer = vec![0_u8; 64 * 1024]; + loop { + match stream.read(&mut buffer) { + Ok(0) => { + saw_remote_end.store(true, Ordering::SeqCst); + let _ = sender.send(JavascriptTcpSocketEvent::End); + if saw_local_shutdown.load(Ordering::SeqCst) + && !close_notified.swap(true, Ordering::SeqCst) + { + let _ = sender.send(JavascriptTcpSocketEvent::Close { had_error: false }); + } + break; + } + Ok(bytes_read) => { + if sender + .send(JavascriptTcpSocketEvent::Data( + buffer[..bytes_read].to_vec(), + )) + .is_err() + { + break; + } + } + Err(error) => { + let code = io_error_code(&error); + let _ = sender.send(JavascriptTcpSocketEvent::Error { + code, + message: error.to_string(), + }); + if !close_notified.swap(true, Ordering::SeqCst) { + let _ = sender.send(JavascriptTcpSocketEvent::Close { had_error: true }); + } + break; + } + } + } + }); +} + +fn terminate_child_process_tree(kernel: &mut SidecarKernel, process: &mut ActiveProcess) { + let listener_ids = process.tcp_listeners.keys().cloned().collect::>(); + for listener_id in listener_ids { + if let Some(listener) = process.tcp_listeners.remove(&listener_id) { + let _ = listener.close(); } } - #[test] - fn created_vm_id_returns_error_for_unexpected_response() { - let error = created_vm_id(DispatchResult { - response: ResponseFrame::new( - 3, - OwnershipScope::session("conn-1", "session-1"), - ResponsePayload::Rejected(RejectedResponse { - code: String::from("invalid_state"), - message: String::from("not owned"), - }), - ), - events: Vec::new(), + let sockets = process.tcp_sockets.keys().cloned().collect::>(); + for socket_id in sockets { + if let Some(socket) = process.tcp_sockets.remove(&socket_id) { + let _ = socket.close(); + } + } + + let unix_listener_ids = process.unix_listeners.keys().cloned().collect::>(); + for listener_id in unix_listener_ids { + if let Some(listener) = process.unix_listeners.remove(&listener_id) { + let _ = listener.close(); + } + } + + let unix_sockets = process.unix_sockets.keys().cloned().collect::>(); + for socket_id in unix_sockets { + if let Some(socket) = process.unix_sockets.remove(&socket_id) { + let _ = socket.close(); + } + } + + let udp_socket_ids = process.udp_sockets.keys().cloned().collect::>(); + for socket_id in udp_socket_ids { + if let Some(mut socket) = process.udp_sockets.remove(&socket_id) { + socket.close(); + } + } + + let child_ids = process.child_processes.keys().cloned().collect::>(); + for child_id in child_ids { + let Some(mut child) = process.child_processes.remove(&child_id) else { + continue; + }; + terminate_child_process_tree(kernel, &mut child); + let _ = kernel.kill_process(EXECUTION_DRIVER_NAME, child.kernel_pid, SIGTERM); + let _ = signal_runtime_process(child.execution.child_pid(), SIGTERM); + child.kernel_handle.finish(0); + let _ = kernel.wait_and_reap(child.kernel_pid); + } +} + +fn javascript_sync_rpc_arg_str<'a>( + args: &'a [Value], + index: usize, + label: &str, +) -> Result<&'a str, SidecarError> { + args.get(index) + .and_then(Value::as_str) + .ok_or_else(|| SidecarError::InvalidState(format!("{label} must be a string argument"))) +} + +fn javascript_sync_rpc_encoding(args: &[Value]) -> Option { + args.get(1).and_then(|value| { + value.as_str().map(str::to_owned).or_else(|| { + value + .get("encoding") + .and_then(Value::as_str) + .map(str::to_owned) }) - .expect_err("unexpected vm payload should return an error"); + }) +} - match error { - SidecarError::InvalidState(message) => { - assert!(message.contains("expected vm_created response")); - assert!(message.contains("Rejected")); - } - other => panic!("expected invalid_state error, got {other:?}"), +fn javascript_sync_rpc_option_bool(args: &[Value], index: usize, key: &str) -> Option { + args.get(index) + .and_then(|value| value.get(key)) + .and_then(Value::as_bool) +} + +fn javascript_sync_rpc_option_u32( + args: &[Value], + index: usize, + key: &str, +) -> Result, SidecarError> { + let Some(value) = args.get(index).and_then(|value| { + if value.is_object() { + value.get(key) + } else if key == "mode" { + Some(value) + } else { + None } + }) else { + return Ok(None); + }; + if value.is_null() { + return Ok(None); } - #[test] - fn configure_vm_instantiates_memory_mounts_through_the_plugin_registry() { - let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = - authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let numeric = value + .as_u64() + .or_else(|| { + value + .as_f64() + .filter(|number| number.is_finite() && *number >= 0.0) + .map(|number| number as u64) + }) + .ok_or_else(|| SidecarError::InvalidState(format!("{key} must be numeric")))?; - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { - entries: vec![ - RootFilesystemEntry { - path: String::from("/workspace"), - kind: RootFilesystemEntryKind::Directory, - ..Default::default() - }, - RootFilesystemEntry { - path: String::from("/workspace/root-only.txt"), - kind: RootFilesystemEntryKind::File, - content: Some(String::from("root bootstrap file")), - ..Default::default() - }, - ], - }), - )) - .expect("bootstrap root workspace"); + u32::try_from(numeric) + .map(Some) + .map_err(|_| SidecarError::InvalidState(format!("{key} must fit within u32"))) +} - sidecar - .dispatch(request( - 5, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/workspace"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("memory"), - config: json!({}), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure mounts"); +fn javascript_sync_rpc_arg_u32( + args: &[Value], + index: usize, + label: &str, +) -> Result { + let value = javascript_sync_rpc_arg_u64(args, index, label)?; + u32::try_from(value) + .map_err(|_| SidecarError::InvalidState(format!("{label} must fit within u32"))) +} - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let hidden = vm - .kernel - .filesystem_mut() - .read_file("/workspace/root-only.txt") - .expect_err("mounted filesystem should hide root-backed file"); - assert_eq!(hidden.code(), "ENOENT"); +fn javascript_sync_rpc_arg_u32_optional( + args: &[Value], + index: usize, + label: &str, +) -> Result, SidecarError> { + javascript_sync_rpc_arg_u64_optional(args, index, label)? + .map(|value| { + u32::try_from(value) + .map_err(|_| SidecarError::InvalidState(format!("{label} must fit within u32"))) + }) + .transpose() +} - vm.kernel - .filesystem_mut() - .write_file("/workspace/from-mount.txt", b"native mount".to_vec()) - .expect("write mounted file"); - assert_eq!( - vm.kernel - .filesystem_mut() - .read_file("/workspace/from-mount.txt") - .expect("read mounted file"), - b"native mount".to_vec() - ); - assert_eq!( - vm.kernel.mounted_filesystems(), - vec![ - MountEntry { - path: String::from("/workspace"), - plugin_id: String::from("memory"), - read_only: false, - }, - MountEntry { - path: String::from("/"), - plugin_id: String::from("root"), - read_only: false, - }, - ] - ); +fn javascript_sync_rpc_arg_u64( + args: &[Value], + index: usize, + label: &str, +) -> Result { + let Some(value) = args.get(index) else { + return Err(SidecarError::InvalidState(format!("{label} is required"))); + }; + + value + .as_u64() + .or_else(|| { + value + .as_f64() + .filter(|number| number.is_finite() && *number >= 0.0) + .map(|number| number as u64) + }) + .ok_or_else(|| SidecarError::InvalidState(format!("{label} must be a numeric argument"))) +} + +fn javascript_sync_rpc_arg_u64_optional( + args: &[Value], + index: usize, + label: &str, +) -> Result, SidecarError> { + let Some(value) = args.get(index) else { + return Ok(None); + }; + if value.is_null() { + return Ok(None); } + javascript_sync_rpc_arg_u64(args, index, label).map(Some) +} - #[test] - fn configure_vm_applies_read_only_mount_wrappers() { - let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = - authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); +fn javascript_sync_rpc_stat_value(stat: VirtualStat) -> Value { + json!({ + "mode": stat.mode, + "size": stat.size, + "blocks": stat.blocks, + "dev": stat.dev, + "rdev": stat.rdev, + "isDirectory": stat.is_directory, + "isSymbolicLink": stat.is_symbolic_link, + "atimeMs": stat.atime_ms, + "mtimeMs": stat.mtime_ms, + "ctimeMs": stat.ctime_ms, + "birthtimeMs": stat.birthtime_ms, + "ino": stat.ino, + "nlink": stat.nlink, + "uid": stat.uid, + "gid": stat.gid, + }) +} - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/readonly"), - read_only: true, - plugin: MountPluginDescriptor { - id: String::from("memory"), - config: json!({}), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure readonly mount"); +fn javascript_sync_rpc_readdir_value(entries: Vec) -> Value { + json!(entries + .into_iter() + .filter(|entry| entry != "." && entry != "..") + .collect::>()) +} - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let error = vm - .kernel - .filesystem_mut() - .write_file("/readonly/blocked.txt", b"nope".to_vec()) - .expect_err("readonly mount should reject writes"); - assert_eq!(error.code(), "EROFS"); +fn javascript_sync_rpc_bytes_arg( + args: &[Value], + index: usize, + label: &str, +) -> Result, SidecarError> { + let Some(value) = args.get(index) else { + return Err(SidecarError::InvalidState(format!("{label} is required"))); + }; + + if let Some(text) = value.as_str() { + return Ok(text.as_bytes().to_vec()); } - #[test] - fn configure_vm_instantiates_host_dir_mounts_through_the_plugin_registry() { - let host_dir = temp_dir("agent-os-sidecar-host-dir"); - fs::write(host_dir.join("hello.txt"), "hello from host").expect("seed host dir"); + let Some(base64_value) = value + .get("__agentOsType") + .and_then(Value::as_str) + .filter(|kind| *kind == "bytes") + .and_then(|_| value.get("base64")) + .and_then(Value::as_str) + else { + return Err(SidecarError::InvalidState(format!( + "{label} must be a string or encoded bytes payload" + ))); + }; - let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = - authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + base64::engine::general_purpose::STANDARD + .decode(base64_value) + .map_err(|error| { + SidecarError::InvalidState(format!("{label} contains invalid base64: {error}")) + }) +} - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { - entries: vec![ - RootFilesystemEntry { - path: String::from("/workspace"), - kind: RootFilesystemEntryKind::Directory, - ..Default::default() - }, - RootFilesystemEntry { - path: String::from("/workspace/root-only.txt"), - kind: RootFilesystemEntryKind::File, - content: Some(String::from("root bootstrap file")), - ..Default::default() - }, - ], - }), +fn javascript_sync_rpc_bytes_value(bytes: &[u8]) -> Value { + json!({ + "__agentOsType": "bytes", + "base64": base64::engine::general_purpose::STANDARD.encode(bytes), + }) +} + +fn service_javascript_sync_rpc( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + socket_paths: &JavascriptSocketPathContext, + kernel: &mut SidecarKernel, + process: &mut ActiveProcess, + request: &JavascriptSyncRpcRequest, + resource_limits: &ResourceLimits, + network_counts: NetworkResourceCounts, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + match request.method.as_str() { + "dns.lookup" | "dns.resolve" | "dns.resolve4" | "dns.resolve6" => { + service_javascript_dns_sync_rpc(bridge, vm_id, dns, request) + } + "net.connect" + | "net.listen" + | "net.poll" + | "net.server_poll" + | "net.server_connections" + | "net.write" + | "net.shutdown" + | "net.destroy" + | "net.server_close" => service_javascript_net_sync_rpc( + bridge, + vm_id, + dns, + socket_paths, + kernel, + process, + request, + resource_limits, + network_counts, + ), + "dgram.createSocket" | "dgram.bind" | "dgram.send" | "dgram.poll" | "dgram.close" => { + service_javascript_dgram_sync_rpc( + bridge, + vm_id, + dns, + socket_paths, + process, + request, + resource_limits, + network_counts, + ) + } + "process.umask" => { + let new_mask = javascript_sync_rpc_arg_u32_optional(&request.args, 0, "process umask")?; + kernel + .umask(EXECUTION_DRIVER_NAME, process.kernel_pid, new_mask) + .map(|mask| json!(mask)) + .map_err(kernel_error) + } + _ => service_javascript_fs_sync_rpc(kernel, process.kernel_pid, request), + } +} + +fn service_javascript_dns_sync_rpc( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + request: &JavascriptSyncRpcRequest, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + match request.method.as_str() { + "dns.lookup" => { + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "dns.lookup requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid dns.lookup payload: {error}")) + }) + })?; + bridge.require_network_access( + vm_id, + NetworkOperation::Dns, + format_dns_resource(&payload.hostname), + )?; + let addresses = filter_dns_ip_addrs( + resolve_dns_ip_addrs(bridge, vm_id, dns, &payload.hostname)?, + payload.family, + )?; + let addresses = filter_dns_safe_ip_addrs(addresses, &payload.hostname)?; + Ok(Value::Array( + addresses + .into_iter() + .map(|ip| { + json!({ + "address": ip.to_string(), + "family": if ip.is_ipv6() { 6 } else { 4 }, + }) + }) + .collect(), )) - .expect("bootstrap root workspace"); + } + "dns.resolve" | "dns.resolve4" | "dns.resolve6" => { + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "dns.resolve requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid dns.resolve payload: {error}")) + }) + })?; + let family = match request.method.as_str() { + "dns.resolve4" => Some(4), + "dns.resolve6" => Some(6), + _ => match payload + .rrtype + .as_deref() + .unwrap_or("A") + .to_ascii_uppercase() + .as_str() + { + "A" => Some(4), + "AAAA" => Some(6), + other => { + return Err(SidecarError::InvalidState(format!( + "unsupported dns rrtype {other}" + ))) + } + }, + }; + bridge.require_network_access( + vm_id, + NetworkOperation::Dns, + format_dns_resource(&payload.hostname), + )?; + let addresses = filter_dns_ip_addrs( + resolve_dns_ip_addrs(bridge, vm_id, dns, &payload.hostname)?, + family, + )?; + let addresses = filter_dns_safe_ip_addrs(addresses, &payload.hostname)?; + Ok(Value::Array( + addresses + .into_iter() + .map(|ip| Value::String(ip.to_string())) + .collect(), + )) + } + other => Err(SidecarError::InvalidState(format!( + "unsupported JavaScript dns sync RPC method {other}" + ))), + } +} + +fn service_javascript_dgram_sync_rpc( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + socket_paths: &JavascriptSocketPathContext, + process: &mut ActiveProcess, + request: &JavascriptSyncRpcRequest, + resource_limits: &ResourceLimits, + network_counts: NetworkResourceCounts, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + match request.method.as_str() { + "dgram.createSocket" => { + check_network_resource_limit( + resource_limits.max_sockets, + network_counts.sockets, + 1, + "socket", + )?; + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "dgram.createSocket requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err( + |error| { + SidecarError::InvalidState(format!( + "invalid dgram.createSocket payload: {error}" + )) + }, + ) + })?; + let family = JavascriptUdpFamily::from_socket_type(&payload.socket_type)?; + let socket_id = process.allocate_udp_socket_id(); + process + .udp_sockets + .insert(socket_id.clone(), ActiveUdpSocket::new(family)); + Ok(json!({ + "socketId": socket_id, + "type": family.socket_type(), + })) + } + "dgram.bind" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "dgram.bind socket id")?; + let payload = request + .args + .get(1) + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "dgram.bind requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid dgram.bind payload: {error}")) + }) + })?; + let socket = process.udp_sockets.get_mut(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown UDP socket {socket_id}")) + })?; + let local_addr = socket.bind(payload.address.as_deref(), payload.port, socket_paths)?; + Ok(json!({ + "localAddress": local_addr.ip().to_string(), + "localPort": local_addr.port(), + "family": socket_addr_family(&local_addr), + })) + } + "dgram.send" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "dgram.send socket id")?; + let chunk = javascript_sync_rpc_bytes_arg(&request.args, 1, "dgram.send payload")?; + let payload = request + .args + .get(2) + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "dgram.send requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid dgram.send payload: {error}")) + }) + })?; + let socket = process.udp_sockets.get_mut(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown UDP socket {socket_id}")) + })?; + let (written, local_addr) = socket.send_to( + bridge, + vm_id, + dns, + payload.address.as_deref().unwrap_or("localhost"), + payload.port, + socket_paths, + &chunk, + )?; + Ok(json!({ + "bytes": written, + "localAddress": local_addr.ip().to_string(), + "localPort": local_addr.port(), + "family": socket_addr_family(&local_addr), + })) + } + "dgram.poll" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "dgram.poll socket id")?; + let wait_ms = + javascript_sync_rpc_arg_u64_optional(&request.args, 1, "dgram.poll wait ms")? + .unwrap_or_default(); + let event = { + let socket = process.udp_sockets.get(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown UDP socket {socket_id}")) + })?; + socket.poll(Duration::from_millis(wait_ms))? + }; + + match event { + Some(JavascriptUdpSocketEvent::Message { data, remote_addr }) => { + let family = JavascriptSocketFamily::from_ip(remote_addr.ip()); + let guest_remote_port = if is_loopback_ip(remote_addr.ip()) { + socket_paths + .guest_udp_port_for_host_port(family, remote_addr.port()) + .unwrap_or(remote_addr.port()) + } else { + remote_addr.port() + }; + Ok(json!({ + "type": "message", + "data": javascript_sync_rpc_bytes_value(&data), + "remoteAddress": remote_addr.ip().to_string(), + "remotePort": guest_remote_port, + "remoteFamily": socket_addr_family(&remote_addr), + })) + } + Some(JavascriptUdpSocketEvent::Error { code, message }) => Ok(json!({ + "type": "error", + "code": code, + "message": message, + })), + None => Ok(Value::Null), + } + } + "dgram.close" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "dgram.close socket id")?; + let mut socket = process.udp_sockets.remove(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown UDP socket {socket_id}")) + })?; + socket.close(); + Ok(Value::Null) + } + other => Err(SidecarError::InvalidState(format!( + "unsupported JavaScript dgram sync RPC method {other}" + ))), + } +} + +fn service_javascript_net_sync_rpc( + bridge: &SharedBridge, + vm_id: &str, + dns: &VmDnsConfig, + socket_paths: &JavascriptSocketPathContext, + kernel: &mut SidecarKernel, + process: &mut ActiveProcess, + request: &JavascriptSyncRpcRequest, + resource_limits: &ResourceLimits, + network_counts: NetworkResourceCounts, +) -> Result +where + B: NativeSidecarBridge + Send + 'static, + BridgeError: fmt::Debug + Send + Sync + 'static, +{ + match request.method.as_str() { + "net.connect" => { + check_network_resource_limit( + resource_limits.max_sockets, + network_counts.sockets, + 1, + "socket", + )?; + check_network_resource_limit( + resource_limits.max_connections, + network_counts.connections, + 1, + "connection", + )?; + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "net.connect requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid net.connect payload: {error}")) + }) + })?; + if let Some(path) = payload.path.as_deref() { + let guest_path = normalize_path(path); + let host_path = resolve_guest_socket_host_path(socket_paths, &guest_path); + let socket = ActiveUnixSocket::connect(&host_path, &guest_path)?; + let socket_id = process.allocate_unix_socket_id(); + process.unix_sockets.insert(socket_id.clone(), socket); + Ok(json!({ + "socketId": socket_id, + "remotePath": guest_path, + })) + } else { + let port = payload.port.ok_or_else(|| { + SidecarError::InvalidState(String::from( + "net.connect requires either a path or port", + )) + })?; + let host = payload.host.as_deref().unwrap_or("localhost"); + bridge.require_network_access( + vm_id, + NetworkOperation::Http, + format_tcp_resource(host, port), + )?; + let socket = + ActiveTcpSocket::connect(bridge, vm_id, dns, host, port, socket_paths)?; + let socket_id = process.allocate_tcp_socket_id(); + let local_addr = socket.guest_local_addr; + let remote_addr = socket.guest_remote_addr; + process.tcp_sockets.insert(socket_id.clone(), socket); + Ok(json!({ + "socketId": socket_id, + "localAddress": local_addr.ip().to_string(), + "localPort": local_addr.port(), + "remoteAddress": remote_addr.ip().to_string(), + "remotePort": remote_addr.port(), + "remoteFamily": socket_addr_family(&remote_addr), + })) + } + } + "net.listen" => { + check_network_resource_limit( + resource_limits.max_sockets, + network_counts.sockets, + 1, + "socket", + )?; + let payload = request + .args + .first() + .cloned() + .ok_or_else(|| { + SidecarError::InvalidState(String::from( + "net.listen requires a request payload", + )) + }) + .and_then(|value| { + serde_json::from_value::(value).map_err(|error| { + SidecarError::InvalidState(format!("invalid net.listen payload: {error}")) + }) + })?; + if let Some(path) = payload.path.as_deref() { + let guest_path = normalize_path(path); + if kernel.exists(&guest_path).map_err(kernel_error)? { + return Err(sidecar_net_error(std::io::Error::from_raw_os_error( + libc::EADDRINUSE, + ))); + } + + let host_path = resolve_guest_socket_host_path(socket_paths, &guest_path); + let on_host_mount = + host_mount_path_for_guest_path_from_mounts(&socket_paths.mounts, &guest_path) + .is_some(); + let listener = ActiveUnixListener::bind(&host_path, &guest_path, payload.backlog)?; + if !on_host_mount { + ensure_kernel_parent_directories(kernel, &guest_path)?; + kernel + .write_file(&guest_path, Vec::new()) + .map_err(kernel_error)?; + } + let listener_id = process.allocate_unix_listener_id(); + process.unix_listeners.insert(listener_id.clone(), listener); + Ok(json!({ + "serverId": listener_id, + "path": guest_path, + })) + } else { + let (family, host) = normalize_tcp_listen_host(payload.host.as_deref())?; + let requested_port = payload.port.unwrap_or(0); + bridge.require_network_access( + vm_id, + NetworkOperation::Listen, + format_tcp_resource(host, requested_port), + )?; + let port = allocate_guest_listen_port( + requested_port, + family, + &socket_paths.used_tcp_guest_ports, + socket_paths.listen_policy, + )?; + let listener = ActiveTcpListener::bind(host, port, payload.backlog)?; + let listener_id = process.allocate_tcp_listener_id(); + let local_addr = listener.guest_local_addr(); + process.tcp_listeners.insert(listener_id.clone(), listener); + Ok(json!({ + "serverId": listener_id, + "localAddress": local_addr.ip().to_string(), + "localPort": local_addr.port(), + "family": socket_addr_family(&local_addr), + })) + } + } + "net.poll" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "net.poll socket id")?; + let wait_ms = + javascript_sync_rpc_arg_u64_optional(&request.args, 1, "net.poll wait ms")? + .unwrap_or_default(); + let event = if let Some(socket) = process.tcp_sockets.get_mut(socket_id) { + socket.poll(Duration::from_millis(wait_ms))? + } else if let Some(socket) = process.unix_sockets.get_mut(socket_id) { + socket.poll(Duration::from_millis(wait_ms))? + } else { + return Err(SidecarError::InvalidState(format!( + "unknown net socket {socket_id}" + ))); + }; + + match event { + Some(JavascriptTcpSocketEvent::Data(chunk)) => Ok(json!({ + "type": "data", + "data": javascript_sync_rpc_bytes_value(&chunk), + })), + Some(JavascriptTcpSocketEvent::End) => Ok(json!({ + "type": "end", + })), + Some(JavascriptTcpSocketEvent::Error { code, message }) => Ok(json!({ + "type": "error", + "code": code, + "message": message, + })), + Some(JavascriptTcpSocketEvent::Close { had_error }) => { + if let Some(socket) = process.tcp_sockets.remove(socket_id) { + if let Some(listener_id) = socket.listener_id.as_deref() { + if let Some(listener) = process.tcp_listeners.get_mut(listener_id) { + listener.release_connection(socket_id); + } + } + } else if let Some(socket) = process.unix_sockets.remove(socket_id) { + if let Some(listener_id) = socket.listener_id.as_deref() { + if let Some(listener) = process.unix_listeners.get_mut(listener_id) { + listener.release_connection(socket_id); + } + } + } + Ok(json!({ + "type": "close", + "hadError": had_error, + })) + } + None => Ok(Value::Null), + } + } + "net.server_poll" => { + let listener_id = + javascript_sync_rpc_arg_str(&request.args, 0, "net.server_poll listener id")?; + let wait_ms = + javascript_sync_rpc_arg_u64_optional(&request.args, 1, "net.server_poll wait ms")? + .unwrap_or_default(); + let tcp_event = if let Some(listener) = process.tcp_listeners.get_mut(listener_id) { + Some(listener.poll(Duration::from_millis(wait_ms))?) + } else { + None + }; + + if let Some(event) = tcp_event { + return match event { + Some(JavascriptTcpListenerEvent::Connection(pending)) => { + if let Err(error) = check_network_resource_limit( + resource_limits.max_sockets, + network_counts.sockets, + 1, + "socket", + ) + .and_then(|()| { + check_network_resource_limit( + resource_limits.max_connections, + network_counts.connections, + 1, + "connection", + ) + }) { + let _ = pending.stream.shutdown(Shutdown::Both); + return Ok(json!({ + "type": "error", + "code": "EAGAIN", + "message": error.to_string(), + })); + } + let socket = ActiveTcpSocket::from_stream( + pending.stream, + Some(listener_id.to_string()), + pending.guest_local_addr, + pending.guest_remote_addr, + )?; + let socket_id = process.allocate_tcp_socket_id(); + if let Some(listener) = process.tcp_listeners.get_mut(listener_id) { + listener.register_connection(&socket_id); + } + process.tcp_sockets.insert(socket_id.clone(), socket); + Ok(json!({ + "type": "connection", + "socketId": socket_id, + "localAddress": pending.guest_local_addr.ip().to_string(), + "localPort": pending.guest_local_addr.port(), + "remoteAddress": pending.guest_remote_addr.ip().to_string(), + "remotePort": pending.guest_remote_addr.port(), + "remoteFamily": socket_addr_family(&pending.guest_remote_addr), + })) + } + Some(JavascriptTcpListenerEvent::Error { code, message }) => Ok(json!({ + "type": "error", + "code": code, + "message": message, + })), + None => Ok(Value::Null), + }; + } + + let event = { + let listener = process.unix_listeners.get_mut(listener_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net listener {listener_id}")) + })?; + listener.poll(Duration::from_millis(wait_ms))? + }; + + match event { + Some(JavascriptUnixListenerEvent::Connection(pending)) => { + if let Err(error) = check_network_resource_limit( + resource_limits.max_sockets, + network_counts.sockets, + 1, + "socket", + ) + .and_then(|()| { + check_network_resource_limit( + resource_limits.max_connections, + network_counts.connections, + 1, + "connection", + ) + }) { + let _ = pending.stream.shutdown(Shutdown::Both); + return Ok(json!({ + "type": "error", + "code": "EAGAIN", + "message": error.to_string(), + })); + } + let socket = ActiveUnixSocket::from_stream( + pending.stream, + Some(listener_id.to_string()), + pending.local_path.clone(), + pending.remote_path.clone(), + )?; + let socket_id = process.allocate_unix_socket_id(); + if let Some(listener) = process.unix_listeners.get_mut(listener_id) { + listener.register_connection(&socket_id); + } + process.unix_sockets.insert(socket_id.clone(), socket); + Ok(json!({ + "type": "connection", + "socketId": socket_id, + "localPath": pending.local_path, + "remotePath": pending.remote_path, + })) + } + Some(JavascriptUnixListenerEvent::Error { code, message }) => Ok(json!({ + "type": "error", + "code": code, + "message": message, + })), + None => Ok(Value::Null), + } + } + "net.server_connections" => { + let listener_id = javascript_sync_rpc_arg_str( + &request.args, + 0, + "net.server_connections listener id", + )?; + if let Some(listener) = process.tcp_listeners.get(listener_id) { + Ok(json!(listener.active_connection_count())) + } else { + let listener = process.unix_listeners.get(listener_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net listener {listener_id}")) + })?; + Ok(json!(listener.active_connection_count())) + } + } + "net.write" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "net.write socket id")?; + let chunk = javascript_sync_rpc_bytes_arg(&request.args, 1, "net.write chunk")?; + if let Some(socket) = process.tcp_sockets.get(socket_id) { + socket.write_all(&chunk).map(|written| json!(written)) + } else { + let socket = process.unix_sockets.get(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net socket {socket_id}")) + })?; + socket.write_all(&chunk).map(|written| json!(written)) + } + } + "net.shutdown" => { + let socket_id = + javascript_sync_rpc_arg_str(&request.args, 0, "net.shutdown socket id")?; + if let Some(socket) = process.tcp_sockets.get(socket_id) { + socket.shutdown_write()?; + } else { + let socket = process.unix_sockets.get(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net socket {socket_id}")) + })?; + socket.shutdown_write()?; + } + Ok(Value::Null) + } + "net.destroy" => { + let socket_id = javascript_sync_rpc_arg_str(&request.args, 0, "net.destroy socket id")?; + if let Some(socket) = process.tcp_sockets.remove(socket_id) { + if let Some(listener_id) = socket.listener_id.as_deref() { + if let Some(listener) = process.tcp_listeners.get_mut(listener_id) { + listener.release_connection(socket_id); + } + } + let _ = socket.close(); + Ok(Value::Null) + } else { + let socket = process.unix_sockets.remove(socket_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net socket {socket_id}")) + })?; + if let Some(listener_id) = socket.listener_id.as_deref() { + if let Some(listener) = process.unix_listeners.get_mut(listener_id) { + listener.release_connection(socket_id); + } + } + let _ = socket.close(); + Ok(Value::Null) + } + } + "net.server_close" => { + let listener_id = + javascript_sync_rpc_arg_str(&request.args, 0, "net.server_close listener id")?; + if let Some(listener) = process.tcp_listeners.remove(listener_id) { + listener.close()?; + Ok(Value::Null) + } else { + let listener = process.unix_listeners.remove(listener_id).ok_or_else(|| { + SidecarError::InvalidState(format!("unknown net listener {listener_id}")) + })?; + listener.close()?; + Ok(Value::Null) + } + } + _ => Err(SidecarError::InvalidState(format!( + "unsupported JavaScript net sync RPC method {}", + request.method + ))), + } +} + +fn service_javascript_fs_sync_rpc( + kernel: &mut SidecarKernel, + kernel_pid: u32, + request: &JavascriptSyncRpcRequest, +) -> Result { + match request.method.as_str() { + "fs.open" | "fs.openSync" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem open path")?; + let flags = javascript_sync_rpc_arg_u32(&request.args, 1, "filesystem open flags")?; + let mode = + javascript_sync_rpc_arg_u32_optional(&request.args, 2, "filesystem open mode")?; + kernel + .fd_open(EXECUTION_DRIVER_NAME, kernel_pid, path, flags, mode) + .map(|fd| json!(fd)) + .map_err(kernel_error) + } + "fs.read" | "fs.readSync" => { + let fd = javascript_sync_rpc_arg_u32(&request.args, 0, "filesystem read fd")?; + let length = usize::try_from(javascript_sync_rpc_arg_u64( + &request.args, + 1, + "filesystem read length", + )?) + .map_err(|_| { + SidecarError::InvalidState( + "filesystem read length must fit within usize".to_string(), + ) + })?; + let position = + javascript_sync_rpc_arg_u64_optional(&request.args, 2, "filesystem read position")?; + let bytes = match position { + Some(offset) => { + kernel.fd_pread(EXECUTION_DRIVER_NAME, kernel_pid, fd, length, offset) + } + None => kernel.fd_read(EXECUTION_DRIVER_NAME, kernel_pid, fd, length), + }; + bytes + .map(|payload| javascript_sync_rpc_bytes_value(&payload)) + .map_err(kernel_error) + } + "fs.write" | "fs.writeSync" => { + let fd = javascript_sync_rpc_arg_u32(&request.args, 0, "filesystem write fd")?; + let contents = + javascript_sync_rpc_bytes_arg(&request.args, 1, "filesystem write contents")?; + let position = javascript_sync_rpc_arg_u64_optional( + &request.args, + 2, + "filesystem write position", + )?; + let written = match position { + Some(offset) => { + kernel.fd_pwrite(EXECUTION_DRIVER_NAME, kernel_pid, fd, &contents, offset) + } + None => kernel.fd_write(EXECUTION_DRIVER_NAME, kernel_pid, fd, &contents), + }; + written.map(|count| json!(count)).map_err(kernel_error) + } + "fs.close" | "fs.closeSync" => { + let fd = javascript_sync_rpc_arg_u32(&request.args, 0, "filesystem close fd")?; + kernel + .fd_close(EXECUTION_DRIVER_NAME, kernel_pid, fd) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.fstat" | "fs.fstatSync" => { + let fd = javascript_sync_rpc_arg_u32(&request.args, 0, "filesystem fstat fd")?; + kernel + .fd_stat(EXECUTION_DRIVER_NAME, kernel_pid, fd) + .map_err(kernel_error)?; + kernel + .dev_fd_stat(EXECUTION_DRIVER_NAME, kernel_pid, fd) + .map(javascript_sync_rpc_stat_value) + .map_err(kernel_error) + } + "fs.readFileSync" | "fs.promises.readFile" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem readFile path")?; + let encoding = javascript_sync_rpc_encoding(&request.args); + kernel + .read_file_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(|content| match encoding.as_deref() { + Some("utf8") | Some("utf-8") => { + Value::String(String::from_utf8_lossy(&content).into_owned()) + } + _ => javascript_sync_rpc_bytes_value(&content), + }) + .map_err(kernel_error) + } + "fs.writeFileSync" | "fs.promises.writeFile" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem writeFile path")?; + let contents = + javascript_sync_rpc_bytes_arg(&request.args, 1, "filesystem writeFile contents")?; + kernel + .write_file_for_process( + EXECUTION_DRIVER_NAME, + kernel_pid, + path, + contents, + javascript_sync_rpc_option_u32(&request.args, 2, "mode")?, + ) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.statSync" | "fs.promises.stat" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem stat path")?; + kernel + .stat_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(javascript_sync_rpc_stat_value) + .map_err(kernel_error) + } + "fs.lstatSync" | "fs.promises.lstat" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem lstat path")?; + kernel + .lstat_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(javascript_sync_rpc_stat_value) + .map_err(kernel_error) + } + "fs.readdirSync" | "fs.promises.readdir" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem readdir path")?; + kernel + .read_dir_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(javascript_sync_rpc_readdir_value) + .map_err(kernel_error) + } + "fs.mkdirSync" | "fs.promises.mkdir" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem mkdir path")?; + let recursive = + javascript_sync_rpc_option_bool(&request.args, 1, "recursive").unwrap_or(false); + kernel + .mkdir_for_process( + EXECUTION_DRIVER_NAME, + kernel_pid, + path, + recursive, + javascript_sync_rpc_option_u32(&request.args, 1, "mode")?, + ) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.accessSync" | "fs.promises.access" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem access path")?; + kernel + .stat_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(|_| Value::Null) + .map_err(kernel_error) + } + "fs.copyFileSync" | "fs.promises.copyFile" => { + let source = + javascript_sync_rpc_arg_str(&request.args, 0, "filesystem copyFile source")?; + let destination = + javascript_sync_rpc_arg_str(&request.args, 1, "filesystem copyFile destination")?; + let contents = kernel + .read_file_for_process(EXECUTION_DRIVER_NAME, kernel_pid, source) + .map_err(kernel_error)?; + kernel + .write_file_for_process( + EXECUTION_DRIVER_NAME, + kernel_pid, + destination, + contents, + None, + ) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.existsSync" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem exists path")?; + kernel + .exists_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(Value::Bool) + .map_err(kernel_error) + } + "fs.readlinkSync" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem readlink path")?; + kernel + .read_link_for_process(EXECUTION_DRIVER_NAME, kernel_pid, path) + .map(Value::String) + .map_err(kernel_error) + } + "fs.symlinkSync" => { + let target = + javascript_sync_rpc_arg_str(&request.args, 0, "filesystem symlink target")?; + let link_path = + javascript_sync_rpc_arg_str(&request.args, 1, "filesystem symlink path")?; + kernel + .symlink(target, link_path) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.linkSync" => { + let source = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem link source")?; + let destination = + javascript_sync_rpc_arg_str(&request.args, 1, "filesystem link path")?; + kernel + .link(source, destination) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.renameSync" | "fs.promises.rename" => { + let source = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem rename source")?; + let destination = + javascript_sync_rpc_arg_str(&request.args, 1, "filesystem rename destination")?; + kernel + .rename(source, destination) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.rmdirSync" | "fs.promises.rmdir" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem rmdir path")?; + kernel + .remove_dir(path) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.unlinkSync" | "fs.promises.unlink" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem unlink path")?; + kernel + .remove_file(path) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.chmodSync" | "fs.promises.chmod" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem chmod path")?; + let mode = javascript_sync_rpc_arg_u32(&request.args, 1, "filesystem chmod mode")?; + kernel + .chmod(path, mode) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.chownSync" | "fs.promises.chown" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem chown path")?; + let uid = javascript_sync_rpc_arg_u32(&request.args, 1, "filesystem chown uid")?; + let gid = javascript_sync_rpc_arg_u32(&request.args, 2, "filesystem chown gid")?; + kernel + .chown(path, uid, gid) + .map(|()| Value::Null) + .map_err(kernel_error) + } + "fs.utimesSync" | "fs.promises.utimes" => { + let path = javascript_sync_rpc_arg_str(&request.args, 0, "filesystem utimes path")?; + let atime_ms = + javascript_sync_rpc_arg_u64(&request.args, 1, "filesystem utimes atime")?; + let mtime_ms = + javascript_sync_rpc_arg_u64(&request.args, 2, "filesystem utimes mtime")?; + kernel + .utimes(path, atime_ms, mtime_ms) + .map(|()| Value::Null) + .map_err(kernel_error) + } + _ => Err(SidecarError::InvalidState(format!( + "unsupported JavaScript sync RPC method {}", + request.method + ))), + } +} + +fn kernel_error(error: KernelError) -> SidecarError { + SidecarError::Kernel(error.to_string()) +} + +fn plugin_error(error: PluginError) -> SidecarError { + SidecarError::Plugin(error.to_string()) +} + +fn javascript_error(error: JavascriptExecutionError) -> SidecarError { + SidecarError::Execution(error.to_string()) +} + +fn wasm_error(error: WasmExecutionError) -> SidecarError { + SidecarError::Execution(error.to_string()) +} + +fn python_error(error: PythonExecutionError) -> SidecarError { + SidecarError::Execution(error.to_string()) +} + +fn vfs_error(error: VfsError) -> SidecarError { + SidecarError::Kernel(error.to_string()) +} + +fn parse_signal(signal: &str) -> Result { + let trimmed = signal.trim(); + if trimmed.is_empty() { + return Err(SidecarError::InvalidState(String::from( + "kill_process requires a non-empty signal", + ))); + } + + if let Ok(value) = trimmed.parse::() { + return match value { + 0 | libc::SIGINT | SIGKILL | SIGTERM | libc::SIGCONT | libc::SIGSTOP => Ok(value), + _ => Err(SidecarError::InvalidState(format!( + "unsupported kill_process signal {signal}" + ))), + }; + } + + let upper = trimmed.to_ascii_uppercase(); + let normalized = upper.strip_prefix("SIG").unwrap_or(&upper); + + signal_number_from_name(normalized).ok_or_else(|| { + SidecarError::InvalidState(format!("unsupported kill_process signal {signal}")) + }) +} + +fn signal_number_from_name(signal: &str) -> Option { + match signal { + "INT" => Some(libc::SIGINT), + "KILL" => Some(SIGKILL), + "TERM" => Some(SIGTERM), + "CONT" => Some(libc::SIGCONT), + "STOP" => Some(libc::SIGSTOP), + _ => None, + } +} + +fn runtime_child_is_alive(child_pid: u32) -> Result { + let wait_flags = WaitPidFlag::WNOHANG + | WaitPidFlag::WNOWAIT + | WaitPidFlag::WEXITED + | WaitPidFlag::WUNTRACED + | WaitPidFlag::WCONTINUED; + match wait_on_child(WaitId::Pid(Pid::from_raw(child_pid as i32)), wait_flags) { + Ok(WaitStatus::StillAlive) + | Ok(WaitStatus::Stopped(_, _)) + | Ok(WaitStatus::Continued(_)) => Ok(true), + Ok(WaitStatus::Exited(_, _)) | Ok(WaitStatus::Signaled(_, _, _)) => Ok(false), + #[cfg(any(target_os = "linux", target_os = "android"))] + Ok(WaitStatus::PtraceEvent(_, _, _) | WaitStatus::PtraceSyscall(_)) => Ok(true), + Err(nix::errno::Errno::ECHILD) => Ok(false), + Err(error) => Err(SidecarError::Execution(format!( + "failed to inspect guest runtime process {child_pid}: {error}" + ))), + } +} + +fn signal_runtime_process(child_pid: u32, signal: i32) -> Result<(), SidecarError> { + if !runtime_child_is_alive(child_pid)? { + return Ok(()); + } + + if signal == 0 { + return Ok(()); + } + + let parsed = Signal::try_from(signal).map_err(|_| { + SidecarError::InvalidState(format!("unsupported kill_process signal {signal}")) + })?; + let result = send_signal(Pid::from_raw(child_pid as i32), Some(parsed)); + + match result { + Ok(()) => Ok(()), + Err(nix::errno::Errno::ESRCH) => Ok(()), + Err(error) => Err(SidecarError::Execution(format!( + "failed to signal guest runtime process {child_pid}: {error}" + ))), + } +} + +fn error_code(error: &SidecarError) -> &'static str { + match error { + SidecarError::InvalidState(_) => "invalid_state", + SidecarError::Unauthorized(_) => "unauthorized", + SidecarError::Unsupported(_) => "unsupported", + SidecarError::FrameTooLarge(_) => "frame_too_large", + SidecarError::Kernel(_) => "kernel_error", + SidecarError::Plugin(_) => "plugin_error", + SidecarError::Execution(_) => "execution_error", + SidecarError::Bridge(_) => "bridge_error", + SidecarError::Io(_) => "io_error", + } +} + +fn guest_errno_code(message: &str) -> Option<&str> { + let (code, _) = message.split_once(':')?; + if code.len() < 2 || !code.starts_with('E') { + return None; + } + code[1..] + .bytes() + .all(|byte| byte.is_ascii_uppercase() || byte.is_ascii_digit() || byte == b'_') + .then_some(code) +} + +fn javascript_sync_rpc_error_code(error: &SidecarError) -> String { + match error { + SidecarError::Execution(message) => guest_errno_code(message) + .unwrap_or("ERR_AGENT_OS_NODE_SYNC_RPC") + .to_owned(), + _ => String::from("ERR_AGENT_OS_NODE_SYNC_RPC"), + } +} + +fn ignore_stale_javascript_sync_rpc_response(error: SidecarError) -> Result<(), SidecarError> { + match error { + SidecarError::Execution(message) + if message.ends_with("is no longer pending") + && message.starts_with("sync RPC request ") => + { + Ok(()) + } + other => Err(other), + } +} + +#[cfg(test)] +mod tests { + #[path = "/home/nathan/a5/crates/bridge/tests/support.rs"] + mod bridge_support; + + use super::*; + use crate::protocol::{ + AuthenticateRequest, BootstrapRootFilesystemRequest, ConfigureVmRequest, CreateVmRequest, + DisposeReason, GetZombieTimerCountRequest, GuestRuntimeKind, MountDescriptor, + MountPluginDescriptor, OpenSessionRequest, OwnershipScope, PermissionDescriptor, + PermissionMode, RequestFrame, RequestPayload, ResponsePayload, RootFilesystemEntry, + RootFilesystemEntryKind, SidecarPlacement, + }; + use crate::s3_plugin::test_support::MockS3Server; + use crate::sandbox_agent_plugin::test_support::MockSandboxAgentServer; + use agent_os_kernel::command_registry::CommandDriver; + use agent_os_kernel::kernel::SpawnOptions; + use agent_os_kernel::mount_table::MountEntry; + use bridge_support::RecordingBridge; + use serde_json::json; + use std::collections::BTreeMap; + use std::fs; + use std::io::{Read, Write}; + use std::net::{Shutdown, SocketAddr, TcpListener, TcpStream}; + use std::path::{Path, PathBuf}; + use std::process::Command; + use std::thread; + use std::time::{SystemTime, UNIX_EPOCH}; + + const TEST_AUTH_TOKEN: &str = "sidecar-test-token"; + const TLS_TEST_KEY_PEM: &str = "-----BEGIN PRIVATE KEY-----\n\ +MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQClvETzHfSyd1Y+\n\ +sjCfGkuyGxFMzwQlYjUrE0iwdMF774LYHFdpvtEo3sLOW6/b1xfXS/55jq+aggxS\n\ +v+vgtjrhGf/y33XzdrjxcVBRWIsgAtxMHsNKO4EQ/uA1g6zlbaSIu+ZWX3bkDuTi\n\ +K45VW69M0XSVyv8XFGYOcf8LTI87gTtXHuT92iej77IM2lHqLXCzQVr+NQ9yvXld\n\ +9yHlA2ZfYqhkSTLdDablqfgirrQIzZzLypSGQwZUU06nCtZ+dg6SNV4TGL4NqekD\n\ +jXR3BvmZu5l4sGAsNfFVjLx6hxsLt8uqn65sCAwBDdfucR+39+pHA+esj6NAWAFO\n\ +J9CB94sfAgMBAAECggEABQTA772x+a98aJSbvU2eCiwgp3tDTGB/bKj+U/2NGFQl\n\ +2aZuDTEugzbPnlEPb7BBNA9EiujDr4GNnvnZyimqecOASRn0J+Wp7wG35Waxe8wq\n\ +YJGz5y0LGPkmz+gHVcEusMdDz8y/PGOpEaIxAquukLxs89Y8SDYhawGPsAdm9O3F\n\ +4a+aosyQwS26mkZ/1WZOTsOVd4A1/1pxBvsANURj+pq7ed/1WqgrZBN/BG1TX5Xm\n\ +DZeYy01kTCMWtcAb4f8PxGpbkSGMvBb+Mj5XtZByvfQeC+Cs5ECXhmJtVaYVUHhT\n\ +vI0oTMGvit9ffoYNds0qTeZpEeineaDH3sD16D037QKBgQDX5b65KfIVH0/WvcbJ\n\ +Gx2Wh7knXdDBky40wdq4buKK+ImzPPRxOsQ+xEMgEaZs8gb7LBapbB0cZ+YsKBOt\n\ +4FY86XQU5V5ju2ntldIIIaugIGgvGS0jdRMH3ux6iEjPZE6Fm7/s8bjIgqB7keWh\n\ +1rcZwDrwMzqwAUoBTJX58OY/fQKBgQDEhT5U7TqgEFVSspYh8c8yVRV9udiphPH3\n\ +3XIbo9iV3xzNFdwtNHC+2eLM+4J3WKjhB0UvzrlIegSqKPIsy+0nD1uzaU+O72gg\n\ +7+NKSh0RT61UDolk+P4s/2+5tnZqSNYO7Sd/svE/rkwIEtDEI5tb1nqq75h/HDEW\n\ +k56GHAxvywKBgGmGmTdmIjZizKJYti4b+9VU15I/T8ceCmqtChw1zrNAkgWy2IPz\n\ +xnIreefV2LPNhM4GGbmL55q3yhBxMlU9nsk9DokcJ4u10ivXnAZvdrTYwjOrKZ34\n\ +HmotcwbdUEFWdO7nVuMYr0oKVyivAj+ddHe4ttYrJBddOe/yoCe/sLr9AoGBAKHL\n\ +IVpCRXXqfJStOzWPI4rIyfzMuTg3oA71XjCrYHFjUw715GPDPN+j+znQB8XCVKeP\n\ +mMKXa6vj6Vs+gsOm0QTLfC/lj/6Z1Bzp4zMSeYP7GTSPE0bySDE7y/wV4L/4X2PC\n\ +lDZqWHyZPzeWZhJVTl754dxBjkd4KmHv/x9ikEqpAoGBAJNA0u0fKhdWDz32+a2F\n\ ++plJ18kQvGuwKFWIIVHBDc0wCxLKWKr5wgkhdcAEpy4mgosiZ09DzV/OpQBBHVWZ\n\ +v/Cn/DwZyoiXIi5onf7AqWIhw+aem+oMbugbSIYqDwYkwnN79tsza0KC1ScphIuf\n\ +vKoOAdY4xOcG9BEZZoKVOa8R\n\ +-----END PRIVATE KEY-----\n"; + const TLS_TEST_CERT_PEM: &str = "-----BEGIN CERTIFICATE-----\n\ +MIIDCTCCAfGgAwIBAgIUJqRgTEIlpbfqbQnyo9hxLyIn3qYwDQYJKoZIhvcNAQEL\n\ +BQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTI2MDQwNTA3MTAwOVoXDTI2MDQw\n\ +NjA3MTAwOVowFDESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEF\n\ +AAOCAQ8AMIIBCgKCAQEApbxE8x30sndWPrIwnxpLshsRTM8EJWI1KxNIsHTBe++C\n\ +2BxXab7RKN7Czluv29cX10v+eY6vmoIMUr/r4LY64Rn/8t9183a48XFQUViLIALc\n\ +TB7DSjuBEP7gNYOs5W2kiLvmVl925A7k4iuOVVuvTNF0lcr/FxRmDnH/C0yPO4E7\n\ +Vx7k/dono++yDNpR6i1ws0Fa/jUPcr15Xfch5QNmX2KoZEky3Q2m5an4Iq60CM2c\n\ +y8qUhkMGVFNOpwrWfnYOkjVeExi+DanpA410dwb5mbuZeLBgLDXxVYy8eocbC7fL\n\ +qp+ubAgMAQ3X7nEft/fqRwPnrI+jQFgBTifQgfeLHwIDAQABo1MwUTAdBgNVHQ4E\n\ +FgQUwViZyKE6S2vgTAkexnZFccSwoPMwHwYDVR0jBBgwFoAUwViZyKE6S2vgTAke\n\ +xnZFccSwoPMwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAadmK\n\ +3Ugrvep6glHAfgPP54um9cjJZQZDPn5I7yvgDr/Zp/u/UMW/OUKSfL1VNHlbAVLc\n\ +Yzq2RVTrJKObiTSoy99OzYkEdgfuEBBP7XBEQlqoOGYNRR+IZXBBiQ+m9CtajNwQ\n\ +G6mr9//zZtV1y2UUBgtxVpry5iOekpkr8iXyDLnGpS2gKL5dwXCzWCKVCO3qVotn\n\ +r6FBg4DCBMkwO6xOVN2yInPd6CPy/JAUPW50zWPnn4DKfeAAU0C+E75HN65jozdi\n\ +12yT4K772P8oSecGPInZhqJgOv1q0BDG8gccOxX1PA4sE00Enqlbvxz7sku9y4zp\n\ +ykAheWCsAteSEWVc0w==\n\ +-----END CERTIFICATE-----\n"; + + fn request( + request_id: u64, + ownership: OwnershipScope, + payload: RequestPayload, + ) -> RequestFrame { + RequestFrame::new(request_id, ownership, payload) + } + + fn create_test_sidecar() -> NativeSidecar { + NativeSidecar::with_config( + RecordingBridge::default(), + NativeSidecarConfig { + sidecar_id: String::from("sidecar-test"), + compile_cache_root: Some(std::env::temp_dir().join("agent-os-sidecar-test-cache")), + expected_auth_token: Some(String::from(TEST_AUTH_TOKEN)), + ..NativeSidecarConfig::default() + }, + ) + .expect("create sidecar") + } + + fn unexpected_response_error(expected: &str, other: ResponsePayload) -> SidecarError { + SidecarError::InvalidState(format!("expected {expected} response, got {other:?}")) + } + + fn authenticated_connection_id(auth: DispatchResult) -> Result { + match auth.response.payload { + ResponsePayload::Authenticated(response) => { + assert_eq!( + auth.response.ownership, + OwnershipScope::connection(&response.connection_id) + ); + Ok(response.connection_id) + } + other => Err(unexpected_response_error("authenticated", other)), + } + } + + fn opened_session_id(session: DispatchResult) -> Result { + match session.response.payload { + ResponsePayload::SessionOpened(response) => Ok(response.session_id), + other => Err(unexpected_response_error("session_opened", other)), + } + } + + fn created_vm_id(response: DispatchResult) -> Result { + match response.response.payload { + ResponsePayload::VmCreated(response) => Ok(response.vm_id), + other => Err(unexpected_response_error("vm_created", other)), + } + } + + fn authenticate_and_open_session( + sidecar: &mut NativeSidecar, + ) -> Result<(String, String), SidecarError> { + let auth = sidecar + .dispatch(request( + 1, + OwnershipScope::connection("conn-1"), + RequestPayload::Authenticate(AuthenticateRequest { + client_name: String::from("service-tests"), + auth_token: String::from(TEST_AUTH_TOKEN), + }), + )) + .expect("authenticate"); + let connection_id = authenticated_connection_id(auth)?; + + let session = sidecar + .dispatch(request( + 2, + OwnershipScope::connection(&connection_id), + RequestPayload::OpenSession(OpenSessionRequest { + placement: SidecarPlacement::Shared { pool: None }, + metadata: BTreeMap::new(), + }), + )) + .expect("open session"); + let session_id = opened_session_id(session)?; + Ok((connection_id, session_id)) + } + + fn create_vm( + sidecar: &mut NativeSidecar, + connection_id: &str, + session_id: &str, + permissions: Vec, + ) -> Result { + create_vm_with_metadata( + sidecar, + connection_id, + session_id, + permissions, + BTreeMap::new(), + ) + } + + fn create_vm_with_metadata( + sidecar: &mut NativeSidecar, + connection_id: &str, + session_id: &str, + permissions: Vec, + metadata: BTreeMap, + ) -> Result { + let response = sidecar + .dispatch(request( + 3, + OwnershipScope::session(connection_id, session_id), + RequestPayload::CreateVm(CreateVmRequest { + runtime: GuestRuntimeKind::JavaScript, + metadata, + root_filesystem: Default::default(), + permissions, + }), + )) + .expect("create vm"); + + created_vm_id(response) + } + + fn temp_dir(prefix: &str) -> PathBuf { + let suffix = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("clock should be monotonic enough for temp paths") + .as_nanos(); + let path = std::env::temp_dir().join(format!("{prefix}-{suffix}")); + fs::create_dir_all(&path).expect("create temp dir"); + path + } + + fn write_fixture(path: &Path, contents: &str) { + fs::write(path, contents).expect("write fixture"); + } + + fn assert_node_available() { + let output = Command::new("node") + .arg("--version") + .output() + .expect("spawn node --version"); + assert!( + output.status.success(), + "node must be available for python dispatch tests" + ); + } + + fn run_javascript_entry( + sidecar: &mut NativeSidecar, + vm_id: &str, + cwd: &Path, + process_id: &str, + allowed_node_builtins: &str, + ) -> (String, String, Option) { + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.to_owned(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.to_owned(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + allowed_node_builtins.to_owned(), + )]), + cwd: cwd.to_path_buf(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(vm_id).expect("javascript vm"); + vm.active_processes.insert( + process_id.to_owned(), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(vm_id).expect("javascript vm"); + vm.active_processes + .get(process_id) + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript process {process_id} disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(vm_id, process_id, event) + .expect("handle javascript event"); + } + + (stdout, stderr, exit_code) + } + + fn start_fake_javascript_process( + sidecar: &mut NativeSidecar, + vm_id: &str, + cwd: &Path, + process_id: &str, + allowed_node_builtins: &str, + ) { + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.to_owned(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.to_owned(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + allowed_node_builtins.to_owned(), + )]), + cwd: cwd.to_path_buf(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + let vm = sidecar.vms.get_mut(vm_id).expect("javascript vm"); + vm.active_processes.insert( + process_id.to_owned(), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + fn call_javascript_sync_rpc( + sidecar: &mut NativeSidecar, + vm_id: &str, + process_id: &str, + request: JavascriptSyncRpcRequest, + ) -> Result { + let bridge = sidecar.bridge.clone(); + let (dns, socket_paths, counts, limits) = { + let vm = sidecar.vms.get(vm_id).expect("javascript vm"); + ( + vm.dns.clone(), + build_javascript_socket_path_context(vm).expect("build socket path context"), + vm.active_processes + .get(process_id) + .expect("javascript process") + .network_resource_counts(), + ResourceLimits::default(), + ) + }; + + let vm = sidecar.vms.get_mut(vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut(process_id) + .expect("javascript process"); + service_javascript_sync_rpc( + &bridge, + vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &request, + &limits, + counts, + ) + } + + #[test] + fn dispose_vm_removes_per_vm_javascript_import_cache_directory() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_a = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm a"); + let vm_b = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm b"); + + let cache_path_a = sidecar + .javascript_engine + .materialize_import_cache_for_vm(&vm_a) + .expect("materialize vm a import cache") + .to_path_buf(); + let cache_path_b = sidecar + .javascript_engine + .materialize_import_cache_for_vm(&vm_b) + .expect("materialize vm b import cache") + .to_path_buf(); + let cache_root_a = cache_path_a + .parent() + .expect("vm a cache parent") + .to_path_buf(); + let cache_root_b = cache_path_b + .parent() + .expect("vm b cache parent") + .to_path_buf(); + + assert_ne!(cache_root_a, cache_root_b); + assert!(cache_root_a.exists(), "vm a cache root should exist"); + assert!(cache_root_b.exists(), "vm b cache root should exist"); + + sidecar + .dispose_vm_internal(&connection_id, &session_id, &vm_a, DisposeReason::Requested) + .expect("dispose vm a"); + + assert!( + !cache_root_a.exists(), + "vm a cache root should be removed on dispose" + ); + assert!( + cache_root_b.exists(), + "vm b cache root should remain until that VM is disposed" + ); + assert!( + sidecar + .javascript_engine + .import_cache_path_for_vm(&vm_a) + .is_none(), + "vm a cache entry should be removed from the engine" + ); + assert_eq!( + sidecar.javascript_engine.import_cache_path_for_vm(&vm_b), + Some(cache_path_b.as_path()) + ); + + sidecar + .dispose_vm_internal(&connection_id, &session_id, &vm_b, DisposeReason::Requested) + .expect("dispose vm b"); + assert!( + !cache_root_b.exists(), + "vm b cache root should be removed on dispose" + ); + } + + #[test] + fn get_zombie_timer_count_reports_kernel_state_before_and_after_waitpid() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + let zombie_pid = { + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + vm.kernel + .register_driver(CommandDriver::new("test-driver", ["test-zombie"])) + .expect("register test driver"); + let process = vm + .kernel + .spawn_process( + "test-zombie", + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from("test-driver")), + ..SpawnOptions::default() + }, + ) + .expect("spawn test process"); + process.finish(17); + assert_eq!(vm.kernel.zombie_timer_count(), 1); + process.pid() + }; + + let zombie_count = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::GetZombieTimerCount(GetZombieTimerCountRequest::default()), + )) + .expect("query zombie count"); + match zombie_count.response.payload { + ResponsePayload::ZombieTimerCount(response) => assert_eq!(response.count, 1), + other => panic!("unexpected zombie count response: {other:?}"), + } + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let waited = vm.kernel.waitpid(zombie_pid).expect("waitpid"); + assert_eq!(waited.pid, zombie_pid); + assert_eq!(waited.status, 17); + assert_eq!(vm.kernel.zombie_timer_count(), 0); + } + + let reaped_count = sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::GetZombieTimerCount(GetZombieTimerCountRequest::default()), + )) + .expect("query reaped zombie count"); + match reaped_count.response.payload { + ResponsePayload::ZombieTimerCount(response) => assert_eq!(response.count, 0), + other => panic!("unexpected zombie count response: {other:?}"), + } + } + + #[test] + fn parse_signal_only_accepts_whitelisted_guest_signals() { + assert_eq!(parse_signal("SIGINT").expect("parse SIGINT"), libc::SIGINT); + assert_eq!(parse_signal("kill").expect("parse SIGKILL"), SIGKILL); + assert_eq!(parse_signal("15").expect("parse numeric SIGTERM"), SIGTERM); + assert_eq!( + parse_signal("SIGCONT").expect("parse SIGCONT"), + libc::SIGCONT + ); + assert_eq!( + parse_signal("SIGSTOP").expect("parse SIGSTOP"), + libc::SIGSTOP + ); + assert_eq!(parse_signal("0").expect("parse signal 0"), 0); + assert!(parse_signal("SIGUSR1").is_err()); + } + + #[test] + fn runtime_child_liveness_only_tracks_owned_children() { + assert!( + !runtime_child_is_alive(std::process::id()).expect("current pid is not a child"), + "current process should not be treated as a guest runtime child" + ); + + let mut child = Command::new("sh") + .arg("-c") + .arg("sleep 10") + .spawn() + .expect("spawn child process"); + let child_pid = child.id(); + + assert!( + runtime_child_is_alive(child_pid).expect("inspect running child"), + "running child should be considered alive" + ); + + signal_runtime_process(child_pid, SIGTERM).expect("signal running child"); + child.wait().expect("wait for signaled child"); + + assert!( + !runtime_child_is_alive(child_pid).expect("inspect reaped child"), + "reaped child should no longer be considered alive" + ); + signal_runtime_process(child_pid, SIGTERM).expect("ignore reaped child"); + } + + #[test] + fn authenticated_connection_id_returns_error_for_unexpected_response() { + let error = authenticated_connection_id(DispatchResult { + response: ResponseFrame::new( + 1, + OwnershipScope::connection("conn-1"), + ResponsePayload::SessionOpened(SessionOpenedResponse { + session_id: String::from("session-1"), + owner_connection_id: String::from("conn-1"), + }), + ), + events: Vec::new(), + }) + .expect_err("unexpected auth payload should return an error"); + + match error { + SidecarError::InvalidState(message) => { + assert!(message.contains("expected authenticated response")); + assert!(message.contains("SessionOpened")); + } + other => panic!("expected invalid_state error, got {other:?}"), + } + } + + #[test] + fn opened_session_id_returns_error_for_unexpected_response() { + let error = opened_session_id(DispatchResult { + response: ResponseFrame::new( + 2, + OwnershipScope::connection("conn-1"), + ResponsePayload::VmCreated(VmCreatedResponse { + vm_id: String::from("vm-1"), + }), + ), + events: Vec::new(), + }) + .expect_err("unexpected session payload should return an error"); + + match error { + SidecarError::InvalidState(message) => { + assert!(message.contains("expected session_opened response")); + assert!(message.contains("VmCreated")); + } + other => panic!("expected invalid_state error, got {other:?}"), + } + } + + #[test] + fn created_vm_id_returns_error_for_unexpected_response() { + let error = created_vm_id(DispatchResult { + response: ResponseFrame::new( + 3, + OwnershipScope::session("conn-1", "session-1"), + ResponsePayload::Rejected(RejectedResponse { + code: String::from("invalid_state"), + message: String::from("not owned"), + }), + ), + events: Vec::new(), + }) + .expect_err("unexpected vm payload should return an error"); + + match error { + SidecarError::InvalidState(message) => { + assert!(message.contains("expected vm_created response")); + assert!(message.contains("Rejected")); + } + other => panic!("expected invalid_state error, got {other:?}"), + } + } + + #[test] + fn configure_vm_instantiates_memory_mounts_through_the_plugin_registry() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { + entries: vec![ + RootFilesystemEntry { + path: String::from("/workspace"), + kind: RootFilesystemEntryKind::Directory, + ..Default::default() + }, + RootFilesystemEntry { + path: String::from("/workspace/root-only.txt"), + kind: RootFilesystemEntryKind::File, + content: Some(String::from("root bootstrap file")), + ..Default::default() + }, + ], + }), + )) + .expect("bootstrap root workspace"); + + sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("memory"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure mounts"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let hidden = vm + .kernel + .filesystem_mut() + .read_file("/workspace/root-only.txt") + .expect_err("mounted filesystem should hide root-backed file"); + assert_eq!(hidden.code(), "ENOENT"); + + vm.kernel + .filesystem_mut() + .write_file("/workspace/from-mount.txt", b"native mount".to_vec()) + .expect("write mounted file"); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/workspace/from-mount.txt") + .expect("read mounted file"), + b"native mount".to_vec() + ); + assert_eq!( + vm.kernel.mounted_filesystems(), + vec![ + MountEntry { + path: String::from("/workspace"), + plugin_id: String::from("memory"), + read_only: false, + }, + MountEntry { + path: String::from("/"), + plugin_id: String::from("root"), + read_only: false, + }, + ] + ); + } + + #[test] + fn configure_vm_applies_read_only_mount_wrappers() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/readonly"), + read_only: true, + plugin: MountPluginDescriptor { + id: String::from("memory"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure readonly mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let error = vm + .kernel + .filesystem_mut() + .write_file("/readonly/blocked.txt", b"nope".to_vec()) + .expect_err("readonly mount should reject writes"); + assert_eq!(error.code(), "EROFS"); + } + + #[test] + fn configure_vm_instantiates_host_dir_mounts_through_the_plugin_registry() { + let host_dir = temp_dir("agent-os-sidecar-host-dir"); + fs::write(host_dir.join("hello.txt"), "hello from host").expect("seed host dir"); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { + entries: vec![ + RootFilesystemEntry { + path: String::from("/workspace"), + kind: RootFilesystemEntryKind::Directory, + ..Default::default() + }, + RootFilesystemEntry { + path: String::from("/workspace/root-only.txt"), + kind: RootFilesystemEntryKind::File, + content: Some(String::from("root bootstrap file")), + ..Default::default() + }, + ], + }), + )) + .expect("bootstrap root workspace"); + + sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("host_dir"), + config: json!({ + "hostPath": host_dir, + "readOnly": false, + }), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure host_dir mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let hidden = vm + .kernel + .filesystem_mut() + .read_file("/workspace/root-only.txt") + .expect_err("mounted host dir should hide root-backed file"); + assert_eq!(hidden.code(), "ENOENT"); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/workspace/hello.txt") + .expect("read mounted host file"), + b"hello from host".to_vec() + ); + + vm.kernel + .filesystem_mut() + .write_file("/workspace/from-vm.txt", b"native host dir".to_vec()) + .expect("write host dir file"); + assert_eq!( + fs::read_to_string(host_dir.join("from-vm.txt")).expect("read host output"), + "native host dir" + ); + + fs::remove_dir_all(host_dir).expect("remove temp dir"); + } + + #[test] + fn configure_vm_js_bridge_mount_preserves_hard_link_identity() { + let mut sidecar = create_test_sidecar(); + sidecar + .bridge + .inspect(|bridge| { + bridge.seed_directory( + "/workspace", + vec![agent_os_bridge::DirectoryEntry { + name: String::from("original.txt"), + kind: FileKind::File, + }], + ); + bridge.seed_file("/workspace/original.txt", b"hello world".to_vec()); + }) + .expect("seed js bridge filesystem"); + + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("js_bridge"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure js_bridge mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + vm.kernel + .filesystem_mut() + .link("/workspace/original.txt", "/workspace/linked.txt") + .expect("create js bridge hard link"); + + let original = vm + .kernel + .filesystem_mut() + .stat("/workspace/original.txt") + .expect("stat original"); + let linked = vm + .kernel + .filesystem_mut() + .stat("/workspace/linked.txt") + .expect("stat linked"); + assert_eq!(original.ino, linked.ino); + assert_eq!(original.nlink, 2); + assert_eq!(linked.nlink, 2); + + vm.kernel + .filesystem_mut() + .write_file("/workspace/linked.txt", b"updated".to_vec()) + .expect("write through hard link"); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/workspace/original.txt") + .expect("read original through shared inode"), + b"updated".to_vec() + ); + + vm.kernel + .filesystem_mut() + .remove_file("/workspace/original.txt") + .expect("remove original hard link"); + assert!(!vm + .kernel + .filesystem() + .exists("/workspace/original.txt") + .expect("check removed original")); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/workspace/linked.txt") + .expect("read surviving hard link"), + b"updated".to_vec() + ); + assert_eq!( + vm.kernel + .filesystem_mut() + .stat("/workspace/linked.txt") + .expect("stat surviving hard link") + .nlink, + 1 + ); + } + + #[test] + fn configure_vm_js_bridge_mount_preserves_metadata_updates() { + let mut sidecar = create_test_sidecar(); + sidecar + .bridge + .inspect(|bridge| { + bridge.seed_directory( + "/workspace", + vec![agent_os_bridge::DirectoryEntry { + name: String::from("original.txt"), + kind: FileKind::File, + }], + ); + bridge.seed_file("/workspace/original.txt", b"hello world".to_vec()); + }) + .expect("seed js bridge filesystem"); + + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("js_bridge"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure js_bridge mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + vm.kernel + .filesystem_mut() + .link("/workspace/original.txt", "/workspace/linked.txt") + .expect("create js bridge hard link"); + + vm.kernel + .filesystem_mut() + .chown("/workspace/original.txt", 2000, 3000) + .expect("update js bridge ownership"); + vm.kernel + .filesystem_mut() + .utimes( + "/workspace/linked.txt", + 1_700_000_000_000, + 1_710_000_000_000, + ) + .expect("update js bridge timestamps"); + + let original = vm + .kernel + .filesystem_mut() + .stat("/workspace/original.txt") + .expect("stat original"); + let linked = vm + .kernel + .filesystem_mut() + .stat("/workspace/linked.txt") + .expect("stat linked"); + + assert_eq!(original.uid, 2000); + assert_eq!(original.gid, 3000); + assert_eq!(linked.uid, 2000); + assert_eq!(linked.gid, 3000); + assert_eq!(original.atime_ms, 1_700_000_000_000); + assert_eq!(original.mtime_ms, 1_710_000_000_000); + assert_eq!(linked.atime_ms, 1_700_000_000_000); + assert_eq!(linked.mtime_ms, 1_710_000_000_000); + } + + #[test] + fn configure_vm_instantiates_sandbox_agent_mounts_through_the_plugin_registry() { + let server = MockSandboxAgentServer::start("agent-os-sidecar-sandbox", None); + fs::write(server.root().join("hello.txt"), "hello from sandbox") + .expect("seed sandbox file"); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { + entries: vec![ + RootFilesystemEntry { + path: String::from("/sandbox"), + kind: RootFilesystemEntryKind::Directory, + ..Default::default() + }, + RootFilesystemEntry { + path: String::from("/sandbox/root-only.txt"), + kind: RootFilesystemEntryKind::File, + content: Some(String::from("root bootstrap file")), + ..Default::default() + }, + ], + }), + )) + .expect("bootstrap root sandbox dir"); + + sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/sandbox"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("sandbox_agent"), + config: json!({ + "baseUrl": server.base_url(), + }), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure sandbox_agent mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let hidden = vm + .kernel + .filesystem_mut() + .read_file("/sandbox/root-only.txt") + .expect_err("mounted sandbox should hide root-backed file"); + assert_eq!(hidden.code(), "ENOENT"); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/sandbox/hello.txt") + .expect("read mounted sandbox file"), + b"hello from sandbox".to_vec() + ); + + vm.kernel + .filesystem_mut() + .write_file("/sandbox/from-vm.txt", b"native sandbox mount".to_vec()) + .expect("write sandbox file"); + assert_eq!( + fs::read_to_string(server.root().join("from-vm.txt")).expect("read sandbox output"), + "native sandbox mount" + ); + } + + #[test] + fn configure_vm_instantiates_s3_mounts_through_the_plugin_registry() { + let server = MockS3Server::start(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { + entries: vec![ + RootFilesystemEntry { + path: String::from("/data"), + kind: RootFilesystemEntryKind::Directory, + ..Default::default() + }, + RootFilesystemEntry { + path: String::from("/data/root-only.txt"), + kind: RootFilesystemEntryKind::File, + content: Some(String::from("root bootstrap file")), + ..Default::default() + }, + ], + }), + )) + .expect("bootstrap root s3 dir"); + + sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/data"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("s3"), + config: json!({ + "bucket": "test-bucket", + "prefix": "service-test", + "region": "us-east-1", + "endpoint": server.base_url(), + "credentials": { + "accessKeyId": "minioadmin", + "secretAccessKey": "minioadmin", + }, + "chunkSize": 8, + "inlineThreshold": 4, + }), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure s3 mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let hidden = vm + .kernel + .filesystem_mut() + .read_file("/data/root-only.txt") + .expect_err("mounted s3 fs should hide root-backed file"); + assert_eq!(hidden.code(), "ENOENT"); + + vm.kernel + .filesystem_mut() + .write_file("/data/from-vm.txt", b"native s3 mount".to_vec()) + .expect("write s3-backed file"); + assert_eq!( + vm.kernel + .filesystem_mut() + .read_file("/data/from-vm.txt") + .expect("read s3-backed file"), + b"native s3 mount".to_vec() + ); + drop(sidecar); + + let requests = server.requests(); + assert!( + requests.iter().any(|request| request.method == "PUT"), + "expected the native plugin to persist data back to S3" + ); + assert!( + requests + .iter() + .any(|request| request.path.contains("filesystem-manifest.json")), + "expected the native plugin to store a manifest object" + ); + } + + #[test] + fn bridge_permissions_map_symlink_operations_to_symlink_access() { + let bridge = SharedBridge::new(RecordingBridge::default()); + let permissions = bridge_permissions(bridge.clone(), "vm-symlink"); + let check = permissions + .filesystem + .as_ref() + .expect("filesystem permission callback"); + + let decision = check(&FsAccessRequest { + vm_id: String::from("ignored-by-bridge"), + op: FsOperation::Symlink, + path: String::from("/workspace/link.txt"), + }); + assert!(decision.allow); + + let recorded = bridge + .inspect(|bridge| bridge.filesystem_permission_requests.clone()) + .expect("inspect bridge"); + assert_eq!( + recorded, + vec![FilesystemPermissionRequest { + vm_id: String::from("vm-symlink"), + path: String::from("/workspace/link.txt"), + access: FilesystemAccess::Symlink, + }] + ); + } + + #[test] + fn parse_resource_limits_reads_filesystem_limits() { + let metadata = BTreeMap::from([ + (String::from("resource.max_sockets"), String::from("8")), + (String::from("resource.max_connections"), String::from("4")), + ( + String::from("resource.max_filesystem_bytes"), + String::from("4096"), + ), + ( + String::from("resource.max_inode_count"), + String::from("128"), + ), + ( + String::from("resource.max_blocking_read_ms"), + String::from("250"), + ), + ( + String::from("resource.max_pread_bytes"), + String::from("8192"), + ), + ( + String::from("resource.max_fd_write_bytes"), + String::from("4096"), + ), + ( + String::from("resource.max_process_argv_bytes"), + String::from("2048"), + ), + ( + String::from("resource.max_process_env_bytes"), + String::from("1024"), + ), + ( + String::from("resource.max_readdir_entries"), + String::from("32"), + ), + (String::from("resource.max_wasm_fuel"), String::from("5000")), + ( + String::from("resource.max_wasm_memory_bytes"), + String::from("131072"), + ), + ( + String::from("resource.max_wasm_stack_bytes"), + String::from("262144"), + ), + ]); + + let limits = parse_resource_limits(&metadata).expect("parse resource limits"); + assert_eq!(limits.max_sockets, Some(8)); + assert_eq!(limits.max_connections, Some(4)); + assert_eq!(limits.max_filesystem_bytes, Some(4096)); + assert_eq!(limits.max_inode_count, Some(128)); + assert_eq!(limits.max_blocking_read_ms, Some(250)); + assert_eq!(limits.max_pread_bytes, Some(8192)); + assert_eq!(limits.max_fd_write_bytes, Some(4096)); + assert_eq!(limits.max_process_argv_bytes, Some(2048)); + assert_eq!(limits.max_process_env_bytes, Some(1024)); + assert_eq!(limits.max_readdir_entries, Some(32)); + assert_eq!(limits.max_wasm_fuel, Some(5000)); + assert_eq!(limits.max_wasm_memory_bytes, Some(131072)); + assert_eq!(limits.max_wasm_stack_bytes, Some(262144)); + } + + #[test] + fn create_vm_applies_filesystem_permission_descriptors_to_kernel_access() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm( + &mut sidecar, + &connection_id, + &session_id, + vec![ + PermissionDescriptor { + capability: String::from("fs"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("fs.read"), + mode: PermissionMode::Deny, + }, + ], + ) + .expect("create vm"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + vm.kernel + .filesystem_mut() + .write_file("/blocked.txt", b"nope".to_vec()) + .expect("write should be allowed"); + + let read_error = vm + .kernel + .filesystem_mut() + .read_file("/blocked.txt") + .expect_err("read should be denied"); + assert_eq!(read_error.code(), "EACCES"); + } + + #[test] + fn configure_vm_mounts_require_fs_write_permission() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + sidecar + .bridge + .set_vm_permissions( + &vm_id, + &[PermissionDescriptor { + capability: String::from("fs.write"), + mode: PermissionMode::Deny, + }], + ) + .expect("set vm permissions"); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("memory"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("dispatch configure vm"); + + match result.response.payload { + ResponsePayload::Rejected(rejected) => { + assert_eq!(rejected.code, "kernel_error"); + assert!( + rejected.message.contains("EACCES"), + "unexpected error: {}", + rejected.message + ); + } + other => panic!("expected rejected response, got {other:?}"), + } + } + + #[test] + fn configure_vm_sensitive_mounts_require_fs_mount_sensitive_permission() { + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + sidecar + .bridge + .set_vm_permissions( + &vm_id, + &[ + PermissionDescriptor { + capability: String::from("fs.write"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("fs.mount_sensitive"), + mode: PermissionMode::Deny, + }, + ], + ) + .expect("set vm permissions"); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/etc"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("memory"), + config: json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("dispatch configure vm"); + + match result.response.payload { + ResponsePayload::Rejected(rejected) => { + assert_eq!(rejected.code, "kernel_error"); + assert!( + rejected.message.contains("EACCES"), + "unexpected error: {}", + rejected.message + ); + assert!( + rejected.message.contains("fs.mount_sensitive"), + "unexpected error: {}", + rejected.message + ); + } + other => panic!("expected rejected response, got {other:?}"), + } + } + + #[test] + fn scoped_host_filesystem_unscoped_target_requires_exact_guest_root_prefix() { + let filesystem = ScopedHostFilesystem::new( + HostFilesystem::new(SharedBridge::new(RecordingBridge::default()), "vm-1"), + "/data", + ); + + assert_eq!( + filesystem.unscoped_target(String::from("/database")), + "/database" + ); + assert_eq!( + filesystem.unscoped_target(String::from("/data/nested.txt")), + "/nested.txt" + ); + assert_eq!(filesystem.unscoped_target(String::from("/data")), "/"); + } + + #[test] + fn scoped_host_filesystem_realpath_preserves_paths_outside_guest_root() { + let bridge = SharedBridge::new(RecordingBridge::default()); + bridge + .inspect(|bridge| { + agent_os_bridge::FilesystemBridge::symlink( + bridge, + SymlinkRequest { + vm_id: String::from("vm-1"), + target_path: String::from("/database"), + link_path: String::from("/data/alias"), + }, + ) + .expect("seed alias symlink"); + }) + .expect("inspect bridge"); + + let filesystem = ScopedHostFilesystem::new(HostFilesystem::new(bridge, "vm-1"), "/data"); + + assert_eq!( + filesystem.realpath("/alias").expect("resolve alias"), + "/database" + ); + } + + #[test] + fn host_filesystem_realpath_fails_closed_on_circular_symlinks() { + let bridge = SharedBridge::new(RecordingBridge::default()); + bridge + .inspect(|bridge| { + agent_os_bridge::FilesystemBridge::symlink( + bridge, + SymlinkRequest { + vm_id: String::from("vm-1"), + target_path: String::from("/loop-b.txt"), + link_path: String::from("/loop-a.txt"), + }, + ) + .expect("seed loop-a symlink"); + agent_os_bridge::FilesystemBridge::symlink( + bridge, + SymlinkRequest { + vm_id: String::from("vm-1"), + target_path: String::from("/loop-a.txt"), + link_path: String::from("/loop-b.txt"), + }, + ) + .expect("seed loop-b symlink"); + }) + .expect("inspect bridge"); + + let filesystem = HostFilesystem::new(bridge, "vm-1"); + let error = filesystem + .realpath("/loop-a.txt") + .expect_err("circular symlink chain should fail closed"); + assert_eq!(error.code(), "ELOOP"); + } + + #[test] + fn configure_vm_host_dir_plugin_fails_closed_for_escape_symlinks() { + let host_dir = temp_dir("agent-os-sidecar-host-dir-escape"); + std::os::unix::fs::symlink("/etc", host_dir.join("escape")).expect("seed escape symlink"); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("host_dir"), + config: json!({ + "hostPath": host_dir, + "readOnly": false, + }), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), + }), + )) + .expect("configure host_dir mount"); + + let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); + let error = vm + .kernel + .filesystem_mut() + .read_file("/workspace/escape/hostname") + .expect_err("escape symlink should fail closed"); + assert_eq!(error.code(), "EACCES"); + + fs::remove_dir_all(host_dir).expect("remove temp dir"); + } + + #[test] + fn execute_starts_python_runtime_instead_of_rejecting_it() { + assert_node_available(); + + let cache_root = temp_dir("agent-os-sidecar-python-cache"); + + let mut sidecar = NativeSidecar::with_config( + RecordingBridge::default(), + NativeSidecarConfig { + sidecar_id: String::from("sidecar-python-test"), + compile_cache_root: Some(cache_root), + expected_auth_token: Some(String::from(TEST_AUTH_TOKEN)), + ..NativeSidecarConfig::default() + }, + ) + .expect("create sidecar"); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::Execute(crate::protocol::ExecuteRequest { + process_id: String::from("proc-python"), + runtime: GuestRuntimeKind::Python, + entrypoint: String::from("print('hello from python')"), + args: Vec::new(), + env: BTreeMap::new(), + cwd: None, + wasm_permission_tier: None, + }), + )) + .expect("dispatch python execute"); + + match result.response.payload { + ResponsePayload::ProcessStarted(response) => { + assert_eq!(response.process_id, "proc-python"); + assert!( + response.pid.is_some(), + "python runtime should expose a child pid" + ); + } + other => panic!("unexpected execute response: {other:?}"), + } + + let vm = sidecar.vms.get(&vm_id).expect("python vm"); + let process = vm + .active_processes + .get("proc-python") + .expect("python process should be tracked"); + assert_eq!(process.runtime, GuestRuntimeKind::Python); + match &process.execution { + ActiveExecution::Python(_) => {} + other => panic!("unexpected active execution variant: {other:?}"), + } + } + + #[test] + fn python_vfs_rpc_requests_proxy_into_the_vm_kernel_filesystem() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-python-vfs-rpc-cwd"); + let pyodide_dir = temp_dir("agent-os-sidecar-python-vfs-rpc-pyodide"); + write_fixture( + &pyodide_dir.join("pyodide.mjs"), + r#" +export async function loadPyodide() { + return { + setStdin(_stdin) {}, + async runPythonAsync(_code) { + await new Promise(() => {}); + }, + }; +} +"#, + ); + write_fixture( + &pyodide_dir.join("pyodide-lock.json"), + "{\"packages\":[]}\n", + ); + + let context = sidecar + .python_engine + .create_context(CreatePythonContextRequest { + vm_id: vm_id.clone(), + pyodide_dist_path: pyodide_dir, + }); + let execution = sidecar + .python_engine + .start_execution(StartPythonExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + code: String::from("print('hold-open')"), + file_path: None, + env: BTreeMap::new(), + cwd: cwd.clone(), + }) + .expect("start fake python execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + vm.kernel + .spawn_process( + PYTHON_COMMAND, + vec![String::from("print('hold-open')")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel python process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + vm.active_processes.insert( + String::from("proc-python-vfs"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::Python, + ActiveExecution::Python(execution), + ), + ); + } + + sidecar + .handle_python_vfs_rpc_request( + &vm_id, + "proc-python-vfs", + PythonVfsRpcRequest { + id: 1, + method: PythonVfsRpcMethod::Mkdir, + path: String::from("/workspace"), + content_base64: None, + recursive: false, + }, + ) + .expect("handle python mkdir rpc"); + sidecar + .handle_python_vfs_rpc_request( + &vm_id, + "proc-python-vfs", + PythonVfsRpcRequest { + id: 2, + method: PythonVfsRpcMethod::Write, + path: String::from("/workspace/note.txt"), + content_base64: Some(String::from("aGVsbG8gZnJvbSBzaWRlY2FyIHJwYw==")), + recursive: false, + }, + ) + .expect("handle python write rpc"); + + let content = { + let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + String::from_utf8( + vm.kernel + .read_file("/workspace/note.txt") + .expect("read bridged file from kernel"), + ) + .expect("utf8 file contents") + }; + assert_eq!(content, "hello from sidecar rpc"); + + let process = { + let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + vm.active_processes + .remove("proc-python-vfs") + .expect("remove fake python process") + }; + let _ = signal_runtime_process(process.execution.child_pid(), SIGTERM); + } + + #[test] + fn javascript_sync_rpc_requests_proxy_into_the_vm_kernel_filesystem() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-sync-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import fs from "node:fs"; + +fs.writeFileSync("/rpc/note.txt", "hello from sidecar rpc"); +fs.mkdirSync("/rpc/subdir", { recursive: true }); +fs.symlinkSync("/rpc/note.txt", "/rpc/link.txt"); +const linkTarget = fs.readlinkSync("/rpc/link.txt"); +const existsBefore = fs.existsSync("/rpc/note.txt"); +const lstat = fs.lstatSync("/rpc/link.txt"); +fs.linkSync("/rpc/note.txt", "/rpc/hard.txt"); +fs.renameSync("/rpc/hard.txt", "/rpc/renamed.txt"); +const contents = fs.readFileSync("/rpc/renamed.txt", "utf8"); +fs.unlinkSync("/rpc/renamed.txt"); +fs.rmdirSync("/rpc/subdir"); +console.log(JSON.stringify({ existsBefore, linkTarget, linkIsSymlink: lstat.isSymbolicLink(), contents })); +await new Promise(() => {}); +"#, + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_NODE_SYNC_RPC_ENABLE"), + String::from("1"), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-sync"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut saw_stdout = false; + for _ in 0..16 { + let event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get("proc-js-sync") + .expect("javascript process should be tracked"); + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript sync rpc event") + .expect("javascript sync rpc event") + }; + + if let ActiveExecutionEvent::Stdout(chunk) = &event { + let stdout = String::from_utf8(chunk.clone()).expect("stdout utf8"); + if stdout.contains("\"contents\":\"hello from sidecar rpc\"") + && stdout.contains("\"existsBefore\":true") + && stdout.contains("\"linkTarget\":\"/rpc/note.txt\"") + && stdout.contains("\"linkIsSymlink\":true") + { + saw_stdout = true; + break; + } + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-sync", event) + .expect("handle javascript sync rpc event"); + } + + let content = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + String::from_utf8( + vm.kernel + .read_file("/rpc/note.txt") + .expect("read bridged file from kernel"), + ) + .expect("utf8 file contents") + }; + assert_eq!(content, "hello from sidecar rpc"); + let link_target = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .read_link("/rpc/link.txt") + .expect("read bridged symlink") + }; + assert_eq!(link_target, "/rpc/note.txt"); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + assert!( + !vm.kernel + .exists("/rpc/renamed.txt") + .expect("renamed file should be gone"), + "expected renamed file to be removed", + ); + assert!( + !vm.kernel + .exists("/rpc/subdir") + .expect("subdir should be gone"), + "expected subdir to be removed", + ); + } + assert!(saw_stdout, "expected guest stdout after sync fs round-trip"); + + let process = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes + .remove("proc-js-sync") + .expect("remove fake javascript process") + }; + let _ = signal_runtime_process(process.execution.child_pid(), SIGTERM); + } + + #[test] + fn python_vfs_rpc_paths_are_scoped_to_workspace_root() { + assert_eq!( + normalize_python_vfs_rpc_path("/workspace/./note.txt") + .expect("normalize workspace path"), + String::from("/workspace/note.txt") + ); + assert!( + normalize_python_vfs_rpc_path("/workspace/../etc/passwd").is_err(), + "workspace escape should be rejected", + ); + assert!( + normalize_python_vfs_rpc_path("/etc/passwd").is_err(), + "non-workspace paths should be rejected", + ); + assert!( + normalize_python_vfs_rpc_path("workspace/note.txt").is_err(), + "relative paths should be rejected", + ); + } + + #[test] + fn javascript_fs_sync_rpc_resolves_proc_self_against_the_kernel_process() { + let mut config = KernelVmConfig::new("vm-js-procfs-rpc"); + config.permissions = Permissions::allow_all(); + let mut kernel = SidecarKernel::new(MountTable::new(MemoryFileSystem::new()), config); + kernel + .register_driver(CommandDriver::new( + EXECUTION_DRIVER_NAME, + [JAVASCRIPT_COMMAND], + )) + .expect("register execution driver"); + + let process = kernel + .spawn_process( + JAVASCRIPT_COMMAND, + Vec::new(), + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + ..SpawnOptions::default() + }, + ) + .expect("spawn javascript kernel process"); + + let link = service_javascript_fs_sync_rpc( + &mut kernel, + process.pid(), + &JavascriptSyncRpcRequest { + id: 1, + method: String::from("fs.readlinkSync"), + args: vec![json!("/proc/self")], + }, + ) + .expect("resolve /proc/self"); + assert_eq!(link, Value::String(format!("/proc/{}", process.pid()))); + + let entries = service_javascript_fs_sync_rpc( + &mut kernel, + process.pid(), + &JavascriptSyncRpcRequest { + id: 2, + method: String::from("fs.readdirSync"), + args: vec![json!("/proc/self/fd")], + }, + ) + .expect("read /proc/self/fd"); + let entry_names = entries + .as_array() + .expect("readdir should return an array") + .iter() + .filter_map(Value::as_str) + .collect::>(); + assert!(entry_names.contains(&"0")); + assert!(entry_names.contains(&"1")); + assert!(entry_names.contains(&"2")); + + process.finish(0); + kernel + .waitpid(process.pid()) + .expect("wait javascript process"); + } + + #[test] + fn javascript_fd_and_stream_rpc_requests_proxy_into_the_vm_kernel_filesystem() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .write_file("/rpc/input.txt", b"abcdefg") + .expect("seed input file"); + } + let cwd = temp_dir("agent-os-sidecar-js-fd-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import fs from "node:fs"; +import { once } from "node:events"; + +const inFd = fs.openSync("/rpc/input.txt", "r"); +const buffer = Buffer.alloc(5); +const bytesRead = fs.readSync(inFd, buffer, 0, buffer.length, 1); +const stat = fs.fstatSync(inFd); +fs.closeSync(inFd); + +const defaultUmask = process.umask(); +const previousUmask = process.umask(0o027); +const outFd = fs.openSync("/rpc/output.txt", "w", 0o666); +const written = fs.writeSync(outFd, Buffer.from("kernel"), 0, 6, 0); +fs.closeSync(outFd); +fs.mkdirSync("/rpc/private", { mode: 0o777 }); +const outputStat = fs.statSync("/rpc/output.txt"); +const privateDirStat = fs.statSync("/rpc/private"); + +const asyncSummary = await new Promise((resolve, reject) => { + fs.open("/rpc/input.txt", "r", (openError, asyncFd) => { + if (openError) { + reject(openError); + return; + } + + const target = Buffer.alloc(5); + fs.read(asyncFd, target, 0, 5, 0, (readError, asyncBytesRead) => { + if (readError) { + reject(readError); + return; + } + + fs.fstat(asyncFd, (statError, asyncStat) => { + if (statError) { + reject(statError); + return; + } + + fs.close(asyncFd, (closeError) => { + if (closeError) { + reject(closeError); + return; + } + + resolve({ + asyncBytesRead, + asyncText: target.toString("utf8"), + asyncSize: asyncStat.size, + }); + }); + }); + }); + }); +}); + +const reader = fs.createReadStream("/rpc/input.txt", { + encoding: "utf8", + start: 0, + end: 4, + highWaterMark: 3, +}); +const streamChunks = []; +reader.on("data", (chunk) => streamChunks.push(chunk)); +await once(reader, "close"); + +const writer = fs.createWriteStream("/rpc/stream.txt", { start: 0 }); +writer.write("ab"); +writer.end("cd"); +await once(writer, "close"); + +let watchCode = ""; +let watchFileCode = ""; +try { + fs.watch("/rpc/input.txt"); +} catch (error) { + watchCode = error.code; +} +try { + fs.watchFile("/rpc/input.txt", () => {}); +} catch (error) { + watchFileCode = error.code; +} + +console.log( + JSON.stringify({ + text: buffer.toString("utf8"), + bytesRead, + size: stat.size, + blocks: stat.blocks, + dev: stat.dev, + rdev: stat.rdev, + written, + defaultUmask, + previousUmask, + outputMode: outputStat.mode & 0o777, + privateDirMode: privateDirStat.mode & 0o777, + asyncSummary, + streamChunks, + watchCode, + watchFileCode, + }), +); +"#, + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"child_process\",\"console\",\"crypto\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-fd"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-fd") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript fd rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript fd process disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-fd", event) + .expect("handle javascript fd rpc event"); + } + + assert_eq!(exit_code, Some(0), "stdout: {stdout}\nstderr: {stderr}"); + assert!(stdout.contains("\"text\":\"bcdef\""), "stdout: {stdout}"); + assert!(stdout.contains("\"bytesRead\":5"), "stdout: {stdout}"); + assert!(stdout.contains("\"size\":7"), "stdout: {stdout}"); + assert!(stdout.contains("\"blocks\":1"), "stdout: {stdout}"); + assert!(stdout.contains("\"dev\":1"), "stdout: {stdout}"); + assert!(stdout.contains("\"rdev\":0"), "stdout: {stdout}"); + assert!(stdout.contains("\"written\":6"), "stdout: {stdout}"); + assert!(stdout.contains("\"defaultUmask\":18"), "stdout: {stdout}"); + assert!(stdout.contains("\"previousUmask\":18"), "stdout: {stdout}"); + assert!(stdout.contains("\"outputMode\":416"), "stdout: {stdout}"); + assert!( + stdout.contains("\"privateDirMode\":488"), + "stdout: {stdout}" + ); + assert!( + stdout.contains("\"asyncText\":\"abcde\""), + "stdout: {stdout}" + ); + assert!(stdout.contains("\"asyncSize\":7"), "stdout: {stdout}"); + assert!( + stdout.contains("\"streamChunks\":[\"abc\",\"de\"]"), + "stdout: {stdout}" + ); + assert!( + stdout.contains("\"watchCode\":\"ERR_AGENT_OS_FS_WATCH_UNAVAILABLE\""), + "stdout: {stdout}" + ); + assert!( + stdout.contains("\"watchFileCode\":\"ERR_AGENT_OS_FS_WATCH_UNAVAILABLE\""), + "stdout: {stdout}" + ); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let output = String::from_utf8( + vm.kernel + .read_file("/rpc/output.txt") + .expect("read fd output file"), + ) + .expect("utf8 output contents"); + assert_eq!(output, "kernel"); + + let stream = String::from_utf8( + vm.kernel + .read_file("/rpc/stream.txt") + .expect("read stream output file"), + ) + .expect("utf8 stream contents"); + assert_eq!(stream, "abcd"); + } + } + + #[test] + fn javascript_fs_promises_rpc_requests_proxy_into_the_vm_kernel_filesystem() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-promises-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import fs from "node:fs/promises"; + +await fs.writeFile("/rpc/note.txt", "hello from sidecar promises rpc"); +const contents = await fs.readFile("/rpc/note.txt", "utf8"); +console.log(contents); +await new Promise(() => {}); +"#, + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-promises"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut saw_stdout = false; + for _ in 0..4 { + let event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get("proc-js-promises") + .expect("javascript process should be tracked"); + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript promises rpc event") + .expect("javascript promises rpc event") + }; + + if let ActiveExecutionEvent::Stdout(chunk) = &event { + let stdout = String::from_utf8(chunk.clone()).expect("stdout utf8"); + if stdout.contains("hello from sidecar promises rpc") { + saw_stdout = true; + break; + } + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-promises", event) + .expect("handle javascript promises rpc event"); + } + + let content = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + String::from_utf8( + vm.kernel + .read_file("/rpc/note.txt") + .expect("read bridged file from kernel"), + ) + .expect("utf8 file contents") + }; + assert_eq!(content, "hello from sidecar promises rpc"); + assert!( + saw_stdout, + "expected guest stdout after fs.promises round-trip" + ); + + let process = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes + .remove("proc-js-promises") + .expect("remove fake javascript process") + }; + let _ = signal_runtime_process(process.execution.child_pid(), SIGTERM); + } + + #[test] + fn javascript_net_rpc_connects_to_host_tcp_server() { + assert_node_available(); + + let listener = TcpListener::bind("127.0.0.1:0").expect("bind tcp listener"); + let port = listener.local_addr().expect("listener address").port(); + let server = thread::spawn(move || { + let (mut stream, _) = listener.accept().expect("accept tcp client"); + let mut received = Vec::new(); + stream + .read_to_end(&mut received) + .expect("read client payload"); + assert_eq!(String::from_utf8(received).expect("client utf8"), "ping"); + stream.write_all(b"pong").expect("write server payload"); + }); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-net-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + &format!( + r#" +import net from "node:net"; + +const socket = net.createConnection({{ host: "127.0.0.1", port: {port} }}); +let data = ""; +socket.setEncoding("utf8"); +socket.on("connect", () => {{ + socket.end("ping"); +}}); +socket.on("data", (chunk) => {{ + data += chunk; +}}); +socket.on("error", (error) => {{ + console.error(error.stack ?? error.message); + process.exit(1); +}}); +socket.on("close", (hadError) => {{ + console.log(JSON.stringify({{ + data, + hadError, + remoteAddress: socket.remoteAddress, + remotePort: socket.remotePort, + localPort: socket.localPort, + }})); + process.exit(hadError ? 1 : 0); +}}); +"#, + ), + ); + + let (stdout, stderr, exit_code) = run_javascript_entry( + &mut sidecar, + &vm_id, + &cwd, + "proc-js-net", + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ); + + server.join().expect("join tcp server"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + assert!(stdout.contains("\"data\":\"pong\""), "stdout: {stdout}"); + assert!(stdout.contains("\"hadError\":false"), "stdout: {stdout}"); + assert!( + stdout.contains(&format!("\"remotePort\":{port}")), + "stdout: {stdout}" + ); + } + + #[test] + fn javascript_dgram_rpc_sends_and_receives_host_udp_packets() { + assert_node_available(); + + let listener = UdpSocket::bind("127.0.0.1:0").expect("bind udp listener"); + let port = listener.local_addr().expect("listener address").port(); + let server = thread::spawn(move || { + let mut buffer = [0_u8; 64 * 1024]; + let (bytes_read, remote_addr) = listener.recv_from(&mut buffer).expect("recv packet"); + assert_eq!( + String::from_utf8(buffer[..bytes_read].to_vec()).expect("udp payload utf8"), + "ping" + ); + listener + .send_to(b"pong", remote_addr) + .expect("send udp response"); + }); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-dgram-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + &format!( + r#" +import dgram from "node:dgram"; + +const socket = dgram.createSocket("udp4"); +const summary = await new Promise((resolve) => {{ +socket.on("error", (error) => {{ + console.error(error.stack ?? error.message); + process.exit(1); +}}); +socket.on("message", (message, rinfo) => {{ + const address = socket.address(); + socket.close(() => {{ + resolve({{ + address, + message: message.toString("utf8"), + rinfo, + }}); + }}); +}}); +socket.bind(0, "127.0.0.1", () => {{ + socket.send("ping", {port}, "127.0.0.1"); +}}); +}}); + +console.log(JSON.stringify(summary)); +"#, + ), + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dgram\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-dgram"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-dgram") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript dgram rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript dgram process disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-dgram", event) + .expect("handle javascript dgram rpc event"); + } + + server.join().expect("join udp server"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + assert!(stdout.contains("\"message\":\"pong\""), "stdout: {stdout}"); + assert!( + stdout.contains("\"address\":{\"address\":\"127.0.0.1\""), + "stdout: {stdout}" + ); + assert!( + stdout.contains(&format!("\"port\":{port}")), + "stdout: {stdout}" + ); + } + + #[test] + fn javascript_dns_rpc_resolves_localhost() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-dns-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import dns from "node:dns"; + +const lookup = await dns.promises.lookup("localhost", { all: true }); +const resolve4 = await dns.promises.resolve4("localhost"); + +console.log(JSON.stringify({ lookup, resolve4 })); +"#, + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-dns"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-dns") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript dns rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript dns process disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-dns", event) + .expect("handle javascript dns rpc event"); + } + + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse dns JSON"); + assert!( + parsed["lookup"] + .as_array() + .is_some_and(|entries| !entries.is_empty()), + "stdout: {stdout}" + ); + assert!( + parsed["resolve4"] + .as_array() + .is_some_and(|entries| entries.iter().any(|entry| entry == "127.0.0.1")), + "stdout: {stdout}" + ); + } + + #[test] + fn javascript_network_ssrf_protection_blocks_private_dns_and_unowned_loopback_targets() { + assert_node_available(); + + let loopback_listener = TcpListener::bind("127.0.0.1:0").expect("bind loopback listener"); + let loopback_port = loopback_listener + .local_addr() + .expect("loopback listener address") + .port(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + Vec::new(), + BTreeMap::from([( + String::from("network.dns.override.metadata.test"), + String::from("169.254.169.254"), + )]), + ) + .expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-ssrf-protection-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + &format!( + r#" +import dns from "node:dns"; +import net from "node:net"; + +const dnsLookup = await (async () => {{ + try {{ + await dns.promises.lookup("metadata.test", {{ family: 4 }}); + return {{ unexpected: true }}; + }} catch (error) {{ + return {{ code: error.code ?? null, message: error.message }}; + }} +}})(); + +const privateConnect = await new Promise((resolve) => {{ + const socket = net.createConnection({{ host: "metadata.test", port: 80 }}); + socket.on("connect", () => {{ + socket.destroy(); + resolve({{ unexpected: true }}); + }}); + socket.on("error", (error) => {{ + resolve({{ code: error.code ?? null, message: error.message }}); + }}); +}}); + +const loopbackConnect = await new Promise((resolve) => {{ + const socket = net.createConnection({{ host: "127.0.0.1", port: {loopback_port} }}); + socket.on("connect", () => {{ + socket.destroy(); + resolve({{ unexpected: true }}); + }}); + socket.on("error", (error) => {{ + resolve({{ code: error.code ?? null, message: error.message }}); + }}); +}}); + +console.log(JSON.stringify({{ dnsLookup, privateConnect, loopbackConnect }})); +process.exit(0); +"#, + ), + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-ssrf-protection"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-ssrf-protection") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript ssrf event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript ssrf process disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } - sidecar - .dispatch(request( - 5, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/workspace"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("host_dir"), - config: json!({ - "hostPath": host_dir, - "readOnly": false, - }), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure host_dir mount"); + sidecar + .handle_execution_event(&vm_id, "proc-js-ssrf-protection", event) + .expect("handle javascript ssrf event"); + } - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let hidden = vm - .kernel - .filesystem_mut() - .read_file("/workspace/root-only.txt") - .expect_err("mounted host dir should hide root-backed file"); - assert_eq!(hidden.code(), "ENOENT"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse ssrf JSON"); assert_eq!( - vm.kernel - .filesystem_mut() - .read_file("/workspace/hello.txt") - .expect("read mounted host file"), - b"hello from host".to_vec() + parsed["dnsLookup"]["code"], + Value::String(String::from("EACCES")) + ); + assert!( + parsed["dnsLookup"]["message"] + .as_str() + .is_some_and(|message| message.contains("169.254.0.0/16")), + "stdout: {stdout}" + ); + assert_eq!( + parsed["privateConnect"]["code"], + Value::String(String::from("EACCES")) + ); + assert!( + parsed["privateConnect"]["message"] + .as_str() + .is_some_and(|message| message.contains("169.254.0.0/16")), + "stdout: {stdout}" + ); + assert_eq!( + parsed["loopbackConnect"]["code"], + Value::String(String::from("EACCES")) + ); + assert!( + parsed["loopbackConnect"]["message"] + .as_str() + .is_some_and(|message| message.contains(LOOPBACK_EXEMPT_PORTS_ENV)), + "stdout: {stdout}" ); - vm.kernel - .filesystem_mut() - .write_file("/workspace/from-vm.txt", b"native host dir".to_vec()) - .expect("write host dir file"); + drop(loopback_listener); + } + + #[test] + fn javascript_dns_rpc_honors_vm_dns_overrides_and_net_connect_uses_sidecar_dns() { + assert_node_available(); + + let listener = TcpListener::bind("127.0.0.1:0").expect("bind tcp listener"); + let port = listener.local_addr().expect("listener address").port(); + let server = thread::spawn(move || { + let (mut stream, _) = listener.accept().expect("accept tcp client"); + let mut received = Vec::new(); + stream + .read_to_end(&mut received) + .expect("read client payload"); + assert_eq!(String::from_utf8(received).expect("client utf8"), "ping"); + stream.write_all(b"pong").expect("write server payload"); + }); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + Vec::new(), + BTreeMap::from([ + ( + format!("env.{LOOPBACK_EXEMPT_PORTS_ENV}"), + serde_json::to_string(&vec![port.to_string()]).expect("serialize exempt ports"), + ), + ( + String::from("network.dns.override.example.test"), + String::from("127.0.0.1"), + ), + ( + String::from(VM_DNS_SERVERS_METADATA_KEY), + String::from("203.0.113.53:5353"), + ), + ]), + ) + .expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-dns-override-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + &format!( + r#" +import dns from "node:dns"; +import net from "node:net"; + +const lookup = await dns.promises.lookup("example.test", {{ family: 4 }}); +const resolved = await dns.promises.resolve4("example.test"); +const socketSummary = await new Promise((resolve, reject) => {{ + const socket = net.createConnection({{ host: "example.test", port: {port} }}); + let data = ""; + socket.setEncoding("utf8"); + socket.on("connect", () => {{ + socket.end("ping"); + }}); + socket.on("data", (chunk) => {{ + data += chunk; + }}); + socket.on("error", reject); + socket.on("close", (hadError) => {{ + resolve({{ + data, + hadError, + remoteAddress: socket.remoteAddress, + remotePort: socket.remotePort, + }}); + }}); +}}); + +console.log(JSON.stringify({{ lookup, resolved, socketSummary }})); +"#, + ), + ); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-dns-override"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..64 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-dns-override") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript dns override rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + panic!("javascript dns override process disappeared before exit"); + }; + + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-dns-override", event) + .expect("handle javascript dns override rpc event"); + } + + server.join().expect("join tcp server"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse dns JSON"); + assert_eq!(parsed["lookup"]["address"], Value::from("127.0.0.1")); + assert_eq!(parsed["lookup"]["family"], Value::from(4)); + assert_eq!(parsed["resolved"][0], Value::from("127.0.0.1")); + assert_eq!(parsed["socketSummary"]["data"], Value::from("pong")); + assert_eq!(parsed["socketSummary"]["hadError"], Value::from(false)); assert_eq!( - fs::read_to_string(host_dir.join("from-vm.txt")).expect("read host output"), - "native host dir" + parsed["socketSummary"]["remoteAddress"], + Value::from("127.0.0.1") + ); + assert_eq!( + parsed["socketSummary"]["remotePort"], + Value::from(u64::from(port)) ); - fs::remove_dir_all(host_dir).expect("remove temp dir"); + let events = sidecar + .with_bridge_mut(|bridge| bridge.structured_events.clone()) + .expect("collect structured events"); + let dns_events = events + .iter() + .filter(|event| event.name == "network.dns.resolved") + .filter(|event| { + event.fields.get("hostname").map(String::as_str) == Some("example.test") + }) + .collect::>(); + assert!( + dns_events.len() >= 3, + "expected dns events for lookup, resolve4, and net.connect: {dns_events:?}" + ); + for event in dns_events { + assert_eq!(event.fields["source"], "override"); + assert_eq!(event.fields["addresses"], "127.0.0.1"); + assert_eq!(event.fields["resolver_count"], "1"); + assert_eq!(event.fields["resolvers"], "203.0.113.53:5353"); + } } #[test] - fn configure_vm_js_bridge_mount_preserves_hard_link_identity() { + fn javascript_network_permission_callbacks_fire_for_dns_lookup_connect_and_listen() { + assert_node_available(); + + let listener = TcpListener::bind("127.0.0.1:0").expect("bind tcp listener"); + let port = listener.local_addr().expect("listener address").port(); + let server = thread::spawn(move || { + let (mut stream, _) = listener.accept().expect("accept tcp client"); + let mut received = Vec::new(); + stream + .read_to_end(&mut received) + .expect("read client payload"); + assert_eq!(String::from_utf8(received).expect("client utf8"), "ping"); + }); + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + Vec::new(), + BTreeMap::from([ + ( + format!("env.{LOOPBACK_EXEMPT_PORTS_ENV}"), + serde_json::to_string(&vec![port.to_string()]).expect("serialize exempt ports"), + ), + ( + String::from("network.dns.override.example.test"), + String::from("127.0.0.1"), + ), + ]), + ) + .expect("create vm"); sidecar .bridge - .inspect(|bridge| { - bridge.seed_directory( - "/workspace", - vec![agent_os_bridge::DirectoryEntry { - name: String::from("original.txt"), - kind: FileKind::File, - }], - ); - bridge.seed_file("/workspace/original.txt", b"hello world".to_vec()); + .clear_vm_permissions(&vm_id) + .expect("clear static vm permissions"); + let cwd = temp_dir("agent-os-sidecar-js-network-permission-callbacks"); + write_fixture( + &cwd.join("entry.mjs"), + &format!( + r#" +import dns from "node:dns"; +import net from "node:net"; + +const lookup = await dns.promises.lookup("example.test", {{ family: 4 }}); +const listenAddress = await new Promise((resolve, reject) => {{ + const server = net.createServer(); + server.on("error", reject); + server.listen(0, "127.0.0.1", () => {{ + const address = server.address(); + server.close((error) => {{ + if (error) {{ + reject(error); + return; + }} + resolve(address); + }}); + }}); +}}); +const connectResult = await new Promise((resolve, reject) => {{ + const socket = net.createConnection({{ host: "127.0.0.1", port: {port} }}); + socket.on("error", reject); + socket.on("connect", () => {{ + socket.end("ping"); + }}); + socket.on("close", (hadError) => {{ + resolve({{ hadError }}); + }}); +}}); + +console.log(JSON.stringify({{ lookup, listenAddress, connectResult }})); +process.exit(0); +"#, + ), + ); + + let (stdout, stderr, exit_code) = run_javascript_entry( + &mut sidecar, + &vm_id, + &cwd, + "proc-js-network-permission-callbacks", + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ); + + server.join().expect("join tcp server"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse callback JSON"); + assert_eq!( + parsed["lookup"]["address"], + Value::String(String::from("127.0.0.1")) + ); + assert_eq!(parsed["connectResult"]["hadError"], Value::Bool(false)); + assert!( + parsed["listenAddress"]["port"] + .as_u64() + .is_some_and(|value| value > 0), + "stdout: {stdout}" + ); + + let expected = [ + format!("net:{vm_id}:{}", format_dns_resource("example.test")), + format!("net:{vm_id}:{}", format_tcp_resource("127.0.0.1", 0)), + format!("net:{vm_id}:{}", format_tcp_resource("127.0.0.1", port)), + ]; + let checks = sidecar + .with_bridge_mut(|bridge| { + bridge + .permission_checks + .iter() + .filter(|entry| entry.starts_with("net:")) + .cloned() + .collect::>() }) - .expect("seed js bridge filesystem"); + .expect("read permission checks"); + for check in expected { + assert!( + checks.iter().any(|entry| entry == &check), + "missing permission check {check:?} in {checks:?}" + ); + } + } + + #[test] + fn javascript_network_permission_denials_surface_eacces_to_guest_code() { + assert_node_available(); + let mut sidecar = create_test_sidecar(); let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + vec![ + PermissionDescriptor { + capability: String::from("fs"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("env"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("child_process"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("network"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("network.dns"), + mode: PermissionMode::Deny, + }, + PermissionDescriptor { + capability: String::from("network.http"), + mode: PermissionMode::Deny, + }, + PermissionDescriptor { + capability: String::from("network.listen"), + mode: PermissionMode::Deny, + }, + ], + BTreeMap::from([( + String::from("network.dns.override.example.test"), + String::from("127.0.0.1"), + )]), + ) + .expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-network-permission-denials"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import dns from "node:dns"; +import net from "node:net"; + +let dnsResult = null; +try { + dnsResult = { unexpected: await dns.promises.lookup("example.test", { family: 4 }) }; +} catch (error) { + dnsResult = { code: error.code ?? null, message: error.message }; +} +const listenResult = (() => { + const server = net.createServer(); + try { + server.listen(0, "127.0.0.1"); + return { unexpected: true }; + } catch (error) { + return { code: error.code ?? null, message: error.message }; + } +})(); +const connectResult = await new Promise((resolve) => { + const socket = net.createConnection({ host: "127.0.0.1", port: 43111 }); + socket.on("connect", () => resolve({ unexpected: true })); + socket.on("error", (error) => { + resolve({ code: error.code ?? null, message: error.message }); + }); +}); + +console.log(JSON.stringify({ dnsResult, listenResult, connectResult })); +process.exit(0); +"#, + ); - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/workspace"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("js_bridge"), - config: json!({}), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure js_bridge mount"); + let (stdout, stderr, exit_code) = run_javascript_entry( + &mut sidecar, + &vm_id, + &cwd, + "proc-js-network-permission-denials", + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"dns\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ); - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - vm.kernel - .filesystem_mut() - .link("/workspace/original.txt", "/workspace/linked.txt") - .expect("create js bridge hard link"); + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse denial JSON"); + for field in ["dnsResult", "listenResult", "connectResult"] { + assert_eq!(parsed[field]["code"], Value::String(String::from("EACCES"))); + assert!( + parsed[field]["message"] + .as_str() + .is_some_and(|message| message.contains("blocked by network.")), + "missing policy detail for {field}: {stdout}" + ); + } + } - let original = vm - .kernel - .filesystem_mut() - .stat("/workspace/original.txt") - .expect("stat original"); - let linked = vm - .kernel - .filesystem_mut() - .stat("/workspace/linked.txt") - .expect("stat linked"); - assert_eq!(original.ino, linked.ino); - assert_eq!(original.nlink, 2); - assert_eq!(linked.nlink, 2); + #[test] + fn javascript_tls_rpc_connects_and_serves_over_guest_net() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-tls-rpc-cwd"); + let entry = format!( + r#" +import tls from "node:tls"; + +const key = {key:?}; +const cert = {cert:?}; + +const summary = await new Promise((resolve, reject) => {{ + const server = tls.createServer({{ key, cert }}, (socket) => {{ + let received = ""; + socket.setEncoding("utf8"); + socket.on("data", (chunk) => {{ + received += chunk; + socket.end(`pong:${{chunk}}`); + }}); + socket.on("error", reject); + socket.on("close", () => {{ + server.close(() => {{ + resolve({{ + authorized: client.authorized, + encrypted: client.encrypted, + hadError: closeState.hadError, + localPort: client.localPort, + received, + remoteAddress: client.remoteAddress, + response, + serverPort: port, + serverSecure: secureConnectionSeen, + }}); + }}); + }}); + }}); + let response = ""; + let port = null; + let secureConnectionSeen = false; + let closeState = {{ hadError: false }}; + let client = null; + + server.on("secureConnection", () => {{ + secureConnectionSeen = true; + }}); + server.on("error", reject); + server.listen(0, "127.0.0.1", () => {{ + port = server.address().port; + client = tls.connect({{ + host: "127.0.0.1", + port, + rejectUnauthorized: false, + }}, () => {{ + client.write("ping"); + }}); + client.setEncoding("utf8"); + client.on("data", (chunk) => {{ + response += chunk; + }}); + client.on("error", reject); + client.on("close", (hadError) => {{ + closeState = {{ hadError }}; + }}); + }}); +}}); + +console.log(JSON.stringify(summary)); +"#, + key = TLS_TEST_KEY_PEM, + cert = TLS_TEST_CERT_PEM, + ); + write_fixture(&cwd.join("entry.mjs"), &entry); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"tls\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-tls"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } - vm.kernel - .filesystem_mut() - .write_file("/workspace/linked.txt", b"updated".to_vec()) - .expect("write through hard link"); - assert_eq!( - vm.kernel - .filesystem_mut() - .read_file("/workspace/original.txt") - .expect("read original through shared inode"), - b"updated".to_vec() - ); + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..192 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-tls") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript tls rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + continue; + }; - vm.kernel - .filesystem_mut() - .remove_file("/workspace/original.txt") - .expect("remove original hard link"); - assert!(!vm - .kernel - .filesystem() - .exists("/workspace/original.txt") - .expect("check removed original")); + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-tls", event) + .expect("handle javascript tls rpc event"); + } + + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse tls JSON"); + assert_eq!(parsed["response"], Value::String(String::from("pong:ping"))); + assert_eq!(parsed["received"], Value::String(String::from("ping"))); + assert_eq!(parsed["serverSecure"], Value::Bool(true)); + assert_eq!(parsed["encrypted"], Value::Bool(true)); + assert_eq!(parsed["hadError"], Value::Bool(false)); assert_eq!( - vm.kernel - .filesystem_mut() - .read_file("/workspace/linked.txt") - .expect("read surviving hard link"), - b"updated".to_vec() + parsed["remoteAddress"], + Value::String(String::from("127.0.0.1")) ); - assert_eq!( - vm.kernel - .filesystem_mut() - .stat("/workspace/linked.txt") - .expect("stat surviving hard link") - .nlink, - 1 + assert!( + parsed["serverPort"].as_u64().is_some_and(|port| port > 0), + "stdout: {stdout}" ); } #[test] - fn configure_vm_js_bridge_mount_preserves_metadata_updates() { - let mut sidecar = create_test_sidecar(); - sidecar - .bridge - .inspect(|bridge| { - bridge.seed_directory( - "/workspace", - vec![agent_os_bridge::DirectoryEntry { - name: String::from("original.txt"), - kind: FileKind::File, - }], - ); - bridge.seed_file("/workspace/original.txt", b"hello world".to_vec()); - }) - .expect("seed js bridge filesystem"); + fn javascript_http_rpc_requests_gets_and_serves_over_guest_net() { + assert_node_available(); + let mut sidecar = create_test_sidecar(); let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-http-rpc-cwd"); + write_fixture( + &cwd.join("entry.mjs"), + r#" +import http from "node:http"; + +const summary = await new Promise((resolve, reject) => { + const requests = []; + let requestResponse = ""; + let getResponse = ""; + + const server = http.createServer((req, res) => { + let body = ""; + req.setEncoding("utf8"); + req.on("data", (chunk) => { + body += chunk; + }); + req.on("end", () => { + requests.push({ + method: req.method, + url: req.url, + body, + }); + res.end(`pong:${req.method}:${body || req.url}`); + }); + }); + + let port = null; + server.on("error", reject); + server.listen(0, "127.0.0.1", () => { + port = server.address().port; + const req = http.request( + { + host: "127.0.0.1", + method: "POST", + path: "/submit", + port, + }, + (res) => { + res.setEncoding("utf8"); + res.on("data", (chunk) => { + requestResponse += chunk; + }); + res.on("end", () => { + http + .get(`http://127.0.0.1:${port}/health`, (getRes) => { + getRes.setEncoding("utf8"); + getRes.on("data", (chunk) => { + getResponse += chunk; + }); + getRes.on("end", () => { + server.close(() => { + resolve({ + getResponse, + port, + requestResponse, + requests, + }); + }); + }); + }) + .on("error", reject); + }); + }, + ); + req.on("error", reject); + req.end("ping"); + }); +}); + +console.log(JSON.stringify(summary)); +"#, + ); - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/workspace"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("js_bridge"), - config: json!({}), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure js_bridge mount"); + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"http\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - vm.kernel - .filesystem_mut() - .link("/workspace/original.txt", "/workspace/linked.txt") - .expect("create js bridge hard link"); + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; - vm.kernel - .filesystem_mut() - .chown("/workspace/original.txt", 2000, 3000) - .expect("update js bridge ownership"); - vm.kernel - .filesystem_mut() - .utimes( - "/workspace/linked.txt", - 1_700_000_000_000, - 1_710_000_000_000, - ) - .expect("update js bridge timestamps"); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-http"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } - let original = vm - .kernel - .filesystem_mut() - .stat("/workspace/original.txt") - .expect("stat original"); - let linked = vm - .kernel - .filesystem_mut() - .stat("/workspace/linked.txt") - .expect("stat linked"); + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..192 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-http") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript http rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + continue; + }; - assert_eq!(original.uid, 2000); - assert_eq!(original.gid, 3000); - assert_eq!(linked.uid, 2000); - assert_eq!(linked.gid, 3000); - assert_eq!(original.atime_ms, 1_700_000_000_000); - assert_eq!(original.mtime_ms, 1_710_000_000_000); - assert_eq!(linked.atime_ms, 1_700_000_000_000); - assert_eq!(linked.mtime_ms, 1_710_000_000_000); + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } + + sidecar + .handle_execution_event(&vm_id, "proc-js-http", event) + .expect("handle javascript http rpc event"); + } + + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse http JSON"); + assert_eq!( + parsed["requestResponse"], + Value::String(String::from("pong:POST:ping")) + ); + assert_eq!( + parsed["getResponse"], + Value::String(String::from("pong:GET:/health")) + ); + assert_eq!( + parsed["requests"][0]["url"], + Value::String(String::from("/submit")) + ); + assert_eq!( + parsed["requests"][1]["url"], + Value::String(String::from("/health")) + ); + assert!( + parsed["port"].as_u64().is_some_and(|port| port > 0), + "stdout: {stdout}" + ); } #[test] - fn configure_vm_instantiates_sandbox_agent_mounts_through_the_plugin_registry() { - let server = MockSandboxAgentServer::start("agent-os-sidecar-sandbox", None); - fs::write(server.root().join("hello.txt"), "hello from sandbox") - .expect("seed sandbox file"); + fn javascript_https_rpc_requests_and_serves_over_guest_tls() { + assert_node_available(); let mut sidecar = create_test_sidecar(); let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-https-rpc-cwd"); + let entry = format!( + r#" +import https from "node:https"; + +const key = {key:?}; +const cert = {cert:?}; + +const summary = await new Promise((resolve, reject) => {{ + let received = ""; + let response = ""; + const server = https.createServer({{ key, cert }}, (req, res) => {{ + req.setEncoding("utf8"); + req.on("data", (chunk) => {{ + received += chunk; + }}); + req.on("end", () => {{ + res.end(`pong:${{req.method}}:${{received}}`); + }}); + }}); + + let port = null; + server.on("error", reject); + server.listen(0, "127.0.0.1", () => {{ + port = server.address().port; + const req = https.request({{ + host: "127.0.0.1", + method: "POST", + path: "/secure", + port, + rejectUnauthorized: false, + }}, (res) => {{ + res.setEncoding("utf8"); + res.on("data", (chunk) => {{ + response += chunk; + }}); + res.on("end", () => {{ + server.close(() => {{ + resolve({{ + port, + received, + response, + }}); + }}); + }}); + }}); + req.on("error", reject); + req.end("ping"); + }}); +}}); + +console.log(JSON.stringify(summary)); +"#, + key = TLS_TEST_KEY_PEM, + cert = TLS_TEST_CERT_PEM, + ); + write_fixture(&cwd.join("entry.mjs"), &entry); - sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { - entries: vec![ - RootFilesystemEntry { - path: String::from("/sandbox"), - kind: RootFilesystemEntryKind::Directory, - ..Default::default() - }, - RootFilesystemEntry { - path: String::from("/sandbox/root-only.txt"), - kind: RootFilesystemEntryKind::File, - content: Some(String::from("root bootstrap file")), - ..Default::default() - }, - ], - }), - )) - .expect("bootstrap root sandbox dir"); + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"https\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); - sidecar - .dispatch(request( - 5, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/sandbox"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("sandbox_agent"), - config: json!({ - "baseUrl": server.base_url(), - }), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure sandbox_agent mount"); + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-https"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..192 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-https") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript https rpc event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + continue; + }; - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let hidden = vm - .kernel - .filesystem_mut() - .read_file("/sandbox/root-only.txt") - .expect_err("mounted sandbox should hide root-backed file"); - assert_eq!(hidden.code(), "ENOENT"); - assert_eq!( - vm.kernel - .filesystem_mut() - .read_file("/sandbox/hello.txt") - .expect("read mounted sandbox file"), - b"hello from sandbox".to_vec() - ); + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => { + exit_code = Some(*code); + } + _ => {} + } - vm.kernel - .filesystem_mut() - .write_file("/sandbox/from-vm.txt", b"native sandbox mount".to_vec()) - .expect("write sandbox file"); + sidecar + .handle_execution_event(&vm_id, "proc-js-https", event) + .expect("handle javascript https rpc event"); + } + + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse https JSON"); + assert_eq!(parsed["received"], Value::String(String::from("ping"))); assert_eq!( - fs::read_to_string(server.root().join("from-vm.txt")).expect("read sandbox output"), - "native sandbox mount" + parsed["response"], + Value::String(String::from("pong:POST:ping")) + ); + assert!( + parsed["port"].as_u64().is_some_and(|port| port > 0), + "stdout: {stdout}" ); } #[test] - fn configure_vm_instantiates_s3_mounts_through_the_plugin_registry() { - let server = MockS3Server::start(); + fn javascript_net_rpc_listens_accepts_connections_and_reports_listener_state() { + assert_node_available(); let mut sidecar = create_test_sidecar(); let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-net-server-cwd"); + write_fixture(&cwd.join("entry.mjs"), "setInterval(() => {}, 1000);"); + start_fake_javascript_process(&mut sidecar, &vm_id, &cwd, "proc-js-server", "[\"net\"]"); + + let listen = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-server", + JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 0, + "backlog": 2, + })], + }, + ) + .expect("listen through sidecar net RPC"); + let server_id = listen["serverId"].as_str().expect("server id").to_string(); + let guest_port = listen["localPort"] + .as_u64() + .and_then(|value| u16::try_from(value).ok()) + .expect("guest listener port"); + let host_port = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-server") + .and_then(|process| process.tcp_listeners.get(&server_id)) + .expect("sidecar tcp listener") + .local_addr() + .port() + }; - sidecar + let response = sidecar .dispatch(request( - 4, + 1, OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { - entries: vec![ - RootFilesystemEntry { - path: String::from("/data"), - kind: RootFilesystemEntryKind::Directory, - ..Default::default() - }, - RootFilesystemEntry { - path: String::from("/data/root-only.txt"), - kind: RootFilesystemEntryKind::File, - content: Some(String::from("root bootstrap file")), - ..Default::default() - }, - ], + RequestPayload::FindListener(FindListenerRequest { + host: Some(String::from("127.0.0.1")), + port: Some(guest_port), + path: None, }), )) - .expect("bootstrap root s3 dir"); + .expect("query sidecar listener"); + match response.response.payload { + ResponsePayload::ListenerSnapshot(snapshot) => { + let listener = snapshot.listener.expect("listener snapshot"); + assert_eq!(listener.process_id, "proc-js-server"); + assert_eq!(listener.host.as_deref(), Some("127.0.0.1")); + assert_eq!(listener.port, Some(guest_port)); + } + other => panic!("unexpected find_listener response payload: {other:?}"), + } - sidecar - .dispatch(request( - 5, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/data"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("s3"), - config: json!({ - "bucket": "test-bucket", - "prefix": "service-test", - "region": "us-east-1", - "endpoint": server.base_url(), - "credentials": { - "accessKeyId": "minioadmin", - "secretAccessKey": "minioadmin", - }, - "chunkSize": 8, - "inlineThreshold": 4, - }), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), - }), - )) - .expect("configure s3 mount"); + let client = thread::spawn(move || { + let mut stream = + TcpStream::connect(("127.0.0.1", host_port)).expect("connect to sidecar listener"); + stream.write_all(b"ping").expect("write client payload"); + stream + .shutdown(Shutdown::Write) + .expect("shutdown client write half"); + let mut received = Vec::new(); + stream + .read_to_end(&mut received) + .expect("read server response"); + assert_eq!( + String::from_utf8(received).expect("server response utf8"), + "pong:ping" + ); + }); - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let hidden = vm - .kernel - .filesystem_mut() - .read_file("/data/root-only.txt") - .expect_err("mounted s3 fs should hide root-backed file"); - assert_eq!(hidden.code(), "ENOENT"); + let accepted = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-server", + JavascriptSyncRpcRequest { + id: 2, + method: String::from("net.server_poll"), + args: vec![json!(server_id), json!(250)], + }, + ) + .expect("accept connection"); + assert_eq!(accepted["type"], Value::from("connection")); + assert_eq!(accepted["localAddress"], Value::from("127.0.0.1")); + assert_eq!(accepted["localPort"], Value::from(guest_port)); + let socket_id = accepted["socketId"] + .as_str() + .expect("socket id") + .to_string(); + + let data = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-server", + JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.poll"), + args: vec![json!(socket_id.clone()), json!(250)], + }, + ) + .expect("poll socket data"); + assert_eq!(data["type"], Value::from("data")); - vm.kernel - .filesystem_mut() - .write_file("/data/from-vm.txt", b"native s3 mount".to_vec()) - .expect("write s3-backed file"); - assert_eq!( + let bytes = base64::engine::general_purpose::STANDARD + .decode(data["data"]["base64"].as_str().expect("base64 payload")) + .expect("decode payload"); + assert_eq!(bytes, b"ping"); + + let written = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-server", + JavascriptSyncRpcRequest { + id: 4, + method: String::from("net.write"), + args: vec![json!(socket_id.clone()), json!("pong:ping")], + }, + ) + .expect("write response"); + assert_eq!(written, Value::from(9)); + + call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-server", + JavascriptSyncRpcRequest { + id: 5, + method: String::from("net.shutdown"), + args: vec![json!(socket_id)], + }, + ) + .expect("shutdown write half"); + client.join().expect("join tcp client"); + } + + #[test] + fn javascript_net_rpc_reports_connection_counts_and_enforces_backlog() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-net-backlog-cwd"); + write_fixture(&cwd.join("entry.mjs"), "setInterval(() => {}, 1000);"); + + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); + + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); vm.kernel - .filesystem_mut() - .read_file("/data/from-vm.txt") - .expect("read s3-backed file"), - b"native s3 mount".to_vec() + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() + }, + ) + .expect("spawn kernel javascript process") + }; + + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-backlog"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } + + let bridge = sidecar.bridge.clone(); + let dns = sidecar.vms.get(&vm_id).expect("javascript vm").dns.clone(); + let limits = ResourceLimits::default(); + let socket_paths = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + build_javascript_socket_path_context(vm).expect("build socket path context") + }; + + let listen = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 0, + "backlog": 1, + })], + }, + &limits, + counts, + ) + .expect("listen through sidecar net RPC") + }; + let server_id = listen["serverId"].as_str().expect("server id").to_string(); + let _port = listen["localPort"] + .as_u64() + .and_then(|value| u16::try_from(value).ok()) + .expect("listener port"); + let host_port = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-backlog") + .and_then(|process| process.tcp_listeners.get(&server_id)) + .expect("host backlog listener") + .local_addr() + .port() + }; + + let first_client = thread::spawn(move || { + let mut stream = + TcpStream::connect(("127.0.0.1", host_port)).expect("connect first backlog client"); + stream + .set_read_timeout(Some(Duration::from_secs(5))) + .expect("set first client timeout"); + let mut received = Vec::new(); + stream + .read_to_end(&mut received) + .expect("read first backlog client EOF"); + assert!( + received.is_empty(), + "first backlog client should not receive data" + ); + }); + + let first_connection = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 2, + method: String::from("net.server_poll"), + args: vec![json!(server_id), json!(250)], + }, + &limits, + counts, + ) + .expect("accept first backlog connection") + }; + let first_socket_id = first_connection["socketId"] + .as_str() + .expect("first socket id") + .to_string(); + + let connection_count = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.server_connections"), + args: vec![json!(server_id)], + }, + &limits, + counts, + ) + .expect("query server connections") + }; + assert_eq!(connection_count, json!(1)); + + let second_client = thread::spawn(move || { + let address = SocketAddr::from(([127, 0, 0, 1], host_port)); + let mut stream = TcpStream::connect_timeout(&address, Duration::from_secs(2)) + .expect("connect second backlog client"); + stream + .set_read_timeout(Some(Duration::from_secs(2))) + .expect("set second client timeout"); + stream + .write_all(b"blocked") + .expect("write second backlog client payload"); + let mut buffer = [0_u8; 16]; + match stream.read(&mut buffer) { + Ok(0) => {} + Ok(bytes_read) => panic!( + "unexpected second backlog payload: {}", + String::from_utf8_lossy(&buffer[..bytes_read]) + ), + Err(error) + if matches!( + error.kind(), + std::io::ErrorKind::ConnectionAborted + | std::io::ErrorKind::ConnectionReset + | std::io::ErrorKind::NotConnected + | std::io::ErrorKind::TimedOut + | std::io::ErrorKind::WouldBlock + ) => {} + Err(error) => panic!("unexpected second backlog read error: {error}"), + } + }); + + let second_poll = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 4, + method: String::from("net.server_poll"), + args: vec![json!(server_id), json!(250)], + }, + &limits, + counts, + ) + .expect("poll second backlog connection") + }; + assert_eq!(second_poll, Value::Null); + second_client.join().expect("join second backlog client"); + + let connection_count = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 5, + method: String::from("net.server_connections"), + args: vec![json!(server_id)], + }, + &limits, + counts, + ) + .expect("query server connections after backlog rejection") + }; + assert_eq!(connection_count, json!(1)); + + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 6, + method: String::from("net.destroy"), + args: vec![json!(first_socket_id)], + }, + &limits, + counts, + ) + .expect("destroy first backlog socket"); + } + first_client.join().expect("join first backlog client"); + + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-backlog")) + .expect("backlog process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-backlog") + .expect("backlog process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 7, + method: String::from("net.server_close"), + args: vec![json!(server_id)], + }, + &limits, + counts, + ) + .expect("close backlog listener"); + } + + sidecar + .dispose_vm_internal( + &connection_id, + &session_id, + &vm_id, + DisposeReason::Requested, + ) + .expect("dispose backlog vm"); + } + + #[test] + fn javascript_network_bind_policy_restricts_hosts_and_ports() { + assert_node_available(); + + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + Vec::new(), + BTreeMap::from([ + ( + String::from(VM_LISTEN_PORT_MIN_METADATA_KEY), + String::from("49152"), + ), + ( + String::from(VM_LISTEN_PORT_MAX_METADATA_KEY), + String::from("49160"), + ), + ]), + ) + .expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-bind-policy-cwd"); + write_fixture(&cwd.join("entry.mjs"), "setInterval(() => {}, 1000);"); + start_fake_javascript_process( + &mut sidecar, + &vm_id, + &cwd, + "proc-js-bind-policy", + "[\"dgram\",\"net\"]", + ); + + let unspecified = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "0.0.0.0", + "port": 49152, + })], + }, + ) + .expect_err("deny unspecified TCP listen host"); + assert!( + unspecified + .to_string() + .contains("must bind to loopback, not unspecified"), + "{unspecified}" + ); + + let privileged = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 2, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 80, + })], + }, + ) + .expect_err("deny privileged port"); + assert!( + privileged + .to_string() + .contains("privileged listen port 80 requires"), + "{privileged}" ); - drop(sidecar); - let requests = server.requests(); + let out_of_range = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 40000, + })], + }, + ) + .expect_err("deny out-of-range port"); assert!( - requests.iter().any(|request| request.method == "PUT"), - "expected the native plugin to persist data back to S3" + out_of_range + .to_string() + .contains("outside the allowed range 49152-49160"), + "{out_of_range}" ); + + let udp_socket = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 4, + method: String::from("dgram.createSocket"), + args: vec![json!({ "type": "udp4" })], + }, + ) + .expect("create udp socket"); + let udp_socket_id = udp_socket["socketId"] + .as_str() + .expect("udp socket id") + .to_string(); + + let udp_unspecified = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 5, + method: String::from("dgram.bind"), + args: vec![ + json!(udp_socket_id), + json!({ + "address": "0.0.0.0", + "port": 49153, + }), + ], + }, + ) + .expect_err("deny unspecified UDP bind host"); assert!( - requests - .iter() - .any(|request| request.path.contains("filesystem-manifest.json")), - "expected the native plugin to store a manifest object" + udp_unspecified + .to_string() + .contains("must bind to loopback, not unspecified"), + "{udp_unspecified}" ); + + let success = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-bind-policy", + JavascriptSyncRpcRequest { + id: 6, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 49155, + })], + }, + ) + .expect("allow loopback listener inside configured range"); + assert_eq!(success["localAddress"], Value::from("127.0.0.1")); + assert_eq!(success["localPort"], Value::from(49155)); } #[test] - fn bridge_permissions_map_symlink_operations_to_symlink_access() { - let bridge = SharedBridge::new(RecordingBridge::default()); - let permissions = bridge_permissions(bridge.clone(), "vm-symlink"); - let check = permissions - .filesystem - .as_ref() - .expect("filesystem permission callback"); - - let decision = check(&FsAccessRequest { - vm_id: String::from("ignored-by-bridge"), - op: FsOperation::Symlink, - path: String::from("/workspace/link.txt"), - }); - assert!(decision.allow); + fn javascript_network_bind_policy_can_allow_privileged_guest_ports() { + assert_node_available(); - let recorded = bridge - .inspect(|bridge| bridge.filesystem_permission_requests.clone()) - .expect("inspect bridge"); - assert_eq!( - recorded, - vec![FilesystemPermissionRequest { - vm_id: String::from("vm-symlink"), - path: String::from("/workspace/link.txt"), - access: FilesystemAccess::Symlink, - }] + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = create_vm_with_metadata( + &mut sidecar, + &connection_id, + &session_id, + Vec::new(), + BTreeMap::from([ + ( + String::from(VM_LISTEN_PORT_MIN_METADATA_KEY), + String::from("1"), + ), + ( + String::from(VM_LISTEN_PORT_MAX_METADATA_KEY), + String::from("128"), + ), + ( + String::from(VM_LISTEN_ALLOW_PRIVILEGED_METADATA_KEY), + String::from("true"), + ), + ]), + ) + .expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-privileged-listen-cwd"); + write_fixture(&cwd.join("entry.mjs"), "setInterval(() => {}, 1000);"); + start_fake_javascript_process( + &mut sidecar, + &vm_id, + &cwd, + "proc-js-privileged", + "[\"net\"]", ); + + let listen = call_javascript_sync_rpc( + &mut sidecar, + &vm_id, + "proc-js-privileged", + JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 80, + })], + }, + ) + .expect("allow privileged guest port"); + assert_eq!(listen["localAddress"], Value::from("127.0.0.1")); + assert_eq!(listen["localPort"], Value::from(80)); } #[test] - fn scoped_host_filesystem_unscoped_target_requires_exact_guest_root_prefix() { - let filesystem = ScopedHostFilesystem::new( - HostFilesystem::new(SharedBridge::new(RecordingBridge::default()), "vm-1"), - "/data", - ); + fn javascript_network_listeners_are_isolated_per_vm_even_with_same_guest_port() { + assert_node_available(); - assert_eq!( - filesystem.unscoped_target(String::from("/database")), - "/database" - ); - assert_eq!( - filesystem.unscoped_target(String::from("/data/nested.txt")), - "/nested.txt" - ); - assert_eq!(filesystem.unscoped_target(String::from("/data")), "/"); + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_a = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm a"); + let vm_b = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm b"); + let cwd_a = temp_dir("agent-os-sidecar-js-net-isolation-a"); + let cwd_b = temp_dir("agent-os-sidecar-js-net-isolation-b"); + write_fixture(&cwd_a.join("entry.mjs"), "setInterval(() => {}, 1000);"); + write_fixture(&cwd_b.join("entry.mjs"), "setInterval(() => {}, 1000);"); + start_fake_javascript_process(&mut sidecar, &vm_a, &cwd_a, "proc-a", "[\"net\"]"); + start_fake_javascript_process(&mut sidecar, &vm_b, &cwd_b, "proc-b", "[\"net\"]"); + + let listen_a = call_javascript_sync_rpc( + &mut sidecar, + &vm_a, + "proc-a", + JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 43111, + })], + }, + ) + .expect("listen on vm a"); + let listen_b = call_javascript_sync_rpc( + &mut sidecar, + &vm_b, + "proc-b", + JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 43111, + })], + }, + ) + .expect("listen on vm b"); + assert_eq!(listen_a["localPort"], Value::from(43111)); + assert_eq!(listen_b["localPort"], Value::from(43111)); + + let connect_a = call_javascript_sync_rpc( + &mut sidecar, + &vm_a, + "proc-a", + JavascriptSyncRpcRequest { + id: 2, + method: String::from("net.connect"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 43111, + })], + }, + ) + .expect("connect within vm a"); + let connect_b = call_javascript_sync_rpc( + &mut sidecar, + &vm_b, + "proc-b", + JavascriptSyncRpcRequest { + id: 2, + method: String::from("net.connect"), + args: vec![json!({ + "host": "127.0.0.1", + "port": 43111, + })], + }, + ) + .expect("connect within vm b"); + assert_eq!(connect_a["remotePort"], Value::from(43111)); + assert_eq!(connect_b["remotePort"], Value::from(43111)); + + let server_id_a = listen_a["serverId"] + .as_str() + .expect("server id a") + .to_string(); + let server_id_b = listen_b["serverId"] + .as_str() + .expect("server id b") + .to_string(); + let accepted_a = call_javascript_sync_rpc( + &mut sidecar, + &vm_a, + "proc-a", + JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.server_poll"), + args: vec![json!(server_id_a), json!(250)], + }, + ) + .expect("accept vm a connection"); + let accepted_b = call_javascript_sync_rpc( + &mut sidecar, + &vm_b, + "proc-b", + JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.server_poll"), + args: vec![json!(server_id_b), json!(250)], + }, + ) + .expect("accept vm b connection"); + assert_eq!(accepted_a["type"], Value::from("connection")); + assert_eq!(accepted_b["type"], Value::from("connection")); + assert_eq!(accepted_a["localPort"], Value::from(43111)); + assert_eq!(accepted_b["localPort"], Value::from(43111)); + + let query_a = sidecar + .dispatch(request( + 50, + OwnershipScope::vm(&connection_id, &session_id, &vm_a), + RequestPayload::FindListener(FindListenerRequest { + host: Some(String::from("127.0.0.1")), + port: Some(43111), + path: None, + }), + )) + .expect("query vm a listener"); + let query_b = sidecar + .dispatch(request( + 51, + OwnershipScope::vm(&connection_id, &session_id, &vm_b), + RequestPayload::FindListener(FindListenerRequest { + host: Some(String::from("127.0.0.1")), + port: Some(43111), + path: None, + }), + )) + .expect("query vm b listener"); + match query_a.response.payload { + ResponsePayload::ListenerSnapshot(snapshot) => { + let listener = snapshot.listener.expect("vm a listener"); + assert_eq!(listener.process_id, "proc-a"); + assert_eq!(listener.host.as_deref(), Some("127.0.0.1")); + assert_eq!(listener.port, Some(43111)); + } + other => panic!("unexpected vm a listener response: {other:?}"), + } + match query_b.response.payload { + ResponsePayload::ListenerSnapshot(snapshot) => { + let listener = snapshot.listener.expect("vm b listener"); + assert_eq!(listener.process_id, "proc-b"); + assert_eq!(listener.host.as_deref(), Some("127.0.0.1")); + assert_eq!(listener.port, Some(43111)); + } + other => panic!("unexpected vm b listener response: {other:?}"), + } } #[test] - fn scoped_host_filesystem_realpath_preserves_paths_outside_guest_root() { - let bridge = SharedBridge::new(RecordingBridge::default()); - bridge - .inspect(|bridge| { - agent_os_bridge::FilesystemBridge::symlink( - bridge, - SymlinkRequest { - vm_id: String::from("vm-1"), - target_path: String::from("/database"), - link_path: String::from("/data/alias"), - }, - ) - .expect("seed alias symlink"); - }) - .expect("inspect bridge"); + fn javascript_net_rpc_listens_and_connects_over_unix_domain_sockets() { + assert_node_available(); - let filesystem = ScopedHostFilesystem::new(HostFilesystem::new(bridge, "vm-1"), "/data"); + let mut sidecar = create_test_sidecar(); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-net-unix-cwd"); + write_fixture(&cwd.join("entry.mjs"), "setInterval(() => {}, 1000);"); - assert_eq!( - filesystem.realpath("/alias").expect("resolve alias"), - "/database" - ); - } + let context = sidecar + .javascript_engine + .create_context(CreateJavascriptContextRequest { + vm_id: vm_id.clone(), + bootstrap_module: None, + compile_cache_root: None, + }); + let execution = sidecar + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { + vm_id: vm_id.clone(), + context_id: context.context_id, + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"crypto\",\"events\",\"fs\",\"net\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), + cwd: cwd.clone(), + }) + .expect("start fake javascript execution"); - #[test] - fn host_filesystem_realpath_fails_closed_on_circular_symlinks() { - let bridge = SharedBridge::new(RecordingBridge::default()); - bridge - .inspect(|bridge| { - agent_os_bridge::FilesystemBridge::symlink( - bridge, - SymlinkRequest { - vm_id: String::from("vm-1"), - target_path: String::from("/loop-b.txt"), - link_path: String::from("/loop-a.txt"), - }, - ) - .expect("seed loop-a symlink"); - agent_os_bridge::FilesystemBridge::symlink( - bridge, - SymlinkRequest { - vm_id: String::from("vm-1"), - target_path: String::from("/loop-a.txt"), - link_path: String::from("/loop-b.txt"), + let kernel_handle = { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .spawn_process( + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], + SpawnOptions { + requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), + cwd: Some(String::from("/")), + ..SpawnOptions::default() }, ) - .expect("seed loop-b symlink"); - }) - .expect("inspect bridge"); + .expect("spawn kernel javascript process") + }; - let filesystem = HostFilesystem::new(bridge, "vm-1"); - let error = filesystem - .realpath("/loop-a.txt") - .expect_err("circular symlink chain should fail closed"); - assert_eq!(error.code(), "ELOOP"); - } + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.active_processes.insert( + String::from("proc-js-unix"), + ActiveProcess::new( + kernel_handle.pid(), + kernel_handle, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), + ); + } - #[test] - fn configure_vm_host_dir_plugin_fails_closed_for_escape_symlinks() { - let host_dir = temp_dir("agent-os-sidecar-host-dir-escape"); - std::os::unix::fs::symlink("/etc", host_dir.join("escape")).expect("seed escape symlink"); + let bridge = sidecar.bridge.clone(); + let dns = sidecar.vms.get(&vm_id).expect("javascript vm").dns.clone(); + let limits = ResourceLimits::default(); + let socket_paths = JavascriptSocketPathContext { + sandbox_root: cwd.clone(), + mounts: Vec::new(), + listen_policy: VmListenPolicy::default(), + loopback_exempt_ports: BTreeSet::new(), + tcp_loopback_guest_to_host_ports: BTreeMap::new(), + udp_loopback_guest_to_host_ports: BTreeMap::new(), + udp_loopback_host_to_guest_ports: BTreeMap::new(), + used_tcp_guest_ports: BTreeMap::new(), + used_udp_guest_ports: BTreeMap::new(), + }; + let socket_path = "/tmp/agent-os.sock"; + let host_socket_path = cwd.join("tmp/agent-os.sock"); - let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = - authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id).expect("create vm"); + let listen = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 1, + method: String::from("net.listen"), + args: vec![json!({ + "path": socket_path, + "backlog": 1, + })], + }, + &limits, + counts, + ) + .expect("listen on unix socket") + }; + let server_id = listen["serverId"].as_str().expect("server id").to_string(); + assert_eq!(listen["path"], Value::String(String::from(socket_path))); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + assert!( + vm.kernel + .exists(socket_path) + .expect("kernel socket placeholder exists"), + "kernel did not expose unix socket path" + ); + } + assert!(host_socket_path.exists(), "host unix socket path missing"); - sidecar + let listener_lookup = sidecar .dispatch(request( - 4, + 2, OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::ConfigureVm(ConfigureVmRequest { - mounts: vec![MountDescriptor { - guest_path: String::from("/workspace"), - read_only: false, - plugin: MountPluginDescriptor { - id: String::from("host_dir"), - config: json!({ - "hostPath": host_dir, - "readOnly": false, - }), - }, - }], - software: Vec::new(), - permissions: Vec::new(), - instructions: Vec::new(), - projected_modules: Vec::new(), + RequestPayload::FindListener(FindListenerRequest { + host: None, + port: None, + path: Some(String::from(socket_path)), }), )) - .expect("configure host_dir mount"); + .expect("query unix listener"); + match listener_lookup.response.payload { + ResponsePayload::ListenerSnapshot(snapshot) => { + let listener = snapshot.listener.expect("listener snapshot"); + assert_eq!(listener.process_id, "proc-js-unix"); + assert_eq!(listener.path.as_deref(), Some(socket_path)); + } + other => panic!("unexpected listener response payload: {other:?}"), + } + + let connect = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 3, + method: String::from("net.connect"), + args: vec![json!({ + "path": socket_path, + })], + }, + &limits, + counts, + ) + .expect("connect to unix listener") + }; + let client_socket_id = connect["socketId"] + .as_str() + .expect("client socket id") + .to_string(); + assert_eq!( + connect["remotePath"], + Value::String(String::from(socket_path)) + ); + + let accepted = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 4, + method: String::from("net.server_poll"), + args: vec![json!(server_id), json!(250)], + }, + &limits, + counts, + ) + .expect("accept unix socket connection") + }; + let server_socket_id = accepted["socketId"] + .as_str() + .expect("server socket id") + .to_string(); + assert_eq!( + accepted["localPath"], + Value::String(String::from(socket_path)) + ); + + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + let connections = service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 5, + method: String::from("net.server_connections"), + args: vec![json!(server_id)], + }, + &limits, + counts, + ) + .expect("query unix server connections"); + assert_eq!(connections, json!(1)); + } + + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 6, + method: String::from("net.write"), + args: vec![ + json!(client_socket_id), + json!({ + "__agentOsType": "bytes", + "base64": "cGluZw==", + }), + ], + }, + &limits, + counts, + ) + .expect("write unix client payload"); + } - let vm = sidecar.vms.get_mut(&vm_id).expect("configured vm"); - let error = vm - .kernel - .filesystem_mut() - .read_file("/workspace/escape/hostname") - .expect_err("escape symlink should fail closed"); - assert_eq!(error.code(), "EACCES"); + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 7, + method: String::from("net.shutdown"), + args: vec![json!(client_socket_id)], + }, + &limits, + counts, + ) + .expect("shutdown unix client write half"); + } - fs::remove_dir_all(host_dir).expect("remove temp dir"); - } + let server_data = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 8, + method: String::from("net.poll"), + args: vec![json!(server_socket_id), json!(250)], + }, + &limits, + counts, + ) + .expect("poll unix server socket data") + }; + assert_eq!( + server_data["data"]["base64"], + Value::String(String::from("cGluZw==")) + ); - #[test] - fn execute_starts_python_runtime_instead_of_rejecting_it() { - assert_node_available(); + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + let server_end = service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 9, + method: String::from("net.poll"), + args: vec![json!(server_socket_id), json!(250)], + }, + &limits, + counts, + ) + .expect("poll unix server socket end"); + assert_eq!(server_end["type"], Value::String(String::from("end"))); + } - let cache_root = temp_dir("agent-os-sidecar-python-cache"); + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 10, + method: String::from("net.write"), + args: vec![ + json!(server_socket_id), + json!({ + "__agentOsType": "bytes", + "base64": "cG9uZw==", + }), + ], + }, + &limits, + counts, + ) + .expect("write unix server payload"); + } - let mut sidecar = NativeSidecar::with_config( - RecordingBridge::default(), - NativeSidecarConfig { - sidecar_id: String::from("sidecar-python-test"), - compile_cache_root: Some(cache_root), - expected_auth_token: Some(String::from(TEST_AUTH_TOKEN)), - ..NativeSidecarConfig::default() - }, - ) - .expect("create sidecar"); - let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id); + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 11, + method: String::from("net.shutdown"), + args: vec![json!(server_socket_id)], + }, + &limits, + counts, + ) + .expect("shutdown unix server write half"); + } - let result = sidecar - .dispatch(request( - 4, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::Execute(crate::protocol::ExecuteRequest { - process_id: String::from("proc-python"), - runtime: GuestRuntimeKind::Python, - entrypoint: String::from("print('hello from python')"), - args: Vec::new(), - env: BTreeMap::new(), - cwd: None, - }), - )) - .expect("dispatch python execute"); + let client_data = { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 12, + method: String::from("net.poll"), + args: vec![json!(client_socket_id), json!(250)], + }, + &limits, + counts, + ) + .expect("poll unix client socket data") + }; + assert_eq!( + client_data["data"]["base64"], + Value::String(String::from("cG9uZw==")) + ); - match result.response.payload { - ResponsePayload::ProcessStarted(response) => { - assert_eq!(response.process_id, "proc-python"); - assert!( - response.pid.is_some(), - "python runtime should expose a child pid" - ); - } - other => panic!("unexpected execute response: {other:?}"), + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + let client_end = service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 13, + method: String::from("net.poll"), + args: vec![json!(client_socket_id), json!(250)], + }, + &limits, + counts, + ) + .expect("poll unix client socket end"); + assert_eq!(client_end["type"], Value::String(String::from("end"))); } - let vm = sidecar.vms.get(&vm_id).expect("python vm"); - let process = vm - .active_processes - .get("proc-python") - .expect("python process should be tracked"); - assert_eq!(process.runtime, GuestRuntimeKind::Python); - match &process.execution { - ActiveExecution::Python(_) => {} - other => panic!("unexpected active execution variant: {other:?}"), + for (id, request_id) in [(&client_socket_id, 14_u64), (&server_socket_id, 15_u64)] { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: request_id, + method: String::from("net.destroy"), + args: vec![json!(id)], + }, + &limits, + counts, + ) + .expect("destroy unix socket"); + } + + { + let counts = sidecar + .vms + .get(&vm_id) + .and_then(|vm| vm.active_processes.get("proc-js-unix")) + .expect("unix process") + .network_resource_counts(); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + let process = vm + .active_processes + .get_mut("proc-js-unix") + .expect("unix process"); + service_javascript_net_sync_rpc( + &bridge, + &vm_id, + &dns, + &socket_paths, + &mut vm.kernel, + process, + &JavascriptSyncRpcRequest { + id: 16, + method: String::from("net.server_close"), + args: vec![json!(server_id)], + }, + &limits, + counts, + ) + .expect("close unix listener"); } + + sidecar + .dispose_vm_internal( + &connection_id, + &session_id, + &vm_id, + DisposeReason::Requested, + ) + .expect("dispose unix vm"); } #[test] - fn python_vfs_rpc_requests_proxy_into_the_vm_kernel_filesystem() { + fn javascript_child_process_rpc_spawns_nested_node_processes_inside_vm_kernel() { assert_node_available(); let mut sidecar = create_test_sidecar(); - let (connection_id, session_id) = authenticate_and_open_session(&mut sidecar); - let vm_id = create_vm(&mut sidecar, &connection_id, &session_id); - let cwd = temp_dir("agent-os-sidecar-python-vfs-rpc-cwd"); - let pyodide_dir = temp_dir("agent-os-sidecar-python-vfs-rpc-pyodide"); + let (connection_id, session_id) = + authenticate_and_open_session(&mut sidecar).expect("authenticate and open session"); + let vm_id = + create_vm(&mut sidecar, &connection_id, &session_id, Vec::new()).expect("create vm"); + let cwd = temp_dir("agent-os-sidecar-js-child-process-cwd"); write_fixture( - &pyodide_dir.join("pyodide.mjs"), + &cwd.join("child.mjs"), r#" -export async function loadPyodide() { - return { - setStdin(_stdin) {}, - async runPythonAsync(_code) { - await new Promise(() => {}); - }, - }; -} +import fs from "node:fs"; + +const note = fs.readFileSync("/rpc/note.txt", "utf8").trim(); +console.log(`${process.argv[2]}:${process.pid}:${process.ppid}:${note}`); "#, ); write_fixture( - &pyodide_dir.join("pyodide-lock.json"), - "{\"packages\":[]}\n", + &cwd.join("entry.mjs"), + r#" +const { execSync, spawn } = require("node:child_process"); + +const child = spawn("node", ["./child.mjs", "spawn"], { + stdio: ["ignore", "pipe", "pipe"], +}); +let spawnOutput = ""; +child.stdout.setEncoding("utf8"); +child.stdout.on("data", (chunk) => { + spawnOutput += chunk; +}); +await new Promise((resolve, reject) => { + child.on("error", reject); + child.on("close", (code) => { + if (code !== 0) { + reject(new Error(`spawn exit ${code}`)); + return; + } + resolve(); + }); +}); + +const execOutput = execSync("node ./child.mjs exec", { + encoding: "utf8", +}).trim(); + +console.log(JSON.stringify({ + parentPid: process.pid, + childPid: child.pid, + spawnOutput: spawnOutput.trim(), + execOutput, +})); +"#, ); + { + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); + vm.kernel + .write_file("/rpc/note.txt", b"hello from nested child".to_vec()) + .expect("seed rpc note"); + } + let context = sidecar - .python_engine - .create_context(CreatePythonContextRequest { + .javascript_engine + .create_context(CreateJavascriptContextRequest { vm_id: vm_id.clone(), - pyodide_dist_path: pyodide_dir, + bootstrap_module: None, + compile_cache_root: None, }); let execution = sidecar - .python_engine - .start_execution(StartPythonExecutionRequest { + .javascript_engine + .start_execution(StartJavascriptExecutionRequest { vm_id: vm_id.clone(), context_id: context.context_id, - code: String::from("print('hold-open')"), - file_path: None, - env: BTreeMap::new(), + argv: vec![String::from("./entry.mjs")], + env: BTreeMap::from([( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from( + "[\"assert\",\"buffer\",\"console\",\"child_process\",\"crypto\",\"events\",\"fs\",\"path\",\"querystring\",\"stream\",\"string_decoder\",\"timers\",\"url\",\"util\",\"zlib\"]", + ), + )]), cwd: cwd.clone(), }) - .expect("start fake python execution"); + .expect("start fake javascript execution"); let kernel_handle = { - let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); vm.kernel .spawn_process( - PYTHON_COMMAND, - vec![String::from("print('hold-open')")], + JAVASCRIPT_COMMAND, + vec![String::from("./entry.mjs")], SpawnOptions { requester_driver: Some(String::from(EXECUTION_DRIVER_NAME)), cwd: Some(String::from("/")), ..SpawnOptions::default() }, ) - .expect("spawn kernel python process") + .expect("spawn kernel javascript process") }; { - let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); + let vm = sidecar.vms.get_mut(&vm_id).expect("javascript vm"); vm.active_processes.insert( - String::from("proc-python-vfs"), - ActiveProcess { - kernel_pid: kernel_handle.pid(), + String::from("proc-js-child"), + ActiveProcess::new( + kernel_handle.pid(), kernel_handle, - runtime: GuestRuntimeKind::Python, - execution: ActiveExecution::Python(execution), - }, + GuestRuntimeKind::JavaScript, + ActiveExecution::Javascript(execution), + ), ); } - sidecar - .handle_python_vfs_rpc_request( - &vm_id, - "proc-python-vfs", - PythonVfsRpcRequest { - id: 1, - method: PythonVfsRpcMethod::Mkdir, - path: String::from("/rpc"), - content_base64: None, - recursive: false, - }, - ) - .expect("handle python mkdir rpc"); - sidecar - .handle_python_vfs_rpc_request( - &vm_id, - "proc-python-vfs", - PythonVfsRpcRequest { - id: 2, - method: PythonVfsRpcMethod::Write, - path: String::from("/rpc/note.txt"), - content_base64: Some(String::from("aGVsbG8gZnJvbSBzaWRlY2FyIHJwYw==")), - recursive: false, - }, - ) - .expect("handle python write rpc"); + let mut stdout = String::new(); + let mut stderr = String::new(); + let mut exit_code = None; + for _ in 0..96 { + let next_event = { + let vm = sidecar.vms.get(&vm_id).expect("javascript vm"); + vm.active_processes + .get("proc-js-child") + .map(|process| { + process + .execution + .poll_event(Duration::from_secs(5)) + .expect("poll javascript child_process event") + }) + .flatten() + }; + let Some(event) = next_event else { + if exit_code.is_some() { + break; + } + continue; + }; - let content = { - let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); - String::from_utf8( - vm.kernel - .read_file("/rpc/note.txt") - .expect("read bridged file from kernel"), - ) - .expect("utf8 file contents") - }; - assert_eq!(content, "hello from sidecar rpc"); + match &event { + ActiveExecutionEvent::Stdout(chunk) => { + stdout.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Stderr(chunk) => { + stderr.push_str(&String::from_utf8_lossy(chunk)); + } + ActiveExecutionEvent::Exited(code) => exit_code = Some(*code), + _ => {} + } - let process = { - let vm = sidecar.vms.get_mut(&vm_id).expect("python vm"); - vm.active_processes - .remove("proc-python-vfs") - .expect("remove fake python process") - }; - let _ = signal_runtime_process(process.execution.child_pid(), SIGTERM); + sidecar + .handle_execution_event(&vm_id, "proc-js-child", event) + .expect("handle javascript child_process event"); + } + + assert_eq!(exit_code, Some(0), "stderr: {stderr}"); + let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse child_process JSON"); + let parent_pid = parsed["parentPid"].as_u64().expect("parent pid") as u32; + let child_pid = parsed["childPid"].as_u64().expect("child pid") as u32; + let spawn_parts = parsed["spawnOutput"] + .as_str() + .expect("spawn output") + .split(':') + .map(str::to_owned) + .collect::>(); + let exec_parts = parsed["execOutput"] + .as_str() + .expect("exec output") + .split(':') + .map(str::to_owned) + .collect::>(); + + assert_eq!(spawn_parts[0], "spawn"); + assert_eq!(spawn_parts[1].parse::().expect("spawn pid"), child_pid); + assert_eq!( + spawn_parts[2].parse::().expect("spawn ppid"), + parent_pid + ); + assert_eq!(spawn_parts[3], "hello from nested child"); + assert_eq!(exec_parts[0], "exec"); + assert_eq!(exec_parts[2].parse::().expect("exec ppid"), parent_pid); + assert_eq!(exec_parts[3], "hello from nested child"); + } + + #[test] + fn javascript_child_process_internal_bootstrap_env_is_allowlisted() { + let filtered = sanitize_javascript_child_process_internal_bootstrap_env(&BTreeMap::from([ + ( + String::from("AGENT_OS_ALLOWED_NODE_BUILTINS"), + String::from("[\"fs\"]"), + ), + ( + String::from("AGENT_OS_GUEST_PATH_MAPPINGS"), + String::from("[]"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_UID"), + String::from("0"), + ), + ( + String::from("AGENT_OS_VIRTUAL_PROCESS_VERSION"), + String::from("v24.0.0"), + ), + ( + String::from("AGENT_OS_VIRTUAL_OS_HOSTNAME"), + String::from("agent-os-test"), + ), + ( + String::from("AGENT_OS_PARENT_NODE_ALLOW_CHILD_PROCESS"), + String::from("1"), + ), + ( + String::from("VISIBLE_MARKER"), + String::from("child-visible"), + ), + ])); + + assert_eq!( + filtered.get("AGENT_OS_ALLOWED_NODE_BUILTINS"), + Some(&String::from("[\"fs\"]")) + ); + assert_eq!( + filtered.get("AGENT_OS_GUEST_PATH_MAPPINGS"), + Some(&String::from("[]")) + ); + assert_eq!( + filtered.get("AGENT_OS_VIRTUAL_PROCESS_UID"), + Some(&String::from("0")) + ); + assert_eq!( + filtered.get("AGENT_OS_VIRTUAL_PROCESS_VERSION"), + Some(&String::from("v24.0.0")) + ); + assert_eq!( + filtered.get("AGENT_OS_VIRTUAL_OS_HOSTNAME"), + Some(&String::from("agent-os-test")) + ); + assert!(!filtered.contains_key("AGENT_OS_PARENT_NODE_ALLOW_CHILD_PROCESS")); + assert!(!filtered.contains_key("VISIBLE_MARKER")); } } diff --git a/crates/sidecar/src/stdio.rs b/crates/sidecar/src/stdio.rs index 54edfe6f1..3b3fc3cb1 100644 --- a/crates/sidecar/src/stdio.rs +++ b/crates/sidecar/src/stdio.rs @@ -12,7 +12,8 @@ use agent_os_bridge::{ WriteFileRequest, }; use agent_os_sidecar::protocol::{ - AuthenticatedResponse, NativeFrameCodec, ProtocolFrame, ResponsePayload, SessionOpenedResponse, + AuthenticatedResponse, NativeFrameCodec, ProtocolCodecError, ProtocolFrame, ResponsePayload, + SessionOpenedResponse, }; use agent_os_sidecar::{NativeSidecar, NativeSidecarConfig}; use nix::poll::{poll, PollFd, PollFlags, PollTimeout}; @@ -145,6 +146,13 @@ fn read_frame( } let declared_len = u32::from_be_bytes(prefix) as usize; + if declared_len > codec.max_frame_bytes() { + return Err(ProtocolCodecError::FrameTooLarge { + size: declared_len, + max: codec.max_frame_bytes(), + } + .into()); + } let total_len = prefix.len().saturating_add(declared_len); let mut bytes = Vec::with_capacity(total_len); bytes.extend_from_slice(&prefix); @@ -168,6 +176,27 @@ fn default_compile_cache_root() -> PathBuf { )) } +#[cfg(test)] +mod tests { + use super::*; + use std::io::Cursor; + + #[test] + fn read_frame_rejects_oversized_prefix_before_allocating_payload() { + let codec = NativeFrameCodec::new(16); + let mut reader = Cursor::new((32_u32).to_be_bytes().to_vec()); + + let error = read_frame(&codec, &mut reader).expect_err("oversized frame should fail"); + let error = error + .downcast::() + .expect("protocol codec error"); + assert!(matches!( + *error, + ProtocolCodecError::FrameTooLarge { size: 32, max: 16 } + )); + } +} + #[derive(Debug, Clone)] struct LocalBridge { started_at: Instant, @@ -311,30 +340,42 @@ impl FilesystemBridge for LocalBridge { impl PermissionBridge for LocalBridge { fn check_filesystem_access( &mut self, - _request: FilesystemPermissionRequest, + request: FilesystemPermissionRequest, ) -> Result { - Ok(PermissionDecision::allow()) + Ok(PermissionDecision::deny(format!( + "no static filesystem policy registered for {}:{}", + request.vm_id, request.path + ))) } fn check_network_access( &mut self, - _request: NetworkPermissionRequest, + request: NetworkPermissionRequest, ) -> Result { - Ok(PermissionDecision::allow()) + Ok(PermissionDecision::deny(format!( + "no static network policy registered for {}:{}", + request.vm_id, request.resource + ))) } fn check_command_execution( &mut self, - _request: CommandPermissionRequest, + request: CommandPermissionRequest, ) -> Result { - Ok(PermissionDecision::allow()) + Ok(PermissionDecision::deny(format!( + "no static child_process policy registered for {}:{}", + request.vm_id, request.command + ))) } fn check_environment_access( &mut self, - _request: EnvironmentPermissionRequest, + request: EnvironmentPermissionRequest, ) -> Result { - Ok(PermissionDecision::allow()) + Ok(PermissionDecision::deny(format!( + "no static env policy registered for {}:{}", + request.vm_id, request.key + ))) } } diff --git a/crates/sidecar/tests/connection_auth.rs b/crates/sidecar/tests/connection_auth.rs index af7ee0a0e..a6ee00631 100644 --- a/crates/sidecar/tests/connection_auth.rs +++ b/crates/sidecar/tests/connection_auth.rs @@ -40,6 +40,7 @@ fn authenticate_ignores_client_connection_hints_and_preserves_existing_owners() cwd.to_string_lossy().into_owned(), )]), root_filesystem: Default::default(), + permissions: Vec::new(), }), )) .expect("dispatch cross-connection create_vm"); diff --git a/crates/sidecar/tests/kill_cleanup.rs b/crates/sidecar/tests/kill_cleanup.rs index b81a77df0..28c209141 100644 --- a/crates/sidecar/tests/kill_cleanup.rs +++ b/crates/sidecar/tests/kill_cleanup.rs @@ -192,6 +192,7 @@ fn dispose_vm_succeeds_even_when_a_guest_process_is_running() { cwd.to_string_lossy().into_owned(), )]), root_filesystem: Default::default(), + permissions: Vec::new(), }), )) .expect("create replacement vm after dispose"); @@ -253,6 +254,7 @@ fn close_session_removes_the_session_and_disposes_owned_vms() { cwd.to_string_lossy().into_owned(), )]), root_filesystem: Default::default(), + permissions: Vec::new(), }), )) .expect("dispatch closed-session create_vm"); diff --git a/crates/sidecar/tests/protocol.rs b/crates/sidecar/tests/protocol.rs index ff7f737d6..c961f57a1 100644 --- a/crates/sidecar/tests/protocol.rs +++ b/crates/sidecar/tests/protocol.rs @@ -88,6 +88,7 @@ fn codec_rejects_invalid_ownership_binding() { runtime: GuestRuntimeKind::JavaScript, metadata: BTreeMap::new(), root_filesystem: Default::default(), + permissions: Vec::new(), }), )); @@ -128,6 +129,7 @@ fn response_tracker_enforces_request_response_correlation_and_duplicate_hardenin runtime: GuestRuntimeKind::JavaScript, metadata: BTreeMap::new(), root_filesystem: Default::default(), + permissions: Vec::new(), }), ); tracker @@ -169,6 +171,7 @@ fn response_tracker_rejects_kind_and_ownership_mismatches() { runtime: GuestRuntimeKind::WebAssembly, metadata: BTreeMap::from([(String::from("runtime"), String::from("wasm"))]), root_filesystem: Default::default(), + permissions: Vec::new(), }), ); tracker @@ -303,6 +306,7 @@ fn schema_supports_configuration_and_structured_events() { package_name: "workspace".to_string(), entrypoint: "/workspace/index.ts".to_string(), }], + command_permissions: BTreeMap::new(), }), )); diff --git a/crates/sidecar/tests/python.rs b/crates/sidecar/tests/python.rs index 53faa2b37..09ad71887 100644 --- a/crates/sidecar/tests/python.rs +++ b/crates/sidecar/tests/python.rs @@ -110,6 +110,7 @@ fn execute_python_entrypoint_with_env( args: Vec::new(), env, cwd: None, + wasm_permission_tier: None, }), )) .expect("start python execution"); @@ -144,6 +145,7 @@ fn execute_javascript_with_env( args, env, cwd: None, + wasm_permission_tier: None, }), )) .expect("start JavaScript execution"); @@ -176,6 +178,7 @@ fn create_vm_with_root_filesystem( cwd.to_string_lossy().into_owned(), )]), root_filesystem, + permissions: Vec::new(), }), )) .expect("create sidecar VM"); @@ -581,7 +584,7 @@ fn python_runtime_reports_syntax_errors_over_stderr() { } #[test] -fn python_runtime_enforces_frozen_time_and_blocks_node_escape_hatches() { +fn python_runtime_blocks_pyodide_js_escape_hatches() { assert_node_available(); let mut sidecar = new_sidecar("python-security"); @@ -606,8 +609,8 @@ fn python_runtime_enforces_frozen_time_and_blocks_node_escape_hatches() { "proc-python-security", r#" import json -import time import js +import pyodide_js def capture(action): try: @@ -622,11 +625,11 @@ def capture(action): } result = { - "time_ms": int(time.time() * 1000), - "date_now_ms": int(js.Date.now()), - "child_process": capture(lambda: js.process.getBuiltinModule("node:child_process")), - "vm": capture(lambda: js.process.getBuiltinModule("node:vm")), - "fetch": capture(lambda: js.fetch("http://127.0.0.1:1/")), + "js_process_env": capture(lambda: js.process.env), + "js_require": capture(lambda: js.require), + "js_process_exit": capture(lambda: js.process.exit), + "js_process_kill": capture(lambda: js.process.kill), + "pyodide_js_eval_code": capture(lambda: pyodide_js.eval_code), } print(json.dumps(result)) @@ -648,30 +651,33 @@ print(json.dumps(result)) ); let parsed: Value = serde_json::from_str(stdout.trim()).expect("parse python security JSON"); - let time_ms = parsed["time_ms"].as_i64().expect("time_ms as i64"); - let date_now_ms = parsed["date_now_ms"].as_i64().expect("date_now_ms as i64"); - assert!( - (time_ms - date_now_ms).abs() <= 1, - "expected frozen Python and JS clocks to stay aligned within 1ms, got {time_ms} vs {date_now_ms}" + for key in [ + "js_process_env", + "js_require", + "js_process_exit", + "js_process_kill", + ] { + assert_eq!(parsed[key]["ok"], Value::Bool(false)); + assert_eq!( + parsed[key]["type"], + Value::String(String::from("RuntimeError")) + ); + assert_eq!(parsed[key]["code"], Value::Null); + assert!(parsed[key]["message"] + .as_str() + .expect("js hardening message") + .contains("js is not available")); + } + assert_eq!(parsed["pyodide_js_eval_code"]["ok"], Value::Bool(false)); + assert_eq!( + parsed["pyodide_js_eval_code"]["type"], + Value::String(String::from("RuntimeError")) ); - assert_eq!(parsed["child_process"]["ok"], Value::Bool(false)); - assert_eq!(parsed["child_process"]["code"], "ERR_ACCESS_DENIED"); - assert!(parsed["child_process"]["message"] - .as_str() - .expect("child_process message") - .contains("node:child_process")); - assert_eq!(parsed["vm"]["ok"], Value::Bool(false)); - assert_eq!(parsed["vm"]["code"], "ERR_ACCESS_DENIED"); - assert!(parsed["vm"]["message"] - .as_str() - .expect("vm message") - .contains("node:vm")); - assert_eq!(parsed["fetch"]["ok"], Value::Bool(false)); - assert_eq!(parsed["fetch"]["code"], "ERR_ACCESS_DENIED"); - assert!(parsed["fetch"]["message"] + assert_eq!(parsed["pyodide_js_eval_code"]["code"], Value::Null); + assert!(parsed["pyodide_js_eval_code"]["message"] .as_str() - .expect("fetch message") - .contains("network access")); + .expect("pyodide_js hardening message") + .contains("pyodide_js is not available")); } #[test] @@ -947,6 +953,7 @@ if (mode === 'write') { permissions: Vec::new(), instructions: Vec::new(), projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), }), )) .expect("configure host_dir workspace mount"); diff --git a/crates/sidecar/tests/security_audit.rs b/crates/sidecar/tests/security_audit.rs new file mode 100644 index 000000000..584a1d816 --- /dev/null +++ b/crates/sidecar/tests/security_audit.rs @@ -0,0 +1,327 @@ +mod support; + +use agent_os_bridge::StructuredEventRecord; +use agent_os_sidecar::protocol::{ + BootstrapRootFilesystemRequest, ConfigureVmRequest, ExecuteRequest, GuestFilesystemCallRequest, + GuestFilesystemOperation, GuestRuntimeKind, KillProcessRequest, MountDescriptor, + MountPluginDescriptor, OwnershipScope, PermissionDescriptor, PermissionMode, RequestPayload, + ResponsePayload, RootFilesystemEntry, RootFilesystemEntryKind, +}; +use support::{ + assert_node_available, authenticate, authenticate_with_token, collect_process_output, + create_vm, open_session, request, temp_dir, write_fixture, RecordingBridge, +}; + +fn structured_events( + sidecar: &agent_os_sidecar::NativeSidecar, +) -> Vec { + sidecar + .with_bridge_mut(|bridge| bridge.structured_events.clone()) + .expect("inspect structured events") +} + +fn find_event<'a>(events: &'a [StructuredEventRecord], name: &str) -> &'a StructuredEventRecord { + events + .iter() + .find(|event| event.name == name) + .unwrap_or_else(|| panic!("missing structured event: {name}")) +} + +fn assert_timestamp(event: &StructuredEventRecord) { + event.fields["timestamp"] + .parse::() + .unwrap_or_else(|error| panic!("invalid audit timestamp: {error}")); +} + +#[test] +fn auth_failures_emit_security_audit_events() { + let mut sidecar = support::new_sidecar("security-audit-auth"); + + let result = authenticate_with_token(&mut sidecar, 1, "conn-hint", "wrong-token"); + match result.response.payload { + ResponsePayload::Rejected(rejected) => { + assert_eq!(rejected.code, "unauthorized"); + assert!(rejected.message.contains("invalid auth token")); + } + other => panic!("unexpected auth failure response: {other:?}"), + } + + let events = structured_events(&sidecar); + let event = find_event(&events, "security.auth.failed"); + assert_eq!(event.vm_id, "sidecar-security-audit-auth"); + assert_eq!(event.fields["source"], "sidecar-tests"); + assert_eq!(event.fields["connection_id"], "conn-hint"); + assert!(event.fields["reason"].contains("invalid auth token")); + assert_timestamp(event); +} + +#[test] +fn filesystem_permission_denials_emit_security_audit_events() { + let mut sidecar = support::new_sidecar("security-audit-permissions"); + let cwd = temp_dir("security-audit-permissions-cwd"); + + let connection_id = authenticate(&mut sidecar, "conn-1"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let (vm_id, _) = create_vm( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + ); + + let denied_vm_id = vm_id.clone(); + let sidecar = &mut sidecar; + let _ = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: Vec::new(), + software: Vec::new(), + permissions: vec![ + PermissionDescriptor { + capability: String::from("fs"), + mode: PermissionMode::Allow, + }, + PermissionDescriptor { + capability: String::from("fs.read"), + mode: PermissionMode::Deny, + }, + ], + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: Default::default(), + }), + )) + .expect("configure vm permissions"); + + let write = sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &denied_vm_id), + RequestPayload::GuestFilesystemCall(GuestFilesystemCallRequest { + operation: GuestFilesystemOperation::WriteFile, + path: String::from("/blocked.txt"), + destination_path: None, + target: None, + content: Some(String::from("blocked")), + encoding: Some(agent_os_sidecar::protocol::RootFilesystemEntryEncoding::Utf8), + recursive: false, + mode: None, + uid: None, + gid: None, + atime_ms: None, + mtime_ms: None, + len: None, + }), + )) + .expect("write blocked file"); + match write.response.payload { + ResponsePayload::GuestFilesystemResult(_) => {} + other => panic!("unexpected write response: {other:?}"), + } + + let read = sidecar + .dispatch(request( + 6, + OwnershipScope::vm(&connection_id, &session_id, &denied_vm_id), + RequestPayload::GuestFilesystemCall(GuestFilesystemCallRequest { + operation: GuestFilesystemOperation::ReadFile, + path: String::from("/blocked.txt"), + destination_path: None, + target: None, + content: None, + encoding: None, + recursive: false, + mode: None, + uid: None, + gid: None, + atime_ms: None, + mtime_ms: None, + len: None, + }), + )) + .expect("dispatch denied read"); + match read.response.payload { + ResponsePayload::Rejected(rejected) => { + assert_eq!(rejected.code, "kernel_error"); + assert!(rejected.message.contains("EACCES")); + } + other => panic!("unexpected read response: {other:?}"), + } + + let events = structured_events(sidecar); + let event = find_event(&events, "security.permission.denied"); + assert_eq!(event.vm_id, denied_vm_id); + assert_eq!(event.fields["operation"], "read"); + assert_eq!(event.fields["path"], "/blocked.txt"); + assert_eq!(event.fields["policy"], "fs.read"); + assert!(event.fields["reason"].contains("fs.read")); + assert_timestamp(event); +} + +#[test] +fn mount_operations_emit_security_audit_events() { + let mut sidecar = support::new_sidecar("security-audit-mounts"); + let cwd = temp_dir("security-audit-mounts-cwd"); + + let connection_id = authenticate(&mut sidecar, "conn-1"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let (vm_id, _) = create_vm( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + ); + + sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::BootstrapRootFilesystem(BootstrapRootFilesystemRequest { + entries: vec![RootFilesystemEntry { + path: String::from("/workspace"), + kind: RootFilesystemEntryKind::Directory, + ..Default::default() + }], + }), + )) + .expect("bootstrap workspace"); + + sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: vec![MountDescriptor { + guest_path: String::from("/workspace"), + read_only: false, + plugin: MountPluginDescriptor { + id: String::from("memory"), + config: serde_json::json!({}), + }, + }], + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: Default::default(), + }), + )) + .expect("mount workspace"); + + sidecar + .dispatch(request( + 6, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::ConfigureVm(ConfigureVmRequest { + mounts: Vec::new(), + software: Vec::new(), + permissions: Vec::new(), + instructions: Vec::new(), + projected_modules: Vec::new(), + command_permissions: Default::default(), + }), + )) + .expect("unmount workspace"); + + let events = structured_events(&sidecar); + let mounted = find_event(&events, "security.mount.mounted"); + assert_eq!(mounted.vm_id, vm_id); + assert_eq!(mounted.fields["guest_path"], "/workspace"); + assert_eq!(mounted.fields["plugin_id"], "memory"); + assert_eq!(mounted.fields["read_only"], "false"); + assert_timestamp(mounted); + + let unmounted = events + .iter() + .rfind(|event| event.name == "security.mount.unmounted") + .expect("missing unmount audit event"); + assert_eq!(unmounted.vm_id, vm_id); + assert_eq!(unmounted.fields["guest_path"], "/workspace"); + assert_eq!(unmounted.fields["plugin_id"], "memory"); + assert_eq!(unmounted.fields["read_only"], "false"); + assert_timestamp(unmounted); +} + +#[test] +fn kill_requests_emit_security_audit_events() { + assert_node_available(); + + let mut sidecar = support::new_sidecar("security-audit-kill"); + let cwd = temp_dir("security-audit-kill-cwd"); + let entry = cwd.join("sleep.cjs"); + write_fixture( + &entry, + "setInterval(() => { process.stdout.write('tick\\n'); }, 1000);\n", + ); + + let connection_id = authenticate(&mut sidecar, "conn-1"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let (vm_id, _) = create_vm( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + ); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::Execute(ExecuteRequest { + process_id: String::from("proc-kill"), + runtime: GuestRuntimeKind::JavaScript, + entrypoint: entry.to_string_lossy().into_owned(), + args: Vec::new(), + env: Default::default(), + cwd: None, + wasm_permission_tier: None, + }), + )) + .expect("start js process"); + match result.response.payload { + ResponsePayload::ProcessStarted(_) => {} + other => panic!("unexpected execute response: {other:?}"), + } + + let result = sidecar + .dispatch(request( + 5, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::KillProcess(KillProcessRequest { + process_id: String::from("proc-kill"), + signal: String::from("SIGTERM"), + }), + )) + .expect("kill js process"); + match result.response.payload { + ResponsePayload::ProcessKilled(_) => {} + other => panic!("unexpected kill response: {other:?}"), + } + + let (_stdout, _stderr, _exit_code) = collect_process_output( + &mut sidecar, + &connection_id, + &session_id, + &vm_id, + "proc-kill", + ); + + let events = structured_events(&sidecar); + let event = find_event(&events, "security.process.kill"); + assert_eq!(event.vm_id, vm_id); + assert_eq!(event.fields["source"], "control_plane"); + assert_eq!(event.fields["source_pid"], "0"); + assert_eq!(event.fields["process_id"], "proc-kill"); + assert_eq!(event.fields["signal"], "SIGTERM"); + assert!(event.fields.contains_key("target_pid")); + assert!(event.fields.contains_key("host_pid")); + assert_timestamp(event); +} diff --git a/crates/sidecar/tests/security_hardening.rs b/crates/sidecar/tests/security_hardening.rs index 54b9f0ae9..ea11f381b 100644 --- a/crates/sidecar/tests/security_hardening.rs +++ b/crates/sidecar/tests/security_hardening.rs @@ -6,12 +6,96 @@ use agent_os_sidecar::protocol::{ use agent_os_sidecar::{NativeSidecar, NativeSidecarConfig}; use serde_json::Value; use std::collections::BTreeMap; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::path::{Path, PathBuf}; use support::{ assert_node_available, authenticate, collect_process_output, create_vm, create_vm_with_metadata, execute, open_session, request, temp_dir, write_fixture, RecordingBridge, TEST_AUTH_TOKEN, }; +const ARG_PREFIX: &str = "ARG="; +const INVOCATION_BREAK: &str = "--END--"; +const NODE_ALLOW_FS_READ_FLAG: &str = "--allow-fs-read="; +const NODE_ALLOW_FS_WRITE_FLAG: &str = "--allow-fs-write="; + +struct EnvVarGuard { + key: &'static str, + previous: Option, +} + +impl EnvVarGuard { + fn set(key: &'static str, value: &Path) -> Self { + let previous = std::env::var(key).ok(); + // SAFETY: These sidecar integration tests mutate process env within a single test scope. + unsafe { + std::env::set_var(key, value); + } + Self { key, previous } + } +} + +impl Drop for EnvVarGuard { + fn drop(&mut self) { + match &self.previous { + Some(value) => unsafe { + std::env::set_var(self.key, value); + }, + None => unsafe { + std::env::remove_var(self.key); + }, + } + } +} + +fn canonical(path: &Path) -> PathBuf { + path.canonicalize() + .unwrap_or_else(|error| panic!("canonicalize {}: {error}", path.display())) +} + +fn write_fake_node_binary(path: &Path, log_path: &Path) { + let script = format!( + "#!/bin/sh\nset -eu\nlog=\"{}\"\nfor arg in \"$@\"; do\n printf 'ARG=%s\\n' \"$arg\" >> \"$log\"\ndone\nprintf '%s\\n' '{}' >> \"$log\"\nexit 0\n", + log_path.display(), + INVOCATION_BREAK, + ); + fs::write(path, script).expect("write fake node binary"); + let mut permissions = fs::metadata(path) + .expect("fake node metadata") + .permissions(); + permissions.set_mode(0o755); + fs::set_permissions(path, permissions).expect("chmod fake node binary"); +} + +fn parse_invocations(log_path: &Path) -> Vec> { + let contents = fs::read_to_string(log_path).expect("read invocation log"); + let separator = format!("{INVOCATION_BREAK}\n"); + contents + .split(&separator) + .filter(|block| !block.trim().is_empty()) + .map(|block| { + block + .lines() + .filter_map(|line| line.strip_prefix(ARG_PREFIX)) + .map(str::to_owned) + .collect::>() + }) + .collect() +} + +fn read_flags(args: &[String]) -> Vec<&str> { + args.iter() + .filter_map(|arg| arg.strip_prefix(NODE_ALLOW_FS_READ_FLAG)) + .collect() +} + +fn write_flags(args: &[String]) -> Vec<&str> { + args.iter() + .filter_map(|arg| arg.strip_prefix(NODE_ALLOW_FS_WRITE_FLAG)) + .collect() +} + #[test] fn sidecar_rejects_oversized_request_frames_before_dispatch() { let root = temp_dir("frame-limit"); @@ -73,7 +157,12 @@ fn guest_execution_clears_host_env_and_blocks_network_and_escape_paths() { const result = { path: process.env.PATH ?? null, home: process.env.HOME ?? null, - marker: process.env.AGENT_OS_ALLOWED ?? null, + marker: process.env.VISIBLE_MARKER ?? null, + internalMarker: process.env.AGENT_OS_ALLOWED ?? null, + guestPathMappings: process.env.AGENT_OS_GUEST_PATH_MAPPINGS ?? null, + importCachePath: process.env.AGENT_OS_NODE_IMPORT_CACHE_PATH ?? null, + hasInternalMarker: 'AGENT_OS_ALLOWED' in process.env, + keys: Object.keys(process.env).filter((key) => key.startsWith('AGENT_OS_')), }; const dataResponse = await fetch('data:text/plain,agent-os-ok'); @@ -107,14 +196,6 @@ fn guest_execution_clears_host_env_and_blocks_network_and_escape_paths() { result.httpImport = { code: error.code ?? null, message: error.message }; } - const fs = require('fs'); - try { - fs.readFileSync('/proc/self/environ', 'utf8'); - result.procEnviron = 'unexpected'; - } catch (error) { - result.procEnviron = { code: error.code ?? null, message: error.message }; - } - console.log(JSON.stringify(result)); })().catch((error) => { console.error(error.stack || String(error)); @@ -132,10 +213,7 @@ fn guest_execution_clears_host_env_and_blocks_network_and_escape_paths() { &session_id, GuestRuntimeKind::JavaScript, &cwd, - BTreeMap::from([( - String::from("env.AGENT_OS_ALLOWED"), - String::from("present"), - )]), + BTreeMap::from([(String::from("env.VISIBLE_MARKER"), String::from("present"))]), ); execute( @@ -164,6 +242,11 @@ fn guest_execution_clears_host_env_and_blocks_network_and_escape_paths() { assert_eq!(parsed["path"], Value::Null); assert_eq!(parsed["home"], Value::Null); assert_eq!(parsed["marker"], Value::String(String::from("present"))); + assert_eq!(parsed["internalMarker"], Value::Null); + assert_eq!(parsed["guestPathMappings"], Value::Null); + assert_eq!(parsed["importCachePath"], Value::Null); + assert_eq!(parsed["hasInternalMarker"], Value::Bool(false)); + assert_eq!(parsed["keys"], Value::Array(Vec::new())); assert_eq!( parsed["dataText"], Value::String(String::from("agent-os-ok")) @@ -188,10 +271,6 @@ fn guest_execution_clears_host_env_and_blocks_network_and_escape_paths() { parsed["httpImport"]["code"], Value::String(String::from("ERR_ACCESS_DENIED")) ); - assert_eq!( - parsed["procEnviron"]["code"], - Value::String(String::from("ERR_ACCESS_DENIED")) - ); } #[test] @@ -247,6 +326,7 @@ console.log("slow"); args: Vec::new(), env: BTreeMap::new(), cwd: None, + wasm_permission_tier: None, }), )) .expect("dispatch second execute"); @@ -291,3 +371,130 @@ console.log("slow"); assert_eq!(stdout.trim(), "fast"); assert!(stderr.is_empty(), "unexpected fast stderr: {stderr}"); } + +#[test] +fn execute_rejects_cwd_outside_vm_sandbox_root() { + let mut sidecar = support::new_sidecar("execute-cwd-validation"); + let cwd = temp_dir("execute-cwd-validation-root"); + let entry = cwd.join("entry.mjs"); + write_fixture(&entry, "console.log('ignored');\n"); + + let connection_id = authenticate(&mut sidecar, "conn-1"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let (vm_id, _) = create_vm( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + ); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::Execute(agent_os_sidecar::protocol::ExecuteRequest { + process_id: String::from("proc-1"), + runtime: GuestRuntimeKind::JavaScript, + entrypoint: entry.to_string_lossy().into_owned(), + args: Vec::new(), + env: BTreeMap::new(), + cwd: Some(String::from("/")), + wasm_permission_tier: None, + }), + )) + .expect("dispatch execute request"); + + match result.response.payload { + ResponsePayload::Rejected(rejected) => { + assert_eq!(rejected.code, "invalid_state"); + assert!(rejected.message.contains("sandbox root")); + assert!(rejected.message.contains(cwd.to_string_lossy().as_ref())); + } + other => panic!("unexpected execute response: {other:?}"), + } +} + +#[test] +fn execute_scopes_node_permission_flags_to_vm_sandbox_root() { + let root = temp_dir("execute-cwd-permission-root"); + let fake_node_path = root.join("fake-node.sh"); + let log_path = root.join("node-args.log"); + write_fake_node_binary(&fake_node_path, &log_path); + let _node_binary = EnvVarGuard::set("AGENT_OS_NODE_BINARY", &fake_node_path); + + let mut sidecar = support::new_sidecar("execute-cwd-permission-root"); + let cwd = root.join("workspace"); + let nested_cwd = cwd.join("nested"); + fs::create_dir_all(&nested_cwd).expect("create nested cwd"); + let entry = cwd.join("entry.mjs"); + write_fixture(&entry, "console.log('ignored');\n"); + + let connection_id = authenticate(&mut sidecar, "conn-1"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let (vm_id, _) = create_vm( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + ); + + let result = sidecar + .dispatch(request( + 4, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::Execute(agent_os_sidecar::protocol::ExecuteRequest { + process_id: String::from("proc-1"), + runtime: GuestRuntimeKind::JavaScript, + entrypoint: entry.to_string_lossy().into_owned(), + args: Vec::new(), + env: BTreeMap::new(), + cwd: Some(nested_cwd.to_string_lossy().into_owned()), + wasm_permission_tier: None, + }), + )) + .expect("dispatch execute request"); + + match result.response.payload { + ResponsePayload::ProcessStarted(response) => { + assert_eq!(response.process_id, "proc-1"); + } + other => panic!("unexpected execute response: {other:?}"), + } + + let (_stdout, stderr, exit_code) = + collect_process_output(&mut sidecar, &connection_id, &session_id, &vm_id, "proc-1"); + assert_eq!(exit_code, 0); + assert!(stderr.is_empty(), "unexpected stderr: {stderr}"); + + let invocations = parse_invocations(&log_path); + assert_eq!( + invocations.len(), + 2, + "expected warmup and execution invocations" + ); + + let sandbox_root = canonical(&cwd).display().to_string(); + let nested_root = canonical(&nested_cwd).display().to_string(); + for args in &invocations { + let read_paths = read_flags(args); + let write_paths = write_flags(args); + assert!( + read_paths.iter().any(|path| *path == sandbox_root.as_str()), + "sandbox root should stay in read allowlist: {args:?}" + ); + assert!( + write_paths + .iter() + .any(|path| *path == sandbox_root.as_str()), + "sandbox root should stay in write allowlist: {args:?}" + ); + assert!( + !write_paths.iter().any(|path| *path == nested_root.as_str()), + "requested cwd should not become a write permission root: {args:?}" + ); + } +} diff --git a/crates/sidecar/tests/session_isolation.rs b/crates/sidecar/tests/session_isolation.rs index ff055ad03..1b49f5ddb 100644 --- a/crates/sidecar/tests/session_isolation.rs +++ b/crates/sidecar/tests/session_isolation.rs @@ -36,6 +36,7 @@ fn sessions_and_vms_reject_cross_connection_access() { cwd.to_string_lossy().into_owned(), )]), root_filesystem: Default::default(), + permissions: Vec::new(), }), )) .expect("dispatch mismatched session create_vm"); diff --git a/crates/sidecar/tests/socket_state_queries.rs b/crates/sidecar/tests/socket_state_queries.rs index dc077d9d1..e04e510cb 100644 --- a/crates/sidecar/tests/socket_state_queries.rs +++ b/crates/sidecar/tests/socket_state_queries.rs @@ -2,14 +2,16 @@ mod support; use agent_os_sidecar::protocol::{ DisposeReason, DisposeVmRequest, EventPayload, FindBoundUdpRequest, FindListenerRequest, - GetSignalStateRequest, GuestRuntimeKind, OwnershipScope, RequestPayload, ResponsePayload, - SignalDispositionAction, + GetSignalStateRequest, GuestRuntimeKind, KillProcessRequest, OwnershipScope, RequestPayload, + ResponsePayload, SignalDispositionAction, }; +use nix::libc; use std::collections::BTreeMap; +use std::fs; use std::time::{Duration, Instant}; use support::{ assert_node_available, authenticate, create_vm_with_metadata, execute, new_sidecar, - open_session, request, temp_dir, write_fixture, + open_session, request, temp_dir, wasm_signal_state_module, write_fixture, }; fn wait_for_process_output( @@ -54,14 +56,14 @@ fn sidecar_queries_listener_udp_and_signal_state() { let cwd = temp_dir("socket-state-queries-cwd"); let tcp_entry = cwd.join("tcp-listener.mjs"); let udp_entry = cwd.join("udp-listener.mjs"); - let signal_entry = cwd.join("signal-state.mjs"); + let signal_entry = cwd.join("signal-state.wasm"); write_fixture( &tcp_entry, [ "import net from 'node:net';", "const server = net.createServer(() => {});", - "server.listen(43111, '0.0.0.0', () => {", + "server.listen(43111, '127.0.0.1', () => {", " console.log('tcp-listening:43111');", "});", ] @@ -72,27 +74,13 @@ fn sidecar_queries_listener_udp_and_signal_state() { [ "import dgram from 'node:dgram';", "const socket = dgram.createSocket('udp4');", - "socket.bind(43112, '0.0.0.0', () => {", + "socket.bind(43112, '127.0.0.1', () => {", " console.log('udp-bound:43112');", "});", ] .join("\n"), ); - write_fixture( - &signal_entry, - [ - "const prefix = '__AGENT_OS_SIGNAL_STATE__:';", - "process.stderr.write(", - " `${prefix}${JSON.stringify({", - " signal: 2,", - " registration: { action: 'user', mask: [15], flags: 0x1234 },", - " })}\\n`,", - ");", - "console.log('signal-registered');", - "setInterval(() => {}, 1000);", - ] - .join("\n"), - ); + fs::write(&signal_entry, wasm_signal_state_module()).expect("write signal-state wasm fixture"); let connection_id = authenticate(&mut sidecar, "conn-1"); let session_id = open_session(&mut sidecar, 2, &connection_id); @@ -109,6 +97,15 @@ fn sidecar_queries_listener_udp_and_signal_state() { allowed_builtins, )]), ); + let (wasm_vm_id, _) = create_vm_with_metadata( + &mut sidecar, + 30, + &connection_id, + &session_id, + GuestRuntimeKind::WebAssembly, + &cwd, + BTreeMap::new(), + ); execute( &mut sidecar, @@ -130,6 +127,52 @@ fn sidecar_queries_listener_udp_and_signal_state() { "tcp-listening:43111", ); + let listener_deadline = Instant::now() + Duration::from_secs(5); + loop { + let listener = sidecar + .dispatch(request( + 7, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::FindListener(FindListenerRequest { + host: Some(String::from("127.0.0.1")), + port: Some(43111), + path: None, + }), + )) + .expect("query tcp listener"); + match listener.response.payload { + ResponsePayload::ListenerSnapshot(snapshot) => { + if let Some(listener) = snapshot.listener { + assert_eq!(listener.process_id, "tcp-listener"); + assert_eq!(listener.host.as_deref(), Some("127.0.0.1")); + assert_eq!(listener.port, Some(43111)); + break; + } + } + other => panic!("unexpected listener response: {other:?}"), + } + assert!( + Instant::now() < listener_deadline, + "timed out waiting for listener snapshot" + ); + std::thread::sleep(Duration::from_millis(25)); + } + + let kill_listener = sidecar + .dispatch(request( + 70, + OwnershipScope::vm(&connection_id, &session_id, &vm_id), + RequestPayload::KillProcess(KillProcessRequest { + process_id: String::from("tcp-listener"), + signal: String::from("SIGTERM"), + }), + )) + .expect("kill tcp listener"); + assert!(matches!( + kill_listener.response.payload, + ResponsePayload::ProcessKilled(_) + )); + execute( &mut sidecar, 5, @@ -155,9 +198,9 @@ fn sidecar_queries_listener_udp_and_signal_state() { 6, &connection_id, &session_id, - &vm_id, + &wasm_vm_id, "signal-state", - GuestRuntimeKind::JavaScript, + GuestRuntimeKind::WebAssembly, &signal_entry, Vec::new(), ); @@ -165,38 +208,17 @@ fn sidecar_queries_listener_udp_and_signal_state() { &mut sidecar, &connection_id, &session_id, - &vm_id, + &wasm_vm_id, "signal-state", "signal-registered", ); - let listener = sidecar - .dispatch(request( - 7, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::FindListener(FindListenerRequest { - host: Some(String::from("0.0.0.0")), - port: Some(43111), - path: None, - }), - )) - .expect("query tcp listener"); - match listener.response.payload { - ResponsePayload::ListenerSnapshot(snapshot) => { - let listener = snapshot.listener.expect("listener snapshot"); - assert_eq!(listener.process_id, "tcp-listener"); - assert_eq!(listener.host.as_deref(), Some("0.0.0.0")); - assert_eq!(listener.port, Some(43111)); - } - other => panic!("unexpected listener response: {other:?}"), - } - let bound_udp = sidecar .dispatch(request( 8, OwnershipScope::vm(&connection_id, &session_id, &vm_id), RequestPayload::FindBoundUdp(FindBoundUdpRequest { - host: Some(String::from("0.0.0.0")), + host: Some(String::from("127.0.0.1")), port: Some(43112), }), )) @@ -205,49 +227,223 @@ fn sidecar_queries_listener_udp_and_signal_state() { ResponsePayload::BoundUdpSnapshot(snapshot) => { let socket = snapshot.socket.expect("bound udp snapshot"); assert_eq!(socket.process_id, "udp-listener"); - assert_eq!(socket.host.as_deref(), Some("0.0.0.0")); + assert_eq!(socket.host.as_deref(), Some("127.0.0.1")); assert_eq!(socket.port, Some(43112)); } other => panic!("unexpected bound udp response: {other:?}"), } - let signal_state = sidecar - .dispatch(request( - 9, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), - RequestPayload::GetSignalState(GetSignalStateRequest { - process_id: String::from("signal-state"), - }), - )) - .expect("query signal state"); - match signal_state.response.payload { - ResponsePayload::SignalState(snapshot) => { - assert_eq!(snapshot.process_id, "signal-state"); - assert_eq!( - snapshot.handlers.get(&2), - Some(&agent_os_sidecar::protocol::SignalHandlerRegistration { - action: SignalDispositionAction::User, - mask: vec![15], - flags: 0x1234, - }) - ); + let signal_deadline = Instant::now() + Duration::from_secs(5); + let wasm_ownership = OwnershipScope::vm(&connection_id, &session_id, &wasm_vm_id); + loop { + let _ = sidecar + .poll_event(&wasm_ownership, Duration::from_millis(25)) + .expect("pump wasm signal-state events"); + let signal_state = sidecar + .dispatch(request( + 9, + wasm_ownership.clone(), + RequestPayload::GetSignalState(GetSignalStateRequest { + process_id: String::from("signal-state"), + }), + )) + .expect("query signal state"); + match signal_state.response.payload { + ResponsePayload::SignalState(snapshot) => { + assert_eq!(snapshot.process_id, "signal-state"); + if snapshot.handlers.get(&2) + == Some(&agent_os_sidecar::protocol::SignalHandlerRegistration { + action: SignalDispositionAction::User, + mask: vec![15], + flags: 0x1234, + }) + { + break; + } + } + other => panic!("unexpected signal state response: {other:?}"), } - other => panic!("unexpected signal state response: {other:?}"), + assert!( + Instant::now() < signal_deadline, + "timed out waiting for signal state" + ); + std::thread::sleep(Duration::from_millis(25)); } let dispose = sidecar .dispatch(request( 10, - OwnershipScope::vm(&connection_id, &session_id, &vm_id), + OwnershipScope::vm(&connection_id, &session_id, &wasm_vm_id), RequestPayload::DisposeVm(DisposeVmRequest { reason: DisposeReason::Requested, }), )) - .expect("dispose vm"); + .expect("dispose wasm vm"); match dispose.response.payload { ResponsePayload::VmDisposed(response) => { - assert_eq!(response.vm_id, vm_id); + assert_eq!(response.vm_id, wasm_vm_id); } other => panic!("unexpected dispose response: {other:?}"), } } + +#[test] +fn sidecar_tracks_javascript_sigchld_and_delivers_it_on_child_exit() { + assert_node_available(); + + let mut sidecar = new_sidecar("socket-state-sigchld"); + let cwd = temp_dir("socket-state-sigchld-cwd"); + let parent_entry = cwd.join("parent.mjs"); + let child_entry = cwd.join("child.mjs"); + + write_fixture( + &child_entry, + [ + "await new Promise((resolve) => setTimeout(resolve, 200));", + "console.log('child-exit');", + ] + .join("\n"), + ); + write_fixture( + &parent_entry, + [ + "import { spawn } from 'node:child_process';", + "let sigchldCount = 0;", + "process.on('SIGCHLD', () => {", + " sigchldCount += 1;", + " console.log(`sigchld:${sigchldCount}`);", + "});", + "console.log('sigchld-registered');", + "const child = spawn('node', ['./child.mjs'], { stdio: ['ignore', 'ignore', 'ignore'] });", + "await new Promise((resolve, reject) => {", + " child.on('error', reject);", + " child.on('close', (code) => {", + " if (code !== 0) {", + " reject(new Error(`child exit ${code}`));", + " return;", + " }", + " resolve();", + " });", + "});", + "const deadline = Date.now() + 2000;", + "while (sigchldCount === 0 && Date.now() < deadline) {", + " await new Promise((resolve) => setTimeout(resolve, 10));", + "}", + "if (sigchldCount === 0) {", + " throw new Error('SIGCHLD was not delivered');", + "}", + "console.log(`sigchld-final:${sigchldCount}`);", + ] + .join("\n"), + ); + + let connection_id = authenticate(&mut sidecar, "conn-sigchld"); + let session_id = open_session(&mut sidecar, 2, &connection_id); + let allowed_builtins = serde_json::to_string(&[ + "assert", + "buffer", + "child_process", + "console", + "crypto", + "events", + "fs", + "path", + "querystring", + "stream", + "string_decoder", + "timers", + "url", + "util", + "zlib", + ]) + .expect("serialize builtins"); + let (vm_id, _) = create_vm_with_metadata( + &mut sidecar, + 3, + &connection_id, + &session_id, + GuestRuntimeKind::JavaScript, + &cwd, + BTreeMap::from([( + String::from("env.AGENT_OS_ALLOWED_NODE_BUILTINS"), + allowed_builtins, + )]), + ); + + execute( + &mut sidecar, + 4, + &connection_id, + &session_id, + &vm_id, + "sigchld-parent", + GuestRuntimeKind::JavaScript, + &parent_entry, + Vec::new(), + ); + + let ownership = OwnershipScope::vm(&connection_id, &session_id, &vm_id); + let deadline = Instant::now() + Duration::from_secs(10); + let mut signal_registered = false; + let mut saw_registered_output = false; + let mut saw_sigchld_output = false; + let mut saw_final_output = false; + let mut exit_code = None; + + while exit_code.is_none() || !signal_registered { + let signal_state = sidecar + .dispatch(request( + 5, + ownership.clone(), + RequestPayload::GetSignalState(GetSignalStateRequest { + process_id: String::from("sigchld-parent"), + }), + )) + .expect("query sigchld signal state"); + match signal_state.response.payload { + ResponsePayload::SignalState(snapshot) => { + if snapshot.handlers.get(&(libc::SIGCHLD as u32)) + == Some(&agent_os_sidecar::protocol::SignalHandlerRegistration { + action: SignalDispositionAction::User, + mask: vec![], + flags: 0, + }) + { + signal_registered = true; + } + } + other => panic!("unexpected signal state response: {other:?}"), + } + + let event = sidecar + .poll_event(&ownership, Duration::from_millis(100)) + .expect("poll SIGCHLD process"); + if let Some(event) = event { + match event.payload { + EventPayload::ProcessOutput(output) if output.process_id == "sigchld-parent" => { + saw_registered_output |= output.chunk.contains("sigchld-registered"); + saw_sigchld_output |= output.chunk.contains("sigchld:1"); + saw_final_output |= output.chunk.contains("sigchld-final:1"); + } + EventPayload::ProcessExited(exited) if exited.process_id == "sigchld-parent" => { + exit_code = Some(exited.exit_code); + } + _ => {} + } + } + + assert!( + Instant::now() < deadline, + "timed out waiting for SIGCHLD registration/output" + ); + } + + assert!(signal_registered, "SIGCHLD should be registered"); + assert!( + saw_registered_output, + "parent should report SIGCHLD registration" + ); + assert!(saw_sigchld_output, "parent should receive SIGCHLD output"); + assert!(saw_final_output, "parent should report final SIGCHLD count"); + assert_eq!(exit_code, Some(0)); +} diff --git a/crates/sidecar/tests/stdio_binary.rs b/crates/sidecar/tests/stdio_binary.rs index ba759307f..1f7283ef6 100644 --- a/crates/sidecar/tests/stdio_binary.rs +++ b/crates/sidecar/tests/stdio_binary.rs @@ -187,6 +187,7 @@ fn native_sidecar_binary_runs_the_framed_protocol_over_stdio() { temp.to_string_lossy().into_owned(), )]), root_filesystem: Default::default(), + permissions: Vec::new(), }), ), ); @@ -527,6 +528,7 @@ fn native_sidecar_binary_runs_the_framed_protocol_over_stdio() { args: Vec::new(), env: BTreeMap::new(), cwd: None, + wasm_permission_tier: None, }), ), ); @@ -624,6 +626,7 @@ fn native_sidecar_binary_supports_js_bridge_host_filesystem_access() { runtime: GuestRuntimeKind::JavaScript, metadata: BTreeMap::new(), root_filesystem: Default::default(), + permissions: Vec::new(), }), ), ); @@ -660,6 +663,7 @@ fn native_sidecar_binary_supports_js_bridge_host_filesystem_access() { permissions: Vec::new(), instructions: Vec::new(), projected_modules: Vec::new(), + command_permissions: BTreeMap::new(), }), ), ); diff --git a/crates/sidecar/tests/support/mod.rs b/crates/sidecar/tests/support/mod.rs index 20fbe421e..33fad7691 100644 --- a/crates/sidecar/tests/support/mod.rs +++ b/crates/sidecar/tests/support/mod.rs @@ -161,6 +161,7 @@ pub fn create_vm_with_metadata( runtime, metadata, root_filesystem: Default::default(), + permissions: Vec::new(), }), )) .expect("create sidecar VM"); @@ -194,6 +195,7 @@ pub fn execute( args, env: BTreeMap::new(), cwd: None, + wasm_permission_tier: None, }), )) .expect("start sidecar execution"); @@ -302,3 +304,46 @@ pub fn wasm_stdout_module() -> Vec { ) .expect("compile wasm fixture") } + +pub fn wasm_signal_state_module() -> Vec { + wat::parse_str( + r#" +(module + (type $fd_write_t (func (param i32 i32 i32 i32) (result i32))) + (type $proc_sigaction_t (func (param i32 i32 i32 i32 i32) (result i32))) + (import "wasi_snapshot_preview1" "fd_write" (func $fd_write (type $fd_write_t))) + (import "host_process" "proc_sigaction" (func $proc_sigaction (type $proc_sigaction_t))) + (memory (export "memory") 1) + (data (i32.const 32) "signal-registered\n") + (func $_start (export "_start") + (local $spin i32) + (drop + (call $proc_sigaction + (i32.const 2) + (i32.const 2) + (i32.const 16384) + (i32.const 0) + (i32.const 4660) + ) + ) + (i32.store (i32.const 0) (i32.const 32)) + (i32.store (i32.const 4) (i32.const 18)) + (drop + (call $fd_write + (i32.const 1) + (i32.const 0) + (i32.const 1) + (i32.const 24) + ) + ) + (local.set $spin (i32.const 5000000)) + (loop $wait + (local.set $spin (i32.sub (local.get $spin) (i32.const 1))) + (br_if $wait (i32.gt_s (local.get $spin) (i32.const 0))) + ) + ) +) +"#, + ) + .expect("compile signal-state wasm fixture") +} diff --git a/packages/browser/src/driver.ts b/packages/browser/src/driver.ts index 626505160..d2fdefc89 100644 --- a/packages/browser/src/driver.ts +++ b/packages/browser/src/driver.ts @@ -179,6 +179,9 @@ export class OpfsFileSystem implements VirtualFileSystem { return { mode: S_IFREG | 0o644, size: file.size, + blocks: file.size === 0 ? 0 : Math.ceil(file.size / 512), + dev: 1, + rdev: 0, isDirectory: false, isSymbolicLink: false, atimeMs: file.lastModified, @@ -198,6 +201,9 @@ export class OpfsFileSystem implements VirtualFileSystem { return { mode: S_IFDIR | 0o755, size: 4096, + blocks: 8, + dev: 1, + rdev: 0, isDirectory: true, isSymbolicLink: false, atimeMs: now, diff --git a/packages/browser/src/os-filesystem.ts b/packages/browser/src/os-filesystem.ts index c08b9e306..f8403701e 100644 --- a/packages/browser/src/os-filesystem.ts +++ b/packages/browser/src/os-filesystem.ts @@ -468,9 +468,13 @@ export class InMemoryFileSystem implements VirtualFileSystem { } private toStat(entry: Entry): VirtualStat { + const size = entry.type === "file" ? entry.data.length : 4096; return { mode: entry.mode, - size: entry.type === "file" ? entry.data.length : 4096, + size, + blocks: size === 0 ? 0 : Math.ceil(size / 512), + dev: 1, + rdev: 0, isDirectory: entry.type === "dir", isSymbolicLink: entry.type === "symlink", atimeMs: entry.atimeMs, diff --git a/packages/browser/src/runtime.ts b/packages/browser/src/runtime.ts index 20ab881fa..657a1d1cb 100644 --- a/packages/browser/src/runtime.ts +++ b/packages/browser/src/runtime.ts @@ -13,6 +13,9 @@ export interface VirtualDirEntry { export interface VirtualStat { mode: number; size: number; + blocks: number; + dev: number; + rdev: number; isDirectory: boolean; isSymbolicLink: boolean; atimeMs: number; diff --git a/packages/core/src/agent-os.ts b/packages/core/src/agent-os.ts index b34336d42..57b4bfd8a 100644 --- a/packages/core/src/agent-os.ts +++ b/packages/core/src/agent-os.ts @@ -1,7 +1,4 @@ -import { - execFileSync, - spawn as spawnChildProcess, -} from "node:child_process"; +import { execFileSync, spawn as spawnChildProcess } from "node:child_process"; import { existsSync, mkdtempSync, @@ -13,21 +10,32 @@ import { } from "node:fs"; import { tmpdir } from "node:os"; import { + sep as hostPathSeparator, join, posix as posixPath, relative as relativeHostPath, resolve as resolveHostPath, - sep as hostPathSeparator, } from "node:path"; import { fileURLToPath } from "node:url"; +import { type ToolKit, validateToolkits } from "./host-tools.js"; +import { generateToolReference } from "./host-tools-prompt.js"; +import { + type HostToolsServer, + startHostToolsServer, +} from "./host-tools-server.js"; import { + createShimFilesystem, + generateMasterShim, + generateToolkitShim, +} from "./host-tools-shims.js"; +import { + type ConnectTerminalOptions, createInMemoryFileSystem, type Kernel, type KernelExecOptions, type KernelExecResult, type ProcessInfo as KernelProcessInfo, type KernelSpawnOptions, - type ConnectTerminalOptions, type ManagedProcess, type OpenShellOptions, type Permissions, @@ -35,17 +43,6 @@ import { type VirtualFileSystem, type VirtualStat, } from "./runtime-compat.js"; -import { type ToolKit, validateToolkits } from "./host-tools.js"; -import { generateToolReference } from "./host-tools-prompt.js"; -import { - startHostToolsServer, - type HostToolsServer, -} from "./host-tools-server.js"; -import { - createShimFilesystem, - generateMasterShim, - generateToolkitShim, -} from "./host-tools-shims.js"; export type { ConnectTerminalOptions } from "./runtime-compat.js"; @@ -98,18 +95,27 @@ export interface AgentRegistryEntry { installed: boolean; } -import { - createNodeHostNetworkAdapter, -} from "./runtime-compat.js"; import { AcpClient } from "./acp-client.js"; +import { AGENT_CONFIGS, type AgentConfig, type AgentType } from "./agents.js"; import { getBaseEnvironment, getBaseFilesystemEntries, } from "./base-filesystem.js"; +import { CronManager } from "./cron/cron-manager.js"; +import type { ScheduleDriver } from "./cron/schedule-driver.js"; +import { TimerScheduleDriver } from "./cron/timer-driver.js"; +import type { + CronEvent, + CronEventHandler, + CronJob, + CronJobInfo, + CronJobOptions, +} from "./cron/types.js"; import { - snapshotVirtualFilesystem, type FilesystemEntry, + snapshotVirtualFilesystem, } from "./filesystem-snapshot.js"; +import { createHostDirBackend } from "./host-dir-mount.js"; import { createSnapshotExport, type LayerStore, @@ -117,56 +123,50 @@ import { type RootSnapshotExport, type SnapshotLayerHandle, } from "./layers.js"; -import { AGENT_CONFIGS, type AgentConfig, type AgentType } from "./agents.js"; -import { createHostDirBackend } from "./host-dir-mount.js"; +import { getOsInstructions } from "./os-instructions.js"; import { type CommandPackageMetadata, + processSoftware, type SoftwareInput, type SoftwareRoot, - processSoftware, } from "./packages.js"; -import { CronManager } from "./cron/cron-manager.js"; -import type { ScheduleDriver } from "./cron/schedule-driver.js"; -import { TimerScheduleDriver } from "./cron/timer-driver.js"; -import type { - CronEvent, - CronEventHandler, - CronJob, - CronJobInfo, - CronJobOptions, -} from "./cron/types.js"; -import { getOsInstructions } from "./os-instructions.js"; +import type { JsonRpcRequest, JsonRpcResponse } from "./protocol.js"; +import { createNodeHostNetworkAdapter } from "./runtime-compat.js"; import { - Session, - type SessionInitData, type AgentCapabilities, type AgentInfo, type GetEventsOptions, type PermissionReply, + type PermissionRequestHandler, type SequencedEvent, + Session, type SessionConfigOption, type SessionEventHandler, + type SessionInitData, type SessionModeState, - type PermissionRequestHandler, } from "./session.js"; -import type { JsonRpcRequest, JsonRpcResponse } from "./protocol.js"; -import type { InProcessSidecarVmAdmin } from "./sidecar/in-process-transport.js"; import { - AgentOsSidecar, - createAgentOsSidecar, - getSharedAgentOsSidecar, - leaseAgentOsSidecarVm, type AgentOsCreateSidecarOptions, type AgentOsSharedSidecarOptions, + type AgentOsSidecar, type AgentOsSidecarConfig, type AgentOsSidecarVmLease, + createAgentOsSidecar, + getSharedAgentOsSidecar, + leaseAgentOsSidecarVm, } from "./sidecar/handle.js"; -import { NativeSidecarKernelProxy, type LocalCompatMount } from "./sidecar/native-kernel-proxy.js"; -import { NativeSidecarProcessClient } from "./sidecar/native-process-client.js"; +import type { InProcessSidecarVmAdmin } from "./sidecar/in-process-transport.js"; import { serializeMountConfigForSidecar } from "./sidecar/mount-descriptors.js"; +import { + type LocalCompatMount, + NativeSidecarKernelProxy, +} from "./sidecar/native-kernel-proxy.js"; +import { serializePermissionsForSidecar } from "./sidecar/permission-descriptors.js"; +import type { RootFilesystemEntry } from "./sidecar/native-process-client.js"; +import { NativeSidecarProcessClient } from "./sidecar/native-process-client.js"; import { serializeRootFilesystemForSidecar } from "./sidecar/root-filesystem-descriptors.js"; import { createStdoutLineIterable } from "./stdout-lines.js"; -import type { RootFilesystemEntry } from "./sidecar/native-process-client.js"; + export type { AgentOsCreateSidecarOptions, AgentOsSharedSidecarOptions, @@ -272,6 +272,11 @@ export interface AgentOsOptions { software?: SoftwareInput[]; /** Loopback ports to exempt from SSRF checks (for testing with host-side mock servers). */ loopbackExemptPorts?: number[]; + /** + * Allowed Node.js builtins for guest Node processes. + * Defaults to the hardened builtin set used by the native sidecar bridge. + */ + allowedNodeBuiltins?: string[]; /** * Host-side CWD for module access resolution. Sets the directory whose * node_modules are projected into the VM at /root/node_modules/. @@ -893,9 +898,9 @@ function convertSidecarRootSnapshotEntries( function ensureNativeSidecarBinary(): string { if ( - ensuredSidecarBinary - && existsSync(ensuredSidecarBinary) - && !sidecarBinaryNeedsBuild() + ensuredSidecarBinary && + existsSync(ensuredSidecarBinary) && + !sidecarBinaryNeedsBuild() ) { return ensuredSidecarBinary; } @@ -1017,7 +1022,9 @@ function collectSidecarMountPlan(options: { hostMounts: HostMountInfo[]; hostPathMappings: HostMountInfo[]; } { - const sidecarMounts: Array> = []; + const sidecarMounts: Array< + ReturnType + > = []; const hostMounts: HostMountInfo[] = []; const hostPathMappings: HostMountInfo[] = []; const seenMounts = new Set(); @@ -1059,7 +1066,9 @@ function collectSidecarMountPlan(options: { pushMount(mount); } - const moduleNodeModules = resolveHostPath(join(options.moduleAccessCwd, "node_modules")); + const moduleNodeModules = resolveHostPath( + join(options.moduleAccessCwd, "node_modules"), + ); if (existsSync(moduleNodeModules)) { pushMount({ path: "/root/node_modules", @@ -1105,7 +1114,9 @@ function collectSidecarMountPlan(options: { } hostMounts.sort((left, right) => right.vmPath.length - left.vmPath.length); - hostPathMappings.sort((left, right) => right.vmPath.length - left.vmPath.length); + hostPathMappings.sort( + (left, right) => right.vmPath.length - left.vmPath.length, + ); return { sidecarMounts, hostMounts, hostPathMappings }; } @@ -1259,13 +1270,14 @@ export class AgentOs { const commandGuestPaths = collectGuestCommandPaths( preparedCommandDirs.commandDirs, ); - const { sidecarMounts, hostMounts, hostPathMappings } = collectSidecarMountPlan({ - mounts: options?.mounts, - moduleAccessCwd, - softwareRoots: processed.softwareRoots, - commandDirs: preparedCommandDirs.commandDirs, - shimDir: toolShimDir, - }); + const { sidecarMounts, hostMounts, hostPathMappings } = + collectSidecarMountPlan({ + mounts: options?.mounts, + moduleAccessCwd, + softwareRoots: processed.softwareRoots, + commandDirs: preparedCommandDirs.commandDirs, + shimDir: toolShimDir, + }); client = NativeSidecarProcessClient.spawn({ cwd: REPO_ROOT, @@ -1274,6 +1286,9 @@ export class AgentOs { frameTimeoutMs: 60_000, }); const session = await client.authenticateAndOpenSession(); + const sidecarPermissions = serializePermissionsForSidecar( + options?.permissions, + ); const nativeVm = await client.createVm(session, { runtime: "java_script", metadata: { @@ -1286,15 +1301,18 @@ export class AgentOs { options?.rootFilesystem, bootstrapLower, ), + permissions: sidecarPermissions, }); await client.waitForEvent( (event) => - event.payload.type === "vm_lifecycle" - && event.payload.state === "ready", + event.payload.type === "vm_lifecycle" && + event.payload.state === "ready", 10_000, ); await client.configureVm(session, nativeVm, { mounts: sidecarMounts, + permissions: sidecarPermissions, + commandPermissions: processed.commandPermissions, }); rootBridge = new NativeSidecarKernelProxy({ @@ -1305,10 +1323,12 @@ export class AgentOs { cwd: "/home/user", localMounts, commandGuestPaths, + wasmCommandPermissions: processed.commandPermissions, hostPathMappings: hostPathMappings.map((mapping) => ({ guestPath: mapping.vmPath, hostPath: mapping.hostPath, })), + allowedNodeBuiltins: options?.allowedNodeBuiltins, loopbackExemptPorts: options?.loopbackExemptPorts, nodeExecutionCwd: "/home/user", onDispose: cleanup, @@ -1646,15 +1666,12 @@ export class AgentOs { } } - async mkdir( - path: string, - options?: { recursive?: boolean }, - ): Promise { + async mkdir(path: string, options?: { recursive?: boolean }): Promise { if (options?.recursive) { return this._mkdirp(path); } this._assertSafeAbsolutePath(path); - return this.kernel.mkdir(path); + return this.#kernel.mkdir(path); } async readdir(path: string): Promise { @@ -2333,7 +2350,7 @@ export class AgentOs { const { iterable, onStdout } = createStdoutLineIterable(); const launchArgs = [...(config.launchArgs ?? []), ...extraArgs]; let launchEnv = { ...config.defaultEnv, ...extraEnv, ...options?.env }; - let sessionCwd = options?.cwd ?? "/home/user"; + const sessionCwd = options?.cwd ?? "/home/user"; const binPath = this._resolveAdapterBin(config.acpAdapter); if ( (agentType === "pi" || agentType === "pi-cli") && @@ -2544,10 +2561,7 @@ export class AgentOs { /** Send a prompt to the agent and wait for the final response. * Returns the raw JSON-RPC response and the accumulated agent text. */ - async prompt( - sessionId: string, - text: string, - ): Promise { + async prompt(sessionId: string, text: string): Promise { const session = this._requireSession(sessionId); // Collect streamed text while the prompt is running diff --git a/packages/core/src/overlay-filesystem.ts b/packages/core/src/overlay-filesystem.ts index bd439d69e..09f3ca2e4 100644 --- a/packages/core/src/overlay-filesystem.ts +++ b/packages/core/src/overlay-filesystem.ts @@ -26,6 +26,12 @@ export interface OverlayBackendOptions { mode?: "ephemeral" | "read-only"; } +const OVERLAY_METADATA_ROOT = "/.agent-os-overlay"; +const OVERLAY_WHITEOUT_DIR = "/.agent-os-overlay/whiteouts"; +const OVERLAY_OPAQUE_DIR = "/.agent-os-overlay/opaque"; + +type OverlayMarkerKind = "whiteout" | "opaque"; + export function createOverlayBackend( options: OverlayBackendOptions, ): VirtualFileSystem { @@ -42,28 +48,109 @@ export function createOverlayBackend( const upper = mode === "read-only" ? null : options.upper ?? createInMemoryFileSystem(); - const whiteouts = new Set(); function normPath(path: string): string { return posixPath.normalize(path); } - function isWhitedOut(path: string): boolean { - return whiteouts.has(normPath(path)); + function isInternalMetadataPath(path: string): boolean { + const normalized = normPath(path); + return normalized === OVERLAY_METADATA_ROOT + || normalized.startsWith(`${OVERLAY_METADATA_ROOT}/`); + } + + function shouldHideDirectoryEntry(path: string, entryName: string): boolean { + return normPath(path) === "/" && entryName === posixPath.basename(OVERLAY_METADATA_ROOT); } - function addWhiteout(path: string): void { - whiteouts.add(normPath(path)); + function markerDirectory(kind: OverlayMarkerKind): string { + return kind === "whiteout" ? OVERLAY_WHITEOUT_DIR : OVERLAY_OPAQUE_DIR; } - function removeWhiteout(path: string): void { - whiteouts.delete(normPath(path)); + function markerPath(kind: OverlayMarkerKind, path: string): string { + return posixPath.join( + markerDirectory(kind), + Buffer.from(normPath(path)).toString("base64url"), + ); } function throwReadOnly(): never { throw new KernelError("EROFS", "read-only file system"); } + function throwMetadataAccessDenied(path: string, op: string): never { + throw new KernelError("EPERM", `operation not permitted, ${op} '${path}'`); + } + + async function ensureMetadataDirectoriesInUpper(path: string): Promise { + if (!upper) { + throwReadOnly(); + } + await upper.mkdir(OVERLAY_METADATA_ROOT, { recursive: true }); + await upper.mkdir(OVERLAY_WHITEOUT_DIR, { recursive: true }); + await upper.mkdir(OVERLAY_OPAQUE_DIR, { recursive: true }); + } + + async function markerExists(kind: OverlayMarkerKind, path: string): Promise { + if (!upper) { + return false; + } + return upper.exists(markerPath(kind, path)); + } + + async function setMarker( + kind: OverlayMarkerKind, + path: string, + present: boolean, + ): Promise { + if (!upper) { + if (present) { + throwReadOnly(); + } + return; + } + + const pathForMarker = markerPath(kind, path); + if (present) { + await ensureMetadataDirectoriesInUpper(path); + await upper.writeFile(pathForMarker, normPath(path)); + return; + } + + if (await upper.exists(pathForMarker)) { + await upper.removeFile(pathForMarker); + } + } + + async function isWhitedOut(path: string): Promise { + return markerExists("whiteout", path); + } + + async function isOpaqueDirectory(path: string): Promise { + return markerExists("opaque", path); + } + + async function addWhiteout(path: string): Promise { + await setMarker("whiteout", path, true); + } + + async function removeWhiteout(path: string): Promise { + await setMarker("whiteout", path, false); + } + + async function markOpaqueDirectory(path: string): Promise { + await setMarker("opaque", path, true); + } + + async function clearOpaqueDirectory(path: string): Promise { + await setMarker("opaque", path, false); + } + + async function clearPathMetadata(path: string): Promise { + await removeWhiteout(path); + await clearOpaqueDirectory(path); + } + async function existsInFilesystem( filesystem: VirtualFileSystem, path: string, @@ -129,7 +216,7 @@ export function createOverlayBackend( } async function mergedLstat(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await hasEntryInUpper(path)) { @@ -146,6 +233,9 @@ export function createOverlayBackend( if (!upper) { throwReadOnly(); } + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "mkdir"); + } const normalized = normPath(path); const parts = normalized.split("/").filter(Boolean); @@ -198,6 +288,7 @@ export function createOverlayBackend( await upper.mkdir(path); await upper.chmod(path, lower.stat.mode); await upper.chown(path, lower.stat.uid, lower.stat.gid); + await markOpaqueDirectory(path); return; } @@ -208,7 +299,7 @@ export function createOverlayBackend( } async function pathExistsInMergedView(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { return false; } if (await hasEntryInUpper(path)) { @@ -219,7 +310,7 @@ export function createOverlayBackend( const backend: VirtualFileSystem = { async readFile(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await existsInUpper(path)) { @@ -233,7 +324,7 @@ export function createOverlayBackend( }, async readTextFile(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await existsInUpper(path)) { @@ -247,26 +338,33 @@ export function createOverlayBackend( }, async readDir(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such directory: ${path}`); } let directoryExists = false; const entries = new Set(); - - for (let index = lowers.length - 1; index >= 0; index--) { - try { - const lowerEntries = await lowers[index].readDir(path); - directoryExists = true; - for (const entry of lowerEntries) { - if (entry === "." || entry === "..") continue; - const childPath = posixPath.join(normPath(path), entry); - if (!isWhitedOut(childPath)) { - entries.add(entry); + const includeLowers = !(await isOpaqueDirectory(path)); + + if (includeLowers) { + for (let index = lowers.length - 1; index >= 0; index--) { + try { + const lowerEntries = await lowers[index].readDir(path); + directoryExists = true; + for (const entry of lowerEntries) { + if ( + entry === "." + || entry === ".." + || shouldHideDirectoryEntry(path, entry) + ) continue; + const childPath = posixPath.join(normPath(path), entry); + if (!(await isWhitedOut(childPath))) { + entries.add(entry); + } } + } catch { + // This lower does not contribute a directory here. } - } catch { - // This lower does not contribute a directory here. } } @@ -275,7 +373,11 @@ export function createOverlayBackend( const upperEntries = await upper.readDir(path); directoryExists = true; for (const entry of upperEntries) { - if (entry === "." || entry === "..") continue; + if ( + entry === "." + || entry === ".." + || shouldHideDirectoryEntry(path, entry) + ) continue; entries.add(entry); } } catch { @@ -291,26 +393,33 @@ export function createOverlayBackend( }, async readDirWithTypes(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such directory: ${path}`); } let directoryExists = false; const entriesByName = new Map(); - - for (let index = lowers.length - 1; index >= 0; index--) { - try { - const lowerEntries = await lowers[index].readDirWithTypes(path); - directoryExists = true; - for (const entry of lowerEntries) { - if (entry.name === "." || entry.name === "..") continue; - const childPath = posixPath.join(normPath(path), entry.name); - if (!isWhitedOut(childPath)) { - entriesByName.set(entry.name, entry); + const includeLowers = !(await isOpaqueDirectory(path)); + + if (includeLowers) { + for (let index = lowers.length - 1; index >= 0; index--) { + try { + const lowerEntries = await lowers[index].readDirWithTypes(path); + directoryExists = true; + for (const entry of lowerEntries) { + if ( + entry.name === "." + || entry.name === ".." + || shouldHideDirectoryEntry(path, entry.name) + ) continue; + const childPath = posixPath.join(normPath(path), entry.name); + if (!(await isWhitedOut(childPath))) { + entriesByName.set(entry.name, entry); + } } + } catch { + // This lower does not contribute a directory here. } - } catch { - // This lower does not contribute a directory here. } } @@ -319,7 +428,11 @@ export function createOverlayBackend( const upperEntries = await upper.readDirWithTypes(path); directoryExists = true; for (const entry of upperEntries) { - if (entry.name === "." || entry.name === "..") continue; + if ( + entry.name === "." + || entry.name === ".." + || shouldHideDirectoryEntry(path, entry.name) + ) continue; entriesByName.set(entry.name, entry); } } catch { @@ -338,10 +451,13 @@ export function createOverlayBackend( path: string, content: string | Uint8Array, ): Promise { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "open"); + } if (!upper) { throwReadOnly(); } - removeWhiteout(path); + await clearPathMetadata(path); if (await findLowerByEntry(path)) { await copyUpPath(path); } else { @@ -351,10 +467,13 @@ export function createOverlayBackend( }, async createDir(path: string): Promise { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "mkdir"); + } if (!upper) { throwReadOnly(); } - removeWhiteout(path); + await clearPathMetadata(path); if (await pathExistsInMergedView(path)) { throw new KernelError("EEXIST", `file exists: ${path}`); } @@ -366,7 +485,10 @@ export function createOverlayBackend( path: string, options?: { recursive?: boolean }, ): Promise { - removeWhiteout(path); + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "mkdir"); + } + await clearPathMetadata(path); if (await pathExistsInMergedView(path)) { const stat = await mergedLstat(path); if (options?.recursive && stat.isDirectory && !stat.isSymbolicLink) { @@ -382,11 +504,14 @@ export function createOverlayBackend( }, async exists(path: string): Promise { + if (isInternalMetadataPath(path)) { + return false; + } return pathExistsInMergedView(path); }, async stat(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await existsInUpper(path)) { @@ -400,7 +525,10 @@ export function createOverlayBackend( }, async removeFile(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "unlink"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } const lower = await findLowerByExists(path); @@ -414,11 +542,15 @@ export function createOverlayBackend( if (upperExists) { await upper.removeFile(path); } - addWhiteout(path); + await clearOpaqueDirectory(path); + await addWhiteout(path); }, async removeDir(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "rmdir"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such directory: ${path}`); } const lower = await findLowerByExists(path); @@ -432,7 +564,8 @@ export function createOverlayBackend( if (upperExists) { await upper.removeDir(path); } - addWhiteout(path); + await clearOpaqueDirectory(path); + await addWhiteout(path); }, async rename(oldPath: string, newPath: string): Promise { @@ -445,7 +578,7 @@ export function createOverlayBackend( }, async realpath(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await existsInUpper(path)) { @@ -459,16 +592,19 @@ export function createOverlayBackend( }, async symlink(target: string, linkPath: string): Promise { + if (isInternalMetadataPath(linkPath)) { + throwMetadataAccessDenied(linkPath, "symlink"); + } if (!upper) { throwReadOnly(); } - removeWhiteout(linkPath); + await clearPathMetadata(linkPath); await ensureAncestorDirectoriesInUpper(linkPath); return upper.symlink(target, linkPath); }, async readlink(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await hasEntryInUpper(path)) { @@ -482,7 +618,7 @@ export function createOverlayBackend( }, async lstat(path: string): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await hasEntryInUpper(path)) { @@ -496,17 +632,23 @@ export function createOverlayBackend( }, async link(oldPath: string, newPath: string): Promise { + if (isInternalMetadataPath(oldPath) || isInternalMetadataPath(newPath)) { + throwMetadataAccessDenied(newPath, "link"); + } if (!upper) { throwReadOnly(); } - removeWhiteout(newPath); + await clearPathMetadata(newPath); await copyUpPath(oldPath); await ensureAncestorDirectoriesInUpper(newPath); return upper.link(oldPath, newPath); }, async chmod(path: string, modeValue: number): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "chmod"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (!upper) { @@ -519,7 +661,10 @@ export function createOverlayBackend( }, async chown(path: string, uid: number, gid: number): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "chown"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (!upper) { @@ -532,7 +677,10 @@ export function createOverlayBackend( }, async utimes(path: string, atime: number, mtime: number): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "utime"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (!upper) { @@ -552,7 +700,10 @@ export function createOverlayBackend( }, async truncate(path: string, length: number): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "truncate"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (!upper) { @@ -569,7 +720,7 @@ export function createOverlayBackend( offset: number, length: number, ): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path) || await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (await existsInUpper(path)) { @@ -587,7 +738,10 @@ export function createOverlayBackend( offset: number, data: Uint8Array, ): Promise { - if (isWhitedOut(path)) { + if (isInternalMetadataPath(path)) { + throwMetadataAccessDenied(path, "pwrite"); + } + if (await isWhitedOut(path)) { throw new KernelError("ENOENT", `no such file: ${path}`); } if (!upper) { diff --git a/packages/core/src/runtime.ts b/packages/core/src/runtime.ts index 6834fe065..48037c473 100644 --- a/packages/core/src/runtime.ts +++ b/packages/core/src/runtime.ts @@ -97,6 +97,9 @@ export interface VirtualDirEntry { export interface VirtualStat { mode: number; size: number; + blocks: number; + dev: number; + rdev: number; isDirectory: boolean; isSymbolicLink: boolean; atimeMs: number; @@ -859,9 +862,13 @@ export class InMemoryFileSystem implements VirtualFileSystem { } private toStat(entry: MemoryEntry): VirtualStat { + const size = entry.type === "file" ? entry.data.length : 4096; return { mode: entry.mode, - size: entry.type === "file" ? entry.data.length : 4096, + size, + blocks: size === 0 ? 0 : Math.ceil(size / 512), + dev: 1, + rdev: 0, isDirectory: entry.type === "dir", isSymbolicLink: entry.type === "symlink", atimeMs: entry.atimeMs, @@ -900,9 +907,17 @@ export class NodeFileSystem implements VirtualFileSystem { } private toStat(stat: fsSync.Stats): VirtualStat { + const posixStat = stat as fsSync.Stats & { + blocks?: number; + dev?: number; + rdev?: number; + }; return { mode: stat.mode, size: stat.size, + blocks: posixStat.blocks ?? (stat.size === 0 ? 0 : Math.ceil(stat.size / 512)), + dev: posixStat.dev ?? 1, + rdev: posixStat.rdev ?? 0, isDirectory: stat.isDirectory(), isSymbolicLink: stat.isSymbolicLink(), atimeMs: Math.trunc(stat.atimeMs), diff --git a/packages/core/src/sidecar/native-kernel-proxy.ts b/packages/core/src/sidecar/native-kernel-proxy.ts index 338057d8b..96533d0ec 100644 --- a/packages/core/src/sidecar/native-kernel-proxy.ts +++ b/packages/core/src/sidecar/native-kernel-proxy.ts @@ -15,26 +15,27 @@ import { join as joinHostPath, posix as posixPath, } from "node:path"; -import { - type ConnectTerminalOptions, - type Kernel, - type KernelExecOptions, - type KernelExecResult, - type KernelSpawnOptions, - type ManagedProcess, - type OpenShellOptions, - type ProcessInfo, - type ShellHandle, - type VirtualFileSystem, - type VirtualStat, +import type { + ConnectTerminalOptions, + Kernel, + KernelExecOptions, + KernelExecResult, + KernelSpawnOptions, + ManagedProcess, + OpenShellOptions, + PermissionTier, + ProcessInfo, + ShellHandle, + VirtualFileSystem, + VirtualStat, } from "../runtime-compat.js"; -import { +import type { + AuthenticatedSession, + CreatedVm, + GuestFilesystemStat, NativeSidecarProcessClient, - type AuthenticatedSession, - type CreatedVm, - type GuestFilesystemStat, - type SidecarSignalHandlerRegistration, - type SidecarSocketStateEntry, + SidecarSignalHandlerRegistration, + SidecarSocketStateEntry, } from "./native-process-client.js"; const SYNTHETIC_PID_BASE = 1_000_000; @@ -45,19 +46,43 @@ const EXTRA_FS_WRITE_PATHS_ENV = "AGENT_OS_EXTRA_FS_WRITE_PATHS"; const ALLOWED_NODE_BUILTINS_ENV = "AGENT_OS_ALLOWED_NODE_BUILTINS"; const LOOPBACK_EXEMPT_PORTS_ENV = "AGENT_OS_LOOPBACK_EXEMPT_PORTS"; const DEFAULT_ALLOWED_NODE_BUILTINS = [ + "assert", + "buffer", + "console", "child_process", - "dgram", + "crypto", "dns", + "events", + "fs", "http", "http2", "https", - "inspector", - "net", + "os", + "path", + "querystring", + "stream", + "string_decoder", + "timers", "tls", - "v8", - "vm", - "worker_threads", + "url", + "util", + "zlib", ] as const; + +function normalizeAllowedNodeBuiltins( + allowedNodeBuiltins?: readonly string[], +): string[] { + if (allowedNodeBuiltins === undefined) { + return [...DEFAULT_ALLOWED_NODE_BUILTINS]; + } + + return [ + ...new Set( + allowedNodeBuiltins.filter((value) => typeof value === "string"), + ), + ]; +} + const PREFERRED_SIGNAL_NAMES = [ "SIGHUP", "SIGINT", @@ -112,9 +137,9 @@ function buildSignalNameByNumber(): Map { } for (const [name, value] of Object.entries(signals)) { if ( - typeof value === "number" - && !NON_CANONICAL_SIGNAL_NAMES.has(name) - && !names.has(value) + typeof value === "number" && + !NON_CANONICAL_SIGNAL_NAMES.has(name) && + !names.has(value) ) { names.set(value, name); } @@ -204,7 +229,9 @@ interface NativeSidecarKernelProxyOptions { cwd: string; localMounts: LocalCompatMount[]; commandGuestPaths: ReadonlyMap; + wasmCommandPermissions?: Readonly>; hostPathMappings: HostPathMapping[]; + allowedNodeBuiltins?: readonly string[]; loopbackExemptPorts?: number[]; nodeExecutionCwd: string; onDispose?: () => Promise; @@ -222,7 +249,9 @@ export class NativeSidecarKernelProxy { private readonly vm: CreatedVm; private readonly localMounts: LocalCompatMount[]; private readonly commandGuestPaths: Map; + private readonly wasmCommandPermissions: Readonly>; private readonly hostPathMappings: HostPathMapping[]; + private readonly allowedNodeBuiltins: readonly string[]; private readonly loopbackExemptPorts: readonly number[]; private readonly nodeExecutionCwd: string; private readonly onDispose: (() => Promise) | undefined; @@ -254,9 +283,15 @@ export class NativeSidecarKernelProxy { (left, right) => right.path.length - left.path.length, ); this.commandGuestPaths = new Map(options.commandGuestPaths); + this.wasmCommandPermissions = Object.freeze({ + ...(options.wasmCommandPermissions ?? {}), + }); this.hostPathMappings = [...options.hostPathMappings].sort( (left, right) => right.guestPath.length - left.guestPath.length, ); + this.allowedNodeBuiltins = normalizeAllowedNodeBuiltins( + options.allowedNodeBuiltins, + ); this.loopbackExemptPorts = [...(options.loopbackExemptPorts ?? [])]; this.nodeExecutionCwd = options.nodeExecutionCwd; this.onDispose = options.onDispose; @@ -281,7 +316,9 @@ export class NativeSidecarKernelProxy { return this.zombieTimerCountValue; } - registerCommandGuestPaths(commandGuestPaths: ReadonlyMap): void { + registerCommandGuestPaths( + commandGuestPaths: ReadonlyMap, + ): void { for (const [name, guestPath] of commandGuestPaths) { this.commandGuestPaths.set(name, guestPath); (this.commands as Map).set(name, "wasmvm"); @@ -512,13 +549,17 @@ export class NativeSidecarKernelProxy { const { onData, ...shellOptions } = options ?? {}; const shell = this.openShell({ ...shellOptions, - onStderr: shellOptions.onStderr ?? ((data) => { - process.stderr.write(data); - }), - }); - const outputHandler = onData ?? ((data: Uint8Array) => { - stdout.write(data); + onStderr: + shellOptions.onStderr ?? + ((data) => { + process.stderr.write(data); + }), }); + const outputHandler = + onData ?? + ((data: Uint8Array) => { + stdout.write(data); + }); const restoreRawMode = stdin.isTTY && typeof stdin.setRawMode === "function"; const onStdinData = (data: Uint8Array | string) => { @@ -692,10 +733,8 @@ export class NativeSidecarKernelProxy { if (!cached?.pending) { this.listenerLookups.set(key, { value: cached?.value ?? null, - pending: this.refreshSocketLookup( - this.listenerLookups, - key, - () => this.client.findListener(this.session, this.vm, request), + pending: this.refreshSocketLookup(this.listenerLookups, key, () => + this.client.findListener(this.session, this.vm, request), ), }); } @@ -711,10 +750,8 @@ export class NativeSidecarKernelProxy { if (!cached?.pending) { this.boundUdpLookups.set(key, { value: cached?.value ?? null, - pending: this.refreshSocketLookup( - this.boundUdpLookups, - key, - () => this.client.findBoundUdp(this.session, this.vm, request), + pending: this.refreshSocketLookup(this.boundUdpLookups, key, () => + this.client.findBoundUdp(this.session, this.vm, request), ), }); } @@ -755,7 +792,10 @@ export class NativeSidecarKernelProxy { this.vm, entry.processId, ); - this.signalStates.set(entry.pid, toKernelSignalState(signalState.handlers)); + this.signalStates.set( + entry.pid, + toKernelSignalState(signalState.handlers), + ); } catch { this.signalStates.set( entry.pid, @@ -792,6 +832,7 @@ export class NativeSidecarKernelProxy { args: execution.args, env: execution.env, cwd: execution.cwd, + wasmPermissionTier: execution.wasmPermissionTier, }); entry.hostPid = started.pid; entry.started = true; @@ -986,6 +1027,7 @@ export class NativeSidecarKernelProxy { args: string[]; cwd?: string; env?: Record; + wasmPermissionTier?: PermissionTier; bootstrap?: () => Promise; }> { if (entry.command === "node") { @@ -1023,14 +1065,15 @@ export class NativeSidecarKernelProxy { const wasmEntrypoint = this.commandGuestPaths.get(entry.command); if (wasmEntrypoint) { - return { - runtime: "web_assembly", - entrypoint: wasmEntrypoint, - args: entry.args, - cwd: entry.cwd, - env: entry.env, - }; - } + return { + runtime: "web_assembly", + entrypoint: wasmEntrypoint, + args: entry.args, + cwd: entry.cwd, + env: entry.env, + wasmPermissionTier: this.wasmCommandPermissions[entry.command], + }; + } throw new Error( `command not found on native sidecar path: ${entry.command}`, @@ -1142,9 +1185,7 @@ export class NativeSidecarKernelProxy { [GUEST_PATH_MAPPINGS_ENV]: JSON.stringify(pathMappings), [EXTRA_FS_READ_PATHS_ENV]: JSON.stringify(extraReadPaths), [EXTRA_FS_WRITE_PATHS_ENV]: JSON.stringify(extraWritePaths), - [ALLOWED_NODE_BUILTINS_ENV]: JSON.stringify( - DEFAULT_ALLOWED_NODE_BUILTINS, - ), + [ALLOWED_NODE_BUILTINS_ENV]: JSON.stringify(this.allowedNodeBuiltins), [LOOPBACK_EXEMPT_PORTS_ENV]: JSON.stringify( this.loopbackExemptPorts.map((port) => String(port)), ), @@ -1614,6 +1655,9 @@ function toVirtualStat(stat: GuestFilesystemStat): VirtualStat { return { mode: stat.mode, size: stat.size, + blocks: stat.blocks, + dev: stat.dev, + rdev: stat.rdev, isDirectory: stat.is_directory, isSymbolicLink: stat.is_symbolic_link, atimeMs: stat.atime_ms, diff --git a/packages/core/src/sidecar/native-process-client.ts b/packages/core/src/sidecar/native-process-client.ts index aaff4130f..8146e5d7d 100644 --- a/packages/core/src/sidecar/native-process-client.ts +++ b/packages/core/src/sidecar/native-process-client.ts @@ -20,6 +20,7 @@ type SidecarPlacement = | { kind: "explicit"; sidecar_id: string }; type GuestRuntimeKind = "java_script" | "web_assembly"; +type WasmPermissionTier = "full" | "read-write" | "read-only" | "isolated"; type RootFilesystemEntryEncoding = "utf8" | "base64"; type RootFilesystemDescriptor = { @@ -73,6 +74,9 @@ type WireRootFilesystemEntry = { export interface GuestFilesystemStat { mode: number; size: number; + blocks: number; + dev: number; + rdev: number; is_directory: boolean; is_symbolic_link: boolean; atime_ms: number; @@ -144,6 +148,7 @@ type RequestPayload = runtime: GuestRuntimeKind; metadata: Record; root_filesystem: WireRootFilesystemDescriptor; + permissions: WirePermissionDescriptor[]; } | { type: "configure_vm"; @@ -152,6 +157,7 @@ type RequestPayload = permissions: WirePermissionDescriptor[]; instructions: string[]; projected_modules: WireProjectedModuleDescriptor[]; + command_permissions: Record; } | { type: "dispose_vm"; @@ -188,6 +194,7 @@ type RequestPayload = args: string[]; env?: Record; cwd?: string; + wasm_permission_tier?: WasmPermissionTier; } | { type: "write_stdin"; @@ -550,6 +557,7 @@ export class NativeSidecarProcessClient { runtime: GuestRuntimeKind; metadata?: Record; rootFilesystem?: RootFilesystemDescriptor; + permissions?: SidecarPermissionDescriptor[]; }, ): Promise { const response = await this.sendRequest({ @@ -563,6 +571,7 @@ export class NativeSidecarProcessClient { runtime: options.runtime, metadata: options.metadata ?? {}, root_filesystem: toWireRootFilesystemDescriptor(options.rootFilesystem), + permissions: (options.permissions ?? []).map(toWirePermissionDescriptor), }, }); if (response.payload.type !== "vm_created") { @@ -585,6 +594,7 @@ export class NativeSidecarProcessClient { permissions?: SidecarPermissionDescriptor[]; instructions?: string[]; projectedModules?: SidecarProjectedModuleDescriptor[]; + commandPermissions?: Record; }, ): Promise { const response = await this.sendRequest({ @@ -605,6 +615,7 @@ export class NativeSidecarProcessClient { projected_modules: (options.projectedModules ?? []).map( toWireProjectedModuleDescriptor, ), + command_permissions: options.commandPermissions ?? {}, }, }); if (response.payload.type !== "vm_configured") { @@ -926,6 +937,7 @@ export class NativeSidecarProcessClient { args?: string[]; env?: Record; cwd?: string; + wasmPermissionTier?: WasmPermissionTier; }, ): Promise<{ pid: number | null }> { const response = await this.sendRequest({ @@ -943,6 +955,9 @@ export class NativeSidecarProcessClient { args: options.args ?? [], ...(options.env ? { env: options.env } : {}), ...(options.cwd ? { cwd: options.cwd } : {}), + ...(options.wasmPermissionTier + ? { wasm_permission_tier: options.wasmPermissionTier } + : {}), }, }); if (response.payload.type !== "process_started") { diff --git a/packages/core/src/sidecar/permission-descriptors.ts b/packages/core/src/sidecar/permission-descriptors.ts new file mode 100644 index 000000000..12d80a073 --- /dev/null +++ b/packages/core/src/sidecar/permission-descriptors.ts @@ -0,0 +1,346 @@ +import type { + NetworkAccessRequest, + PermissionDecision, + Permissions, +} from "../runtime-compat.js"; +import type { SidecarPermissionDescriptor } from "./native-process-client.js"; + +type SidecarPermissionMode = SidecarPermissionDescriptor["mode"]; + +interface FsPermissionSample { + capability: string; + requests: Array<{ path: string; operation: string }>; +} + +interface NetworkPermissionSample { + capability: string; + requests: NetworkAccessRequest[]; +} + +const DEFAULT_SIDE_CAR_PERMISSIONS: SidecarPermissionDescriptor[] = [ + { capability: "fs", mode: "allow" }, + { capability: "network", mode: "allow" }, + { capability: "child_process", mode: "allow" }, + { capability: "env", mode: "allow" }, +]; + +const FS_PERMISSION_SAMPLES: FsPermissionSample[] = [ + { + capability: "fs.read", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "read" }, + { path: "/tmp/policy-probe.txt", operation: "read" }, + ], + }, + { + capability: "fs.write", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "write" }, + { path: "/tmp/policy-probe.txt", operation: "write" }, + ], + }, + { + capability: "fs.create_dir", + requests: [ + { path: "/workspace/policy-probe-dir", operation: "mkdir" }, + { path: "/tmp/policy-probe-dir", operation: "mkdir" }, + ], + }, + { + capability: "fs.create_dir", + requests: [ + { path: "/workspace/policy-probe-dir", operation: "createDir" }, + { path: "/tmp/policy-probe-dir", operation: "createDir" }, + ], + }, + { + capability: "fs.readdir", + requests: [ + { path: "/workspace", operation: "readdir" }, + { path: "/tmp", operation: "readdir" }, + ], + }, + { + capability: "fs.stat", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "stat" }, + { path: "/tmp/policy-probe.txt", operation: "stat" }, + ], + }, + { + capability: "fs.rm", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "rm" }, + { path: "/tmp/policy-probe.txt", operation: "rm" }, + ], + }, + { + capability: "fs.rename", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "rename" }, + { path: "/tmp/policy-probe.txt", operation: "rename" }, + ], + }, + { + capability: "fs.stat", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "exists" }, + { path: "/tmp/policy-probe.txt", operation: "exists" }, + ], + }, + { + capability: "fs.symlink", + requests: [ + { path: "/workspace/policy-probe-link.txt", operation: "symlink" }, + { path: "/tmp/policy-probe-link.txt", operation: "symlink" }, + ], + }, + { + capability: "fs.readlink", + requests: [ + { path: "/workspace/policy-probe-link.txt", operation: "readlink" }, + { path: "/tmp/policy-probe-link.txt", operation: "readlink" }, + ], + }, + { + capability: "fs.write", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "link" }, + { path: "/tmp/policy-probe.txt", operation: "link" }, + ], + }, + { + capability: "fs.chmod", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "chmod" }, + { path: "/tmp/policy-probe.txt", operation: "chmod" }, + ], + }, + { + capability: "fs.write", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "chown" }, + { path: "/tmp/policy-probe.txt", operation: "chown" }, + ], + }, + { + capability: "fs.write", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "utimes" }, + { path: "/tmp/policy-probe.txt", operation: "utimes" }, + ], + }, + { + capability: "fs.truncate", + requests: [ + { path: "/workspace/policy-probe.txt", operation: "truncate" }, + { path: "/tmp/policy-probe.txt", operation: "truncate" }, + ], + }, + { + capability: "fs.mount_sensitive", + requests: [ + { path: "/etc", operation: "mountSensitive" }, + { path: "/proc", operation: "mountSensitive" }, + ], + }, +] as const; + +const NETWORK_PERMISSION_SAMPLES: NetworkPermissionSample[] = [ + { + capability: "network.fetch", + requests: [ + { + url: "https://example.test/fetch", + host: "example.test", + port: 443, + protocol: "https", + }, + { + url: "http://127.0.0.1:4318/fetch", + host: "127.0.0.1", + port: 4318, + protocol: "http", + }, + ], + }, + { + capability: "network.http", + requests: [ + { + url: "https://example.test/http", + host: "example.test", + port: 443, + protocol: "https", + }, + { + url: "http://127.0.0.1:4318/http", + host: "127.0.0.1", + port: 4318, + protocol: "http", + }, + ], + }, + { + capability: "network.dns", + requests: [ + { host: "example.test", protocol: "dns" }, + { host: "localhost", protocol: "dns" }, + ], + }, + { + capability: "network.listen", + requests: [ + { host: "127.0.0.1", port: 3000, protocol: "tcp" }, + { host: "0.0.0.0", port: 3001, protocol: "tcp" }, + ], + }, +] as const; + +function normalizeDecision(decision: PermissionDecision): SidecarPermissionMode { + if (typeof decision === "boolean") { + return decision ? "allow" : "deny"; + } + return decision.allowed ? "allow" : "deny"; +} + +function inferUniformMode( + label: string, + check: ((request: T) => PermissionDecision) | undefined, + requests: readonly T[], +): SidecarPermissionMode | null { + if (!check) { + return null; + } + const [firstRequest, ...rest] = requests; + if (firstRequest === undefined) { + return null; + } + const mode = normalizeDecision(check(firstRequest)); + for (const request of rest) { + if (normalizeDecision(check(request)) !== mode) { + throw new Error( + `${label} permission callback varies by resource and cannot be serialized for the native sidecar`, + ); + } + } + return mode; +} + +function inferFsDescriptors( + permissions: NonNullable, +): SidecarPermissionDescriptor[] { + const descriptorModes = new Map(); + for (const sample of FS_PERMISSION_SAMPLES) { + const mode = inferUniformMode(sample.capability, permissions, sample.requests); + if (!mode) { + continue; + } + const existingMode = descriptorModes.get(sample.capability); + if (existingMode && existingMode !== mode) { + throw new Error( + `${sample.capability} permission callback varies by operation and cannot be serialized for the native sidecar`, + ); + } + descriptorModes.set(sample.capability, mode); + } + const descriptors = [...descriptorModes.entries()].map(([capability, mode]) => ({ + capability, + mode, + })); + + if (descriptors.length === 0) { + return []; + } + + const [firstDescriptor, ...rest] = descriptors; + if ( + firstDescriptor && + rest.every((descriptor) => descriptor.mode === firstDescriptor.mode) + ) { + return [{ capability: "fs", mode: firstDescriptor.mode }]; + } + + return descriptors; +} + +function inferNetworkDescriptors( + permissions: NonNullable, +): SidecarPermissionDescriptor[] { + const descriptors = NETWORK_PERMISSION_SAMPLES.map((sample) => ({ + capability: sample.capability, + mode: inferUniformMode(sample.capability, permissions, sample.requests), + })).filter( + ( + descriptor, + ): descriptor is SidecarPermissionDescriptor & { + mode: SidecarPermissionMode; + } => descriptor.mode !== null, + ); + + if (descriptors.length === 0) { + return []; + } + + const [firstDescriptor, ...rest] = descriptors; + if ( + firstDescriptor && + rest.every((descriptor) => descriptor.mode === firstDescriptor.mode) + ) { + return [{ capability: "network", mode: firstDescriptor.mode }]; + } + + return descriptors; +} + +export function serializePermissionsForSidecar( + permissions?: Permissions, +): SidecarPermissionDescriptor[] { + if (permissions === undefined) { + return [...DEFAULT_SIDE_CAR_PERMISSIONS]; + } + + const descriptors: SidecarPermissionDescriptor[] = []; + + if (permissions.fs) { + descriptors.push(...inferFsDescriptors(permissions.fs)); + } else { + descriptors.push({ capability: "fs", mode: "allow" }); + } + + if (permissions.network) { + descriptors.push(...inferNetworkDescriptors(permissions.network)); + } else { + descriptors.push({ capability: "network", mode: "allow" }); + } + + if (permissions.childProcess) { + const mode = inferUniformMode( + "child_process", + permissions.childProcess, + [ + { command: "node", args: ["-v"] }, + { command: "bash", args: ["-lc", "true"] }, + ], + ); + if (mode) { + descriptors.push({ capability: "child_process", mode }); + } + } else { + descriptors.push({ capability: "child_process", mode: "allow" }); + } + + if (permissions.env) { + const mode = inferUniformMode("env.read", permissions.env, [ + { name: "HOME", value: "/home/user" }, + { name: "SECRET_KEY", value: "hidden" }, + ]); + if (mode) { + descriptors.push({ capability: "env", mode }); + } + } else { + descriptors.push({ capability: "env", mode: "deny" }); + } + + return descriptors; +} diff --git a/packages/core/tests/allowed-node-builtins.test.ts b/packages/core/tests/allowed-node-builtins.test.ts new file mode 100644 index 000000000..59a8bc772 --- /dev/null +++ b/packages/core/tests/allowed-node-builtins.test.ts @@ -0,0 +1,113 @@ +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, test, vi } from "vitest"; +import type { AgentOsOptions } from "../src/index.js"; +import { NativeSidecarKernelProxy } from "../src/sidecar/native-kernel-proxy.js"; +import type { + AuthenticatedSession, + CreatedVm, + NativeSidecarProcessClient, +} from "../src/sidecar/native-process-client.js"; + +describe("AgentOsOptions.allowedNodeBuiltins", () => { + let proxy: NativeSidecarKernelProxy | null = null; + let fixtureRoot: string | null = null; + + afterEach(async () => { + await proxy?.dispose(); + proxy = null; + if (fixtureRoot) { + rmSync(fixtureRoot, { recursive: true, force: true }); + fixtureRoot = null; + } + }); + + function createMockClient() { + let stopped = false; + const execute = vi.fn( + async ( + _session: AuthenticatedSession, + _vm: CreatedVm, + _execution: { env?: Record }, + ) => { + throw new Error("stop after capture"); + }, + ); + const client = { + waitForEvent: vi.fn(async () => { + while (!stopped) { + await new Promise((resolve) => setTimeout(resolve, 1)); + } + throw new Error("mock stopped"); + }), + execute, + disposeVm: vi.fn(async () => { + stopped = true; + }), + dispose: vi.fn(async () => { + stopped = true; + }), + } as unknown as NativeSidecarProcessClient; + + return { client, execute }; + } + + async function captureAllowedNodeBuiltins( + options: Partial = {}, + ) { + fixtureRoot = mkdtempSync(join(tmpdir(), "agent-os-allowed-builtins-")); + const { client, execute } = createMockClient(); + + proxy = new NativeSidecarKernelProxy({ + client, + session: { + connectionId: "conn-1", + sessionId: "session-1", + } as AuthenticatedSession, + vm: { vmId: "vm-1" } as CreatedVm, + env: { HOME: "/workspace" }, + cwd: "/workspace", + localMounts: [], + commandGuestPaths: new Map(), + hostPathMappings: [ + { + guestPath: "/workspace", + hostPath: fixtureRoot, + }, + ], + allowedNodeBuiltins: options.allowedNodeBuiltins, + nodeExecutionCwd: "/workspace", + }); + + const proc = proxy.spawn("node", ["/workspace/entry.mjs"], { + cwd: "/workspace", + env: { HOME: "/workspace" }, + }); + const exitCode = await proc.wait(); + + expect(exitCode).toBe(1); + expect(execute).toHaveBeenCalledTimes(1); + return execute.mock.calls[0]?.[2]?.env?.AGENT_OS_ALLOWED_NODE_BUILTINS; + } + + test("overrides the native sidecar Node builtin allowlist for guest executions", async () => { + const options: AgentOsOptions = { + allowedNodeBuiltins: ["worker_threads"], + }; + + expect(await captureAllowedNodeBuiltins(options)).toBe( + JSON.stringify(options.allowedNodeBuiltins), + ); + }); + + test("uses the hardened default allowlist when guest executions do not override it", async () => { + const builtins = JSON.parse(await captureAllowedNodeBuiltins()); + expect(builtins).toContain("os"); + expect(builtins).toContain("dns"); + expect(builtins).toContain("http"); + expect(builtins).toContain("http2"); + expect(builtins).toContain("https"); + expect(builtins).toContain("tls"); + }); +}); diff --git a/packages/core/tests/native-sidecar-process.test.ts b/packages/core/tests/native-sidecar-process.test.ts index 30128af76..2beda455b 100644 --- a/packages/core/tests/native-sidecar-process.test.ts +++ b/packages/core/tests/native-sidecar-process.test.ts @@ -335,7 +335,7 @@ describe("native sidecar process client", () => { ); test( - "queries listener, UDP, and signal state through the real sidecar protocol", + "queries listener and UDP through the real sidecar protocol and ignores forged signal-state stderr", async () => { const fixtureRoot = mkdtempSync(join(tmpdir(), "agent-os-native-sidecar-")); cleanupPaths.push(fixtureRoot); @@ -447,17 +447,12 @@ describe("native sidecar process client", () => { runtime: "java_script", entrypoint: "./signal-state.mjs", }); - const signalState = await waitFor( - () => client.getSignalState(session, vm, "signal-state"), - { - isReady: (value) => value.handlers.get(2)?.flags === 0x1234, - }, + const signalState = await client.getSignalState( + session, + vm, + "signal-state", ); - expect(signalState.handlers.get(2)).toEqual({ - action: "user", - mask: [15], - flags: 0x1234, - }); + expect(signalState.handlers.size).toBe(0); await client.killProcess(session, vm, "tcp-listener"); await client.waitForEvent( @@ -563,13 +558,9 @@ describe("native sidecar process client", () => { () => signalStdout, { isReady: (value) => value.includes("registered") }, ); - const registration = await waitFor( - () => kernel.processTable.getSignalState(signalProc.pid).handlers.get(2), - { isReady: (value) => value?.flags === 0x4321 }, + expect(kernel.processTable.getSignalState(signalProc.pid).handlers.get(2)).toBe( + undefined, ); - expect(registration?.action).toBe("user"); - expect(registration?.mask).toEqual(new Set([15])); - expect(registration?.flags).toBe(0x4321); tcpServer.kill(15); udpServer.kill(15); diff --git a/packages/core/tests/overlay-backend.test.ts b/packages/core/tests/overlay-backend.test.ts index b13696380..25dac2bef 100644 --- a/packages/core/tests/overlay-backend.test.ts +++ b/packages/core/tests/overlay-backend.test.ts @@ -142,6 +142,22 @@ describe("OverlayBackend (layer behavior)", () => { expect(text).toBe("resurrected"); }); + test("whiteouts persist when reopening with the same writable upper", async () => { + await overlay.removeFile("/data/base.txt"); + + const reopened = createOverlayBackend({ lower, upper }); + + expect(await reopened.exists("/data/base.txt")).toBe(false); + expect(await reopened.readDir("/data")).not.toContain("base.txt"); + }); + + test("directory copy-up marks the upper directory opaque", async () => { + await overlay.chmod("/data", 0o700); + + expect(await overlay.readDir("/data")).toEqual([]); + expect(await overlay.readDir("/")).not.toContain(".agent-os-overlay"); + }); + test("pread falls through to lower", async () => { const chunk = await overlay.pread("/data/base.txt", 5, 6); expect(new TextDecoder().decode(chunk)).toBe("conten"); diff --git a/packages/core/tests/sidecar-permission-descriptors.test.ts b/packages/core/tests/sidecar-permission-descriptors.test.ts new file mode 100644 index 000000000..0d38b570b --- /dev/null +++ b/packages/core/tests/sidecar-permission-descriptors.test.ts @@ -0,0 +1,50 @@ +import { describe, expect, test } from "vitest"; +import type { Permissions } from "../src/runtime-compat.js"; +import { serializePermissionsForSidecar } from "../src/sidecar/permission-descriptors.js"; + +describe("serializePermissionsForSidecar", () => { + test("uses allow-all descriptors when permissions are omitted", () => { + expect(serializePermissionsForSidecar()).toEqual([ + { capability: "fs", mode: "allow" }, + { capability: "network", mode: "allow" }, + { capability: "child_process", mode: "allow" }, + { capability: "env", mode: "allow" }, + ]); + }); + + test("serializes per-operation fs restrictions and preserves env deny-by-default on partial policies", () => { + const permissions: Permissions = { + fs: ({ operation }) => operation === "read", + network: () => false, + childProcess: () => false, + }; + + expect(serializePermissionsForSidecar(permissions)).toEqual([ + { capability: "fs.read", mode: "allow" }, + { capability: "fs.write", mode: "deny" }, + { capability: "fs.create_dir", mode: "deny" }, + { capability: "fs.readdir", mode: "deny" }, + { capability: "fs.stat", mode: "deny" }, + { capability: "fs.rm", mode: "deny" }, + { capability: "fs.rename", mode: "deny" }, + { capability: "fs.symlink", mode: "deny" }, + { capability: "fs.readlink", mode: "deny" }, + { capability: "fs.chmod", mode: "deny" }, + { capability: "fs.truncate", mode: "deny" }, + { capability: "fs.mount_sensitive", mode: "deny" }, + { capability: "network", mode: "deny" }, + { capability: "child_process", mode: "deny" }, + { capability: "env", mode: "deny" }, + ]); + }); + + test("rejects resource-dependent permission callbacks that the native sidecar cannot serialize", () => { + const permissions: Permissions = { + fs: ({ path }) => path.startsWith("/workspace"), + }; + + expect(() => serializePermissionsForSidecar(permissions)).toThrow( + /varies by resource/, + ); + }); +}); diff --git a/packages/core/tests/wasm-permission-tiers.test.ts b/packages/core/tests/wasm-permission-tiers.test.ts new file mode 100644 index 000000000..487c4277f --- /dev/null +++ b/packages/core/tests/wasm-permission-tiers.test.ts @@ -0,0 +1,87 @@ +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, test, vi } from "vitest"; +import { NativeSidecarKernelProxy } from "../src/sidecar/native-kernel-proxy.js"; +import type { + AuthenticatedSession, + CreatedVm, + NativeSidecarProcessClient, +} from "../src/sidecar/native-process-client.js"; + +describe("WASM command permission tiers", () => { + let proxy: NativeSidecarKernelProxy | null = null; + let fixtureRoot: string | null = null; + + afterEach(async () => { + await proxy?.dispose(); + proxy = null; + if (fixtureRoot) { + rmSync(fixtureRoot, { recursive: true, force: true }); + fixtureRoot = null; + } + }); + + function createMockClient() { + let stopped = false; + const execute = vi.fn(async () => { + throw new Error("stop after capture"); + }); + const client = { + waitForEvent: vi.fn(async () => { + while (!stopped) { + await new Promise((resolve) => setTimeout(resolve, 1)); + } + throw new Error("mock stopped"); + }), + execute, + disposeVm: vi.fn(async () => { + stopped = true; + }), + dispose: vi.fn(async () => { + stopped = true; + }), + } as unknown as NativeSidecarProcessClient; + + return { client, execute }; + } + + test("propagates per-command WASM tiers into sidecar execute requests", async () => { + fixtureRoot = mkdtempSync(join(tmpdir(), "agent-os-wasm-tiers-")); + const { client, execute } = createMockClient(); + + proxy = new NativeSidecarKernelProxy({ + client, + session: { + connectionId: "conn-1", + sessionId: "session-1", + } as AuthenticatedSession, + vm: { vmId: "vm-1" } as CreatedVm, + env: { HOME: "/workspace" }, + cwd: "/workspace", + localMounts: [], + commandGuestPaths: new Map([["grep", "/__agentos/commands/000/grep"]]), + wasmCommandPermissions: { grep: "read-only" }, + hostPathMappings: [ + { + guestPath: "/workspace", + hostPath: fixtureRoot, + }, + ], + nodeExecutionCwd: "/workspace", + }); + + const proc = proxy.spawn("grep", ["needle", "haystack.txt"], { + cwd: "/workspace", + }); + const exitCode = await proc.wait(); + + expect(exitCode).toBe(1); + expect(execute).toHaveBeenCalledTimes(1); + expect(execute.mock.calls[0]?.[2]).toMatchObject({ + runtime: "web_assembly", + entrypoint: "/__agentos/commands/000/grep", + wasmPermissionTier: "read-only", + }); + }); +}); diff --git a/registry/AGENTS.md b/registry/AGENTS.md new file mode 120000 index 000000000..681311eb9 --- /dev/null +++ b/registry/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file diff --git a/registry/CLAUDE.md b/registry/CLAUDE.md index 62c04a535..eccd57d09 100644 --- a/registry/CLAUDE.md +++ b/registry/CLAUDE.md @@ -132,12 +132,16 @@ make publish-clean # Clear publish cache make clean # Remove dist/ and wasm/ from all packages ``` +## Testing + +- External-network registry tests should stay behind `AGENTOS_E2E_NETWORK=1`, probe host connectivity up front so CI can skip cleanly when the internet is unavailable, and retry the in-VM command itself for transient outbound failures instead of hard-failing on the first flaky request. + ## Native Source All WASM command source code lives in `native/`: - `native/crates/commands/` -- Rust command crates (105 commands) - `native/crates/libs/` -- shared Rust libraries (grep engine, awk engine, etc.) -- `native/crates/wasi-ext/` -- WASI extension traits +- `native/crates/wasi-ext/` -- WASI extension traits. Host-import wrappers here, matching wasi-libc patches, and uucore stubs should validate every guest buffer length crossing (`usize` -> `u32`) and reject host-returned lengths that exceed the supplied buffer; `poll()` wrappers should also enforce the exact 8-byte-per-`pollfd` layout. - `native/c/programs/` -- C command source (curl, wget, sqlite3, zip, unzip) - `native/patches/` -- Rust std patches for WASI - `native/Makefile` -- Rust build system diff --git a/registry/native/c/programs/sqlite3_cli.c b/registry/native/c/programs/sqlite3_cli.c index b763a8a3d..ba878b956 100644 --- a/registry/native/c/programs/sqlite3_cli.c +++ b/registry/native/c/programs/sqlite3_cli.c @@ -159,7 +159,24 @@ static int wasiFullPathname(sqlite3_vfs *pVfs, const char *zName, int nOut, char static int wasiRandomness(sqlite3_vfs *pVfs, int nByte, char *zOut) { (void)pVfs; - /* Simple deterministic fill — no /dev/urandom in WASM */ + int fd = open("/dev/urandom", O_RDONLY); + if (fd >= 0) { + int total = 0; + while (total < nByte) { + ssize_t read_len = read(fd, zOut + total, (size_t)(nByte - total)); + if (read_len <= 0) { + break; + } + total += (int)read_len; + } + close(fd); + if (total == nByte) { + return nByte; + } + nByte = total; + } + + /* Fallback only if urandom is unexpectedly unavailable in the runtime. */ for (int i = 0; i < nByte; i++) zOut[i] = (char)(i * 37 + 17); return nByte; } diff --git a/registry/native/crates/wasi-ext/src/lib.rs b/registry/native/crates/wasi-ext/src/lib.rs index f49c9ea87..ab41943ac 100644 --- a/registry/native/crates/wasi-ext/src/lib.rs +++ b/registry/native/crates/wasi-ext/src/lib.rs @@ -19,6 +19,28 @@ pub const ERRNO_NOSYS: Errno = 52; pub const ERRNO_NOENT: Errno = 44; pub const ERRNO_SRCH: Errno = 71; // No such process pub const ERRNO_CHILD: Errno = 10; // No child processes +const POLLFD_BYTES: usize = 8; + +fn checked_u32_len(len: usize) -> Result { + u32::try_from(len).map_err(|_| ERRNO_INVAL) +} + +fn validate_returned_len(len: u32, capacity: usize) -> Result { + match usize::try_from(len) { + Ok(len) if len <= capacity => Ok(len as u32), + _ => Err(ERRNO_INVAL), + } +} + +fn validate_poll_buffer_len(buffer_len: usize, nfds: u32) -> Result<(), Errno> { + let nfds = usize::try_from(nfds).map_err(|_| ERRNO_INVAL)?; + let expected = nfds.checked_mul(POLLFD_BYTES).ok_or(ERRNO_INVAL)?; + if buffer_len == expected { + Ok(()) + } else { + Err(ERRNO_INVAL) + } +} // ============================================================ // host_process module — process management and FD operations @@ -159,17 +181,20 @@ pub fn spawn( cwd: &[u8], ) -> Result { let mut pid: u32 = 0; + let argv_len = checked_u32_len(argv.len())?; + let envp_len = checked_u32_len(envp.len())?; + let cwd_len = checked_u32_len(cwd.len())?; let errno = unsafe { proc_spawn( argv.as_ptr(), - argv.len() as u32, + argv_len, envp.as_ptr(), - envp.len() as u32, + envp_len, stdin_fd, stdout_fd, stderr_fd, cwd.as_ptr(), - cwd.len() as u32, + cwd_len, &mut pid, ) }; @@ -492,7 +517,8 @@ pub fn socket(domain: u32, sock_type: u32, protocol: u32) -> Result /// `addr` is a serialized address string (e.g. "host:port"). /// Returns `Ok(())` on success, `Err(errno)` on failure. pub fn connect(fd: u32, addr: &[u8]) -> Result<(), Errno> { - let errno = unsafe { net_connect(fd, addr.as_ptr(), addr.len() as u32) }; + let addr_len = checked_u32_len(addr.len())?; + let errno = unsafe { net_connect(fd, addr.as_ptr(), addr_len) }; if errno == ERRNO_SUCCESS { Ok(()) } else { @@ -504,10 +530,11 @@ pub fn connect(fd: u32, addr: &[u8]) -> Result<(), Errno> { /// /// Returns `Ok(bytes_sent)` on success, `Err(errno)` on failure. pub fn send(fd: u32, buf: &[u8], flags: u32) -> Result { + let buf_len = checked_u32_len(buf.len())?; let mut sent: u32 = 0; - let errno = unsafe { net_send(fd, buf.as_ptr(), buf.len() as u32, flags, &mut sent) }; + let errno = unsafe { net_send(fd, buf.as_ptr(), buf_len, flags, &mut sent) }; if errno == ERRNO_SUCCESS { - Ok(sent) + validate_returned_len(sent, buf.len()) } else { Err(errno) } @@ -517,10 +544,11 @@ pub fn send(fd: u32, buf: &[u8], flags: u32) -> Result { /// /// Returns `Ok(bytes_received)` on success, `Err(errno)` on failure. pub fn recv(fd: u32, buf: &mut [u8], flags: u32) -> Result { + let buf_len = checked_u32_len(buf.len())?; let mut received: u32 = 0; - let errno = unsafe { net_recv(fd, buf.as_mut_ptr(), buf.len() as u32, flags, &mut received) }; + let errno = unsafe { net_recv(fd, buf.as_mut_ptr(), buf_len, flags, &mut received) }; if errno == ERRNO_SUCCESS { - Ok(received) + validate_returned_len(received, buf.len()) } else { Err(errno) } @@ -543,19 +571,21 @@ pub fn net_close_socket(fd: u32) -> Result<(), Errno> { /// Writes the resolved address into `buf` and returns the number of bytes written. /// Returns `Ok(len)` on success, `Err(errno)` on failure. pub fn getaddrinfo(host: &[u8], port: &[u8], buf: &mut [u8]) -> Result { - let mut len: u32 = buf.len() as u32; + let host_len = checked_u32_len(host.len())?; + let port_len = checked_u32_len(port.len())?; + let mut len = checked_u32_len(buf.len())?; let errno = unsafe { net_getaddrinfo( host.as_ptr(), - host.len() as u32, + host_len, port.as_ptr(), - port.len() as u32, + port_len, buf.as_mut_ptr(), &mut len, ) }; if errno == ERRNO_SUCCESS { - Ok(len) + validate_returned_len(len, buf.len()) } else { Err(errno) } @@ -565,7 +595,8 @@ pub fn getaddrinfo(host: &[u8], port: &[u8], buf: &mut [u8]) -> Result Result<(), Errno> { - let errno = unsafe { net_setsockopt(fd, level, optname, optval.as_ptr(), optval.len() as u32) }; + let optval_len = checked_u32_len(optval.len())?; + let errno = unsafe { net_setsockopt(fd, level, optname, optval.as_ptr(), optval_len) }; if errno == ERRNO_SUCCESS { Ok(()) } else { @@ -578,10 +609,10 @@ pub fn setsockopt(fd: u32, level: u32, optname: u32, optval: &[u8]) -> Result<() /// Writes the serialized address into `buf` and returns the number of bytes written. /// Returns `Ok(len)` on success, `Err(errno)` on failure. pub fn getsockname(fd: u32, buf: &mut [u8]) -> Result { - let mut len: u32 = buf.len() as u32; + let mut len = checked_u32_len(buf.len())?; let errno = unsafe { net_getsockname(fd, buf.as_mut_ptr(), &mut len) }; if errno == ERRNO_SUCCESS { - Ok(len) + validate_returned_len(len, buf.len()) } else { Err(errno) } @@ -592,10 +623,10 @@ pub fn getsockname(fd: u32, buf: &mut [u8]) -> Result { /// Writes the serialized address into `buf` and returns the number of bytes written. /// Returns `Ok(len)` on success, `Err(errno)` on failure. pub fn getpeername(fd: u32, buf: &mut [u8]) -> Result { - let mut len: u32 = buf.len() as u32; + let mut len = checked_u32_len(buf.len())?; let errno = unsafe { net_getpeername(fd, buf.as_mut_ptr(), &mut len) }; if errno == ERRNO_SUCCESS { - Ok(len) + validate_returned_len(len, buf.len()) } else { Err(errno) } @@ -607,7 +638,8 @@ pub fn getpeername(fd: u32, buf: &mut [u8]) -> Result { /// After success, `send`/`recv` on this fd use the encrypted TLS stream. /// Returns `Ok(())` on success, `Err(errno)` on failure. pub fn tls_connect(fd: u32, hostname: &[u8]) -> Result<(), Errno> { - let errno = unsafe { net_tls_connect(fd, hostname.as_ptr(), hostname.len() as u32) }; + let hostname_len = checked_u32_len(hostname.len())?; + let errno = unsafe { net_tls_connect(fd, hostname.as_ptr(), hostname_len) }; if errno == ERRNO_SUCCESS { Ok(()) } else { @@ -621,6 +653,7 @@ pub fn tls_connect(fd: u32, hostname: &[u8]) -> Result<(), Errno> { /// `timeout_ms` is the timeout: 0=non-blocking, -1=block forever, >0=milliseconds. /// Returns `Ok(ready_count)` on success, `Err(errno)` on failure. pub fn poll(fds: &mut [u8], nfds: u32, timeout_ms: i32) -> Result { + validate_poll_buffer_len(fds.len(), nfds)?; let mut ready: u32 = 0; let errno = unsafe { net_poll(fds.as_mut_ptr(), nfds, timeout_ms, &mut ready) }; if errno == ERRNO_SUCCESS { @@ -635,7 +668,8 @@ pub fn poll(fds: &mut [u8], nfds: u32, timeout_ms: i32) -> Result { /// `addr` is a serialized address string (e.g. "host:port" or "/path/to/socket"). /// Returns `Ok(())` on success, `Err(errno)` on failure. pub fn bind(fd: u32, addr: &[u8]) -> Result<(), Errno> { - let errno = unsafe { net_bind(fd, addr.as_ptr(), addr.len() as u32) }; + let addr_len = checked_u32_len(addr.len())?; + let errno = unsafe { net_bind(fd, addr.as_ptr(), addr_len) }; if errno == ERRNO_SUCCESS { Ok(()) } else { @@ -663,10 +697,10 @@ pub fn listen(fd: u32, backlog: u32) -> Result<(), Errno> { /// Returns `Err(errno)` on failure. pub fn accept(fd: u32, addr_buf: &mut [u8]) -> Result<(u32, u32), Errno> { let mut new_fd: u32 = 0; - let mut addr_len: u32 = addr_buf.len() as u32; + let mut addr_len = checked_u32_len(addr_buf.len())?; let errno = unsafe { net_accept(fd, &mut new_fd, addr_buf.as_mut_ptr(), &mut addr_len) }; if errno == ERRNO_SUCCESS { - Ok((new_fd, addr_len)) + Ok((new_fd, validate_returned_len(addr_len, addr_buf.len())?)) } else { Err(errno) } @@ -677,20 +711,22 @@ pub fn accept(fd: u32, addr_buf: &mut [u8]) -> Result<(u32, u32), Errno> { /// `addr` is the destination address string (e.g. "host:port"). /// Returns `Ok(bytes_sent)` on success, `Err(errno)` on failure. pub fn sendto(fd: u32, buf: &[u8], flags: u32, addr: &[u8]) -> Result { + let buf_len = checked_u32_len(buf.len())?; + let addr_len = checked_u32_len(addr.len())?; let mut sent: u32 = 0; let errno = unsafe { net_sendto( fd, buf.as_ptr(), - buf.len() as u32, + buf_len, flags, addr.as_ptr(), - addr.len() as u32, + addr_len, &mut sent, ) }; if errno == ERRNO_SUCCESS { - Ok(sent) + validate_returned_len(sent, buf.len()) } else { Err(errno) } @@ -701,13 +737,14 @@ pub fn sendto(fd: u32, buf: &[u8], flags: u32, addr: &[u8]) -> Result Result<(u32, u32), Errno> { + let buf_len = checked_u32_len(buf.len())?; let mut received: u32 = 0; - let mut addr_len: u32 = addr_buf.len() as u32; + let mut addr_len = checked_u32_len(addr_buf.len())?; let errno = unsafe { net_recvfrom( fd, buf.as_mut_ptr(), - buf.len() as u32, + buf_len, flags, &mut received, addr_buf.as_mut_ptr(), @@ -715,7 +752,10 @@ pub fn recvfrom(fd: u32, buf: &mut [u8], flags: u32, addr_buf: &mut [u8]) -> Res ) }; if errno == ERRNO_SUCCESS { - Ok((received, addr_len)) + Ok(( + validate_returned_len(received, buf.len())?, + validate_returned_len(addr_len, addr_buf.len())?, + )) } else { Err(errno) } @@ -796,10 +836,29 @@ pub fn is_atty(fd: u32) -> Result { /// Returns `Ok(len)` on success, `Err(errno)` on failure. pub fn get_pwuid(uid: u32, buf: &mut [u8]) -> Result { let mut len: u32 = 0; - let errno = unsafe { getpwuid(uid, buf.as_mut_ptr(), buf.len() as u32, &mut len) }; + let buf_len = checked_u32_len(buf.len())?; + let errno = unsafe { getpwuid(uid, buf.as_mut_ptr(), buf_len, &mut len) }; if errno == ERRNO_SUCCESS { - Ok(len) + validate_returned_len(len, buf.len()) } else { Err(errno) } } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn poll_buffer_validation_requires_exact_pollfd_capacity() { + assert_eq!(validate_poll_buffer_len(POLLFD_BYTES, 1), Ok(())); + assert_eq!(validate_poll_buffer_len(POLLFD_BYTES - 1, 1), Err(ERRNO_INVAL)); + assert_eq!(validate_poll_buffer_len(POLLFD_BYTES + 1, 1), Err(ERRNO_INVAL)); + } + + #[test] + fn returned_lengths_must_fit_in_the_supplied_buffer() { + assert_eq!(validate_returned_len(4, 4), Ok(4)); + assert_eq!(validate_returned_len(5, 4), Err(ERRNO_INVAL)); + } +} diff --git a/registry/native/patches/wasi-libc/0007-getpwuid.patch b/registry/native/patches/wasi-libc/0007-getpwuid.patch index fbfd1340b..32e5febd4 100644 --- a/registry/native/patches/wasi-libc/0007-getpwuid.patch +++ b/registry/native/patches/wasi-libc/0007-getpwuid.patch @@ -48,6 +48,7 @@ index 0000000..c80a680 + &ret_len + ); + if (err != 0) return NULL; ++ if (ret_len >= sizeof(pw_buf)) return NULL; + + pw_buf[ret_len] = '\0'; + diff --git a/registry/native/stubs/uucore/src/lib/features/entries.rs b/registry/native/stubs/uucore/src/lib/features/entries.rs index 443d91a9d..65c5efdaf 100644 --- a/registry/native/stubs/uucore/src/lib/features/entries.rs +++ b/registry/native/stubs/uucore/src/lib/features/entries.rs @@ -312,7 +312,11 @@ mod wasi_impl { if errno != 0 || len == 0 { return None; } - let s = core::str::from_utf8(&buf[..len as usize]).ok()?; + let len = usize::try_from(len).ok()?; + if len > buf.len() { + return None; + } + let s = core::str::from_utf8(&buf[..len]).ok()?; let (name, passwd, pw_uid, pw_gid, gecos, home, shell) = parse_passwd_string(s)?; Some(Passwd { name, diff --git a/registry/tests/wasmvm/curl.test.ts b/registry/tests/wasmvm/curl.test.ts index f88814bef..eb532b77b 100644 --- a/registry/tests/wasmvm/curl.test.ts +++ b/registry/tests/wasmvm/curl.test.ts @@ -29,7 +29,11 @@ import { type ServerResponse, } from 'node:http'; import { createServer as createHttpsServer, type Server as HttpsServer } from 'node:https'; -import { createServer as createTcpServer, type Server as TcpServer } from 'node:net'; +import { + createConnection, + createServer as createTcpServer, + type Server as TcpServer, +} from 'node:net'; import { execSync } from 'node:child_process'; import { existsSync, unlinkSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; @@ -43,6 +47,14 @@ const hasHttpGetTest = hasWasmBinaries && existsSync(resolve(COMMANDS_DIR, 'http const hasPackagedCurl = existsSync(resolve(CURL_PACKAGE_DIR, 'curl')); const hasCurl = hasPackagedCurl || (hasWasmBinaries && existsSync(resolve(COMMANDS_DIR, 'curl'))); const runExternalNetwork = process.env.AGENTOS_E2E_NETWORK === '1'; +const EXTERNAL_HOST = 'example.com'; +const EXTERNAL_TCP_PORT = 80; +const EXTERNAL_HTTP_URL = `http://${EXTERNAL_HOST}/`; +const EXTERNAL_HTTPS_URL = `https://${EXTERNAL_HOST}/`; +const EXTERNAL_EXPECTED_BODY = 'Example Domain'; +const EXTERNAL_RETRY_ATTEMPTS = 3; +const EXTERNAL_RETRY_DELAY_MS = 1_000; +const EXTERNAL_PROBE_TIMEOUT_MS = 8_000; let hasOpenssl = false; try { @@ -52,6 +64,91 @@ try { hasOpenssl = false; } +function sleep(ms: number): Promise { + return new Promise((resolveSleep) => setTimeout(resolveSleep, ms)); +} + +function formatError(error: unknown): string { + if (error instanceof Error) return error.message; + return String(error); +} + +async function retryExternal(run: () => Promise, attempts = EXTERNAL_RETRY_ATTEMPTS): Promise { + let lastError: unknown; + for (let attempt = 1; attempt <= attempts; attempt += 1) { + try { + return await run(); + } catch (error) { + lastError = error; + if (attempt < attempts) { + await sleep(EXTERNAL_RETRY_DELAY_MS); + } + } + } + + throw lastError ?? new Error('external network probe failed'); +} + +async function probeExternalTcp(): Promise { + await new Promise((resolveConnect, rejectConnect) => { + const socket = createConnection({ + host: EXTERNAL_HOST, + port: EXTERNAL_TCP_PORT, + }); + let settled = false; + + const finish = (callback: () => void) => { + if (settled) return; + settled = true; + callback(); + }; + + socket.setTimeout(EXTERNAL_PROBE_TIMEOUT_MS); + socket.once('connect', () => { + finish(() => { + socket.end(); + resolveConnect(); + }); + }); + socket.once('timeout', () => { + finish(() => { + socket.destroy(); + rejectConnect(new Error(`timed out connecting to ${EXTERNAL_HOST}:${EXTERNAL_TCP_PORT}`)); + }); + }); + socket.once('error', (error) => { + finish(() => { + socket.destroy(); + rejectConnect(error); + }); + }); + }); +} + +async function probeExternalHttps(): Promise { + const response = await fetch(EXTERNAL_HTTPS_URL, { + signal: AbortSignal.timeout(EXTERNAL_PROBE_TIMEOUT_MS), + }); + if (!response.ok) { + throw new Error(`host probe failed with HTTP ${response.status}`); + } + await response.arrayBuffer(); +} + +const externalNetworkSkipReason = runExternalNetwork + ? await (async () => { + try { + await retryExternal(async () => { + await probeExternalTcp(); + await probeExternalHttps(); + }); + return false as const; + } catch (error) { + return `external network unavailable: ${formatError(error)}`; + } + })() + : 'set AGENTOS_E2E_NETWORK=1 to enable external-network coverage'; + function generateSelfSignedCert(): { key: string; cert: string } { const keyPath = join(tmpdir(), `curl-test-key-${process.pid}-${Date.now()}.pem`); try { @@ -327,6 +424,19 @@ describe.skipIf(!hasCurl && !hasHttpGetTest)('curl and socket layer', () => { return kernel; } + async function execWithRetry(command: string) { + let lastResult: Awaited> | undefined; + for (let attempt = 1; attempt <= EXTERNAL_RETRY_ATTEMPTS; attempt += 1) { + lastResult = await kernel.exec(command); + if (lastResult.exitCode === 0) return lastResult; + if (attempt < EXTERNAL_RETRY_ATTEMPTS) { + await sleep(EXTERNAL_RETRY_DELAY_MS); + } + } + + return lastResult!; + } + afterEach(async () => { await kernel?.dispose(); }); @@ -582,17 +692,28 @@ describe.skipIf(!hasCurl && !hasHttpGetTest)('curl and socket layer', () => { expect(Date.now() - startedAt).toBeLessThan(8000); }, 15000); - it.skipIf(!hasCurl || !runExternalNetwork)('curl reaches httpbin over real external HTTP', async () => { + it.skipIf(!hasHttpGetTest || externalNetworkSkipReason)('http_get_test reaches an external host over real TCP', async () => { + await createKernelWithNet(); + const result = await execWithRetry(`http_get_test ${EXTERNAL_HOST} ${EXTERNAL_TCP_PORT} /`); + expect(result.exitCode).toBe(0); + expect(result.stdout).toMatch(/HTTP\/1\.[01] (200|301|302)/); + }, 30000); + + it.skipIf(!hasCurl || externalNetworkSkipReason)('curl reaches a real external HTTP endpoint', async () => { await createKernelWithNet(); - const result = await kernel.exec('curl -sS --max-time 20 http://httpbin.org/get'); + const result = await execWithRetry( + `curl -fsSL --retry 2 --retry-delay 1 --retry-all-errors --connect-timeout 10 --max-time 30 ${EXTERNAL_HTTP_URL}`, + ); expect(result.exitCode).toBe(0); - expect(result.stdout).toContain('"url": "http://httpbin.org/get"'); + expect(result.stdout).toContain(EXTERNAL_EXPECTED_BODY); }, 30000); - it.skipIf(!hasCurl || !runExternalNetwork)('curl reaches httpbin over real external HTTPS', async () => { + it.skipIf(!hasCurl || externalNetworkSkipReason)('curl reaches a real external HTTPS endpoint', async () => { await createKernelWithNet(); - const result = await kernel.exec('curl -sS --max-time 20 https://httpbin.org/get'); + const result = await execWithRetry( + `curl -fsSL --retry 2 --retry-delay 1 --retry-all-errors --connect-timeout 10 --max-time 30 ${EXTERNAL_HTTPS_URL}`, + ); expect(result.exitCode).toBe(0); - expect(result.stdout).toContain('"url": "https://httpbin.org/get"'); + expect(result.stdout).toContain(EXTERNAL_EXPECTED_BODY); }, 30000); }); diff --git a/scripts/ralph/AGENTS.md b/scripts/ralph/AGENTS.md new file mode 120000 index 000000000..681311eb9 --- /dev/null +++ b/scripts/ralph/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file diff --git a/scripts/ralph/archive/2026-04-04-04-01-feat_rust_kernel_sidecar/prd.json b/scripts/ralph/archive/2026-04-04-04-01-feat_rust_kernel_sidecar/prd.json new file mode 100644 index 000000000..d0cffc2af --- /dev/null +++ b/scripts/ralph/archive/2026-04-04-04-01-feat_rust_kernel_sidecar/prd.json @@ -0,0 +1,694 @@ +{ + "project": "agentOS", + "branchName": "ralph/runtime-isolation-hardening", + "description": "Port the original JS kernel's proven isolation model to the Rust sidecar — kernel-backed polyfills for all Node.js builtins, virtualized process global, Pyodide sandbox hardening, and defense-in-depth resource limits", + "userStories": [ + { + "id": "US-001", + "title": "Remove dangerous builtins from DEFAULT_ALLOWED_NODE_BUILTINS", + "description": "As a security engineer, I want builtins without kernel-backed polyfills removed from the allow list so that guest code cannot fall through to real host modules", + "acceptanceCriteria": [ + "DEFAULT_ALLOWED_NODE_BUILTINS in native-kernel-proxy.ts only includes builtins with kernel-backed polyfills (fs, path, url, child_process, stream, events, buffer, crypto, util, zlib, string_decoder, querystring, assert, timers, console)", + "dgram, dns, http, http2, https, net, tls, vm, worker_threads, inspector, v8 are removed from DEFAULT_ALLOWED_NODE_BUILTINS", + "os, cluster, diagnostics_channel, module, trace_events are added to DENIED_BUILTINS in node_import_cache.rs", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 1, + "passes": false, + "notes": "Zero-effort highest-value security fix. Every builtin without a polyfill currently falls through to real host module via nextResolve()." + }, + { + "id": "US-002", + "title": "Block Pyodide import js FFI sandbox escape", + "description": "As a security engineer, I want Python code blocked from accessing JS globals via import js so that Pyodide cannot escape its sandbox", + "acceptanceCriteria": [ + "Python code doing `import js; js.process.env` raises an error or returns a safe proxy", + "Python code doing `import pyodide_js` is similarly blocked or proxied", + "js.require, js.process.kill, js.process.exit are not accessible from Python", + "Existing Python execution tests pass", + "Typecheck passes" + ], + "priority": 2, + "passes": false, + "notes": "CRITICAL: import js exposes all JS globals including process.env, process.kill(), require. Full sandbox escape." + }, + { + "id": "US-003", + "title": "Enable Node.js --permission flag for Pyodide host process", + "description": "As a security engineer, I want the --permission flag applied to the Pyodide host Node.js process so that OS-level backstop protections are active", + "acceptanceCriteria": [ + "python.rs no longer sets enable_permissions=false (line ~622)", + "--permission flag is applied to the Pyodide host process with appropriate --allow-fs-read/--allow-fs-write scoped to the sandbox root", + "Pyodide execution still functions correctly with permissions enabled", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 3, + "passes": false, + "notes": "Currently python.rs:622 explicitly disables --permission. This removes the defense-in-depth OS-level backstop." + }, + { + "id": "US-004", + "title": "Scrub AGENT_OS_* environment variables from guest process.env", + "description": "As a security engineer, I want internal AGENT_OS_* environment variables hidden from guest code so that host implementation details are not leaked", + "acceptanceCriteria": [ + "Guest code accessing process.env does not see any AGENT_OS_* keys", + "AGENT_OS_GUEST_PATH_MAPPINGS (which reveals real host paths) is not visible to guest", + "AGENT_OS_NODE_IMPORT_CACHE_PATH is not visible to guest", + "process.env is replaced with a proxy or filtered copy in the runner/loader setup", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 4, + "passes": false, + "notes": "process.env currently leaks all AGENT_OS_* internal control variables to guest code." + }, + { + "id": "US-005", + "title": "Virtualize process.cwd() to return kernel CWD", + "description": "As a security engineer, I want process.cwd() to return the kernel's virtual CWD instead of the real host path so that the host filesystem layout is hidden", + "acceptanceCriteria": [ + "process.cwd() returns the guest virtual path (e.g. /root) not the host path (e.g. /tmp/agent-os-xxx/workspace)", + "process.chdir() is intercepted and routed through the kernel or denied", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 5, + "passes": false, + "notes": "process.cwd() currently returns real host path like /tmp/agent-os-xxx/workspace." + }, + { + "id": "US-006", + "title": "Virtualize process.execPath, argv[0], pid, ppid, getuid, getgid", + "description": "As a security engineer, I want host-revealing process properties replaced with virtual values so that the guest cannot observe the host environment", + "acceptanceCriteria": [ + "process.execPath returns a virtual path (e.g. /usr/bin/node) not the real host binary path", + "process.argv[0] returns a virtual path", + "process.pid returns the kernel PID, not the real host OS PID", + "process.ppid returns the kernel parent PID, not the sidecar's PID", + "process.getuid() and process.getgid() return virtualized values (e.g. 0 for root)", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 6, + "passes": false, + "notes": "Multiple process properties leak real host state: execPath, argv[0], pid, ppid, getuid, getgid." + }, + { + "id": "US-007", + "title": "Intercept process signal handlers and deny native addon loading", + "description": "As a security engineer, I want guest signal handler registration intercepted and native addon loading denied so that the guest cannot interfere with process lifecycle or run arbitrary native code", + "acceptanceCriteria": [ + "process.on('SIGINT'/SIGTERM/etc) is intercepted — guest cannot prevent sidecar from terminating the process", + "process.dlopen() throws ERR_ACCESS_DENIED", + "Module._extensions['.node'] throws ERR_ACCESS_DENIED when attempting to load .node files", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 7, + "passes": false, + "notes": "Guest can register signal handlers that prevent clean termination. Native addons (.node files) are arbitrary native code on the host." + }, + { + "id": "US-008", + "title": "Fix exec/execSync bypass in wrapChildProcessModule", + "description": "As a security engineer, I want exec and execSync intercepted with the same protections as spawn/execFile so that shell commands cannot bypass path translation and permission checks", + "acceptanceCriteria": [ + "child_process.exec() applies path translation and --permission injection", + "child_process.execSync() applies path translation and --permission injection", + "Guest code calling execSync('cat /etc/passwd') does NOT read the real host /etc/passwd", + "Existing child_process tests pass", + "Typecheck passes" + ], + "priority": 8, + "passes": false, + "notes": "exec/execSync are currently passed through as bare .bind() calls with ZERO interception. Guest can run arbitrary host commands." + }, + { + "id": "US-009", + "title": "Translate host paths in require.resolve() and error messages", + "description": "As a security engineer, I want host filesystem paths scrubbed from require.resolve() results and error messages so that the host layout is not revealed to guest code", + "acceptanceCriteria": [ + "require.resolve() returns guest-visible paths, not real host paths like /tmp/agent-os-node-import-cache-1/...", + "Module-not-found error messages have host paths translated to guest-visible paths", + "Loader error stack traces have host paths translated", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 9, + "passes": false, + "notes": "require.resolve() and error messages currently expose real host filesystem paths." + }, + { + "id": "US-010", + "title": "Replace in-band control message parsing with side channel", + "description": "As a security engineer, I want all control messages (exit codes, metrics, signal state) moved to a dedicated side channel so that guest code cannot inject fake control messages via stdout/stderr", + "acceptanceCriteria": [ + "__AGENT_OS_PYTHON_EXIT__ parsing removed from stderr — exit detection uses a dedicated mechanism", + "__AGENT_OS_SIGNAL_STATE__ parsing removed from stderr", + "__AGENT_OS_NODE_IMPORT_CACHE_METRICS__ parsing removed from stderr", + "Control data flows through a dedicated pipe/fd or separate IPC channel", + "Guest code writing these prefixes to stderr has no effect on sidecar state", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 10, + "passes": false, + "notes": "Guest code can write magic prefixes to stderr to inject fake control messages. Affects Python exit detection, signal state, and import cache metrics." + }, + { + "id": "US-011", + "title": "Make ALLOWED_NODE_BUILTINS configurable from AgentOsOptions", + "description": "As a developer, I want to configure which Node.js builtins are allowed per-VM so that different VMs can have different isolation profiles", + "acceptanceCriteria": [ + "AgentOsOptions accepts an optional allowedNodeBuiltins field", + "The field flows through to the sidecar bridge and overrides DEFAULT_ALLOWED_NODE_BUILTINS", + "When not specified, uses the hardened default from US-001", + "Fix --allow-worker inconsistency: only pass --allow-worker when worker_threads is in the allowed list", + "Typecheck passes" + ], + "priority": 11, + "passes": false, + "notes": "Currently hardcoded. Different use cases need different builtin profiles." + }, + { + "id": "US-012", + "title": "Build SharedArrayBuffer RPC bridge for synchronous kernel syscalls", + "description": "As a developer, I want a SharedArrayBuffer + Atomics.wait RPC bridge between guest Node.js processes and the Rust sidecar so that synchronous polyfill methods (readFileSync, etc.) can call the kernel", + "acceptanceCriteria": [ + "SharedArrayBuffer-based sync RPC channel established between guest process and sidecar", + "Guest-side bridge exposes callSync(method, args) that blocks via Atomics.wait until sidecar responds", + "Sidecar-side bridge reads requests, dispatches to kernel, writes responses, and notifies via Atomics.notify", + "Round-trip latency is under 1ms for simple operations (e.g. stat)", + "Bridge handles serialization of paths, buffers, and error codes", + "Pattern matches the proven Pyodide VFS bridge implementation", + "Typecheck passes" + ], + "priority": 12, + "passes": false, + "notes": "Foundation for all sync polyfills. Same pattern as existing Pyodide VFS bridge. Original JS kernel used this for fs, net, etc." + }, + { + "id": "US-013", + "title": "Port os module polyfill with kernel-provided values", + "description": "As a developer, I want the os module to return kernel-provided values instead of real host information so that the guest sees the virtual OS environment", + "acceptanceCriteria": [ + "os.hostname() returns the kernel hostname (e.g. agent-os), not the real host hostname", + "os.cpus() returns configured virtual CPU info, not real host CPUs", + "os.totalmem()/os.freemem() return configured virtual memory values", + "os.networkInterfaces() returns virtual network interfaces, not real host interfaces", + "os.homedir() returns the kernel home directory", + "os.userInfo() returns virtual user info", + "os.platform()/os.type()/os.release() return linux values", + "os module is added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 13, + "passes": false, + "notes": "Simple polyfill (~100 lines). os module currently leaks real host info (hostname, CPUs, memory, network interfaces)." + }, + { + "id": "US-014", + "title": "Port fs.promises async methods through kernel VFS RPC", + "description": "As a developer, I want fs.promises methods to route through the kernel VFS via async IPC so that async filesystem operations are fully virtualized", + "acceptanceCriteria": [ + "fs.promises.readFile routes through kernel VFS, not real node:fs", + "fs.promises.writeFile routes through kernel VFS", + "fs.promises.stat, lstat, readdir, mkdir, rmdir, unlink, rename, copyFile, chmod, chown, utimes route through kernel VFS", + "fs.promises.access routes through kernel VFS with permission checks", + "Path arguments are translated from guest paths to kernel VFS paths", + "Error codes match POSIX (ENOENT, EACCES, EEXIST, etc.)", + "Typecheck passes" + ], + "priority": 14, + "passes": false, + "notes": "~20 async methods with direct kernel VFS counterparts. Uses async IPC messages to sidecar." + }, + { + "id": "US-015", + "title": "Port fs sync methods through SharedArrayBuffer bridge", + "description": "As a developer, I want synchronous fs methods (readFileSync, writeFileSync, etc.) to route through the kernel VFS via the SharedArrayBuffer sync RPC bridge", + "acceptanceCriteria": [ + "fs.readFileSync routes through kernel VFS via sync RPC, not real node:fs", + "fs.writeFileSync routes through kernel VFS via sync RPC", + "fs.statSync, lstatSync, readdirSync, mkdirSync, rmdirSync, unlinkSync, renameSync route through kernel VFS", + "fs.existsSync routes through kernel VFS", + "fs.readlinkSync, symlinkSync, linkSync route through kernel VFS", + "Sync methods block correctly via Atomics.wait until kernel responds", + "Typecheck passes" + ], + "priority": 15, + "passes": false, + "notes": "Depends on US-012 (SharedArrayBuffer RPC bridge). Sync methods use Atomics.wait to block until kernel responds." + }, + { + "id": "US-016", + "title": "Port fs fd-based operations and streams through kernel VFS", + "description": "As a developer, I want fd-based fs operations and streams to route through the kernel VFS so that all file I/O is fully virtualized", + "acceptanceCriteria": [ + "fs.open/fs.openSync return kernel-managed file descriptors", + "fs.read/fs.readSync on opened fds route through kernel fd_read", + "fs.write/fs.writeSync on opened fds route through kernel fd_write", + "fs.close/fs.closeSync route through kernel fd_close", + "fs.fstat/fs.fstatSync route through kernel fd_stat", + "fs.createReadStream returns a readable stream backed by kernel fd operations", + "fs.createWriteStream returns a writable stream backed by kernel fd operations", + "fs.watch/fs.watchFile are stubbed (kernel has no file-watching API) with clear error message", + "Typecheck passes" + ], + "priority": 16, + "passes": false, + "notes": "Depends on US-012. Fd-based ops map to kernel fd_open/fd_read/fd_write/fd_close. Streams built on top of polyfilled fd ops." + }, + { + "id": "US-017", + "title": "Port child_process polyfill through kernel process table", + "description": "As a developer, I want child_process.spawn/exec/execFile to route through the kernel process table so that child processes are fully virtualized", + "acceptanceCriteria": [ + "child_process.spawn routes through kernel.spawn_process(), not real host child_process", + "child_process.execFile routes through kernel process table", + "child_process.exec routes through kernel process table", + "child_process.execSync routes through kernel process table via sync RPC", + "Returned ChildProcess object is a synthetic EventEmitter backed by kernel pipe fds for stdio", + "Exit/close events are wired through kernel waitpid", + ".kill() method routes through kernel kill_process", + "Replace wrapChildProcessModule() entirely — no more path-translating wrapper over real child_process", + "Typecheck passes" + ], + "priority": 17, + "passes": false, + "notes": "Depends on US-012. Replace the current path-translating wrapper with a full kernel-backed polyfill." + }, + { + "id": "US-018", + "title": "Port net.Socket polyfill via kernel socket table", + "description": "As a developer, I want net.Socket to be a Duplex stream backed by the kernel socket table so that TCP connections are fully virtualized", + "acceptanceCriteria": [ + "net.Socket is a Duplex stream backed by kernel socket table operations via RPC", + "net.connect/net.createConnection create kernel-managed sockets", + "Socket.write sends data through kernel socket send", + "Socket data event fires from kernel socket recv", + "Socket connect/close/error events work correctly", + "Loopback connections stay entirely in-kernel", + "External connections route through HostNetworkAdapter", + "net module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 18, + "passes": false, + "notes": "Depends on US-012. Kernel already has socket table + HostNetworkAdapter. Original JS kernel had kernel.socketTable.create/connect/send/recv." + }, + { + "id": "US-019", + "title": "Port net.createServer polyfill via kernel socket listen/accept", + "description": "As a developer, I want net.createServer to create servers backed by the kernel socket table so that TCP servers are fully virtualized", + "acceptanceCriteria": [ + "net.createServer returns a server backed by kernel socket listen/accept", + "Server.listen binds to a kernel-managed socket", + "Incoming connections fire connection event with kernel-backed net.Socket instances", + "Server.close properly tears down kernel socket", + "Server.address() returns the bound address from kernel", + "Typecheck passes" + ], + "priority": 19, + "passes": false, + "notes": "Depends on US-018 (net.Socket polyfill)." + }, + { + "id": "US-020", + "title": "Port dgram polyfill via kernel socket table", + "description": "As a developer, I want dgram.createSocket to be backed by the kernel socket table so that UDP is fully virtualized", + "acceptanceCriteria": [ + "dgram.createSocket('udp4'/'udp6') creates a kernel-managed UDP socket", + "socket.send routes through kernel socket send", + "socket.on('message') fires from kernel socket recv", + "socket.bind routes through kernel socket bind", + "socket.close properly tears down kernel socket", + "dgram module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 20, + "passes": false, + "notes": "Depends on US-012. Similar pattern to net.Socket polyfill but for UDP." + }, + { + "id": "US-021", + "title": "Port dns polyfill via kernel DNS resolver", + "description": "As a developer, I want dns.resolve and dns.lookup to route through the kernel DNS resolver so that name resolution is fully virtualized", + "acceptanceCriteria": [ + "dns.lookup routes through kernel DNS resolver, not libuv getaddrinfo", + "dns.resolve/dns.resolve4/dns.resolve6 route through kernel DNS resolver", + "dns.promises.lookup and dns.promises.resolve work correctly", + "DNS results match what the kernel's resolver returns", + "dns module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 21, + "passes": false, + "notes": "dns.lookup uses libuv getaddrinfo internally, not node:net — needs its own interception." + }, + { + "id": "US-022", + "title": "Port tls polyfill via kernel networking", + "description": "As a developer, I want TLS socket creation to route through kernel networking so that encrypted connections are fully virtualized", + "acceptanceCriteria": [ + "tls.connect creates a TLS socket backed by kernel networking", + "tls.createServer creates a TLS server backed by kernel networking", + "TLS handshake and data transfer work correctly through kernel", + "tls module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 22, + "passes": false, + "notes": "Depends on US-018 (net.Socket polyfill). TLS wraps the underlying TCP socket." + }, + { + "id": "US-023", + "title": "Port http/https/http2 on top of polyfilled net and tls", + "description": "As a developer, I want http/https/http2 modules to work through polyfilled networking so that HTTP is fully virtualized", + "acceptanceCriteria": [ + "Investigate whether real node:http uses the polyfilled net module when loader hooks intercept require('net') inside http internals", + "If yes: verify http.request, http.get, http.createServer work correctly on top of polyfilled net", + "If no: implement http.request/http.get as kernel-level fetch-style RPC calls", + "https works on top of polyfilled tls", + "http, https, http2 modules added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 23, + "passes": false, + "notes": "Depends on US-018 (net), US-022 (tls). May work automatically if Node.js internal require('net') is intercepted by loader hooks." + }, + { + "id": "US-024", + "title": "Add Drop impl, timeout, and kill for PythonExecution", + "description": "As a developer, I want PythonExecution to clean up properly on drop and support timeouts so that orphaned Pyodide processes don't leak", + "acceptanceCriteria": [ + "PythonExecution implements Drop that kills the child process if still running", + "wait() accepts an optional timeout parameter", + "A cancel()/kill() method exists for in-flight Python executions", + "Orphaned processes (~200MB+ each) are reliably cleaned up", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 24, + "passes": false, + "notes": "Currently no Drop impl. Orphaned Node+Pyodide processes leak ~200MB+ each." + }, + { + "id": "US-025", + "title": "Add Python spawn_waiter thread and bounded stdout/stderr buffering", + "description": "As a developer, I want Python execution to use a dedicated waiter thread and bounded output buffers so that exit detection is reliable and large output doesn't cause OOM", + "acceptanceCriteria": [ + "Dedicated spawn_waiter thread for exit detection (matching JS/WASM pattern), replacing fragile stderr parsing + try_wait polling", + "stdout/stderr buffers capped at a configurable max size", + "OOM is prevented on large Python output", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 25, + "passes": false, + "notes": "Exit detection currently relies on fragile stderr magic prefix parsing. All output accumulated in memory with no cap." + }, + { + "id": "US-026", + "title": "Add VFS RPC path validation and sync bridge timeout", + "description": "As a security engineer, I want VFS RPC operations scoped to the guest CWD and sync bridge calls to have timeouts so that Pyodide cannot access arbitrary kernel paths or hang forever", + "acceptanceCriteria": [ + "VFS RPC operations in service.rs validate that request.path is within the guest's permitted scope", + "Kernel permission checks are applied to VFS RPC paths", + "Synchronous VFS RPC bridge calls have a configurable timeout (default 30s)", + "Timeout produces a clear error, not a hang", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 26, + "passes": false, + "notes": "service.rs:2394-2470 passes request.path directly to kernel with no validation. readSync blocks forever if Rust never responds." + }, + { + "id": "US-027", + "title": "Wire options.permissions through to sidecar bridge", + "description": "As a developer, I want AgentOsOptions.permissions to actually control kernel permission policy so that the declared permission model is enforced", + "acceptanceCriteria": [ + "AgentOsOptions.permissions is serialized and sent to the sidecar bridge", + "Sidecar applies the permission policy to kernel operations", + "LocalBridge no longer defaults to allowAll", + "When permissions restrict fs access, guest fs operations are denied appropriately", + "When permissions restrict network, guest network operations are denied", + "Typecheck passes" + ], + "priority": 27, + "passes": false, + "notes": "permissions field is accepted but never consumed. LocalBridge allows everything. PermissionDescriptor exists on Rust side but TS always sends empty array." + }, + { + "id": "US-028", + "title": "Validate CWD within sandbox root", + "description": "As a security engineer, I want the execution CWD validated against the sandbox root so that setting cwd=/ cannot grant host-wide filesystem access", + "acceptanceCriteria": [ + "service.rs validates that the Execute request's cwd is within the configured sandbox root", + "Setting cwd=/ is rejected with a clear error", + "cwd is not directly used as real host current_dir without validation", + "--allow-fs-read/--allow-fs-write are scoped to sandbox root, not the raw cwd", + "Typecheck passes" + ], + "priority": 28, + "passes": false, + "notes": "service.rs:2195-2206 uses cwd directly as real host current_dir AND adds it to --allow-fs-read/--allow-fs-write. No validation." + }, + { + "id": "US-029", + "title": "Per-VM import cache paths to prevent cross-VM poisoning", + "description": "As a security engineer, I want each VM to use isolated import cache paths so that one VM cannot poison another VM's module resolution", + "acceptanceCriteria": [ + "Each VM instance gets a unique import cache directory", + "flushCacheState does not merge shared on-disk cache across VMs", + "A poisoned resolution entry in VM-A's cache cannot affect VM-B", + "Cache cleanup happens on VM shutdown", + "Typecheck passes" + ], + "priority": 29, + "passes": false, + "notes": "flushCacheState reads/merges/writes a shared cache. Two VMs sharing the same cache root enables cross-VM cache poisoning." + }, + { + "id": "US-030", + "title": "Fix --allow-child-process unconditional escalation", + "description": "As a security engineer, I want --allow-child-process and --allow-worker only passed to child Node processes when the parent was explicitly granted those permissions", + "acceptanceCriteria": [ + "prependNodePermissionArgs checks parent process permissions before adding --allow-child-process", + "prependNodePermissionArgs checks parent process permissions before adding --allow-worker", + "A guest process without child_process permission cannot spawn children that have it", + "Recursive escalation chain is broken", + "Typecheck passes" + ], + "priority": 30, + "passes": false, + "notes": "Currently --allow-child-process and --allow-worker are passed unconditionally to all child Node processes." + }, + { + "id": "US-031", + "title": "Resolve symlinks before permission checks and fix link/exists gaps", + "description": "As a security engineer, I want permission checks to use resolved paths so that symlinks cannot bypass access control", + "acceptanceCriteria": [ + "PermissionedFileSystem resolves symlinks before checking permissions", + "link() checks permissions on both source and destination paths", + "Symlinks are prevented from targeting paths across mount boundaries", + "exists() returns false on EACCES instead of leaking file existence", + "Typecheck passes" + ], + "priority": 31, + "passes": false, + "notes": "permissions.rs checks caller-supplied path, then inner fs resolves symlinks independently. TOCTOU bypass if mounts expose host paths." + }, + { + "id": "US-032", + "title": "Fix host PID reuse in signal_runtime_process and dup2 bounds", + "description": "As a security engineer, I want process signaling to verify child liveness and fd operations to validate bounds so that PID reuse and fd overflow are prevented", + "acceptanceCriteria": [ + "signal_runtime_process checks child liveness before sending kill(2)", + "Allowed signals whitelisted to SIGTERM, SIGKILL, SIGINT, SIGCONT, signal-0", + "dup2 validates new_fd < MAX_FDS_PER_PROCESS before proceeding", + "open_with validates fd bounds", + "PTY foreground PGID changes validate target PGID belongs to same session", + "Typecheck passes" + ], + "priority": 32, + "passes": false, + "notes": "Sidecar sends real kill(2) to host PIDs. PID reuse could kill wrong host process. dup2 skips fd bounds check." + }, + { + "id": "US-033", + "title": "Add filesystem size and inode limits to ResourceLimits", + "description": "As a security engineer, I want configurable filesystem size and inode count limits so that guest code cannot write to OOM", + "acceptanceCriteria": [ + "max_filesystem_bytes added to ResourceLimits with configurable default", + "max_inode_count added to ResourceLimits with configurable default", + "Write operations check total filesystem size before proceeding", + "File/directory creation checks inode count before proceeding", + "truncate and pwrite validate against size limits before resizing (prevents OOM)", + "Exceeding limits returns ENOSPC", + "Typecheck passes" + ], + "priority": 33, + "passes": false, + "notes": "All file data is in-memory with no cap. Guest can write until host OOM. truncate/pwrite with large values cause immediate OOM." + }, + { + "id": "US-034", + "title": "Add WASM fuel/memory limits and socket/connection limits", + "description": "As a security engineer, I want WASM execution and network resource limits so that guest code cannot exhaust compute or connection resources", + "acceptanceCriteria": [ + "WASM execution fuel limits are configurable and enforced", + "WASM memory growth caps are configurable and enforced", + "WASM stack size is bounded", + "Socket count limit added to ResourceLimits", + "Connection count limit added to ResourceLimits", + "Pipe/PTY read operations have configurable timeout (no infinite blocking on leaked write end)", + "read_frame checks declared_len against max_frame_bytes before allocation (prevents OOM)", + "Typecheck passes" + ], + "priority": 34, + "passes": false, + "notes": "No WASM fuel/memory/stack limits. No socket/connection limits. pipe.read/pty.read block forever if write end leaks." + }, + { + "id": "US-035", + "title": "Fix Pyodide hardening order and VFS RPC queue bounds", + "description": "As a security engineer, I want Pyodide hardening applied before loadPyodide and VFS RPC queue bounded so that cached API references and unbounded queues cannot be exploited", + "acceptanceCriteria": [ + "Hardening code (global restrictions, API removals) runs BEFORE loadPyodide()", + "Pyodide cannot cache references to dangerous APIs before hardening", + "VFS RPC request queue has a configurable bound (e.g. 1000 pending requests)", + "Exceeding queue bound returns an error, not silent accumulation", + "Typecheck passes" + ], + "priority": 35, + "passes": false, + "notes": "Hardening currently runs AFTER loadPyodide. VFS RPC queue is unbounded." + }, + { + "id": "US-036", + "title": "Add missing Pyodide integration tests", + "description": "As a developer, I want comprehensive Pyodide tests so that isolation guarantees are verified by the test suite", + "acceptanceCriteria": [ + "Test frozen time — Python sees deterministic/controlled time", + "Test node:child_process and node:vm are inaccessible from Python", + "Test zero network requests during Pyodide init", + "Test kill (SIGTERM) terminates Python execution", + "Test concurrent Python executions don't interfere", + "Test cross-runtime file visibility (Python can see files written by JS and vice versa)", + "All new tests pass", + "Typecheck passes" + ], + "priority": 36, + "passes": false, + "notes": "Multiple Pyodide Phase 1/3 acceptance criteria have no test coverage." + }, + { + "id": "US-037", + "title": "Add security audit logging", + "description": "As a security engineer, I want structured logging for security-relevant events so that breaches and policy violations are observable", + "acceptanceCriteria": [ + "Auth failures are logged with structured data (timestamp, source, reason)", + "Permission denials are logged (path, operation, policy)", + "Mount/unmount operations are logged", + "Process kill operations are logged (source PID, target PID, signal)", + "Logs use structured format (JSON or similar) suitable for aggregation", + "Typecheck passes" + ], + "priority": 37, + "passes": false, + "notes": "No security event logging exists. Auth failures, permission denials, mounts, kills are all silent." + }, + { + "id": "US-038", + "title": "Fix plugin SSRF and add mount permission checks", + "description": "As a security engineer, I want plugin URLs validated and mount operations permission-checked so that plugins cannot reach internal services and mounts cannot bypass access control", + "acceptanceCriteria": [ + "Google Drive plugin validates token_url and api_base_url against expected hosts", + "S3 plugin validates endpoint against private IP ranges (169.254.x.x, 10.x.x.x, etc.)", + "mount_filesystem in kernel.rs checks caller permissions, not just assert_not_terminated", + "Mounting at sensitive paths (/, /etc, /proc) requires elevated permission", + "Typecheck passes" + ], + "priority": 38, + "passes": false, + "notes": "Plugins accept arbitrary URLs. mount_filesystem only checks assert_not_terminated, no path or caller validation." + }, + { + "id": "US-039", + "title": "Fix host_dir TOCTOU, setpgid cross-driver, and mutex poison policy", + "description": "As a developer, I want kernel correctness issues fixed so that path resolution, process groups, and mutex handling are robust", + "acceptanceCriteria": [ + "host_dir mount uses O_NOFOLLOW/openat-style resolution to prevent symlink TOCTOU", + "setpgid validates that target PGID's owning driver matches requester", + "Single mutex poison policy applied consistently (lock_or_recover everywhere OR .expect everywhere)", + "Typecheck passes" + ], + "priority": 39, + "passes": false, + "notes": "fs::canonicalize + ensure_within_root has TOCTOU race. setpgid allows cross-driver group joining. Inconsistent mutex handling." + }, + { + "id": "US-040", + "title": "Fix hardenProperty fallback and zombie reaper exit code handling", + "description": "As a developer, I want property hardening to throw on failure and zombie reaping to preserve exit codes so that security and correctness are maintained", + "acceptanceCriteria": [ + "hardenProperty throws instead of falling back to mutable assignment", + "Zombie reaper preserves exit codes for zombies with living parents that haven't called waitpid", + "Typecheck passes" + ], + "priority": 40, + "passes": false, + "notes": "hardenProperty silently falls back to mutable. Zombie reaper loses exit codes." + }, + { + "id": "US-041", + "title": "Enforce WASM permission tiers", + "description": "As a security engineer, I want WASM commands restricted based on their declared permission tier so that read-only commands cannot write files or spawn processes", + "acceptanceCriteria": [ + "WASI preopens restricted based on declared permission tier (read-only, read-write, full)", + "host_process imports only provided to full-tier commands", + "read-only tier commands cannot write files", + "read-write tier commands cannot spawn processes or make network requests", + "Typecheck passes" + ], + "priority": 41, + "passes": false, + "notes": "Permission tiers are declared in descriptors but not enforced at runtime." + }, + { + "id": "US-042", + "title": "Extract Pyodide embedded JS and deduplicate cross-runtime code", + "description": "As a developer, I want embedded JS extracted to files and shared code deduplicated so that the codebase is maintainable", + "acceptanceCriteria": [ + "~870 lines of embedded JS in python.rs extracted to a .js file loaded at build time", + "~300 lines of duplicated code across python.rs/wasm.rs/javascript.rs extracted to a shared module", + "NodeImportCache temp directories cleaned up on crash (add cleanup-on-startup logic)", + "Typecheck passes" + ], + "priority": 42, + "passes": false, + "notes": "Large embedded JS strings are hard to maintain. Significant duplication across runtime implementations." + }, + { + "id": "US-043", + "title": "Low-priority robustness fixes", + "description": "As a developer, I want minor correctness and safety issues fixed so that edge cases don't cause panics or undefined behavior", + "acceptanceCriteria": [ + "read_dir uses tree structure instead of linear scan for directory children lookup", + "collect_snapshot_entries uses iteration with depth limit instead of unbounded recursion", + "nlink uses saturating_sub to prevent underflow", + "allocate_fd uses bounded scan to prevent potential infinite loop", + "SQLite WASM VFS uses kernel random_get instead of deterministic randomness", + "WASM FFI poll buffer validation, getpwuid buffer trust, usize-to-u32 truncation checks added", + "Typecheck passes" + ], + "priority": 43, + "passes": false, + "notes": "Collection of minor issues that individually have low impact but collectively improve robustness." + } + ] +} diff --git a/scripts/ralph/prd.json b/scripts/ralph/prd.json new file mode 100644 index 000000000..2d9dd3843 --- /dev/null +++ b/scripts/ralph/prd.json @@ -0,0 +1,1293 @@ +{ + "project": "agentOS", + "branchName": "ralph/runtime-isolation-hardening", + "description": "Port the original JS kernel's proven isolation model to the Rust sidecar \u2014 kernel-backed polyfills for all Node.js builtins, virtualized process global, Pyodide sandbox hardening, and defense-in-depth resource limits", + "userStories": [ + { + "id": "US-001", + "title": "Remove dangerous builtins from DEFAULT_ALLOWED_NODE_BUILTINS", + "description": "As a security engineer, I want builtins without kernel-backed polyfills removed from the allow list so that guest code cannot fall through to real host modules", + "acceptanceCriteria": [ + "DEFAULT_ALLOWED_NODE_BUILTINS in native-kernel-proxy.ts only includes builtins with kernel-backed polyfills (fs, path, url, child_process, stream, events, buffer, crypto, util, zlib, string_decoder, querystring, assert, timers, console)", + "dgram, dns, http, http2, https, net, tls, vm, worker_threads, inspector, v8 are removed from DEFAULT_ALLOWED_NODE_BUILTINS", + "os, cluster, diagnostics_channel, module, trace_events are added to DENIED_BUILTINS in node_import_cache.rs", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 1, + "passes": true, + "notes": "Zero-effort highest-value security fix. Every builtin without a polyfill currently falls through to real host module via nextResolve()." + }, + { + "id": "US-002", + "title": "Block Pyodide import js FFI sandbox escape", + "description": "As a security engineer, I want Python code blocked from accessing JS globals via import js so that Pyodide cannot escape its sandbox", + "acceptanceCriteria": [ + "Python code doing `import js; js.process.env` raises an error or returns a safe proxy", + "Python code doing `import pyodide_js` is similarly blocked or proxied", + "js.require, js.process.kill, js.process.exit are not accessible from Python", + "Existing Python execution tests pass", + "Typecheck passes" + ], + "priority": 2, + "passes": true, + "notes": "CRITICAL: import js exposes all JS globals including process.env, process.kill(), require. Full sandbox escape." + }, + { + "id": "US-003", + "title": "Enable Node.js --permission flag for Pyodide host process", + "description": "As a security engineer, I want the --permission flag applied to the Pyodide host Node.js process so that OS-level backstop protections are active", + "acceptanceCriteria": [ + "python.rs no longer sets enable_permissions=false (line ~622)", + "--permission flag is applied to the Pyodide host process with appropriate --allow-fs-read/--allow-fs-write scoped to the sandbox root", + "Pyodide execution still functions correctly with permissions enabled", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 3, + "passes": true, + "notes": "Currently python.rs:622 explicitly disables --permission. This removes the defense-in-depth OS-level backstop." + }, + { + "id": "US-004", + "title": "Scrub AGENT_OS_* environment variables from guest process.env", + "description": "As a security engineer, I want internal AGENT_OS_* environment variables hidden from guest code so that host implementation details are not leaked", + "acceptanceCriteria": [ + "Guest code accessing process.env does not see any AGENT_OS_* keys", + "AGENT_OS_GUEST_PATH_MAPPINGS (which reveals real host paths) is not visible to guest", + "AGENT_OS_NODE_IMPORT_CACHE_PATH is not visible to guest", + "process.env is replaced with a proxy or filtered copy in the runner/loader setup", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 4, + "passes": true, + "notes": "process.env currently leaks all AGENT_OS_* internal control variables to guest code." + }, + { + "id": "US-005", + "title": "Virtualize process.cwd() to return kernel CWD", + "description": "As a security engineer, I want process.cwd() to return the kernel's virtual CWD instead of the real host path so that the host filesystem layout is hidden", + "acceptanceCriteria": [ + "process.cwd() returns the guest virtual path (e.g. /root) not the host path (e.g. /tmp/agent-os-xxx/workspace)", + "process.chdir() is intercepted and routed through the kernel or denied", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 5, + "passes": true, + "notes": "process.cwd() currently returns real host path like /tmp/agent-os-xxx/workspace." + }, + { + "id": "US-006", + "title": "Virtualize process.execPath, argv[0], pid, ppid, getuid, getgid", + "description": "As a security engineer, I want host-revealing process properties replaced with virtual values so that the guest cannot observe the host environment", + "acceptanceCriteria": [ + "process.execPath returns a virtual path (e.g. /usr/bin/node) not the real host binary path", + "process.argv[0] returns a virtual path", + "process.pid returns the kernel PID, not the real host OS PID", + "process.ppid returns the kernel parent PID, not the sidecar's PID", + "process.getuid() and process.getgid() return virtualized values (e.g. 0 for root)", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 6, + "passes": true, + "notes": "Multiple process properties leak real host state: execPath, argv[0], pid, ppid, getuid, getgid." + }, + { + "id": "US-007", + "title": "Intercept process signal handlers and deny native addon loading", + "description": "As a security engineer, I want guest signal handler registration intercepted and native addon loading denied so that the guest cannot interfere with process lifecycle or run arbitrary native code", + "acceptanceCriteria": [ + "process.on('SIGINT'/SIGTERM/etc) is intercepted \u2014 guest cannot prevent sidecar from terminating the process", + "process.dlopen() throws ERR_ACCESS_DENIED", + "Module._extensions['.node'] throws ERR_ACCESS_DENIED when attempting to load .node files", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 7, + "passes": true, + "notes": "Guest can register signal handlers that prevent clean termination. Native addons (.node files) are arbitrary native code on the host." + }, + { + "id": "US-008", + "title": "Fix exec/execSync bypass in wrapChildProcessModule", + "description": "As a security engineer, I want exec and execSync intercepted with the same protections as spawn/execFile so that shell commands cannot bypass path translation and permission checks", + "acceptanceCriteria": [ + "child_process.exec() applies path translation and --permission injection", + "child_process.execSync() applies path translation and --permission injection", + "Guest code calling execSync('cat /etc/passwd') does NOT read the real host /etc/passwd", + "Existing child_process tests pass", + "Typecheck passes" + ], + "priority": 8, + "passes": true, + "notes": "exec/execSync are currently passed through as bare .bind() calls with ZERO interception. Guest can run arbitrary host commands." + }, + { + "id": "US-009", + "title": "Translate host paths in require.resolve() and error messages", + "description": "As a security engineer, I want host filesystem paths scrubbed from require.resolve() results and error messages so that the host layout is not revealed to guest code", + "acceptanceCriteria": [ + "require.resolve() returns guest-visible paths, not real host paths like /tmp/agent-os-node-import-cache-1/...", + "Module-not-found error messages have host paths translated to guest-visible paths", + "Loader error stack traces have host paths translated", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 9, + "passes": true, + "notes": "require.resolve() and error messages currently expose real host filesystem paths." + }, + { + "id": "US-010", + "title": "Replace in-band control message parsing with side channel", + "description": "As a security engineer, I want all control messages (exit codes, metrics, signal state) moved to a dedicated side channel so that guest code cannot inject fake control messages via stdout/stderr", + "acceptanceCriteria": [ + "__AGENT_OS_PYTHON_EXIT__ parsing removed from stderr \u2014 exit detection uses a dedicated mechanism", + "__AGENT_OS_SIGNAL_STATE__ parsing removed from stderr", + "__AGENT_OS_NODE_IMPORT_CACHE_METRICS__ parsing removed from stderr", + "Control data flows through a dedicated pipe/fd or separate IPC channel", + "Guest code writing these prefixes to stderr has no effect on sidecar state", + "Existing tests pass", + "Typecheck passes" + ], + "priority": 10, + "passes": true, + "notes": "Guest code can write magic prefixes to stderr to inject fake control messages. Affects Python exit detection, signal state, and import cache metrics." + }, + { + "id": "US-011", + "title": "Make ALLOWED_NODE_BUILTINS configurable from AgentOsOptions", + "description": "As a developer, I want to configure which Node.js builtins are allowed per-VM so that different VMs can have different isolation profiles", + "acceptanceCriteria": [ + "AgentOsOptions accepts an optional allowedNodeBuiltins field", + "The field flows through to the sidecar bridge and overrides DEFAULT_ALLOWED_NODE_BUILTINS", + "When not specified, uses the hardened default from US-001", + "Fix --allow-worker inconsistency: only pass --allow-worker when worker_threads is in the allowed list", + "Typecheck passes" + ], + "priority": 11, + "passes": true, + "notes": "Currently hardcoded. Different use cases need different builtin profiles." + }, + { + "id": "US-012", + "title": "Build SharedArrayBuffer RPC bridge for synchronous kernel syscalls", + "description": "As a developer, I want a SharedArrayBuffer + Atomics.wait RPC bridge between guest Node.js processes and the Rust sidecar so that synchronous polyfill methods (readFileSync, etc.) can call the kernel", + "acceptanceCriteria": [ + "SharedArrayBuffer-based sync RPC channel established between guest process and sidecar", + "Guest-side bridge exposes callSync(method, args) that blocks via Atomics.wait until sidecar responds", + "Sidecar-side bridge reads requests, dispatches to kernel, writes responses, and notifies via Atomics.notify", + "Round-trip latency is under 1ms for simple operations (e.g. stat)", + "Bridge handles serialization of paths, buffers, and error codes", + "Pattern matches the proven Pyodide VFS bridge implementation", + "Typecheck passes" + ], + "priority": 12, + "passes": true, + "notes": "Foundation for all sync polyfills. Same pattern as existing Pyodide VFS bridge. Original JS kernel used this for fs, net, etc." + }, + { + "id": "US-013", + "title": "Port os module polyfill with kernel-provided values", + "description": "As a developer, I want the os module to return kernel-provided values instead of real host information so that the guest sees the virtual OS environment", + "acceptanceCriteria": [ + "os.hostname() returns the kernel hostname (e.g. agent-os), not the real host hostname", + "os.cpus() returns configured virtual CPU info, not real host CPUs", + "os.totalmem()/os.freemem() return configured virtual memory values", + "os.networkInterfaces() returns virtual network interfaces, not real host interfaces", + "os.homedir() returns the kernel home directory", + "os.userInfo() returns virtual user info", + "os.platform()/os.type()/os.release() return linux values", + "os module is added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 13, + "passes": true, + "notes": "Simple polyfill (~100 lines). os module currently leaks real host info (hostname, CPUs, memory, network interfaces)." + }, + { + "id": "US-014", + "title": "Port fs.promises async methods through kernel VFS RPC", + "description": "As a developer, I want fs.promises methods to route through the kernel VFS via async IPC so that async filesystem operations are fully virtualized", + "acceptanceCriteria": [ + "fs.promises.readFile routes through kernel VFS, not real node:fs", + "fs.promises.writeFile routes through kernel VFS", + "fs.promises.stat, lstat, readdir, mkdir, rmdir, unlink, rename, copyFile, chmod, chown, utimes route through kernel VFS", + "fs.promises.access routes through kernel VFS with permission checks", + "Path arguments are translated from guest paths to kernel VFS paths", + "Error codes match POSIX (ENOENT, EACCES, EEXIST, etc.)", + "Typecheck passes" + ], + "priority": 14, + "passes": true, + "notes": "~20 async methods with direct kernel VFS counterparts. Uses async IPC messages to sidecar." + }, + { + "id": "US-015", + "title": "Port fs sync methods through SharedArrayBuffer bridge", + "description": "As a developer, I want synchronous fs methods (readFileSync, writeFileSync, etc.) to route through the kernel VFS via the SharedArrayBuffer sync RPC bridge", + "acceptanceCriteria": [ + "fs.readFileSync routes through kernel VFS via sync RPC, not real node:fs", + "fs.writeFileSync routes through kernel VFS via sync RPC", + "fs.statSync, lstatSync, readdirSync, mkdirSync, rmdirSync, unlinkSync, renameSync route through kernel VFS", + "fs.existsSync routes through kernel VFS", + "fs.readlinkSync, symlinkSync, linkSync route through kernel VFS", + "Sync methods block correctly via Atomics.wait until kernel responds", + "Typecheck passes" + ], + "priority": 15, + "passes": true, + "notes": "Depends on US-012 (SharedArrayBuffer RPC bridge). Sync methods use Atomics.wait to block until kernel responds." + }, + { + "id": "US-016", + "title": "Port fs fd-based operations and streams through kernel VFS", + "description": "As a developer, I want fd-based fs operations and streams to route through the kernel VFS so that all file I/O is fully virtualized", + "acceptanceCriteria": [ + "fs.open/fs.openSync return kernel-managed file descriptors", + "fs.read/fs.readSync on opened fds route through kernel fd_read", + "fs.write/fs.writeSync on opened fds route through kernel fd_write", + "fs.close/fs.closeSync route through kernel fd_close", + "fs.fstat/fs.fstatSync route through kernel fd_stat", + "fs.createReadStream returns a readable stream backed by kernel fd operations", + "fs.createWriteStream returns a writable stream backed by kernel fd operations", + "fs.watch/fs.watchFile are stubbed (kernel has no file-watching API) with clear error message", + "Typecheck passes" + ], + "priority": 16, + "passes": true, + "notes": "Depends on US-012. Fd-based ops map to kernel fd_open/fd_read/fd_write/fd_close. Streams built on top of polyfilled fd ops." + }, + { + "id": "US-017", + "title": "Port child_process polyfill through kernel process table", + "description": "As a developer, I want child_process.spawn/exec/execFile to route through the kernel process table so that child processes are fully virtualized", + "acceptanceCriteria": [ + "child_process.spawn routes through kernel.spawn_process(), not real host child_process", + "child_process.execFile routes through kernel process table", + "child_process.exec routes through kernel process table", + "child_process.execSync routes through kernel process table via sync RPC", + "Returned ChildProcess object is a synthetic EventEmitter backed by kernel pipe fds for stdio", + "Exit/close events are wired through kernel waitpid", + ".kill() method routes through kernel kill_process", + "Replace wrapChildProcessModule() entirely \u2014 no more path-translating wrapper over real child_process", + "Typecheck passes" + ], + "priority": 17, + "passes": true, + "notes": "Depends on US-012. Replace the current path-translating wrapper with a full kernel-backed polyfill." + }, + { + "id": "US-018", + "title": "Port net.Socket polyfill via kernel socket table", + "description": "As a developer, I want net.Socket to be a Duplex stream backed by the kernel socket table so that TCP connections are fully virtualized", + "acceptanceCriteria": [ + "net.Socket is a Duplex stream backed by kernel socket table operations via RPC", + "net.connect/net.createConnection create kernel-managed sockets", + "Socket.write sends data through kernel socket send", + "Socket data event fires from kernel socket recv", + "Socket connect/close/error events work correctly", + "Loopback connections stay entirely in-kernel", + "External connections route through HostNetworkAdapter", + "net module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 18, + "passes": true, + "notes": "Depends on US-012. Kernel already has socket table + HostNetworkAdapter. Original JS kernel had kernel.socketTable.create/connect/send/recv." + }, + { + "id": "US-019", + "title": "Port net.createServer polyfill via kernel socket listen/accept", + "description": "As a developer, I want net.createServer to create servers backed by the kernel socket table so that TCP servers are fully virtualized", + "acceptanceCriteria": [ + "net.createServer returns a server backed by kernel socket listen/accept", + "Server.listen binds to a kernel-managed socket", + "Incoming connections fire connection event with kernel-backed net.Socket instances", + "Server.close properly tears down kernel socket", + "Server.address() returns the bound address from kernel", + "Typecheck passes" + ], + "priority": 19, + "passes": true, + "notes": "Depends on US-018 (net.Socket polyfill)." + }, + { + "id": "US-020", + "title": "Port dgram polyfill via kernel socket table", + "description": "As a developer, I want dgram.createSocket to be backed by the kernel socket table so that UDP is fully virtualized", + "acceptanceCriteria": [ + "dgram.createSocket('udp4'/'udp6') creates a kernel-managed UDP socket", + "socket.send routes through kernel socket send", + "socket.on('message') fires from kernel socket recv", + "socket.bind routes through kernel socket bind", + "socket.close properly tears down kernel socket", + "dgram module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 20, + "passes": true, + "notes": "Depends on US-012. Similar pattern to net.Socket polyfill but for UDP." + }, + { + "id": "US-021", + "title": "Port dns polyfill via kernel DNS resolver", + "description": "As a developer, I want dns.resolve and dns.lookup to route through the kernel DNS resolver so that name resolution is fully virtualized", + "acceptanceCriteria": [ + "dns.lookup routes through kernel DNS resolver, not libuv getaddrinfo", + "dns.resolve/dns.resolve4/dns.resolve6 route through kernel DNS resolver", + "dns.promises.lookup and dns.promises.resolve work correctly", + "DNS results match what the kernel's resolver returns", + "dns module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 21, + "passes": true, + "notes": "dns.lookup uses libuv getaddrinfo internally, not node:net \u2014 needs its own interception." + }, + { + "id": "US-022", + "title": "Port tls polyfill via kernel networking", + "description": "As a developer, I want TLS socket creation to route through kernel networking so that encrypted connections are fully virtualized", + "acceptanceCriteria": [ + "tls.connect creates a TLS socket backed by kernel networking", + "tls.createServer creates a TLS server backed by kernel networking", + "TLS handshake and data transfer work correctly through kernel", + "tls module added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 22, + "passes": true, + "notes": "Depends on US-018 (net.Socket polyfill). TLS wraps the underlying TCP socket." + }, + { + "id": "US-023", + "title": "Port http/https/http2 on top of polyfilled net and tls", + "description": "As a developer, I want http/https/http2 modules to work through polyfilled networking so that HTTP is fully virtualized", + "acceptanceCriteria": [ + "Investigate whether real node:http uses the polyfilled net module when loader hooks intercept require('net') inside http internals", + "If yes: verify http.request, http.get, http.createServer work correctly on top of polyfilled net", + "If no: implement http.request/http.get as kernel-level fetch-style RPC calls", + "https works on top of polyfilled tls", + "http, https, http2 modules added to BUILTIN_ASSETS and removed from DENIED_BUILTINS", + "Typecheck passes" + ], + "priority": 23, + "passes": true, + "notes": "Depends on US-018 (net), US-022 (tls). May work automatically if Node.js internal require('net') is intercepted by loader hooks." + }, + { + "id": "US-024", + "title": "Add Drop impl, timeout, and kill for PythonExecution", + "description": "As a developer, I want PythonExecution to clean up properly on drop and support timeouts so that orphaned Pyodide processes don't leak", + "acceptanceCriteria": [ + "PythonExecution implements Drop that kills the child process if still running", + "wait() accepts an optional timeout parameter", + "A cancel()/kill() method exists for in-flight Python executions", + "Orphaned processes (~200MB+ each) are reliably cleaned up", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 37, + "passes": true, + "notes": "Currently no Drop impl. Orphaned Node+Pyodide processes leak ~200MB+ each." + }, + { + "id": "US-025", + "title": "Add Python spawn_waiter thread and bounded stdout/stderr buffering", + "description": "As a developer, I want Python execution to use a dedicated waiter thread and bounded output buffers so that exit detection is reliable and large output doesn't cause OOM", + "acceptanceCriteria": [ + "Dedicated spawn_waiter thread for exit detection (matching JS/WASM pattern), replacing fragile stderr parsing + try_wait polling", + "stdout/stderr buffers capped at a configurable max size", + "OOM is prevented on large Python output", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 38, + "passes": true, + "notes": "Exit detection currently relies on fragile stderr magic prefix parsing. All output accumulated in memory with no cap." + }, + { + "id": "US-026", + "title": "Add VFS RPC path validation and sync bridge timeout", + "description": "As a security engineer, I want VFS RPC operations scoped to the guest CWD and sync bridge calls to have timeouts so that Pyodide cannot access arbitrary kernel paths or hang forever", + "acceptanceCriteria": [ + "VFS RPC operations in service.rs validate that request.path is within the guest's permitted scope", + "Kernel permission checks are applied to VFS RPC paths", + "Synchronous VFS RPC bridge calls have a configurable timeout (default 30s)", + "Timeout produces a clear error, not a hang", + "Existing Python tests pass", + "Typecheck passes" + ], + "priority": 39, + "passes": true, + "notes": "service.rs:2394-2470 passes request.path directly to kernel with no validation. readSync blocks forever if Rust never responds." + }, + { + "id": "US-027", + "title": "Wire options.permissions through to sidecar bridge", + "description": "As a developer, I want AgentOsOptions.permissions to actually control kernel permission policy so that the declared permission model is enforced", + "acceptanceCriteria": [ + "AgentOsOptions.permissions is serialized and sent to the sidecar bridge", + "Sidecar applies the permission policy to kernel operations", + "LocalBridge no longer defaults to allowAll", + "When permissions restrict fs access, guest fs operations are denied appropriately", + "When permissions restrict network, guest network operations are denied", + "Typecheck passes" + ], + "priority": 24, + "passes": true, + "notes": "permissions field is accepted but never consumed. LocalBridge allows everything. PermissionDescriptor exists on Rust side but TS always sends empty array." + }, + { + "id": "US-028", + "title": "Validate CWD within sandbox root", + "description": "As a security engineer, I want the execution CWD validated against the sandbox root so that setting cwd=/ cannot grant host-wide filesystem access", + "acceptanceCriteria": [ + "service.rs validates that the Execute request's cwd is within the configured sandbox root", + "Setting cwd=/ is rejected with a clear error", + "cwd is not directly used as real host current_dir without validation", + "--allow-fs-read/--allow-fs-write are scoped to sandbox root, not the raw cwd", + "Typecheck passes" + ], + "priority": 25, + "passes": true, + "notes": "service.rs:2195-2206 uses cwd directly as real host current_dir AND adds it to --allow-fs-read/--allow-fs-write. No validation." + }, + { + "id": "US-029", + "title": "Per-VM import cache paths to prevent cross-VM poisoning", + "description": "As a security engineer, I want each VM to use isolated import cache paths so that one VM cannot poison another VM's module resolution", + "acceptanceCriteria": [ + "Each VM instance gets a unique import cache directory", + "flushCacheState does not merge shared on-disk cache across VMs", + "A poisoned resolution entry in VM-A's cache cannot affect VM-B", + "Cache cleanup happens on VM shutdown", + "Typecheck passes" + ], + "priority": 32, + "passes": true, + "notes": "flushCacheState reads/merges/writes a shared cache. Two VMs sharing the same cache root enables cross-VM cache poisoning." + }, + { + "id": "US-030", + "title": "Fix --allow-child-process unconditional escalation", + "description": "As a security engineer, I want --allow-child-process and --allow-worker only passed to child Node processes when the parent was explicitly granted those permissions", + "acceptanceCriteria": [ + "prependNodePermissionArgs checks parent process permissions before adding --allow-child-process", + "prependNodePermissionArgs checks parent process permissions before adding --allow-worker", + "A guest process without child_process permission cannot spawn children that have it", + "Recursive escalation chain is broken", + "Typecheck passes" + ], + "priority": 26, + "passes": true, + "notes": "Currently --allow-child-process and --allow-worker are passed unconditionally to all child Node processes." + }, + { + "id": "US-031", + "title": "Resolve symlinks before permission checks and fix link/exists gaps", + "description": "As a security engineer, I want permission checks to use resolved paths so that symlinks cannot bypass access control", + "acceptanceCriteria": [ + "PermissionedFileSystem resolves symlinks before checking permissions", + "link() checks permissions on both source and destination paths", + "Symlinks are prevented from targeting paths across mount boundaries", + "exists() returns false on EACCES instead of leaking file existence", + "Typecheck passes" + ], + "priority": 27, + "passes": true, + "notes": "permissions.rs checks caller-supplied path, then inner fs resolves symlinks independently. TOCTOU bypass if mounts expose host paths." + }, + { + "id": "US-032", + "title": "Fix host PID reuse in signal_runtime_process and dup2 bounds", + "description": "As a security engineer, I want process signaling to verify child liveness and fd operations to validate bounds so that PID reuse and fd overflow are prevented", + "acceptanceCriteria": [ + "signal_runtime_process checks child liveness before sending kill(2)", + "Allowed signals whitelisted to SIGTERM, SIGKILL, SIGINT, SIGCONT, signal-0", + "dup2 validates new_fd < MAX_FDS_PER_PROCESS before proceeding", + "open_with validates fd bounds", + "PTY foreground PGID changes validate target PGID belongs to same session", + "Typecheck passes" + ], + "priority": 33, + "passes": true, + "notes": "Sidecar sends real kill(2) to host PIDs. PID reuse could kill wrong host process. dup2 skips fd bounds check." + }, + { + "id": "US-033", + "title": "Add filesystem size and inode limits to ResourceLimits", + "description": "As a security engineer, I want configurable filesystem size and inode count limits so that guest code cannot write to OOM", + "acceptanceCriteria": [ + "max_filesystem_bytes added to ResourceLimits with configurable default", + "max_inode_count added to ResourceLimits with configurable default", + "Write operations check total filesystem size before proceeding", + "File/directory creation checks inode count before proceeding", + "truncate and pwrite validate against size limits before resizing (prevents OOM)", + "Exceeding limits returns ENOSPC", + "Typecheck passes" + ], + "priority": 30, + "passes": true, + "notes": "All file data is in-memory with no cap. Guest can write until host OOM. truncate/pwrite with large values cause immediate OOM." + }, + { + "id": "US-034", + "title": "Add WASM fuel/memory limits and socket/connection limits", + "description": "As a security engineer, I want WASM execution and network resource limits so that guest code cannot exhaust compute or connection resources", + "acceptanceCriteria": [ + "WASM execution fuel limits are configurable and enforced", + "WASM memory growth caps are configurable and enforced", + "WASM stack size is bounded", + "Socket count limit added to ResourceLimits", + "Connection count limit added to ResourceLimits", + "Pipe/PTY read operations have configurable timeout (no infinite blocking on leaked write end)", + "read_frame checks declared_len against max_frame_bytes before allocation (prevents OOM)", + "Typecheck passes" + ], + "priority": 31, + "passes": true, + "notes": "No WASM fuel/memory/stack limits. No socket/connection limits. pipe.read/pty.read block forever if write end leaks." + }, + { + "id": "US-035", + "title": "Fix Pyodide hardening order and VFS RPC queue bounds", + "description": "As a security engineer, I want Pyodide hardening applied before loadPyodide and VFS RPC queue bounded so that cached API references and unbounded queues cannot be exploited", + "acceptanceCriteria": [ + "Hardening code (global restrictions, API removals) runs BEFORE loadPyodide()", + "Pyodide cannot cache references to dangerous APIs before hardening", + "VFS RPC request queue has a configurable bound (e.g. 1000 pending requests)", + "Exceeding queue bound returns an error, not silent accumulation", + "Typecheck passes" + ], + "priority": 40, + "passes": true, + "notes": "Hardening currently runs AFTER loadPyodide. VFS RPC queue is unbounded." + }, + { + "id": "US-036", + "title": "Add missing Pyodide integration tests", + "description": "As a developer, I want comprehensive Pyodide tests so that isolation guarantees are verified by the test suite", + "acceptanceCriteria": [ + "Test frozen time \u2014 Python sees deterministic/controlled time", + "Test node:child_process and node:vm are inaccessible from Python", + "Test zero network requests during Pyodide init", + "Test kill (SIGTERM) terminates Python execution", + "Test concurrent Python executions don't interfere", + "Test cross-runtime file visibility (Python can see files written by JS and vice versa)", + "All new tests pass", + "Typecheck passes" + ], + "priority": 41, + "passes": true, + "notes": "Multiple Pyodide Phase 1/3 acceptance criteria have no test coverage." + }, + { + "id": "US-037", + "title": "Add security audit logging", + "description": "As a security engineer, I want structured logging for security-relevant events so that breaches and policy violations are observable", + "acceptanceCriteria": [ + "Auth failures are logged with structured data (timestamp, source, reason)", + "Permission denials are logged (path, operation, policy)", + "Mount/unmount operations are logged", + "Process kill operations are logged (source PID, target PID, signal)", + "Logs use structured format (JSON or similar) suitable for aggregation", + "Typecheck passes" + ], + "priority": 43, + "passes": true, + "notes": "No security event logging exists. Auth failures, permission denials, mounts, kills are all silent." + }, + { + "id": "US-038", + "title": "Fix plugin SSRF and add mount permission checks", + "description": "As a security engineer, I want plugin URLs validated and mount operations permission-checked so that plugins cannot reach internal services and mounts cannot bypass access control", + "acceptanceCriteria": [ + "Google Drive plugin validates token_url and api_base_url against expected hosts", + "S3 plugin validates endpoint against private IP ranges (169.254.x.x, 10.x.x.x, etc.)", + "mount_filesystem in kernel.rs checks caller permissions, not just assert_not_terminated", + "Mounting at sensitive paths (/, /etc, /proc) requires elevated permission", + "Typecheck passes" + ], + "priority": 28, + "passes": true, + "notes": "Plugins accept arbitrary URLs. mount_filesystem only checks assert_not_terminated, no path or caller validation." + }, + { + "id": "US-039", + "title": "Fix host_dir TOCTOU, setpgid cross-driver, and mutex poison policy", + "description": "As a developer, I want kernel correctness issues fixed so that path resolution, process groups, and mutex handling are robust", + "acceptanceCriteria": [ + "host_dir mount uses O_NOFOLLOW/openat-style resolution to prevent symlink TOCTOU", + "setpgid validates that target PGID's owning driver matches requester", + "Single mutex poison policy applied consistently (lock_or_recover everywhere OR .expect everywhere)", + "Typecheck passes" + ], + "priority": 34, + "passes": true, + "notes": "fs::canonicalize + ensure_within_root has TOCTOU race. setpgid allows cross-driver group joining. Inconsistent mutex handling." + }, + { + "id": "US-040", + "title": "Fix hardenProperty fallback and zombie reaper exit code handling", + "description": "As a developer, I want property hardening to throw on failure and zombie reaping to preserve exit codes so that security and correctness are maintained", + "acceptanceCriteria": [ + "hardenProperty throws instead of falling back to mutable assignment", + "Zombie reaper preserves exit codes for zombies with living parents that haven't called waitpid", + "Typecheck passes" + ], + "priority": 35, + "passes": true, + "notes": "hardenProperty silently falls back to mutable. Zombie reaper loses exit codes." + }, + { + "id": "US-041", + "title": "Enforce WASM permission tiers", + "description": "As a security engineer, I want WASM commands restricted based on their declared permission tier so that read-only commands cannot write files or spawn processes", + "acceptanceCriteria": [ + "WASI preopens restricted based on declared permission tier (read-only, read-write, full)", + "host_process imports only provided to full-tier commands", + "read-only tier commands cannot write files", + "read-write tier commands cannot spawn processes or make network requests", + "Typecheck passes" + ], + "priority": 29, + "passes": true, + "notes": "Permission tiers are declared in descriptors but not enforced at runtime." + }, + { + "id": "US-042", + "title": "Extract Pyodide embedded JS and deduplicate cross-runtime code", + "description": "As a developer, I want embedded JS extracted to files and shared code deduplicated so that the codebase is maintainable", + "acceptanceCriteria": [ + "~870 lines of embedded JS in python.rs extracted to a .js file loaded at build time", + "~300 lines of duplicated code across python.rs/wasm.rs/javascript.rs extracted to a shared module", + "NodeImportCache temp directories cleaned up on crash (add cleanup-on-startup logic)", + "Typecheck passes" + ], + "priority": 42, + "passes": true, + "notes": "Large embedded JS strings are hard to maintain. Significant duplication across runtime implementations." + }, + { + "id": "US-043", + "title": "Low-priority robustness fixes", + "description": "As a developer, I want minor correctness and safety issues fixed so that edge cases don't cause panics or undefined behavior", + "acceptanceCriteria": [ + "read_dir uses tree structure instead of linear scan for directory children lookup", + "collect_snapshot_entries uses iteration with depth limit instead of unbounded recursion", + "nlink uses saturating_sub to prevent underflow", + "allocate_fd uses bounded scan to prevent potential infinite loop", + "SQLite WASM VFS uses kernel random_get instead of deterministic randomness", + "WASM FFI poll buffer validation, getpwuid buffer trust, usize-to-u32 truncation checks added", + "Typecheck passes" + ], + "priority": 36, + "passes": true, + "notes": "Collection of minor issues that individually have low impact but collectively improve robustness." + }, + { + "id": "US-044", + "title": "Implement kernel-controlled DNS resolver instead of host delegation", + "description": "As a security engineer, I want DNS resolution to go through the kernel rather than delegating to the host system resolver so that the isolation invariant (all syscalls through kernel) is maintained", + "acceptanceCriteria": [ + "dns.lookup() and dns.resolve() route through a kernel DNS forwarding layer, not host to_socket_addrs()", + "net.connect(hostname) resolves DNS through the kernel resolver, not directly via host", + "resolve_tcp_connect_addr in service.rs uses kernel DNS instead of (host, port).to_socket_addrs()", + "resolve_dns_ip_addrs in service.rs uses kernel DNS instead of (hostname, 0).to_socket_addrs()", + "Per-VM DNS configuration is possible (custom resolvers, overrides)", + "DNS results are kernel-observable and auditable", + "Existing networking tests pass", + "Typecheck passes" + ], + "priority": 44, + "passes": true, + "notes": "DNS currently delegates to host system resolver via Rust to_socket_addrs(). Functional but violates isolation invariant. Both net.connect(\"example.com\") and dns.lookup() resolve through host." + }, + { + "id": "US-045", + "title": "Implement real getConnections() and enforce server backlog", + "description": "As a developer, I want net.Server.getConnections() to return actual connection count and listen backlog to be enforced so that server resource management works correctly", + "acceptanceCriteria": [ + "server.getConnections(callback) returns actual active connection count instead of 0", + "Sidecar tracks active connections per listener", + "server.listen({ backlog }) is validated and enforced by the sidecar", + "Typecheck passes" + ], + "priority": 45, + "passes": true, + "notes": "getConnections() currently stubs to 0. Backlog parameter accepted but ignored in service.rs (let _ = payload.backlog)." + }, + { + "id": "US-046", + "title": "Add Unix domain socket support to net polyfill", + "description": "As a developer, I want Unix domain sockets supported in the net polyfill so that Node.js apps that use socket files work inside the VM", + "acceptanceCriteria": [ + "net.connect({ path }) creates a kernel-managed Unix domain socket", + "net.createServer().listen({ path }) binds a Unix domain socket", + "Unix socket files appear in the kernel VFS", + "Typecheck passes" + ], + "priority": 46, + "passes": true, + "notes": "Currently throws unsupported error. Many Node.js apps and frameworks assume Unix domain socket support." + }, + { + "id": "US-047", + "title": "Add external networking CI tests", + "description": "As a developer, I want external network connectivity tested in CI so that outbound connection regressions are caught automatically", + "acceptanceCriteria": [ + "At least one CI test validates outbound TCP connection to an external host", + "At least one CI test validates outbound HTTP/HTTPS request", + "Tests are robust to transient network failures (retry, skip on network unavailable)", + "curl.test.ts external network tests enabled in CI or equivalent coverage added" + ], + "priority": 47, + "passes": true, + "notes": "External network tests in curl.test.ts are skipped unless runExternalNetwork=true. No CI validation of outbound connectivity." + }, + { + "id": "US-048", + "title": "Audit and verify network permission checks on socket operations", + "description": "As a security engineer, I want network permission callbacks verified at socket operation time so that the permission model is actually enforced", + "acceptanceCriteria": [ + "NetworkAccessRequest callbacks are invoked on net.connect(), net.listen(), dns.lookup()", + "Permission denial returns proper error to guest code", + "Test that a VM with network permissions denied cannot make connections", + "Test that a VM with network permissions denied cannot bind servers", + "Typecheck passes" + ], + "priority": 48, + "passes": true, + "notes": "Permission framework exists (NetworkAccessRequest, NetworkOperation enums) but needs audit to confirm callbacks fire at socket operation time, not just policy setup." + }, + { + "id": "US-049", + "title": "Block remaining process properties that leak host information", + "description": "As a security engineer, I want process.config, process.versions, process.memoryUsage(), process.uptime(), process.platform, and process.arch replaced with virtual values so that no host build/runtime info is exposed", + "acceptanceCriteria": [ + "process.config returns a safe stub object (not host build config)", + "process.versions returns virtual versions (not host openssl/v8/zlib versions)", + "process.memoryUsage() returns virtual memory values", + "process.uptime() returns VM uptime, not host process uptime", + "process.platform returns 'linux' (not leaking host platform)", + "process.arch returns virtual arch value", + "process.release returns safe stub (not host release info)", + "The Proxy fallback in createGuestProcess no longer uses Reflect.get(source, key, source) for unhandled properties", + "Typecheck passes" + ], + "priority": 49, + "passes": true, + "notes": "Audit finding: guest process proxy only overrides 5 properties (execPath, pid, ppid, getuid, getgid). All others pass through via Reflect.get() fallback, leaking host build config, memory usage, uptime, etc." + }, + { + "id": "US-050", + "title": "Prevent CJS require() from resolving host node_modules", + "description": "As a security engineer, I want createGuestRequire() to only resolve from guest-visible paths so that host node_modules cannot be loaded by guest code", + "acceptanceCriteria": [ + "createGuestRequire() does not delegate to Module.createRequire() with host paths", + "Guest require('lodash') resolves from VM-visible node_modules only, not host node_modules", + "Module._resolveFilename is patched to translate paths before resolution", + "require.cache keys use guest paths, not host paths", + "Existing module resolution tests pass", + "Typecheck passes" + ], + "priority": 50, + "passes": true, + "notes": "Audit finding: createGuestRequire() uses Module.createRequire() + baseRequire() which resolves packages from HOST node_modules. Guest code can load arbitrary host packages." + }, + { + "id": "US-051", + "title": "Fix os polyfill fallbacks that default to host values", + "description": "As a security engineer, I want os.homedir(), os.userInfo(), os.tmpdir(), and os.hostname() to never fall back to real host environment variables when AGENT_OS_VIRTUAL_OS_* vars are unset", + "acceptanceCriteria": [ + "os.homedir() returns a safe default (e.g. /root) when AGENT_OS_VIRTUAL_OS_HOMEDIR is unset, never HOST_PROCESS_ENV.HOME", + "os.userInfo().username returns a safe default (e.g. root) when AGENT_OS_VIRTUAL_OS_USER is unset, never HOST_PROCESS_ENV.USER", + "os.tmpdir() returns /tmp when AGENT_OS_VIRTUAL_OS_TMPDIR is unset, never HOST_PROCESS_ENV.TMPDIR", + "os.hostname() returns 'agent-os' when AGENT_OS_VIRTUAL_OS_HOSTNAME is unset, never HOST_PROCESS_ENV.HOSTNAME", + "Shell returns /bin/sh when AGENT_OS_VIRTUAL_OS_SHELL is unset, never HOST_PROCESS_ENV.SHELL", + "Typecheck passes" + ], + "priority": 51, + "passes": true, + "notes": "Audit finding: os polyfill uses HOST_PROCESS_ENV.HOME/USER/SHELL/TMPDIR as fallback when AGENT_OS_VIRTUAL_OS_* not set, leaking host username, home dir, temp dir, shell path." + }, + { + "id": "US-052", + "title": "Strip AGENT_OS_* variables from child process spawn environments", + "description": "As a security engineer, I want AGENT_OS_* internal variables stripped from child process environments so that spawned children cannot reconstruct the guest/host path mapping", + "acceptanceCriteria": [ + "Child processes spawned via kernel child_process polyfill do not receive AGENT_OS_GUEST_PATH_MAPPINGS", + "Child processes do not receive AGENT_OS_VIRTUAL_PROCESS_EXEC_PATH or AGENT_OS_VIRTUAL_PROCESS_UID/GID", + "Child processes do not receive AGENT_OS_VIRTUAL_OS_* configuration variables", + "INTERNAL_ENV_KEYS merging in child spawn only passes keys actually needed for child bootstrap", + "Typecheck passes" + ], + "priority": 52, + "passes": true, + "notes": "Audit finding: child process env merging passes through all AGENT_OS_* and AGENT_OS_VIRTUAL_OS_* variables, allowing child processes to reconstruct the full guest/host mapping." + }, + { + "id": "US-053", + "title": "Add permission check to unmount_filesystem", + "description": "As a security engineer, I want unmount_filesystem to require permission checks so that guest code cannot unmount sensitive paths", + "acceptanceCriteria": [ + "unmount_filesystem() checks fs write permission on the mount path before proceeding", + "Unmounting sensitive paths (/, /etc, /proc) requires fs.mount_sensitive permission", + "Attempted unmount of denied path returns EACCES", + "Existing mount/unmount tests pass", + "Typecheck passes" + ], + "priority": 53, + "passes": true, + "notes": "Audit finding: unmount_filesystem() calls .inner_mut().inner_mut().unmount() directly, bypassing all permission checks. Guest can unmount any filesystem including /, /etc, /proc." + }, + { + "id": "US-054", + "title": "Change KernelVmConfig default permissions to deny-all", + "description": "As a security engineer, I want KernelVmConfig::new() to default to deny-all permissions so that forgetting to set permissions doesn't grant unrestricted access", + "acceptanceCriteria": [ + "KernelVmConfig::new() uses Permissions::default() (deny-all) instead of Permissions::allow_all()", + "All call sites that need allow_all explicitly set it", + "Tests that depend on allow_all are updated to explicitly request it", + "Typecheck passes" + ], + "priority": 54, + "passes": true, + "notes": "Audit finding: KernelVmConfig::new() defaults to Permissions::allow_all(). Any code creating a VM without explicit permissions gets unrestricted access." + }, + { + "id": "US-055", + "title": "Add SSRF protection with private IP address validation on outbound connections", + "description": "As a security engineer, I want outbound TCP/UDP connections validated against private IP ranges so that guest code cannot reach cloud metadata endpoints or internal services", + "acceptanceCriteria": [ + "net.connect() validates target address against blocked ranges before connecting", + "Blocked ranges: 169.254.0.0/16 (link-local/cloud metadata), 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 (private), 127.0.0.0/8 (loopback except exempt ports), ::1/128, fc00::/7, fe80::/10", + "DNS resolution results are validated against same blocked ranges before returning to guest", + "Loopback exempt ports still work (for mock LLM servers etc.)", + "Blocked connection attempts return EACCES with clear message", + "Typecheck passes" + ], + "priority": 55, + "passes": true, + "notes": "Audit finding: DNS resolution and TCP/UDP connections have zero address validation. Guest can SSRF to cloud metadata (169.254.169.254), internal databases, host services, etc." + }, + { + "id": "US-056", + "title": "Add per-operation size limits for pread, fd_write, env, and argv", + "description": "As a security engineer, I want individual read/write operations and process spawn arguments bounded so that a single operation cannot exhaust host memory", + "acceptanceCriteria": [ + "pread() length parameter capped at a configurable max (e.g. 64MB) to prevent OOM", + "fd_write() data size capped at a configurable max per-operation", + "Environment variables passed to spawn_process have total size limit", + "Command arguments passed to spawn_process have total size limit", + "readdir results are paginated or have a max batch size", + "truncate/pwrite validate target size against max_filesystem_bytes BEFORE allocating memory", + "Exceeding limits returns EINVAL or ENOMEM", + "Typecheck passes" + ], + "priority": 56, + "passes": true, + "notes": "Audit finding: pread(fd, 0, usize::MAX) allocates unbounded memory. fd_write accepts arbitrary data size. spawn_process env/args have no size limit. readdir returns all entries at once. truncate allocates before checking FS limits." + }, + { + "id": "US-057", + "title": "Protect RPC channel FDs from guest manipulation", + "description": "As a security engineer, I want sync RPC and control channel file descriptors protected from guest code so that RPC messages cannot be forged or channels disrupted", + "acceptanceCriteria": [ + "RPC channel FDs are remapped to high FD numbers (e.g. 1000+) out of guest FD range (0-255)", + "Guest code cannot close, dup2, read, or write to RPC channel FDs", + "Control channel FD is similarly protected", + "FD numbers are not exposed in environment variables readable by guest (or are protected by the guest-env proxy)", + "Forged writes to RPC request pipe have no effect on sidecar state", + "Typecheck passes" + ], + "priority": 57, + "passes": true, + "notes": "Audit finding: RPC FD numbers passed via env vars with FD_CLOEXEC cleared. Guest can close(), dup2(), read/write to forge RPC requests/responses, or break sidecar communication." + }, + { + "id": "US-058", + "title": "Add WASM module parser size limits and DoS protection", + "description": "As a security engineer, I want WASM module file size and section counts bounded so that malicious modules cannot cause parser DoS", + "acceptanceCriteria": [ + "WASM module file size capped before reading (e.g. 256MB max)", + "Import section count validated against a reasonable max before iteration", + "Memory section count validated similarly", + "varuint parsing has iteration bounds", + "Malformed modules produce clear error messages, not panics", + "Typecheck passes" + ], + "priority": 58, + "passes": true, + "notes": "Audit finding: fs::read() on module path has no size limit (can OOM). Import section iteration is unbounded if import_count is huge. varuint parsing has shift overflow check but no iteration cap." + }, + { + "id": "US-059", + "title": "Implement SIGCHLD delivery on child process exit", + "description": "As a developer, I want SIGCHLD delivered to parent processes when children exit so that async child reaping works correctly", + "acceptanceCriteria": [ + "SIGCHLD (signal 17) is delivered to parent process when child exits", + "SIGCHLD is delivered to parent when child is killed", + "Per-process signal handler registry tracks registered signals", + "Guest code can register SIGCHLD handler via process.on('SIGCHLD')", + "If no handler is registered, signal is silently ignored (POSIX default)", + "Typecheck passes" + ], + "priority": 59, + "passes": true, + "notes": "Audit finding: No SIGCHLD implementation. Only SIGTERM(15) and SIGKILL(9) are defined. Parent processes cannot receive async notification of child termination." + }, + { + "id": "US-060", + "title": "Implement SIGPIPE delivery on broken pipe write", + "description": "As a developer, I want SIGPIPE delivered when writing to a broken pipe so that the standard POSIX broken-pipe contract is honored", + "acceptanceCriteria": [ + "Writing to a pipe whose read end is closed delivers SIGPIPE to the writer", + "If SIGPIPE is ignored/blocked, write returns EPIPE (existing behavior preserved)", + "SIGPIPE delivery respects signal masks when implemented", + "Typecheck passes" + ], + "priority": 60, + "passes": true, + "notes": "Audit finding: pipe_manager returns EPIPE error but does not deliver SIGPIPE signal. Linux requires both signal delivery AND EPIPE error." + }, + { + "id": "US-061", + "title": "Implement waitpid flags: WNOHANG, WUNTRACED, WCONTINUED, and process group waits", + "description": "As a developer, I want full waitpid semantics so that shells and process managers can reap children correctly", + "acceptanceCriteria": [ + "waitpid supports WNOHANG flag (returns immediately if no exited child)", + "waitpid supports WUNTRACED flag (also reports stopped children)", + "waitpid supports WCONTINUED flag (also report continued children)", + "waitpid supports pid=-1 (wait for any child)", + "waitpid supports negative PID (wait for any child in process group)", + "waitpid supports pid=0 (wait for any child in caller's process group)", + "Typecheck passes" + ], + "priority": 61, + "passes": true, + "notes": "Audit finding: waitpid(pid: u32) only blocks indefinitely on single process. No WNOHANG, WUNTRACED, WCONTINUED, negative PID, or pid=-1 support." + }, + { + "id": "US-062", + "title": "Implement advisory file locking (flock)", + "description": "As a developer, I want advisory file locking so that git, npm, and other tools that use lock files work correctly inside the VM", + "acceptanceCriteria": [ + "flock() syscall implemented with LOCK_SH (shared), LOCK_EX (exclusive), LOCK_UN (unlock)", + "LOCK_NB (non-blocking) flag supported — returns EWOULDBLOCK if lock unavailable", + "Locks are per-FD (not per-process) following POSIX semantics", + "Locks are released when FD is closed", + "Lock conflicts between processes are properly detected", + "Typecheck passes" + ], + "priority": 62, + "passes": true, + "notes": "Audit finding: Neither flock() nor fcntl(F_SETLK) implemented anywhere. Git, npm, and many tools depend on file locking. This is the #1 compatibility blocker for agent tools." + }, + { + "id": "US-063", + "title": "Implement O_CREAT|O_EXCL atomicity and O_APPEND atomic writes", + "description": "As a developer, I want atomic file creation and atomic append writes so that concurrent operations don't cause data corruption", + "acceptanceCriteria": [ + "O_CREAT|O_EXCL is a single atomic operation (no TOCTOU between exists check and creation)", + "O_APPEND writes atomically seek to EOF and write in a single locked operation", + "Concurrent O_CREAT|O_EXCL calls on the same path — exactly one succeeds, others get EEXIST", + "Concurrent O_APPEND writes don't interleave data", + "Typecheck passes" + ], + "priority": 63, + "passes": true, + "notes": "Audit finding: O_CREAT|O_EXCL checks exists() then creates (TOCTOU race). O_APPEND reads file size, then seeks, then writes (race condition). Both are critical for git ref creation and concurrent log writes." + }, + { + "id": "US-064", + "title": "Implement non-blocking I/O (O_NONBLOCK) and PIPE_BUF atomicity", + "description": "As a developer, I want non-blocking I/O and atomic pipe writes so that event loops and IPC work correctly", + "acceptanceCriteria": [ + "O_NONBLOCK flag tracked per-FD in the FD table", + "Non-blocking read on empty pipe returns EAGAIN instead of blocking", + "Non-blocking write on full pipe returns EAGAIN instead of blocking", + "Non-blocking connect returns EINPROGRESS", + "Pipe writes <= PIPE_BUF (4096 bytes) are atomic — not interleaved with other writes", + "Typecheck passes" + ], + "priority": 64, + "passes": true, + "notes": "Audit finding: O_NONBLOCK not implemented. Pipe writes not atomic at any size. Non-blocking I/O is required for event loops, Node.js internals, and many CLI tools." + }, + { + "id": "US-065", + "title": "Implement select/poll for FD multiplexing", + "description": "As a developer, I want FD multiplexing so that processes can wait on multiple file descriptors simultaneously", + "acceptanceCriteria": [ + "poll() syscall implemented for pipes, PTYs, and sockets", + "POLLIN (readable), POLLOUT (writable), POLLERR, POLLHUP events supported", + "Timeout parameter works correctly (0=non-blocking check, -1=block forever, N=timeout in ms)", + "Can poll across different FD types (pipe + PTY + socket)", + "Typecheck passes" + ], + "priority": 65, + "passes": true, + "notes": "Audit finding: No select/poll/epoll mechanism in kernel. Cannot multiplex I/O across FDs. Breaks event loops, shell I/O multiplexing, and server accept loops." + }, + { + "id": "US-066", + "title": "Implement process reparenting to init and fix process group kill", + "description": "As a developer, I want orphaned processes reparented to init and process group kill to reach all process states so that process lifecycle matches Linux", + "acceptanceCriteria": [ + "When a parent process exits, its children are reparented to PID 1 (init/kernel)", + "Orphaned process groups receive SIGHUP+SIGCONT per POSIX", + "kill(-pgid) reaches processes in Running, Stopped, AND Zombie states (not just Running)", + "Zombie processes count against max_processes resource limit (prevent zombie storm bypass)", + "Typecheck passes" + ], + "priority": 66, + "passes": true, + "notes": "Audit finding: No reparenting — orphaned children become standalone zombies. Process group kill filters for ProcessStatus::Running only, missing stopped/zombie. Zombie processes bypass max_processes since only running_processes is checked." + }, + { + "id": "US-067", + "title": "Implement OverlayFS opaque directories and persistent whiteouts", + "description": "As a developer, I want OverlayFS opaque directory markers and durable whiteouts so that overlay semantics match Linux OverlayFS", + "acceptanceCriteria": [ + "Copy-up of a directory marks it opaque in the upper layer", + "Opaque directories hide all entries from lower layers", + "Whiteout state is stored durably (in upper layer metadata, not in-memory Set)", + "Whiteouts survive snapshot/restore cycles", + "S3 and other remote upper layers persist whiteout markers", + "Typecheck passes" + ], + "priority": 67, + "passes": true, + "notes": "Audit finding: No opaque directory markers — lower layer entries leak through after copy-up. Whiteouts stored in in-memory Set, lost on snapshot/persistence. S3 mount doesn't persist whiteouts." + }, + { + "id": "US-068", + "title": "Fix overlay hardlink copy-up, rmdir ENOTEMPTY, and cross-mount hardlink", + "description": "As a developer, I want overlay hardlink operations and rmdir to be correct so that filesystem operations don't silently corrupt data", + "acceptanceCriteria": [ + "link() after copy-up references the correct upper layer path (not the original lower path)", + "rmdir() checks children in BOTH upper and lower layers before removing (returns ENOTEMPTY if lower has children)", + "Hardlink across mount boundaries returns EXDEV (mount_table.rs link() checks old_index == new_index)", + "rename() in overlay is crash-safe (use rename-in-upper, not read+write+delete)", + "Typecheck passes" + ], + "priority": 68, + "passes": true, + "notes": "Audit finding: Hardlink copy-up resolves wrong path. removeDir succeeds even when lower layer has children. Hardlink across mounts doesn't check mount index. Rename uses non-atomic read+write+delete." + }, + { + "id": "US-069", + "title": "Implement /proc filesystem with essential entries", + "description": "As a developer, I want a /proc filesystem so that tools that inspect process state work correctly inside the VM", + "acceptanceCriteria": [ + "/proc/self is a symlink to /proc/[current_pid]", + "/proc/[pid]/fd/ lists open file descriptors as symlinks", + "/proc/[pid]/cmdline contains null-separated command line", + "/proc/[pid]/environ contains null-separated environment", + "/proc/[pid]/cwd is a symlink to the process working directory", + "/proc/[pid]/stat contains basic process status info", + "/proc/mounts lists mounted filesystems", + "Typecheck passes" + ], + "priority": 69, + "passes": true, + "notes": "Audit finding: /proc is read-only and returns generic error. No /proc/self, /proc/[pid]/fd, /proc/[pid]/cmdline, /proc/mounts, etc. Many tools read /proc to discover process state." + }, + { + "id": "US-070", + "title": "Fix /dev/zero and /dev/urandom to return requested byte count", + "description": "As a developer, I want device reads to return the requested number of bytes so that reads from /dev/zero and /dev/urandom behave like Linux", + "acceptanceCriteria": [ + "/dev/zero read returns exactly the requested number of zero bytes (not fixed 4096)", + "/dev/urandom read returns exactly the requested number of random bytes (not fixed 4096)", + "Reading 5 bytes from /dev/zero returns 5 bytes", + "Reading 1MB from /dev/urandom returns 1MB (up to a sane max)", + "Typecheck passes" + ], + "priority": 70, + "passes": true, + "notes": "Audit finding: device_layer.rs returns vec![0; 4096] and random_bytes(4096) regardless of requested length. Should return requested length." + }, + { + "id": "US-071", + "title": "Implement shebang parsing for script execution", + "description": "As a developer, I want the kernel to parse #! shebangs so that script files can be executed directly", + "acceptanceCriteria": [ + "When exec() encounters a file starting with #!, it parses the interpreter path and arguments", + "#!/bin/sh script.sh executes as sh script.sh", + "#!/usr/bin/env node executes the script with node", + "Shebang line is limited to a reasonable max length (256 bytes)", + "Missing interpreter returns ENOENT", + "Typecheck passes" + ], + "priority": 71, + "passes": true, + "notes": "Audit finding: Kernel doesn't parse shebang lines. Scripts starting with #!/bin/sh won't execute. Common pattern in agent workflows." + }, + { + "id": "US-072", + "title": "Add JavaScript sync RPC timeout and response backpressure", + "description": "As a developer, I want JavaScript sync RPC calls to have timeouts and response writes to have backpressure so that slow guests cannot hang the sidecar", + "acceptanceCriteria": [ + "Sync RPC requests have a configurable timeout (default 30s, matching Python VFS bridge)", + "Timeout produces a clear error response, not a hang", + "Response writer has backpressure — if guest slow-reads, sidecar does not block indefinitely", + "RPC response writer uses a bounded buffer with timeout", + "Typecheck passes" + ], + "priority": 72, + "passes": true, + "notes": "Audit finding: JavaScript sync RPC in service.rs dispatches to kernel without timeout. Response writer can deadlock if guest slow-reads. Python VFS bridge has 30s timeout but JS bridge does not." + }, + { + "id": "US-073", + "title": "Add network port binding restrictions and VM network isolation", + "description": "As a security engineer, I want port binding restricted and VMs isolated from each other's network so that guest code cannot expose services on host interfaces or interfere with other VMs", + "acceptanceCriteria": [ + "Guest code cannot bind to 0.0.0.0 (only 127.0.0.1 or :: loopback)", + "Port range restrictions configurable per-VM (e.g. only ephemeral ports 49152-65535)", + "Privileged ports (< 1024) denied unless explicitly allowed", + "Two VMs cannot interfere via shared host port bindings", + "socket_host_matches() no longer treats 0.0.0.0 as matching loopback", + "Typecheck passes" + ], + "priority": 73, + "passes": true, + "notes": "Audit finding: Guest can bind to ANY port on ANY interface including 0.0.0.0. Two VMs can interfere via shared host socket table. socket_host_matches() is overly permissive." + }, + { + "id": "US-074", + "title": "Fix guestVisiblePathFromHostPath to never fall back to raw host path", + "description": "As a security engineer, I want path translation to return a safe default instead of the raw host path when mapping fails so that unmapped paths never leak to guest code", + "acceptanceCriteria": [ + "guestVisiblePathFromHostPath returns a safe placeholder (e.g. '/unknown') when no mapping matches, never the raw host path", + "INITIAL_GUEST_CWD returns a safe default (e.g. /root or /workspace) when HOST_CWD has no mapping, never HOST_CWD itself", + "translateTextToGuest uses the same safe default for unmapped paths in error messages", + "Error stack traces never contain host filesystem paths", + "Typecheck passes" + ], + "priority": 74, + "passes": true, + "notes": "Audit finding: guestVisiblePathFromHostPath ?? value falls back to host path. INITIAL_GUEST_CWD ?? HOST_CWD falls back to host CWD. Both leak host filesystem layout." + }, + { + "id": "US-075", + "title": "Implement SIGSTOP/SIGCONT job control and SIGWINCH for PTY resize", + "description": "As a developer, I want SIGSTOP/SIGCONT for job control and SIGWINCH for terminal resize so that interactive shells and terminal apps work correctly", + "acceptanceCriteria": [ + "SIGSTOP transitions a process to ProcessStatus::Stopped", + "SIGCONT resumes a stopped process back to Running", + "PTY resize generates SIGWINCH to the foreground process group", + "^Z in PTY delivers SIGTSTP (already partially implemented)", + "Shell bg/fg commands can use SIGCONT to resume stopped jobs", + "Typecheck passes" + ], + "priority": 75, + "passes": true, + "notes": "Audit finding: ProcessStatus::Stopped exists but is unreachable. No SIGSTOP/SIGCONT mechanism. No SIGWINCH on PTY resize. Shell job control broken." + }, + { + "id": "US-076", + "title": "Add missing errno checks: EISDIR, ENOTDIR, ENAMETOOLONG, EROFS", + "description": "As a developer, I want correct errno values for common error cases so that tools that check errno values behave correctly", + "acceptanceCriteria": [ + "Writing to a directory returns EISDIR (not ENOENT or generic error)", + "Path component that is a file returns ENOTDIR (e.g. stat('/file/child') when /file is regular file)", + "Path exceeding max length returns ENAMETOOLONG (add configurable max, e.g. 4096)", + "Write to read-only filesystem returns EROFS (not EACCES)", + "Typecheck passes" + ], + "priority": 76, + "passes": true, + "notes": "Audit finding: EISDIR not returned for write-on-directory. ENOTDIR not checked in path components. ENAMETOOLONG not implemented. EROFS not distinguished from EACCES." + }, + { + "id": "US-077", + "title": "Implement umask and stat blocks/dev fields", + "description": "As a developer, I want umask support and complete stat fields so that file creation modes and stat output match Linux expectations", + "acceptanceCriteria": [ + "umask() syscall implemented per-process (default 0o022)", + "File/directory creation applies umask to permission bits", + "stat() returns st_blocks field (allocated 512-byte blocks)", + "stat() returns st_dev field (device ID identifying the filesystem)", + "stat() returns st_rdev for device files (major:minor)", + "atime is updated on all read operations (not just pread)", + "ctime is updated on all metadata changes", + "Typecheck passes" + ], + "priority": 77, + "passes": true, + "notes": "Audit finding: No umask implementation. stat missing blocks/dev fields. atime only updated on pread, not general reads. ctime inconsistently updated." + }, + { + "id": "US-078", + "title": "Add WASM module path symlink TOCTOU protection and prewarm timeout", + "description": "As a security engineer, I want WASM module path resolution to be safe from symlink TOCTOU and prewarm to have timeouts so that module loading cannot be exploited or hang", + "acceptanceCriteria": [ + "resolved_module_path() canonicalizes paths consistently with normalize_path() in permission setup", + "Module validation and execution use the same resolved path (no TOCTOU window between them)", + "File fingerprint uses inode+dev instead of size+mtime to prevent swap attacks", + "ensure_materialized() has a configurable timeout (default 30s)", + "Prewarm phase has a separate timeout from execution", + "Typecheck passes" + ], + "priority": 78, + "passes": true, + "notes": "Audit finding: resolved_module_path() doesn't canonicalize while normalize_path() does — TOCTOU between validation and execution. File fingerprint uses size+mtime (swappable). ensure_materialized() can hang with no timeout." + }, + { + "id": "US-079", + "title": "Add Pyodide process memory and execution timeout limits", + "description": "As a security engineer, I want Pyodide processes bounded by memory and execution time so that runaway Python code cannot exhaust host resources", + "acceptanceCriteria": [ + "Configurable memory limit for Pyodide Node.js host process (e.g. --max-old-space-size)", + "Configurable execution timeout per Python run (default 5 minutes)", + "Timeout kills the process cleanly and returns a timeout error", + "Memory limit produces a clear OOM error, not a host crash", + "Recursion depth stays at Python default (~1000) or is configurable", + "Typecheck passes" + ], + "priority": 79, + "passes": true, + "notes": "Audit finding: No memory limit on Pyodide process. No execution timeout at Python level. Recursion depth only limited by Python default. Pyodide is otherwise well-secured but resource limits are missing." + }, + { + "id": "US-080", + "title": "Enforce WASM runtime memory limits and pass fuel to Node.js runtime", + "description": "As a security engineer, I want WASM memory and fuel limits actually enforced at runtime so that guest WASM code cannot exhaust host memory or compute", + "acceptanceCriteria": [ + "WASM_MAX_MEMORY_BYTES_ENV is passed to the Node.js runtime process (not just used for compile-time validation)", + "Node.js WASI runtime enforces max memory pages matching the configured limit", + "WASM memory.grow() beyond the limit fails at runtime (not just at module load)", + "WASM fuel limit is per-instruction metering, not just a coarse process timeout", + "If per-instruction fuel is not feasible, document the gap and ensure process timeout is tight", + "Typecheck passes" + ], + "priority": 80, + "passes": true, + "notes": "Audit finding: WASM_MAX_MEMORY_BYTES_ENV only validated at compile time in validate_module_limits(). Not passed to Node.js runtime. Fuel converted to millisecond timeout with 10ms granularity. Guest WASM can grow memory unbounded at runtime." + }, + { + "id": "US-081", + "title": "Make WASI conditional based on permission tier", + "description": "As a security engineer, I want WASI disabled for restricted permission tiers so that isolated WASM commands cannot access host resources via WASI", + "acceptanceCriteria": [ + "allow_wasi parameter in harden_node_command is derived from permission tier, not hardcoded true", + "Isolated tier: WASI disabled (allow_wasi = false)", + "ReadOnly tier: WASI enabled with read-only preopens only", + "ReadWrite tier: WASI enabled with read-write preopens", + "Full tier: WASI enabled with all preopens", + "Typecheck passes" + ], + "priority": 81, + "passes": true, + "notes": "Audit finding: wasm.rs line 612 hardcodes allow_wasi = true for all WASM execution regardless of permission tier. Even Isolated tier gets WASI." + } + ] +} diff --git a/scripts/ralph/progress.txt b/scripts/ralph/progress.txt new file mode 100644 index 000000000..942cdad5c --- /dev/null +++ b/scripts/ralph/progress.txt @@ -0,0 +1,1546 @@ +# Ralph Progress Log +## Codebase Patterns +- WASM permission tiers should drive both guest-side preopen behavior and host Node permission flags in `crates/execution/src/wasm.rs`; `Isolated` must keep `--allow-wasi` off entirely, while `ReadOnly` / `ReadWrite` / `Full` differ through the WASI layer's read/write scope. +- Pyodide runtime hardening knobs should stay in reserved `AGENT_OS_PYTHON_*` execution env keys: apply heap caps to both prewarm and execution host launches, and make `PythonExecution::wait(None)` honor the configured per-run timeout instead of treating `None` as unbounded. +- WASM runtime limits span both `crates/execution/src/wasm.rs` and the generated `wasm-runner.mjs` in `crates/execution/src/node_import_cache.rs`: pass `AGENT_OS_WASM_MAX_*` through reserved env, keep the Node argv flags in sync, and cap the module memory section before `WebAssembly.compile()` so `memory.grow()` obeys the configured limit even when the module omits a maximum. +- Per-process filesystem state such as `umask` belongs in `ProcessContext` / `ProcessTable`; when guest Node code needs it, thread it through `crates/kernel/src/kernel.rs`, `crates/sidecar/src/service.rs`, and `crates/execution/src/node_import_cache.rs` together instead of reading host `process`. +- `VirtualStat` field additions must be propagated as one bundle across kernel stat producers, sidecar protocol serialization, mount/plugin adapters, and the TypeScript `VirtualStat` / `GuestFilesystemStat` surfaces or some callers will silently keep incomplete metadata. +- Filesystem errno hardening usually needs both layers updated together: enforce fast-fail guest-path validation in `crates/kernel/src/permissions.rs` so overlong paths do not degrade into permission errors, and keep `crates/kernel/src/vfs.rs` path traversal authoritative for semantic errors like `ENOTDIR`. +- Job-control signal state transitions should be split by layer: `crates/kernel/src/process_table.rs` owns `SIGSTOP`/`SIGTSTP`/`SIGCONT` status changes and `waitpid` notifications, while `crates/kernel/src/kernel.rs` should emit PTY-driven `SIGWINCH` after the PTY layer reports the foreground process group. +- Guest path scrubbing in `crates/execution/src/node_import_cache.rs` should treat `HOST_CWD` as an implicit runtime-only mapping to the virtual guest cwd for entrypoint loading and stack traces, and only fall back to `/unknown` for absolute host paths outside visible mappings or internal cache roots. +- Sidecar-managed loopback `net.listen` / `dgram.bind` now separate guest-visible ports from hidden host-bound ports; use guest ports in RPC responses and snapshots, but use the actual host listener port when a host-side test client needs to connect directly. +- JavaScript sync RPC timeout and backpressure belong in `crates/execution/src/javascript.rs`: track the pending request ID on the host, auto-emit `ERR_AGENT_OS_NODE_SYNC_RPC_TIMEOUT` there, queue replies through a bounded async writer so slow guest reads cannot block the sidecar thread, and let `crates/sidecar/src/service.rs` ignore stale `sync RPC request ... is no longer pending` races after timeout. +- Active JavaScript/Python/WASM executions must retain a `NodeImportCache` cleanup guard until the child exits; otherwise dropping the engine can delete `timing-bootstrap.mjs` and related cached runner assets while the host runtime is still importing them. +- Direct script execution in `crates/kernel/src/kernel.rs` should first map registered `/bin/*` and `/usr/bin/*` command stubs back to their command drivers, and only parse shebangs for real file paths; otherwise stub executables like `/bin/sh` recurse into their own wrapper. +- Stream devices in `crates/kernel/src/device_layer.rs` should share one length-aware helper, and exact Linux-style byte-count behavior for `/dev/zero` / `/dev/urandom` should be asserted through `pread` / `fd_read` rather than `read_file()`. +- Synthetic procfs entries in `crates/kernel/src/kernel.rs` should authorize the guest-visible `/proc/...` path directly; if procfs checks go through `PermissionedFileSystem::check_path(...)`, missing backing `/proc` directories in the mounted root can accidentally break the virtual proc layer. +- OverlayFS mutating ops should not trust merged `read_dir()` for emptiness once copy-up marks directories opaque; raw upper/lower listings are required for `rmdir`, and rename-like moves should stage source entries into the upper and then use the upper filesystem's native `rename` to preserve hardlinks/inode identity. +- Process-table exit-path changes should be implemented as one bundle: reparent orphaned children, reevaluate orphaned stopped groups for `SIGHUP`/`SIGCONT`, and keep `max_processes` enforcement counting unreaped zombies so lifecycle semantics and resource limits stay aligned. +- Overlay whiteout and opaque-directory state should live under a reserved hidden metadata root in the writable upper, and every merged overlay listing or snapshot path must filter that metadata root back out of user-visible results. +- Cross-resource kernel readiness waits should use the shared `PollNotifier` in `crates/kernel/src/poll.rs`; when pipe or PTY state changes, notify it alongside the manager condvar so mixed-FD `poll_fds` calls do not miss wakeups. +- Kernel filesystem semantic additions must be threaded through every wrapper layer together: `VirtualFileSystem`, `PermissionedFileSystem`, `DeviceLayer`, `MountTable`/`MountedFileSystem`, and the root/overlay delegates, or mounted/device-backed paths silently keep the old behavior. +- Per-FD status bits such as `O_NONBLOCK` belong on `FdEntry` / `ProcessFdTable`, while shared `FileDescription.flags()` should stay limited to open-file-description semantics such as access mode and `O_APPEND`; use `/dev/fd/N` duplication when you need a differently flagged view of the same description before a real `fcntl(F_SETFL)` surface exists. +- PID-aware POSIX signal side effects belong at `KernelVm` syscall entrypoints, not low-level resource managers: `PipeManager` should stay signal-agnostic and let `crates/kernel/src/kernel.rs` `fd_write` translate broken-pipe `EPIPE` into `SIGPIPE`. +- Parent-aware `waitpid` state tracking belongs in `crates/kernel/src/process_table.rs`: queue stop/continue notifications there, and let `crates/kernel/src/kernel.rs` clean up resources only after an exited child is actually reaped. +- Advisory `flock` state should be kernel-global but owned by the shared open-file-description (`FileDescription.id()`), keyed by the opened file identity, and released only when the last refcounted FD closes so dup/fork inheritance shares locks while separate opens still conflict. +- WebAssembly parser hardening in `crates/execution/src/wasm.rs` should stat module files before `fs::read()`, cap section entry counts before iteration, and bound varuint byte length so malformed modules fail closed without parser DoS. +- Child-facing control/RPC pipes in `crates/execution` should keep their original `pipe2(O_CLOEXEC)` FDs private and use `ExportedChildFds` in `crates/execution/src/node_process.rs` to duplicate only the child ends into reserved `1000+` FD numbers right before `Command::spawn()`. +- `KernelVmConfig::new()` is deny-all by default; any kernel or browser-sidecar fixture that expects unrestricted filesystem/process access must opt in with `config.permissions = Permissions::allow_all()`. +- Per-operation memory guards belong in `ResourceLimits`; when adding one, enforce it in the kernel entrypoint that materializes data and keep the matching `resource.max_*` metadata parsing in `crates/sidecar/src/service.rs` in sync. +- Sidecar JavaScript network policy should read internal bootstrap env like `AGENT_OS_LOOPBACK_EXEMPT_PORTS` from `CreateVmRequest.metadata` `env.*` entries, not `vm.guest_env`, because `guest_env` is permission-filtered and may be empty. +- Kernel mount and unmount entrypoints in `crates/kernel/src/kernel.rs` should both route through `check_mount_permissions(...)` so `fs.write` and `fs.mount_sensitive` stay consistent for `/`, `/etc`, and `/proc`. +- Guest `child_process` internals should never ride in `options.env`: strip `AGENT_OS_*` keys in `crates/execution/src/node_import_cache.rs`, carry only the Node bootstrap allowlist in `options.internalBootstrapEnv`, and let `crates/sidecar/src/service.rs` re-inject that allowlisted map only for nested JavaScript runtimes. +- The guest `os` polyfill in `crates/execution/src/node_import_cache.rs` should only honor explicit `AGENT_OS_VIRTUAL_OS_*` overrides; safe defaults like `agent-os`, `/root`, `/tmp`, and `/bin/sh` must not fall back to host env vars. +- JavaScript sync-RPC networking in `crates/sidecar/src/service.rs` bypasses the kernel permission wrappers, so `dns.lookup`/`net.connect`/`net.listen` must enforce `network.dns`/`network.http`/`network.listen` there directly, and errno-style failures should be preserved into `respond_javascript_sync_rpc_error(...)` so guest code sees `EACCES` instead of a generic sync-RPC code. +- Guest-visible `process` virtualization in `crates/execution/src/node_import_cache.rs` is safest when you harden properties on the real `process` first and let the guest proxy fall through with `Reflect.get(..., proxy)`; using the host `process` as the fallback receiver can leak unsanitized accessor state. +- Sidecar TCP/Unix socket readers should treat peer EOF as a half-close, not a full close: emit `End` immediately, but only emit `Close` after the local write half has also been shut down, or guest `socket.end(...)` flows can turn into resets. +- Native sidecar security telemetry should use `bridge.emit_structured_event(...)` with a `timestamp` field and stable keys like `policy`, `path`, `reason`, `source_pid`, and `target_pid`; this makes sidecar tests assertable without scraping free-form logs. +- Sidecar VM-scoped DNS policy is driven from `CreateVmRequest.metadata`: use `network.dns.servers` for comma-separated upstream resolvers and `network.dns.override.` for fixed answers, and emit `network.dns.resolved` / `network.dns.resolve_failed` structured events so resolution is observable in tests. +- Execution host-runner scripts that `NodeImportCache` materializes should live in `crates/execution/assets/runners/` and be loaded with `include_str!`; for temp-cache cleanup regressions, construct the cache with `NodeImportCache::new_in(...)` so the one-time sweep is scoped to the test root. +- Real bundled-Pyodide coverage belongs in `crates/execution/src/node_import_cache.rs` materialized-runner tests, and those helpers should load `timing-bootstrap.mjs` so frozen `Date`/`performance` behavior matches real execution launches; use `crates/execution/tests/python.rs` for fake-`pyodide.mjs` bootstrap regressions. +- Sidecar `host_dir` mounts should anchor guest path resolution with `openat2(..., RESOLVE_BENEATH | RESOLVE_NO_MAGICLINKS)` and translate kernel `EXDEV` escape rejections back to guest-facing `EACCES`. +- Python VFS RPCs are intentionally scoped to `/workspace`; normalize and reject anything outside that guest root in `crates/sidecar/src/service.rs` before touching the kernel VFS. +- Pyodide VFS RPC timeouts are safer to enforce in `crates/execution/src/python.rs` against pending request IDs than inside the embedded runner; touching the bundled Python runner can perturb real Pyodide bootstrap behavior. +- Pyodide bootstrap hardening in `crates/execution/src/node_import_cache.rs` must stay staged: `globalThis` guards can go in before `loadPyodide()`, but mutating `process` before `loadPyodide()` breaks the bundled Pyodide runtime under Node `--permission`. +- Non-reaping host child liveness checks in `crates/sidecar/src/service.rs` should use `waitid(..., WNOWAIT | WNOHANG | WEXITED | WSTOPPED | WCONTINUED)`; `waitpid` does not provide a safe non-reaping probe for the PID-reuse hardening path. +- `crates/execution/src/node_import_cache.rs` hardening helpers should fail closed: if `Object.defineProperty(...)` cannot lock down a guest-visible property, throw immediately instead of falling back to mutable assignment. +- Kernel zombie cleanup in `crates/kernel/src/process_table.rs` should only reap exited children once they no longer have a living parent in the table; otherwise reschedule them so `waitpid` can still observe their exit code. +- Native execution engines should own `NodeImportCache` state per `vm_id`, and sidecar VM disposal should call each engine's `dispose_vm`; a single engine-wide cache leaks module state across VMs. +- WASM runtime hardening is split across three layers together: `ResourceLimits` / sidecar metadata parsing, `crates/sidecar/src/service.rs` injecting reserved `AGENT_OS_WASM_*` env keys, and `crates/execution/src/wasm.rs` validating or enforcing the actual limit before guest code runs. +- Sidecar `ResourceLimits` parsing should start from `ResourceLimits::default()` and only override metadata keys that are present; rebuilding the struct from sparse metadata silently drops default filesystem byte/inode caps. +- WASM command permission tiers have to be threaded through all three layers together: `packages/core` command metadata, sidecar protocol/service request fields (`command_permissions` and per-exec `wasm_permission_tier`), and `StartWasmExecutionRequest.permission_tier`; top-level exec and JS `child_process` launches use separate paths. +- Native WASM host-import wrappers in `registry/native/crates/wasi-ext`, the matching wasi-libc patches, and the uucore WASI stubs should validate every guest buffer length crossing (`usize` -> `u32`) and reject host-returned lengths that exceed the supplied buffer; `poll()` wrappers should also enforce the exact 8-byte-per-`pollfd` layout. +- Sensitive mount paths are gated separately from ordinary writes: kernel mount APIs require `fs.write` on the mount target, and `/`, `/etc`, `/proc` also require `fs.mount_sensitive`; in sidecar tests, `configure_vm` reconciles mounts before `payload.permissions`, so mount-time policy must already be installed on the VM (for example via `bridge.set_vm_permissions(...)`). +- Filesystem permission checks in `crates/kernel/src/permissions.rs` should resolve the deepest existing ancestor before authorizing create/probe paths, make `exists()` fail closed, and stay aligned with `crates/kernel/src/mount_table.rs` rejecting cross-mount symlink targets with `EXDEV`. +- Python execution in `crates/execution/src/python.rs` should keep `poll_event()` blocked until a real guest-visible event arrives or the caller timeout expires; filtered stderr/control traffic is internal noise, and `wait()` should cap buffered stdio via the hidden `AGENT_OS_PYTHON_OUTPUT_BUFFER_MAX_BYTES` env knob instead of growing unbounded buffers. +- Native sidecar permission policy must be serialized into `CreateVmRequest`, not just `configure_vm`, because guest env filtering and bootstrap driver registration both happen during VM construction. +- Sidecar execute flows should validate host `cwd` against `vm.cwd` before spawn, then pass the sandbox root to the Node permission layer separately from the runtime `current_dir`; the host process can start in a subdirectory without widening `--allow-fs-read/--allow-fs-write`. +- Node builtin hardening is split between `packages/core/src/sidecar/native-kernel-proxy.ts` and four generated surfaces in `crates/execution/src/node_import_cache.rs` (loader, Node runner, Python runner, denied asset materialization); update all of them together when changing builtin policy. +- CJS module isolation in `crates/execution/src/node_import_cache.rs` has to patch `Module._resolveFilename` and the guest-facing `Module._cache` / `require.cache` view together; wrapping only `createGuestRequire()` leaves local `require()` inside loaded `.cjs` modules free to walk host `node_modules`. +- Host `node:http`, `node:https`, and `node:http2` do not pick up patched `net`/`tls` internals automatically; keep them guest-owned by wrapping the host client/server surface and forwarding guest sockets into the host server via `connection`/`secureConnection` exactly once. +- `AGENT_OS_ALLOWED_NODE_BUILTINS` is the shared source of truth for guest Node capability gating, but permissioned top-level JavaScript executions on Node v24 still need `--allow-worker` because `register(loader)` spins an internal loader worker; keep that runtime requirement separate from guest `worker_threads` exposure, and keep child-process permission args aligned with the allowed builtin set. +- Permissioned Pyodide host launches need the same `--allow-worker` treatment as JavaScript in `crates/execution/src/python.rs`; Node's internal loader worker is a host runtime requirement there too, not guest `worker_threads` exposure. +- Guest-owned Node builtin polyfills that need both ESM and CJS coverage should be wired in three places together: loader import rewriting/asset resolution, the generated Node runner’s `process.getBuiltinModule` and `Module._load` hooks, and the core bridge’s default allowlist in `packages/core/src/sidecar/native-kernel-proxy.ts`. +- When a Node builtin port is landing in phases, inherit untouched exports from a snapped host module and override only the RPC-backed surface for the current story; this keeps helper APIs working while the follow-on stories replace the remaining host-backed entrypoints. +- Node `net` server behavior is split between the guest runner in `crates/execution/src/node_import_cache.rs` and the sidecar TCP state machine in `crates/sidecar/src/service.rs`; changes to `listen`, `getConnections`, backlog handling, or close semantics need updates and regressions on both sides. +- When a guest Node networking port stops using real host listeners, mirror that state in `crates/sidecar/src/service.rs` `ActiveProcess` tracking and consult it from `find_listener`/socket snapshot queries before falling back to `/proc/[pid]/net/*`; procfs only sees host-owned sockets, not sidecar-managed polyfill listeners. +- UDP guest ports follow the same rule as TCP listeners: keep sidecar-managed datagram sockets on `ActiveProcess`, create the real `UdpSocket` lazily on `bind()`/first `send()`, and answer `find_bound_udp` from that tracked state because `/proc/[pid]/net/udp*` never sees sidecar-owned sockets. +- Guest Node `tls` should stay layered on the guest `net` polyfill: client connections pass a preconnected guest socket into `tls.connect({ socket })`, and TLS servers should wrap accepted guest sockets with `new TLSSocket(..., { isServer: true })` and treat the wrapped socket's `secure` event as `secureConnection`. +- Pyodide guest hardening that must not rewrite user code belongs in `crates/execution/src/node_import_cache.rs` as a `pyodide.runPython(...)` bootstrap in the embedded Python runner, installed after package preloads and before `runPythonAsync()`. +- The Pyodide host Node process is hardened with Node `--permission` in `crates/execution/src/python.rs`; keep its read allowlist scoped to the import-cache root, compile-cache dir, Pyodide bundle, and sandbox cwd, and keep writes limited to the cache paths plus sandbox cwd. +- Node guest env hardening in `crates/execution/src/node_import_cache.rs` should snapshot `AGENT_OS_*` control vars first, then replace `process.env` with a filtered proxy so runtime internals keep working while guest enumeration/access stays scrubbed; when `node:module` is denied, bootstrap the runner via `process.getBuiltinModule('node:module')` instead of importing it through the guest loader. +- Node guest process virtualization in `crates/execution/src/node_import_cache.rs` should snapshot the host `process.cwd()` before hardening, use that snapshot for internal module resolution/`createRequire(...)`, and derive guest-visible paths from `AGENT_OS_GUEST_PATH_MAPPINGS` for user-facing `process.*` APIs. +- Guest-visible `process` identity in `crates/execution/src/node_import_cache.rs` should be virtualized through a `globalThis.process` proxy after bootstrap setup, while `require('node:process')` and `process.getBuiltinModule('node:process')` are routed back to that same proxy; keep internal host-only values in snapped constants like `HOST_EXEC_PATH`. +- In the generated Node runner, host-only builtin lookups needed for bootstrap/hardening should go through snapped `hostRequire(...)` rather than guest-visible ESM imports, and wrapped `process` methods that return `this` must translate the captured host target back to `guestProcess` after the proxy swap. +- Nested JavaScript child executions should propagate host Node `--permission` escalation via explicit `AGENT_OS_PARENT_NODE_ALLOW_*` markers in `crates/execution/src/javascript.rs` and `crates/execution/src/node_import_cache.rs`; do not infer child `--allow-worker` or `--allow-child-process` from `AGENT_OS_ALLOWED_NODE_BUILTINS` alone, because top-level loader requirements and child inheritance are different concerns. +- `wrapChildProcessModule` in `crates/execution/src/node_import_cache.rs` can only sandbox `exec`/`execSync` safely for simple Node-runtime commands; parse shell-free argv and delegate to `execFile`, but deny arbitrary shell strings because host shells bypass Node `--permission`. +- Guest-visible module path scrubbing in `crates/execution/src/node_import_cache.rs` has to cover both the ESM loader and the generated Node runner: translate `error.message`, `error.stack`, and `requireStack`, and import guest entrypoints through guest-mapped file URLs so top-level stack traces never start on host paths. +- Execution control data that affects host state should move over the shared `AGENT_OS_CONTROL_PIPE_FD` side channel in `crates/execution/src/node_process.rs`; if a runtime still surfaces compatible debug/control prefixes, strip matching guest `stderr` lines before exposing them so forged prefixes never drive host behavior. +- Guest-visible signal registration that the sidecar needs to observe should ride the shared control pipe from `crates/execution/src/node_import_cache.rs` into `JavascriptExecutionEvent::SignalState` and `crates/sidecar/src/service.rs` `vm.signal_states`; keeping the last snapshot after exit avoids fast-process query races. +- The JavaScript sync syscall bridge in `crates/execution/src/node_import_cache.rs` should keep request writes on the guest main thread and use a worker only for blocking response reads plus `SharedArrayBuffer` wakeups; under the current Node permission model, worker-thread writes to the inherited request FD fail with `EBADF`. +- Guest Node `fs` and `fs/promises` polyfills now share the same JavaScript sync-RPC transport; async methods should dispatch as `fs.promises.*` RPC calls, and guest-visible `readdir` results must filter the kernel VFS `.` / `..` entries back out to match Node semantics. +- Non-fd guest `fs` sync methods should be overridden onto the wrapped module via a dedicated sync-RPC helper in `crates/execution/src/node_import_cache.rs`; keep fd/stream APIs on the translated host module until their kernel-backed port is implemented, and add matching `fs.*Sync` dispatch arms in `crates/sidecar/src/service.rs`. +- Guest Node `fs` fd/stream support should stay on the shared sync-RPC bridge end-to-end: `open/read/write/close/fstat` and `createReadStream`/`createWriteStream` all use the same RPC surface, while runner-internal sync-RPC pipe writes must use snapped host `node:fs` bindings because `syncBuiltinModuleExports(...)` mutates builtin modules for guest code. +- Synthetic guest `ChildProcess` handles in `crates/execution/src/node_import_cache.rs` must stay ref'd by default and only `unref()` their poll timer when guest code explicitly asks; otherwise `exec()`/top-level `await` can terminate early with Node's unsettled-top-level-await exit. +- When a newly allowed Node builtin still exposes bypass-capable host-owned helpers or constructors, replace those exports with guest shims or explicit unsupported stubs before adding the builtin to `DEFAULT_ALLOWED_NODE_BUILTINS`; `dns.Resolver` and `dns.promises.Resolver` are the model for this rule. +- Registry external-network tests should stay behind `AGENTOS_E2E_NETWORK=1`, preflight host connectivity before enabling CI coverage, and retry the in-VM outbound command so transient internet issues skip or self-heal instead of creating flaky regressions. + +Started: Sat Apr 4 07:06:17 PM PDT 2026 +--- +## 2026-04-05 12:29:56 PDT - US-081 +- What was implemented +- Derived the `allow_wasi` argument for `harden_node_command(...)` from `StartWasmExecutionRequest.permission_tier` in `crates/execution/src/wasm.rs`, so `Isolated` launches no longer get host Node `--allow-wasi` while the other tiers keep WASI enabled. +- Added a permission-flags regression in `crates/execution/tests/permission_flags.rs` that runs all four WASM permission tiers and asserts `--allow-wasi` is only present for `ReadOnly`, `ReadWrite`, and `Full`. +- Added a reusable WASM permission-tier note to `CLAUDE.md`. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/permission_flags.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: WASM permission tiers must gate both host Node permission flags and guest-side WASI preopens; changing only the guest runtime layer still leaves `Isolated` executions with host WASI access. + - Gotchas encountered: WASM warmup caching reuses identical module paths across contexts, so permission-flag tests that expect one prewarm plus one execution per launch need tier-specific module paths to avoid collapsing invocations. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-execution`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, and `cargo test -p agent-os-execution --test wasm -- --test-threads=1` pass after this change. +--- +## 2026-04-05 10:33:43 PDT - US-072 +- What was implemented +- Added host-side JavaScript sync RPC timeout tracking in `crates/execution/src/javascript.rs`, so pending requests now auto-expire with `ERR_AGENT_OS_NODE_SYNC_RPC_TIMEOUT` instead of waiting forever for a sidecar response. +- Replaced direct sync-RPC response pipe writes with a bounded async writer queue and timeout-based enqueueing so slow guest reads cannot block sidecar request handling indefinitely. +- Updated `crates/sidecar/src/service.rs` to ignore stale post-timeout sync-RPC replies instead of surfacing a second failure after the timeout response has already been sent. +- Added focused execution regressions for timeout-response emission and bounded response-queue backpressure. +- Files changed +- `AGENTS.md` +- `crates/execution/src/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: JavaScript sync RPC should mirror Python VFS RPC timeout handling at the execution-host layer, but its response path also needs a bounded async queue because the sidecar thread can otherwise block on a slow-reading guest pipe. + - Gotchas encountered: Once the host-side timeout has emitted an error response, later sidecar attempts to reply will race and surface `sync RPC request ... is no longer pending`; that stale response needs to be ignored in `crates/sidecar/src/service.rs`. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-execution`, `cargo check -p agent-os-sidecar`, `cargo test -p agent-os-execution javascript::tests -- --nocapture`, and `cargo test -p agent-os-sidecar javascript_sync_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --nocapture` pass after this change. +--- +## 2026-04-05 11:31:04 PDT - US-076 +- What was implemented +- Added a shared `MAX_PATH_LENGTH` / `validate_path(...)` guard in `crates/kernel/src/vfs.rs` and threaded it through the permission wrapper so overlong guest paths now fail closed with `ENAMETOOLONG` before permission fallback or generic lookup errors. +- Tightened `MemoryFileSystem::resolve_path_with_options(...)` in `crates/kernel/src/vfs.rs` so intermediate non-directory components now raise `ENOTDIR` during traversal instead of falling through to `ENOENT`. +- Added an API-surface regression in `crates/kernel/tests/api_surface.rs` that covers `EISDIR`, `ENOTDIR`, `ENAMETOOLONG`, and `EROFS`, and shortened the deep-tree fixture in `crates/kernel/tests/root_fs.rs` so the existing overlay depth-limit test still exercises snapshot depth rather than the new path-length guard. +- Files changed +- `crates/kernel/src/permissions.rs` +- `crates/kernel/src/vfs.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/root_fs.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Filesystem errno hardening spans both `PermissionedFileSystem` and the underlying VFS; path-length checks belong in the permission layer for fast-fail behavior, while semantic traversal errors like `ENOTDIR` belong in `MemoryFileSystem`. + - Gotchas encountered: Existing deep-tree regressions can accidentally start testing `ENAMETOOLONG` once path-length guards land, so depth-focused fixtures should keep segment names intentionally short. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel`, `cargo test -p agent-os-kernel --test api_surface filesystem_operations_return_linux_errno_values_for_common_failures -- --exact`, `cargo test -p agent-os-kernel --test root_fs overlay_rename_rejects_directory_trees_that_exceed_snapshot_depth_limit -- --exact`, and `cargo test -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 02:40:37 PDT - US-033 +- What was implemented +- Added filesystem resource accounting in `crates/kernel/src/resource_accounting.rs`, including default `max_filesystem_bytes` / `max_inode_count` limits and a recursive usage walker that measures visible bytes plus unique inodes. +- Hardened kernel filesystem mutation paths in `crates/kernel/src/kernel.rs` so `write_file`, `create_dir`, `mkdir`, `symlink`, `truncate`, `fd_pwrite`, `fd_write`, and `O_CREAT` / `O_TRUNC` open flows enforce the new limits and fail with `ENOSPC` before resize-driven growth. +- Updated sidecar metadata parsing in `crates/sidecar/src/service.rs` so sparse VM metadata preserves `ResourceLimits::default()` and only overrides resource keys that are explicitly present. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/resource_accounting.rs` +- `crates/kernel/tests/resource_accounting.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Filesystem resource accounting should scan the raw filesystem beneath `PermissionedFileSystem` / `DeviceLayer`; using the permission-wrapped view couples internal accounting to guest read policy and special `/dev/*` entries. + - Gotchas encountered: Sidecar resource parsing has to preserve `ResourceLimits::default()` when metadata is sparse, or new default caps like filesystem bytes/inodes get silently disabled. +- Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test resource_accounting -- --test-threads=1`, `cargo test -p agent-os-kernel --test api_surface kernel_fd_surface_supports_open_seek_positional_io_dup_and_dev_fd_views -- --exact`, `cargo test -p agent-os-sidecar service::tests::parse_resource_limits_reads_filesystem_limits -- --exact`, and `cargo check -p agent-os-kernel -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 02:56:51 PDT - US-034 +- What was implemented +- Extended `ResourceLimits` with socket/connection caps, configurable blocking read timeout, and reserved WASM runtime limit fields; kernel `fd_read()` now uses bounded pipe/PTTY reads so leaked write ends return `EAGAIN` instead of hanging forever. +- Hardened the native sidecar to parse the new resource metadata, count network resources across the active process tree, reject oversized framed stdio prefixes before allocation, and thread `max_wasm_*` limits into the execution layer through reserved `AGENT_OS_WASM_*` env keys. +- Added execution-side WASM enforcement in `crates/execution/src/wasm.rs`: configurable fuel budget timeout, configurable Node stack-size flag, and pre-spawn module validation for declared memory maximums when a memory cap is set. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/execution/tests/wasm.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/pipe_manager.rs` +- `crates/kernel/src/pty.rs` +- `crates/kernel/src/resource_accounting.rs` +- `crates/kernel/tests/resource_accounting.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/src/stdio.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: WASM runtime hardening is split across `ResourceLimits`, sidecar env injection, and execution-time validation; changing only one layer silently leaves the limit unenforced. + - Gotchas encountered: Node's `WASI.start()` expects the guest module to export `memory`, so even timeout-only WASM regression fixtures need a memory export to exercise the runtime path cleanly. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test resource_accounting -- --test-threads=1`, `cargo test -p agent-os-execution --test wasm -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, `cargo test -p agent-os-sidecar parse_resource_limits_reads_filesystem_limits -- --exact`, `cargo test -p agent-os-sidecar stdio::tests::read_frame_rejects_oversized_prefix_before_allocating_payload -- --exact`, and `cargo check -p agent-os-kernel -p agent-os-execution -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 01:10:03 PDT - US-028 +- What was implemented +- Added host-side `cwd` validation in `crates/sidecar/src/service.rs` so `ExecuteRequest.cwd` is normalized against the VM sandbox root and rejected when it escapes, including the `cwd=/` host-root case called out in the PRD. +- Threaded the VM sandbox root into the Node permission setup for JavaScript, Python, and WASM host launches so `--allow-fs-read` and `--allow-fs-write` stay pinned to the sandbox root even when the runtime starts in a nested working directory. +- Added sidecar security regressions that verify both the rejection path and the permission-flag scoping behavior with a fake Node binary. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/wasm.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/security_hardening.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: The execution engines already separate `current_dir(...)` from permission flag construction, so sidecar hardening can flow sandbox metadata through a reserved env key without changing the public execution request types. +- Gotchas encountered: The Rust permission-flag tests mutate `AGENT_OS_NODE_BINARY`, so they need single-threaded execution (`-- --test-threads=1`) to avoid test-process env races. +- Useful context: `cargo test -p agent-os-sidecar --test security_hardening execute_rejects_cwd_outside_vm_sandbox_root -- --exact`, `cargo test -p agent-os-sidecar --test security_hardening execute_scopes_node_permission_flags_to_vm_sandbox_root -- --exact`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, and `cargo check -p agent-os-sidecar -p agent-os-execution` all pass after this change. +--- +## 2026-04-05 07:36:14 PDT - US-055 +- What was implemented +- Added SSRF validation to sidecar JavaScript DNS/TCP handling in `crates/sidecar/src/service.rs`, blocking private/link-local IPv4 and IPv6 ranges from `dns.lookup` / `dns.resolve*` results before they reach guest code. +- Hardened `net.connect` target selection so literal or DNS-resolved loopback/private addresses fail closed with `EACCES`, while VM-owned loopback listeners and explicitly exempt host loopback ports from `AGENT_OS_LOOPBACK_EXEMPT_PORTS` still connect successfully. +- Added focused sidecar regressions for blocked metadata/loopback targets and updated existing DNS/permission callback tests to prove the exempt-port path still works. +- Files changed +- `AGENTS.md` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Sidecar-only bootstrap settings such as `AGENT_OS_LOOPBACK_EXEMPT_PORTS` must be read from `CreateVmRequest.metadata` `env.*` entries rather than `vm.guest_env`, because env permissions can legally filter `guest_env` down to nothing. + - Gotchas encountered: The sidecar lib tests that materialize Node import-cache runners are reliable when run sequentially, but parallel spot-checks can trip temp-cache races and report `register.mjs` missing. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-sidecar javascript_network_ssrf_protection_blocks_private_dns_and_unowned_loopback_targets -- --nocapture`, `cargo test -p agent-os-sidecar javascript_dns_rpc_honors_vm_dns_overrides_and_net_connect_uses_sidecar_dns -- --nocapture`, `cargo test -p agent-os-sidecar javascript_network_permission_callbacks_fire_for_dns_lookup_connect_and_listen -- --nocapture`, `cargo test -p agent-os-sidecar javascript_dns_rpc_resolves_localhost -- --nocapture`, and `cargo fmt --check --all` all pass after this change. +--- +## 2026-04-05 01:17:31 PDT - US-024 +- What was implemented +- Added `PythonExecution::kill()` / `cancel()`, a timeout-aware `wait(timeout)` API, and a `Drop` cleanup path in `crates/execution/src/python.rs` so in-flight Pyodide host processes are explicitly reaped instead of leaking after timeout or early handle drops. +- Tightened Python exit handling so a `PythonExit` control message immediately surfaces `PythonExecutionEvent::Exited`, which keeps polling callers from hanging behind an internal control-only state transition. +- Restored the Python runner's Node permission bootstrap by always keeping `--allow-worker` enabled for the host-side loader worker, and added regressions for wait-time cleanup and explicit kill behavior. +- Files changed +- `AGENTS.md` +- `crates/execution/src/python.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/execution/tests/python.rs` +- `crates/execution/tests/python_prewarm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Pyodide host executions need the same Node internal loader-worker permission as JavaScript hosts, even when guest `worker_threads` remains denied. + - Gotchas encountered: `PythonExecution::poll_event()` should emit `Exited` immediately when the control pipe reports `PythonExit`; returning `None` there looks like a timeout to polling callers and leaves tests waiting on a later synthetic exit. +- Useful context: `cargo test -p agent-os-execution --test python -- --test-threads=1`, `cargo test -p agent-os-execution --test python_prewarm -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, and `cargo check -p agent-os-execution` all pass after this change. +--- +## 2026-04-05 09:17:51 PDT - US-065 +- What was implemented +- Added a new kernel poll surface in `crates/kernel/src/poll.rs` plus `KernelVm::poll_fds(...)` in `crates/kernel/src/kernel.rs` so callers can multiplex across multiple FDs with `POLLIN`, `POLLOUT`, `POLLERR`, `POLLHUP`, and timeout handling for `0`, finite millisecond waits, and `-1`. +- Wired `PipeManager` and `PtyManager` readiness reporting into that syscall, including a shared `PollNotifier` so mixed pipe/PTy waits wake correctly when buffers, waiter queues, or peer-close state changes. +- Added focused kernel regressions covering pipe readability/writability, hangup/error signaling, mixed pipe+PTY polling, and finite timeout behavior. +- Files changed +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/lib.rs` +- `crates/kernel/src/pipe_manager.rs` +- `crates/kernel/src/poll.rs` +- `crates/kernel/src/pty.rs` +- `crates/kernel/tests/poll.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Kernel-wide `poll` support is easiest to keep race-free when every special-FD manager shares one notifier and emits wakeups for buffer changes, waiter-queue changes, and peer-close transitions. + - Gotchas encountered: PTY and pipe waiter queues affect write readiness, not just buffered bytes, so `poll` wakeups have to fire when reads start waiting or time out, not only on reads and writes that move payload bytes. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel`, `cargo test -p agent-os-kernel --test poll -- --nocapture`, and `cargo test -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 09:25:08 PDT - US-066 +- What was implemented +- Updated `crates/kernel/src/process_table.rs` so exiting parents reparent children to PID 1 when available, newly orphaned stopped process groups receive `SIGHUP` followed by `SIGCONT`, and negative-PID group kills target stopped and exited members instead of only running ones. +- Updated process resource enforcement in `crates/kernel/src/resource_accounting.rs` so unreaped zombies count against `max_processes`. +- Added kernel regressions for reparenting to PID 1, orphaned-group signal delivery, negative-PID group kills spanning stopped/zombie members, and zombie-aware process limits. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/src/resource_accounting.rs` +- `crates/kernel/tests/process_table.rs` +- `crates/kernel/tests/resource_accounting.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Process-table exit-path changes should keep reparenting, orphaned stopped-group signaling, and zombie-aware process limits aligned or Linux lifecycle behavior drifts in subtle ways. +- Gotchas encountered: Tests that use PID 1 as an ordinary parent will trigger init-style orphan-group handling, so lifecycle regressions should create a separate synthetic init process when they need a non-init parent in the same session. +- Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test process_table -- --nocapture`, `cargo test -p agent-os-kernel --test resource_accounting -- --nocapture`, `cargo check -p agent-os-kernel`, and `cargo test -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 12:06:52 PDT - US-078 +- What was implemented +- Hardened `crates/execution/src/wasm.rs` so WASM executions resolve the module path once through the same canonicalized path shape used by Node permission setup, reuse that resolved path for validation/warmup/runtime launch, and use a dedicated `AGENT_OS_WASM_PREWARM_TIMEOUT_MS` instead of reusing the execution fuel timeout. +- Switched warmup fingerprints in `crates/execution/src/runtime_support.rs` to `dev:ino`, added a bounded `ensure_materialized_with_timeout(...)` path in `crates/execution/src/node_import_cache.rs` with a 30s default, and added a keepalive cleanup guard so active JS/Python/WASM executions do not lose their materialized runner assets when the engine drops. +- Added focused regressions for canonical symlink resolution, import-cache materialization timeout handling, separate prewarm timeout behavior, and symlink-target warmup invalidation with same-size modules. +- Files changed +- `AGENTS.md` +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/runtime_support.rs` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/wasm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Active runtime handles must retain the `NodeImportCache` cleanup guard until the child exits; otherwise dropping the execution engine can delete `timing-bootstrap.mjs` and related assets during module import. + - Gotchas encountered: The broader `agent-os-execution` benchmark integration test currently fails in an unrelated JavaScript permission scenario (`hot-projected-package-file-import` reading `/root/node_modules/typescript/lib/typescript.js`), so WASM verification is more reliable with `--test-threads=1` plus focused execution suites. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-execution`, `cargo test -p agent-os-execution --lib -- --test-threads=1`, and `cargo test -p agent-os-execution --test wasm -- --test-threads=1` all pass after this change. +--- +## 2026-04-04 19:11:19 PDT - US-001 +- What was implemented +- Hardened the native sidecar default Node builtin allowlist to only kernel-backed/polyfilled modules. +- Expanded the Rust import-cache deny policy to block `os`, `cluster`, `diagnostics_channel`, `module`, and `trace_events` everywhere the guest runtime hardens builtin access. +- Added a regression test that verifies all denied builtin asset shims are materialized and still throw `ERR_ACCESS_DENIED`. +- Files changed +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `crates/execution/src/node_import_cache.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: The sidecar’s default builtin policy is injected through `AGENT_OS_ALLOWED_NODE_BUILTINS`, so JS-side allowlist changes must stay aligned with Rust-side deny shims. +- Gotchas encountered: Repo-wide `pnpm exec tsc -p packages/core/tsconfig.json --noEmit` already fails in unrelated files (`packages/core/src/agent-os.ts`, `packages/core/src/host-tools-server.ts`, `packages/core/src/sidecar/client.ts`), and `pnpm --dir packages/core exec vitest run tests/native-sidecar-process.test.ts` currently fails because `agent-os-sidecar` does not compile due to missing `DiagnosticsRequest` protocol imports. +- Useful context: `cargo test -p agent-os-execution node_import_cache::tests` is the focused verification target for `crates/execution/src/node_import_cache.rs` hardening changes. +--- +## 2026-04-04 19:19:59 PDT - US-002 +- What was implemented +- Added a Python bootstrap blocklist in the embedded Pyodide runner so `import js` and `import pyodide_js` resolve to denied proxy modules before guest code executes. +- Added a real-bundled-Pyodide regression test in `agent-os-execution` that verifies `js.process.env`, `js.require`, `js.process.exit`, `js.process.kill`, and `pyodide_js.eval_code` are inaccessible from Python. +- Updated the sidecar Python security test to assert the blocked Pyodide FFI escape hatches instead of relying on `import js`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/sidecar/tests/python.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Pyodide import interception needs both a `sys.modules` override and a builtin `__import__` wrapper to make blocked module behavior deterministic across Pyodide’s import path. +- Gotchas encountered: Double-underscore helper names inside Python bootstrap classes get name-mangled and can accidentally turn intended `RuntimeError` denials into `NameError`s. +- Useful context: `cargo test -p agent-os-execution node_import_cache::tests` and `cargo test -p agent-os-execution --test python` pass for this change, while `cargo test -p agent-os-sidecar ...` is still blocked by unrelated pre-existing compile errors in `crates/sidecar/src/service.rs` (`DiagnosticsRequest`/`DiagnosticsSnapshotResponse` imports and nearby test code). +--- +## 2026-04-04 19:23:48 PDT - US-003 +- What was implemented +- Enabled Node `--permission` hardening for the Pyodide host process in `crates/execution/src/python.rs`, with the existing read/write allowlists now applied to both prewarm and execution launches. +- Updated the execution permission regression test to assert Python prewarm and exec both receive scoped fs read/write flags for the sandbox cwd, Pyodide bundle, and shared import-cache paths. +- Files changed +- `crates/execution/src/python.rs` +- `crates/execution/tests/permission_flags.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Python execution uses the same `harden_node_command(...)` helper as JS/WASM, so Pyodide permission changes should be tested via `crates/execution/tests/permission_flags.rs` rather than ad-hoc process spawning checks. +- Gotchas encountered: `cargo test -p agent-os-execution` still hits an unrelated pre-existing benchmark failure in `crates/execution/tests/benchmark.rs` on Node `v24.13.0` (`node:module` default export assumption in `runner.mjs`); the focused Python/permission suites pass. +- Useful context: `cargo test -p agent-os-execution --test permission_flags`, `cargo test -p agent-os-execution --test python_prewarm`, and `cargo test -p agent-os-execution --test python` are the relevant passing checks for Pyodide host-process permission changes. +--- +## 2026-04-04 19:31:16 PDT - US-004 +- What was implemented +- Replaced the Node guest runner’s `process.env` with a filtered proxy that strips every `AGENT_OS_*` key from direct access, `in` checks, and enumeration while preserving non-internal guest env vars. +- Snapshotted the runner’s internal `AGENT_OS_*` control vars before the scrub so loader/bootstrap wiring still works, and routed the runner’s own `node:module` access through `process.getBuiltinModule(...)` so it remains compatible with the hardened deny list and Node `v24.13.0`. +- Added execution and sidecar security regression coverage so guest code now verifies `AGENT_OS_GUEST_PATH_MAPPINGS`, `AGENT_OS_NODE_IMPORT_CACHE_PATH`, and other `AGENT_OS_*` keys are hidden. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/tests/security_hardening.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: If the Node runner needs denied builtins such as `node:module` for its own bootstrap, it must grab them from `process.getBuiltinModule(...)` before guest hardening rather than importing them through the guest loader. +- Gotchas encountered: `cargo test -p agent-os-execution --test javascript` is reliable on this branch when run serially with `-- --test-threads=1`; the targeted sidecar security test is still blocked by unrelated pre-existing compile errors in `crates/sidecar/src/service.rs` (`DiagnosticsRequest` / `DiagnosticsSnapshotResponse` imports). +- Useful context: `javascript_execution_ignores_guest_overrides_for_internal_node_env` in `crates/execution/tests/javascript.rs` is the focused regression for hidden `AGENT_OS_*` env keys, and `crates/sidecar/tests/security_hardening.rs` now has the end-to-end assertions ready once the sidecar crate compiles again. +--- +## 2026-04-04 19:38:58 PDT - US-005 +- What was implemented +- Virtualized the Node guest runner’s `process.cwd()` so it returns the guest path derived from `AGENT_OS_GUEST_PATH_MAPPINGS` instead of the host working directory. +- Denied `process.chdir()` from guest code and kept internal loader/`createRequire(...)` resolution pinned to a snapped host cwd so module loading still resolves against the real sandbox path. +- Added a regression test that verifies a mapped host cwd is exposed as `/root` to guest code and that `process.chdir()` throws `ERR_ACCESS_DENIED`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Guest-facing process virtualization should translate host paths back through `AGENT_OS_GUEST_PATH_MAPPINGS`, while internal Node bootstrap code continues using a captured host cwd to avoid breaking resolution. +- Gotchas encountered: `cargo test -p agent-os-execution --test javascript -- --test-threads=1` still shows pre-existing flaky cache-metric assertions (`javascript_execution_invalidates_bare_package_resolution_when_package_metadata_changes`, `javascript_execution_preserves_source_changes_with_cached_resolution`) even though those cases pass when rerun individually; the new cwd regression and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` pass. +- Useful context: The cwd hardening lives in the embedded runner source inside `crates/execution/src/node_import_cache.rs`, not in `crates/execution/src/javascript.rs`, because the visible `process` object is constructed inside the generated `runner.mjs`. +--- +## 2026-04-05 07:10:45 PDT - US-053 +- What was implemented +- Routed `KernelVm::unmount_filesystem` through the same `check_mount_permissions(...)` helper already used by mount operations, so unmounts now require `fs.write` and sensitive unmounts additionally require `fs.mount_sensitive`. +- Added kernel permission regressions covering denied unmounts at `/workspace` and denied sensitive unmounts at `/etc`, asserting the exact permission probes and `EACCES` behavior. +- Files changed +- `crates/kernel/src/kernel.rs` +- `crates/kernel/tests/permissions.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Mount lifecycle policy in the kernel belongs at the `KernelVm` entrypoints; tests can seed mounts through `filesystem_mut().inner_mut().inner_mut().mount(...)` when they need to bypass permission wrappers and assert policy behavior in isolation. +- Gotchas encountered: `MountTable::unmount` still returns `EINVAL` for non-mount targets and `/`; the permission gate must run before that raw mount-table call so denied paths fail closed with `EACCES`. +- Useful context: `cargo test -p agent-os-kernel --test permissions` and `cargo test -p agent-os-kernel mount_table` cover the touched behavior and both pass on this branch. +--- +## 2026-04-05 07:15:15 PDT - US-054 +- What was implemented +- Changed `KernelVmConfig::new()` in `crates/kernel/src/kernel.rs` to use deny-all `Permissions::default()` instead of implicit `allow_all()`. +- Updated kernel test fixtures and browser-sidecar tests that need unrestricted behavior to set `config.permissions = Permissions::allow_all()` explicitly, and added a regression in `crates/kernel/tests/permissions.rs` that verifies the default config denies filesystem writes. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/kernel_integration.rs` +- `crates/kernel/tests/permissions.rs` +- `crates/kernel/tests/resource_accounting.rs` +- `crates/sidecar-browser/tests/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: `KernelVmConfig::new()` should remain deny-all; broad-access fixtures need to opt into `Permissions::allow_all()` explicitly so security-sensitive defaults do not get reintroduced accidentally. + - Gotchas encountered: Kernel tests that only exercise process, PTY, or fd APIs still rely on filesystem and child-process permissions under the hood, so they fail closed after this default flips unless the fixture sets permissions explicitly. + - Useful context: `cargo test -p agent-os-kernel`, `cargo test -p agent-os-sidecar-browser`, and `cargo check -p agent-os-kernel -p agent-os-sidecar-browser` all pass after this change. +--- +## 2026-04-04 19:57:51 PDT - US-006 +- What was implemented +- Virtualized the Node guest runner’s `process.execPath`, `process.argv[0]`, `process.pid`, `process.ppid`, `process.getuid()`, and `process.getgid()` so guest code sees configured virtual values instead of host state. +- Added `AGENT_OS_VIRTUAL_PROCESS_*` execution env hooks so upstream callers can inject kernel-derived process identity without exposing those control vars to guest `process.env`. +- Routed `require('node:process')` and `process.getBuiltinModule('node:process')` back to the same guest `process` proxy, and switched the ESM `child_process` builtin asset to re-export the runner’s wrapped module instead of rebuilding from scrubbed `AGENT_OS_*` env. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Some Node `process` properties are refreshed or non-configurable, so stable guest identity virtualization works more reliably by swapping `globalThis.process` to a proxy after bootstrap setup than by relying on direct property replacement alone. +- Gotchas encountered: `process.argv0` is non-configurable in Node v24, so this story can safely virtualize `process.argv[0]` but not the separate `process.argv0` property without violating Proxy invariants. +- Useful context: `cargo test -p agent-os-execution --test javascript -- --test-threads=1` and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` both pass after this change on the current branch. +## 2026-04-04 20:03:57 PDT - US-007 +- What was implemented +- Hardened the generated Node guest runner to deny `process.on`/`addListener`/`once`/`prepend*` registrations for real OS signal events while leaving non-signal process events usable. +- Denied native addon loading by overriding `Module._extensions['.node']` to throw `ERR_ACCESS_DENIED`, complementing the existing `process.dlopen` denial. +- Added an execution regression test that verifies signal-handler registration is blocked, non-signal listeners still work, and both `process.dlopen(...)` and `require('./addon.node')` fail with `ERR_ACCESS_DENIED`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Runner bootstrap code that needs host-only builtin state should use snapped `hostRequire(...)`, because guest-loader ESM imports can be redirected into denied builtin assets once hardening is active. +- Gotchas encountered: Wrapped `process` EventEmitter methods return the host `process` object by default; after the guest proxy swap they need to remap that return value back to `guestProcess` or user code will observe the wrong identity. +- Useful context: `cargo test -p agent-os-execution --test javascript -- --test-threads=1` and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` both pass for this story on the current branch. +--- +## 2026-04-04 20:13:48 PDT - US-008 +- What was implemented +- Replaced the guest `child_process.exec` and `execSync` pass-throughs in `wrapChildProcessModule` with a shell-free parser that routes simple Node-runtime commands through `execFile`/`execFileSync`, preserving the existing guest path translation and Node `--permission` injection logic. +- Denied unsupported shell strings for `exec`/`execSync` so commands like `cat /etc/passwd` no longer fall through to a real host shell. +- Added a regression test that verifies both async `exec` and sync `execSync` launch hardened Node children, and that direct shell access is rejected with `ERR_ACCESS_DENIED`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: For guest `child_process.exec` compatibility, preserve the callback-based API by delegating supported commands to `execFile` and wrapping denied async callbacks rather than inventing a parallel child-process path. +- Gotchas encountered: `util.promisify(exec)` depends on Node’s built-in custom promisify hook, so execution regressions should exercise the raw callback contract instead of assuming a `{ stdout, stderr }` promise shape from wrapped `exec`. +- Useful context: `cargo test -p agent-os-execution --test javascript -- --test-threads=1` and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` both pass after this change. +--- +## 2026-04-04 20:22:11 PDT - US-009 +- What was implemented +- Added host-to-guest path scrubbing helpers to the embedded Node ESM loader and guest runner so `require.resolve()` returns guest-visible paths and guest-facing error surfaces rewrite host paths out of `message`, `stack`, `path`, `filename`, `url`, and `requireStack`. +- Switched guest entrypoint/bootstrap imports to use guest-mapped file URLs, which keeps top-level loader/parser stack traces anchored to guest paths instead of the real sandbox path. +- Added JavaScript regressions covering guest-visible `require.resolve()` results, translated CJS module-not-found errors, and translated top-level loader stack traces. +- Files changed +- `crates/execution/src/node_import_cache.rs` + +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Host-path scrubbing for Node guests is incomplete unless both the generated loader and the generated runner rewrite errors; CJS `require(...)` and top-level ESM imports leak through different surfaces. + - Gotchas encountered: The repo root hit `ENOSPC` during the broader JavaScript suite because thousands of stale `/tmp/agent-os-node-import-cache-*` directories had accumulated; clearing those temp caches restored the real test signal. + - Useful context: `cargo test -p agent-os-execution --test javascript -- --test-threads=1` and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` both pass after this change. +--- +## 2026-04-04 20:38:57 PDT - US-010 +- What was implemented +- Added a shared `AGENT_OS_CONTROL_PIPE_FD` execution side channel in `agent-os-execution` and routed Node import-cache metrics, Pyodide exit reporting, and WASM signal registrations through structured control messages instead of parsing guest `stderr`. +- Updated the JavaScript, Python, and WASM execution wrappers to ignore guest-forged control prefixes on `stderr`, while still surfacing trusted debug metrics from the control pipe where tests expect them. +- Replaced sidecar signal-state updates with structured WASM execution events and updated regression coverage so forged `stderr` no longer mutates signal state while real WASM `proc_sigaction` registrations still do. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/node_process.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/javascript.rs` +- `crates/execution/tests/python.rs` +- `crates/execution/tests/wasm.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/socket_state_queries.rs` +- `crates/sidecar/tests/support/mod.rs` +- `packages/core/tests/native-sidecar-process.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Cross-runtime control flow between spawned Node hosts and Rust should use structured JSON lines over `AGENT_OS_CONTROL_PIPE_FD`, then be translated back into runtime-specific events at the Rust boundary instead of teaching the sidecar to parse text streams. +- Gotchas encountered: `agent-os-sidecar` remains blocked by pre-existing compile failures unrelated to this story (`DiagnosticsRequest`/`DiagnosticsSnapshotResponse` imports in `crates/sidecar/src/service.rs` plus existing lib-test mismatches around `authenticate_and_open_session(...)`), so Rust sidecar tests and the `packages/core` real-sidecar spec still cannot build on this branch. +- Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_ignores_forged_import_cache_metrics_written_to_stderr -- --exact`, `cargo test -p agent-os-execution --test python python_execution_ignores_forged_exit_control_written_to_stderr -- --exact`, and `cargo test -p agent-os-execution --test wasm wasm_execution_emits_signal_state_from_control_channel -- --exact` all pass after this change. +--- +## 2026-04-04 20:51:16 PDT - US-011 +- What was implemented +- Added `allowedNodeBuiltins?: string[]` to `AgentOsOptions` and threaded it into `NativeSidecarKernelProxy` so guest Node executions can override the hardened default builtin allowlist per VM. +- Gated Node `--allow-worker` permission injection off the resolved builtin allowlist in both Rust host launchers and the generated `wrapChildProcessModule(...)` bridge, so worker permissions only appear when `worker_threads` is explicitly allowed. +- Added a `packages/core` bridge regression that verifies the configured allowlist reaches guest execution env, plus a Rust permission-flags regression for the `worker_threads`/`--allow-worker` linkage. +- Fixed a pre-existing `packages/core` typecheck typo in `AgentOs.mkdir()` (`this.kernel` -> `this.#kernel`) so `pnpm --dir packages/core exec tsc --noEmit` passes again. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/node_process.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/permission_flags.rs` +- `packages/core/src/agent-os.ts` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/tests/allowed-node-builtins.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Per-VM Node builtin overrides are owned by the JS bridge (`NativeSidecarKernelProxy`) rather than VM-create metadata; tests can validate the flow by mocking `NativeSidecarProcessClient.execute(...)` and inspecting the emitted guest env without compiling the Rust sidecar binary. +- Gotchas encountered: `packages/core` end-to-end VM tests that call `AgentOs.create()` still trip the branch’s unrelated `agent-os-sidecar` compile failure in `crates/sidecar/src/service.rs`, so bridge-level tests are the reliable verification path until that crate is fixed. +- Useful context: `cargo test -p agent-os-execution --test permission_flags node_permission_flags_only_allow_workers_when_worker_threads_is_enabled -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `pnpm --dir packages/core exec tsc --noEmit`, and `pnpm --dir packages/core exec vitest run tests/allowed-node-builtins.test.ts` all pass after this change. +--- +## 2026-04-04 21:17:24 PDT - US-012 +- What was implemented +- Added a SharedArrayBuffer-backed JavaScript sync RPC bridge in `agent-os-execution` that surfaces synchronous guest Node fs requests as structured Rust events and accepts structured success/error responses over dedicated sync-RPC pipes. +- Wired the sidecar execution loop to dispatch `fs.readFileSync`, `fs.writeFileSync`, `fs.statSync`, `fs.readdirSync`, and `fs.mkdirSync` through the kernel VFS, and added focused execution and sidecar regressions that exercise the bridge end to end. +- Files changed +- `crates/execution/src/benchmark.rs` +- `crates/execution/src/javascript.rs` +- `crates/execution/src/lib.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/socket_state_queries.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Permissioned Node v24 guest launches that bootstrap through `register(loader)` need `--allow-worker` even when guest `worker_threads` remains denied, because Node uses an internal loader worker before user code runs. +- Gotchas encountered: The SAB bridge only worked reliably once the child sync-RPC pipe fds stayed alive through `spawn()`, and the guest had to keep request writes on the main thread while a worker blocked on the response pipe; trying to write requests from the worker hit `EBADF`. +- Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_runs_bootstrap_and_streams_stdio -- --exact`, `cargo test -p agent-os-execution --test javascript javascript_execution_surfaces_shared_array_buffer_sync_rpc_requests -- --exact`, `cargo test -p agent-os-execution --test permission_flags node_permission_flags_allow_workers_for_internal_javascript_loader_runtime -- --exact`, `cargo check -p agent-os-sidecar`, and `cargo test -p agent-os-sidecar service::tests::javascript_sync_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --exact` all pass after this change. +--- +## 2026-04-04 21:28:39 PDT - US-013 +- What was implemented +- Added a guest-owned `node:os` polyfill in `crates/execution/src/node_import_cache.rs` that virtualizes hostname, CPU, memory, loopback networking, home directory, user info, and blocks host-priority mutation via `os.setPriority()`. +- Routed `node:os` through the generated loader asset pipeline plus the runner’s `require(...)`/`process.getBuiltinModule(...)` hooks, removed `os` from the denied builtin asset set, and enabled it in the core bridge’s default Node builtin allowlist. +- Added regression coverage for the new builtin asset materialization, direct JavaScript execution of the virtualized `os` surface, the default allowlist propagation, and updated the repo instruction tables so the `os` status is no longer stale. +- Files changed +- `CLAUDE.md` +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/tests/allowed-node-builtins.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Guest builtin ports like `os` need both import-cache asset coverage and runtime hook coverage; only doing one leaves either ESM imports or CJS/builtin lookups leaking back to the host module. +- Gotchas encountered: The Rust `JavascriptExecutionEngine` does not supply the core bridge’s default builtin allowlist on its own, so direct execution tests must pass `AGENT_OS_ALLOWED_NODE_BUILTINS` explicitly when they exercise opt-in builtins like `os`. +- Useful context: `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript javascript_execution_virtualizes_os_module -- --exact`, `pnpm --dir packages/core exec tsc --noEmit`, and `pnpm --dir packages/core exec vitest run tests/allowed-node-builtins.test.ts` all pass after this change. +--- +## 2026-04-04 21:42:38 PDT - US-014 +- What was implemented +- Routed the `fs.promises` story surface through the Node RPC bridge by adding guest-path normalization, stat proxies, encoding/time normalization, and `fs.promises.*` dispatch in both the generated Node runner and the materialized `node:fs` builtin asset. +- Extended the sidecar JavaScript RPC handler to service `fs.promises.readFile`, `writeFile`, `stat`, `lstat`, `readdir`, `mkdir`, `rmdir`, `unlink`, `rename`, `copyFile`, `chmod`, `chown`, `utimes`, and `access` against the kernel VFS, including Node-facing `readdir` filtering for `.` and `..`. +- Enabled the JavaScript sync-RPC bridge for guest Node executions by default so `fs.promises` no longer depends on opt-in env wiring, and added focused execution and sidecar regressions for the async path alongside the existing sync bridge checks. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Guest Node `fs.promises` rides the same JavaScript sync-RPC transport as sync `fs` today; new async VFS methods should use `fs.promises.*` method names and only resolve guest paths, leaving host-path translation to the sidecar/kernel boundary. +- Gotchas encountered: The direct `JavascriptExecutionEngine` test harness maps the guest cwd to `/`, not `/workspace`, so relative-path RPC assertions need to match `/note.txt`/`/subdir` rather than the sidecar VM’s mounted workspace paths. +- Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_routes_fs_promises_through_sync_rpc -- --exact`, `cargo test -p agent-os-execution --test javascript javascript_execution_surfaces_shared_array_buffer_sync_rpc_requests -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::javascript_fs_promises_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --exact`, and `cargo test -p agent-os-sidecar service::tests::javascript_sync_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --exact` all pass after this change. +--- +## 2026-04-04 22:17:44 PDT - US-016 +- What was implemented +- Routed guest `fs.open/openSync`, `read/readSync`, `write/writeSync`, `close/closeSync`, and `fstat/fstatSync` through the shared JavaScript sync-RPC bridge and sidecar kernel fd APIs. +- Added RPC-backed `createReadStream` and `createWriteStream` implementations plus explicit `fs.watch`/`fs.watchFile` stubs that throw `ERR_AGENT_OS_FS_WATCH_UNAVAILABLE`. +- Hardened the generated Node runner and `node:fs` builtin asset so ESM, CJS, warmup, and internal sync-RPC plumbing all keep using the correct host-vs-guest fs bindings. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Guest fd APIs and fs streams should share the same sync-RPC surface as path-based fs methods; once the runner mutates builtin exports for guests, any internal pipe/control writes must keep snapped host `node:fs` bindings to avoid recursive RPC calls. + - Gotchas encountered: The materialized `node:fs` ESM asset can run during prewarm before guest hardening is installed, so it needs a safe fallback to `process.getBuiltinModule('node:fs')` instead of assuming the guest wrapper globals already exist. + - Useful context: `cargo test -p agent-os-execution --test javascript -- --test-threads=1`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::javascript_fd_and_stream_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --exact`, and `pnpm --dir packages/core exec tsc --noEmit` all pass after this change. +--- +## 2026-04-04 21:52:54 PDT - US-015 +- What was implemented +- Ported the non-fd guest `fs` sync surface onto the SharedArrayBuffer sync-RPC bridge in `crates/execution/src/node_import_cache.rs`, covering `readFileSync`, `writeFileSync`, `statSync`, `lstatSync`, `readdirSync`, `mkdirSync`, `existsSync`, `readlinkSync`, `symlinkSync`, `linkSync`, `renameSync`, `unlinkSync`, `rmdirSync`, plus sync aliases for `access`, `copyFile`, `chmod`, `chown`, and `utimes`. +- Added matching `fs.*Sync` dispatch arms in `crates/sidecar/src/service.rs` so those guest calls execute against the kernel VFS, and expanded the focused execution/sidecar regressions to verify both request surfacing and end-to-end kernel behavior. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Sync `fs` methods should share the same JS bridge as `fs.promises`, but they need a separate override layer on the wrapped module so fd/stream APIs can remain on the old host-backed path until US-016 lands. + - Gotchas encountered: `readdirSync({ withFileTypes: true })` cannot reuse the old synthetic dirent helper for RPC-backed paths; it needs per-entry `lstatSync` round-trips to reconstruct Dirent-like type methods without falling back to host `node:fs`. + - Useful context: `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript javascript_execution_surfaces_shared_array_buffer_sync_rpc_requests -- --exact`, `cargo test -p agent-os-execution --test javascript javascript_execution_redirects_computed_node_fs_imports_through_builtin_assets -- --exact`, `cargo test -p agent-os-sidecar service::tests::javascript_sync_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --exact`, `cargo check -p agent-os-execution`, and `cargo check -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-04 22:48:51 PDT - US-017 +- What was implemented +- Replaced the guest `wrapChildProcessModule(...)` path-translating wrapper with an RPC-backed `child_process` polyfill that routes `spawn`, `exec`, `execFile`, `spawnSync`, `execSync`, `execFileSync`, and `fork` through the shared Agent OS sync bridge. +- Added sidecar child-process RPC handlers that resolve nested guest commands into kernel-managed runtime launches, stream stdio through synthetic `ChildProcess` objects, route `.kill()` through kernel/runtime signaling, and tear down nested children when the parent process exits. +- Added focused execution and sidecar regressions covering callback-based `exec`/`execSync` behavior and nested `node` child processes reading the VM filesystem through the kernel. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: The guest `child_process` polyfill now rides the same JS sync-RPC bridge as `fs`, with async lifecycle methods split into `child_process.spawn`, `child_process.poll`, `child_process.write_stdin`, `child_process.close_stdin`, and `child_process.kill` on the sidecar. +- Gotchas encountered: Synthetic child processes must keep their polling timer ref'd until `child.unref()` is called, and `exec`/`execFile` should default collected output to `utf8` strings to match Node's callback API. +- Useful context: `cargo check -p agent-os-sidecar`, `cargo test -p agent-os-execution --test javascript javascript_execution_hardens_exec_and_execsync_child_process_calls -- --exact`, and `cargo test -p agent-os-sidecar --lib javascript_child_process_rpc_spawns_nested_node_processes_inside_vm_kernel -- --nocapture` all pass for this story. +--- +## 2026-04-04 23:18:04 PDT - US-018 +- What was implemented +- Added a guest `node:net` builtin asset and runner polyfill that routes `net.connect` and `net.createConnection` through the shared JavaScript sync-RPC bridge while preserving untouched host `net` helpers for APIs owned by later stories. +- Added sidecar-managed TCP socket state with `net.connect`, `net.poll`, `net.write`, `net.shutdown`, and `net.destroy` RPC handlers, including background read polling, close/error propagation, and process-teardown cleanup. +- Added focused regressions for net builtin materialization, guest-side sync-RPC request flow, and a sidecar end-to-end TCP round-trip against a real host `TcpListener`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Partial builtin ports like `net` can safely extend a snapped host module and override only the story-owned RPC surface, which keeps unaffected helpers and later-story APIs available without duplicating the whole builtin up front. +- Gotchas encountered: `Duplex` must be snapped from `node:stream` explicitly in the generated Node runner, and `socket.end(...)` drives both `net.shutdown` and a later `net.destroy`, so guest-side sync-RPC regressions need to account for both lifecycle calls. +- Useful context: `cargo check -p agent-os-execution`, `cargo check -p agent-os-sidecar`, `cargo test -p agent-os-execution ensure_materialized_writes_net_builtin_asset -- --exact`, `cargo test -p agent-os-execution --test javascript javascript_execution_routes_net_connect_through_sync_rpc -- --exact`, and `cargo test -p agent-os-sidecar javascript_net_rpc_connects_to_host_tcp_server -- --exact` all pass after this change. +--- +## 2026-04-04 23:38:01 PDT - US-019 +- What was implemented +- Added a guest `net.createServer`/`net.Server` polyfill in `crates/execution/src/node_import_cache.rs` that routes `listen`, accept polling, `address()`, and `close()` through the existing JavaScript sync-RPC bridge while handing accepted connections off as kernel-backed `net.Socket` instances. +- Extended `crates/sidecar/src/service.rs` with sidecar-managed TCP listener state, `net.listen`/`net.server_poll`/`net.server_close` RPC handlers, accepted-socket promotion into the existing TCP socket table, and listener snapshot lookup from `ActiveProcess` state before the legacy `/proc` fallback. +- Added focused execution and sidecar regressions for `net.createServer`, plus stabilized the socket-state integration test around the new sidecar-managed listener path. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/socket_state_queries.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Sidecar-managed Node listeners must be stored on `ActiveProcess` and surfaced through `find_listener`; once a builtin port stops binding a real host socket, `/proc/[pid]/net/*` no longer reflects guest listener state. +- Gotchas encountered: Mixed socket-state tests can become noisy once an idle `net.createServer` starts long-polling `net.server_poll`; verify or tear down the listener once its snapshot is asserted, and poll signal-state snapshots until the separate control event has been observed. +- Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_routes_net_create_server_through_sync_rpc -- --exact`, `cargo test -p agent-os-sidecar javascript_net_rpc_listens_accepts_connections_and_reports_listener_state -- --exact`, `cargo test -p agent-os-sidecar javascript_net_rpc_connects_to_host_tcp_server -- --exact`, `cargo test -p agent-os-sidecar --test socket_state_queries sidecar_queries_listener_udp_and_signal_state -- --exact`, `cargo check -p agent-os-execution`, and `cargo check -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-04 23:53:49 PDT - US-020 +- What was implemented +- Added a guest `node:dgram` builtin asset and runner polyfill in `crates/execution/src/node_import_cache.rs` that routes `createSocket`, `bind`, `send`, message polling, and `close` through the shared JavaScript sync-RPC bridge while preserving the existing allowlist/deny behavior. +- Extended `crates/sidecar/src/service.rs` with sidecar-managed UDP socket state, `dgram.createSocket`/`dgram.bind`/`dgram.send`/`dgram.poll`/`dgram.close` RPC handlers, lazy host `UdpSocket` binding, and `find_bound_udp` lookup from `ActiveProcess` state before the `/proc` fallback. +- Added focused execution and sidecar regressions for the new `dgram` RPC surface, builtin asset materialization, a real host UDP round-trip, and revalidated the socket snapshot integration test against the sidecar-managed path. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: UDP ports should mirror the staged `net` approach: keep guest-facing JS state unbound until `bind()` or first `send()`, then route all subsequent message delivery through the shared sync-RPC poll loop instead of host Node’s `dgram` module. + - Gotchas encountered: Sidecar-managed UDP bindings never show up in `/proc/[pid]/net/udp*`, so `find_bound_udp` has to consult `ActiveProcess` state first, and the existing mixed socket-state integration test can still flake on the unrelated signal-state polling step and may need a rerun. + - Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_routes_dgram_through_sync_rpc -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::javascript_dgram_rpc_sends_and_receives_host_udp_packets -- --exact`, `cargo test -p agent-os-sidecar --test socket_state_queries sidecar_queries_listener_udp_and_signal_state -- --exact`, and `cargo check -p agent-os-execution -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 00:04:05 PDT - US-021 +- What was implemented +- Added a guest-owned `node:dns` polyfill in `crates/execution/src/node_import_cache.rs` that routes `dns.lookup`, `dns.resolve`, `dns.resolve4`, `dns.resolve6`, and the matching `dns.promises.*` APIs through the JavaScript sync-RPC bridge, while replacing bypass-capable resolver constructors with guest shims instead of inheriting the host module. +- Extended `crates/sidecar/src/service.rs` with `dns.lookup` / `dns.resolve*` RPC handlers backed by sidecar DNS resolution, and added focused execution, sidecar, and import-cache coverage for the new builtin asset and runtime path. +- Added `dns` to the core bridge default allowlist plus a regression in `packages/core/tests/allowed-node-builtins.test.ts` so newly created VMs expose the hardened DNS polyfill by default. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/tests/allowed-node-builtins.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Newly allowed Node builtins must not inherit host-owned constructors or helpers that can bypass the kernel-backed surface; replace them with guest shims or explicit unsupported stubs before exposing the builtin by default. + - Gotchas encountered: `packages/core` verification is blocked in this checkout because the workspace is missing installable dependencies and `pnpm install` fails with `ERR_PNPM_WORKSPACE_PKG_NOT_FOUND` for `@rivet-dev/agent-os`, so the TypeScript and Vitest checks for the updated core files could not be executed here. + - Useful context: `cargo test -p agent-os-execution --test javascript javascript_execution_routes_dns_through_sync_rpc -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-sidecar --lib service::tests::javascript_dns_rpc_resolves_localhost -- --exact`, and `cargo check -p agent-os-execution -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 00:22:29 PDT - US-022 +- What was implemented +- Added a guest-owned `node:tls` builtin in `crates/execution/src/node_import_cache.rs` that rewrites ESM imports to a materialized TLS asset, exposes the module through `process.getBuiltinModule(...)`/`Module._load`, and wraps the existing guest `net` sockets for both `tls.connect` and `tls.createServer`. +- Enabled `tls` in the core bridge default builtin allowlist, added builtin asset/import coverage in `agent-os-execution`, and added a sidecar end-to-end regression that performs a full guest-to-guest TLS handshake over the kernel-backed `net` transport using a self-signed cert. +- Updated the repo instructions so the TLS row and Node builtin porting guidance no longer describe `node:tls` as a host fallthrough. +- Files changed +- `AGENTS.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/tests/allowed-node-builtins.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: TLS can stay guest-owned without new sidecar RPC methods by layering host TLS state over preconnected guest `net` sockets; `tls.connect({ socket })` and server-side `new TLSSocket(socket, { isServer: true, ... })` are the safe entrypoints. +- Gotchas encountered: Server-side wrapped `TLSSocket`s signal handshake readiness on the `secure` event, not `secureConnect`, and the local `packages/core` toolchain is still unavailable in this checkout (`pnpm --dir packages/core exec tsc --noEmit` / `vitest` both fail because the commands are not installed). +- Useful context: `cargo check -p agent-os-execution -p agent-os-sidecar`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript javascript_execution_imports_tls_builtin_when_allowed -- --exact`, and `cargo test -p agent-os-sidecar javascript_tls_rpc_connects_and_serves_over_guest_net -- --exact` all pass after this change. +--- +## 2026-04-05 00:44:32 PDT - US-023 +- What was implemented +- Added guest-owned `http`, `https`, and `http2` builtin wrappers in `crates/execution/src/node_import_cache.rs`, wired them through the loader, Node runner, builtin asset materialization, and `process.getBuiltinModule` / `Module._load` hooks, and exposed them from the default sidecar allowlist. +- Implemented transport-backed `http` / `https` client and server shims on top of the existing guest `net` / `tls` polyfills, plus `http2.connect`, `createServer`, and `createSecureServer` wrappers so the modules no longer fall through to host builtins. +- Added regression coverage for builtin asset materialization, direct JavaScript import of the new modules, and VM-level `http.request` / `http.get` / `http.createServer` plus `https.request` / `https.createServer` behavior. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/tests/allowed-node-builtins.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Host `http` / `https` / `http2` do not automatically honor patched builtin dependencies, so the stable pattern is to keep the host parser/stream implementation but bridge guest sockets into host servers with `connection` / `secureConnection` forwarding and to force client requests through guest-owned `createConnection`. +- Gotchas encountered: When bridging host servers to guest transports, do not register both a transport-server callback and a forwarded event for the same socket event; double delivery replays requests for `http` and triggers `ERR_HTTP2_SOCKET_BOUND` for `http2`. +- Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript javascript_execution_imports_http_builtins_when_allowed -- --exact`, `cargo test -p agent-os-sidecar service::tests::javascript_http_rpc_requests_gets_and_serves_over_guest_net -- --exact`, `cargo test -p agent-os-sidecar service::tests::javascript_https_rpc_requests_and_serves_over_guest_tls -- --exact`, `cargo test -p agent-os-sidecar service::tests::javascript_tls_rpc_connects_and_serves_over_guest_net -- --exact`, `pnpm exec tsc --noEmit` (run from `packages/core` after `pnpm install --ignore-workspace --ignore-scripts` there), and `pnpm exec vitest run tests/allowed-node-builtins.test.ts` (also from `packages/core`) all passed after this change. +--- +## 2026-04-05 01:02:29 PDT - US-027 +- What was implemented +- Serialized `AgentOsOptions.permissions` into sidecar permission descriptors in `packages/core`, passed them through both `create_vm` and `configure_vm`, and added descriptor inference that rejects resource-dependent callbacks the native sidecar cannot faithfully encode. +- Extended the sidecar `CreateVmRequest` schema with permissions, applied a per-VM static permission policy before guest env filtering and kernel bootstrap, and cleared that policy on VM disposal. +- Added focused regression coverage for descriptor serialization, protocol compilation, and sidecar filesystem enforcement under a denied `fs.read` policy. +- Files changed +- `AGENTS.md` +- `crates/sidecar/src/protocol.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/src/stdio.rs` +- `crates/sidecar/tests/connection_auth.rs` +- `crates/sidecar/tests/kill_cleanup.rs` +- `crates/sidecar/tests/protocol.rs` +- `crates/sidecar/tests/python.rs` +- `crates/sidecar/tests/session_isolation.rs` +- `crates/sidecar/tests/stdio_binary.rs` +- `crates/sidecar/tests/support/mod.rs` +- `packages/core/src/agent-os.ts` +- `packages/core/src/sidecar/native-process-client.ts` +- `packages/core/src/sidecar/permission-descriptors.ts` +- `packages/core/tests/sidecar-permission-descriptors.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Native-sidecar permission policy has to be present during `create_vm`; sending it only in `configure_vm` is too late because guest env filtering and bootstrap driver registration both run while the VM is being constructed. + - Gotchas encountered: Denying `fs.write` at VM creation time blocks the sidecar’s own `/bin/*` bootstrap stub registration, so enforcement tests should deny `fs.read` or otherwise leave bootstrap writes allowed unless the kernel gains a post-bootstrap permission swap. + - Useful context: `pnpm --dir packages/core exec vitest run tests/sidecar-permission-descriptors.test.ts`, `pnpm --dir packages/core exec tsc --noEmit`, `cargo test -p agent-os-sidecar --test protocol`, `cargo test -p agent-os-sidecar service::tests::bridge_permissions_map_symlink_operations_to_symlink_access -- --exact`, and `cargo test -p agent-os-sidecar service::tests::create_vm_applies_filesystem_permission_descriptors_to_kernel_access -- --exact` all pass after this change. `pnpm --dir packages/core exec biome format --write ...` could not run in this checkout because `biome` is not installed. +--- +## 2026-04-05 01:29:38 PDT - US-025 +- What was implemented +- Replaced Python execution’s main-thread exit polling with a dedicated waiter thread that watches the shared child handle, joins the stdout/stderr readers before emitting `Exited`, and leaves `kill()`/`Drop` able to terminate the same child safely. +- Updated `poll_event()` to keep consuming control-only and filtered-stderr traffic within the caller timeout so interactive callers no longer see spurious `None` results from internal Python control messages. +- Added bounded stdout/stderr accumulation to `wait()` with a default 1 MiB per-stream cap and a hidden per-execution override via `AGENT_OS_PYTHON_OUTPUT_BUFFER_MAX_BYTES`, plus a regression that verifies truncation under a small cap. +- Files changed +- `AGENTS.md` +- `crates/execution/src/python.rs` +- `crates/execution/tests/python.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Python execution has two different consumers: streaming callers use `poll_event()`, while buffered callers use `wait()`. Internal control/filter noise should be transparent to the streaming API, but buffered `wait()` still needs a hard cap to avoid OOM on large guest output. + - Gotchas encountered: Python still needs direct access to its own `Child` handle for `kill()`/`Drop`, so the waiter thread cannot consume the child the same way JS/WASM do; the safe compromise here is a dedicated waiter loop over the shared handle, with `kill()` continuing to own the final `wait()`. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test python`, `cargo test -p agent-os-execution --test python_prewarm`, and `cargo check -p agent-os-execution -p agent-os-sidecar` pass after this change. `cargo test -p agent-os-execution --test permission_flags` still has an existing invocation-count assumption (`expected ... 5`, got `4`), and `cargo test -p agent-os-sidecar --test python` currently fails in this checkout during bundled Pyodide warmup with `Error [ERR_ACCESS_DENIED]: process.binding`. +--- +## 2026-04-05 01:38:58 PDT - US-030 +- What was implemented +- Added explicit `AGENT_OS_PARENT_NODE_ALLOW_CHILD_PROCESS` and `AGENT_OS_PARENT_NODE_ALLOW_WORKER` markers to JavaScript host launches so nested Node executions only inherit `--allow-child-process` and `--allow-worker` when the parent host process was explicitly allowed to pass them through. +- Updated the generated `child_process` polyfill to forward those markers into nested spawn envs, keeping top-level loader-only worker permission separate from child-process escalation decisions. +- Added a permission-flags regression that simulates nested Node child executions and verifies denied parents do not pass either flag while explicitly allowed parents still do. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/permission_flags.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Nested JavaScript child executions need explicit parent-permission markers for Node `--permission` escalation; `AGENT_OS_ALLOWED_NODE_BUILTINS` alone is not enough because top-level loader workers are a runtime requirement, not a guest capability grant. + - Gotchas encountered: Top-level JavaScript executions still need host `--allow-worker` on Node v24 for `register(loader)`, so child-permission propagation has to be modeled separately instead of reusing the top-level host flag state. + - Useful context: `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1` and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` both pass after this change. +--- +## 2026-04-05 01:46:34 PDT - US-031 +- What was implemented +- Hardened `PermissionedFileSystem` so read/write/stat-like permission checks canonicalize the resolved target path, create/probe checks canonicalize the deepest existing ancestor, `exists()` fails closed instead of leaking denied targets, and `link()` checks both source and destination paths. +- Hardened `MountTable::symlink()` to reject targets that resolve into a different mount, closing the mount-boundary bypass called out in the PRD. +- Added kernel regressions covering symlink-resolved permission subjects, dual-path hardlink checks, fail-closed `exists()`, and cross-mount symlink rejection. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/mount_table.rs` +- `crates/kernel/src/permissions.rs` +- `crates/kernel/tests/mount_table.rs` +- `crates/kernel/tests/permissions.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Permission checks for filesystem paths should split into two cases: existing-target operations use `realpath`, while create/probe operations resolve the deepest existing ancestor and then append the unresolved suffix so missing paths still inherit symlink-aware policy. + - Gotchas encountered: `PermissionedFileSystem::exists()` is part of kernel open/create flows, so it must stay fail-closed for denied or missing paths without surfacing `ENOENT` back to callers that expect a simple boolean probe. + - Useful context: `cargo test -p agent-os-kernel -- --test-threads=1`, `cargo test -p agent-os-kernel --test permissions -- --test-threads=1`, `cargo test -p agent-os-kernel --test mount_table -- --test-threads=1`, and `cargo check -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 02:06:50 PDT - US-038 +- What was implemented +- Added kernel mount authorization so `mount_filesystem` and `mount_boxed_filesystem` now require ordinary write permission on the mount target, and sensitive targets under `/`, `/etc`, and `/proc` also require a separate `fs.mount_sensitive` capability. +- Hardened the Google Drive and S3 native mount plugins against SSRF by validating Google OAuth/API hosts and rejecting private/local S3 endpoint IPs, while still allowing the loopback mock servers used by unit tests under `cfg(test)`. +- Extended sidecar permission serialization/tests to emit `fs.mount_sensitive`, added kernel and sidecar regressions for mount gating, and added plugin regressions for the new URL validation paths. +- Files changed +- `AGENTS.md` +- `Cargo.lock` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/permissions.rs` +- `crates/kernel/tests/permissions.rs` +- `crates/sidecar/Cargo.toml` +- `crates/sidecar/src/google_drive_plugin.rs` +- `crates/sidecar/src/s3_plugin.rs` +- `crates/sidecar/src/service.rs` +- `packages/core/src/sidecar/permission-descriptors.ts` +- `packages/core/tests/sidecar-permission-descriptors.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Sensitive mount checks piggyback on the existing filesystem permission model instead of a separate mount subsystem; ordinary mount checks should reuse the resolved-path write subject, and only the elevated path list needs the extra capability. + - Gotchas encountered: In the Rust sidecar, `ConfigureVm` mounts are applied before `payload.permissions`, so mount-denial tests must seed the VM permission map before dispatch rather than relying on the request body. + - Useful context: `cargo test -p agent-os-kernel --test permissions`, `cargo test -p agent-os-sidecar google_drive_plugin_rejects_`, `cargo test -p agent-os-sidecar s3_plugin_rejects_private_ip_endpoints`, `cargo test -p agent-os-sidecar configure_vm_mounts_require_fs_write_permission`, `cargo test -p agent-os-sidecar configure_vm_sensitive_mounts_require_fs_mount_sensitive_permission`, `cargo test -p agent-os-sidecar create_vm_applies_filesystem_permission_descriptors_to_kernel_access -- --test-threads=1`, `pnpm --dir packages/core exec vitest run tests/sidecar-permission-descriptors.test.ts`, and `pnpm --dir packages/core exec tsc --noEmit` pass. A broader `cargo test -p agent-os-sidecar` run still hits unrelated existing failures in host-dir, Python warmup, child-process worker permissions, and TCP runtime tests on this branch. +--- +## 2026-04-05 02:25:19 PDT - US-041 +- What was implemented +- Propagated per-command WASM permission tiers from `packages/core` into the native sidecar flow, including VM configuration state, top-level execute requests, and JS `child_process` launches that resolve to WASM commands. +- Added runtime enforcement in the WASM execution engine so `read-only` / `isolated` tiers deny mutating WASI filesystem imports, `read-write` keeps workspace writes but still withholds `host_process`, and only `full` tier exposes the `host_process` import surface. +- Added regression coverage for tier propagation and enforcement across the TS proxy layer and Rust WASM execution tests. +- Files changed +- `crates/execution/src/lib.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/execution/tests/wasm.rs` +- `crates/sidecar/src/protocol.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/protocol.rs` +- `crates/sidecar/tests/python.rs` +- `crates/sidecar/tests/security_hardening.rs` +- `crates/sidecar/tests/stdio_binary.rs` +- `crates/sidecar/tests/support/mod.rs` +- `packages/core/src/agent-os.ts` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/src/sidecar/native-process-client.ts` +- `packages/core/tests/wasm-permission-tiers.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: The sidecar needs both durable per-VM command tier metadata and per-execution tier hints, because direct `exec()` and JS `child_process` launches reach WASM through different call paths. + - Gotchas encountered: Node `--permission` still leaves the process `cwd` writable, so `read-only` WASM tiers must also harden the WASI import surface itself instead of relying on `--allow-fs-write` alone. + - Useful context: `NODE_WASM_RUNNER_SOURCE` in `crates/execution/src/node_import_cache.rs` is the enforcement point for tier-specific preopens/imports, while `packages/core/tests/wasm-permission-tiers.test.ts` is the focused TS regression that proves the tier reaches sidecar execute requests. +--- +## 2026-04-05 03:06:56 PDT - US-029 +- What was implemented +- Replaced the single engine-wide `NodeImportCache` instances in the JavaScript, Python, and WASM execution engines with per-VM caches keyed by `vm_id`, while still reusing the cache across contexts inside the same VM. +- Wired sidecar VM disposal to drop per-VM execution-engine cache state, and added `NodeImportCache` cleanup on drop so the per-VM cache directories are removed from disk when a VM shuts down. +- Added a sidecar regression that proves two VMs get different JavaScript import-cache directories and that disposing one VM removes only its own cache root. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/wasm.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Python's bundled Pyodide path should be resolved from the VM-scoped import cache before context creation; otherwise Python silently reintroduces shared cache state even if the JavaScript engine is isolated. + - Gotchas encountered: Sidecar unit tests cannot call crate-private execution-engine helpers from another crate, so cross-crate regressions either need public hidden debug accessors or need to live inside the execution crate itself. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --no-run`, `cargo test -p agent-os-sidecar --no-run`, `cargo test -p agent-os-sidecar dispose_vm_removes_per_vm_javascript_import_cache_directory -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, `cargo test -p agent-os-execution --test python_prewarm -- --test-threads=1`, `cargo test -p agent-os-execution --test wasm wasm_execution_reuses_shared_warmup_path_across_contexts -- --test-threads=1`, and `cargo test -p agent-os-execution --test wasm wasm_execution_times_out_when_fuel_budget_is_exhausted -- --test-threads=1` pass. A full `cargo test -p agent-os-execution --test wasm -- --test-threads=1` run hit a transient timeout in `wasm_execution_times_out_when_fuel_budget_is_exhausted`, but that case passed immediately when rerun in isolation. +--- +## 2026-04-05 03:16:19 PDT - US-032 +- What was implemented +- Hardened sidecar runtime signaling in `crates/sidecar/src/service.rs` by whitelisting guest-exposed signals to `SIGTERM`, `SIGKILL`, `SIGINT`, `SIGCONT`, and signal `0`, and by probing child liveness with a non-reaping `waitid(...)` check before sending a real host signal. +- Added kernel-side fd bound validation so `dup2` and `open_with` reject target descriptors at or above `MAX_FDS_PER_PROCESS`, and tightened `pty_set_foreground_pgid` so foreground process groups must belong to the caller's session. +- Added focused regressions for fd-bound enforcement, same-session PTY foreground enforcement, and the sidecar signal/liveness hardening path. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/fd_table.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/fd_table.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Host child liveness checks that must not steal exit state from the real waiter should use `waitid(..., WNOWAIT | WNOHANG | WEXITED | WSTOPPED | WCONTINUED)` instead of `waitpid`. + - Gotchas encountered: `waitpid` rejects the `WNOWAIT` combination here, so using it for PID-reuse hardening returns `EINVAL` and silently leaves the sidecar without a safe non-reaping probe. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test fd_table --test api_surface`, `cargo test -p agent-os-sidecar service::tests::parse_signal_only_accepts_whitelisted_guest_signals -- --exact`, `cargo test -p agent-os-sidecar service::tests::runtime_child_liveness_only_tracks_owned_children -- --exact`, and `cargo check -p agent-os-kernel -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 03:33:42 PDT - US-026 +- What was implemented +- Scoped Python VFS RPC handling in `crates/sidecar/src/service.rs` to the guest `/workspace` root, normalizing paths before dispatch and rejecting escape attempts before they reach the kernel. +- Added host-side Python VFS RPC timeout tracking in `crates/execution/src/python.rs`; pending request IDs now auto-expire with `ERR_AGENT_OS_PYTHON_VFS_RPC_TIMEOUT` instead of leaving the guest blocked forever if no response arrives. +- Added focused regressions for Python VFS path scoping and timeout behavior, and updated the existing sidecar Python VFS unit coverage to use the real `/workspace` bridge root. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/python.rs` +- `crates/execution/tests/python.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Python VFS RPC validation belongs on the sidecar boundary, not in the guest bridge alone; normalizing `/workspace` paths centrally prevents `..` escapes before kernel permission checks run. + - Gotchas encountered: Trying to push the timeout into the embedded Pyodide runner can break real bundled Pyodide bootstrap (`process.binding` access during warmup); the safer timeout enforcement point is the Rust execution layer where pending RPC IDs are already visible. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test python`, and `cargo test -p agent-os-sidecar python_vfs_rpc -- --test-threads=1` pass after this change. `cargo test -p agent-os-sidecar --test python -- --test-threads=1` is still failing on the pre-existing Pyodide hardening-order regression tracked separately by `US-035` (`process.binding` denied during warmup). +--- +## 2026-04-05 03:49:26 PDT - US-039 +- What was implemented +- Replaced the sidecar `host_dir` mount’s `canonicalize`-then-open flow with anchored `openat2(..., RESOLVE_BENEATH | RESOLVE_NO_MAGICLINKS)` resolution plus descriptor-relative `mkdirat`/`unlinkat`/`renameat`/`linkat`/`readlinkat` handling, so symlink swaps cannot race guest paths out of the mounted host root. +- Hardened `KernelVm::setpgid` to reject joining a live process group owned by a different driver, and added a kernel unit test that exercises the cross-driver join attempt directly. +- Normalized `crates/kernel/src/kernel.rs` onto the recover-on-poison mutex policy already used by the other kernel managers by replacing the remaining lock poisoning `.expect(...)` sites with shared helpers. +- Files changed +- `crates/sidecar/src/host_dir_plugin.rs` +- `crates/kernel/src/kernel.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Safe native `host_dir` operations are easiest to keep correct if the code either stays on the `openat2`-resolved descriptor directly or uses its `/proc/self/fd/` anchor while that descriptor remains open. + - Gotchas encountered: Linux reports `RESOLVE_BENEATH` escape attempts as `EXDEV`; the guest-facing sidecar layer should translate that back to `EACCES` so callers keep treating path escapes as access denial. + - Useful context: `cargo test -p agent-os-sidecar host_dir_plugin -- --test-threads=1`, `cargo test -p agent-os-kernel setpgid_rejects_joining_a_process_group_owned_by_another_driver -- --test-threads=1`, and `cargo check -p agent-os-kernel -p agent-os-sidecar` pass after this change. +--- +## 2026-04-05 03:55:46 PDT - US-040 +- What was implemented +- Removed the mutable-assignment fallback from both generated `hardenProperty` helpers in `crates/execution/src/node_import_cache.rs`, so guest hardening now fails closed if `Object.defineProperty(...)` cannot lock down a property. +- Updated the kernel zombie reaper in `crates/kernel/src/process_table.rs` to keep exited children with living parents in the table and reschedule their cleanup, preserving exit codes until `waitpid` can reap them. +- Added focused regressions for the preserved child-exit-code path, eventual reaping after the parent exits, and a JavaScript startup regression that confirms the stricter hardening path still boots successfully. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/tests/process_table.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Guest hardening in `node_import_cache.rs` should fail closed; if a property cannot be made non-writable/non-configurable, treat that as a startup error instead of silently keeping a mutable escape hatch. + - Gotchas encountered: The process-table zombie TTL is still useful for parentless/orphaned exits, but child zombies with a live parent must be requeued or their exit code disappears before `waitpid`. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test process_table`, `cargo test -p agent-os-execution --test javascript -- --test-threads=1`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, and `cargo check -p agent-os-kernel -p agent-os-execution` all pass after this change. +--- +## 2026-04-05 04:06:43 PDT - US-043 +- What was implemented +- Switched `MemoryFileSystem::read_dir_with_types()` to a prefix-bounded `BTreeMap::range(...)` walk instead of scanning the full path index, hardened inode link-count decrements with `saturating_sub`, and made FD allocation wrap within the per-process limit so a freed low-numbered FD is reused even after `next_fd` has advanced past 255. +- Reworked overlay snapshot collection in `crates/kernel/src/overlay_fs.rs` to use an explicit stack with a depth cap, added a regression that exercises the rename limit on deeply nested lower trees, and kept the rest of the overlay rename behavior unchanged. +- Hardened the native WASM boundary by validating `usize` -> `u32` length conversions and host-returned buffer lengths in `registry/native/crates/wasi-ext/src/lib.rs`, adding `poll()` buffer-shape checks, rejecting overlong `getpwuid` responses in both the wasi-libc patch and the uucore WASI stub, and switching the SQLite WASM VFS randomness source to `/dev/urandom` with a deterministic fallback only if that device is unavailable. +- Files changed +- `crates/kernel/src/fd_table.rs` +- `crates/kernel/src/overlay_fs.rs` +- `crates/kernel/src/vfs.rs` +- `crates/kernel/tests/fd_table.rs` +- `crates/kernel/tests/root_fs.rs` +- `registry/AGENTS.md` +- `registry/native/c/programs/sqlite3_cli.c` +- `registry/native/crates/wasi-ext/src/lib.rs` +- `registry/native/patches/wasi-libc/0007-getpwuid.patch` +- `registry/native/stubs/uucore/src/lib/features/entries.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Native WASM host-import wrappers should validate both directions of every buffer contract: checked `usize` -> `u32` casts before the syscall, then a returned-length bounds check before treating guest memory as initialized. + - Gotchas encountered: Running `cargo test` directly against `registry/native/crates/wasi-ext/Cargo.toml` can refresh `registry/native/Cargo.lock`; if the story does not change native manifests, restore the lockfile before committing. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test fd_table --test vfs`, `cargo test -p agent-os-kernel --test root_fs`, `cargo test -p agent-os-kernel --test root_fs overlay_rename_rejects_directory_trees_that_exceed_snapshot_depth_limit -- --exact`, and `cargo test --manifest-path /home/nathan/a5/registry/native/crates/wasi-ext/Cargo.toml` all pass after this change. +--- +## 2026-04-05 04:18:27 PDT - US-035 +- What was implemented +- Split the embedded Pyodide hardening in `crates/execution/src/node_import_cache.rs` so safe `globalThis` guards are installed before `loadPyodide()`, while `process`-level denials stay deferred until after Pyodide bootstrap and package preload work complete. +- Added bounded Python VFS RPC backlog handling in `crates/execution/src/python.rs`, including a configurable `AGENT_OS_PYTHON_VFS_RPC_MAX_PENDING_REQUESTS` limit and explicit `ERR_AGENT_OS_PYTHON_VFS_RPC_QUEUE_FULL` responses instead of unbounded request accumulation. +- Added regressions that verify cached pre-load access to hardened globals is blocked and that overflowing the Python VFS RPC request queue returns explicit queue-full errors without surfacing extra requests to the host. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/python.rs` +- `crates/execution/tests/python.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Pyodide bootstrap hardening has to be staged; `globalThis` guards can go in before `loadPyodide()`, but mutating `process` before bootstrap breaks the real bundled Pyodide runtime under Node `--permission`. + - Gotchas encountered: `cargo test -p agent-os-sidecar --test python -- --test-threads=1` is still red on the pre-existing Pyodide warmup `process.binding` denial path and an unrelated cross-runtime workspace test, so the focused execution suites remain the reliable story-level verification targets for this area. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test python -- --test-threads=1`, `cargo test -p agent-os-execution node_import_cache::tests::materialized_python_runner_hardens_builtin_access_before_load_pyodide -- --exact --test-threads=1`, and `cargo check -p agent-os-execution -p agent-os-sidecar` pass after this change. +--- +## 2026-04-05 04:31:32 PDT - US-036 +- What was implemented +- Added a real bundled-Pyodide regression in `crates/execution/src/node_import_cache.rs` that verifies Python sees the frozen millisecond timestamp and that Python-side access to `node:child_process` and `node:vm` stays blocked through the `js` escape hatch. +- Aligned the materialized Python-runner test helpers with production by loading `timing-bootstrap.mjs`, so runner-level tests now exercise the same frozen `Date`/`performance` behavior as actual executions. +- Added an execution-engine regression in `crates/execution/tests/python.rs` that proves `loadPyodide()` cannot make outbound network requests during bootstrap. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/python.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** +- Patterns discovered: Real Pyodide behavior is easiest to validate in `node_import_cache` materialized-runner tests, while bootstrap hardening scenarios are cheaper and clearer with fake `pyodide.mjs` fixtures in `crates/execution/tests/python.rs`. +- Gotchas encountered: The materialized Python-runner test helpers must import `timing-bootstrap.mjs`; without that, frozen-time assertions measure the bare runner rather than the real execution path. +- Useful context: Focused checks that passed for this story were `cargo test -p agent-os-execution --test python python_execution_blocks_network_requests_during_pyodide_init -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests::materialized_python_runner_blocks_pyodide_js_escape_modules -- --exact`, `cargo test -p agent-os-execution node_import_cache::tests::materialized_python_runner_exposes_frozen_time_to_python -- --exact`, and `cargo check -p agent-os-execution`. +--- +## 2026-04-05 04:41:51 PDT - US-042 +- What was implemented +- Extracted the Pyodide host runner into the checked-in asset `crates/execution/assets/runners/python-runner.mjs` and switched `crates/execution/src/node_import_cache.rs` to materialize it via `include_str!` instead of keeping the runtime embedded inline in Rust. +- Added `crates/execution/src/runtime_support.rs` to share Node runtime helpers across `python.rs`, `javascript.rs`, and `wasm.rs`, covering compile-cache setup, sandbox-root/cache-root resolution, warmup marker hashing, feature-flag parsing, and shared file fingerprinting. +- Added startup cleanup for stale `agent-os-node-import-cache-*` directories keyed by temp-root, plus a regression that exercises isolated cleanup through `NodeImportCache::new_in(...)`. +- Files changed +- `AGENTS.md` +- `crates/execution/assets/runners/python-runner.mjs` +- `crates/execution/src/javascript.rs` +- `crates/execution/src/lib.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/runtime_support.rs` +- `crates/execution/src/wasm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Execution-host runner scripts are easier to maintain as checked-in assets loaded with `include_str!` than as multi-hundred-line Rust string literals, and the shared runtime helper module is the right place for cross-runtime Node warmup/compile-cache/path utilities. + - Gotchas encountered: Temp-cache cleanup needs to be keyed by the chosen base directory instead of a single global one-shot, otherwise tests cannot exercise cleanup safely after other `NodeImportCache::default()` calls have already happened in-process. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-execution`, `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, `cargo test -p agent-os-execution --test python -- --test-threads=1`, `cargo test -p agent-os-execution --test wasm -- --test-threads=1`, and `cargo test -p agent-os-execution --test javascript -- --test-threads=1` all pass after this change. +--- +## 2026-04-05 05:01:06 PDT - US-037 +- What was implemented +- Added structured security audit events in `crates/sidecar/src/service.rs` for invalid auth tokens, filesystem permission denials, mount/unmount reconciliation, and process kill requests. +- Added a focused integration test suite in `crates/sidecar/tests/security_audit.rs` that asserts the emitted audit records and their structured fields. +- Preserved the reusable sidecar audit-logging pattern in `AGENTS.md` and marked the story complete in the PRD. +- Files changed +- `AGENTS.md` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/security_audit.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Native sidecar security telemetry is easiest to keep stable by emitting `StructuredEventRecord`s with a shared `timestamp` field and event-specific keys, rather than trying to parse free-form log messages later. + - Gotchas encountered: Sidecar kill paths see parsed numeric signals internally, so audit fields that need the caller-facing signal name should capture the original request string before `parse_signal(...)`. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-sidecar`, `cargo test -p agent-os-sidecar --test security_audit -- --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::create_vm_applies_filesystem_permission_descriptors_to_kernel_access -- --exact --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::configure_vm_mounts_require_fs_write_permission -- --exact --test-threads=1`, and `cargo test -p agent-os-sidecar service::tests::configure_vm_sensitive_mounts_require_fs_mount_sensitive_permission -- --exact --test-threads=1` all pass. +--- +## 2026-04-05 05:18:49 PDT - US-044 +- What was implemented +- Replaced the sidecar’s host `to_socket_addrs()` DNS path with a Hickory-based in-process resolver so `dns.lookup()`, `dns.resolve*()`, and `net.connect(hostname)` now resolve through sidecar-controlled logic instead of delegating to the host resolver. +- Added VM-scoped DNS metadata parsing in `crates/sidecar/src/service.rs` with `network.dns.servers` for upstream resolvers and `network.dns.override.` for fixed answers, and emitted `network.dns.resolved` / `network.dns.resolve_failed` structured events for auditable resolution. +- Added a focused regression test that proves a VM-local DNS override drives both `node:dns` and `node:net` hostname connects and records the structured DNS events. +- Files changed +- `Cargo.lock` +- `crates/sidecar/Cargo.toml` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: VM-specific sidecar behavior can be added without widening the public API by parsing `CreateVmRequest.metadata`, which is useful when the actor/client parity requirements would otherwise force cross-repo API work. + - Gotchas encountered: Hickory’s `Resolver::builder_tokio()` keeps DNS resolution in-process and off the host libc resolver, but custom upstreams still need per-connection port overrides applied to every `NameServerConfig.connections` entry. +- Useful context: `cargo fmt --all`, `cargo check -p agent-os-sidecar`, and `cargo test -p agent-os-sidecar javascript_dns_rpc -- --test-threads=1` all pass after this change. `cargo test -p agent-os-sidecar javascript_ -- --test-threads=1` still shows unrelated/pre-existing instability in older JS net/child-process tests on this branch, but the DNS-focused slice and the new override/connect regression are green. +--- +## 2026-04-05 05:41:47 PDT - US-045 +- What was implemented +- Added a real guest `net.Server.getConnections(callback)` path in `crates/execution/src/node_import_cache.rs` that queries the sidecar instead of returning a stubbed `0`. +- Taught the sidecar TCP listener state in `crates/sidecar/src/service.rs` to track active listener-owned connections, expose them through a new `net.server_connections` sync RPC, and enforce `listen({ backlog })` by rejecting excess accepted connections once the configured listener limit is reached. +- Added focused regressions for the runner/server RPC path and a direct sidecar backlog test that proves listener counts and backlog enforcement work against real TCP sockets. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Node `net` server behavior spans both the guest runner RPC shims and the sidecar’s TCP socket lifecycle bookkeeping, so listener features need paired changes and paired regressions in both layers. + - Gotchas encountered: Guest stream teardown can easily turn a graceful FIN into a reset if the runner keeps a stale socket id after the sidecar reports close, so normal close/finalize paths should avoid issuing an extra `net.destroy`. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test javascript javascript_execution_routes_net_create_server_through_sync_rpc -- --test-threads=1`, and `cargo test -p agent-os-sidecar javascript_net_rpc_reports_connection_counts_and_enforces_backlog -- --test-threads=1` pass after this change. +--- +## 2026-04-05 06:08:50 PDT - US-046 +- What was implemented +- Added guest `node:net` IPC support in `crates/execution/src/node_import_cache.rs` so `net.connect({ path })` and `net.createServer().listen({ path })` now route through the sync RPC bridge, preserve guest-resolved socket paths, and expose string `address()` results for Unix sockets. +- Extended the native sidecar in `crates/sidecar/src/service.rs` with Unix listener/socket tracking, guest-path-to-host-path resolution, active listener lookup by socket path, and kernel-visible placeholder files for non-mounted Unix socket paths. +- Added focused regressions for the guest runner IPC RPC surface and a direct sidecar Unix-socket round-trip that verifies connect/listen behavior plus socket-file visibility in the kernel VFS. +- Files changed +- `AGENTS.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Guest Node `net` Unix sockets need the same two-layer treatment as TCP: update both the generated runner RPC shims and the sidecar’s active socket bookkeeping, then cover both layers with regressions. + - Gotchas encountered: Unix socket paths only exist on the host when they resolve onto a host-backed path or the sidecar mirrors them under the VM sandbox root, so non-mounted IPC listeners also need a kernel VFS placeholder for guest `fs` visibility. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test javascript javascript_execution_routes_net_ -- --test-threads=1`, `cargo test -p agent-os-sidecar service::tests::javascript_net_rpc_listens_and_connects_over_unix_domain_sockets -- --exact --test-threads=1`, and `cargo check -p agent-os-execution -p agent-os-sidecar` all pass after this change. +--- +## 2026-04-05 06:16:15 PDT - US-047 +- What was implemented +- Enabled external-network coverage in `registry/tests/wasmvm/curl.test.ts` with a host-side availability probe, retry helpers, a new raw external TCP regression through `http_get_test`, and more stable external HTTP/HTTPS curl assertions against `example.com`. +- Updated `.github/workflows/ci.yml` to run the test suite with `AGENTOS_E2E_NETWORK=1`, and recorded the reusable external-network test pattern in `registry/CLAUDE.md`. +- Files changed +- `.github/workflows/ci.yml` +- `registry/CLAUDE.md` +- `registry/tests/wasmvm/curl.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Registry external-network coverage is safest when CI opt-in (`AGENTOS_E2E_NETWORK=1`) is paired with a host-side preflight probe and command-level retries inside the VM. + - Gotchas encountered: The root workspace install is currently broken by unrelated package metadata (`examples/quickstart` expects `@rivet-dev/agent-os`, while `packages/core` is named `@rivet-dev/agent-os-core`), so focused registry verification has to use `pnpm install --dir registry --ignore-workspace --no-lockfile`. + - Useful context: `cargo fmt --all --check` passes. `AGENTOS_E2E_NETWORK=1 registry/node_modules/.bin/vitest run registry/tests/wasmvm/curl.test.ts` passes syntactically but skips locally because the required WASM artifacts are not built in this checkout. Root `pnpm install --frozen-lockfile` fails pre-existingly with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`, and root `pnpm install --no-frozen-lockfile` also fails pre-existingly because the workspace contains a missing `@rivet-dev/agent-os` package reference in `examples/quickstart`. +--- +## 2026-04-05 06:31:02 PDT - US-048 +- What was implemented +- Enforced per-VM network permissions in the sidecar JavaScript sync-RPC paths for `dns.lookup`/`dns.resolve*`, TCP `net.connect`, and TCP `net.listen`, using operation-time checks instead of only relying on policy setup. +- Preserved errno-style sync-RPC failures back to guest JavaScript so denied network operations now surface `EACCES` instead of the generic `ERR_AGENT_OS_NODE_SYNC_RPC`. +- Added sidecar regressions that verify the bridge callback path is exercised for `dns.lookup`, `net.connect`, and `net.listen`, and that a VM with denied network permissions cannot resolve DNS, open outbound TCP connections, or bind TCP listeners. +- Files changed +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: JavaScript networking sync RPCs are one of the places where sidecar code can bypass kernel permission wrappers, so that layer needs its own explicit permission enforcement and guest-visible errno preservation. + - Gotchas encountered: `dns.promises.lookup(...)` in the guest can still throw synchronously when the underlying sync RPC fails, so denial regressions should use `try`/`catch` instead of assuming a rejected promise path. + - Useful context: `cargo fmt --check`, `cargo test -p agent-os-sidecar javascript_network_permission_callbacks_fire_for_dns_lookup_connect_and_listen -- --nocapture`, `cargo test -p agent-os-sidecar javascript_network_permission_denials_surface_eacces_to_guest_code -- --nocapture`, and `cargo test -p agent-os-sidecar javascript_dns_rpc_resolves_localhost -- --nocapture` pass after this change. `cargo test -p agent-os-sidecar -- --test-threads=1` is still red on pre-existing failures outside this story, including the bundled Pyodide warmup `process.binding` denial path and unstable older sidecar net/child-process tests. +--- +## 2026-04-05 06:39:43 PDT - US-049 +- What was implemented +- Hardened the guest Node `process` surface in `crates/execution/src/node_import_cache.rs` so `config`, `versions`, `release`, `version`, `platform`, `arch`, `memoryUsage()`, and `uptime()` now return virtualized values instead of host runtime/build details. +- Reworked the guest `process` proxy fallback to resolve properties through the guest proxy receiver rather than the raw host `process`, which closes accessor-based leaks while preserving the existing hardened property overrides. +- Added a JavaScript regression that verifies `globalThis.process`, `require("node:process")`, and `process.getBuiltinModule("node:process")` all expose the sanitized surface. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Guest-visible `process` virtualization is more reliable when the real host `process` is hardened first and the guest proxy only controls the receiver path for fallthrough properties. + - Gotchas encountered: `process.memoryUsage` in Node also exposes a `rss()` helper on the function object, so replacing the method needs to preserve that nested API or guest compatibility regresses. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test javascript -- --test-threads=1`, and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` all pass after this change. +--- +## 2026-04-05 06:50:49 PDT - US-050 +- What was implemented +- Hardened the generated Node runner’s CommonJS loader in `crates/execution/src/node_import_cache.rs` so `Module._resolveFilename` now translates guest paths before resolution and rejects resolved host files outside guest-visible mappings or the current execution root. +- Swapped the guest-facing `require.cache` surface onto a translated proxy over `Module._cache`, keeping cache keys in guest path space while preserving host-path internals for Node’s loader. +- Added a JavaScript regression that loads a CommonJS module from a mapped guest workspace, verifies a package under guest-visible `node_modules` still loads, and confirms a hidden ancestor `node_modules/host-only-pkg` outside the mapping is blocked. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: CommonJS isolation has to happen at the loader level, because local `require()` inside a loaded `.cjs` module does not use the top-level `createGuestRequire()` wrapper. + - Gotchas encountered: Node’s ESM-to-CJS bridge does not expose a stable local `require.cache` surface for assertions, so cache translation regressions are more reliable when checked from the guest global `require`. + - Useful context: `cargo fmt --check`, `cargo test -p agent-os-execution --test javascript -- --test-threads=1`, and `cargo test -p agent-os-execution node_import_cache::tests -- --test-threads=1` all pass after this change. +--- +## 2026-04-05 06:57:59 PDT - US-051 +- What was implemented +- Hardened the guest `os` polyfill in `crates/execution/src/node_import_cache.rs` so `hostname`, `homedir`, `tmpdir`, `userInfo`, and shell defaults now come only from `AGENT_OS_VIRTUAL_OS_*` overrides or safe VM defaults, never host `HOME`/`USER`/`TMPDIR`/`HOSTNAME`/`SHELL` fallbacks. +- Updated the existing `os` virtualization regression to set `AGENT_OS_VIRTUAL_OS_SHELL` explicitly, matching the new contract that plain host `SHELL` must be ignored. +- Added a JavaScript regression that feeds host-looking env vars into the guest and verifies `node:os` still returns `agent-os`, `/root`, `/tmp`, `root`, and `/bin/sh`. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: `node:os` virtualization should treat host env vars as implementation detail leakage; only explicit `AGENT_OS_VIRTUAL_OS_*` knobs are valid inputs for guest-visible overrides. + - Gotchas encountered: The JavaScript execution tests can trip the import-cache temp-root cleanup path if multiple `cargo test` invocations run concurrently, so this suite is more reliable when executed sequentially. + - Useful context: `cargo test -p agent-os-execution javascript_execution_virtualizes_os_module -- --test-threads=1`, `cargo test -p agent-os-execution javascript_execution_os_module_safe_defaults_ignore_host_env -- --test-threads=1`, and `cargo test -p agent-os-execution --test javascript -- --test-threads=1` pass after this change. +--- +## 2026-04-05 07:07:44 PDT - US-052 +- What was implemented +- Stripped all `AGENT_OS_*` keys from the guest `child_process` polyfill’s public `options.env` payload in `crates/execution/src/node_import_cache.rs`, including caller-supplied overrides, and moved the nested-Node bootstrap state into a separate `internalBootstrapEnv` RPC field. +- Updated `crates/sidecar/src/service.rs` to sanitize that sidecar-only bootstrap map with an allowlist and re-inject it only when starting a nested JavaScript runtime, leaving non-Node child environments free of Agent OS control vars. +- Added regressions that verify the child-process RPC payload excludes internal env keys, that the sidecar bootstrap allowlist rejects stray keys, and that nested Node child-process execution still works after the split. +- Files changed +- `AGENTS.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Nested Node child processes should receive Agent OS bootstrap state through sidecar-only RPC metadata, not through the child environment that shell/WASM children inherit. + - Gotchas encountered: The sidecar child-process regression is more stable when it stays at the RPC/bootstrap layer; trying to assert non-Node env contents through extra command fixtures introduces unrelated import-cache and command-availability noise. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test javascript child_process -- --test-threads=1`, and `cargo test -p agent-os-sidecar javascript_child_process -- --test-threads=1` all pass after this change. +--- +## 2026-04-05 07:46:10 PDT - US-056 +- What was implemented +- Extended `ResourceLimits` with configurable caps for `pread`, `fd_write`/`fd_pwrite`, merged spawn `argv`/`env`, and `readdir` batches, with safe defaults in the kernel. +- Enforced those limits in `KernelVm` entrypoints and added `read_dir_limited(...)` support through the core VFS delegation stack so common in-memory and overlay listings fail closed before returning oversized batches. +- Threaded the new `resource.max_*` keys through sidecar metadata parsing and documented the pattern in the repo instructions. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/device_layer.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/mount_table.rs` +- `crates/kernel/src/overlay_fs.rs` +- `crates/kernel/src/permissions.rs` +- `crates/kernel/src/resource_accounting.rs` +- `crates/kernel/src/root_fs.rs` +- `crates/kernel/src/vfs.rs` +- `crates/kernel/tests/resource_accounting.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Per-operation memory caps should be enforced at the kernel entrypoint that materializes guest-visible buffers, not only in downstream VFS helpers, so oversized calls fail before extra copies or file reads happen. + - Gotchas encountered: `AGENTS.md` at the repo root is a symlink to `CLAUDE.md`, so instruction updates appear as a tracked `CLAUDE.md` diff. + - Useful context: `cargo test -p agent-os-kernel --test resource_accounting`, `cargo test -p agent-os-kernel --test api_surface`, `cargo test -p agent-os-kernel --test vfs`, and `cargo test -p agent-os-sidecar parse_resource_limits_reads_filesystem_limits --lib` all pass for this change. +--- +## 2026-04-05 08:01:50 PDT - US-057 +- What was implemented +- Added `ExportedChildFds` in `crates/execution/src/node_process.rs` so control and RPC pipes stay `O_CLOEXEC` on their original low-numbered descriptors and only get duplicated into reserved `1000+` FDs immediately before `Command::spawn()`. +- Switched JavaScript sync RPC, Python VFS RPC, and the shared Node control channel wiring to export those reserved high FDs instead of inheriting the original pipe ends, which also keeps the parent-side duplicates closed automatically after spawn. +- Added a unit regression for the shared FD exporter and verified the affected execution paths with focused JavaScript, Python, and WASM runtime tests. +- Files changed +- `crates/execution/src/javascript.rs` +- `crates/execution/src/node_process.rs` +- `crates/execution/src/python.rs` +- `crates/execution/src/wasm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: FD remapping for guest-visible control channels belongs in the shared `node_process` spawn helpers so JavaScript, Python, and WASM launches all inherit the same protected `1000+` descriptor policy. + - Gotchas encountered: This repo’s pinned `nix` API still takes raw `RawFd` values for `fcntl`, so shared FD helpers need to duplicate with `source_fd.as_raw_fd()` instead of newer `AsFd`-style calls. + - Useful context: `cargo check -p agent-os-execution`, `cargo test -p agent-os-execution node_process::tests -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript javascript_execution_surfaces_shared_array_buffer_sync_rpc_requests -- --test-threads=1`, `cargo test -p agent-os-execution --test python python_execution_surfaces_vfs_rpc_requests_and_resumes_after_responses -- --test-threads=1`, and `cargo test -p agent-os-execution --test wasm wasm_execution_emits_signal_state_from_control_channel -- --test-threads=1` all pass after this change. +--- +## 2026-04-05 08:06:20 PDT - US-058 +- What was implemented +- Added explicit WebAssembly parser guardrails in `crates/execution/src/wasm.rs`: module files are size-checked via `metadata()` before `fs::read()`, import and memory section counts are capped before iteration, and varuint decoding now has a hard byte-length bound plus checked `usize` conversions. +- Added focused regressions in `crates/execution/tests/wasm.rs` for oversized sparse module files, excessive import entries, excessive memory entries, and malformed overlong varuint encodings so parser failures stay explicit and non-panicking. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/wasm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: WASM parser hardening belongs in the lightweight preflight path before Node/V8 spawn, and sparse files are an efficient way to regression-test file-size caps without allocating the capped bytes. + - Gotchas encountered: The old `shift >= 64` guard still fired before the new varuint byte cap until the continued-10th-byte case was rejected explicitly; test the exact overlong encoding path, not just malformed-section overflow in general. + - Useful context: `cargo test -p agent-os-execution --test wasm -- --test-threads=1` and `cargo check -p agent-os-execution` both pass for this change. +--- +## 2026-04-05 08:24:58 PDT - US-059 +- What was implemented +- Added kernel-level `SIGCHLD` delivery in `crates/kernel/src/process_table.rs` so living parents are signaled when child processes exit or are killed, with updated stub/mock driver behavior and kernel regressions covering both paths. +- Allowed guest Node `process.on('SIGCHLD')` registration in `crates/execution/src/node_import_cache.rs`, emitted signal-state updates over the shared control pipe, and surfaced those updates through `JavascriptExecutionEvent::SignalState` so the sidecar can observe JavaScript signal handlers. +- Updated `crates/sidecar/src/service.rs` to track JavaScript signal registrations, send a real host `SIGCHLD` to parent runtime processes when nested `child_process` children exit, and retain the last signal-state snapshot after process exit so `get_signal_state` queries stay deterministic. +- Files changed +- `crates/execution/src/benchmark.rs` +- `crates/execution/src/javascript.rs` +- `crates/execution/src/lib.rs` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/tests/process_table.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/socket_state_queries.rs` +- `crates/sidecar/tests/support/mod.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: JavaScript signal support needs all three layers updated together: guest runner registration/hardening, execution-event plumbing, and sidecar state/delivery logic. + - Gotchas encountered: Fast-exiting processes can clear `vm.signal_states` before tests or clients query them; retaining the last snapshot after exit makes signal-state inspection deterministic without affecting live delivery. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test process_table -- --test-threads=1`, `cargo test -p agent-os-execution --test javascript -- --test-threads=1`, `cargo test -p agent-os-sidecar --test socket_state_queries -- --test-threads=1`, and `cargo check -p agent-os-kernel -p agent-os-execution -p agent-os-sidecar` all pass. The focused `javascript_execution_denies_process_signal_handlers_and_native_addons` test hit the known temp import-cache race once (`register.mjs` missing) and passed on immediate rerun; the full `agent-os-execution` javascript suite passed in this session. +--- +## 2026-04-05 08:28:25 PDT - US-060 +- What was implemented +- Added `SIGPIPE` to `crates/kernel/src/process_table.rs` and taught `crates/kernel/src/kernel.rs` `fd_write` to deliver that signal when a pipe write fails with `EPIPE`, while preserving the existing broken-pipe error return. +- Added a kernel integration regression that closes a pipe's read end, verifies the write still fails with `EPIPE`, and asserts the writer records `SIGPIPE` and exits with the corresponding signal status. +- Files changed +- `AGENTS.md` +- `CLAUDE.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/tests/kernel_integration.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: PID-aware POSIX signal side effects belong in `KernelVm` syscall entrypoints; low-level helpers like `PipeManager` should keep returning primitive errno results and let the syscall layer add process-table behavior such as `SIGPIPE`. + - Gotchas encountered: `PipeManager::write(...)` can only report `EPIPE`; the kernel layer has to translate that into signal delivery after the pipe lock is released, or exit cleanup risks re-entering the same primitive while it is still borrowed. + - Useful context: `cargo test -p agent-os-kernel --test kernel_integration`, `cargo test -p agent-os-kernel --test pipe_manager`, `cargo test -p agent-os-kernel --test process_table`, `cargo test -p agent-os-kernel`, and `cargo check -p agent-os-kernel` all pass for this change. +--- +## 2026-04-05 08:37:17 PDT - US-061 +- What was implemented +- Added queued wait-state tracking in `crates/kernel/src/process_table.rs` so parent-aware waits can report `WNOHANG`, `WUNTRACED`, `WCONTINUED`, `pid=-1`, `pid=0`, and negative-process-group selectors without reaping stopped or continued children. +- Added `KernelVm::waitpid_with_options(...)` in `crates/kernel/src/kernel.rs`, keeping the existing single-PID `waitpid(...)` reap path stable while only cleaning up resources after exited children are actually collected. +- Added kernel regressions covering non-blocking waits, stop/continue reporting, process-group selectors, and the public kernel wait API. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/process_table.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Parent-aware `waitpid` bookkeeping belongs in `ProcessTable`; queue stop/continue notifications there and keep `KernelVm` focused on post-reap cleanup. + - Gotchas encountered: `WUNTRACED` and `WCONTINUED` need one-shot queued events, not just current `ProcessStatus`, or a child that stops and resumes before the parent waits loses observable state transitions. + - Useful context: `cargo fmt --all` and `cargo test -p agent-os-kernel` both pass for this change. +--- +## 2026-04-05 08:48:38 PDT - US-062 +- What was implemented +- Added kernel advisory locking support in `crates/kernel/src/fd_table.rs` and `crates/kernel/src/kernel.rs`, including `LOCK_SH`, `LOCK_EX`, `LOCK_UN`, `LOCK_NB`, a kernel-global lock manager, and a public `KernelVm::fd_flock(...)` surface keyed by opened-file identity. +- Wired advisory locks into FD lifecycle cleanup so dup/fork-inherited descriptors share the same lock ownership and the lock is released only when the last refcounted FD closes or the owning process is reaped. +- Added focused regressions for lock parsing, shared/exclusive conflicts, nonblocking `EWOULDBLOCK`, dup inheritance, fork inheritance, and last-close release behavior. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/fd_table.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/fd_table.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Advisory `flock` ownership belongs to the shared open-file-description, not the PID, so the kernel should key conflicts by file identity while using `FileDescription.id()` as the owner token and releasing on the last refcounted close. + - Gotchas encountered: Lock release has to run through the same last-close path used by dup2 replacement and process-reap cleanup; closing an individual FD is not enough when other dup/fork references still point at the same `FileDescription`. + - Useful context: `cargo fmt --package agent-os-kernel`, `cargo test -p agent-os-kernel --test fd_table --test api_surface`, `cargo check -p agent-os-kernel`, and `cargo test -p agent-os-kernel` all pass for this change. +--- +## 2026-04-05 08:57:03 PDT - US-063 +- What was implemented +- Added explicit `create_file_exclusive(...)` and `append_file(...)` filesystem operations, then threaded them through the kernel wrappers (`PermissionedFileSystem`, `DeviceLayer`, `MountTable`, `RootFileSystem`, `OverlayFileSystem`) so `fd_open(... O_CREAT|O_EXCL ...)` and `fd_write(... O_APPEND ...)` stop using split `exists/stat/read/write` sequences. +- Updated `KernelVm::prepare_fd_open(...)` to route `O_CREAT|O_EXCL` through a single exclusive-create call and `KernelVm::fd_write(...)` to route append-mode writes through a single append operation that updates the shared cursor after the append completes. +- Added regression coverage in `crates/kernel/tests/api_surface.rs` with a probe filesystem that simulates stale `exists` and stale append snapshots, proving the kernel now uses the atomic code paths instead of overwriting a competing creator or dropping a competing append. +- Files changed +- `crates/kernel/src/device_layer.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/mount_table.rs` +- `crates/kernel/src/overlay_fs.rs` +- `crates/kernel/src/permissions.rs` +- `crates/kernel/src/root_fs.rs` +- `crates/kernel/src/vfs.rs` +- `crates/kernel/tests/api_surface.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Kernel filesystem semantic additions have to be propagated through every wrapper layer, not just `MemoryFileSystem`, or mounted/device-backed paths keep the old behavior. + - Gotchas encountered: `MountTable` has its own `MountedFileSystem` abstraction separate from `VirtualFileSystem`, so new VFS entrypoints must be added to both traits and to `MountedVirtualFileSystem` forwarding. + - Useful context: `cargo fmt --all --check`, `cargo test -p agent-os-kernel --test api_surface -- --test-threads=1`, `cargo check -p agent-os-kernel`, and `cargo test -p agent-os-kernel` all pass for this change. +--- +## 2026-04-05 09:05:53 PDT - US-064 +- What was implemented +- Added FD-level `O_NONBLOCK` tracking in `crates/kernel/src/fd_table.rs`, keeping shared `FileDescription.flags()` scoped to open-file-description state while `fd_stat` reports the combined view. +- Taught `crates/kernel/src/kernel.rs` to honor per-FD nonblocking mode for pipe and PTY reads, to route pipe writes through a blocking/nonblocking-aware pipe path, and to let `/dev/fd/N` duplication layer `O_NONBLOCK` onto a duplicate FD. +- Hardened `crates/kernel/src/pipe_manager.rs` so blocking small writes wait until the full `PIPE_BUF` chunk fits, preserving atomic writes up to 4096 bytes while nonblocking writes still fail fast with `EAGAIN`. +- Added focused regressions for FD-level nonblocking flags, nonblocking pipe duplicates through `/dev/fd`, and `PIPE_BUF` atomicity. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/fd_table.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/pipe_manager.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/fd_table.rs` +- `crates/kernel/tests/pipe_manager.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Per-FD status bits such as `O_NONBLOCK` should live on `FdEntry`, while shared `FileDescription.flags()` should keep only open-file-description semantics like access mode and `O_APPEND`. + - Gotchas encountered: Without a `fcntl(F_SETFL)` API, the practical way to obtain a nonblocking view of an existing pipe in this codebase is duplicating `/dev/fd/N` and layering `O_NONBLOCK` onto the duplicate entry. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test fd_table -- --test-threads=1`, `cargo test -p agent-os-kernel --test pipe_manager -- --test-threads=1`, `cargo test -p agent-os-kernel --test api_surface -- --test-threads=1`, and `cargo check -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 09:35:57 PDT - US-067 +- What was implemented +- Replaced the Rust kernel overlay’s in-memory whiteout tracking with durable marker files in the writable upper, added opaque-directory markers for copied-up directories, and hid the reserved overlay metadata root from merged reads. +- Applied the same durable-marker scheme to the TypeScript overlay backend so reopening an overlay with the same writable upper preserves whiteouts and opaque-directory state across persistent or remote uppers. +- Added focused Rust and Vitest regressions for upper-marker persistence, opaque-directory behavior, and metadata-root filtering. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/overlay_fs.rs` +- `packages/core/src/overlay-filesystem.ts` +- `packages/core/tests/overlay-backend.test.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Durable overlay state can live in ordinary VFS files as long as it is kept under a reserved hidden metadata root and the merged overlay view consistently filters that root back out. + - Gotchas encountered: Overlay durability has two separate concerns: reopening with the same writable upper must preserve live whiteouts and opaque markers, while sealing a layer should still snapshot the merged view so those markers materialize away in frozen lower layers. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel --quiet`, `cargo test -p agent-os-kernel -- --nocapture`, `pnpm --dir packages/core exec tsc --noEmit --pretty false`, and `pnpm --dir packages/core exec vitest run tests/overlay-backend.test.ts tests/layers.test.ts` all pass after this change. +--- +## 2026-04-05 09:43:59 PDT - US-068 +- What was implemented +- Updated `crates/kernel/src/overlay_fs.rs` so `remove_dir()` checks raw upper and lower entries instead of the merged `read_dir()` view, which prevents opaque copy-up directories from incorrectly dropping lower children. +- Reworked overlay `rename()` to stage source entries into the writable upper layer, copy overlay subtree markers onto the destination, and then call the upper filesystem's native `rename()` so hardlinks keep their inode identity across moves. +- Added kernel regressions proving lower-file hardlink copy-up survives a later rename, opaque directory `rmdir` still returns `ENOTEMPTY`, and mount-table cross-mount hardlinks return `EXDEV`. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/overlay_fs.rs` +- `crates/kernel/tests/mount_table.rs` +- `crates/kernel/tests/root_fs.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Overlay mutations that preserve identity should first materialize the source subtree in the writable upper and then defer the actual move to the upper filesystem's native `rename`; rebuilding destinations with read/write/delete breaks hardlinks. + - Gotchas encountered: Once a lower directory is copied up and marked opaque, merged directory iteration intentionally hides lower children, so `rmdir` emptiness checks must inspect raw upper and lower layers directly instead of reusing merged `read_dir()`. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test root_fs -- --nocapture`, `cargo test -p agent-os-kernel --test mount_table -- --nocapture`, `cargo test -p agent-os-kernel -- --nocapture`, and `cargo check -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 10:07:52 PDT - US-069 +- What was implemented +- Added a kernel-backed procfs surface in `crates/kernel/src/kernel.rs` for `/proc`, `/proc/self`, `/proc/[pid]`, `/proc/[pid]/fd`, `/proc/[pid]/cmdline`, `/proc/[pid]/environ`, `/proc/[pid]/cwd`, `/proc/[pid]/stat`, and `/proc/mounts`, with live process/FD/mount metadata and synthetic stats/symlink targets. +- Made procfs read-only and threaded the virtual path handling through direct kernel filesystem APIs, proc-aware FD opens/reads/stats, and the sidecar JavaScript sync-RPC filesystem bridge. +- Added focused kernel regressions for live procfs process metadata and mount listings, plus a sidecar unit regression proving JS `fs.readlinkSync('/proc/self')` resolves against the calling kernel PID. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/permissions.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/security_hardening.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Synthetic procfs entries should authorize the guest-visible `/proc/...` path directly; if procfs checks go through `PermissionedFileSystem::check_path(...)`, missing backing `/proc` directories in the mounted root can accidentally break the virtual proc layer. + - Gotchas encountered: The sidecar’s JavaScript sync-RPC procfs coverage is more stable as a direct `service_javascript_fs_sync_rpc(...)` unit test than as a late assertion inside a full guest-runtime security script, because unrelated denied-builtin paths can dispose the Node sync bridge first. + - Useful context: `cargo check -p agent-os-kernel -p agent-os-sidecar`, `cargo test -p agent-os-kernel --test api_surface proc_filesystem_exposes_live_process_metadata_and_fd_symlinks -- --nocapture`, `cargo test -p agent-os-kernel --test api_surface proc_mounts_lists_root_and_active_mounts -- --nocapture`, `cargo test -p agent-os-sidecar javascript_fs_sync_rpc_resolves_proc_self_against_the_kernel_process -- --nocapture`, and `cargo test -p agent-os-sidecar --test security_hardening guest_execution_clears_host_env_and_blocks_network_and_escape_paths -- --nocapture` all pass after this change. +--- +## 2026-04-05 10:14:13 PDT - US-070 +- What was implemented +- Centralized `/dev/null`, `/dev/zero`, and `/dev/urandom` reads in `crates/kernel/src/device_layer.rs` so both `read_file()` and `pread()` use the same length-aware device-byte helper instead of duplicating stream-device logic. +- Updated `crates/kernel/tests/device_layer.rs` to assert exact 5-byte zero reads and exact 1 MiB urandom reads at the VFS layer. +- Added a kernel FD regression in `crates/kernel/tests/api_surface.rs` that opens `/dev/zero` and `/dev/urandom` and verifies `fd_read()` returns the requested byte counts. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/device_layer.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/device_layer.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Stream devices should keep their byte-generation logic in one helper so `pread()` / FD reads stay aligned with any bounded `read_file()` fallback. + - Gotchas encountered: Exact-byte Linux semantics for `/dev/zero` and `/dev/urandom` need to be asserted on length-aware read surfaces (`pread`, `fd_read`), because `read_file()` has no request-size parameter. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel`, `cargo test -p agent-os-kernel --test device_layer -- --nocapture`, `cargo test -p agent-os-kernel --test api_surface kernel_fd_surface_reads_exact_byte_counts_from_device_nodes -- --exact --nocapture`, and `cargo test -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 10:21:58 PDT - US-071 +- What was implemented +- Added shebang-aware command resolution in `crates/kernel/src/kernel.rs` so direct path execution now dispatches `#!/bin/sh ...` and `#!/usr/bin/env node ...` scripts through registered interpreters, enforces a 256-byte shebang cap, and returns `ENOEXEC`/`ENOENT` for malformed or missing interpreters. +- Added kernel integration coverage for direct shell and Node shebang execution plus missing-interpreter and overlong-shebang failures. +- Files changed +- `CLAUDE.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/tests/kernel_integration.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Direct script execution should resolve registered `/bin/*` and `/usr/bin/*` command stubs before parsing file contents; otherwise stub executables like `/bin/sh` loop back through their own shebang wrapper. + - Gotchas encountered: `#!/usr/bin/env ...` shebangs need interpreter extraction at parse time rather than generic basename dispatch if the proc cmdline should reflect the real target interpreter (`node`, not `env`). + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel`, `cargo test -p agent-os-kernel --test kernel_integration -- --nocapture`, and `cargo test -p agent-os-kernel` all pass after this change. +--- +## 2026-04-05 11:01:43 PDT - US-073 +- What was implemented +- Hardened sidecar-managed Node networking in `crates/sidecar/src/service.rs` so TCP and UDP binds only allow loopback hosts, guest listen ports can be constrained per VM with `network.listen.port_min`, `network.listen.port_max`, and `network.listen.allow_privileged`, and `socket_host_matches()` no longer treats `0.0.0.0` as equivalent to loopback. +- Added guest-port to host-port translation for sidecar-managed loopback listeners so separate VMs can reuse the same guest-visible port without colliding on real host sockets; listener snapshots and RPC responses stay guest-visible while host-side probes use the hidden bound port. +- Updated the Node import-cache polyfill defaults in `crates/execution/src/node_import_cache.rs` so `server.listen(0)` and `dgram.bind(0)` default to loopback instead of unspecified addresses, and refreshed socket-state coverage to query `127.0.0.1`. +- Files changed +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/sidecar/src/service.rs` +- `crates/sidecar/tests/socket_state_queries.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Sidecar-managed loopback listeners should keep a guest-visible port mapping separate from the hidden host-bound port so VM-local semantics and host-side test probes can both work. + - Gotchas encountered: Existing unit tests that connect from the host must use the actual listener socket stored in `ActiveProcess.tcp_listeners`, not the guest-visible port returned to Node. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-sidecar -p agent-os-execution`, `cargo test -p agent-os-sidecar javascript_network_ -- --test-threads=1 --nocapture`, `cargo test -p agent-os-sidecar javascript_net_rpc_listens_accepts_connections_and_reports_listener_state -- --test-threads=1 --nocapture`, `cargo test -p agent-os-sidecar javascript_net_rpc_reports_connection_counts_and_enforces_backlog -- --test-threads=1 --nocapture`, `cargo test -p agent-os-sidecar javascript_net_rpc_listens_and_connects_over_unix_domain_sockets -- --test-threads=1 --nocapture`, and `cargo test -p agent-os-sidecar --test socket_state_queries -- --test-threads=1 --nocapture` all pass after this change. +--- +## 2026-04-05 11:15:18 PDT - US-074 +- What was implemented +- Hardened both generated Node import-cache templates in `crates/execution/src/node_import_cache.rs` so host-to-guest path translation never falls back to raw host paths, uses the virtual guest cwd as an implicit runtime-only mapping for the real `HOST_CWD`, and redacts other unmapped absolute host paths to `/unknown`. +- Preserved loader/runtime usability by mapping internal import-cache guest paths back to their host cache roots, and by treating explicit virtual OS paths like `/bin/bash` as already guest-visible instead of scrubbing them. +- Added JavaScript regressions that verify `process.cwd()` and `require.resolve()` fall back to `/root` with no guest path mappings, and that top-level errors redact an injected unmapped host path to `/unknown`. +- Files changed +- `AGENTS.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/tests/javascript.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Guest path scrubbing should treat the actual `HOST_CWD` as an implicit runtime-only mapping to the virtual guest cwd so entrypoint loading, `process.cwd()`, and stack traces stay coherent without revealing the host path. + - Gotchas encountered: Internal Node import-cache asset paths and explicit virtual OS paths are already guest-visible surfaces; scrubbing them to `/unknown` breaks loader startup (`register.mjs` / `timing-bootstrap.mjs`) and regresses `os.userInfo().shell`. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-execution`, and `cargo test -p agent-os-execution --test javascript -- --nocapture --test-threads=1` all pass after this change. +--- +## 2026-04-05 11:23:54 PDT - US-075 +- What was implemented +- Updated `crates/kernel/src/process_table.rs` so `kill(...)` now treats `SIGSTOP` and `SIGTSTP` as stop transitions, treats `SIGCONT` as a resume transition, and queues the matching `waitpid` stop/continue notifications instead of leaving `ProcessStatus::Stopped` unreachable. +- Added PTY window-size state in `crates/kernel/src/pty.rs` plus a new `KernelVm::pty_resize(...)` entrypoint in `crates/kernel/src/kernel.rs` that emits `SIGWINCH` to the foreground process group only when the PTY size actually changes. +- Widened `crates/sidecar/src/service.rs` guest signal parsing so sidecar `killProcess(..., "SIGSTOP")` matches the hardened kernel semantics. +- Added focused regressions for process-table job control transitions, PTY resize `SIGWINCH`, the PTY unit surface, the sidecar signal parser, and the existing native-sidecar end-to-end `SIGSTOP`/`SIGCONT` process-control path. +- Files changed +- `AGENTS.md` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/src/pty.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/process_table.rs` +- `crates/sidecar/src/service.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Job-control signal state transitions should be split by layer: `ProcessTable::kill(...)` owns `SIGSTOP`/`SIGTSTP`/`SIGCONT` status changes and `waitpid` notifications, while `KernelVm` entrypoints should emit PTY-driven `SIGWINCH` after the PTY layer reports the foreground process group. + - Gotchas encountered: The kernel’s stub driver treats any signal as fatal unless explicitly exempted, so kernel tests that add non-terminating signals such as `SIGSTOP`, `SIGCONT`, `SIGTSTP`, or `SIGWINCH` must keep `StubDriverProcess::kill(...)` aligned with Linux job-control semantics. + - Useful context: `cargo fmt --all`, `cargo check -p agent-os-kernel -p agent-os-sidecar`, `cargo test -p agent-os-kernel --test process_table -- --nocapture`, `cargo test -p agent-os-kernel --test api_surface -- --nocapture`, `cargo test -p agent-os-kernel --test pty -- --nocapture`, `cargo test -p agent-os-sidecar parse_signal_only_accepts_whitelisted_guest_signals -- --nocapture`, and `pnpm --dir packages/core exec vitest run tests/native-sidecar-process.test.ts -t "delivers SIGSTOP and SIGCONT through killProcess"` all pass after this change. +--- +## 2026-04-05 11:49:44 PDT - US-077 +- What was implemented +- Added per-process `umask` state in `crates/kernel/src/process_table.rs`, exposed it through `KernelVm::umask(...)`, and applied it to file and directory creation paths in `crates/kernel/src/kernel.rs`, including `O_CREAT`, direct write helpers, and recursive `mkdir`. +- Extended `VirtualStat` with `blocks`, `dev`, and `rdev`, filled those fields across kernel VFS stats, synthetic `/dev` and `/proc` entries, sidecar host-dir and sandbox-agent mounts, sidecar protocol serialization, and the TypeScript runtime adapters in `packages/core` and `packages/browser`. +- Updated filesystem timestamp behavior so directory reads refresh `atime` and metadata-changing operations such as `link`, `rename`, and unlink-style removals refresh `ctime`, then added regressions covering kernel umask behavior, stat field propagation, timestamp updates, and guest Node `process.umask()` plus `fs` integration. +- Files changed +- `AGENTS.md` +- `CLAUDE.md` +- `crates/execution/src/node_import_cache.rs` +- `crates/kernel/src/device_layer.rs` +- `crates/kernel/src/kernel.rs` +- `crates/kernel/src/process_table.rs` +- `crates/kernel/src/vfs.rs` +- `crates/kernel/tests/api_surface.rs` +- `crates/kernel/tests/vfs.rs` +- `crates/sidecar/src/host_dir_plugin.rs` +- `crates/sidecar/src/protocol.rs` +- `crates/sidecar/src/sandbox_agent_plugin.rs` +- `crates/sidecar/src/service.rs` +- `packages/browser/src/driver.ts` +- `packages/browser/src/os-filesystem.ts` +- `packages/browser/src/runtime.ts` +- `packages/core/src/runtime.ts` +- `packages/core/src/sidecar/native-kernel-proxy.ts` +- `packages/core/src/sidecar/native-process-client.ts` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Per-process filesystem state such as `umask` belongs in `ProcessContext` / `ProcessTable`; if guest Node code needs it, the kernel entrypoint and the JS sync-RPC bridge have to move together. + - Gotchas encountered: `VirtualStat` changes are easy to land incompletely because synthetic kernel stats, sidecar mount/plugin adapters, protocol structs, and TypeScript runtime types all have their own copy paths. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-kernel --test vfs --test api_surface`, `cargo check -p agent-os-kernel -p agent-os-sidecar -p agent-os-execution`, `cargo test -p agent-os-sidecar javascript_fd_and_stream_rpc_requests_proxy_into_the_vm_kernel_filesystem -- --nocapture`, and `pnpm -C /home/nathan/a5 --filter @rivet-dev/agent-os-core run check-types` all pass after this change. `pnpm -C /home/nathan/a5 --filter @rivet-dev/agent-os-browser run check-types` is blocked in this checkout because `packages/browser` has no local `node_modules` and fails with `tsc: not found`. +--- +## 2026-04-05 12:14:22 PDT - US-079 +- What was implemented +- Added reserved Pyodide runtime knobs in `crates/execution/src/python.rs` for a per-run execution timeout (`AGENT_OS_PYTHON_EXECUTION_TIMEOUT_MS`, default 5 minutes) and a Node heap cap (`AGENT_OS_PYTHON_MAX_OLD_SPACE_MB`), made `PythonExecution::wait(None)` enforce the configured timeout, and kill the child before returning `TimedOut`. +- Applied `--max-old-space-size` to both Pyodide prewarm and execution launches so the Node host process is bounded even before guest code runs. +- Added regressions covering the implicit timeout path, heap-cap flag injection for both launches, and OOM stderr surfacing. +- Files changed +- `AGENTS.md` +- `crates/execution/src/python.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/execution/tests/python.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: Pyodide execution-level hardening fits the existing reserved-env pattern; runtime-only knobs should be read from `StartPythonExecutionRequest.env`, consumed host-side, and stripped from guest-visible `process.env`. + - Gotchas encountered: `wait(None)` is part of the security boundary for Python runs now; treating `None` as “unbounded” would silently bypass the per-run timeout even when the request config set one. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test python -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, and `cargo check -p agent-os-execution` all pass after this change. The Python suite showed one transient `loader.mjs`-missing import-cache race on the first run, then passed cleanly on rerun. +--- +## 2026-04-05 12:24:42 PDT - US-080 +- What was implemented +- Wired WASM runtime limit propagation in `crates/execution/src/wasm.rs` so Node child processes receive `AGENT_OS_WASM_MAX_MEMORY_BYTES` / `AGENT_OS_WASM_MAX_FUEL`, apply `--wasm-max-mem-pages`, and use a tighter 1ms timeout poll when fuel falls back to wall-clock enforcement. +- Hardened the generated WASM runner in `crates/execution/src/node_import_cache.rs` to rewrite the memory section before `WebAssembly.compile()`, capping declared or undeclared memories to the configured page limit so runtime `memory.grow()` fails at the configured cap. +- Added regressions in `crates/execution/tests/wasm.rs` and `crates/execution/tests/permission_flags.rs` covering runtime memory growth enforcement plus Node env/flag propagation for prewarm and execution launches. +- Files changed +- `crates/execution/src/node_import_cache.rs` +- `crates/execution/src/wasm.rs` +- `crates/execution/tests/permission_flags.rs` +- `crates/execution/tests/wasm.rs` +- `scripts/ralph/prd.json` +- `scripts/ralph/progress.txt` +- **Learnings for future iterations:** + - Patterns discovered: WASM memory caps are a two-layer enforcement problem here; compile-time validation alone is insufficient, so the Node-hosted runner must cap the memory section it compiles when the module omits or overstates its maximum. + - Gotchas encountered: `cargo test -p agent-os-execution --test wasm` is reliable with `--test-threads=1`; parallel runs can hit the known shared import-cache cleanup race and fail with missing `timing-bootstrap.mjs`. + - Useful context: `cargo fmt --all`, `cargo test -p agent-os-execution --test wasm -- --test-threads=1`, `cargo test -p agent-os-execution --test permission_flags -- --test-threads=1`, `cargo test -p agent-os-execution --lib wasm::tests::wasm_memory_limit_no_longer_requires_declared_module_maximum -- --exact`, and `cargo check -p agent-os-execution` all pass after this change. +---