feat: Add self-verification with verified:true only on assertion pass

README.md

@@ -4,14 +4,16 @@ Ephemeral, zero-credential, self-verifying execution for untrusted or agent-writ
 
 ## What this demonstrates
 
-An airlock is the safe way to run code you do not trust: an LLM-generated snippet, a plugin, a user-submitted function. This repo builds that primitive from the ground up in TypeScript. The guarantee is that a caller never reads an output unless the run stayed inside its resource ceilings and its output satisfies a post-condition the caller supplied. Untrusted code is guilty until proven correct, and the type system makes you prove it before you can touch the value.
+An airlock is the safe way to run code you do not trust: an LLM-generated snippet, a plugin, a user-submitted function. This repo builds that primitive from the ground up in TypeScript. The guarantee is that a caller never reads an output as trusted unless the run stayed inside its resource ceilings and its output satisfies a post-condition the caller supplied. Untrusted code is guilty until proven correct, and the type system makes you prove it before you can touch a verified value.
 
-The first slice was the contract and the in-process runner that enforces it. The second slice added `run(code, opts)`, which executes untrusted source in a fresh `node:vm` context with no ambient authority. The third slice added `runInWorker(code, opts)`: the same contract on a `worker_threads` isolate with frozen globals and an empty env. The fourth slice hardens the **resource limit** layer: wall-clock deadline with hard terminate on abort, V8 heap cap, and output size caps. This slice adds a **deny-by-default module loader**: untrusted code has no `require` unless the caller opts in with `allowedModules`, and only the listed builtin or package ids resolve. Relative and absolute paths are always refused. Later slices add a Docker-backed tier and a growing suite of documented escape-attempt tests.
+The first slice was the contract and the in-process runner that enforces it. The second slice added `run(code, opts)`, which executes untrusted source in a fresh `node:vm` context with no ambient authority. The third slice added `runInWorker(code, opts)`: the same contract on a `worker_threads` isolate with frozen globals and an empty env. The fourth slice hardens the **resource limit** layer: wall-clock deadline with hard terminate on abort, V8 heap cap, and output size caps. The fifth slice adds a **deny-by-default module loader**: untrusted code has no `require` unless the caller opts in with `allowedModules`. This slice makes **self-verification** explicit: every result carries a literal `verified: true | false`, and `verified: true` is returned only when the caller-supplied assertion passes. Later slices add a Docker-backed tier and a growing suite of documented escape-attempt tests.
 
 ## Concepts demonstrated
 
-- **Verification-gated results.** The output value is reachable only through the `ok` variant of a discriminated union, so an unverified run is unrepresentable at the call site.
-- **Post-condition contracts.** A run is trusted when a caller-supplied assertion holds over its output, a design-by-contract style check applied to untrusted code.
+- **Self-verification.** Execution and verification are separate phases. A pure `selfVerify` post-condition tags a produced value as trusted only when a caller-supplied assertion holds; runners return `verified: true` exclusively on that path.
+- **Verification-gated results.** Every `RunResult` arm carries a literal `verified: true | false`. The trusted value is reachable only through the `ok` + `verified: true` arm of a discriminated union, so an unverified run cannot be treated as success at the type level.
+- **Post-condition contracts.** A run is trusted when a caller-supplied assertion holds over its output, a design-by-contract style check applied to untrusted code. Assertions may return a boolean or `{ pass, reason? }` for diagnostic refusal reasons.
+- **Predicate composition.** `allAssertions` (conjunction) and `anyAssertion` (disjunction) build compound post-conditions without re-running the untrusted task.
 - **Deadline enforcement with cooperative cancellation.** An internal timer races the task and aborts the `AbortSignal` it runs under, composed with any caller-owned signal.
 - **Hard preemption via worker termination.** On the isolate tier, the wall-clock deadline and caller abort both call `worker.terminate()`, reclaiming the OS thread instead of abandoning a hung task.
 - **Resource isolation and ceilings.** Three independent budgets gate every run: wall-clock time, V8 old-generation heap (`maxOldGenerationSizeMb`), and measured UTF-8 output size (`maxOutputBytes`). Each maps to a distinct result status (`timeout`, `out-of-memory`, `output-too-large`).
@@ -32,18 +34,20 @@ The first slice was the contract and the in-process runner that enforces it. The
 
 ```
 runVerified(task, { timeoutMs, assert, signal?, maxOutputBytes? }) -> RunResult
+selfVerify(value, assert) -> { verified: true, value } | { verified: false, value, reason? }
 ```
 
-- The value is returned **only** as `{ status: "ok", value, durationMs }`, and only when the task finished before `timeoutMs`, the payload stayed under `maxOutputBytes` when set, and `assert(value)` returned true.
-- Every other outcome is an explicit refusal: `timeout`, `assertion-failed` (carries the value for diagnostics, never as trusted), `output-too-large`, `out-of-memory`, or `error`.
+- The value is trusted **only** as `{ status: "ok", verified: true, value, durationMs }`, and only when the task finished before `timeoutMs`, the payload stayed under `maxOutputBytes` when set, and `assert(value)` passed.
+- Every other outcome is an explicit refusal with `verified: false`: `timeout`, `assertion-failed` (carries the value and optional `reason` for diagnostics, never as trusted), `output-too-large`, `out-of-memory`, or `error`.
+- `isVerified(result)` narrows on the literal `verified: true` arm so callers cannot read a trusted value without a type-level proof.
 - The task is handed an `AbortSignal` that fires on the deadline or on the caller's own signal, so well-behaved async work can stop early. On the worker tier that same abort path also terminates the isolate.
 
 The in-process tier cannot preempt code that blocks the event loop with a synchronous spin; that is what the isolate and container tiers are for. This tier defines the contract those tiers implement.
 
 ## Usage
 
 ```ts
-import { runVerified, isVerified } from "airlock";
+import { runVerified, isVerified, allAssertions, selfVerify } from "airlock";
 
 const result = await runVerified(
   async (signal) => {
@@ -53,15 +57,26 @@ const result = await runVerified(
   {
     timeoutMs: 2000,
     maxOutputBytes: 64 * 1024,
-    assert: (data) => Number.isInteger(data.total) && data.total >= 0,
+    assert: allAssertions(
+      (data) => Number.isInteger(data.total),
+      (data) =>
+        data.total >= 0
+          ? { pass: true }
+          : { pass: false, reason: "total must be non-negative" },
+    ),
   },
 );
 
 if (isVerified(result)) {
+  // result.verified is literally true here
   console.log("trusted output:", result.value.total);
 } else {
-  console.warn("refused:", result.status);
+  console.warn("refused:", result.status, result.verified); // always false
 }
+
+// Pure re-check of an already-produced value (no re-execution):
+const check = await selfVerify({ total: 3 }, (d) => d.total > 0);
+// check.verified === true
 ```
 
 To run untrusted **source code** instead of a trusted closure, use `run`. The code executes with no ambient authority, so `process`, `require`, `fetch`, and timers are all undefined inside it. Any capability it needs is passed explicitly through `grant`:
@@ -165,3 +180,4 @@ pnpm run build
 - `src/worker.ts`: `runInWorker(code, opts)` runs untrusted source in a `worker_threads` isolate started with an empty `process.env` and frozen globals, caps the heap with `maxOldGenerationSizeMb` (reported as `out-of-memory`), and hard-kills the thread on the deadline so a sync spin and a never-settling async task are both preempted. An escape-attempt test confirms the constructor walk that reaches the host realm in-process reaches only the credential-free worker realm here.
 - `src/limits.ts`: shared resource ceilings for every tier. Wall-clock timeout aborts the task signal and, on the worker tier, calls `worker.terminate()` on both deadline and caller abort. Heap cap via V8 `resourceLimits`. Output size caps (`maxOutputBytes`) measure UTF-8 payload with a budgeted walk (cycle-safe, early-exit) and refuse with `output-too-large` before the post-condition runs.
 - `src/modules.ts`: deny-by-default module loader with an explicit `allowedModules` allowlist. Omitted means no `require`; `[]` or a list injects `createGatedRequire` over the host/worker require. Exact match only (bare and `node:` equivalent), path specifiers always refused, and the gate wins over a grant-supplied `require`. Wired into both `run` and `runInWorker`.
+- Self-verification: run a supplied assertion, return `verified: true` only if it passes. `src/verify.ts` owns the pure post-condition phase (`selfVerify`, `allAssertions` / `anyAssertion`, structured `{ pass, reason? }` outcomes). Every `RunResult` arm carries a literal `verified` flag; `isVerified` narrows on `verified: true`. Wired through `runVerified`, `run`, and `runInWorker`.

src/contract.ts

@@ -1,3 +1,5 @@
+import type { AssertionFn } from "./verify.js";
+
 export type RunStatus =
   | "ok"
   | "timeout"
@@ -7,22 +9,38 @@ export type RunStatus =
   | "output-too-large";
 
 /**
- * A run only counts as verified when it carries `status: "ok"`. Every other
- * variant is a refusal, so a caller cannot read `value` without first proving
- * the run passed both the deadline and the post-condition.
+ * A run only counts as verified when it carries `verified: true`. Every other
+ * variant is a refusal, so a caller cannot treat a value as trusted without
+ * first proving the run passed both the deadline and the post-condition.
+ * The literal `true` / `false` on each arm makes the trust boundary visible
+ * at the type level, not only as a status string.
  */
 export type RunResult<T> =
-  | { status: "ok"; value: T; durationMs: number }
-  | { status: "timeout"; timeoutMs: number }
-  | { status: "assertion-failed"; value: T }
-  | { status: "error"; error: unknown }
-  | { status: "out-of-memory"; maxOldGenerationSizeMb: number }
-  | { status: "output-too-large"; maxOutputBytes: number; actualBytes: number };
+  | { status: "ok"; verified: true; value: T; durationMs: number }
+  | { status: "timeout"; verified: false; timeoutMs: number }
+  | {
+      status: "assertion-failed";
+      verified: false;
+      value: T;
+      reason?: string;
+    }
+  | { status: "error"; verified: false; error: unknown }
+  | {
+      status: "out-of-memory";
+      verified: false;
+      maxOldGenerationSizeMb: number;
+    }
+  | {
+      status: "output-too-large";
+      verified: false;
+      maxOutputBytes: number;
+      actualBytes: number;
+    };
 
 export type Task<T> = (signal: AbortSignal) => T | Promise<T>;
 
-/** Post-condition. A run's output is trusted only if this returns true. */
-export type Assertion<T> = (value: T) => boolean | Promise<boolean>;
+/** Post-condition. A run's output is trusted only if this returns a pass. */
+export type Assertion<T> = AssertionFn<T>;
 
 export interface VerifiedRunOptions<T> {
   timeoutMs: number;
@@ -35,6 +53,6 @@ export interface VerifiedRunOptions<T> {
 
 export function isVerified<T>(
   result: RunResult<T>,
-): result is Extract<RunResult<T>, { status: "ok" }> {
-  return result.status === "ok";
+): result is Extract<RunResult<T>, { verified: true }> {
+  return result.verified === true;
 }

src/index.ts

@@ -1,5 +1,16 @@
 export { runVerified } from "./run.js";
 export { isVerified } from "./contract.js";
+export {
+  selfVerify,
+  normalizeAssertOutcome,
+  allAssertions,
+  anyAssertion,
+} from "./verify.js";
+export type {
+  AssertOutcome,
+  AssertionFn,
+  VerifyResult,
+} from "./verify.js";
 export {
   run,
   probeAmbientAuthority,

src/run.ts

@@ -1,11 +1,13 @@
 import type { RunResult, Task, VerifiedRunOptions } from "./contract.js";
 import { checkOutputSize, validateResourceLimits } from "./limits.js";
+import { selfVerify } from "./verify.js";
 
 const DEADLINE = Symbol("deadline");
 
 /**
- * The core airlock primitive. Runs `task` under a deadline, then checks the
- * supplied post-condition, and hands back the value only when both pass.
+ * The core airlock primitive. Runs `task` under a deadline, then self-verifies
+ * the produced value with the supplied post-condition, and hands back
+ * `verified: true` only when both the deadline and the assertion pass.
  *
  * The deadline is enforced by racing an internal timer and aborting the signal
  * the task receives. That stops async and cooperative work, but a task that
@@ -47,7 +49,7 @@ export async function runVerified<T>(
   try {
     const outcome = await Promise.race([running, deadline]);
     if (outcome === DEADLINE) {
-      return { status: "timeout", timeoutMs };
+      return { status: "timeout", verified: false, timeoutMs };
     }
 
     const value = outcome as T;
@@ -56,18 +58,32 @@ export async function runVerified<T>(
       if (size.exceeded) {
         return {
           status: "output-too-large",
+          verified: false,
           maxOutputBytes,
           actualBytes: size.bytes,
         };
       }
     }
 
-    const passed = await assert(value);
-    return passed
-      ? { status: "ok", value, durationMs: performance.now() - started }
-      : { status: "assertion-failed", value };
+    const check = await selfVerify(value, assert);
+    if (check.verified) {
+      return {
+        status: "ok",
+        verified: true,
+        value: check.value,
+        durationMs: performance.now() - started,
+      };
+    }
+    return check.reason !== undefined
+      ? {
+          status: "assertion-failed",
+          verified: false,
+          value: check.value,
+          reason: check.reason,
+        }
+      : { status: "assertion-failed", verified: false, value: check.value };
   } catch (error) {
-    return { status: "error", error };
+    return { status: "error", verified: false, error };
   } finally {
     if (timer !== undefined) clearTimeout(timer);
     signal?.removeEventListener("abort", relayAbort);

src/sandbox.ts

@@ -124,7 +124,7 @@ export async function run<T>(
   try {
     script = new vm.Script(code, { filename: filename ?? "airlock-sandbox.js" });
   } catch (error) {
-    return { status: "error", error };
+    return { status: "error", verified: false, error };
   }
 
   const result = await runVerified<T>(
@@ -142,7 +142,7 @@ export async function run<T>(
   );
 
   if (result.status === "error" && isSyncTimeout(result.error)) {
-    return { status: "timeout", timeoutMs };
+    return { status: "timeout", verified: false, timeoutMs };
   }
   return result;
 }

src/verify.ts

@@ -0,0 +1,86 @@
+/**
+ * Self-verification is a separate phase from execution: take a produced value
+ * and a caller-supplied post-condition, and tag the value as trusted only when
+ * the assertion holds. Execution tiers (vm, worker, later Docker) hand results
+ * here; this module never runs untrusted code itself.
+ */
+
+export type AssertOutcome =
+  | boolean
+  | { pass: true }
+  | { pass: false; reason?: string };
+
+export type AssertionFn<T> = (
+  value: T,
+) => AssertOutcome | Promise<AssertOutcome>;
+
+export type VerifyResult<T> =
+  | { verified: true; value: T }
+  | { verified: false; value: T; reason?: string };
+
+export function normalizeAssertOutcome(outcome: AssertOutcome): {
+  passed: boolean;
+  reason?: string;
+} {
+  if (typeof outcome === "boolean") {
+    return { passed: outcome };
+  }
+  if (outcome.pass) {
+    return { passed: true };
+  }
+  return outcome.reason !== undefined
+    ? { passed: false, reason: outcome.reason }
+    : { passed: false };
+}
+
+/**
+ * Pure post-condition check. `verified: true` is returned only when the
+ * assertion resolves to a passing outcome; every other result is untrusted.
+ */
+export async function selfVerify<T>(
+  value: T,
+  assert: AssertionFn<T>,
+): Promise<VerifyResult<T>> {
+  const outcome = await assert(value);
+  const { passed, reason } = normalizeAssertOutcome(outcome);
+  if (passed) {
+    return { verified: true, value };
+  }
+  return reason !== undefined
+    ? { verified: false, value, reason }
+    : { verified: false, value };
+}
+
+/** Conjoin assertions: every one must pass for the value to be trusted. */
+export function allAssertions<T>(
+  ...asserts: readonly AssertionFn<T>[]
+): AssertionFn<T> {
+  return async (value) => {
+    for (const assert of asserts) {
+      const { passed, reason } = normalizeAssertOutcome(await assert(value));
+      if (!passed) {
+        return reason !== undefined
+          ? { pass: false, reason }
+          : { pass: false };
+      }
+    }
+    return { pass: true };
+  };
+}
+
+/** Disjoin assertions: one passing check is enough. */
+export function anyAssertion<T>(
+  ...asserts: readonly AssertionFn<T>[]
+): AssertionFn<T> {
+  return async (value) => {
+    let lastReason: string | undefined;
+    for (const assert of asserts) {
+      const { passed, reason } = normalizeAssertOutcome(await assert(value));
+      if (passed) return { pass: true };
+      if (reason !== undefined) lastReason = reason;
+    }
+    return lastReason !== undefined
+      ? { pass: false, reason: lastReason }
+      : { pass: false };
+  };
+}