Add support for Pi

CHANGELOG.md

@@ -2,6 +2,33 @@
 
 ## Unreleased
 
+- **Fix: retry Pi OAuth validation during credential refresh.** Shared
+  Codex auth validation retries transient reads while Pi holds its lock,
+  preventing concurrent starts from rejecting a valid store.
+
+- **Feature: configurable Pi request timeout and in-session retry.**
+  The `pi` driver now writes `httpIdleTimeoutMs` from
+  `PI_HTTP_IDLE_TIMEOUT_MS` (default 300000, `0` disables) and enables
+  Pi's own retry when `PI_MAX_RETRIES` is positive. Both are forwarded
+  from the host environment or `docker_args`. Fixes agents exiting with
+  `Request timed out.` when a slow local backend queues a request for
+  more than five minutes. Defaults are unchanged.
+
+- **Feature: support Codex subscriptions through Pi.** Add explicit
+  `auth: "chatgpt"` with `openai-codex/<model>` and a dedicated writable
+  `PI_AUTH_DIR`. Share Pi's auth lock and rotated credentials across
+  containers without importing unrelated host credentials or sessions.
+  Add validation, a live subscription smoke config, and a network-isolated
+  real-Pi test proving two agents share one OAuth refresh.
+
+- **Feature: add the `pi` driver.** Run Pi headlessly or through its
+  native interactive UI, with effort selection, system-prompt injection,
+  JSON activity/usage parsing, and structured-error detection. Support
+  Anthropic API keys/OAuth and provider-qualified models with explicit
+  keys. Add `pi_version` image pinning, unit coverage, a live smoke
+  config, and Docker E2E checks using the real Pi CLI and a local test
+  API. No existing driver or dashboard controls change.
+
 - **Rename: `claude-swarm` to `swarm-core`.** Update project links,
   installation examples, issue-template branding, pre-push stash
   labels, and test fixtures. Document URL updates for existing clones

Dockerfile

@@ -32,9 +32,9 @@ RUN if echo ",$SWARM_AGENTS," | grep -q ",claude-code,"; then \
     fi
 ENV PATH="/home/agent/.local/bin:${PATH}"
 
-# --- Node.js (shared by Gemini CLI and Codex CLI) ---
+# --- Node.js (shared by Gemini CLI, Codex CLI, and Pi) ---
 USER root
-RUN if echo ",$SWARM_AGENTS," | grep -qE ",(gemini-cli|codex-cli),"; then \
+RUN if echo ",$SWARM_AGENTS," | grep -qE ",(gemini-cli|codex-cli|pi),"; then \
         curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
         && apt-get install -y --no-install-recommends nodejs \
         && rm -rf /var/lib/apt/lists/*; \
@@ -52,6 +52,17 @@ RUN if echo ",$SWARM_AGENTS," | grep -q ",codex-cli,"; then \
         && mkdir -p /home/agent/.codex \
         && chown agent:agent /home/agent/.codex; \
     fi
+
+# --- Pi ---
+ARG PI_VERSION=
+RUN if echo ",$SWARM_AGENTS," | grep -q ",pi,"; then \
+        apt-get update \
+        && apt-get install -y --no-install-recommends fd-find ripgrep \
+        && ln -s /usr/bin/fdfind /usr/local/bin/fd \
+        && rm -rf /var/lib/apt/lists/* \
+        && npm install -g --ignore-scripts \
+            "@earendil-works/pi-coding-agent${PI_VERSION:+@$PI_VERSION}"; \
+    fi
 USER agent
 
 # Trust mounted bare repos and allow file:// transport for submodules.

README.md

@@ -23,7 +23,7 @@ Based on the agent-team pattern from
 - bash (5.0+), git, jq, bc
 - flock (util-linux on Linux; `brew install flock` on macOS)
 - tput (ncurses) — used by the dashboard
-- An Anthropic API key, OAuth token, or compatible endpoint
+- Credentials for the selected [agent/provider](USAGE.md#auth-modes)
 
 For development: `shellcheck` for linting.
 
@@ -126,10 +126,10 @@ Place a `swarm.json` in your repo root:
 }
 ```
 
-Groups without `api_key` use `ANTHROPIC_API_KEY` or
-`CLAUDE_CODE_OAUTH_TOKEN` from the environment. A named group
-with omitted `count` or `count: 0` is useful as an
-interactive-only profile.
+Credentials are driver-specific. Claude Code groups without `api_key`
+use `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` from the environment;
+see [Auth modes](USAGE.md#auth-modes) for other drivers. A named group
+with omitted `count` or `count: 0` is an interactive-only profile.
 
 **Per-group fields:** `name`, `model`, `count`, `effort`, `context`,
 `prompt`, `auth`, `api_key`, `auth_token`, `base_url`, `tag`,
@@ -156,6 +156,11 @@ Each driver implements a fixed role interface:
 | `agent_activity_jq` | jq filter for activity display |
 
 Built-in drivers: `claude-code` (default), `gemini-cli`,
-`codex-cli`, `fake` (test double).  See
+`codex-cli`, `pi` ([Pi](https://pi.dev)), `fake` (test double).  See
 [USAGE.md](USAGE.md#writing-a-new-driver) for the full interface
 and guide to writing a new driver.
+
+For Pi, set `"driver": "pi"` and use an Anthropic model ID or a Pi
+`provider/model` ID. Codex subscriptions use `openai-codex/<model>`,
+`"auth": "chatgpt"`, and a dedicated `PI_AUTH_DIR` with Pi OAuth login.
+See [Pi setup and authentication](USAGE.md#pi).

USAGE.md

@@ -79,6 +79,11 @@ Credentials stay as env vars (not in shell history).
 | `OPENAI_API_KEY` | | OpenAI API key (for Codex CLI driver). |
 | `CODEX_AUTH_JSON` | `~/.codex/auth.json` | Path to Codex auth file (ChatGPT subscription). |
 | `GEMINI_API_KEY` | | Google API key (for Gemini CLI driver). |
+| `PI_API_KEY` | | Pi key for the selected provider; explicit group `api_key` takes precedence. |
+| `PI_AUTH_DIR` | | Dedicated writable Pi home for `auth: "chatgpt"`; contains only Codex OAuth credentials. |
+| `ANTHROPIC_OAUTH_TOKEN` | | Pi Anthropic OAuth token; falls back to `CLAUDE_CODE_OAUTH_TOKEN`. |
+| `PI_HTTP_IDLE_TIMEOUT_MS` | `300000` | Pi driver: `httpIdleTimeoutMs` written to the container's Pi settings. Bounds the wait for response headers and stream data; `0` disables. See [Pi timeouts](#pi-timeouts-and-retries). |
+| `PI_MAX_RETRIES` | `0` | Pi driver: enable Pi's in-session request retry with this many attempts. `0` leaves retries to the harness. |
 | `SWARM_CONFIG` | | Path to swarmfile (or place `swarm.json` in repo root). |
 | `SWARM_TITLE` | | Dashboard title override. |
 | `SWARM_RUNTIME_DIR` | | See [Runtime directory](#runtime-directory). |
@@ -110,7 +115,7 @@ Per-group fields in `swarm.json` `agents` array:
 | `auth` | `apikey`, `oauth`, `chatgpt`, omit | Which host credential to inject (see [Auth modes](#auth-modes)). |
 | `api_key` | key or `$VAR` | Per-group API key for third-party endpoints. |
 | `auth_token` | key or `$VAR` | Per-group Bearer token (OpenRouter-style). |
-| `base_url` | URL | Per-group API endpoint. |
+| `base_url` | URL | Per-group API endpoint. Registers the provider (built-in or custom) with full model metadata; tune via `PI_API_KIND`, `PI_CONTEXT_WINDOW`, `PI_MAX_TOKENS`. |
 | `tag` | string or `$VAR` | Label for grouping runs (default: top-level). |
 | `driver` | driver name | Agent driver override (default: top-level or `claude-code`). |
 
@@ -119,11 +124,13 @@ Per-group fields in `swarm.json` `agents` array:
 - Claude Code: `low`, `medium`, `high`, `max` (Opus only).
 - Codex CLI: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`.
 - Gemini CLI: ignored.
+- Pi: `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`;
+  `none` maps to `off`. Pi clamps levels to the selected model's support.
 
 Top-level fields: `prompt`, `setup`, `max_idle` (default: `3`),
 `max_retry_wait`, `driver`, `inject_git_rules`,
 `git_user` (`name`, `email`, `signing_key`),
-`claude_code_version`, `codex_cli_version`, `title`, `tag`,
+`claude_code_version`, `codex_cli_version`, `pi_version`, `title`, `tag`,
 `pricing`, `docker_args`, `post_process`.
 
 ### Interactive profiles
@@ -366,8 +373,8 @@ do not count toward numbered-agent completion.
 | `p` | Start post-process after confirmation; replaces an exited run. |
 
 The Model column appends the agent's reasoning effort as a
-parenthesised letter: `(h)` high, `(m)` medium, `(l)` low,
-`(x)` xhigh, `(n)` none, and `(M)` max. Models configured
+parenthesised letter: `(h)` high, `(m)` medium or minimal, `(l)` low,
+`(x)` xhigh, `(n)` none, `(o)` off (Pi), and `(M)` max. Models configured
 without an `effort` show no suffix.
 
 ## Activity streaming
@@ -466,6 +473,23 @@ Unit tests (no Docker or API key):
 ./tests/test_harness.sh        # Stat extraction.
 ./tests/test_harvest.sh        # Harvest git ops.
 ./tests/test_launch.sh         # Launch logic.
+./tests/test_pi.sh             # Pi driver and build integration.
+```
+
+`--all` also runs `./tests/runtime_pi.sh`: the installed Pi CLI in Docker
+against test APIs, without real credentials or paid model calls. It
+checks the four tools, Git commits/pushes, idle exit, post-processing,
+harvest, activity, usage stats, and structured API failures. Two additional
+network-isolated Pi containers verify shared Codex OAuth refresh and
+persisted credentials using a test-only transport. With Python
+3 available on the host, it also checks native UI startup, `/quit`, and
+interactive branch push through a PTY; otherwise that check is skipped.
+This is not a substitute for live-provider verification.
+
+For a live Pi OAuth smoke test (billable Anthropic extra usage):
+
+```bash
+./tests/test.sh --config tests/configs/pi-only.json
 ```
 
 ## Post-processing
@@ -597,13 +621,13 @@ Three credential mechanisms serve different purposes:
   the container.  Values: `apikey`, `oauth`, `chatgpt`, or
   omit (auto-detect).
 
-- **`api_key`** — Per-group API key for third-party endpoints
-  (MiniMax, etc.).  Passed as `ANTHROPIC_API_KEY` inside the
-  container.  Supports `$VAR` references to host env vars.
+- **`api_key`** — Per-group provider API key, mapped to the
+  selected driver's credential mechanism. Supports `$VAR`
+  references to host env vars.
 
-- **`auth_token`** — Per-group Bearer token for endpoints
-  that use `ANTHROPIC_AUTH_TOKEN` (OpenRouter-style).  Clears
-  `ANTHROPIC_API_KEY` so Claude Code enters third-party mode.
+- **`auth_token`** — Per-group Bearer token for Anthropic-compatible
+  endpoints through Claude Code or Pi (`ANTHROPIC_AUTH_TOKEN`).
+  Claude Code clears `ANTHROPIC_API_KEY` to enter third-party mode.
   Supports `$VAR` references.
 
 ### Claude Code
@@ -648,6 +672,149 @@ then set `"auth": "chatgpt"` in your swarm config:
 The auth file is bind-mounted read-only into containers.
 Override the path with `CODEX_AUTH_JSON=/path/to/auth.json`.
 
+### Pi
+
+Set `"driver": "pi"` for numbered agents, interactive profiles, or
+post-processing. Pi runs inside the existing Docker boundary. The image
+installs Node.js, Pi, `fd`, and `ripgrep`. API-key and Anthropic-token runs
+do not need host Pi; Codex subscription setup uses Pi's `/login`.
+
+Bare model IDs select Anthropic. For another built-in Pi provider, use
+`provider/model`, including any slashes inside the model ID:
+
+```json
+{
+  "driver": "pi",
+  "pi_version": "0.86.1",
+  "prompt": "tasks/work.md",
+  "agents": [
+    { "count": 1, "model": "anthropic/claude-sonnet-4-6",
+      "auth": "oauth", "effort": "low" },
+    { "name": "operator", "count": 0, "model": "openai/gpt-5.4",
+      "api_key": "$OPENAI_API_KEY", "effort": "high" }
+  ]
+}
+```
+
+Run `./launch.sh interactive operator` for Pi's native UI. As with the
+other interactive drivers, the profile prompt's path is displayed, not
+automatically submitted. Git instructions are appended to the system
+prompt in both modes.
+
+| Credential selection | Behavior |
+|---|---|
+| Group `api_key` | Explicit key for the selected Pi provider. |
+| Group `auth_token` | Anthropic Bearer token (`ANTHROPIC_AUTH_TOKEN`). |
+| `auth: "apikey"` | `PI_API_KEY`, then `ANTHROPIC_API_KEY`; no OAuth fallback. |
+| `auth: "oauth"` | `ANTHROPIC_OAUTH_TOKEN`, then `CLAUDE_CODE_OAUTH_TOKEN`. |
+| `auth: "chatgpt"` | Shared writable `PI_AUTH_DIR`; requires `openai-codex/<model>`. |
+| Omitted `auth` | `PI_API_KEY`, then Anthropic OAuth, then `ANTHROPIC_API_KEY`. |
+
+Group credentials override `auth`. Anthropic environment credentials
+apply only to Anthropic. Other providers require an explicit `api_key`
+or `PI_API_KEY`, except for the Codex subscription mode below. Pi's
+normal host auth/session directory is not automatically mounted.
+Supplied Anthropic OAuth tokens must remain valid for the run.
+
+**Anthropic subscription auth in third-party apps uses paid extra
+usage, not plan limits.** Enable/fund extra usage or use an API key.
+An extra-usage rejection is fatal, not a reason to retry indefinitely.
+
+`base_url` registers the selected provider in Pi's container-local
+`models.json` as a full provider definition — built-in prefixes
+(`openrouter/...`) get their endpoint redirected, and custom prefixes
+(`ethereum-foundation/...`) resolve as first-class providers with the
+model declared (`api` defaults to `openai-completions`; metadata via
+`PI_API_KIND`, `PI_CONTEXT_WINDOW`, `PI_MAX_TOKENS`). This override is
+not allowed with `auth: "chatgpt"`. For API-key/token modes, secrets
+stay in environment variables; generated auth files contain
+environment references outside the worktree.
+
+Headless runs use fresh, unsaved sessions with `read`, `write`, `edit`,
+and `bash`. Project Pi settings/extensions are not automatically trusted;
+the native interactive UI retains its trust prompt. Pi reads root
+`AGENTS.md`/`CLAUDE.md` natively; if neither exists (nor
+`AGENTS.override.md`), the driver appends `.claude/CLAUDE.md` when present.
+Existing context modes still only strip `.claude/`, not `.pi/` or root
+context files.
+
+Startup network checks and telemetry are disabled. By default the harness
+owns retries; Pi's automatic retry and cache warming are disabled. Costs are
+Pi's provider-reported usage multiplied by its catalog rates, not a
+subscription invoice. Timing uses the harness's wall-clock fallback.
+
+#### Pi timeouts and retries
+
+Pi's `httpIdleTimeoutMs` (default 300000) also bounds how long a request
+may wait for response headers. A local backend that queues requests and
+only starts streaming when it begins processing can exceed five minutes
+under load, and the session then fails with `Request timed out.` Two
+environment variables adjust the generated container settings; they are
+forwarded from the host environment or set through `docker_args`:
+
+```json
+"docker_args": ["-e", "PI_HTTP_IDLE_TIMEOUT_MS=1800000",
+                "-e", "PI_MAX_RETRIES=2"]
+```
+
+`PI_HTTP_IDLE_TIMEOUT_MS=0` disables the timeout inside Pi. A positive
+`PI_MAX_RETRIES` enables Pi's own retry, which re-issues the request inside
+the same session and keeps the conversation; the harness `max_retry_wait`
+fallback restarts the task from the prompt instead. Only
+`httpIdleTimeoutMs` is written, never `retry.provider.timeoutMs`. Keep
+`SWARM_ACTIVITY_TIMEOUT` above the backend's longest silent period, since a
+long prefill emits no log lines.
+
+#### Codex subscription through Pi
+
+Use Pi's `openai-codex` provider for a ChatGPT subscription, not its
+`openai` API-key provider. Log in with host Pi into a **dedicated**
+directory outside the worktree:
+
+```bash
+export PI_AUTH_DIR="$HOME/.config/swarm-core/pi-codex"
+install -d -m 700 "$PI_AUTH_DIR"
+PI_CODING_AGENT_DIR="$PI_AUTH_DIR" pi
+# In Pi: /login, select OpenAI Codex, finish login, then /quit.
+```
+
+Then configure the swarm:
+
+```json
+{
+  "driver": "pi",
+  "pi_version": "0.86.1",
+  "agents": [
+    { "count": 1, "model": "openai-codex/gpt-5.5",
+      "auth": "chatgpt", "effort": "low" }
+  ]
+}
+```
+
+Choose a model enabled for your account. The live smoke config uses
+`gpt-5.5`; model availability is not guaranteed by a successful login.
+This mode works for headless agents and native interactive profiles.
+
+The whole dedicated directory is mounted read-write at
+`/home/agent/.pi/agent`. Keep it private and writable by the container's
+agent user (UID 1000); `auth.json` should have mode 600 and contain only
+an `openai-codex` OAuth entry. Do not use your normal `~/.pi/agent`, add
+unrelated credentials/extensions, or copy the same refresh token into
+independent stores for parallel runs. `CODEX_AUTH_JSON` belongs to the
+separate `codex-cli` driver and is not used here.
+
+Pi shares `auth.json` and its adjacent lock across containers, refreshes
+expired credentials, and persists rotated tokens back to this directory.
+The driver replaces its `settings.json` at startup, uses SSE transport,
+and keeps new interactive sessions container-local. If refresh fails,
+log in again in the dedicated directory; there is no API-key fallback.
+
+Verify with a live subscription call:
+
+```bash
+./tests/test.sh --config tests/configs/pi-chatgpt.json
+```
+
 ### General rules
 
 Groups with `api_key` or `auth_token` ignore the `auth`
@@ -709,6 +876,7 @@ Built-in drivers:
 | `claude-code` | `claude` | Yes |
 | `gemini-cli` | `gemini` | |
 | `codex-cli` | `codex` | |
+| `pi` | `pi` | |
 | `fake` | (none) | Test double for unit testing |
 
 Set the driver globally in `swarm.json`:
@@ -755,6 +923,19 @@ The value is forwarded to `npm install -g @openai/codex@<ver>`
 inside the image build.  Leave the field unset (or empty) to
 keep the default "latest published release" behavior.
 
+### Pinning Pi version
+
+Set `pi_version` to pin `@earendil-works/pi-coding-agent` in the image:
+
+```json
+{ "pi_version": "0.86.1" }
+```
+
+Omit it to install the latest published release. The driver is tested
+with Pi 0.86.1 (Node.js 22.19+); older Pi releases may lack required
+CLI flags or event fields. npm lifecycle scripts are disabled during
+installation.
+
 ### Writing a new driver
 
 Create `lib/drivers/<name>.sh` implementing these functions: