You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Run Pi headlessly or through its native interactive UI, with effort selection, system-prompt injection, JSON activity/usage parsing, and structured-error detection.
Supports Anthropic API keys/OAuth and provider-qualified models with explicit keys.
Add pi_version image pinning, unit coverage, a live smoke config, and Docker E2E checks using the real Pi CLI and a local test API.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Unresolved correctness issues affect prompt handling, usage accounting, custom-model reasoning, and concurrent access to shared Pi configuration and credentials.
No open issues 🎉
5 previously missed issues
In code that hasn't changed since last review
Lock auth.json reads to prevent partial credential loads
lib/drivers/pi.sh:23
The shared auth.json is read without acquiring Pi's lock. Pi 0.86.1 persists refreshed tokens with writeFileSync, which truncates the file before rewriting it, so a concurrent startup or harness session can read empty/partial JSON here. _pi_validate_chatgpt then rejects valid credentials and terminates the agent instead of waiting for refresh. Validate a snapshot under the same auth-file lock, or delegate the read to Pi's locked credential storage.
Escape prompts starting with @ to avoid file-input parsing
lib/drivers/pi.sh:93
Pi 0.86.1 interprets positional arguments beginning with @ as file inputs even after --. If the prompt file's contents begin with @README.md or @maintainer please review..., Pi treats the entire prompt as a filename rather than literal task text, potentially failing with File not found. Prefix only @-leading prompts with a newline to avoid file-input parsing without changing the reaper's stdin handling. Add a regression case alongside the leading-dash prompt test.
Keep generated settings container-local to prevent cross-run overrides
lib/drivers/pi.sh:164
In ChatGPT mode, pi_home is the same bind-mounted directory for every container sharing PI_AUTH_DIR. Atomic replacement prevents partial JSON, but concurrent runs with different timeout/retry overrides still overwrite each other's settings: if one writes its settings before another Pi process loads them, that process uses the wrong timeout and retry policy. Keep generated settings container-local while sharing the OAuth store and its lock.
Include reasoning capability in custom model definitions
lib/drivers/pi.sh:212
Custom model definitions omit reasoning, which Pi 0.86.1 defaults to false. Consequently, a reasoning-capable model registered through a custom provider/base URL cannot honor an explicit effort: "high" (or another non-off level): Pi treats it as a non-thinking model. None of the exposed metadata overrides changes this. Allow reasoning capability to be configured, or derive it from an explicitly requested non-off effort, and include it in the generated model definition.
Parse compaction_end usage in harness statistics
lib/drivers/pi.sh:246
Pi 0.86.1 reports automatic compaction usage through compaction_end.result.usage, not an entry_appended compaction event. This parser ignores compaction_end, so sessions that compact omit the summarization request's tokens and cost from the harness stats and cost reports. Include that event's usage without incrementing assistant turns, and cover the real event shape in the unit fixture.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Explicit API overrides are ignored for catalog models, so a model configured for OpenAI completions can still send Anthropic Messages requests to the overridden endpoint.
Unit suite: 14 files / 1560 tests, 0 failed (incl. new test_pi.sh 151, test_costs.sh 38, test_launch.sh 240, test_dashboard.sh 166).
Shellcheck: clean at --severity=warning on all scripts incl. the new lib/drivers/pi.sh.
Docker runtime: runtime_pi.sh 5/5 and runtime_signal_trap.sh 16/16 — real Pi CLI (0.86.1, pinned in image) + local deterministic test APIs: tools, commits, idle exit, post-process, harvest, activity/cost/token accounting, zero-exit API error -> failed container, Codex subscription auth + SSE, native Pi UI startup + /quit + interactive branch push.
Integration matrix: 12/12 passed with real Claude agents (claude-opus-4-6 / claude-sonnet-4-6), OAuth 2/2 (1-agent-oauth + 2-agents-mixed-auth) using a Claude Code OAuth token.
Commit-lint: master is an ancestor of head, no fixup/squash/wip. Semver: VERSION unchanged -> skip.
One note: on first run 2-agents-context-bare failed because the context=none Claude agent committed its own .claude/CLAUDE.md outside test-results/; re-running that case passed. That is real-LLM nondeterminism, not a PR regression.
The Pi driver works (unit + Docker E2E against the real CLI and deterministic local APIs). Caveat: the Pi E2E uses mock APIs, not a live provider — the paid matrix exercises the Claude Code driver, so Pi's live-provider path is unverified (the PR's runtime test explicitly does not claim live-provider coverage).
All CI gates that can run locally pass. Approving / suggesting merge.
The pi driver wrote settings.json with Pi's default httpIdleTimeoutMs
of 300000 and retry disabled, with no way to change either from the
swarm config. Local backends that queue requests and only send headers
once processing starts can exceed five minutes under load; the session
then fails with "Request timed out." and the container exits.
Read PI_HTTP_IDLE_TIMEOUT_MS (default 300000, 0 disables) and
PI_MAX_RETRIES (default 0) in agent_settings, validate them, and write
httpIdleTimeoutMs and retry.enabled/maxRetries. Forward both from the
host environment in the start, interactive and post-process docker
paths. Only httpIdleTimeoutMs is set: Pi maps 0 there to disabled,
whereas retry.provider.timeoutMs would pass 0 through literally.
Defaults are unchanged. Document the variables in USAGE.md and add
unit coverage for the generated settings and the launcher wiring.
A per-group base_url now writes a complete provider definition
into the container's models.json (endpoint, API kind, model
metadata, env-ref API key) instead of a bare baseUrl override.
Built-in prefixes keep working with their endpoint redirected;
non-built-in prefixes such as ethereum-foundation/ now resolve
as first-class custom providers, so the dashboard label matches
the actual endpoint. Metadata defaults via PI_API_KIND,
PI_CONTEXT_WINDOW, PI_MAX_TOKENS.
Pi 0.86.1 still treats arguments beginning with @ as file attachments after --.
The harness passes the contents of the configured prompt file here, so a prompt
starting with an @mention is interpreted as a filename instead of task text and
can fail before making a request.
Pass it as literal text, for example by prefixing a newline to prevent
attachment parsing.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Exported `PI_API_KIND`, `PI_CONTEXT_WINDOW`, and `PI_MAX_TOKENS` values are not
forwarded to containers, so endpoint overrides silently use the default protocol
and limits.
For example, a custom provider configured on the host with
`PI_API_KIND=anthropic-messages` still receives `openai-completions`.
Forward these variables through this shared auth hook for numbered agents,
interactive profiles, and post-processing.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
An exported `SWARM_CONFIG` takes precedence over this fixture's `swarm.json`,
so the runtime test can fail on an unrelated path or launch a different
configuration using real provider credentials. Export the generated config path
here to keep the start, post-process, and interactive invocations on the local
test API.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Clear the host timeout/retry overrides before checking their defaults. With the
documented `PI_HTTP_IDLE_TIMEOUT_MS=1800000` or `PI_MAX_RETRIES=2` exported,
the early `agent_settings` calls inherit those values, but the assertions at
lines 209–218 still require retries disabled and a 300000 ms timeout. This
makes the unit suite fail for a valid host configuration; unset both variables
here before the explicit override cases.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Pi can finish truncated-response recovery with a `compaction_end` overflow
error, `willRetry: false`, and a zero exit code while the last assistant
message has `stopReason: "length"`. Filtering only assistant messages drops
that terminal failure, so the harness treats an incomplete session as
successful. Track terminal overflow errors too, replacing that state when a
later assistant response arrives.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Pi 0.86.1 reports compaction usage in `compaction_end.result.usage`, not an
`entry_appended` compaction event. This omits compaction requests from the
harness's token and cost totals. Count the completion event without
incrementing assistant turns, and update the fixture to use that event shape.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
PR #127 moved the bare repo from /tmp/${PROJECT}-upstream.git into the
private runtime directory (swarm_runtime_dir/upstream.git). runtime_pi.sh
still used the legacy path, so the error and codex containers mounted a
nonexistent upstream and the zero-exit error test failed.
Point BARE at the runtime dir via swarm_runtime_dir, and clean up the whole
runtime directory on teardown.
Model: deepseek-v4-flash (m5m-ds4)
Tools: Pi 1.4.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
radubahmata
left a comment
Run Pi headlessly or through its native interactive UI, with effort selection, system-prompt injection, JSON activity/usage parsing, and structured-error detection.
pi_versionimage pinning, unit coverage, a live smoke config, and Docker E2E checks using the real Pi CLI and a local test API.No existing driver or dashboard controls change.