Skip to content

fix(sbx): acq run does not wait for kit startup-phase config writes before attach, causing a stale-config race #506

Description

@wz-gsa

Summary

On the sbx backend, acq run <agent> <path> can attach an agent to a session before that agent's own kit-managed config file has finished being written by a startup-phase command. This is a real, live-reproduced race, not a theoretical one — confirmed with the exact sequencing acq itself performs before attach.

Why this matters

usai-provider's merge-global-config.mjs (a plain, non-background startup-phase command) writes ~/.config/opencode/opencode.jsonc — the file OpenCode reads for its model/provider configuration. If OpenCode attaches before that write completes, it can start with an absent, partial, or stale provider configuration. This affects the kit as shipped today, independent of any new work — it is not specific to any in-flight design.

Root cause

Per Docker's own kit-reference documentation (https://docs.docker.com/ai/sandboxes/customize/kit-reference/, "Execution order" section):

"Startup commands are non-interactive. They run before the agent attaches, with no terminal connected... They also don't gate the agent's entrypoint: the agent launches once startup commands have been dispatched, regardless of background. A value of false waits within the startup dispatcher before it runs the next command; it doesn't delay the agent entrypoint."

So on sbx, non-background startup commands are sequenced relative to each other, but none of them are guaranteed to finish before the container's entrypoint (and thus the agent binary) starts. acq's own control flow (acq_backend_provision → ensure_valid_key → ensure_opencode_postinstall → acq_backend_attach, see acq:1029-1065) does not close this gap: ensure_opencode_postinstall (acq.backends/common.sh:1565) only probes opencode --version to confirm the binary is executable — it does not wait for, or know about, any kit's config-writing command.

On msb, by contrast, acq's own bash loop (_acq_msb_apply_kit_dir at acq.backends/msb.sh:1508, iterated sequentially before acq_backend_attach is called) happens to serialize every kit's startup phase before attach — closing the race there. This is acq's own orchestration accident, not a guarantee from msb itself, and it does not exist on the sbx path at all.

Reproduction (live, this session, sbx v0.43.0)

A minimal probe kit with a single startup command that sleeps 4 seconds, then writes a distinguishing opencode.json:

schemaVersion: "2"
kind: mixin
name: slow-config-writer
setup:
  startup:
    - command:
        - sh
        - -c
        - |
          echo "$(date +%s.%N) STARTUP_BEGIN" >> /tmp/entrypoint-race.log
          sleep 4
          mkdir -p /home/agent/.config/opencode
          cat > /home/agent/.config/opencode/opencode.json <<'JSON'
          {"$schema":"https://opencode.ai/config.json","provider":{"racetest":{"npm":"@ai-sdk/openai-compatible","options":{"baseURL":"https://example.invalid/v1"},"models":{"race-marker-model":{"name":"Race Marker Model"}}}}}
          JSON
          echo "$(date +%s.%N) STARTUP_CONFIG_WRITTEN" >> /tmp/entrypoint-race.log

Steps, simulating exactly what acq run opencode <path> does before attach:

sbx create --name race-verify --kit <usai-provider-kit-dir> --kit <slow-config-writer-kit-dir> opencode <path>
# (host returns)
sbx exec race-verify -- opencode --version   # the EXACT probe ensure_opencode_postinstall runs
sbx exec race-verify -- cat /home/agent/.config/opencode/opencode.json

Observed timeline:

T+0.00s   host: sbx create returns
T+1.33s   host: opencode --version succeeds ("1.18.23") — this is the exact check
                acq's ensure_opencode_postinstall performs before attach
                → config file AT THIS INSTANT still shows the baseline
                  MCP-gateway placeholder content, not our kit's data
T+4.01s   guest: slow-config-writer's startup command finally finishes;
                config file now shows the real written content

opencode --version succeeded 2.7 seconds before the config-writing kit finished. ensure_opencode_postinstall is not a barrier against this race — it happens to run concurrently with it and can (and did) win.

Impact

  • Affects the already-shipped usai-provider kit's merge-global-config.mjs startup command on the sbx backend.
  • Severity in practice depends on real-world timing margin between kit startup dispatch and a human's actual attach — this reproduction used a synthetic 4s delay to make the race observable, not to claim merge-global-config.mjs itself takes 4 seconds. The structural gap is proven; the real-world window for the specific shipped script is not yet measured.
  • Any future kit relying on a startup-phase script to finish writing config before the agent's first read (e.g. a models-orchestrator kit) would inherit the same gap on sbx.

Proposed fix

Per discussion, the preferred fix is not a new host-side pre-creation rendering mechanism (which would weaken the pinned-kit-ref provenance model and add a new acq-level hook). Instead: close the gap using acq's existing exec plumbing, the same way the msb path already accidentally does — add an explicit barrier/completion-check on the sbx path between provision and attach, so acq run does not proceed to attach until every kit's startup-phase commands have observably finished (not just "the entrypoint binary is executable").

Concretely, one viable approach: have acq (not the kit) poll for a completion marker (e.g. a file, or sbx's own startup-log completion state if inspectable) after sbx create returns and before calling ensure_opencode_postinstall/acq_backend_attach — mirroring the sequencing that already exists organically in _acq_msb_apply_kit_dir. This keeps the fix backend-neutral in intent, requires no kit-schema change, and does not touch kit provenance.

Verification once fixed

  • Re-run this exact reproduction; confirm opencode --version's success point is now after the config-writing kit's completion timestamp, not before.
  • Add an offline bats regression test asserting the ordering (via stubbed sbx/msb) rather than a timing-based test, per this repo's own guidance that sleep-based CI assertions are flaky — assert the sequence of calls acq makes, not wall-clock timing.
  • Keep the reproduction's probe kit (or an equivalent) as a live scripts/verify-* check so this can be re-verified on future sbx/msb version bumps, matching the existing scripts/verify-ports-live pattern.

References

  • Docker's kit-reference documentation on startup-command/entrypoint ordering (fetched live during investigation): https://docs.docker.com/ai/sandboxes/customize/kit-reference/
  • usai-provider kit's startup command (the real-world instance of this pattern): integrations/isolation/acq-kits/usai-provider/spec.yaml in agentic-coding-patterns.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions