feat: port standalone Secrets CLI and provider backends - #1
Conversation
protocol-server.ts already parsed and typed context.allowedBundles/scope but called through to ./lib/secrets/access.js, a module that was never created — every request over the pipe would throw at runtime. Add access.ts: an unscoped caller passes through unchanged (today's trusted local client), a scoped caller may only reach bundles.* ops it can prove are bound to an allowed bundle name, listBundles is filtered rather than denied, and everything else (raw store access, index/agent/remote/push/ sync/rc-hygiene, bulk ops like migrateLegacyBundles) fails closed. Also finishes the AWS credential-precedence, exact-UTF-8-decoding and cancellation tightening that was mid-edit in providers/aws.ts and sdk.ts.
|
Picked this back up from an idle, uncommitted state (continuing session 01a0765a → the merged plan PR #3499 on agi-cli). Fixed a real defect in the already-pushed commits: Added Honest state of this PR, not just the diff I added: Next real step is decoupling those ~40 imports into the backend/schema/host contracts the plan specifies, which is multi-day work better run as a fleet team than solo — not something I'm going to half-finish with more point-fixes. |
The raw port still imported ~40 agents-cli modules (feed, daemon, hosts,
devices, resource profiles, analytics, state). Give the package its own
leaves instead of dragging those in:
- lib/state.ts: SECRETS_HOME root (default ~/.secrets) with the engine's
existing sub-layout, so pointing it at ~/.agents adopts stores in place;
secrets.yaml carries backend/policy/agent defaults and provider connections
- lib/events.ts: the value-free JSONL event log that replaces the agents feed;
lib/secrets/usage.ts derives per-bundle usage from it (no side database)
- lib/launch.ts + lib/version.ts: re-invoke *this* binary, read *its* version
- lib/helper-versions.ts: keychain helper floor; still fetched from the agi-cli
train because the Keychain ACLs are bound to that signing identity
- broker (secrets/agent.ts): one lifecycle owned here — no daemon hosting,
no legacy launchd retirement, `secrets stop` ends the broker process
- removed: reserved-sync (fleet auth-bundle policy stays in agents),
usage-db (analytics), resource-profile filtering (the pipe's allowlist is
the policy seam), daemon boot from the keychain spawn wrapper
Ported verbatim: fs-atomic, format, text/*, platform/{exec,process},
exec-bounded, help, onepassword (op CLI), app-bundle-install,
helper-download, sha256-asset, rush-session, pwsh.
82 -> 18 tsc errors; what remains is the SSH transport (remote/push),
which becomes transports/ssh.ts, plus --device->--host and the OpenClaw
subcommand in commands/secrets.ts.
- src/index.ts: the `secrets` executable; intercepts __serve (private pipe), __vault-age-helper and the __secrets-* broker clients before commander - commands/secrets.ts registers on the root program; --device/--devices are --host/--hosts (REMOTE-1: an OpenSSH alias or user@host, no registry); unlock --agent is --scope (CTX-1: the package takes an opaque scope, it does not know harness names); `secrets start/stop` own the broker directly; OpenClaw config migration leaves (harness policy stays in agents); help text no longer names agents commands or ~/.agents paths
… package - scripts/install.sh: side-by-side dev install as `secrets-dev`, never `secrets` - scripts/release.sh: dry-run by default; --apply gates on a clean default branch, single-step semver over the published latest, a CHANGELOG entry, typecheck, the suite and a test-free tarball, then publishes and proves the registry install injects a canary through `secrets exec` - .github/workflows/tests.yml: ubuntu + macos, typecheck/build/test plus an installed-binary canary with no other CLI on PATH
- state root: file store, key dir and read-backoff derive from SECRETS_HOME - event log locks its own file, so it is created before the first append (lockSync spun on ENOENT until its own timeout — the "hung vitest") - broker toggle is SECRETS_NO_BROKER, platform gate lives in ensureAgentRunning - launch keeps the Bun single-file handling the sync-client tests pin - dropped the agents-only blocks: daemon-hosted broker, legacy launchd retire, menubar helper, agents command-registry help, feed levels/milestones - new: events.test.ts, usage.test.ts; command tests spawn `src/index.ts` directly with SECRETS_HOME=<home>/.agents (tsx added as a dev dependency) 20 touched suites: 316 passed, 4 skipped.
…e library Credential transport refuses an unpinned destination, so the CLI needs the one explicit way to pin a host: ssh-keyscan into the managed known_hosts under SECRETS_HOME (REMOTE-1). Error hints across the library now name `secrets …` commands; the helper-missing hint points at `secrets status`, which downloads the keychain helper on demand.
…); remaining hint assertions use the secrets verbs
…tches the transport)
Add src/lib/transports/ssh.ts — the standalone CLI's single hardened ssh choke point, ported from agi-cli@95562110 (ssh-exec, devices/known-hosts, hosts/remote-cmd, hosts/remote-os) with ControlMaster multiplexing removed wholesale (REMOTE-1: no multiplexed credential transport). Keeps bounded timeouts, the 16MiB stdout cap, chunk-safe UTF-8 accumulation, terminal save/restore, managed known_hosts pinning under secretsHome(), and a per-process uname -s OS probe replacing the (absent) device registry. Rewire secrets/remote.ts and secrets/push.ts to import only from ../transports/ssh.js: resolveHostSshTarget validates-and-returns (no registry), resolveRemoteOsSync -> resolveRemoteOs, user-facing 'agents secrets' -> 'secrets' and --device -> --host, host-pin hint -> 'secrets hosts pin <target>'. isDangerousRemoteEnvKey unchanged.
…kfile honest - the command tests build the tree on the root program (it IS `secrets`) and assert the --host/--scope surface - stderr notices say [secrets], not [agents] - install.sh: keep devDependencies declared in the dev prefix so the committed bun.lock still matches and `bun install --production --frozen-lockfile` succeeds on macOS (it refused the stripped package.json)
AGENTS_SECRETS_* → SECRETS_*, plus SECRETS_SYNC_PASSPHRASE, SECRETS_BACKUP_PASSPHRASE, SECRETS_VAULT_PATH, SECRETS_CRED_*, SECRETS_NO_USAGE_TRACK and SECRETS_SCOPE (the ambient grant scope that replaces AGENTS_AGENT_NAME — the package takes an opaque scope, CTX-1). Kept on purpose: the signed keychain helper's AGENTS_KEYCHAIN_* variables (its contract, republishing it is a separate train) and AGENTS_RUNTIME / AGENTS_SESSION_ID / CLAUDECODE as recognized agent-context markers for materialization refusal (MAT-1). Full suite: 1008 passed, 14 skipped.
A fresh Mac has no helper, and the keychain hot path is sync + network-free by design, so `secrets exec` on even a file-backend bundle died with "npm package shipped without the bundled keychain helper" (the store probe asks the keychain first). Run the one download-capable installer once at startup on darwin — a current install is a no-op; a failure is reported and the file/vault backends keep working. SECRETS_SKIP_HELPER_INSTALL=1 opts out.
|
CI green on both platforms at 4b11247 (https://github.com/phnx-labs/secrets-cli/actions/runs/34037866007): typecheck, build, full suite, and the installed-binary canary ( Local full suite on the merged branch: 1008 passed / 14 skipped. Non-author review in progress; merge to main follows its verdict. |
…NTEXT, pinned transfers - `__secrets-get` delivers the bundle only on an inherited private pipe (fd 3), never stdout; without the pipe it exits 2 before touching the broker, so a bare `secrets __secrets-get prod` from a shell or an agent's Bash tool prints nothing (MAT-1). The parent spawns it with the pipe (agent.ts syncClient). - `SECRETS_CONTEXT=agent` is an agent-context marker in isAgentInvocationContext, so every materializing guard (view --reveal, export --plaintext, get <item>) honours the neutral signal the docs promise. - `export --host`, `import --from-ssh --host` and `exec --host` refuse an unpinned destination before any credential moves (REMOTE-1); the pinning seam existed but nothing called it. - `get <item>` refuses a bundle's own keychain item in an agent context or on a non-TTY — the raw-item exception is for an ad-hoc token, not a bundle door. - Touch ID prompt and helper hints name this product, not agents-cli; duplicate HELPER_RELEASE_REPO removed; stale --agent/agents.yaml comments; README documents --scope. - New protocol-server.test.ts drives handleRequest end to end on a real file store: handshake, version/shape rejection, scoped vs unscoped access, raw store gating, error text never carrying provider details.
|
Non-author review (built-in adversarial reviewer, run against this branch) returned three blockers and five should-fix findings; all addressed in commit "fix(security): review blockers — private-pipe sync client, SECRETS_CONTEXT, pinned transfers":
tsc 0 errors; touched suites 148 passed. Full suite re-run and CI follow before merge. |
…runtimes own an internal FIFO at fd 3
…ts-cli strings; --scope in docs
|
Non-author review verdict: APPROVE (built-in adversarial reviewer, re-verified read-only against c2f6f19 after the fix commits).
The one cosmetic item it named ( |
1 participant
Implements the approved extraction plan from phnx-labs/agi-cli#3499 (PHNX-3989): the standalone
secretsexecutable + SDK, with no runtime dependency on agents-cli.State (2026-09-06, this branch):
tsc --noEmit0 errors (from 82). Builtdist/index.jspasses the end-to-end canary on Linux with nothing else on PATH —secrets create canary --backend file,secrets add canary CANARY --value release-ok,secrets exec canary -- printenv CANARY→release-ok,secrets list, and a__servehandshake over fds 3/4 returning the operation table. Full suite: 990 passed / 14 skipped, remaining failures are the command-level suites being adapted to the root program (in progress).What landed here, in order:
__serve, protocol v1).access.ts: bundle-allowlist policy on the pipe, fail-closed for scoped callers (was a dangling import).lib/state.ts(SECRETS_HOME, adopts an existing~/.agentslayout in place),lib/events.ts(value-free JSONL replacing the feed) +secrets/usage.ts(derived, no side DB),lib/launch.ts,lib/version.ts,lib/helper-versions.ts(keychain helper still fetched from the agi-cli train — the Keychain ACLs are bound to that signing identity); broker lifecycle owned here (no daemon hosting, no launchd retire); removed reserved-sync, usage-db, resource-profile filtering, OpenClaw migration, daemon boot.src/index.ts),--device→--host,unlock --agent→--scope,secrets hosts pin|list,secrets start|stopown the broker.install.shassecrets-dev,release.shwith a registry install canary), README, CHANGELOG,docs/secrets.md, CI workflow (ubuntu + macos, with an installed-binary canary).Still to do before this merges to main: command-suite adaptation,
AGENTS_*→SECRETS_*environment names, macOS dev-install smoke (in progress on the signing box), then publish 0.1.0 and verify the registry install. The agents-cli consumer cutover is a separate PR series on agi-cli.