Skip to content

Typed operations, engines, workspace, query & MCP (#689)#690

Open
tony wants to merge 158 commits into
masterfrom
engine-ops
Open

Typed operations, engines, workspace, query & MCP (#689)#690
tony wants to merge 158 commits into
masterfrom
engine-ops

Conversation

@tony

@tony tony commented Jun 21, 2026

Copy link
Copy Markdown
Member

Summary

Implements the typed operations + engines architecture under libtmux.experimental — an inert, statically-typed operation spine; a family of interchangeable engines (subprocess, concrete, control-mode, their async variants, and the native imsg easter-egg); lazy/async plans with ;-folding chainability; pure object-graph snapshots; a typed read surface; engine-typed facades; a declarative tmuxp-style workspace builder; a live pane query DSL; tmux 3.7 floating panes; an optional Model Context Protocol server; and a docs catalog generated from the registry.

Operationalizes #688 (architecture) per the plan in #689. Touches no existing public API — everything is additive under libtmux.experimental (explicitly outside the versioning policy). Nothing is generated at runtime; everything is statically typed and checker-clean.

What's delivered

The spine — libtmux.experimental.ops (pure, no tmux):

  • Operation[ResultT]: frozen, keyword-only, class-vars as the single source of truth (kind/command/scope/result_cls/effects/safety/chainable/version gates). Pure render() with declarative version gating; build_result() adapts raw output to a typed result (version-threaded so read parsing matches the gated render).
  • Typed Result hierarchy with opt-in raise_for_status(): AckResult, SplitWindowResult/CreateResult (captured ids), CapturePaneResult, ListPanes/Windows/Sessions/ClientsResult (snapshot-deriving rows), plus HasSessionResult, DisplayMessageResult, ShowOptionsResult, ShowBufferResult.
  • Closed Target sum, fail-closed OperationRegistry, stdlib serialization, and catalog() (registry-derived docs data).
  • LazyPlan (record → resolve SlotRef forward refs → execute). Folding is planner-based: pass a Planner — sequential (one tmux call per op), FoldingPlanner (;-chained tmux a ; b), or MarkedPlanner ({marked}-fold). All yield an identical PlanResult; only the dispatch count differs, and failure attribution matches tmux's cmdq_remove_group (first failed, rest skipped).
  • Read seam: ListPanes/ListWindows/ListSessions/ListClients render the same -F template neo uses (imported, not copied) and parse into models snapshots — a typed read surface parallel to neo, leaving the ORM untouched.
  • 58 operations across client/pane/window/session/server scopes.

Engines — libtmux.experimental.engines (all behind TmuxEngine/AsyncTmuxEngine, all returning the same CommandResult):

Family Sync Async
Subprocess (classic) SubprocessEngine AsyncSubprocessEngine
Concrete (in-memory) ConcreteEngine AsyncConcreteEngine
Control mode (tmux -C) ControlModeEngine AsyncControlModeEngine (event stream via subscribe())
Native imsg (binary protocol) ImsgEngine (opt-in easter egg)

Control engines use an I/O-free bytes ControlModeParser with FIFO/skip correlation (startup-ACK consumed up front; unsolicited hook blocks skipped), report their tmux version for runtime gating, reap the throwaway session a bare tmux -C implies, and end event subscribers cleanly on engine death. The imsg engine speaks tmux's binary peer protocol directly (AF_UNIX + SCM_RIGHTS, PROTOCOL_VERSION 8) with a live parity test vs the subprocess engine.

Models — libtmux.experimental.models: frozen Pane/Window/Session/Client/ServerSnapshot (typed core + raw field tail), from_pane_rows() builds the whole tree from one list-panes -a query, round-trips to plain dicts.

Facades — libtmux.experimental.facade ("mode lives in the type"): the full eager / lazy / async × Server/Session/Window/Pane/Client matrix over the same ops; control mode is just an engine choice.

Declarative workspace — libtmux.experimental.workspace: a tmuxp-style Workspace declares a session as a tree of windows and panes and lowers to a Core LazyPlan. build/abuild fold the dispatches by default (a multi-pane window collapses to a handful of ;-chained + {marked} calls, identical result to an unfolded build); host steps (per-command sleeps, the opt-in wait_pane anti-race) stay hard fold boundaries. Threads env/shell/options through the IR, round-trips via to_dict, emits a build-event stream, supports floating panes (including cross-window overlays wired through a graphlib symbol table), and ships a load CLI for .tmuxp.yaml.

Live pane query — libtmux.experimental.query: panes() opens a lazy, chainable query over the panes a running server has (filter/order_by/limit/map, including filter(floating=True)), reading nothing until a terminal call. commands() records per-pane ops (send keys, resize, select, respawn, clear history, kill) into a LazyPlan that folds to one tmux dispatch. Resolves against a live engine or a plain sequence of snapshots (offline in tests).

Floating panes (tmux 3.7): available from the operation layer (a new-pane op with absolute geometry and zoom), the pane facades (new_pane()), the MCP surface (a curated tool), and workspace specs (Workspace float declarations).

MCP server — libtmux.experimental.mcp (optional libtmux[mcp] extra): a framework-agnostic tool projection (no hard MCP/pydantic dependency) plus a thin FastMCP adapter, launched as libtmux-engine-mcp. Three tiers: a curated vocabulary of intuitive verbs (~40) that mirror the ORM, a per-operation tool for every operation (hidden by default), and plan tools that preview or build a whole workspace. It is caller-aware (discovers the launching pane and refuses to kill or respawn its own pane/window/session), gates mutating and destructive tools behind a tiered LIBTMUX_SAFETY level until opted in, adds middleware (audit logging, read-only retry, tail-preserving output limits), serves a needle-free wait_for_output monitor and tmux:// resources, and ships recipe prompts.

Docs: an in-repo tmuxop-catalog Sphinx directive renders catalog() into the operation reference (exercised by the docs gate), so the reference can't drift from the code.

Testing

  • A large experimental suite (~2,260 repo tests) + doctests. The pure spine/models/concrete/query tests need no tmux; the classic/control/async/imsg engines, the facades, workspace builds, and the MCP surface are validated against a real tmux server via the libtmux fixtures.
  • Cross-engine contract suite: same typed result across engines; serialization round-trips.
  • Full repo gate green: ruff, ruff format, ty, pytest, build-docs.

Design notes

  • Revises Design typed operations and engines #688: execution mode lives in the facade type, not a runtime-bound engine attribute (return types differ by mode).
  • Per-engine error policy: classic reproduces today's behavior; newer engines return typed results with opt-in raise_for_status(). Same result shape across engines.
  • Core is stdlib-dataclass-only; the MCP surface sits behind the optional mcp extra, with no MCP/pydantic dependency in the core projection.
  • imsg is opt-in and non-default: it depends on tmux's internal protocol (v8), is POSIX-only, and cannot host attach (which falls back to a local spawn).

Refs #688, #689.

@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.12245% with 1287 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.83%. Comparing base (765697a) to head (e6b44ac).

Files with missing lines Patch % Lines
scripts/mcp_swap.py 26.72% 314 Missing and 15 partials ⚠️
src/libtmux/experimental/engines/imsg/base.py 51.60% 163 Missing and 33 partials ⚠️
src/libtmux/experimental/engines/control_mode.py 67.98% 69 Missing and 28 partials ⚠️
...libtmux/experimental/engines/async_control_mode.py 78.97% 58 Missing and 20 partials ⚠️
...rc/libtmux/experimental/mcp/vocabulary/_resolve.py 60.95% 46 Missing and 11 partials ⚠️
src/libtmux/experimental/engines/imsg/v8.py 76.17% 39 Missing and 17 partials ⚠️
src/libtmux/experimental/mcp/middleware.py 77.15% 32 Missing and 13 partials ⚠️
docs/_ext/tmuxop.py 18.18% 36 Missing ⚠️
src/libtmux/experimental/mcp/vocabulary/pane.py 76.15% 32 Missing and 4 partials ⚠️
src/libtmux/experimental/mcp/__init__.py 47.61% 28 Missing and 5 partials ⚠️
... and 59 more
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #690       +/-   ##
===========================================
+ Coverage   51.78%   75.83%   +24.04%     
===========================================
  Files          25      228      +203     
  Lines        3638    13538     +9900     
  Branches      733     1741     +1008     
===========================================
+ Hits         1884    10266     +8382     
- Misses       1448     2618     +1170     
- Partials      306      654      +348     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony tony changed the title Typed operations and engines: inert op spine (#689) Typed operations and engines: spine + 4 engines + facades (#689) Jun 21, 2026
@tony tony changed the title Typed operations and engines: spine + 4 engines + facades (#689) Typed operations and engines Jun 21, 2026
@tony tony changed the title Typed operations and engines Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Jun 21, 2026
tony added a commit that referenced this pull request Jun 21, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 2 issues:

  1. LazyPlan resolves a forward SlotRef only for target, never for src_target, so a dual-target op (JoinPane, SwapPane, MovePane, BreakPane, SwapWindow, MoveWindow, LinkWindow) whose src_target comes from an earlier plan.add(...) reaches render() with the slot unresolved and raises TypeError: cannot render an unresolved SlotRef. (bug: _resolve() substitutes operation.target but not operation.src_target, even though serialize.py already handles both)

def _resolve(
operation: Operation[t.Any],
bindings: dict[int, str],
) -> Operation[t.Any]:
"""Substitute a :class:`SlotRef` target with a captured concrete id."""
target = operation.target
if not isinstance(target, SlotRef):
return operation
try:
concrete = bindings[target.slot] + target.suffix
except KeyError as error:
msg = (
f"slot {target.slot} has no captured id yet; a plan step can only "
f"target an earlier step that creates an object"
)
raise OperationError(msg) from error
return dataclasses.replace(operation, target=_target_from_id(concrete))

  1. SaveBuffer declares contradictory metadata: safety = "mutating" together with effects = Effects(read_only=True), where read_only is documented as "does not change tmux state". Its read peer ShowBuffer uses safety = "readonly", and a consumer filtering registry.select(lambda s: s.safety == "readonly") would omit save_buffer despite effects.read_only=True. (bug: inconsistent safety/effects declarations)

result_cls = AckResult
safety = "mutating"
effects = Effects(read_only=True)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 1 issue:

  1. LazyPlan resolves forward SlotRefs on every dispatch path except the {marked} fold's decorates. _drive resolves the create op but builds decorates raw, so a chainable dual-target decorate (SwapPane/JoinPane/MovePane) whose src_target is a forward slot reaches render_marked unresolved and raises TypeError: cannot render an unresolved SlotRef. The single-op and chain paths both call _resolve; this one does not. (bug: decorates = [self._operations[i] for i in decorate_idx] skips _resolve, so src_target SlotRefs survive into render under MarkedPlanner)

create_idx, *decorate_idx = step.indices
create = _resolve(self._operations[create_idx], bindings)
decorates = [self._operations[i] for i in decorate_idx]
merged: CommandResult = yield _Chain(
render_marked(create, decorates, version),
)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

1 similar comment
@tony

tony commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jun 27, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jun 28, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 4, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
@tony tony changed the title Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Typed operations, engines, workspace, query & MCP (#689) Jul 4, 2026
@tony

tony commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Fresh review of the current HEAD (4393b02a) — the ops / engines / facade / workspace / query / MCP surface was checked for bugs and AGENTS.md compliance and is clean. (The prior review comments predate ~12 days of commits.)

🤖 Generated with Claude Code

tony added a commit that referenced this pull request Jul 5, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added a commit that referenced this pull request Jul 11, 2026
why: Record the experimental operations/engines layer for the
upcoming release so the unreleased section tracks what landed.

what:
- Add a "What's new" deliverable under the unreleased 0.59.x section
  for the experimental operations and engines layer (#690)
- Defer the release lead paragraph until the version is cut
tony added 2 commits July 12, 2026 06:25
why: Operationalizes the typed-operations/engines architecture
(issues 688, 689) with the pure substrate that was absent from every
prototype branch: an inert, statically-typed operation value that
renders tmux commands, carries its result type, and serializes without
a live tmux server. Engines stay transport-agnostic over it. None of
this touches or changes existing public APIs.

what:
- Add libtmux.experimental.{ops,engines} packages (experimental, not
  under the versioning policy)
- ops: frozen Operation[ResultT] with class-level metadata as the
  single source of truth; pure render() with declarative version gating
  (LooseVersion); build_result() adapting raw output to typed results
- ops: typed Result base + raise_for_status() (CPython/requests
  precedent), SplitWindowResult/CapturePaneResult payloads
- ops: closed Target sum (PaneId/WindowId/SessionId/ClientName/NameRef/
  IndexRef/Special/SlotRef) with fail-closed validation
- ops: fail-closed OperationRegistry keyed by kind, with OpSpec views
  and predicate listing; stdlib dict serialization with round-trips
- ops: four seed operations (split-window, capture-pane, send-keys,
  select-layout) registered via @register
- engines: TmuxEngine/AsyncTmuxEngine protocols, CommandRequest/
  CommandResult, EngineSpec; run()/arun() execute bridge sharing one
  render/build path (sync vs await is the only divergence)
- tests: 111 pure, fixture-parametrizable unit tests + doctests, all
  runnable without a tmux server
why: Proves the operation/result contract is transport-agnostic -- the
same typed result whether produced by a real tmux subprocess or an
in-memory simulator -- and provides the offline engine that lets ops
doctests and tests run without a tmux server (issue 689 phases 2-3).

what:
- engines.subprocess: classic SubprocessEngine mirroring tmux_cmd
  (has-session stderr fold, backslashreplace, trailing-blank strip;
  tmux failure returned as data, only missing binary raises), with
  for_server() deriving -L/-S/-f/-2 flags from a live Server
- engines.concrete: deterministic in-memory engine (fabricated pane/
  window/session ids, canned capture lines) for tests and docs
- engines.registry: name-keyed engine registry (register/create/
  available), seeded with subprocess + concrete
- tests/experimental/contract: engine-agnostic operation contract run
  offline via concrete, plus classic-vs-concrete parity against a real
  tmux server (same result type + argv, payload may differ)
tony added 29 commits July 12, 2026 06:25
why: A window/session option or environment value typed as an int
in YAML (e.g. `main-pane-height: 35`) reached the ;-chain renderer
as an int and raised AttributeError in _escape_arg. The IR declares
these Mapping[str, str] and tmux wants string args (classic libtmux
str()s every arg), so analyze must coerce at ingest.

what:
- Add _str_map(): stringify the keys and values of an options or
  environment mapping
- Apply it to all 7 ingest sites (session/window/pane
  options/options_after/global_options/environment)
- Add parametrized coercion tests + a live folded-build regression
why: NewPane defaults to detached (-d), which does not focus the new
pane. The {marked} fold's untargeted `select-pane -m` then marks the
old active pane, so a NewPane >> SendKeys(slot) plan under the default
MarkedPlanner sent keys into the wrong pane. SplitWindow is unaffected
because it focuses its new pane.

what:
- _marked_decorates skips creators with detach=True; they dispatch
  alone so their decorates bind the captured pane id via the slot
- Add a parametrized regression: split and focused NewPane still mark,
  detached NewPane does not
why: subscribe() gated only on _closing, but a dead reader sets _dead,
not _closing. A subscribe() after the engine died -- e.g. the output
monitor or pull ring reconnecting once tmux closed stdout -- registered
a fresh queue and blocked forever on queue.get(), the exact hang the
close-subscribers work removed for the aclose() path.

what:
- Gate subscribe() on `_closing or _dead is not None`
- Add a regression mirroring the after-close test but marking dead first
why: AGENTS.md's namespace-import rule exempts only dataclass/field
from the dataclasses module; replace is a plain function and should be
called as dataclasses.replace, matching the sibling snapshots.py.

what:
- import dataclasses; use dataclasses.replace in filter/order_by/limit
why: AGENTS.md's shipped-vs-branch rule keeps never-shipped,
branch-internal lineage out of docstrings; the "chainable-commands"
and "libtmux-protocol-engines" prototypes are sibling branches the
reader never experienced. Describe only current state.

what:
- Trim prototype references in ops/plan.py, ops/_chain.py,
  engines/base.py, engines/registry.py
why: window_shell (a window's custom first-pane shell) rode new-window
for windows 2..N, but window 0 is created by new-session, which had no
shell parameter -- so the first window's first pane silently booted the
default shell instead of the configured one.

what:
- Add window_shell to NewSession (a trailing shell-command, mirroring
  how NewWindow appends it for later windows)
- compiler passes ws.windows[0].window_shell to NewSession
- Test that window 0's window_shell rides new-session
why: The wait_pane readiness guard skipped the wait when a first pane
set `shell=`, but the first pane's own shell is never applied (only
window_shell reaches its creator). So the pane launched the default
shell and send-keys raced the prompt -- the exact race wait_pane
exists to prevent.

what:
- _emit_pane_commands takes the shell the creator actually applies
  instead of re-deriving pane.shell|window_shell: first pane ->
  window_shell, split -> pane.shell|window_shell, float -> pane.shell
- Parametrized regression: a first-pane pane.shell still waits, a
  first-pane window_shell skips
why: AGENTS.md requires working doctests on all methods; the async
twins arun (fluent) and aexecute (plan) had none, unlike their
documented siblings run and astream.

what:
- Add runnable async doctests (asyncio.run over AsyncConcreteEngine)
why: EagerWindow and LazyWindow expose select_layout, but AsyncWindow
lacked it -- `await window.select_layout(...)` raised AttributeError,
breaking eager/lazy/async parity for the layout op.

what:
- Add AsyncWindow.select_layout mirroring the eager/lazy method
- Assert it in the async facade parity test
why: ImsgEngine executes against a live tmux server but did not
implement tmux_version(), unlike every other real-server engine. So
resolve_engine_version() fell back to "assume latest", and a
version-gated op run over imsg against an older server rendered a
too-new flag that tmux rejects -- diverging from the subprocess and
control-mode engines on the same server.

what:
- Add ImsgEngine.tmux_version(): memoized tmux -V via the resolved
  binary, mirroring SubprocessEngine
- Test it satisfies SupportsTmuxVersion and reports/memoizes a version
why: The run_and_wait / diagnose_failing_pane / interrupt_gracefully
prompts steer the agent to call wait_for_output, but that tool is
registered only on a streaming control-mode async server. On the sync
server (and non-streaming async) the agent was told to call a tool that
does not exist -> tool-not-found.

what:
- register_prompts(events_enabled=): register the wait_for_output
  prompts only when events streaming is enabled; build_dev_workspace
  always registers
- build_async_server passes its computed events_enabled; the sync
  build_server leaves it False
- Parametrized gate regression + a sync-server integration check
why: The audit summary redacted a sensitive arg (keys/text/command/...)
only when it was a str or dict; a non-conformant client sending e.g.
command=["secret"] fell through to the raw-passthrough branch and logged
the payload verbatim into the INFO audit record -- contradicting the
documented "payload-bearing arguments never reach the log raw" promise.
Schema validation rejects such calls, but _summarize_args runs first.

what:
- Shape-redact any other-typed sensitive value via _redacted_value_shape
- Doctest that a list-valued sensitive arg is redacted
why: AGENTS.md shipped-vs-branch keeps never-shipped, branch-internal
narrative out of docstrings; round 1 missed a few.

what:
- control_mode.py: drop the "chainable-commands runner /
  libtmux-protocol-engines parser" lineage sentence
- concrete.py: drop "preserving the historical behaviour" (an
  intermediate in-branch state)
- new_pane.py: drop the "first operation gated by min_version" ordinal;
  keep only the current-state VersionUnsupported clause
why: register_plan_tools' docstring said build_workspace is registered
only on the synchronous server, but the async branch registers it too
(dispatching to abuild_workspace) -- misleading a maintainer about tool
exposure.

what:
- State it registers on both servers, per-engine dispatch
why: The session start_directory lands on window 0's first pane, but
confirm() read the window's active_pane. When a non-first pane is
focused (or a split leaves focus off pane 0) with a different cwd,
confirm falsely reported "first pane cwd != declared".

what:
- Check the first window's first pane (index 0), not active_pane
- Live regression: a focused non-first pane at a different cwd still
  confirms ok
why: A tmux restart or socket blip killed the async control-mode
engine permanently -- the reader EOF'd, the engine went dead, and
subscribers had to be torn down. A supervisor that self-heals keeps
the event stream alive across the gap and lets subscribe() rely on
_closing alone instead of a sticky _dead gate.

what:
- Replace the one-shot start()/reader with a supervisor that spawns
  tmux -C, replays desired subscriptions/attach, runs the reader
  inline (one at a time), and reconnects with deterministic jittered
  backoff when the reader returns on EOF.
- Add add_subscription()/set_attach_targets() declarative state,
  replayed on every (re)connect; bump _generation per connect and
  reset parser/pending/attach before the new proc's bytes flow.
- Surface first-connect failure to every start() caller; reset the
  backoff on a healthy connect.
- Gate subscribe() on _closing only: a reconnecting (_dead) engine
  keeps its subscriber so the post-reconnect reader feeds it.
- Cover supervisor reconnect, attach replay, and attach reset;
  drop the now-obsolete _dead-gate sentinel test.
why: The engine's supervisor now reconnects on a tmux restart or
socket blip, but _broadcast_stream_end ends the subscribe stream on
the disconnect, so the pull ring's drain task completes. A bare
``is None`` guard never re-subscribed, silently freezing the cursor
after a reconnect.

what:
- Restart the drain in _EventRing._ensure_started when the prior task
  has completed (not just when unset).
- Clear a prior drain error at the start of _drain (a fresh attempt)
  rather than on restart, so a still-unread failure is not wiped
  before since() surfaces it -- a persistently dead stream must not
  read as empty-but-healthy.
- Cover restart-only-if-done and persistent-error surfacing with
  parametrized tests.
why: "Facade" is not idiomatic Python for these engine-typed classes.
They are the domain-shaped tmux nouns -- server/session/window/pane/
client -- so the package reads as libtmux.experimental.objects, and
an individual engine-bound class is a "wrapper" in prose. Retires the
facade/handle vocabulary.

what:
- Rename src/libtmux/experimental/facade -> objects and its tests,
  keeping the Eager/Lazy/Async class names unchanged.
- Reword package docstrings from "facade"/"handle" to "object".
- Update the three stray package references (ops.results,
  ops.new_pane, docs/experimental) to "wrapper".
why: A proc that connects then instantly dies still consumes its
startup ACK, so the supervisor counted it as a healthy connection and
reset the reconnect backoff to zero every loop. A persistently
flapping proc (fatal config, permission, or resource error) then
fork-stormed tmux at ~10 Hz instead of backing off.

what:
- Reset the backoff only for a connection that survived a minimum
  lifetime; a connect-then-immediately-die counts as a failed attempt,
  so the backoff escalates.
- Add _HEALTHY_CONNECTION_SECONDS and a regression test asserting a
  connect-then-die loop escalates instead of pinning at _backoff(0).
why: A reader that returned via an exception (not a clean EOF) leaves
the tmux -C process alive. The supervisor then reconnected and _spawn
overwrote _proc with the new process without terminating the old one,
orphaning a control client on every such reconnect.

what:
- Terminate a still-alive prior proc at the top of _spawn before
  replacing it; a clean-EOF proc has already exited (no-op).
- Add a regression test that _spawn terminates a live prior proc.
why: Quantify the experimental workspace builder's build cost across
engines and answer "which engine, how fast" reproducibly, without ever
touching the developer's live tmux server.

what:
- Add scripts/bench_engines.py, a self-contained PEP 723 script (uv
  run) that sweeps scenarios x engines x wait-modes and reports
  min/avg/median/p90/p95/p99/max as rich tables + JSON.
- Engines: classic; the builder on subprocess/control_mode/imsg/
  concrete; and a pipelined prototype that batches independent creates
  via run_batch.
- Sandboxed: per-run isolated sockets under a throwaway dir, TMUX
  unset, atexit teardown + orphan backstop -- the default server is
  never contacted.
- Subcommands: run (in-process grid), cell (one build, for hyperfine),
  profile (cProfile).
why: Version the measured build cost across engines (and the pipelining
prototype) alongside the harness so the numbers are reproducible and
reviewable.

what:
- Add scripts/bench-results/ with RESULTS.md (analysis: the engine
  grid, with/without shell-readiness wait, profile, and the ~79x
  reconciliation) plus grid.json / wait.json (raw per-run data from
  scripts/bench_engines.py).
why: The committed benchmark script failed a repo-wide `mypy .` sweep
with 5 errors. Four stem from the `typer` PEP 723 inline dependency,
which the repo's mypy environment cannot resolve (cascading into
untyped-decorator errors); one is a genuine Server|None narrowing gap.
The configured scope (files = [src, tests]) hides them, but a broad
`mypy .` surfaces them.

what:
- Add a file-level disable-error-code for the two typer environment
  artifacts (import-not-found, untyped-decorator), keeping every other
  check strict.
- Accept `Server | None` in ImsgForServer and assert non-None so the
  make_engine callable type narrows correctly.
why: "Concrete" named the in-memory simulator engine, but every real
engine (subprocess, control_mode, imsg) is equally a concrete
implementation. Across the Python/Rust ecosystem "Concrete*" is a
test-only convention for "a minimal instantiable ABC subclass", the
opposite of this docs/doctest workhorse, and the word already carries
its id/type sense throughout ops/query/objects. "Mock" names the engine
by its role: the no-tmux, in-memory stand-in.

what:
- Rename ConcreteEngine -> MockEngine and AsyncConcreteEngine ->
  AsyncMockEngine; module concrete.py -> mock.py
- EngineKind.CONCRETE ("concrete") -> EngineKind.MOCK ("mock");
  EngineSpec.concrete() -> EngineSpec.mock(); registry key becomes
  "mock" (available_engines() re-sorts accordingly)
- Rename the benchmark engine key/label; update RESULTS.md and
  grid.json to match
- Keep docstrings describing the in-memory simulation so the name's
  test-double flavor does not mislead doctest readers
- Leave "concrete" untouched where it means a concrete id/target/pane
  handle (ops, query, objects, workspace)
why: Every engine re-derived the same two things before it could
dispatch: which tmux binary to exec, and which tmux server to point
at. Five copies of the -L/-S/-f/-2/-8 flag construction, three copies
of the memoized shutil.which + `tmux -V` probe, and one drifted copy
in control_mode that called shutil.which unguarded and unmemoized on
every connect.

what:
- Add engines/connection.py with a frozen ServerConnection value
  object owning the connection flags, memoized binary resolution,
  memoized version probe, and argv assembly
- ServerConnection.from_server() is the single mirror of
  Server.cmd()'s flag construction; every engine's for_server() is
  now built on it
- SubprocessEngine, AsyncSubprocessEngine, ControlModeEngine,
  AsyncControlModeEngine and ImsgEngine hold a connection instead of
  re-deriving one; tmux_bin/server_args stay as read-only properties
- Export ServerConnection from libtmux.experimental.engines
why: _chain re-inlined tmux's success rule (`returncode == 0 and not
stderr`) at two attribution sites, so the definitive rule in
ops/results.py could drift out from under them. Worse,
ensure_chainable -- the fail-closed guard against folding an op that
captures output or creates an object -- was dead code: no rendering
path called it, so a capturing op folded into a `;` chain would have
silently mis-attributed its stdout.

what:
- Import status_for from ops/results.py in attribute() and
  attribute_marked() instead of re-inlining the success rule
- Run ensure_chainable over every op in render_chain()
- Run ensure_chainable over render_marked()'s decorates (the create is
  the one non-chainable op the fold tolerates: its captured id is read
  back from stdout[0])
- Document the guard on the module and on both rendering paths, with
  doctests that show the fold failing closed
why: Six helpers had been re-implemented rather than imported, so a
fix to one copy silently left the others behind: the docstring
first-line summary (3 clones), the optional-target resolver, the
workspace on_exists preflight, the caller's socket-path
reconstruction, the plan-outcome projection, and the pane-readiness
poll (2 clones, one of which had already drifted its poll budget).

what:
- Add ops/_docstring.py first_line(); catalog.py, mcp/registry.py and
  mcp/fastmcp_adapter.py import it (the adapter keeps its `| None`
  contract at the call site)
- Add experimental/_wait_pane.py holding the cursor-probe poll;
  fluent.py and workspace/runner.py import wait_pane/await_pane
- mcp/vocabulary/option.py imports _resolve.opt_target
- workspace/sets.py imports runner.py's _preflight_sync/_preflight_async
- mcp/vocabulary/_caller.py extracts socket_path_of() and
  same_socket_path() from the two copies in socket_matches /
  socket_could_match
- mcp/plan_tools.py adds PlanOutcome.to_dict() and _to_outcome(); the
  adapter's plan tools return outcome.to_dict()
why: Six pieces of surface were written and never read. The imsg
protocol's six msg_* accessors and pack_message() were declared in the
ImsgProtocolCodec Protocol and implemented in v8 with no caller, so
the Protocol overstated what a codec must supply. EngineSpec.extra was
never consulted, and the engine registry's docstring advertised a
create_engine(EngineSpec) overload that does not exist. The MCP
descriptor carried effects/version_gates/version_gate that no
projection reads, and middleware kept a tool-batch summarizer for a
shape it no longer sees.

what:
- Drop msg_exit/msg_write_open/msg_write/msg_write_close/msg_read_open/
  msg_exiting and pack_message from the ImsgProtocolCodec Protocol and
  from ProtocolV8Codec
- Drop EngineSpec.extra; correct engines/registry.py's docstring to
  describe the EngineKind it actually accepts
- Drop ToolDescriptor.effects/version_gates and
  ParamDescriptor.version_gate plus the registry lines populating them
- Drop mcp/middleware.py's _summarize_tool_batch_operation_args and the
  _summarize_nested_operation_args indirection over it
- Register the imsg engine under EngineKind.IMSG.value, not a bare string
why: Control mode does not forward pane output verbatim: tmux writes
every non-printable byte -- and the backslash itself -- as a backslash
plus three octal digits. Any reader that scans %output for raw bytes
must undo that first or it can never match. That decoding is transport
knowledge, so it belongs next to render_control_line in the engines
layer rather than in whichever consumer notices the escaping first.

what:
- Add unescape_control_output() to engines/base.py, the inbound
  counterpart to render_control_line: octal escapes decode back to the
  bytes the pane wrote, and bytes tmux left alone pass through, so an
  already-raw payload is unaffected.
- Cover it with doctests over printable text, an OSC escape sequence,
  and multi-byte UTF-8.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant