Skip to content

Absorb user-DO resets in the Overseer's session capabilities - #162

Merged
ndisidore merged 2 commits into
mainfrom
fix/overseer-userdo-recover
Aug 11, 2026
Merged

Absorb user-DO resets in the Overseer's session capabilities#162
ndisidore merged 2 commits into
mainfrom
fix/overseer-userdo-recover

Conversation

@ndisidore

@ndisidore ndisidore commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

When a user DO resets under an open workspace, the session stays broken until the browser reconnects. Chat sends fail, the model picker and gatekeeper setup wedge until a reload, and pin/title updates stop propagating. The cause is the same one #133 fixed at the Worker layer: the Overseer minted its user-DO stubs once at open() and reused them for the whole session, and a stub is permanently broken once its incarnation dies.

Changes

The session capability classes now store the user ID and create a fresh stub for each call, through the same telemetry wrapper the Worker uses.

Deliberately out of scope

No retries, matching #133. No call deadline either: a user DO wedged behind a stuck input gate hangs newChat silently for an unbounded-but-usually-finite window (i.e. the 30s DO timeout), then rejects (and now recovers). Will add as a followup if we think its necessary

The Worker side already re-resolves its user-DO stub on every call
(#133), but the Overseer kept the older design: open() minted owner and
clientUser stubs once and the session capability classes
(OverseerClientInterface, UseOverseerInterface, GadgetClientImpl,
UseGadgetClientInterface) retained them for the session's lifetime. A
stub is bound to one incarnation and permanently broken once it resets,
so after a routine user-DO reset every user-DO-carrying call on the
still-open session -- newChat, sendChatMessage, listModels, setPinned,
createGadget, connector creation, owner metadata -- kept failing until
the WebSocket reconnected.

Apply the same fix one layer down: the capability classes store the
user ID and mint a fresh stub per call through #owner/#clientUser
getters, wrapped in wrapDoStubForTelemetry. The wrapper now takes an
optional logger so Overseer-observed resets attribute to
workshop.overseer with the gadgetId, distinct from the Worker
chokepoint's workshop.server events.

The getters must be native #-private: capnweb exposes prototype getters
over RPC (instance properties are blocked, getters are the documented
opt-in), so a TypeScript-private getter would have handed browsers the
raw user-DO stub -- including getChatContext(), which returns API keys.

GadgetClient creation sites pass the ID, not a stub: handing a child
capability the getter's value would give it a fresh stub that it then
stores for its lifetime, reintroducing the bug at gadget scope.

Cross-call e-order to the user DO is lost by design, matching the
Worker layer: the capability calls are pure reads or guarded writes
(updateTitle/updatePinned throw if the record is missing rather than
upsert), and the one multi-call sequence in open() awaits sequentially.

Deliberately NO retries (per #133) and NO call deadline: a user DO
wedged behind a stuck input gate still hangs newChat indefinitely.
That needs a bounded-timeout mechanism and is left as a follow-up.

The integration test pins the asymmetric reset the retained-stub design
could not absorb: the user DO alone is aborted via runInDurableObject
(abortAllDurableObjects() would kill the Overseer too), then chat,
model listing, and gadget creation succeed through the same retained
workspace capability. It fails against the previous overseer.ts with
the poisoned-stub rejection and passes with this change.
@github-actions github-actions Bot added the kernel Changes to the Workshop kernel label Aug 11, 2026
@ask-bonk

ask-bonk Bot commented Aug 11, 2026

Copy link
Copy Markdown

This is consistent. The whitelist matches the USER_DO_ABORT_REASON constant, and the comment cross-reference is accurate.

I've completed a thorough review. Let me summarize my assessment.

Review

This is a clean, well-motivated fix. The change mirrors the existing AuthenticatedApiImpl.#user pattern (server.ts:96) in the Overseer's session capabilities: instead of holding a single user-DO stub captured at open(), each of OverseerClientInterface, UseOverseerInterface, GadgetClientImpl, and UseGadgetClientInterface now stores the user ID and mints a fresh telemetry-wrapped stub per call. That correctly absorbs a user-DO-only reset, which the retained-stub design couldn't.

Verification points I checked:

  • All prior this.owner / this.clientUser field references are gone — no dangling uses (grep confirms zero).
  • impl.users, impl.ownerId, and impl.logger all exist with matching types (impl.logger is ReturnType<typeof createWorkshopLogger>, matching the new wrapDoStubForTelemetry param).
  • The #owner getter's this.impl.ownerId! non-null assertion is no less safe than the old open() code, which used the same ownerId! to mint the stub. deleteSelf reads #owner before nulling ownerId.
  • .id.toString() analytics calls still work: .id is a non-function property, returned directly by the proxy.
  • The do-telemetry.ts logger-injection change is backward compatible (defaults to the Worker logger) and threads the Overseer's gadget-scoped logger through, satisfying the PR's telemetry-attribution goal.
  • The new integration test correctly isolates the user-DO reset via runInDurableObject + state.abort() (rather than abortAllDurableObjects, which would kill the Overseer too), and the onUnhandledError whitelist matches the USER_DO_ABORT_REASON constant.

Observations (non-blocking, no action needed):

  • The analytics .id.toString() sites now construct a throwaway stub solely to read the id, which is slightly wasteful but harmless and consistent with the pattern.
  • UseOverseerInterface still calls this.#clientUser.whoami() directly for presence while OverseerClientInterface memoizes via #getClientProfile(). This asymmetry predates the PR and is preserved, not introduced.
  • The "no call deadline" gap is explicitly and correctly deferred in the PR description.

LGTM!

github run

@ndisidore
ndisidore marked this pull request as ready for review August 11, 2026 22:12
Comment thread packages/workshop-backend/src/overseer.ts
Two findings from an audit of the fresh-stub change:

The #owner getters read this.impl.ownerId with a non-null assertion,
but deleteSelf() clears ownerId ~100ms before the revocation restart
aborts the object, so a concurrent call on a still-open session could
hit idFromString(undefined) and throw a raw TypeError. Guard with the
same "Workspace has been deleted." error #ownerUserStub() uses.

Losing cross-call e-order makes one frontend sequence newly reversible:
the in-workspace rename (handleSaveTitle) runs on the long-lived
session with nothing disabling the save affordance mid-flight, so two
rapid saves could land at the user DO in either order and leave the
home-page listing title diverged from the workspace title. Ignore
saves while one is in flight, matching the in-flight guards on the
blueprint pin and quick-model toggles.
@github-actions github-actions Bot added the workshop/frontend Changes to the Workshop frontend label Aug 11, 2026
@ask-bonk

ask-bonk Bot commented Aug 11, 2026

Copy link
Copy Markdown

The #owner getter is used in whoami/metadata paths and setTitle/deleteSelf/getCreatedBy. If ownerId is undefined, the workspace has been deleted, and throwing "Workspace has been deleted." is a correct behavior (previously the ownerId! would have created a stub with idFromString(undefined) which would throw a less meaningful error anyway). This is an improvement.

Note #clientUser getter has no such guard, but clientUserId is a constructor-injected constant that's always set, so no guard is needed. Correct.

I've completed a thorough review. Let me consolidate my findings.

Review Summary

This is a well-motivated, correctly-implemented fix. The core change replaces retained user-DO stubs (captured once at open()) with per-call fresh-stub getters (#owner, #clientUser) across the four session capability classes, mirroring the existing AuthenticatedApiImpl.#user pattern that #133 established at the Worker layer. A broken (dead-incarnation) stub can no longer wedge an open session; the first post-reset call transparently restarts the object.

Points I verified:

  • No dangling references: grep confirms zero remaining this.owner / this.clientUser field references. The owner/clientUser locals that remain in open() (overseer.ts:6404-6407, 6458-6477) are one-shot connect-time operations, correctly left out of the retained-stub scope.
  • Constructor call sites updated to pass clientUserId: string instead of stubs; GadgetClientImpl/UseGadgetClientInterface likewise threaded the id through.
  • deleteSelf ordering is safe (overseer.ts:7360-7365): this.#owner is read at line 7361, and ownerId is nulled at 7364 within the same blockConcurrencyWhile — the getter evaluates before the null.
  • #owner deleted-workspace guard (the audit-findings commit) is a strict improvement: it throws a meaningful "Workspace has been deleted." instead of the old ownerId! producing idFromString(undefined).
  • .id.toString() analytics sites still work: .id is a non-function property returned directly by the telemetry Proxy (do-telemetry.ts:33), so the throwaway stub is harmless.
  • do-telemetry.ts logger injection is backward compatible (defaults to the Worker logger) and threads the gadget-scoped impl.logger (ReturnType<typeof createWorkshopLogger>, matching the param type) for correct attribution.
  • Integration test correctly isolates the user-DO reset via runInDurableObject + state.abort() (not abortAllDurableObjects, which would kill the Overseer too); it lives in a new non-skipped describe block, and the onUnhandledError whitelist string matches the USER_DO_ABORT_REASON constant.
  • GadgetEditor.tsx in-flight guard is a correct, minimal idempotency hardening (set before the await, cleared in finally) for the now-retriable setTitle.

Non-blocking observations (no action needed):

  • The UseOverseerInterface presence path calls this.#clientUser.whoami() directly while OverseerClientInterface memoizes via #getClientProfile(). This asymmetry predates the PR and the fresh-stub cost is one-shot at construction, so it's immaterial.
  • The "no call deadline" gap (a user DO wedged behind a stuck input gate hangs until the ~30s DO timeout) is explicitly and reasonably deferred in the PR description.

LGTM!

github run

@ndisidore
ndisidore merged commit 213ea6a into main Aug 11, 2026
8 checks passed
@ndisidore
ndisidore deleted the fix/overseer-userdo-recover branch August 11, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel Changes to the Workshop kernel workshop/frontend Changes to the Workshop frontend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants