Absorb user-DO resets in the Worker - #133
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
4b76f09 to
58866bc
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
GPT comments: |
Routine user-DO resets (storage timeouts, overload aborts — the errors seen in production logs) were wedging sessions: a stub is bound to one incarnation and permanently broken once it resets (DO error-handling docs), so a cached stub turned one reset into every subsequent call failing until reload. - Fresh stub per call: #user re-resolves on every access, so a post-reset call simply restarts the object and the session self-heals structurally. - Every user-DO RPC goes through the #userCall / #userCommand choke points — a naked this.#user.x() is a review defect. Typed #query/#command Proxies route the ~35 plain delegation sites through them while deriving the telemetry operation name from the DO method. The split is about ordering: commands are effectful and serialize on a per-session chain (per-call stubs forfeit cross-stub e-order, and overlapping optimistic writes like rapid pin toggles must arrive in issue order); queries stay concurrent. - Deliberately NO retries: fresh stubs already fix the wedged-session mode, what remains is only the call in flight at the reset moment, workerd's structured reset flags reach the client (which classifies and quiets them, #110), and the DO platform is moving toward transparent recovery. do-reset.ts keeps the flag predicate (durableObjectReset/retryable, never bare overloaded) purely to classify surfaced resets for telemetry (user_do.reset.surfaced) — the volume check on this design's thesis. - authenticate() rejects corrupt base64 tokens as coded auth failures instead of leaking the decoder's SyntaxError. Integration tests pin that local vitest-pool-workers aborts reject FLAGLESS (so the flag predicate is unit-tested with production-shaped synthetic errors, and a pool upgrade that adds real flags fails loudly) and cover same-session recovery across abortAllDurableObjects(). Deliberately deferred: self-healing of connected-accounts subscriptions across resets (the DO's in-memory registrations die with the incarnation, so a subscribed browser silently stops receiving updates until it re-subscribes or the socket reconnects). A TODO(deferred) at the subscribe chokepoint marks the gap; the implementation is split out to the follow-up branch feat/do-reset-subscription-self-heal.
58866bc to
1b25391
Compare
Re: the GPT comments (Maximo-Guk)I traced all three against the actual diff. All three describe the superseded #104 changeset (which shipped a 1. " 2. "queries overtake earlier commands, regressing e-order" — deliberate and documented. 3. "auth retries resets without emitting telemetry" — no auth retries exist. On the earlier review nits
Net: no in-scope code change needed. The three GPT items are all stale against #104's retry design; the fresh-stub-no-retry design here is intentionally free of the idempotency/overload/telemetry hazards they raise. |
|
Posted my analysis to PR #133. SummaryI reviewed the three actionable GPT comments (from Maximo-Guk) against the actual diff. All three describe the superseded #104 changeset, not this PR — the retry layer was built and removed on review, so none of the hazards apply:
I also verified the two stale ask-bonk nits: No in-scope code changes were warranted — making spurious edits would have been the wrong call given the comments are all stale against #104's retry design. |
Routine Durable Object resets — storage timeouts, overload aborts; ~3k over a recent 4-day window in production logs — were wedging sessions: a stub is permanently broken once its incarnation resets, so a cached stub turned one reset into every subsequent call failing until reload, surfacing as scary terminal errors.
What changed
Fresh stub per call. The session re-resolves its user-DO stub on every access, so a post-reset call simply restarts the object
Deliberately no retries. With fresh stubs, the only thing a reset can still break is the call in flight at that moment; workerd's structured reset flags reach the client
Deliberately deferred / out of scope
Testing
abortAllDurableObjects(), the non-graceful teardown): the same session recovers transparently after a reset via fresh stubs, and the local-aborts-reject-flagless shape is pinned so the flag path can graduate from synthetic unit tests to real-reset integration tests if a future pool upgrade attaches the production flags.debugAbort()injector, composed with the Classify RPC errors client-side and quiet recoverable failures #110 client): run on the pre-split branch — 72 aborts across idle, reload-race, and SPA-navigation-storm scenarios with every page functional and zero application console output; a chat send racing a workspace-DO abort showed the inline hint (Classify RPC errors client-side and quiet recoverable failures #110's UI), preserved the draft, and did not double-send. The subscription-specific results move to the follow-up PR.