Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/durable-chats-recover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
"agents": minor
"@cloudflare/ai-chat": minor
"@cloudflare/think": minor
---

Make durable chat recovery unconditional for `AIChatAgent` and `Think`.

Every chat turn now runs in a recovery fiber, including WebSocket, programmatic, retry, and continuation paths. `chatRecovery` accepts `true` or a configuration object; `false` is no longer supported. Previously compiled JavaScript that still supplies `false` safely receives the default recovery configuration.

To keep durable bookkeeping while preventing automatic inference after an interruption, return `{ continue: false }` from `onChatRecovery()`. Use durable cancellation, side-effect, or spend state in that hook and tune `chatRecovery` budgets when retries must be bounded.
2 changes: 1 addition & 1 deletion design/chat-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ export class MyAgent extends AIChatAgent<Env> {
| `this.messages` | `ChatMessage[]` | Current conversation history |
| `maxPersistedMessages` | `number \| undefined` | Storage cap |
| `messageConcurrency` | `MessageConcurrency` | Overlap strategy |
| `chatRecovery` | `boolean` | Fiber-wrapped turns |
| `chatRecovery` | `ChatRecoveryConfig` | Tune always-on durable recovery |
| `waitForMcpConnections` | `boolean \| { timeout }` | MCP wait |
| `saveMessages(msgs)` | method | Programmatic turn |
| `continueLastTurn(body?)` | method | Continue last assistant message |
Expand Down
2 changes: 2 additions & 0 deletions design/chat-shared-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,8 @@ Pure functions for aligning client messages with server state during persistence

**`ChatRecoveryEngine`** owns the **shared durable chat-recovery orchestration** — the sequence both `AIChatAgent` and `Think` run when a Durable Object wakes and finds an interrupted chat turn (a `runFiber` that died mid-stream from hibernation, process death, or deploy churn). This state machine was previously duplicated across both packages, and the duplication was already drifting (better fixes landing in one but not the other).

Durable chat recovery is an invariant in both hosts: every chat entry path runs inside a recovery fiber. `ChatRecoveryConfig` accepts `true` or a tuning object, not `false`; previously compiled JavaScript that still supplies `false` is resolved to the default configuration. This guarantees that agent-tool child inspection can use durable recovery state after a restart instead of relying on instance-local abort controllers or stream managers.

**Two host-supplied seams:**

- **`ChatRecoveryAdapter`** — the incident/budget I/O the engine drives (read/write/sweep incidents, read progress, emit lifecycle events, resolve the recovery stream, give-up/exhaust). Stable across a session.
Expand Down
2 changes: 1 addition & 1 deletion design/think-vs-aichat.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Related:
| **Programmatic turns** | `saveMessages(messages)` | `saveMessages(messages)` (same) |
| **Continuation** | `continueLastTurn(body?)` — appends to existing message (chunk rewriting) | `continueLastTurn(body?)` — creates new message (append deferred) |
| **Concurrency** | `messageConcurrency` (queue/latest/merge/drop/debounce) | `messageConcurrency` (same strategies, merge is non-destructive) |
| **Durability** | `chatRecovery` + `runFiber` | `chatRecovery` + `runFiber` (same) |
| **Durability** | Always-on recovery fibers; `chatRecovery` tunes budgets | Always-on recovery fibers; `chatRecovery` tunes budgets (same) |
| **Stability** | `waitUntilStable()` / `hasPendingInteraction()` | `waitUntilStable()` / `hasPendingInteraction()` (same) |
| **Turn reset** | `resetTurnState()` (protected) | `resetTurnState()` (protected) |
| **onStart** | Must call `super.onStart()` | Constructor wrapping — no `super.onStart()` needed |
Expand Down
25 changes: 13 additions & 12 deletions docs/agents/chat-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -590,17 +590,11 @@ If you do not pass `abortSignal` to `streamText`, the LLM call will continue run

### Stream Recovery

When a Durable Object is evicted mid-stream (code update, inactivity timeout, resource limit), the LLM connection is severed permanently and the in-memory streaming state is lost. `chatRecovery` wraps each chat turn in a [`runFiber()`](./durable-execution.md), providing automatic `keepAlive` during streaming and a recovery hook on restart.
When a Durable Object is evicted mid-stream (code update, inactivity timeout, resource limit), the LLM connection is severed permanently and the in-memory streaming state is lost. Durable recovery wraps every `AIChatAgent` and `Think` chat turn in a [`runFiber()`](./durable-execution.md), providing automatic `keepAlive` during streaming and a recovery hook on restart.

```typescript
export class ChatAgent extends AIChatAgent {
override chatRecovery = true;
}
```

When enabled, every `onChatMessage` call runs inside a fiber. If the agent is evicted mid-stream, the fiber row survives in SQLite. On the next activation, the framework detects the interrupted fiber, reconstructs the partial response from buffered stream chunks, and calls `onChatRecovery`.
If the agent is evicted mid-stream, the fiber row survives in SQLite. On the next activation, the framework detects the interrupted fiber, reconstructs the partial response from buffered stream chunks, and calls `onChatRecovery`.

`AIChatAgent` defaults `chatRecovery` to `false` so existing chat agents only get client reconnect/resumable-stream behavior. `Think` defaults it to `true`.
Durable recovery is always enabled. Use `chatRecovery` only to tune its budgets and terminal behavior.

> **Assign `chatRecovery` as a class field or in the constructor — never in `onStart()`.** On every wake the SDK evaluates recovery budgets (and may seal an interrupted turn, firing `onExhausted`) _before_ your `onStart()` body runs. A config produced inside `onStart()` is therefore read as the built-in defaults at the moment recovery decides, so your `maxRecoveryWork` / `shouldKeepRecovering` / `onExhausted` silently never apply to the recovery that matters. The SDK logs a one-time warning if it detects `chatRecovery` being assigned during `onStart()`.

Expand All @@ -610,8 +604,6 @@ Override to implement provider-specific recovery. The default behavior persists

```typescript
export class ChatAgent extends AIChatAgent {
override chatRecovery = true;

override async onChatRecovery(
ctx: ChatRecoveryContext
): Promise<ChatRecoveryOptions> {
Expand Down Expand Up @@ -659,7 +651,16 @@ Settled work is never dropped: `persist: false` only suppresses persistence of a

When recovery happens before any stream chunks were written, there is no partial assistant message to continue. If the latest persisted message is still the unanswered user message from the interrupted turn, the framework retries that turn automatically unless `continue` is `false`.

`chatRecovery` can also be configured with budgets and terminal behavior:
#### Controlling automatic continuation

Durable bookkeeping stays enabled even when automatic continuation is not appropriate:

- **Retries or side effects are unsafe:** override `onChatRecovery()` and return `{ continue: false }`. Persist idempotency keys or completion records before external side effects so a recovered turn can tell whether work already happened.
- **Cancellation must survive eviction:** an `AbortSignal` only cancels the current in-memory turn. Also persist cancellation intent in agent state or SQL, read it in `onChatRecovery()`, and return `{ continue: false }` when cancellation was requested.
- **Cost must be bounded:** set `maxAttempts`, `noProgressTimeoutMs`, `maxRecoveryWork`, and `maxOomRetries`. Use `shouldKeepRecovering` with durable spend data to stop later attempts. The predicate is not bound to the agent instance, so read spend from a store keyed by `ctx.recoveryRootRequestId`.
- **A provider can resume without a new model call:** use `this.stash()` to save its response ID, retrieve that response in `onChatRecovery()`, and return `{ persist: false, continue: false }`.

`chatRecovery` can be configured with budgets and terminal behavior:

```typescript
override chatRecovery = {
Expand Down
2 changes: 1 addition & 1 deletion docs/agents/durable-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,7 +466,7 @@ Key points:

### Chat recovery

`AIChatAgent` builds on fibers for LLM streaming recovery. When `chatRecovery` is enabled, each chat turn is wrapped in a fiber automatically. The framework handles the internal recovery path and exposes `onChatRecovery` for provider-specific strategies. See [Long-Running Agents: Recovering interrupted LLM streams](./long-running-agents.md#recovering-interrupted-llm-streams) and the [`forever-chat` example](https://github.com/cloudflare/agents/tree/main/experimental/forever-chat).
`AIChatAgent` and `Think` build on fibers for LLM streaming recovery. Every chat turn is wrapped in a fiber automatically. The framework handles the internal recovery path and exposes `onChatRecovery` for provider-specific strategies. See [Long-Running Agents: Recovering interrupted LLM streams](./long-running-agents.md#recovering-interrupted-llm-streams) and the [`forever-chat` example](https://github.com/cloudflare/agents/tree/main/experimental/forever-chat).

## Concurrent fibers

Expand Down
2 changes: 1 addition & 1 deletion docs/agents/human-in-the-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ See the complete example: [guides/human-in-the-loop/](https://github.com/cloudfl

### Surviving restarts while waiting for a human

A Durable Object can be evicted at any time (a deploy, an inactivity timeout, a resource limit), including while a turn is paused on an approval prompt or a client-side tool call. When [`chatRecovery`](./chat-agents.md#stream-recovery) is enabled (the default for `Think`), the SDK recognizes that such a turn is _waiting on the human_, not stuck, and does **not** seal it: the no-progress window, attempt cap, `maxRecoveryWork`, and `shouldKeepRecovering` are all suspended while the interaction is pending. Recovery parks the turn instead of failing it, and the user's eventual approval or `tool_result` resumes the conversation through the normal continuation path. A user who takes minutes to respond to a prompt that was interrupted by a deploy therefore does not see a spurious "session interrupted" error.
A Durable Object can be evicted at any time (a deploy, an inactivity timeout, a resource limit), including while a turn is paused on an approval prompt or a client-side tool call. Durable [`chatRecovery`](./chat-agents.md#stream-recovery) is always enabled. The SDK recognizes that such a turn is _waiting on the human_, not stuck, and does **not** seal it: the no-progress window, attempt cap, `maxRecoveryWork`, and `shouldKeepRecovering` are all suspended while the interaction is pending. Recovery parks the turn instead of failing it, and the user's eventual approval or `tool_result` resumes the conversation through the normal continuation path. A user who takes minutes to respond to a prompt that was interrupted by a deploy therefore does not see a spurious "session interrupted" error.

This protection applies to interactions only the client can resolve — `approval-requested` parts and `input-available` parts for client-side tools (those without a server `execute`). A server tool whose `execute()` was killed mid-flight is a genuine orphan and recovers through the normal transcript-repair path instead.

Expand Down
6 changes: 2 additions & 4 deletions docs/agents/long-running-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -537,7 +537,7 @@ For chat-oriented sub-agents, [Think](https://github.com/cloudflare/agents/blob/

The patterns above handle the project manager's coordination work — scheduling, delegating, polling. But the project manager also uses an LLM directly: generating plans, summarizing progress, drafting status emails. Those LLM calls stream tokens over a connection that cannot be resumed if the agent is evicted mid-response.

For chat-oriented agents built on `AIChatAgent`, this is an even sharper problem — the user is watching the response stream in real time and sees it stop mid-sentence. `chatRecovery` wraps each chat turn in a `runFiber`, providing automatic `keepAlive` during streaming and a recovery hook when the agent restarts:
For chat-oriented agents built on `AIChatAgent` or `Think`, this is an even sharper problem — the user is watching the response stream in real time and sees it stop mid-sentence. Durable recovery wraps every chat turn in a `runFiber`, providing automatic `keepAlive` during streaming and a recovery hook when the agent restarts:

```typescript
import { AIChatAgent } from "@cloudflare/ai-chat";
Expand All @@ -547,8 +547,6 @@ import type {
} from "@cloudflare/ai-chat";

class ProjectChat extends AIChatAgent<Env> {
override chatRecovery = true;

override async onChatRecovery(
ctx: ChatRecoveryContext
): Promise<ChatRecoveryOptions> {
Expand All @@ -573,7 +571,7 @@ The right recovery strategy depends on the LLM provider:

For a complete multi-provider implementation with full code for each strategy, see the [`forever-chat` example](https://github.com/cloudflare/agents/tree/main/experimental/forever-chat) and the [`forever.md` design doc](https://github.com/cloudflare/agents/tree/main/experimental/forever.md).

[Think](https://github.com/cloudflare/agents/blob/main/docs/think/index.md) enables `chatRecovery` by default. The default path persists partial output and auto-continues or retries the turn when safe, so many apps do not need a custom hook. Override `onChatRecovery` when a provider has a better recovery strategy, or configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal user experience.
`AIChatAgent` and [Think](https://github.com/cloudflare/agents/blob/main/docs/think/index.md) always enable durable recovery. The default path persists partial output and auto-continues or retries the turn when safe, so many apps do not need a custom hook. Override `onChatRecovery` when a provider has a better recovery strategy, or configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal user experience.

If the agent is interrupted before any assistant stream chunks are written, there is no partial assistant message to continue. When the latest persisted message is still the unanswered user message from that turn, chat recovery retries the turn automatically unless `onChatRecovery` returns `{ continue: false }`.

Expand Down
Loading
Loading