RFC: Deprecate chat() in favor of composable agentLoop() and atomic text() activities

[jherr]

Summary

This RFC proposes to deprecate the existing chat() activity in favor of a new architecture that clearly separates atomic text generation and agentic workflows:

• text() activity: A simple, atomic, one-shot text generation function with no tools or loop logic. Smaller bundle, predictable behavior, and matches expectations for LLM completion APIs.
• agentLoop() function: A composable orchestrator for agentic workflows — multi-turn, tools, approval flows, and custom agentic logic. Accepts a TextCreator (adapter or composable callback) for maximum flexibility.
• Deprecation of chat(): The combined function is deprecated and users are directed to migrate to agentLoop() (for tools/agentic flow) or text() (for atomic text generation).
• See PR: [WIP] Feat: experimental agent loop #175 ([WIP] Feat: experimental agent loop)

Motivation

The current chat() activity combines two distinct responsibilities:

1. Text generation: Making a single request to an LLM and returning a (streaming or non-streaming) result
2. Agent orchestration: Multi-turn tool execution, approval flows, and configurable looping

This led to:

• More complex code and API: Difficult to tree-shake or reason about
• Large bundles: Users who want only one-shot completions pay for agentic machinery
• Poor composability: No way to "bring your own loop" or layer additional router/middleware logic

Proposal: New Architecture

text() activity (atomic request)

import { experimental_text as text } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

// Streaming
for await (const chunk of text({ adapter: openaiText('gpt-4o'), messages: [{ role:  'user', content: 'Hello!' }]})) {
  // ... 
}
// Non-streaming
await text({ adapter: openaiText('gpt-4o'), messages, stream: false })
// Structured output
await text({ adapter: openaiText('gpt-4o'), messages, outputSchema: z.object({... }) })

• Atomic LLM request (no multi-turn, no tools)
• Smaller bundle (can be tree-shaken)

agentLoop() (agentic orchestration)

Brings the agent loop to the top-level, composable API:

import { experimental_agentLoop as agentLoop } from '@tanstack/ai'
import { openaiText } from '@tanstack/ai-openai'

for await (const chunk of agentLoop({
  adapter: openaiText('gpt-4o'),
  messages,
  tools: [weatherTool],
})) {
  // ...
}

// With structured output
await agentLoop({
  adapter: openaiText('gpt-4o'),
  messages,
  tools: [researchTool],
  outputSchema: z. object({ summary: z.string() })
})

// With callback-style TextCreator for more control
for await (const chunk of agentLoop((opts) => text({
  adapter: openaiText('gpt-4o'),
  ... opts,
  temperature: 0.9,
  modelOptions: { frequencyPenalty:  0.5 }
}), {
  messages,
  tools:  [weatherTool],
})) {
  // ...
}

• All existing agentic features (tools/loop/strategy/approvals)
• Accepts a TextCreator callback for power users (custom adapters, logging, middleware, etc.)
• Standardizes on StreamChunk format for full composability

Migration Examples

1-to-1:

- for await (const chunk of chat({ adapter, messages, tools, ... })) { ... }
+ for await (const chunk of agentLoop({ adapter, messages, tools, ... })) { ... }

Full control:

for await (const chunk of agentLoop((opts) => text({ adapter, ...opts, temperature: 0.9 }), {
  messages,
  tools,
})) { ... }

No tools: use text()

- await chat({ adapter, messages })
+ await text({ adapter, messages })

Deprecation

• chat() is deprecated and will be removed in a future major version
• Users should migrate to agentLoop() or text() as appropriate
• PR [WIP] Feat: experimental agent loop #175 implements the split for feedback and iteration

Benefits

• Tree-shakeability: Tool/loop code isn't included unless you need it
• Composability: Use agentLoop with any compatible function; bring your own agent loop; use router/middleware
• Separation of concerns: Clear, stable contracts for atomic vs. orchestration logic

See also

• PR #175
• The new experimental_text and experimental_agentLoop exports