otelMiddleware: structured-output calls with no tools emit no iteration span — no generation record and captureContent is a silent no-op

[tombeckenham]

Summary

otelMiddleware emits no iteration span for a structured-output call that has no tools. Those runs produce a root chat span and nothing else, so:

• no per-generation span is exported (backends that key off it — PostHog's $ai_generation — see the call as a bare trace with no generation inside it), and
• captureContent captures nothing, because prompt/completion content is only ever attached to iteration spans.

Token usage still lands on the root span, so the run looks present but empty: you can see that a model was called and how many tokens it burned, but not what was sent, what came back, or a generation record to attribute cost against.

This is the common shape for non-chat LLM work — schema-constrained extraction/classification with no tools — so in an app built mostly from outputSchema calls, generation-level observability is close to empty.

Root cause

The agent loop is skipped when a separate structured-output finalization will run (packages/ai/src/activities/chat/index.ts:850):

const skipAgentLoop =
  !!this.finalStructuredOutput &&
  this.tools.length === 0 &&
  this.finalStructuredOutput.nativeCombined !== true

if (!skipAgentLoop) {
  do {
    if (this.cyclePhase === 'processText') {
      // Run onConfig before each model call (phase = beforeModel)
      this.middlewareCtx.phase = 'beforeModel'
      ...

otelMiddleware opens its iteration span from that hook, and only that hook (packages/ai/src/middlewares/otel.ts:402):

onConfig(ctx, config) {
  if (ctx.phase !== 'beforeModel') return
  ...
  const info: OtelSpanInfo<'iteration'> = { kind: 'iteration', ctx, iteration: ctx.iteration }

runStructuredFinalization does re-run the general-purpose config hook (index.ts:2110), but it sets a different phase first (index.ts:2086):

this.middlewareCtx.phase = 'structuredOutput'

So the finalization request — the one that actually calls the provider on this path — reaches otelMiddleware.onConfig and is dropped by the phase !== 'beforeModel' guard. No iteration span is opened, so nothing downstream of it runs: no gen_ai.input.messages / gen_ai.output.messages (otel.ts:549, otel.ts:617), no gen_ai.choice events, no per-generation span.

Reproduction

const otel = otelMiddleware({ tracer, captureContent: true })

// No tools + outputSchema → one root span, no iteration span, no content.
await chat({
  adapter: openRouterText('x-ai/grok-4.5'),
  messages: [{ role: 'user', content: 'Describe this scene' }],
  outputSchema: z.object({ description: z.string() }),
  middleware: [otel],
  stream: false,
})

// Same call without outputSchema → root + iteration span, content captured.

Observed

Production data from an app where every structured call goes through this path. $ai_trace is what PostHog creates from the root span; $ai_generation from the iteration span:

span name	traces	generations
phase-3-visual-prompts	5	0
recommendStylesForScript	2	0
phase-1-scene-splitting	1	0
script-enhance (no schema)	1	1

Only the one call without outputSchema produced a generation. Content matches: the generation carries 12,187 chars of input and 1,569 of output; every structured call has input/output_choices null.

Expected

A structured-output finalization is a real model call and should be observable as one. Either:

1. Open an iteration span for the finalization request. Have otelMiddleware.onConfig also handle phase === 'structuredOutput', or have the engine drive the finalization's onConfig with phase = 'beforeModel' so it looks like the model call it is. This seems closest to intent — the span tree then always has a leaf per provider call.
2. Failing that, mirror input/output onto the root span when no iteration span was opened, so captureContent isn't silently a no-op on this path.

Option 1 also fixes the metric side: gen_ai.client.token.usage is recorded off the iteration path, so these calls are missing from the token histogram too.

Happy to open a PR if you agree with the direction — option 1, guarded so a run that did go through the agent loop doesn't double-count.

Environment

• @tanstack/ai 0.42.0 (0.43.0 changelog shows no related change)
• @tanstack/ai-openrouter 0.15.10
• Cloudflare Workers (workerd), private BasicTracerProvider + OTLP HTTP exporter