otelMiddleware: captureContent drops image/audio/video/document parts, recording only '[image]' placeholders #1525

Summary

With captureContent: true, otelMiddleware replaces every non-text content part with a placeholder string before writing gen_ai.input.messages / langfuse.observation.input. A vision call's recorded input reads:

[{"role":"user","content":"Describe this reference image. [image]"}]

The image is gone from the trace. For vision classification this is the part you most need when debugging. For example: why did the model call this image an illustration? You can see the model's answer but not what it looked at.

Root cause

serializeContent (packages/ai/src/middlewares/otel.ts:185) flattens each message to one string. Image, audio, video and document parts become '[image]', '[audio]' and so on, and source is discarded:

case 'image':
  parts.push('[image]')
  break

The result is written as { role, content: string } (line ~584), so there is nowhere to put a structured part even if one survived.

There is no way to work around this from outside the middleware:

• attributeEnricher runs before the captureContent block, so anything it sets on gen_ai.input.messages gets overwritten.
• redact only ever sees the already-flattened string.

Proposal

Keep content parts structured in gen_ai.input.messages, following the OTel GenAI semconv parts shape, which PostHog, Langfuse and Datadog all render:

• text → { type: 'text', content }
• image / audio / video / document with source.type === 'url' → { type: 'uri', modality, uri: source.value, mime_type? }
• source.type === 'data' (inline base64) → keep the placeholder by default, since it would blow maxContentLength and span attribute limits. Optionally add an opt-in hook (e.g. serializePart?: (part) => unknown) so an app can upload the blob and substitute a URL.

redact should still apply to the text parts.

If changing the attribute shape is too breaking for 0.x, an opt-in structuredContent: true flag would also work.

Version

@tanstack/ai 0.58.0 (current main has the same code).

I'm happy to open a PR.