Skip to content

fix(ai-anthropic): pass system prompts as TextBlockParam[] to support prompt caching#393

Open
imsherrill wants to merge 2 commits intoTanStack:mainfrom
imsherrill:fix/anthropic-system-prompt-caching
Open

fix(ai-anthropic): pass system prompts as TextBlockParam[] to support prompt caching#393
imsherrill wants to merge 2 commits intoTanStack:mainfrom
imsherrill:fix/anthropic-system-prompt-caching

Conversation

@imsherrill
Copy link
Contributor

@imsherrill imsherrill commented Mar 21, 2026

Changes

The Anthropic adapter joins system prompts into a plain string before passing them to the API:

system: options.systemPrompts?.join('\n'),

This prevents using Anthropic's prompt caching on system prompts. The Anthropic SDK accepts system as either string or TextBlockParam[], and only the array form supports cache_control.

This PR converts systemPrompts to TextBlockParam[] instead of joining to a string. Users who want cache_control can pass structured blocks via modelOptions.system, which already takes priority via the provider options spread.

  • No breaking changes — existing systemPrompts: ['...'] usage works identically
  • The InternalTextProviderOptions.system type already accepts string | Array<TextBlockParam>
  • Other adapters (OpenAI, Gemini, Groq, etc.) are unaffected — this is Anthropic-specific

Closes #379

Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested this code locally with pnpm run test:pr.

Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Summary by CodeRabbit

  • Refactor

    • Improved system prompt handling in the Anthropic adapter to process multiple prompts as structured blocks instead of joined text.
  • Tests

    • Added test coverage for system prompt array conversion and model options precedence validation.

… prompt caching

The Anthropic adapter joined system prompts into a plain string before
passing them to the API. This prevented using Anthropic's prompt caching
on system prompts, since only the array form (TextBlockParam[]) supports
cache_control. Convert systemPrompts to TextBlockParam[] so that users
can override with cache_control via modelOptions.system.
…ersion

Verify that systemPrompts are passed as TextBlockParam[] (not a joined
string) and that modelOptions.system with cache_control takes precedence.
@nx-cloud
Copy link

nx-cloud bot commented Mar 21, 2026

View your CI Pipeline Execution ↗ for commit 7ad955e

Command Status Duration Result
nx run-many --targets=build --exclude=examples/** ✅ Succeeded 43s View ↗
nx affected --targets=test:sherif,test:knip,tes... ✅ Succeeded 10s View ↗

☁️ Nx Cloud last updated this comment at 2026-03-21 13:44:53 UTC

@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 21, 2026

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/@tanstack/ai@393

@tanstack/ai-anthropic

npm i https://pkg.pr.new/@tanstack/ai-anthropic@393

@tanstack/ai-client

npm i https://pkg.pr.new/@tanstack/ai-client@393

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/@tanstack/ai-devtools-core@393

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/@tanstack/ai-elevenlabs@393

@tanstack/ai-event-client

npm i https://pkg.pr.new/@tanstack/ai-event-client@393

@tanstack/ai-fal

npm i https://pkg.pr.new/@tanstack/ai-fal@393

@tanstack/ai-gemini

npm i https://pkg.pr.new/@tanstack/ai-gemini@393

@tanstack/ai-grok

npm i https://pkg.pr.new/@tanstack/ai-grok@393

@tanstack/ai-groq

npm i https://pkg.pr.new/@tanstack/ai-groq@393

@tanstack/ai-ollama

npm i https://pkg.pr.new/@tanstack/ai-ollama@393

@tanstack/ai-openai

npm i https://pkg.pr.new/@tanstack/ai-openai@393

@tanstack/ai-openrouter

npm i https://pkg.pr.new/@tanstack/ai-openrouter@393

@tanstack/ai-preact

npm i https://pkg.pr.new/@tanstack/ai-preact@393

@tanstack/ai-react

npm i https://pkg.pr.new/@tanstack/ai-react@393

@tanstack/ai-react-ui

npm i https://pkg.pr.new/@tanstack/ai-react-ui@393

@tanstack/ai-solid

npm i https://pkg.pr.new/@tanstack/ai-solid@393

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/@tanstack/ai-solid-ui@393

@tanstack/ai-svelte

npm i https://pkg.pr.new/@tanstack/ai-svelte@393

@tanstack/ai-vue

npm i https://pkg.pr.new/@tanstack/ai-vue@393

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/@tanstack/ai-vue-ui@393

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/@tanstack/preact-ai-devtools@393

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/@tanstack/react-ai-devtools@393

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/@tanstack/solid-ai-devtools@393

commit: 7ad955e

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 61ae4c84-f59d-44ee-bc66-714ced332e97

📥 Commits

Reviewing files that changed from the base of the PR and between c3583e3 and 7ad955e.

📒 Files selected for processing (2)
  • packages/typescript/ai-anthropic/src/adapters/text.ts
  • packages/typescript/ai-anthropic/tests/anthropic-adapter.test.ts

📝 Walkthrough

Walkthrough

The Anthropic adapter's system prompt handling was updated to pass system prompts as an array of text blocks ({ type: 'text', text } objects) instead of a joined string, enabling support for Anthropic's prompt caching feature on system prompts. Tests were added to validate the new array format and cache control behavior.

Changes

Cohort / File(s) Summary
Adapter Implementation
packages/typescript/ai-anthropic/src/adapters/text.ts
Modified system parameter construction to convert options.systemPrompts into an array of TextBlockParam objects when non-empty, replacing the previous join('\n') approach. Sets system to undefined when empty.
Test Coverage
packages/typescript/ai-anthropic/tests/anthropic-adapter.test.ts
Added two test cases validating system prompt conversion to array format and cache control precedence behavior when modelOptions.system overrides systemPrompts.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 Hops of joy with whisker wiggles
System prompts now dance as arrays bright,
Cache control brings savings so delightful,
No more string joins—just elegant blocks!
Ninety percent savings? That's magical might!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: converting system prompts from joined strings to TextBlockParam[] to enable prompt caching support.
Description check ✅ Passed The PR description follows the repository template with completed 'Changes' section and both checklist sections marked as done. All required information is present.
Linked Issues check ✅ Passed The PR fully addresses issue #379 by converting system prompts to TextBlockParam[] format instead of joined strings, enabling Anthropic prompt caching support while maintaining backward compatibility.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the linked issue: adapter implementation in text.ts and comprehensive tests in anthropic-adapter.test.ts. No unrelated modifications detected.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@tanstack/ai-anthropic doesn't support Anthropic prompt caching on system prompts

1 participant