Prompt caching for Claude

[tpaulshippy]

What this does

Support prompt caching in both Anthropic and Bedrock providers for Claude models that support it.

Caching system prompts:

chat = RubyLLM.chat
chat.with_instructions("You are a helpful assistant.")
chat.cache_prompts(system: true)

Caching user prompts:

chat = RubyLLM.chat
chat.with_instructions("You are a helpful assistant.")
chat.cache_prompts(system: true, user: true)
chat.ask("What is the capital of France?")

Caching tool definitions:

chat = RubyLLM.chat
chat.with_instructions("You are a helpful assistant.")
chat.with_tool(MyTool)
chat.cache_prompts(tools: true)
chat.ask("What is the capital of France?")

Type of change

• New feature

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• New public methods/classes

Related issues

Resolves #13

[tpaulshippy]

@crmne As I don't have an Anthropic key, I'll need you to generate the VCR cartridges for that provider. Hoping everything just works, but let me know if not.

[crmne]

@tpaulshippy this would be great to have! Will you be willing to enable it on all providers?

I'll do a proper review when I can.

[tpaulshippy]

My five minutes of research indicates that at least OpenAI and Gemini take the approach of automatically caching for you based on the size and structure of your request. So the only support I think we'd really need for those two is to populate the cached token counts on the response messages. Unless we want to try to support explicit caching on the Gemini API but that looks complex and not as commonly needed.

Do you know of other providers that require payload changes for prompt caching?

[tpaulshippy]

Realizing this might cause errors on older models that do not support caching. If it does, we could raise here, or just let the API validation handle it. I'm torn on whether the capabilities check complexity is worth it as these models are probably so rarely used.

[tpaulshippy]

@crmne As I don't have an Anthropic key, I'll need you to generate the VCR cartridges for that provider. Hoping everything just works, but let me know if not.

Scratch that. I decided to stop being a cheapskate and just pay Anthropic their $5.