Add Databricks Native Anthropic Messages API support (#15960) #15961
Conversation
- Implement DatabricksAnthropicMessagesConfig with BaseAnthropicMessagesConfig - Support native Anthropic Messages API protocol (no transformation needed) - Use Bearer token authentication (vs x-api-key) for Databricks endpoints - Add databricks_anthropic/ provider prefix recognition - Support all Anthropic features including Prompt Caching, tool calling, etc. - Add 5 comprehensive unit tests (all passing) - Full pass-through of Anthropic Messages API requests/responses Fixes BerriAI#15960
|
Someone is attempting to deploy a commit to the CLERKIEAI Team on Vercel. A member of the Team first needs to authorize it. |
|
Ubuntu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
| """Get complete URL - Databricks native endpoint doesn't need /v1/messages suffix.""" | ||
| return api_base or "" | ||
|
|
||
| def validate_anthropic_messages_environment( |
There was a problem hiding this comment.
i believe our dbrx chat completion validation leverages env vars, which would be missing in this implementation
| model, custom_llm_provider | ||
| ) | ||
|
|
||
| if model.startswith("databricks_anthropic/"): |
There was a problem hiding this comment.
can we avoid introducing a new provider name?
from what i understand, you're just trying to use the native route on v1/messages.
since this is possible for vertex + bedrock without new provider names, the same should be possible for databricks
There was a problem hiding this comment.
thanks, I will double with vertex + bedrock. and in fact, we checked the Anthropic provider firstly, and found it ony support api toke authtication (ANTHROPIC_API_KEY), no support for auth token way(ANTHROPIC_AUTH_TOKEN) and databricks's Anthropic endpoint only support Auth Token, that's why we decied to add such support to databricks side.
There was a problem hiding this comment.
I double check it, and it seems, to introduce a new provider is still a better way.
Because the 'Authropic' provider currently doesn't support ANTHROPIC_AUTH_TOKEN, if can update it in 'Authropic' level, it will affect more, and takes more test.
for vertex + bedrock, they have different URL/fields with databricks, so it is better to add new.
and I also checked the possible to add if/else against databricks.
| Dimension | New Provider (databricks_anthropic) |
In-Class Branch (Rejected) |
|---|---|---|
| Separation of protocols | Strong | Mixed (OpenAI + Anthropic in one class) |
| Cognitive load | Lower | Higher (conditional logic sprawl) |
| Testing | Orthogonal test matrix | Cross-cutting scenarios |
| Risk of regression | Lower | Higher |
| Future divergence (new messages features) | Easy | Hard |
| Monitoring / rate-limit segmentation | Native via provider key | Needs secondary flags |
| Header handling clarity | Dedicated | Conditional removal/insertion |
Conclusion: A distinct provider maximizes maintainability and correctness.
3 participants
Fixes #15960
Title
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes