Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.NET: Bug: Anthropic connector doesn't work for VertexAI #10012

Open
Vifill opened this issue Dec 18, 2024 · 1 comment
Open

.NET: Bug: Anthropic connector doesn't work for VertexAI #10012

Vifill opened this issue Dec 18, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@Vifill
Copy link

Vifill commented Dec 18, 2024

Describe the bug
The Anthropic VertexAI needs the anthropic-version to be passed in the body (as anthropic_version) instead of the header, as is done with the Anthropic API.

(See documentation here: https://docs.anthropic.com/en/api/claude-on-vertex-ai)
In Vertex, anthropic_version is passed in the request body (rather than as a header), and must be set to the value vertex-2023-10-16.

This is not done in the connector logic, it reuses the AnthropicRequest model which doesn't have the anthropic_version in the body.

When trying to use the AddAnthropicVertextAIChatCompletion out of the box it returns

{
  "type" : "error",
  "error" : {
    "type" : "invalid_request_error",
    "message" : "anthropic_version: Field required"
  }
}

Sadly this seems to be a hard blocker since I see no good way to add this into the body, but I would love if anyone had a workaround.

UPDATE: After trying a hacky workaround by intercepting the request and editing the json before sending it I noticed a new error. The 'model' property is not allowed, so that also needs to be removed from the body.

Additional context
Not sure if it's a bug or intented, but unlike the GeminiVertexAi connector, the AddAnthropicVertextAIChatCompletion (it also has a typo) isn't creating the full endpoint for you. So you need to create the full URL like so:

var endpoint =  new Uri($"https://{location}-aiplatform.googleapis.com/v1/projects/{projectId}/locations/{location}/publishers/anthropic/models/{_modelId}:rawPredict");
builder.AddAnthropicVertextAIChatCompletion(_modelId, () => new ValueTask<string>(bearerKey), endpoint);

Which I think could be improved to be more intuitive, and be similar to the Gemini VertexAI connector.

@Vifill Vifill added the bug Something isn't working label Dec 18, 2024
@Vifill Vifill changed the title Bug: Anthropic connector doesn't work for VertexAI .NET: Anthropic connector doesn't work for VertexAI Dec 19, 2024
@Vifill Vifill changed the title .NET: Anthropic connector doesn't work for VertexAI .NET: Bug: Anthropic connector doesn't work for VertexAI Dec 19, 2024
@Vifill
Copy link
Author

Vifill commented Dec 19, 2024

After trying a hacky workaround by intercepting the request and editing the json before sending it I noticed a new error. The 'model' property is not allowed, so that also needs to be removed from the body.

https://docs.anthropic.com/en/api/claude-on-vertex-ai
In Vertex, model is not passed in the request body. Instead, it is specified in the Google Cloud endpoint URL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Bug
Development

No branches or pull requests

3 participants