Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Reasoning Content is not included in response from bedrock converse for sonnet 3.7 #8777

Closed
talalryz opened this issue Feb 25, 2025 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@talalryz
Copy link

talalryz commented Feb 25, 2025

What happened?

Making the curl request:

curl ... --data '{
  "model": "sonnet-3.7",
  "max_tokens": 128000,
  "thinking": {
        "type": "enabled",
        "budget_tokens": 3200
    },
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "what is 1+2?"
    }
  ]
}'

the response is:

{"id":"chatcmpl-99a791ff-b69e-4cac-83ca-1f3726fd934b","created":1740441961,"model":"us.anthropic.claude-3-7-sonnet-20250219-v1:0","object":"chat.completion","system_fingerprint":null,"choices":[{"finish_reason":"stop","index":0,"message":{"content":"The sum of 1 and 2 is 3.","role":"assistant","tool_calls":null,"function_call":null}}],"usage":{"completion_tokens":51,"prompt_tokens":49,"total_tokens":100,"completion_tokens_details":null,"prompt_tokens_details":null}}

and does have a thinking or reasoning block e.g., I would expected to have had something like this in the response:

{
            "type": "thinking",
            "thinking": " ..",
},

My litellm config looks like:

model_list:
  - model_name: sonnet-3.7
    litellm_params:
      model: bedrock/converse/us.anthropic.claude-3-7-sonnet-20250219-v1:0

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.61.15-nightly

Twitter / LinkedIn details

No response

@talalryz talalryz added the bug Something isn't working label Feb 25, 2025
@krrishdholakia
Copy link
Contributor

Hey @talalryz adding this for the Anthropic API now - #8778

Will work on bedrock support next

@krrishdholakia krrishdholakia self-assigned this Feb 25, 2025
@talalryz
Copy link
Author

Thank you!! I'll close the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants