-
-
Notifications
You must be signed in to change notification settings - Fork 4.3k
fix(ollama/chat): correctly map reasoning_effort to think in requests #15186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…o kowyo/fix-ollama-think
@kowyo is attempting to deploy a commit to the CLERKIEAI Team on Vercel. A member of the Team first needs to authorize it. |
Hey @kowyo i believe that parameter value (low/medium/high) only works for gpt-oss. Can we be more careful on when to use low/medium/high vs. a 'True' based on model? |
Hey @krrishdholakia, you are correct, thanks for the feedback. I have made some adjustments and will only set thinking level for Let me know if further work is needed. |
@krrishdholakia Hello, this fix is important to me. Anything else I can do to push forward the progress? Thanks for the consideration. |
Title
fix(ollama/chat): correctly map reasoning_effort to think in requests
Relevant issues
Fixes #15059
Fixes #11680 (#11680 (comment))
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/
directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit
Type
🐛 Bug Fix
Changes
Parameter mapping improvements:
Updated
map_openai_params
in both chat/transformation.py and completion/transformation.py so thereasoning_effort
parameter maps directly to the Ollamathink
param (high/medium/low) instead of always defaulting to True. As in ollama/ollama-python@aa4b476 and ollama/ollama#11752.Parameter mapping correction:
pop the
think
parameter fromoptional_params
, so that it maps correctly to ollama API https://github.com/ollama/ollama/blob/main/docs/api.md, fixes the warninglevel=WARN source=types.go:737 msg="invalid option provided" option=think
Effect
Before this PR:
reasoning_content
is missing from output, indicating thatthink
param is not mapped correctly:#15059
With this PR:
reasoning_content
is return correctly.Testing Pass
Gemini and Cohere related test should not be affected in this PR