Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama-python: enable passing messages with "control" role to enable granite3.2 thinking control functionality #462

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rylativity
Copy link

Ollama's python client currently only permits messages with roles of either 'user', 'assistant', 'system', or 'tool'.

As described here - ollama/ollama#8955 (comment) - this prevents sending messages with the "control" role, which was introduced by IBM's granite3.2 to enable turning the "thinking" functionality on and off as needed.

This pull request adds "control" to the list of valid values for a message's "role" field/attribute so that these messages can be passed through to ollama.

Note: ollama server already handles messages with this role appropriately, but the python client currently does not allow sending them to ollama server. This resolves the issue in the python client.

… to enable granite3.2 thinking control functionality
@lemassykoi
Copy link

thanks, but I did your modification on my computer, and enabled full traceback.
The problem is from langchain_core/messages/utils.py at func _create_message_from_message_type

image

@rylativity
Copy link
Author

rylativity commented Mar 6, 2025

This PR addresses the original issue you reported within the ollama-python client. I was able to reproduce your original issue with the original code that you shared, and it does not use langchain.

The new traceback you shared above is different and is a separate error coming from langchain-core, which is a different project/codebase.

Will you please share the code or command that you ran that resulted in the langchain error above? I'm happy to take a look and try to open a PR for langchain-core as well.

Copy link

@gabe-l-hart gabe-l-hart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One quick addition if we're going to go the route of expanding the set of literals, we should consider including "document" as well.

@lemassykoi
Copy link

This PR addresses the original issue you reported within the ollama-python client. I was able to reproduce your original issue with the original code that you shared, and it does not use langchain.

The new traceback you shared above is different and is a separate error coming from langchain-core, which is a different project/codebase.

Will you please share the code or command that you ran that resulted in the langchain error above? I'm happy to take a look and try to open a PR for langchain-core as well.

Thanks, this is my orighinal post with original code :

langchain-ai/langchain#30122 (comment)

@rylativity rylativity changed the title enable passing messages with "control" role to enable granite3.2 thinking control functionality ollama-python: enable passing messages with "control" role to enable granite3.2 thinking control functionality Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants