Model Settings: Add Tool Choice ["auto", "required", "none"]

[Kurry]

Is your feature request related to a problem? Please describe.
When creating an LlmAgent with custom tools in the Google ADK, the agent often hallucinates plausible JSON responses rather than actually invoking the tool code. For example, when I query for real data (e.g., "Get company profile for MSFT"), it sometimes fabricates empty or incorrect JSON instead of executing my FunctionTool-wrapped API calls. Despite using skip_summarization=True and explicit instructions to return only the tool output, actual tool code execution is not always enforced.

Not having strict and configurable tool invocation makes the Google ADK unusable for a large population of developers, especially in production or evaluation settings. For example, I want to run the Agent locally with Ollama. If I want to switch models or experiment with different providers, I would need to run 100+ evaluations per agent version just to double-check that tool calls are being made correctly and consistently—this is expensive, inefficient and error-prone.

Describe the solution you'd like
I would like the ADK to support a model-level configuration system (similar to the OpenAI Agents SDK ModelSettings tool_choice option) where I can explicitly require tool execution for user queries. Specifically:

• Add a setting like tool_choice with options such as "auto", "required", or "none".
  • "required" would guarantee that the agent always calls a tool and never returns a model-generated (potentially hallucinated) response when tools are available.
  • If tool execution fails, the agent should error gracefully rather than hallucinate output.
• Allow this and other model settings (temperature, max_tokens, tool call parallelism, etc.) to be passed simply at agent or query time.
• These options should be part of a first-class ModelSettings dataclass or similar configuration, as in the OpenAI Agents SDK.

Describe alternatives you've considered

• Adding stricter prompts/instructions to the agent to enforce tool usage (has not reliably worked)
• Custom middleware or output verification logic to check if output was from a tool or LLM (adds complexity and feels like a workaround)
• Calling tools directly and avoiding the agent pattern, which loses multi-tool orchestration benefits

Additional context
The OpenAI Agents SDK ModelSettings provides a pythonic and explicit way to control this behavior. Here is a summary:

• With tool_choice="required", I can ensure queries always trigger tool calls and results are not fabricated by the model.
• Parallel tool calls and other advanced options are easily configured.
• See my attached code: in ADK, despite best efforts (after_tool_callback, skip_summarization, strict instructions), the agent continues to hallucinate responses.

Not having robust, configurable tool invocation makes reliable agent evaluation and deployment across multiple models or providers (like running locally with Ollama) impractical. Developers are forced into excessive, manual evaluation just to verify core agent functionality.

Having a native ModelSettings-like configuration with explicit tool_choice enforcement in ADK would resolve these UX and reliability issues and make tool use far more robust for API integration and retrieval-augmented agent scenarios.

[iamulya]

The following is generated by an ADK bot I am testing to provide guidance for the features, bugs submitted by the users

Thank you for this excellent and detailed feature request! You've clearly identified a critical pain point in developing robust and reliable LLM agents, particularly when it comes to ensuring consistent tool invocation over hallucinated responses. Your suggestion to introduce a ModelSettings class, inspired by the OpenAI Agents SDK's tool_choice mechanism, is excellent and aligns perfectly with the ADK's goals of providing flexibility and control.

This is a high-value feature that would significantly improve the developer experience and the reliability of agents built with ADK.

Here's a breakdown of the proposed enhancement based on your input and how it could be integrated into the ADK:

1. ModelSettings Dataclass

A new dataclass (or Pydantic BaseModel) should be introduced, for example, in src/google/adk/models/model_settings.py:

# src/google/adk/models/model_settings.py
from typing import Optional, Union, List, Literal, Dict
from pydantic import BaseModel, Field
from google.genai.types import FunctionCallingConfig # For Gemini specific mode

class SpecificToolChoice(BaseModel):
    type: Literal["function"] = "function"
    function: Dict[str, str] = Field(..., example={"name": "my_tool_name"})

class ModelSettings(BaseModel):
    """
    Configuration settings for the language model's generation behavior.
    """
    temperature: Optional[float] = Field(default=None, ge=0.0, le=2.0, description="Controls randomness. Lower is more deterministic.")
    max_output_tokens: Optional[int] = Field(default=None, gt=0, description="Maximum number of tokens to generate.")
    top_p: Optional[float] = Field(default=None, ge=0.0, le=1.0, description="Nucleus sampling parameter.")
    top_k: Optional[int] = Field(default=None, gt=0, description="Top-k sampling parameter.")
    stop_sequences: Optional[List[str]] = Field(default=None, description="Sequences where the model will stop generating.")

    tool_choice: Optional[Union[
        Literal["auto", "required", "none"],
        SpecificToolChoice,
        # List[SpecificToolChoice] # Could be considered for restricting to a subset
    ]] = Field(default="auto", description="Controls how the model selects tools.")

    # Consider adding a field for parallel tool calls if not implicitly handled
    # by the LLM provider when tool_choice is "auto" or "required".
    # For Gemini, this is part of ToolConfig.
    # allow_parallel_tool_calls: bool = True

    # Advanced: Directly pass provider-specific configurations if needed,
    # though the goal is to abstract this away as much as possible.
    # provider_specific_config: Optional[Dict[str, Any]] = None

    class Config:
        extra = 'forbid'

2. Integration into LlmAgent and RunConfig

• LlmAgent:
  Modify src/google/adk/agents/llm_agent.py to include model_settings as an optional initialization parameter.

  from ..models.model_settings import ModelSettings # New import
  
  class LlmAgent(BaseAgent):
      # ... existing fields ...
      model_settings: Optional[ModelSettings] = None
      # ...

• RunConfig:
  Modify src/google/adk/agents/run_config.py to allow overriding or specifying ModelSettings at runtime.

  from ..models.model_settings import ModelSettings # New import
  
  class RunConfig(BaseModel):
      # ... existing fields ...
      model_settings: Optional[ModelSettings] = None
      # ...

  The Runner (in runners.py) would then need to be updated to resolve these settings. Runtime settings in RunConfig should take precedence over agent-defined settings.

3. Implementing tool_choice in LLM Connectors

This is the most crucial part. The abstract ModelSettings need to be translated into concrete API parameters for each supported LLM.

• GoogleLlm (Gemini) in src/google/adk/models/google_llm.py:
  • The generate_content_async method (and potentially connect for live mode) needs to map ModelSettings.tool_choice to google.genai.types.FunctionCallingConfig.Mode.
  • tool_choice="auto" -> mode=FunctionCallingConfig.Mode.AUTO
  • tool_choice="none" -> mode=FunctionCallingConfig.Mode.NONE
  • tool_choice="required" -> mode=FunctionCallingConfig.Mode.ANY.
    • ADK Framework Logic for "required": If Mode.ANY is used and the Gemini response does not include a function call (but tool_choice="required" was specified by the user and tools were available in the LlmRequest), the BaseLlmFlow or a dedicated response processor must detect this.
    • Instead of returning the LLM's text response, it should raise a specific error, e.g., ToolChoiceRequiredError("Model did not call a required tool, despite tools being available and tool_choice='required'.").
  • tool_choice=SpecificToolChoice(function={"name": "my_tool"}) -> mode=FunctionCallingConfig.Mode.ANY and set allowed_function_names=["my_tool"] in the FunctionCallingConfig.
  • Other ModelSettings like temperature, max_output_tokens, etc., would populate the GenerateContentConfig.
• LiteLlm in src/google/adk/models/lite_llm.py:
  • litellm.completion and litellm.acompletion directly accept a tool_choice parameter that mirrors OpenAI's options (including "auto", "required", "none", or {"type": "function", "function": {"name": "my_tool"}}).
  • The ADK's LiteLlm wrapper would pass the resolved tool_choice from ModelSettings directly to LiteLLM.
  • ADK Framework Logic for "required": Similar to Gemini, if tool_choice="required" is used and the underlying model (via LiteLLM) returns a text response instead of a tool call, the ADK framework must detect this and raise ToolChoiceRequiredError.
• AnthropicLlm in src/google/adk/models/anthropic_llm.py:
  • Anthropic's API (as of Claude 3.5) supports a tool_choice parameter with options like {"type": "auto"}, {"type": "any"} (for forcing one tool call), and {"type": "tool", "name": "specific_tool_name"}.
  • tool_choice="auto" -> Anthropic {"type": "auto"}
  • tool_choice="none" -> This might require omitting the tools parameter entirely or if there's a specific "none" equivalent.
  • tool_choice="required" -> Anthropic {"type": "any"}. The same ADK framework logic for erroring if no tool call is made would apply.
  • tool_choice=SpecificToolChoice(function={"name": "my_tool"}) -> Anthropic {"type": "tool", "name": "my_tool"}.

4. Updating BaseLlmFlow

• Located in src/google/adk/flows/llm_flows/base_llm_flow.py.
• The _call_llm_async method, or more likely, a pre-processing step that populates the LlmRequest.config (like in src/google/adk/flows/llm_flows/basic.py), would need to:
  1. Resolve the effective ModelSettings (from RunConfig or agent's default).
  2. Populate the LLM-specific configuration (e.g., GenerateContentConfig for Gemini) based on these ModelSettings.
• A post-processing step in _postprocess_async (or a new dedicated response processor) would be responsible for checking the tool_choice="required" condition and raising ToolChoiceRequiredError if the model fails to make a tool call.

Benefits Addressed:

• Strict Tool Invocation: tool_choice="required" would provide the desired guarantee.
• Graceful Erroring: The proposed ToolChoiceRequiredError handles cases where the model doesn't comply with "required". If a tool is called but fails, existing error propagation should handle that.
• Simplified Model Settings: The ModelSettings class provides a single, clear place for various model parameters.
• Model Agnosticism: The ModelSettings class is an abstraction, and each LLM connector (GoogleLlm, LiteLlm, etc.) implements the translation to its specific API.

Example Usage:

from google.adk.agents import Agent, RunConfig
from google.adk.models.model_settings import ModelSettings, SpecificToolChoice

# Agent-level default
strong_tool_agent = Agent(
    name="strong_tool_user",
    model="gemini-1.5-flash",
    tools=[my_tool_function],
    model_settings=ModelSettings(tool_choice="required")
)

# Query-time override
runner = Runner(agent=strong_tool_agent, ...)
events_required = await runner.run_async(
    ...,
    run_config=RunConfig(
        model_settings=ModelSettings(tool_choice="required")
    )
)

events_specific_tool = await runner.run_async(
    ...,
    run_config=RunConfig(
        model_settings=ModelSettings(
            tool_choice=SpecificToolChoice(function={"name": "my_tool_function"})
        )
    )
)

events_no_tools = await runner.run_async(
    ...,
    run_config=RunConfig(
        model_settings=ModelSettings(tool_choice="none")
    )
)

This enhancement would indeed make tool usage in ADK far more robust, predictable, and easier to evaluate, especially across different LLM providers. It's a solid proposal that aligns well with the needs of developers building production-grade agents.

[robert-at-pretension-io]

Wow... Which adk is that? ^

[robert-at-pretension-io]

@iamulya , Hi! Could I see the code for that agent? I'm working on something similar!

[seanzhou1023]

@iamulya thank you so much for the details. It really helps and makes sense to improve quality of agent. I'll put it as highest priority to implement. Please stay tuned.