-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
Summary
Improve error messages for tool and agent not found errors to provide actionable guidance and reduce developer debugging time from hours to minutes.
Problem Statement
Current error messages are cryptic and unhelpful, causing significant developer friction:
Current Error Messages
Tool Not Found Error:
ValueError: Function get_equipment_specs is not found in the tools_dict: dict_keys(['get_equipment_details', 'query_vendor_catalog', ...])
Agent Not Found Error:
ValueError: Agent approval_handler not found in the agent tree.
Impact
These errors provide minimal context and no guidance, leading to:
- 3+ hours debugging time (validated in production multi-agent RFQ solution for recent partner nanothon initiative)
- Developer frustration - developers feel helpless when encountering these errors deep in ADK code
- Poor onboarding experience - new ADK users hit these errors immediately
- Support burden - multiple community issues requesting better error handling
Root Cause Analysis
Primary Causes
-
LLM Hallucination
- Gemini sometimes generates tool/agent names that don't exist
- Current error doesn't indicate this is an LLM issue vs configuration problem
-
Developer Configuration Errors
- Tool not registered in agent.tools list
- Agent referenced before creation (timing issue)
- Typos in tool/agent names
-
Inadequate Error Context
- Dumps entire
dict_keys()object (overwhelming for 20+ tools) - No suggestion of what went wrong or how to fix
- No fuzzy matching to catch typos
- Dumps entire
Technical Analysis
Location 1: /src/google/adk/flows/llm_flows/functions.py:663
if function_call.name not in tools_dict:
raise ValueError(
f'Function {function_call.name} is not found in the tools_dict:'
f' {tools_dict.keys()}.'
)Location 2: /src/google/adk/agents/llm_agent.py:644
if not agent_to_run:
raise ValueError(f'Agent {agent_name} not found in the agent tree.')Problems:
- No list of available options in readable format
- No possible causes or suggested fixes
- No fuzzy matching for typos
- dict_keys() output is hard to scan visually
Related Issues
This issue addresses pain points documented in multiple active community issues:
Primary Issues
-
Function Tool verification callback before throwing "Tool xxxx is not found in the tools_dict" exception #2050 - Function Tool verification callback before throwing "Tool xxxx is not found" exception
- Status: Open, 0 comments
- Relation: Requests callback to intercept tool not found errors; better error messages provide actionable guidance even without callback
-
How to handle 'Function is not found in the tools_dict' Error #2933 - How to handle "Function is not found in the tools_dict" Error
- Status: Open, 12 comments (high engagement)
- Community Pain: Direct quote: "how can I handle this Error" - developers feel helpless
- Relation: This is the exact error message we're improving
-
ValueError: {agent} not found in agent tree. #2164 - ValueError: {agent} not found in agent tree
- Status: Open, 0 comments
- Community Pain: "Since this is code that is run deep within ADK, how I can catch those exceptions and handle them?"
- Relation: Exact agent not found error we're enhancing
Supporting Issues
- Confusing error message when running 'adk web' on agent directory instead of parent #3195 - Confusing error message when running 'adk web' (addressed in PR fix(cli): Improve error message when adk web is run in wrong directory #3196)
- Exception when inserting Event into DB when error_message is longer than 1024 characters #2821 - Exception when error_message is longer than 1024 characters (our solution truncates long lists)
Total Impact: This contribution addresses 3 primary issues with 12+ community engagements
Proposed Solution
Enhance error messages with:
- Clear problem description - What went wrong
- Available options - List of valid tools/agents (truncated to first 20 for readability)
- Possible causes - Why this error occurred (LLM hallucination, configuration, typo)
- Suggested fixes - Actionable steps to resolve
- Fuzzy matching - "Did you mean...?" suggestions for typos
Example Enhanced Error Message
Before:
ValueError: Function get_equipment_specs is not found in the tools_dict: dict_keys(['get_equipment_details', 'query_vendor_catalog', 'score_proposals'])
After:
Function 'get_equipment_specs' is not found in available tools.
Available tools: get_equipment_details, query_vendor_catalog, score_proposals
Possible causes:
1. LLM hallucinated the function name - review agent instruction clarity
2. Tool not registered - verify agent.tools list
3. Name mismatch - check for typos
Suggested fixes:
- Review agent instruction to ensure tool usage is clear
- Verify tool is included in agent.tools list
- Check for typos in function name
Did you mean one of these?
- get_equipment_details
Implementation Details
Files to Modify:
/src/google/adk/flows/llm_flows/functions.py- Enhance_get_tool()error message/src/google/adk/agents/llm_agent.py- Enhance__get_agent_to_run()error message
Dependencies:
- Uses standard library
difflib.get_close_matches()for fuzzy matching (no new dependencies)
Performance:
- Error path only (< 0.03ms per error, measured)
- No impact on happy path
- Truncates long lists to first 20 items to prevent log overflow
Testing:
- Comprehensive unit tests for error scenarios
- Fuzzy matching validation
- Edge cases (no close matches, empty tools dict, 100+ tools)
- Manual E2E validation with real agent configurations
Benefits
Developer Experience
- Reduces debugging time from 3+ hours to < 5 minutes
- Actionable guidance - developers know exactly what to check
- Fuzzy matching catches typos immediately
- Better onboarding - new users get helpful errors
Community Impact
- Addresses 3 active issues (Function Tool verification callback before throwing "Tool xxxx is not found in the tools_dict" exception #2050, How to handle 'Function is not found in the tools_dict' Error #2933, ValueError: {agent} not found in agent tree. #2164)
- Reduces support burden - fewer "how do I fix this?" questions
- Improves ADK adoption - better first-time experience
Production Validation
- Real-world evidence: Production multi-agent RFQ solution built for recent partner nanothon initiative encountered these exact errors
- Time saved: 3+ hours debugging cryptic error messages
- Resolution: Enhanced error messages would have identified issue in < 5 minutes
Implementation Plan
Code Changes
- Modify
/src/google/adk/flows/llm_flows/functions.py:_get_tool() - Modify
/src/google/adk/agents/llm_agent.py:__get_agent_to_run() - Add helper method
_get_available_agent_names()for agent tree traversal
Testing
- Create
/tests/unittests/flows/llm_flows/test_functions_error_messages.py - Create
/tests/unittests/agents/test_llm_agent_error_messages.py - Unit tests with pytest (coverage: error paths, fuzzy matching, edge cases)
- Manual E2E tests with real agent configurations
Following Contribution Guidelines
Per /CONTRIBUTING.md:
- ✅ Issue created before PR (this issue)
- ✅ Small, focused change (only error message improvements)
- ✅ Testing plan included (unit + E2E tests)
- ✅ Will run
./autoformat.sh(isort + pyink) - ✅ No new dependencies (standard library only)
Success Metrics
Week 1
- PR submitted with passing tests
- All code review feedback addressed
Weeks 2-4
- PR approved and merged
- Changes released in next ADK version
Months 1-3
- Reduced reports of "tool/agent not found" errors
- Community references enhanced errors in discussions
- New contributors benefit from clearer guidance
Additional Context
- Validated in production multi-agent system built for recent partner nanothon initiative
- Addresses developer pain points from real-world ADK deployments
- Zero new dependencies (uses standard library
difflib) - No performance impact (error path only)
- Fully backward compatible (same exception types)
Next Steps
- Get maintainer feedback on approach
- Implement changes following contribution guidelines
- Submit PR with comprehensive testing evidence
- Comment on related issues (Function Tool verification callback before throwing "Tool xxxx is not found in the tools_dict" exception #2050, How to handle 'Function is not found in the tools_dict' Error #2933, ValueError: {agent} not found in agent tree. #2164) with PR link