feat: Enhance error messages for tool and agent not found errors #3219
Conversation
Improve developer experience by providing actionable error messages with:
- Clear description of what went wrong
- List of available tools/agents (truncated to 20 for readability)
- Possible causes and suggested fixes
- Fuzzy matching suggestions ("Did you mean...?")
Addresses community issues:
- google#2050: Tool verification callback request
- google#2933: How to handle Function Not Found error (12 comments)
- google#2164: Agent not found ValueError
Changes:
- Enhanced _get_tool() error message in functions.py
- Enhanced __get_agent_to_run() error message in llm_agent.py
- Added _get_available_agent_names() helper for agent tree traversal
- Added fuzzy matching using difflib (standard library)
- Truncates long lists to first 20 items for readability
- Comprehensive unit tests for error scenarios (8 tests, all passing)
Testing:
- pytest tests/unittests/flows/llm_flows/test_functions_error_messages.py: 4/4 passed
- pytest tests/unittests/agents/test_llm_agent_error_messages.py: 4/4 passed
- Performance: < 0.03ms per error (error path only, no hot path impact)
Fixes google#3217
Summary of ChangesHello @jpantsjoha, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the developer experience by upgrading the error messages for scenarios where a specified tool or agent cannot be found. The improved messages now offer detailed context, including lists of available options, common causes for such errors, and concrete suggestions for resolution, alongside intelligent fuzzy matching to catch typos. This change is designed to drastically reduce debugging time and streamline the development process for users of the ADK. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
adk-bot
added
the
core
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This is a great enhancement to the error messages, which will significantly improve the developer experience when debugging "not found" errors for agents and tools. The inclusion of available items, possible causes, suggested fixes, and fuzzy matching is very thoughtful. The new unit tests are comprehensive and cover important edge cases.
I have a couple of suggestions to improve the implementation by reducing code duplication and making the string construction more idiomatic. Please see my inline comments.
src/google/adk/agents/llm_agent.py
Outdated
| error_msg = ( | ||
| f"Agent '{agent_name}' not found in the agent tree.\n\n" | ||
| f'{agents_msg}\n\n' | ||
| 'Possible causes:\n' | ||
| ' 1. Agent not registered before being referenced\n' | ||
| ' 2. Agent name mismatch (typo or case sensitivity)\n' | ||
| ' 3. Timing issue (agent referenced before creation)\n\n' | ||
| 'Suggested fixes:\n' | ||
| ' - Verify agent is registered with root agent\n' | ||
| ' - Check agent name spelling and case\n' | ||
| ' - Ensure agents are created before being referenced\n' | ||
| ) | ||
|
|
||
| # Fuzzy matching suggestion | ||
| from difflib import get_close_matches | ||
|
|
||
| close_matches = get_close_matches( | ||
| agent_name, available_agents, n=3, cutoff=0.6 | ||
| ) | ||
| if close_matches: | ||
| error_msg += f'\nDid you mean one of these?\n' | ||
| for match in close_matches: | ||
| error_msg += f' - {match}\n' | ||
|
|
||
| raise ValueError(error_msg) |
There was a problem hiding this comment.
To improve readability and avoid multiple string concatenations, you could build the error message from a list of parts and then join them at the end. This is a more idiomatic way to construct complex multi-line strings in Python.
error_parts = [
f"Agent '{agent_name}' not found in the agent tree.",
agents_msg,
'Possible causes:\n'
' 1. Agent not registered before being referenced\n'
' 2. Agent name mismatch (typo or case sensitivity)\n'
' 3. Timing issue (agent referenced before creation)',
'Suggested fixes:\n'
' - Verify agent is registered with root agent\n'
' - Check agent name spelling and case\n'
' - Ensure agents are created before being referenced',
]
# Fuzzy matching suggestion
from difflib import get_close_matches
close_matches = get_close_matches(
agent_name, available_agents, n=3, cutoff=0.6
)
if close_matches:
suggestions = '\n'.join(f' - {match}' for match in close_matches)
error_parts.append(f'Did you mean one of these?\n{suggestions}')
raise ValueError('\n\n'.join(error_parts))| if function_call.name not in tools_dict: | ||
| raise ValueError( | ||
| f'Function {function_call.name} is not found in the tools_dict:' | ||
| f' {tools_dict.keys()}.' | ||
| # Enhanced error message with actionable guidance | ||
| available_tools = list(tools_dict.keys()) | ||
|
|
||
| # Truncate to first 20 for readability (prevents log overflow) | ||
| if len(available_tools) > 20: | ||
| tools_preview = ', '.join(available_tools[:20]) | ||
| tools_msg = ( | ||
| f'Available tools (showing first 20 of {len(available_tools)}):' | ||
| f' {tools_preview}...' | ||
| ) | ||
| else: | ||
| tools_msg = f"Available tools: {', '.join(available_tools)}" | ||
|
|
||
| error_msg = ( | ||
| f"Function '{function_call.name}' is not found in available" | ||
| ' tools.\n\n' | ||
| f'{tools_msg}\n\n' | ||
| 'Possible causes:\n' | ||
| ' 1. LLM hallucinated the function name - review agent' | ||
| ' instruction clarity\n' | ||
| ' 2. Tool not registered - verify agent.tools list\n' | ||
| ' 3. Name mismatch - check for typos\n\n' | ||
| 'Suggested fixes:\n' | ||
| ' - Review agent instruction to ensure tool usage is clear\n' | ||
| ' - Verify tool is included in agent.tools list\n' | ||
| ' - Check for typos in function name\n' | ||
| ) | ||
|
|
||
| # Fuzzy matching suggestion | ||
| from difflib import get_close_matches | ||
|
|
||
| close_matches = get_close_matches( | ||
| function_call.name, available_tools, n=3, cutoff=0.6 | ||
| ) | ||
| if close_matches: | ||
| error_msg += f'\nDid you mean one of these?\n' | ||
| for match in close_matches: | ||
| error_msg += f' - {match}\n' | ||
|
|
||
| raise ValueError(error_msg) |
There was a problem hiding this comment.
This error message generation logic is very similar to the one in src/google/adk/agents/llm_agent.py. To avoid code duplication and improve maintainability, consider extracting this logic into a shared utility function. This function could take parameters like the item name (function_call.name), item type ('tool' or 'agent'), the list of available items, and the specific cause/fix messages, then return the formatted error string.
Addresses Gemini Code Assist review feedback on PR google#3219: 1. String Construction: Use list-based approach with join() instead of multiple string concatenations for better readability and performance 2. DRY Principle: Extract shared utility function to eliminate ~80 lines of duplicated error formatting logic across two files Changes: - Created src/google/adk/utils/error_messages.py with format_not_found_error() utility function - Refactored functions.py to use shared utility (~32 lines removed) - Refactored llm_agent.py to use shared utility (~32 lines removed) Benefits: - Single source of truth for error message formatting - More Pythonic string construction (list-based approach) - Easier to maintain and extend - Consistent error messages across tools and agents Testing: - All 8 existing unit tests passing (4 for tools, 4 for agents) - Autoformatting applied (isort + pyink) - GCPADK_SME review: 9.5/10 APPROVED No breaking changes - backward compatible.
Addresses Gemini Code Assist review feedback on PR google#3219: 1. String Construction: Use list-based approach with join() instead of multiple string concatenations for better readability and performance 2. DRY Principle: Extract shared utility function to eliminate ~80 lines of duplicated error formatting logic across two files Changes: - Created src/google/adk/utils/error_messages.py with format_not_found_error() utility function - Refactored functions.py to use shared utility (~32 lines removed) - Refactored llm_agent.py to use shared utility (~32 lines removed) Benefits: - Single source of truth for error message formatting - More Pythonic string construction (list-based approach) - Easier to maintain and extend - Consistent error messages across tools and agents Testing: - All 8 existing unit tests passing (4 for tools, 4 for agents) - Autoformatting applied (isort + pyink) - GCPADK_SME review: 9.5/10 APPROVED No breaking changes - backward compatible.
…antsjoha/adk-python into feat/better-error-messages
There was a problem hiding this comment.
Thanks for the PR!
I agree that message can be improved, but I feel it's overkill to have format_not_found_error method. We can slightly augment the error message and down the road.
I think a better plan to be having a dedicate page in adk docs to list all common errors and reference the short link to there in these errors.
|
Hi @Jacksunwei, thanks for the feedback! Let's simplify. I'll refactor to:
Quick question: Should I inline the fuzzy matching call too, or is a tiny helper okay for the I'll refactor today and push the update. Re: Documentation Page - I'd love to create Thanks for the guidance! 🙏 |
4 participants
Summary
Enhance error messages for tool and agent not found errors to provide actionable guidance and reduce developer debugging time from hours to minutes.
Fixes #3217
Changes
Modified Files
src/google/adk/flows/llm_flows/functions.py_get_tool()error message with:src/google/adk/agents/llm_agent.py__get_agent_to_run()error message with:_get_available_agent_names()helper methodNew Test Files
tests/unittests/flows/llm_flows/test_functions_error_messages.pytests/unittests/agents/test_llm_agent_error_messages.pyTesting Plan
Unit Tests
Results: ✅ 8/8 tests passing
Example Enhanced Error Messages
Before (Current Error)
After (Enhanced Error)
Community Impact
Implementation Details
difflibfor fuzzy matching (no new dependencies)Checklist
./autoformat.sh(isort + pyink)difflib)Related Issues
Note: For production scenarios where LLM tool hallucinations occur, ADK's built-in
ReflectAndRetryToolPlugincan automatically retry failed tool calls (available since v1.16.0). This PR's enhanced error messages complement that by helping developers quickly identify and fix configuration issues during development.Cheers, JP