Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Error reporting message strips away useful error type #835

Open
rht opened this issue Feb 28, 2025 · 1 comment
Open

[BUG] Error reporting message strips away useful error type #835

rht opened this issue Feb 28, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@rht
Copy link

rht commented Feb 28, 2025

Describe the bug
I tried to intentionally tell CodeAgent to output code with syntax error in its tool calling code. It did show the error message, but it omitted the informative error/exception type, i.e. SyntaxError. This is just an illustration, as it could as well be KeyError, NotImplementedError, ImportError, etc.

Code to reproduce the error

from smolagents.agents import CodeAgent
from smolagents import tool, HfApiModel

model = HfApiModel("Qwen/Qwen2.5-72B-Instruct")


@tool
def give_yes(query: str) -> str:
    """
    Give a yes string as an answer
    Args:
        query: any question you have
    """

    return f"yes, {query}"


agent = CodeAgent(tools=[give_yes], additional_authorized_imports=["os"], model=model)
query = "can you call give_yes, giving the content of os.environ['PWD'] as an input to it, but intentionally made a syntax error, and tell me what the tool says?"
print(agent.run(query))

Error logs (if any)

╭───────────────────────────────────────────────────────── New run ─────────────────────────────────────────────────────────╮
│                                                                                                                           │
│ can you call give_yes, giving the content of os.environ['PWD'] as an input to it, but intentionally made a syntax error,  │
│ and tell me what the tool says?                                                                                           │
│                                                                                                                           │
╰─ HfApiModel - Qwen/Qwen2.5-72B-Instruct ──────────────────────────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 ─ Executing parsed code: ──────────────────────────────────────────────────────────────────────────────────────────────────
  give_yes(query os.environ['PWD'])
 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Code parsing failed on line 1 due to: SyntaxError
give_yes(query os.environ['PWD'])
          ^
Error: invalid syntax. Perhaps you forgot a comma? (<unknown>, line 1)
[Step 0: Duration 2.57 seconds| Input tokens: 2,091 | Output tokens: 50]

Expected behavior
SyntaxError: invalid syntax. Perhaps you forgot a comma? (<unknown>, line 1)

Packages version:

Name: smolagents
Version: 1.9.2
@rht rht added the bug Something isn't working label Feb 28, 2025
@rht
Copy link
Author

rht commented Feb 28, 2025

Ah, I didn't see the message Code parsing failed on line 1 due to: SyntaxError. Maybe it should be bundled together with the message like in the usual Python console output?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant