Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do you skip errors such as StructuredOutputError? #210

Open
Lawouach opened this issue May 14, 2024 · 1 comment
Open

How do you skip errors such as StructuredOutputError? #210

Lawouach opened this issue May 14, 2024 · 1 comment

Comments

@Lawouach
Copy link

Hi,

It seems to me that when an error such as StructuredOutputError is raised, the whole process terminates. Is there a way to make it more forgiving? I'd personally wouldn't mind if a few retrieval failed during streaming if I could keep these that succeeded.

Am I looking at this the wrong way?

@jackmpcollins
Copy link
Owner

Some tips on dealing with StructuredOutputError from https://magentic.dev/structured-outputs/#json-schema

If a StructuredOutputError is raised often, this indicates that the LLM is failing to match the schema. The traceback for these errors includes the underlying pydantic ValidationError which shows in what way the received response was invalid. To combat these errors there are several options

  • Add descriptions or examples for individual fields to demonstrate valid values.
  • Simplify the output schema, including using more flexible types (e.g. str instead of datetime) or allowing fields to be nullable with | None.
  • Switch to a "more intelligent" LLM. See Configuration for how to do this.

You could also retry the request using https://github.com/jd/tenacity or https://github.com/hynek/stamina . This is less effective for @prompt_chain if it is failing at some intermediate step.

There's an open issue for having the LLM adjust its output based on the error #166

If none of the above helps, could you please share some example code to recreate the issue. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants