Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AGE-272] [Bug] Improve errors in evaluation view #1300

Closed
mmabrouk opened this issue Jan 29, 2024 · 3 comments
Closed

[AGE-272] [Bug] Improve errors in evaluation view #1300

mmabrouk opened this issue Jan 29, 2024 · 3 comments
Assignees
Labels
Backend bug Something isn't working evaluation Frontend High Priority linear Created by Linear-GitHub Sync
Milestone

Comments

@mmabrouk
Copy link
Member

mmabrouk commented Jan 29, 2024

When errors happen in the evaluation view, there are a couple of problems:

  • The error in the llm app are shown in black and not in red as expected
    • The issue is in the LLM app response and not in the UI
  • The error in the llm app is very general (500 internal error) and does not provide details about the message in the error as we do have in the playground (❌ Model gpt-3.5-turbo does not support JSON response format)
    • Unclear whether this is still relevant
  • The error for the evaluation is the same as in the llm app. Instead we should say "LLM app failed" or something similar.

image

From SyncLinear.com | AGE-272

@mmabrouk mmabrouk added the linear Created by Linear-GitHub Sync label May 28, 2024
@mmabrouk mmabrouk changed the title [Bug] Improve errors in evaluation view [AGE-272] [Bug] Improve errors in evaluation view May 28, 2024
@mmabrouk mmabrouk added this to the Evaluation view v2 milestone May 28, 2024
@mmabrouk mmabrouk modified the milestones: Evaluation view v2, v.52 Jul 1, 2024
@mmabrouk mmabrouk added the bug Something isn't working label Jul 2, 2024
@mmabrouk mmabrouk changed the title [AGE-272] [Bug] Improve errors in evaluation view [AGE-272] [Bug] Errors are not shown correctly in the evaluation Jul 2, 2024
@mmabrouk mmabrouk changed the title [AGE-272] [Bug] Errors are not shown correctly in the evaluation [AGE-272] [Bug] Improve errors in evaluation view Jul 2, 2024
@mmabrouk
Copy link
Member Author

mmabrouk commented Jul 4, 2024

@jp this issue should actually be fixed in the new version of the SDK (we still are returning things as 200 messages but with another output). However the templates have not been updated / deployed yet.

@mmabrouk
Copy link
Member Author

mmabrouk commented Jul 5, 2024

not sure tbh on the interface here

@mmabrouk
Copy link
Member Author

mmabrouk commented Jul 5, 2024

My guess is that FastAPI should return 401 with details

@mmabrouk mmabrouk closed this as completed Jul 5, 2024
@mmabrouk mmabrouk reopened this Jul 5, 2024
@mmabrouk mmabrouk closed this as completed Jul 5, 2024
@mmabrouk mmabrouk reopened this Jul 5, 2024
@mmabrouk mmabrouk closed this as completed Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backend bug Something isn't working evaluation Frontend High Priority linear Created by Linear-GitHub Sync
Projects
None yet
Development

No branches or pull requests

3 participants