Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing eval: automatic alert #3567

Open
jacopo-chevallard opened this issue Jan 28, 2025 — with Linear · 1 comment
Open

Parsing eval: automatic alert #3567

jacopo-chevallard opened this issue Jan 28, 2025 — with Linear · 1 comment
Assignees

Comments

Copy link
Collaborator

jacopo-chevallard commented Jan 28, 2025

The goal is to be alerted if the parsing metrics go below a certain threshold.

For this we need to:

  1. Define metrics used for alerts: global metrics, or individual ones?
  2. Define thresholds for the different metrics: we need to run a few initial tests to setup these thresholds, once we have defined which metrics to use for the alerts
  3. Create the alert service, or use service already included in exp. tracking platforms.
  4. Send alerts to Slack or other channels
@jacopo-chevallard jacopo-chevallard self-assigned this Jan 28, 2025
Copy link

linear bot commented Jan 28, 2025

@jacopo-chevallard jacopo-chevallard changed the title Parsing: automatic alert Parsing eval: automatic alert Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant