Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft PR: Scoring plots #98

Merged
merged 7 commits into from
Nov 1, 2024
Merged

Conversation

SamuelBrand1
Copy link
Collaborator

@SamuelBrand1 SamuelBrand1 commented Nov 1, 2024

This draft PR tracks the contribution of these scoring plots/tables:

  • Table of model performance over season. Required columns: Model, abs WIS, rel. WIS, MAE, 50% coverage and 95% coverage.
  • line plot of relative WIS by each week: two curves, 1 is flat (cdc flat baseline) the other is pyrenew-hew
  • Heatmap plot of relative WIS by each state over season.
  • plot_wis showing relative contribution of under/overprediction and dispersion for cdc flat baseline and pyrenew-hew.

@SamuelBrand1
Copy link
Collaborator Author

I've added a function to summarise the scores by model (over all days) which gives the columns as described at the top.

@SamuelBrand1
Copy link
Collaborator Author

I've added a plotting function for (epi)weekly WIS scores. This might need a bit of tuning for visual niceness.

@dylanhmorris dylanhmorris merged commit 37efbed into dhm-run-scoringutils Nov 1, 2024
1 check failed
@dylanhmorris dylanhmorris deleted the scoring-plots branch November 1, 2024 18:19
@dylanhmorris dylanhmorris restored the scoring-plots branch November 1, 2024 18:36
@dylanhmorris dylanhmorris deleted the scoring-plots branch November 19, 2024 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants