Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add postprocessing collation helper, minor prod/util edits #250

Merged
merged 20 commits into from
Dec 30, 2024

Conversation

dylanhmorris
Copy link
Contributor

@dylanhmorris dylanhmorris commented Dec 16, 2024

  • Create script postprocess_forecast_batches.py to automate plot collation, hubverse table creation, etc
  • Fix some import bugs in utils.py
  • Make setup_prod_job.py default to a job with COVID-19 and Influenza as the diseases
  • Add test of postprocessing via postprocess_forecast_batches.py to the end to end test, and make the end to end test try multiple locations x multiple diseases, including US national forecasts.

Copy link

codecov bot commented Dec 16, 2024

Codecov Report

Attention: Patch coverage is 0% with 81 lines in your changes missing coverage. Please review.

Project coverage is 18.54%. Comparing base (d5c0199) to head (886d440).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pipelines/postprocess_forecast_batches.py 0.00% 65 Missing ⚠️
hewr/R/process_state_forecast.R 0.00% 12 Missing ⚠️
pipelines/forecast_state.py 0.00% 2 Missing ⚠️
pipelines/utils.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #250      +/-   ##
==========================================
- Coverage   19.57%   18.54%   -1.04%     
==========================================
  Files          20       21       +1     
  Lines        1364     1440      +76     
==========================================
  Hits          267      267              
- Misses       1097     1173      +76     
Flag Coverage Δ
hewr 55.79% <0.00%> (-1.10%) ⬇️
pipelines 0.00% <0.00%> (ø)
pyrenew_hew 1.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@damonbayer
Copy link
Collaborator

Closes #163 ?

@dylanhmorris
Copy link
Contributor Author

dylanhmorris commented Dec 17, 2024

Closes #163 ?

Prerequisite but as written doesn't close because it's just a local helper for now and I didn't yet do anything with task dependencies for an azure prod job.

My instinct is "land this and then do that as a separate PR", but lmk if you think otherwise.

@damonbayer damonbayer changed the title Add postprocessing collation helper (Draft) Add postprocessing collation helper Dec 23, 2024
@dylanhmorris dylanhmorris marked this pull request as ready for review December 23, 2024 22:40
@dylanhmorris dylanhmorris changed the title (Draft) Add postprocessing collation helper Add postprocessing collation helper Dec 23, 2024
@dylanhmorris
Copy link
Contributor Author

@damonbayer ready for review.

@dylanhmorris dylanhmorris changed the title Add postprocessing collation helper Add postprocessing collation helper, minor prod/util edits Dec 23, 2024
Copy link
Collaborator

@damonbayer damonbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be possible to do a postprocess test in CI in this PR? I think that would put us in a really good position for making sure all the PyRenew-HE changes continue to work.

@damonbayer damonbayer self-requested a review December 27, 2024 15:42
Copy link
Collaborator

@damonbayer damonbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming this is still WIP, but formally requesting changes to address the CI failure.

@dylanhmorris
Copy link
Contributor Author

All CI now passes; ready for re-review @damonbayer (or @sbidari since @damonbayer is out).

Copy link
Collaborator

@sbidari sbidari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a very thorough review but it looks mostly good to me!

@dylanhmorris dylanhmorris merged commit 2510e26 into main Dec 30, 2024
13 of 15 checks passed
@dylanhmorris dylanhmorris deleted the dhm-add-postprocess-helper branch December 30, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants