📊 Adding global health data for mpox #3229
Conversation
|
Quick links (staging server):
Login: chart-diff: ✅No charts for review.data-diff: ❌ Found differences+ Dataset garden/health/latest/global_health_mpox
+ + Table global_health_mpox
+ + Column reported_cases
+ + Column suspected_cases_cumulative
= Dataset garden/who/latest/monkeypox
~ Table monkeypox (changed metadata)
- - title: Africa CDC Epidemic Intelligence Weekly Reports
+ + title: Mpox - 2024
~ Dim country
+ + New values: 1 / 85252 (0.00%)
date country
2024-08-22 Gabon
- - Removed values: 2 / 85252 (0.00%)
date country
2024-08-25 Congo
2024-08-12 Democratic Republic of Congo
~ Dim date
+ + New values: 1 / 85252 (0.00%)
country date
Gabon 2024-08-22
- - Removed values: 2 / 85252 (0.00%)
country date
Congo 2024-08-25
Democratic Republic of Congo 2024-08-12
~ Column annotation (new data, changed data)
+ + New values: 1 / 85252 (0.00%)
country date annotation
Gabon 2024-08-22
- - Removed values: 2 / 85252 (0.00%)
country date annotation
Congo 2024-08-25
Democratic Republic of Congo 2024-08-12 Includes only confirmed deaths as reported by WHO
~ Column iso_code (new data, changed data)
+ + New values: 1 / 85252 (0.00%)
country date iso_code
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date iso_code
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Changed values: 5004 / 85252 (5.87%)
country date iso_code - iso_code +
Africa 2023-11-27 OWID_AFR NaN
Africa 2024-02-26 OWID_AFR NaN
Asia 2022-07-31 OWID_ASI NaN
Asia 2023-06-18 OWID_ASI NaN
South America 2022-09-04 OWID_SAM NaN
~ Column new_cases (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_cases
Gabon 2024-08-22 <NA>
- - Removed values: 2 / 85252 (0.00%)
country date new_cases
Congo 2024-08-25 <NA>
Democratic Republic of Congo 2024-08-12 <NA>
~ Column new_cases_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_cases_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_cases_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column new_cases_smoothed (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_cases_smoothed
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_cases_smoothed
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column new_cases_smoothed_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_cases_smoothed_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_cases_smoothed_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column new_deaths (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_deaths
Gabon 2024-08-22 <NA>
- - Removed values: 2 / 85252 (0.00%)
country date new_deaths
Congo 2024-08-25 <NA>
Democratic Republic of Congo 2024-08-12 <NA>
~ Column new_deaths_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_deaths_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_deaths_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column new_deaths_smoothed (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_deaths_smoothed
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_deaths_smoothed
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column new_deaths_smoothed_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date new_deaths_smoothed_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date new_deaths_smoothed_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column suspected_cases_cumulative (changed metadata, new data, changed data)
- - description: |-
- - Suspected mpox cases are those which are likely to be mpox based on symptoms, but the infection has not been confirmed by a laboratory.
+ + - producer: Global.health
+ + title: Mpox - 2024
+ + citation_full: Global.health Mpox (accessed on 2024-09-02)
+ + url_main: https://global.health/
+ + url_download: https://mpox-2024.s3.eu-central-1.amazonaws.com/latest.csv
- - - producer: Africa Centres for Disease Control and Prevention
- - title: Africa CDC Epidemic Intelligence Weekly Reports
- - citation_full: Africa CDC, Epidemic Intelligence Weekly Reports (2024)
- - attribution: Africa CDC
- - attribution_short: Africa CDC
- - url_main: https://africacdc.org/thematic-area/surveillance-and-disease-intelligence/africa-cdc-weekly-event-based-surveillance/
- - url_download: https://docs.google.com/spreadsheets/d/1KlqUXISKxEpvHxxG8rSzqT3YK5w0mEpJH0ttyAOGLwo/pub?output=csv
- - date_accessed: '2024-08-30'
? ^ -
+ + date_accessed: '2024-09-02'
? ^ +
- - date_published: '2024-08-16'
? ^ ^^
+ + date_published: '2024-09-02'
? ^ ^^
+ + name: CC BY 4.0
+ + url: https://global.health/terms-of-use/
- - name: © All rights reserved Africa CDC
- - licenses:
- - - name: © All rights reserved Africa CDC
- - display:
- - numDecimalPlaces: 0
+ + New values: 1 / 85252 (0.00%)
country date suspected_cases_cumulative
Gabon 2024-08-22 5
- - Removed values: 2 / 85252 (0.00%)
country date suspected_cases_cumulative
Congo 2024-08-25 215
Democratic Republic of Congo 2024-08-12 27136
~ Changed values: 26 / 85252 (0.03%)
country date suspected_cases_cumulative - suspected_cases_cumulative +
Cameroon 2024-03-29 125 <NA>
Cameroon 2024-08-16 <NA> 143
Congo 2024-03-19 98 <NA>
Congo 2024-06-01 163 164
Democratic Republic of Congo 2024-03-20 15928 <NA>
~ Column total_cases (new data)
+ + New values: 1 / 85252 (0.00%)
country date total_cases
Gabon 2024-08-22 <NA>
- - Removed values: 2 / 85252 (0.00%)
country date total_cases
Congo 2024-08-25 <NA>
Democratic Republic of Congo 2024-08-12 <NA>
~ Column total_cases_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date total_cases_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date total_cases_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
~ Column total_deaths (new data)
+ + New values: 1 / 85252 (0.00%)
country date total_deaths
Gabon 2024-08-22 <NA>
- - Removed values: 2 / 85252 (0.00%)
country date total_deaths
Congo 2024-08-25 <NA>
Democratic Republic of Congo 2024-08-12 <NA>
~ Column total_deaths_per_million (new data)
+ + New values: 1 / 85252 (0.00%)
country date total_deaths_per_million
Gabon 2024-08-22 NaN
- - Removed values: 2 / 85252 (0.00%)
country date total_deaths_per_million
Congo 2024-08-25 NaN
Democratic Republic of Congo 2024-08-12 NaN
Legend: +New ~Modified -Removed =Identical Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippetAutomatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included Edited: 2024-09-02 13:05:48 UTC |
Damn, staging servers weren't fetching the most recent |
scripts/update-global-health-mpox.sh
Outdated
| start_time=$(date +%s) | ||
|
|
||
| echo '--- Update Mpox - Global Health' | ||
| cd /home/owid/etl |
There was a problem hiding this comment.
Could you update file scripts/update-monkeypox.sh instead and add those lines there? (i.e. it would be executing two snapshot files and doing single commit) It'll be nicer to have it in one place.
There was a problem hiding this comment.
Sure thing - I just added "poetry run python snapshots/health/latest/global_health_mpox.py" to the existing bash script and then deleted this one.
Ah awesome, thanks! |
|
@Marigold - is there a way to show the new data from the explorer step in the preview of the explorer? In the current preview there are suspected cases for Central African Republic, but there shouldn't be any in the new dataset (or at least there isn't when I run it locally). |
Ah I found something you had previously said and sorted it, thanks past Mojmir! |
|
Hey @Marigold! Am I okay to merge this? It looks like the failing CI has to do with an unrelated reason. But wanted to check before I press the big green button. |
I think based on the conversation I just saw in tech-issues it's okay to merge, so I'll go for it. |
Hey @Marigold
Here, I'm trying to use the data from [Global.health] (https://global.health/) for suspected cases instead of the fast-track dataset. However the Mpox explorer on this staging server seems to be one from a few weeks ago and doesn't show suspected cases.
Is it possible to force the one on staging to be in line with the one on live?