Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 fix duplicate values in covid sequences #3180

Merged
merged 2 commits into from
Aug 23, 2024
Merged

Conversation

Marigold
Copy link
Collaborator

No description provided.

@Marigold Marigold marked this pull request as ready for review August 23, 2024 07:34
@owidbot
Copy link
Contributor

owidbot commented Aug 23, 2024

Quick links (staging server):

Site Admin Wizard

Login: ssh owid@staging-site-fix-covid-sequence

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences
= Dataset garden/who/latest/monkeypox
  = Table monkeypox
    ~ Dim country
+       + New values: 366 / 85201 (0.43%)
                date  country
          2024-08-09 Barbados
          2022-05-03   Brazil
          2022-06-05   Brazil
          2024-08-12 Colombia
          2024-08-11 Paraguay
-       - Removed values: 29 / 85201 (0.03%)
                date country
          2022-06-30 Bahamas
          2022-07-09 Bahamas
          2022-07-11 Bahamas
          2022-07-18 Bahamas
          2022-07-21 Bahamas
    ~ Dim date
+       + New values: 366 / 85201 (0.43%)
           country       date
          Barbados 2024-08-09
            Brazil 2022-05-03
            Brazil 2022-06-05
          Colombia 2024-08-12
          Paraguay 2024-08-11
-       - Removed values: 29 / 85201 (0.03%)
          country       date
          Bahamas 2022-06-30
          Bahamas 2022-07-09
          Bahamas 2022-07-11
          Bahamas 2022-07-18
          Bahamas 2022-07-21
    ~ Column annotation (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date annotation
          Barbados 2024-08-09           
            Brazil 2022-05-03           
            Brazil 2022-06-05           
          Colombia 2024-08-12           
          Paraguay 2024-08-11           
-       - Removed values: 29 / 85201 (0.03%)
          country       date annotation
          Bahamas 2022-06-30           
          Bahamas 2022-07-09           
          Bahamas 2022-07-11           
          Bahamas 2022-07-18           
          Bahamas 2022-07-21
    ~ Column iso_code (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date iso_code
          Barbados 2024-08-09      BRB
            Brazil 2022-05-03      BRA
            Brazil 2022-06-05      BRA
          Colombia 2024-08-12      COL
          Paraguay 2024-08-11      PRY
-       - Removed values: 29 / 85201 (0.03%)
          country       date iso_code
          Bahamas 2022-06-30      BHS
          Bahamas 2022-07-09      BHS
          Bahamas 2022-07-11      BHS
          Bahamas 2022-07-18      BHS
          Bahamas 2022-07-21      BHS
    ~ Column new_cases (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_cases
          Barbados 2024-08-09          0
            Brazil 2022-05-03        716
            Brazil 2022-06-05          0
          Colombia 2024-08-12          0
          Paraguay 2024-08-11          0
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_cases
          Bahamas 2022-06-30          0
          Bahamas 2022-07-09          0
          Bahamas 2022-07-11          0
          Bahamas 2022-07-18          0
          Bahamas 2022-07-21          0
        ~ Changed values: 849 / 85201 (1.00%)
                country       date  new_cases -  new_cases +
                 Canada 2024-06-25            6            7
                Ecuador 2022-07-12            0            2
                Ecuador 2022-12-20            0           11
                Ecuador 2023-07-25            0            2
          North America 2022-09-20         1209         1214
    ~ Column new_cases_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_cases_per_million
          Barbados 2024-08-09                  0.000
            Brazil 2022-05-03                  3.328
            Brazil 2022-06-05                  0.000
          Colombia 2024-08-12                  0.000
          Paraguay 2024-08-11                  0.000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_cases_per_million
          Bahamas 2022-06-30                    0.0
          Bahamas 2022-07-09                    0.0
          Bahamas 2022-07-11                    0.0
          Bahamas 2022-07-18                    0.0
          Bahamas 2022-07-21                    0.0
        ~ Changed values: 828 / 85201 (0.97%)
                country       date  new_cases_per_million -  new_cases_per_million +
                   Asia 2023-07-03                    0.000                    0.004
                 Brazil 2024-02-13                    0.000                    0.152
                 Mexico 2024-05-07                    0.015                    0.000
          United States 2022-09-20                    2.643                    2.657
          United States 2022-10-04                    1.656                    1.668
    ~ Column new_cases_smoothed (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_cases_smoothed
          Barbados 2024-08-09            0.000000
            Brazil 2022-05-03          102.290001
            Brazil 2022-06-05            0.570000
          Colombia 2024-08-12            0.140000
          Paraguay 2024-08-11            0.000000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_cases_smoothed
          Bahamas 2022-06-30                0.14
          Bahamas 2022-07-09                0.00
          Bahamas 2022-07-11                0.00
          Bahamas 2022-07-18                0.00
          Bahamas 2022-07-21                0.00
        ~ Changed values: 5743 / 85201 (6.74%)
                country       date  new_cases_smoothed -  new_cases_smoothed +
                 Africa 2024-07-22             32.430000             33.000000
                 Canada 2024-06-25              0.860000              1.000000
                Ecuador 2023-10-11              0.000000              0.290000
                 Mexico 2024-07-06              0.710000              0.140000
          United States 2022-08-26            308.709991            313.140015
    ~ Column new_cases_smoothed_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_cases_smoothed_per_million
          Barbados 2024-08-09                           0.000
            Brazil 2022-05-03                           0.475
            Brazil 2022-06-05                           0.003
          Colombia 2024-08-12                           0.003
          Paraguay 2024-08-11                           0.000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_cases_smoothed_per_million
          Bahamas 2022-06-30                           0.341
          Bahamas 2022-07-09                           0.000
          Bahamas 2022-07-11                           0.000
          Bahamas 2022-07-18                           0.000
          Bahamas 2022-07-21                           0.000
        ~ Changed values: 4796 / 85201 (5.63%)
                country       date  new_cases_smoothed_per_million -  new_cases_smoothed_per_million +
                 Brazil 2023-09-02                             0.000                             0.009
                Ecuador 2022-12-25                             0.000                             0.087
                   Peru 2023-08-12                             0.000                             0.013
          United States 2022-05-31                             0.022                             0.020
          United States 2023-10-08                             0.021                             0.022
    ~ Column new_deaths (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_deaths
          Barbados 2024-08-09           0
            Brazil 2022-05-03           2
            Brazil 2022-06-05           0
          Colombia 2024-08-12           0
          Paraguay 2024-08-11           0
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_deaths
          Bahamas 2022-06-30           0
          Bahamas 2022-07-09           0
          Bahamas 2022-07-11           0
          Bahamas 2022-07-18           0
          Bahamas 2022-07-21           0
        ~ Changed values: 85 / 85201 (0.10%)
                country       date  new_deaths -  new_deaths +
                 Brazil 2022-07-12             0             3
                 Brazil 2022-11-29             2             0
          South America 2022-09-20             0             1
          South America 2022-11-29             4             2
                  World 2022-08-02             8             7
    ~ Column new_deaths_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_deaths_per_million
          Barbados 2024-08-09                  0.0000
            Brazil 2022-05-03                  0.0093
            Brazil 2022-06-05                  0.0000
          Colombia 2024-08-12                  0.0000
          Paraguay 2024-08-11                  0.0000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_deaths_per_million
          Bahamas 2022-06-30                     0.0
          Bahamas 2022-07-09                     0.0
          Bahamas 2022-07-11                     0.0
          Bahamas 2022-07-18                     0.0
          Bahamas 2022-07-21                     0.0
        ~ Changed values: 85 / 85201 (0.10%)
                country       date  new_deaths_per_million -  new_deaths_per_million +
                 Brazil 2022-07-12                   0.00000                   0.01393
                 Brazil 2022-11-29                   0.00927                   0.00000
          South America 2022-09-20                   0.00000                   0.00229
          South America 2022-11-29                   0.00913                   0.00457
                  World 2022-08-02                   0.00100                   0.00088
    ~ Column new_deaths_smoothed (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_deaths_smoothed
          Barbados 2024-08-09                 0.00
            Brazil 2022-05-03                 0.29
            Brazil 2022-06-05                 0.00
          Colombia 2024-08-12                 0.00
          Paraguay 2024-08-11                 0.00
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_deaths_smoothed
          Bahamas 2022-06-30                  0.0
          Bahamas 2022-07-09                  0.0
          Bahamas 2022-07-11                  0.0
          Bahamas 2022-07-18                  0.0
          Bahamas 2022-07-21                  0.0
        ~ Changed values: 592 / 85201 (0.69%)
                country       date  new_deaths_smoothed -  new_deaths_smoothed +
                Ecuador 2022-09-22                   0.00                   0.14
          South America 2022-07-05                   0.00                   0.29
          South America 2022-08-04                   0.14                   0.00
          South America 2022-12-06                   0.29                   0.14
                  World 2024-05-11                   0.29                   0.00
    ~ Column new_deaths_smoothed_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  new_deaths_smoothed_per_million
          Barbados 2024-08-09                          0.00000
            Brazil 2022-05-03                          0.00135
            Brazil 2022-06-05                          0.00000
          Colombia 2024-08-12                          0.00000
          Paraguay 2024-08-11                          0.00000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  new_deaths_smoothed_per_million
          Bahamas 2022-06-30                              0.0
          Bahamas 2022-07-09                              0.0
          Bahamas 2022-07-11                              0.0
          Bahamas 2022-07-18                              0.0
          Bahamas 2022-07-21                              0.0
        ~ Changed values: 592 / 85201 (0.69%)
                country       date  new_deaths_smoothed_per_million -  new_deaths_smoothed_per_million +
                Ecuador 2022-09-22                            0.00000                            0.00776
          South America 2022-07-05                            0.00000                            0.00066
          South America 2022-08-04                            0.00032                            0.00000
          South America 2022-12-06                            0.00066                            0.00032
                  World 2024-05-11                            0.00004                            0.00000
    ~ Column suspected_cases_cumulative (new data)
+       + New values: 366 / 85201 (0.43%)
           country       date  suspected_cases_cumulative
          Barbados 2024-08-09                        <NA>
            Brazil 2022-05-03                        <NA>
            Brazil 2022-06-05                        <NA>
          Colombia 2024-08-12                        <NA>
          Paraguay 2024-08-11                        <NA>
-       - Removed values: 29 / 85201 (0.03%)
          country       date  suspected_cases_cumulative
          Bahamas 2022-06-30                        <NA>
          Bahamas 2022-07-09                        <NA>
          Bahamas 2022-07-11                        <NA>
          Bahamas 2022-07-18                        <NA>
          Bahamas 2022-07-21                        <NA>
    ~ Column total_cases (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  total_cases
          Barbados 2024-08-09            1
            Brazil 2022-05-03          716
            Brazil 2022-06-05          884
          Colombia 2024-08-12         4257
          Paraguay 2024-08-11          126
-       - Removed values: 29 / 85201 (0.03%)
          country       date  total_cases
          Bahamas 2022-06-30            1
          Bahamas 2022-07-09            1
          Bahamas 2022-07-11            1
          Bahamas 2022-07-18            1
          Bahamas 2022-07-21            1
        ~ Changed values: 8124 / 85201 (9.54%)
                country       date  total_cases -  total_cases +
                Bahamas 2023-02-17              2              1
                Bahamas 2023-06-04              2              3
                 Brazil 2023-03-22          10890          10586
                 Brazil 2024-03-01          11212          11474
          North America 2024-03-06          39090          39161
    ~ Column total_cases_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  total_cases_per_million
          Barbados 2024-08-09                 3.542000
            Brazil 2022-05-03                 3.328000
            Brazil 2022-06-05                 4.107000
          Colombia 2024-08-12                81.283997
          Paraguay 2024-08-11                18.112000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  total_cases_per_million
          Bahamas 2022-06-30                    2.439
          Bahamas 2022-07-09                    2.439
          Bahamas 2022-07-11                    2.439
          Bahamas 2022-07-18                    2.438
          Bahamas 2022-07-21                    2.438
        ~ Changed values: 8102 / 85201 (9.51%)
                country       date  total_cases_per_million -  total_cases_per_million +
                 Africa 2024-08-01                   3.790000                   3.796000
                Ecuador 2023-04-26                  29.356001                  35.634998
                 Panama 2023-06-15                  52.625999                  54.417000
          South America 2023-02-03                  49.981998                  49.668999
          United States 2023-03-27                  89.477997                  89.462997
    ~ Column total_deaths (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  total_deaths
          Barbados 2024-08-09             0
            Brazil 2022-05-03             2
            Brazil 2022-06-05             2
          Colombia 2024-08-12             0
          Paraguay 2024-08-11             0
-       - Removed values: 29 / 85201 (0.03%)
          country       date  total_deaths
          Bahamas 2022-06-30             0
          Bahamas 2022-07-09             0
          Bahamas 2022-07-11             0
          Bahamas 2022-07-18             0
          Bahamas 2022-07-21             0
        ~ Changed values: 6052 / 85201 (7.10%)
                country       date  total_deaths -  total_deaths +
                 Brazil 2023-12-07              16              15
                  Chile 2023-06-10               2               3
          South America 2023-10-01              44              45
                  World 2022-08-20              35              43
                  World 2023-05-08             160             165
    ~ Column total_deaths_per_million (new data, changed data)
+       + New values: 366 / 85201 (0.43%)
           country       date  total_deaths_per_million
          Barbados 2024-08-09                   0.00000
            Brazil 2022-05-03                   0.00930
            Brazil 2022-06-05                   0.00929
          Colombia 2024-08-12                   0.00000
          Paraguay 2024-08-11                   0.00000
-       - Removed values: 29 / 85201 (0.03%)
          country       date  total_deaths_per_million
          Bahamas 2022-06-30                       0.0
          Bahamas 2022-07-09                       0.0
          Bahamas 2022-07-11                       0.0
          Bahamas 2022-07-18                       0.0
          Bahamas 2022-07-21                       0.0
        ~ Changed values: 6052 / 85201 (7.10%)
                country       date  total_deaths_per_million -  total_deaths_per_million +
                 Brazil 2023-12-07                     0.07375                     0.06914
                  Chile 2023-06-10                     0.10189                     0.15284
          South America 2023-10-01                     0.09988                     0.10215
                  World 2022-08-20                     0.00438                     0.00539
                  World 2023-05-08                     0.01991                     0.02054


Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2024-08-23 07:38:19 UTC
Execution time: 14.54 seconds

@Marigold Marigold merged commit 0a5fba4 into master Aug 23, 2024
8 checks passed
@Marigold Marigold deleted the fix-covid-sequence branch August 23, 2024 07:49
@lucasrodes lucasrodes mentioned this pull request Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants