Skip to content

Patch improperly failed files 2021-09-29--2021-10-04 for 7 indicators #1292

Open
@krivard

Description

@krivard

On 2021-09-29, we deployed changes to the data exporter and archive differ to automatically populate missingness columns to indicate deleted rows or typically-unavailable columns in the files we submit to delphi-epidata acquisition. From this day through 2021-10-04, acquisition improperly failed files that included the resulting missingness annotations.

This affected the following indicators:

  • chng
  • covid-act-now
  • hhs
  • hospital-admissions
  • jhu-csse
  • nchs-mortality
  • usa-facts

This Kibana visualization shows how many files were improperly failed for each source:
image

We need to generate a patch for each of the affected indicators over these dates and upload it using the batch issue uploader. For everything except hospital-admissions, this can be done using the S3 ArchiveDiffer history. For hospital-admissions, we'll need to call in Maria.

Metadata

Metadata

Assignees

No one assigned

    Labels

    data qualityMissing data, weird data, broken dataneeds-coordinationThis work should be assigned to a coordinator and split up into several subtasks

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions