Skip to content

A real-time dashboard for Virginia COVID-19 case reports and vaccinations with end-to-end data collection, storage, and processing pipeline

License

Notifications You must be signed in to change notification settings

crystal-ctrl/engineering_project

Repository files navigation

Exploring the Vax Potential

Real-Time Dashboard for Virginia COVID-19 Report and Vaccination

Metis engineering project

Abstract

The goal of this project built an end-to-end data storage and processing pipeline that created a dashboard to visualize Virginia COVID-19 case reports and vaccinations with real-time data. The impact of this project is to provide a user-friendly interactive dashboard that allows users to visualize both real-time COVID-19 case reports and vaccinations. The pipeline created was able to collect data from API, store data in SQL database and preprocess data, and was automated with cron job for real-time data. The data were then used to build a Dash dashboard app. The web app dashboard consists of two main components: State overview and County summary. The dashboard is live and can be view here.

Design

As the Delta variant of COVID-19 spread looming over US, CDC and healthcare experts are warning poorly vaccinated regions to be prepared for the renewed danger. Upon researching for my current state Virginia's status, I realized there are mostly cases surveillance dashboard. Although there are a few vaccination dashboard, they are separated from the cases surveillance, which can be inconvinient for users if they want to explore the relationship between cases and vaccinations.

So the goal of this project is to build a pipeline for a dashboard to visualize Viriginia COVID-19 case reports and vaccinations with real-time data.

Data

  • The Virginia Vaccine Administered dataset (267,000 data points) and COVID cases dataset (60,600 data points) from Virginia Open Data. The data are updated every day so the database is constantly growing.
  • Cense data for Virginia counties population

Methodology

Data Engineering Workflow:

Purpose Connects Main script Function libary
Update content database API -> Database preprocess_data.py collect_data.py
Launch Dash app Database -> App index.py app.py
state.py
county.py
Deploy app on Heroku App -> Web Procfile requirements.txt
  • API calling daily to update the dashboard with cron job

Data Visualization & Application Production:

  • Plotly, Dash

  • Deployed the Dash app on Heroku

Techonologies

  • SODA API
  • SQL, sqlachemy
  • Python(pandas,numpy)
  • Dash
  • Heroku

Communications

In addition to the presentation slides, you can check out the app here.

Notebooks can be found here.

About

A real-time dashboard for Virginia COVID-19 case reports and vaccinations with end-to-end data collection, storage, and processing pipeline

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published