Skip to content

Latest commit

 

History

History
22 lines (12 loc) · 961 Bytes

data-pipeline-project.md

File metadata and controls

22 lines (12 loc) · 961 Bytes

Data Pipeline Project

In this project you are going to build a data pipeline that is processing the Green Taxi Trips of the NYC Taxi Trip Dataset.

  1. The first task is to write a script that is directly uploading the data of the first 3 months of the year 2021 to a GCS bucket.

  2. The second task is to write a ETL or ELT pipeline that is taking the data from the GCS bucket and that is processing the data and calculating the revenue per day. This can be done in the Google Cloud or on your local machine.

Bonus task if you have the time:

  1. Use prefect for the Workflow Orechestration.

Questions:

  1. What are the steps you took to complete the project?
  2. What are the challenges you faced?
  3. What are the things you would do differently if you had more time?

Submission:

Please submit your solution as a link to a github repository. The repository should contain the scripts and a README.md file that is answering the questions above.