Skip to content

ETL asteroids data extractor using some Google Cloud services

License

Notifications You must be signed in to change notification settings

rubnsbarbosa/nasa-asteroids-extractor

Repository files navigation

Comet

Retrieve asteroids based on their closest approach to Earth data extractor from NASA API.

Introduction

This repository provides an example of a data extractor etl:

  • Extracts data from Asteroids - NeoWs API. GET https://api.nasa.gov/neo/rest/v1/feed?start_date=START_DATE&end_date=END_DATE&api_key=API_KEY
  • Transform the JSON object, then load the csv file into a gs://bucket
  • Load to Big Query from Google Cloud Storage

Prerequisites

Ensure you have the following:

  • Python 3.8 or later installed
  • Google Cloud SDK installed and configured
  • A Google Cloud project with billing enabled

Setup

  1. Clone the repository:
git clone https://github.com/rubnsbarbosa/nasa-asteroids-extractor.git
cd nasa-asteroids-extractor
  1. Create a virtual environment and install dependencies:
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

License

MIT.

About

ETL asteroids data extractor using some Google Cloud services

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published