Skip to content
/ Pyspark-parquet-to-csv Public

Convert Parquet files to CSV stored in S3 Using AWS Glue

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

sumairz/Pyspark-parquet-to-csv

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
README.md
README.md
 
 
main.py
main.py
 
 

Repository files navigation

This script convert parquet files to csv stored in s3. Run the script in AWS Glue as ETL Job.

It uses pendulum library to break timestamps into date range so that we can loop through s3 folders strcuture like year/month/day/hour

About

Convert Parquet files to CSV stored in S3 Using AWS Glue

Topics

python csv spark glue parquet

Resources

Readme
Activity

Stars

0 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages