README.md

Providence Bike Share

The goal of this project is to collect data from different bikeshare/scooter programs in Providence, RI through the gbfs location API. Then the data will be cleaned in Python and processed to track changes in location by bike_id to discern trips. The final clean, processed dataset will be analyzed further with Python & visualized using Python, QGIS and Tableau.

This is NOT offical trip data from any of the providers.

Prerequisites

You must have Python 3 installed. You can download it here.
Routing requires Graphhopper, OpenStreetMap.

To use AWS Lambda you must have an AWS account with IAM setup.
To download the files from your s3 bucket you will need AWS CLI.

Usage

Code is run from the command line with one requirement
providers list: jump | bird | lime | veoride | spin

--provider a bike/scooter company.
Optional -s or --sync to download newer files from s3 bucket.
Optional -t or --test runs a small sample of 2,000 files through the pipeline.

example: python main.py --provider jump

example: python main.py --provider lime --sync

AWS Cloud

I chose to use AWS Lambda to run the Python code collecting the data. The Lambda script ran every 5 minutes by a CloudWatch Events trigger. The Python script would then process the location gbfs feed api into a .json file and add it into an AWS S3 Bucket.

Data Features

clean_trips.csv

        - trip_id
        - battery_end
        - battery_start
        - lat_end
        - lat_start
        - lon_end
        - lon_start
        - timestamp_end
        - timestamp_start
        - neghbor_end
        - neghbor_start
        - ward_end
        - ward_start
        - bike_id
        - type
        - duration
        - duration_min
        - distance
        - time
        - speed_mph

report_bike_details.csv

        - bike_id
        - trips_count
        - charges_count
        - min_date
        - max_date
        - date_diff

report_daily_trips.csv

        - date
        - count
        - avg_wind
        - precipitation
        - max_temp

report_ward_trips.csv

        - ward_start
        - ward_end
        - count
        - percent

report_neighborhood_trips.csv

        - neghbor_start
        - neghbor_end
        - count
        - percent

clean_bike_routes.geojson

        - trip_id
        - geometry (LINESTRING)

report_trip_lines_straight.geojson

        - trip_id
        - lon_end
        - lon_start
        - lat_end
        - lat_start
        - battery_end
        - battery_start
        - timestamp_end
        - timestamp_start
        - neghbor_end
        - neghbor_start
        - ward_end
        - bike_id
        - duration
        - duration_min
        - type
        - distance
        - time
        - speed_mph
        - geometry (LINESTRING)

GBFS Location API

• Spin Scooters
• VeoRide Scooters (removed)
• Jump Bikes (removed)
• Lime Scooters (removed)
• Bird Scooters (removed)

Creations With Data

• JUMP Tableau Dashboard
• Collection of Maps
  • all routed trips
  • all straight lines
  • pvd-fest routed trips

References

• Weather Data (NOAA)
• Routing Engine Graphhopper
• Routing Map OpenStreetMap