Data fetching and visualization with BI tools

Demo project employing data fetching, visualization and analysis.

Description

This is analysis of my Discover Weekly playlist suggested by Spotify. Firstly, fetch the playlist using Spotify API with Python. Then, visualize my data with Preset platform (Apache Superset spin-off) by creating an interactive dashboard.

You can interact with the final dashboard here (Note: Login into Preset is required.)

Dashboard contents can be found in dashboard/ directory.

Getting Started

Dependencies

• OS: WSL2 run on Windows 10
• Data processing: Conda enviroment with required Python packages (See conda_env.yml file).
• Data Viz: Apache Superset or Preset.

Installing

• For data fetching and processing, a Conda environment spotipy in conda_env.yml is used. For the installation type in terminal:

  conda env create -f conda_env.yml
  

• Personal creditantials must be stored in .env that is not version contorolled. To get the creditantials see this tutorial. The files strucutre is:

  USER_ID=...
  PLAYLIST_ID=...
  CLIENT_ID=...
  CLIENT_SECRET=...
  

  Paste requrired Spotify IDs instead of the dots.

• The easiset way to get Apache Superset is with Docker by dowloading the latest Docker image and running it on your machine. Alternatively, use Preset - a cloud based version of Apache Superset.

Executing program

• Once the Conda enviroment is installed and Spotify IDs are saved in .env file, the Python sript can be run to fetch a playlist's data.

1. Activate the Conda environment

   conda activate spotipy
   

2. Run the Python script

   python src/fetch_data.py
   

   The playlist data will be saved in .csv format in data/ directory - data/spotify_playlist.csv.

3. Upload the data to Apache Superset or Preset. I've used Preset services.

Authors

Igors Dubanevics - igordub

Acknowledgments

Inspiration, code snippets, etc.

• awesome-readme
• PurpleBooth
• dbader
• zenorocha
• fvcproductions