songnet is sequence model that gives personalized song recommendations based on the listening patterns of a specific user.
When listening to a particular song on Spotify, I find myself thinking about "related" songs. It is hard to quantitatively descrive "related". It needs to be understood that "related" is not necessarliy "similar" (which is the way Spotify does it). For example, sometimes I go from one genre of songs to another, other times I go from happy songs to sad songs by. Usually, I would just search for the song and listen. I wanted something that can generate personalized playlist(s) given initial few songs based on my listening patterns. This project is an attempt to solve this problem of learning listening patterns of a user.
Currently, this project uses two things from Spotify:
- Song Features (Easy to get using Spotify API)
- Spoitfy Streaming History (Not possible to get using Spotify API)
The only way to get your streaming history is by requesting Spotify your personal data and they process it and send it back to you after a few days. Since it is currently not possible to extract this streaming history from the API, making this network generic is a lot harder than usual for an end-user. Although, the code itself can be used to train the model for any particular user given that they provide their streaming history. There is no active maintainence to make this code ready to use for any end-user.
For attempting to train the network with your own streaming history, take a look at train_spotify.py
The streaming data is converted into variable length sequences of songs. To mitigate the problem of long-term streaming activity, we only consider a song sequence to be a sequence if they were streamed in an single hour. This time-span choice is arbitrary but can be tuned personally based on the user themselves.
The model is a simple LSTM based sequence model, that takes in sequences of songs to generate a target song. Each song in the sequence is represented by its features extracted from the Spotify API such as acousticness, valence etc.
Once the model is trained, we generate personalized playlist by bootstrapping the model by giving initial few songs. And then use the generated song index to add more songs to the sequence.
Since the purpose of the model is very subjective and there is not objective way to determine the recommendations generated by the model, It is hard to quantitively analyze the models correctness. However, here are some playlists generated by the model for those who are interested in the results. Overall, based on personal streaming history, the output is fairly in line with my expectations. In the following playlist, the model is seeded by the first three songs (indicated in bold) and the rest are generated by the model.
| Title | Artist |
|---|---|
| Faded | Alan Walker |
| Fearless Pt. II | Lost Sky |
| Don't You Worry Child | Swedish House Mafia |
| Manchala | Vishal-Shekhar |
| Infectious | Tobu |
| Tera Ban Jaunga | Akhil Sachdeva |
| Heading Home | Alan Walker |
| Animals | Maroon 5 |
| Galliyan (From "Ek Villain") | Ankit Tiwari |
| Link | Jim Yosef |
| Burn | Ellie Goulding |
| Stereo Love - Edit | Edward Maya |
| Green Light | Lorde |
| Saathiya | A.R. Rahman |
| Make Me Move | Culture Code |
| Love Story (Taylor’s Version) | Taylor Swift |
| Phoenix (Blanke Remix) | League of Legends |
| Candyland | Tobu |
| Sunburst | Tobu |
| Until We Get There | Lucius |
| Title | Artist |
|---|---|
| Beete Lamhein | KK |
| Main Hoon Na(Sad) | Abhijeet |
| Ae Dil Hai Mushkil Title Track | Pritam |
| Tere Liye - Hip Hop Mix | Sachin Gupta |
| this is me trying | Taylor Swift |
| Tere Liye - Hip Hop Mix | Sachin Gupta |
| Hamari Sansoon Men | Noor Jehan |
| Bolna (From "Kapoor & Sons (Since 1921)") | Tanishk Bagchi |
| Yakeen | Atif Aslam |
| Heat Waves | Meric Again |
| Infinity | Jaymes Young |
| Well Meet Again | TheFatRat |
| Walk Thru Fire | Vicetone |
| So Close To Magic | Aquilo |
| Heading Home | Alan Walker |
| Lose You To Love Me | Selena Gomez |
| The World We Knew | Daniel Olsén |
| Dancing With Your Ghost | Sasha Alex Sloan |
| Superheroes | The Script |
| Theyre Just Ghosts | Motion Blur |
The inital idea behind this project was that of incorporating listening pattern while generating recommendation along with the genre of the songs (Unlike typical Spotify recommendations in my experience). We can see how the model transitions from one genre to another although we seeded it with songs from one genre. And this transitions looks a lot like a transition that sometimes I do a lot while searching and listening to songs manually.
-
Add an interface using Spotify API so that we can add weekly generated recommendations to one's library provided some initial seed songs leveraging the model to its limits.
-
Spotify API features while being easy to get are arbitrary i.e. they are based on expert knowledge and are hand-crafted. To achieve the true power of Deep-Learning, we must fingerprint i.e. extract features of a song based on its waveform. There are some scripts that try to this but are currently very limited and do not perform well.
-
Also, if possible, the language of the song should be added as a feature because in my personal experience, streaming multi-language songs in a single session is rare. So the model needs to make an informed decision while generating recommendation