Refresh saved tracks in the background #502
Conversation
|
Exciting that you started thinking about this! This is a big one. I do think since refreshing the tracks takes such a long time (and also at some point might stall the program) given that it's paging through some large number of tracks, I think we ought to come up with a more comprehensive solution. Here are my thoughts so far... The solution I'm centering on so far involves caching the users save tracks and only fetching recently saved tracks upon initialization while also give a user a button to force resync all of their tracks. The hope is that if a user only uses Psst, then there will be no need to refresh the user saved tracks since everything will be controlled and observed by Psst. However, in the case that a user goes into the Spotify client and unlikes a bunch of very old liked songs, there would be no way to identify which those songs were that should be removed unless we refresh their entire liked tracked library. Saving some users like songs will take a very long time using the Spotify API since we can only query about 50 tracks at a time (see spec). For 8k songs that's 160 calls - we might be able to do some of those in parallel since we can stagger the calls, but we risk getting rate limited. Once we have this many songs, we don't want to repeat this process again. So, the second part of this feature will have be caching of the users tracks. Once we get all of this done, we can then have the tracks that get saved by user just simply get appended to the user's saved track list cache as well as making an API request that saves it to the user's library. |
|
Yes, the refreshing takes an unreasonable amount of time, I agree. However, sadly, the safest thing to do is assume that users interact with their saved tracks a lot just in case someone does and it becomes annoying. So I have an idea which builds off yours: we could cache only the first 50 or 100 songs so that it doesn't take up much space, and for the rest of the songs, we implement some lazy loading or off-screen loading which will mean that the user won't even be able to feel the rest of the songs being loaded in. For the change, we can perhaps, as you suggest, compare it at the start of the program, or maybe (I'm not sure if this is possible) we could update it while the user has scrolled away from it or have a refresh button... Or all three! |
|
Are you using etags when refreshing your cached data? Saves a lot of traffic and deserialisation work which speeds things up. In Mopidy-Spotify I found that a big (background) cache load of everything at the start is not noticeable to users, and then doing etag-aware refresh after that is speedy. I also bump up the expiry time of some assets (i.e. tracks, albums) since they don't realistically change. I had plans to persist the cache data between app runs but I found it wasn't necessary once the background refresh was implemented. However, that's partly due to the way Mopidy is run (asna service, at startup). Their endpoint for saved tracks is rubbish, it is dying for a way to specify the required response fields (like you can with playlists). |
|
@kingosticks I don't think so... we really should be caching every query this way. Would you be interesting in working on a PR? |
|
Sorry, no, I don't actually use psst. I'm just lurking in projects that implement Spotify streaming. |
This solves issue #501 by simply refreshing the saved tracks in the background after the user has pressed the save button.