String is empty or null (Parameter 'refreshToken') during authentication

[gsuberland]

I've had a script working for many years with the following code:

	private static async Task Start()
	{
		var json = await File.ReadAllTextAsync(CredentialsPath);
		var token = JsonConvert.DeserializeObject<PKCETokenResponse>(json);
		
		var authenticator = new PKCEAuthenticator(clientId!, token!);
		authenticator.TokenRefreshed += (sender, token) => File.WriteAllText(CredentialsPath, JsonConvert.SerializeObject(token));

		var config = SpotifyClientConfig.CreateDefault()
		  .WithAuthenticator(authenticator);

		var spotify = new SpotifyClient(config);

		var me = await spotify.UserProfile.Current();
		// ...

With the following code for the initial authentication:

	private static async Task StartAuthentication()
	{
		var (verifier, challenge) = PKCEUtil.GenerateCodes();

		await _server.Start();
		_server.AuthorizationCodeReceived += async (sender, response) =>
		{
			await _server.Stop();
			var token = await new OAuthClient().RequestToken(
				new PKCETokenRequest(clientId!, response.Code, _server.BaseUri, verifier)
			);

			await File.WriteAllTextAsync(CredentialsPath, JsonConvert.SerializeObject(token));
			await Start();
		};

		var request = new LoginRequest(_server.BaseUri, clientId!, LoginRequest.ResponseType.Code)
		{
			CodeChallenge = challenge,
			CodeChallengeMethod = "S256",
			Scope = new List<string> { UserReadEmail, UserReadPrivate, PlaylistReadPrivate, PlaylistReadCollaborative, PlaylistModifyPrivate, PlaylistModifyPublic }
		};

		var uri = request.ToUri();
		try
		{
			BrowserUtil.Open(uri);
		}
		catch (Exception)
		{
			Logger.Error($"Unable to open URL, manually open: {uri}");
		}
		
		await Task.Delay(30000);
	}

This is based on the documented examples.

The problem is that, since Spotify changed some stuff on their end, I'm finding that my script is broken and now randomly throws an ArgumentException when first attempting to use the API, and then gets stuck in that state.

The exception is "String is empty or null (Parameter 'refreshToken')", and the stack trace is:

   at SpotifyAPI.Web.Ensure.ArgumentNotNullOrEmptyString(String value, String name)
   at SpotifyAPI.Web.PKCETokenRefreshRequest..ctor(String clientId, String refreshToken)
   at SpotifyAPI.Web.PKCEAuthenticator.Apply(IRequest request, IAPIConnector apiConnector)
   at SpotifyAPI.Web.Http.APIConnector.ApplyAuthenticator(IRequest request)
   at SpotifyAPI.Web.Http.APIConnector.DoRequest(IRequest request, CancellationToken cancel)
   at SpotifyAPI.Web.Http.APIConnector.DoSerializedRequest[T](IRequest request, CancellationToken cancel)
   at SpotifyAPI.Web.Http.APIConnector.SendAPIRequest[T](Uri uri, HttpMethod method, IDictionary`2 parameters, Object body, IDictionary`2 headers, CancellationToken cancel)

I have verified that the refreshToken is indeed null in the JSON. If I delete the credential file and manually re-auth in the browser, it works again for a while.

The script runs hourly in a scheduled task. What I think is the root cause is that Spotify shortened the expiry time on the tokens to 1 hour (the token shows ExpiresIn set to 3600 seconds) at the same time they recently changed their authentication policy to disallow http://localhost as a callback target (it now has to be 127.0.0.1). My suspicion is that SpotifyAPI-NET is seeing a non-expired token (PKCETokenResponse.IsExpired = false) at that one hour mark, but by the time the network request is in-flight it is seen as expired by Spotify, which causes the breakage and for some reason deletes the token. I might be wrong though - just a hunch.

Any ideas?

[gsuberland]

Actually, I think I found the issue. Spotify seems to have changed how they do this. Their docs at https://developer.spotify.com/documentation/web-api/tutorials/code-pkce-flow#request-an-access-token say:

A refresh token that can be used to request new access tokens. When refreshing an access token, the response might not include a new refresh token. If it does not, continue using the existing token. See refreshing tokens.

So they now return null refresh tokens when you should continue using the existing one. That's the change that broke this, and the example code should probably be updated to demonstrate this problem.