Add --force-refresh flag to auth token command (#4767)

mihaimitrea-db · simonfaltum · web-flow · commit d151b00568b8 · 2026-03-27T16:47:50.000Z
Wire a new `--force-refresh` CLI flag that delegates to the SDK's `ForceRefreshToken()` method, bypassing the cached token validity check. The default path through `Token()` is unchanged. > **Note:** this will not compile until the SDK ships `ForceRefreshToken()` ([databricks/databricks-sdk-go#1552](databricks/databricks-sdk-go#1552)) and the CLI''s Go SDK dependency is bumped to v0.126. ## Changes - Add a `--force-refresh` boolean flag to `databricks auth token`. When set, the command calls `PersistentAuth.ForceRefreshToken()` instead of `PersistentAuth.Token()`, which always performs a token refresh against the IdP regardless of the cached token's remaining TTL. - The error handling is shared between both paths: invalid refresh tokens produce the same actionable "To reauthenticate, run..." message, and the backward-compat `cache.ErrNotFound` rewrite is preserved. - Add three acceptance tests covering the force-refresh flag end-to-end: - **`force-refresh-success`**: pre-populated cache with a valid token, `--force-refresh` returns the server's new token (not the cached one). - **`force-refresh-invalid-refresh-token`**: server returns 401 with invalid refresh token; asserts the actionable re-login error message. - **`force-refresh-no-cache`**: no cached token exists; asserts the backward-compat error message is preserved for `--force-refresh`. - Add unit tests verifying the default path still returns a cached valid token without refreshing, and that `--force-refresh` correctly delegates to `ForceRefreshToken()` for both success and failure cases. ## Why [#4564](#4564) reports that external consumers using `databricks auth token` as a credential helper (e.g. Claude Code via `apiKeyHelper`) can receive near-expired tokens that expire before they can be used. [databricks/databricks-sdk-go#1535](databricks/databricks-sdk-go#1535) addressed the common case by adding a 5-minute proactive refresh buffer to `Token()`. However, that proactive refresh is intentionally best-effort: `Token()` still returns the existing access token when it is valid and a proactive refresh fails, because callers did not explicitly ask for a fresh token. For integrations that treat the CLI as a token minter or want to manage their own cache/TTL policy, "return a still-usable token" is different from "refresh now and give me a newly minted token or fail." The `--force-refresh` flag gives those integrations an explicit way to guarantee a fresh token. ## Tests - Unit tests in `cmd/auth/token_test.go`: table-driven cases covering default-path cache reuse, force-refresh success, and force-refresh error preservation. - Acceptance tests in `acceptance/cmd/auth/token/force-refresh-{success,invalid-refresh-token,no-cache}/`: end-to-end tests against the mock OIDC server, run for both `terraform` and `direct` engine variants. --------- Co-authored-by: simon <simon.faltum@databricks.com>
diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md
@@ -5,6 +5,7 @@
 ### Notable Changes
 
 ### CLI
+* Add `--force-refresh` flag to `databricks auth token` to force a token refresh even when the cached token is still valid ([#4767](https://github.com/databricks/cli/pull/4767)).
 
 ### Bundles
 * engine/direct: Fix drift in grants resource due to privilege reordering ([#4794](https://github.com/databricks/cli/pull/4794))
diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/out.test.toml b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/out.test.toml
diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/output.txt
@@ -0,0 +1,4 @@
+Error: A new access token could not be retrieved because the refresh token is invalid. To reauthenticate, run the following command:
+  $ databricks auth login --profile test-profile
+
+Exit code: 1
diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/script
@@ -0,0 +1,4 @@
+setup_test_profile
+setup_test_token_cache
+
+errcode $CLI auth token --profile test-profile --force-refresh
diff --git a/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/test.toml b/acceptance/cmd/auth/token/force-refresh-invalid-refresh-token/test.toml
@@ -0,0 +1,4 @@
+[[Server]]
+Pattern = "POST /oidc/v1/token"
+Response.StatusCode = 401
+Response.Body = '{"error": "invalid_request", "error_description": "Refresh token is invalid"}'
diff --git a/acceptance/cmd/auth/token/force-refresh-no-cache/out.test.toml b/acceptance/cmd/auth/token/force-refresh-no-cache/out.test.toml
diff --git a/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt b/acceptance/cmd/auth/token/force-refresh-no-cache/output.txt
@@ -0,0 +1,3 @@
+Error: cache: databricks OAuth is not configured for this host. Try logging in again with `databricks auth login --profile test-profile` before retrying. If this fails, please report this issue to the Databricks CLI maintainers at https://github.com/databricks/cli/issues/new
+
+Exit code: 1
diff --git a/acceptance/cmd/auth/token/force-refresh-no-cache/script b/acceptance/cmd/auth/token/force-refresh-no-cache/script
@@ -0,0 +1,3 @@
+setup_test_profile
+
+errcode $CLI auth token --profile test-profile --force-refresh
diff --git a/acceptance/cmd/auth/token/force-refresh-success/out.test.toml b/acceptance/cmd/auth/token/force-refresh-success/out.test.toml
diff --git a/acceptance/cmd/auth/token/force-refresh-success/output.txt b/acceptance/cmd/auth/token/force-refresh-success/output.txt
@@ -0,0 +1,3 @@
+
+>>> [CLI] auth token --profile test-profile --force-refresh
+"oauth-token"
diff --git a/acceptance/cmd/auth/token/force-refresh-success/script b/acceptance/cmd/auth/token/force-refresh-success/script
@@ -0,0 +1,6 @@
+setup_test_profile
+setup_test_token_cache
+
+# The cached token is "cached-access-token". The mock OIDC server returns
+# "oauth-token", so this confirms --force-refresh fetched a fresh token.
+trace $CLI auth token --profile test-profile --force-refresh | jq .access_token
diff --git a/acceptance/cmd/auth/token/script.prepare b/acceptance/cmd/auth/token/script.prepare
@@ -0,0 +1,31 @@
+setup_test_profile() {
+    export DATABRICKS_HOST_ORIG="$DATABRICKS_HOST"
+
+    sethome "./home"
+    unset DATABRICKS_HOST
+    unset DATABRICKS_TOKEN
+    unset DATABRICKS_CONFIG_PROFILE
+
+    cat > "./home/.databrickscfg" <<ENDCFG
+[test-profile]
+host = $DATABRICKS_HOST_ORIG
+auth_type = databricks-cli
+ENDCFG
+}
+
+setup_test_token_cache() {
+    mkdir -p "./home/.databricks"
+    cat > "./home/.databricks/token-cache.json" <<ENDCACHE
+{
+  "version": 1,
+  "tokens": {
+    "test-profile": {
+      "access_token": "cached-access-token",
+      "token_type": "Bearer",
+      "refresh_token": "test-refresh-token",
+      "expiry": "2099-01-01T00:00:00Z"
+    }
+  }
+}
+ENDCACHE
+}
diff --git a/cmd/auth/in_memory_test.go b/cmd/auth/in_memory_test.go
@@ -10,20 +10,28 @@ type inMemoryTokenCache struct {
 }
 
 // Lookup implements TokenCache.
+// Returns a copy to match real (file-backed) cache behavior, where each
+// Lookup deserializes a fresh struct. Without the copy, callers that
+// mutate the returned token (e.g. clearing RefreshToken) would corrupt
+// entries shared across test cases.
 func (i *inMemoryTokenCache) Lookup(key string) (*oauth2.Token, error) {
 	token, ok := i.Tokens[key]
 	if !ok {
 		return nil, cache.ErrNotFound
 	}
-	return token, nil
+	cp := *token
+	return &cp, nil
 }
 
 // Store implements TokenCache.
+// Stores a copy to prevent callers from mutating cached entries after Store
+// returns (mirrors file-backed cache semantics).
 func (i *inMemoryTokenCache) Store(key string, t *oauth2.Token) error {
 	if t == nil {
 		delete(i.Tokens, key)
 	} else {
-		i.Tokens[key] = t
+		cp := *t
+		i.Tokens[key] = &cp
 	}
 	return nil
 }
diff --git a/cmd/auth/token.go b/cmd/auth/token.go
@@ -54,15 +54,20 @@ func newTokenCommand(authArguments *auth.AuthArguments) *cobra.Command {
 		Use:   "token [HOST_OR_PROFILE]",
 		Short: "Get authentication token",
 		Long: `Get authentication token from the local cache in ~/.databricks/token-cache.json.
-Refresh the access token if it is expired. Note: This command only works with
-U2M authentication (using the 'databricks auth login' command). M2M authentication
-using a client ID and secret is not supported.`,
+Refresh the access token if it is expired or close to expiry. Use --force-refresh
+to bypass expiry checks. Note: This command only works with U2M authentication
+(using the 'databricks auth login' command). M2M authentication using a client ID
+and secret is not supported.`,
 	}
 
 	var tokenTimeout time.Duration
 	cmd.Flags().DurationVar(&tokenTimeout, "timeout", defaultTimeout,
 		"Timeout for acquiring a token.")
 
+	var forceRefresh bool
+	cmd.Flags().BoolVar(&forceRefresh, "force-refresh", false,
+		"Force a token refresh even if the cached token is still valid.")
+
 	cmd.RunE = func(cmd *cobra.Command, args []string) error {
 		ctx := cmd.Context()
 		profileName := ""
@@ -76,6 +81,7 @@ using a client ID and secret is not supported.`,
 			profileName:        profileName,
 			args:               args,
 			tokenTimeout:       tokenTimeout,
+			forceRefresh:       forceRefresh,
 			profiler:           profile.DefaultProfiler,
 			persistentAuthOpts: nil,
 		})
@@ -106,6 +112,9 @@ type loadTokenArgs struct {
 	// tokenTimeout is the timeout for retrieving (and potentially refreshing) an OAuth token.
 	tokenTimeout time.Duration
 
+	// forceRefresh forces a token refresh even if the cached token is still valid.
+	forceRefresh bool
+
 	// profiler is the profiler to use for reading the host and account ID from the .databrickscfg file.
 	profiler profile.Profiler
 
@@ -248,7 +257,12 @@ func loadToken(ctx context.Context, args loadTokenArgs) (*oauth2.Token, error) {
 		helpMsg := helpfulError(ctx, args.profileName, oauthArgument)
 		return nil, fmt.Errorf("%w. %s", err, helpMsg)
 	}
-	t, err := persistentAuth.Token()
+	var t *oauth2.Token
+	if args.forceRefresh {
+		t, err = persistentAuth.ForceRefreshToken()
+	} else {
+		t, err = persistentAuth.Token()
+	}
 	if err != nil {
 		if errors.Is(err, cache.ErrNotFound) {
 			// The error returned by the SDK when the token cache doesn't exist or doesn't contain a token
diff --git a/cmd/auth/token_test.go b/cmd/auth/token_test.go
@@ -2,6 +2,7 @@ package auth
 
 import (
 	"context"
+	"errors"
 	"net/http"
 	"testing"
 	"time"
@@ -16,6 +17,12 @@ import (
 	"golang.org/x/oauth2"
 )
 
+type failOnCallTransport struct{}
+
+func (failOnCallTransport) RoundTrip(*http.Request) (*http.Response, error) {
+	return nil, errors.New("unexpected HTTP call")
+}
+
 var refreshFailureTokenResponse = fixtures.HTTPFixture{
 	MatchAny: true,
 	Status:   401,
@@ -135,6 +142,10 @@ func TestToken_loadToken(t *testing.T) {
 				Host:                 "https://m2m.cloud.databricks.com",
 				HasClientCredentials: true,
 			},
+			{
+				Name: "valid-token",
+				Host: "https://valid-token.cloud.databricks.com",
+			},
 		},
 	}
 	tokenCache := &inMemoryTokenCache{
@@ -181,11 +192,16 @@ func TestToken_loadToken(t *testing.T) {
 				RefreshToken: "dup1",
 				Expiry:       time.Now().Add(1 * time.Hour),
 			},
+			"valid-token": {
+				AccessToken:  "cached-access-token",
+				RefreshToken: "valid-token",
+				Expiry:       time.Now().Add(1 * time.Hour),
+			},
 		},
 	}
-	validateToken := func(resp *oauth2.Token) {
-		assert.Equal(t, "new-access-token", resp.AccessToken)
-		assert.Equal(t, "Bearer", resp.TokenType)
+	validateToken := func(got *oauth2.Token) {
+		assert.Equal(t, "new-access-token", got.AccessToken)
+		assert.Equal(t, "Bearer", got.TokenType)
 	}
 
 	cases := []struct {
@@ -699,6 +715,59 @@ func TestToken_loadToken(t *testing.T) {
 			},
 			validateToken: validateToken,
 		},
+		{
+			name: "default path reuses valid cached token without refresh",
+			args: loadTokenArgs{
+				authArguments: &auth.AuthArguments{},
+				profileName:   "valid-token",
+				args:          []string{},
+				tokenTimeout:  1 * time.Hour,
+				profiler:      profiler,
+				persistentAuthOpts: []u2m.PersistentAuthOption{
+					u2m.WithTokenCache(tokenCache),
+					u2m.WithOAuthEndpointSupplier(&MockApiClient{}),
+					u2m.WithHttpClient(&http.Client{Transport: failOnCallTransport{}}),
+				},
+			},
+			validateToken: func(got *oauth2.Token) {
+				assert.Equal(t, "cached-access-token", got.AccessToken)
+			},
+		},
+		{
+			name: "force refresh refreshes valid cached token",
+			args: loadTokenArgs{
+				authArguments: &auth.AuthArguments{},
+				profileName:   "valid-token",
+				args:          []string{},
+				tokenTimeout:  1 * time.Hour,
+				forceRefresh:  true,
+				profiler:      profiler,
+				persistentAuthOpts: []u2m.PersistentAuthOption{
+					u2m.WithTokenCache(tokenCache),
+					u2m.WithOAuthEndpointSupplier(&MockApiClient{}),
+					u2m.WithHttpClient(&http.Client{Transport: fixtures.SliceTransport{refreshSuccessTokenResponse}}),
+				},
+			},
+			validateToken: validateToken,
+		},
+		{
+			name: "force refresh preserves error handling on refresh failure",
+			args: loadTokenArgs{
+				authArguments: &auth.AuthArguments{},
+				profileName:   "valid-token",
+				args:          []string{},
+				tokenTimeout:  1 * time.Hour,
+				forceRefresh:  true,
+				profiler:      profiler,
+				persistentAuthOpts: []u2m.PersistentAuthOption{
+					u2m.WithTokenCache(tokenCache),
+					u2m.WithOAuthEndpointSupplier(&MockApiClient{}),
+					u2m.WithHttpClient(&http.Client{Transport: fixtures.SliceTransport{refreshFailureTokenResponse}}),
+				},
+			},
+			wantErr: `A new access token could not be retrieved because the refresh token is invalid. To reauthenticate, run the following command:
+  $ databricks auth login --profile valid-token`,
+		},
 	}
 	for _, c := range cases {
 		t.Run(c.name, func(t *testing.T) {

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	+Error: cache: databricks OAuth is not configured for this host. Try logging in again with `databricks auth login --profile test-profile` before retrying. If this fails, please report this issue to the Databricks CLI maintainers at https://github.com/databricks/cli/issues/new
	`2`	`+`
	`3`	`+Exit code: 1`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+setup_test_profile`
	`2`	`+`
	`3`	`+errcode $CLI auth token --profile test-profile --force-refresh`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+`
	`2`	`+>>> [CLI] auth token --profile test-profile --force-refresh`
	`3`	`+"oauth-token"`