Conversation
…ommit When commitMigration fails after resources.json is pushed but before the remote terraform.tfstate is deleted (e.g. on a 403 from DELETE), the workspace was left with both state files. The next deploy found the stray resources.json and never re-migrated cleanly. Add rollbackRemoteMigration: on commit failure, delete remote resources.json (ignore ErrNotExist) and, if the remote terraform.tfstate is gone (its DELETE can succeed even when the response is lost), restore it from the .backup pushDirectState wrote. The workspace is left terraform-authoritative so the next deploy retries the migration from scratch. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: e00ad7b
Top 3 slowest tests (at least 2 minutes):
|
Splitting commitMigration into pushMigrationToRemote (remote-only) and finalizeLocalMigration (local-only) so rollback fires only on a remote-push failure. Rolling back after the remote push already succeeded but a local rename failed would revert the workspace to terraform while this checkout's local direct state (higher serial) still wins on the next deploy, diverging from the workspace. A local-finalize failure is instead left to converge on retry, since the workspace is already direct-authoritative. Co-authored-by: Isaac <no-reply@databricks.com>
Once resources.json lands on the workspace (serial tf+1), any leftover terraform state files cannot affect engine selection because they carry a lower serial. The terraform backup/delete steps are cleanup, not commit. Simplify the commit path: pushDirectState only uploads resources.json. Reuse BackupRemoteTerraformState (already fail-open with log.Warnf) for the remote terraform cleanup, and make the local renames fail-open too. Remove rollbackRemoteMigration — there is nothing to roll back. Co-authored-by: Isaac <no-reply@databricks.com>
…esources.json committed Co-authored-by: Isaac <no-reply@databricks.com>
denik
changed the title
Co-authored-by: Isaac <no-reply@databricks.com>
andrewnester
approved these changes
Sep 21, 2026
denik
added a commit
that referenced
this pull request
Sep 21, 2026
Adopt the approach from #6772: pushing resources.json (serial tf+1) is the hard commit — once it lands, leftover terraform state carries a lower serial and can no longer win engine selection, so backing up/deleting the terraform state (remote and local) is best-effort cleanup, not part of the commit. pushDirectState now only uploads resources.json. finalizeLocalMigration places the converted state locally (still required — the deploy reads it) and then best-effort cleans up the terraform state via BackupRemoteTerraformState and a warn-on-failure local rename. Remove rollbackRemoteMigration — there is nothing to roll back once resources.json has committed. Only a resources.json push failure aborts the migration (fall back to terraform); a cleanup failure lets the migration succeed with a warning. auto-migrate-tfbackup-failure now asserts the migration succeeds with a warning and keeps the .backup, instead of falling back. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 21, 2026
PR #6772 makes the terraform state cleanup after a migration commit fail-open on main's post-deploy path. This branch already adopts the same approach in the before-deploy path, so keep this branch's direct_migration.go (which supersedes the post-deploy MigrateToDirect that #6772 edits) and take #6772's changelog fragment. Stacking on #6772 so it lands first. Co-authored-by: Isaac <no-reply@databricks.com>
denik
enabled auto-merge
September 21, 2026 15:16
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Once
resources.jsonlands on the workspace (serialtf+1), any leftover terraform state files cannot affect engine selection because they carry a lower serial. The terraform backup/delete steps are cleanup, not commit.Simplify:
pushDirectStateonly uploadsresources.json. ReuseBackupRemoteTerraformState(already fail-open) for the remote terraform cleanup, and make the local renames fail-open too. RemoverollbackRemoteMigration— there is nothing to roll back.