Update our EDD process documentation

docs/contributing/database-migrations/edd.mdx

@@ -1,16 +1,24 @@
 import Tabs from "@theme/Tabs";
 import TabItem from "@theme/TabItem";
 
-# Evolutionary Database Design
+import refactoringPhases from "./stages_refactoring.jpg";
+
+# Evolutionary database design
 
 At Bitwarden we follow
 [Evolutionary Database Design (EDD)](https://en.wikipedia.org/wiki/Evolutionary_database_design).
 EDD describes a process where the database schema is continuously updated while still ensuring
 compatibility with older releases by using database transition phases.
 
-In short the Database Schema for the Bitwarden Server **must** support the previous release of the
-server. The database migrations will be performed before the code deployment, and in the event of a
-release rollback the database schema will **not** be updated.
+Additional requirements include:
+
+- **Zero-downtime deployments**: Which means that multiple versions of the application will be
+  running concurrently during the deployment window.
+- **Code rollback**: Critical defects in code should be able to be rolled back to the previous
+  version.
+
+To fulfill these additional requirements the database schema **must** support the previous release
+of the server.
 
 <bitwarden>
 
@@ -24,22 +32,69 @@ For background on this decision please see the [Evolutionary Database Design RFD
 
 ## Design
 
-### Nullable
-
-Database tables, views and stored procedures should almost always use either nullable fields or have
-a default value. Since this will allow stored procedures to omit columns, which is a requirement
-when running both old and new code.
-
-### EDD Process
-
-The EDD breaks up each database migration into three phases. _Start_, _Transition_ and _End_.
-
-![Refactoring Stages](./stages_refactoring.jpg)
-[https://www.martinfowler.com/articles/evodb.html#TransitionPhase](https://www.martinfowler.com/articles/evodb.html#TransitionPhase)
-
-This necessitates two different database migrations. The first migration adds new content and is
-backwards compatible with the existing code. The second migration removes content and is not
-backwards compatible with that same code prior to the first migration.
+Database changes can be categorized into two categories: destructive and non-destructive
+\[[1](./edd#further-reading)\]. A destructive change prevents existing functionality from working as
+expected without an accompanying code change. A non-destructive change is one that does not require
+a code change to allow the application to continue working as expected.
+
+### Non-destructive database changes
+
+Non-destructive changes almost always use either nullable fields or default values in the database
+tables, views, and stored procedures. This ensures that the stored procedures can be called without
+the new columns which allows it to run with both the old and new code.
+
+### Destructive changes
+
+While a new column can be added without being a destructive change, there are times when a new
+column will be considered a destructive change if the default value of the column is a non-constant
+value that needs to be computed from elsewhere.
+
+Destructive database changes are handled elegantly by breaking them up into three phases: _Start_,
+_Transition_ and _End_.
+
+<div style={{ margin: "1em" }}>
+  <img src={refactoringPhases} alt="Refactoring Phases" />
+  <div style={{ fontSize: 12, textAlign: "center" }}>
+    Refactoring Phases [<a href="./edd#further-reading">1</a>]
+  </div>
+</div>
+
+To add terminology to compliment the [above diagram](./edd#destructive-changes), migrations that are
+a part of "Deploy new changes, migrate data, put in scaffolding code" are considered _Initial_
+migrations. Migrations that are run during the Transition Phase are considered _Transition_
+migrations. And the migrations that run as a part of "Remove old schema, scaffolding code" are
+considered _Finalization_ migrations. The definitions of each are helpful when discussing the type
+of migration in relation to orchestrating them during a deployment.
+
+### Initial migrations
+
+- Compatible with _X.1.0_ **and** _X.2.0_ application code changes
+- Represents the beginning of a database change
+- Updates the database schema to support any new functionality while also maintaining old
+  functionality
+- Supports both the previous version of code and the one being upgraded to
+- Run during upgrade
+- Must execute quickly to minimize downtime.
+
+### Transition migrations
+
+- Compatible with _X.1.0_ **and** _X.2.0_ application code changes
+- The time between initial migration and finalization migration
+- Exists to provide an opportunity to rollback server to _X.1.0_ version prior to breaking changes
+- Only data population migrations may be run at this time, if they are needed
+  - Optional step, required only when migrating data would be too slow to execute during the initial
+    migration. This might be a column population, index creation, anything to prepare the database
+    for the _X.2.0_ version
+  - Must be run as a background task during the Transition phase.
+  - These MUST run in a way where the database stays responsive during the full migration
+- Schema changes are NOT to be run during this phase.
+
+### Finalization migrations
+
+- Only compatible with _X.2.0_ application code; represents the point of no return for this
+  migration
+- Removes columns, data, and fallback code required to support _X.1.0_ version
+- Should be run as a typical migration either during a subsequent upgrade
 
 ### Example
 
@@ -73,7 +128,7 @@ actions.
 :::
 
 <Tabs>
-<TabItem value="first" label="First Migration" default>
+<TabItem value="first" label="Initial Migration" default>
 
 ```sql
 -- Add Column
@@ -120,7 +175,7 @@ END
 ```
 
 </TabItem>
-<TabItem value="data" label="Data Migration">
+<TabItem value="data" label="Transition Migration">
 
 ```sql
 UPDATE [dbo].Customer SET
@@ -129,7 +184,7 @@ WHERE FirstName IS NULL
 ```
 
 </TabItem>
-<TabItem value="second" label="Second Migration">
+<TabItem value="second" label="Finalization Migration">
 
 ```sql
 -- Remove Column
@@ -173,65 +228,95 @@ END
 </TabItem>
 </Tabs>
 
-## Workflow
+## Deployment orchestration
+
+There are some important constraints to the implementation of the process:
+
+- Bitwarden Production environments are required to be on at all times
+- Self-host instances must support the same database change process; however, they do not have the
+  same always-on application constraint
+- Minimization of manual steps in the process
+
+The process to support all of these constraints is a complex one. Below is an image of a state
+machine that will hopefully help visualize the process and what it supports. It assumes that all
+database changes follow the standards that are laid out in [Migrations](./).
+
+---
+
+![Bitwarden EDD State Machine](./edd_state_machine.jpg) \[Open Image in a new tab for better
+viewing\]
+
+---
 
-The Bitwarden specific workflow for writing migrations are described below.
+### Online environments
 
-### Developer
+Schema migrations and data migrations as just migrations. The underlying implementation issue is
+orchestrating the runtime constraints on the migration. Eventually, all migrations will end up in
+`DbScripts`. However, to orchestrate the running of _Transition_ and associated _Finalization_
+migrations, they are kept outside of `DbScripts` until the correct timing.
 
-The development flow is described in [Migrations](./).
+In environments with always-on applications, _Transition_ scripts must be run after the new code has
+been rolled out. To execute a full deploy, all new migrations in `DbScripts` are run, the new code
+is rolled out, and then all _Transition_ migrations in the `DbScripts_transition` directory are run
+as soon as all of the new code services are online. In the case of a critical failure after the new
+code is rolled out, a Rollback would be conducted (see Rollbacks below). _Finalization_ migrations
+will not be run until the start of the next deploy when they are moved into `DbScripts`.
 
-### Devops
+After this deploy, to prep for the next release, all migrations in `DbScripts_transition` are moved
+to `DbScripts` and then all migrations in `DbScripts_finalization` are moved to `DbScripts`,
+conserving their execution order for a clean install. For the current branching strategy, ]Rs will
+be open against `master` when `rc` is cut to prep for this release. This PR automation will also
+handle renaming the migration file and updating any reference of `[dbo_finalization]` to `[dbo]`.
 
-#### On `rc` cut
+The next deploy will pick up the newly added migrations in `DbScripts` and set the previously
+repeatable _Transition_ migrations to no longer be repeatable, execute the _Finalization_
+migrations, and then execute any new migrations associated with the code changes that are about to
+go out.
 
-Create a PR moving the future scripts.
+The state of migrations in the different directories at any one time is is saved and versioned in
+the Migrator Utility which supports the phased migration process in both types of environments.
 
-- `DbScripts_future` to `DbScripts`, prefix the script with the current date, but retain the
-  existing date.
-- `dbo_future` to `dbo`.
-  <bitwarden>
-    <li>
-      Create a ticket in Jira with a `Due Date` of the release date to ensure future migrations are
-      merged in and ready to be executed. Set the ticket that created the future migration as a
-      blocker.
-    </li>
-  </bitwarden>
+### Offline environments
 
-#### After server release
+The process for offline environments is similar to the always-on ones. However, since they do not
+have the constraint of always being on, the _Initial_ and _Transition_ migrations will be run one
+after the other:
 
-1. Run whatever data migration scripts might be needed. (This might need to be batched and executed
-   until all the data has been migrated)
-2. After having the server run for a while execute the future migration script to clean up the
-   database.
+- Stop the Bitwarden stack as done today
+- Start the database
+- Run all new migrations in `DbScripts` (both _Finalization_ migrations from the last deploy and any
+  _Initial_ migrations from the deploy currently going out)
+- Run all _Transition_ migrations
+- Restart the Bitwarden stack.
 
 ## Rollbacks
 
 In the event the server release failed and needs to be rolled back, it should be as simple as just
 re-deploying the previous version again. The database will **stay** in the transition phase until a
-hotfix can be released, and the server can be updated.
+patch can be released, and the server can be updated. Once a patch is ready to go out, it is
+deployed the _Transition_ migrations are rerun to verify that the DB is in the state that it is
+required to be in.
 
-The goal is to resolve the issue quickly and re-deploy the fixed code to minimize the time the
-database stays in the transition phase. Should a feature need to be completely pulled, a new
-migration needs to be written to undo the database changes and the future migration will also need
-to be updated to work with the database changes. This is generally not recommended since pending
-migrations (for other releases) will need to be revisited.
+Should a feature need to be completely pulled, a new migration needs to be written to undo the
+database changes and the future migration will also need to be updated to work with the database
+changes. This is generally not recommended since pending migrations (for other releases) will need
+to be revisited.
 
 ## Testing
 
 Prior to merging a PR please ensure that the database changes run well on the currently released
 version. We currently do not have an automated test suite for this and it’s up to the developers to
 ensure their database changes run correctly against the currently released version.
 
-## Further Reading
+## Further reading
 
-- [Evolutionary Database Design](https://martinfowler.com/articles/evodb.html) (Particularly
-  [All database changes are database refactorings](https://martinfowler.com/articles/evodb.html#AllDatabaseChangesAreMigrations))
-- [The Agile Data (AD) Method](http://agiledata.org/) (Particularly
-  [Catalog of Database Refactorings](http://agiledata.org/essays/databaseRefactoringCatalog.html))
-- [Refactoring Databases: Evolutionary Database](https://databaserefactoring.com/)
-- Refactoring Databases: Evolutionary Database Design (Addison-Wesley Signature Series (Fowler))
-  ISBN-10: 0321774515
+1. [Evolutionary Database Design](https://martinfowler.com/articles/evodb.html) (Particularly
+   [All database changes are database refactorings](https://martinfowler.com/articles/evodb.html#AllDatabaseChangesAreMigrations))
+2. [The Agile Data (AD) Method](http://agiledata.org/) (Particularly
+   [Catalog of Database Refactorings](http://agiledata.org/essays/databaseRefactoringCatalog.html))
+3. [Refactoring Databases: Evolutionary Database](https://databaserefactoring.com/)
+4. Refactoring Databases: Evolutionary Database Design (Addison-Wesley Signature Series (Fowler))
+   ISBN-10: 0321774515
 
 [edd-rfd]:
   https://bitwarden.atlassian.net/wiki/spaces/PIQ/pages/177701412/Adopt+Evolutionary+database+design

docs/contributing/database-migrations/index.md

@@ -2,9 +2,9 @@
 sidebar_position: 2
 ---
 
-# Database Migrations
+# Database migrations
 
-## Applying Migrations
+## Applying migrations
 
 We use a `migrate.ps1` PowerShell script to apply migrations to the local development database. This
 script handles the different database providers that we support.
@@ -13,12 +13,12 @@ For instructions on how to use `migrate.ps1`, see the Getting Started section fo
 [MSSQL](../../getting-started/server/database/mssql/index.md#updating-the-database) and
 [Entity Framework](../../getting-started/server/database/ef/index.mdx#migrations)
 
-## Creating Migrations for New Changes
+## Creating migrations for new changes
 
 Any database change must be scripted as a migration for both our primary DBMS - MSSQL - as well as
 for Entity Framework. Follow the instructions below for each provider.
 
-### MSSQL Migrations
+### MSSQL migrations
 
 :::tip
 
@@ -37,24 +37,24 @@ It is possible that a change may not require a non-backwards-compatible end phas
 may be backwards-compatible in their final form). In that case, only one phase of changes is
 required.
 
-#### Backwards Compatible Migration
+#### Backwards compatible migration
 
 1. Modify the source `.sql` files in `src/Sql/dbo`.
 2. Write a migration script, and place it in `util/Migrator/DbScripts`. Each script must be prefixed
    with the current date.
 
-#### Non-Backwards Compatible Migration
+#### Non-backwards compatible migration
 
-1. Copy the relevant `.sql` files from `src/Sql/dbo` to `src/Sql/dbo_future`.
+1. Copy the relevant `.sql` files from `src/Sql/dbo` to `src/Sql/dbo_finalization`.
 2. Remove the backwards compatibility that is no longer needed.
-3. Write a new Migration and place it in `src/Migrator/DbScripts_future`. Name it
-   `YYYY-0M-FutureMigration.sql`.
+3. Write a new Migration and place it in `src/Migrator/DbScripts_finalization`. Name it
+   `YYYY-0M-FinalizationMigration.sql`.
    - Typically migrations are designed to be run in sequence. However since the migrations in
      DbScripts_future can be run out of order, care must be taken to ensure they remain compatible
      with the changes to DbScripts. In order to achieve this we only keep a single migration, which
      executes all backwards incompatible schema changes.
 
-### EF Migrations
+### EF migrations
 
 If you alter the database schema, you must create an EF migration script to ensure that EF databases
 keep pace with these changes. Developers must do this and include the migrations with their PR.
@@ -72,4 +72,15 @@ pwsh ef_migrate.ps1 [NAME_OF_MIGRATION]
 
 This will generate the migrations, which should then be included in your PR.
 
+### [Not Yet Implemented] Manual MSSQL migrations
+
+There may be a need for a migration to be run outside of our normal update process. These types of
+migrations should be saved for very exceptional purposes. One such reason could be an Index rebuild.
+
+1. Write a new Migration with a prefixed current date and place it in
+   `src/Migrator/DbScripts_manual`
+2. After it has been run against our Cloud environments and we are satisfied with the outcome,
+   create a PR to move it to `DbScripts`. This will enable it to be run by our Migrator processes in
+   self-host and clean installs of both cloud and self-host environments
+
 [code-style-sql]: ../code-style/sql.md