2024 11 02 (Saturday) Deployment

General Notes

This deployment contains the latest assortment of work from Flexion. See the bug fixes and story below.

Feature Stories

• https://github.com/flexion/ef-cms/issues/10480

Bug Fixes

• https://github.com/flexion/ef-cms/issues/10074
• https://github.com/flexion/ef-cms/issues/10363

Observations

Timeline

(2024-10-31)

• 23:34 - Created the Pull Request

(2024-11-01)

• 11:52 - New postgres-related secrets added to the production Secrets Manager
• 12:02 - Docker container version 4.3.15 built and published to the production ECR
• 12:05 - Batch zipper feature flag added to the deploy table
• 12:09 - Account-specific terraform deployment completes
• 12:35 - All-colors terraform deployment completes
• 12:36 - Migration-related fields modified in the deploy table
• 12:37 - Database users created in the newly-deployed postgres cluster
• 12:55 - Merged the PR - CircleCI Build
• 13:24 - Deploy step fails - Batch Compute Environment in unexpected state
• 14:01 - Previously-failing terraform redeployed
• 14:02 - Deployment restarted - CircleCI Build
• 14:10 - Deploy step completes
• 14:19 - Dynamo migration begins
• 15:14 - Dynamo migration completes
• 15:15 - OpenSearch indexing/Postgres migration begins
• 16:56 - OpenSearch indexing/Postgres migration completes
• 17:02 - Load tests pass
• 17:07 - Read-only smoketests pass
• 17:09 - Wait for color switch step reached
• 19:52 - Manually verified that the source/destination counts match

(2024-11-02)

• 23:55 - Manually verified that the source/destination counts match
• 23:58 - Wait for color switch step manually approved

(2024-11-03)

• 00:02 - Switched colors
• 00:19 - Cleanup step completes
• 00:37 - Begin script to remove message records from Dynamo
• 01:19 - Message cleanup completes