Transition Web Servers to the new Redis Central Service after making sure the current jobs are completed

[carolyncole]

What maintenance needs to be done?

We will have a new central Redis server for PDC describe after pulibrary/princeton_ansible#5343. We need to point our web servers at our new central server with care so that we do not lose any jobs on our current individual Redis servers.

Level of urgency

• High
• Moderate
• Low

Why is this maintenance needed?

We want to not longer back up a single server when data gets deposited. Sharing Redis will allow all servers to work the central queue

Acceptance criteria

• Check the current status of the sidekiq on each server (one at a time)
  • pdc-describe-prod1
    • turn off nginx
    • allow the queue to drain
    • run the playbook limiting to just this server
    • stop and disable local redis sudo systemctl disable redis-server
    • start nginx
  • pdc-describe-prod2
    • turn off nginx
    • allow the queue to drain
    • run the playbook limiting to just this server
    • stop and disable local redis sudo systemctl disable redis-server
    • start nginx
  • pdc-describe-prod3
    • turn off nginx
    • allow the queue to drain
    • run the playbook limiting to just this server
    • stop and disable local redis sudo systemctl disable redis-server
    • start nginx

Implementation notes, if any

Still have some cleanup to do, but at this point all servers should be pointing at the new central Redis