|
| 1 | +--- |
| 2 | +sidebar_label: "Maintenance Mode" |
| 3 | +title: "Maintenance Mode" |
| 4 | +description: "Learn how to enable and use maintenance mode to cordon and drain nodes." |
| 5 | +hide_table_of_contents: false |
| 6 | +sidebar_position: 240 |
| 7 | +tags: ["clusters", "cluster management"] |
| 8 | +--- |
| 9 | + |
| 10 | +Similar to `kubectl` commands `cordon` and `drain`, maintenance mode allows you to temporarily disable scheduling for an |
| 11 | +active control plane or worker node. When a node is placed in maintenance mode, workloads are migrated automatically to |
| 12 | +other healthy nodes in the cluster without services being disrupted. Using maintenance mode makes it easier to perform |
| 13 | +necessary maintenance tasks, address node issues, and optimize workload distribution while maintaining the desired level |
| 14 | +of performance and availability. |
| 15 | + |
| 16 | +## Prerequistes |
| 17 | + |
| 18 | +- An active Palette host cluster with more than one control plane node and worker node. |
| 19 | + |
| 20 | +- Alternate nodes with sufficient resources available where processes from maintenance nodes can be provisioned. |
| 21 | + |
| 22 | +## Limitations |
| 23 | + |
| 24 | +<!-- prettier-ignore --> |
| 25 | +- Static pods and DaemonSets are not evicted from the node when activating maintenance mode. |
| 26 | + |
| 27 | +- Scans cannot be performed on the cluster when any node in the cluster is in maintenance mode. |
| 28 | + |
| 29 | +- Nodes in maintenance mode are not included in the backup process, which also means they cannot be restored. |
| 30 | + |
| 31 | +- Changes to add-on profiles are not applied to nodes in maintenance mode. |
| 32 | + |
| 33 | +- Certain changes to infrastructure profiles, such as Kubernetes version upgrades, require nodes to be recreated, |
| 34 | + removing maintenance nodes in the process. |
| 35 | + |
| 36 | +## Activate Maintenance Mode |
| 37 | + |
| 38 | +<!-- prettier-ignore --> |
| 39 | +1. Log in to [Palette](https://console.spectrocloud.com). |
| 40 | + |
| 41 | +2. Navigate to the left **Main Menu** and select **Clusters**. |
| 42 | + |
| 43 | +3. Select the desired cluster and navigate to the **Nodes** tab of the cluster. |
| 44 | + |
| 45 | +4. Beside the node that needs maintenance, select the **three-dot Menu** and **Turn on maintenance mode**. |
| 46 | + |
| 47 | +5. When maintenance mode is activated, the **Health** icon changes to a set of tools, and the tooltip states |
| 48 | + **Maintenance Mode: Initiated**. When Maintenance Mode is finished, the tooltip changes to **Maintenance Mode: |
| 49 | + Complete**. |
| 50 | + |
| 51 | +Palette reminds you in several locations that you have a node in maintenance mode: |
| 52 | + |
| 53 | +- Beside the **Settings** drop-down while viewing your cluster. |
| 54 | + |
| 55 | +- On the cluster’s **Overview** tab beneath **Health** status. |
| 56 | + |
| 57 | +- On the cluster’s **Nodes** tab in the node’s **Health** column. |
| 58 | + |
| 59 | + |
| 60 | + |
| 61 | +### Validate |
| 62 | + |
| 63 | +1. Log in to [Palette](https://console.spectrocloud.com). |
| 64 | + |
| 65 | +2. Navigate to the left **Main Menu** and select **Clusters**. |
| 66 | + |
| 67 | +3. Select the cluster with maintenance mode active and download the [kubeconfig](./palette-webctl.md) file. |
| 68 | + |
| 69 | + |
| 70 | + |
| 71 | +4. Open a terminal window and set the environment variable `KUBECONFIG` to point to the kubeconfig file you downloaded. |
| 72 | + |
| 73 | + ```bash |
| 74 | + export KUBECONFIG=~/Downloads/admin.aws-maintenance-test.kubeconfig |
| 75 | + ``` |
| 76 | + |
| 77 | +5. Confirm that the node is in a maintenance state, indicated by a `STATUS` of `SchedulingDisabled`. |
| 78 | + |
| 79 | + ```bash |
| 80 | + kubectl get nodes |
| 81 | + ``` |
| 82 | + |
| 83 | + ```bash hideClipboard {4} |
| 84 | + NAME STATUS ROLES AGE VERSION |
| 85 | + ip-10-0-1-174.ec2.internal Ready control-plane 177m v1.30.6 |
| 86 | + ip-10-0-1-26.ec2.internal Ready <none> 174m v1.30.6 |
| 87 | + ip-10-0-1-235.ec2.internal Ready,SchedulingDisabled <none> 174m v1.30.6 |
| 88 | + ``` |
| 89 | + |
| 90 | +## Disable Maintenance Mode |
| 91 | + |
| 92 | +<!-- prettier-ignore --> |
| 93 | +1. Log in to [Palette](https://console.spectrocloud.com). |
| 94 | + |
| 95 | +2. Navigate to the left **Main Menu** and select **Clusters**. |
| 96 | + |
| 97 | +3. Select the desired cluster and navigate to the **Nodes** tab of the cluster. |
| 98 | + |
| 99 | +4. Select the **three-dot Menu** beside the maintenance node and **Turn off maintenance mode**. |
| 100 | + |
| 101 | +5. When maintenance mode is disabled, the **Health** icon reverts to a checkmark. |
| 102 | + |
| 103 | +:::warning |
| 104 | + |
| 105 | +Taking a node out of maintenance mode does not automatically rebalance workloads. |
| 106 | + |
| 107 | +::: |
| 108 | + |
| 109 | +### Validate |
| 110 | + |
| 111 | +1. Log in to [Palette](https://console.spectrocloud.com). |
| 112 | + |
| 113 | +2. Navigate to the left **Main Menu** and select **Clusters**. |
| 114 | + |
| 115 | +3. Select the desired cluster and download the [kubeconfig](./palette-webctl.md) file. |
| 116 | + |
| 117 | + |
| 118 | + |
| 119 | +4. Open a terminal window and set the environment variable `KUBECONFIG` to point to the kubeconfig file you downloaded. |
| 120 | + |
| 121 | + ```bash |
| 122 | + export KUBECONFIG=~/Downloads/admin.aws-maintenance-test.kubeconfig |
| 123 | + ``` |
| 124 | + |
| 125 | +5. Confirm that scheduling is no longer disabled for the node, indicated by a `STATUS` of `Ready`. |
| 126 | + |
| 127 | + ```bash |
| 128 | + kubectl get nodes |
| 129 | + ``` |
| 130 | + |
| 131 | + ```bash hideClipboard |
| 132 | + NAME STATUS ROLES AGE VERSION |
| 133 | + ip-10-0-1-174.ec2.internal Ready control-plane 177m v1.30.6 |
| 134 | + ip-10-0-1-26.ec2.internal Ready <none> 174m v1.30.6 |
| 135 | + ip-10-0-1-235.ec2.internal Ready <none> 174m v1.30.6 |
| 136 | + ``` |
0 commit comments