You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug #1883 changed the grace-period default from 1 to 0. This is a dangerous & unrecommended operation, and we've seen enterprise operations using k9s to trigger force delete and caused data corruption/lose for stateful workloads.
Force delete has very different implication between grace-period=0 vs. >0, with 0 it will basically skip validating that the pod is successfully SIGKILLed before removing the Pod object from the API server. This very dangerous for stateful workload, where in case of split-brain (e.g. kubelet down) a new Pod can get created (e.g. by statefulset controller) with the same "identity".
Expected behavior
Default grace-period to 1.
Bonus: provide a more hidden way to trigger grace-period=0, with enough warning to let user aware of the danger & underlying implication.
The text was updated successfully, but these errors were encountered:
Describe the bug
#1883 changed the grace-period default from 1 to 0. This is a dangerous & unrecommended operation, and we've seen enterprise operations using k9s to trigger force delete and caused data corruption/lose for stateful workloads.
Force delete has very different implication between grace-period=0 vs. >0, with 0 it will basically skip validating that the pod is successfully SIGKILLed before removing the Pod object from the API server. This very dangerous for stateful workload, where in case of split-brain (e.g. kubelet down) a new Pod can get created (e.g. by statefulset controller) with the same "identity".
kubectl also defaults to non-zero and give very clear warning if user want to explicitly set to 0: https://github.com/kubernetes/kubectl/blob/master/pkg/cmd/delete/delete.go#L62C3-L67C71
Some related articles why force delete could be problematic: https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/#statefulset-considerations
Expected behavior
Default grace-period to 1.
Bonus: provide a more hidden way to trigger grace-period=0, with enough warning to let user aware of the danger & underlying implication.
The text was updated successfully, but these errors were encountered: