operator tries to shrink volume after storage size update if value is in Mi #5651
Comments
|
Can you show the size of your PVCs (both TiKV and PD)? |
|
The problem happens on create too, not just on update. I created a new TidbCluster and here's all the manifests and logs: I created TidbCluster CR with the same storage sizes as before (15675Mi and 1740Mi). Here's the CR copied from Lens after creation: TiKV pvc: PD pvc: operator logs: You can ignore logs for other CRs. The TidbInitializer only creates an "app" database and does nothing else. As you can see, operator creates the cluster with correct pvc sizes but then tries to shrink from a higher size to the original correct size. |
|
from PVC status:
phase: Bound
accessModes:
- ReadWriteOnce
capacity:
storage: 16Gi
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 15675MiIt seems the CSI driver created a 16Gi volume instead of a 15675Mi one. |
Interesting. Maybe if operator looks at pvc spec.resources.requests.storage which is 15675Mi instead of pvc status.capacity.storage which is 16Gi, the problem is fixed. I'm not an expert but I think spec is a more solid thing to look at than status |
When reszing a PVC, operator need to check the status to ensure whether the expanding operation is finished. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
What version of Kubernetes are you using?
Client Version: v1.28.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.6
What version of TiDB Operator are you using?
TiDB Operator Version: version.Info{GitVersion:"v1.5.3", GitCommit:"2c9e4dad0abaa4400afdef9ceff3084e71510ecb", GitTreeState:"clean", BuildDate:"2024-04-18T03:46:15Z", GoVersion:"go1.21.6", Compiler:"gc", Platform:"linux/amd64"}
What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?
cephfs.csi.ceph.com
What's the status of the TiDB cluster pods?
Running
What did you do?
I tried to update storage size in TidbCluster for PD and TiKV to 1740Mi and 15675Mi. The operator updated the volumes correctly but then in operator logs, I get these errors:
I set the disk size in Mi but after the volumes are updated, the operator seems to round up and use Gi and then complain about not being able to shrink the disk size from Gi value to Mi value that I've set.
What did you expect to see?
I expected the operator NOT to try to shrink volumes after it updated the volumes.
What did you see instead?
I saw the operator incorrectly assuming the storage size is 16Gi instead of 15675Mi and try to shrink the volumes.
The text was updated successfully, but these errors were encountered: