Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ticdc: pvc_cleaner doesn't reclaim PVC after ticdc scaled in #4943

Open
mahjonp opened this issue Mar 28, 2023 · 0 comments · May be fixed by #4944
Open

ticdc: pvc_cleaner doesn't reclaim PVC after ticdc scaled in #4943

mahjonp opened this issue Mar 28, 2023 · 0 comments · May be fixed by #4944

Comments

@mahjonp
Copy link
Contributor

mahjonp commented Mar 28, 2023 •
edited
Loading

Bug Report

tidb-operator/pkg/manager/member/pvc_cleaner.go

Lines 71 to 103 in b004b9a

func (c *realPVCCleaner) reclaimPV(meta metav1.Object) (map[string]string, error) {
var clusterType string
switch meta := meta.(type) {
case *v1alpha1.TidbCluster:
if !meta.IsPVReclaimEnabled() {
return nil, nil
}
clusterType = "tidbcluster"
case *v1alpha1.DMCluster:
if !meta.IsPVReclaimEnabled() {
return nil, nil
}
clusterType = "dmcluster"
}
ns := meta.GetNamespace()
metaName := meta.GetName()
skipReason := map[string]string{}
pvcs, err := c.listAllPVCs(meta)
if err != nil {
return skipReason, err
}
runtimeMeta := meta.(runtime.Object)
for _, pvc := range pvcs {
pvcName := pvc.GetName()
l := label.Label(pvc.Labels)
if !(l.IsPD() || l.IsTiKV() || l.IsTiFlash() || l.IsDMMaster() || l.IsDMWorker()) {
skipReason[pvcName] = skipReasonPVCCleanerIsNotTarget
continue
}

What did you do?

Create a tidb-cluster with 3 ticdc replicas, then scale in 1 replicas

What did you expect to see?

Only 1 ticdc data pvc left.

In our k8s environment, the storageclass we use is extremely valuable. Each node has only 2-3 PVs created, yet we run hundreds of TiDB clusters for testing purposes. Therefore, it is imperative that scaled-in PVs are reclaimed as soon as possible.

What did you see instead?

ticdc-data-tc-ticdc-0      Bound    pvc-8d6d669a-c631-40da-9c14-dd108825b331   500Gi      RWO            local-path     11m
ticdc-data-tc-ticdc-1      Bound    pvc-53e1eeee-bab5-4d48-a3a9-ab91fb4c4111   500Gi      RWO            local-path     11m
ticdc-data-tc-ticdc-2      Bound    pvc-e61a2b51-7b71-421b-820a-81ae2d2e20fb   500Gi      RWO            local-path     11m
@mahjonp mahjonp linked a pull request Mar 28, 2023 that will close this issue
Open
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pvc_cleaner: reclaim ticdc pvc mahjonp/tidb-operator
1 participant
@mahjonp