Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changefeeds are not distributed evenly over TiCDC nodes #11331

Open
King-Dylan opened this issue Jun 20, 2024 · 4 comments
Open

Changefeeds are not distributed evenly over TiCDC nodes #11331

King-Dylan opened this issue Jun 20, 2024 · 4 comments
Labels
area/ticdc Issues or PRs related to TiCDC. type/bug This is a bug.

Comments

@King-Dylan
Copy link

King-Dylan commented Jun 20, 2024

What did you do?

Creating multiple changefeeds where each one contains only one table results in all table tasks being sent to a single capture. This is unreasonable. I think before we bind a table to a capture, we should add more judgment conditions, such as considering the current tasks already bound to it. Then, we can choose the best capture to bind the table to, which will be better for this scenario.
https://github.com/pingcap/tiflow/blob/master/cdc/scheduler/internal/v3/scheduler/scheduler_basic.go#L81-L82

What did you expect to see?

The tasks evenly distirbuted on different ticdc nodes.

What did you see instead?

All of them are bound on specific node.

Versions of the cluster

TiCDC version (execute cdc version):

v7.1.1
@King-Dylan King-Dylan added area/ticdc Issues or PRs related to TiCDC. type/bug This is a bug. labels Jun 20, 2024
@github-actions github-actions bot added this to Need Triage in Question and Bug Reports Jun 20, 2024
@King-Dylan
Copy link
Author

#11331 seems just fixed the uneven distribute scenario for one changefeed muti-tables. This scenario is more like multiple changefeeds with one table task.

@fubinzh
Copy link

fubinzh commented Jun 21, 2024

#11331 seems just fixed the uneven distribute scenario for one changefeed muti-tables. This scenario is more like multiple changefeeds with one table task.

@King-Dylan The link in the comment is wrong?

@choudharypankaj
Copy link

#9847
This is the previous issue that @King-Dylan was referring to.

@fubinzh
Copy link

fubinzh commented Jun 28, 2024

This looks like an enhancement? Please @flowbehappy take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ticdc Issues or PRs related to TiCDC. type/bug This is a bug.
Projects
Development

No branches or pull requests

3 participants