Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce the overhead of DXF task executor manager on idle state #59344

Closed
D3Hunter opened this issue Feb 8, 2025 · 0 comments · Fixed by #59345
Closed

reduce the overhead of DXF task executor manager on idle state #59344

D3Hunter opened this issue Feb 8, 2025 · 0 comments · Fixed by #59345
Labels
component/ddl This issue is related to DDL of TiDB. component/disttask distributed execute framework of tidb type/enhancement The issue or PR belongs to an enhancement.

Comments

@D3Hunter
Copy link
Contributor

D3Hunter commented Feb 8, 2025

Enhancement

task executor manager is run on all tidb node, and it will run a complex join every 300ms. This might make tidb takes some mount of CPU even on idle state, and produce some slow query when the cluster has a large number of nodes and heavy workload.

we pick 300ms to make DXF more responsive to make small tasks runs faster as we don't have a mechanism(except PD) to send events across tidb now, and tidb doesn't support prepared stmt for internal statement, so the SQL is compiled every time.

maybe we can change the join to two separate single table query to make it cost less.

@D3Hunter D3Hunter added component/ddl This issue is related to DDL of TiDB. type/enhancement The issue or PR belongs to an enhancement. component/disttask distributed execute framework of tidb labels Feb 8, 2025
@ti-chi-bot ti-chi-bot bot closed this as completed in fbdaef5 Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/ddl This issue is related to DDL of TiDB. component/disttask distributed execute framework of tidb type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant