Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore DDL based on regex. #895

Open
subkanthi opened this issue Nov 2, 2024 · 2 comments
Open

Ignore DDL based on regex. #895

subkanthi opened this issue Nov 2, 2024 · 2 comments
Labels
dev-complete Development completed lightweight Issues related to Lightweight version p1
Milestone

Comments

@subkanthi
Copy link
Collaborator

Add functionality so that the DDL regex defined in a configuration variable will be ignored and not executed.

@subkanthi subkanthi added the lightweight Issues related to Lightweight version label Nov 2, 2024
@subkanthi subkanthi added this to the 2.4.1 milestone Nov 2, 2024
@subkanthi subkanthi added the dev-complete Development completed label Nov 4, 2024
@subkanthi
Copy link
Collaborator Author

Add to config.yml

#ignore.ddl.regex: If set, the connector will ignore DDL events that match the regex.
ignore.ddl.regex: "(?i)(ANALYZE PARTITION).*"

Tested:
MySQL:

CREATE TABLE sales (     id INT NOT NULL,     sale_date DATE NOT NULL,     amount DECIMAL(10, 2),     PRIMARY KEY (id, sale_date) ) PARTITION BY RANGE (YEAR(sale_date)) (     PARTITION p2020 VALUES LESS THAN (2021),     PARTITION p2021 VALUES LESS THAN (2022),     PARTITION p2022 VALUES LESS THAN (2023),     PARTITION pfuture VALUES LESS THAN MAXVALUE );

alter table sales analyze partition p2022;
clickhouse-sink-connector-lt_1  | 2024-11-05 02:02:32.685 INFO  - Ignoring DDL: alter table sales analyze partition p2022 as it matches the regex: (?i)(ANALYZE PARTITION).*
clickhouse-sink-connector-lt_1  | 2024-11-05 02:02:32.685 INFO  - Ignored Source DB DDL: alter table sales analyze partition p2022 Snapshot:false

@subkanthi
Copy link
Collaborator Author

subkanthi commented Nov 14, 2024

Added fix in #916, if the ignore.ddl.regex is not defined, sink connector should not throw an error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev-complete Development completed lightweight Issues related to Lightweight version p1
Projects
None yet
Development

No branches or pull requests

1 participant