scan.incremental.snapshot.enabled和snapshot.select.statement.overrides为什么不能同时启用

[lykpass]

@wuchong @leonardBang scan.incremental.snapshot.enabled为true时,通过snapshot.select.statement.overrides传入的statement不生效，为false时生效。两个功能能同时支持吗？开启增量快照，能加快快照速度这个功能超级好用，如果能同时支持snapshot.select.statement.overrides传个过滤sql，只快照自己需要的数据范围就更好了，那样就能进一步较少快照的时间，和快照的数据量了，减少了数据库的压力。

[dz902]

你好，这个部分的代码是在哪里呢，我设置了 overrides 但是没生效，代码里面也没看到是在哪里设置的 @lykpass

[yuxiqian]

Hi @lykpass and @dz902, CDC Incremental snapshot framework does not depend on Debezium, so snapshot.select.statement.overrides won't work since it's Debezium specific.

On the bright side, a similar feature has just been implemented by @uicosp in #3776, which brought a new option called scan.snapshot.filters, which could apply custom predicates during snapshot phase.

[dz902]

Hi @yuxiqian , thanks for the prompt reply! I'm using DataStream API and looking for a way to use snapshot overrides. Is it possible to disable incremental snapshot and use snapshot.select.statement.overrides when using DataStream API? I have traced the code to

flink-cdc/flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/debezium/task/MySqlSnapshotSplitReadTask.java

Line 245 in c2e3012

final String selectSql =

but I could not locate the overrides.

Issues like #24 and #187 suggests that the passthrough parameter should work but I could not get it to work, nor can I find it in the source code so I'm unable to debug it myself.

[yuxiqian]

Hi @dz902,

I think it’s possible to call MySQL Builder in org.apache.flink.cdc.connectors.mysql (instead of newer version in *.source) to use legacy source where you can use debeziumProperties to override specific configs.