Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates from postgres (update in postgres -> kafka -> clickhouse) do NOT work at all #899

Open
salisbury-espinosa opened this issue Nov 4, 2024 · 0 comments · May be fixed by #900
Open
Labels
dev-complete Development completed
Milestone

Comments

@salisbury-espinosa
Copy link
Contributor

https://github.com/Altinity/clickhouse-sink-connector/blob/develop/sink-connector/src/main/java/com/altinity/clickhouse/sink/connector/db/batch/PreparedStatementExecutor.java#L334

ps.setLong(columnNameToIndexMap.get(versionColumn),  record.getSequenceNumber());

this always sets the value to -1

so the version in clickhouse is always the same ( 18446744073709551615 ):

ch-1S_1K :) select _version from test limit 10;

SELECT _version
FROM test
LIMIT 10

Query id: 477120a0-fc78-43bc-b713-3a85f0ba291b

    ┌─────────────_version─┐
 1. │ 184467440737095516152. │ 184467440737095516153. │ 184467440737095516154. │ 184467440737095516155. │ 184467440737095516156. │ 184467440737095516157. │ 184467440737095516158. │ 184467440737095516159. │ 1844674407370955161510. │ 18446744073709551615 │
    └──────────────────────┘

10 rows in set. Elapsed: 0.002 sec. 

==>

ch-1S_1K :) select CAST(-1, 'UInt64');

SELECT CAST(-1, 'UInt64')

Query id: bb3c5db5-c027-4110-9cd7-5d740f57cfd2

   ┌───CAST(-1, 'UInt64')─┐
1. │ 18446744073709551615-- 18.45 quintillion
   └──────────────────────┘

1 row in set. Elapsed: 0.001 sec.

so need use LSN

salisbury-espinosa added a commit to salisbury-espinosa/clickhouse-sink-connector that referenced this issue Nov 4, 2024
@subkanthi subkanthi added this to the 2.5.0 milestone Nov 4, 2024
@subkanthi subkanthi added the dev-complete Development completed label Nov 4, 2024
@subkanthi subkanthi modified the milestones: 2.5.0, 2.6.0 Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev-complete Development completed
Projects
None yet
2 participants