Skip to content

cdc/changefeeds/new metadata env: crdb_internal_table_id in source object #143371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
rohan-joshi opened this issue Mar 24, 2025 · 2 comments · May be fixed by #147341
Open

cdc/changefeeds/new metadata env: crdb_internal_table_id in source object #143371

rohan-joshi opened this issue Mar 24, 2025 · 2 comments · May be fixed by #147341
Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-cdc

Comments

@rohan-joshi
Copy link

rohan-joshi commented Mar 24, 2025 •
edited by rharding6373
Loading

Requirements:

  • Add the following field to the source object in the metadata env:
    -- crdb_internal_table_id: The table ID that CRDB assigns to this table internally. Downstream can be used to uniquely identify the table, even if table names change; for Replicator will be important for our KV store key

Jira issue: CRDB-48789
@rohan-joshi rohan-joshi added A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-cdc labels Mar 24, 2025
@blathers-crl blathers[crl]
Copy link

blathers-crl bot commented Mar 24, 2025

cc @cockroachdb/cdc

@rharding6373
Copy link
Collaborator

rharding6373 commented Mar 24, 2025 •
edited
Loading

Could we have more context on this issue? Range ID is non-deterministic (i.e., for a given row it can change due to range splits/merges) and it's generally not a good idea to expose those kinds of internal implementation details. We decided that we would not be including range ID and edited the original issue comment to reflect this.

@rharding6373 rharding6373 changed the title cdc/changefeeds/new metadata env: range_id + crdb_internal_table_id in source object cdc/changefeeds/new metadata env: crdb_internal_table_id in source object May 9, 2025
elizaMkraule added a commit to elizaMkraule/cockroach that referenced this issue May 27, 2025
@elizaMkraule
cdc: include crdb_internal_table_id in changefeed source metadata
4800aa3 
Previously, the metadata env of changefeed source objects did not include
the CockroachDB internal table ID. This field is necessary downstream to
uniquely identify tables even if their names change. This change adds the
crdb_internal_table_id field to the source object within the metadata env,
ensuring consistent identification of tables across renames and supporting
downstream integrations.

Fixes cockroachdb#143371
Epic: CRDB-48789

Release note (general change): Changefeed source metadata now includes the
new crdb_internal_table_id field, enabling downstream consumers to uniquely
identify tables even if table names change.
@elizaMkraule elizaMkraule linked a pull request May 27, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-cdc
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cdc: include crdb_internal_table_id in changefeed source metadata elizaMkraule/cockroach
3 participants
@asg0451 @rohan-joshi @rharding6373