Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for renaming fields #293

Open
Fryuni opened this issue Aug 22, 2020 · 3 comments
Open

Support for renaming fields #293

Fryuni opened this issue Aug 22, 2020 · 3 comments

Comments

@Fryuni
Copy link

Fryuni commented Aug 22, 2020

Avro schema evolution supports field renaming by setting the old name as an alias of the new name on the new schema.

If a field is missing on BigQuery is missing from the schema but is an alias to some other field, currently, this causes an impossible update, but there is no need to update the table schema since this is a backward-compatible schema evolution for Avro.

@saumyasuhagiya
Copy link

@Fryuni
Copy link
Author

Fryuni commented May 4, 2022

The problem is not the data, we do use a transformer to make the data itself compatible with the previous version (a custom one).

The connector reads directly from the registry the latest version and tries to update the schema with the new column, this happens before attempting to write the data.

@Fryuni
Copy link
Author

Fryuni commented May 4, 2022

I reopened this issue on the new repo at confluentinc/kafka-connect-bigquery#203

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants