Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't change partition spec for BigQuery tables #590

Open
lewish opened this issue Feb 3, 2020 · 0 comments
Open

Can't change partition spec for BigQuery tables #590

lewish opened this issue Feb 3, 2020 · 0 comments

Comments

@lewish
Copy link
Collaborator

lewish commented Feb 3, 2020

bigquery error: Cannot replace a table with a different partitioning spec. Instead, DROP the table, and then recreate it.

Current workaround is to manually delete the table from within BigQuery.

Options:

  • Always drop and recreate tables
  • Drop and recreate tables only when full-refresh is defined
  • Drop and recreate tables when the partition spec (or cluster by spec when added) exists on the table definition, or on the existing table in the warehouse.

This will significantly increase the number of statements we need to execute for BigQuery, so I propose we do option 3. It should be possible to get the current partition by spec when we compute warehouse state, and use this in the SQL generation logic.

@lewish lewish added the P2 label Feb 3, 2020
@BenBirt BenBirt added P3 and removed P2 labels Jul 6, 2020
@Ekrekr Ekrekr added UX Papercut and removed P3 labels Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants