Skip to content

Conversation

@raminqaf
Copy link
Contributor

@raminqaf raminqaf commented Nov 5, 2025

What is the purpose of the change

This pull request implements the CREATE OR ALTER MATERIALIZED TABLE syntax as proposed in FLIP-546. This new command provides an idempotent way to manage materialized tables, enabling declarative deployment patterns for CI/CD pipelines and infrastructure-as-code workflows.

The command intelligently routes to either CREATE or ALTER logic:

  • If the materialized table does not exist, it creates a new table (behaves like CREATE MATERIALIZED TABLE)
  • If the materialized table already exists, it modifies the query definition (behaves like ALTER MATERIALIZED TABLE AS)

This eliminates the need for complex DROP-IF-EXISTS patterns and makes materialized table management more robust and predictable in automated deployment scenarios.

Brief change log

  • Added CREATE OR ALTER MATERIALIZED TABLE SQL syntax to the Flink SQL parser
  • Extended SqlCreateOrAlterMaterializedTable to handle both create and alter operations based on table existence
  • Refactored materialized table parsing and building logic into SqlCreateOrAlterMaterializedTableConverter for better encapsulation
  • Added getOriginalQuery() method to CatalogMaterializedTable interface (aligning with CatalogView interface)
  • Added comprehensive tests for CREATE OR ALTER behavior in both create and alter scenarios
  • Updated documentation to include CREATE OR ALTER MATERIALIZED TABLE syntax and examples

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests in MaterializedTableStatementParserTest to verify SQL parsing of CREATE OR ALTER syntax
  • Added integration tests in SqlMaterializedTableNodeToOperationConverterTest:
    • testCreateOrAlterMaterializedTable() - verifies CREATE behavior when table doesn't exist
    • testCreateOrAlterMaterializedTableForExistingTable() - verifies ALTER behavior when table exists with schema evolution
  • Extended integration tests in SqlGatewayRestEndpointMaterializedTableITCase to validate end-to-end behavior via REST API
  • Verified that existing CREATE and ALTER tests continue to pass, ensuring backward compatibility
  • Manually verified idempotent behavior by running the same CREATE OR ALTER statement multiple times

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes - CatalogMaterializedTable interface extended with getOriginalQuery() method
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs - Added new section in docs/content/docs/dev/table/materialized-table/statements.md explaining CREATE OR ALTER syntax, behavior, examples, and use cases

@raminqaf raminqaf changed the title [FLINK-38355] Support CreateOrAlter MATERIALIZED TABLE [FLINK-38355][table][FLIP-546] Support CREATE OR ALTER MATERIALIZED TABLE Nov 5, 2025
@raminqaf raminqaf changed the title [FLINK-38355][table][FLIP-546] Support CREATE OR ALTER MATERIALIZED TABLE [FLINK-38355][table][FLIP-546] Support CREATE OR ALTER MATERIALIZED TABLE syntax Nov 5, 2025
@flinkbot
Copy link
Collaborator

flinkbot commented Nov 5, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@github-actions github-actions bot added the community-reviewed PR has been reviewed by the community. label Nov 6, 2025
@raminqaf raminqaf force-pushed the FLINK-38355 branch 6 times, most recently from ba9286e to 2cd5eae Compare November 13, 2025 10:53
return context.getCatalogManager().getTable(identifier).isPresent();
}

private CatalogMaterializedTable buildNewCatalogMaterializedTableFromOldTable(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally this method should be in alterMaterializedTable converter
may be we can move it in future when that converter becomes more mature

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better if we move it later with the corresponding PR.

@raminqaf raminqaf force-pushed the FLINK-38355 branch 2 times, most recently from c5fbef2 to 8b4d426 Compare November 14, 2025 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants