Skip to content

[SPARK-52159][SQL] Properly handle table existence check for jdbc dialects #50835

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from

Conversation

milastdbx
Copy link
Contributor

@milastdbx milastdbx commented May 8, 2025

What changes were proposed in this pull request?

In this PR, I propose that we rethrow exception when we are doing table existence check in jdbc dialect, if exception is not related to table/schema not being found. I propose this because currently all exceptions get swallowed and method returns false. From the perspective of the system its as if table doesn't exist, which is a wrong message (e.g, we can get table does not exist even if it was network failure).

This issue is mostly exposed when TableCatalog API is used,

  override def loadTable(ident: Identifier): Table = {
    if (!tableExists(ident)) {
      throw QueryCompilationErrors.noSuchTableError(ident)
    }

Error code links:

How its implemented today, tableExist cannot throw anything, so every exception gets converted to noSuchTableError which is wrong.

Does this PR introduce any user-facing change?

Customers will get proper error messages.

How was this patch tested?

Tests

Was this patch authored or co-authored using generative AI tooling?

Generetad by: COPILOT

@github-actions github-actions bot added the SQL label May 8, 2025
@milastdbx milastdbx changed the title add handling for table not found Properly handle table existence check for jdbc dialects May 8, 2025
@milastdbx
Copy link
Contributor Author

I will create SPARK jira ticket shortly

@milastdbx milastdbx changed the title Properly handle table existence check for jdbc dialects [SPARK-52159] Properly handle table existence check for jdbc dialects May 15, 2025
Copy link
Member

@MaxGekk MaxGekk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@milastdbx Could you fix the test failure, please. I think it is related to your changes:

[info] JDBCTableCatalogSuite:
[info] - show tables (61 milliseconds)
[info] - drop a table and test whether the table exists *** FAILED *** (86 milliseconds)
[info]   "[SCHEMA]_NOT_FOUND" did not equal "[TABLE_OR_VIEW]_NOT_FOUND" (SparkFunSuite.scala:346)
[info]   Analysis:
[info]   "[SCHEMA]_NOT_FOUND" -> "[TABLE_OR_VIEW]_NOT_FOUND"

Also add the [SQL] tag to PR's title.

@milastdbx milastdbx changed the title [SPARK-52159] Properly handle table existence check for jdbc dialects [SPARK-52159] [SQL] Properly handle table existence check for jdbc dialects May 16, 2025
@milastdbx milastdbx changed the title [SPARK-52159] [SQL] Properly handle table existence check for jdbc dialects [SPARK-52159][SQL] Properly handle table existence check for jdbc dialects May 16, 2025
@milastdbx milastdbx requested a review from MaxGekk May 16, 2025 11:47
Copy link
Contributor

@alekjarmov alekjarmov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to add comments with the relevant error code links in the override function like here

@MaxGekk
Copy link
Member

MaxGekk commented May 19, 2025

Would be nice to add comments with the relevant error code links

It would be nice to have the links but they can become outdated, unfortunately, and this is out of our control.

+1, LGTM. Merging to master.
Thank you, @milastdbx and @alekjarmov for review.

@MaxGekk MaxGekk closed this in a08b3fd May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants