Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28292: Optimize SHOW TABLES|VIEWS statements #5273

Merged
merged 7 commits into from
Sep 16, 2024

Conversation

wecharyu
Copy link
Contributor

@wecharyu wecharyu commented May 31, 2024

What changes were proposed in this pull request?

Make some improvements in SHOW TABLES statement:

  1. Push down table pattern to metastore
  2. Request table names instead of table objects
  3. Change index UNIQUETABLE(TBL_NAME, DB_ID) to UNIQUETABLE (DB_ID, TBL_NAME) and drop index TBLS_N49(DB_ID)

Why are the changes needed?

Improve the performance of SHOW TABLES|VIEWS statements.

Does this PR introduce any user-facing change?

No.

Is the change a dependency upgrade?

No.

How was this patch tested?

Passing existing tests and do benchmark tests.

 java -jar hmsbench.jar -H localhost --savedata /tmp/benchdata --sanitize -N 100 -N 10000 -o bench_results.csv -C -d testbench_http --params=100 -M 'listTables.*'
  • Before this patch
Operation       Mean    Med     Min     Max     Err%
listTables      1.52074 1.50483 1.39134 1.90837 4.07209
listTables.100  1.68746 1.68214 1.57137 1.87610 2.93073
listTables.10000        16.9628 16.9202 16.4026 19.1795 1.77736
  • After this patch
Operation       Mean    Med     Min     Max     Err%
listTables      1.50856 1.49130 1.39420 1.76155 4.03957
listTables.100  1.66239 1.65599 1.52580 1.96438 3.59206
listTables.10000        10.5922 10.2885 9.74368 15.3253 9.76028

tables = client.getTables("d*", "*");
Assert.assertEquals("All tables size", 7, tables.size());
tables = client.getTables("def*", "*");
Assert.assertEquals("All tables size", 5, tables.size());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these tests, discussed in #4326 (comment).

@wecharyu
Copy link
Contributor Author

wecharyu commented Jun 4, 2024

@dengzhhu653 @deniskuzZ @zhangbutao Could you please help review this patch?

Copy link

sonarcloud bot commented Jun 29, 2024

Copy link
Member

@deniskuzZ deniskuzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, sorry for a few month delay. 1 suggestion on regex simplification

Copy link
Member

@deniskuzZ deniskuzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1, pending tests

Copy link

sonarcloud bot commented Sep 15, 2024

@deniskuzZ deniskuzZ merged commit 6b361ac into apache:master Sep 16, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants