Skip to content

[fix](systable) fix infomation_schema.partitions table getVisiableVersion spends too much time in cloud mode #58515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[fix](systable) fix infomation_schema.partitions table getVisiableVersion spends too much time in cloud mode #58515

wants to merge 2 commits into from

Conversation

@koarz
Copy link
Contributor

@koarz koarz commented Nov 28, 2025 •
edited by gavinchou
Loading

No description provided.

@koarz
Copy link
Contributor Author

koarz commented Nov 28, 2025

run buildall

@koarz koarz changed the title [fix](systable)fix partitions table getVisiableVersion cost too many … [fix](systable)fix partitions table getVisiableVersion cost too many time Nov 28, 2025
@gavinchou gavinchou changed the title [fix](systable)fix partitions table getVisiableVersion cost too many time [fix](systable) fix infomation_schema.partitions table getVisiableVersion spends too much time in cloud mode Nov 28, 2025
gavinchou
gavinchou previously approved these changes Nov 28, 2025
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 28, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 28, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 28, 2025

PR approved by anyone and no changes requested.

@koarz
Copy link
Contributor Author

koarz commented Nov 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 28, 2025

TPC-H: Total hot run time: 34023 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a706a391884c6560ad150793eca28ee397e0812, data reload: false

------ Round 1 ----------------------------------
q1	17612	5048	4869	4869
q2	2050	317	204	204
q3	10257	1330	737	737
q4	10230	932	359	359
q5	7541	2483	2205	2205
q6	184	174	141	141
q7	950	795	638	638
q8	9346	1381	982	982
q9	7084	5349	5331	5331
q10	6814	2229	1855	1855
q11	503	310	281	281
q12	336	381	233	233
q13	17767	3621	3021	3021
q14	254	237	212	212
q15	565	516	508	508
q16	854	864	815	815
q17	573	804	431	431
q18	7608	7253	7123	7123
q19	1091	951	570	570
q20	347	343	220	220
q21	3594	3243	2327	2327
q22	1029	1002	961	961
Total cold run time: 106589 ms
Total hot run time: 34023 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4974	4903	4904	4903
q2	318	411	325	325
q3	2141	2615	2275	2275
q4	1324	1750	1300	1300
q5	4189	4321	4675	4321
q6	213	177	128	128
q7	2043	2031	1838	1838
q8	2624	2533	2487	2487
q9	7478	7543	7463	7463
q10	3031	3269	2823	2823
q11	590	525	505	505
q12	695	758	617	617
q13	3488	4005	3226	3226
q14	270	288	281	281
q15	568	498	516	498
q16	912	963	873	873
q17	1186	1436	1466	1436
q18	8041	7807	7511	7511
q19	843	828	801	801
q20	2056	2069	1949	1949
q21	4858	4635	4252	4252
q22	1062	1027	986	986
Total cold run time: 52904 ms
Total hot run time: 50798 ms

@doris-robot
Copy link

doris-robot commented Nov 28, 2025

TPC-DS: Total hot run time: 181510 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a706a391884c6560ad150793eca28ee397e0812, data reload: false

query1	1069	402	399	399
query2	6581	1190	1162	1162
query3	6831	246	225	225
query4	25232	23271	22792	22792
query5	5075	654	507	507
query6	348	240	240	240
query7	4654	524	302	302
query8	327	259	248	248
query9	8723	2583	2575	2575
query10	550	342	309	309
query11	15467	15168	14816	14816
query12	192	121	111	111
query13	1687	575	438	438
query14	10118	5974	5897	5897
query15	214	199	189	189
query16	7692	735	530	530
query17	1322	782	612	612
query18	2036	443	328	328
query19	207	199	172	172
query20	130	126	126	126
query21	215	132	114	114
query22	4104	4229	4038	4038
query23	33063	32010	31912	31912
query24	8311	2392	2384	2384
query25	582	523	445	445
query26	1233	283	165	165
query27	2727	505	339	339
query28	4324	2116	2113	2113
query29	811	615	501	501
query30	311	237	216	216
query31	830	703	611	611
query32	88	71	68	68
query33	596	376	333	333
query34	818	878	545	545
query35	809	840	751	751
query36	885	909	837	837
query37	124	112	86	86
query38	3897	3810	3736	3736
query39	1455	1397	1393	1393
query40	230	132	120	120
query41	65	63	62	62
query42	127	112	113	112
query43	437	435	409	409
query44	1294	761	771	761
query45	197	194	188	188
query46	878	999	654	654
query47	1721	1749	1682	1682
query48	398	423	330	330
query49	810	492	453	453
query50	651	679	403	403
query51	3984	3840	3904	3840
query52	114	114	111	111
query53	244	262	194	194
query54	296	299	280	280
query55	99	95	92	92
query56	333	317	308	308
query57	1165	1157	1089	1089
query58	299	291	288	288
query59	2336	2422	2330	2330
query60	355	354	331	331
query61	185	162	158	158
query62	804	736	661	661
query63	230	196	196	196
query64	4419	1183	884	884
query65	4011	3972	3965	3965
query66	1107	450	333	333
query67	15171	15237	14931	14931
query68	3986	972	634	634
query69	523	348	310	310
query70	1097	967	1020	967
query71	446	332	326	326
query72	5889	5053	5077	5053
query73	658	582	347	347
query74	8842	8792	8563	8563
query75	3034	3040	2521	2521
query76	3331	1139	733	733
query77	514	420	331	331
query78	9460	9630	8878	8878
query79	2279	871	583	583
query80	1702	669	491	491
query81	566	267	239	239
query82	409	137	112	112
query83	373	258	253	253
query84	260	111	96	96
query85	912	469	438	438
query86	473	289	285	285
query87	3989	4082	3901	3901
query88	2972	2302	2258	2258
query89	388	334	288	288
query90	1922	224	216	216
query91	183	167	146	146
query92	85	72	68	68
query93	1788	987	654	654
query94	774	454	350	350
query95	510	417	404	404
query96	511	548	281	281
query97	2602	2713	2580	2580
query98	246	220	212	212
query99	1325	1561	1262	1262
Total cold run time: 267647 ms
Total hot run time: 181510 ms

@doris-robot
Copy link

doris-robot commented Nov 28, 2025

ClickBench: Total hot run time: 27.18 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a706a391884c6560ad150793eca28ee397e0812, data reload: false

query1	0.06	0.05	0.05
query2	0.11	0.05	0.05
query3	0.25	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.25	0.25
query6	1.18	0.65	0.63
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.56	0.51	0.49
query10	0.56	0.56	0.56
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.63	0.60	0.61
query14	0.99	0.99	0.99
query15	0.82	0.80	0.80
query16	0.40	0.38	0.39
query17	1.00	1.02	1.01
query18	0.23	0.21	0.21
query19	1.96	1.85	1.89
query20	0.02	0.02	0.02
query21	15.44	0.29	0.13
query22	4.83	0.04	0.05
query23	15.86	0.25	0.10
query24	1.32	0.54	0.18
query25	0.12	0.05	0.08
query26	0.14	0.13	0.13
query27	0.06	0.04	0.05
query28	4.77	1.21	1.02
query29	12.60	3.90	3.20
query30	0.28	0.13	0.11
query31	2.82	0.61	0.40
query32	3.24	0.54	0.47
query33	2.98	2.98	3.03
query34	16.79	5.23	4.53
query35	4.58	4.59	4.59
query36	0.69	0.50	0.50
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.04	0.03
query42	0.05	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 98.12 s
Total hot run time: 27.18 s

gavinchou
gavinchou reviewed Nov 28, 2025
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/tablefunction/MetadataGenerator.java
.getBucketNum())); // BUCKET_NUM
trow.addToColumnValue(new TCell().setLongVal(partition.getCommittedVersion())); // COMMITTED_VERSION
trow.addToColumnValue(new TCell().setLongVal(partition.getVisibleVersion())); // VISIBLE_VERSION
trow.addToColumnValue(
Copy link
Contributor

@gavinchou gavinchou Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a session var or configuration to control the the behavior

@gavinchou gavinchou self-requested a review November 28, 2025 11:53
@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 28, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou Awaiting requested review from gavinchou

1 more reviewer

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. cloud dev/3.1.x dev/4.0.x reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@koarz @doris-robot @hello-stephen @gavinchou @Yukang-Lian