MatrixOne-v2.0.2

Release date: January 25, 2025
MatrixOne version: v2.0.2

MatrixOne 2.0.2 introduces a series of improvements and bug fixes that enhance system performance, stability, and usability. Below are the major updates and new features.

Key Improvements

1. Point query performance optimization
2. Replay implementation optimization to reduce TN restart time
3. Support querying statistical information of subscribed tables
4. ETLMerge optimization
5. Optimize logtail reconnection to minimize impact on CN operations and reduce CN downtime
6. Implement json_set

Bugfix

1. Fix bugs related to full-text indexing
2. Resolve the issue of CN hanging during upgrades
3. Fix some operator-related bugs
4. Fix bugs related to high-concurrency DDL
5. Fix bugs in window functions

What's Changed

• cp to 2.0 "[Tech Request]: optimize filters on composite columns (#20695)" by @badboynt1 in #20703
• fix a bug for case when expression by @badboynt1 in #20710
• support json_set/json_insert/json_replace(#2.0-dev) by @YANGGMM in #20592
• Optimize analyze code 2.0 dev by @qingxinhome in #20638
• cp to 2.0 "optimize stats for runtime filters when table not flushed" by @badboynt1 in #20719
• Lock mo_databases for DDL/DML(2.0-dev) by @ouyuanning in #20617
• fileservice: more disk cache events logging by @reusee in #20706
• to 2.0: fix show accounts double counting size. by @gouhongshen in #20714
• fix bad write format error - 2.0 by @aptend in #20716
• [improvement] logtail: do not abort txns when reconnect. by @volgariver6 in #20732
• rm tag 20544(2.0-dev) by @Ariznawlll in #20709
• Blob supports maximum storage64 m data 2.0 dev by @qingxinhome in #20720
• [improvement] logservice: add some logs by @volgariver6 in #20704
• [improvement] stats: add some logs and UT by @volgariver6 in #20620
• fix stats when merge filters on composite key by @badboynt1 in #20749
• to 2.0: apply reader pk filter on workspace entry. by @gouhongshen in #20750
• fix tests by @zhangxu19830126 in #20746
• fix task concurrency control by @w-zr in #20757
• optimize operator of fill by @iamlinjunhong in #20723
• Refactor fault injection by @Wenbin1002 in #20523
• to 2.0: optimization of the reader Equal filter. by @gouhongshen in #20533
• Print index name when doing explain or explain analyze by @qingxinhome in #20741
• improve lock meta table for DML (#20763) by @ouyuanning in #20774
• fix get role name of mo admin role in non-sys account by @ck89119 in #20776
• optimize workspace dump memory usage by @Wenbin1002 in #20779
• Fix panic caused by resolve index table in subscription by @ck89119 in #20793
• remove some logic from multi_update (#20788) by @ouyuanning in #20795
• refine string by @daviszhen in #20798
• Cdc task auto detect db/tbl by @ck89119 in #20701
• align merge with main by @w-zr in #20488
• fileservice: drain response reader to ensure connection reuse by @reusee in #20792
• fix ReWriteCheckpointAndBlockFromKey by @jiangxinmeng1 in #20807
• [improvement] logtail: some refactor and add UTs by @volgariver6 in #20756
• refine bytes converting by @daviszhen in #20805
• add date to bytes by @daviszhen in #20814
• [improvement] clusterservice: at least one tn in the service. by @volgariver6 in #20811
• to 2.0: support subscription in mo table size/rows. by @gouhongshen in #20784
• fix a bug that cause panic on loop join by @badboynt1 in #20826
• fix merge start log by @w-zr in #20832
• cherrypick from 2.0.1hotfix to 2.0dev: use fixed function id by @daviszhen in #20830
• [improvement] proxy: filter duplicate set stmts. by @volgariver6 in #20829
• fix w-w conflict for add/drop FK (#20804) by @ouyuanning in #20836
• optimize workspace dump 2.0 by @Wenbin1002 in #20772
• composite date bytes by @daviszhen in #20823
• reset create node and delete node in replay by @jiangxinmeng1 in #20824
• add pitr checking by @ck89119 in #20838
• fix panic on 2.0-dev by @w-zr in #20841
• revert maximum value of max_allowed_packet to 1G by @ck89119 in #20851
• Locate sql parser error new 2.0 dev by @qingxinhome in #20758
• Revert check pitr in cdc by @ck89119 in #20854
• optimize replay by @jiangxinmeng1 in #20860
• fix 2.0-dev merge oom by @w-zr in #20861
• remove unused fields in s3writer(2.0-dev) by @ouyuanning in #20862
• TxnExecuter context missing roleId when executes the DDL statement by @qingxinhome in #20858
• Dynamic workspace threshold by @Wenbin1002 in #20848
• enable multiple merges on one table at once by @w-zr in #20847
• change code owner by @ouyuanning in #20875
• delete bvt tage #19912 by @aressu1985 in #20820
• 2.0 dev mo set json by @Songxx-7 in #20872
• Fix the bug that the file cannot be found when restoring using incremental backup-2.0 by @LeftHandCold in #20878
• [improvement] logservice: add truncate log by @volgariver6 in #20882
• refine convertslice by @daviszhen in #20863
• Fix data race for ParallelJobScheduler 2.0 by @LeftHandCold in #20885
• fail expired task by @w-zr in #20891
• fix big txn dump by @Wenbin1002 in #20900
• abort lock table when tn restart by @iamlinjunhong in #20914
• fix getChangedTableList panic caused by null requests. by @gouhongshen in #20906
• fix opt bug: not ResolveAlias for sk (#20903) by @ouyuanning in #20918
• make sure the default lock timeout is large enough to reduce sensitivity to 2.0 by @zhangxu19830126 in #20920
• Fix get snapshot panic for GC by @LeftHandCold in #20924
• Add stats io time consumed for stats 2.0 dev by @qingxinhome in #20800
• fix panic in on duplicate key update by @ouyuanning in #20884
• Remove GC's panic(), use Warn log instead: GC-PANIC by @LeftHandCold in #20910
• fileservice: more logs for memory cache updating by @reusee in #20889
• Update schema upgrade version to V2.0.2-2.0-dev by @qingxinhome in #20905
• [bug] fix ut TestSpeedupAbortAllTxn by @volgariver6 in #20928
• fix remote lock EOF and OOM when lock table bind by @iamlinjunhong in #20939
• [bug] logtail: remove "set ready" in init method by @volgariver6 in #20934
• fix a bug in apply index by @badboynt1 in #20946
• Revert "Cdc task auto detect db/tbl (#20701)" by @ck89119 in #20952
• Fix bug: eng is nil in multiupdate by @ouyuanning in #20961
• [bug] datasync: fix ut TestCompleteData by @volgariver6 in #20960
• fix keep tn version in lock service by @iamlinjunhong in #20963
• [Tech Request]: optimize non-pk index query for sysbench by @badboynt1 in #20975
• reorder catalog cache to mitigate the searching cost 2.0 by @aptend in #20955
• add session system var delete_opt_to_truncate by @huby2358 in #20950
• Revert "abort lock table when tn restart (#20914)" by @iamlinjunhong in #20980
• Add logs for upgrade framework 2.0 dev by @qingxinhome in #20977
• chore: improve mo_logger write big sql in 2.0-dev by @xzxiong in #20969
• [improvement] change the default timeout of allocate in incr. by @volgariver6 in #20988
• enable retry when first lock table by @iamlinjunhong in #20991
• use atomic variables by @daviszhen in #20995
• [improvement] modify the default interval of truncate hakeeper logs. by @volgariver6 in #20983
• remove getRelationById in some case by @ouyuanning in #20984
• optimize stats when table not flushed yet by @badboynt1 in #21008
• update replay by @jiangxinmeng1 in #21001
• optimize stats when table not flushed yet step II by @badboynt1 in #21016
• add debug log to lock service heartbeat to 2.0 by @zhangxu19830126 in #21019
• add bvt tag for issue #21024 by @aressu1985 in #21029
• [bug] logservice: fix context timeout for heartbeat by @volgariver6 in #21027
• fix rollback database ddl 2.0 by @aptend in #21035
• ignore base test cases by @sukki37 in #21054
• feat: mo gc tool by @Wenbin1002 in #21004
• stop cn merge by @w-zr in #21041
• add columns to mo_pubs/subs by @ck89119 in #21036
• fix one err handling by @XuPeng-SH in #21046
• fix dn crash 2.0 by @aptend in #21026
• fix deadlock by @daviszhen in #20970
• fix isValidRemoteTxn by @iamlinjunhong in #21076
• Update policyOverlap.go by @w-zr in #21082
• fix bug: prefix_xx get incorrect result by @ouyuanning in #21075
• Fix skip entry in replay by @jiangxinmeng1 in #21091
• Fix bug: incorrect pk filter for string column (#21085) by @ouyuanning in #21096
• [bug] proxy: migrate session skip error when db is dropped by @volgariver6 in #21098
• disable logs by @daviszhen in #21089
• check col existence of mo_pubs/subs by @ck89119 in #21095
• fix replay by @jiangxinmeng1 in #21108
• skip the TS check when getChangedTableList. by @gouhongshen in #21084
• fix incomplete cache 2.0 by @aptend in #21068
• fix need retry in engine 2.0 by @aptend in #21053
• fix merge by @XuPeng-SH in #21119
• fix bug: recall NewExpressionExecutor in prepare by @ouyuanning in #21120
• fix replay by @jiangxinmeng1 in #21128
• fix createdInTxn impl 2.0 by @aptend in #21137
• [#21049] Fix-bug-cherry-pick by @triump2020 in #21117
• W zr patch 1 by @w-zr in #21142
• fix bvt by @ck89119 in #21147
• [Cherry-pick] use full db.table path in insert stmt by @ck89119 in #21152
• optimize readers when there is false filter by @badboynt1 in #21158
• chore: add account filter-condition to speedup etlmerge check-exist by @xzxiong in #21155
• If the schema version has reached the final version, mark and return by @qingxinhome in #21165
• to 2.0: fix partition state pinned by mo table stats and adding some metrics. by @gouhongshen in #21168
• improve catalog by @jiangxinmeng1 in #21166
• fulltext index bugs port from main to 2.0-dev by @cpegeric in #21139
• Change the upgraded panic error to fatal by @qingxinhome in #21170
• fix disorder in reconnecting logtail 2.0 by @aptend in #21175
• restore upgrade tenant Code by @qingxinhome in #21184
• fix a bug that cause panic in shuffle by @badboynt1 in #21178
• [CherryPick]Fix bug for 21179 by @triump2020 in #21191
• [improvement] proxy: add logs when connect/disconnect by @volgariver6 in #21190
• fix unlock when version of bind changed by @iamlinjunhong in #21206
• fix enum type parse stage(#2.0-dev) by @YANGGMM in #21216
• malloc: default to malloc by @reusee in #21211
• fix the err missing when GetChangedTableList. by @gouhongshen in #21224
• fix TestReplay8 2.0 by @aptend in #21200
• fix data copy in replaying from checkpoint by @aptend in #21232
• [bug] stats: fix hang issue when get stats by @volgariver6 in #21234
• malloc: tune glibc malloc by @reusee in #21239
• to 2.0: reduce the memory consumption by using pointer as btree item in the partition state by @gouhongshen in #21245
• Fix Transient VPool leak by @LeftHandCold in #21247
• to 2.0: skip to subscribe to the deleted table when calculating the mo table stats. by @gouhongshen in #21254
• malloc: fix macos build by @reusee in #21250
• Revert "malloc: default to malloc (#21211)" by @fengttt in #21262
• to 2.0: notify global stats when subscribed table. by @gouhongshen in #21263
• fix mo_table_stats task panic and release mpool. by @gouhongshen in #21273
• bug fix: Invalid search string converted to empty pattern to panic by @cpegeric in #21275
• fix operator of partition by @iamlinjunhong in #21274
• fix txn client deadlock 2.0 by @zhangxu19830126 in #21280
• support dump for vecf32 and vecf64(#2.0-dev) by @YANGGMM in #21282
• add log of stack for retry lock by @iamlinjunhong in #21287
• fileservice: more logs for debugging EOF in Write by @reusee in #21286
• Fix the bug that onBlockSelectedFn does not return error by @LeftHandCold in #21306
• restore moerr code by @iamlinjunhong in #21309
• fix TestCannotBusyLoopIfWriteCIsFull failed to 2.0 by @zhangxu19830126 in #21313
• cleam remote txn when stop cn by @iamlinjunhong in #21321
• bugfix: adjust mo_logger use server side max_allowed_packet config in 2.0 by @xzxiong in #21316
• lower possibility of failure in receiving logtail 2.0 by @aptend in #21323
• fileservice: return AWS region detection error in ObjectStorageArgume… by @reusee in #21299
• fix top container reset (#21208) by @ouyuanning in #21335

Full Changelog: v2.0.1-hotfix-20250102...v2.0.2