23.3.13 Altinity Stable

Changes compared to upstream's v23.3.13.6-lts

Bug fixes

• Fix key condition on duplicate primary keys (ClickHouse#48838 by @amosbird via #275 )
• Fix excessive memory usage for FINAL (ClickHouse#50429 by @azat via #307)
• Fixed DNS-related crash (ClickHouse#52634 by @arthurpassos via #295)
• Fix parquet array inconsistent offsets (ClickHouse#54370 by @arthurpassos via #303)
• Fixed rare bug when we produced invalid read ranges for queries with FINAL. Resulted in Cannot read out of marks range exception (ClickHouse#54934 by @nickitat via #313)
• Fixed reading from sparse columns after changing setting ratio_of_defaults_for_sparse_serialization back to 1.0 and restarting the server. (ClickHouse#49660 by @CurtizJ via #314)

Performance improvement

• Do not build sets for skip indexes if they are not used (ClickHouse#48299 by @CurtizJ via #276)

Improvement

• Disabled unsafe "compile_expressions" setting by default (ClickHouse#51368 by @alexey-milovidov via #306)
• Row policy for database (ClickHouse#47640 by @ilejn via #300)

Build/Testing/Packaging Improvement

• Bring back DNS tests (ClickHouse#53286 - @arthurpassos via #296)
• Made builds and tests possible in Altinity's infrastructure
• Support of Aarch64 builds (#311)
  • building Aarch64 binaries
  • multi-arch docker containers (Amd64, Aarch64)
  • running stateles, stateful tests against Aarch64
  • running clickhouse-regression tests against Aarch64

Upstream changes between v23.3.8.21-lts (1675f2264f3) and v23.3.13.6-lts (25635e27551)

Performance Improvement

• Backported in #52213: Do not store blocks in ANY hash join if nothing is inserted. #48633 (vdimir).
• Backported in #52826: Fix incorrect projection analysis which invalidates primary keys. This issue only exists when query_plan_optimize_primary_key = 1, query_plan_optimize_projection = 1 . This fixes #48823 . This fixes #51173 . #52308 (Amos Bird).

Build/Testing/Packaging Improvement

• Backported in #53019: Packing inline cache into docker images sometimes causes strange special effects. Since we don't use it at all, it's good to go. #53008 (Mikhail f. Shiryaev).
• Backported in #53288: The compiler's profile data (-ftime-trace) is uploaded to ClickHouse Cloud., the second attempt after #53100. #53213 (Alexey Milovidov).
• Backported in #53461: Preserve environment parameters in clickhouse start command. Fixes #51962. #53418 (Mikhail f. Shiryaev).
• Get rid of describe_parameters for the best robot token #53833 (Mikhail f. Shiryaev).

Bug Fix (user-visible misbehavior in an official stable release)

• Fix optimization to move functions before sorting. #51481 (Nikolai Kochetov).
• Fix Block structure mismatch in Pipe::unitePipes for FINAL #51492 (Nikita Taranov).
• Fix binary arithmetic for Nullable(IPv4) #51642 (Yakov Olkhovskiy).
• Fix async insert with deduplication for ReplicatedMergeTree using merging algorithms #51676 (Antonio Andelic).
• Support IPv4 and IPv6 as dictionary attributes #51756 (Yakov Olkhovskiy).
• Fix deadlock on DatabaseCatalog shutdown #51908 (Alexander Tokmakov).
• Fix ORDER BY tuple of WINDOW functions #52145 (Alexey Milovidov).
• Disable expression templates for time intervals #52335 (Alexander Tokmakov).
• Fix countSubstrings() hang with empty needle and a column haystack #52409 (Sergei Trifonov).
• Fixed inserting into Buffer engine #52440 (Vasily Nemkov).
• The implementation of AnyHash was non-conformant. #52448 (Alexey Milovidov).
• init and destroy ares channel on demand.. #52634 (Arthur Passos).
• Fix crash in function tuple with one sparse column argument #52659 (Anton Popov).
• clickhouse-keeper: fix implementation of server with poll() #52833 (Andy Fiddaman).
• Fix password leak in show create mysql table #52962 (Duc Canh Le).
• Not-ready Set #53162 (Nikolai Kochetov).
• Fix incorrect normal projection AST format #53347 (Amos Bird).
• Fix loading lazy database during system.table select query #53372 (SmitaRKulkarni).
• Fix wrong columns order for queries with parallel FINAL. #53489 (Nikolai Kochetov).
• Fix crash in join on sparse column #53548 (vdimir).
• Fix: interpolate expression takes source column instead of same name aliased from select expression. #53572 (Yakov Olkhovskiy).
• Correctly handle totals and extremes with DelayedSource #53644 (Antonio Andelic).
• Fix: sorted distinct with sparse columns #53711 (Igor Nikonov).
• Fix rows_before_limit_at_least for DelayedSource. #54122 (Nikolai Kochetov).
• Backport moving/ part checking code from master #54157 (Mikhail f. Shiryaev).
• Fix crash in comparison functions due to incorrect query analysis #52172 (Alexey Milovidov).
• Fix deadlocks in StorageTableFunctionProxy #52626 (Alexander Tokmakov).

NOT FOR CHANGELOG / INSIGNIFICANT

• Disable test_reverse_dns_query/test.py #53195 (Alexander Tokmakov).
• Disable test_host_regexp_multiple_ptr_records/test.py #53211 (Alexander Tokmakov).
• Fix broken 02862_sorted_distinct_sparse_fix #53738 (Antonio Andelic).

Packages

Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb, .rpm, or .tgz

Docker images

Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.3.13.7.altinitystable