22.8.20 Altinity Stable

Changes Compared to Upstream v22.8.20.11-lts

Bug fix

• Don't execute and/or/if/multiIf on LowCardinality dictionary (#44469 by @Avogar via #283)
• Do not remove inputs from maybe compiled DAG. (#48303 by @KochetovNicolai via #278)
• Arrow wouldn't abort process in case of error in input file (#45478 by @Avogar via #242)
• Fix nullptr dereference in DB::VolumeJBOD::reserve (#41483 by @amosbird via #241)
• Fix base58Encode / base58Decode handling leading 0 / '1' (#40620 by @zvonand via #224)
• Flatten list type arrow chunks on parsing (#43297 by @arthurpassos via #209)

Improvement

• Added support of chunked arrow/parquet arrays (#40485 by @arthurpassos via #223)
• Increase request_timeout_ms for s3 disks (#42321 by @filimonov via #222)
• Improved stale replica recovery process for ReplicatedMergeTree (#42134 by @tavplubix, #41945 by @alesapin via #221)

Security

• Fixed CVEs in the clickhouse-diagnostics utility

Build/Testing/Packaging Improvement

• Made builds and tests possible in Altinity's infrastructure

Changes in upstream from v22.8.15.23-lts to v22.8.20.11-lts

Bug Fix (user-visible misbehavior in an official stable release)

• Fix broken index analysis when binary operator contains a null constant argument #50177 (Amos Bird).
• Fix incorrect constant folding #50536 (Alexey Milovidov).
• Fix fuzzer failure in ActionsDAG #51301 (Alexey Milovidov).
• Fix segfault in MathUnary #51499 (Ilya Yatsishin).
• Fix subtly broken copy-on-write of ColumnLowCardinality dictionary #51064 (Michael Kolupaev).
• Generate safe IVs #51086 (Salvatore Mesoraca).
• Fix bad cast from LowCardinality column when using short circuit function execution #43311 (Kruglov Pavel).
• Fix msan issue in randomStringUTF8(uneven number) #49750 (Robert Schulze).
• JIT compilation not equals NaN fix #50056 (Maksim Kita).
• Fix crash with multiIf and constant condition and nullable arguments #50123 (Anton Popov).
• Fixed type conversion from Date/Date32 to DateTime64 when querying with DateTime64 index #50280 (Lucas Chang).
• Fix Keeper deadlock on exception when preprocessing requests. #50387 (frinkr).
• Fix Log family table return wrong rows count after truncate #50585 (flynn).
• Do not read all the columns from right GLOBAL JOIN table. #50721 (Nikolai Kochetov).
• Fix explain graph with projection #47473 (flynn).
• Disable compile_aggregate_expressions by default #48195 (Alexey Milovidov).
• Fix possible segfault in cache #48469 (Kseniia Sumarokova).
• Fix bug in Keeper when a node is not created with scheme auth in ACL sometimes. #48595 (Aleksei Filatov).
• Fix bug in zero-copy replication disk choice during fetch #47010 (alesapin).
• Fix query parameters #47488 (Alexey Milovidov).
• Fix wait for zero copy lock during move #47631 (alesapin).
• Fix crash in polygonsSymDifferenceCartesian #47702 (pufit).
• Backport to 22.8: Fix moving broken parts to the detached for the object storage disk on startup #48273 (Aleksei Filatov).

Performance Improvement

• Backported in #49214: Fixed excessive reading in queries with FINAL. #47801 (Nikita Taranov).

Improvement

• Backported in #48157: Fixed UNKNOWN_TABLE exception when attaching to a materialized view that has dependent tables that are not available. This might be useful when trying to restore state from a backup. #47975 (MikhailBurdukov).

Packages

23.3.8 Altinity Stable

Changes compared to Upstream v23.3.8.21-lts

Bug fixes

• Fix key condition on duplicate primary keys (ClickHouse#48838 by @amosbird via #275 )

Performance improvement

• Do not build sets for skip indexes if they are not used (ClickHouse#48299 by @CurtizJ via #276)

Build/Testing/Packaging Improvement

• Made builds and tests possible in Altinity's infrastructure

Full release notes

Packages

22.8.15 Altinity Stable

Changes Compared to Upstream v22.8.15.23-lts

Bug fix

• Fix base58Encode / base58Decode handling leading 0 / '1' (ClickHouse#40620 by @zvonand via #224)
• arrow wouldn't abort process in case of error in input file (ClickHouse#45478 by @Avogar via #242)
• Fix possible server crash when using the JBOD feature. (ClickHouse#41483 by @amosbird via #241)

Improvement

• Increase request_timeout_ms for s3 disks (ClickHouse#42321 by @filimonov via #222)
• Improved stale replica recovery process for ReplicatedMergeTree (ClickHouse#42134 by @tavplubix, ClickHouse#41945 by @alesapin via #221)
• Added support of chunked arrow/parquet arrays (ClickHouse#40485 by @arthurpassos via #223)

CI/CD

• executing https://github.com/Altinity/clickhouse-regression/ tests against the newly built binaries (#249 by @MyroTk)

Other

• unit-test for arrow lib errors (ClickHouse#47958 by @arthurpassos via #243)
• Do not ship clickhouse-diagnostics in clickhouse-common-static package due to golang CVEs (#257)

Changes in upstream from v22.8.13.20-lts to v22.8.15.23-lts

Performance Improvement

• Backported in #45845: Fixed performance of short SELECT queries that read from tables with large number ofArray/Map/Nested columns. #45630 (Anton Popov).
• Backported in #46374: Fix too big memory usage for vertical merges on non-remote disk. Respect max_insert_delayed_streams_for_parallel_write for the remote disk. #46275 (Nikolai Kochetov).
• Backported in #46358: Allow using Vertical merge algorithm with parts in Compact format. This will allow ClickHouse server to use much less memory for background operations. This closes #46084. #46282 (Anton Popov).

Build/Testing/Packaging Improvement

• Backported in #46112: Remove the dependency on the adduser tool from the packages, because we don't use it. This fixes #44934. #45011 (Alexey Milovidov).
• Backported in #46482: Get rid of unnecessary build for standalone clickhouse-keeper. #46367 (Mikhail f. Shiryaev).
• Backported in #46505: Some time ago the ccache compression was changed to zst, but gz archives are downloaded by default. It fixes it by prioritizing zst archive. #46490 (Mikhail f. Shiryaev).

Improvement

• Backported in #46981: - Apply ALTER TABLE table_name ON CLUSTER cluster MOVE PARTITION|PART partition_expr TO DISK|VOLUME 'disk_name' to all replicas. Because ALTER TABLE t MOVE is not replicated. #46402 (lizhuoyu5).

Bug Fix

• Backported in #45908: Fixed bug with non-parsable default value for EPHEMERAL column in table metadata. #44026 (Yakov Olkhovskiy).
• Backported in #46238: A couple of seg faults have been reported around c-ares. All of the recent stack traces observed fail on inserting into std::unodered_set<>. I believe I have found the root cause of this, it seems to be unprocessed queries. Prior to this PR, CH calls poll to wait on the file descriptors in the c-ares channel. According to the poll docs, a negative return value means an error has ocurred. Because of this, we would abort the execution and return failure. The problem is that poll will also return a negative value if a system interrupt occurs. A system interrupt does not mean the processing has failed or ended, but we would abort it anyways because we were checking for negative values. Once the execution is aborted, the whole stack is destroyed, which includes the std::unordered_set<std::string> passed to the void * parameter of the c-ares callback. Once c-ares completed the request, the callback would be invoked and would access an invalid memory address causing a segfault. #45629 (Arthur Passos).
• Backported in #45727: Fix key description when encountering duplicate primary keys. This can happen in projections. See #45590 for details. #45686 (Amos Bird).
• Backported in #46394: Fix SYSTEM UNFREEZE queries failing with the exception CANNOT_PARSE_INPUT_ASSERTION_FAILED. #46325 (Aleksei Filatov).
• Backported in #46442: Fix possible LOGICAL_ERROR in asynchronous inserts with invalid data sent in format VALUES. #46350 (Anton Popov).
• Backported in #46674: Fix an invalid processing of constant LowCardinality argument in function arrayMap. This bug could lead to a segfault in release, and logical error Bad cast in debug build. #46569 (Alexey Milovidov).
• Backported in #46879: Fix MSan report in the maxIntersections function. This closes #43126. #46847 (Alexey Milovidov).
• Backported in #46871: Fix a bug in the Map data type. This closes #46855. #46856 (Alexey Milovidov).
• Backported in #47336: Sometimes after changing a role that could be not reflected on the access rights of a user who uses that role. This PR fixes that. #46772 (Vitaly Baranov).
• Backported in #46901: - Fix incorrect alias recursion in QueryNormalizer. #46609 (Raúl Marín).
• Backported in #47156: - Fix arithmetic operations in aggregate optimization with min and max. #46705 (Duc Canh Le).
• Backported in #46987: Fix result of LIKE predicates which translate to substring searches and contain quoted non-LIKE metacharacters. #46875 (Robert Schulze).
• Backported in #47357: Fix possible deadlock on distributed query cancellation. #47161 (Kruglov Pavel).

NOT FOR CHANGELOG / INSIGNIFICANT

• Another attempt to fix automerge, or at least to have debug footprint #45476 (Mikhail f. Shiryaev).
• Add check for running workflows to merge_pr.py #45803 (Mikhail f. Shiryaev).
• Get rid of progress timestamps in release publishing #45818 (Mikhail f. Shiryaev).
• Add necessary dependency for sanitizers #45959 (Mikhail f. Shiryaev).
• Add helping logging to auto-merge script #46080 (Mikhail f. Shiryaev).
• Fix write buffer destruction order for vertical merge. #46205 (Nikolai Kochetov).
• Get rid of legacy DocsReleaseChecks #46665 (Mikhail f. Shiryaev).
• Reduce updates of Mergeable Check #46781 (Mikhail f. Shiryaev...

22.8.13 Altinity Stable

Changes Compared to Upstream v22.8.13.20-lts

Bug Fix

• Fix base58Encode / base58Decode handling leading 0 / '1' (ClickHouse#40620 by @zvonand via #224)
• fixed c-ares related crash (ClickHouse#45629 by @arthurpassos via #231)

Improvement

• Increase request_timeout_ms for s3 disks (ClickHouse#42321 by @filimonov via #222)
• Improved stale replica recovery process for ReplicatedMergeTree (ClickHouse#42134 by @tavplubix, ClickHouse#41945 by @alesapin via #221)
• Added support of chunked arrow/parquet arrays (ClickHouse#40485 by @arthurpassos via #223)

Full release notes

22.3.15.34 Altinity Stable

Based on upstream/v22.3.15.33-lts

Changes Compared to upstream v22.3.15.33-lts

Bug fixes

• Fix for exponential time decaying window functions. Now respecting boundaries of the window ClickHouse#36944 by @excitoon (via #164)
• Fixes for objects removal in S3ObjectStorage ClickHouse#37882 by @excitoon (via #164)
• Build/Testing/Packaging Improvement: Allow Github workflows to run on Altinity's infrastructure
• Fixed Unknown identifier (aggregate-function) exception ClickHouse#39762 by @quickhouse (via #189)
• Fixed point of origin for exponential decay window functions ClickHouse#39593 by @quickhouse (via #190)
• Reverted: 'Fix seeking while reading from encrypted disk' ClickHouse#39687, known to cause issues in 22.3 (via #193)
• Reverted: 'Enable using constants from with and select in aggregate function parameters.' ClickHouse#27531, causes problems on 22.3 (via #194)
• Fix memory leak while pushing to MVs w/o query context (from Kafka/...) ClickHouse#40732 by @azat (via #202)
• Fix arrow column dictionary to ch lc ClickHouse#40037 by @arthurpassos (via #209)

Improvements

• Improvement: Optimized processing of ORDER BY in window functions ClickHouse#34632 by @excitoon (via #164)
• Improvement: Support batch_delete capability for GCS ClickHouse#37659 by @frew (via #164)
• Add support for extended (chunked) arrays for Parquet format (ClickHouse#40485, ClickHouse#43297) by @arthurpassos (via #203, #211)

changes in upstream

Packages

https://builds.altinity.cloud/

22.3.12.20 Altinity Stable

Based on upstream/v22.3.12.19-lts

Changes Compared to Community Build v22.3.12.19-lts

Bug fix

• Fix for exponential time decaying window functions. Now respecting boundaries of the window ClickHouse#36944 by @excitoon (via #164)
• Fixes for objects removal in S3ObjectStorage ClickHouse#37882 by @excitoon (via #164)
• Fixed Unknown identifier (aggregate-function) exception ClickHouse#39762 by @quickhouse (via #189)
• Fixed point of origin for exponential decay window functions ClickHouse#39593 by @quickhouse (via #190)
• Fix unused unknown columns introduced by WITH statement ClickHouse#39131 by @amosbird
• Fix memory leak while pushing to MVs w/o query context (from Kafka/...) by @azat

Improvement

• Optimized processing of ORDER BY in window functions ClickHouse#34632 by @excitoon (via #164)
• Support batch_delete capability for GCS ClickHouse#37659 by @frew (via #164)
• Add support for extended (chunked) arrays for Parquet format ClickHouse#40485 by @arthurpassos

Build/Testing/Packaging Improvement

• Build/Testing/Packaging Improvement: Allow Github workflows to run on Altinity's infrastructure

Changes in upstream from v22.3.10.22-lts to v22.3.12.19-lts

Bug fix

• Fix clickhouse-server doinst.sh. ClickHouse#40681 by @Felixoid.
• fix heap buffer overflow by limiting http chunk size. ClickHouse#40292 by @CheSema.
• Fix possible segfault in CapnProto input format'. ClickHouse#40241 by @Avogar.
• Fix insufficient argument check for encryption functions'. ClickHouse#40194 by @alexey-milovidov.
• Fix HashMethodOneNumber with const column. ClickHouse#40020 by @canhld94.
• Fix seeking while reading from encrypted disk. ClickHouse#39687 by @vitlibar.
• Fix number of threads for pushing to views'. ClickHouse#39253 by @azat.

Other

• collectFilesToSkip() in MutateTask now supports new index file extension .idx2 for MinMax. ClickHouse#40122 by @robot-ch-test-poll2.
• Clean out clickhouse-server.service from /etc. ClickHouse#39323 by @Felixoid.

Full list of changes

Packages

https://builds.altinity.cloud/

22.3.10.24 Altinity Stable

Based on upstream/v22.3.10.22-lts

Changes Compared to Community Build v22.3.10.22-lts

Bug fix

• Fix for exponential time decaying window functions. Now respecting boundaries of the window ClickHouse#36944 by @excitoon (via #164)
• Fixes for objects removal in S3ObjectStorage ClickHouse#37882 by @excitoon (via #164)
• Fixed Unknown identifier (aggregate-function) exception ClickHouse#39762 by @quickhouse (via #189)
• Fixed point of origin for exponential decay window functions ClickHouse#39593 by @quickhouse (via #190)
• Reverted 'Fix seeking while reading from encrypted disk' ClickHouse#39687, known to cause issues in 22.3 (via ##193)
• Reverted: 'Enable using constants from with and select in aggregate function parameters.' ClickHouse#27531, causes problems on 22.3 (via #194)

Improvement

• Improvement: Optimized processing of ORDER BY in window functions ClickHouse#34632 by @excitoon (via #164)
• Improvement: Support batch_delete capability for GCS ClickHouse#37659 by @frew (via #164)

Build/Testing/Packaging Improvement

• Build/Testing/Packaging Improvement: Allow Github workflows to run on Altinity's infrastructure

Changes in upstream from v22.3.8.39-lts to v22.3.10.22-lts

Bug fix

• Fix CANNOT_READ_ALL_DATA with pread_threadpool.. #39800 (Anton Popov).
• Fix seeking while reading from encrypted disk. #39687 (Vitaly Baranov).
• Fix bug with maxsplit in the splitByChar. #39552 (filimonov).
• Fix data race and possible heap-buffer-overflow in Avro format. #39498 (Kruglov Pavel).
• Fix incorrect postgresql query which fetches tables list. #39283 (Kseniia Sumarokova).
• Fix segfault in materialised postgresql. #39272 (Kseniia Sumarokova).
• removed default queue arguments. #39259 (rnbondarenko).
• Fix reading of sparse columns from s3 #37978 via #39206 (robot-ch-test-poll).
• Fix reading from s3 in some corner cases #38239. via #39206 (robot-ch-test-poll).
• prewhere action should preserve sorting columns. #39157 (Yakov Olkhovskiy).
• Fix insert into MV with enabled extremes.. #39125 (Nikolai Kochetov).
• Fix for fs cache. #39070 (Kseniia Sumarokova).
• Remove logging from OvercommitTracker to avoid deadlocks. #39030 (Dmitry Novik).
• Fix reading of sparse columns from s3. #37978 (Anton Popov).
• Fix some issues with low cardinality and async reads from remote fs. #36763 (Kseniia Sumarokova).
• Fix performance regression of scalar query. #35986 (Amos Bird).

Other

• Change mysql-odbc url. #39702 (Mikhail f. Shiryaev).
• Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread. #39619 (Nikolai Kochetov).
• Add docker_server.py running to backport and release CIs. #39011 (Mikhail f. Shiryaev).
• Retry docker buildx commands with progressive sleep in between. #38898 (Mikhail f. Shiryaev).

Full list of changes

Packages

https://builds.altinity.cloud/

22.3.8 Altinity Stable

Based on upstream/v22.3.8.39-lts

Changes Compared to Community Build v22.3.8.39-lts

• Bug fix: Fix for exponential time decaying window functions. Now respecting boundaries of the window ClickHouse#36944 by @excitoon (via #164)
• Bug fix: Fixes for objects removal in S3ObjectStorage ClickHouse#37882 by @excitoon (via #164)
• Improvement: Optimized processing of ORDER BY in window functions ClickHouse#34632 by @excitoon (via #164)
• Improvement: Support batch_delete capability for GCS ClickHouse#37659 by @frew (via #164)

Packages

https://builds.altinity.cloud/

21.1.11 Altinity Stable

Based on upstream/v21.1.9.41-stable

Changes Compared to Community Build v21.1.9.41-stable

• Security fix: CVE-2021-43304, CVE-2021-43305 (ClickHouse#27743)
• Security fix: CVE-2021-42387, CVE-2021-42388, CVE-2021-42389, CVE-2021-42390, CVE-2021-42391 (ClickHouse#27136)
• background_fetches_pool_size is set to 8. This is better for production usage with frequent small insertions or slow ZooKeeper clusters. (via 579bf72)
• Bug fix: Fixed the error ‘Cannot find column in ActionsDAG result', which may happen if subquery uses untuple. ClickHouse#22290 (via 9f3e0b1)
• Bug fix: Fixed quantile(s)TDigest that could produce non-accurate results in some edge cases: ClickHouse#22408 (via 312180e)
• Bug fix: Fixed a bug with accessing S3 buckets when the object URL contained ‘+’ sign: ClickHouse#23822 (via a0418cb)

Packages

https://builds.altinity.cloud/

22.3 Altinity Stable

Based on upstream v22.3.5-lts

Changes compared to Community Build v22.3.5-lts

• Building/Packaging fixes