23.8.16 Altinity Stable
Enmk
released this
26 Sep 00:16
·
21 commits
to releases/23.8.16
since this release
Changes compared to upstream's v23.8.16.40-lts (e143a90)
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346) - Fix expiration in RoleCache (ClickHouse#67748 by @vitibar via #451)
- Fix crash with invalid access to AccessControl instance (ClickHouse#65051 by @davenger via #465)
- fix SIGSEGV due to profiler (ClickHouse#63865 by @azat via #466)
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Improvement
- Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
- Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)
- Engine
Merge
filters the records according to the row policies of the underlying tables (ClickHouse#50209 by @ilejn via #446)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)- Changed "version is not official" log message on failure (#369 by @Enmk)
- Fix kerberized_hadoop docker image build issue (#394 by @ilejn)
- Reuse unchanged images #455 by @MyroTk
- Pinned version of pip packet requests==2.31.0 in attempt to fix: 'Error while fetching server API version: Not supported URL scheme http+docker' (#409 by @Enmk)
- Aarch64 builds with better introspection (
-no-pie
) (#396 by @Enmk) - Fixed the docker tag for docker server test image (#463 by @MyroTk)
- Add
SignAarch64
job to match SignRelease (#462 by @MyroTk) - Fix
test_distributed_directory_monitor_split_batch_on_failure_OFF
(#464 by @Enmk) - Fix
kererberized_hadoop
tests (#468, #459 by @ilejn) - Updated reference to clickhouse-regression suite (#467 by @MyroTk)
- Scheduled run + CI/CD fixups (#402 by @MyroTk)
upstream ClickHouse changes between v23.8.16.40-lts (e143a90) and v23.8.11.28-lts (31879d2)
Critical Bug Fix (crash, LOGICAL_ERROR, data loss, RBAC))
- Fix crash with
UniqInjectiveFunctionsEliminationPass
anduniqCombined
. (#65188 by Raúl Marín via #65281) - Fix a bug in ClickHouse Keeper that causes digest mismatch during closing session. (#65198 by Aleksei Filatov via #65368)
- Fix crash in maxIntersections. (#65689 by Raúl Marín via #65743)
Bug Fix (user-visible misbehavior in an official stable release))
- The query cache now denies access to entries when the user is re-created or assumes another role. This improves prevents attacks where 1. an user with the same name as a dropped user may access the old user's cache entries or 2. a user with a different role may access cache entries of a role with a different row policy. (#58611 by Robert Schulze via #61439)
- Fix string search with constant start position which previously could lead to memory corruption. (#61547 by Antonio Andelic via #61572)
- Fix crash in
multiSearchAllPositionsCaseInsensitiveUTF8
when specifying incorrect UTF-8 sequence. Example: #61714. (#61749 by pufit via #61854) - Fixed a bug in zero-copy replication (an experimental feature) that could cause
The specified key does not exist
errors and data loss after REPLACE/MOVE PARTITION. A similar issue might happen with TTL-moves between disks. (#54193 by Alexander Tokmakov via #62898) - Fix the ATTACH query with the ON CLUSTER clause when the database does not exist on the initiator node. Closes #55009. (#61365 by Nikolay Degterinsky via #61964)
- Fix data race between
MOVE PARTITION
query and merges resulting in intersecting parts. (#61610 by János Benjamin Antal via #62527) - Fix skipping escape sequence parsing errors during JSON data parsing while using
input_format_allow_errors_num/ratio
settings. (#61883 by Kruglov Pavel via #62238) - Fix segmentation fault when using Hive table engine. Reference #62154, #62560. (#62578 by Nikolay Degterinsky via #62673)
- Setting server_name might help with recently reported SSL handshake error when connecting to MongoDB Atlas:
Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: error:10000438:SSL routines:OPENSSL_internal:TLSV1_ALERT_INTERNAL_ERROR
. (#63122 by Alexander Gololobov via #63172) - The wire protocol version check for MongoDB used to try accessing "config" database, but this can fail if the user doesn't have permissions for it. The fix is to use the database name provided by user. (#63126 by Alexander Gololobov via #63164)
- Fixes #59989: runs init scripts when force-enabled or when no database exists, rather than the inverse. (#59991 by jktng via #64422)
- Fix "Invalid storage definition in metadata file" for parameterized views. (#60708 by Azat Khuzhin via #64016)
- Fix the issue where the function
addDays
(and similar functions) reports an error when the first parameter isDateTime64
. (#61561 by Shuai li via #63456) - Fix crash with untuple and unresolved lambda. (#63131 by Raúl Marín via #63289)
- Fix
X-ClickHouse-Timezone
header returning wrong timezone when usingsession_timezone
as query level setting. (#63377 by Andrey Zvonov via #63512) query_plan_remove_redundant_distinct
can break queries with WINDOW FUNCTIONS (withallow_experimental_analyzer
is on). Fixes #62820. (#63776 by Igor Nikonov via #63902)- Deserialize untrusted binary inputs in a safer way. (#64024 by Robert Schulze via #64104)
- Prevent LOGICAL_ERROR on CREATE TABLE as MaterializedView. (#64174 by Raúl Marín via #64265)
- Fixed memory possible incorrect memory tracking in several kinds of queries: queries that read any data from S3, queries via http protocol, asynchronous inserts. (#64844 by Anton Popov via #64867)
- Fix possible abort on uncaught exception in ~WriteBufferFromFileDescriptor in StatusFile. (#64206 by Kruglov Pavel via #65351)
- Fix crash on destroying AccessControl: add explicit shutdown. (#64993 by Vitaly Baranov via #66037)
- Fixed bug in MergeJoin. Column in sparse serialisation might be treated as a column of its nested type though the required conversion wasn't performed. (#65632 by Nikita Taranov via #65782)
- For queries that read from
PostgreSQL
, cancel the internalPostgreSQL
query if the ClickHouse query is finished. Otherwise,ClickHouse
query cannot be canceled until the internalPostgreSQL
query is finished. (#65771 by Maksim Kita via #65926) - Fix a bug in short circuit logic when old analyzer and
dictGetOrDefault
is used. (#65802 by jsc0218 via #65822) - Fixed a bug in ZooKeeper client: a session could get stuck in unusable state after receiving a hardware error from ZooKeeper. For example, this might happen due to "soft memory limit" in ClickHouse Keeper. (#66140 by Alexander Tokmakov via #66449)
- Correctly track memory for
Allocator::realloc
. (#66548 by Antonio Andelic via #66717) - Fix bug with session closing in Keeper. (#65735 by Antonio Andelic via #65913)
- Fix data race in
S3::ClientCache
. (#66644 by Konstantin Morozov via #66853)
Improvement
- Fixed accounting of memory allocated before attaching thread to a query or a user. (#56089 by Nikita Taranov via #61930)
- Added support for parameterized view with analyzer to not analyze create parameterized view. Refactor existing parameterized view logic to not analyze create parameterized view. (#54211 by SmitaRKulkarni via #66962)
- Reload certificate chain during certificate reload. (#61671 by Pervakov Grigorii via #65461)
- Always start Keeper with sufficient amount of threads in global thread pool. (#64444 by Duc Canh Le via #65880)
- Respect cgroup CPU limit in Keeper. (#65819 by Antonio Andelic via #65912)
Build/Testing/Packaging Improvement
- Remove from the Keeper Docker image the volumes at
/etc/clickhouse-keeper
and/var/log/clickhouse-keeper
. (#61683 by Tristan via #62007) - The Dockerfile is reviewed by the docker official library in docker-library/official-images#15846. (#63400 by Mikhail f. Shiryaev via #63621)
- Decrease the
unit-test
image a few times. (#65102 by Mikhail f. Shiryaev via #65153) - fix
02124_insert_deduplication_token_multiple_blocks
. (#63950 by Han Fei via #63965)
CI Fix or Improvement (changelog entry is not required))
- Remove unnecessary layers from clickhouse/cctools (#61374 by Mikhail f. Shiryaev via #61428)
- CI: fix checkout action version (#61441 by Max K. via #61484)
00002_log_and_exception_messages_formatting
: exclude one more format string (#62190 by Konstantin Bogdanov via #62201)- We won't fail the job when GH fails to retrieve the job ID and URLs. (#62651 by Mikhail f. Shiryaev via #62796)
- Unflake
02813_func_now_and_alias
(#62932 by Robert Schulze via #62968) - 'Installation test has wrong check_state'. (#63994 by Mikhail f. Shiryaev).
- The commit url has different pattern. (#63331 by Mikhail f. Shiryaev via #63343)
- Do not create new release in release branch automatically. (#64039 by Mikhail f. Shiryaev via #64043)
- Pin requests to fix the integration tests. (#65183 by Mikhail f. Shiryaev)
NO CL CATEGORY
- Revert "Revert "ReplicatedMergeTree invalid metadata_version fix"" (#60078 by Maksim Kita via #62585).
- Revert "Revert "Do not remove server constants from GROUP BY key for secondary query."" #63415 by Nikolai Kochetov via #63704)
NOT FOR CHANGELOG / INSIGNIFICANT
- Throw on query timeout in ZooKeeperRetries (#60922 by Antonio Andelic)
- Follow up to #56541. (#57141 by Kseniia Sumarokova via #65080)
Build report
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.16.41.altinitystable