Flaky test: ClickHouseStatementTest.testAsyncInsert (jdbc-v1) races the server async-insert flush

[polyglotAI-bot]

Description

com.clickhouse.jdbc.ClickHouseStatementTest.testAsyncInsert fails intermittently in the
JDBC driver + CH 26.3 (apache_http_client) and JDBC driver + CH latest (apache_http_client)
integration jobs. It has been red on unrelated PRs since about 2026-08-04.

The second half of the test (clickhouse-jdbc/src/test/java/com/clickhouse/jdbc/ClickHouseStatementTest.java:541-549)
inserts one row with async_insert=1,wait_for_async_insert=0 and then asserts that the row is
not yet queryable. That only holds while the server has not flushed its async-insert buffer,
so the test depends on a server timeout it does not control. The test's own message admits it
("Server was probably busy at that time"), and the //TODO: I'm not sure this is a valid test...
comment on line 539 predicts exactly this.

Steps to reproduce

1. Run the single test against a recent server, on an idle machine: it passes.
2. Run it again while the machine is under CPU load (as a CI runner is):
   mvn -B -pl clickhouse-jdbc -DskipUTs=true -Dit.test=ClickHouseStatementTest#testAsyncInsert -Dfailsafe.failIfNoSpecifiedTests=false verify
3. Repeat step 2 a few times. It fails in a large fraction of runs.

Measured on ClickHouse 26.7.3.19, 4 vCPU:

• idle: 0 failures in 6 runs
• with 12 busy-loop processes on 4 vCPU: 3 failures in 5 runs, with the identical assertion CI reports

Error Log or Exception StackTrace

[ERROR] com.clickhouse.jdbc.ClickHouseStatementTest.testAsyncInsert -- Time elapsed: 0.237 s <<< FAILURE!
[ERROR]   ClickHouseStatementTest.testAsyncInsert:548 Server was probably busy at that time, so the row was inserted before your query expected [false] but found [true]
[ERROR] Tests run: 231, Failures: 1, Errors: 0, Skipped: 0

230 of 231 tests pass in every observed failure; only testAsyncInsert fails.

Evidence of non-determinism in CI

Three independent runs, on PRs whose diffs do not touch async insert (JavaCC/ANTLR parser paths
and INSERT ... TABLE FUNCTION routing), and the same check passes on other open PRs:

• Fix jdbc-v2: reparse the parameters of a recovered ANTLR4 parse tree #3026 - https://github.com/ClickHouse/clickhouse-java/actions/runs/30880305039/job/91901348954
• Fix jdbc-v2: keep INSERT INTO [TABLE] FUNCTION off the RowBinary insert path #3016 - https://github.com/ClickHouse/clickhouse-java/actions/runs/30869518158/job/91869635687
• Fix jdbc-v2: NPE when the JavaCC parser cannot parse an INSERT VALUES list #3014 - https://github.com/ClickHouse/clickhouse-java/actions/runs/30865833294 (both CH 26.3 and CH latest jobs)

Mechanism

The server flushes an async-insert buffer after async_insert_busy_timeout_*. On 26.x the
adaptive timeout is on by default and starts at the minimum:

async_insert_use_adaptive_busy_timeout   1
async_insert_busy_timeout_min_ms         50
async_insert_busy_timeout_max_ms         200

So with a single async insert on an otherwise idle server the row becomes visible about 50 ms
after the INSERT. The test's assertion therefore only passes if the driver issues the following
SELECT within that ~50 ms window. That gap is pure client-side latency (statement splitting,
two more HTTP round trips, JIT warm-up) and is not bounded by anything, so on a slow or loaded
runner it exceeds 50 ms and the row is already there.

Reproduced at the protocol level, without the driver, by inserting an equivalent gap
(TRUNCATE / async INSERT / SELECT as three HTTP requests, default settings): the row was
already visible in 24 of 30 runs. Both outcomes occur, which is the flake.

Expected Behaviour

testAsyncInsert should pass deterministically. Verifying that wait_for_async_insert=0
returns before the data is queryable is a reasonable thing to test - but the window has to be
made deterministic instead of relying on the server being slow.

Suggested fix

Pin the buffer open for the duration of the assertion, rather than adding a retry or a sleep,
by extending the custom params on line 541:

props.setProperty(ClickHouseHttpOption.CUSTOM_PARAMS.getKey(),
        "async_insert=1,wait_for_async_insert=0"
        + ",async_insert_use_adaptive_busy_timeout=0"
        + ",async_insert_busy_timeout_ms=30000"
        + ",async_insert_busy_timeout_max_ms=30000");

Verified at the protocol level with those settings and a deliberate 500 ms delay before the
SELECT (10x the current window): the row was visible in 0 of 15 runs. The assertion becomes
a real invariant instead of a race.

Optionally, the test could then also prove the other half of the contract - run
SYSTEM FLUSH ASYNC INSERT QUEUE and assert the row does appear - so the test covers both
states deterministically.

Note the first half of the test (wait_for_async_insert=1, lines 524-536) is not affected and
should keep its current behavior.

Configuration

Environment

• Cloud (this half of the test is skipped on Cloud)
• Client version: main @ 601ade1
• Language version: OpenJDK 17.0.18, Maven 3.9.15
• OS: Ubuntu 24.04, 4 vCPU

ClickHouse Server

• ClickHouse Server version: 26.7.3.19 locally; CI fails on 26.3 and on latest
• Non-default settings: none - the defaults above are what make it race
• CREATE TABLE:

CREATE TABLE test_async_insert(id UInt32, s String) ENGINE = MergeTree ORDER BY id;

---

Found by our PR monitor, which saw this check fail on three unrelated PRs of ours, and then
verified locally: it fails 3 of 5 runs under CPU load and 0 of 6 when idle.

[polyglotAI-bot]

Small correction to the numbers above: the loaded loop finished after the issue was filed. Final
rate is 3 failures in 6 runs under load (runs 2, 4 and 5 failed; 1, 3 and 6 passed), against
0 failures in 6 runs idle. Everything else is unchanged.

[mshustov]

@polyglotAI-bot fix this