test: disable connector telemetry in test runs to remove CI ERROR-log noise#1500
Open
sd-db wants to merge 1 commit into
Open
test: disable connector telemetry in test runs to remove CI ERROR-log noise#1500sd-db wants to merge 1 commit into
sd-db wants to merge 1 commit into
Conversation
The databricks-sql connector (4.2.6, the locked version) enables telemetry by
default and runs its POSTs on a background thread. When those POSTs fail in CI
they spam ERROR logs ("--- Logging error ---" blocks and literal
"HTTP request error: %s" lines) on otherwise-green runs, inflating log artifacts
and burying real failures.
Test runs gain nothing from telemetry, so pass enable_telemetry=False through
the test profile's connection_parameters; the connector then falls back to its
NoopTelemetryClient and emits no telemetry POSTs.
Collaborator
Author
|
/integration-test |
|
Integration tests dispatched for PR #1500 by @sd-db. Track progress in the Actions tab. |
Coverage report
This PR does not seem to contain any modification to coverable code. |
jprakash-db
approved these changes
Jun 5, 2026
|
Integration results for PR #1500 — UC cluster ❌ failure · SQL warehouse ✅ success · All-purpose cluster ✅ success · Shard coverage ✅ success |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On fully successful CI runs, the integration-test logs are flooded with ERROR-level noise —
--- Logging error ---+ traceback blocks and literalHTTP request error: %slines — that buries real failures and inflates log artifacts (~4,000 of ~9,200 lines were noise in a sample green job,78949595712).Root cause of the bulk of it: the
databricks-sql-connector(4.2.6, our locked version) enables telemetry by default and runs its POSTs on a backgroundThreadPoolExecutor. When those POSTs fail in CI (runner egress to the telemetry endpoint), the connector logs retry-exhaustion at ERROR on a background thread. Under pytest those records hit a capture handler whose stream has been closed at teardown, degrading into the--- Logging error ---blocks.Fix
Test runs gain nothing from connector telemetry. Pass
enable_telemetry=Falsethrough the test profiles'connection_parameters(which flows todbsql.connect), so the connector uses itsNoopTelemetryClientand emits no telemetry POSTs — removing the noise at the source.This is scoped to
tests/profiles.pyonly; no product code changes.Verification
telemetry_enabled is Falseand usesNoopTelemetryClient.27020859727): across all 8 e2e shards (cluster / uc-cluster / sqlwarehouse), the logs contained zero stray telemetry lines — no--- Logging error ---blocks, noERROR:databricks.sql…lines, noI/O operation on closed file, noHTTP request error: %s. (The original sample job carried ~4,000 such lines.)test_remove_foreign_key_constraint) due to a transient server-sideDatabase Error [INTERNAL_ERROR]creating a foreign-key model — not related to telemetry or this change.Follow-up (separate PR)
There is a related, independent user-facing bug — the logging bridge in
dbt/adapters/databricks/logging.pyforwards the raw%stemplate instead of the rendered message, so connector logs with placeholders render a literal%son real dbt runs (e.g.Token exchange failed, using external token: %s, still visible in the run above). That fix (record.getMessage(), plus stoppingdatabricks.sqlrecords from propagating to the root logger) is deferred to its own PR.