deps: duckdb 1.5.5 + pyducklake 1.0.17 — leak-free ducklake channel, drift-gated

[jghoman]

Why

The 1.5.2 extension channel is frozen at ducklake build 415a9ebd, which leaks per-connection native memory proportional to catalog metadata churn (A/B-verified in viaduck: ~1MiB per snapshot-advancing bind; OOM every 4-6h there). Millpond's long-lived writer connections against a busy shared catalog have the same exposure. The 1.5.5 channel serves d8a1881e, which is flat. pyducklake (ours) rides along as a deliberate lockstep canary — it hard-pins duckdb, so a one-sided bump of either package fails uv lock instead of silently splitting the two DuckLake stacks. Nothing imports it yet.

The drift gate

The frozen 1.5.2 channel was accidental drift protection; 1.5.5's channel is live and can re-serve new builds under the same URL. The release workflow runs in parallel with CI, so the test canary alone can't stop a drifted build from reaching the fleet via the mutable tag + autobounce. The Dockerfile now asserts the ducklake build SHA in the extension-install layer: an unexpected ducklake build cannot produce an image at all. Verified both directions (correct SHA builds; --build-arg DUCKLAKE_SHA=deadbeef fails the layer with the drift message).

Behavior changes under d8a1881e (all verified against real catalogs)

• The (INT64_MAX, UINT64_MAX] shred-rejection window and write-time error signature are unchanged — the fix: fall back to string-only when the VARIANT write fails #118 guard and backstop hold. The inlined-path deferred failure now reports INT128 (was UINT64); the width-agnostic matcher still catches it.
• JSON numbers outside [INT64_MIN, UINT64_MAX] in a VARIANT become lossy DOUBLE (were digit-preserving VARCHAR). Deliberate policy: follow native DuckLake semantics; the string column stays authoritative (now asserted). Rows written across the version boundary mix both representations in the companion permanently.
• No catalog metadata migration: old/new builds read and write the same catalogs interchangeably (probed both directions, incl. shredded Parquet and inlined data), so mixed fleets and co-readers (viaduck, duckgres) are unaffected.
• duckdb_databases() no longer lists the attach's metadata db as its own row — test-fixture-only; no prod code touches it.

New engine-behavior contract test

tests/integration/test_ducklake_engine_contract.py pins what the variant guard assumes of the pinned build, probed the way the 2026-08-12 incident executed (guard bypassed, real shredded Parquet writes): the containment invariant (engine-rejected ⊆ guard-rewritten, through the real sanitize_variant_sources chain), its converse (shreddable probes round-trip untouched), both window boundaries exactly, nested/array/sci-notation shapes, error-signature recognition, and the inlined commit-then-detonate path. The next bump that shifts any of this fails at the PR instead of on the fleet.

Supply chain

pyducklake 1.0.17 published 2026-08-13 → fixed-date exclude-newer-package carve-out from the 7-day cooldown (future releases re-enter it until deliberately bumped); sha256-pinned in uv.lock. Added to the dependency-review allow-list (PEP 639-only license metadata, same class as the opentelemetry entries; Apache-2.0).

Testing

• 740 unit+integration tests pass, including the new 16 contract cases; e2e suite (4/4) against the built image; ruff clean.
• Image built locally for arm64 with the gate; negative build verified to fail.
• Cross-version probes run in side-by-side 1.5.2/1.5.5 environments (linux containers) for the mixed-fleet analysis.

Authored with agent assistance; two adversarial review passes (correctness + prod-impact/deployment) ran over the diff and their findings are incorporated (drift gate, allow-list entry, lockstep comment, nested/array probes, stale-comment sweep).