Repository navigation
perf(sessions): batch canonical observation body reads - #4739
Closed
ScriptedAlchemy wants to merge 1 commit into
Closed
ScriptedAlchemy wants to merge 1 commit into
ScriptedAlchemy wants to merge 1 commit into
Conversation
Replace per-reference awaited SQL reads in both observation decoders with one indexed read on the caller's existing connection or snapshot. Preserve missing-body order and canonical size/hash verification. Add real SQLite regressions for query counts, exact reconstruction, inline fast paths and typed missing/corrupt body refusals. Native SQLite mechanism probes passed; Rust and Bazel validation is pending on hosted CI.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Both observation hydration adapters awaited one SQL round trip for each distinct canonical body. The temporal projector already batches observation rows, but then reintroduced a per-body waterfall while decoding each row. Concurrent tasks on the same retained snapshot would not remove that dispatch overhead.
Change
Regression coverage
Added a behavioral regression in each adapter using the actual SQLite-backed
TestConnection. A query-counting wrapper forwards every call to the real reader. Tests cover two distinct bodies plus a shared duplicate, exact reconstruction, one body query instead of two, no query for inline content, and typed missing/corrupt-body refusals. The global adapter also checks that an unrelated corrupt body is not fetched. Existing tests remain intact.Validation actually performed
Passed six native SQLite 3.46.1 mechanism checks using the exact SQL extracted from the changed source: 32 referenced bodies required one statement rather than 32, identity/deflate/Unicode rows matched scalar reads, missing rows and ordering were preserved, unrelated rows were excluded, 100 randomized reference subsets matched, and query-only execution left data unchanged.
EXPLAIN QUERY PLANuses the canonical-body primary-key index for each requested hash. The supplementary Python verifier exercised corruption/declared-size cases; it does not validate the Rust implementation.The local source copies were verified against their original Git blob hashes before editing.
git diff --checkpassed. Published blobs match the reviewed local files. A separate source-review pass checked snapshot ownership, error mapping/order, integrity checks and both production callers. Publication is based on refreshed masteradbf0e39; its intervening Doctor merge does not touch these files.Not run: Rust compilation, Rust tests, rustfmt, Bazel, production latency/RSS benchmarks, or the full native matrix. This environment has no supplied full checkout or Rust/Bazel toolchain. Keep this draft until hosted validation executes the owning global-db, session-temporal-store and store unit tests. No wall-clock speedup is claimed from the statement-count reduction.
This is one focused result of the requested async hot-path audit. It does not replace the independent retention/GC work in #4718 or modify its branch. Existing PR repair/merge automation owns CI follow-through and authorized merging.