perf(models): validate only the matched variant for discriminated unions (#1649)#1695
Open
Echolonius wants to merge 1 commit into
Open
Conversation
construct_type validated the whole RawMessageStreamEvent union on every streaming event, even though the wire data already names its variant via the `type` discriminator. Validating a union forces pydantic to consider every member; resolving the variant first and validating just that one returns an identical object for roughly half the CPU. The variant is resolved from the (already-cached) discriminator metadata before the full-union validate. On a 4k-delta stream this is ~1.8x faster on the decode leg (~16.1 -> 8.8 us/event, ~45% less CPU, pydantic 2.12 / py3.9). Invalid or non-discriminated data falls through to the existing whole-union path unchanged, so behavior is identical in every other case. Addresses the union-decode leg of anthropics#1649 (complementary to anthropics#1663, which covers the build_events / accumulate_event legs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
construct_typevalidated the wholeRawMessageStreamEventunion on every streaming event, even though each wire event already names its variant via thetypediscriminator. Validating a union forces pydantic to consider every member; resolving the variant up-front and validating just that one returns an identical object for roughly half the CPU.This addresses the union-decode leg of #1649 — the largest cost in that issue's profile (
construct_type→validate_python, ~45s of the ~100s). It is complementary to #1663, which covers thebuild_events/accumulate_eventlegs in_messages.pyand does not touch the union decode in_models.py.Change
In the
is_unionbranch ofconstruct_type, when the union is discriminated and the value carries a known discriminator string, validate just the matched variant before falling back to the whole-union validate:The discriminator metadata is already cached in
DISCRIMINATOR_CACHE, so variant resolution is a dict lookup after warm-up. The existing post-failure discriminator block is reused (thevariant_typeis now resolved once at the top), so the diff is small.Why it's safe (behavior preserved)
new == old == construct_typefor every event in the benchmark).pass, and fall through to the exact existing path (whole-union validate → unvalidated.construct()of the matched variant). The pre-existingtest_discriminated_unions_invalid_data*tests pin this and still pass.variant_typestaysNone, fast path is skipped, whole-union path runs unchanged.Benchmark
Realistic stream (4000
content_block_deltaevents + start/stop), pydantic 2.12.5 / CPython 3.9, best of 7:Every streaming consumer pays this leg (iteration and
get_final_message()-only), and the same fast path benefits all discriminated-union decoding across the SDK — not just streaming.Tests
tests/test_models.py— added 4 tests:.construct()pathAll existing
test_models.py+ streaming tests pass (62 model tests, 280 across models/streaming/client). No new pyright/ruff findings.