fix(contrib/pydantic): reuse TypeAdapters across payloads - #1703
fix(contrib/pydantic): reuse TypeAdapters across payloads#1703nathan-gage wants to merge 1 commit into
Conversation
PydanticJSONPlainPayloadConverter.from_payload constructed a fresh pydantic TypeAdapter for every payload, rebuilding the core schema each time for non-class hints such as discriminated unions and generic collections. Cache adapters per converter instance, keyed on hashable type hints; unhashable hints keep constructing fresh adapters. The cache is unbounded by default and configurable via the new keyword-only max_cached_type_adapters option on PydanticJSONPlainPayloadConverter and PydanticPayloadConverter (positive bounds with LRU eviction, zero disables caching, negative raises ValueError). Fixes temporalio#1695
|
I would prefer this to be opt in behavior, at least at first, and mark it as experimental please. |
|
@tconley1428 happy to make that change, but, I'd like to offer some context first, since I'd consider this closer to a bug fix than a new feature. Pydantic recommends creating a
rebuilding it per payload is ~45–50× slower in my benchmarks (three-member discriminated union), and was a lot faster in our internal worker, was extremely noticeable in my usage. wrt memory risk, I measured per-adapter process RSS (Python heap + Rust core-schema, pydantic 2.13):
a typical app with 20–100 distinct workflow/activity type hints caches well under 1 MiB on the long-lived client/worker converter. the only real regression path is runtime-generated hints (dynamic models, parametrized perhaps we take a middle path; default i'd also note that opt-in is awkward with the current API — that said, first contribution here, so if opt-in + experimental is the standard for behavior changes, totally no objection! just let me know the preferred shape (e.g. default |
What was changed
PydanticJSONPlainPayloadConverter.from_payloadconstructed a freshpydantic.TypeAdapterfor every payload,rebuilding the core schema each time for non-class hints such as discriminated unions and generic collections (measured
~46x slower than validation alone on a three-member discriminated union).
Adapters are now cached per converter instance, keyed on hashable type hints; unhashable hints keep constructing fresh
adapters. The cache is unbounded by default since registered workflow/activity hint sets are finite. A new keyword-only
max_cached_type_adaptersoption onPydanticJSONPlainPayloadConverterandPydanticPayloadConverterbounds the cache(LRU eviction), disables it (
0), or rejects negative values.Why?
Fixes #1695
Checklist
composite-converter forwarding, threaded first-use of a
defer_build=Truemodeltests/contrib/pydantic/suite passespoe lint(ruff check/format, pyright, mypy, basedpyright, pydocstyle) passesDataConverter; CHANGELOG updated under Unreleased/Changed.