Use Pydantic TypeAdapter for encoding/decoding JSON fields

[jmbledsoe]

Is your feature request related to a problem? Please describe.

I am using Postgres JSONB columns for storing complex data structures, and I have Pydantic models representing those data structures. Where the column itself is a Pydantic model, I can use JSONField to encode and decode the model:

• When encoding, the field uses model_dump to convert the model to a Python object here
• When decoding, the field uses the __init__ to construct a model from a Python dictionary here

However, some other columns are not themselves Pydantic models but are composite objects that include Pydantic models e.g. list[SomePydanticModel] or dict[str, SomePydanticModel]. I cannot use JSONField for these columns because the instances themselves are not Pydantic models i.e. they do not have the model_dump function and the field type is not ModelMetaclass.

Describe the solution you'd like
Pydantic provides the TypeAdapter class for working with composite objects like this. Its dump_python function behaves like model_dump, and its validate_python function behaves like a model's __init__ function. Using the functions on TypeAdapter would allow JSONField to handle composite models in the same way that it handles simple models.

Describe alternatives you've considered
I currently have an implementation of a Tortoise ORM field that uses TypeAdapter. My implementation bypasses the encoder and decoder functions, opting instead to use dump_json and validate_json directly instead. I intend to open a PR with this implementation, but I'm open to guidance to do something differently to bring this capability into Tortoise ORM.

Additional context
I am not 100% sure of the performance cost of instantiating TypeAdapter, so we may want to include some form of caching constructed instances.

[LanceMoe]

The changes there were primarily made to generate better OpenAPI documentation through TypeHints( #1702 ). I didn’t take more complex cases. If you have a better solution, I think submitting a PR would be worth a try. :)

[henadzit]

Can you please post here your implementation of your field?

[jmbledsoe]

Can you please post here your implementation of your field?

YMMV, but this is the implementation that is working for me today. We are converting from SQLAlchemy/SQLModel to Tortoise ORM so I may adapt it as I convert more tables:

from typing import TypeVar

import tortoise
from pydantic import TypeAdapter
from tortoise.exceptions import FieldError


T = TypeVar("T")


class ModelField(tortoise.fields.JSONField[T]):
    """ JSONB field that is serialized from/deserialized to a Pydantic type

    JSONField defined by Tortoise ORM includes the optional behavior of serializing from/deserializing to Pydantic
    models using `model_validate` and `model_dump` respectively, but this doesn't work for composite Python types that
    include Pydantic models, such as lists or dictionaries. To work around this issue, ModelField uses Pydantic's
    `TypeAdapter`, which has functions for validating and dumping models, whether they are simple or composite.
    """

    def to_db_value(
        self,
        value: T | str | bytes | dict | list | None,
        instance: type[tortoise.Model] | tortoise.Model,
    ) -> str | None:
        self.validate(value)
        if value is None:
            return None

        if isinstance(value, (str, bytes)):
            try:
                self.to_python_value(value)
            except Exception:
                raise FieldError(f"Value {value!r} is invalid json value.")
            if isinstance(value, bytes):
                return value.decode()
            return value

        return TypeAdapter(self.field_type).dump_json(value).decode()

    def to_python_value(self, value: T | str | bytes | dict | list | None) -> T | None:
        if value is None:
            return None

        if isinstance(value, (str, bytes)):
            return TypeAdapter(self.field_type).validate_json(value)

        return TypeAdapter(self.field_type).validate_python(value)