Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with JsonField in ORM Causing Data Retrieval Failure #905

Open
murabo opened this issue Sep 16, 2024 · 0 comments
Open

Issue with JsonField in ORM Causing Data Retrieval Failure #905

murabo opened this issue Sep 16, 2024 · 0 comments
Assignees
Labels
api: spanner Issues related to the googleapis/python-spanner-django API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@murabo
Copy link

murabo commented Sep 16, 2024

I am encountering an issue when using JsonField in a Django model with Google Spanner. After saving data into a table that contains a JsonField through the Django ORM, I am unable to retrieve the data due to a TypeError.

The error occurs when attempting to query the table, and the following traceback is returned:

from django.db import models
class JsonTest(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    body = models.JSONField(null=True, blank=True)
    is_processed = models.BooleanField(default=False)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 256, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 280, in __iter__
    self._fetch_all()
  File "/Users/murabo/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/query.py", line 68, in __iter__
    for row in compiler.results_iter(results):
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1116, in apply_converters
    value = converter(value, expression, connection)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/p3_1/lib/python3.12/site-packages/django/db/models/fields/json.py", line 83, in from_db_value
    return json.loads(value, cls=self.decoder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/.pyenv/versions/3.12.0/lib/python3.12/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not JsonObject

It seems like the JsonField data type is not being handled correctly when reading from the database, possibly related to how the data is being serialized and deserialized.

Steps to reproduce:

Create a model with a JsonField.
Save data using the Django ORM.
Attempt to query the table containing the JsonField.
Expected behavior: The data should be retrieved and deserialized correctly.

Actual behavior: The query fails with a TypeError, indicating that the JSON object must be str, bytes, or bytearray, but JsonObject is being returned.

Any help on resolving this issue would be greatly appreciated!

@murabo murabo added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Sep 16, 2024
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner-django API. label Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner-django API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

2 participants