id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
31973	TypeError loading data in JSONField if DB has native JSON support	Adam Alton	nobody	"Following today's security release, I've upgraded from Django 3.1 to 3.1.1 with no other code changes, and suddenly I'm getting this error when loading a model instance from the DB:

{{{
list(MyModel.objects.all())
Traceback (most recent call last):
  File ""<console>"", line 1, in <module>
  File ""/usr/local/lib/python3.8/site-packages/django/db/models/query.py"", line 287, in __iter__
    self._fetch_all()
  File ""/usr/local/lib/python3.8/site-packages/django/db/models/query.py"", line 1303, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File ""/usr/local/lib/python3.8/site-packages/django/db/models/query.py"", line 70, in __iter__
    for row in compiler.results_iter(results):
  File ""/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py"", line 1100, in apply_converters
    value = converter(value, expression, connection)
  File ""/usr/local/lib/python3.8/site-packages/django/db/models/fields/json.py"", line 74, in from_db_value
    return json.loads(value, cls=self.decoder)
  File ""/usr/local/lib/python3.8/json/__init__.py"", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not list
}}}

From looking at the recent changes between 3.0 and 3.1, I'm pretty sure that the problem was introduced in this commit: https://github.com/django/django/commit/0be51d2226fce030ac9ca840535a524f41e9832c

Specifically, in django/db/models/fields/json.py, the removal of this from `from_db_value`:


{{{
    if connection.features.has_native_json_field and self.decoder is None:
            return value
}}}



I'm pretty sure that this means that if the value has already been decoded and the DB has got native JSON support, then it continues through this function, attempting to decode the already-decoded data, and therefore gets a `TypeError`. But note that the exception catching around `json.loads` only catches `json.JSONDecodeError`, not `TypeError`, and hence it dies.

I will see if I can write a test to prove this (or run the existing tests with postgres, which has native JSON support). But I'm reporting this now, as it seems like a fairly serious breaking change in 3.1.1. And maybe someone else who understands the changes in the mentioned commit can fix this faster than I can."	Bug	closed	Database layer (models, ORM)	3.1	Normal	invalid		Thiago Bellini Ribeiro giacomo	Unreviewed	0	0	0	0	0	0