JSONFIeld decode error with Python 3.9

[Alan D. Snow]

https://code.djangoproject.com/ticket/31973

Traceback:

venv/lib/python3.9/site-packages/django/db/models/fields/json.py:83: in from_db_value
    return json.loads(value, cls=self.decoder)

    def loads(s, *, cls=None, object_hook=None, parse_float=None,
            parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
        """Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance
        containing a JSON document) to a Python object.
    
        ``object_hook`` is an optional function that will be called with the
        result of any object literal decode (a ``dict``). The return value of
        ``object_hook`` will be used instead of the ``dict``. This feature
        can be used to implement custom decoders (e.g. JSON-RPC class hinting).
    
        ``object_pairs_hook`` is an optional function that will be called with the
        result of any object literal decoded with an ordered list of pairs.  The
        return value of ``object_pairs_hook`` will be used instead of the ``dict``.
        This feature can be used to implement custom decoders.  If ``object_hook``
        is also defined, the ``object_pairs_hook`` takes priority.
    
        ``parse_float``, if specified, will be called with the string
        of every JSON float to be decoded. By default this is equivalent to
        float(num_str). This can be used to use another datatype or parser
        for JSON floats (e.g. decimal.Decimal).
    
        ``parse_int``, if specified, will be called with the string
        of every JSON int to be decoded. By default this is equivalent to
        int(num_str). This can be used to use another datatype or parser
        for JSON integers (e.g. float).
    
        ``parse_constant``, if specified, will be called with one of the
        following strings: -Infinity, Infinity, NaN.
        This can be used to raise an exception if invalid JSON numbers
        are encountered.
    
        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
        kwarg; otherwise ``JSONDecoder`` is used.
        """
        if isinstance(s, str):
            if s.startswith('\ufeff'):
                raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
                                      s, 0)
        else:
            if not isinstance(s, (bytes, bytearray)):
>               raise TypeError(f'the JSON object must be str, bytes or bytearray, '
                                f'not {s.__class__.__name__}')
E               TypeError: the JSON object must be str, bytes or bytearray, not list

../../miniconda/envs/midas/lib/python3.9/json/__init__.py:339: TypeError

Suggested fix is to catch TypeError as well as JSONDecodeError here: ​https://github.com/django/django/blob/adb4100e58d9ea073ee8caa454bb7c885b6a83ed/django/db/models/fields/json.py#L82-L85

        try:
            return json.loads(value, cls=self.decoder)
        except (json.JSONDecodeError, TypeError):
            return value

[Mariusz Felisiak]

Duplicate of #31973.

[Alan D. Snow]

This is not a duplicate. It is related to #31973, but a fix has not been applied yet.

[Mariusz Felisiak]

#31973 is closed as invalid, so "fix" will not be applied.

[Alan D. Snow]

​https://github.com/django/django/pull/15083

[Mariusz Felisiak]

Alan, please stop and don't reopen closed tickets without providing any new details.

[Alan D. Snow]

I am confused. What is the resolution?

[Alan D. Snow]

Is there something wrong with this?

                                          Table "public.table"
      Column      |           Type           | Collation | Nullable |                           Default                            
------------------+--------------------------+-----------+----------+--------------------------------------------------------------
 id               | integer                  |           | not null | nextval('table_id_seq'::regclass)
 metadata         | jsonb                    |           | not null | 

[Alan D. Snow]

When I do:

class Table(models.Model):
    metadata = JSONField(
        blank=True,
        default=dict,
        help_text="Any additional metadata.",
    )

And create a model and query for it:

table = Table.objects.create(metadata={"A": "B"})
found = table.objects.first()

I get an error.
If I set found.metadata = "", and query fort it, the error goes away.

[Alan D. Snow]

There is no issue with Python 3.8.

[Alan D. Snow]

Current workaround:

class JSONFieldPatch(models.JSONField):
    """
    Patch for Python 3.9 since json returns TypeError
    """

    def from_db_value(self, value, expression, connection):
        try:
            return super().from_db_value(value, expression, connection)
        except TypeError:
            return value

[Mariusz Felisiak]

The provided model works for me with Python 3.8 and Python 3.9. I don't think you've explained the issue in enough detail to confirm a bug in Django. Please reopen the ticket if you can debug your issue and provide details about why and where Django is at fault.