Opened 4 years ago
Last modified 4 years ago
#32542 closed Bug
Native postgres json field support is broken — at Initial Version
Reported by: | Artem | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 3.1 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
In Django 3.1.1, there was a refactoring of encoders/decoders of DB fields which affected postgres json/jsonb field support.
Before:
Postgres json and jsonb field values are encoded/decoded into python native format by psycopg2. JSONField had it's own decoder but if was omitted if db has native json support.
After:
Jsonb field is no longer decoded by psycopg2 (an empty stub is used instead of json.joads
), all decoding stuff is delegated to JSONField.from_db_value(...)
.
But postgres json field is still decoded by psycopg2, which causes 'TypeError' while trying to decode it again by JSONField
.
Breaking commit: https://github.com/django/django/commit/0be51d2226fce030ac9ca840535a524f41e9832c
Example:
class CustomJSONField(models.JSONField): def db_type(self, *args, **kwargs): # stored as-is, keys order is preserved return 'json' class TestModel(models.Model) json_data = CustomJSONField(default=dict) TestModel(json_data={"foo": "bar"}).save() TestModel.objects.last()
Expected behavior: code snippet works
Actual behavior:
TypeError: the JSON object must be str, bytes or bytearray, not dict
The issue was mentioned in few tickets (#31973, #31956) but eventually their authors found workarounds (e.g. by switching to jsonb field) or still using Django <= 3.1. I think support of postgres json (not only jsonb) is important too.
The possible fix is to add the decoder stub for json field too:
Before (https://github.com/django/django/blob/main/django/db/backends/postgresql/base.py#L218):
psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)
After:
psycopg2.extras.register_default_json(conn_or_curs=connection, loads=lambda x: x) psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)