Opened 2 years ago

Closed 2 years ago

#33086 closed Bug (wontfix)

ArrayField with JSONField and custom encoder raises error on serialization with "django.core.serializers.json".

Reported by: Anudeep Samaiya Owned by: Anudeep Samaiya
Component: contrib.postgres Version: 3.2
Severity: Normal Keywords: postgres, json
Cc: Anudeep Samaiya Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Anudeep Samaiya)

The encoder is not supported currently, which causes the serialization of uuid fields missing.

Current Implementation:

def value_to_string(self, obj):
    values = []
    vals = self.value_from_object(obj)
    base_field = self.base_field

    for val in vals:
        if val is None:
            values.append(None)
        else:
            obj = AttributeSetter(base_field.attname, val)
            values.append(base_field.value_to_string(obj))
    return json.dumps(values)

Suggested Implementation:

def value_to_string(self, obj, encoder=DjangoJSONEncoder):
    values = []
    vals = self.value_from_object(obj)
    base_field = self.base_field

    for val in vals:
        if val is None:
            values.append(None)
        else:
            obj = AttributeSetter(base_field.attname, val)
            values.append(base_field.value_to_string(obj))
    return json.dumps(values, cls=encoder)

How to reproduce:

import uuid
from django.contrib.postgres.fields import ArrayField, JSONField
from django.core.serializers.json import DjangoJSONEncoder

class Test(models.Model):
    uuid = models.UUIDField(unique=True, default=uuid.uuid4, editable=False)
    data = ArrayField(
        JSONField(encoder=DjangoJSONEncoder),
        max_length=200,
        blank=True,
        default=list,
    )

Now to test do the following:

from decimal import Decimal
from django.core import serializers
from test.models import Test

test = Test(data=[{'threshold_amount_usd': Decimal('0'), 'taint_percent_threshold': Decimal('0')}])
serializers.serialize("json", (test,))

test = Test(data=[{'id': uuid4()}])
serializers.serialize("json", (test,))

Change History (4)

comment:1 by Anudeep Samaiya, 2 years ago

Cc: Anudeep Samaiya added
Owner: changed from < anudeepsamaiya > to Anudeep Samaiya

comment:2 by Mariusz Felisiak, 2 years ago

Easy pickings: unset
Has patch: unset
Resolution: needsinfo
Status: assignedclosed
Summary: Postgres Contrib ArrayField raises error on serialization with "django.core.serializers.json"ArrayField with UUIDField raises error on serialization with "django.core.serializers.json"

Thanks for this report, however serialization works for me with:

  • uuid = models.UUIDField(default=uuid.uuid4)
  • array = ArrayField(models.UUIDField(), default=list)
  • nested_array = ArrayField(ArrayField(models.UUIDField()), null=True)

I don't think you've explained the issue in enough detail to confirm a bug in Django. Please reopen the ticket if you can debug your issue and provide details about why and where Django is at fault. If you're having trouble understanding how Django works, see TicketClosingReasons/UseSupportChannels for ways to get help.

comment:3 by Anudeep Samaiya, 2 years ago

Description: modified (diff)
Easy pickings: set
Has patch: set
Needs documentation: set
Needs tests: set
Patch needs improvement: set
Resolution: needsinfo
Status: closednew

comment:4 by Mariusz Felisiak, 2 years ago

Easy pickings: unset
Has patch: unset
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Resolution: wontfix
Status: newclosed
Summary: ArrayField with UUIDField raises error on serialization with "django.core.serializers.json"ArrayField with JSONField and custom encoder raises error on serialization with "django.core.serializers.json".

Thanks for extra details. Unfortunately proposed change is backward incompatible, we also cannot fix this by changing to_python() and value_to_string() for JSONField (see discussions in PR9801, PR9809, and PR11538). JSONField can store JSON arrays so you should be able to use JSONField(encoder=DjangoJSONEncoder) instead of ArrayField(JSONField(encoder=DjangoJSONEncoder)). It also works with the db round-trip

>>> test = Test.objects.create(data=[{'id': uuid.uuid4()}])
>>> test.refresh_from_db()
>>> serializers.serialize('json', (test,))
'[{"model": "test_33086.test", "pk": 16, "fields": {"array": "[]", "nested_array": null, "data": "[{\\"id\\": \\"2a7cc8f4-a541-49ce-8310-c73c6cc2dc4c\\"}]"}}]'
Note: See TracTickets for help on using tickets.
Back to Top