JSONField improperly escaped in admin interface during normal usage

[Jeff Cook]

The line of code at ​https://github.com/django/django/blob/68de48c96328e13d5dbdb1f3006e4a1ca74f3c34/django/contrib/postgres/forms/jsonb.py#L45 results in the escaping of a plain JSON string. That's because the string is already in valid JSON when run through json.dumps. This is user-facing and visible in the Django admin interface when a JSONField is edited.

With the current code, the line in the admin looks like this: ​http://imgur.com/fw29t1c.jpg

If we just return value instead of json.dumps(value), we get this: ​http://imgur.com/msBT1tI.jpg

(Someone may want to attach those jpgs directly to this report for posterity; I don't know what the convention is on this bug tracker)

In fact, looking at the code now, I wonder if we just need to swap lines 44 and 45. It would seem that if a value is invalid JSON, as the check on line 43 implies, that we *should* be dumping it through the json serializer, whereas if it's valid JSON, we shouldn't. Maybe the logic just got swapped there.

#25532 deals with a similar topic.

[Tim Graham]

Are you using Django 1.10? What are the steps to reproduce?

[Jeff Cook]

It should be noted that I am also using Mezzanine and seeing this issue through Mezzanine's admin interface, though I haven't found this issue to exist in Mezzanine's code -- as I said, the fix that's working for me is in contrib/postgres/forms/jsonb.py. I haven't tested non-Mezzanine. I'm using PgSQL 9.5.4, Django 1.10.1, Mezzanine git head (06b3df5380a5a), Python 3.5.2.

Steps to reproduce: create a django.contrib.postgres.fields.JSONField in a model. Migrate. Make sure the model is registered for admin. Load up the admin. Input valid JSON into the JSONField and save. Open the newly-added record in the admin. Observe that the JSONField is improperly escaped.

[Tim Graham]

I still can't reproduce. What's your valid JSON input? It would be useful to know if you can reproduce without Mezzanine to rule out an issue there.

[ErikW]

I too have been impacted by this bug. I'm not using either Mezzanine or django admin.

I have the field declared on my like like so:

    from django.contrib.postgres.fields import JSONField

    class FeedStatus(models.Model)
        #.....
        data = JSONField(default=dict)
        #.....

When I try to access feed_status.data -- it returns JSON as a string (ie "{}" instead of {}}. Each time the object is saved, the empty string is re-serialized (ie '"\\"{}\\""').

In 18:44:50 [30]: f = FeedStatus.objects.get(pk=25806)

In 18:44:54 [31]: f.data = {}

In 18:44:56 [32]: f.save()

In 18:44:59 [33]: f = FeedStatus.objects.get(pk=25806)

In 18:45:01 [34]: f.data
Out 18:45:01 [34]: '{}'

In 18:45:03 [35]: f.save()

In 18:45:05 [36]: f = FeedStatus.objects.get(pk=25806)

In 18:45:07 [37]: f.data
Out 18:45:07 [37]: '"{}"'

In 18:45:10 [38]: f.save()

In 18:45:11 [39]: f = FeedStatus.objects.get(pk=25806)

In 18:45:16 [40]: f.data
Out 18:45:16 [40]: '"\\"{}\\""'

In 18:45:19 [41]: f.save()

In 18:45:20 [42]: f = FeedStatus.objects.get(pk=25806)

In 18:45:23 [43]: f.data
Out 18:45:23 [43]: '"\\"\\\\\\"{}\\\\\\"\\""'

This of course leads to incredibly bloated columns in the database (I've found some that are 10's of MB in size when they should just be a blank dict!!!)

I'm on Django 1.10.7.

[Tim Graham]

Are you using django-jsonfield? See ticket:27675#comment:8.