JSONField form field double encodes invalid values

[David Szotten]

the JSONField form field double-encodes invalid json strings,

e.g. entering the string {"foo"} and submitting the form, it comes back with an error, but the form field populated with the string "{\"foo\"}"

not sure how to fix, since prepare_value doesn't know if we have valid json (and we, like postgres, but unlike the json spec consider bare strings as valid json)

[Claude Paroz]

What about not calling dumps on string values?

[David Szotten]

Replying to claudep:

What about not calling dumps on string values?

because bare strings are allowed as valid json (e.g. "foo") this would instead cause problems for such values, which would get stripped of their quotes

[Claude Paroz]

I made a tentative patch here: ​https://github.com/django/django/compare/master...claudep:25532

If we would choose this way, we would have to provide a deprecation path for custom fields without kwargs for prepare_value, even if it seems currently undocumented.

[David S]

Replying to claudep:

Looks like it fixes the problem. Can't speak to whether this is the best solution; what's the next step here?

[Eric]

Just got burned by this too

[Tommy Beadle]

This is how I fixed the issue when I ran across it.

[Claude Paroz]

@tbeadle On invalid input, json.loads() will produce a ValueError, how would that solve the issue?

[Claude Paroz]

I see now, your patch is a fix when the form is redisplayed because of another invalid value, but not when the JSON input itself is wrong, right?

[Claude Paroz]

​PR. Thanks Tommy, you gave me a good track!

[Tim Graham]

Left a few comments for improvement.

[Claude Paroz <claude@…>]

In db196195:

Fixed #25532 -- Properly redisplayed JSONField form input values

Thanks David Szotten for the report and Tommy Beadle for code inspiration.
Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In 6a8ba2e:

[1.9.x] Fixed #25532 -- Properly redisplayed JSONField form input values

Thanks David Szotten for the report and Tommy Beadle for code inspiration.
Thanks Tim Graham for the review.
Partial backport of db19619545 from master.

[Tim Graham]

The new test_formfield_disabled test is failing on the 1.9 branch with TypeError: the JSON object must be str, not 'list'.

[Claude Paroz]

I knew that test was failing on 1.9 and it was not supposed to be committed for that reason. Sorry for the mess.

[Claude Paroz <claude@…>]

In 97ccab12:

[1.9.x] Refs #25532 -- Removed a failing test on Django 1.9

That test is failing on Django 1.9, that issue has been fixed on master
only as it touches a part of form validation that is too important to
touch for a stable release.

[Tim Graham <timograham@…>]

In a5f85d89:

Fixed #26917 -- Fixed crash in disabled ModelChoiceFields.

Partially reverted refs #25532 to fix a regression in Django 1.10.
This reintroduces a crash for disabled forms.JSONField (refs #26949),
however, that issue is also present on Django 1.9.

Thanks Ryan Schave for the test.

[Tim Graham <timograham@…>]

In 3744fc1:

[1.10.x] Fixed #26917 -- Fixed crash in disabled ModelChoiceFields.

Partially reverted refs #25532 to fix a regression in Django 1.10.
This reintroduces a crash for disabled forms.JSONField (refs #26949),
however, that issue is also present on Django 1.9.

Thanks Ryan Schave for the test.

Backport of a5f85d891b51d7ceb4f9e422e3e4f5c741062288 from master