JSONField form field double encodes invalid values

[David Szotten]

the JSONField form field double-encodes invalid json strings,

e.g. entering the string {"foo"} and submitting the form, it comes back with an error, but the form field populated with the string "{\"foo\"}"

not sure how to fix, since prepare_value doesn't know if we have valid json (and we, like postgres, but unlike the json spec consider bare strings as valid json)

[Claude Paroz]

What about not calling dumps on string values?

[David Szotten]

Replying to claudep:

What about not calling dumps on string values?

because (as i mentioned) bare strings are allowed as valid json (e.g. "foo") this would instead cause problems for such values, which would get stripped of their quotes

[Claude Paroz]

I made a tentative patch here: ​https://github.com/django/django/compare/master...claudep:25532

If we would choose this way, we would have to provide a deprecation path for custom fields without kwargs for prepare_value, even if it seems currently undocumented.

[David S]

Replying to claudep:

Looks like it fixes the problem. Can't speak to whether this is the best solution; what's the next step here?

[Eric]

Just got burned by this too

[Tommy Beadle]

This is how I fixed the issue when I ran across it.

[Claude Paroz]

@tbeadle On invalid input, json.loads() will produce a ValueError, how would that solve the issue?

[Claude Paroz]

I see now, your patch is a fix when the form is redisplayed because of another invalid value, but not when the JSON input itself is wrong, right?

[Claude Paroz]

​PR. Thanks Tommy, you gave me a good track!

[Tim Graham]

Left a few comments for improvement.

[Claude Paroz <claude@…>]

In db196195:

Fixed #25532 -- Properly redisplayed JSONField form input values

Thanks David Szotten for the report and Tommy Beadle for code inspiration.
Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In 6a8ba2e:

[1.9.x] Fixed #25532 -- Properly redisplayed JSONField form input values

Thanks David Szotten for the report and Tommy Beadle for code inspiration.
Thanks Tim Graham for the review.
Partial backport of db19619545 from master.

[Tim Graham]

The new test_formfield_disabled test is failing on the 1.9 branch with TypeError: the JSON object must be str, not 'list'.

[Claude Paroz]

I knew that test was failing on 1.9 and it was not supposed to be committed for that reason. Sorry for the mess.

[Claude Paroz <claude@…>]

In 97ccab12:

[1.9.x] Refs #25532 -- Removed a failing test on Django 1.9

That test is failing on Django 1.9, that issue has been fixed on master
only as it touches a part of form validation that is too important to
touch for a stable release.

[Tim Graham <timograham@…>]

In a5f85d89:

Fixed #26917 -- Fixed crash in disabled ModelChoiceFields.

Partially reverted refs #25532 to fix a regression in Django 1.10.
This reintroduces a crash for disabled forms.JSONField (refs #26949),
however, that issue is also present on Django 1.9.

Thanks Ryan Schave for the test.

[Tim Graham <timograham@…>]

In 3744fc1:

[1.10.x] Fixed #26917 -- Fixed crash in disabled ModelChoiceFields.

Partially reverted refs #25532 to fix a regression in Django 1.10.
This reintroduces a crash for disabled forms.JSONField (refs #26949),
however, that issue is also present on Django 1.9.

Thanks Ryan Schave for the test.

Backport of a5f85d891b51d7ceb4f9e422e3e4f5c741062288 from master