Code

Opened 6 years ago

Closed 6 years ago

Last modified 3 years ago

#7131 closed (fixed)

Update simplejson to 1.8.1 for better Unicode support

Reported by: jmillikin Owned by: nobody
Component: Core (Serialization) Version: master
Severity: Keywords: simplejson, json, unicode
Cc: mattimustang@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Simplejson 1.8.1 features greatly improved Unicode support and performance over the version included in Django trunk. In particular, the version in trunk mangles astral unicode when serializing.

If it's not feasible to include the most recent version, I would be willing to backport as much as I could to Django's version if that would be preferable.

Attachments (1)

simplejson-1.9.2.patch (2.1 KB) - added by Matthew Flanagan <mattimustang@…> 6 years ago.
patch simplejson-1.9.2 to turn it into django.utils.simplejson

Download all attachments as: .zip

Change History (7)

comment:1 Changed 6 years ago by mattimustang@…

  • Cc mattimustang@… added
  • Has patch set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

+1
Except to say that simplejson 1.9.1 is out now.
I'm seeing very poor performance in simplejson 1.5 in Django when it comes to using it in serving RESTful JSON data from my models. Serializing anything more than a dozen objects with a few fields becomes noticeably slow to users. Being able to take advantage of the C "speedups" in the later versions of simplejson would be very beneficial.

Just how to include the module so that users can build the C extension themselves will be tricky.

Two alternatives would be to decouple simplejson from Django as requested in #4843 or change the simplejson import in django.core.serializers.json to be:

try:
    import simplejson
except ImportError:
    from django.utils import simplejson

Either way would allow users to build their own simplejson and use it.

comment:2 Changed 6 years ago by Simon Greenhill

  • Has patch unset
  • Triage Stage changed from Unreviewed to Design decision needed

comment:3 Changed 6 years ago by mtredinnick

  • Triage Stage changed from Design decision needed to Accepted

Jacob and I had a chat about this and we'll do it at some point (moving to the latest version). What will be useful in the interim, however, is if somebody could work out what, if any, backwards incompatibilities there are between simplejson 1.5 and simplejson 1.9 that might affect Django's serialisers. Just drop a note in this ticket with any discoveries so that we can update the wiki page when we make the update.

comment:4 Changed 6 years ago by Matthew Flanagan <mattimustang@…>

  • milestone set to 1.0 beta

I've had a look through and I can't see any backward incompatibilities and I've dropped simplejson-1.9.2 in as a replacement for django.utils.simplejson in a large application that I have that uses serialization quite heavily and I haven't had any problems. All django tests pass too.

I'll attach a patch to simplejson-1.9.2 to bring it into Django.

We should also note in the serialization docs that you can build the C _speedups.so from the original simplejson source and copy it to django/utils/simplejson to get better serialization performance.

Changed 6 years ago by Matthew Flanagan <mattimustang@…>

patch simplejson-1.9.2 to turn it into django.utils.simplejson

comment:5 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8124]) Fixed #7131 -- Updated included simplejson code to match the simplejson-1.9.2
release. This should be fully backwards-compatible for people using the public
interfaces.

comment:6 Changed 3 years ago by jacob

  • milestone 1.0 beta deleted

Milestone 1.0 beta deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.