Opened 10 years ago
Closed 10 years ago
#22821 closed New feature (wontfix)
DjangoJSONEncoder no longer supports simplejson
Reported by: | Owned by: | nobody | |
---|---|---|---|
Component: | Core (Serialization) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
For better or worse, DjangoJSONEncoder
, despite living in the core.serializers
package (and to my knowledge not being public API), is useful to support the few extra datatypes in standard json usage, and doesn't have any negative side affects. So it gets used.
Previously, in Django 1.4, core.serializers.json
relied upon simplejson
, but starting with 1.5 only used json
.
Since then, simplejson
has moved forward and has a slightly different API that prevents using DjangoJSONEncoder
, because it subclasses json.JSONEncoder
rather than simplejson.JSONEncoder
, thus this worked in 1.4:
import simplejson as json from django.core.serializers.json import DjangoJSONEncoder json.dumps("{}", cls=DjangoJSONEncoder)
but in 1.5+, it yields a TypeError, with simplejson==3.5.2
:
Traceback (most recent call last): File "<console>", line 1, in <module> File "/path/to/python2.7/site-packages/simplejson/__init__.py", line 382, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'namedtuple_as_object'
I think it ought to be possible to decouple the implementation such that people wanting to continue using simplejson could still benefit from the implementation of default, by doing something like (untested, pseudo-code):
class RichEncoder(object): def default(self, o): ... return (RichEncoder, self).default(o) class DjangoJSONEncoder(RichEncoder, json.JSONEncoder): pass DateTimeAwareJSONEncoder = DjangoJSONEncoder
meanwhile, userland implementations would replace the json.JSONEncoder
part with simplejson.JSONEncoder
, I guess.
This came up and bit someone in the IRC channel, and I'm reporting it because I think it's supportable, though it'd benefit me nought.
Change History (4)
comment:1 by , 10 years ago
comment:2 by , 10 years ago
Perhaps I'm not understanding the nuances of the problem (it wouldn't be the first time), but I'm not convinced that solving this particular problem is difficult (periphery around it, perhaps)
The following code-bomb works for me using simplejson
versions (the last PATCH releases of every MINOR version):
- 2.1.6
- 2.6.2
- 3.0.9
- 3.3.3
- 3.4.1
- 3.5.2
and the json
module in Python 2.7.6 (I'm guessing/hoping the least Py3K compatible part of this is using xrange
, and print
as a keyword)
import simplejson import json import decimal import datetime # So this is taken straight out of the DjangoJSONEncoder implementation # in master as of 7548aa8ffd46eb6e0f73730d1b2eb515ba581f95 class RichEncoder(object): def default(self, o): # See "Date Time String Format" in the ECMA-262 specification. if isinstance(o, datetime.datetime): r = o.isoformat() if o.microsecond: r = r[:23] + r[26:] if r.endswith('+00:00'): r = r[:-6] + 'Z' return r elif isinstance(o, datetime.date): return o.isoformat() elif isinstance(o, datetime.time): if is_aware(o): raise ValueError("JSON can't represent timezone-aware times.") r = o.isoformat() if o.microsecond: r = r[:12] return r elif isinstance(o, decimal.Decimal): return str(o) else: return super(RichEncoder, self).default(o) # this line is what would end up as DjangoJSONEncoder # and also aliased as DateTimeAwareJSONEncoder class stdlibJSONEncoder(RichEncoder, json.JSONEncoder): pass # this would by necessity be a userland implementation, but is obviously less # code to copy-paste & maintain. class simpleJSONEncoder(RichEncoder, simplejson.JSONEncoder): pass data = {'decimal': decimal.Decimal('4.011'), 'datetime': datetime.datetime.today()} print "Naive:" # naive, stdlib try: json.dumps(data) except TypeError as e: print e # naive, simplejson try: simplejson.dumps(data) except TypeError as e: print e print "\n\nEncoding from Django:" print json.dumps(data, cls=stdlibJSONEncoder) print simplejson.dumps(data, cls=simpleJSONEncoder) # allow decimals to be strings print simplejson.dumps(data, cls=simpleJSONEncoder, use_decimal=False) class CustomJSONEncoder(object): def default(self, o): # contrived example ... if hasattr(o, '__iter__'): return tuple(o) return super(CustomJSONEncoder, self).default(o) # so you need to encode *other* things, but you want the date/decimal handling # both of these would obviously be userland implementations class stdlibCustomJSONEncoder(CustomJSONEncoder, stdlibJSONEncoder): pass class simpleCustomJSONEncoder(CustomJSONEncoder, simpleJSONEncoder): pass # long-winded version. class simpleCustomJSONEncoder2(CustomJSONEncoder, RichEncoder, simplejson.JSONEncoder): pass more_data = {'xrange': xrange(0, 10), 'datetime': datetime.datetime.today()} print "\n\nEncoding from Django, + extra stuff:" print json.dumps(more_data, cls=stdlibCustomJSONEncoder) print simplejson.dumps(more_data, cls=simpleCustomJSONEncoder) # long-winded version, as above print simplejson.dumps(more_data, cls=simpleCustomJSONEncoder2) print "\n\nWithout our custom encoder:" last_data = {'xrange': xrange(0, 10)} try: json.dumps(last_data) except TypeError as e: print e try: simplejson.dumps(last_data) except TypeError as e: print e
None of these exhibit the namedtuple_as_object
issue presented in the ticket itself, because their encoders match their expectations.
If simplejson
is still buggy, 30-40 releases later, as the mentioned ticket indicated, that's fine; that's for an end-user to deal with, or for simplejson
to handle in future releases, but as long as the contract for a JSONEncoder instance's default is just accept o
and return a serializable version of it, I'd think the above would continue to work.
By separating the custom default
object introspection out of the encoder itself, it at least grants the possibility of users getting the best of both Django's code and simplejson's, if that is their desire. The danger of passing a json.JSONEncoder
to simplejson.dumps()
as that ticket describes, is then circumventable, though left as an exercise to the user.
comment:3 by , 10 years ago
If this broke in 1.5 and hasn't been raised since then, I am thinking it may not be worth it. I'm also not sure Django should get further into the serializer business.
Consider the the maintenance burden of adding simplejson to the test dependencies. Also the latest version doesn't support Python 3.2 so we'd have to skip the tests there.
comment:4 by , 10 years ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
See #18023 for why this is hard.
If I remember correctly, the short version is that simplejson broke both backwards and forwards compatibility with itself, making it impossible to be compatible with arbitrary versions of simplejson.