Opened 3 weeks ago
Closed 8 days ago
#37108 closed Cleanup/optimization (fixed)
DjangoJSONEncoder encodes time inconsistently depending on microseconds
| Reported by: | Roman Donchenko | Owned by: | SnippyCodes |
|---|---|---|---|
| Component: | Core (Serialization) | Version: | 6.0 |
| Severity: | Normal | Keywords: | |
| Cc: | Triage Stage: | Ready for checkin | |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Consider this script:
import json from datetime import datetime, time, UTC from django.core.serializers.json import DjangoJSONEncoder print(json.dumps(datetime.fromtimestamp(0, UTC), cls=DjangoJSONEncoder)) print(json.dumps(datetime.fromtimestamp(0.000001, UTC), cls=DjangoJSONEncoder)) print(json.dumps(time(), cls=DjangoJSONEncoder)) print(json.dumps(time(microsecond=1), cls=DjangoJSONEncoder))
It outputs the following:
"1970-01-01T00:00:00Z" "1970-01-01T00:00:00.000Z" "00:00:00" "00:00:00.000"
In other words, even though the encoded value is logically the same, DjangoJSONEncoder either adds ".000" or doesn't depending on the number of microseconds.
This is not a big problem, but it is mildly annoying. Imagine you save a fixture with dumpdata, load it back, then save again. The first time, the encoder adds ".000", then after loading the number of microseconds is set to 0, so after the second save, the encoder doesn't add ".000". Now there's an unnecessary change in the saved fixture.
Seems like the encoder should either always add ".000" or always omit it.
Change History (9)
comment:1 by , 3 weeks ago
comment:2 by , 3 weeks ago
FWIW, nowadays isoformat supports a timespec parameter that can control how the microseconds are represented - including an option to truncate to millisecond precision.
comment:3 by , 3 weeks ago
| Triage Stage: | Unreviewed → Accepted |
|---|---|
| Type: | Uncategorized → Cleanup/optimization |
I think we should always omit .000, something like:
-
django/core/serializers/json.py
diff --git a/django/core/serializers/json.py b/django/core/serializers/json.py index b955939e0d..1cf24a1211 100644
a b class DjangoJSONEncoder(json.JSONEncoder): 90 90 def default(self, o): 91 91 # See "Date Time String Format" in the ECMA-262 specification. 92 92 if isinstance(o, datetime.datetime): 93 r = o.isoformat() 94 if o.microsecond: 95 r = r[:23] + r[26:] 93 r = o.isoformat(sep="T", timespec="milliseconds") 94 r = r.replace(".000", "") 96 95 if r.endswith("+00:00"): 97 96 r = r.removesuffix("+00:00") + "Z"
This is a slight breaking change and so I would recommend a release note
comment:4 by , 3 weeks ago
| Has patch: | set |
|---|---|
| Owner: | set to |
| Status: | new → assigned |
comment:5 by , 2 weeks ago
| Needs tests: | set |
|---|---|
| Patch needs improvement: | set |
comment:6 by , 2 weeks ago
| Needs documentation: | set |
|---|---|
| Needs tests: | unset |
| Patch needs improvement: | unset |
comment:7 by , 2 weeks ago
| Patch needs improvement: | set |
|---|
comment:8 by , 13 days ago
| Needs documentation: | unset |
|---|---|
| Patch needs improvement: | unset |
| Triage Stage: | Accepted → Ready for checkin |
To add some more information after looking into this, this behavior is due to the truncation of microseconds which was added as part of the timezone effort back in 2011:
https://github.com/django/django/commit/9b1cb755a28f020e27d4268c214b25315d4de42e#diff-45e847dda304240b8c519e6b57aa635ae806341986b14e0f17367450f1cbe0e9R47
i.e.
When you specify a timestamp of 0.000001, it is be stepped down to the logical ".000" you mentioned. However, whether or not microseconds is included is a function of the standard
datetimelibrary and not the actualDjangoJSONEncoder.i.e.
I'm not sure
DjangoJSONEncodershould be responsible for providing default behavior that might override this.