Opened 3 weeks ago

Closed 8 days ago

#37108 closed Cleanup/optimization (fixed)

DjangoJSONEncoder encodes time inconsistently depending on microseconds

Reported by: Roman Donchenko Owned by: SnippyCodes
Component: Core (Serialization) Version: 6.0
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Consider this script:

import json
from datetime import datetime, time, UTC
from django.core.serializers.json import DjangoJSONEncoder

print(json.dumps(datetime.fromtimestamp(0, UTC), cls=DjangoJSONEncoder))
print(json.dumps(datetime.fromtimestamp(0.000001, UTC), cls=DjangoJSONEncoder))

print(json.dumps(time(), cls=DjangoJSONEncoder))
print(json.dumps(time(microsecond=1), cls=DjangoJSONEncoder))

It outputs the following:

"1970-01-01T00:00:00Z"
"1970-01-01T00:00:00.000Z"
"00:00:00"
"00:00:00.000"

In other words, even though the encoded value is logically the same, DjangoJSONEncoder either adds ".000" or doesn't depending on the number of microseconds.

This is not a big problem, but it is mildly annoying. Imagine you save a fixture with dumpdata, load it back, then save again. The first time, the encoder adds ".000", then after loading the number of microseconds is set to 0, so after the second save, the encoder doesn't add ".000". Now there's an unnecessary change in the saved fixture.

Seems like the encoder should either always add ".000" or always omit it.

Change History (9)

comment:1 by Richard Autry, 3 weeks ago

To add some more information after looking into this, this behavior is due to the truncation of microseconds which was added as part of the timezone effort back in 2011:

https://github.com/django/django/commit/9b1cb755a28f020e27d4268c214b25315d4de42e#diff-45e847dda304240b8c519e6b57aa635ae806341986b14e0f17367450f1cbe0e9R47

i.e.

>>> now.isoformat()[:23] + now.isoformat()[26:]
'2026-05-19T15:50:51.060'

When you specify a timestamp of 0.000001, it is be stepped down to the logical ".000" you mentioned. However, whether or not microseconds is included is a function of the standard datetime library and not the actual DjangoJSONEncoder.

i.e.

>>> time().isoformat()
'00:00:00'
>>> time(microsecond=1).isoformat()
'00:00:00.000001'

I'm not sure DjangoJSONEncoder should be responsible for providing default behavior that might override this.

comment:2 by Roman Donchenko, 3 weeks ago

FWIW, nowadays isoformat supports a timespec parameter that can control how the microseconds are represented - including an option to truncate to millisecond precision.

comment:3 by Sarah Boyce, 3 weeks ago

Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

I think we should always omit .000, something like:

  • django/core/serializers/json.py

    diff --git a/django/core/serializers/json.py b/django/core/serializers/json.py
    index b955939e0d..1cf24a1211 100644
    a b class DjangoJSONEncoder(json.JSONEncoder):  
    9090    def default(self, o):
    9191        # See "Date Time String Format" in the ECMA-262 specification.
    9292        if isinstance(o, datetime.datetime):
    93             r = o.isoformat()
    94             if o.microsecond:
    95                 r = r[:23] + r[26:]
     93            r = o.isoformat(sep="T", timespec="milliseconds")
     94            r = r.replace(".000", "")
    9695            if r.endswith("+00:00"):
    9796                r = r.removesuffix("+00:00") + "Z"

This is a slight breaking change and so I would recommend a release note

comment:4 by SnippyCodes, 3 weeks ago

Has patch: set
Owner: set to SnippyCodes
Status: newassigned

comment:5 by Sarah Boyce, 2 weeks ago

Needs tests: set
Patch needs improvement: set

comment:6 by Sarah Boyce, 2 weeks ago

Needs documentation: set
Needs tests: unset
Patch needs improvement: unset

comment:7 by Sarah Boyce, 2 weeks ago

Patch needs improvement: set

comment:8 by Sarah Boyce, 13 days ago

Needs documentation: unset
Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:9 by Sarah Boyce <42296566+sarahboyce@…>, 8 days ago

Resolution: fixed
Status: assignedclosed

In ea9742c:

Fixed #37108 -- Made DjangoJSONEncoder consistently omit .000 microseconds.

Note: See TracTickets for help on using tickets.
Back to Top