Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#26324 closed Bug (fixed)

DurationField stores certain values incorrectly in sqlite

Reported by: chausner Owned by: nobody
Component: Database layer (models, ORM) Version: 1.8
Severity: Release blocker Keywords: sqlite, DurationField
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by chausner)

When encoding certain timedelta values in a DurationField, saving the object to a sqlite database and then reading it again, DurationField returns None.

class Foo(models.Model):
    bar = models.DurationField()

Foo.objects.create(bar=timedelta(seconds=2.05))
Foo.objects.create(bar=timedelta(seconds=3.05))

print str(Foo.objects.all()[0].bar) # 'None'
print str(Foo.objects.all()[1].bar) # '3.05'

I can reproduce this bug with Django 1.8.5 and 1.9.3 using sqlite as database backend. It does not occur with MySQL, so it seems to be related to how sqlite stores bigints. It renders DurationField essentially useless with sqlite.

Attachments (1)

26324-test.diff (621 bytes) - added by Tim Graham 4 years ago.

Download all attachments as: .zip

Change History (10)

comment:1 Changed 4 years ago by chausner

Description: modified (diff)

comment:2 Changed 4 years ago by Tim Graham

Severity: NormalRelease blocker
Triage Stage: UnreviewedAccepted
Version: 1.91.8

Problem seems to be in parse_duration(). It can't parse the resulting value and so converts it to None.

Accepting as a release blocker due to possible data corruption. Regression test attached.

Changed 4 years ago by Tim Graham

Attachment: 26324-test.diff added

comment:3 Changed 4 years ago by chausner

Not sure if it's of any help but this bug report here looks related: https://github.com/potatolondon/djangae/issues/512.

comment:4 Changed 4 years ago by Neraste

I am currently running into the very same problem (Django 1.8.6).

The get_db_prep_value method of DurationField class always returns a float to the database:

return value.total_seconds() * 1000000

But according to the doc, it has to be stored as a BigInt in SQLite database.

Analyzing Django debug logs shows the SQLite driver always receive float values. Some are harmless (eg: 119019000.0), others are harmfull (eg: 133592000.00000001). The last ones come from the funny side effects of floating point computation:

>>> 133.592 * 1000000
133592000.00000001

I think it can be fixed with:

return int(round(value.total_seconds() * 1000000))
Last edited 4 years ago by Neraste (previous) (diff)

comment:5 Changed 4 years ago by Tim Graham

Has patch: set

comment:6 Changed 4 years ago by Simon Charette

Triage Stage: AcceptedReady for checkin

comment:7 Changed 4 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In 4f0cd0fd:

Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

comment:8 Changed 4 years ago by Tim Graham <timograham@…>

In 5b6c7512:

[1.9.x] Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

Backport of 4f0cd0fd162122da96978b357ac9fc9534529410 from master

comment:9 Changed 4 years ago by Tim Graham <timograham@…>

In c7764ca3:

[1.8.x] Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

Backport of 4f0cd0fd162122da96978b357ac9fc9534529410 from master

Note: See TracTickets for help on using tickets.
Back to Top