Opened 3 years ago

Closed 22 months ago

Last modified 22 months ago

#18766 closed Cleanup/optimization (fixed)

Models with datetime fields fail to save when date is below 1900 and USE_TZ is True

Reported by: void Owned by: aaugustin
Component: Documentation Version: master
Severity: Normal Keywords: timezone, tz
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have model with datetime field in it. When I save this model in admin site and the filled date is below 1900, I am getting the following traceback:

  File "/Users/voidus/Documents/workspace/django-trunk/django/contrib/admin/options.py", line 709, in save_model
    obj.save()
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/base.py", line 524, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/base.py", line 595, in save_base
    rows = manager.using(using).filter(pk=pk_val)._update(values)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/query.py", line 573, in _update
    return query.get_compiler(self.db).execute_sql(None)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 991, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 813, in execute_sql
    sql, params = self.as_sql()
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 956, in as_sql
    val = field.get_db_prep_save(val, connection=self.connection)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 303, in get_db_prep_save
    prepared=False)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 834, in get_db_prep_value
    value = self.get_prep_value(value)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 820, in get_prep_value
    if value is not None and settings.USE_TZ and timezone.is_naive(value):
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 271, in is_naive
    return value.tzinfo is None or value.tzinfo.utcoffset(value) is None
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 71, in utcoffset
    if self._isdst(dt):
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 89, in _isdst
    stamp = _time.mktime(tt)
ValueError: year out of range

I think that is because utcoffset and some other functions use _isdst function, which calls time.mktime and it have some limitation on the date range it can handle.

time module can handle dates in the more narrow range (1969-2038) then datetime module (1-9999).

Change History (20)

comment:1 Changed 3 years ago by void

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

As wikipedia article says, DST was not used before at least 1916. So, maybe simple check like

if dt.year < 1916:
    return False

will work

comment:2 Changed 3 years ago by aaugustin

  • Owner changed from nobody to aaugustin

comment:3 Changed 3 years ago by void

The other corner case is dates after 2038. While simple check for < 1916 will work for dates before 1900, as there were no DST at that time, after 2038 we will probably still have DST. 2038 is not near, but I can imagine a calendar application with events in future.

comment:4 Changed 3 years ago by aaugustin

Indeed, this is a limitation of django.utils.tzinfo.LocalTimezone.

Could you try installing pytz (as strongly recommeded by the docs)? If that fixes the problem, I'd lean towards a documentation fix rather than extending the huge hack that is LocalTimezone.

(I know that it works for me, I'd like to know if it works for you.)

Note that dates after 2038 are handled by returning incorrect data rather than crashing. I deeply dislike that, but I can't change it easily, for backwards-compatibility reasons.

Last edited 3 years ago by aaugustin (previous) (diff)

comment:5 Changed 3 years ago by aaugustin

See also #17262.

comment:6 Changed 3 years ago by aaugustin

  • Resolution set to needsinfo
  • Status changed from new to closed

Waiting to know if installing pytz resolves the problem...

comment:7 Changed 3 years ago by anonymous

Yes, installing pytz resolves the problem.

comment:8 Changed 3 years ago by void

Sorry, that was me, was logged out somehow.

comment:9 Changed 3 years ago by aaugustin

  • Component changed from Uncategorized to Documentation
  • Resolution needsinfo deleted
  • Status changed from closed to reopened
  • Triage Stage changed from Unreviewed to Accepted
  • Type changed from Bug to Cleanup/optimization

I'm accepting the ticket primarily as a documentation issue. This is related to #17262. I still have to decide how to deal with this.

comment:10 Changed 3 years ago by void

Maybe it would be OK to require pytz installed if user set USE_I18N = True in settings.py?
Or maybe we can even set a hard dependency? I think it's now commonly consented that django should make more use of pip - at least localflavor will be separated, etc. So, maybe pytz would be the first thing to require?

comment:11 Changed 3 years ago by aaugustin

USE_TZ = True is in the default settings files (for a good reason) and we want to keep Django usable without having to learn pip / virtualenv first (also for a good reason). I went to great lengths to ensure that pytz wouldn't be a mandatory.

I think it's more promising to raise a more explicit error in this case, ie. "value outside of supported range, please install pytz".

I

comment:12 Changed 3 years ago by void

Yeah, it may be a good solution. But may this behaviour be considered as a regression? Imagine, you have some values outside of supported range in database, then upgrade django one version upper, set USE_TZ=True, as docs insist. And live being happy until someday you see in logs such message about pytz.

comment:13 Changed 2 years ago by aaugustin

  • Status changed from reopened to new

comment:14 Changed 2 years ago by dracos

Just to back up void's comment, I have just upgraded a site from Django 1.1 to 1.5. One of the few things that only bit me once it went live (and thanks as ever for the good documentation on upgrades and consistency :) ) was that I have some very old datetimes in the live database (back to the 17th century) that were working fine, that then started to throw "ValueError: year out of range" 500s due to this issue. It does indeed resolve if pytz is installed (or if USE_TZ is set back to False, which I've done for now).

comment:15 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

  • Resolution set to fixed
  • Status changed from new to closed

In ded11aa620e5b38c78ff92e691a459d1977a2539:

Fixed #18766 -- Pointed to pytz when LocalTimezone fails.

Thanks void for the report.

comment:16 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

In 7c31e195db963ad1ebb28265e721097000fa84d1:

[1.6.x] Fixed #18766 -- Pointed to pytz when LocalTimezone fails.

Thanks void for the report.

Backport of ded11aa6 from master.

comment:17 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

In 1a1e14786aa35b4d80b46be747b5128800cf0f97:

Hardened the test introduced in ded11aa6. Refs #18766.

Inputs acceptable to time.mktime are platform-dependent.

comment:18 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

In 0035a0ce2ec243ff9255157117c20caf41a61284:

[1.6.x] Hardened the test introduced in ded11aa6. Refs #18766.

Inputs acceptable to time.mktime are platform-dependent.

Backport of 1a1e1478 from master.

comment:19 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

In c687bf0620d67e0579302515904ffe169581d036:

Further hardening. Refs #18766.

comment:20 Changed 22 months ago by Aymeric Augustin <aymeric.augustin@…>

In b085e7c3033d49e4d7525bacb1f34340730918a8:

[1.6.x] Further hardening. Refs #18766.

Backport of c687bf0 from master.

Note: See TracTickets for help on using tickets.
Back to Top