No support for dates before 1900

[Antti Kaihola]

See logviewer for a ​short discussion.

#1070 changed date validation so dates before 1900 raise an error. But that's only to avoid problems further down the road.

>>> datetime(1805,12,24).strftime('%Y-%m-%d')
ValueError: year=1805 is before 1900; the datetime strftime() methods require year >= 1900

Using PHP-style date formatting is broken too:

>>> dateformat.format(datetime(1899,4,12), 'Y-m-d')
/usr/local/lib/djangomr2458/django/utils/tzinfo.py in _isdst(self, dt)
---> 50         stamp = time.mktime(tt)
ValueError: year out of range

The problem is deeper in Python (or C, or the OS, or whatever):

>>> time.mktime((1901,12,14,0,0,0,0,0,0))
-2147477992.0
>>> time.mktime((1901,12,13,0,0,0,0,0,0))
OverflowError: mktime argument out of range
>>> time.mktime((1899,12,31,0,0,0,0,0,0))
ValueError: year out of range

Would mx.DateTime be a solution?

>>> mx.DateTime.Date(1543,12,24).strftime('%Y-%m-%d')
'1543-12-24'

I realize that adding another external module dependency is not a Good Thing. Maybe it's worth checking if mx.DateTime is similar enough to be used as a drop-in replacement for datetime in case it's installed on the system? This way those who need to go back to 19th century or before could just install mx.DateTime and enjoy.

[Adrian Holovaty]

I'm marking this as a wontfix for now, because adding support for mx.DateTime would introduce too much complexity.

[Gary Wilson <gary.wilson@…>]

I don't think that we necessarily need to use mx.DateTime. At least we could fix the admin interface by not using strftime when saving values to the database. datetime.datetime has no problem creating dates before 1900.

[Chris Beaven]

patch for handling of pre-1900 strftime (with tests)

[Chris Beaven]

updated for recent change to date validator

[Simon G. <dev@…>]

Marked as Accepted as I do think dates prior to 1900 will be an issue in a non-trivial proportion of applications. It's unfortunate that Python drops the ball here, and this adds another layer of complexity but the version here by SmileyChris doesn't require the mx.DateTime addition, and appears to handle everything.

Some issues prior to check in though:
Are there any potential backwards compat. issues here?
Can we directly replace datetime with datetime_pg (& there are datetime calls all over the code)?
Is this going to be a performance issue?
Alternatively - is it possible to get users who need these date ranges to install mx.DateTime (e.g. use the python version by default, and override with this/mx.DateTime on a user-setting basis)?

[brian.mabry.edwards@…]

update patch for oldforms

[anonymous]

I'm also having problems with dates before 1900, using the current SVN version and newforms:

Traceback (most recent call last):
File "C:\Python25\lib\site-packages\django\core\handlers\base.py" in get_response
  77. response = callback(request, *callback_args, **callback_kwargs)
File "C:\Documents and Settings\Usu�rio\Desktop\projeto1\exemplos\projeto1\cadastro\views.py" in lista_cliente
  37. form.save()
File "C:\Python25\lib\site-packages\django\newforms\models.py" in save
  56. return save_instance(self, instance, fields, fail_message, commit)
File "C:\Python25\lib\site-packages\django\newforms\models.py" in save_instance
  35. instance.save()
File "C:\Python25\lib\site-packages\django\db\models\base.py" in save
  217. db_values = [f.get_db_prep_save(f.pre_save(self, False)) for f in non_pks]
File "C:\Python25\lib\site-packages\django\db\models\fields\__init__.py" in get_db_prep_save
  496. value = value.strftime('%Y-%m-%d')

  ValueError at /cadastro/clientes/8/
  year=1000 is before 1900; the datetime strftime() methods require year >= 1900 

[Fredrik Lundh <fredrik@…>]

Since you're just overriding a couple of (broken) portions of the core implementation, I don't really see any performance or compatibility issues here. But that's me.

[Fredrik Lundh <fredrik@…>]

On the other hand, it might be better to just put the robust strptime/strftime implementations in a support library, and make sure that Django uses those for all datetime formatting. I'll check with the sprint IRC folks.

[James Bennett]

#3306 is a duplicate but also has a patch.

[Fredrik Lundh <fredrik@…>]

The #3306 approach is more of a hack, though; it just fakes its way around the C time limitations, in order to support 1900-9999. But that doesn't help; I have dates between 1500 and 1900 in my databases... :-/

No time to dig deeper into this right now, but I'm now convinced that putting Dalke's code in a support library, and making sure to use that whenever the Django core converts dates, is the right solution. Introducing a Django-specific datetime type/module may be elegant, but it's also rather fragile if you have a view that gets data from non-Django sources, and want to render it through Django.

I can check the license with Andrew and adapt Chris' patch, if there's support for that approach.

[Jeremy Dunck]

@Fredrik: +1

[jbronn]

@Fredrik: +1 (I need this support as well)

[Gijs Nijholt <gijs.nijholt@…>]

Replying to jbronn:

@Fredrik: +1 (I need this support as well)

me too.. I'd really like to see this fixed
years before 1900 is something that python/django should support by default imho
is this platform specific btw? (i've read that it's an issue on osx for example)

thanks

[jbronn]

datetime patch from Chris Green

[Philippe Raoult]

updated patch against [7322]

[Philippe Raoult]

tests are working, we're all good.

[Chris Beaven]

PhiR - I'm actually not that keen on that implementation. I can see some updates I could make to my original patch (wow, that was a long time ago) and still think it's a better way. Could you bring it up on django-dev please?

[Chris Beaven]

(oops)

[jetsaredim]

FYI, I tried to apply both the datetime_pre_1900.2.patch and the datetime_pre_1900.2.2.patch and both generate a bunch of rejected diffs. I tried to hand-patch the rejects from 1900.2, but when I go into the admin site and edit an entry to change the date to before 1900, I get a ValueError exception saying "year=1786 is before 1900; the datetime strftime() methods require year >= 1900". I'm guessing that its just really hard to patch a moving target and that trunk has just progressed too far from when these patches were generated to be valid now.

I'm not terribly knowledgeable about the internals of django beyond the basics. I don't really have the time to study the code to get the proper expertise needed to truly patch this into current trunk (as it is this simple site I'm working on is taking up most of what little free time I have). I'm certainly willing to apply whatever patch someone can generate (SmileyChris or whoever) and report back my findings. My code is not terribly sophisticated and I'm really just needing this to be able to store basic date information for some historical data (cemetery records, actually), so if there are specific test that people want performed, that information would be good to know about as well.

Thanks in advance.

[jetsaredim]

Also, just to be clear. I didn't apply both patches to the same tree (though I'm not entirely sure that would be bad since 1900.2.2 is supposed to just additionally fix oldforms, but I digress...). I re-checked-out trunk between applying 1900.2.2 and then subsequently applying 1900.2 and then proceeding to attempt to hand-patch.

Sorry for the spam, but just wanted to clarify that.

[Chris Beaven]

New and improved!

[Chris Beaven]

Ok, so my naming sucked - the new module is called datetime_safe. The methods that you'll normally use are just datetime_safe.new_date and datetime_safe.new_datetime.

It should handle all form (old and new) and database saving fine. Django only uses a datetime_safe object when it needs to, so if you're doing your own strftime calls on pre-1900 dates, you'll need to convert the object yourself.

[jetsaredim]

Just tried the new patch. Applied cleanly to current trunk (rev 7407).

I was able to change the dates in my objects to pre-1900 dates via the admin interface without error. I was also able to retrieve the data via my previously-existing (generic) view code - think object_detail - without error.

If anyone can come up with other tests that should be performed, I'll oblige whatever I can.

[Simon Litchfield <simon@…>]

The way newforms treats < 1900 dates, with or without mx.DateTime, is another consideration for this ticket.

See also #7014 for a patch with tests to raise ValidationError instead of ValueError for dates before 1900 on newforms DateField/DateTimeFields.

[dan]

patch no longer cleanly applies (looks like a whitespace issue), and it fails unit tests:

Error while importing datetime_safe:  File "./runtests.py", line 130, in django_tests
    mod = load_app(model_label)
  File "/Users/dan/src/django/tests/django/db/models/loading.py", line 72, in load_app
    mod = __import__(app_name, {}, {}, ['models'])
ImportError: No module named datetime_safe

would it help for me to reroll this, or does SmileyChris have this under control?

[Chris Beaven]

Feel free to re-roll it.

[Adrian Holovaty]

(In [7946]) Fixed #1443 -- Django's various bits now support dates before 1900. Thanks to SmileyChris, Chris Green, Fredrik Lundh and others for patches and design help