Django

Code

Ticket #1443 (closed: fixed)

Opened 3 years ago

Last modified 4 months ago

No support for dates before 1900

Reported by: akaihola Assigned to: adrian
Milestone: Component: Core framework
Version: SVN Keywords: date 1900 datetime
Cc: rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au Triage Stage: Accepted
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

See logviewer for a short discussion.

#1070 changed date validation so dates before 1900 raise an error. But that's only to avoid problems further down the road.

>>> datetime(1805,12,24).strftime('%Y-%m-%d')
ValueError: year=1805 is before 1900; the datetime strftime() methods require year >= 1900

Using PHP-style date formatting is broken too:

>>> dateformat.format(datetime(1899,4,12), 'Y-m-d')
/usr/local/lib/djangomr2458/django/utils/tzinfo.py in _isdst(self, dt)
---> 50         stamp = time.mktime(tt)
ValueError: year out of range

The problem is deeper in Python (or C, or the OS, or whatever):

>>> time.mktime((1901,12,14,0,0,0,0,0,0))
-2147477992.0
>>> time.mktime((1901,12,13,0,0,0,0,0,0))
OverflowError: mktime argument out of range
>>> time.mktime((1899,12,31,0,0,0,0,0,0))
ValueError: year out of range

Would mx.DateTime be a solution?

>>> mx.DateTime.Date(1543,12,24).strftime('%Y-%m-%d')
'1543-12-24'

I realize that adding another external module dependency is not a Good Thing. Maybe it's worth checking if mx.DateTime? is similar enough to be used as a drop-in replacement for datetime in case it's installed on the system? This way those who need to go back to 19th century or before could just install mx.DateTime and enjoy.

Attachments

datetime_pre_1900.patch (17.0 kB) - added by SmileyChris on 09/23/06 17:14:51.
patch for handling of pre-1900 strftime (with tests)
datetime_pre_1900.2.patch (17.1 kB) - added by SmileyChris on 09/24/06 17:25:24.
updated for recent change to date validator
datetime_pre_1900.2.2.patch (17.1 kB) - added by brian.mabry.edwards@gmail.com on 02/04/07 13:09:28.
update patch for oldforms
date1900.diff (4.2 kB) - added by jbronn on 12/18/07 16:00:40.
datetime patch from Chris Green
1443_updated.diff (4.2 kB) - added by PhiR on 03/19/08 14:48:02.
updated patch against [7322]
datetime_safe.diff (13.6 kB) - added by SmileyChris on 04/07/08 21:11:04.
New and improved!

Change History

03/02/06 08:22:39 changed by adrian

  • status changed from new to closed.
  • resolution set to wontfix.

I'm marking this as a wontfix for now, because adding support for mx.DateTime? would introduce too much complexity.

09/22/06 18:05:54 changed by Gary Wilson <gary.wilson@gmail.com>

  • cc set to gary.wilson@gmail.com.
  • status changed from closed to reopened.
  • version set to SVN.
  • resolution deleted.

I don't think that we necessarily need to use mx.DateTime. At least we could fix the admin interface by not using strftime when saving values to the database. datetime.datetime has no problem creating dates before 1900.

09/23/06 17:14:51 changed by SmileyChris

  • attachment datetime_pre_1900.patch added.

patch for handling of pre-1900 strftime (with tests)

09/24/06 17:06:28 changed by SmileyChris

  • summary changed from no support for dates before 1900 to [patch] no support for dates before 1900.

09/24/06 17:25:24 changed by SmileyChris

  • attachment datetime_pre_1900.2.patch added.

updated for recent change to date validator

01/18/07 02:56:11 changed by Simon G. <dev@simon.net.nz>

  • keywords set to date 1900 datetime.
  • stage changed from Unreviewed to Accepted.

Marked as Accepted as I do think dates prior to 1900 will be an issue in a non-trivial proportion of applications. It's unfortunate that Python drops the ball here, and this adds another layer of complexity but the version here by SmileyChris? doesn't require the mx.DateTime? addition, and appears to handle everything.

Some issues prior to check in though: Are there any potential backwards compat. issues here? Can we directly replace datetime with datetime_pg (& there are datetime calls all over the code)? Is this going to be a performance issue? Alternatively - is it possible to get users who need these date ranges to install mx.DateTime? (e.g. use the python version by default, and override with this/mx.DateTime? on a user-setting basis)?

02/03/07 23:14:45 changed by anonymous

  • cc changed from gary.wilson@gmail.com to gary.wilson@gmail.com, brian.mabry.edwards@gmail.com.

02/04/07 13:09:28 changed by brian.mabry.edwards@gmail.com

  • attachment datetime_pre_1900.2.2.patch added.

update patch for oldforms

05/29/07 21:02:34 changed by anonymous

  • cc changed from gary.wilson@gmail.com, brian.mabry.edwards@gmail.com to rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com.

I'm also having problems with dates before 1900, using the current SVN version and newforms:

Traceback (most recent call last): File "C:\Python25\lib\site-packages\django\core\handlers\base.py" in get_response

  1. response = callback(request, *callback_args, **callback_kwargs)

File "C:\Documents and Settings\Usu�rio\Desktop\projeto1\exemplos\projeto1\cadastro\views.py" in lista_cliente

  1. form.save()

File "C:\Python25\lib\site-packages\django\newforms\models.py" in save

  1. return save_instance(self, instance, fields, fail_message, commit)

File "C:\Python25\lib\site-packages\django\newforms\models.py" in save_instance

  1. instance.save()

File "C:\Python25\lib\site-packages\django\db\models\base.py" in save

  1. db_values = [f.get_db_prep_save(f.pre_save(self, False)) for f in non_pks]

File "C:\Python25\lib\site-packages\django\db\models\fields\init.py" in get_db_prep_save

  1. value = value.strftime('%Y-%m-%d')

ValueError? at /cadastro/clientes/8/ year=1000 is before 1900; the datetime strftime() methods require year >= 1900

09/15/07 11:44:12 changed by Fredrik Lundh <fredrik@pythonware.com>

Since you're just overriding a couple of (broken) portions of the core implementation, I don't really see any performance or compatibility issues here. But that's me.

09/15/07 12:48:51 changed by Fredrik Lundh <fredrik@pythonware.com>

On the other hand, it might be better to just put the robust strptime/strftime implementations in a support library, and make sure that Django uses those for all datetime formatting. I'll check with the sprint IRC folks.

09/16/07 11:04:06 changed by ubernostrum

#3306 is a duplicate but also has a patch.

(follow-up: ↓ 12 ) 09/16/07 11:11:02 changed by Fredrik Lundh <fredrik@pythonware.com>

The #3306 approach is more of a hack, though; it just fakes its way around the C time limitations, in order to support 1900-9999. But that doesn't help; I have dates between 1500 and 1900 in my databases... :-/

No time to dig deeper into this right now, but I'm now convinced that putting Dalke's code in a support library, and making sure to use that whenever the Django core converts dates, is the right solution. Introducing a Django-specific datetime type/module may be elegant, but it's also rather fragile if you have a view that gets data from non-Django sources, and want to render it through Django.

I can check the license with Andrew and adapt Chris' patch, if there's support for that approach.

12/01/07 13:47:20 changed by jdunck

@Fredrik: +1

(in reply to: ↑ 10 ; follow-up: ↓ 13 ) 12/01/07 13:47:49 changed by jbronn

@Fredrik: +1 (I need this support as well)

(in reply to: ↑ 12 ) 12/11/07 14:54:55 changed by Gijs Nijholt <gijs.nijholt@gmail.com>

Replying to jbronn:

@Fredrik: +1 (I need this support as well)

me too.. I'd really like to see this fixed years before 1900 is something that python/django should support by default imho is this platform specific btw? (i've read that it's an issue on osx for example)

thanks

12/18/07 16:00:40 changed by jbronn

  • attachment date1900.diff added.

datetime patch from Chris Green

03/19/08 14:48:02 changed by PhiR

  • attachment 1443_updated.diff added.

updated patch against [7322]

03/19/08 14:49:39 changed by PhiR

  • stage changed from Accepted to Ready for checkin.

tests are working, we're all good.

03/19/08 17:35:59 changed by SmileyChris

  • status changed from reopened to closed.
  • resolution set to fixed.
  • stage changed from Ready for checkin to Design decision needed.

PhiR - I'm actually not that keen on that implementation. I can see some updates I could make to my original patch (wow, that was a long time ago) and still think it's a better way. Could you bring it up on django-dev please?

03/19/08 19:07:50 changed by SmileyChris

  • status changed from closed to reopened.
  • resolution deleted.

(oops)

03/31/08 22:36:45 changed by jetsaredim

  • cc changed from rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com to rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com.

04/02/08 04:34:11 changed by jetsaredim

  • needs_better_patch set to 1.

FYI, I tried to apply both the datetime_pre_1900.2.patch and the datetime_pre_1900.2.2.patch and both generate a bunch of rejected diffs. I tried to hand-patch the rejects from 1900.2, but when I go into the admin site and edit an entry to change the date to before 1900, I get a ValueError? exception saying "year=1786 is before 1900; the datetime strftime() methods require year >= 1900". I'm guessing that its just really hard to patch a moving target and that trunk has just progressed too far from when these patches were generated to be valid now.

I'm not terribly knowledgeable about the internals of django beyond the basics. I don't really have the time to study the code to get the proper expertise needed to truly patch this into current trunk (as it is this simple site I'm working on is taking up most of what little free time I have). I'm certainly willing to apply whatever patch someone can generate (SmileyChris? or whoever) and report back my findings. My code is not terribly sophisticated and I'm really just needing this to be able to store basic date information for some historical data (cemetery records, actually), so if there are specific test that people want performed, that information would be good to know about as well.

Thanks in advance.

04/02/08 04:37:35 changed by jetsaredim

Also, just to be clear. I didn't apply both patches to the same tree (though I'm not entirely sure that would be bad since 1900.2.2 is supposed to just additionally fix oldforms, but I digress...). I re-checked-out trunk between applying 1900.2.2 and then subsequently applying 1900.2 and then proceeding to attempt to hand-patch.

Sorry for the spam, but just wanted to clarify that.

04/07/08 21:11:04 changed by SmileyChris

  • attachment datetime_safe.diff added.

New and improved!

04/07/08 21:14:16 changed by SmileyChris

  • needs_better_patch deleted.
  • summary changed from [patch] no support for dates before 1900 to No support for dates before 1900.

Ok, so my naming sucked - the new module is called datetime_safe. The methods that you'll normally use are just datetime_safe.new_date and datetime_safe.new_datetime.

It should handle all form (old and new) and database saving fine. Django only uses a datetime_safe object when it needs to, so if you're doing your own strftime calls on pre-1900 dates, you'll need to convert the object yourself.

04/07/08 21:14:37 changed by SmileyChris

  • owner changed from nobody to SmileyChris.
  • status changed from reopened to new.

04/07/08 21:23:00 changed by jetsaredim

Just tried the new patch. Applied cleanly to current trunk (rev 7407).

I was able to change the dates in my objects to pre-1900 dates via the admin interface without error. I was also able to retrieve the data via my previously-existing (generic) view code - think object_detail - without error.

If anyone can come up with other tests that should be performed, I'll oblige whatever I can.

04/13/08 21:09:03 changed by Simon Litchfield <simon@quo.com.au>

  • cc changed from rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com to rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au.

The way newforms treats < 1900 dates, with or without mx.DateTime?, is another consideration for this ticket.

See also #7014 for a patch with tests to raise ValidationError? instead of ValueError? for dates before 1900 on newforms DateField/DateTimeFields?.

06/16/08 00:01:57 changed by dan <dan@possumpalace.org>

  • cc changed from rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au to rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au, dan@possumpalace.org.

06/26/08 22:29:44 changed by dan90

patch no longer cleanly applies (looks like a whitespace issue), and it fails unit tests:

Error while importing datetime_safe:  File "./runtests.py", line 130, in django_tests
    mod = load_app(model_label)
  File "/Users/dan/src/django/tests/django/db/models/loading.py", line 72, in load_app
    mod = __import__(app_name, {}, {}, ['models'])
ImportError: No module named datetime_safe

would it help for me to reroll this, or does SmileyChris? have this under control?

06/28/08 04:39:43 changed by SmileyChris

Feel free to re-roll it.

07/17/08 22:13:02 changed by adrian

  • owner changed from SmileyChris to adrian.
  • status changed from new to assigned.
  • stage changed from Design decision needed to Accepted.

07/17/08 22:38:45 changed by dan90

  • cc changed from rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au, dan@possumpalace.org to rafaelbm_sbo@yahoo.com.br, gary.wilson@gmail.com, brian.mabry.edwards@gmail.com, greenwaldjared@gmail.com, simon@quo.com.au.

07/17/08 22:47:28 changed by adrian

  • status changed from assigned to closed.
  • resolution set to fixed.

(In [7946]) Fixed #1443 -- Django's various bits now support dates before 1900. Thanks to SmileyChris?, Chris Green, Fredrik Lundh and others for patches and design help


Add/Change #1443 (No support for dates before 1900)




Change Properties
Action