Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#2315 closed defect (fixed)

[patch] Timezone bug in development server using a Windows environment

Reported by: ils_rules@… Owned by: Adrian Holovaty
Component: django-admin.py runserver Version: master
Severity: normal Keywords: timezone datetime
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Django seems to screw up the python time settings. I put TIME_ZONE = 'Europe/Amsterdam' in my current project. The datetime.datetime.now() then returns a different time that if i just run python and run a datetime.datetime.now() (which returns the CORRECT time in Europe/Amsterdam). The difference is exactly one hour.

This difference is VERY annoying, because I can not do synchornization, different imports, etc. :(

Attachments (4)

windows_timezone_fix.patch (1.0 KB) - added by Chris Beaven 10 years ago.
Fix for windows timezone problems with development server
bug2315_docs.diff (1.3 KB) - added by Marc Fargas <telenieko@…> 10 years ago.
Docs for this ticket, see comments below.
windows_timezone_fix.2.patch (2.2 KB) - added by Chris Beaven 10 years ago.
import moved, docs included
time_zone_doc.patch (1.1 KB) - added by chad@… 10 years ago.
doc tweak: postgres' options don't always correlate

Download all attachments as: .zip

Change History (33)

comment:1 Changed 10 years ago by Malcolm Tredinnick

priority: highnormal

Since all Django is doing here is setting the TZ environment variable, this is strange; it should just work. If you are on a Unix-based system (Linux / Solaris / Mac OSX / ...), could you please supply the results of the following. The first and third (at least) should probably work on Windows, too. Not sure how to change the environment-local timezone for Windows, though. Everywhere there is "..." below, please supply the value your system shows.

python
>>> import datetime, os, time
>>> os.environ.get('TZ')
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...
TZ=Europe/Amsterdam python
>>> import datetime
>>> datetime.datetime.now()
...

and finally

./manage.py shell
>>> import datetime, os, time
>>> os.environ['TZ']
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...

In theory, if your machine is set to Europe/Amsterdam time, these should all return the same things for now() and the latter two should return "Europe/Amsterdam" for the timezone, whilst the first one may or may not return a value for the TZ environment variable. The first test will also show what your machine currently things is the difference between its local timezone and UTC.

comment:2 Changed 10 years ago by ils_rules@…

Severity: criticalnormal

Ok this is getting stranger every minute.. I'm running on a windows box, with time settings set to Europe/Amsterdam.

Local time on box is 11:19

running on the 'runserver' in django:

datetime.now(): 2006-07-10 10:19:05.234000
time.localtime(): (2006, 7, 10, 10, 19, 5, 0, 191, 1)
time.gztime(): (2006, 7, 10, 9, 19, 5, 0, 191, 0)

running in apache:

datetime.now(): 2006-07-10 11:20:03.406000
time.localtime(): (2006, 7, 10, 11, 20, 3, 0, 191, 1)
time.gztime(): (2006, 7, 10, 9, 20, 3, 0, 191, 0)

so there is something wrong with the built-in server ??

comment:3 Changed 10 years ago by dan

I'm having the same problem. I'm in Ann Arbor, Michigan. I tried setting the time zone code to both Detroit and Chicago in the settings file. And it just gives me GMT. This is on Windows ME using the test server. But it works correctly on Linux/Apache.

comment:4 Changed 10 years ago by dan

One more thing. I'm using SQLite with Windows and the test server. And PostgreSQL with Linux/Apache.

comment:5 Changed 10 years ago by Malcolm Tredinnick

Arg! Windows! :-(

This is not going to be possible to solve in a way that isn't a little bit painful. Timezone strings on Windows are a mess. The database isn't as complete as the strings used on Unix variants and there are differences in the common areas. On Linux, it is possible, if you set the wrong string to always get the "non daylight-savings" version and it looks like the same thing is possible on Windows, too.

This link and the associated thread explains the pain with trying to make this portable: http://mail.python.org/pipermail/python-dev/2003-March/034062.html

Messing with the environment via TZ in Windows is probably a doomed exercise. We really need somebody with Windows expertise to write a win32 patch that does The Right Thing(tm) here. None of the main Django developers seem to use Windows, so this is going to be fixed by somebody who is comfortable on the platform scratching their own itch and submitting a patch.

comment:6 Changed 10 years ago by django@…

Keywords: timezone datetime added
Version: SVN

This problem occurs on MYSQL SQLite3 and postgresql on Windows. the TZ stuff works fine until you import django.db
Once that happens the TZ for the process is set to UTC. Why? no clue, but it is something that django is doing and is not DB specific. It is windows specific.

Have not had time to track down what the %#%%$@ django is doing on windows to cause this but it has caused some major problems for us as it throws off matplotlib (used to generate dynamic graphs) for our date based graphing. As this is effecting the C runtime of the process this even causes problems for SciPy.

This bug needs to be highlited in the documentation! I have talked to 5 people on teh irc channel who were effected by this.
The solution has been to have a special settings variable that the other code uses to skew data (sometimes the querey result data, sometimes lie about the TZ to matplotlib), and set it special if we are windows.

Changed 10 years ago by Chris Beaven

Attachment: windows_timezone_fix.patch added

Fix for windows timezone problems with development server

comment:7 Changed 10 years ago by Chris Beaven

Summary: Django causes incorrect .now() behaviour[patch] Timezone bug in development server using a Windows environment

Ok, patch attached disables the settings.TIME_ZONE functionality in Windows, since it doesn't work anyway. As people have reported (and I can confirm) this causes the built-in development server to treat all times as UTC no matter what TIME_ZONE is set to.

On a side note, time.tzset() should probably be set after the os.environ['TZ'] = self.TIME_ZONE line (according to Python documentation) but I didn't want to change anything more in this patch than getting this Windows bug fixed.

comment:8 Changed 10 years ago by Chris Beaven

To clarify, this doesn't disable settings.TIME_ZONE in Windows, it just doesn't use it for time and datetime modules (because it didn't work anyway, and caused problems with the development server).

It is still required for the django.db stuff.

comment:9 Changed 10 years ago by Chris Beaven

Ok, in a further bout of weirdness (and further testing), it's not even removing the os.environ['TZ'] = self.TIME_ZONE which fixes this problem. Seemingly all that is required to fix it in windows is to import time. Not actually use it - just import it.

Weird.

comment:10 Changed 10 years ago by Adrian Holovaty

Component: Core frameworkdjango-admin.py runserver

comment:11 Changed 10 years ago by Chris Beaven

Ok, here's my findings so far:

  • It's not just in the development server! I just had this happen on my Windows server (running Apache & MySQL).
  • On my development machine (while running Apache & PostgreSQL) I did not encounter the same problem. But it IS a problem with Python.
  • I made a testtime view which just consisted of return HttpResponse(str(datetime.now()))
    • First call to that, time was correct (UTC+13)
    • Called a database method
    • Second call to testtime view was incorrect (UTC)
  • I tried just using import time and it didn't fix it
  • I tried the patch attached to this ticket and it fixed it

comment:12 Changed 10 years ago by angelo@…

hey, just checked to be sure if i have the newest version, but yeah. this patch didnt get into svn so far, so i add my note.

i had the same problem on my windows dev env, and the patch helped. thanks alot.

comment:13 Changed 10 years ago by Chris Beaven

Needs documentation: set
Triage Stage: UnreviewedAccepted

Accepted because it's a confirmed bug.

In fact, I'm tempted to mark as ready for checkin since it's not going to have any side effects on other platforms, and Windows just plain doesn't work with os.environ['TZ'].

Probably needs some sort of documentation addition to explain that Windows people should always just set settings.TIME_ZONE to their local time unless they want problems.

Changed 10 years ago by Marc Fargas <telenieko@…>

Attachment: bug2315_docs.diff added

Docs for this ticket, see comments below.

comment:14 Changed 10 years ago by Marc Fargas <telenieko@…>

Needs documentation: unset

Docs added, does two things:

  • Adds a note to the settings.py template about that, I didn't like the idea of placing a "windows specific" warning there, but as people can get in trouble with this on windows maybe we can let this one line be there (unless we hope that newcomers will google to reach the settings documentation and read the note there).
  • Adds a note on the corresponding settings.txt section.

As always, I leave the stage untouched for triage review :)

comment:15 Changed 10 years ago by Chris Beaven

Triage Stage: AcceptedReady for checkin

Thanks Marc. Let's see what the core have to say about this... I say it's a rather common use-case so the Windows note is justified. However personally I'm not sure about having documentation reference a ticket.

comment:16 Changed 10 years ago by Marc Fargas <telenieko@…>

I didn't want to reference the ticket also, but option 2 was to create a "windows.txt" titled "Windows specific issues" which for the newcomer could be translated to "Oh hell, this thing called Django has trouble with windows!!" but it's an option, maybe if there arise more Windows specific issues we'll have to create a windows.txt file hehe.

comment:17 Changed 10 years ago by mir@…

just as a note, #2625 is probably a duplicate of this.

comment:18 Changed 10 years ago by Malcolm Tredinnick

One question before I commit this: how sensitive is the solution to where the "import time" statement is? I would like to have it just before or just after "import os" in order to keep things in a logical order (system imports all together).

I have no problem with referencing a bug in the docs, by the way. The bug report doesn't contain crucial information -- just backup supporting evidence. So I'll leave that reference there; you can still read the docs and make sense of them without referring to the bug report.

comment:19 in reply to:  18 Changed 10 years ago by Marc Fargas <telenieko@…>

Triage Stage: Ready for checkinDesign decision needed

Changing status due to last comment by mtredinnick. Could be ready for checking when it has an answer ;)

Changed 10 years ago by Chris Beaven

import moved, docs included

comment:20 Changed 10 years ago by Chris Beaven

Triage Stage: Design decision neededReady for checkin

I tested the new patch, still works fine.

comment:21 Changed 10 years ago by Malcolm Tredinnick

When applying this and reading everything over, the docs read better without the ticket reference, so I removed it (it's in the code, though, so we can track back why we're doing this funky stuff). Sorry for the change-of-mind, guys.

comment:22 in reply to:  20 ; Changed 10 years ago by Marc Fargas <telenieko@…>

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

comment:23 Changed 10 years ago by Malcolm Tredinnick

Resolution: fixed
Status: newclosed

(In [4487]) Fixed #2315 -- added work around for Windows timezone setting (i.e. we can't do
it). This will work until somebody wants to write some full Win32 timezone changing code for us. Thanks to Marc Fargas and SmileyChris for the combined patch.

comment:24 in reply to:  22 ; Changed 10 years ago by Malcolm Tredinnick

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

comment:25 in reply to:  22 Changed 10 years ago by Marc Fargas <telenieko@…>

Forget about my last comment, patch already checked-in! ;)

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

comment:26 in reply to:  24 Changed 10 years ago by Marc Fargas <telenieko@…>

Replying to mtredinnick:

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

Sorry, didn't spot the new upload ;)

Changed 10 years ago by chad@…

Attachment: time_zone_doc.patch added

doc tweak: postgres' options don't always correlate

comment:27 Changed 10 years ago by anonymous

Resolution: fixed
Status: closedreopened

I'm seeing similar problems on FreeBSD 6. Turns out that FreeBSD doesn't recognize 'US/Eastern' as an option for TIME_ZONE. I've attached a patch that clarifies this in the settings.py docs.

comment:28 Changed 10 years ago by Malcolm Tredinnick

Resolution: fixed
Status: reopenedclosed

(In [4678]) Fixed #2315 -- Clarified that the available PostgreSQL timezone options may
provide more options than are strictly available. This is probably the best we
can do for such a varied area of standardisation.

comment:29 Changed 10 years ago by Malcolm Tredinnick

I did not apply the docs patch attached here, because /etc/localtime is a binary file and asking people to reverse engineer /usr/share/zoneinfo/ is a bit of a hurdle. We can't really include a full tutorial on setting timezone options, so I've included a warning about the slight variability instead.

Note: See TracTickets for help on using tickets.
Back to Top