Code

Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#2315 closed defect (fixed)

[patch] Timezone bug in development server using a Windows environment

Reported by: ils_rules@… Owned by: adrian
Component: django-admin.py runserver Version: master
Severity: normal Keywords: timezone datetime
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Django seems to screw up the python time settings. I put TIME_ZONE = 'Europe/Amsterdam' in my current project. The datetime.datetime.now() then returns a different time that if i just run python and run a datetime.datetime.now() (which returns the CORRECT time in Europe/Amsterdam). The difference is exactly one hour.

This difference is VERY annoying, because I can not do synchornization, different imports, etc. :(

Attachments (4)

windows_timezone_fix.patch (1.0 KB) - added by SmileyChris 8 years ago.
Fix for windows timezone problems with development server
bug2315_docs.diff (1.3 KB) - added by Marc Fargas <telenieko@…> 7 years ago.
Docs for this ticket, see comments below.
windows_timezone_fix.2.patch (2.2 KB) - added by SmileyChris 7 years ago.
import moved, docs included
time_zone_doc.patch (1.1 KB) - added by chad@… 7 years ago.
doc tweak: postgres' options don't always correlate

Download all attachments as: .zip

Change History (33)

comment:1 Changed 8 years ago by mtredinnick

  • priority changed from high to normal

Since all Django is doing here is setting the TZ environment variable, this is strange; it should just work. If you are on a Unix-based system (Linux / Solaris / Mac OSX / ...), could you please supply the results of the following. The first and third (at least) should probably work on Windows, too. Not sure how to change the environment-local timezone for Windows, though. Everywhere there is "..." below, please supply the value your system shows.

python
>>> import datetime, os, time
>>> os.environ.get('TZ')
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...
TZ=Europe/Amsterdam python
>>> import datetime
>>> datetime.datetime.now()
...

and finally

./manage.py shell
>>> import datetime, os, time
>>> os.environ['TZ']
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...

In theory, if your machine is set to Europe/Amsterdam time, these should all return the same things for now() and the latter two should return "Europe/Amsterdam" for the timezone, whilst the first one may or may not return a value for the TZ environment variable. The first test will also show what your machine currently things is the difference between its local timezone and UTC.

comment:2 Changed 8 years ago by ils_rules@…

  • Severity changed from critical to normal

Ok this is getting stranger every minute.. I'm running on a windows box, with time settings set to Europe/Amsterdam.

Local time on box is 11:19

running on the 'runserver' in django:

datetime.now(): 2006-07-10 10:19:05.234000
time.localtime(): (2006, 7, 10, 10, 19, 5, 0, 191, 1)
time.gztime(): (2006, 7, 10, 9, 19, 5, 0, 191, 0)

running in apache:

datetime.now(): 2006-07-10 11:20:03.406000
time.localtime(): (2006, 7, 10, 11, 20, 3, 0, 191, 1)
time.gztime(): (2006, 7, 10, 9, 20, 3, 0, 191, 0)

so there is something wrong with the built-in server ??

comment:3 Changed 8 years ago by dan

I'm having the same problem. I'm in Ann Arbor, Michigan. I tried setting the time zone code to both Detroit and Chicago in the settings file. And it just gives me GMT. This is on Windows ME using the test server. But it works correctly on Linux/Apache.

comment:4 Changed 8 years ago by dan

One more thing. I'm using SQLite with Windows and the test server. And PostgreSQL with Linux/Apache.

comment:5 Changed 8 years ago by mtredinnick

Arg! Windows! :-(

This is not going to be possible to solve in a way that isn't a little bit painful. Timezone strings on Windows are a mess. The database isn't as complete as the strings used on Unix variants and there are differences in the common areas. On Linux, it is possible, if you set the wrong string to always get the "non daylight-savings" version and it looks like the same thing is possible on Windows, too.

This link and the associated thread explains the pain with trying to make this portable: http://mail.python.org/pipermail/python-dev/2003-March/034062.html

Messing with the environment via TZ in Windows is probably a doomed exercise. We really need somebody with Windows expertise to write a win32 patch that does The Right Thing(tm) here. None of the main Django developers seem to use Windows, so this is going to be fixed by somebody who is comfortable on the platform scratching their own itch and submitting a patch.

comment:6 Changed 8 years ago by django@…

  • Keywords timezone datetime added
  • Version set to SVN

This problem occurs on MYSQL SQLite3 and postgresql on Windows. the TZ stuff works fine until you import django.db
Once that happens the TZ for the process is set to UTC. Why? no clue, but it is something that django is doing and is not DB specific. It is windows specific.

Have not had time to track down what the %#%%$@ django is doing on windows to cause this but it has caused some major problems for us as it throws off matplotlib (used to generate dynamic graphs) for our date based graphing. As this is effecting the C runtime of the process this even causes problems for SciPy.

This bug needs to be highlited in the documentation! I have talked to 5 people on teh irc channel who were effected by this.
The solution has been to have a special settings variable that the other code uses to skew data (sometimes the querey result data, sometimes lie about the TZ to matplotlib), and set it special if we are windows.

Changed 8 years ago by SmileyChris

Fix for windows timezone problems with development server

comment:7 Changed 8 years ago by SmileyChris

  • Summary changed from Django causes incorrect .now() behaviour to [patch] Timezone bug in development server using a Windows environment

Ok, patch attached disables the settings.TIME_ZONE functionality in Windows, since it doesn't work anyway. As people have reported (and I can confirm) this causes the built-in development server to treat all times as UTC no matter what TIME_ZONE is set to.

On a side note, time.tzset() should probably be set after the os.environ['TZ'] = self.TIME_ZONE line (according to Python documentation) but I didn't want to change anything more in this patch than getting this Windows bug fixed.

comment:8 Changed 8 years ago by SmileyChris

To clarify, this doesn't disable settings.TIME_ZONE in Windows, it just doesn't use it for time and datetime modules (because it didn't work anyway, and caused problems with the development server).

It is still required for the django.db stuff.

comment:9 Changed 8 years ago by SmileyChris

Ok, in a further bout of weirdness (and further testing), it's not even removing the os.environ['TZ'] = self.TIME_ZONE which fixes this problem. Seemingly all that is required to fix it in windows is to import time. Not actually use it - just import it.

Weird.

comment:10 Changed 8 years ago by adrian

  • Component changed from Core framework to django-admin.py runserver

comment:11 Changed 8 years ago by SmileyChris

Ok, here's my findings so far:

  • It's not just in the development server! I just had this happen on my Windows server (running Apache & MySQL).
  • On my development machine (while running Apache & PostgreSQL) I did not encounter the same problem. But it IS a problem with Python.
  • I made a testtime view which just consisted of return HttpResponse(str(datetime.now()))
    • First call to that, time was correct (UTC+13)
    • Called a database method
    • Second call to testtime view was incorrect (UTC)
  • I tried just using import time and it didn't fix it
  • I tried the patch attached to this ticket and it fixed it

comment:12 Changed 7 years ago by angelo@…

hey, just checked to be sure if i have the newest version, but yeah. this patch didnt get into svn so far, so i add my note.

i had the same problem on my windows dev env, and the patch helped. thanks alot.

comment:13 Changed 7 years ago by SmileyChris

  • Needs documentation set
  • Triage Stage changed from Unreviewed to Accepted

Accepted because it's a confirmed bug.

In fact, I'm tempted to mark as ready for checkin since it's not going to have any side effects on other platforms, and Windows just plain doesn't work with os.environ['TZ'].

Probably needs some sort of documentation addition to explain that Windows people should always just set settings.TIME_ZONE to their local time unless they want problems.

Changed 7 years ago by Marc Fargas <telenieko@…>

Docs for this ticket, see comments below.

comment:14 Changed 7 years ago by Marc Fargas <telenieko@…>

  • Needs documentation unset

Docs added, does two things:

  • Adds a note to the settings.py template about that, I didn't like the idea of placing a "windows specific" warning there, but as people can get in trouble with this on windows maybe we can let this one line be there (unless we hope that newcomers will google to reach the settings documentation and read the note there).
  • Adds a note on the corresponding settings.txt section.

As always, I leave the stage untouched for triage review :)

comment:15 Changed 7 years ago by SmileyChris

  • Triage Stage changed from Accepted to Ready for checkin

Thanks Marc. Let's see what the core have to say about this... I say it's a rather common use-case so the Windows note is justified. However personally I'm not sure about having documentation reference a ticket.

comment:16 Changed 7 years ago by Marc Fargas <telenieko@…>

I didn't want to reference the ticket also, but option 2 was to create a "windows.txt" titled "Windows specific issues" which for the newcomer could be translated to "Oh hell, this thing called Django has trouble with windows!!" but it's an option, maybe if there arise more Windows specific issues we'll have to create a windows.txt file hehe.

comment:17 Changed 7 years ago by mir@…

just as a note, #2625 is probably a duplicate of this.

comment:18 follow-up: Changed 7 years ago by mtredinnick

One question before I commit this: how sensitive is the solution to where the "import time" statement is? I would like to have it just before or just after "import os" in order to keep things in a logical order (system imports all together).

I have no problem with referencing a bug in the docs, by the way. The bug report doesn't contain crucial information -- just backup supporting evidence. So I'll leave that reference there; you can still read the docs and make sense of them without referring to the bug report.

comment:19 in reply to: ↑ 18 Changed 7 years ago by Marc Fargas <telenieko@…>

  • Triage Stage changed from Ready for checkin to Design decision needed

Changing status due to last comment by mtredinnick. Could be ready for checking when it has an answer ;)

Changed 7 years ago by SmileyChris

import moved, docs included

comment:20 follow-up: Changed 7 years ago by SmileyChris

  • Triage Stage changed from Design decision needed to Ready for checkin

I tested the new patch, still works fine.

comment:21 Changed 7 years ago by mtredinnick

When applying this and reading everything over, the docs read better without the ticket reference, so I removed it (it's in the code, though, so we can track back why we're doing this funky stuff). Sorry for the change-of-mind, guys.

comment:22 in reply to: ↑ 20 ; follow-ups: Changed 7 years ago by Marc Fargas <telenieko@…>

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

comment:23 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [4487]) Fixed #2315 -- added work around for Windows timezone setting (i.e. we can't do
it). This will work until somebody wants to write some full Win32 timezone changing code for us. Thanks to Marc Fargas and SmileyChris for the combined patch.

comment:24 in reply to: ↑ 22 ; follow-up: Changed 7 years ago by mtredinnick

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

comment:25 in reply to: ↑ 22 Changed 7 years ago by Marc Fargas <telenieko@…>

Forget about my last comment, patch already checked-in! ;)

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

comment:26 in reply to: ↑ 24 Changed 7 years ago by Marc Fargas <telenieko@…>

Replying to mtredinnick:

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

Sorry, didn't spot the new upload ;)

Changed 7 years ago by chad@…

doc tweak: postgres' options don't always correlate

comment:27 Changed 7 years ago by anonymous

  • Resolution fixed deleted
  • Status changed from closed to reopened

I'm seeing similar problems on FreeBSD 6. Turns out that FreeBSD doesn't recognize 'US/Eastern' as an option for TIME_ZONE. I've attached a patch that clarifies this in the settings.py docs.

comment:28 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [4678]) Fixed #2315 -- Clarified that the available PostgreSQL timezone options may
provide more options than are strictly available. This is probably the best we
can do for such a varied area of standardisation.

comment:29 Changed 7 years ago by mtredinnick

I did not apply the docs patch attached here, because /etc/localtime is a binary file and asking people to reverse engineer /usr/share/zoneinfo/ is a bit of a hurdle. We can't really include a full tutorial on setting timezone options, so I've included a warning about the slight variability instead.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.