Opened 16 years ago

Closed 16 years ago

#10439 closed (fixed)

Test suite side-effects cause dateformat tests to fail

Reported by: Malcolm Tredinnick Owned by: Malcolm Tredinnick
Component: Uncategorized Version: dev
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Running the full tests (as of r10008) shows a failure in the dateformat tests. Running various tests in isolation plus the dateformat test does not generate the failure. So this is some kind of test suite unintended side-effect. Seems to happen under every backend.

The failing test reports this

======================================================================
FAIL: Doctest: regressiontests.dateformat.tests
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/django/test/_doctest.py", line 2180, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for regressiontests.dateformat.tests
  File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 0, in tests

----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 26, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'O') == '+0100'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 30, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'r') == 'Sun, 8 Jul 1979 22:00:00 +0100'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 38, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'T') == 'CET'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 40, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'U') == '300531600'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 52, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'Z') == '3600'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 57, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(summertime, 'O') == '+0200'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 61, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(wintertime, 'O') == '+0100'
Expected:
    True
Got:
    False

I snarfed that output from the buildbot, but I'm seeing the same results locally.

For anybody investigating this: the wintertime/summertime times are set up to cross a daylight savings boundary from 2005 in Copenhagen (we're very tricky when we write our tests for things like that). The results should be (and have been, until recently) independent of the timezone of the machine doing the testing.

Attachments (1)

10439-Dont-tzset-in-every-Settings-isntance-r10008.diff (1.3 KB ) - added by Ramiro Morales 16 years ago.
One attempt at fixing this by modifying django.conf.Settings.init

Download all attachments as: .zip

Change History (11)

comment:1 by Malcolm Tredinnick, 16 years ago

Triage Stage: UnreviewedAccepted

Should mention that I haven't run every combination of tests to work out the side-effect. My laptop is slow enough and this is low-priority enough that I've decided to punt for now.

One way to debug this is to run the tests with -v2, capture the output and notice the order. Then run half of the test preceding the dateformat one, plus dateformat and see if the problem appears, etc. Alternatively, get lucky and work out by eye-inspection which test is causing the problem (or work out which revision introduced the failure and work from there).

comment:2 by Alex Gaynor, 16 years ago

I get a full test suite passage so this is probably somewhat platform dependent. I'm in the default Chicago timezone, so that may be it.

comment:3 by Karen Tracey, 16 years ago

I'm not seeing this either (using Ubuntu & MySQL/InnoDB, US Eastern Daylight Time). Malcolm you say it happens under every backend but buildbot shows green for sqlite so it doesn't seem to happen there? Currently running the test suite using MyISAM since that's what the failing buildbot is running but that'll take a while to finish...

comment:4 by Malcolm Tredinnick, 16 years ago

I saw it locally (and repeatedly) under SQLite, initially. Then I verified it happened under PostgreSQL and say the buildbot failing similarly for MySQL.

Most odd. I'll look a bit further when I have time. Does anything change if you set Django's timezone setting to something east of UTC (UTC + some hours, instead of minus some hours, as you'd get in the US)?

comment:5 by Ramiro Morales, 16 years ago

More information: I'm seeing this when running the test suite agains PostgreSQL 8.1 (that's the version I have access to now, Debian etch). Running bisection shows the problem was introduced in r9994, and this sounds logical and consistent with what Malcolm says above because the regressiontests/app_loading test added in that commit is run before regressiontests/dateformat. It seems the former isn't leaving things in a state expected by the latter (maybe the way it is installing new settings has something to do with this?).

This can be reproduced (by people already seeing it) with ./runtests.py app_loading dateformat

in reply to:  5 comment:6 by Karen Tracey, 16 years ago

Replying to ramiro:

This can be reproduced (by people already seeing it) with ./runtests.py app_loading dateformat

Huh. If I run just those two tests, I see the error (using both sqlite, and mysql with either storage engine). If I run the full suite I can't recreate it. Time zone setting doesn't seem to matter as I can see the error with settings both east and west of GMT if I run just those two tests, and cannot see the error even with a time zone set east of GMT when I run the full suite. Odd.

in reply to:  5 comment:7 by Ramiro Morales, 16 years ago

Replying to ramiro:

regressiontests/app_loading test added in that commit is run before regressiontests/dateformat. It seems the former isn't leaving things in a state expected by the latter (maybe the way it is installing new settings has something to do with this?).

Problem seems to be related to the fact that django.conf.Settings' __init__ touches global state and it's being called to set up the regressiontests/app_loading tests

by Ramiro Morales, 16 years ago

One attempt at fixing this by modifying django.conf.Settings.init

comment:8 by Ramiro Morales, 16 years ago

Has patch: set

comment:9 by Malcolm Tredinnick, 16 years ago

Owner: changed from nobody to Malcolm Tredinnick
Status: newassigned

Awesome, Ramiro! Everything makes sense and it was a recent change that caused it, not something else that changed on my system. Patch looks like roughly the right thing, too.

comment:10 by Malcolm Tredinnick, 16 years ago

Resolution: fixed
Status: assignedclosed

(In [10016]) [1.0.X] Fixed #10439 -- Fixed a subtle test failure caused by r9995.

Thanks to Ramiro Morales for debugging what was going on here.

Backport of r10015 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top