Code

Opened 5 years ago

Closed 5 years ago

#10439 closed (fixed)

Test suite side-effects cause dateformat tests to fail

Reported by: mtredinnick Owned by: mtredinnick
Component: Uncategorized Version: master
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Running the full tests (as of r10008) shows a failure in the dateformat tests. Running various tests in isolation plus the dateformat test does not generate the failure. So this is some kind of test suite unintended side-effect. Seems to happen under every backend.

The failing test reports this

======================================================================
FAIL: Doctest: regressiontests.dateformat.tests
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/django/test/_doctest.py", line 2180, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for regressiontests.dateformat.tests
  File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 0, in tests

----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 26, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'O') == '+0100'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 30, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'r') == 'Sun, 8 Jul 1979 22:00:00 +0100'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 38, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'T') == 'CET'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 40, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'U') == '300531600'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 52, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(my_birthday, 'Z') == '3600'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 57, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(summertime, 'O') == '+0200'
Expected:
    True
Got:
    False
----------------------------------------------------------------------
File "/var/buildbot/slave/parts/ubuntu-8.04-python2.4-mysql5.0/django-trunk_ubuntu-8.04-python2.4-mysql5.0/
build/tests/regressiontests/dateformat/tests.py", line 61, in regressiontests.dateformat.tests
Failed example:
    no_tz or format(wintertime, 'O') == '+0100'
Expected:
    True
Got:
    False

I snarfed that output from the buildbot, but I'm seeing the same results locally.

For anybody investigating this: the wintertime/summertime times are set up to cross a daylight savings boundary from 2005 in Copenhagen (we're very tricky when we write our tests for things like that). The results should be (and have been, until recently) independent of the timezone of the machine doing the testing.

Attachments (1)

10439-Dont-tzset-in-every-Settings-isntance-r10008.diff (1.3 KB) - added by ramiro 5 years ago.
One attempt at fixing this by modifying django.conf.Settings.init

Download all attachments as: .zip

Change History (11)

comment:1 Changed 5 years ago by mtredinnick

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

Should mention that I haven't run every combination of tests to work out the side-effect. My laptop is slow enough and this is low-priority enough that I've decided to punt for now.

One way to debug this is to run the tests with -v2, capture the output and notice the order. Then run half of the test preceding the dateformat one, plus dateformat and see if the problem appears, etc. Alternatively, get lucky and work out by eye-inspection which test is causing the problem (or work out which revision introduced the failure and work from there).

comment:2 Changed 5 years ago by Alex

I get a full test suite passage so this is probably somewhat platform dependent. I'm in the default Chicago timezone, so that may be it.

comment:3 Changed 5 years ago by kmtracey

I'm not seeing this either (using Ubuntu & MySQL/InnoDB, US Eastern Daylight Time). Malcolm you say it happens under every backend but buildbot shows green for sqlite so it doesn't seem to happen there? Currently running the test suite using MyISAM since that's what the failing buildbot is running but that'll take a while to finish...

comment:4 Changed 5 years ago by mtredinnick

I saw it locally (and repeatedly) under SQLite, initially. Then I verified it happened under PostgreSQL and say the buildbot failing similarly for MySQL.

Most odd. I'll look a bit further when I have time. Does anything change if you set Django's timezone setting to something east of UTC (UTC + some hours, instead of minus some hours, as you'd get in the US)?

comment:5 follow-ups: Changed 5 years ago by ramiro

More information: I'm seeing this when running the test suite agains PostgreSQL 8.1 (that's the version I have access to now, Debian etch). Running bisection shows the problem was introduced in r9994, and this sounds logical and consistent with what Malcolm says above because the regressiontests/app_loading test added in that commit is run before regressiontests/dateformat. It seems the former isn't leaving things in a state expected by the latter (maybe the way it is installing new settings has something to do with this?).

This can be reproduced (by people already seeing it) with ./runtests.py app_loading dateformat

comment:6 in reply to: ↑ 5 Changed 5 years ago by kmtracey

Replying to ramiro:

This can be reproduced (by people already seeing it) with ./runtests.py app_loading dateformat

Huh. If I run just those two tests, I see the error (using both sqlite, and mysql with either storage engine). If I run the full suite I can't recreate it. Time zone setting doesn't seem to matter as I can see the error with settings both east and west of GMT if I run just those two tests, and cannot see the error even with a time zone set east of GMT when I run the full suite. Odd.

comment:7 in reply to: ↑ 5 Changed 5 years ago by ramiro

Replying to ramiro:

regressiontests/app_loading test added in that commit is run before regressiontests/dateformat. It seems the former isn't leaving things in a state expected by the latter (maybe the way it is installing new settings has something to do with this?).

Problem seems to be related to the fact that django.conf.Settings' __init__ touches global state and it's being called to set up the regressiontests/app_loading tests

Changed 5 years ago by ramiro

One attempt at fixing this by modifying django.conf.Settings.init

comment:8 Changed 5 years ago by ramiro

  • Has patch set

comment:9 Changed 5 years ago by mtredinnick

  • Owner changed from nobody to mtredinnick
  • Status changed from new to assigned

Awesome, Ramiro! Everything makes sense and it was a recent change that caused it, not something else that changed on my system. Patch looks like roughly the right thing, too.

comment:10 Changed 5 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [10016]) [1.0.X] Fixed #10439 -- Fixed a subtle test failure caused by r9995.

Thanks to Ramiro Morales for debugging what was going on here.

Backport of r10015 from trunk.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.