Opened 15 years ago

Closed 12 years ago

#12408 closed Bug (fixed)

test runner needs to reorder tests more explicitly

Reported by: sebleier@… Owned by: nobody
Component: Testing framework Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Right now the test runner partitions the tests into two bins, either the TestCase bin or the catch all bin. The ordering of the catch all bin then depends on the order in which the tests are seen. The problem is that TransactionTestCase tests sometimes write to the test database without cleaning up after itself and that can sometimes affect a doctest that runs right after it since the doctest doesn't flush the database before it runs. Granted, doctests leave themselves vulnerable to each other since they can modify the database and don't flush before or after they complete. I think adding a little more protection by specifying that doctests should run before TransactionTestCase tests would help a little for the case when people write TransactionTestCase tests that modify data that later doctests are not expecting.

I suspect this won't be an issue once a resolution of #5624 is found.

Attachments (1)

test_ordering.diff (986 bytes ) - added by sebleier@… 15 years ago.

Download all attachments as: .zip

Change History (7)

by sebleier@…, 15 years ago

Attachment: test_ordering.diff added

comment:1 by Eric Holscher, 15 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Matt McClanahan, 14 years ago

Severity: Normal
Type: Bug

comment:3 by Julien Phalip, 14 years ago

Needs documentation: set
Needs tests: set

comment:4 by Aymeric Augustin, 13 years ago

UI/UX: unset

Change UI/UX from NULL to False.

comment:5 by Aymeric Augustin, 13 years ago

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:6 by Ramiro Morales <cramm0@…>, 12 years ago

Resolution: fixed
Status: newclosed

In [f758bdab5eec3e615598948dd5bcf9bb7b910c9d]:

Fixed #18271 -- Changed stage at which TransactionTestCase flushes DB tables.

Previously, the flush was done before the test case execution and now
it is performed after it.

Other changes to the testing infrastructure include:

  • TransactionTestCase now doesn't reset autoincrement sequences either (previous behavior can achieved by using reset_sequences.) With this, no implicit such reset is performed by any of the provided TestCase classes.
  • New ordering of test cases: All unittest tes cases are run first and doctests are run at the end.

THse changes could be backward-incompatible with test cases that relied
on some kind of state being preserved between tests. Please read the
relevant sections of the release notes and testing documentation for
further details.

Thanks Andreas Pelme for the initial patch. Karen Tracey and Anssi
Kääriäinen for the feedback and Anssi for reviewing.

This also fixes #12408.

Note: See TracTickets for help on using tickets.
Back to Top