#20590 closed Bug (fixed)

Documentation of "test execution order" in 1.6

Reported by: apollo13 Owned by: nobody
Component: Documentation Version: master
Severity: Release blocker Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Current docs (https://docs.djangoproject.com/en/dev/topics/testing/overview/#order-in-which-tests-are-executed) suggest that there is no particular order defined in which tests are executed; this is not true according to https://github.com/django/django/blob/master/django/test/runner.py#L18 which reorders the tests to execute TestCase before TransactionTestCase

Change History (4)

comment:1 Changed 21 months ago by aaugustin

Note that the deprecated DjangoTestSuiteRunner behaves according to the documentation: https://github.com/django/django/blob/master/django/test/simple.py#L253. This could be an unintended side effect of the new test discovery.

However, the behavior of the current code is useful for fixing #20579, and I'd prefer to keep it if possible at all.

comment:2 Changed 21 months ago by carljm

It was not intended for discovery to change test case reordering, and based on my reading of the code there is no change. In both the old and new runners, the call that is made is reorder_suite(suite, (TestCase,)). The only difference is that in the new runner the value (TestCase,) is pulled out into a runner class attribute for easier modification by subclasses, but the default reordering behavior is the same in both runners.

comment:3 Changed 21 months ago by aaugustin

There's a small perversity here, it isn't the same TestCase in the two classes:

  • in the old (deprecated) runner, TestCase is unittest.TestCase (it's explicitly spelled like that).
  • in the new runner, TestCase is from django.test import TestCase.

comment:4 Changed 20 months ago by Andrew Godwin <andrew@…>

  • Resolution set to fixed
  • Status changed from new to closed

In e26b589b8cd2c46c0e6af360abaacdb2fb0af27d:

Fixed #20590: Documented new test case ordering

Note: See TracTickets for help on using tickets.
Back to Top