Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#15051 closed (wontfix)

django.test.reorder_suite isn't a stable sort

Reported by: w004dal Owned by: nobody
Component: Testing framework Version: 1.2
Severity: Keywords:
Cc: w004dal@… Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

I have run into cases where tests must run in a specific order (ie. if I'm going to test terminating the web server and see how the web services behind it respond, then I want that test to run last).

The reordering and grouping of tests together in django.test.reorder_suite isn't a "stable sort".

If all of the tests are of the same type, then if I have an array of tests called 'x', then for all permutations of the elements of 'x', stable_sort(x) == x.

Change History (3)

comment:1 Changed 5 years ago by w004dal

  • Cc w004dal@… added
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 5 years ago by russellm

  • Resolution set to wontfix
  • Status changed from new to closed

Unit tests, by definition, can be run in any order. If you have tests that need to run in a particular order, then you don't have unit tests.

comment:3 Changed 5 years ago by w004dal

What about a general wisdom to "fail early and fail fast"? Wouldn't it be a 'nice-to-have' to run them in ascending order of cost?

Note: See TracTickets for help on using tickets.
Back to Top