Change DiscoverRunner always to run _FailedTest "tests" first

[Chris Jerdonek]

Currently, DiscoverRunner's default test order is to run _FailedTest objects in a last "catch-all" test-type group, and to order them in no particular way within that last group. Since Django already knows those tests have failed, it seems like it would be better for the user if Django always ran / reported those failed tests first. That would give the user quicker feedback. Also, when --failfast is used, it would result in tests failing faster (namely right away), instead of possibly waiting for many tests to complete before getting to the _FailedTest.

This could be implemented by changing ​reorder_tests() to prepend its list of classes with a bin for _FailedTest objects.

[Carlton Gibson]

Hey Chris — I'm happy to accept this.

With pytest I find the -x --lf combo to be quite effective. (That's not quite what you're proposing here I think, since --lf will select only the tests that previously failed — but "Carry on if all those passed" is probably better behaviour...)

I don't know if folks will have thoughts on the exact behaviour they're after, but I suspect we'll find out in review 😀

Thanks.

[Chris Jerdonek]

PR: ​https://github.com/django/django/pull/14191

[Mariusz Felisiak <felisiak.mariusz@…>]

In dcb06c2c:

Fixed #32591 -- Made DiscoverRunner order _FailedTest objects first.

Failures detected when loading tests are ordered before all of the
above for quicker feedback. This includes things like test modules that
couldn't be found or that couldn't be loaded due to syntax errors.