id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
32140	`startTest` result hook not called until after tests already started when `--parallel` used	Bob Whitelock	nobody	"Given a custom Django test runner that uses a custom result class, like this:

{{{
from unittest.runner import TextTestRunner, TextTestResult

from django.test.runner import DiscoverRunner


class CustomResult(TextTestResult):

    def startTest(self, test):
        print(f""startTest called for {test}"")
        super().startTest(test)


class CustomRunner(TextTestRunner):
    resultclass = CustomResult


class CustomDjangoRunner(DiscoverRunner):
    test_runner = CustomRunner
}}}

And some tests, like this:

{{{
from django.test import TestCase


class FirstTest(TestCase):
    def test_first(self):
        print(""Running test_first"")


class SecondTest(TestCase):
    def test_second(self):
        print(""Running test_second"")
}}}

When the tests are run without `--parallel`, the expected behaviour occurs, and the `startTest` hook is called just before starting the test:

{{{
$ ./manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
startTest called for test_first (parallel_test.tests.FirstTest)
Running test_first
.startTest called for test_second (parallel_test.tests.SecondTest)
Running test_second
.
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK
Destroying test database for alias 'default'...
}}}

When the tests are run with `--parallel`, the `startTest` hook appears to be called after the test has already been run:

{{{
$ ./manage.py test --parallel
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
System check identified no issues (0 silenced).
Running test_first
Running test_second
startTest called for test_second (parallel_test.tests.SecondTest)
.startTest called for test_first (parallel_test.tests.FirstTest)
.
----------------------------------------------------------------------
Ran 2 tests in 0.017s

OK
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
}}}

This is surprising, as I would expect the hook to a. be called in a consistent place when running a test, whether this is in parallel or serial, and b. be called before/as the test is started. This inconsistent behaviour can lead to issues when writing custom Django test runners, as people use this hook to run some code just as each test is started - for example see https://github.com/xmlrunner/unittest-xml-reporting/issues/229 and https://stackoverflow.com/a/39758653/2620402.

See https://github.com/bobwhitelock/django_parallel_test_issue_repro for a minimal repro of the issue including the code samples above.

I've confirmed this issue is present in both Django 3.1 and 2.2.

Happy to help with fixing this, but I'd need some pointers as to where to start. Thanks!"	Bug	closed	Testing framework	3.1	Normal	needsinfo	startTest parallel test runner	Adam Johnson	Unreviewed	0	0	0	0	0	0