id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
32114	Workaround for subtest issue with parallel test runner	Jordan Ephron	nobody	"Django's ParallelTestSuite requires that all test cases be
pickleable. When a SubTest fails with an exception, it's
pickled and shipped back to the main process. Unfortunately,
Django's TestCases aren't always pickleable (for instance,
they may contain an instance of django.test.Client which can't
be cleanly pickled if an exception was raised by a view)

When the following test case is run in a parallel environment, 
the test runner is unable to serialize the exception:
{{{
#!div style=""font-size: 80%""
{{{#!python
class SubtestBugTestCase(TestCase):
    def test_subtest_bug(self):
        with self.subTest(""I am the subtest""):
            self.not_pickleable = lambda: 0
            self.assertTrue(False)

# AttributeError: Can't pickle local object 'SubtestBugTestCase.test_subtest_bug.<locals>.<lambda>'
}}}
}}}


Luckily, Django's DiscoverRunner only actually cares about a
small subset of the fields on TestCase, and those fields are 
all of pickleable types. (This is also true of the PyCharm
test runner, which we use)

We work around the subtest issue by wrapping up the subtest 
as follows:
{{{
#!div style=""font-size: 80%""
{{{#!python
class TestCaseDTO:
    def __init__(self, test):
        self._m_id = test.id()
        self._m_str = str(test)
        self._m_shortDescription = test.shortDescription()
        if hasattr(test, ""test_case""):
            self.test_case = TestCaseDTO(test.test_case)
        if hasattr(test, ""_subDescription""):
            self._m_subDescription = test._subDescription()

    def _subDescription(self):
        """"""conforming to _SubTest""""""
        return self._m_subDescription

    def id(self):
        return self._m_id

    def shortDescription(self):
        return self._m_shortDescription

    def __str__(self):
        return self._m_str


class SubtestSerializingRemoteTestResult(RemoteTestResult):
    def addSubTest(self, test, subtest, err):
        subtest = TestCaseDTO(subtest)
        super().addSubTest(test, subtest, err)


class OurRemoteTestRunner(RemoteTestRunner):
    resultclass = SubtestSerializingRemoteTestResult


class OurTestSuite(ParallelTestSuite):
    runner_class = OurRemoteTestRunner


class OurDiscoverRunner(DiscoverRunner):
    parallel_test_suite = OurTestSuite
}}}
}}}

Now, to be fair, it's possible that some test runners do care
about fields that aren't captured by TestCaseDTO, so I'm 
not sure whether this is truly a universal solution, but this 
issue was a significant impediment to parallelizing our tests.

Would it be worthwhile to have this behavior in Django core?

"	Uncategorized	new	Uncategorized	3.1	Normal		Test subtest parallel		Unreviewed	0	0	0	0	0	0