id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
35849	ParallelTestSuite reports error occurred in arbitrary test instead of setUpClass	David Winiecki	David Winiecki	"When running tests in parallel, if an error occurs in a test suite (`TestCase`) before the first test method runs, for example in `setUpClass`, then the shell output incorrectly states that the error occurred in an arbitrary test method, rather than, for example, `setUpClass`.

This is a problem because a user reading the test output may not notice the real error location (for example, `setUpClass`) in the traceback and may not run the other tests in the test suite.

This is especially problematic if automation parses the error location to automatically re-run tests (to address flaky, intermittently failing tests), because the other tests in the suite may never be run.

Repro:

1. Create a Django project for testing:

Instructions to create the project yourself are below. **Or you can use this existing project** in the `bugdemo/` directory at https://github.com/dcki/django/blob/demo_report_test_error_bug/bugdemo/README.md

To create the project yourself:

A. Install Python.
B. Create a virtualenv and activate it: `mkdir my_project && cd my_project && python3 -m venv .venv && . .venv/bin/activate`
C. Install Django and create a Django project: `pip install django && django-admin startproject mysite .`
D. Create a tests directory and an `__init__.py` file inside: `mkdir mysite/tests && touch mysite/tests/__init__.py`
E. Install tblib: `pip install tblib`
F. Add these tests to the project:

`mysite/tests/test_things.py`

{{{
from django.test import SimpleTestCase


class AlwaysFailTest(SimpleTestCase):
    @classmethod
    def setUpClass(cls) -> None:
        super().setUpClass()
        try:
            raise Exception('Intentional error')
        except:
            super().tearDownClass()
            raise

    def test_should_pass_a(self) -> None:
        pass

    def test_should_pass_b(self) -> None:
        pass


# Exists so that, when an attempt is made to run tests in parallel, and at
# least two TestCases are specified, then tests actually run in parallel.
# (As of this writing, Django runs tests serially if there is only one test
# suite to run, even if `--parallel=2` is specified.)
class AlwaysPassTest(SimpleTestCase):
    def test_should_pass_a(self) -> None:
        pass

    def test_should_pass_b(self) -> None:
        pass
}}}

2. Run the tests: `python3 manage.py test --parallel=2`

Expected output:

{{{
Found 4 test(s).
System check identified no issues (0 silenced).
E..
======================================================================
ERROR: setUpClass (mysite.tests.test_things.AlwaysFailTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ""/Users/dwiniecki/.pyenv/versions/3.11.9/lib/python3.11/unittest/suite.py"", line 166, in _handleClassSetUp
    setUpClass()
  File ""/Users/dwiniecki/src/other/dcki-django/bugdemo2/mysite/tests/test_things.py"", line 9, in setUpClass
    raise Exception('Intentional error')
    ^^^^^^^^^^^^^^^^^
Exception: Intentional error

----------------------------------------------------------------------
Ran 2 tests in 0.214s

FAILED (errors=1)
}}}

Actual output:

{{{
Found 4 test(s).
System check identified no issues (0 silenced).
E..
======================================================================
ERROR: test_should_pass_b (mysite.tests.test_things.AlwaysFailTest.test_should_pass_b)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ""/Users/dwiniecki/.pyenv/versions/3.11.9/lib/python3.11/unittest/suite.py"", line 166, in _handleClassSetUp
    setUpClass()
  File ""/Users/dwiniecki/src/other/dcki-django/bugdemo/mysite/tests/test_things.py"", line 9, in setUpClass
    raise Exception('Intentional error')
    ^^^^^^^^^^^^^^^^^
Exception: Intentional error

----------------------------------------------------------------------
Ran 2 tests in 0.221s

FAILED (errors=1)
}}}"	Cleanup/optimization	closed	Testing framework	dev	Normal	fixed		David Winiecki Adam Johnson Jacob Walls	Accepted	1	0	0	0	0	0