AttributeError: '_io.TextIOWrapper' object has no attribute 'getvalue' when setUpTestData raises and tests are run with –parallel –buffer

[Rafael Carlos Soriano Marmol]

Similar issue reported here: https://code.djangoproject.com/ticket/35794

If a test class raises an exception in setUpTestData(), the failure is reported correctly when tests are executed

• in sequence, or
• in parallel without buffering (--parallel only).

However, running the same suite with both --parallel and --buffer causes the test run to abort with an unrelated AttributeError, and the original exception is lost.

Minimal reproducer

# tests/test_issue.py
from django.test import TestCase


class AbortingTest(TestCase):
    @classmethod
    def setUpTestData(cls):
        raise RuntimeError("Boo!")

    def test_pass(self):
        pass


class AnotherAbortingTest(TestCase):
    @classmethod
    def setUpTestData(cls):
        raise RuntimeError("Boo Too!")

    def test_pass(self):
        pass

the errors are reported clearly when running tests in sequence or --parallel without --buffer

======================================================================
ERROR: setUpClass (investment.investment.tests.test_issue.AnotherAbortingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 166, in _handleClassSetUp
    setUpClass()
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/testcases.py", line 1426, in setUpClass
    cls.setUpTestData()
    ^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/investment/investment/tests/test_issue.py", line 16, in setUpTestData
    raise RuntimeError("Boo Too!")
    ^^^^^^^^^^^^^^^^^
RuntimeError: Boo Too!

======================================================================
ERROR: setUpClass (investment.investment.tests.test_issue.AbortingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 166, in _handleClassSetUp
    setUpClass()
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/testcases.py", line 1426, in setUpClass
    cls.setUpTestData()
    ^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/investment/investment/tests/test_issue.py", line 7, in setUpTestData
    raise RuntimeError("Boo!")
    ^^^^^^^^^^^^^^^^^
RuntimeError: Boo!

----------------------------------------------------------------------
Ran 0 tests in 4.715s

FAILED (errors=2)

❌ Fails (parallel with buffer)

python manage.py test tests.test_issue --parallel --buffer

Destroying test database for alias 'default'...
Traceback (most recent call last):
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/manage.py", line 34, in <module>
    main()
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/manage.py", line 29, in main
    execute_from_command_line(sys.argv)
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/commands/test.py", line 24, in run_from_argv
    super().run_from_argv(argv)
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/base.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/commands/test.py", line 63, in handle
    failures = test_runner.run_tests(test_labels)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 1099, in run_tests
    result = self.run_suite(suite)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 1026, in run_suite
    return runner.run(suite)
           ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/runner.py", line 240, in run
    test(result)
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 562, in run
    self.handle_event(result, tests, event)
  File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 586, in handle_event
    handler(test, *args)
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/runner.py", line 101, in addError
    super(TextTestResult, self).addError(test, err)
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 17, in inner
    return method(self, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 116, in addError
    self.errors.append((test, self._exc_info_to_string(err, test)))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 195, in _exc_info_to_string
    output = sys.stdout.getvalue()
             ^^^^^^^^^^^^^^^^^^^
AttributeError: '_io.TextIOWrapper' object has no attribute 'getvalue'

• The traceback contains no reference to the original RuntimeError("Boo!") / RuntimeError("Boo Too!").

Expected behaviour

With --parallel --buffer the test runner should report the same clear RuntimeError stack traces shown when buffering is disabled.

Actual behaviour

The run aborts with an AttributeError deep inside unittest.result, and the real test failures are swallowed.

Additional details

• tblib installed (latest version == 3.1.0)
• logging => logging.StreamHandler
• Removing either --parallel or --buffer avoids the issue.

Similar issue reported here: https://code.djangoproject.com/ticket/35794

Edit:

issue also happens with exceptions inside setUpClass (--parallel --buffer)

class AbortingTest(TestCase):
    @classmethod
    def setUpClass(cls):
        raise RuntimeError("Boo!")

    def test_pass(self):
        pass

[Rafael Carlos Soriano Marmol]

issue also happens with exceptions inside setUpClass (--parallel --buffer)

class AbortingTest(TestCase):
    @classmethod
    def setUpClass(cls):
        raise RuntimeError("Boo!")

    def test_pass(self):
        pass

[Simon Charette]

Managed to reproduce by using unitest.TestCase as well

from unittest import TestCase

class AbortingTest(TestCase):
    @classmethod
    def setUpClass(cls):
        raise RuntimeError("Boo!")

    def test_pass(self):
        pass


class AbortingTest2(TestCase):
    @classmethod
    def setUpClass(cls):
        raise RuntimeError("Boo!")

    def test_pass(self):
        pas

the problem seems due to the fact that the normal unittest ​suite setup makes sure to call TestResutl._setupStdout ​method prior to calling setUpClass which allows its addError ​handler to not crash when it takes its if self.buffer ​branch in _exc_info_to_string.

Given the buffering should happened in spawned worker process and we basically only use the TestResult provided to ParallelTestSuite.run for replicating the events sent from the pool in the main process and there's no test output to capture on the main process (no tests are run in the main process) a potential solution could be to simply disable buffer on the accumulator test result.

django/test/runner.py

@@ -561 +561 @@
         ]
         test_results = pool.imap_unordered(self.run_subsuite.__func__, args)
 
+        # Disable buffering on the local test result that will accumulate
+        # remote suites results as each process will take care of its own
+        # buffering and there's nothing to capture on the main process.
+        result.buffer = False
+
         while True:
             if result.shouldStop:
                 pool.terminate()

[Simon Charette]

Adam, adding you as you worked on adding --buffer support for parallel tests in #31370.

[Shubham Akhilesh Singh]

Fix PR ​https://github.com/django/django/pull/19853