Opened 2 months ago
Last modified 2 months ago
#36491 new Bug
AttributeError: '_io.TextIOWrapper' object has no attribute 'getvalue' when setUpTestData raises and tests are run with –parallel –buffer
Reported by: | Rafael Carlos Soriano Marmol | Owned by: | |
---|---|---|---|
Component: | Testing framework | Version: | 5.2 |
Severity: | Normal | Keywords: | tests, parallel, buffer |
Cc: | Adam Johnson | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Similar issue reported here: https://code.djangoproject.com/ticket/35794
If a test class raises an exception in setUpTestData(), the failure is reported correctly when tests are executed
- in sequence, or
- in parallel without buffering (--parallel only).
However, running the same suite with both --parallel and --buffer causes the test run to abort with an unrelated AttributeError, and the original exception is lost.
Minimal reproducer
# tests/test_issue.py from django.test import TestCase class AbortingTest(TestCase): @classmethod def setUpTestData(cls): raise RuntimeError("Boo!") def test_pass(self): pass class AnotherAbortingTest(TestCase): @classmethod def setUpTestData(cls): raise RuntimeError("Boo Too!") def test_pass(self): pass
the errors are reported clearly when running tests in sequence or --parallel without --buffer
====================================================================== ERROR: setUpClass (investment.investment.tests.test_issue.AnotherAbortingTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 166, in _handleClassSetUp setUpClass() File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/testcases.py", line 1426, in setUpClass cls.setUpTestData() ^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/investment/investment/tests/test_issue.py", line 16, in setUpTestData raise RuntimeError("Boo Too!") ^^^^^^^^^^^^^^^^^ RuntimeError: Boo Too! ====================================================================== ERROR: setUpClass (investment.investment.tests.test_issue.AbortingTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 166, in _handleClassSetUp setUpClass() File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/testcases.py", line 1426, in setUpClass cls.setUpTestData() ^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/investment/investment/tests/test_issue.py", line 7, in setUpTestData raise RuntimeError("Boo!") ^^^^^^^^^^^^^^^^^ RuntimeError: Boo! ---------------------------------------------------------------------- Ran 0 tests in 4.715s FAILED (errors=2)
❌ Fails (parallel with buffer)
python manage.py test tests.test_issue --parallel --buffer
Destroying test database for alias 'default'... Traceback (most recent call last): File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/manage.py", line 34, in <module> main() File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/Indiana/manage.py", line 29, in main execute_from_command_line(sys.argv) File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line utility.execute() File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/commands/test.py", line 24, in run_from_argv super().run_from_argv(argv) File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/base.py", line 416, in run_from_argv self.execute(*args, **cmd_options) File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/base.py", line 460, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/core/management/commands/test.py", line 63, in handle failures = test_runner.run_tests(test_labels) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 1099, in run_tests result = self.run_suite(suite) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 1026, in run_suite return runner.run(suite) ^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/runner.py", line 240, in run test(result) File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/suite.py", line 84, in __call__ return self.run(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 562, in run self.handle_event(result, tests, event) File "/Users/rafaelsm@backbase.com/dell/home/rsoriano/repositorios/docker-indiana/venvs/indiana_3_12/lib/python3.12/site-packages/django/test/runner.py", line 586, in handle_event handler(test, *args) File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/runner.py", line 101, in addError super(TextTestResult, self).addError(test, err) File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 17, in inner return method(self, *args, **kw) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 116, in addError self.errors.append((test, self._exc_info_to_string(err, test))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.11/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/result.py", line 195, in _exc_info_to_string output = sys.stdout.getvalue() ^^^^^^^^^^^^^^^^^^^ AttributeError: '_io.TextIOWrapper' object has no attribute 'getvalue'
- The traceback contains no reference to the original RuntimeError("Boo!") / RuntimeError("Boo Too!").
Expected behaviour
With --parallel --buffer the test runner should report the same clear RuntimeError stack traces shown when buffering is disabled.
Actual behaviour
The run aborts with an AttributeError deep inside unittest.result, and the real test failures are swallowed.
Additional details
- tblib installed (latest version == 3.1.0)
- logging => logging.StreamHandler
- Removing either --parallel or --buffer avoids the issue.
Similar issue reported here: https://code.djangoproject.com/ticket/35794
Edit:
issue also happens with exceptions inside setUpClass (--parallel --buffer)
class AbortingTest(TestCase): @classmethod def setUpClass(cls): raise RuntimeError("Boo!") def test_pass(self): pass
Attachments (2)
Change History (6)
by , 2 months ago
Attachment: | test_issue.py added |
---|
by , 2 months ago
comment:1 by , 2 months ago
Description: | modified (diff) |
---|
comment:2 by , 2 months ago
comment:3 by , 2 months ago
Triage Stage: | Unreviewed → Accepted |
---|
Managed to reproduce by using unitest.TestCase
as well
from unittest import TestCase class AbortingTest(TestCase): @classmethod def setUpClass(cls): raise RuntimeError("Boo!") def test_pass(self): pass class AbortingTest2(TestCase): @classmethod def setUpClass(cls): raise RuntimeError("Boo!") def test_pass(self): pas
the problem seems due to the fact that the normal unittest
suite setup makes sure to call TestResutl._setupStdout
method prior to calling setUpClass
which allows its addError
handler to not crash when it takes its if self.buffer
branch in _exc_info_to_string
.
Given the buffering should happened in spawned worker process and we basically only use the TestResult
provided to ParallelTestSuite.run
for replicating the events sent from the pool in the main process and there's no test output to capture on the main process (no tests are run in the main process) a potential solution could be to simply disable buffer
on the accumulator test result.
-
django/test/runner.py
diff --git a/django/test/runner.py b/django/test/runner.py index 3e5c319ade..a44180b3b6 100644
a b def run(self, result): 561 561 ] 562 562 test_results = pool.imap_unordered(self.run_subsuite.__func__, args) 563 563 564 # Disable buffering on the local test result that will accumulate 565 # remote suites results as each process will take care of its own 566 # buffering and there's nothing to capture on the main process. 567 result.buffer = False 568 564 569 while True: 565 570 if result.shouldStop: 566 571 pool.terminate()
comment:4 by , 2 months ago
Cc: | added |
---|
Adam, adding you as you worked on adding --buffer
support for parallel tests in #31370.
issue also happens with exceptions inside setUpClass (--parallel --buffer)