Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#33891 closed Bug (fixed)

TypeError when trying to run tests in parallel with spawn.

Reported by: Kevin Renskers Owned by: Mariusz Felisiak
Component: Testing framework Version: 4.1
Severity: Release blocker Keywords:
Cc: Carlton Gibson Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

After updating to Django 4.1 I can no longer run my tests in parallel. I get seemingly endless errors in my terminal.

(env) $ ./manage.py test --parallel
Found 124 test(s).
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
System check identified no issues (0 silenced).
Process SpawnPoolWorker-7:
Process SpawnPoolWorker-2:
Process SpawnPoolWorker-6:
Process SpawnPoolWorker-3:
Process SpawnPoolWorker-1:
Process SpawnPoolWorker-8:
Process SpawnPoolWorker-5:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 109, in worker
    initializer(*initargs)
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Users/kevin/Workspace/cn-django/env/lib/python3.10/site-packages/django/test/runner.py", line 420, in _init_worker
    process_setup(*process_setup_args)
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 109, in worker
    initializer(*initargs)
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Users/kevin/Workspace/cn-django/env/lib/python3.10/site-packages/django/test/runner.py", line 420, in _init_worker
    process_setup(*process_setup_args)
Traceback (most recent call last):
TypeError: ParallelTestSuite.process_setup() missing 1 required positional argument: 'self'

That traceback and error then keeps repeating.

If I run the tests just with ./manage.py test, everything works fine.

Change History (4)

comment:1 by Mariusz Felisiak, 2 years ago

Cc: Carlton Gibson added
Owner: changed from nobody to Mariusz Felisiak
Severity: NormalRelease blocker
Status: newassigned
Summary: TypeError when trying to run tests in parallelTypeError when trying to run tests in parallel with spawn.
Triage Stage: UnreviewedAccepted

It looks that I overcomplicated 41c4cb253c137edf5a96b7408ea55d57d6e0602a, sorry. Does the following patch work for you?

  • django/test/runner.py

    diff --git a/django/test/runner.py b/django/test/runner.py
    index 270ddf5b59..24ce766ce1 100644
    a b def _run_subsuite(args):  
    443443    return subsuite_index, result.events
    444444
    445445
     446def _process_setup_stub(*args):
     447    """Stub method to simplify run() implementation."""
     448    pass
     449
     450
    446451class ParallelTestSuite(unittest.TestSuite):
    447452    """
    448453    Run a series of tests in parallel in several processes.
    class ParallelTestSuite(unittest.TestSuite):  
    461466
    462467    # In case someone wants to modify these in a subclass.
    463468    init_worker = _init_worker
     469    process_setup = _process_setup_stub
    464470    process_setup_args = ()
    465471    run_subsuite = _run_subsuite
    466472    runner_class = RemoteTestRunner
    class ParallelTestSuite(unittest.TestSuite):  
    474480        self.serialized_contents = None
    475481        super().__init__()
    476482
    477     def process_setup(self, *args):
    478         """
    479         Stub method to simplify run() implementation. "self" is never actually
    480         passed because a function implementing this method (__func__) is
    481         always used, not the method itself.
    482         """
    483         pass
    484 
    485483    def run(self, result):
    486484        """
    487485        Distribute test cases across workers.

comment:2 by Mariusz Felisiak, 2 years ago

Has patch: set

comment:3 by GitHub <noreply@…>, 2 years ago

Resolution: fixed
Status: assignedclosed

In d4db417c:

Fixed #33891 -- Fixed test command crash when running in parallel using spawn.

Thanks Kevin Renskers for the report.

Regression in 41c4cb253c137edf5a96b7408ea55d57d6e0602a.

comment:4 by Mariusz Felisiak <felisiak.mariusz@…>, 2 years ago

In ab860d4e:

[4.1.x] Fixed #33891 -- Fixed test command crash when running in parallel using spawn.

Thanks Kevin Renskers for the report.

Regression in 41c4cb253c137edf5a96b7408ea55d57d6e0602a.
Backport of d4db417c8e6935222d198925dd4c5ac1ca737909 from main

Note: See TracTickets for help on using tickets.
Back to Top