#34010 closed Bug (fixed)
Parallel tests don't work with spawn.
| Reported by: | Kevin Renskers | Owned by: | Adam Johnson |
|---|---|---|---|
| Component: | Testing framework | Version: | 4.1 |
| Severity: | Release blocker | Keywords: | |
| Cc: | Carlton Gibson, David Smith | Triage Stage: | Ready for checkin |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
I've upgraded to Django 4.1.1 and parallel tests don't work, but now with a different error as what I got in 4.1 (see #33891).
$ ./manage.py test --parallel
Found 153 test(s).
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
System check identified no issues (0 silenced).
Process SpawnPoolWorker-2:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 114, in worker
task = get()
File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/queues.py", line 367, in get
return _ForkingPickler.loads(res)
File "/Users/kevin/Workspace/cn-django/criticalnotes/auth/test_models.py", line 5, in <module>
from criticalnotes.auth.models import User
File "/Users/kevin/Workspace/cn-django/criticalnotes/auth/models.py", line 10, in <module>
class User(models.Model):
File "/Users/kevin/Workspace/cn-django/env/lib/python3.10/site-packages/django/db/models/base.py", line 127, in __new__
app_config = apps.get_containing_app_config(module)
File "/Users/kevin/Workspace/cn-django/env/lib/python3.10/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
self.check_apps_ready()
File "/Users/kevin/Workspace/cn-django/env/lib/python3.10/site-packages/django/apps/registry.py", line 138, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
(That error then repeat 26 times in my case).
If I run the tests just with ./manage.py test, everything works fine.
Attachments (1)
Change History (16)
comment:1 follow-up: 2 Changed 9 months ago by
| Cc: | Carlton Gibson added |
|---|
comment:2 Changed 9 months ago by
Replying to Mariusz Felisiak:
I'm confused you confirmed that d4db417c8e6935222d198925dd4c5ac1ca737909 works for you. Can you provide a sample project?
I'm not sure why that seemed to work. Maybe I made a mistake updating Django to the commit with the fix, or maybe I ran the tests without the parallel flag 🤔
I can email you a closed source project, if it won't be shared with other people. Just let me know where to send the project.
comment:3 follow-up: 4 Changed 9 months ago by
| Resolution: | → needsinfo |
|---|---|
| Status: | new → closed |
| Summary: | Parallel tests don't work → Parallel tests don't work with spawn. |
We also confirmed that #33891 is fixed so it must be something different. You can send a sample project to me or Carlton to emails from GitHub profiles. Thanks.
comment:4 Changed 9 months ago by
Replying to Mariusz Felisiak:
We also confirmed that #33891 is fixed so it must be something different. You can send a sample project to me or Carlton to emails from GitHub profiles. Thanks.
Yep I sent a project to Carlton on the 15th. Not sure why this ticket got closed though, since I have a reproducible problem running parallel tests?
comment:5 Changed 9 months ago by
Yep I sent a project to Carlton on the 15th. Not sure why this ticket got closed though, since I have a reproducible problem running parallel tests?
Don't worry, we will reopen the ticket as soon as Carlton confirms it's an issue in Django. Django Con Europe starts today so it might take a little longer but it's on the radar.
comment:6 Changed 9 months ago by
| Resolution: | needsinfo |
|---|---|
| Status: | closed → new |
comment:7 Changed 9 months ago by
Yes, confirming that. I’m currently reducing the example project to work out exactly what the issue is. Thanks.
comment:8 Changed 8 months ago by
| Cc: | David Smith added |
|---|---|
| Severity: | Normal → Release blocker |
| Triage Stage: | Unreviewed → Accepted |
OK, thanks for the report Kevin. We're not setting the ParallelTestSuite.process_setup and process_setup_args correctly, instead being the _process_setup_stub no-op. We'll need an implementation for that.
(If your — i.e. my — test project doesn't come out of partition_suite_by_case() the right way you don't trigger this. I shall upload a minimal project later.)
This was an issue in the original patch 3b3f38b3b09b0f2373e51406ecb8c9c45d36aebc, rather than the other referenced issues.
Changed 8 months ago by
| Attachment: | parallel_tests.zip added |
|---|
Minimal reproduce. Install Django and manage.py test --parallel on macOS (and presumably Windows)
comment:9 Changed 8 months ago by
| Has patch: | set |
|---|---|
| Owner: | changed from nobody to Carlton Gibson |
| Status: | new → assigned |
comment:11 Changed 8 months ago by
| Triage Stage: | Accepted → Ready for checkin |
|---|
I'm confused you confirmed that d4db417c8e6935222d198925dd4c5ac1ca737909 works for you. Can you provide a sample project?