Better error message for RuntimeError: populate() isn't reentrant

[Thomas Güttler]

We were hit by an exception like this in production environment

Traceback (most recent call last):
  File "foobar-fm/foobar_fm/apache/django_wsgi.py", line 4, in <module>
    application = wsgi.get_handler()
  File "djangotools/djangotools/utils/wsgi.py", line 54, in get_handler
    return get_wsgi_application()
  File "lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
    django.setup()
  File "lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "lib/python2.7/site-packages/django/apps/registry.py", line 78, in populate
    raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant

To debug this, I am missing an very important part:

The Stacktrace of the other one trying to load the config

Here is the code part

            # app_config should be pristine, otherwise the code below won't
            # guarantee that the order matches the order in INSTALLED_APPS.
            if self.app_configs:
                raise RuntimeError("populate() isn't reentrant")

My idea: the first one, who passes above check stores its current stracktrace to a variable.

The if the second fails, and RuntimeError("populate() isn't reentrant") gets executed, the exception can show the stacktrace of the first run.

With this you can debug it. Otherwise you are lost.

What do you think?

[Tim Graham]

I'm not sure. Aymeric, what's your opinion?

[Aymeric Augustin]

I understand the reasoning behind this suggestion, but in my experience, trying to be smart with error handling often backfires, and my gut feeling is that this is too magical.

Besides Python 3's exception chaining solves the problem entirely... Let's say that my motivation for doing something is non-existent, given that I don't plan to ever use Python 2 again.

[Tim Graham]

The rationale makes sense to me.

[Thomas Güttler]

Python 3's exception chaining solves the problem entirely.

I only used Python2 up to now. I guess it is time to upgrade.

Thank you for your feedback.

[Thomas Güttler]

5 years later, running Python 3.8 I come across the same error again. On the Python command line it works, there I see the real error: ModuleNotFoundError: No module named 'psycopg2'

But if I run this via PyCharm "execute file". I get the meaningless error message.

Here is an answer to see the real error message, if you don't see it: ​https://stackoverflow.com/a/55929118/633961 (replace the exection in registry.py with self.app_configs = {})

[Adrian Torres]

Is there a way that we can reopen this ticket? It's still a constant issue and debugging the actual root cause of the issue can prove to be problematic in remote environments, the linked solution in StackOverflow has over 100k views which just goes to show how many people have run into this issue and still do.

Don't know why Aymeric mentions Python 3's exception chaining by default, it clearly does not happen here, at least not when executing commands such as python3 manage.py migrate.

[Claude Paroz]

If you can provide a patch with some concrete proposals that improves things without regressions, I guess this could be reconsidered.

[Bruno Henrique de Paula]

Hey guys, is there a way to fix this? This is a well known issue in the Python community, it's not practical to keep editing the registry.py file and doing the workaround proposed here -> ​https://stackoverflow.com/a/55929118/633961 in order to see the real problem. How could we solve this?

[Aymeric Augustin]

I wasn't aware that this could hide errors in settings. That's a problem indeed.

[Natalia Bidart]

I have read the StackOverflow post, but I'm still unclear on how to reproduce this in a local development environment. I have tried running Django with Gunicorn and forcing errors such as:

• An import error from the admin (no issues; I receive the traceback in the console).
• An ImproperlyConfigured exception raised (no issues here either).

I will close this as needsinfo until someone can provide a small Django project with instructions to reproduce this issue. Ideally, providing a test case would also be helpful. In either case, please feel free to reopen for us to re-triage. Thank you!

[Aymeric Augustin]

Based on the StackOverflow question, it seemed as simple as "add invalid code to settings", but indeed that's not enough. Thank you for double checking. Hopefully someone who hits the problem will report back here instead of on StackOverflow.

[yzupnick]

I just hit this by installing django_allauth[socialaccount], and adding these apps to the INSTALLED_APPS list, but without adding pyjwt, cryptography, or requests to my python environment.

1. "allauth",
2. "allauth.account",
3. "allauth.socialaccount",
4. "allauth.socialaccount.providers.google"

(Also I am building using nix and dream2nix... not sure if that is causing the problem...)