Better error message for RuntimeError: populate() isn't reentrant

[Thomas Güttler]

We were hit by an exception like this in production environment

Traceback (most recent call last):
  File "foobar-fm/foobar_fm/apache/django_wsgi.py", line 4, in <module>
    application = wsgi.get_handler()
  File "djangotools/djangotools/utils/wsgi.py", line 54, in get_handler
    return get_wsgi_application()
  File "lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
    django.setup()
  File "lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "lib/python2.7/site-packages/django/apps/registry.py", line 78, in populate
    raise RuntimeError("populate() isn't reentrant")
RuntimeError: populate() isn't reentrant

To debug this, I am missing an very important part:

The Stacktrace of the other one trying to load the config

Here is the code part

            # app_config should be pristine, otherwise the code below won't
            # guarantee that the order matches the order in INSTALLED_APPS.
            if self.app_configs:
                raise RuntimeError("populate() isn't reentrant")

My idea: the first one, who passes above check stores its current stracktrace to a variable.

The if the second fails, and RuntimeError("populate() isn't reentrant") gets executed, the exception can show the stacktrace of the first run.

With this you can debug it. Otherwise you are lost.

What do you think?

[Tim Graham]

I'm not sure. Aymeric, what's your opinion?

[Aymeric Augustin]

I understand the reasoning behind this suggestion, but in my experience, trying to be smart with error handling often backfires, and my gut feeling is that this is too magical.

Besides Python 3's exception chaining solves the problem entirely... Let's say that my motivation for doing something is non-existent, given that I don't plan to ever use Python 2 again.

[Tim Graham]

The rationale makes sense to me.

[Thomas Güttler]

Python 3's exception chaining solves the problem entirely.

I only used Python2 up to now. I guess it is time to upgrade.

Thank you for your feedback.

[Thomas Güttler]

5 years later, running Python 3.8 I come across the same error again. On the Python command line it works, there I see the real error: ModuleNotFoundError: No module named 'psycopg2'

But if I run this via PyCharm "execute file". I get the meaningless error message.

Here is an answer to see the real error message, if you don't see it: ​https://stackoverflow.com/a/55929118/633961 (replace the exection in registry.py with self.app_configs = {})

[Adrian Torres]

Is there a way that we can reopen this ticket? It's still a constant issue and debugging the actual root cause of the issue can prove to be problematic in remote environments, the linked solution in StackOverflow has over 100k views which just goes to show how many people have run into this issue and still do.

Don't know why Aymeric mentions Python 3's exception chaining by default, it clearly does not happen here, at least not when executing commands such as python3 manage.py migrate.

[Claude Paroz]

If you can provide a patch with some concrete proposals that improves things without regressions, I guess this could be reconsidered.

[Bruno Henrique de Paula]

Hey guys, is there a way to fix this? This is a well known issue in the Python community, it's not practical to keep editing the registry.py file and doing the workaround proposed here -> ​https://stackoverflow.com/a/55929118/633961 in order to see the real problem. How could we solve this?