wsgi.py does not overwrite DJANGO_SETTINGS_MODULE in apache prefork

[schaefer@…]

The wsgi.py Django WSGI application shipped with Django 1.4 contains the following piece of code:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ project_name }}.settings")

(Obviously adapted to the respective project)

Using setdefault() here is wrong. When using a preforked Apache, different WSGI-application can be run consecutively in the same process. If they use different setting modules, the second application will not overwrite the settings module of the first, resulting in the wrong Django application to be shown.

Reproduction

• Create two Django projects
• Host each of them on a different virtual host
• Reload one a number of times to "seed" the processes
• Reload the other one. Notice that (randomly) it shows the *first* project instead of the second

Fix

Replace the aforementioned line with the following:

os.environ["DJANGO_SETTINGS_MODULE"] = "{{ project_name }}.settings"

There is no reason to try and avoid overwriting the environment variable.

[Carl Meyer]

There is a reason to try to avoid overwriting the environment variable - there are quite a few use cases where you might want to set the environment variable yourself externally, if you are using multiple different settings files in your project (a common practice). The value hardcoded in the wsgi file is intended only as a fallback default. In general, hardcoded values should not override values set explicitly in the environment.

Running multiple different Django sites in the same process under preforked Apache is an edge case, and it's easy enough to change the default wsgi file in your project (or use SetEnv in the virtualhost) if you are in that situation.

[Graham.Dumpleton@…]

You cannot use SetEnv to do anything. That does not affect process wide environment variables when using mod_wsgi, only per request WSGI environ which Django doesn't as far as I know pay attention to for this because it is still stuck with this global variable to define what the settings are.

The root trigger for this problem has been documented in mod_wsgi documentation for years:

​http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Application_Environment_Variables

Use of multiple sub interpreters is not an edge case, it is actually the default use case in mod_wsgi in that for every distinct WSGIScriptAlias, each runs in a separate sub interpreter of the same process unless you deliberately work out how to use daemon mode and force each WSGI application into a separate daemon process group. Because the majority of people don't read documentation, they don't usually use daemon mode and so will be hit with this problem as soon as they try and host two Django applications.

[Carl Meyer]

Thanks for clarifying that SetEnv is not useful here. I still think that, since this whole issue is based on the peculiarities of mod_wsgi, it does not provide justification for Django to ship a wsgi.py that overrides an explicitly-set environment variable. It might, however, provide a justification for a patch to the "hosting with mod_wsgi" documentation, explaining why this issue occurs and how one can address it by modifying the default wsgi.py. (Yes, I know you said people don't read documentation. At some point, that's their problem.)

[Graham.Dumpleton@…]

See ticket #18518 for documentation change.

[Graham.Dumpleton@…]

Silly cut and paste. See ticket #18559.

[Russell Keith-Magee]

Discussion on Twitter, and a duplicate as #18559 makes it clear that there is some contention over this wontfix. At the very least, the limitations that exist with prefork need to be documented (as suggested on #18559); a strong recommendation to use daemon mode (obviating the "default config is prefork" problem) might also be appropriate.

[schaefer@…]

What, exactly, would be the use case for setting DJANGO_SETTINGS_MODULE prior to wsgi.py seeing it? SetEnv does not work, so you can not set the environment variable from the Apache config. Where, exactly, would a user set it? It seems to me that setting D_S_M outside of wsgi.py would be the niche case, in which case the user can edit wsgi.py for his needs?

[benjaoming]

mod_wsgi is very important to Django, and an abundance of Django server platforms are running it with Apache prefork. So it's a bit bizarre to introduce a default behaviour that does not support it, nor even mention the incompatibility. I did not expect it and walked right into a trap with hours of debugging, because I run older (working!) WSGI scripts mixed with a broken Django 1.4 auto-generated script, and because there are 30 server processes, I only had these random processes with an incorrect DJANGO_SETTINGS_MODULE.

I suggest a commented-out example of how to directly set DJANGO_SETTINGS_MODULE from wsgi.py so the user is aware and can take action.

# Default behaviour is to respect the outside environment
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')

# Enable the below line if you are running multiple sites sharing the same environment.
# This is necessary with for instance Apache prefork + mod_wsgi.
#os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

I would much prefer having the second line commented in, and the setdefault as an example. I agree with schaefer that setting the environment outside of the script is probably a niche -- since wsgi.py is included in a django project, then it's already a project-specific script!

I could live with a big warning box in the documentation about how to properly configure the environment and an example of how to customize wsgi.py -- but having out-of-the-box good behaviour is my preference :)

[Carl Meyer]

This boils down to an issue of how much Django wants to privilege the peculiar needs of mod_wsgis unusual Python execution environment over other WSGI app servers that execute Python in a more usual way (one environment, one process). If you're using one of the latter (gunicorn, for instance), it's pretty natural to, for instance, have separate "development" and "production" settings modules and choose one by setting DJANGO_SETTINGS_MODULE explicitly, for instance DJANGO_SETTINGS_MODULE=myproject.settings.production gunicorn ..., and it would be quite surprising (even anti-social) for a Django script to presume to unilaterally override DJANGO_SETTINGS_MODULE to a fixed value, disregarding what is set in the environment.

That said, I recognize that there are a lot of mod_wsgi users, and the current behavior interacts quite badly with its default settings. I am still opposed to changing the default Django wsgi script to accommodate thate, but I'd certainly be in favor of a strong documentation fix, even including a comment directly above that line in the file.

[Aymeric Augustin]

FWIW I agree with Carl's last comment. I would be extremely surprised if DJANGO_SETTINGS_MODULE was arbitrarily overridden.

[benjaoming]

I've suggested a commented out example of how things should look with WSGI. Would that work together with a comment in the documentation?

[Carl Meyer <carl@…>]

In 751a7d0c32746dc6774f1b561db523b25365148a:

Fixed #18518 -- Add warning re mod_wsgi and wsgi.py environ handling.

[Carl Meyer]

Ah shoot, I forgot to thank people appropriately in the commit message; my apologies. Thanks to schaefer for the report and Graham and benjaoming for further discussion and clarification. I've closed the ticket with this prominent documentation warning (including a comment in the generated wsgi.py), but that doesn't rule out the possibility of someone opening a new ticket or pull request (or reopening this one) if there's a brilliant idea for how to better support mod_wsgi sub-interpreters directly, without sacrificing the ability to set DJANGO_SETTINGS_MODULE externally.