Document changes to WSGI application loading sequence in 1.7

[yscumc]

After upgrading Django from 1.6 to 1.7, I found that there were some undocumented differences in how the wsgi loading is handled.

I had the following in Django 1.6 in my wsgi.py:

from django.core.wsgi import get_wsgi_application
_application = get_wsgi_application()

# Update DJANGO_SETTINGS_MODULE os env variable from internal Apache env variable, set by "SetEnv" in httpd.conf
def application(environ, start_response):
    if 'DJANGO_SETTINGS_MODULE' in environ:
        os.environ['DJANGO_SETTINGS_MODULE'] = environ['DJANGO_SETTINGS_MODULE']

    return _application(environ, start_response)

This will get the env from the Apache conf and set it before a request to make sure that the proper settings file is loaded.

However, in Django 1.7, this broke because django.setup() is now run as part of django.core.wsgi.get_wsgi_application() and this causes the settings file to be loaded before the first application() call (first request).

Previously, the loading of the settings file seems to happen with the first application() call, django.core.wsgi.get_wsgi_application()().

Debugging this was actually more difficult than it seems. I initially just got a 400 BAD REQUEST error after upgrading, with no errors in the logs. It took a long time to realize my my custom settings file wasn't being loaded and so ALLOWED_HOSTS weren't being loaded either.

After finding out it was a changed behavior in wsgi loading that was causing the problem, fixing the problem was straightforward, but I recommend updating the documentation with this change in Django 1.7 so that others wouldn't run into this issue.

Here's the fixed version that also works with 1.7 (and earlier) if someone is interested:

# Update DJANGO_SETTINGS_MODULE os env variable from internal Apache env variable, set by "SetEnv" in httpd.conf
def application(environ, start_response):
    if 'DJANGO_SETTINGS_MODULE' in environ:
        os.environ['DJANGO_SETTINGS_MODULE'] = environ['DJANGO_SETTINGS_MODULE']

    # Only attempt to execute get_wsgi_application() once
    if not application._app:
        application._app = get_wsgi_application()

    return application._app(environ, start_response)
application._app = None

[Tim Graham]

If you could propose a patch, I'll be happy to review it.

[Paul Rentschler]

I ran into this problem as well and created a pull request (​https://github.com/django/django/pull/5271) to address the issue. Although I realize I didn't address the original submitter's request for documentation about how wsgi loading is handled, I did provide changes to the documentation at explains and provides a working solution that is also backwards compatible for doing what the original submitter was trying to do along with showing how to pass arbitrary variables from the Apache configuration to Django.

[Tim Graham <timograham@…>]

In 47016d4:

Fixed #25203 -- Documented how to pass Apache environment variables to Django.

[Tim Graham <timograham@…>]

In c877349:

[1.8.x] Fixed #25203 -- Documented how to pass Apache environment variables to Django.

Backport of 47016d4322574860f90431e1c87d19f7a1f778c6 from master

[Carl Meyer]

Commented on the PR here: ​https://github.com/django/django/commit/47016d4322574860f90431e1c87d19f7a1f778c6#commitcomment-13192876

I'm not convinced this is a technique that the Django documentation should promote.

[Carl Meyer]

Graham Dumpleton (author of mod_wsgi) has confirmed that this is not a technique we should be recommending in our docs: ​https://twitter.com/GrahamDumpleton/status/642714837466906624

He says once he is back from vacation he will recommend better alternatives that we could document.

I think we should roll back this patch and wait for his recommendation.

[Tim Graham <timograham@…>]

In 83ea3bc7:

Reverted "Fixed #25203 -- Documented how to pass Apache environment variables to Django."

As discussed on the ticket, this isn't a pattern we should recommend.

This reverts commit 47016d4322574860f90431e1c87d19f7a1f778c6.

[Tim Graham <timograham@…>]

In d0d2567:

[1.8.x] Reverted "Fixed #25203 -- Documented how to pass Apache environment variables to Django."

As discussed on the ticket, this isn't a pattern we should recommend.

This reverts commit c8773493b62083b9ca5476a6040737f5cca79944.

[Graham Dumpleton]

Where you need a process level global variable inherited from some key set in the Apache configuration, what you are best to do is ensure you are running the Django application in a mod_wsgi daemon process group of its own (forced to use main interpreter application group), and then base any in code decisions on the name of the daemon process group you used in the Apache configuration.

For example, you could even have the name of the Django settings module be the name of the daemon process group.

WSGIDaemonProcess mysite.production.settings
WSGIScriptAlias / /some/path/wsgi.py process-group=mysite.production.settings application-group=%{GLOBAL}

In the wsgi.py file you can then say:

try:
    import mod_wsgi
    os.environ['DJANGO_SETTINGS_MODULE'] = mod_wsgi.process_group
except ImportError:
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.development.settings')

This can be placed before any other module imports as it is dependent on information available at import time and not at the time of a request.

[Jon Dufresne]

FWIW, this unexpected change has occurred in a previous Django release as well. See ticket #21486. The response in that ticket was to avoid importing settings.py during a call to get_wsgi_application() such that the original pattern would continue to work. I'm not sure if this is still feasible today.

I appreciate the expertise and knowledge Graham Dumpleton brings to the table. But it feels a tad limiting that one can't freely pass an arbitrary amount of environment variables from Apache to Django. Some deployment strategies, such as Twelve-Factor App (Not Django specific ​http://12factor.net/config) specifically recommend configuring web apps through environment variables. Is using environment variables really just not possible with mod_wsgi?

If my application has exactly one variable to read, it is difficult to make multiple decisions on this. For example, my application loads settings slightly differently in production vs debug/development mode. This is to ease configuration for other, perhaps new, developers. With just one available variable that is always set it becomes difficult to do this.