mod_wsgi authentication/authorization docs result in extra memory usage

[Graham Dumpleton]

The documentation at:

• ​https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/apache-auth/

uses the examples:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

Although these will work, they cause the Django application code to be loaded more than once into the same process. That is, you end up with an instance in one sub interpreter which handles web request, and another instance in another sub interpreter which handles just authentication and authorisation.

Ideally you do no want two copies as that simply increases overall memory usage.

The original mod_wsgi documentation does point to this in the statements:

By default the auth providers are executed in context of first interpreter created by Python, ie., '%{GLOBAL}' and always in the Apache child processes, never in a daemon process. The interpreter can be overridden using the 'application-group' option to the script directive. The namespace for authentication groups is shared with that for application groups defined by WSGIApplicationGroup.

Because the auth provider is always run in the Apache child processes and never in the context of a mod_wsgi daemon process, if the authentication check is making use of the internals of some Python web framework, it is recommended that the application using that web framework also be run in embedded mode and the same application group. This is the case as the Python web frameworks often bring in a huge amount of code even if using only one small part of them. This will result in a lot of memory being used in the Apache child processes just to support the auth provider.

To combat this double of memory usage the mod_wsgi documentation gives the example of:

WSGIAuthUserScript /usr/local/django/mysite/apache/auth.wsgi application-group=django

It is this use of the application-group argument for WSGIAuthUserScript and WSGIAuthGroupScript that the Django documentation doesn't use.

The Django examples therefore should really use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    Require group secret-agents
    Require valid-user
</Location>

On the presumption that there is only the one Django application, it may actually be better to turn that around and force the Django application to run in the main interpreter anyway. This is better as some third party extension modules for Python do not work properly in sub interpreters.

Thus may be better to use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

[Aymeric Augustin]

I believe we should provide a copy-paste-proof example for the simple case of "first time I'm touching Django and Apache, how do I run my test project?" and shell out to mod_wsgi's own docs for anything more complicated.

For that use case, we can assume that there's only one Django application.

We just need:

• to make sure our documentation stays internally consistent — we don't want the mod_wsgi auth docs to have subtle differences with the mod_wsgi main docs.
• to add a warning about running multiple Django projets in the same server — which Apache's lovely handling of env variables makes hard anyway — and leave that part of the explaination up to mod_wsgi :)

[Daniele Procida]

I've marked this ticket as especially suitable for people following the ​Don't be afraid to commit tutorial at the DjangoCon US 2014 sprints. If you're tackling this ticket, please don't hesitate to ask me for guidance if you'd like any, either here or on the Django IRC channels, where I can be found as EvilDMP.

[Daniel Craigmile]

I made the changes listed above here: ​https://github.com/django/django/pull/3176

[Collin Anderson]

GrahamDumpleton approves the change, though doesn't address aaugustin's concerns.

[Tim Graham]

I am not an expert, but reading through the ticket, it seems like a warning to the effect of "The use of WSGIApplicationGroup %{GLOBAL} presumes that your Apache instance is running only one Django application. If you are running more than Django application, please refer to the mod_wsgi docs for more information about this setting."

[Tim Graham <timograham@…>]

In c7f7432be54695ea16a2ba1cc5f8e5e1facf43f7:

Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

[Tim Graham <timograham@…>]

In 5767bc722ff9d65b20226a0aff438b38b46cc594:

[1.6.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master

[Tim Graham <timograham@…>]

In 84b50718abf1cd9883e39faff6aa7fc59fd5b75d:

[1.7.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master