Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#23350 closed Cleanup/optimization (fixed)

mod_wsgi authentication/authorization docs result in extra memory usage

Reported by: Graham Dumpleton Owned by: Daniel Craigmile
Component: Documentation Version: 1.6
Severity: Normal Keywords: afraid-to-commit
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

The documentation at:

uses the examples:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

Although these will work, they cause the Django application code to be loaded more than once into the same process. That is, you end up with an instance in one sub interpreter which handles web request, and another instance in another sub interpreter which handles just authentication and authorisation.

Ideally you do no want two copies as that simply increases overall memory usage.

The original mod_wsgi documentation does point to this in the statements:

By default the auth providers are executed in context of first interpreter created by Python, ie., '%{GLOBAL}' and always in the Apache child processes, never in a daemon process. The interpreter can be overridden using the 'application-group' option to the script directive. The namespace for authentication groups is shared with that for application groups defined by WSGIApplicationGroup.

Because the auth provider is always run in the Apache child processes and never in the context of a mod_wsgi daemon process, if the authentication check is making use of the internals of some Python web framework, it is recommended that the application using that web framework also be run in embedded mode and the same application group. This is the case as the Python web frameworks often bring in a huge amount of code even if using only one small part of them. This will result in a lot of memory being used in the Apache child processes just to support the auth provider.

To combat this double of memory usage the mod_wsgi documentation gives the example of:

WSGIAuthUserScript /usr/local/django/mysite/apache/auth.wsgi application-group=django

It is this use of the application-group argument for WSGIAuthUserScript and WSGIAuthGroupScript that the Django documentation doesn't use.

The Django examples therefore should really use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    Require group secret-agents
    Require valid-user
</Location>

On the presumption that there is only the one Django application, it may actually be better to turn that around and force the Django application to run in the main interpreter anyway. This is better as some third party extension modules for Python do not work properly in sub interpreters.

Thus may be better to use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

Change History (9)

comment:1 Changed 2 years ago by Aymeric Augustin

Component: UncategorizedDocumentation
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

I believe we should provide a copy-paste-proof example for the simple case of "first time I'm touching Django and Apache, how do I run my test project?" and shell out to mod_wsgi's own docs for anything more complicated.

For that use case, we can assume that there's only one Django application.

We just need:

  • to make sure our documentation stays internally consistent — we don't want the mod_wsgi auth docs to have subtle differences with the mod_wsgi main docs.
  • to add a warning about running multiple Django projets in the same server — which Apache's lovely handling of env variables makes hard anyway — and leave that part of the explaination up to mod_wsgi :)

comment:2 Changed 2 years ago by Daniele Procida

Easy pickings: set
Keywords: afraid-to-commit added

I've marked this ticket as especially suitable for people following the ​Don't be afraid to commit tutorial at the DjangoCon US 2014 sprints. If you're tackling this ticket, please don't hesitate to ask me for guidance if you'd like any, either at the sprints themselves, or here or on the Django IRC channels, where I can be found as EvilDMP.

Last edited 2 years ago by Daniele Procida (previous) (diff)

comment:3 Changed 2 years ago by Daniel Craigmile

Owner: changed from nobody to Daniel Craigmile
Status: newassigned

comment:4 Changed 2 years ago by Daniel Craigmile

I made the changes listed above here: https://github.com/django/django/pull/3176

comment:5 Changed 2 years ago by Collin Anderson

Has patch: set
Triage Stage: AcceptedReady for checkin

GrahamDumpleton approves the change, though doesn't address aaugustin's concerns.

comment:6 Changed 2 years ago by Tim Graham

I am not an expert, but reading through the ticket, it seems like a warning to the effect of "The use of WSGIApplicationGroup %{GLOBAL} presumes that your Apache instance is running only one Django application. If you are running more than Django application, please refer to the mod_wsgi docs for more information about this setting."

comment:7 Changed 2 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In c7f7432be54695ea16a2ba1cc5f8e5e1facf43f7:

Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

comment:8 Changed 2 years ago by Tim Graham <timograham@…>

In 5767bc722ff9d65b20226a0aff438b38b46cc594:

[1.6.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master

comment:9 Changed 2 years ago by Tim Graham <timograham@…>

In 84b50718abf1cd9883e39faff6aa7fc59fd5b75d:

[1.7.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master

Note: See TracTickets for help on using tickets.
Back to Top