Opened 8 months ago

Closed 8 months ago

Last modified 8 months ago

#23350 closed Cleanup/optimization (fixed)

mod_wsgi authentication/authorization docs result in extra memory usage

Reported by: GrahamDumpleton Owned by: x110dc
Component: Documentation Version: 1.6
Severity: Normal Keywords: afraid-to-commit
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

The documentation at:

uses the examples:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

Although these will work, they cause the Django application code to be loaded more than once into the same process. That is, you end up with an instance in one sub interpreter which handles web request, and another instance in another sub interpreter which handles just authentication and authorisation.

Ideally you do no want two copies as that simply increases overall memory usage.

The original mod_wsgi documentation does point to this in the statements:

By default the auth providers are executed in context of first interpreter created by Python, ie., '%{GLOBAL}' and always in the Apache child processes, never in a daemon process. The interpreter can be overridden using the 'application-group' option to the script directive. The namespace for authentication groups is shared with that for application groups defined by WSGIApplicationGroup.

Because the auth provider is always run in the Apache child processes and never in the context of a mod_wsgi daemon process, if the authentication check is making use of the internals of some Python web framework, it is recommended that the application using that web framework also be run in embedded mode and the same application group. This is the case as the Python web frameworks often bring in a huge amount of code even if using only one small part of them. This will result in a lot of memory being used in the Apache child processes just to support the auth provider.

To combat this double of memory usage the mod_wsgi documentation gives the example of:

WSGIAuthUserScript /usr/local/django/mysite/apache/auth.wsgi application-group=django

It is this use of the application-group argument for WSGIAuthUserScript and WSGIAuthGroupScript that the Django documentation doesn't use.

The Django examples therefore should really use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup django

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py application-group=django
    Require group secret-agents
    Require valid-user
</Location>

On the presumption that there is only the one Django application, it may actually be better to turn that around and force the Django application to run in the main interpreter anyway. This is better as some third party extension modules for Python do not work properly in sub interpreters.

Thus may be better to use:

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py
WSGIPythonPath /path/to/mysite.com

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    Require valid-user
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
</Location>

and

WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py

WSGIProcessGroup %{GLOBAL}
WSGIApplicationGroup %{GLOBAL}

<Location "/secret">
    AuthType Basic
    AuthName "Top Secret"
    AuthBasicProvider wsgi
    WSGIAuthUserScript /path/to/mysite.com/mysite/wsgi.py
    WSGIAuthGroupScript /path/to/mysite.com/mysite/wsgi.py
    Require group secret-agents
    Require valid-user
</Location>

Change History (9)

comment:1 Changed 8 months ago by aaugustin

  • Component changed from Uncategorized to Documentation
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Type changed from Uncategorized to Cleanup/optimization

I believe we should provide a copy-paste-proof example for the simple case of "first time I'm touching Django and Apache, how do I run my test project?" and shell out to mod_wsgi's own docs for anything more complicated.

For that use case, we can assume that there's only one Django application.

We just need:

  • to make sure our documentation stays internally consistent — we don't want the mod_wsgi auth docs to have subtle differences with the mod_wsgi main docs.
  • to add a warning about running multiple Django projets in the same server — which Apache's lovely handling of env variables makes hard anyway — and leave that part of the explaination up to mod_wsgi :)

comment:2 Changed 8 months ago by evildmp

  • Easy pickings set
  • Keywords afraid-to-commit added

I've marked this ticket as especially suitable for people following the ​Don't be afraid to commit tutorial at the DjangoCon US 2014 sprints. If you're tackling this ticket, please don't hesitate to ask me for guidance if you'd like any, either at the sprints themselves, or here or on the Django IRC channels, where I can be found as EvilDMP.

Last edited 8 months ago by evildmp (previous) (diff)

comment:3 Changed 8 months ago by x110dc

  • Owner changed from nobody to x110dc
  • Status changed from new to assigned

comment:4 Changed 8 months ago by x110dc

I made the changes listed above here: https://github.com/django/django/pull/3176

comment:5 Changed 8 months ago by collinanderson

  • Has patch set
  • Triage Stage changed from Accepted to Ready for checkin

GrahamDumpleton approves the change, though doesn't address aaugustin's concerns.

comment:6 Changed 8 months ago by timgraham

I am not an expert, but reading through the ticket, it seems like a warning to the effect of "The use of WSGIApplicationGroup %{GLOBAL} presumes that your Apache instance is running only one Django application. If you are running more than Django application, please refer to the mod_wsgi docs for more information about this setting."

comment:7 Changed 8 months ago by Tim Graham <timograham@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In c7f7432be54695ea16a2ba1cc5f8e5e1facf43f7:

Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

comment:8 Changed 8 months ago by Tim Graham <timograham@…>

In 5767bc722ff9d65b20226a0aff438b38b46cc594:

[1.6.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master

comment:9 Changed 8 months ago by Tim Graham <timograham@…>

In 84b50718abf1cd9883e39faff6aa7fc59fd5b75d:

[1.7.x] Fixed #23350 -- Updated mod_wsgi auth example to use less memory.

Thanks Graham Dumpleton for the report.

Backport of c7f7432be5 from master

Note: See TracTickets for help on using tickets.
Back to Top