[patch] CGI Support for django

The CGI Handler classes

The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)

The CGI Server classes

An alternative cgi script to Martin's using extisting WSGI

I took a differnet approach to Martin. Rather that add another handler the script I've added uses the existing WSGI handler. A CGi->WSGI shim layer if you like.

Tested it on the standard Apache 1.3 install that comes with Mac OS X, and python 2.4. I don't know of any reason it shouldn't work on other platforms, but you've been warned.

Has anyone tested this?

Recently I switch to the "django.cgi" approach and it works fine in my setup.

Which of the patches is "the good one" ? About the last one:

• It enforces the project to be under /home/mycode/mysite, maybe it should work as the FastCGI wrapper placing a file in you cgi-bin that set's the needed environment and then calls run_with_cgi() , so run_with_cgi() should be somewhere in the django's codebase (as the FastCGI equivalent) and the rest of the file (the last lines) in the cgi-bin/ file. It's almost how the FastCGI method works.

Setting "Patch needs improvement" for that.

Side note: for the one who writes the docs for that: consider putting a big big big note about the performance of running Django (or anything larger than a Hello World) in CGI mode.

to be placed in django/core/servers

to be placed in docs/

cgi.py and cgi.txt is the django.cgi splitted, notes:

• It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
• The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
• How to write a test for this? no idea ;)

Hope it's ok.

I tested Marc's cgi.py code, which works fine for me with django 0.95.1.

I think that Django should include this server interface, but there should be a note somewhere explaining that when using the CGI interface all dynamic page views slow down the more models and applications that your website contains, even if the page itself doesn't use them. The timing may be fine on a simple blog, but my site has 5 sub-applications and 31 tables (including all the admin ones) and it's currently taking 6 seconds to return a dynamic page, against under 0.5 seconds for a static GIF.

Works for me using 0.95.1, aside from the same issue that's behind the old, dismissed bug #285. Since not having a working admin side would have defeated most of the point in using Django, and the blithe suggestion of a global rewrite rule was somewhere between distasteful and impossible here, I had to find a fix which allows Django to work with this as a CGI, but as usual there's a price to be paid (pseudo patch for the 1/22 cgi.py):

+    # kluge to give Django the full local part of the URL
+    # fixes problems like #285 at the cost of injecting a prefix into urls.py
+    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ.get('PATH_INFO', '')

     result = application(environ, start_response)

This fixes the issue with the admin interface going off to points unknown after the login screen; it may also handle other places where a non-relative URL is generated. It does break the CGI standard meaning of PATH_INFO, so if this were to be adopted I should advise fixing the root of the problem in wsgi.py where it stupidly assumes that PATH_INFO is all there is to the URL. I assume this is a relic of the way Django has traditionally been deployed (see for example the discouragment of serving anything but Django in the mod_python setup docs, etc.) I would further speculate that some similar fix would deal as well with #285 and any related issues for FCGI, SCGI, ...?

Oh, the cost. It is, of course, that the top-level urls.py has to embed the added prefix (SCRIPT_NAME) to every url. In my own experiemntal work I renamed urls.py to real_urls.py, and made a "dummy" urls.py that matches the prefix and chains to real_urls.py. It's a bit ugly, and, once again assuming that the project cares about working cleanly in a non-mod_python environment, should get better support. I have some handwaving-grade ideas about that; perhaps I'll work it up into a patch later. Since this is for a copious spare time project, it's likely I'll just continue as-is for at least a while.

• Silly 1: there is no official documentation about how to run django with CGI :( I known, CGI is not the preferred setup.
• Silly 2: in this ticket are too many files. What way is the best?

I tried http://code.djangoproject.com/attachment/ticket/2407/django.cgi But it doesn't run, if the URL is empty, example:

• ".../django.cgi" works not
• ".../django.cgi/foo" works.

It's because, the http://code.djangoproject.com/browser/django/trunk/django/core/handlers/wsgi.py file access to "PATH_INFO" in the os.environ. But my Apache2 put only this requested path into the environ if it is not empty.

Here a small patch, for this:

Index: wsgi.py
===================================================================
--- wsgi.py	(revision 4556)
+++ wsgi.py	(working copy)
@@ -73,7 +73,7 @@
 class WSGIRequest(http.HttpRequest):
     def __init__(self, environ):
         self.environ = environ
-        self.path = environ['PATH_INFO']
+        self.path = environ.get('PATH_INFO','/')
         self.META = environ
         self.method = environ['REQUEST_METHOD'].upper()

Also it would work, if we change the http://code.djangoproject.com/attachment/ticket/2407/django.cgi file and insert:

...
 def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

uhmm.. as per my last comment:

    cgi.py and cgi.txt is the django.cgi splitted, notes:

    * It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
    * The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
    * How to write a test for this? no idea ;) 

So, the official docs for this patch are on the cgi.txt file, they are not in the "Documentation" section of djangoproject.com as the patch is still pending to be applied. And cgi.py is the python code to run the thing ;) The best should be the cgi.py file as it's the best choice as it's the last patch and the candidate to be checked in. Read the cgi.txt file for set-up information (it's almost like the fcgi wrapper).

About the files:

• cgi.py (2.2 kB) - to be placed in django/core/servers
• cgi.txt (1.3 kB) - to be placed in docs/

Hope this helps.

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module! What's about cgi_server.py?

Replying to Jedie:

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module! What's about cgi_server.py?

from django.core.servers.cgi import runcgi import django.core.servers.cgi.runcgi

is not

import cgi

This cgi.py is deep inside django and the documentation instructs you to import "runcgi" not "cgi" so you should have no trouble (I see no reason you would import the builtin "cgi" and this "cgi" on the wrapper script anyway). But it can be renamed anyway, something more to be discussed!

Call to volunteers: Write tests for this ticket! ;)

Yes, you are right. I have not put cgi.py into django/core/servers. I saved it into the root and make "from cgi import runcgi"... So i get an funny error, because the builtin file was overwritten ;)

Two problems remaining:

1. I must use use the environ['PATH_INFO'] = environ.get('PATH_INFO',"/") fix from above.

2. My http headers was not send in the right order:

• The first Head line is Status: 500 INTERNAL SERVER ERROR
• second line: Vary: Cookie
• and the last line: Content-Type: text/html

So i have made a silly patch:

...
    def send_content_type(response_headers):
        for no, header in enumerate(response_headers):
            if header[0].lower() == "content-type":
                sys.stdout.write('%s: %s\r\n' % header)
                del(response_headers[no])
                return response_headers

        sys.stdout.write('Content-Type: text/html\r\n')
        sys.stdout.write('Warning: Content Type not send!') # Bullshit?!?


    def write(data):
        if not headers_set:
            raise AssertionError("write() before start_response()")

        elif not headers_sent:
            # Before the first output, send the stored headers
            status, response_headers = headers_sent[:] = headers_set

            response_headers = send_content_type(response_headers) # Send Content-Type first

            sys.stdout.write('Status: %s\r\n' % status)
            for header in response_headers:
                sys.stdout.write('%s: %s\r\n' % header)
            sys.stdout.write('\r\n')

        sys.stdout.write(data)
        sys.stdout.flush()
...

FWIW, when using django.cgi, I had to specify the PATH_INFO like this:

environ['PATH_INFO']         = environ.get('REDIRECT_URL',"/") 

instead of :

def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

so that I got the correct path passed to django. Perhaps there is a different way to do this, but it worked for me.

Why in god's name a patch for django? There are couple of CGI-WSGI wrappers out there. nobody has to patch django for CGI support :-/

Why not include the small CGI Handler into django directly? It's small! So CGI guys need no extra files from somewhere.

The empty PATH_INFO problem is the same with SCGI or fastCGI. There exist the ticket:3414 for this.

print in python code should go stderr (logfile):

===> diff -u cgi.py ~modarch/django/trunk/django/core/servers/cgi.py --- cgi.py 2007-05-07 16:49:30.313178117 +0200 +++ /home/modarch/django/trunk/django/core/servers/cgi.py 2007-05-07 17:11:09.788411072 +0200 @@ -10,6 +10,7 @@

def runcgi():

environ = dict(os.environ.items())

+ environPATH_INFO? = environ.get('PATH_INFO',"/")

environwsgi.input? = sys.stdin environwsgi.errors? = sys.stderr environwsgi.version? = (1,0)

@@ -26,6 +27,9 @@

headers_set = [] headers_sent = []

+ + stdout=sys.stdout + sys.stdout=sys.stderr # print should go to stderr (logfile)

def write(data):

if not headers_set:

@@ -34,13 +38,13 @@

elif not headers_sent:

# Before the first output, send the stored headers status, response_headers = headers_sent[:] = headers_set

- sys.stdout.write('Status: %s\r\n' % status) + stdout.write('Status: %s\r\n' % status)

for header in response_headers:

- sys.stdout.write('%s: %s\r\n' % header) - sys.stdout.write('\r\n') + stdout.write('%s: %s\r\n' % header) + stdout.write('\r\n') - sys.stdout.write(data) - sys.stdout.flush() + stdout.write(data) + stdout.flush()

def start_response(status,response_headers,exc_info=None):

if exc_info:

Why ?? stdout is stdout, and stderr is stderr they are and mean different things.

Why should stdout = stderr ? If you are using print statemens for debuggin please use the logging module ;)

Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)

Django just isn't designed to run under CGI.

It won't run under OS/2, either.