Django

Code

Ticket #2407 (closed: wontfix)

Opened 2 years ago

Last modified 8 months ago

[patch] CGI Support for django

Reported by: Martin Glueck <martin.glueck@gmail.com> Assigned to: nobody
Milestone: Component: Core framework
Version: SVN Keywords: cgi
Cc: django_trac@jensdiemer.de Triage Stage: Design decision needed
Has patch: 1 Needs documentation: 0
Needs tests: 1 Patch needs improvement: 1

Description

The attached files add CGI support for django. Maybe this is useful for somebody else as well.

Attachments

CGI.py (8.0 kB) - added by Martin on 07/23/06 20:52:13.
The CGI Handler classes
CGI.2.py (6.8 kB) - added by Martin on 07/23/06 20:55:23.
The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)
CGI_Server.py (1.2 kB) - added by Martin on 07/23/06 20:58:58.
The CGI Server classes
django.cgi (3.4 kB) - added by Paul Sargent on 08/03/06 07:49:53.
An alternative cgi script to Martin's using extisting WSGI
cgi.py (2.2 kB) - added by Marc Fargas <telenieko@telenieko.com> on 01/22/07 18:22:51.
to be placed in django/core/servers
cgi.txt (1.3 kB) - added by Marc Fargas <telenieko@telenieko.com> on 01/22/07 18:23:14.
to be placed in docs/
cgi.2.py (2.8 kB) - added by Thomas Güttler on 05/08/07 03:35:12.
Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)

Change History

07/23/06 20:52:13 changed by Martin

  • attachment CGI.py added.

The CGI Handler classes

07/23/06 20:55:23 changed by Martin

  • attachment CGI.2.py added.

The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)

07/23/06 20:58:58 changed by Martin

  • attachment CGI_Server.py added.

The CGI Server classes

08/03/06 07:49:53 changed by Paul Sargent

  • attachment django.cgi added.

An alternative cgi script to Martin's using extisting WSGI

08/03/06 07:58:24 changed by Paul Sargent

  • summary changed from CGI Support for django to [patch] CGI Support for django.

I took a differnet approach to Martin. Rather that add another handler the script I've added uses the existing WSGI handler. A CGi->WSGI shim layer if you like.

Tested it on the standard Apache 1.3 install that comes with Mac OS X, and python 2.4. I don't know of any reason it shouldn't work on other platforms, but you've been warned.

01/18/07 04:00:00 changed by Simon G. <dev@simon.net.nz>

  • keywords set to cgi.
  • needs_docs set to 1.
  • needs_tests set to 1.

Has anyone tested this?

01/18/07 12:47:29 changed by martin.glueck@gmail.com

Recently I switch to the "django.cgi" approach and it works fine in my setup.

01/22/07 18:07:08 changed by Marc Fargas <telenieko@telenieko.com>

  • needs_better_patch set to 1.

Which of the patches is "the good one" ? About the last one:

  • It enforces the project to be under /home/mycode/mysite, maybe it should work as the FastCGI wrapper placing a file in you cgi-bin that set's the needed environment and then calls run_with_cgi() , so run_with_cgi() should be somewhere in the django's codebase (as the FastCGI equivalent) and the rest of the file (the last lines) in the cgi-bin/ file. It's almost how the FastCGI method works.

Setting "Patch needs improvement" for that.

Side note: for the one who writes the docs for that: consider putting a big big big note about the performance of running Django (or anything larger than a Hello World) in CGI mode.

01/22/07 18:22:51 changed by Marc Fargas <telenieko@telenieko.com>

  • attachment cgi.py added.

to be placed in django/core/servers

01/22/07 18:23:14 changed by Marc Fargas <telenieko@telenieko.com>

  • attachment cgi.txt added.

to be placed in docs/

01/22/07 18:25:36 changed by Marc Fargas <telenieko@telenieko.com>

  • needs_better_patch deleted.
  • needs_docs deleted.

cgi.py and cgi.txt is the django.cgi splitted, notes:

  • It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
  • The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
  • How to write a test for this? no idea ;)

Hope it's ok.

02/04/07 00:16:54 changed by Andrew Johnson <anjohnson@iee.org>

I tested Marc's cgi.py code, which works fine for me with django 0.95.1.

I think that Django should include this server interface, but there should be a note somewhere explaining that when using the CGI interface all dynamic page views slow down the more models and applications that your website contains, even if the page itself doesn't use them. The timing may be fine on a simple blog, but my site has 5 sub-applications and 31 tables (including all the admin ones) and it's currently taking 6 seconds to return a dynamic page, against under 0.5 seconds for a static GIF.

02/04/07 02:18:39 changed by Michael Radziej <mir@noris.de>

  • stage changed from Unreviewed to Design decision needed.

02/07/07 19:37:24 changed by maney at two14 net

  • needs_better_patch set to 1.

Works for me using 0.95.1, aside from the same issue that's behind the old, dismissed bug #285. Since not having a working admin side would have defeated most of the point in using Django, and the blithe suggestion of a global rewrite rule was somewhere between distasteful and impossible here, I had to find a fix which allows Django to work with this as a CGI, but as usual there's a price to be paid (pseudo patch for the 1/22 cgi.py):

+    # kluge to give Django the full local part of the URL
+    # fixes problems like #285 at the cost of injecting a prefix into urls.py
+    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ.get('PATH_INFO', '')

     result = application(environ, start_response)

This fixes the issue with the admin interface going off to points unknown after the login screen; it may also handle other places where a non-relative URL is generated. It does break the CGI standard meaning of PATH_INFO, so if this were to be adopted I should advise fixing the root of the problem in wsgi.py where it stupidly assumes that PATH_INFO is all there is to the URL. I assume this is a relic of the way Django has traditionally been deployed (see for example the discouragment of serving anything but Django in the mod_python setup docs, etc.) I would further speculate that some similar fix would deal as well with #285 and any related issues for FCGI, SCGI, ...?

Oh, the cost. It is, of course, that the top-level urls.py has to embed the added prefix (SCRIPT_NAME) to every url. In my own experiemntal work I renamed urls.py to real_urls.py, and made a "dummy" urls.py that matches the prefix and chains to real_urls.py. It's a bit ugly, and, once again assuming that the project cares about working cleanly in a non-mod_python environment, should get better support. I have some handwaving-grade ideas about that; perhaps I'll work it up into a patch later. Since this is for a copious spare time project, it's likely I'll just continue as-is for at least a while.

02/21/07 11:52:17 changed by jedie

  • Silly 1: there is no official documentation about how to run django with CGI :( I known, CGI is not the preferred setup.
  • Silly 2: in this ticket are too many files. What way is the best?

I tried http://code.djangoproject.com/attachment/ticket/2407/django.cgi But it doesn't run, if the URL is empty, example:

  • ".../django.cgi" works not
  • ".../django.cgi/foo" works.

It's because, the http://code.djangoproject.com/browser/django/trunk/django/core/handlers/wsgi.py file access to "PATH_INFO" in the os.environ. But my Apache2 put only this requested path into the environ if it is not empty.

Here a small patch, for this:

Index: wsgi.py
===================================================================
--- wsgi.py	(revision 4556)
+++ wsgi.py	(working copy)
@@ -73,7 +73,7 @@
 class WSGIRequest(http.HttpRequest):
     def __init__(self, environ):
         self.environ = environ
-        self.path = environ['PATH_INFO']
+        self.path = environ.get('PATH_INFO','/')
         self.META = environ
         self.method = environ['REQUEST_METHOD'].upper()

Also it would work, if we change the http://code.djangoproject.com/attachment/ticket/2407/django.cgi file and insert:

...
 def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

02/21/07 13:45:21 changed by Marc Fargas <telenieko@telenieko.com>

uhmm.. as per my last comment:

    cgi.py and cgi.txt is the django.cgi splitted, notes:

    * It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
    * The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
    * How to write a test for this? no idea ;) 

So, the official docs for this patch are on the cgi.txt file, they are not in the "Documentation" section of djangoproject.com as the patch is still pending to be applied. And cgi.py is the python code to run the thing ;) The best should be the cgi.py file as it's the best choice as it's the last patch and the candidate to be checked in. Read the cgi.txt file for set-up information (it's almost like the fcgi wrapper).

About the files:

  • cgi.py (2.2 kB) - to be placed in django/core/servers
  • cgi.txt (1.3 kB) - to be placed in docs/

Hope this helps.

(follow-up: ↓ 12 ) 02/22/07 06:22:58 changed by Jedie

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module! What's about cgi_server.py?

(in reply to: ↑ 11 ) 02/22/07 06:33:59 changed by Marc Fargas <telenieko@telenieko.com>

Replying to Jedie:

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module! What's about cgi_server.py?

from django.core.servers.cgi import runcgi import django.core.servers.cgi.runcgi

is not

import cgi

This cgi.py is deep inside django and the documentation instructs you to import "runcgi" not "cgi" so you should have no trouble (I see no reason you would import the builtin "cgi" and this "cgi" on the wrapper script anyway). But it can be renamed anyway, something more to be discussed!

Call to volunteers: Write tests for this ticket! ;)

02/22/07 06:52:47 changed by Jedie

Yes, you are right. I have not put cgi.py into django/core/servers. I saved it into the root and make "from cgi import runcgi"... So i get an funny error, because the builtin file was overwritten ;)

Two problems remaining:

1. I must use use the environ['PATH_INFO'] = environ.get('PATH_INFO',"/") fix from above.

2. My http headers was not send in the right order:

  • The first Head line is Status: 500 INTERNAL SERVER ERROR
  • second line: Vary: Cookie
  • and the last line: Content-Type: text/html

So i have made a silly patch:

...
    def send_content_type(response_headers):
        for no, header in enumerate(response_headers):
            if header[0].lower() == "content-type":
                sys.stdout.write('%s: %s\r\n' % header)
                del(response_headers[no])
                return response_headers

        sys.stdout.write('Content-Type: text/html\r\n')
        sys.stdout.write('Warning: Content Type not send!') # Bullshit?!?


    def write(data):
        if not headers_set:
            raise AssertionError("write() before start_response()")

        elif not headers_sent:
            # Before the first output, send the stored headers
            status, response_headers = headers_sent[:] = headers_set

            response_headers = send_content_type(response_headers) # Send Content-Type first

            sys.stdout.write('Status: %s\r\n' % status)
            for header in response_headers:
                sys.stdout.write('%s: %s\r\n' % header)
            sys.stdout.write('\r\n')

        sys.stdout.write(data)
        sys.stdout.flush()
...

03/11/07 21:18:03 changed by anonymous

FWIW, when using django.cgi, I had to specify the PATH_INFO like this:

environ['PATH_INFO']         = environ.get('REDIRECT_URL',"/") 

instead of :

def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

so that I got the correct path passed to django. Perhaps there is a different way to do this, but it worked for me.

04/08/07 03:59:03 changed by anonymous

Why in god's name a patch for django? There are couple of CGI-WSGI wrappers out there. nobody has to patch django for CGI support :-/

04/09/07 02:57:40 changed by anonymous

Why not include the small CGI Handler into django directly? It's small! So CGI guys need no extra files from somewhere.

05/04/07 07:57:38 changed by anonymous

The empty PATH_INFO problem is the same with SCGI or fastCGI. There exist the ticket:3414 for this.

05/07/07 10:15:19 changed by Thomas Güttler

print in python code should go stderr (logfile):

===> diff -u cgi.py ~modarch/django/trunk/django/core/servers/cgi.py --- cgi.py 2007-05-07 16:49:30.313178117 +0200 +++ /home/modarch/django/trunk/django/core/servers/cgi.py 2007-05-07 17:11:09.788411072 +0200 @@ -10,6 +10,7 @@

def runcgi():

environ = dict(os.environ.items())

+ environPATH_INFO? = environ.get('PATH_INFO',"/")

environwsgi.input? = sys.stdin environwsgi.errors? = sys.stderr environwsgi.version? = (1,0)

@@ -26,6 +27,9 @@

headers_set = [] headers_sent = []

+ + stdout=sys.stdout + sys.stdout=sys.stderr # print should go to stderr (logfile)

def write(data):

if not headers_set:

@@ -34,13 +38,13 @@

elif not headers_sent:

# Before the first output, send the stored headers status, response_headers = headers_sent[:] = headers_set

- sys.stdout.write('Status: %s\r\n' % status) + stdout.write('Status: %s\r\n' % status)

for header in response_headers:

- sys.stdout.write('%s: %s\r\n' % header) - sys.stdout.write('\r\n') + stdout.write('%s: %s\r\n' % header) + stdout.write('\r\n') - sys.stdout.write(data) - sys.stdout.flush() + stdout.write(data) + stdout.flush()

def start_response(status,response_headers,exc_info=None):

if exc_info:

05/07/07 10:47:06 changed by Marc Fargas <telenieko@telenieko.com>

Why ?? stdout is stdout, and stderr is stderr they are and mean different things.

Why should stdout = stderr ? If you are using print statemens for debuggin please use the logging module ;)

05/08/07 03:35:12 changed by Thomas Güttler

  • attachment cgi.2.py added.

Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)

07/04/07 02:35:51 changed by Jens Diemer

  • cc set to django_trac@jensdiemer.de.

11/30/07 15:46:17 changed by jacob

  • status changed from new to closed.
  • resolution set to wontfix.

Django just isn't designed to run under CGI.

It won't run under OS/2, either.


Add/Change #2407 ([patch] CGI Support for django)




Change Properties
Action