Opened 18 years ago

Closed 13 years ago

#2407 closed enhancement (wontfix)

CGI Support for django

Reported by: Martin Glueck <martin.glueck@…> Owned by: nobody
Component: Core (Other) Version: 1.1
Severity: normal Keywords: cgi
Cc: django_trac@… Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

The attached files add CGI support for django. Maybe this is useful for somebody else as well.

Attachments (8)

CGI.py (8.0 KB ) - added by Martin 18 years ago.
The CGI Handler classes
CGI.2.py (6.8 KB ) - added by Martin 18 years ago.
The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)
CGI_Server.py (1.2 KB ) - added by Martin 18 years ago.
The CGI Server classes
django.cgi (3.4 KB ) - added by Paul Sargent 18 years ago.
An alternative cgi script to Martin's using extisting WSGI
cgi.py (2.2 KB ) - added by Marc Fargas <telenieko@…> 17 years ago.
to be placed in django/core/servers
cgi.txt (1.3 KB ) - added by Marc Fargas <telenieko@…> 17 years ago.
to be placed in docs/
cgi.2.py (2.8 KB ) - added by Thomas Güttler 17 years ago.
Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)
dj_wsgiref_cgihandler.py (255 bytes ) - added by limon 16 years ago.
run django as cgi with wsgiref CGIHandler

Download all attachments as: .zip

Change History (32)

by Martin, 18 years ago

Attachment: CGI.py added

The CGI Handler classes

by Martin, 18 years ago

Attachment: CGI.2.py added

The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)

by Martin, 18 years ago

Attachment: CGI_Server.py added

The CGI Server classes

by Paul Sargent, 18 years ago

Attachment: django.cgi added

An alternative cgi script to Martin's using extisting WSGI

comment:1 by Paul Sargent, 18 years ago

Summary: CGI Support for django[patch] CGI Support for django

I took a differnet approach to Martin. Rather that add another handler the script I've added uses the existing WSGI handler. A CGi->WSGI shim layer if you like.

Tested it on the standard Apache 1.3 install that comes with Mac OS X, and python 2.4. I don't know of any reason it shouldn't work on other platforms, but you've been warned.

comment:2 by Simon G. <dev@…>, 17 years ago

Keywords: cgi added
Needs documentation: set
Needs tests: set

Has anyone tested this?

comment:3 by martin.glueck@…, 17 years ago

Recently I switch to the "django.cgi" approach and it works fine in my setup.

comment:4 by Marc Fargas <telenieko@…>, 17 years ago

Patch needs improvement: set

Which of the patches is "the good one" ? About the last one:

  • It enforces the project to be under /home/mycode/mysite, maybe it should work as the FastCGI wrapper placing a file in you cgi-bin that set's the needed environment and then calls run_with_cgi() , so run_with_cgi() should be somewhere in the django's codebase (as the FastCGI equivalent) and the rest of the file (the last lines) in the cgi-bin/ file. It's almost how the FastCGI method works.

Setting "Patch needs improvement" for that.

Side note: for the one who writes the docs for that: consider putting a big big big note about the performance of running Django (or anything larger than a Hello World) in CGI mode.

by Marc Fargas <telenieko@…>, 17 years ago

Attachment: cgi.py added

to be placed in django/core/servers

by Marc Fargas <telenieko@…>, 17 years ago

Attachment: cgi.txt added

to be placed in docs/

comment:5 by Marc Fargas <telenieko@…>, 17 years ago

Needs documentation: unset
Patch needs improvement: unset

cgi.py and cgi.txt is the django.cgi splitted, notes:

  • It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
  • The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
  • How to write a test for this? no idea ;)

Hope it's ok.

comment:6 by Andrew Johnson <anjohnson@…>, 17 years ago

I tested Marc's cgi.py code, which works fine for me with django 0.95.1.

I think that Django should include this server interface, but there should be a note somewhere explaining that when using the CGI interface all dynamic page views slow down the more models and applications that your website contains, even if the page itself doesn't use them. The timing may be fine on a simple blog, but my site has 5 sub-applications and 31 tables (including all the admin ones) and it's currently taking 6 seconds to return a dynamic page, against under 0.5 seconds for a static GIF.

comment:7 by Michael Radziej <mir@…>, 17 years ago

Triage Stage: UnreviewedDesign decision needed

comment:8 by maney at two14 net, 17 years ago

Patch needs improvement: set

Works for me using 0.95.1, aside from the same issue that's behind the old, dismissed bug #285. Since not having a working admin side would have defeated most of the point in using Django, and the blithe suggestion of a global rewrite rule was somewhere between distasteful and impossible here, I had to find a fix which allows Django to work with this as a CGI, but as usual there's a price to be paid (pseudo patch for the 1/22 cgi.py):

+    # kluge to give Django the full local part of the URL
+    # fixes problems like #285 at the cost of injecting a prefix into urls.py
+    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ.get('PATH_INFO', '')

     result = application(environ, start_response)

This fixes the issue with the admin interface going off to points unknown after the login screen; it may also handle other places where a non-relative URL is generated. It does break the CGI standard meaning of PATH_INFO, so if this were to be adopted I should advise fixing the root of the problem in wsgi.py where it stupidly assumes that PATH_INFO is all there is to the URL. I assume this is a relic of the way Django has traditionally been deployed (see for example the discouragment of serving anything but Django in the mod_python setup docs, etc.) I would further speculate that some similar fix would deal as well with #285 and any related issues for FCGI, SCGI, ...?

Oh, the cost. It is, of course, that the top-level urls.py has to embed the added prefix (SCRIPT_NAME) to every url. In my own experiemntal work I renamed urls.py to real_urls.py, and made a "dummy" urls.py that matches the prefix and chains to real_urls.py. It's a bit ugly, and, once again assuming that the project cares about working cleanly in a non-mod_python environment, should get better support. I have some handwaving-grade ideas about that; perhaps I'll work it up into a patch later. Since this is for a copious spare time project, it's likely I'll just continue as-is for at least a while.

comment:9 by jedie, 17 years ago

  • Silly 1: there is no official documentation about how to run django with CGI :( I known, CGI is not the preferred setup.
  • Silly 2: in this ticket are too many files. What way is the best?

I tried http://code.djangoproject.com/attachment/ticket/2407/django.cgi
But it doesn't run, if the URL is empty, example:

  • ".../django.cgi" works not
  • ".../django.cgi/foo" works.

It's because, the http://code.djangoproject.com/browser/django/trunk/django/core/handlers/wsgi.py file access to "PATH_INFO" in the os.environ. But my Apache2 put only this requested path into the environ if it is not empty.

Here a small patch, for this:

Index: wsgi.py
===================================================================
--- wsgi.py	(revision 4556)
+++ wsgi.py	(working copy)
@@ -73,7 +73,7 @@
 class WSGIRequest(http.HttpRequest):
     def __init__(self, environ):
         self.environ = environ
-        self.path = environ['PATH_INFO']
+        self.path = environ.get('PATH_INFO','/')
         self.META = environ
         self.method = environ['REQUEST_METHOD'].upper()

Also it would work, if we change the http://code.djangoproject.com/attachment/ticket/2407/django.cgi file and insert:

...
 def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

comment:10 by Marc Fargas <telenieko@…>, 17 years ago

uhmm.. as per my last comment:

    cgi.py and cgi.txt is the django.cgi splitted, notes:

    * It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
    * The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
    * How to write a test for this? no idea ;) 

So, the official docs for this patch are on the cgi.txt file, they are not in the "Documentation" section of djangoproject.com as the patch is still pending to be applied. And cgi.py is the python code to run the thing ;)
The best should be the cgi.py file as it's the best choice as it's the last patch and the candidate to be checked in. Read the cgi.txt file for set-up information (it's almost like the fcgi wrapper).

About the files:

  • cgi.py (2.2 kB) - to be placed in django/core/servers
  • cgi.txt (1.3 kB) - to be placed in docs/

Hope this helps.

comment:11 by Jedie, 17 years ago

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module!
What's about cgi_server.py?

in reply to:  11 comment:12 by Marc Fargas <telenieko@…>, 17 years ago

Replying to Jedie:

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module!
What's about cgi_server.py?

from django.core.servers.cgi import runcgi
import django.core.servers.cgi.runcgi

is not

import cgi

This cgi.py is deep inside django and the documentation instructs you to import "runcgi" not "cgi" so you should have no trouble (I see no reason you would import the builtin "cgi" and this "cgi" on the wrapper script anyway). But it can be renamed anyway, something more to be discussed!

Call to volunteers: Write tests for this ticket! ;)

comment:13 by Jedie, 17 years ago

Yes, you are right. I have not put cgi.py into django/core/servers. I saved it into the root and make "from cgi import runcgi"... So i get an funny error, because the builtin file was overwritten ;)

Two problems remaining:

  1. I must use use the environ['PATH_INFO'] = environ.get('PATH_INFO',"/") fix from above.
  1. My http headers was not send in the right order:
    • The first Head line is Status: 500 INTERNAL SERVER ERROR
    • second line: Vary: Cookie
    • and the last line: Content-Type: text/html

So i have made a silly patch:

...
    def send_content_type(response_headers):
        for no, header in enumerate(response_headers):
            if header[0].lower() == "content-type":
                sys.stdout.write('%s: %s\r\n' % header)
                del(response_headers[no])
                return response_headers

        sys.stdout.write('Content-Type: text/html\r\n')
        sys.stdout.write('Warning: Content Type not send!') # Bullshit?!?


    def write(data):
        if not headers_set:
            raise AssertionError("write() before start_response()")

        elif not headers_sent:
            # Before the first output, send the stored headers
            status, response_headers = headers_sent[:] = headers_set

            response_headers = send_content_type(response_headers) # Send Content-Type first

            sys.stdout.write('Status: %s\r\n' % status)
            for header in response_headers:
                sys.stdout.write('%s: %s\r\n' % header)
            sys.stdout.write('\r\n')

        sys.stdout.write(data)
        sys.stdout.flush()
...

comment:14 by anonymous, 17 years ago

FWIW, when using django.cgi, I had to specify the PATH_INFO like this:

environ['PATH_INFO']         = environ.get('REDIRECT_URL',"/") 

instead of :

def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

so that I got the correct path passed to django. Perhaps there is a different way to do this, but it worked for me.

comment:15 by anonymous, 17 years ago

Why in god's name a patch for django? There are couple of CGI-WSGI wrappers out there. nobody has to patch django for CGI support :-/

comment:16 by anonymous, 17 years ago

Why not include the small CGI Handler into django directly? It's small! So CGI guys need no extra files from somewhere.

comment:17 by anonymous, 17 years ago

The empty PATH_INFO problem is the same with SCGI or fastCGI. There exist the ticket:3414 for this.

comment:18 by Thomas Güttler, 17 years ago

print in python code should go stderr (logfile):

===> diff -u cgi.py ~modarch/django/trunk/django/core/servers/cgi.py
--- cgi.py 2007-05-07 16:49:30.313178117 +0200
+++ /home/modarch/django/trunk/django/core/servers/cgi.py 2007-05-07 17:11:09.788411072 +0200
@@ -10,6 +10,7 @@

def runcgi():

environ = dict(os.environ.items())

+ environPATH_INFO = environ.get('PATH_INFO',"/")

environwsgi.input = sys.stdin
environwsgi.errors = sys.stderr
environwsgi.version = (1,0)

@@ -26,6 +27,9 @@

headers_set = []
headers_sent = []

+
+ stdout=sys.stdout
+ sys.stdout=sys.stderr # print should go to stderr (logfile)

def write(data):

if not headers_set:

@@ -34,13 +38,13 @@

elif not headers_sent:

# Before the first output, send the stored headers
status, response_headers = headers_sent[:] = headers_set

  • sys.stdout.write('Status: %s\r\n' % status)

+ stdout.write('Status: %s\r\n' % status)

for header in response_headers:

  • sys.stdout.write('%s: %s\r\n' % header)
  • sys.stdout.write('\r\n')

+ stdout.write('%s: %s\r\n' % header)
+ stdout.write('\r\n')

  • sys.stdout.write(data)
  • sys.stdout.flush()

+ stdout.write(data)
+ stdout.flush()

def start_response(status,response_headers,exc_info=None):

if exc_info:

comment:19 by Marc Fargas <telenieko@…>, 17 years ago

Why ??
stdout is stdout, and stderr is stderr they are and mean different things.

Why should stdout = stderr ? If you are using print statemens for debuggin please use the logging module ;)

by Thomas Güttler, 17 years ago

Attachment: cgi.2.py added

Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)

comment:20 by Jens Diemer, 17 years ago

Cc: django_trac@… added

comment:21 by Jacob, 16 years ago

Resolution: wontfix
Status: newclosed

Django just isn't designed to run under CGI.

It won't run under OS/2, either.

comment:22 by limon, 16 years ago

Use wsgiref module for django cgi support

by limon, 16 years ago

Attachment: dj_wsgiref_cgihandler.py added

run django as cgi with wsgiref CGIHandler

comment:23 by DremLIN, 13 years ago

Resolution: wontfix
Status: closedreopened
Summary: [patch] CGI Support for djangoCGI Support for django
Version: SVN1.1

Hello,

In my hosting, I have: Python 2.3 run with Apache 1.3 as CGI. I install Django 1.1.2, but can't run "mysite" application.

I try dj_wsgiref_cgihandler.py by limon and I get error:
Status: 500 Dude, this is whack! A server error occurred. Please contact the administrator.

Please give me working example index.cgi for CGI.2.py (6.8 kB) by Martin or CGI_Server.py (1.2 kB) by Martin.

My e-mail: dremlin.ru [at] gmail.com

comment:24 by Russell Keith-Magee, 13 years ago

Resolution: wontfix
Status: reopenedclosed

Django doesn't support CGI for deployment, and Trac isn't a user support forum. If you have a howto question, please ask on Django-users, but be advised -- Django doesn't support CGI for a very good reason.

Note: See TracTickets for help on using tickets.
Back to Top