Opened 9 years ago

Closed 5 years ago

#2407 closed enhancement (wontfix)

CGI Support for django

Reported by: Martin Glueck <martin.glueck@…> Owned by: nobody
Component: Core (Other) Version: 1.1
Severity: normal Keywords: cgi
Cc: django_trac@… Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

The attached files add CGI support for django. Maybe this is useful for somebody else as well.

Attachments (8)

CGI.py (8.0 KB) - added by Martin 9 years ago.
The CGI Handler classes
CGI.2.py (6.8 KB) - added by Martin 9 years ago.
The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)
CGI_Server.py (1.2 KB) - added by Martin 9 years ago.
The CGI Server classes
django.cgi (3.4 KB) - added by Paul Sargent 9 years ago.
An alternative cgi script to Martin's using extisting WSGI
cgi.py (2.2 KB) - added by Marc Fargas <telenieko@…> 8 years ago.
to be placed in django/core/servers
cgi.txt (1.3 KB) - added by Marc Fargas <telenieko@…> 8 years ago.
to be placed in docs/
cgi.2.py (2.8 KB) - added by Thomas Güttler 8 years ago.
Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)
dj_wsgiref_cgihandler.py (255 bytes) - added by limon 7 years ago.
run django as cgi with wsgiref CGIHandler

Download all attachments as: .zip

Change History (32)

Changed 9 years ago by Martin

The CGI Handler classes

Changed 9 years ago by Martin

The CGI Handler classes (replaces the old CGI.py which had still my standard file-header)

Changed 9 years ago by Martin

The CGI Server classes

Changed 9 years ago by Paul Sargent

An alternative cgi script to Martin's using extisting WSGI

comment:1 Changed 9 years ago by Paul Sargent

  • Summary changed from CGI Support for django to [patch] CGI Support for django

I took a differnet approach to Martin. Rather that add another handler the script I've added uses the existing WSGI handler. A CGi->WSGI shim layer if you like.

Tested it on the standard Apache 1.3 install that comes with Mac OS X, and python 2.4. I don't know of any reason it shouldn't work on other platforms, but you've been warned.

comment:2 Changed 8 years ago by Simon G. <dev@…>

  • Keywords cgi added
  • Needs documentation set
  • Needs tests set

Has anyone tested this?

comment:3 Changed 8 years ago by martin.glueck@…

Recently I switch to the "django.cgi" approach and it works fine in my setup.

comment:4 Changed 8 years ago by Marc Fargas <telenieko@…>

  • Patch needs improvement set

Which of the patches is "the good one" ? About the last one:

  • It enforces the project to be under /home/mycode/mysite, maybe it should work as the FastCGI wrapper placing a file in you cgi-bin that set's the needed environment and then calls run_with_cgi() , so run_with_cgi() should be somewhere in the django's codebase (as the FastCGI equivalent) and the rest of the file (the last lines) in the cgi-bin/ file. It's almost how the FastCGI method works.

Setting "Patch needs improvement" for that.

Side note: for the one who writes the docs for that: consider putting a big big big note about the performance of running Django (or anything larger than a Hello World) in CGI mode.

Changed 8 years ago by Marc Fargas <telenieko@…>

to be placed in django/core/servers

Changed 8 years ago by Marc Fargas <telenieko@…>

to be placed in docs/

comment:5 Changed 8 years ago by Marc Fargas <telenieko@…>

  • Needs documentation unset
  • Patch needs improvement unset

cgi.py and cgi.txt is the django.cgi splitted, notes:

  • It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
  • The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
  • How to write a test for this? no idea ;)

Hope it's ok.

comment:6 Changed 8 years ago by Andrew Johnson <anjohnson@…>

I tested Marc's cgi.py code, which works fine for me with django 0.95.1.

I think that Django should include this server interface, but there should be a note somewhere explaining that when using the CGI interface all dynamic page views slow down the more models and applications that your website contains, even if the page itself doesn't use them. The timing may be fine on a simple blog, but my site has 5 sub-applications and 31 tables (including all the admin ones) and it's currently taking 6 seconds to return a dynamic page, against under 0.5 seconds for a static GIF.

comment:7 Changed 8 years ago by Michael Radziej <mir@…>

  • Triage Stage changed from Unreviewed to Design decision needed

comment:8 Changed 8 years ago by maney at two14 net

  • Patch needs improvement set

Works for me using 0.95.1, aside from the same issue that's behind the old, dismissed bug #285. Since not having a working admin side would have defeated most of the point in using Django, and the blithe suggestion of a global rewrite rule was somewhere between distasteful and impossible here, I had to find a fix which allows Django to work with this as a CGI, but as usual there's a price to be paid (pseudo patch for the 1/22 cgi.py):

+    # kluge to give Django the full local part of the URL
+    # fixes problems like #285 at the cost of injecting a prefix into urls.py
+    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ.get('PATH_INFO', '')

     result = application(environ, start_response)

This fixes the issue with the admin interface going off to points unknown after the login screen; it may also handle other places where a non-relative URL is generated. It does break the CGI standard meaning of PATH_INFO, so if this were to be adopted I should advise fixing the root of the problem in wsgi.py where it stupidly assumes that PATH_INFO is all there is to the URL. I assume this is a relic of the way Django has traditionally been deployed (see for example the discouragment of serving anything but Django in the mod_python setup docs, etc.) I would further speculate that some similar fix would deal as well with #285 and any related issues for FCGI, SCGI, ...?

Oh, the cost. It is, of course, that the top-level urls.py has to embed the added prefix (SCRIPT_NAME) to every url. In my own experiemntal work I renamed urls.py to real_urls.py, and made a "dummy" urls.py that matches the prefix and chains to real_urls.py. It's a bit ugly, and, once again assuming that the project cares about working cleanly in a non-mod_python environment, should get better support. I have some handwaving-grade ideas about that; perhaps I'll work it up into a patch later. Since this is for a copious spare time project, it's likely I'll just continue as-is for at least a while.

comment:9 Changed 8 years ago by jedie

  • Silly 1: there is no official documentation about how to run django with CGI :( I known, CGI is not the preferred setup.
  • Silly 2: in this ticket are too many files. What way is the best?

I tried http://code.djangoproject.com/attachment/ticket/2407/django.cgi
But it doesn't run, if the URL is empty, example:

  • ".../django.cgi" works not
  • ".../django.cgi/foo" works.

It's because, the http://code.djangoproject.com/browser/django/trunk/django/core/handlers/wsgi.py file access to "PATH_INFO" in the os.environ. But my Apache2 put only this requested path into the environ if it is not empty.

Here a small patch, for this:

Index: wsgi.py
===================================================================
--- wsgi.py	(revision 4556)
+++ wsgi.py	(working copy)
@@ -73,7 +73,7 @@
 class WSGIRequest(http.HttpRequest):
     def __init__(self, environ):
         self.environ = environ
-        self.path = environ['PATH_INFO']
+        self.path = environ.get('PATH_INFO','/')
         self.META = environ
         self.method = environ['REQUEST_METHOD'].upper()

Also it would work, if we change the http://code.djangoproject.com/attachment/ticket/2407/django.cgi file and insert:

...
 def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

comment:10 Changed 8 years ago by Marc Fargas <telenieko@…>

uhmm.. as per my last comment:

    cgi.py and cgi.txt is the django.cgi splitted, notes:

    * It seems that the http://www.djangoproject.com/documentation/ page is not in SVN so I cannot add a link to /cgi/ somebody will need to do that!
    * The reporter or some volunteer could please try the patch? the cgi.txt doc explains how to set the wrapper.
    * How to write a test for this? no idea ;) 

So, the official docs for this patch are on the cgi.txt file, they are not in the "Documentation" section of djangoproject.com as the patch is still pending to be applied. And cgi.py is the python code to run the thing ;)
The best should be the cgi.py file as it's the best choice as it's the last patch and the candidate to be checked in. Read the cgi.txt file for set-up information (it's almost like the fcgi wrapper).

About the files:

  • cgi.py (2.2 kB) - to be placed in django/core/servers
  • cgi.txt (1.3 kB) - to be placed in docs/

Hope this helps.

comment:11 follow-up: Changed 8 years ago by Jedie

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module!
What's about cgi_server.py?

comment:12 in reply to: ↑ 11 Changed 8 years ago by Marc Fargas <telenieko@…>

Replying to Jedie:

It's not a good idea to name a file cgi.py! So you override python's builtin cgi module!
What's about cgi_server.py?

from django.core.servers.cgi import runcgi
import django.core.servers.cgi.runcgi

is not

import cgi

This cgi.py is deep inside django and the documentation instructs you to import "runcgi" not "cgi" so you should have no trouble (I see no reason you would import the builtin "cgi" and this "cgi" on the wrapper script anyway). But it can be renamed anyway, something more to be discussed!

Call to volunteers: Write tests for this ticket! ;)

comment:13 Changed 8 years ago by Jedie

Yes, you are right. I have not put cgi.py into django/core/servers. I saved it into the root and make "from cgi import runcgi"... So i get an funny error, because the builtin file was overwritten ;)

Two problems remaining:

  1. I must use use the environ['PATH_INFO'] = environ.get('PATH_INFO',"/") fix from above.
  1. My http headers was not send in the right order:
    • The first Head line is Status: 500 INTERNAL SERVER ERROR
    • second line: Vary: Cookie
    • and the last line: Content-Type: text/html

So i have made a silly patch:

...
    def send_content_type(response_headers):
        for no, header in enumerate(response_headers):
            if header[0].lower() == "content-type":
                sys.stdout.write('%s: %s\r\n' % header)
                del(response_headers[no])
                return response_headers

        sys.stdout.write('Content-Type: text/html\r\n')
        sys.stdout.write('Warning: Content Type not send!') # Bullshit?!?


    def write(data):
        if not headers_set:
            raise AssertionError("write() before start_response()")

        elif not headers_sent:
            # Before the first output, send the stored headers
            status, response_headers = headers_sent[:] = headers_set

            response_headers = send_content_type(response_headers) # Send Content-Type first

            sys.stdout.write('Status: %s\r\n' % status)
            for header in response_headers:
                sys.stdout.write('%s: %s\r\n' % header)
            sys.stdout.write('\r\n')

        sys.stdout.write(data)
        sys.stdout.flush()
...

comment:14 Changed 8 years ago by anonymous

FWIW, when using django.cgi, I had to specify the PATH_INFO like this:

environ['PATH_INFO']         = environ.get('REDIRECT_URL',"/") 

instead of :

def run_with_cgi(application):
 
     environ                      = dict(os.environ.items())
+     environ['PATH_INFO']         = environ.get('PATH_INFO',"/")
     environ['wsgi.input']        = sys.stdin
     environ['wsgi.errors']       = sys.stderr
     environ['wsgi.version']      = (1,0)

so that I got the correct path passed to django. Perhaps there is a different way to do this, but it worked for me.

comment:15 Changed 8 years ago by anonymous

Why in god's name a patch for django? There are couple of CGI-WSGI wrappers out there. nobody has to patch django for CGI support :-/

comment:16 Changed 8 years ago by anonymous

Why not include the small CGI Handler into django directly? It's small! So CGI guys need no extra files from somewhere.

comment:17 Changed 8 years ago by anonymous

The empty PATH_INFO problem is the same with SCGI or fastCGI. There exist the ticket:3414 for this.

comment:18 Changed 8 years ago by Thomas Güttler

print in python code should go stderr (logfile):

===> diff -u cgi.py ~modarch/django/trunk/django/core/servers/cgi.py
--- cgi.py 2007-05-07 16:49:30.313178117 +0200
+++ /home/modarch/django/trunk/django/core/servers/cgi.py 2007-05-07 17:11:09.788411072 +0200
@@ -10,6 +10,7 @@

def runcgi():

environ = dict(os.environ.items())

+ environPATH_INFO? = environ.get('PATH_INFO',"/")

environwsgi.input? = sys.stdin
environwsgi.errors? = sys.stderr
environwsgi.version? = (1,0)

@@ -26,6 +27,9 @@

headers_set = []
headers_sent = []

+
+ stdout=sys.stdout
+ sys.stdout=sys.stderr # print should go to stderr (logfile)

def write(data):

if not headers_set:

@@ -34,13 +38,13 @@

elif not headers_sent:

# Before the first output, send the stored headers
status, response_headers = headers_sent[:] = headers_set

  • sys.stdout.write('Status: %s\r\n' % status)

+ stdout.write('Status: %s\r\n' % status)

for header in response_headers:

  • sys.stdout.write('%s: %s\r\n' % header)
  • sys.stdout.write('\r\n')

+ stdout.write('%s: %s\r\n' % header)
+ stdout.write('\r\n')

  • sys.stdout.write(data)
  • sys.stdout.flush()

+ stdout.write(data)
+ stdout.flush()

def start_response(status,response_headers,exc_info=None):

if exc_info:

comment:19 Changed 8 years ago by Marc Fargas <telenieko@…>

Why ??
stdout is stdout, and stderr is stderr they are and mean different things.

Why should stdout = stderr ? If you are using print statemens for debuggin please use the logging module ;)

Changed 8 years ago by Thomas Güttler

Changes to cgi.py: include SCRIPT_NAME in PATH_INFO and connect stdout to stderr (print to logfile)

comment:20 Changed 8 years ago by Jens Diemer

  • Cc django_trac@… added

comment:21 Changed 7 years ago by jacob

  • Resolution set to wontfix
  • Status changed from new to closed

Django just isn't designed to run under CGI.

It won't run under OS/2, either.

comment:22 Changed 7 years ago by limon

Use wsgiref module for django cgi support

Changed 7 years ago by limon

run django as cgi with wsgiref CGIHandler

comment:23 Changed 5 years ago by DremLIN

  • Resolution wontfix deleted
  • Status changed from closed to reopened
  • Summary changed from [patch] CGI Support for django to CGI Support for django
  • Version changed from SVN to 1.1

Hello,

In my hosting, I have: Python 2.3 run with Apache 1.3 as CGI. I install Django 1.1.2, but can't run "mysite" application.

I try dj_wsgiref_cgihandler.py by limon and I get error:
Status: 500 Dude, this is whack! A server error occurred. Please contact the administrator.

Please give me working example index.cgi for CGI.2.py (6.8 kB) by Martin or CGI_Server.py (1.2 kB) by Martin.

My e-mail: dremlin.ru [at] gmail.com

comment:24 Changed 5 years ago by russellm

  • Resolution set to wontfix
  • Status changed from reopened to closed

Django doesn't support CGI for deployment, and Trac isn't a user support forum. If you have a howto question, please ask on Django-users, but be advised -- Django doesn't support CGI for a very good reason.

Note: See TracTickets for help on using tickets.
Back to Top