Twisted support for Django with Defer enable

[cenyongh@…]

In #172,jnoetzelman@… has submited a method to make Django run on top of twisted.He use twsited WSGIResource to wrap the django'WSGIHandler.But this method can not support Defer. The whole structure of this method is similiar to this diagram:

    (Twisted)                 (Django)
    WSGIResource
         |
        \/
    WSGIHandler     -->      WSGIHandler

When http request come in WSGIResource will create a new Twisted WSGIHandler then this new created Handler will ask Django's WSGIHandler to process the request,after the response send back by django's handler,twisted handler will write it back to the client.

But if the django's handler send back a defer,the whole train will break.

My method has to subclass all the class in the diagram

class WrapDjangoWSGIHandler( BaseHandler ):
    def __call__( self, environ, start_response ):
        from django.conf import settings

        if settings.ENABLE_PSYCO:
            import psyco
            psyco.profile()

        # Set up middleware if needed. We couldn't do this earlier,because settings weren't available.
        if self._request_middleware is None:
            self.load_middleware ()

        dispatcher.send( signal=signals.request_started )
        try:
            request = WSGIRequest( environ )

            #   modified by cenyongh to support defer
            di = defer.maybeDeferred( self.get_response, request.path,request )
            def cb( response ):
                return self.done( response, start_response )
            di.addCallback( cb )
        finally:
            dispatcher.send( signal=signals.request_finished )

        return di;

    def done( self, response, start_response ):
        # Apply response middleware
        for middleware_method in self._response_middleware:
            response = middleware_method( request, response )
        try:
            status_text = STATUS_CODE_TEXT[ response.status_code]
        except KeyError:
            status_text = 'UNKNOWN STATUS CODE'
        status = '%s %s' % ( response.status_code, status_text )
        response_headers = response.headers.items ()
        for c in response.cookies.values():
            response_headers.append( ( 'Set-Cookie', c.output(header='' ) ) )
        start_response( status, response_headers )
        return response

class WrapTW2WSGIResource( wsgi.WSGIResource ):
    def renderHTTP( self, req ):
        from twisted.internet import reactor
        # Do stuff with WSGIHandler.

        #   modified by cenyongh to support defer
        handler = WrapTW2WSGIHandler( self.application, req )

        # Get deferred
        d = handler.responseDeferred
        # Run it in a thread
        reactor.callInThread( handler.run )
        return d

class WrapTW2WSGIHandler( wsgi.WSGIHandler ):
    def run( self ):
        from twisted.internet import reactor
        # Called in application thread
        try:
            result = self.application ( self.environment,self.startWSGIResponse )

            #   modified by cenyongh to support defer
            def cb( response ):
                self.handleResult( [response.content] )
            result.addCallback( cb )

        except Exception, e:
            if not self.headersSent:
                reactor.callFromThread( self.__error, failure.Failure())
            else:
                reactor.callFromThread( self.stream.finish,failure.Failure() )

---

Then the server could be construt as #172 did.But the follow lines

test = wsgi.WSGIResource(AdminMediaHandler(WSGIHandler()))
site = server.Site(res)

should be change to

test = WrapTW2WSGIResource( WrapDjangoWSGIHandler() )
site = server.Site( test )

---

I have test with the views.py like this? And it get work.

def index( request ):
    return render_to_response( 'world/index.html', {} )


def dindex( request ):
    def cb( raw_html ):
        return render_to_response( 'world/index.html', {} )

    d = client.getPage( "http://www.gmail.com" )
    d.addCallback( cb )
    return d

[smurf]

It's not that easy.

Open problems:

• doesn't work as written (done() needs a request parameter)

• calls dispatcher.send(signal=signals.request_finished) too early (before running the response middleware).

• missing error handling (cf. addErrback)

• interacts badly with response middleware that blocks, or tries to finish a database transaction (because it doesn't run in the Django thread any more), or ...

In other words: bad idea.

Better idea:

Your Django code runs in a thread and blocks. The Twisted code that returns a Deferred runs in the main thread and doesn't. Thus, you really should run the offending, deferred-returning code in the main thread and not have Django deal with Deferreds at all.

A recipe for doing this is in ​http://twistedmatrix.com/trac/ticket/1042.