id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
13214	Random OperationalError when using FastCGI (+ possible solutions)	Henrique C. Alves	nobody	"Possible solution: http://groups.google.com/group/django-users/browse_thread/thread/2c7421cdb9b99e48

> Until recently I was curious to test
> this on Django 1.1.1. Will this 
> exception be thrown again... surprise,
> there it was again.  It took me some
> time to debug this, helpful hint was
> that it only  shows when (pre)forking.
> So for those who getting randomly
> those exceptions, I can say... fix 
> your code :)  Ok.. seriously, there
> are always few ways of doing this, so
> let me  firs explain where is a
> problem first.  If you access database
> when any of your modules will import
> as, e.g.  reading configuration from
> database then you will get this error.
> When your fastcgi-prefork application
> starts, first it imports all  modules,
> and only after this forks children. 
> If you have established db connection
> during import all children  processes
> will have an exact copy of that
> object.  This connection is being
> closed at the end of request phase 
> (request_finished signal).  So first
> child which will be called to process
> your request, will  close this
> connection.  But what will happen to
> the rest of the child processes?  They
> will believe that they have open and
> presumably working  connection to the
> db, so any db operation will cause an
> exception.  Why this is not showing in
> threaded execution model? I suppose
> because  threads are using same object
> and know when any other thread is 
> closing connection.  How to fix this? 
> Best way is to fix your code... but
> this can be difficult sometimes. 
> Other option, in my opinion quite
> clean, is to write somewhere in your 
> application small piece of code:

{{{
    from django.db import connection 
    from django.core import signals 
    def close_connection(**kwargs): 
        connection.close() 
    signals.request_started.connect(close_connection) 
}}}

Not ideal thought, connecting twice to the DB is a workaround at best.

----------

Possible solution: using connection pooling (pgpool, pgbouncer), so you have DB connections pooled and stable, and handed fast to your FCGI daemons.

The problem is that this triggers another bug, psycopg2 raising an *InterfaceError* because it's trying to disconnect twice (pgbouncer already handled this).

Now the culprit is Django signal *request_finished* triggering *connection.close()*, and failing loud even if it was already disconnected. I don't think this behavior is desired, as if the request already finished, we don't care about the DB connection anymore. A patch for correcting this should be simple.

The relevant traceback:

{{{
     /usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/core/handlers/wsgi.py in __call__(self=<django.core.handlers.wsgi.WSGIHandler object at 0x24fb210>, environ={'AUTH_TYPE': 'Basic', 'DOCUMENT_ROOT': '/storage/test', 'GATEWAY_INTERFACE': 'CGI/1.1', 'HTTPS': 'off', 'HTTP_ACCEPT': 'application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate', 'HTTP_AUTHORIZATION': 'Basic dGVzdGU6c3VjZXNzbw==', 'HTTP_CONNECTION': 'keep-alive', 'HTTP_COOKIE': '__utma=175602209.1371964931.1269354495.126938948...none); sessionid=a1990f0d8d32c78a285489586c510e8c', 'HTTP_HOST': 'www.rede-colibri.com', ...}, start_response=<function start_response at 0x24f87d0>)
      246                 response = self.apply_response_fixes(request, response)
      247         finally:
      248             signals.request_finished.send(sender=self.__class__)
      249 
      250         try:
    global signals = <module 'django.core.signals' from '/usr/local/l.../Django-1.1.1-py2.6.egg/django/core/signals.pyc'>, signals.request_finished = <django.dispatch.dispatcher.Signal object at 0x1975710>, signals.request_finished.send = <bound method Signal.send of <django.dispatch.dispatcher.Signal object at 0x1975710>>, sender undefined, self = <django.core.handlers.wsgi.WSGIHandler object at 0x24fb210>, self.__class__ = <class 'django.core.handlers.wsgi.WSGIHandler'>
     /usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/dispatch/dispatcher.py in send(self=<django.dispatch.dispatcher.Signal object at 0x1975710>, sender=<class 'django.core.handlers.wsgi.WSGIHandler'>, **named={})
      164 
      165         for receiver in self._live_receivers(_make_id(sender)):
      166             response = receiver(signal=self, sender=sender, **named)
      167             responses.append((receiver, response))
      168         return responses
    response undefined, receiver = <function close_connection at 0x197b050>, signal undefined, self = <django.dispatch.dispatcher.Signal object at 0x1975710>, sender = <class 'django.core.handlers.wsgi.WSGIHandler'>, named = {}
     /usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/__init__.py in close_connection(**kwargs={'sender': <class 'django.core.handlers.wsgi.WSGIHandler'>, 'signal': <django.dispatch.dispatcher.Signal object at 0x1975710>})
       63 # when a Django request is finished.
       64 def close_connection(**kwargs):
       65     connection.close()
       66 signals.request_finished.connect(close_connection)
       67 
    global connection = <django.db.backends.postgresql_psycopg2.base.DatabaseWrapper object at 0x17b14c8>, connection.close = <bound method DatabaseWrapper.close of <django.d...ycopg2.base.DatabaseWrapper object at 0x17b14c8>>
     /usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/backends/__init__.py in close(self=<django.db.backends.postgresql_psycopg2.base.DatabaseWrapper object at 0x17b14c8>)
       74     def close(self):
       75         if self.connection is not None:
       76             self.connection.close()
       77             self.connection = None
       78 
    self = <django.db.backends.postgresql_psycopg2.base.DatabaseWrapper object at 0x17b14c8>, self.connection = <connection object at 0x1f80870; dsn: 'dbname=co...st=127.0.0.1 port=6432 user=postgres', closed: 2>, self.connection.close = <built-in method close of psycopg2._psycopg.connection object at 0x1f80870>
}}}

Exception handling here could add more leniency:

{{{
**/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/__init__.py**

       63 # when a Django request is finished.
       64 def close_connection(**kwargs):
       65     connection.close()
       66 signals.request_finished.connect(close_connection)
}}}

Or it could be handled better on psycopg2, so to not throw fatal errors if all we're trying to do is disconnect and it already is:

{{{
**/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/backends/__init__.py**

       74     def close(self):
       75         if self.connection is not None:
       76             self.connection.close()
       77             self.connection = None
}}}

Other than that, I'm short on ideas. Not sure if this is supposed to be fixed on 1.2, maybe multi-db support refactored things and dealt with this issue. I'll try testing with a 1.2 enviro.
"		closed	Core (Other)	1.1		invalid		hcarvalhoalves@…	Unreviewed	0	0	0	0	0	0