InterfaceError: connection already closed in PostgreSQL

[Ilya Antipenko]

I have 1.6.5 django with psycopg 2.5.3.
This exception happened when connection was closed by postgres.

Related issue #21553

2014-08-19 13:32:46.414848 - ERROR:apscheduler.executors.default:Job "expire_key_notify (trigger: interval[0:01:00], next run at: 2014-08-19 13:33:46 EEST)" raised an exception
2014-08-19 13:32:46.414934 - Traceback (most recent call last):
2014-08-19 13:32:46.415001 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/apscheduler/executors/base.py", line 108, in run_job
2014-08-19 13:32:46.415056 -     retval = job.func(*job.args, **job.kwargs)
2014-08-19 13:32:46.415105 -   File "./test/test_app/workflow/monitor.py", line 508, in expire_key_notify
2014-08-19 13:32:46.415151 -     for event in Event.objects.filter(done=False, event_after__lte=TimeManager.get_time()).all():
2014-08-19 13:32:46.415199 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/models/query.py", line 96, in __iter__
2014-08-19 13:32:46.415245 -     self._fetch_all()
2014-08-19 13:32:46.415293 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/models/query.py", line 854, in _fetch_all
2014-08-19 13:32:46.415338 -     self._result_cache = list(self.iterator())
2014-08-19 13:32:46.415385 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
2014-08-19 13:32:46.415431 -     for row in compiler.results_iter():
2014-08-19 13:32:46.415524 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
2014-08-19 13:32:46.415574 -     for rows in self.execute_sql(MULTI):
2014-08-19 13:32:46.415620 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 785, in execute_sql
2014-08-19 13:32:46.415667 -     cursor = self.connection.cursor()
2014-08-19 13:32:46.415711 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 160, in cursor
2014-08-19 13:32:46.415757 -     cursor = self.make_debug_cursor(self._cursor())
2014-08-19 13:32:46.415802 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 134, in _cursor
2014-08-19 13:32:46.415848 -     return self.create_cursor()
2014-08-19 13:32:46.415896 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
2014-08-19 13:32:46.415943 -     six.reraise(dj_exc_type, dj_exc_value, traceback)
2014-08-19 13:32:46.415992 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 134, in _cursor
2014-08-19 13:32:46.416078 -     return self.create_cursor()
2014-08-19 13:32:46.416127 -   File "/home/user/projects/virtualenvs/test-system2/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 137, in create_cursor
2014-08-19 13:32:46.416176 -     cursor = self.connection.cursor()
2014-08-19 13:32:46.416227 - InterfaceError: connection already closed

Additional, I have other exception when postgres server was restarted

ERROR:apscheduler.executors.default:Job "sync_db_with_queue (trigger: interval[0:02:00], next run at: 2014-08-20 16:20:18 EEST)" raised an exception
Traceback (most recent call last):
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/apscheduler/executors/base.py", line 108, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/user/projects/test-system/test/test_app/workflow/monitor.py", line 369, in sync_db_with_queue
    for task in JobProcessReceipt.objects.filter(status=JobProcessReceipt.enum_receipt_status.queue, create_ts__lt=int(time.time()-30)):
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/models/query.py", line 96, in __iter__
    self._fetch_all()
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/user/projects/.test-system/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
OperationalError: terminating connection due to administrator command
SSL connection has been closed unexpectedly

[Tim Graham]

Why do you think this is a bug in Django? How do you expect Django to react when the database terminates the connection?

[Aymeric Augustin]

Yes, unless I missed something, this is the expected behavior.

[Ilya Antipenko]

Replying to timgraham:

Why do you think this is a bug in Django? How do you expect Django to react when the database terminates the connection?

I think Django should do reconnect, when connection is lost. Or I should use somethink like this ​http://www.tryolabs.com/Blog/2014/02/12/long-time-running-process-and-django-orm/ ?

I have worker, which work with database(Model.objects.all()) every 2 minutes and I got "connection already closed" error from time to time

Server restart - it's example.

I think this is like #15802

[Aymeric Augustin]

Yes, it's a duplicate of #15802. Until today I thought it was impossible, but I just had a new idea, so I'm reopened that ticket.