psycopg2.InterfaceError in sequential tests after HTTP 404

[alubbock]

I'm getting a curious error, where if I have two test methods, one using a force_login after a test with an HTTP 404, the second one fails. I've added a minimal reproducible example below. If the first method instead returns an HTTP 200, no error arises. If I use SQLite instead of Postgres, no error arises.

I'm using a custom user model with the custom_user package, with email instead of username, in case that's relevant.

Key package versions:
django 1.11.10, python 3.6.4, postgres 9.6, custom_user 0.7, django allauth 0.35.0

from django.test import TestCase
from django.contrib.auth import get_user_model


class TestViews(TestCase):
    @classmethod
    def setUpTestData(cls):
        cls.user = get_user_model().objects.create(email='test@example.com')

    def test_404(self):
        resp = self.client.get('_non_existent_URL')
        self.assertEquals(resp.status_code, 404)

    def test_login(self):
        # fails with postgres
        self.client.force_login(self.user)

Here's the traceback:

Traceback (most recent call last):
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/base/base.py", line 231, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 220, in create_cursor
    cursor = self.connection.cursor()
psycopg2.InterfaceError: connection already closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/alex/git/VU/myproj-web/myprojweb/tests/test_logins.py", line 16, in test_login
    self.client.force_login(self.user)
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/test/client.py", line 645, in force_login
    self._login(user, backend)
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/test/client.py", line 654, in _login
    if self.session:
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/test/client.py", line 459, in session
    session.save()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 81, in save
    return self.create()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 50, in create
    self._session_key = self._get_new_session_key()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 164, in _get_new_session_key
    if not self.exists(session_key):
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 46, in exists
    return self.model.objects.filter(session_key=session_key).exists()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/models/query.py", line 670, in exists
    return self.query.has_results(using=self.db)
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/models/sql/query.py", line 517, in has_results
    return compiler.has_results()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 858, in has_results
    return bool(self.execute_sql(SINGLE))
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 887, in execute_sql
    cursor = self.connection.cursor()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/base/base.py", line 231, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/base/base.py", line 231, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/Users/alex/miniconda3/envs/myproj/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 220, in create_cursor
    cursor = self.connection.cursor()
django.db.utils.InterfaceError: connection already closed

[Tim Graham]

I can't reproduce the problem with the test file you provided. Perhaps I'm missing some other detail. Can you please provide a sample project that reproduces the issue (without any third-party dependencies to rule out a problem there)?

[alubbock]

Project demonstrating the error

[alubbock]

The problem appears to be related to the presence of a custom 404 view. Please see attached project, minimally modified from a starter project. Thanks for taking a look.

[Tim Graham]

The issue doesn't occur until Django 1.10, after 5233b70070f8979f41ca1da2c1b1d78c8e30944e.

Your handler404 has a mistake where it passes the result of django.shortcuts.render() (which returns an HttpResponse) to HttpResponseNotFound. I'm surprised this works at all. The exception will be fixed if you change the handler to return HttpResponseNotFound(render(request, 'error404.html').content).

I'm not sure if Django should try to detect this mistake.

[alubbock]

Thanks so much, that fixes the problem.