id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
31905	ASGI: Dead persistent postgres connections are not closed when the database is accessed in middleware	Michael Galler	Michael Galler	"How to reproduce:

Create a project with django-admin

Added persitent postgres connection


{{{
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'test_db',
        'USER': 'postgres',
        'PASSWORD': '1234',
        'HOST': 'localhost',
        'CONN_MAX_AGE': 60
    }
}

}}}

Add a middleware that accesses the database


{{{
from django.contrib.auth.models import User
from django.utils.deprecation import MiddlewareMixin


class FoobarMiddleware(MiddlewareMixin):

    def process_request(self, request):
        print(User.objects.all())
}}}

Add the middleware to settings, apply migrations.
Start ASGI Server for example with:

{{{
uvicorn --loop asyncio --workers 1 foobar.asgi:application --host 0.0.0.0 --reload
}}}

Visit localhost:8000 to open persistent connection

Restart Postgres, after that all connections get the following error


{{{
Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 3.1
Python Version: 3.8.5
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'foobar.middleware.FoobarMiddleware']



Traceback (most recent call last):
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\base\base.py"", line 237, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File ""c:\code\project\_lib\venv\lib\site-packages\django\utils\asyncio.py"", line 26, in inner
    return func(*args, **kwargs)
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\postgresql\base.py"", line 233, in create_cursor
    cursor = self.connection.cursor()

The above exception (connection already closed) was the direct cause of the following exception:
  File ""c:\code\project\_lib\venv\lib\site-packages\asgiref\sync.py"", line 330, in thread_handler
    raise exc_info[1]
  File ""c:\code\project\_lib\venv\lib\site-packages\django\core\handlers\exception.py"", line 38, in inner
    response = await get_response(request)
  File ""c:\code\project\_lib\venv\lib\site-packages\django\utils\deprecation.py"", line 126, in __acall__
    response = await sync_to_async(self.process_request)(request)
  File ""c:\code\project\_lib\venv\lib\site-packages\asgiref\sync.py"", line 296, in __call__
    ret = await asyncio.wait_for(future, timeout=None)
  File ""C:\Program Files\Python38\lib\asyncio\tasks.py"", line 455, in wait_for
    return await fut
  File ""C:\Program Files\Python38\lib\concurrent\futures\thread.py"", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File ""c:\code\project\_lib\venv\lib\site-packages\asgiref\sync.py"", line 334, in thread_handler
    return func(*args, **kwargs)
  File "".\foobar\middleware.py"", line 8, in process_request
    print(User.objects.all())
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\models\query.py"", line 263, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\models\query.py"", line 269, in __len__
    self._fetch_all()
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\models\query.py"", line 1303, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\models\query.py"", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\models\sql\compiler.py"", line 1152, in execute_sql
    cursor = self.connection.cursor()
  File ""c:\code\project\_lib\venv\lib\site-packages\django\utils\asyncio.py"", line 26, in inner
    return func(*args, **kwargs)
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\base\base.py"", line 259, in cursor
    return self._cursor()
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\base\base.py"", line 237, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\utils.py"", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\base\base.py"", line 237, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File ""c:\code\project\_lib\venv\lib\site-packages\django\utils\asyncio.py"", line 26, in inner
    return func(*args, **kwargs)
  File ""c:\code\project\_lib\venv\lib\site-packages\django\db\backends\postgresql\base.py"", line 233, in create_cursor
    cursor = self.connection.cursor()

Exception Type: InterfaceError at /
Exception Value: connection already closed
}}}


Only a restart of the application fixes the problem

This problem occurs only with ASGI, with WSGI for example the devserver this does not occur.
"	Bug	closed	Database layer (models, ORM)	3.1	Release blocker	fixed	ASGI Postgres Persistent	Andrew Godwin	Ready for checkin	1	0	0	0	0	0