Opened 6 months ago
Closed 6 months ago
#35455 closed Bug (invalid)
psycopg3 warns about connections not being closed
Reported by: | HTErik | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 5.0 |
Severity: | Normal | Keywords: | psycopg |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
When running
- a Django 5.0.6 application
- psycopg 3.1.19
- with db
"ENGINE": "django.db.backends.postgresql"
- with Python warnings as errors enabled
This occasionally prints following error from psycopg:
ResourceWarning: connection <psycopg.Connection [IDLE] (REDACTED) at 0x710d29abe120> was deleted while still open. Please use 'with' or '.close()' to close the connection
This warning comes from the __del__
function in psycopg.connection.BaseConnection
.
I have no idea how to reproduce this.
I tried patching psycopg to print a stack trace whenever a connection is opened and closed. This tells me that the connection that was deleted before being closed, was opened through the following code path:
File "blah.py", line 193, in __heartbeat MyModel.objects.bulk_update(items_to_run, ["last_update"]) File "python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "python3.12/site-packages/django/db/models/query.py", line 922, in bulk_update with transaction.atomic(using=self.db, savepoint=False): File "python3.12/site-packages/django/db/transaction.py", line 198, in __enter__ if not connection.get_autocommit(): File "python3.12/site-packages/django/db/backends/base/base.py", line 450, in get_autocommit self.ensure_connection() File "python3.12/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "python3.12/site-packages/django/db/backends/base/base.py", line 275, in ensure_connection self.connect() File "python3.12/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "python3.12/site-packages/django/db/backends/base/base.py", line 256, in connect self.connection = self.get_new_connection(conn_params)
Change History (2)
comment:1 by , 6 months ago
comment:2 by , 6 months ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
Hi HTErik, I recommend you share more details of this in the async section of the forum: https://forum.djangoproject.com/c/internals/async/8, as this is the place for discussion and support.
If there's an agreement that there is a bug in Django or that the docs need updating, come back and I'll be happy to accept improvements 👍
I have identified the problem now.
It happens when one performs Django queries from a separate thread. The DB connections in Django are thread local inside the
ConnectionHandler
, so when the thread finishes and is eventually GCd, the thread locals holding the DB connections in Django are also GCd, without any priorclose()
.To solve it, one must call this in a
finally
clause of all threads that run Django queries.I'm not sure if this is expected from users or if this is something Django should handle internally.
The documentation on this topic could use some more details. https://docs.djangoproject.com/en/5.0/ref/databases/#connection-management