id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
31717	WSGI calling close causes premature close_if_unusable_or_obsolete calls during transaction?	Yu Li	nobody	"I'm having an issue with uwsgi+postgresql+django combo. This issue does not exist when I use the manage runserver development server. 

The issue is, I'm constantly receiving this error from django
{{{
prod_1     | Traceback (most recent call last):
prod_1     |   File ""/usr/local/lib/python3.8/dist-packages/django/db/backends/base/base.py"", line 253, in _cursor
prod_1     |     return self._prepare_cursor(self.create_cursor(name))
prod_1     |   File ""/usr/local/lib/python3.8/dist-packages/django/utils/asyncio.py"", line 26, in inner
prod_1     |     return func(*args, **kwargs)
prod_1     |   File ""/usr/local/lib/python3.8/dist-packages/django/db/backends/postgresql/base.py"", line 231, in create_cursor
prod_1     |     cursor = self.connection.cursor()
prod_1     | psycopg2.InterfaceError: connection already closed
}}}

Possibly related:
https://code.djangoproject.com/ticket/25362
https://code.djangoproject.com/ticket/15802

However, I backtraced the stack of this error and found out Django is calling {{{BaseDatabaseWrapper.close()}}} right before this error.
Specifically, this line:
{{{
def close_if_unusable_or_obsolete(self):
        """"""
        Close the current connection if unrecoverable errors have occurred
        or if it outlived its maximum age.
        """"""
        if self.connection is not None:
            # If the application didn't restore the original autocommit setting,
            # don't take chances, drop the connection.
            if self.get_autocommit() != self.settings_dict[""AUTOCOMMIT""]:
                self.close()
                return
}}}

And when the error happens, we have {{{self.get_autocommit()}}} returning {{{False}}}, and {{{self.settings_dict[""AUTOCOMMIT""]}}} returning {{{True}}}.

Then I backtraced where {{{self.set_autocommit(False)}}} was called and found out that this happened in {{{Atomic.__enter__()}}}
{{{
    def __enter__(self):
        connection = get_connection(self.using)

        if not connection.in_atomic_block:
            # Reset state when entering an outermost atomic block.
            connection.commit_on_exit = True
            connection.needs_rollback = False
            if not connection.get_autocommit():
                # Pretend we're already in an atomic block to bypass the code
                # that disables autocommit to enter a transaction, and make a
                # note to deal with this case in __exit__.
                connection.in_atomic_block = True
                connection.commit_on_exit = False

        if connection.in_atomic_block:
            # We're already in a transaction; create a savepoint, unless we
            # were told not to or we're already waiting for a rollback. The
            # second condition avoids creating useless savepoints and prevents
            # overwriting needs_rollback until the rollback is performed.
            if self.savepoint and not connection.needs_rollback:
                sid = connection.savepoint()
                connection.savepoint_ids.append(sid)
            else:
                connection.savepoint_ids.append(None)
        else:
            connection.set_autocommit(False, force_begin_transaction_with_broken_autocommit=True)
            connection.in_atomic_block = True
}}}


I think something bad is happening when wsgi calls {{{HttpResponseBase.close()}}} during a transaction, because

{{{
def close_old_connections(**kwargs):
    for conn in connections.all():
        conn.close_if_unusable_or_obsolete()


signals.request_started.connect(close_old_connections)
signals.request_finished.connect(close_old_connections)
}}}

and 

{{{

    # The WSGI server must call this method upon completion of the request.
    # See http://blog.dscpl.com.au/2012/10/obligations-for-calling-close-on.html
    def close(self):
        for closer in self._resource_closers:
            try:
                closer()
            except Exception:
                pass
        # Free resources that were still referenced.
        self._resource_closers.clear()
        self.closed = True
        signals.request_finished.send(sender=self._handler_class)

}}}

causes {{{close_if_unusable_or_obsolete()}}} to be called during this transaction, so {{{self.get_autocommit() != self.settings_dict[""AUTOCOMMIT""]}}} closed the connection.

"	Uncategorized	closed	Uncategorized	3.0	Normal	needsinfo		rm_	Unreviewed	0	0	0	0	0	0