Improve handling of database connection and cursor resources

[Jon Dufresne]

Some areas of the code do not always close a database connection or cursor object after use. Instead it waits for the Python garbage collector to remove it.

Like file resources, database resource ownership and lifetime should be deliberate and deterministic.

A common example is the _nodb_connection property which opens a connection but is rarely closed.

​https://github.com/django/django/blob/335c9c94acf263901fb023404408880245b0c4b4/django/db/backends/base/base.py#L609-L618

[Jon Dufresne]

​https://github.com/django/django/pull/12414

[Simon Charette]

I think the _nodb_cursor cursor approach makes a lot of sense. Have we experienced with trying to elevated RessourceWarnings to errors during the suite's execution?

[Jon Dufresne]

Have we experienced with trying to elevated RessourceWarnings to errors during the suite's execution?

That would be great. Unfortunately these typically occur inside a __del__ method, so this doesn't work out so well in practice. Here is the discussion the last time this idea was raised:

​https://github.com/django/django/pull/7676#issuecomment-266226305

If we can overcome this limitation, I'm very interested.

[Simon Charette]

Exceptions within __del__ are effectively turned into stdout messages and not raised but elevating them to errors make messages way more useful and explicit about their origin

e.g.

warnings.filterwarnings('error', 'unclosed', ResourceWarning)

Revealed a ton of other wise silenced messages of the following form when running the suite of a large Django project

ResourceWarning: unclosed file <_io.TextIOWrapper name='path-to-file.json' mode='r' encoding='UTF-8'>

[Jon Dufresne]

Makes sense. I've done that in PR ​https://github.com/django/django/pull/12423. This did not reveal any new ResourceWarning in the test suite for me.

[GitHub <noreply@…>]

In 2905b416:

Refs #31233 -- Added "error" filter for RuntimeWarning during tests.

[Mariusz Felisiak <felisiak.mariusz@…>]

In f48f671:

Refs #31233 -- Changed DatabaseWrapper._nodb_connection to _nodb_cursor().

It is now a method instead of a property and returns a context manager
that yields a cursor on entry and closes the cursor and connection upon
exit.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 3259983:

Fixed #31233 -- Closed database connections and cursors after use.