Opened 12 months ago
Last modified 10 months ago
#35918 closed Cleanup/optimization
SQLCompiler.execute_sql result_type CURSOR usage can be minimized — at Version 1
| Reported by: | Raphael Gaschignard | Owned by: | Raphael Gaschignard |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | dev |
| Severity: | Normal | Keywords: | |
| Cc: | Raphael Gaschignard | Triage Stage: | Ready for checkin |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description (last modified by )
(This refactor is motivated by an ongoing experiment to integrate async cursor work into the ORM, by simplifying some cursor management)
In django.cdb.models.sql.compiler.Compiler.execute_sql, we can pass in the following result types: SINGLE, MULTI, NO_RESULTS, and CURSOR.
execute_sql's docstring to that effect does not reflect this.
SINGLEreturns a single row. It closes the cursor it uses to query.MULTIreturns many rows (wrapped in a cursor iterator). This either closes the cursor it uses to query, or returns an iterator that takes ownership of the cursor to close the cursor once reading of all the results are done.NO_RESULTSreturns nothing. It closes the cursor it uses to query.CURSORreturns the cursor, without closing the cursor, effectively making the caller in charge of closing the cursor
CURSOR returns an unclosed cursor that has to be manage by the caller. In practice, though, apart from a single test usage, Django's codebase currently only uses CURSOR to do one thing: get the number of rows, then close the cursor.
To simplify cursor resource management, I have a two-pronged proposal:
- a new result type,
ROW_COUNT, returns the rows and closes the cursor for you. This covers all non-test usage ofCURSORin Django currently. CURSORis renamed toLEAK_CURSOR, as a way to more strongly indicate that you are now in charge of the cursor
Main point here is to reduce the number of places an open cursor might come into play.