Opened 6 months ago
Last modified 5 months ago
#36380 closed Bug
SQL is formatted for logging regardless of whether it will be logged — at Initial Version
| Reported by: | Jacob Walls | Owned by: | Jacob Walls | 
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | dev | 
| Severity: | Release blocker | Keywords: | debug-sql format_debug_sql | 
| Cc: | Tim Graham, Simon Charette, Jacob Walls, Mariusz Felisiak | Triage Stage: | Ready for checkin | 
| Has patch: | yes | Needs documentation: | no | 
| Needs tests: | no | Patch needs improvement: | no | 
| Easy pickings: | no | UI/UX: | no | 
Description
I have an endpoint that generates several large SQL strings. It performs 10x worse (300ms -> 3000ms) on the main branch. Traced it to eager formatting of SQL.
To reproduce, print or throw inside format_debug_sql(), and run a project without logging queries (i.e. without a django.db entry in LOGGING configured at the DEBUG level).
- 
      
django/db/backends/base/operations.py
diff --git a/django/db/backends/base/operations.py b/django/db/backends/base/operations.py index fea73bc1e4..4c106f59e3 100644
a b class BaseDatabaseOperations: 791 791 792 792 def format_debug_sql(self, sql): 793 793 # Hook for backends (e.g. NoSQL) to customize formatting. 794 print('eager formatting...') 794 795 return sqlparse.format(sql, reindent=True, keyword_case="upper")  
Regression in d8f093908c504ae0dbc39d3f5231f7d7920dde37.
We might want a benchmark in django-asv for views with long SQL strings?
This fix works for me; I can submit a patch with tests in the next few days:
- 
      
django/db/backends/utils.py
diff --git a/django/db/backends/utils.py b/django/db/backends/utils.py index 568f510a67..ae810ffd12 100644
a b from hashlib import md5 10 10 from django.apps import apps 11 11 from django.db import NotSupportedError 12 12 from django.utils.dateparse import parse_time 13 from django.utils.functional import lazy 13 14 14 15 logger = logging.getLogger("django.db.backends") 15 16 … … class CursorDebugWrapper(CursorWrapper): 151 152 logger.debug( 152 153 "(%.3f) %s; args=%s; alias=%s", 153 154 duration, 154 self.db.ops.format_debug_sql(sql),155 lazy(self.db.ops.format_debug_sql)(sql), 155 156 params, 156 157 self.db.alias, 157 158 extra={