#35294 closed Bug (fixed)
Queryset explain can be truncated.
Reported by: | Gordon Wrigley | Owned by: | Adam Johnson |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 4.2 |
Severity: | Normal | Keywords: | explain |
Cc: | Gordon Wrigley | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Django 4.2.10
Python 3.10.13
Postgres 15.6
Psycopg2 2.9.9
I have some very complex querysets I'm trying to optimize and what I've run into is explain output is being truncated at 100 lines. As far as I can tell this is because of this function on django.db.models.sql.compiler.SQLCompiler.
def explain_query(self): result = list(self.execute_sql()) # Some backends return 1 item tuples with strings, and others return # tuples with integers and strings. Flatten them out into strings. format_ = self.query.explain_info.format output_formatter = json.dumps if format_ and format_.lower() == "json" else str for row in result[0]: if not isinstance(row, str): yield " ".join(output_formatter(c) for c in row) else: yield row
Where result[0]
is ignoring additional results. Monkey patching it to
def explain_query(self): results = list(self.execute_sql()) # Some backends return 1 item tuples with strings, and others return # tuples with integers and strings. Flatten them out into strings. format_ = self.query.explain_info.format output_formatter = json.dumps if format_ and format_.lower() == "json" else str for result in results: for row in result: if not isinstance(row, str): yield " ".join(output_formatter(c) for c in row) else: yield row
Gets me the full explain output.
Change History (10)
comment:1 by , 8 months ago
Type: | Uncategorized → Bug |
---|
comment:2 by , 8 months ago
comment:3 by , 8 months ago
Yes, although I don't know if I did it right. I'm not clear where the system is getting psycopg2 from. I installed 3 and nothing changed. But I don't know if at that point it was using 2 or 3.
comment:4 by , 8 months ago
Summary: | Queryset explain truncated → Queryset explain can be truncated. |
---|---|
Triage Stage: | Unreviewed → Accepted |
Seems valid. Would like like to prepare a patch? (a regression test can be tricky.)
comment:5 by , 8 months ago
Has patch: | set |
---|---|
Owner: | changed from | to
Status: | new → assigned |
comment:6 by , 8 months ago
I made a PR but the test is ugly and a bit hacky to force a 100+ line explain plan by joining 100 1-row generate_series()
. Happy to hear any alternatives.
comment:7 by , 8 months ago
Patch needs improvement: | set |
---|
comment:8 by , 8 months ago
Patch needs improvement: | unset |
---|---|
Triage Stage: | Accepted → Ready for checkin |
Interesting, do you observe the same with psycopg 3+?