Opened 4 months ago
Closed 4 months ago
#35559 closed Cleanup/optimization (fixed)
Calling list() on an empty sliced union still causes a database query
Reported by: | Lucidiot | Owned by: | Simon Charette |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 5.0 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
When upgrading from Django 4.1 to 4.2, some of our unit tests failed because of unexpected SQL queries, and those queries are still present in Django 5.0.
We have a Django REST Framework API view, which relies on Django's Paginator
to do its pagination. It uses a QuerySet
which involves a UNION
of two other queries that both have some filters. Those filters can cause the whole union to be empty. The Paginator
knows that the result count is 0, so it returns a page of results by slicing with [0:0]
.
In Django 4.1, calling list()
on that sliced union would cause no query and an empty list is returned, but starting with 4.2, a query runs:
qs = Thing.objects.filter(id__in=Thing.objects.none()) list(qs) # 0 queries list(qs[0:0]) # 0 queries list(qs.union(qs)) # 0 queries list(qs.union(qs)[0:0]) # 1 query
( SELECT "app_thing"."id" AS "col1" FROM "app_thing" WHERE 0 = 1 ) UNION ( SELECT "app_thing"."id" AS "col1" FROM "app_thing" WHERE 0 = 1 )
I think this extra query was added in the fix for #34125.
Change History (6)
comment:1 by , 4 months ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
Triage Stage: | Unreviewed → Accepted |
Type: | Bug → Cleanup/optimization |
comment:2 by , 4 months ago
Has patch: | set |
---|
comment:3 by , 4 months ago
Needs tests: | set |
---|
comment:4 by , 4 months ago
Needs tests: | unset |
---|
Wrong assessment, test coverage exists as the crash detailed in #34125 that this ticket change the approach for (to prevent unnecessary queries) only manifests itself on Postgres.
comment:5 by , 4 months ago
Triage Stage: | Accepted → Ready for checkin |
---|
I confirm that c2cc80756b8949cdd87b88bbfdfee698ced441e0 focused on the correctness aspect of the problem (avoiding the crash) without focusing on some edge cases that might still allow the query eliding to take place.