Opened 3 years ago

Last modified 3 years ago

#32573 closed Bug

Query optimization in YearLookup breaks filtering by "__iso_year" — at Version 1

Reported by: Florian Demmer Owned by: Florian Demmer
Component: Database layer (models, ORM) Version: 2.2
Severity: Normal Keywords:
Cc: Sigurd Ljødal Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Florian Demmer)

The optimization to use BETWEEN instead of the EXTRACT operation in YearLookup is also registered for the "__iso_year" lookup, which breaks the functionality provided by ExtractIsoYear when used via the lookup.

This has unfortunately been broken ever since ExtractIsoYear was introduced in Django 2.2 via #28649 and wasn't easy to track down since ExtractIsoYear when used by itself eg. in an annotation works perfectly fine. Just when using the lookup in a filter, the optimization is used (even when explicitly using an annotation):

# annotation works
>>> qs = DTModel.objects.annotate(extracted=ExtractIsoYear('start_date')).only('id')
>>> print(qs.query)
SELECT "db_functions_dtmodel"."id", EXTRACT('isoyear' FROM "db_functions_dtmodel"."start_date") AS "extracted" FROM "db_functions_dtmodel"

# explicit annotation used in filter does not use "extracted" and adds BETWEEN
>>> print(qs.filter(extracted=2020).query)
SELECT "db_functions_dtmodel"."id", EXTRACT('isoyear' FROM "db_functions_dtmodel"."start_date") AS "extracted" FROM "db_functions_dtmodel" WHERE "db_functions_dtmodel"."start_date" BETWEEN 2020-01-01 AND 2020-12-31

# implicit lookup uses BETWEEN
print(DTModel.objects.filter(start_date__iso_year=2020).only('id').query)
SELECT "db_functions_dtmodel"."id" FROM "db_functions_dtmodel" WHERE "db_functions_dtmodel"."start_date" BETWEEN 2020-01-01 AND 2020-12-31

This results in the wrong data being returned by filters using iso_year.

This PR fixes the behaviour, reverts the invalid changes to the tests and extends one test to catch this problem: https://github.com/django/django/pull/14157

Change History (1)

comment:1 by Florian Demmer, 3 years ago

Description: modified (diff)
Has patch: set
Owner: changed from nobody to Florian Demmer
Status: newassigned
Note: See TracTickets for help on using tickets.
Back to Top