Opened 4 years ago

Closed 4 years ago

#31977 closed Bug (duplicate)

Transforms silently don't work in F and OuterRef expressions.

Reported by: Gordon Wrigley Owned by: nobody
Component: Database layer (models, ORM) Version: 3.1
Severity: Normal Keywords: function, lookup, transform, F, OuterRef
Cc: Adam Johnson Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I've seen this on different databases and different Django versions.

Here's Sqlite with Django 3.1.1

>>> qs = Order.objects.filter(submitted_time__date=F("submitted_time__date"))
>>> qs
<QuerySet []>
>>> print(qs.query)
SELECT <REDACTED> FROM "core_order" WHERE django_datetime_cast_date("core_order"."submitted_time", 'UTC', 'UTC') = "core_order"."submitted_time"

With a JSONField

>>> qs = Order.objects.filter(cruft__bob=F("cruft__fred"))
>>> print(qs.query)
SELECT <REDACTED> FROM "core_order" WHERE JSON_EXTRACT("core_order"."cruft", $."bob") = "core_order"."cruft"

With JSON in an OuterRef

>>> qs = Order.objects.annotate(prev=Subquery(Order.objects.filter(cruft__bob=OuterRef("cruft__fred")).values("pk")[:1]))
>>> print(qs.query)
SELECT <REDACTED>, (SELECT U0."id" FROM "core_order" U0 WHERE JSON_EXTRACT(U0."cruft", $."bob") = "core_order"."cruft" LIMIT 1) AS "prev" FROM "core_order"

And on Postgres

>>> qs = Order.objects.filter(submitted_time__date=F("submitted_time__date"))
>>> qs
<QuerySet []>
>>> print(qs.query)
SELECT <REDACTED> FROM "core_order" WHERE ("core_order"."submitted_time" AT TIME ZONE 'UTC')::date = "core_order"."submitted_time"

It does the same thing on Django 2.2 as well.

All these failures are totally silent, you can create and evaluate the queryset and the only indication that something is amiss is you get the wrong results. And I couldn't find any reference to this behaviour in the documentation or other tickets, although it's not an easy thing to search for.

Change History (3)

comment:1 by Adam Johnson, 4 years ago

Cc: Adam Johnson added

comment:2 by Gordon Wrigley, 4 years ago

Incidentally (and for anyone who ends up here trying to find a work around) functions work, so you can do Order.objects.filter(submitted_time__date=TruncDate("submitted_time")) for the OuterRef case you can annotate the transformed value onto the outer queryset and OuterRef to the annotated field.

comment:3 by Mariusz Felisiak, 4 years ago

Resolution: duplicate
Status: newclosed
Type: UncategorizedBug

Duplicate of #31639.

Note: See TracTickets for help on using tickets.
Back to Top