Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#34527 closed Bug (invalid)

Unexpected behavior with division in aggregation

Reported by: Egor R Owned by: nobody
Component: Database layer (models, ORM) Version: 3.2
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Egor R)

I came across an unexpected behavior (code taken from a real project, and the models are renamed):

django.__version__
# '3.2.18'

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
    / Count("relatedmodel__id", filter=Q(relatedmodel__owner=user)),
).values_list("score", flat=True)
# <QuerySet [None, 10.0, 8.0, None, 9.0, None]>

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
).values_list("score", flat=True)
# <QuerySet [None, 40.0, 35.0, None, 37.0, None]>

MyModel.objects.all().annotate(
    score=Count("relatedmodel__id", filter=Q(relatedmodel__owner=user),),
).values_list("score", flat=True)
# <QuerySet [0, 4, 4, 0, 4, 0]>

Since we're specifying output_field=FloatField() for Sum, I expected to get 10, 8.75 and 9.25 as the results of the first query, but I'm getting 10.0/8.0/9.0.
I looked into SQL code generated by the query - there's no casting of Sum to float there, so it's somewhat understandable why it is happening. But, shouldn't Django cast Sum to float in SQL?

Explicitly casting Count as FloatField works, though:

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
    / Cast(Count("relatedmodel__id", filter=Q(relatedmodel__owner=user)), FloatField())
).values_list("score", flat=True)
# <QuerySet [None, 10.0, 8.75, None, 9.25, None]>

Relevant snippets of the models:

class MyModel(models.Model):
    ...

class RelatedModel(models.Model):
    points = models.PositiveSmallIntegerField(
        "description", default=10, null=True, blank=True
    )
    mymodel = models.ForeignKey("myapp.MyModel", on_delete=models.CASCADE)
    owner = models.ForeignKey(User, on_delete=models.CASCADE)
    ...

I created an MRE, will attach it to the ticket.
Tested on Postgres 12 (real project) and 15 (MRE).

PS Real project uses django-tenants, and I included it in the MRE as I tried to track down the bug. Can retest and create MRE without django-tenants if needed.

Attachments (1)

dj_agg_bug_clean.zip (9.3 KB ) - added by Egor R 2 years ago.
zip of the MRE

Download all attachments as: .zip

Change History (6)

by Egor R, 2 years ago

Attachment: dj_agg_bug_clean.zip added

zip of the MRE

comment:1 by Egor R, 2 years ago

Description: modified (diff)

comment:2 by David Sanders, 2 years ago

Hi,

Casting is required here, output_field just lets Django know what type the returned data is:

The output_field argument requires a model field instance, like IntegerField() or BooleanField(), into which Django will load the value after it’s retrieved from the database.

Ref: https://docs.djangoproject.com/en/4.2/ref/models/expressions/#aggregate-expressions

comment:3 by Egor R, 2 years ago

Thanks, got it. Not quite obvious, though.

Should I close the ticket?

comment:4 by Mariusz Felisiak, 2 years ago

Resolution: invalid
Status: newclosed

comment:5 by David Sanders, 2 years ago

Thanks, got it. Not quite obvious, though.
Should I close the ticket?

Looks like Mariusz beat me to it xD

If there's anything in the docs that you think could be improved we're always open to suggestions! Best place to start a conversation is on the Django Forum: https://code.djangoproject.com/wiki/DevelopersMailingList

Note: See TracTickets for help on using tickets.
Back to Top