Opened 13 months ago

Closed 13 months ago

Last modified 11 months ago

#32007 closed Bug (fixed)

QuerySet.annotate() with WhereNode() crashes with AttributeError.

Reported by: Gordon Wrigley Owned by: Mariusz Felisiak
Component: Database layer (models, ORM) Version: 3.1
Severity: Release blocker Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I think I've found a regression in 3.1, I'm seeing this across databases and Python versions but only on Django 3.1.
With reference to the polls app the queryset I hit this on is essentially:

Question.objects.values("question_text").annotate(
    pub_date__count=Count("pub_date", distinct=True),
    pub_date__is_null=ExpressionWrapper(Q(pub_date=None), output_field=BooleanField())
).values("question_text", "pub_date__count", "pub_date__is_null")

This results in:

~\.venv\django_tutorial\lib\site-packages\django\db\models\query.py in annotate(self, *args, **kwargs)
   1121                     clone.query.group_by = True
   1122                 else:
-> 1123                     clone.query.set_group_by()
   1124                 break
   1125

~\.venv\django_tutorial\lib\site-packages\django\db\models\sql\query.py in set_group_by(self, allow_aliases)
   1979                     if not allow_aliases or alias in column_names:
   1980                         alias = None
-> 1981                     group_by_cols = annotation.get_group_by_cols(alias=alias)
   1982                 group_by.extend(group_by_cols)
   1983         self.group_by = tuple(group_by)

~\.venv\django_tutorial\lib\site-packages\django\db\models\expressions.py in get_group_by_cols(self, alias)
    867
    868     def get_group_by_cols(self, alias=None):
--> 869         expression = self.expression.copy()
    870         expression.output_field = self.output_field
    871         return expression.get_group_by_cols(alias=alias)

AttributeError: 'WhereNode' object has no attribute 'copy'

It's a bit of a dumb query but given my usecase https://pypi.org/project/django-data-browser/ that's a little beyond my control.

Still it's kinda surprising that it worked on 3.0 and doesn't on 3.1.

It can be simplified a bit and still get essentially the same result:

Question.objects.annotate(
    pub_date__count=Count("pub_date", distinct=True),
    pub_date__is_null=ExpressionWrapper(Q(pub_date=None), output_field=BooleanField())
)

Removing either of the annotate clauses "fixes" it.

Change History (11)

comment:1 Changed 13 months ago by Mariusz Felisiak

Component: UncategorizedDatabase layer (models, ORM)
Severity: NormalRelease blocker
Summary: AttributeError: 'WhereNode' object has no attribute 'copy'QuerySet.annotate() with WhereNode() crashes with AttributeError.
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

Thanks for the report.

Regression in 8a6df55f2dd5131282084a4edfd48f63fbf8c69a.
Reproduced at bcc2befd0e9c1885e45b46d0b0bcdc11def8b249.

comment:2 Changed 13 months ago by Simon Charette

The best way of resolving this is likely to add a WhereNode.copy method.

comment:3 Changed 13 months ago by Gordon Wrigley

Here is a more useful variant, this is a type thing I actually do on a regular basis, and I guess also means work won't be upgrading to 3.1 until I find a work around.

Question.objects.annotate(
    pub_date__is_null=ExpressionWrapper(Q(pub_date=None), output_field=BooleanField())
).values("pub_date__is_null").annotate(
    id__count=Count("id", distinct=True)
).values("pub_date__is_null", "id__count")

comment:4 Changed 13 months ago by Gordon Wrigley

Also as a total aside regarding that last snippet it'd be nice if Q objects just were boolean expressions.

comment:5 Changed 13 months ago by Mariusz Felisiak

Owner: changed from nobody to Mariusz Felisiak
Status: newassigned

comment:6 Changed 13 months ago by Mariusz Felisiak

Has patch: set

comment:7 Changed 13 months ago by GitHub <noreply@…>

Resolution: fixed
Status: assignedclosed

In eaf9764d:

Fixed #32007 -- Fixed queryset crash with Q() annotation and aggregation.

Thanks Gordon Wrigley for the report.

Regression in 8a6df55f2dd5131282084a4edfd48f63fbf8c69a.

comment:8 Changed 13 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 1afc9b3:

[3.1.x] Fixed #32007 -- Fixed queryset crash with Q() annotation and aggregation.

Thanks Gordon Wrigley for the report.

Regression in 8a6df55f2dd5131282084a4edfd48f63fbf8c69a.
Backport of eaf9764d3bb25970da89de5799d8d308715628ba from master

comment:9 Changed 13 months ago by GitHub <noreply@…>

In 3a9f192:

Refs #32007 -- Skipped test_q_expression_annotation_with_aggregation on Oracle.

comment:10 Changed 13 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 5a03e14d:

[3.1.x] Refs #32007 -- Skipped test_q_expression_annotation_with_aggregation on Oracle.

Backport of 3a9f192b131f7a9b0fe5783c684b23015fa67cc8 from master

comment:11 Changed 11 months ago by Gordon Wrigley

I think this is related to https://code.djangoproject.com/ticket/32200

Note: See TracTickets for help on using tickets.
Back to Top