Opened 4 hours ago

Last modified 48 minutes ago

#37473 new Bug

QuerySet.count() and aggregate() crash when values() selects only aggregate annotations — at Version 2

Reported by: cheek-walnut Owned by:
Component: Database layer (models, ORM) Version: 5.2
Severity: Low Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by cheek-walnut)

When values() selects only an aggregate annotation, a following aggregate() over another field generates a subquery with an empty SELECT list.

class Author(models.Model):
    name = models.CharField(max_length=20)
    age = models.IntegerField()

class Book(models.Model):
    title = models.CharField(max_length=20)
    pages = models.IntegerField()
    author = models.ForeignKey(Author, models.CASCADE, related_name="books")

With two authors (ages 40 and 30) and one book each:

>>> Author.objects.annotate(c=Count("books")).values("c").aggregate(m=Max("age"))
Traceback (most recent call last):
  ...
  File "django/db/backends/sqlite3/base.py", line 365, in execute
    return super().execute(query, params)
django.db.utils.OperationalError: near "FROM": syntax error

Generated SQL (SQLite):

SELECT MAX("__col1") FROM (SELECT  FROM "app_author" LEFT OUTER JOIN "app_book" ON ("app_author"."id" = "app_book"."author_id") GROUP BY "app_author"."id", "app_author"."name", "app_author"."age") subquery

Expected {'m': 40}, as on Django 5.1, where the inner query selects "app_author"."age" AS "__col1". values_list("c") fails the same way; values("name", "c") and aggregate(m=Max("c")) work.

Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a (Refs #28900): the query works on its parent and fails on it. Reproduced on 5.2.19, 6.0.10, 6.1.3 and main with SQLite; 5.1.16 is not affected.

The cause appears to be if self.selected: in Query.set_annotation_mask() (django/db/models/sql/query.py:2568) and Query.add_annotation() (:1248), which treats an empty selected dict like None: after get_aggregation() masks the unused c annotation, selected is {} and the __col1 column added for the aggregate is never added to it.

AI disclosure: found during a code audit with Claude Code 2.1.317 (Claude Opus 5.5), which also reduced the test case and ran it on the versions above and on 65ad4ade74 and its parent. I reviewed this report and re-ran the reproduction on main and 5.1.16.

Change History (2)

comment:1 by cheek-walnut, 3 hours ago

Summary: QuerySet.aggregate() crashes when values() selects only an aggregate annotation → QuerySet.count() and aggregate() crash when values() selects only aggregate annotations

On SQLite, the same crash is easier to hit through count(), e.g. with Paginator:

>>> qs = Author.objects.annotate(n=Count("books")).values_list("n", flat=True)
>>> qs.count()
django.db.utils.OperationalError: near "FROM": syntax error
>>> Paginator(qs.order_by("n"), 10).count
django.db.utils.OperationalError: near "FROM": syntax error

The generated SQL is again SELECT COUNT(*) FROM (SELECT FROM "app_author" ... GROUP BY ...) subquery; on 5.1.15 the inner query selects "app_author"."id" AS "col1". aggregate() also fails with a filter on the annotation (.filter(n__gt=0).values("n").aggregate(m=Max("age"))) and with aggregate(avg=Avg("age")).

All of these work on 5.1.15 and on the parent of 65ad4ade74, and fail on 65ad4ade74, 5.2.19, 6.0.10, 6.1.3 and main.

AI disclosure: these variants were also found and run with Claude Code 2.1.317 (Claude Opus 5.5); I re-ran them on main and stable/5.1.x.

Last edited 3 hours ago by cheek-walnut (previous) (diff)

comment:2 by cheek-walnut, 3 hours ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top