Opened 81 minutes ago
Last modified 45 minutes ago
#37473 new Bug
count() and aggregate() build an empty SELECT when values() selects only aggregate annotations
| Reported by: | cheek-walnut | Owned by: | |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | 5.2 |
| Severity: | Low | Keywords: | |
| Cc: | Triage Stage: | Unreviewed | |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description (last modified by )
When values() selects only an aggregate annotation, a following count(), or aggregate() over another field, generates a subquery with an empty SELECT list.
class Author(models.Model): name = models.CharField(max_length=20) age = models.IntegerField() class Book(models.Model): title = models.CharField(max_length=20) pages = models.IntegerField() author = models.ForeignKey(Author, models.CASCADE, related_name="books")
With two authors (ages 40 and 30) and one book each:
>>> Author.objects.annotate(c=Count("books")).values("c").aggregate(m=Max("age"))
Traceback (most recent call last):
...
File "django/db/backends/sqlite3/base.py", line 365, in execute
return super().execute(query, params)
django.db.utils.OperationalError: near "FROM": syntax error
Generated SQL (SQLite):
SELECT MAX("__col1") FROM (SELECT FROM "app_author" LEFT OUTER JOIN "app_book" ON ("app_author"."id" = "app_book"."author_id") GROUP BY "app_author"."id", "app_author"."name", "app_author"."age") subquery
Expected {'m': 40}, as on Django 5.1, where the inner query selects "app_author"."age" AS "__col1". values_list("c") fails the same way; values("name", "c") and aggregate(m=Max("c")) work.
Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a (Refs #28900): the query works on its parent and fails on it. Reproduced on 5.2.19, 6.0.10, 6.1.3 and main with SQLite; 5.1.15 is not affected. On PostgreSQL (checked on 16), which accepts an empty SELECT list, count() returns the right number but aggregate(m=Max("age")) fails with column "__col1" does not exist.
The cause appears to be if self.selected: in Query.set_annotation_mask() (django/db/models/sql/query.py:2568 at 5bd95c57a2) and Query.add_annotation() (:1248), which treats an empty selected dict like None: after get_aggregation() masks the unused c annotation, selected is {} and the __col1 column added for the aggregate is never added to it. count() also goes through the fallback that selects the primary key when nothing else is selected (query.py:595-605), which only sets inner_query.select and is ignored once selected is {}, so changing the two checks to is not None fixes aggregate() but not count().
AI disclosure: found during a code audit with Claude Code 2.1.317 (Claude Opus 5.5), which also reduced the test case and ran it on the versions above and on 65ad4ade74 and its parent. The PostgreSQL behavior and the count() note came from an independent review with ChatGPT Pro and were re-run with Claude Code. I reviewed this report and re-ran the reproduction on main and stable/5.1.x.
Change History (3)
comment:1 by , 64 minutes ago
| Summary: | QuerySet.aggregate() crashes when values() selects only an aggregate annotation → QuerySet.count() and aggregate() crash when values() selects only aggregate annotations |
|---|
comment:2 by , 59 minutes ago
| Description: | modified (diff) |
|---|
comment:3 by , 45 minutes ago
| Description: | modified (diff) |
|---|---|
| Summary: | QuerySet.count() and aggregate() crash when values() selects only aggregate annotations → count() and aggregate() build an empty SELECT when values() selects only aggregate annotations |
On SQLite, the same crash is easier to hit through
count(), e.g. withPaginator:The generated SQL is again
SELECT COUNT(*) FROM (SELECT FROM "app_author" ... GROUP BY ...) subquery; on 5.1.15 the inner query selects"app_author"."id" AS "col1".aggregate()also fails with a filter on the annotation (.filter(n__gt=0).values("n").aggregate(m=Max("age"))) and withaggregate(avg=Avg("age")).All of these work on 5.1.15 and on the parent of 65ad4ade74, and fail on 65ad4ade74, 5.2.19, 6.0.10, 6.1.3 and main.
AI disclosure: these variants were also found and run with Claude Code 2.1.317 (Claude Opus 5.5); I re-ran them on main and stable/5.1.x.