Opened 59 minutes ago
#37355 new Bug
Filtering on a window expression drops duplicated values() columns from the result
| Reported by: | Dave Gaeddert | Owned by: | |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | 5.2 |
| Severity: | Normal | Keywords: | window expressions |
| Cc: | Triage Stage: | Unreviewed | |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Filtering against a window function wraps the query (SQLCompiler.get_qualify_sql()). If the same column is selected through two lookup paths, the outer query only selects it once, so rows come back one column short:
class Classification(models.Model):
pass
class Employee(models.Model):
name = models.CharField(max_length=40)
department = models.CharField(max_length=40)
salary = models.PositiveIntegerField()
classification = models.ForeignKey(Classification, models.CASCADE, null=True)
Employee.objects.annotate(
rank=Window(Rank(), partition_by="department", order_by="-salary")
).filter(rank=1).values_list("classification_id", "classification__id", "name")
# [(1, 'Adams'), (1, 'Wilkinson'), ...]
# expected [(1, 1, 'Adams'), (1, 1, 'Wilkinson'), ...]
Asked for three fields, got 2-tuples, no exception. Adding .order_by("department") turns it into ORDER BY position 3 is not in select list on PostgreSQL.
Reproduces on 5.2, 6.0, and main, on PostgreSQL 16 and SQLite.
Cause: get_qualify_sql() collects the aliases to keep in the outer query in a dict keyed by expression ({expr: alias for expr, _, alias in self.get_select(with_col_aliases=True)[0]}), so two selections of an equal expression collapse into one entry. This is the duplicate-expression half of #36288, which fixed the duplicate-field name case (values_list("x", "x")) from the same regressing commit.
Found while verifying the fix for #37222. Jacob Walls agreed this is a bug on the PR thread: https://github.com/django/django/pull/21659#discussion_r3899304732
Fix with a regression test on my fork: https://github.com/davegaeddert/django/pull/5 — happy to open it against main if accepted.
(AI assistance: Claude Code was used to find and reproduce this and draft the fix; I verified the reproducer and test results.)