#37355 new Bug

Filtering on a window expression drops duplicated values() columns from the result

Reported by: Dave Gaeddert Owned by:
Component: Database layer (models, ORM) Version: 5.2
Severity: Normal Keywords: window expressions
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Filtering against a window function wraps the query (SQLCompiler.get_qualify_sql()). If the same column is selected through two lookup paths, the outer query only selects it once, so rows come back one column short:

class Classification(models.Model):
    pass

class Employee(models.Model):
    name = models.CharField(max_length=40)
    department = models.CharField(max_length=40)
    salary = models.PositiveIntegerField()
    classification = models.ForeignKey(Classification, models.CASCADE, null=True)

Employee.objects.annotate(
    rank=Window(Rank(), partition_by="department", order_by="-salary")
).filter(rank=1).values_list("classification_id", "classification__id", "name")
# [(1, 'Adams'), (1, 'Wilkinson'), ...]
# expected [(1, 1, 'Adams'), (1, 1, 'Wilkinson'), ...]

Asked for three fields, got 2-tuples, no exception. Adding .order_by("department") turns it into ORDER BY position 3 is not in select list on PostgreSQL.

Reproduces on 5.2, 6.0, and main, on PostgreSQL 16 and SQLite.

Cause: get_qualify_sql() collects the aliases to keep in the outer query in a dict keyed by expression ({expr: alias for expr, _, alias in self.get_select(with_col_aliases=True)[0]}), so two selections of an equal expression collapse into one entry. This is the duplicate-expression half of #36288, which fixed the duplicate-field name case (values_list("x", "x")) from the same regressing commit.

Found while verifying the fix for #37222. Jacob Walls agreed this is a bug on the PR thread: https://github.com/django/django/pull/21659#discussion_r3899304732

Fix with a regression test on my fork: https://github.com/davegaeddert/django/pull/5 — happy to open it against main if accepted.

(AI assistance: Claude Code was used to find and reproduce this and draft the fix; I verified the reproducer and test results.)

Change History (0)

Note: See TracTickets for help on using tickets.
Back to Top