Opened 43 minutes ago

Last modified 3 minutes ago

#37311 assigned Bug

In/Range lookups on annotations consume iterator rhs (regression in 6.1)

Reported by: Oleksandr Tatarinov Owned by: Yassin Bahri
Component: Database layer (models, ORM) Version: 6.1
Severity: Release blocker Keywords: regression
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Passing an iterator to __in silently produces an empty result on Django 6.1. Regression in 7b54ddd5e6 (refs #36025). FieldGetDbPrepValueIterableMixin.get_prep_lookup() now iterates self.rhs twice: the any() call exhausts it, the second iteration sees nothing, and the lookup compiles to an empty IN ().
On 6.0 the method made a single pass, and the __in docs say "in a given iterable". One possible fix is to materialize the rhs before the any() check.

Change History (4)

comment:1 by Yassin Bahri, 27 minutes ago

Triage Stage: UnreviewedAccepted

I reproduced this on current main, although the issue is narrower than the current description suggests.

Iterator right-hand sides for ordinary field lookups already work because Query.build_filter() materializes them:

Article.objects.filter(id__in=iter([article_1.id, article_2.id]))

The regression occurs when the left-hand side resolves to an annotation or alias:

queryset = Article.objects.alias(article_id=F("id"))

queryset.filter(
    article_id__in=iter([article_1.id, article_2.id]),
)
# Incorrectly returns no rows.

queryset.filter(
    article_id__range=iter([article_1.id, article_2.id]),
)
# Raises ValueError: not enough values to unpack.

In this path, the iterator reaches FieldGetDbPrepValueIterableMixin.get_prep_lookup() directly. The any() expression consumes it, leaving no values for the following iteration.

I confirmed the regression boundary:

  • Both regression tests pass immediately before 7b54ddd5e6.
  • Both fail at 7b54ddd5e6.
  • in returns an empty result, while range raises ValueError.

The ticket is valid and actionable. I suggest narrowing the summary to:

In/Range lookups on annotations consume iterator rhs (regression in 6.1)

comment:2 by Yassin Bahri, 26 minutes ago

Owner: set to Yassin Bahri
Status: newassigned

comment:3 by Yassin Bahri, 25 minutes ago

Has patch: set

comment:4 by Oleksandr Tatarinov, 3 minutes ago

Summary: In/Range lookups silently return no results when rhs is an iterator (regression in 6.1)In/Range lookups on annotations consume iterator rhs (regression in 6.1)
Note: See TracTickets for help on using tickets.
Back to Top