In/Range lookups on annotations consume iterator rhs

[Oleksandr Tatarinov]

Passing an iterator to __in silently produces an empty result on Django 6.1. Regression in ​7b54ddd5e6 (refs #36025). FieldGetDbPrepValueIterableMixin.get_prep_lookup() now iterates self.rhs twice: the any() call exhausts it, the second iteration sees nothing, and the lookup compiles to an empty IN ().
On 6.0 the method made a single pass, and the __in docs say "in a given iterable". One possible fix is to materialize the rhs before the any() check.

[Yassin Bahri]

I reproduced this on current main, although the issue is narrower than the current description suggests.

Iterator right-hand sides for ordinary field lookups already work because Query.build_filter() materializes them:

Article.objects.filter(id__in=iter([article_1.id, article_2.id]))

The regression occurs when the left-hand side resolves to an annotation or alias:

queryset = Article.objects.alias(article_id=F("id"))

queryset.filter(
    article_id__in=iter([article_1.id, article_2.id]),
)
# Incorrectly returns no rows.

queryset.filter(
    article_id__range=iter([article_1.id, article_2.id]),
)
# Raises ValueError: not enough values to unpack.

In this path, the iterator reaches FieldGetDbPrepValueIterableMixin.get_prep_lookup() directly. The any() expression consumes it, leaving no values for the following iteration.

I confirmed the regression boundary:

• Both regression tests pass immediately before 7b54ddd5e6.
• Both fail at 7b54ddd5e6.
• in returns an empty result, while range raises ValueError.

The ticket is valid and actionable. I suggest narrowing the summary to:

In/Range lookups on annotations consume iterator rhs (regression in 6.1)

[Jacob Walls]

Thanks for the report! Accepting as this is in the same spirit as #21739. (Otherwise, ORM functionality that doesn't support multiple iterations has been documented with that caveat, e.g. in #36426.)

[Sarah Boyce <42296566+sarahboyce@…>]

In 0751d5f:

Fixed #37311 -- Prevented consumption of rhs iterators in annotation lookups.

Regression in 7b54ddd5e64c96c641b70bb5f0e958a9e2035fb2.

Thank you Oleksandr Tatarinov for the report and Clifford Gama for the review.

[Sarah Boyce <42296566+sarahboyce@…>]

In fc805c6:

[6.1.x] Fixed #37311 -- Prevented consumption of rhs iterators in annotation lookups.

Regression in 7b54ddd5e64c96c641b70bb5f0e958a9e2035fb2.

Thank you Oleksandr Tatarinov for the report and Clifford Gama for the review.

Backport of 0751d5fed159fd9ff7edba522ece13c8c9d2a0d0 from main.