ModelChoiceIterator triggers duplicate queries when choices are cast to list

[François Freitag]

When there are no prefetch_related() on ModelChoiceIterator's queryset, it attempts to use the iterator() method to avoid loading all choices in memory. (​because iterator() and prefetch_related() don't make sense together.)

That results in a duplicate query if the choices have been evaluated before, because calling iterator() will clone the QuerySet, resetting its result cache in the process.

class TestModelChoiceField(TestCase):  
    def test_queryset_result_cache_is_reused(self):
        choice = ChoiceOptionModel.objects.create(name="choice 1")
        f = ModelChoiceField(ChoiceOptionModel.objects.all())
        with self.assertNumQueries(1):
            self.assertEqual(
                # list calls both __len__ and __iter__
                list(f.choices),
                [('', '---------'), (choice.pk, str(choice))],
            )

Fails with:

FAIL: test_queryset_result_cache_is_reused (forms_tests.test_tmp.TestModelChoiceField)                                                                         
----------------------------------------------------------------------                                                                                         
Traceback (most recent call last):                                                                                                                             
  File "django/tests/forms_tests/test_tmp.py", line 19, in test_queryset_result_cache_is_reused
    [('', '---------'), (choice.pk, str(choice))],
  File "django/django/test/testcases.py", line 80, in __exit__
    '%d. %s' % (i, query['sql']) for i, query in enumerate(self.captured_queries, start=1)
AssertionError: 2 != 1 : 2 queries executed, 1 expected
Captured queries were:
1. SELECT "forms_tests_choiceoptionmodel"."id", "forms_tests_choiceoptionmodel"."name" FROM "forms_tests_choiceoptionmodel" ORDER BY "forms_tests_choiceoptionmodel"."name" ASC
2. SELECT "forms_tests_choiceoptionmodel"."id", "forms_tests_choiceoptionmodel"."name" FROM "forms_tests_choiceoptionmodel" ORDER BY "forms_tests_choiceoption$odel"."name" ASC

[François Freitag]

​PR

[Tim Graham <timograham@…>]

In a2e97abd:

Fixed #29159 -- Made ModelChoiceIterator reuse QuerySet result cache.

When len() is called (e.g. when casting to list or tuple), the
QuerySet is evaluated and the result cache populated. iterator()
shouldn't be called on the QuerySet after that, as it would reset the
result cache and trigger a second query.