id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
32244	ORM inefficiency: ModelFormSet executes a single-object SELECT query per formset instance when saving/validating	Lushen Wu	nobody	"Conceptual summary of the issue:
Let's say we have a Django app with {{{Author}}} and {{{Book}}} models, and use a {{{BookFormSet}}} to add / modify / delete books that belong to a given {{{Author}}}. The problem is when the {{{BookFormSet}}} is validated, {{{ModelChoiceField.to_python()}}} ends up calling {{{self.queryset.get(id=123)}}} which results in a single-object SELECT query for **each** book in the formset. That means if I want to update 15 books, Django performs 15 separate SELECT queries, which seems incredibly inefficient. (Our actual app is an editor that can update any number of objects in a single formset, e.g. 50+). 

My failed attempts to solve this:
1. First I tried passing a queryset to the {{{BookFormSet}}}, i.e. {{{formset = BookFormSet(data=request.POST, queryset=Book.objects.filter(author=1))}}}, but the `ModelChoiceField` still does its single-object SELECT queries.
2. Then I tried to see where the {{{ModelChoiceField}}} defines its queryset, which seems to be in {{{BaseModelFormSet.add_fields()}}}. I tried initiating the {{{ModelChoiceField}}} with the same queryset that I passed to the formset, e.g. {{{Book.objects.filter(author=1)}}} instead of the original code which would be {{{Book._default_manager.get_queryset()}}}. But this doesn't help because I guess the new queryset I defined isn't actually linked to what was passed to the formset (and already evaluated). So the multiple SELECT queries still happen. (Note: I realize {{{_default_manager.get_queryset()}}} might be necessary in cases where the formset can be used to switch one Model instance to another instance which might not be in the original queryset passed to the {{{BaseModelFormset}}}, but this is not our use case)
3. I noticed that {{{BaseModelFormSet._existing_object()}}} actually provides a way to check whether an object exists in the queryset that was giving to the formset constructor, which means that queryset is evaluated at most once and the results stored in {{{BaseModelFormSet._object_dict}}}. I thought there might be some way to have {{{ModelChoiceField.to_python()}}} do something similar before calling {{{self.queryset.get(id=123)}}}, but I don't think {{{ModelChoiceField}}} is aware of {{{BaseModelFormSet}}}, and it would seem an anti-pattern to reach up the hierarchy like this. 

The easiest solution seems to me to pass {{{BaseModelFormSet._object_dict}}} in some way to each {{{ModelForm}}} that's created, and then allow the {{{ModelChoiceField}}} to check this {{{_object_dict}}} before making another SELECT query.

Even if this could be solved with some form of DB caching, it still seems inefficient application-layer logic."	Cleanup/optimization	new	Database layer (models, ORM)	3.1	Normal		formsets		Unreviewed	0	0	0	0	0	0