Race Condition in ModelChoiceIterator (affects ModelChoiceField and ModelMultipleChoiceField)
|Reported by:||esaj||Owned by:||nobody|
|Severity:||Keywords:||race condition threading ModelChoiceIterator generator ValueError|
|Has patch:||yes||Needs documentation:|
|Needs tests:||Patch needs improvement:|
I have discovered a race condition in ModelChoiceIterator: when ModelChoiceField is initialised with a queryset, this same queryset is kept across multiple requests (due to the fact that form fields are generally only initialised once as class members). As a result, there is a race condition when multiple requests are made concurrently, and more than one request tries to iterate over the same queryset. This generally appears in the form of a ValueError: generator already executing occurring.
The Select widget also seems to keep the same ModelChoiceIterator instance in its choices attribute across multiple requests. Under high load, this ModelChoiceIterator may be consumed by multiple threads simultaneously. However, as this queryset instance is the same for these multiple threads, it causes a ValueError: generator already executing error when the second or third thread attempts to consume it.
The patch is extremely simple for this: just call .all() on the queryset every time the ModelChoiceIterator is consumed. I've tested this under high load and it works perfectly.