Race Condition in ModelChoiceIterator (affects ModelChoiceField and ModelMultipleChoiceField)
|Reported by:||Jason Davies||Owned by:||nobody|
|Severity:||Keywords:||race condition threading ModelChoiceIterator generator ValueError|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||no|
I have discovered a race condition in
ModelChoiceField is initialised with a queryset, this same queryset is kept across multiple requests (due to the fact that form fields are generally only initialised once as class members). As a result, there is a race condition when multiple requests are made concurrently, and more than one request tries to iterate over the same queryset. This generally appears in the form of a
ValueError: generator already executing occurring.
Select widget also seems to keep the same
ModelChoiceIterator instance in its
choices attribute across multiple requests. Under high load, this
ModelChoiceIterator may be consumed by multiple threads simultaneously. However, as this queryset instance is the same for these multiple threads, it causes a
ValueError: generator already executing error when the second or third thread attempts to consume it.
The patch is extremely simple for this: just call
.all() on the queryset every time the
ModelChoiceIterator is consumed. I've tested this under high load and it works perfectly.