MultiValueDictKeyError exception has large performance penalty
|Reported by:||margieroginski||Owned by:||nobody|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||no|
I am seeing a large potential performance penalty due to a string argument representing the entire POST dict getting created when a key is not found in the POST dict.
In datastructures.py, in class MultiValueDict, __getitem__() has the following code:
try: list_ = super(!MultiValueDict, self).__getitem__(key) except !KeyError: raise !MultiValueDictKeyError("Key %r not found in %r" % (key, self))
During form validation, this exception is raised for each field/key not found in the POST dict. If the POST dict is very large, the %r associated with self is very large and can have a very significant time penalty. In a case where I had 300 of these exceptions, the time penalty of creating this string was almost 2 seconds.
It is not that unusual to have POST keys missing that are checked in this way. Here are a few of the common cases I've seen:
- checkboxes that are unchecked
- url arguments that could be POSTED but which are omitted
- fields that are dynamic - ie, they may not be there unless some client code creates them.
Do we need to provide a string argument at all in this case, given the potential time penalty and the frequency of getting values from the POST dict?
Change History (7)
comment:1 Changed 4 years ago by adrian
- Needs documentation unset
- Needs tests unset
- Patch needs improvement unset
- Triage Stage changed from Unreviewed to Accepted
comment:4 Changed 4 years ago by thejaswi_puthraya
- Component changed from Uncategorized to HTTP handling
- Easy pickings unset
- UI/UX unset
comment:6 Changed 18 months ago by Tim Graham <timograham@…>
- Resolution set to fixed
- Status changed from new to closed