Combining Q objects with | is inefficient; lots of deep cloning
|Reported by:||Jeff Balogh||Owned by:||nobody|
|Component:||Database layer (models, ORM)||Version:||1.2-alpha|
|Cc:||Wil Clouser||Triage Stage:||Accepted|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||yes|
I have a view that was pegging the CPU for way longer than it should, so I went to do some profiling. It turns out that the view was spending 1.4 seconds in various parts of deepcopy, and I tracked that down to ORing a bunch of Q objects. Whenever two Q objects are combined with |, the lhs is cloned so that a new, combined Q object is returned. The view was combining 50+ Q objects (lame, I know) and was bogged down in cloning.
The attached patch adds a
Q.or_ class method that takes any number of Q objects and keyword arguments and ORs them together without any cloning.
I thought about adding
Q.and_ for symmetry, but that's the normal operating of Q and filter to begin with, so I didn't want to add another way to do it.
Sad note: for whatever reason, the view in question is only 120ms slower than the optimized
Q.or_ version today. But 120ms is still a lot.
Change History (8)
comment:4 Changed 7 years ago by
|Patch needs improvement:||set|
|Triage Stage:||Unreviewed → Accepted|