Combining Q objects with | is inefficient; lots of deep cloning
|Reported by:||jbalogh||Owned by:||nobody|
|Component:||Database layer (models, ORM)||Version:||1.2-alpha|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||yes|
I have a view that was pegging the CPU for way longer than it should, so I went to do some profiling. It turns out that the view was spending 1.4 seconds in various parts of deepcopy, and I tracked that down to ORing a bunch of Q objects. Whenever two Q objects are combined with |, the lhs is cloned so that a new, combined Q object is returned. The view was combining 50+ Q objects (lame, I know) and was bogged down in cloning.
The attached patch adds a Q.or_ class method that takes any number of Q objects and keyword arguments and ORs them together without any cloning.
I thought about adding Q.and_ for symmetry, but that's the normal operating of Q and filter to begin with, so I didn't want to add another way to do it.
Sad note: for whatever reason, the view in question is only 120ms slower than the optimized Q.or_ version today. But 120ms is still a lot.
Change History (8)
Changed 6 years ago by jbalogh
comment:1 follow-up: ↓ 2 Changed 6 years ago by Alex
- Needs documentation unset
- Needs tests unset
- Patch needs improvement unset
comment:4 Changed 5 years ago by russellm
- milestone set to 1.2
- Patch needs improvement set
- Triage Stage changed from Unreviewed to Accepted