#29254 closed Cleanup/optimization (invalid)
Improve paginator count performance by removing select_related from the query
Reported by: | hakib | Owned by: | nobody |
---|---|---|---|
Component: | Core (Other) | Version: | 2.0 |
Severity: | Normal | Keywords: | pagination |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | yes | UI/UX: | no |
Description
The default paginator is initiated with an object_list
and uses .count
or len
on it to get the total number of rows required to calculate the current page, next page etc...
Most of the time the object_list will be a QuerySet. When this is the case we can get an easy performace boost by removing any select_related from the query. select_related are adding an outer join so it cannot effect the total number of rows in query set - select_related is basically dead weight if you only want to count the rows.
Paginator.count
is pretty simple:
def count(self): """ Returns the total number of objects, across all pages. """ try: return self.object_list.count() except (AttributeError, TypeError): # AttributeError if object_list has no count() method. # TypeError if object_list.count() requires arguments # (i.e. is of type list). return len(self.object_list)
By changing to this:
return self.object_list.select_related(None).count()
we can significantly improve the performace of the count when the original query has many select_related models (as often seen in admin pages with admin_select_related and views that show tables of data).
Only reservation I have is that Paginator
is currently designed to support any kind of object that implements either *.count
or len(*)
- the tests also include some tests for random objects that implement count. This can easily be solved using isinstance(object_list, QuerySet)
, sniffing for hasatrr(select_related, object_list)
or similar to how it's implemented today using AttributeError
.
Change History (2)
comment:1 by , 7 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
This is similar to #23771