Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#29254 closed Cleanup/optimization (invalid)

Improve paginator count performance by removing select_related from the query

Reported by: hakib Owned by: nobody
Component: Core (Other) Version: 2.0
Severity: Normal Keywords: pagination
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

The default paginator is initiated with an object_list and uses .count or len on it to get the total number of rows required to calculate the current page, next page etc...

Most of the time the object_list will be a QuerySet. When this is the case we can get an easy performace boost by removing any select_related from the query. select_related are adding an outer join so it cannot effect the total number of rows in query set - select_related is basically dead weight if you only want to count the rows.

Paginator.count is pretty simple:

    def count(self):
        """
        Returns the total number of objects, across all pages.
        """
        try:
            return self.object_list.count()
        except (AttributeError, TypeError):
            # AttributeError if object_list has no count() method.
            # TypeError if object_list.count() requires arguments
            # (i.e. is of type list).
            return len(self.object_list)

By changing to this:

            return self.object_list.select_related(None).count()

we can significantly improve the performace of the count when the original query has many select_related models (as often seen in admin pages with admin_select_related and views that show tables of data).

Only reservation I have is that Paginator is currently designed to support any kind of object that implements either *.count or len(*) - the tests also include some tests for random objects that implement count. This can easily be solved using isinstance(object_list, QuerySet), sniffing for hasatrr(select_related, object_list) or similar to how it's implemented today using AttributeError.

Change History (2)

comment:1 by hakib, 6 years ago

Resolution: invalid
Status: newclosed

comment:2 by Ramiro Morales, 6 years ago

This is similar to #23771

Note: See TracTickets for help on using tickets.
Back to Top