Page.__len__ could skip a query if self.paginator.count == 0

[Jacob Walls]

With code like this,

paginator = Paginator(unevaluated_queryset, 25)
page = paginator.get_page(1)  # causes count query via validate_number()
if not page:  # causes select query
    return {}

I get an additional query versus:

paginator = Paginator(unevaluated_queryset, 25)
if not paginator.count:  # causes count query
    return {}

---

In the case of no data, the second SELECT query is unnecessary if we already know the paginator is empty. Since paginator.count is cached, would it be worth optimizing out this additional query? Otherwise, you have to dig into the internals to discover that the two examples above do not perform equivalently.

django/core/paginator.py

@@ -188 +188 @@
         return "<Page %s of %s>" % (self.number, self.paginator.num_pages)
 
     def __len__(self):
+        if self.paginator.count == 0:
+            return 0
         return len(self.object_list)
 
     def __getitem__(self, index):

[Simon Charette]

If we are going to do this I think should use paginator.__dict__.get('count') instead to avoid causing the inverse problem if count is not already cached? Or maybe it's not necessary because a page cannot be created without count being cached in the first place?

I think that a similar optimization could be used for any count that is smaller than the paginator's page size.

[Jacob Walls]

Or maybe it's not necessary because a page cannot be created without count being cached in the first place?

That's more or less what I found, except that it is possible to manually instantiate a Page, just unlikely enough that it may not be worth coding around?

The recommend interface is to create the Page via Paginator.page() (or Paginator.get_page() and Paginator.__iter__() which call it), any of which in turn cache the count via validate_number():

You usually won't construct ``Page`` objects by hand -- you'll get them by
iterating :class:`Paginator`, or by using :meth:`Paginator.page`.

[Simon Charette]

In this case I think it's safe to move forward with the optimization, the closest ticket I could find is #23771.

Note that a similar optimization could have done the other way around though where when a page is requested if len(page.object_list) < paginator.per_page then count can be deduced from self.paginator.count = (self.number - 1) * self.paginator.per_page + len(self.object_list).

django/core/paginator.py

@@ -89 +89 @@
         number = self.validate_number(number)
         bottom = (number - 1) * self.per_page
         top = bottom + self.per_page
-        if top + self.orphans >= self.count:
+        if self.orphans and top + self.orphans >= self.count:
             top = self.count
         return self._get_page(self.object_list[bottom:top], number, self)
 
@@ -188 +188 @@
         return "<Page %s of %s>" % (self.number, self.paginator.num_pages)
 
     def __len__(self):
-        return len(self.object_list)
+        if (paginator_count := self.paginator.__dict__.get("count")) == 0:
+            return 0
+        object_list_len = len(self.object_list)
+        if paginator_count is None and object_list_len < self.paginator.per_page:
+            self.paginator.count = (
+                self.number - 1
+            ) * self.paginator.per_page + object_list_len
+        return object_list_len
 
     def __getitem__(self, index):
         if not isinstance(index, (int, slice)):

Unfortunately because Paginator.get_page validates against self.num_pages which triggers the calculation of self.count this is not feasible. On the bright side though, this means that get_page will systematically trigger self.count calculation which strengthen the assumption that it's safe to use directly in Page.__len__.

[Jacob Walls]

In writing a test case, I discovered that I failed to confirm the no-data case does any additional select query. In the no-data case, a slice of the queryset from [0:0] is taken, which the ORM knows not to perform a query for. So there is nothing to optimize here.

In Paginator.page(), bottom and top are both 0 if the count is 0:

        return self._get_page(self.object_list[bottom:top], number, self)