CacheHandler initialize unused caches.

[Alexander Ebral]

After the commit: ​https://github.com/django/django/commit/98e05ccde440cc9b768952cc10bc8285f4924e1f
logic of the method "all" from CacheHandler class was changed.

Before:

    def all(self):
        return getattr(self._caches, 'caches', {}).values()

This method returned connections that were created in __getitem__

Now:

    def all(self):
        return [self[alias] for alias in self]

Connections return for all "CACHES" from settings.py (in case of absence - they are forcibly created in self[alias])

Which version of this method seems to be right?

In my case this unnecessary mass initialization of custom diskcache-classes leads to io-lags.

Snippet that helped me:

import django.core.cache


def cache_getitem(self, alias, exists_only=False):
    try:
        return getattr(self._connections, alias)
    except AttributeError:
        if alias not in self.settings:
            raise self.exception_class(f"The connection '{alias}' doesn't exist.")
        if exists_only:
            return
    conn = self.create_connection(alias)
    setattr(self._connections, alias, conn)
    return conn


def cache_all(self):
    connections = [self.__getitem__(alias, exists_only=True) for alias in self]
    return [conn for conn in connections if conn is not None]


django.core.cache.CacheHandler.all = cache_all
django.core.cache.CacheHandler.__getitem__ = cache_getitem

[Mariusz Felisiak]

Thanks for the report. I agree this can cause a performance regression, we shouldn't initialize unused caches unnecessarily. As far as I'm aware we can restore the previous behavior with:

diff --git a/django/core/cache/__init__.py b/django/core/cache/__init__.py
index 05ef3897d0..c008ed1125 100644
--- a/django/core/cache/__init__.py
+++ b/django/core/cache/__init__.py
@@ -43,6 +43,8 @@ class CacheHandler(BaseConnectionHandler):
             ) from e
         return backend_cls(location, params)
 
+    def all(self):
+        return [self[alias] for alias in self if hasattr(self._connections, alias)]
 
 caches = CacheHandler()
 

Regression in 98e05ccde440cc9b768952cc10bc8285f4924e1f.

[Mariusz Felisiak]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 958cdf65:

Fixed #32747 -- Prevented initialization of unused caches.

Thanks Alexander Ebral for the report.

Regression in 98e05ccde440cc9b768952cc10bc8285f4924e1f.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 41e2aa7:

[3.2.x] Fixed #32747 -- Prevented initialization of unused caches.

Thanks Alexander Ebral for the report.

Regression in 98e05ccde440cc9b768952cc10bc8285f4924e1f.

Backport of 958cdf65ae90d26236d1815bbba804729595ec7a from main