Opened 8 hours ago

Last modified 8 hours ago

#36590 new Cleanup/optimization

Redis, Memcached, and Database cache backends lack specialized many operation implementations

Reported by: Brian Atkinson Owned by:
Component: Core (Cache system) Version: 5.2
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The Redis cache backend has a specialized implementation of get_many and set_many which perform the get/set in one command to the backend. However the similar aget_many and aset_many methods are not specialized. This results in the performance of these async batch methods being significantly slower than they could be. We've seen significant performance improvements by adding the following methods to a sub-class of the built-in RedisCache class:

    async def aget_many(self, keys, version=None):
        return await sync_to_async(self.get_many)(keys, version=version)

    async def aset_many(self, data, timeout=DEFAULT_TIMEOUT, version=None):
        return await sync_to_async(self.set_many)(data, timeout=timeout, version=version)

These were the only ones we've overridden, but there might be a few others which could benefit from an improved implementation in a similar way. Happy to send a PR for a change similar to above for these missing specializations if desired.

Change History (1)

comment:1 by Simon Charette, 8 hours ago

Component: UncategorizedCore (Cache system)
Summary: Redis cache lacks specialized aget_many and aset_many implementationsRedis, Memcached, and Database cache backends lack specialized many operation implementations
Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

Memcached and database backends are also affected as well as delete_many

I wonder if we could potentially have BaseDatabaseCache.__init_subclass__ automatically add specialized aget_many, aset_many, adelete_many when one is declared by a subclass.

Note: See TracTickets for help on using tickets.
Back to Top