Opened 8 hours ago
Last modified 8 hours ago
#36590 new Cleanup/optimization
Redis, Memcached, and Database cache backends lack specialized many operation implementations
Reported by: | Brian Atkinson | Owned by: | |
---|---|---|---|
Component: | Core (Cache system) | Version: | 5.2 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
The Redis cache backend has a specialized implementation of get_many and set_many which perform the get/set in one command to the backend. However the similar aget_many and aset_many methods are not specialized. This results in the performance of these async batch methods being significantly slower than they could be. We've seen significant performance improvements by adding the following methods to a sub-class of the built-in RedisCache class:
async def aget_many(self, keys, version=None): return await sync_to_async(self.get_many)(keys, version=version) async def aset_many(self, data, timeout=DEFAULT_TIMEOUT, version=None): return await sync_to_async(self.set_many)(data, timeout=timeout, version=version)
These were the only ones we've overridden, but there might be a few others which could benefit from an improved implementation in a similar way. Happy to send a PR for a change similar to above for these missing specializations if desired.
Change History (1)
comment:1 by , 8 hours ago
Component: | Uncategorized → Core (Cache system) |
---|---|
Summary: | Redis cache lacks specialized aget_many and aset_many implementations → Redis, Memcached, and Database cache backends lack specialized many operation implementations |
Triage Stage: | Unreviewed → Accepted |
Type: | Uncategorized → Cleanup/optimization |
Memcached and database backends are also affected as well as
delete_many
I wonder if we could potentially have
BaseDatabaseCache.__init_subclass__
automatically add specializedaget_many
,aset_many
,adelete_many
when one is declared by a subclass.