Redis, Memcached, and Database cache backends lack specialized many operation implementations
The Redis cache backend has a specialized implementation of get_many and set_many which perform the get/set in one command to the backend. However the similar aget_many and aset_many methods are not specialized. This results in the performance of these async batch methods being significantly slower than they could be. We've seen significant performance improvements by adding the following methods to a sub-class of the built-in RedisCache class:
async def aget_many(self, keys, version=None):
return await sync_to_async(self.get_many)(keys, version=version)
async def aset_many(self, data, timeout=DEFAULT_TIMEOUT, version=None):
return await sync_to_async(self.set_many)(data, timeout=timeout, version=version)
These were the only ones we've overridden, but there might be a few others which could benefit from an improved implementation in a similar way. Happy to send a PR for a change similar to above for these missing specializations if desired.
Change History
(7)
Component: |
Uncategorized → Core (Cache system)
|
Summary: |
Redis cache lacks specialized aget_many and aset_many implementations → Redis, Memcached, and Database cache backends lack specialized many operation implementations
|
Triage Stage: |
Unreviewed → Accepted
|
Type: |
Uncategorized → Cleanup/optimization
|
Owner: |
set to H. White
|
Status: |
new → assigned
|
Patch needs improvement: |
set
|
Patch needs improvement: |
unset
|
Patch needs improvement: |
set
|
Patch needs improvement: |
unset
|
Memcached and database backends are also affected as well as
delete_many
I wonder if we could potentially have
BaseDatabaseCache.__init_subclass__
automatically add specializedaget_many
,aset_many
,adelete_many
when one is declared by a subclass.