Opened 17 months ago
Last modified 17 months ago
#34681 closed Cleanup/optimization
Optimize memcache_key_warnings() — at Version 2
Reported by: | Adam Johnson | Owned by: | Adam Johnson |
---|---|---|---|
Component: | Core (Cache system) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
The cache functino memcache_key_warnings() iterates the key character-by-character, a slow operation in Python because it has to create and destroy many individual str
objects. Instead, we can search the string with a regular expression for a ~3x speedup on a reasonably sized error-free cache key.
IPython session comparing old and new approaches:
In [1]: import re ...: ...: MEMCACHE_MAX_KEY_LENGTH = 250 ...: ...: ...: def old(key): ...: if len(key) > MEMCACHE_MAX_KEY_LENGTH: ...: yield ( ...: "Cache key will cause errors if used with memcached: %r " ...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH) ...: ) ...: for char in key: ...: if ord(char) < 33 or ord(char) == 127: ...: yield ( ...: "Cache key contains characters that will cause errors if " ...: "used with memcached: %r" % key ...: ) ...: break ...: ...: memcached_error_chars_re = re.compile(r"[\x00-\x20\x7f]") ...: ...: ...: def new(key): ...: if len(key) > MEMCACHE_MAX_KEY_LENGTH: ...: yield ( ...: "Cache key will cause errors if used with memcached: %r " ...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH) ...: ) ...: if memcached_error_chars_re.search(key): ...: yield ( ...: "Cache key contains characters that will cause errors if " ...: "used with memcached: %r" % key ...: ) In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391')) 1.35 µs ± 1.05 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391')) 475 ns ± 1.37 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [4]: %timeit list(old('homepage\n')) 546 ns ± 2.88 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [5]: %timeit list(new('homepage\n')) 413 ns ± 1.35 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
Change History (2)
comment:1 by , 17 months ago
Owner: | changed from | to
---|
comment:2 by , 17 months ago
Description: | modified (diff) |
---|
Note:
See TracTickets
for help on using tickets.