Opened 17 months ago
Last modified 17 months ago
#34681 closed Cleanup/optimization
Optimize memcache_key_warnings() — at Initial Version
Reported by: | Adam Johnson | Owned by: | nobody |
---|---|---|---|
Component: | Core (Cache system) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
The cache functino memcache_key_warnings() iterates the key character-by-character, a slow operation in Python because it has to create and destroy many individual str
objects. Instead, we can search the string with a regular expression for a 6x speedup.
IPython session comparing old and new approaches:
In [1]: import re ...: ...: MEMCACHE_MAX_KEY_LENGTH = 250 ...: ...: ...: def old(key): ...: if len(key) > MEMCACHE_MAX_KEY_LENGTH: ...: yield ( ...: "Cache key will cause errors if used with memcached: %r " ...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH) ...: ) ...: for char in key: ...: if ord(char) < 33 or ord(char) == 127: ...: yield ( ...: "Cache key contains characters that will cause errors if " ...: "used with memcached: %r" % key ...: ) ...: break ...: ...: memcached_error_chars_re = re.compile(r"[\x00-\x32\x127]") ...: ...: ...: def new(key): ...: if len(key) > MEMCACHE_MAX_KEY_LENGTH: ...: yield ( ...: "Cache key will cause errors if used with memcached: %r " ...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH) ...: ) ...: if memcached_error_chars_re.match(key): ...: yield ( ...: "Cache key contains characters that will cause errors if " ...: "used with memcached: %r" % key ...: ) ...: In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391')) 1.35 µs ± 1.09 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391')) 212 ns ± 1.42 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [4]: %timeit list(old('homepage\n')) 545 ns ± 1.17 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [5]: %timeit list(new('homepage\n')) 209 ns ± 1.03 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
Note:
See TracTickets
for help on using tickets.