Opened 2 years ago
Last modified 2 years ago
#34681 closed Cleanup/optimization
Optimize memcache_key_warnings() — at Version 2
| Reported by: | Adam Johnson | Owned by: | Adam Johnson |
|---|---|---|---|
| Component: | Core (Cache system) | Version: | dev |
| Severity: | Normal | Keywords: | |
| Cc: | Triage Stage: | Ready for checkin | |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description (last modified by )
The cache functino memcache_key_warnings() iterates the key character-by-character, a slow operation in Python because it has to create and destroy many individual str objects. Instead, we can search the string with a regular expression for a ~3x speedup on a reasonably sized error-free cache key.
IPython session comparing old and new approaches:
In [1]: import re
...:
...: MEMCACHE_MAX_KEY_LENGTH = 250
...:
...:
...: def old(key):
...: if len(key) > MEMCACHE_MAX_KEY_LENGTH:
...: yield (
...: "Cache key will cause errors if used with memcached: %r "
...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
...: )
...: for char in key:
...: if ord(char) < 33 or ord(char) == 127:
...: yield (
...: "Cache key contains characters that will cause errors if "
...: "used with memcached: %r" % key
...: )
...: break
...:
...: memcached_error_chars_re = re.compile(r"[\x00-\x20\x7f]")
...:
...:
...: def new(key):
...: if len(key) > MEMCACHE_MAX_KEY_LENGTH:
...: yield (
...: "Cache key will cause errors if used with memcached: %r "
...: "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
...: )
...: if memcached_error_chars_re.search(key):
...: yield (
...: "Cache key contains characters that will cause errors if "
...: "used with memcached: %r" % key
...: )
In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391'))
1.35 µs ± 1.05 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391'))
475 ns ± 1.37 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
In [4]: %timeit list(old('homepage\n'))
546 ns ± 2.88 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
In [5]: %timeit list(new('homepage\n'))
413 ns ± 1.35 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
Change History (2)
comment:1 by , 2 years ago
| Owner: | changed from to |
|---|
comment:2 by , 2 years ago
| Description: | modified (diff) |
|---|
Note:
See TracTickets
for help on using tickets.