Opened 17 months ago

Last modified 17 months ago

#34681 closed Cleanup/optimization

Optimize memcache_key_warnings() — at Initial Version

Reported by: Adam Johnson Owned by: nobody
Component: Core (Cache system) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The cache functino memcache_key_warnings() iterates the key character-by-character, a slow operation in Python because it has to create and destroy many individual str objects. Instead, we can search the string with a regular expression for a 6x speedup.

IPython session comparing old and new approaches:

In [1]: import re
   ...:
   ...: MEMCACHE_MAX_KEY_LENGTH = 250
   ...:
   ...:
   ...: def old(key):
   ...:     if len(key) > MEMCACHE_MAX_KEY_LENGTH:
   ...:         yield (
   ...:             "Cache key will cause errors if used with memcached: %r "
   ...:             "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
   ...:         )
   ...:     for char in key:
   ...:         if ord(char) < 33 or ord(char) == 127:
   ...:             yield (
   ...:                 "Cache key contains characters that will cause errors if "
   ...:                 "used with memcached: %r" % key
   ...:             )
   ...:             break
   ...:
   ...: memcached_error_chars_re = re.compile(r"[\x00-\x32\x127]")
   ...:
   ...:
   ...: def new(key):
   ...:     if len(key) > MEMCACHE_MAX_KEY_LENGTH:
   ...:         yield (
   ...:             "Cache key will cause errors if used with memcached: %r "
   ...:             "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
   ...:         )
   ...:     if memcached_error_chars_re.match(key):
   ...:         yield (
   ...:             "Cache key contains characters that will cause errors if "
   ...:             "used with memcached: %r" % key
   ...:         )
   ...:

In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391'))
1.35 µs ± 1.09 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391'))
212 ns ± 1.42 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [4]: %timeit list(old('homepage\n'))
545 ns ± 1.17 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [5]: %timeit list(new('homepage\n'))
209 ns ± 1.03 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

Change History (0)

Note: See TracTickets for help on using tickets.
Back to Top