Optimize memcache_key_warnings()

[Adam Johnson]

The cache functino memcache_key_warnings() iterates the key character-by-character, a slow operation in Python because it has to create and destroy many individual str objects. Instead, we can search the string with a regular expression for a ~3x speedup on a reasonably sized error-free cache key.

IPython session comparing old and new approaches:

In [1]: import re
   ...:
   ...: MEMCACHE_MAX_KEY_LENGTH = 250
   ...:
   ...:
   ...: def old(key):
   ...:     if len(key) > MEMCACHE_MAX_KEY_LENGTH:
   ...:         yield (
   ...:             "Cache key will cause errors if used with memcached: %r "
   ...:             "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
   ...:         )
   ...:     for char in key:
   ...:         if ord(char) < 33 or ord(char) == 127:
   ...:             yield (
   ...:                 "Cache key contains characters that will cause errors if "
   ...:                 "used with memcached: %r" % key
   ...:             )
   ...:             break
   ...:
   ...: memcached_error_chars_re = re.compile(r"[\x00-\x20\x7f]")
   ...:
   ...:
   ...: def new(key):
   ...:     if len(key) > MEMCACHE_MAX_KEY_LENGTH:
   ...:         yield (
   ...:             "Cache key will cause errors if used with memcached: %r "
   ...:             "(longer than %s)" % (key, MEMCACHE_MAX_KEY_LENGTH)
   ...:         )
   ...:     if memcached_error_chars_re.search(key):
   ...:         yield (
   ...:             "Cache key contains characters that will cause errors if "
   ...:             "used with memcached: %r" % key
   ...:         )

In [2]: %timeit list(old('acme-bookstore-user-1234567-book-78910391'))
1.35 µs ± 1.05 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [3]: %timeit list(new('acme-bookstore-user-1234567-book-78910391'))
475 ns ± 1.37 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [4]: %timeit list(old('homepage\n'))
546 ns ± 2.88 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [5]: %timeit list(new('homepage\n'))
413 ns ± 1.35 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

[Mariusz Felisiak]

Agreed. I'd only use _lazy_re_compile().

[Adam Johnson]

Yup, done in the patch.

[Mariusz Felisiak]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 1dbcf9a:

Fixed #34681 -- Optimized memcache_key_warnings().