Allowed testing MemcachedCache and PyLibMCCache during the same test run

[Ed Morley]

Currently jenkins only tests against MemcachedCache (ie python-memcached).

In order to make changes like those in #20892 easier to test, it would be useful if PyLibMCCache were tested on jenkins too.

This will likely involve:

• Adding a PyLibMCCache config to the jenkins configs (not sure whether these are in a public repo somewhere?)
• Refactoring tests/cache/tests.py, such that:
  • MemcachedCacheTests is split into a base class (that isn't a subclass of TestCase) plus MemcachedCacheTests and PyLibMCCacheTests
  • the vast majority of tests are kept in the base class, with just the binding-specific tests in the subclasses
  • the subclasses have appropriate @skipUnless() and @override_settings()

For example:

MemcachedCache_params = {}
PyLibMCCache_params = {}
for _cache_params in settings.CACHES.values():
    backend = _cache_params['BACKEND']
    if backend == 'django.core.cache.backends.memcached.MemcachedCache':
        MemcachedCache = _cache_params
    elif backend == 'django.core.cache.backends.memcached.PyLibMCCache':
        PyLibMCCache_params = _cache_params

# ...

class BaseMemcachedTests(BaseCacheTests):

    def test_foo(self):
        # ...

@unittest.skipUnless(MemcachedCache_params, "MemcachedCache backend not configured")
@override_settings(CACHES=caches_setting_for_tests(
    base=MemcachedCache_params,
    exclude=memcached_excluded_caches,
))
class MemcachedCacheTests(BaseMemcachedTests, TestCase):

    def test_python_memcached__foo(self):
        # ...

@unittest.skipUnless(PyLibMCCache_params, "PyLibMCCache backend not configured")
@override_settings(CACHES=caches_setting_for_tests(
    base=PyLibMCCache_params,
    exclude=memcached_excluded_caches,
))
class PyLibMCCacheTests(BaseMemcachedTests, TestCase):

    def test_pylibmc_foo(self):
        # ...

However, there are both class level uses of @override_settings() and also per-test uses, and the per-test uses are for tests that will be in the base class BaseMemcachedTests.

What's the preferred way to inherit settings from the class-level @override_settings() usage of the subclasses (eg PyLibMCCacheTests), and to then modify them further on the tests run on the base class?

[Tim Graham]

I don't have much guidance about the settings overrides. Whatever works.

I'll take care of updating the Jenkins configuration. I'd guess it'll end up looking something like this:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
    },
    'memcached': {...},
    'pylibmc': {...}
}

Ideally, we might have some feature flags on the classes so as not to hardcode class paths like "django.core.cache.backends.memcached.MemcachedCache". That may make the test suite reusable for third-party subclasses.

[Ed Morley]

I don't have much guidance about the settings overrides. Whatever works.

Ah I've just seen that override_settings can be used as a context manager too and not just as a decorator. I didn't realise that from reading ​https://docs.djangoproject.com/en/1.10/topics/testing/tools/#django.test.override_settings (I didn't scroll up to see the similar entry for modify_settings().

That's fine then, for the per-test overrides we can use it to retrieve the backend specific settings from the base class at runtime (which wouldn't have worked with the decorator).

[Ed Morley]

Should I also add pylibmc to the py2.txt/py3.txt test requirements files? (I'm not sure whether Jenkins uses those?)

Installation of pylibmc requires libmemcached, so we'll need that on the Jenkins nodes too.

[Tim Graham]

Yes, I installed libmemcached-dev on Jenkins.

[Tim Graham <timograham@…>]

In 5d978c46:

Refs #27132 -- Added pylibmc to test requirements.

[Tim Graham <timograham@…>]

In 047c1d4:

Fixed #27132 -- Allowed testing MemcachedCache and PyLibMCCache during the same test run.

[Ed Morley <emorley@…>]

In 547c7e6:

[1.10.x] Refs #27132 -- Added pylibmc to test requirements.

Backport of 5d978c46216df53884fbca590a9abe660a739774 from master

[Ed Morley <emorley@…>]

In 306545d8:

[1.10.x] Fixed #27132 -- Allowed testing MemcachedCache and PyLibMCCache during the same test run.

Backport of 047c1d48a613cc2a16f078a9094cc799f06e6b0c from master