Django

Code

Ticket #7398 (closed: fixed)

Opened 6 months ago

Last modified 4 months ago

Making it easier to add custom cache backends

Reported by: lau@iola.dk Assigned to: nobody
Milestone: post-1.0 Component: Cache system
Version: SVN Keywords:
Cc: lau@iola.dk Triage Stage: Accepted
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

I ran into an inconvenience with the Django cache. I wanted to add my own custom cache backend without having to muck about with the Django core. This proved to be a bit challenging since the backend importer uses a complete path to the module (e.g. "django.core.cache.backends.memcached").

I did a small patch which lets you define extra backend modules without touching the Django folder. Essentially it uses the BACKENDS dictionary in core/cache/init.py to keep information about the full import paths, like this: 

BACKENDS = {
    # name for use in settings file --> path to the backend module
    'memcached': 'django.core.cache.backends.memcached',
    'locmem': 'django.core.cache.backends.locmem',
    'file': 'django.core.cache.backends.filebased',
    'db': 'django.core.cache.backends.db',
    'dummy': 'django.core.cache.backends.dummy',
}

if hasattr(settings, "CACHE_BACKEND_MODULES"):
    BACKENDS.update(settings.CACHE_BACKEND_MODULES)

So if I wanted to add a new backend in my project, I'd do something like:

settings.py: 

CACHE_BACKEND_MODULES = {
    'penandpaper': 'backends.cache.penandpaper'
}

and place the backend in my_app/backends/cache/.

There has been a bit of talk about it lately on the developer mailing list:

http://groups.google.com/group/django-developers/browse_thread/thread/e0fbaff909c44336

Attachments

ease-custom-cache-backends.diff (1.4 kB) - added by lau@iola.dk on 06/09/08 09:57:58.
Ease custom cache backends
ease-custom-cache-backends-with-documentation.diff (2.2 kB) - added by lau@iola.dk on 06/09/08 15:18:44.
Added documentation in docs/settings.txt
importable-backends.diff (4.2 kB) - added by theillustratedlife on 07/12/08 20:40:26.
Allows CACHE_BACKEND to be a path - doc formatting corrected

Change History

06/09/08 09:57:58 changed by lau@iola.dk

  • attachment ease-custom-cache-backends.diff added.

Ease custom cache backends

06/09/08 11:17:28 changed by jacob

  • needs_better_patch changed.
  • needs_tests changed.
  • needs_docs set to 1.

06/09/08 15:18:44 changed by lau@iola.dk

  • attachment ease-custom-cache-backends-with-documentation.diff added.

Added documentation in docs/settings.txt

06/09/08 15:40:13 changed by mattmcc

  • needs_docs deleted.

06/10/08 10:48:31 changed by lau@iola.dk

Let me know if there's anything else in the way of this ticket.

06/19/08 04:36:26 changed by lau@iola.dk

Any news on getting this in?

07/10/08 16:49:57 changed by theillustratedlife

  • stage changed from Unreviewed to Ready for checkin.

07/12/08 08:23:00 changed by mtredinnick

  • needs_better_patch set to 1.
  • stage changed from Ready for checkin to Accepted.

I think a whole extra setting and a dictionary like this is too much for this feature. Let's handle it like database backends: if the cache name (the bit before the colon, ":") is not one of the standard names, it's treated as an import path. Everything after the colon is passed through as an argument, just like now.

So extra caches can be added anywhere on the Python path and we don't need an extra setting. Being able to redefine the existing names to mean something else isn't needed here. If you want to have "some other memcached", just import it under a different name using the extension mechanism in the above paragraph. The point is that every time we add an extra setting, it's one more bit of documentation, one more thing for people to wonder if they need, etc. Let's go for simplicity first.

(follow-up: ↓ 8 ) 07/12/08 18:07:09 changed by theillustratedlife

So, you're proposing something like this in settings.py: 

CACHE_BACKEND = "backends.cache.penandpaper://"

with something like this in django.core.conf: 

backend_import = scheme
if hasattr(BACKENDS, scheme):
    backend_import = BACKENDS[scheme]

    cache_class = getattr(__import__(backend_import, {}, {}, ['']), 'CacheClass')

Correct?

(in reply to: ↑ 7 ) 07/12/08 18:08:54 changed by theillustratedlife

*django.core.cache

Sorry for the typo (and the additional post).

(follow-up: ↓ 10 ) 07/12/08 18:23:14 changed by mtredinnick

Yes, that's the idea.

07/12/08 20:40:26 changed by theillustratedlife

  • attachment importable-backends.diff added.

Allows CACHE_BACKEND to be a path - doc formatting corrected

(in reply to: ↑ 9 ) 07/12/08 22:54:54 changed by theillustratedlife

  • needs_better_patch deleted.

I put together a patch to implement what we discussed earlier. Please take a peek and let me know what you think. =)

07/14/08 17:08:43 changed by Lau Bech Lauritzen <lau@iola.dk>

Okay then, sounds good. Looking forward to seeing it in svn.

07/22/08 12:14:12 changed by mir

  • milestone set to post-1.0.

07/25/08 13:51:32 changed by mtredinnick

  • status changed from new to closed.
  • resolution set to fixed.

(In [8075]) Fixed #7398 -- Allow for custom cache-backends to be used.

Based on a patch from Lau Bech Lauritzen and Brenton Simpson.

07/26/08 13:28:44 changed by theillustratedlife

Thanks!

07/30/08 06:58:45 changed by Lau Bech Lauritzen <lau@iola.dk>

And thanks from here as well!

Add/Change #7398 (Making it easier to add custom cache backends)




Change Properties
Action