Code

Opened 6 years ago

Closed 6 years ago

Last modified 5 years ago

#7398 closed (fixed)

Making it easier to add custom cache backends

Reported by: lau@… Owned by: nobody
Component: Core (Cache system) Version: master
Severity: Keywords:
Cc: lau@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

I ran into an inconvenience with the Django cache. I wanted to add my own custom cache backend without having to muck about with the Django core. This proved to be a bit challenging since the backend importer uses a complete path to the module (e.g. "django.core.cache.backends.memcached").

I did a small patch which lets you define extra backend modules without touching the Django folder. Essentially it uses the BACKENDS dictionary in core/cache/init.py to keep information about the full import paths, like this:

BACKENDS = {
    # name for use in settings file --> path to the backend module
    'memcached': 'django.core.cache.backends.memcached',
    'locmem': 'django.core.cache.backends.locmem',
    'file': 'django.core.cache.backends.filebased',
    'db': 'django.core.cache.backends.db',
    'dummy': 'django.core.cache.backends.dummy',
}

if hasattr(settings, "CACHE_BACKEND_MODULES"):
    BACKENDS.update(settings.CACHE_BACKEND_MODULES)

So if I wanted to add a new backend in my project, I'd do something like:

settings.py:

CACHE_BACKEND_MODULES = {
    'penandpaper': 'backends.cache.penandpaper'
}

and place the backend in my_app/backends/cache/.

There has been a bit of talk about it lately on the developer mailing list:

http://groups.google.com/group/django-developers/browse_thread/thread/e0fbaff909c44336

Attachments (3)

ease-custom-cache-backends.diff (1.4 KB) - added by lau@… 6 years ago.
Ease custom cache backends
ease-custom-cache-backends-with-documentation.diff (2.2 KB) - added by lau@… 6 years ago.
Added documentation in docs/settings.txt
importable-backends.diff (4.2 KB) - added by theillustratedlife 6 years ago.
Allows CACHE_BACKEND to be a path - doc formatting corrected

Download all attachments as: .zip

Change History (19)

Changed 6 years ago by lau@…

Ease custom cache backends

comment:1 Changed 6 years ago by jacob

  • Needs documentation set
  • Needs tests unset
  • Patch needs improvement unset

Changed 6 years ago by lau@…

Added documentation in docs/settings.txt

comment:2 Changed 6 years ago by mattmcc

  • Needs documentation unset

comment:3 Changed 6 years ago by lau@…

Let me know if there's anything else in the way of this ticket.

comment:4 Changed 6 years ago by lau@…

Any news on getting this in?

comment:5 Changed 6 years ago by theillustratedlife

  • Triage Stage changed from Unreviewed to Ready for checkin

comment:6 Changed 6 years ago by mtredinnick

  • Patch needs improvement set
  • Triage Stage changed from Ready for checkin to Accepted

I think a whole extra setting and a dictionary like this is too much for this feature. Let's handle it like database backends: if the cache name (the bit before the colon, ":") is not one of the standard names, it's treated as an import path. Everything after the colon is passed through as an argument, just like now.

So extra caches can be added anywhere on the Python path and we don't need an extra setting. Being able to redefine the existing names to mean something else isn't needed here. If you want to have "some other memcached", just import it under a different name using the extension mechanism in the above paragraph. The point is that every time we add an extra setting, it's one more bit of documentation, one more thing for people to wonder if they need, etc. Let's go for simplicity first.

comment:7 follow-up: Changed 6 years ago by theillustratedlife

So, you're proposing something like this in settings.py:

CACHE_BACKEND = "backends.cache.penandpaper://"

with something like this in django.core.conf:

backend_import = scheme
if hasattr(BACKENDS, scheme):
    backend_import = BACKENDS[scheme]

    cache_class = getattr(__import__(backend_import, {}, {}, ['']), 'CacheClass')

Correct?

comment:8 in reply to: ↑ 7 Changed 6 years ago by theillustratedlife

*django.core.cache

Sorry for the typo (and the additional post).

comment:9 follow-up: Changed 6 years ago by mtredinnick

Yes, that's the idea.

Changed 6 years ago by theillustratedlife

Allows CACHE_BACKEND to be a path - doc formatting corrected

comment:10 in reply to: ↑ 9 Changed 6 years ago by theillustratedlife

  • Patch needs improvement unset

I put together a patch to implement what we discussed earlier. Please take a peek and let me know what you think. =)

comment:11 Changed 6 years ago by Lau Bech Lauritzen <lau@…>

Okay then, sounds good. Looking forward to seeing it in svn.

comment:12 Changed 6 years ago by mir

  • milestone set to post-1.0

comment:13 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8075]) Fixed #7398 -- Allow for custom cache-backends to be used.

Based on a patch from Lau Bech Lauritzen and Brenton Simpson.

comment:14 Changed 6 years ago by theillustratedlife

Thanks!

comment:15 Changed 6 years ago by Lau Bech Lauritzen <lau@…>

And thanks from here as well!

comment:16 Changed 5 years ago by anonymous

  • milestone post-1.0 deleted

Milestone post-1.0 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.