#31938 closed New feature (duplicate)
Add a mechanism for page cache invalidation.
Reported by: | Laurent Tramoy | Owned by: | nobody |
---|---|---|---|
Component: | Core (Cache system) | Version: | 3.1 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
I've recently started to use caching for some of my pages, through the cache_page
decorator, everything works fine.
However, as soon as any dev implements a caching mechanism, they also need to think about cache invalidation, and that's my problem: there is no easy way to invalidate a specific page in Django, since the key relies on the hash cached request's full URL
I think it would be nice for Django to provide helpers to invalidate:
- a specific url with specific query params, no matter the headers
- a specific url with any query params, for one or more specific headers (among others)
- a group of urls
and the same list with a path instead of a full URL, in case the app uses a single scheme, host, and port.
Problem is, we would need to have several distinct cache keys to be able to do all this, which implies more keys, more calls to the cache and / or bigger keys. Since some people may choose performance over cache control, a good solution might be to make _generate_cache_key
and get_cache_key
customizable.
Here is the code I wrote to implement some of these features, after looking at django.utils.cache
(I use a Redis cache with django-redis
):
import re from typing import Dict, Optional from django.conf import settings from django.core.cache import cache from django.core.handlers.wsgi import WSGIRequest from django.utils.cache import get_cache_key def invalidate_view_cache( path: str, *, vary_headers: Optional[Dict[str, str]] = None, key_prefix: Optional[str] = None, invalidate_whole_prefix: bool = False, ) -> int: """ This function first creates a fake WSGIRequest to compute the cache key. The key looks like: views.decorators.cache.cache_page.key_prefix.GET.0fcb3cd9d5b34c8fe83f615913d8509b.c4ca4238a0b923820dcc509a6f75849b.en-us.UTC The first hash corresponds to the full url (including query params), the second to the header values if invalid_whole_prefix is True, we will invalidate all the keys matching the prefix that also match the header hash and prefix. This mean we will deletes all the keys returned by views.decorators.cache.cache_page.key_prefix.GET.????????????????????????????????.c4ca4238a0b923820dcc509a6f75849b.en-us.UTC To be safe, we won't do it if key_prefix is None or '' vary_headers should be a dict of every header used for this particular view In local environment, we have two defined renderers (default of DRF), thus DRF adds 'Accept` to the Vary headers Note: If LocaleMiddleware is used, we'll need to use the same language code as the one in the cached request """ request = WSGIRequest( { "PATH_INFO": path, "REQUEST_METHOD": "GET", "HTTP_HOST": settings.HOST, "wsgi.input": None, "wsgi.url_scheme": f"http{'s'*settings.IS_DEPLOYED_ENVIRONMENT}", } ) if vary_headers: request.META.update(vary_headers) cache_key = get_cache_key(request, key_prefix=key_prefix) if cache_key is None: return 0 if invalidate_whole_prefix and key_prefix: # specific to the redis implementation keys = cache.keys(re.sub("[0-9a-f]{32}", "[0-9a-f]" * 32, cache_key, 1)) return cache.delete_many(keys) return cache.delete(cache_key)
I can try to work on a more generic implementation, if you think this idea is worth it.
Change History (2)
comment:1 by , 4 years ago
Component: | Utilities → Core (Cache system) |
---|---|
Resolution: | → duplicate |
Status: | new → closed |
Summary: | page cache invalidation → Add a mechanism for page cache invalidation. |
Triage Stage: | Unreviewed → Accepted |
comment:2 by , 4 years ago
Hi Carlton,
I'm sorry I never answered, I thought I would receive an alert by email, but I did not. I'll send my suggestions to the mailing list soon !
Hi Laurent.
Thanks for the report. I was going to Accept this as a new feature — I've spent a good amount of time rifling through cache keys to work out which one to delete by hand. I think this would be a good addition.
But it looks to me like #5815 is the same issue, so let's close as a duplicate.
Can I ask you take your idea to the DevelopersMailingList and ask for some input of your thoughts for an implementation? I would be good if you want to take this on to push it forwards! 👍