Opened 17 years ago
Last modified 5 months ago
#5815 assigned New feature
Adds per-view cache refreshing (clearing)
Reported by: | k0001 | Owned by: | Ahter Sönmez |
---|---|---|---|
Component: | Core (Cache system) | Version: | dev |
Severity: | Normal | Keywords: | cache refresh clear |
Cc: | Petr Přikryl | Triage Stage: | Accepted |
Has patch: | yes | Needs documentation: | no |
Needs tests: | yes | Patch needs improvement: | yes |
Easy pickings: | no | UI/UX: | no |
Description
This patch adds the possibility to clear(refresh) cached data per view.
SmileyChris sugested it would be good to add another key to the cached keys where we could keep track of all the different cache_page_key hashes, so we that can delete those then.
It is possible to clear the cache for an specific view, to achieve that, you
should tell your view's url path (and optionally, a key_prefix) to
clear_cache_for_path(path, key_prefix=None)
.
from django.utils.cache import clear_cache_for_path # clear the all cached data for path '/blog/posts/2/' clear_cache_for_path('/blog/posts/2/')
Note that it will delete every page cached from matching this path, Vary
headers
doesn't matter.
NOTE: Thanks SmileyChris for your help and supervision with this.
Attachments (1)
Change History (18)
by , 17 years ago
Attachment: | add_cache_clear_feature.diff added |
---|
comment:1 by , 17 years ago
Triage Stage: | Unreviewed → Design decision needed |
---|
comment:2 by , 17 years ago
Needs tests: | set |
---|---|
Patch needs improvement: | set |
Triage Stage: | Design decision needed → Accepted |
I like the idea. However, I'm -1 on the implementation -- adding another key to the cache seems unnecessary.
comment:3 by , 17 years ago
I don't see how we can get around adding a key to index the view's cache keys. Since they are hashed, there's no way of retrieving them with the current cache API (you'd need some type of .get_starts_with()
method which I'm not sure would be even possible)
comment:4 by , 16 years ago
One could argue that you are doing something wrong if you need per-site or per-view cache clearing.
If you have a page that changes more than once in a month, you are better off using the cache template tag. This allows you to share cached content between views and the keys can be deleted more easily when something got updated. With per-view caching you always cache the whole page, for every page. Imagine how often you cache your header or footer that way if you have a larger site.
comment:5 by , 14 years ago
Severity: | → Normal |
---|---|
Type: | → New feature |
comment:8 by , 12 years ago
Triage Stage: | Accepted → Design decision needed |
---|
Setting to DDN since there's some disagreement between Jacob and Chris.
Explicit cache invalidation can be useful in some scenarios, it'd be nice to support this.
comment:9 by , 12 years ago
Triage Stage: | Design decision needed → Accepted |
---|
If there's really not a better implementation (perhaps there isn't) then I'm OK with this going in as-is.
comment:11 by , 2 years ago
Just to add a data points here, I spent a bunch of time today trying to figure out which key to delete to nuke a cached page and eventually gave up and just nuked a larger part of the cache in hopes of getting lucky (I did).
Having a helper util just to figure out what the cache key value is would be really nice.
Here's a summary of my failed research:
- "Expire a view-cache in Django?" (https://stackoverflow.com/q/2268417/64911) — Says it works with < django 1.7. I didn't try it, but I don't think it'd work.
- "How to invalidate cache_page in Django?" (https://stackoverflow.com/q/33749369/64911) — Accepted answer just says to cache the queryset instead of using cache_page.
- "How to manually clear/update a cached view in django" (https://stackoverflow.com/q/47040438/64911) — Has a solution, but it's quite old and doesn't work. This is when I gave up.
The reason I need this (and couldn't wait for the cache to just expire) is because I use the db cache to cache really big sitemaps that don't change very often. The cache is usually on the order of a couple of weeks.
comment:12 by , 2 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:13 by , 22 months ago
#34271 was a duplicate with a suggestion to make the cache key just depend on the request path:
from pathlib import Path from django.conf import settings from django.core.cache import caches default_cache = caches[settings.CACHE_MIDDLEWARE_ALIAS] cache_key = ".".join(Path(request.path[1:]).parts) if not (response := default_cache.get(cache_key)): response = view_func(request, *args, **kwargs) response.add_post_render_callback( lambda r: default_cache.set(cache_key, r, timeout) )
That would be easy to invalidate outside the request context.
Allowing/documenting easier make_key customisation might allow us to move this forward? 🤔
comment:14 by , 21 months ago
Cc: | added |
---|
comment:15 by , 21 months ago
I haven't had a chance to look at this since the sprint in DjangoCon Europe (nearly 6 months ago). As far as I remember, I already had "something" working.
In the next few days, let me check what was done, and share some progress here.
comment:16 by , 17 months ago
Here's a quick update on this:
I think this can be done. However, there are a few things to take into consideration.
- Django's core caching code is quite old, 18+ years. I believe big part of it should be redesigned and refactored so it's easy to work on it and extend it in non-hacky ways. While this refactor is out of scope for this PR, I think it might be beneficial. I think this might be why this ticket is quite old.
- Even when we solve this problem, due to the way cache keys are generated, this will be only supported by some cache backends (ie. Redis) which allow filtering by keys or part of keys.
- There are better design patterns that would allow more intelligent caching in the projects/apps that use Django. Especially projects that use layered architecture. However, this is not applicable to Django as a library as it cannot be opinionated about how devs would like to structure their code in their apps.
I finally now have a chance to have a look at this again in DjangoCon Europe 2023 sprints. This time we've also paired with Marco Silva and we're tackling the issues together. I'm hoping to work on this more (given we're now two) and get something out as a PR soon.
comment:17 by , 5 months ago
I would like to my use case to widen the perspective.
I am running a website with ~4k RPM and clearing cache for some view could result in timeouts due to cache hammering.
Multiple requests would bump into the cleared cache at the same time and it would result in too many DB requests which would slow down the single database significantly (the requests would take tens of seconds).
So clearing the cache even for single page/view is not an option for me.
The solution for me would be to directly inject new keys.
adds per-view clear cache support