Opened 3 years ago
Closed 3 years ago
#33252 closed Bug (fixed)
CacheMiddleware and FetchFromCacheMiddleware are not thread safe.
Reported by: | Iuri de Silvio | Owned by: | Iuri de Silvio |
---|---|---|---|
Component: | Core (Cache system) | Version: | 3.2 |
Severity: | Normal | Keywords: | |
Cc: | Nick Pope | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
CacheMiddleware persist self.cache = caches[cache_alias]
on startup and it is not thread safe. https://github.com/django/django/blob/main/django/middleware/cache.py#L186
I found that after some production errors with pylibmc and uwsgi threaded. Created a small project to reproduce it. Nothing fancy, just pylibmc cache and a @cache_page
cached view. It fails even with development server, with concurrent requests.
Traceback (most recent call last): File "versions/pylibmcbug/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "versions/pylibmcbug/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "versions/pylibmcbug/lib/python3.9/site-packages/django/utils/decorators.py", line 122, in _wrapped_view result = middleware.process_request(request) File "versions/pylibmcbug/lib/python3.9/site-packages/django/middleware/cache.py", line 145, in process_request cache_key = get_cache_key(request, self.key_prefix, 'GET', cache=self.cache) File "versions/pylibmcbug/lib/python3.9/site-packages/django/utils/cache.py", line 362, in get_cache_key headerlist = cache.get(cache_key) File "versions/pylibmcbug/lib/python3.9/site-packages/django/core/cache/backends/memcached.py", line 77, in get return self._cache.get(key, default) pylibmc.ConnectionError: error 3 from memcached_get(:1:views.decorators.cache.cache_): (0x7f290400bd60) FAILURE, poll() returned a value that was not dealt with, host: localhost:11211 -> libmemcached/io.cc:254
Looking for git history, it is this way since 2010. https://github.com/django/django/commit/673e6fc7fb243ed44841b9969d26a161c25733b3
Change History (8)
comment:1 by , 3 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:3 by , 3 years ago
Has patch: | set |
---|
comment:4 by , 3 years ago
Component: | Uncategorized → Core (Cache system) |
---|
comment:5 by , 3 years ago
Cc: | added |
---|---|
Summary: | CacheMiddleware is not thread safe → CacheMiddleware and FetchFromCacheMiddleware are not thread safe. |
Triage Stage: | Unreviewed → Accepted |
Type: | Uncategorized → Bug |
Thanks for the report!
comment:6 by , 3 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
PR https://github.com/django/django/pull/15044