Opened 7 years ago

Closed 6 years ago

Last modified 4 years ago

#9644 closed (fixed)

Cache miss (maybe locmem cache backend is not thread-safe?)

Reported by: pihentagy Owned by: nobody
Component: Core (Cache system) Version: 1.0
Severity: Keywords: thread safe, cache, exception
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

On my site I get errors like the following (4-5 errors per day, the key is random).
Using Django 1.0.

CACHE_BACKEND = 'locmem:///?max_entries=30000'
Traceback (most recent call last):

  File "/home/profilia/.python/lib/python2.5/site-packages/django/core/handlers/base.py", line 86, in get_response

  File "/var/www/profila.hu/www/profila-site-wsgi/auction/frontend/views.py", line 250, in toc_item_list
    bookmark=False, request=request,

  File "/home/profilia/.python/lib/python2.5/site-packages/django/views/generic/list_detail.py", line 101, in object_list

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 97, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 24, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 111, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 334, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 925, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 148, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 255, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/templatetags/cache.py", line 27, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/core/cache/backends/locmem.py", line 61, in get

KeyError: u'auctionlot-row:hu:-5002021'

Attachments (1)

locmem_key_error.diff (646 bytes) - added by mrts 7 years ago.
Avoid KeyErrors when another thread has deleted the key (btw, double try: needed for except: and finally: before Python 2.5, see PEP-0341)

Download all attachments as: .zip

Change History (10)

comment:1 Changed 7 years ago by pihentagy

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Summary changed from Cache miss (maybe cache is not thread-safe? to Cache miss (maybe cache is not thread-safe?)

comment:2 Changed 7 years ago by ubernostrum

  • Summary changed from Cache miss (maybe cache is not thread-safe?) to Cache miss (maybe locmem cache backend is not thread-safe?)

comment:3 Changed 7 years ago by mrts

Perhaps the reason lies in lines 58-59, where first a reading lock is released and subsequently a writing lock acquired when the value has expired. Between these two operations another thread may have deleted the key, i.e. the following scenario occurs:

Thread 1: foo = cache.get('foo')
Thread 1: acquire read lock
 ...
Thread 1: if expiration > time.time(): False, i.e. value is expired 
Thread 1: release read lock
Thread 2: foo = cache.get('foo')
Thread 2: acquire read lock
 ...
Thread 2: if expiration > time.time(): False, i.e. value is expired 
Thread 2: release read lock
Thread 2: acquire write lock
Thread 2: delete key from cache and return
Thread 1: acquire write lock
Thread 1: delete key from cache and return --> KeyError

Changed 7 years ago by mrts

Avoid KeyErrors when another thread has deleted the key (btw, double try: needed for except: and finally: before Python 2.5, see PEP-0341)

comment:4 Changed 7 years ago by anonymous

  • Has patch set

comment:5 Changed 7 years ago by mrts

pihentagy, this should fix it.

comment:6 Changed 7 years ago by pihentagy

  • Keywords exception added; 404 not found removed

Thanks for the fix.

(I feared to peak in to the source, but that was an easy fix)

comment:7 Changed 6 years ago by jacob

  • milestone set to 1.1
  • Triage Stage changed from Unreviewed to Accepted

comment:8 Changed 6 years ago by jacob

  • Resolution set to fixed
  • Status changed from new to closed

(In [10334]) [1.0.X] Fixed #9644: fix a thread sync issue in the locmem cache. Thanks, mrts. Backport of r10333 from trunk.

comment:9 Changed 4 years ago by jacob

  • milestone 1.1 deleted

Milestone 1.1 deleted

Note: See TracTickets for help on using tickets.
Back to Top