Cache miss (maybe locmem cache backend is not thread-safe?)

[Gergely Kontra]

On my site I get errors like the following (4-5 errors per day, the key is random).
Using Django 1.0.

CACHE_BACKEND = 'locmem:///?max_entries=30000'

Traceback (most recent call last):

  File "/home/profilia/.python/lib/python2.5/site-packages/django/core/handlers/base.py", line 86, in get_response

  File "/var/www/profila.hu/www/profila-site-wsgi/auction/frontend/views.py", line 250, in toc_item_list
    bookmark=False, request=request,

  File "/home/profilia/.python/lib/python2.5/site-packages/django/views/generic/list_detail.py", line 101, in object_list

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 97, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 24, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/loader_tags.py", line 111, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 176, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 334, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 925, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 148, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/defaulttags.py", line 255, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 768, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/template/__init__.py", line 781, in render_node

  File "/home/profilia/.python/lib/python2.5/site-packages/django/templatetags/cache.py", line 27, in render

  File "/home/profilia/.python/lib/python2.5/site-packages/django/core/cache/backends/locmem.py", line 61, in get

KeyError: u'auctionlot-row:hu:-5002021'

[mrts]

Perhaps the reason lies in lines 58-59, where first a reading lock is released and subsequently a writing lock acquired when the value has expired. Between these two operations another thread may have deleted the key, i.e. the following scenario occurs:

Thread 1: foo = cache.get('foo')
Thread 1: acquire read lock
 ...
Thread 1: if expiration > time.time(): False, i.e. value is expired 
Thread 1: release read lock
Thread 2: foo = cache.get('foo')
Thread 2: acquire read lock
 ...
Thread 2: if expiration > time.time(): False, i.e. value is expired 
Thread 2: release read lock
Thread 2: acquire write lock
Thread 2: delete key from cache and return
Thread 1: acquire write lock
Thread 1: delete key from cache and return --> KeyError

[mrts]

Avoid KeyErrors when another thread has deleted the key (btw, double try: needed for except: and finally: before Python 2.5, see PEP-0341)

[mrts]

pihentagy, this should fix it.

[Gergely Kontra]

Thanks for the fix.

(I feared to peak in to the source, but that was an easy fix)

[Jacob]

(In [10334]) [1.0.X] Fixed #9644: fix a thread sync issue in the locmem cache. Thanks, mrts. Backport of r10333 from trunk.

[Jacob]

Milestone 1.1 deleted