Code


Version 8 (modified by anonymous, 6 years ago) (diff)

--

Part of DjangoSpecifications

Threading improvements in Django

According to tickets #5632, #6950 and discussions http://groups.google.com/group/django-users/browse_frm/thread/a7d42475b66530bd, http://groups.google.com/group/django-developers/browse_thread/thread/fbcfa88c997d1bb3, at least the following components of Django are not entirely thread-safe:

  • django.template.loader
  • django.db.models
  • django.contrib.sessions

This specification outlines the changes that need to be implemented to solve these issues.

A related task is to identify other components not mentioned here that have threading issues.

See #1442 and http://groups.google.com/group/django-developers/browse_thread/thread/905f79e350525c95 for threading discussions.

Proposal

Instead of ad-hoc solutions like the one proposed in #5632, there should be a locking module that can be reused.

Five types of locking primitives could be implemented:

  • Django-wide global locks
  • module-level locks
  • class-level locks
  • instance-level locks
  • function-level locks

I don't see any use for a Django-wide lock. Module-level lock is needed for sessions. Class-level locks seem appropriate for models. Template loader can use a function-level lock.

FIXME: module and class level locks missing. The implementation is a modified copy of http://www.phyast.pitt.edu/~micheles/python/documentation.html and should perhaps reside in django.utils.locking:

from django.utils._threading_local import RLock

def getattr_(obj, name, default_thunk):
    "Similar to .setdefault in dictionaries."
    try:
        return getattr(obj, name)
    except AttributeError:
        default = default_thunk()
        setattr(obj, name, default)
        return default

def locked(func, *args, **kw):
   lock = getattr_(func, "__lock", RLock)
   lock.acquire()
   try:
       result = func(*args, **kw)
   finally:
       lock.release()
   return result

Database model locking

All functions that have .alters_data = True should be wrapped with a locked block.

Notes

Once older python version support will be dropped in distant future, locking should be implemented with with: http://docs.python.org/lib/with-locks.html

Although this sounds like a horrible hack, perhaps a portable wrapper around mmap.mmap can be used for emulating shared memory.