Part of DjangoSpecifications

Threading improvements in Django

According to tickets #5632, #6950 and discussions ​http://groups.google.com/group/django-users/browse_frm/thread/a7d42475b66530bd, ​http://groups.google.com/group/django-developers/browse_thread/thread/fbcfa88c997d1bb3, at least the following components of Django are not entirely thread-safe:

• django.template.loader
• django.db.models
• django.contrib.sessions

This specification outlines the changes that need to be implemented to solve these issues.

A related task is to identify other components not mentioned here that have threading issues.

Proposal

Instead of ad-hoc solutions like the one proposed in #5632, there should be a locking module that can be reused.

Five types of locking primitives could be implemented:

• Django-wide global locks
• module-level locks
• class-level locks
• instance-level locks
• function-level locks

I don't see any use for a Django-wide lock. Module-level lock is needed for sessions. Class-level locks seem appropriate for models. Template loader can use a function-level lock.

FIXME: module and class level locks missing. The implementation is a modified copy of ​http://www.phyast.pitt.edu/~micheles/python/documentation.html and should perhaps reside in django.utils.locking:

from django.utils._threading_local import RLock

def getattr_(obj, name, default_thunk):
    "Similar to .setdefault in dictionaries."
    try:
        return getattr(obj, name)
    except AttributeError:
        default = default_thunk()
        setattr(obj, name, default)
        return default

def locked(func, *args, **kw):
   lock = getattr_(func, "__lock", RLock)
   lock.acquire()
   try:
       result = func(*args, **kw)
   finally:
       lock.release()
   return result

Database model locking

All functions that have .alters_data = True should be wrapped with a locked block.

Notes

Once older python version support will be dropped in distant future, locking should be implemented with with: ​http://docs.python.org/lib/with-locks.html