Opened 9 years ago

Closed 9 years ago

Last modified 8 years ago

#1442 closed defect (fixed)

[patch] Fixing multithreading problem with various database backends

Reported by: eugene@… Owned by: adrian
Component: Database layer (models, ORM) Version:
Severity: normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

This patch supersedes tickets #463, #900, and #1237. Instead of lock-based connection pools it uses TLS (thread local storage). TLS is implemented by python 2.4 natively (threading.local), and by stub for python 2.3. The stub is taken verbatim from python 2.4.2 source distribution. I checked the license and it looks totally compatible with BSD license used by Django. Thank you Joseph Kocherhans (#1268).

The patch solves multithreading problems for MySQL (#463) and PostGreSQL (#900). I couldn't find a ticket for ADO/MSSQL and SQLite, so I didn't convert them. In general the fix is unbelievably simple:

try:
    # only exists in python 2.4+
    from threading import local
except ImportError:
    # import copy of _thread_local.py from python 2.4
    from django.utils._threading_local import local

class DatabaseWrapper(local):
...

It can be easily added for other backends, if required. (Additionally MySQL patch implements pinging connections.)

Attachments (4)

django.threading.patch (9.2 KB) - added by eugene@… 9 years ago.
patch for Django trunk
magic.threading.patch (9.1 KB) - added by eugene@… 9 years ago.
patch for magic-removal branch
django.threading.2.patch (10.6 KB) - added by eugene@… 9 years ago.
patch for trunk (includes all database backends)
magic.threading.2.patch (10.5 KB) - added by eugene@… 9 years ago.
patch for magic-removal (includes all database backends)

Download all attachments as: .zip

Change History (10)

Changed 9 years ago by eugene@…

patch for Django trunk

Changed 9 years ago by eugene@…

patch for magic-removal branch

comment:1 Changed 9 years ago by eugene@…

I tested it on Python 2.4 and Python 2.3 (trunk) with MySQL (DreamHost setup). Please try it with PostGreSQL and magic-removal version. I imagine it just works because the patch is so small, and can be veriied manually. But just in case.

comment:2 Changed 9 years ago by eugene@…

  • Summary changed from [patch] multithreading problem with various backends to [patch] Fixing multithreading problem with various database backends

Changed 9 years ago by eugene@…

patch for trunk (includes all database backends)

Changed 9 years ago by eugene@…

patch for magic-removal (includes all database backends)

comment:3 Changed 9 years ago by eugene@…

The patches are updated to include all 4 database backends. So far they worked on my sites (MySQL) and were reported by Ivan Sagalaev as working on Postgres. Nobody reported any negative experience. I assume we can roll them in at this point.

comment:4 Changed 9 years ago by eugene@…

Tickets #463, #900, and #1237 are closed as duplicates --- let's reduce a workload for developers.

comment:5 Changed 9 years ago by adrian

  • Resolution set to fixed
  • Status changed from new to closed

(In [2579]) Fixed #1442 -- Fixed multithreading problem with various database backends. Thanks, Eugene Lazutkin

comment:6 Changed 9 years ago by adrian

(In [2580]) Added missing file from [2579]. Refs #1442

Note: See TracTickets for help on using tickets.
Back to Top