Opened 18 years ago

Closed 18 years ago

Last modified 17 years ago

#1442 closed defect (fixed)

[patch] Fixing multithreading problem with various database backends

Reported by: eugene@… Owned by: Adrian Holovaty
Component: Database layer (models, ORM) Version:
Severity: normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

This patch supersedes tickets #463, #900, and #1237. Instead of lock-based connection pools it uses TLS (thread local storage). TLS is implemented by python 2.4 natively (threading.local), and by stub for python 2.3. The stub is taken verbatim from python 2.4.2 source distribution. I checked the license and it looks totally compatible with BSD license used by Django. Thank you Joseph Kocherhans (#1268).

The patch solves multithreading problems for MySQL (#463) and PostGreSQL (#900). I couldn't find a ticket for ADO/MSSQL and SQLite, so I didn't convert them. In general the fix is unbelievably simple:

try:
    # only exists in python 2.4+
    from threading import local
except ImportError:
    # import copy of _thread_local.py from python 2.4
    from django.utils._threading_local import local

class DatabaseWrapper(local):
...

It can be easily added for other backends, if required. (Additionally MySQL patch implements pinging connections.)

Attachments (4)

django.threading.patch (9.2 KB ) - added by eugene@… 18 years ago.
patch for Django trunk
magic.threading.patch (9.1 KB ) - added by eugene@… 18 years ago.
patch for magic-removal branch
django.threading.2.patch (10.6 KB ) - added by eugene@… 18 years ago.
patch for trunk (includes all database backends)
magic.threading.2.patch (10.5 KB ) - added by eugene@… 18 years ago.
patch for magic-removal (includes all database backends)

Download all attachments as: .zip

Change History (10)

by eugene@…, 18 years ago

Attachment: django.threading.patch added

patch for Django trunk

by eugene@…, 18 years ago

Attachment: magic.threading.patch added

patch for magic-removal branch

comment:1 by eugene@…, 18 years ago

I tested it on Python 2.4 and Python 2.3 (trunk) with MySQL (DreamHost setup). Please try it with PostGreSQL and magic-removal version. I imagine it just works because the patch is so small, and can be veriied manually. But just in case.

comment:2 by eugene@…, 18 years ago

Summary: [patch] multithreading problem with various backends[patch] Fixing multithreading problem with various database backends

by eugene@…, 18 years ago

Attachment: django.threading.2.patch added

patch for trunk (includes all database backends)

by eugene@…, 18 years ago

Attachment: magic.threading.2.patch added

patch for magic-removal (includes all database backends)

comment:3 by eugene@…, 18 years ago

The patches are updated to include all 4 database backends. So far they worked on my sites (MySQL) and were reported by Ivan Sagalaev as working on Postgres. Nobody reported any negative experience. I assume we can roll them in at this point.

comment:4 by eugene@…, 18 years ago

Tickets #463, #900, and #1237 are closed as duplicates --- let's reduce a workload for developers.

comment:5 by Adrian Holovaty, 18 years ago

Resolution: fixed
Status: newclosed

(In [2579]) Fixed #1442 -- Fixed multithreading problem with various database backends. Thanks, Eugene Lazutkin

comment:6 by Adrian Holovaty, 18 years ago

(In [2580]) Added missing file from [2579]. Refs #1442

Note: See TracTickets for help on using tickets.
Back to Top