Ticket #18251: workaround.py

File workaround.py, 3.0 KB (added by harm, 8 years ago)

workaround (not a fix) wrapper around DjangoWSGIHandler

Line 
1from __future__ import with_statement
2from  django.core.handlers.wsgi import WSGIHandler as DjangoWSGIHandler
3
4from threading import Lock
5
6class WSGIHandler(DjangoWSGIHandler):
7    """
8    This provides a threadsafe drop-in replacement of django's WSGIHandler.
9
10    Initialisation of django via a multithreaded wsgi handler is not safe.
11    It is vulnerable to a A-B B-A deadlock.
12
13    When two threads bootstrap django via different urls you have a change to hit
14    the following deadlock.
15
16      thread 1                                               thread  2
17        view A                                                  view B
18         import file foo            import lock foo               import file bar  import lock bar
19               bootstrap django     lock AppCache.write_lock
20                    import file bar import lock bar  <-- blocks
21                                                                     bootstrap django    lock AppCache.write_lock  <----- deadlock
22
23    workaround for an AB BA deadlock:  wrap it in a lock C.
24
25            lock C                      lock C
26                lock A                      lock B
27                lock B                      lock A
28                release B                   release A
29                release A                   release A
30            release C                   release C
31               
32 
33    Thats exactly what this class does,  but... only for the first few calls.
34    After that we remove the lock C.  as the AppCache.write_lock is only held when django is booted.
35
36    If we would not remove the lock C after the first few calls, that would make the whole app single threaded again.
37
38    Usage:   
39        in your wsgi file replace   the following lines
40                import django.core.handlers.wsgi.WSGIHandler
41                application = django.core.handlers.wsgi.WSGIHandler
42        by
43                import threadsafe_wsgi
44                application = threadsafe_wsgi.WSGIHandler
45
46
47    FAQ:
48        Q: why would you want threading in the first place ?               
49        A: to reduce memory. Big apps can consume hundeds of megabytes each.  adding processes is then much more expensive than threads.
50           that memory is better spend caching, when threads are almost free.
51
52        Q: this deadlock, it looks far-fetched, is this real ?
53        A: yes we had this problem on production machines.
54    """
55    __initLock = Lock()  # lock C
56    __initialized = 0
57
58    def __call__(self, environ, start_response):
59        # the first calls (4) we squeeze everybody through lock C
60        # this basically serializes all threads
61        MIN_INIT_CALLS = 4
62        if self.__initialized < MIN_INIT_CALLS:
63            with self.__initLock:
64                ret = DjangoWSGIHandler.__call__(self, environ, start_response)
65                self.__initialized += 1
66                return ret
67        else:
68            # we are safely bootrapped, skip lock C
69            # now we are running multi-threaded again
70            return  DjangoWSGIHandler.__call__(self, environ, start_response)
Back to Top