#17998 closed Bug (worksforme)
database threading issue with dev server and sqlite
Reported by: | anonymous | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 1.4 |
Severity: | Normal | Keywords: | |
Cc: | anssi.kaariainen@… | Triage Stage: | Unreviewed |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Steps to reproduce:
- Run dev server in the standard way "./manage.py runserver"
- Request a Django-served page in a browser
- Likely important that the page has links to static media files served by the dev server so the browser initiates multiple simultaneous requests
- There is middleware on the site that hits the session table in the db for every request, including the static media files
- You may need to flush the browser cache when trying to repeatedly see the error
Expected behavior:
- All files served normally
Observed behavior:
- Non-deterministic, some requests cause an error
- Error response is the following: "A server error occurred. Please contact the administrator."
- The exception in the error log reads:
DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 4374663168 and this is thread id 4385443840.
- Note that the Django site in question is only using the Django ORM, not reaching deeper into the django.db library, and it does not explicitly spawn any threads
- I'm including a full backtrace from the dev server below.
Environment:
- Mac OS X 10.6.8
- python27 @2.7.2_1 (MacPorts)
- py27-sqlite @2.6.3_0 (MacPorts)
- sqlite3 @3.7.11_0 (MacPorts)
- Django 1.4.0-final0 (pip)
- Firefox 11.0
Full backtrace:
DEBUG:django.db.backends:(0.003) SELECT "django_session"."session_key", "django_session"."session_data", "django_session"."expire_date" FROM "django_session" WHERE ("django_session"."session_key" = 02462878adc69e05312600990114b569 AND "django_session"."expire_date" > 2012-03-28 10:29:14.458986 ); args=('02462878adc69e05312600990114b569', u'2012-03-28 10:29:14.458986') DEBUG:django.db.backends:(0.000) SELECT (1) AS "a" FROM "django_session" WHERE "django_session"."session_key" = 02462878adc69e05312600990114b569 LIMIT 1; args=('02462878adc69e05312600990114b569',) DEBUG:django.db.backends:(0.000) UPDATE "django_session" SET "session_data" = NDY1YWVmMjI5ZTZiOGIxODUyMjVlMTAxOGQ2MjUzNTEzOWRmMWQ1NzqAAn1xAS4= , "expire_date" = 2012-03-28 10:59:14.471961 WHERE "django_session"."session_key" = 02462878adc69e05312600990114b569 ; args=('NDY1YWVmMjI5ZTZiOGIxODUyMjVlMTAxOGQ2MjUzNTEzOWRmMWQ1NzqAAn1xAS4=\n', u'2012-03-28 10:59:14.471961', '02462878adc69e05312600990114b569') Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run self.result = application(self.environ, self.start_response) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__ return self.application(environ, start_response) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 243, in __call__ signals.request_finished.send(sender=self.__class__) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 172, in send response = receiver(signal=self, sender=sender, **named) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django_digest/backend/db.py", line 54, in close_connection self.connection.close() File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 319, in close self.validate_thread_sharing() File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/__init__.py", line 136, in validate_thread_sharing % (self.alias, self._thread_ident, thread.get_ident())) DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 4374663168 and this is thread id 4385443840. [28/Mar/2012 10:29:14] "GET / HTTP/1.1" 500 59
Thanks very much, Django is great software!
Change History (7)
comment:1 by , 13 years ago
Cc: | added |
---|---|
Triage Stage: | Unreviewed → Accepted |
comment:2 by , 13 years ago
Triage Stage: | Accepted → Unreviewed |
---|
I am going to revert the accepted triaging. I have been trying to reproduce this, and I can't. The method I have been using is a test view which does a database query and then sleeps a little. Then I hit that view continuously with high-concurrency AB benchmark. No errors.
In addition I have been reading the code, and I can't see a way the connection would get shared between threads.
So, more info is needed.
comment:3 by , 13 years ago
Resolution: | → worksforme |
---|---|
Status: | new → closed |
I can't reproduce this and I haven't seen other complaints about this. Closing this as worksforme.
comment:4 by , 12 years ago
The bug turned out to be in the django_digest package that we were using.
See some discussion here:
https://bitbucket.org/akoha/django-digest/issue/10/conflict-with-global-databasewrapper
A patched version of django-digest that works for me can be found here:
https://github.com/trey0/django-digest
Sorry for reporting this in the wrong place...
follow-up: 6 comment:5 by , 12 years ago
got the same exception when enable
from gevent import monkey; monkey.patch_all()
comment:6 by , 12 years ago
Replying to anonymous:
got the same exception when enable
from gevent import monkey; monkey.patch_all()
Could be fixed by not patching threads:
gevent.monkey.patch_all(socket=True, dns=True, time=True, select=True,thread=False, os=True, ssl=True, httplib=False, aggressive=True)
comment:7 by , 11 years ago
I am recently had this issue occur after I increased the logging level on a mod_wsgi instance.... I log quite a bit in debug mode, but rarely with that setting on anything other than in my development environment (which is usually running via runserver).
I have looked into this a little, but I can't see where the connection is shared between the threads. Any ideas? It should be impossible to get the same
DatabaseWrapper
in different threads unless explicitly shared. Of course, it is possible there is a bug making it not so impossible, which would be more serious.While I haven't reproduced this bug, I am still going to mark this accepted, as I have seen some strange threading errors, too (caused by using ipdb in my case, though).