./manage.py testserver with in-memory sqlite database fails on 1.4

[Henrique C. Alves]

Trying to run ./manage.py testserver on Django 1.4, Python 2.7.1 and sqlite 2.6 fails with the following traceback:

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/

Django Version: 1.4
Python Version: 2.7.1
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'taggit',
 'taggitext',
 'tinymce',
 'omelete.content',
 'omelete.video',
 'omelete.artwork')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Traceback:
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/admin/sites.py" in wrapper
  213.                 return self.admin_view(view, cacheable)(*args, **kwargs)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  91.                     response = view_func(request, *args, **kwargs)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  89.         response = view_func(request, *args, **kwargs)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner
  190.             if not self.has_permission(request):
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/admin/sites.py" in has_permission
  143.         return request.user.is_active and request.user.is_staff
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/utils/functional.py" in inner
  184.             self._setup()
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/utils/functional.py" in _setup
  248.         self._wrapped = self._setupfunc()
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/auth/middleware.py" in <lambda>
  16.         request.user = SimpleLazyObject(lambda: get_user(request))
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/auth/middleware.py" in get_user
  8.         request._cached_user = auth.get_user(request)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/auth/__init__.py" in get_user
  98.         user_id = request.session[SESSION_KEY]
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in __getitem__
  39.         return self._session[key]
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py" in _get_session
  165.                 self._session_cache = self.load()
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/contrib/sessions/backends/db.py" in load
  19.                 expire_date__gt=timezone.now()
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/manager.py" in get
  131.         return self.get_query_set().get(*args, **kwargs)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/query.py" in get
  361.         num = len(clone)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/query.py" in __len__
  85.                 self._result_cache = list(self.iterator())
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/query.py" in iterator
  291.         for row in compiler.results_iter():
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in results_iter
  763.         for rows in self.execute_sql(MULTI):
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  818.         cursor.execute(sql, params)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/backends/util.py" in execute
  40.             return self.cursor.execute(sql, params)
File "/Users/hcalves/.virtualenvs/omelete.v2/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py" in execute
  337.             return Database.Cursor.execute(self, query, params)

Exception Type: DatabaseError at /admin/
Exception Value: no such table: django_session

I was unable to pinpoint the actual problem. Changing the settings to use the special :memory: database and running ./manage.py runserver also fails with the same traceback. The problem is not specific to the session app: it will also fail on django_site table or others. Apparently it doesn't create any tables for the apps on INSTALLED_APPS.

[DNX]

I can confirm this bug under following environment:

Ubuntu (10.04 or 11.10) 64bit

Clean installation of Django 1.4 in a clean virtualenv.

Start a project with startproject

Enable "admin" in INSTALLED_APPS and urls.py

Run ./manage.py testserver

Navigate to ​http://localhost:8000/admin/

You will see the above reported error.

As temporary fix, you can set the TEST_NAME for your DATABASE in settings.py

Example:

if 'testserver' in sys.argv:
    DATABASES['default']['TEST_NAME'] = '/tmp/testserver.db'

[Henrique C. Alves]

Replying to DNX:

I can confirm this bug under following environment:

...

As temporary fix, you can set the TEST_NAME for your DATABASE in settings.py

Example:

if 'testserver' in sys.argv:
    DATABASES['default']['TEST_NAME'] = '/tmp/testserver.db'

Thank you for the workaround DNX. I believe this is a regression then?

[Claude Paroz]

Bisecting the issue brought me to the changeset r16427. So I suspect this is related to threading. It is likely that when threading is enabled, memory databases are not shared between threads. Patching should be trivial.

[Claude Paroz]

use_threading=False for testserver usage

[Claude Paroz]

Note that this command is totally untested. Testing the creation of the :memory: test database when there is already a test database setup might reveal tricky...

[Claude Paroz]

Use test_db_allows_multiple_connections to determine threading capabilities

[Anssi Kääriäinen]

The patch looks good to me, and I can verify it fixes the testserver issue.

It is a bit unfortunate that we don't have a way to test these issues.

[Claude Paroz <claude@…>]

In [bb4452f212e211bca7b6b57904d59270ffd7a503]:

Fixed #18019 -- Use threaded runserver only when database supports it.

[Claude Paroz <claude@…>]

In [6bb85d98b0574aec1aec9fb97c26ce16bc060522]:

[1.4.x] Fixed #18019 -- Use threaded runserver only when database supports it.