Django on Windows

(work in progress)

TOC

Test suite failures

As of r7787 the test suite is failing in two places when run on Windows + Python 2.5.2:

• If setuptools is installed, on a test from tests/regressiontests/templates/loaders.py related to trying to load a template from an egg (from a report on #django-dev, it seems this is a test that is also failing on OS X).
• On a test query from (tests/regressiontests/queries/models.py) when using the sqlite3 Django DB backend

tests/regressiontests/templates/loaders.py

TBD

tests/regressiontests/queries/models.py

This test from tests/regressiontests/queries/models.py fails when running test suite:

Bug #7087 -- dates with extra select columns
>>> Item.objects.dates('created', 'day').extra(select={'a': 1})
[datetime.datetime(2007, 12, 19, 0, 0), datetime.datetime(2007, 12, 20, 0, 0)] 

Error is:

======================================================================
FAIL: Doctest: regressiontests.queries.models.__test__.API_TESTS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\ramiro\django-trunk\django\test\_doctest.py", line 2180, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for regressiontests.queries.models.__test__.API_TESTS
  File "C:\ramiro\django-trunk\tests\regressiontests\queries\models.py", line unknown line number, in API_TESTS

----------------------------------------------------------------------
File "C:\ramiro\django-trunk\tests\regressiontests\queries\models.py", line ?, in regressiontests.queries.models.__test__.API_TESTS
Failed example:
    Item.objects.dates('created', 'day').extra(select={'a': 1})
Exception raised:
    Traceback (most recent call last):
      File "C:\ramiro\django-trunk\django\test\_doctest.py", line 1267, in __run
        compileflags, 1) in test.globs
      File "<doctest regressiontests.queries.models.__test__.API_TESTS[169]>", line 1, in <module>
        Item.objects.dates('created', 'day').extra(select={'a': 1})
      File "c:\ramiro\django-trunk\django\db\models\query.py", line 129, in __repr__
        return repr(list(self))
      File "c:\ramiro\django-trunk\django\db\models\query.py", line 141, in __len__
        self._result_cache.extend(list(self._iter))
      File "c:\ramiro\django-trunk\django\db\models\sql\subqueries.py", line 351, in results_iter
        for rows in self.execute_sql(MULTI):
      File "c:\ramiro\django-trunk\django\db\models\sql\query.py", line 1607, in execute_sql
        cursor.execute(sql, params)
      File "c:\ramiro\django-trunk\django\db\backends\sqlite3\base.py", line 136, in execute
        return Database.Cursor.execute(self, query, params)
    OperationalError: ORDER BY terms must not be non-integer constants


----------------------------------------------------------------------
Ran 253 tests in 433.032s

FAILED (failures=1)

This is a minimal models.py that shows the problem (extracted from the above regression test):

"""
>>> time1 = datetime.datetime(2007, 12, 19, 22, 25, 0)
>>> time2 = datetime.datetime(2007, 12, 19, 21, 0, 0)
>>> time3 = datetime.datetime(2007, 12, 20, 22, 25, 0)
>>> time4 = datetime.datetime(2007, 12, 20, 21, 0, 0)
>>> i1 = Item(name='one', created=time1, modified=time1)
>>> i1.save()
>>> i2 = Item(name='two', created=time2)
>>> i2.save()
>>> i3 = Item(name='three', created=time3)
>>> i3.save()
>>> i4 = Item(name='four', created=time4)
>>> i4.save()

>>> Item.objects.dates('created', 'day').extra(select={'a': 1})
[datetime.datetime(2007, 12, 19, 0, 0), datetime.datetime(2007, 12, 20, 0, 0)]

"""

from django.db import models
import datetime

class Item(models.Model):
    name = models.CharField(max_length=10)
    created = models.DateTimeField()
    modified = models.DateTimeField(blank=True, null=True)

    class Meta:
        ordering = ['name']

    def __unicode__(self):
        return self.name

The SQL being generated by Django is correct:

>>> Item.objects.dates('created', 'day').extra(select={'a': 1}).query.as_sql()
('SELECT DISTINCT (1) AS "a", django_date_trunc("day", "sqlite3_dates_item"."created") FROM "sqlite3_dates_item" ORDER BY 1 ASC', ())

Problems seems to be related to some bug in the version of pysqlite2 or sqlite3 (main suspect) included with the official Python 2.5.2 win32 installer (2.3.2 and 3.3.4 respectively):

Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from sqlite3 import dbapi2
>>> print dbapi2.version_info
(2, 3, 2)
>>> print dbapi2.sqlite_version_info
(3, 3, 4)

The same test works flawlessly on the following platforms:

• GNU/Debian Linux 4.0 (Python 2.4.4 + pysqlite2 2.3.2 + sqlite3 3.3.8)
• GNU/Debian Linux Sid as of Jun 29, 2008 (Python 2.5.2 + builtin pysqlite2 aka sqlite3 2.3.2 + sqlite3 3.5.9)

this may be thanks to the fact that on these platforms the shared sqlite3 library being used by the pysqlite2 (Python 2.4) or sqlite3 (Python 2.5) Python modules is the system wide one and so it hasn't been frozen at 3.3.4.

Possible solution

As sqlite/pysqlite development continues, the pysqlite project keeps publishing new win32 binary installers for Python 2.5. Latest as of today June 29, 2008 is version 2.4.1 (pysqlite-2.4.1.win32-py2.5.exe) that uses sqlite version 3.5.2:

>>> from pysqlite2 import dbapi2
>>> print dbapi2.version_info
(2, 4, 1)
>>> print dbapi2.sqlite_version_info
(3, 5, 2)
>>>

The error doesn't show itself when running the tests if one manages to force Django to use this newer version of pysqlite (see below). This seems to indicate that something was fixed on sqlite between versions 3.3.4 and 3.5.2 (see ​http://sqlite.org/changes.html).

How does this affect Django

The problem doesn't lie with Django, but leads to ask if the order being used to try loading the sqlite-related DB-API2 modules by the sqlite3 backed (django/db/backends/sqlite3/base.py) shouldn't be inverted: Try loading pysqlite2 first and if it fails then try loading sqlite3:

base.

@@ -3 +3 @@
 
 Python 2.3 and 2.4 require pysqlite2 (http://pysqlite.org/).
 
-Python 2.5 and later use the sqlite3 module in the standard library.
+Python 2.5 and later use pysqlite2 or the sqlite3 module in the standard
+library.
 """
 
 from django.db.backends import BaseDatabaseWrapper, BaseDatabaseFeatures, BaseDatabaseOperations, util
 try:
     try:
-        from sqlite3 import dbapi2 as Database
-    except ImportError:
         from pysqlite2 import dbapi2 as Database
+    except ImportError:
+        from sqlite3 import dbapi2 as Database
 except ImportError, e:
     import sys
     from django.core.exceptions import ImproperlyConfigured

Reasoning behind this is that this would allow the user to take advantage of newer pysqlite2/sqlite3 versions he/she may have installed even if using Python 2.5. This might be true regardless of the platform.

See also

• WindowsInstall