unicode strings in model doctest silently prevent the test from being run

[Dave Naffziger]

Here is the model that I created to demonstrate the bug. If you create a new project with a new models.py file that looks like model file below, and try to run the test runner: python manage.py test

You'll see that 0 tests are run. If you remove the last line from the doctest and rerun the same command, the test runs and fails.

from django.db import models

class PublicationManager(models.Manager):

    def get_printable_titles(self):
        titles = map(lambda x: x.title, self.all())
        printable_titles = '\n'.join(titles)
        return printable_titles

class Publication(models.Model):
    """
    # Create a couple of Publications.
    >>> p1 = Publication(id=None, title='The Python Journal')
    >>> p1.save()
    >>> p2 = Publication(id=None, title='Science News')
    >>> p2.save()
    >>> p3 = Publication(id=None, title='Science Weekly')
    >>> p3.save()
    >>> Publication.objects.get_printable_titles() 
    u'The Python Journal\nScience News\nScience Weekly'
    """
    title = models.CharField(max_length=30)
    objects = PublicationManager()

This problem can be corrected by making the docstring a raw docstring (preceding the string with an 'r'). However, the current 'silently fail to run the test' creates a dangerous scenario where users are unlikely to know that their tests aren't being run.

[Malcolm Tredinnick]

We're using a copy of Python's standard doctest module which has some well-known issues with non-ASCII data (although I didn't know about this one). We don't really want to deviate too far from the Python version for maintainability reasons, so if this were to turn out to be really fiddly to solve, we may punt by just saying "don't do that" in the docs. But I haven't read the code in question yet and it may turn out to be easy to fix (although we're rarely that lucky).

[Russell Keith-Magee]

This isn't a problem with Django - it's a problem with the Python doctest module. It should be reported on the issue tracker for the Python language. There is little to be gained by Django maintaining a parallel implementation of Doctest.