Broken model unicode methods can lead to bare exceptions

[Karen Tracey]

Given this model registerd with the admin:

# -*- coding: utf-8 -*-
from django.db import models

class BrokenUnicodeMethod(models.Model):
    name = models.CharField(max_length=6)
    def __unicode__(self):
        return 'Názov: %s' % self.name

attempting to add one in the admin (with DEBUG set to True) generates a bare traceback:

Traceback (most recent call last):

  File "/home/kmt/django/trunk/django/core/servers/basehttp.py", line 278, in run
    self.result = application(self.environ, self.start_response)

  File "/home/kmt/django/trunk/django/core/servers/basehttp.py", line 635, in __call__
    return self.application(environ, start_response)

  File "/home/kmt/django/trunk/django/core/handlers/wsgi.py", line 239, in __call__
    response = self.get_response(request)

  File "/home/kmt/django/trunk/django/core/handlers/base.py", line 128, in get_response
    return self.handle_uncaught_exception(request, resolver, exc_info)

  File "/home/kmt/django/trunk/django/core/handlers/base.py", line 148, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)

  File "/home/kmt/django/trunk/django/views/debug.py", line 39, in technical_500_response
    html = reporter.get_traceback_html()

  File "/home/kmt/django/trunk/django/views/debug.py", line 97, in get_traceback_html
    'exception_value': smart_unicode(self.exc_value, errors='replace'),

  File "/home/kmt/django/trunk/django/utils/encoding.py", line 35, in smart_unicode
    return force_unicode(s, encoding, strings_only, errors)

  File "/home/kmt/django/trunk/django/utils/encoding.py", line 70, in force_unicode
    raise DjangoUnicodeDecodeError(s, *e.args)

DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128). You passed in DjangoUnicodeDecodeError('ascii', 'N\xc3\xa1zov: ', 1, 2, 'ordinal not in range(128)') (<class 'django.utils.encoding.DjangoUnicodeDecodeError'>)

The original DjanoUnicodeDecodeError comes from the admin trying a force_unicode on the new object for the log message. That gets caught and get_traceback_html attempts to calls smart_unicode on the exception value, which ultimately calls the __str__ method for DjangoUnicodeDecodeError:

    def __str__(self):
        original = UnicodeDecodeError.__str__(self)
        return '%s. You passed in %r (%s)' % (original, self.obj,
                type(self.obj))

self.obj is the origin object where __unicode__ generated an exception. But the attempt to use its repr is going to call unicode on it again, since __repr__ for an model is (from django/db/models/base.py):

    def __repr__(self):
        return smart_str(u'<%s: %s>' % (self.__class__.__name__, unicode(self)))

and we get another UnicodeDecodeException.

Of course the model's __unicode__ method is broken, returning non-ASCII data in a bytestring instead of a unicode object, but Django shouldn't obscure that error by generating an error itself trying to say that. Specifically I think the model __repr__ ought to guard against unicode(self) generating an exception and substitute placeholder text if that happens, but I'm not entirely sure of that so I didn't just do it. Other opinions?

The original problem was reported in this thread: ​http://groups.google.com/group/django-users/browse_thread/thread/4209cc8f669f5e12

and involved a __unicode__ method that looked like this:

    def __unicode__(self):
        return "%s %s" % (self.vrstva, self.nazov)

Here 'vrstva' is a foreign key, with a valid __unicode__ method that was returning non-ASCII data, converted to a utf-8 bytestring by the default Model __str__ method:

    def __str__(self):
        if hasattr(self, '__unicode__'):
            return force_unicode(self).encode('utf-8')
        return '%s object' % self.__class__.__name__

[Malcolm Tredinnick]

I wouldn't want to diguise this error too much, but you're right about bullet-proofing the debug page. Making repr() immune to the problem sounds like a good fix, since that will help with interactive debugging as well. Catching unicode encode errors and replacing it with something like "[Bad Unicode data]" would work for me.

[Karen Tracey]

(In [9475]) Fixed #9608: Ensured a Model's default repr() is printable even if its unicode method raises a Unicode error.

[Karen Tracey]

(In [9477]) [1.0.X] Fixed #9608: Ensured a Model's default repr() is printable even if its unicode method raises a Unicode error.

r9475 from trunk.