UnicodeDecodeError in newforms-admin

[Ville Säävuori]

Im getting following error from a model admin that I just converted from oldforms admin to newforms:

Traceback (most recent call last):

 File "/home/unessanet/django/django/core/handlers/base.py", line 85, in get_response
   response = callback(request, *callback_args, **callback_kwargs)

 File "/home/unessanet/django/django/contrib/admin/sites.py", line 144, in root
   return self.model_page(request, *url.split('/', 2))

 File "/home/unessanet/django/django/views/decorators/cache.py", line 44, in _wrapped_view_func
   response = view_func(request, *args, **kwargs)

 File "/home/unessanet/django/django/contrib/admin/sites.py", line 161, in model_page
   return admin_obj(request, rest_of_url)

 File "/home/unessanet/django/django/contrib/admin/options.py", line 252, in __call__
   return self.change_view(request, unquote(url))

 File "/home/unessanet/django/django/contrib/admin/options.py", line 570, in change_view
   'title': _('Change %s') % opts.verbose_name,

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

The object in question has a proper __unicode__-method and a title with a unicode (non-ascii) character in it. The old admin worked just fine.

I'm running newest trunk (r8114) with MySQL, Apache and mod_python.

[Ville Säävuori]

The problem is not in the model (database) data but the Meta-data. I have a verbose_name on the model which is not explicitly set as unicode, which triggers the problem. This did work with the old admin, but does not with the new. Should it work now or should verbose_name and verbose_name_plural be explicitly set as unicode strings in model definition?

[Malcolm Tredinnick]

Are you running python 2.3, by any chance?

[Ville Säävuori]

No, python 2.5

But tested it on another machine and the error and solutions are the same: when I change the models Meta-attribute verbose_name from 'Foo Ää' to u'Foo Ää' it works with the newforms-admin. Both work with old admin.

[anonymous]

Some for me on python 2.4

[Karen Tracey <kmtracey@…>]

#8016 is a dup with a patch to django/contrib/admin/options.py that wraps references to verbose_name in force_unicode. Looks like this code perhaps didn't get properly unicodified when unicode support was merged from trunk to newforms-admin, probably because the code was new or moved from another file. A quick check of the pre-newforms-admin trunk code looks like the references to verbose_name were wrapped in force_unicode there.

[Malcolm Tredinnick]

The patch in #8016 looks pretty much correct. You can't substitute UTF-8 bytestrings containing non-ASCII characters (which are valid in verbose_name) into unicode strings (which are returned from ugettext()).

[Malcolm Tredinnick]

(In [8144]) Fixed #7992 -- Interpolate UTF-8 verbose_names correctly into unicode strings
in the admin interface. Patch from aabele. Refs #8016.

[Malcolm Tredinnick]

By the way, I tried very quickly to write a test for this by adding a non-ASCII, UTF-8 encoded verbose_name to one of the models in regressiontests/admin_views/models.py and it killed the test framework due to what is arguably a bug there. I did test this commit manually, though, and it failed before the patch as indicated in the report, and worked afterwards.

I need to debug the test framework problem and fix that before doing the next bit.

[John-Scott Atlakson]

Hey Malcolm, is there a separate ticket for the test framework bug? I don't want to hijack a closed ticket, but when doing something simple in my TestCase like self.assertContains(response, u'') I'm getting a UnicodeDecodeError. So if there is a separate ticket I should keep an eye on let me know. Thanks!

[Malcolm Tredinnick]

@john_scott: no, I never got around to opening one because I didn't have a small test case to demonstrate it yet. Feel free to make one if you like.

[John-Scott Atlakson]

Replying to mtredinnick:

@john_scott: no, I never got around to opening one because I didn't have a small test case to demonstrate it yet. Feel free to make one if you like.

After much investigating, I believe my issue is unrelated. I'll have to hit django-users to have someone straighten me out.