Opened 8 years ago

Closed 8 years ago

#5953 closed (fixed)

UnicodeDecodeError in blocktrans

Reported by: Cedric Rossi <cedric@…> Owned by: nobody
Component: Template system Version: master
Severity: Keywords:
Cc: cedric@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


In [6675] the following:

{% blocktrans with title as the_title %}{{ the_title }}{% endblocktrans %}

will fail with the following traceback, if title is a string containing non ascii characters:

Traceback (most recent call last):
File "/Users/cedric/src/koffeeweb/django/template/" in render_node
  814. result = node.render(context)
File "/Users/cedric/src/koffeeweb/django/templatetags/" in render
  77. result = re.sub('%(?!\()', '%%', result) % context

  UnicodeDecodeError at /fail/
  'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

On the other hand, this:

{% blocktrans with title|escape as the_title %}{{ the_title }}{% endblocktrans %}

works as expected.

I couldn't reproduce the problem in [6636], so this has probably been introduced recently (auto-escaping stuff?)

Please see the attached project, which defines two urls: /fail/ and /pass/ showing the problem

Attachments (1) (5.7 KB) - added by Cedric Rossi <cedric@…> 8 years ago.

Download all attachments as: .zip

Change History (5)

Changed 8 years ago by Cedric Rossi <cedric@…>

comment:1 Changed 8 years ago by telenieko

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 8 years ago by mtredinnick

What is the contents of "title" at this point when you trigger the error. The data involved is as important as the code for a case like this. Can you print type(title) and repr(title) in your view just before displaying the template please?

comment:3 Changed 8 years ago by Cedric Rossi <cedric@…>

Sorry, I could have been more precise in the description, but I believed attaching a working minimal project would make it easier for you to look at.

Here's the content of the view:

# coding: utf-8

from django.shortcuts import render_to_response

def test_pass(request):
    return render_to_response('test-pass.django.html',
                              {'title': "é"})

def test_fail(request):
    return render_to_response('test-fail.django.html',
                              {'title': "é"})

In case Trac mangles it, that's a &eaccute; The file is saved as utf-8.

print type(title) 
> <type 'str'>
print repr(title)
> '\xc3\xa9'

replacing "é" by u"é" fixes the problem, as expected, so my example was badly choosen, sorry about that.

Now the problem is, in my real project, title is a models.CharField(); DEFAULT_CHARSET is 'utf-8'; my table (MySQL) is properly defined as utf-8, and still, I've got (from a real example):

print type(title) 
> <type 'str'>
print repr(title)
> '\xc3\x8ele de Batz'

Is that expected?

I've tested again in [6636], and print and repr give the same results; still, inside the blocktrans, the problem originally reported remains in [6675] and not in [6636].

comment:4 Changed 8 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [6682]) Fixed a few problems with variable resolving inside of blocktrans tags. A couple of these were exposed by the auto-escaping changes, but I suspect the other one has been hiding in plain sight for a while.

Fixed #5952, #5953

Note: See TracTickets for help on using tickets.
Back to Top