Opened 10 years ago

Closed 10 years ago

#5953 closed (fixed)

UnicodeDecodeError in blocktrans

Reported by: Cedric Rossi <cedric@…> Owned by: nobody
Component: Template system Version: master
Severity: Keywords:
Cc: cedric@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

In [6675] the following:

{% blocktrans with title as the_title %}{{ the_title }}{% endblocktrans %}

will fail with the following traceback, if title is a string containing non ascii characters:

Traceback (most recent call last):
File "/Users/cedric/src/koffeeweb/django/template/__init__.py" in render_node
  814. result = node.render(context)
File "/Users/cedric/src/koffeeweb/django/templatetags/i18n.py" in render
  77. result = re.sub('%(?!\()', '%%', result) % context

  UnicodeDecodeError at /fail/
  'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

On the other hand, this:

{% blocktrans with title|escape as the_title %}{{ the_title }}{% endblocktrans %}

works as expected.

I couldn't reproduce the problem in [6636], so this has probably been introduced recently (auto-escaping stuff?)

Please see the attached project, which defines two urls: /fail/ and /pass/ showing the problem

Attachments (1)

blocktranstest.zip (5.7 KB) - added by Cedric Rossi <cedric@…> 10 years ago.

Download all attachments as: .zip

Change History (5)

Changed 10 years ago by Cedric Rossi <cedric@…>

Attachment: blocktranstest.zip added

comment:1 Changed 10 years ago by Marc Fargas

Triage Stage: UnreviewedAccepted

comment:2 Changed 10 years ago by Malcolm Tredinnick

What is the contents of "title" at this point when you trigger the error. The data involved is as important as the code for a case like this. Can you print type(title) and repr(title) in your view just before displaying the template please?

comment:3 Changed 10 years ago by Cedric Rossi <cedric@…>

Sorry, I could have been more precise in the description, but I believed attaching a working minimal project would make it easier for you to look at.

Here's the content of the view:

# coding: utf-8

from django.shortcuts import render_to_response

def test_pass(request):
    return render_to_response('test-pass.django.html',
                              {'title': "é"})

def test_fail(request):
    return render_to_response('test-fail.django.html',
                              {'title': "é"})

In case Trac mangles it, that's a &eaccute; The file is saved as utf-8.

print type(title) 
> <type 'str'>
print repr(title)
> '\xc3\xa9'

replacing "é" by u"é" fixes the problem, as expected, so my example was badly choosen, sorry about that.

Now the problem is, in my real project, title is a models.CharField(); DEFAULT_CHARSET is 'utf-8'; my table (MySQL) is properly defined as utf-8, and still, I've got (from a real example):

print type(title) 
> <type 'str'>
print repr(title)
> '\xc3\x8ele de Batz'

Is that expected?

I've tested again in [6636], and print and repr give the same results; still, inside the blocktrans, the problem originally reported remains in [6675] and not in [6636].

comment:4 Changed 10 years ago by Malcolm Tredinnick

Resolution: fixed
Status: newclosed

(In [6682]) Fixed a few problems with variable resolving inside of blocktrans tags. A couple of these were exposed by the auto-escaping changes, but I suspect the other one has been hiding in plain sight for a while.

Fixed #5952, #5953

Note: See TracTickets for help on using tickets.
Back to Top