Opened 11 years ago

Closed 11 years ago

Last modified 7 years ago

#8726 closed (fixed)

Unicode-errors with url-tags after changeset 8760

Reported by: Ville Säävuori Owned by: Malcolm Tredinnick
Component: Core (Other) Version: master
Severity: Keywords: url, urls, unicode 1.0-blocker
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


I'm using url-templatetag across my projects. After Changeset r8760 many of (all?) those tags raise an UnicodeDecodeError.

Here is a snippet from the exception:

Exception Type: TemplateSyntaxError at /blogi/
Exception Value: Caught an exception while rendering: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Original Traceback (most recent call last):
  File "/Library/Python/2.5/site-packages/django/template/", line 71, in render_node
    result = node.render(context)
  File "/Library/Python/2.5/site-packages/django/template/", line 168, in render
  File "/Library/Python/2.5/site-packages/django/template/", line 168, in render
  File "/Library/Python/2.5/site-packages/django/template/", line 393, in render
    url = reverse(self.view_name, args=args, kwargs=kwargs)
  File "/Library/Python/2.5/site-packages/django/core/", line 250, in reverse
    *args, **kwargs)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Tag used:

{% url aino.views.tag_detail tag=tag %}

Related line in URLConf:

url(r'^tagit/(?P<tag>\S+)/$', "aino.views.tag_detail", name="tag_detail"),

(Tags in above code may contain unicode characters.)

My test environment is on Mac OS X 10.5, python 2.5, every file is utf-8 encoded.

Change History (14)

comment:1 Changed 11 years ago by Ville Säävuori

Resolution: duplicate
Status: newclosed

This is a duplicate of #8725

comment:2 in reply to:  1 Changed 11 years ago by Karen Tracey

Replying to Uninen:

This is a duplicate of #8725

Are you sure? The exception in #8725 is entirely different and doesn't appear to have anything to do with UnicodeDecode errors. This looks to me like a different problem, though yes it was introduced in the same changeset.

comment:3 Changed 11 years ago by Ville Säävuori

On another look, it may be a different problem. Not sure, though. Can someone duplicate these UnicodeDecodeErrors?

comment:4 Changed 11 years ago by Carl Meyer

I'm pretty sure this is a separate problem from #8725. I'm not able to duplicate it, but I'm not trying to use Unicode characters in a URL slug.

comment:5 Changed 11 years ago by Malcolm Tredinnick

Resolution: duplicate
Status: closedreopened

comment:6 Changed 11 years ago by Malcolm Tredinnick

Owner: changed from nobody to Malcolm Tredinnick
Status: reopenednew

comment:7 Changed 11 years ago by Malcolm Tredinnick

(In [8766]) Added a couple of extra tests to ensure url tag robustness. These are an
attempt to verify #8726.

Refs #8726.

comment:8 Changed 11 years ago by Malcolm Tredinnick

We're going need to some more information to be able to repeat this, because the description doesn't have the critical piece. You can see in [8766] that there are now tests which do exactly what is described (there were previously tests that were very similar and using non-ASCII characters, but I added a couple that were even closer to this ticket) and they don't fail.

So if you could come up with a variation on those tests that does fail, that would be great. Failing that, a small example of what you are passing to the URL tag and what you URLConf looks like. If the critical URLConf is included inside another one, we need them all the way back to the top in case it is a variant on #8725.

If the problem's been fixed after [8763], also, please close this ticket.

comment:9 Changed 11 years ago by Ville Säävuori

Thank you for specifying the problem. It seems that the core of this problem is that the argument for url-tag is *not* unicode but a regular string. Following test produces exactly the same error that I have on some of my project URLs:

(in /tests/regressiontests/templates/ )

'url08': (u'{% url метка_оператора v %}', {'v': 'Ω'}, '/url_tag/%D0%AE%D0%BD%D0%B8%D0%BA%D0%BE%D0%B4/%CE%A9/'),

Why this is, I don't know (yet), because in my case the arguments are normal model fields or model methods which should in every case give unicode strings. But still, I think we should be liberal in what to accept for the url-tag (ie. force arguments to unicode instead of raising an exception), right? At least with previous revisions all of my url-tags worked OK.

comment:10 Changed 11 years ago by Jacob

Component: UncategorizedCore framework
Keywords: 1.0-blocker added
Triage Stage: UnreviewedAccepted

comment:11 Changed 11 years ago by Ville Säävuori

As a clarification, above test fails with r8759, too. I still think it *should* work, though.

Furthermore, I studied my projects more carefully and I found that the original problem originated when I called the templatetag with the instance of django-tagging object as the argument (like {% url viewname tag=tag %}). When I changed the argument to the name field (like {% url viewname %}), it worked fine. The django-tagging model has an __unicode__ method (that returns, not explicitly converting name to unicode). Hope this helps.

And again, these same urls worked fine before r8760.

comment:12 Changed 11 years ago by Malcolm Tredinnick

Thanks for the test case. Now I can work with something. But if django-tagging is attempting to return something from a __unicode__ method that isn't unicode that's an outright bug in django-tagging. Unicode methods must return unicode strings. That isn't optional. Still, using UTF-8 strings in legal situations should work, so this is still a bug. But the situation you have it arising is indicative of a bug somewhere else.

comment:13 Changed 11 years ago by Malcolm Tredinnick

Resolution: fixed
Status: newclosed

(In [8777]) Fixed #8726 -- When doing reverse URL resolving, make sure we're consistently
dealing with unicode strings throughout by promoting up from UTF-8 as necessary.

comment:14 Changed 7 years ago by Jacob

milestone: 1.0

Milestone 1.0 deleted

Note: See TracTickets for help on using tickets.
Back to Top