Opened 7 years ago

Closed 7 years ago

Last modified 4 years ago

#8726 closed (fixed)

Unicode-errors with url-tags after changeset 8760

Reported by: Uninen Owned by: mtredinnick
Component: Core (Other) Version: master
Severity: Keywords: url, urls, unicode 1.0-blocker
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

I'm using url-templatetag across my projects. After Changeset r8760 many of (all?) those tags raise an UnicodeDecodeError.

Here is a snippet from the exception:

Exception Type: TemplateSyntaxError at /blogi/
Exception Value: Caught an exception while rendering: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Original Traceback (most recent call last):
  File "/Library/Python/2.5/site-packages/django/template/debug.py", line 71, in render_node
    result = node.render(context)
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 168, in render
    nodelist.append(node.render(context))
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 168, in render
    nodelist.append(node.render(context))
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 393, in render
    url = reverse(self.view_name, args=args, kwargs=kwargs)
  File "/Library/Python/2.5/site-packages/django/core/urlresolvers.py", line 250, in reverse
    *args, **kwargs)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Tag used:

{% url aino.views.tag_detail tag=tag %}

Related line in URLConf:

url(r'^tagit/(?P<tag>\S+)/$', "aino.views.tag_detail", name="tag_detail"),

(Tags in above code may contain unicode characters.)

My test environment is on Mac OS X 10.5, python 2.5, every file is utf-8 encoded.

Change History (14)

comment:1 follow-up: Changed 7 years ago by Uninen

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to duplicate
  • Status changed from new to closed

This is a duplicate of #8725

comment:2 in reply to: ↑ 1 Changed 7 years ago by kmtracey

Replying to Uninen:

This is a duplicate of #8725

Are you sure? The exception in #8725 is entirely different and doesn't appear to have anything to do with UnicodeDecode errors. This looks to me like a different problem, though yes it was introduced in the same changeset.

comment:3 Changed 7 years ago by Uninen

On another look, it may be a different problem. Not sure, though. Can someone duplicate these UnicodeDecodeErrors?

comment:4 Changed 7 years ago by carljm

I'm pretty sure this is a separate problem from #8725. I'm not able to duplicate it, but I'm not trying to use Unicode characters in a URL slug.

comment:5 Changed 7 years ago by mtredinnick

  • Resolution duplicate deleted
  • Status changed from closed to reopened

comment:6 Changed 7 years ago by mtredinnick

  • Owner changed from nobody to mtredinnick
  • Status changed from reopened to new

comment:7 Changed 7 years ago by mtredinnick

(In [8766]) Added a couple of extra tests to ensure url tag robustness. These are an
attempt to verify #8726.

Refs #8726.

comment:8 Changed 7 years ago by mtredinnick

We're going need to some more information to be able to repeat this, because the description doesn't have the critical piece. You can see in [8766] that there are now tests which do exactly what is described (there were previously tests that were very similar and using non-ASCII characters, but I added a couple that were even closer to this ticket) and they don't fail.

So if you could come up with a variation on those tests that does fail, that would be great. Failing that, a small example of what you are passing to the URL tag and what you URLConf looks like. If the critical URLConf is included inside another one, we need them all the way back to the top in case it is a variant on #8725.

If the problem's been fixed after [8763], also, please close this ticket.

comment:9 Changed 7 years ago by Uninen

Thank you for specifying the problem. It seems that the core of this problem is that the argument for url-tag is *not* unicode but a regular string. Following test produces exactly the same error that I have on some of my project URLs:

(in /tests/regressiontests/templates/tests.py )

'url08': (u'{% url метка_оператора v %}', {'v': 'Ω'}, '/url_tag/%D0%AE%D0%BD%D0%B8%D0%BA%D0%BE%D0%B4/%CE%A9/'),

Why this is, I don't know (yet), because in my case the arguments are normal model fields or model methods which should in every case give unicode strings. But still, I think we should be liberal in what to accept for the url-tag (ie. force arguments to unicode instead of raising an exception), right? At least with previous revisions all of my url-tags worked OK.

comment:10 Changed 7 years ago by jacob

  • Component changed from Uncategorized to Core framework
  • Keywords 1.0-blocker added
  • Triage Stage changed from Unreviewed to Accepted

comment:11 Changed 7 years ago by Uninen

As a clarification, above test fails with r8759, too. I still think it *should* work, though.

Furthermore, I studied my projects more carefully and I found that the original problem originated when I called the templatetag with the instance of django-tagging object as the argument (like {% url viewname tag=tag %}). When I changed the argument to the name field (like {% url viewname tag=tag.name %}), it worked fine. The django-tagging model has an __unicode__ method (that returns self.name, not explicitly converting name to unicode). Hope this helps.

And again, these same urls worked fine before r8760.

comment:12 Changed 7 years ago by mtredinnick

Thanks for the test case. Now I can work with something. But if django-tagging is attempting to return something from a __unicode__ method that isn't unicode that's an outright bug in django-tagging. Unicode methods must return unicode strings. That isn't optional. Still, using UTF-8 strings in legal situations should work, so this is still a bug. But the situation you have it arising is indicative of a bug somewhere else.

comment:13 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8777]) Fixed #8726 -- When doing reverse URL resolving, make sure we're consistently
dealing with unicode strings throughout by promoting up from UTF-8 as necessary.

comment:14 Changed 4 years ago by jacob

  • milestone 1.0 deleted

Milestone 1.0 deleted

Note: See TracTickets for help on using tickets.
Back to Top