Django

Code

Ticket #8726 (closed: fixed)

Opened 3 months ago

Last modified 3 months ago

Unicode-errors with url-tags after changeset 8760

Reported by: Uninen Assigned to: mtredinnick
Milestone: 1.0 Component: Core framework
Version: SVN Keywords: url, urls, unicode 1.0-blocker
Cc: Triage Stage: Accepted
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

I'm using url-templatetag across my projects. After Changeset r8760 many of (all?) those tags raise an UnicodeDecodeError?.

Here is a snippet from the exception:

Exception Type: TemplateSyntaxError at /blogi/
Exception Value: Caught an exception while rendering: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Original Traceback (most recent call last):
  File "/Library/Python/2.5/site-packages/django/template/debug.py", line 71, in render_node
    result = node.render(context)
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 168, in render
    nodelist.append(node.render(context))
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 168, in render
    nodelist.append(node.render(context))
  File "/Library/Python/2.5/site-packages/django/template/defaulttags.py", line 393, in render
    url = reverse(self.view_name, args=args, kwargs=kwargs)
  File "/Library/Python/2.5/site-packages/django/core/urlresolvers.py", line 250, in reverse
    *args, **kwargs)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128)

Tag used:

{% url aino.views.tag_detail tag=tag %}

Related line in URLConf:

url(r'^tagit/(?P<tag>\S+)/$', "aino.views.tag_detail", name="tag_detail"),

(Tags in above code may contain unicode characters.)

My test environment is on Mac OS X 10.5, python 2.5, every file is utf-8 encoded.

Attachments

Change History

(follow-up: ↓ 2 ) 08/31/08 08:07:54 changed by Uninen

  • status changed from new to closed.
  • needs_better_patch changed.
  • resolution set to duplicate.
  • needs_tests changed.
  • needs_docs changed.

This is a duplicate of #8725

(in reply to: ↑ 1 ) 08/31/08 08:16:06 changed by kmtracey

Replying to Uninen:

This is a duplicate of #8725

Are you sure? The exception in #8725 is entirely different and doesn't appear to have anything to do with UnicodeDecode errors. This looks to me like a different problem, though yes it was introduced in the same changeset.

08/31/08 08:24:35 changed by Uninen

On another look, it may be a different problem. Not sure, though. Can someone duplicate these UnicodeDecodeErrors??

08/31/08 11:53:02 changed by carljm

I'm pretty sure this is a separate problem from #8725. I'm not able to duplicate it, but I'm not trying to use Unicode characters in a URL slug.

08/31/08 12:16:06 changed by mtredinnick

  • status changed from closed to reopened.
  • resolution deleted.

08/31/08 12:16:12 changed by mtredinnick

  • owner changed from nobody to mtredinnick.
  • status changed from reopened to new.

08/31/08 12:29:50 changed by mtredinnick

(In [8766]) Added a couple of extra tests to ensure url tag robustness. These are an attempt to verify #8726.

Refs #8726.

08/31/08 12:32:58 changed by mtredinnick

We're going need to some more information to be able to repeat this, because the description doesn't have the critical piece. You can see in [8766] that there are now tests which do exactly what is described (there were previously tests that were very similar and using non-ASCII characters, but I added a couple that were even closer to this ticket) and they don't fail.

So if you could come up with a variation on those tests that does fail, that would be great. Failing that, a small example of what you are passing to the URL tag and what you URLConf looks like. If the critical URLConf is included inside another one, we need them all the way back to the top in case it is a variant on #8725.

If the problem's been fixed after [8763], also, please close this ticket.

08/31/08 13:30:45 changed by Uninen

Thank you for specifying the problem. It seems that the core of this problem is that the argument for url-tag is *not* unicode but a regular string. Following test produces exactly the same error that I have on some of my project URLs:

(in /tests/regressiontests/templates/tests.py )

'url08': (u'{% url метка_оператора v %}', {'v': 'Ω'}, '/url_tag/%D0%AE%D0%BD%D0%B8%D0%BA%D0%BE%D0%B4/%CE%A9/'),

Why this is, I don't know (yet), because in my case the arguments are normal model fields or model methods which should in every case give unicode strings. But still, I think we should be liberal in what to accept for the url-tag (ie. force arguments to unicode instead of raising an exception), right? At least with previous revisions all of my url-tags worked OK.

08/31/08 16:03:33 changed by jacob

  • keywords changed from url, urls, unicode to url, urls, unicode 1.0-blocker.
  • component changed from Uncategorized to Core framework.
  • stage changed from Unreviewed to Accepted.

08/31/08 16:40:40 changed by Uninen

As a clarification, above test fails with r8759, too. I still think it *should* work, though.

Furthermore, I studied my projects more carefully and I found that the original problem originated when I called the templatetag with the instance of django-tagging object as the argument (like {% url viewname tag=tag %}). When I changed the argument to the name field (like {% url viewname tag=tag.name %}), it worked fine. The django-tagging model has an __unicode__ method (that returns self.name, not explicitly converting name to unicode). Hope this helps.

And again, these same urls worked fine before r8760.

08/31/08 17:18:07 changed by mtredinnick

Thanks for the test case. Now I can work with something. But if django-tagging is attempting to return something from a __unicode__ method that isn't unicode that's an outright bug in django-tagging. Unicode methods must return unicode strings. That isn't optional. Still, using UTF-8 strings in legal situations should work, so this is still a bug. But the situation you have it arising is indicative of a bug somewhere else.

08/31/08 17:47:25 changed by mtredinnick

  • status changed from new to closed.
  • resolution set to fixed.

(In [8777]) Fixed #8726 -- When doing reverse URL resolving, make sure we're consistently dealing with unicode strings throughout by promoting up from UTF-8 as necessary.


Add/Change #8726 (Unicode-errors with url-tags after changeset 8760)




Change Properties
Action