newforms UnicodeEncodeError in EmailField on non-successful validation

Just found that it happens only if settings.LANGUAGE_CODE == 'pl'

However, when settings.LANGUAGE_CODE == 'en' and after translation.activate( 'pl' ) - it DOES NOT happen (and I could except it happens, as most of values are translated to Polish).

Fixed formatting in description.

see #3395, it contains a patch... its due to ValidationErrors? containing unicode error messages that cannot be handled by some template tags, or other functions in this case...

Happens with norwegian translation too, any way to fix this, should translations not be in utf-8?

django rev 4490

traceback

Traceback (most recent call last):
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/template/__init__.py" in render_node
  718. result = node.render(context)
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/template/__init__.py" in render
  768. output = self.filter_expression.resolve(context)
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/template/__init__.py" in resolve
  561. obj = resolve_variable(self.var, context)
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/template/__init__.py" in resolve_variable
  655. current = current()
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/newforms/forms.py" in as_ul
  135. return self._html_output(u'<li>%(errors)s%(label)s %(field)s%(help_text)s</li>', u'<li>%s</li>', '</li>', u' %s', False)
File "/var/www/vhosts/amc-info.com/httpdocs/magic-removal/django/newforms/forms.py" in _html_output
  116. output.append(normal_row % {'errors': bf_errors, 'label': label, 'field': unicode(bf), 'help_text': help_text})

  UnicodeEncodeError at /spoersmaal/
  'ascii' codec can't encode character u'\xe5' in position 43: ordinal not in range(128)

local vars

bf  	
<django.newforms.forms.BoundField object at 0xb60ec86c>
bf_errors 	
[u'Dette feltet er p\xe5krevd.']
error_row 	
u'<li>%s</li>'
errors_on_separate_row 	
False
field 	
<django.newforms.fields.CharField object at 0xb60e8ecc>
help_text 	
u''
help_text_html 	
u' %s'
hidden_fields 	
[]
label 	
u'Overskrift:'
name 	
'overskrift'
normal_row 	
u'<li>%(errors)s%(label)s %(field)s%(help_text)s</li>'
output 	
[]
row_ender 	
'</li>'
self 	
<django.newforms.models.SpoersmaalForm object at 0xb620c48c>
top_errors 	
[]

temporary solution for utf-8 gettext translations with newforms, encodes as xmlcharrefs

I'm not sure whether StrAndUnicode is the right level to fix the bug, since it's used in many places. Are you sure this doesn't somehow put xmlcharrefs in your database?

Quite sure, this is only done for validationerrors. Can seem like StrAndUnicode? from the patch, but it's not. :)

utf-8 decode before unicode , attached patch to wrong ticket

The attached patch produces incorrect output for me (I get the XML character entities in the validation message, instead of the swedish characters). I will attach a new patch that fixes the issue without doing going to/from ascii encoding for no reason, and it works :P

smaller fix that does not needlessly convert to/from ascii

This has been fixed as a result of the changes in the unicode branch, at least as best I can tell from the description, since nobody has posted a test case. I can render forms with invalid Email fields and missing content in languages using non-ASCII characters in the translated errors without problems.

I'll close this ticket once the unicode branch is merged back to trunk.

(In [5609]) Merged Unicode branch into trunk (r4952:5608). This should be fully backwards compatible for all practical purposes.

Fixed #2391, #2489, #2996, #3322, #3344, #3370, #3406, #3432, #3454, #3492, #3582, #3690, #3878, #3891, #3937, #4039, #4141, #4227, #4286, #4291, #4300, #4452, #4702