UnicodeEncodeError when validating forms with non-ascii field names

[Alan Justino da Silva]

I am creating forms dynamically, so users can type field names.

Django is raising UnicodeEncodeError when some field name has a non-ascii name. Is simple not acceptable to constrain users to ASCII only.

Baked example:

class UtilizacaoForm(forms.Form):
    pass

ut = UtilizacaoForm()
ut.fields[u'Diâmetro'] = forms.CharField(max_length=3)

ut.is_valid()
File "django/forms/forms.py" in is_valid
  120.         return self.is_bound and not bool(self.errors)
File "django/forms/forms.py" in _get_errors
  111.             self.full_clean()
File "django/forms/forms.py" in full_clean
  242.                 if hasattr(self, 'clean_%s' % name):

Exception Type: UnicodeEncodeError at /novakoski/modelos/detalhe/1/
Exception Value: ('ascii', u'clean_Di\xe2metro', 8, 9, 'ordinal not in range(128)')

Occurs because hasattr(obj, str) can't handle non-ascii at str, before preparing to call clean_fieldname.

Bug found at 1.1.1 (r11612), but stays alive at actual TRUNK (r12264).

Original traceback paste: ​http://dpaste.com/hold/147583/

Quick fix added.

[Alan Justino da Silva]

Bugfix - use a try-catch to workarround hasattr ASCII limitation

[unbracketed]

Patch doesn't cover all cases. For example, printing a form created with the same conditions (dynamically added field with a non-ASCII name) will also generate an exception.

[Alan Justino da Silva]

Replying to unbracketed:

Patch doesn't cover all cases. For example, printing a form created with the same conditions (dynamically added field with a non-ASCII name) will also generate an exception.

Isn't this because of console encoding issues?. My console is configured to UTF-8, as seen:

alanjds@lgpill:~$ locale
LANG=pt_BR.UTF-8        
LANGUAGE=pt_BR: pt: en_US: en
LC_CTYPE="pt_BR.UTF-8"
(...)
LC_ALL="pt_BR.UTF-8"

When locale config was set as LANG=C, printing from ipython or simple python console throws exceptions yet.

Could you post an example? I would like to try to reproduce this error.

[jkocherhans]

In general I agree that it's not acceptable, but that's partly a limitation of Python itself. You can't use non-ascii values for python variables, so you can't create model fields, or declaratively create form fields with non-ascii values. What you're doing here is essentially a hack to get around that fact. You can still change the label of the field to "Diâmetro" if you want.