id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
5875	Non-ascii chars in source code causing UnicodeEncodeError for default 500 error handler, resulting in non-descriptive traceback-only error page	tt@…	nobody	"Project developed for pre-unicode django version, not utilizing i18n and using national chars in source with iso-8859-1 encoding.

Django configured for serving with DEFAULT_CHARSET and FILE_CHARSET set to ""iso-8859-1"".

Django upgraded to current SVN, and the mod_apache error handler appears, giving !UnicodeEncodeError deep in Django.

{{{
Traceback (most recent call last):

  File ""/usr/lib/python2.3/site-packages/django/core/servers/basehttp.py"", line 279, in run
    self.finish_response()

  File ""/usr/lib/python2.3/site-packages/django/core/servers/basehttp.py"", line 317, in finish_response
    for data in self.result:

  File ""/usr/lib/python2.3/site-packages/django/http/__init__.py"", line 332, in next
    chunk = chunk.encode(self._charset)

UnicodeEncodeError: 'latin-1' codec can't encode characters in position 5735-5737: ordinal not in range(256)
}}}

The core problem seems to be chars from the non-allowed range enter the unicode-only path for django, for example in the case of __str__ model methods not returning unicode. 

That is obviously a problem in itself, but should not trip Django up this badly.

Other examples have also beed observed, but all relate to database or form data being touched by non-unicode strings.


Catching !UnicodeEncodeError and using replace mechanism at least enables developer to find offending chars and convert code to unicode.
I don't know if this patch does the right thing, but it gives a workable error message.


See attached patch for details."		closed	Core (Other)	dev		wontfix			Accepted	0	0	0	0	0	0