Unicode problem loading 500 template using Python 3.3

[adj7388@…]

Environment:
Python 3.3
Django-1.5a1
Windows XP

To reproduce this error, work through the tutorial until you get to "Write your first view" section at: ​https://docs.djangoproject.com/en/1.5/intro/tutorial03/#write-your-first-view

Introduce an intentional error, for example in 'views.py' change:
from django.http import HttpResponse
to
from django.http import HttpResponseFoo

This produces an error as expected, but while processing the error, another exception is generated:

"During handling of the above exception, another exception occurred:"
... <snip> ...
  File "C:\home\alan\venvs\Django-1.5a1\lib\site-packages\django\views\debug.py", line 390, in get_traceback_frames
    pre_context_lineno, pre_context, context_line, post_context = self._get_lines_from_file(filename, lineno, 7, loader, module_name)
  File "C:\home\alan\venvs\Django-1.5a1\lib\site-packages\django\views\debug.py", line 361, in _get_lines_from_file
    match = re.search(br'coding[:=]\s*([-\w.]+)', line)
  File "C:\home\alan\venvs\Django-1.5a1\lib\re.py", line 161, in search
    return _compile(pattern, flags).search(string)
TypeError: can't use a bytes pattern on a string-like object

Line 361 in debug.py is where things start to go wrong:

    match = re.search(br'coding[:=]\s*([-\w.]+)', line)

The 'line' argument is a unicode string (type: str), but the regex is a byte pattern. You can get past this error by encoding line as 'ascii':

    match = re.search(br'coding[:=]\s*([-\w.]+)', line.encode('ascii'))

but then you run into a similar problem a few lines later (debug.py line 365):

    source = [six.text_type(sline, encoding, 'replace') for sline in source]

which produces the error "TypeError: decoding str is not supported" because 'six.text_type' is actually the Python 'str()' function, and the code is telling 'str()' to decode 'sline' using the 'ascii' codec.

Basically the 'source' list is read in from a file that produces unicode, but subsequent code thinks lines from the 'source' list are bytes and tries to apply a byte pattern regex (debug.py lines 361) or tries to decode the unicode (debug.py lines 365).

I would submit a patch if I knew how to fix the underlying problem, but I'm not sure if the fix needs to happen in 'debug.py' or in the loader that gets the 'source' list.

[Claude Paroz]

I think this has been fixed in [3629a159f97b03f1142acdea03bdfa91edecea8d]. Reopen if you can reproduce it in master.

[Aymeric Augustin]

Note that the fix was backported to the 1.5.x branch in [7b6978553aa8cde493f02ddd93bbd711afbf28ae]. You can also test a checkout of that branch.