id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
23010	UnicodeDecodeError in makemessages’ call to os.walk	alub	Claude Paroz	"In my project, using Python 2.7.8, if I try the following command:
{{{
./manage.py makemessages -l en
}}}
I get this error:
{{{#!pytb
Traceback (most recent call last):
  File ""./manage.py"", line 10, in <module>
    execute_from_command_line(sys.argv)
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/__init__.py"", line 385, in execute_from_command_line
    utility.execute()
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/__init__.py"", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/base.py"", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/base.py"", line 337, in execute
    output = self.handle(*args, **options)
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/base.py"", line 532, in handle
    return self.handle_noargs(**options)
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/commands/makemessages.py"", line 288, in handle_noargs
    potfiles = self.build_potfiles()
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/commands/makemessages.py"", line 304, in build_potfiles
    file_list = self.find_files(""."")
  File ""/srv/http/webapp/venv/lib/python2.7/site-packages/Django-1.7c1-py2.7.egg/django/core/management/commands/makemessages.py"", line 355, in find_files
    for dirpath, dirnames, filenames in os.walk(root, topdown=True, followlinks=self.symlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 286, in walk
    if isdir(join(top, name)):
  File ""/srv/http/webapp/venv/lib/python2.7/posixpath.py"", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0x96 in position 4: ordinal not in range(128)
}}}
It works in Django 1.6.5, but breaks since 1.7rc1. I think it is related to the commit dbb48d2bb99a5f660cf2d85f137b8d87fc12d99f with the introduction of {{{unicode_literals}}} in this command because of this:
{{{#!pytb
Python 2.7.8 (default, Jul  1 2014, 17:30:21) 
[GCC 4.9.0 20140604 (prerelease)] on linux2
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
>>> import os
>>> _ = list(os.walk('.'))
>>> from __future__ import unicode_literals
>>> _ = list(os.walk('.'))
Traceback (most recent call last):
  File ""<stdin>"", line 1, in <module>
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File ""/srv/http/webapp/venv/lib/python2.7/os.py"", line 286, in walk
    if isdir(join(top, name)):
  File ""/srv/http/webapp/venv/lib/python2.7/posixpath.py"", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0x96 in position 4: ordinal not in range(128)
}}}
So I don’t really know if it’s a Django or Python bug, but it seems like a regression."	Bug	closed	Core (Management commands)	1.7-rc-1	Release blocker	fixed			Ready for checkin	1	0	0	0	0	0