compilemessages management-command fails with UnicodeDecodeError

[Tarun Gaba]

When running compilemessages command on a django project which contains Unicode characters in filenames (for example in static files), it fails with a UnicodeError:

Traceback (most recent call last):
  File "manage.py", line 62, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/compilemessages.py", line 70, in handle
    for dirpath, dirnames, filenames in os.walk('.', topdown=True):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 23: ordinal not in range(128)

The 'walk' functionality in compilemessages was introduced in django 1.9: (​https://github.com/django/django/blob/master/django/core/management/commands/compilemessages.py#L67)

[Tim Graham]

Django 1.9 is no longer supported. Please reopen if you can reproduce with Django's master branch, which supports only Python 3.

[Daniel Browne]

I can reproduce this issue on Django 1.11 which is still supported. Can the '.' passed to os.walk not become b'.' in order to bypass the unicode_literals import? I think this would solve the problem.
(​https://github.com/django/django/blob/stable/1.11.x/django/core/management/commands/compilemessages.py#L70)

[Tim Graham]

Per our ​supported versions policy, 1.11 is only receiving data loss and security fixes. Can the issue be reproduce on Django's master branch (which supports Python 3 only).

[Sebastien Renard]

Got the same bug on django 1.11. One simple way to workaround this is to cd to your locale/ directory before calling compilemessages. Btw that also speed up things, avoiding to search locale folders everywhere.