Opened 5 months ago

Closed 3 months ago

#29246 closed Bug (needsinfo)

compilemessages management-command fails with UnicodeDecodeError

Reported by: Tarun Gaba Owned by: Liuyang Qin
Component: Internationalization Version: 1.11
Severity: Normal Keywords: compilemessages UnicodeDecodeError
Cc: Daniel Browne Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

When running compilemessages command on a django project which contains Unicode characters in filenames (for example in static files), it fails with a UnicodeError:

Traceback (most recent call last):
  File "manage.py", line 62, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/compilemessages.py", line 70, in handle
    for dirpath, dirnames, filenames in os.walk('.', topdown=True):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 23: ordinal not in range(128)

The 'walk' functionality in compilemessages was introduced in django 1.9: (https://github.com/django/django/blob/master/django/core/management/commands/compilemessages.py#L67)

Change History (5)

comment:1 Changed 5 months ago by Tim Graham

Component: UncategorizedInternationalization
Resolution: invalid
Status: newclosed

Django 1.9 is no longer supported. Please reopen if you can reproduce with Django's master branch, which supports only Python 3.

comment:2 Changed 3 months ago by Daniel Browne

Cc: Daniel Browne added
Resolution: invalid
Status: closednew
Version: 1.91.11

I can reproduce this issue on Django 1.11 which is still supported. Can the '.' passed to os.walk not become b'.' in order to bypass the unicode_literals import? I think this would solve the problem.
(https://github.com/django/django/blob/stable/1.11.x/django/core/management/commands/compilemessages.py#L70)

comment:3 Changed 3 months ago by Tim Graham

Per our supported versions policy, 1.11 is only receiving data loss and security fixes. Can the issue be reproduce on Django's master branch (which supports Python 3 only).

comment:4 Changed 3 months ago by Liuyang Qin

Owner: changed from nobody to Liuyang Qin
Status: newassigned

comment:5 Changed 3 months ago by Tim Graham

Resolution: needsinfo
Status: assignedclosed
Note: See TracTickets for help on using tickets.
Back to Top