Code

Opened 11 months ago

Closed 8 months ago

#20990 closed Bug (fixed)

compilemessages fails

Reported by: gregoire.astruc+django@… Owned by: nobody
Component: Internationalization Version: 1.5
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

System: WinXP, django 1.5.2 (through virtual-env), python 2.7.3, using Aptana 3.4.2 shell.

When somebody run the following sequence:

$ ./manage.py makemessages -l en
$ ./manage.py compilemessages

compilemessages fails with the following output:

processing file django.po in d:\Trunk\src\locale\en\LC_MESSAGES
Traceback (most recent call last):
  File "d:\v-env\lib\site-packages\django\core\management\base.py", line 222, in run_from_a
    self.execute(*args, **options.__dict__)
  File "d:\v-env\lib\site-packages\django\core\management\base.py", line 255, in execute
    output = self.handle(*args, **options)
  File "d:\v-env\lib\site-packages\django\core\management\commands\compilemessages.py", lin
    compile_messages(self.stderr, locale=locale)
  File "d:\v-env\lib\site-packages\django\core\management\commands\compilemessages.py", lin
    if f.endswith('.po'):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 1: ordinal not in range(128)

Running ./manage.py compilemessages -l en solves the issue and ./manage.py compilemessages appears to work afterwards.

Attachments (0)

Change History (7)

comment:1 Changed 11 months ago by claudep

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

I've been able to reproduce a similar error in one of my projects.
It might be related to settings.LOCALE_PATH containing non-unicode paths, as os.walk returns unicode results only if it has been fed with an unicode parameter.

Could you test this patch:

--- a/django/core/management/commands/compilemessages.py
+++ b/django/core/management/commands/compilemessages.py
@@ -6,7 +6,7 @@ from optparse import make_option
 
 from django.core.management.base import BaseCommand, CommandError
 from django.core.management.utils import find_command, popen_wrapper
-from django.utils._os import npath
+from django.utils._os import npath, upath
 
 def has_bom(fn):
     with open(fn, 'rb') as f:
@@ -23,7 +23,7 @@ def compile_messages(stdout, locale=None):
     basedirs = [os.path.join('conf', 'locale'), 'locale']
     if os.environ.get('DJANGO_SETTINGS_MODULE'):
         from django.conf import settings
-        basedirs.extend(settings.LOCALE_PATHS)
+        basedirs.extend([upath(path) for path in settings.LOCALE_PATHS])
 
     # Gather existing directories.
     basedirs = set(map(os.path.abspath, filter(os.path.isdir, basedirs)))

comment:2 Changed 11 months ago by gregoire.astruc+django@…

At least it compiles.

But there's a little quirk: it now tries to find every single .po on my whole D: drive, which is quite annoying :)

This :

# Gather existing directories.
basedirs = set(map(os.path.abspath, filter(os.path.isdir, basedirs)))
print basedirs

shows:
set([u'd:\\myproject\\myapp\\src\\locale', u'd:\\'])

Hopefully it helps.

comment:3 Changed 11 months ago by claudep

What's in your settings.LOCALE_PATHS?

comment:4 Changed 11 months ago by gregoire.astruc+django@…

LOCALE_PATHS = (
    os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', 'locale'))
)

My settings is actually a python module, hence the ../.., so it translates to <project_root>/locale.

comment:5 follow-up: Changed 11 months ago by claudep

LOCALE_PATHS has to be a list/tuple. Can you test with that fixed?

comment:6 in reply to: ↑ 5 Changed 8 months ago by furins

Replying to claudep:

LOCALE_PATHS has to be a list/tuple. Can you test with that fixed?

I've tried to replicate the problem (on a Mac OS 10.7, Django 1.5.5, Python 2.7.5 in a virtualenv).

When settings.LOCALE_PATHS is not a tuple (missing the trailing comma), then compilemessages raises another error, but unfortunately it is a little confusing:

example (with LOCALE_PATHS set erroneously to a string instead of a tuple):

$ python2.7 ./manage.py compilemessages --settings=hotspot.settings
processing file django.po in /Users/stefano/sviluppo/siti/hotspot/locale/it_IT/LC_MESSAGES
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 23: ordinal not in range(128)

as you can see, there are no unicode chars in my path (second line) and also in my .po file (I can assure you). Strange enough, the correct path of my .po file has been identified (/Users/stefano/sviluppo/siti/hotspot/locale/it_IT/LC_MESSAGES) thus at a first glance it seems that compilemessages had read the correct path from the settings. Despite that, the only way to solve this error is to correct settings.LOCALE_PATHS, making it a tuple.

If settings.LOCALE_PATHS is a tuple, a list, or a function returning a similar object - and at least one of the path strings contains unicode chars - then I receive the same error of Gregoire, and I can solve it using Claude's patch.

At the very end: it seems to me that Claude's patch solves the issue. Having LOCALE_PATHS set erroneously to a str object generate another type of error.

comment:7 Changed 8 months ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In dffcc5e97988d92b2b8a3bba23a49bcb3cf5d040:

Fixed #20990 -- Ensured unicode paths in compilemessages

Thanks Gregoire Astruc for the report and furins for the review.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.