makemessages command fails on windows

[pasha.savchenko@…]

Following the direction for i18n using gettext, I've installed gettext (tried 0.17 and 0.18.X) on several Windows machines and always fail with the same error:

CommandError: errors happened while running xgettext on __init__.py
xgettext: error while opening ".\__init__.py" for reading: No such file or directory

The most obvious suspect is a bad relative path, so I've traced the problem to this area (lines 189-204) in core/management/commands/makemessage.py:

            content = templatize(src_data, orig_file[2:])
            f = open(os.path.join(dirpath, thefile), "w")
            try:
                f.write(content)
            finally:
                f.close()
        work_file = os.path.join(dirpath, thefile)
        cmd = (
            'xgettext -d %s -L Python %s %s --keyword=gettext_noop '
            '--keyword=gettext_lazy --keyword=ngettext_lazy:1,2 '
            '--keyword=ugettext_noop --keyword=ugettext_lazy '
            '--keyword=ungettext_lazy:1,2 --keyword=pgettext:1c,2 '
            '--keyword=npgettext:1c,2,3 --keyword=pgettext_lazy:1c,2 '
            '--keyword=npgettext_lazy:1c,2,3 --from-code UTF-8 '
            '--add-comments=Translators -o - "%s"' %
            (domain, wrap, location, work_file))

The fix was simple, replacing work_file = os.path.join(dirpath, thefile) with work_file = os.path.abspath(os.path.join(dirpath, thefile)) solved the issue.

Please look into making all paths absolute in management commands (especially ones that run external utils). Attached my fixed makemessages.py file.

[pasha.savchenko@…]

fixed makemessages.py

[Karen Tracey]

I cannot recreate this problem. Using this level of gettext retrieved/"installed" as per the docs:

C:\Users\kmtracey\software\web\playground>xgettext --version
xgettext (GNU gettext-tools) 0.17
Copyright (C) 1995-1998, 2000-2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Ulrich Drepper.

makemessages works fine for my test project:

C:\Users\kmtracey\software\web\playground>manage.py makemessages --locale de
processing language de

and produces output:

C:\Users\kmtracey\software\web\playground>dir locale\de\LC_MESSAGES\django.po
 Volume in drive C is Windows7_OS
 Volume Serial Number is EEFA-02D9

 Directory of C:\Users\kmtracey\software\web\playground\locale\de\LC_MESSAGES

07/01/2012  07:39 PM               659 django.po
               1 File(s)            659 bytes
               0 Dir(s)  202,910,855,168 bytes free

(there's only one translatable string in this project).

This is a Windows 7 box. Near as I can tell the command is producing pathnames relative to the current directory, and that is working fine. While it may work also to make paths absolute, I don't understand why that would be necessary. Can you explain anything about your environment and how it differs from what I have tried successfully that might explain why the current code doesn't work in your case?

[pasha.savchenko@…]

After a few more tests I've traced the fault to Popen not being able to run the first file to process, which is given as such:

cmd = 'xgettext -d django -L Python   --keyword=gettext_noop --keyword=gettext_lazy --keyword=ngettext_lazy:1,2'
      ' --keyword=ugettext_noop --keyword=ugettext_lazy --keyword=ungettext_lazy:1,2 --keyword=pgettext:1c,2'
      ' --keyword=npgettext:1c,2,3 --keyword=pgettext_lazy:1c,2 --keyword=npgettext_lazy:1c,2,3 --from-code UTF-8'
      ' --add-comments=Translators -o - ".\\__init__.py"'
p = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE, close_fds=os.name != 'nt', universal_newlines=True)
p.communicate()

For me this command always failed with:
xgettext: error while opening ".\__init__.py" for reading: No such file or directory

Curiously when changing last xgettext argument to - "__init__.py" the command successively parsed the arguments.

So I started digging in the ​subprocess docs and found this statement:

The executable argument specifies the program to execute. It is very seldom needed: Usually, the program to execute is defined by the args argument. If shell=True, the executable argument specifies which shell to use. On Unix, the default shell is /bin/sh. On Windows, the default shell is specified by the COMSPEC environment variable. The only reason you would need to specify shell=True on Windows is where the command you wish to execute is actually built in to the shell, eg dir, copy. You don’t need shell=True to run a batch file, nor to run a console-based executable.

What is interesting is that by default executable=None and we're not passing this argument.
Sure enough, removing shell=True solved the issue and all of my locales were processed fine.

---

By the way:

1. Popen documentation documentation strongly discourages against using shell=True for security reasons (see ​red box in their FAQ)
2. I could not find any mention of why shell=True was used since it's useless without setting the executable arg. Only relevant commit I found was for when ​_popen wrapper was created, unfortunately the original already included shell=True

The attached patch which removes shell=True is attached, if you wish I could submit the pull request on github as well.

[pasha.savchenko@…]

A patch to fix makemessages issue on windows

[Łukasz Rekucki]

I also can't reproduce this problem. Example command passed to Popen:

'''
xgettext -d django -L Python 
--keyword=gettext_noop --keyword=gettex_lazy --keyword=ngettext_lazy:1,2 
--keyword=ugettext_noop --keyword=ugettext_lazy --keyword=ungettext_lazy:1,2 
--keyword=pgettext:1c,2 --keyword=npgettext:1c,2,3 --keyword=pgettext_lazy:1c,2 
--keyword=npgettext_lazy:1c,2,3 --from-code UTF-8 --add-comments=Translators 
-o - ".\\manage.py"
'''

This works fine with or without shell=True (which, btw, isn't a security issue here because Django generates all input passed to it). We don't need to pass executable, it is inferred from the command string.

Unless given more details about the environment and a way to reproduce the problem, there is not much we can do.