Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#26645 closed Bug (fixed)

Errors when running i18n makemessages tests on Windows

Reported by: Ramiro Morales Owned by: Ramiro Morales
Component: Internationalization Version: 1.10
Severity: Normal Keywords: windows makemessages
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Ramiro Morales)

Follow-up to report makemessages (literal translation extraction functionality) problems made visible after #25677 / fa08d27fb714534670b431fde0cd04a17d637585 but which aren't actually related to the issue reported there and probably nor with the commit either.

Example output whe running tests with Python 2.7 (outcome when using 3.4 and 3.5 is similar).

gettext version is 0.19.6.

Edited to show only errors relevant to this ticket:

======================================================================
ERROR: test_blocktrans_trimmed (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 223, in test_blocktrans_trimmed
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_comments_extractor (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 181, in test_comments_extractor
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_context_in_single_quotes (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 306, in test_context_in_single_quotes
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_extraction_warning (i18n.test_extraction.BasicExtractorTests)
test xgettext warning about multiple bare interpolation placeholders
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 265, in test_extraction_warning
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_special_char_extracted (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 215, in test_special_char_extracted
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_template_comments (i18n.test_extraction.BasicExtractorTests)
Template comment tags on the same line of other constructs (#19552)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 325, in test_template_comments
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_template_message_context_extractor (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 275, in test_template_message_context_extractor
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_unicode_decode_error (i18n.test_extraction.BasicExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 255, in test_unicode_decode_error
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_copy_plural_forms (i18n.test_extraction.CopyPluralFormsExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 575, in test_copy_plural_forms
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_override_plural_forms (i18n.test_extraction.CopyPluralFormsExtractorTests)
Ticket #20311.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 584, in test_override_plural_forms
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\es\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_trans_and_plural_blocktrans_collision (i18n.test_extraction.CopyPluralFormsExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 598, in test_trans_and_plural_blocktrans_collision
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_ignore_directory (i18n.test_extraction.IgnoredExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 491, in test_ignore_directory
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_ignore_file_patterns (i18n.test_extraction.IgnoredExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 507, in test_ignore_file_patterns
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_ignore_subdirectory (i18n.test_extraction.IgnoredExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 500, in test_ignore_subdirectory
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_media_static_dirs_ignored (i18n.test_extraction.IgnoredExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\django\test\utils.py", line 208, in inner
    return func(*args, **kwargs)
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 516, in test_media_static_dirs_ignored
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_default_root_settings (i18n.test_extraction.JavascriptExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\django\test\utils.py", line 208, in inner
    return func(*args, **kwargs)
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 483, in test_default_root_settings
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\djangojs.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_javascript_literals (i18n.test_extraction.JavascriptExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 451, in test_javascript_literals
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\djangojs.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_media_static_dirs_ignored (i18n.test_extraction.JavascriptExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\django\test\utils.py", line 208, in inner
    return func(*args, **kwargs)
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 474, in test_media_static_dirs_ignored
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 69, in _run_makemessages
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\djangojs.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_keep_pot_disabled_by_default (i18n.test_extraction.KeepPotFileExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 682, in test_keep_pot_disabled_by_default
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_keep_pot_enabled (i18n.test_extraction.KeepPotFileExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 694, in test_keep_pot_enabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_keep_pot_explicitly_disabled (i18n.test_extraction.KeepPotFileExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 688, in test_keep_pot_explicitly_disabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_location_comments_for_templatized_files (i18n.test_extraction.LocationCommentsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 658, in test_location_comments_for_templatized_files
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_no_location_disabled (i18n.test_extraction.LocationCommentsTests)
Behavior is correct if --no-location switch isn't specified.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 647, in test_no_location_disabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_no_location_enabled (i18n.test_extraction.LocationCommentsTests)
Behavior is correct if --no-location switch is specified. See #16903.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 640, in test_no_location_enabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_multiple_locales (i18n.test_extraction.MultipleLocaleExtractionTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 715, in test_multiple_locales
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_no_wrap_disabled (i18n.test_extraction.NoWrapExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 623, in test_no_wrap_disabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error



======================================================================
ERROR: test_no_wrap_enabled (i18n.test_extraction.NoWrapExtractorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\django\upstream\tests\i18n\test_extraction.py", line 611, in test_no_wrap_enabled
  File "C:\django\upstream\django\core\management\__init__.py", line 130, in call_command
    return command.execute(*args, **defaults)
  File "C:\django\upstream\django\core\management\base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 343, in handle
    self.write_po_file(potfile, locale)
  File "C:\django\upstream\django\core\management\commands\makemessages.py", line 577, in write_po_file
    "errors happened while running msgmerge\n%s" % errors)
CommandError: errors happened while running msgmerge
C:\django\upstream\tests\i18n\commands\locale\de\LC_MESSAGES\django.po:2:47: syntax error

msgmerge: found 1 fatal error

Change History (8)

comment:1 by Claude Paroz, 9 years ago

Triage Stage: Unreviewed → Accepted

Would be nice to know what's the reported syntax error. With a well-placed breakpoint, you should be able to inspect the po file (and maybe manually run the msgmerge command) to find out the syntax issue.

comment:2 by Ramiro Morales, 9 years ago

Description: modified (diff)
Status: new → assigned

comment:3 by Ramiro Morales, 9 years ago

AFAICS what happens on Windows is that by not passing universal_newlines to subprocess.Popen() as we are doing since fa08d27fb714534670b431fde0cd04a17d637585 the in -memory representation of text content we capture from the standard output of xgettext(1), msgmerge(1), etc. on this platform contains native line ending sequences and not simply \n.

(All examples below are running the i18n.test_extraction.BasicExtractorTests.test_blocktrans_trimmed test case)

Blob of text that reaches django.core.management.commands.makemessages.write_pot_file():

(Pdb) msgs
u'# SOME DESCRIPTIVE TITLE.\r\n# Copyright (C) YEAR THE PACKAGE\'S COPYRIGHT HOLDER\r\n# This file is distributed under the same license as the PACKAGE package.\r\n# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.\r\n#\r\n#, fuzzy\r\nmsgid ""\r\nmsgstr ""\r\n

Note the native \r\n line ending sequences.

Then, the temporary .pot file (test\i18n\test_extraction\commands\locale\django.pot) is written. As we open it in text mode (we can't open it in either binary or universal newlines modes because we are passing the encoding parameter to io.open()) the \n sequences are replaced with \r\n ones resulting in lines separated by \r\r\n, e.g. (output of hd(1) on Linux, django.pot transferred from the Windows system):

$ hd -n 240 django.pot
00000000  23 20 53 4f 4d 45 20 44  45 53 43 52 49 50 54 49  |# SOME DESCRIPTI|
00000010  56 45 20 54 49 54 4c 45  2e 0d 0d 0a 23 20 43 6f  |VE TITLE....# Co|
00000020  70 79 72 69 67 68 74 20  28 43 29 20 59 45 41 52  |pyright (C) YEAR|
00000030  20 54 48 45 20 50 41 43  4b 41 47 45 27 53 20 43  | THE PACKAGE'S C|
00000040  4f 50 59 52 49 47 48 54  20 48 4f 4c 44 45 52 0d  |OPYRIGHT HOLDER.|
00000050  0d 0a 23 20 54 68 69 73  20 66 69 6c 65 20 69 73  |..# This file is|
00000060  20 64 69 73 74 72 69 62  75 74 65 64 20 75 6e 64  | distributed und|
00000070  65 72 20 74 68 65 20 73  61 6d 65 20 6c 69 63 65  |er the same lice|
00000080  6e 73 65 20 61 73 20 74  68 65 20 50 41 43 4b 41  |nse as the PACKA|
00000090  47 45 20 70 61 63 6b 61  67 65 2e 0d 0d 0a 23 20  |GE package....# |
000000a0  46 49 52 53 54 20 41 55  54 48 4f 52 20 3c 45 4d  |FIRST AUTHOR <EM|
000000b0  41 49 4c 40 41 44 44 52  45 53 53 3e 2c 20 59 45  |AIL@ADDRESS>, YE|
000000c0  41 52 2e 0d 0d 0a 23 0d  0d 0a 23 2c 20 66 75 7a  |AR....#...#, fuz|
000000d0  7a 79 0d 0d 0a 6d 73 67  69 64 20 22 22 0d 0d 0a  |zy...msgid ""...|
000000e0  6d 73 67 73 74 72 20 22  22 0d 0d 0a 22 50 72 6f  |msgstr ""..."Pro|

From this stage of the message extraction process onwards:

  1. With each update of the on-disk ancillary temporary POT file additional \r chars are accumulated. In the mentioned test case it ends with \r\r\r\r\n line separators.
  2. As further popen_wrapper calls are performed (i.e. to call msgmerge(1) or msguniq(1)), problem 1 above gets carried to the final .po file(s) and somehow results in multiple \r\n inserted:

This is the final test\i18n\test_extraction\commands\locale\de\LC_MESSAGES\django.po:

$ hd -n 312 django.po
00000000  23 20 53 4f 4d 45 20 44  45 53 43 52 49 50 54 49  |# SOME DESCRIPTI|
00000010  56 45 20 54 49 54 4c 45  2e 0d 0a 22 50 6c 75 72  |VE TITLE..."Plur|
00000020  61 6c 2d 46 6f 72 6d 73  3a 20 6e 70 6c 75 72 61  |al-Forms: nplura|
00000030  6c 73 3d 32 3b 20 70 6c  75 72 61 6c 3d 28 6e 20  |ls=2; plural=(n |
00000040  21 3d 20 31 29 3b 5c 6e  22 0d 0a 0d 0a 0d 0a 0d  |!= 1);\n".......|
00000050  0a 23 20 43 6f 70 79 72  69 67 68 74 20 28 43 29  |.# Copyright (C)|
00000060  20 59 45 41 52 20 54 48  45 20 50 41 43 4b 41 47  | YEAR THE PACKAG|
00000070  45 27 53 20 43 4f 50 59  52 49 47 48 54 20 48 4f  |E'S COPYRIGHT HO|
00000080  4c 44 45 52 0d 0a 0d 0a  0d 0a 0d 0a 23 20 54 68  |LDER........# Th|
00000090  69 73 20 66 69 6c 65 20  69 73 20 64 69 73 74 72  |is file is distr|
000000a0  69 62 75 74 65 64 20 75  6e 64 65 72 20 74 68 65  |ibuted under the|
000000b0  20 73 61 6d 65 20 6c 69  63 65 6e 73 65 20 61 73  | same license as|
000000c0  20 74 68 65 20 50 41 43  4b 41 47 45 20 70 61 63  | the PACKAGE pac|
000000d0  6b 61 67 65 2e 0d 0a 0d  0a 0d 0a 0d 0a 23 20 46  |kage.........# F|
000000e0  49 52 53 54 20 41 55 54  48 4f 52 20 3c 45 4d 41  |IRST AUTHOR <EMA|
000000f0  49 4c 40 41 44 44 52 45  53 53 3e 2c 20 59 45 41  |IL@ADDRESS>, YEA|
00000100  52 2e 0d 0a 0d 0a 0d 0a  0d 0a 23 20 0d 0a 0d 0a  |R.........# ....|
00000110  0d 0a 0d 0a 23 2c 20 66  75 7a 7a 79 0d 0a 0d 0a  |....#, fuzzy....|
00000120  6d 73 67 69 64 20 22 22  0d 0a 0d 0a 6d 73 67 73  |msgid ""....msgs|
00000130  74 72 20 22 22 0d 0a 0d                           |tr ""...|
  1. The newly added newlines confuse copy_plural_forms() (see above) and the code in charge setting of the charset value on the PO header to utf-8. This is what causes the symptom reported by this ticket: django.po:2:47: syntax error -- msgmerge: found 1 fatal error

#25667 was about the inability to handle error output of msgfmt(1) which is actually invoked from compilemessages buf the removal of the gettext_popen_wrapper function (which was the one passing universal_newlines to subprocess.Popen()) introduced this unintended effect.

One possible path would be to re-introduce universal_newlines only for the handling of the gettext tools used by makemessages which, unlike the compilemenssages/msgmft(1), do actually spew POT and PO content in-band; but I'm afraid to do that given the difficulties we've experienced on this front, see #20271, 6fb9dee470d57882e378247fd2706d5f9867b5f9 and 57202a112a966593857725071ecd652a87c157fb for some examples.

I'm working on an alternate approach. Will update this ticket with the link to a PR.

comment:5 by Tim Graham, 9 years ago

Has patch: set
Patch needs improvement: set

comment:6 by Tim Graham, 9 years ago

Patch needs improvement: unset
Triage Stage: Accepted → Ready for checkin
Version: master → 1.10

comment:7 by Ramiro Morales <cramm0@…>, 9 years ago

Resolution: → fixed
Status: assigned → closed

In 6ab0d13:

Fixed #26645 -- Fixed potential makemessages failures on Windows.

This was reflected by 27 i18n test errors (not failures) on such
platform caused by corrupt intermediate PO catalog files fed to
msgmerge(1).

Made it handle in-memory PO/POT file contents with normalized line
separators ('\n').

We need to perform manually this after using subprocess.Popen as we
can't pass it universal_newlines=True.

Also fixes #26670 in the process as this commit refactors the same code
section.

comment:8 by Tim Graham <timograham@…>, 9 years ago

In fc64407:

[1.10.x] Fixed #26645 -- Fixed potential makemessages failures on Windows.

This was reflected by 27 i18n test errors (not failures) on such
platform caused by corrupt intermediate PO catalog files fed to
msgmerge(1).

Made it handle in-memory PO/POT file contents with normalized line
separators ('\n').

We need to perform manually this after using subprocess.Popen as we
can't pass it universal_newlines=True.

Also fixes #26670 in the process as this commit refactors the same code
section.

Backport of 6ab0d1358fc78077064aab88a4fb0a47ca116391 from master

Note: See TracTickets for help on using tickets.
Back to Top