Opened 59 minutes ago
#37447 assigned Bug
makemessages fails in UTF-8 mode when processing non-UTF-8 filenames
| Reported by: | Jacob Walls | Owned by: | Jacob Walls |
|---|---|---|---|
| Component: | Internationalization | Version: | 6.1 |
| Severity: | Release blocker | Keywords: | makemessages, locale, utf-8 |
| Cc: | Sarah Boyce | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Surfaced on CI just now after bumping Windows jobs to Python 3.15, now that UTF-8 mode is the default.
ERROR: test_unicode_file_name (i18n.test_extraction.BasicExtractorTests.test_unicode_file_name) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.15.1\x64\Lib\unittest\case.py", line 58, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.15.1\x64\Lib\unittest\case.py", line 691, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.15.1\x64\Lib\unittest\case.py", line 637, in _callTestMethod result = method() ^^^^^^^^^^^^^^^ File "D:\a\django\django\tests\i18n\test_extraction.py", line 372, in test_unicode_file_name management.call_command("makemessages", locale=[LOCALE], verbosity=0) ^^^^^^^^^^^^^^^ File "D:\a\django\django\django\core\management\__init__.py", line 195, in call_command return command.execute(*args, **defaults) ^^^^^^^^^^^^^^^^^ File "D:\a\django\django\django\core\management\base.py", line 471, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^ File "D:\a\django\django\django\core\management\commands\makemessages.py", line 423, in handle potfiles = self.build_potfiles() ^^^^^^^^^^^ File "D:\a\django\django\django\core\management\commands\makemessages.py", line 504, in build_potfiles self.process_files(file_list) ^^^^^^^^^^^^^^^ File "D:\a\django\django\django\core\management\commands\makemessages.py", line 596, in process_files self.process_locale_dir(locale_dir, files) ^^^^^^^^^^^ File "D:\a\django\django\django\core\management\commands\makemessages.py", line 677, in process_locale_dir raise CommandError( ^^^^^^^ django.core.management.base.CommandError: errors happened while running xgettext on .\__init__.py .\ignore_dir\ignored.html.py .\media_root\media_ignored.html.py .\static\static_ignored.html.py .\templates\empty.html.py .\templates\subdir\ignored.html.py .\templates\test.html.py .\templates\xxx_ignored.html.py .\vidéo.txt.py xgettext: error while opening ".\vidéo.txt.py" for reading: No such file or directory
I believe the fix is to write using the system locale, which we should have been doing all along (linter rule on this topic)
-
django/core/management/commands/makemessages.py
diff --git a/django/core/management/commands/makemessages.py b/django/core/management/commands/makemessages.py index 5a4bd55bb8..9db5a6ef2a 100644
a b class Command(BaseCommand): 663 663 return 664 664 665 665 input_files = [bf.work_path for bf in build_files] 666 with NamedTemporaryFile(mode="w+") as input_files_list: 667 input_files_list.write("\n".join(input_files)) 666 with NamedTemporaryFile(mode="w+b") as input_files_list: 667 input_files_list.write( 668 "\n".join(input_files).encode(DEFAULT_LOCALE_ENCODING) 669 ) 668 670 input_files_list.flush() 669 671 args.extend(["--files-from", input_files_list.name]) 670 672 args.extend(self.xgettext_options)
If a triager confirms the approach, I can open the PR. I propose backporting to 6.1 as part of 3.15 support, but with a release note, since it affects released behavior and would have been visible on prior Python versions when running in UTF-8 mode.
Note:
See TracTickets
for help on using tickets.