Opened 9 years ago

Closed 5 years ago

#6073 closed Bug (fixed)

compile-messages.py parse error for utf-8 files with signature

Reported by: Wonlay Owned by: Ramiro Morales
Component: Internationalization Version: master
Severity: Normal Keywords: msgfmt, BOM
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If your po files are saved in utf-8 encoding with byte order marks, the compile-messages.py will fails with:
LC_MESSAGES/django.po:1:2: parse error
msgfmt: found 1 fatal error

Maybe we can remove the byte order marks before running msgfmt.

Attachments (2)

compile-messages.py.diff (2.6 KB) - added by Wonlay 9 years ago.
6073-dont-allow-BOM-in-po-files-1.diff (2.1 KB) - added by Ramiro Morales 7 years ago.
Patch implementing requirements outlined by Malcom

Download all attachments as: .zip

Change History (15)

Changed 9 years ago by Wonlay

Attachment: compile-messages.py.diff added

comment:1 Changed 9 years ago by Wonlay

Needs tests: set
Status: newassigned
Triage Stage: UnreviewedAccepted

My patch create a temp file for the .po file, if the file has the unicode byte order marks.
And, generate the .mo file from the tmp file.

comment:2 Changed 9 years ago by Wonlay

Needs tests: unset

comment:3 Changed 9 years ago by Wonlay

Keywords: BOM added

comment:4 Changed 9 years ago by Malcolm Tredinnick

Patch needs improvement: set

This patch is massive overkill. PO files for Django must be saved as UTF-8, so that is the only thing you need to worry about.

What broken piece of software is including a BOM when saving in UTF-8 format anyway? It's totally redundant. I'd probably prefer that we just say people should save things without the BOM so that we can just use the tools correctly, rather than having to work around oddities in particular external programs. It's very annoying to have to work around something that just adds extra bytes out the front of a file for precisely no added benefit.

comment:5 Changed 9 years ago by Malcolm Tredinnick

Triage Stage: AcceptedDesign decision needed

Changed 7 years ago by Ramiro Morales

Patch implementing requirements outlined by Malcom

comment:6 Changed 7 years ago by Ramiro Morales

Owner: changed from Wonlay to Ramiro Morales
Patch needs improvement: unset
Status: assignednew

I've just uplaoded a patch that implements rejection of .po files that contain a BOM and adds a note documenting that behavior to the relevant section of i18n docs.

comment:7 Changed 7 years ago by Ramiro Morales

milestone: 1.2

comment:8 Changed 7 years ago by Jannis Leidel

Needs tests: set

comment:9 Changed 7 years ago by James Bennett

milestone: 1.21.3

UTF-8 with a BOM is, as far as I know, so rare that there's probably no urgent need to deal with it in Django.

comment:10 Changed 6 years ago by Ramiro Morales

Resolution: fixed
Status: newclosed

(In [14126]) [1.2.X] Fixed #6073 -- Made compilemessages 18n management command reject PO files with BOM.

Backport of [14125] from trunk

comment:11 Changed 5 years ago by Jacob

milestone: 1.3

Milestone 1.3 deleted

comment:12 Changed 5 years ago by anonymous

Easy pickings: unset
Needs documentation: set
Resolution: fixed
Severity: Normal
Status: closedreopened
Type: Bug
UI/UX: unset
Version: SVN1.4

comment:13 Changed 5 years ago by Ramiro Morales

Resolution: fixed
Status: reopenedclosed
Version: 1.4SVN
Note: See TracTickets for help on using tickets.
Back to Top