Opened 16 years ago

Closed 12 years ago

#6073 closed Bug (fixed)

compile-messages.py parse error for utf-8 files with signature

Reported by: Wonlay Owned by: Ramiro Morales
Component: Internationalization Version: dev
Severity: Normal Keywords: msgfmt, BOM
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If your po files are saved in utf-8 encoding with byte order marks, the compile-messages.py will fails with:
LC_MESSAGES/django.po:1:2: parse error
msgfmt: found 1 fatal error

Maybe we can remove the byte order marks before running msgfmt.

Attachments (2)

compile-messages.py.diff (2.6 KB ) - added by Wonlay 16 years ago.
6073-dont-allow-BOM-in-po-files-1.diff (2.1 KB ) - added by Ramiro Morales 14 years ago.
Patch implementing requirements outlined by Malcom

Download all attachments as: .zip

Change History (15)

by Wonlay, 16 years ago

Attachment: compile-messages.py.diff added

comment:1 by Wonlay, 16 years ago

Needs tests: set
Status: newassigned
Triage Stage: UnreviewedAccepted

My patch create a temp file for the .po file, if the file has the unicode byte order marks.
And, generate the .mo file from the tmp file.

comment:2 by Wonlay, 16 years ago

Needs tests: unset

comment:3 by Wonlay, 16 years ago

Keywords: BOM added

comment:4 by Malcolm Tredinnick, 16 years ago

Patch needs improvement: set

This patch is massive overkill. PO files for Django must be saved as UTF-8, so that is the only thing you need to worry about.

What broken piece of software is including a BOM when saving in UTF-8 format anyway? It's totally redundant. I'd probably prefer that we just say people should save things without the BOM so that we can just use the tools correctly, rather than having to work around oddities in particular external programs. It's very annoying to have to work around something that just adds extra bytes out the front of a file for precisely no added benefit.

comment:5 by Malcolm Tredinnick, 16 years ago

Triage Stage: AcceptedDesign decision needed

by Ramiro Morales, 14 years ago

Patch implementing requirements outlined by Malcom

comment:6 by Ramiro Morales, 14 years ago

Owner: changed from Wonlay to Ramiro Morales
Patch needs improvement: unset
Status: assignednew

I've just uplaoded a patch that implements rejection of .po files that contain a BOM and adds a note documenting that behavior to the relevant section of i18n docs.

comment:7 by Ramiro Morales, 14 years ago

milestone: 1.2

comment:8 by Jannis Leidel, 14 years ago

Needs tests: set

comment:9 by James Bennett, 14 years ago

milestone: 1.21.3

UTF-8 with a BOM is, as far as I know, so rare that there's probably no urgent need to deal with it in Django.

comment:10 by Ramiro Morales, 13 years ago

Resolution: fixed
Status: newclosed

(In [14126]) [1.2.X] Fixed #6073 -- Made compilemessages 18n management command reject PO files with BOM.

Backport of [14125] from trunk

comment:11 by Jacob, 12 years ago

milestone: 1.3

Milestone 1.3 deleted

comment:12 by anonymous, 12 years ago

Easy pickings: unset
Needs documentation: set
Resolution: fixed
Severity: Normal
Status: closedreopened
Type: Bug
UI/UX: unset
Version: SVN1.4

comment:13 by Ramiro Morales, 12 years ago

Resolution: fixed
Status: reopenedclosed
Version: 1.4SVN
Note: See TracTickets for help on using tickets.
Back to Top