Code

Opened 6 years ago

Closed 2 years ago

#6073 closed Bug (fixed)

compile-messages.py parse error for utf-8 files with signature

Reported by: Wonlay Owned by: ramiro
Component: Internationalization Version: master
Severity: Normal Keywords: msgfmt, BOM
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If your po files are saved in utf-8 encoding with byte order marks, the compile-messages.py will fails with:
LC_MESSAGES/django.po:1:2: parse error
msgfmt: found 1 fatal error

Maybe we can remove the byte order marks before running msgfmt.

Attachments (2)

compile-messages.py.diff (2.6 KB) - added by Wonlay 6 years ago.
6073-dont-allow-BOM-in-po-files-1.diff (2.1 KB) - added by ramiro 4 years ago.
Patch implementing requirements outlined by Malcom

Download all attachments as: .zip

Change History (15)

Changed 6 years ago by Wonlay

comment:1 Changed 6 years ago by Wonlay

  • Needs documentation unset
  • Needs tests set
  • Patch needs improvement unset
  • Status changed from new to assigned
  • Triage Stage changed from Unreviewed to Accepted

My patch create a temp file for the .po file, if the file has the unicode byte order marks.
And, generate the .mo file from the tmp file.

comment:2 Changed 6 years ago by Wonlay

  • Needs tests unset

comment:3 Changed 6 years ago by Wonlay

  • Keywords msgfmt, BOM added; msgfmt removed

comment:4 Changed 6 years ago by mtredinnick

  • Patch needs improvement set

This patch is massive overkill. PO files for Django must be saved as UTF-8, so that is the only thing you need to worry about.

What broken piece of software is including a BOM when saving in UTF-8 format anyway? It's totally redundant. I'd probably prefer that we just say people should save things without the BOM so that we can just use the tools correctly, rather than having to work around oddities in particular external programs. It's very annoying to have to work around something that just adds extra bytes out the front of a file for precisely no added benefit.

comment:5 Changed 6 years ago by mtredinnick

  • Triage Stage changed from Accepted to Design decision needed

Changed 4 years ago by ramiro

Patch implementing requirements outlined by Malcom

comment:6 Changed 4 years ago by ramiro

  • Owner changed from Wonlay to ramiro
  • Patch needs improvement unset
  • Status changed from assigned to new

I've just uplaoded a patch that implements rejection of .po files that contain a BOM and adds a note documenting that behavior to the relevant section of i18n docs.

comment:7 Changed 4 years ago by ramiro

  • milestone set to 1.2

comment:8 Changed 4 years ago by jezdez

  • Needs tests set

comment:9 Changed 4 years ago by ubernostrum

  • milestone changed from 1.2 to 1.3

UTF-8 with a BOM is, as far as I know, so rare that there's probably no urgent need to deal with it in Django.

comment:10 Changed 4 years ago by ramiro

  • Resolution set to fixed
  • Status changed from new to closed

(In [14126]) [1.2.X] Fixed #6073 -- Made compilemessages 18n management command reject PO files with BOM.

Backport of [14125] from trunk

comment:11 Changed 3 years ago by jacob

  • milestone 1.3 deleted

Milestone 1.3 deleted

comment:12 Changed 2 years ago by anonymous

  • Easy pickings unset
  • Needs documentation set
  • Resolution fixed deleted
  • Severity set to Normal
  • Status changed from closed to reopened
  • Type set to Bug
  • UI/UX unset
  • Version changed from SVN to 1.4

comment:13 Changed 2 years ago by ramiro

  • Resolution set to fixed
  • Status changed from reopened to closed
  • Version changed from 1.4 to SVN

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.