Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#15848 closed Bug (fixed)

Regression in `makessages -d django` when processing multi-line comments with non-ASCII characters

Reported by: ramiro Owned by: nobody
Component: Internationalization Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX:

Description

As reproted Denis Drescher via email:

The fix committed in [15882] causes problems with comments that contain Unicode characters. The error is:

...
File ".../django/utils/translation/trans_real.py", line 450, in templatize
   content = u''.join(comment)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
131: ordinal not in range(128)

To reproduce it, just add a comment block to a template that contains a string such as this one, where the apostrophe is non-ASCII: “It doesn’t work properly on our page.”

To sort of fix it, I removed the three “u”s in lines 450, 457, and 459 of the file, but I read somewhere that the way to go is to use Unicode consistently everywhere, so this solution may not be ideal.

Attachments (1)

15848-tests.diff (753 bytes) - added by ramiro 4 years ago.
Tests for this bug

Download all attachments as: .zip

Change History (5)

Changed 4 years ago by ramiro

Tests for this bug

comment:1 Changed 4 years ago by ramiro

  • Resolution set to fixed
  • Status changed from new to closed

In [16038]:

Fixed #15848 -- Fixed regression introduced in [15882] in makemessages management command when processing multi-line comments that contain non-ASCCI characters in templates. Thanks for the report Denis Drescher.

comment:2 Changed 4 years ago by ramiro

In [16040]:

[1.3.X] Fixed #15848 -- Fixed regression introduced in [15882] in makemessages management command when processing multi-line comments that contain non-ASCCI characters in templates. Thanks for the report Denis Drescher.

Backport of r16038/r16039 from trunk.

comment:3 follow-up: Changed 4 years ago by anonymous

  • Easy pickings unset

I'm using offical 1.3 (pip install django) and still get this error. Should I upgrade to trunk from svn?

  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/makemessages.py", line 365, in handle_noargs
    make_messages(locale, domain, verbosity, process_all, extensions, symlinks, ignore_patterns, no_wrap, no_obsolete)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/makemessages.py", line 233, in make_messages
    f.write(templatize(src, orig_file[2:]))
  File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py", line 127, in templatize
    return _trans.templatize(src, origin)
  File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", line 450, in templatize
    content = u''.join(comment)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

comment:4 in reply to: ↑ 3 Changed 4 years ago by ramiro

Replying to anonymous:

I'm using offical 1.3 (pip install django) and still get this error. Should I upgrade to trunk from svn?

As you can see from the two commit messages before your question, the issue is fixed both in trunk and the releases/1.3.X branch. You can use any of them although I'd suggest the latter because it receives only this kind of bug fixes.

Note: See TracTickets for help on using tickets.
Back to Top