Code

Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#2588 closed defect (wontfix)

[patch] Django unicode patches

Reported by: Victor Ng <victor.ng@…> Owned by: adrian
Component: Internationalization Version:
Severity: normal Keywords:
Cc: victor.ng@… Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

This patch supercedes the patches in #2543, #2554.

I've removed the body of encode_output from VariableNode as it's not correct. We should be maintaining unicode strings throughout the Django code and converting to a charset only at the last possible moment.

I've modified django.http.init to encode the content to settings.DEFAULT_CHARSET which so that we never get a case of mixed unicode/non-unicode strings.

Attachments (2)

r3641_unicode.patch (7.3 KB) - added by Victor Ng <victor.ng@…> 8 years ago.
patch against django r3641 to fix various unicode problems
r3645_unicode.patch (8.4 KB) - added by Victor Ng <victor.ng@…> 8 years ago.
additional fix for FormWrapper objects in the context. patched against django r3645

Download all attachments as: .zip

Change History (9)

Changed 8 years ago by Victor Ng <victor.ng@…>

patch against django r3641 to fix various unicode problems

comment:1 Changed 8 years ago by anonymous

  • Cc victor.ng@… added

Changed 8 years ago by Victor Ng <victor.ng@…>

additional fix for FormWrapper objects in the context. patched against django r3645

comment:2 Changed 8 years ago by anonymous

  • Component changed from Admin interface to Internationalization
  • Owner changed from adrian to hugo

comment:3 Changed 8 years ago by Victor Ng <victor.ng@…>

I've got a bunch of unit tests in my app that excercise this stuff to make sure it works, still have to pull them out into a patch against django though.

comment:4 Changed 8 years ago by hugo

  • Owner changed from hugo to adrian

I move this back to adrian, as it patches many core files which aren't really part of the i18n subproject.

comment:5 Changed 7 years ago by anonymous

I tried applying to 0.95 but it failed, even when fixing obvious stuff manually ... i get unicodedecodeerrors (even on completely cleanly created database using syncdb)

Can we expect any progress on this one soon?

comment:6 Changed 7 years ago by Victor Ng <victor.ng@…>

  • Resolution set to wontfix
  • Status changed from new to closed

After wrestling with the Unicode errors in Django for about 6 months now, I've just settled on using a middleware layer to decode the input string using the settings.DEFAULT_CHARSET and then re-encoding the data back to UTF-8.

If you modify the request.POST and the request.GET MultiValueDict instances as well as the request.path, you'll get ASCII clean UTF-8 data guaranteed to flow through Django.

This 'solves' the unicode problem in the existing 0.95 codebase by forcing all internal data to be UTF-8 data. This way you don't have to modify calls of str() to unicode(), you don't have to use the "u" qualifier for string data and you don't have to worry about what any particular database backend is going to do.

In other words, you don't have to touch any other parts of django and as long as you assume that internal data is UTF8 and not just plain ASCII, everything should work fine.

comment:7 Changed 7 years ago by Thomas Güttler <hv@…>

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.