Opened 11 years ago

Closed 10 years ago

#1356 closed defect (duplicate)

[patch] database charset and mysql backend

Reported by: little Owned by: Adrian Holovaty
Component: Database layer (models, ORM) Version:
Severity: normal Keywords: mysql charset utf8
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

This is moved over from #1355:

  1. In django.core.db.backends.mysql.DatabaseWrapper in method cursor()

When you doing

            self.connection = Database.connect(**kwargs)
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")

After 'set names ..' the charset of connection was changed.
But self.connection.charset property (property of MySQLdb.connection) was not changed. It is always 'Latin1'
So, when i pass u"unicode national characters" into database, MySQLdb tryed to convert it into Latin1 and exception raises

To fix it, i must set .charset propery manually:

...
            self.connection = Database.connect(**kwargs)
            self.connection.charset = 'utf8'
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")
...

Attachments (2)

mysql.4_1.unicode.diff (831 bytes) - added by Philipp Keller <philipp.keller@…> 10 years ago.
test.html (40.2 KB) - added by anonymous 10 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 11 years ago by hugo

Component: Admin interfaceDatabase wrapper
Reporter: changed from hugo to little

Changed 10 years ago by Philipp Keller <philipp.keller@…>

Attachment: mysql.4_1.unicode.diff added

comment:2 Changed 10 years ago by Philipp Keller <philipp.keller@…>

I ran into the same problem as reported in this ticket.
My posted diff is slightly different than the proposed change. The diff is really small and really shouldn't cause any problems.

comment:3 Changed 10 years ago by Philipp Keller <philipp.keller@…>

Owner: changed from Adrian Holovaty to anonymous
Status: newassigned
Summary: database charset and mysql backend[patch] database charset and mysql backend

comment:4 Changed 10 years ago by Philipp Keller <philipp.keller@…>

Owner: changed from anonymous to Adrian Holovaty
Status: assignednew

Changed 10 years ago by anonymous

Attachment: test.html added

comment:5 Changed 10 years ago by Simon G. <dev@…>

Keywords: mysql charset utf8 added
Triage Stage: UnreviewedReady for checkin

I've checked and this doesn't appear to conflict with #3151 ([4267]).

comment:6 Changed 10 years ago by anton@…

Guys, please take a look at #3370
this line:

self.connection.charset = 'utf8'

does not work in my environment (MySQL 5.0), and it only works when I pass 'charset': 'utf8' to kwargs when doing:

self.connection = Database.connect(**kwargs)

comment:7 Changed 10 years ago by Michael Radziej <mir@…>

Needs tests: set
Patch needs improvement: set
Triage Stage: Ready for checkinDesign decision needed

We have a bit of chaos here ... Tickets #3370, #1356 and probably #952 all are about this problem, all are accepted, and #3370 and #1356 have very similar patches. I ask everybody to continue discussion in django-developers ("unicode issues in multiple tickets"), and I ask the authors of these three tickets to work together to find out how to proceed.

As long as it's not clear which path to take, I mark all bugs as "design decision needed." (I assume that the other reviews were not aware of the competing tickets.)

http://groups.google.com/group/django-developers/browse_thread/thread/4b71be8257d42faf

comment:8 Changed 10 years ago by Malcolm Tredinnick

I wouldn't be too surprised to discover that #2635 fixes this particular problem and some of the others.

comment:9 Changed 10 years ago by Michael Radziej <mir@…>

Resolution: duplicate
Status: newclosed

duplicate of #3754 (and fixed). It's still a bug for mysql_old, but mysql_old is deprecated.

Note: See TracTickets for help on using tickets.
Back to Top