Opened 18 years ago

Closed 17 years ago

#1356 closed defect (duplicate)

[patch] database charset and mysql backend

Reported by: little Owned by: Adrian Holovaty
Component: Database layer (models, ORM) Version:
Severity: normal Keywords: mysql charset utf8
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

This is moved over from #1355:

  1. In django.core.db.backends.mysql.DatabaseWrapper in method cursor()

When you doing

            self.connection = Database.connect(**kwargs)
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")

After 'set names ..' the charset of connection was changed.
But self.connection.charset property (property of MySQLdb.connection) was not changed. It is always 'Latin1'
So, when i pass u"unicode national characters" into database, MySQLdb tryed to convert it into Latin1 and exception raises

To fix it, i must set .charset propery manually:

...
            self.connection = Database.connect(**kwargs)
            self.connection.charset = 'utf8'
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")
...

Attachments (2)

mysql.4_1.unicode.diff (831 bytes ) - added by Philipp Keller <philipp.keller@…> 17 years ago.
test.html (40.2 KB ) - added by anonymous 17 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 by hugo, 18 years ago

Component: Admin interfaceDatabase wrapper
Reporter: changed from hugo to little

by Philipp Keller <philipp.keller@…>, 17 years ago

Attachment: mysql.4_1.unicode.diff added

comment:2 by Philipp Keller <philipp.keller@…>, 17 years ago

I ran into the same problem as reported in this ticket.
My posted diff is slightly different than the proposed change. The diff is really small and really shouldn't cause any problems.

comment:3 by Philipp Keller <philipp.keller@…>, 17 years ago

Owner: changed from Adrian Holovaty to anonymous
Status: newassigned
Summary: database charset and mysql backend[patch] database charset and mysql backend

comment:4 by Philipp Keller <philipp.keller@…>, 17 years ago

Owner: changed from anonymous to Adrian Holovaty
Status: assignednew

by anonymous, 17 years ago

Attachment: test.html added

comment:5 by Simon G. <dev@…>, 17 years ago

Keywords: mysql charset utf8 added
Triage Stage: UnreviewedReady for checkin

I've checked and this doesn't appear to conflict with #3151 ([4267]).

comment:6 by anton@…, 17 years ago

Guys, please take a look at #3370
this line:

self.connection.charset = 'utf8'

does not work in my environment (MySQL 5.0), and it only works when I pass 'charset': 'utf8' to kwargs when doing:

self.connection = Database.connect(**kwargs)

comment:7 by Michael Radziej <mir@…>, 17 years ago

Needs tests: set
Patch needs improvement: set
Triage Stage: Ready for checkinDesign decision needed

We have a bit of chaos here ... Tickets #3370, #1356 and probably #952 all are about this problem, all are accepted, and #3370 and #1356 have very similar patches. I ask everybody to continue discussion in django-developers ("unicode issues in multiple tickets"), and I ask the authors of these three tickets to work together to find out how to proceed.

As long as it's not clear which path to take, I mark all bugs as "design decision needed." (I assume that the other reviews were not aware of the competing tickets.)

http://groups.google.com/group/django-developers/browse_thread/thread/4b71be8257d42faf

comment:8 by Malcolm Tredinnick, 17 years ago

I wouldn't be too surprised to discover that #2635 fixes this particular problem and some of the others.

comment:9 by Michael Radziej <mir@…>, 17 years ago

Resolution: duplicate
Status: newclosed

duplicate of #3754 (and fixed). It's still a bug for mysql_old, but mysql_old is deprecated.

Note: See TracTickets for help on using tickets.
Back to Top