Opened 9 years ago

Closed 8 years ago

#1356 closed defect (duplicate)

[patch] database charset and mysql backend

Reported by: little Owned by: adrian
Component: Database layer (models, ORM) Version:
Severity: normal Keywords: mysql charset utf8
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

This is moved over from #1355:

  1. In django.core.db.backends.mysql.DatabaseWrapper in method cursor()

When you doing

            self.connection = Database.connect(**kwargs)
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")

After 'set names ..' the charset of connection was changed.
But self.connection.charset property (property of MySQLdb.connection) was not changed. It is always 'Latin1'
So, when i pass u"unicode national characters" into database, MySQLdb tryed to convert it into Latin1 and exception raises

To fix it, i must set .charset propery manually:

...
            self.connection = Database.connect(**kwargs)
            self.connection.charset = 'utf8'
        cursor = self.connection.cursor()
        if self.connection.get_server_info() >= '4.1':
            cursor.execute("SET NAMES 'utf8'")
...

Attachments (2)

mysql.4_1.unicode.diff (831 bytes) - added by Philipp Keller <philipp.keller@…> 8 years ago.
test.html (40.2 KB) - added by anonymous 8 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 9 years ago by hugo

  • Component changed from Admin interface to Database wrapper
  • Reporter changed from hugo to little

Changed 8 years ago by Philipp Keller <philipp.keller@…>

comment:2 Changed 8 years ago by Philipp Keller <philipp.keller@…>

I ran into the same problem as reported in this ticket.
My posted diff is slightly different than the proposed change. The diff is really small and really shouldn't cause any problems.

comment:3 Changed 8 years ago by Philipp Keller <philipp.keller@…>

  • Owner changed from adrian to anonymous
  • Status changed from new to assigned
  • Summary changed from database charset and mysql backend to [patch] database charset and mysql backend

comment:4 Changed 8 years ago by Philipp Keller <philipp.keller@…>

  • Owner changed from anonymous to adrian
  • Status changed from assigned to new

Changed 8 years ago by anonymous

comment:5 Changed 8 years ago by Simon G. <dev@…>

  • Keywords mysql charset utf8 added
  • Triage Stage changed from Unreviewed to Ready for checkin

I've checked and this doesn't appear to conflict with #3151 ([4267]).

comment:6 Changed 8 years ago by anton@…

Guys, please take a look at #3370
this line:

self.connection.charset = 'utf8'

does not work in my environment (MySQL 5.0), and it only works when I pass 'charset': 'utf8' to kwargs when doing:

self.connection = Database.connect(**kwargs)

comment:7 Changed 8 years ago by Michael Radziej <mir@…>

  • Needs tests set
  • Patch needs improvement set
  • Triage Stage changed from Ready for checkin to Design decision needed

We have a bit of chaos here ... Tickets #3370, #1356 and probably #952 all are about this problem, all are accepted, and #3370 and #1356 have very similar patches. I ask everybody to continue discussion in django-developers ("unicode issues in multiple tickets"), and I ask the authors of these three tickets to work together to find out how to proceed.

As long as it's not clear which path to take, I mark all bugs as "design decision needed." (I assume that the other reviews were not aware of the competing tickets.)

http://groups.google.com/group/django-developers/browse_thread/thread/4b71be8257d42faf

comment:8 Changed 8 years ago by mtredinnick

I wouldn't be too surprised to discover that #2635 fixes this particular problem and some of the others.

comment:9 Changed 8 years ago by Michael Radziej <mir@…>

  • Resolution set to duplicate
  • Status changed from new to closed

duplicate of #3754 (and fixed). It's still a bug for mysql_old, but mysql_old is deprecated.

Note: See TracTickets for help on using tickets.
Back to Top