id summary reporter owner description type status component version severity resolution keywords cc stage has_patch needs_docs needs_tests needs_better_patch easy ui_ux 13758 MySQLdb utf8_bin and django causes UnicodeDecodeError sam.vevang@… nobody "Issue: I have a Model with a FileField. When I delete that instances of that model that have unicode characters in their filenames, I get a {{{ 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128) }}} I finally traced the problem back to my database collation: utf8_bin. I chose utf8_bin so I could order the strings in a case-sensitive manner. FYI, MySQLdb does not return python unicode strings with a utf8_bin collation, it returns utf8 bytestrings. for a brief description of that issue see: http://code.djangoproject.com/ticket/8340#comment:4 The traceback from my exception reveals the exception being thrown in ""django/db/models/fields/files.py"" in get_prep_value (line 248). FileField is a subclass of Field, but implements the same backend MySQL type (varchar) as a CharField. However it seems that FileField and CharField have completely different implementations of get_prep_db. Here is CharField's implementation: {{{ def to_python(self, value): if isinstance(value, basestring) or value is None: return value return smart_unicode(value) def get_prep_value(self, value): return self.to_python(value) }}} Here is Filefield's: {{{ def get_prep_value(self, value): ""Returns field's value prepared for saving into a database."" # Need to convert File objects provided via a form to unicode for database insertion if value is None: return None return unicode(value) }}} My experimentations revealed that if I replace the FileField implementation of get_prep_value with CharField's implementation, the exception goes away. The issue is that the default encoding is ascii and so unicode() called on a utf8 byte str blows up. The CharField implementation simply checks if the value is an instance of basestring and quietly passes it through. " Bug closed Database layer (models, ORM) dev Normal fixed utf8_binMySQLdb collation unicode bytestring Accepted 1 0 1 0 0 0