Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#9952 closed (wontfix)

to_python being called with unicode text breaks Unpickler

Reported by: Mark Jones Owned by: nobody
Component: Database layer (models, ORM) Version: 1.0
Severity: Keywords: CustomFields
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


I have a database column that contains a pickle. When the model tries to access it as a custom field, I run into this problem:

>>> QuizQuestion.objects.get(pk=1)
class= <type 'unicode'>

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "c:\Python25\Lib\site-packages\django\db\models\", line 93, in get
    return self.get_query_set().get(*args, **kwargs)
  File "c:\Python25\Lib\site-packages\django\db\models\", line 304, in get
    num = len(clone)
  File "c:\Python25\Lib\site-packages\django\db\models\", line 160, in __len__
    self._result_cache = list(self.iterator())
  File "c:\Python25\Lib\site-packages\django\db\models\", line 280, in iterator
    obj = self.model(*row[index_start:])
  File "c:\Python25\Lib\site-packages\django\db\models\", line 213, in __init__
    setattr(self, field.attname, val)
  File "c:\Python25\Lib\site-packages\django\db\models\fields\", line 32, in __set__
    obj.__dict__[] = self.field.to_python(value)
  File "P:\home\mark\Python\intomec\..\intomec\tenq\", line 42, in to_python
    return pickle.loads(value)
  File "c:\Python25\lib\", line 1374, in loads
    return Unpickler(file).load()
  File "c:\Python25\lib\", line 858, in load
KeyError: '\x00'

the class= part (right before the Traceback) is printed by the class right before the problem in pickle.loadds

Why is the db output converted into unicode before to_python is called? the KeyError is because the Unpickler expects a bytestream, not a unicode stream.....

Change History (2)

comment:1 Changed 9 years ago by Jacob

Resolution: wontfix
Status: newclosed

I suspect this is because you're putting pickle data into a TextField or CharField, which expects text, not binary. If you're storing pickled data, you'll want to write a custom field (

If you're not using a TextField/CharField, please feel free to reopen.

comment:2 in reply to:  1 Changed 9 years ago by freeav8r

The following works well for unpickling if you don't want to modify the field:

object = loads(pickledstring.encode('utf-8'))

Note: See TracTickets for help on using tickets.
Back to Top