Code

Opened 6 years ago

Closed 5 years ago

Last modified 5 years ago

#9952 closed (wontfix)

to_python being called with unicode text breaks Unpickler

Reported by: mark0978 Owned by: nobody
Component: Database layer (models, ORM) Version: 1.0
Severity: Keywords: CustomFields
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

I have a database column that contains a pickle. When the model tries to access it as a custom field, I run into this problem:

>>> QuizQuestion.objects.get(pk=1)
class= <type 'unicode'>
(I1
I2
I3
I4
tp0
.

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "c:\Python25\Lib\site-packages\django\db\models\manager.py", line 93, in get
    return self.get_query_set().get(*args, **kwargs)
  File "c:\Python25\Lib\site-packages\django\db\models\query.py", line 304, in get
    num = len(clone)
  File "c:\Python25\Lib\site-packages\django\db\models\query.py", line 160, in __len__
    self._result_cache = list(self.iterator())
  File "c:\Python25\Lib\site-packages\django\db\models\query.py", line 280, in iterator
    obj = self.model(*row[index_start:])
  File "c:\Python25\Lib\site-packages\django\db\models\base.py", line 213, in __init__
    setattr(self, field.attname, val)
  File "c:\Python25\Lib\site-packages\django\db\models\fields\subclassing.py", line 32, in __set__
    obj.__dict__[self.field.name] = self.field.to_python(value)
  File "P:\home\mark\Python\intomec\..\intomec\tenq\models.py", line 42, in to_python
    return pickle.loads(value)
  File "c:\Python25\lib\pickle.py", line 1374, in loads
    return Unpickler(file).load()
  File "c:\Python25\lib\pickle.py", line 858, in load
    dispatch[key](self)
KeyError: '\x00'
>>>

the class= part (right before the Traceback) is printed by the class right before the problem in pickle.loadds

Why is the db output converted into unicode before to_python is called? the KeyError is because the Unpickler expects a bytestream, not a unicode stream.....

Attachments (0)

Change History (2)

comment:1 follow-up: Changed 5 years ago by jacob

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to wontfix
  • Status changed from new to closed

I suspect this is because you're putting pickle data into a TextField or CharField, which expects text, not binary. If you're storing pickled data, you'll want to write a custom field (http://docs.djangoproject.com/en/dev/howto/custom-model-fields/).

If you're not using a TextField/CharField, please feel free to reopen.

comment:2 in reply to: ↑ 1 Changed 5 years ago by freeav8r

The following works well for unpickling if you don't want to modify the field:

object = loads(pickledstring.encode('utf-8'))

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.