Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#20212 closed Bug (fixed)

Error pickling SimpleLazyObject

Reported by: Iru Hwang <iru@…> Owned by: daniellindsley
Component: Core (Serialization) Version: 1.5
Severity: Release blocker Keywords:
Cc: iru@…, bmispelon@…, daniellindsley Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Pickling SimpleLazyObject doesn't seem to work.
I've noticed that new classmethod "newobj" is added in 1.5 but seems it prevents the object from pickling.
This affects some cases when request.user needs to be pickled.

>>> from django.utils.functional import SimpleLazyObject
>>> from django.contrib.auth.models import User
>>> u = User.objects.all()[0]
>>> s = SimpleLazyObject(lambda: u)
>>> import pickle
>>> pickle.dumps(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 400, in save_reduce
    save(func)
  File "/usr/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects
>>> 

Attachments (1)

simple_lazy_object.patch (796 bytes) - added by Iru Hwang <iru@…> 3 years ago.

Download all attachments as: .zip

Change History (10)

Changed 3 years ago by Iru Hwang <iru@…>

Attachment: simple_lazy_object.patch added

comment:1 Changed 3 years ago by Iru Hwang <iru@…>

Cc: iru@… added
Has patch: set
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset

comment:2 Changed 3 years ago by Baptiste Mispelon

Cc: bmispelon@… added
Needs tests: set
Patch needs improvement: set
Triage Stage: UnreviewedAccepted

I can reproduce the reported issue. And it is indeed a regression (the code in the report works in 1.4).

I traced the error to the following commit: b430e1db5f26d1de3a5cd0d6eae18603070fdab4

The attached patch is not valid however because it does not work with python3.

Some regression tests will also need to be included.

comment:3 Changed 3 years ago by Iru Hwang <iru@…>

also checked the object is not compatible with cPickle.

>>> u = User.objects.all()[0]
>>> s = SimpleLazyObject(lambda: u)
>>> import cPickle
>>> cPickle.dumps(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

comment:4 Changed 3 years ago by Simon Charette

Severity: NormalRelease blocker

Marking as Release blocker since this has been identifier as a regression from 1.4 to 1.5.

comment:5 Changed 3 years ago by Iru Hwang

in python 2.x, pickle uses protocol 0 if it is not specified. (in 3.3 it uses 3 as default which is defined as pickle.DEFAULT_PROTOCOL)

http://docs.python.org/2/library/pickle.html#data-stream-format

pickle.dumps({'method':method, 'arg':arg}, -1)
    or
pickle.dumps({'method':method, 'arg':arg}, pickle.HIGHEST_PROTOCOL)

This solves the problem so I don't think this is a blocker issue.

comment:6 Changed 3 years ago by Iru Hwang

sorry for make you guys confused ;) can we close this? or is django code still need to be checked?

comment:7 Changed 3 years ago by daniellindsley

Cc: daniellindsley added
Owner: changed from nobody to daniellindsley
Status: newassigned

comment:8 Changed 3 years ago by Daniel Lindsley <daniel@…>

Resolution: fixed
Status: assignedclosed

In e24d486fbc0b1c42abe8b54217ff428e449c48cc:

Fixed #20212 - reduce should only be defined for Py3+.

comment:9 Changed 3 years ago by Daniel Lindsley <daniel@…>

In cb9aaac91fb41b29b3d0d94f3cd208123c02a2ca:

[1.5.x] Fixed #20212 - reduce should only be defined for Py3+.

Note: See TracTickets for help on using tickets.
Back to Top