Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#20212 closed Bug (fixed)

Error pickling SimpleLazyObject

Reported by: Iru Hwang <iru@…> Owned by: daniellindsley
Component: Core (Serialization) Version: 1.5
Severity: Release blocker Keywords:
Cc: iru@…, bmispelon@…, daniellindsley Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Pickling SimpleLazyObject doesn't seem to work.
I've noticed that new classmethod "newobj" is added in 1.5 but seems it prevents the object from pickling.
This affects some cases when request.user needs to be pickled.

>>> from django.utils.functional import SimpleLazyObject
>>> from django.contrib.auth.models import User
>>> u = User.objects.all()[0]
>>> s = SimpleLazyObject(lambda: u)
>>> import pickle
>>> pickle.dumps(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 400, in save_reduce
    save(func)
  File "/usr/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects
>>> 

Attachments (1)

simple_lazy_object.patch (796 bytes) - added by Iru Hwang <iru@…> 2 years ago.

Download all attachments as: .zip

Change History (10)

Changed 2 years ago by Iru Hwang <iru@…>

comment:1 Changed 2 years ago by Iru Hwang <iru@…>

  • Cc iru@… added
  • Has patch set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 2 years ago by bmispelon

  • Cc bmispelon@… added
  • Needs tests set
  • Patch needs improvement set
  • Triage Stage changed from Unreviewed to Accepted

I can reproduce the reported issue. And it is indeed a regression (the code in the report works in 1.4).

I traced the error to the following commit: b430e1db5f26d1de3a5cd0d6eae18603070fdab4

The attached patch is not valid however because it does not work with python3.

Some regression tests will also need to be included.

comment:3 Changed 2 years ago by Iru Hwang <iru@…>

also checked the object is not compatible with cPickle.

>>> u = User.objects.all()[0]
>>> s = SimpleLazyObject(lambda: u)
>>> import cPickle
>>> cPickle.dumps(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected string or Unicode object, NoneType found

comment:4 Changed 2 years ago by charettes

  • Severity changed from Normal to Release blocker

Marking as Release blocker since this has been identifier as a regression from 1.4 to 1.5.

comment:5 Changed 2 years ago by iru

in python 2.x, pickle uses protocol 0 if it is not specified. (in 3.3 it uses 3 as default which is defined as pickle.DEFAULT_PROTOCOL)

http://docs.python.org/2/library/pickle.html#data-stream-format

pickle.dumps({'method':method, 'arg':arg}, -1)
    or
pickle.dumps({'method':method, 'arg':arg}, pickle.HIGHEST_PROTOCOL)

This solves the problem so I don't think this is a blocker issue.

comment:6 Changed 2 years ago by iru

sorry for make you guys confused ;) can we close this? or is django code still need to be checked?

comment:7 Changed 2 years ago by daniellindsley

  • Cc daniellindsley added
  • Owner changed from nobody to daniellindsley
  • Status changed from new to assigned

comment:8 Changed 2 years ago by Daniel Lindsley <daniel@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In e24d486fbc0b1c42abe8b54217ff428e449c48cc:

Fixed #20212 - reduce should only be defined for Py3+.

comment:9 Changed 2 years ago by Daniel Lindsley <daniel@…>

In cb9aaac91fb41b29b3d0d94f3cd208123c02a2ca:

[1.5.x] Fixed #20212 - reduce should only be defined for Py3+.

Note: See TracTickets for help on using tickets.
Back to Top