Opened 9 years ago
Closed 9 years ago
#25426 closed Bug (duplicate)
pickling SimpleLazyObject fails just after accessing related object of wrapped model instance.
Reported by: | Iru Hwang | Owned by: | Iru Hwang |
---|---|---|---|
Component: | Core (Serialization) | Version: | 1.8 |
Severity: | Release blocker | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Reproduction step :
import pickle import django django.setup() from django.contrib.auth.models import User from django.utils.functional import SimpleLazyObject u = User.objects.select_related('profile').get(id=153) o = SimpleLazyObject(lambda :u) pickle.dumps(o) # FAILS Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/pickle.py", line 1374, in dumps Pickler(file, protocol).dump(obj) File "/usr/local/lib/python2.7/pickle.py", line 224, in dump self.save(obj) File "/usr/local/lib/python2.7/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/usr/local/lib/python2.7/pickle.py", line 419, in save_reduce save(state) File "/usr/local/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/local/lib/python2.7/pickle.py", line 649, in save_dict self._batch_setitems(obj.iteritems()) File "/usr/local/lib/python2.7/pickle.py", line 661, in _batch_setitems for k, v in items: RuntimeError: dictionary changed size during iteration pickle.dumps(o) # SUCCEEDS
This also fails :
u = User.objects.get(id=153) o = SimpleLazyObject(lambda :u) o.profile pickle.dumps(o)
This looks to be related with change in Model.reduce in django 1.8
adding DJANGO_VERSION_PICKLE_KEY in dict
def __reduce__(self): """ Provides pickling support. Normally, this just dispatches to Python's standard handling. However, for models with deferred field loading, we need to do things manually, as they're dynamically created classes and only module-level classes can be pickled by the default path. """ data = self.__dict__ data[DJANGO_VERSION_PICKLE_KEY] = get_version() if not self._deferred: class_id = self._meta.app_label, self._meta.object_name return model_unpickle, (class_id, [], simple_class_factory), data defers = [] for field in self._meta.fields: if isinstance(self.__class__.__dict__.get(field.attname), DeferredAttribute): defers.append(field.attname) model = self._meta.proxy_for_model class_id = model._meta.app_label, model._meta.object_name return (model_unpickle, (class_id, defers, deferred_class_factory), data)
Attachments (1)
Change History (12)
comment:1 by , 9 years ago
Description: | modified (diff) |
---|
comment:2 by , 9 years ago
Description: | modified (diff) |
---|
comment:3 by , 9 years ago
Severity: | Normal → Release blocker |
---|
comment:5 by , 9 years ago
This happens always when cPickle is used. Does not happen when proto >=2 with python implementation of pickle.
comment:6 by , 9 years ago
Has patch: | set |
---|
by , 9 years ago
Attachment: | patch_25426.diff added |
---|
comment:7 by , 9 years ago
the patch is for saving django version info as unpickler's argument instead of setting it as model instance's attribute which results in RuntimeError depending on pickling protocol.
comment:8 by , 9 years ago
added test for this issue and updated existing one
https://github.com/django/django/compare/master...knifenomad:ticket_25426
comment:9 by , 9 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:10 by , 9 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:11 by , 9 years ago
Resolution: | → duplicate |
---|---|
Status: | assigned → closed |
closing since this duplicates https://code.djangoproject.com/ticket/25389
This is how this happens :
SimpleLazyObject.__reduce_ex__ #instancemethod
)Model.__reduce__ #instancemethod
)Model.__reduce__ #instancemethod
)