Opened 3 years ago

Last modified 2 years ago

#25361 new Bug

Unpickling of QuerySet fails in presence of abstract intermediate model

Reported by: Torsten Bronger Owned by: nobody
Component: Database layer (models, ORM) Version: 1.8
Severity: Normal Keywords:
Cc: bronger@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Consider the following model hierarchy:

from django.db import models

class Pizza(models.Model):
    timestamp = models.DateTimeField("last modified", auto_now=True)

class SpecialPizza(Pizza):
    class Meta:
        abstract = True
        ordering = ["timestamp"]

class MyPizza(SpecialPizza):

class Topping(models.Model):
    pizza = models.ForeignKey(MyPizza, related_name="toppings")

    class Meta:
        ordering = ["pizza"]

QuerySet objects containing MyPizza objects with Toppings cannot be pickled and unpickled:

my_pizza = models.MyPizza.objects.create()
pickled = pickle.dumps(my_pizza.toppings.all())
response = pickle.loads(pickled)

The traceback is:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/" in inner
  145.                     return func(*args, **kwargs)
File "/tmp/myproject/myproject/myapp/" in test
  9.     response = pickle.loads(pickled)
File "/usr/lib/python2.7/" in loads
  1383.     return Unpickler(file).load()
File "/usr/lib/python2.7/" in load
  858.                 dispatch[key](self)
File "/usr/lib/python2.7/" in load_reduce
  1134.         value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/" in _load_field
  66.     return apps.get_model(app_label, model_name)._meta.get_field(field_name)
File "/usr/local/lib/python2.7/dist-packages/django/apps/" in get_model
  202.         return self.get_app_config(app_label).get_model(model_name.lower())
File "/usr/local/lib/python2.7/dist-packages/django/apps/" in get_model
  162.                 "App '%s' doesn't have a '%s' model." % (self.label, model_name))

Exception Type: LookupError at /
Exception Value: App 'myapp' doesn't have a 'specialpizza' model.

If one removes one of the "ordering" fields, the unpickler uses "MyPizza" instead of "SpecialPizza", which is correct and should always be the case.

Change History (2)

comment:1 Changed 3 years ago by Simon Charette

Triage Stage: UnreviewedAccepted

I guess we should special case fields of abstract models in Field.__reduce__.

The code added in #19635 could be either adjusted to raise a RuntimeError or return another unpickling function that relies on the __path__ of the abstract model to retrieve the associated class and call _meta.get_field() on it.

comment:2 Changed 2 years ago by Samuel Spencer

I just got bit by this and I can't say I've come across the best solution, but I was able to resolve it with this code in /django/db/models/fields/

---	2016-09-22 06:07:36.167725186 +0000
+++	2016-09-22 06:08:19.175742789 +0000
@@ -66,11 +66,6 @@
     return apps.get_model(app_label, model_name)._meta.get_field(field_name)
-def _load_field_for_abstract(app_label, model_name, field_name):
-    from django.utils.module_loading import import_string
-    return import_string('%s.models.%s'%(app_label, model_name))._meta.get_field(field_name)
 # A guide to Field parameters:
 #   * name:      The name of the field specified in the model.
@@ -508,11 +503,7 @@
             # Deferred model will not be found from the app registry. This
             # could be fixed by reconstructing the deferred model on unpickle.
             raise RuntimeError("Fields of deferred models can't be reduced")
-        if self.model._meta.abstract:
-            func = _load_field_for_abstract
-        else:
-            func = _load_field
-        return func, (self.model._meta.app_label, self.model._meta.object_name,
+        return _load_field, (self.model._meta.app_label, self.model._meta.object_name,
     def get_pk_value_on_save(self, instance): is the original from Django 1.8.14. I'm not sure where to begin with submitting and testing this, but I'd like to help if someone could give me some pointers.

Edit: I've submitted a PR for this:
Edit 2: The Above PR was closed - but if you need to get this working, you can pretty easily override the pickle method *without* having to Monkey Patch! -

Last edited 2 years ago by Samuel Spencer (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top