Can't pickle queryset

[Paul Garner]

Was working ok until we upgraded from 1.1.1 to 1.2-beta-1 this morning.
Started getting the "can't pickle function object" error from this code:

class RatingDimensionManager(models.Manager):
    use_for_related_fields = True
    
    def get_for(self, obj):
        ct = ContentType.objects.get_for_model(obj)# cached by contentypes framework
        def from_db(obj):
            result = self.filter(content_type=ct.pk)
            cache.set('ratings:dimensions_for_type:%s:%s' % (hash(str(self.all().query)), ct.pk), result, settings.RATINGS_DIMENSIONS_FOR_CT_TTL)
            return result
        value = cache.get('ratings:dimensions_for_type:%s:%s' % (hash(str(self.all().query)), ct.pk), None)
        if value is None:
            value = from_db(obj)
        return value

class RatingDimension(models.Model):
    content_type = models.ForeignKey(ContentType)
    label = models.CharField(max_length=256)
    sort_weight = models.IntegerField(default=100)
    average = models.FloatField(editable=False, default=0)
    
    objects = RatingDimensionManager()
    
    def __unicode__(self):
        return self.label

    class Meta():
        ordering = ('sort_weight','label',)

Seems related to ​http://code.djangoproject.com/ticket/7506

[Russell Keith-Magee]

Works for me:

>>> import pickle
>>> pickle.dumps(RatingDimension.objects.all())
ccopy_reg
_reconstructor
p0
(cdjango.db.models.query
QuerySet
p1
...

Two suggestions for providing bug reports:

1. Provide an example of the code that is failing. "Pickling" is a very vague operation, and as I've demonstrated, it does work under some circumstances. You need to give specific instructions for how to generate a failure. Remember that Django has a fairly extensive test suite, and there are lots of pickling tests. If there is a problem, it's more than likely going to be in edge case usage.
2. An example is good, but a *minimal* example is better. Is the RatingDimensionManager necessary to stimulate this problem? What about the Meta? The unicode call? I don't doubt that these are necessary for your code, but they don't assist the debugging process.

Looking at your models, there's a good chance that this is a duplicate of #12924. I'm going to close this ticket worksforme (because it does). If it turns out that this isn't a duplicate of #12924, please reopen with a minimal example and a sample of how to induce the error.

[noisyboiler]

Does not work for me with memcached

[Paul Garner]

Thanks noisyboiler. It was the cache.set() call that was failing for me, and I am using memcached too.

However Russell is correct I believe, if I change this line:

result = self.filter(content_type=ct.pk)

to

result = self.filter(content_type__pk=ct.pk)

(as per #12924) then it starts working again. And also, that second form seems more correct anyway. Though as I said, it used to work in 1.1.1

[noisyboiler]

Doesn't Pickle:
cars = Car.objects.filter(advertcarisnull=False)
Does Pickle:
cars = Car.objects.filter(advertcarpkisnull=False)

Both yield same results. Thank You Russell and anentropic.

[Ash Christopher]

This isn't really a solution to the problem, or if it is a solution to the problem then the documentation on QuerySets is wrong. Either way, I don't believe this is solved.

According to documentation, this it valid:

the_region = TripRegion.objects.all()[0]
guides = DestinationGuide.objects.active().filter(region=the_region)

And sure, it is a valid way to perform a QuerySet lookup, however in order to make use of say caching which needs to pickle the query set we shouldn't have to change the QS lookup to:

guides = estinationGuide.objects.active().filter(regionpk=the_region.pk)

[Karen Tracey]

PLEASE use preview (and WikiFormatting). Most of the code posted in the previous few updates is completely indecipherable.

[Ash Christopher]

Fail.

*sigh*

This isn't really a solution to the problem, or if it is a solution to the problem then the documentation on Query Sets is incomplete. Either way, I don't believe this is solved.

    the_region = TripRegion.objects.all()[0]
    guides = DestinationGuide.objects.active().filter(region=the_region)   

This statement is valid, but will not work when pickled

    pickle.dumps(guides)

    TypeError::can't pickle function objects

In order to get it to work, you need to change it to:

    the_region = TripRegion.objects.all()[0]
    guides = DestinationGuide.objects.active().filter(region__pk=the_region.pk)   

I specifically get this problem when trying to cache query set results.

It seems weird that I would have to change the way I do a lookup depending on whether I needed to cache the results or not.

[Russell Keith-Magee]

I can't reproduce the problem reported by @ashchristopher on r12974. I used the following test in the queryset_pickle regression test:

    def test_related_field_by_query(self):
        Group.objects.create(name="Ponies Who Own Maybachs")
        g = Group.objects.all()[0]
        self.assert_pickles(Event.objects.filter(group=g))