Opened 14 years ago

Closed 14 years ago

Last modified 12 years ago

#12924 closed (fixed)

Pickling QuerySet with related fields fails

Reported by: Dmitry Shevchenko Owned by: Alex Gaynor
Component: Database layer (models, ORM) Version: 1.2-beta
Severity: Keywords:
Cc: dmishe@…, sj@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Assume we have two related models:

from django.db import models

class Group(models.Model):
    title = models.CharField(max_length=10)
        
class Event(models.Model):
    group = models.ForeignKey('Group')

Pickling queryset with filtering by group then fails:

>>>pickle.dumps(Event.objects.filter(group=1))
PicklingError: Can't pickle <function _curried at 0x101fdd5f0>: it's not found as django.utils.functional._curried

However, if i specify that i'm filtering on __pk lookup, then it suddenly works:

pickle.dumps(Event.objects.filter(group__pk=1))

Attachments (3)

translation-wtf-party.diff (5.3 KB ) - added by Alex Gaynor 14 years ago.
Initial version, seems to be working, fixes all teh tickets about pickling translatable strings as well.
translation-wtf-party.2.diff (7.0 KB ) - added by Alex Gaynor 14 years ago.
Added some forgotten files (tests), also updated some comments.
translation-wtf-party.3.diff (8.5 KB ) - added by Alex Gaynor 14 years ago.
Killed some, now dead, code.

Download all attachments as: .zip

Change History (12)

comment:1 by Dmitry Shevchenko, 14 years ago

Version: 1.11.2-beta

comment:2 by Dmitry Shevchenko, 14 years ago

Ok found this evil function :)

In [61]: Event.objects.filter(group=1).query.where.children[0].children[0][0].field.related_query_name
Out[61]: <function _curried at 0x101fdd5f0>

comment:3 by Russell Keith-Magee, 14 years ago

milestone: 1.2
Triage Stage: UnreviewedAccepted

comment:4 by Russell Keith-Magee, 14 years ago

Component: UncategorizedDatabase layer (models, ORM)

comment:5 by Alex Gaynor, 14 years ago

Owner: changed from nobody to Alex Gaynor
Status: newassigned

by Alex Gaynor, 14 years ago

Attachment: translation-wtf-party.diff added

Initial version, seems to be working, fixes all teh tickets about pickling translatable strings as well.

comment:6 by mrts, 14 years ago

Let me point out that there's an important difference in behaviour in of delayed loading in Alex's patch. Namely, as with Alex's, Malcolm's code installs the reference to delayed_loader() as a proxy for every real translation function. But when delayed_loader() runs, it substitutes the references to itself with the actual functions from the imported real translation module in Malcolm's version. So it will be called exactly once and function calls will be fast subsequently.

As of now, Alex's version retains the references so that all calls to translation functions actually pass through delayed_loader(). As

  • calls to translation functions are frequent and
  • imports inside a function make calling it rather slow compared with a simple proxy function call (which is somewhat unexpected -- one would assume that only the first call to import is expensive and others cheap once the module is loaded -- but only timeit will tell as they say :) )

I recommend retaining Malcolm's optimized version. Currying is fine though and makes the code more readable IMHO.

See discussion at http://botland.oebfare.com/logger/django-dev/2010/3/20/3/ .

comment:7 by Stephan Jaensch, 14 years ago

Cc: sj@… added

by Alex Gaynor, 14 years ago

Added some forgotten files (tests), also updated some comments.

by Alex Gaynor, 14 years ago

Killed some, now dead, code.

comment:8 by Russell Keith-Magee, 14 years ago

Resolution: fixed
Status: assignedclosed

(In [12866]) Fixed #12769, #12924 -- Corrected the pickling of curried and lazy objects, which was preventing queries with translated or related fields from being pickled. And lo, Alex Gaynor didst slayeth the dragon.

comment:13 by Jacob, 12 years ago

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top