Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#12924 closed (fixed)

Pickling QuerySet with related fields fails

Reported by: dmishe Owned by: Alex
Component: Database layer (models, ORM) Version: 1.2-beta
Severity: Keywords:
Cc: dmishe@…, sj@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Assume we have two related models:

from django.db import models

class Group(models.Model):
    title = models.CharField(max_length=10)
        
class Event(models.Model):
    group = models.ForeignKey('Group')

Pickling queryset with filtering by group then fails:

>>>pickle.dumps(Event.objects.filter(group=1))
PicklingError: Can't pickle <function _curried at 0x101fdd5f0>: it's not found as django.utils.functional._curried

However, if i specify that i'm filtering on __pk lookup, then it suddenly works:

pickle.dumps(Event.objects.filter(group__pk=1))

Attachments (3)

translation-wtf-party.diff (5.3 KB) - added by Alex 5 years ago.
Initial version, seems to be working, fixes all teh tickets about pickling translatable strings as well.
translation-wtf-party.2.diff (7.0 KB) - added by Alex 5 years ago.
Added some forgotten files (tests), also updated some comments.
translation-wtf-party.3.diff (8.5 KB) - added by Alex 5 years ago.
Killed some, now dead, code.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 5 years ago by dmishe

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Version changed from 1.1 to 1.2-beta

comment:2 Changed 5 years ago by dmishe

Ok found this evil function :)

In [61]: Event.objects.filter(group=1).query.where.children[0].children[0][0].field.related_query_name
Out[61]: <function _curried at 0x101fdd5f0>

comment:3 Changed 5 years ago by russellm

  • milestone set to 1.2
  • Triage Stage changed from Unreviewed to Accepted

comment:4 Changed 5 years ago by russellm

  • Component changed from Uncategorized to Database layer (models, ORM)

comment:5 Changed 5 years ago by Alex

  • Owner changed from nobody to Alex
  • Status changed from new to assigned

Changed 5 years ago by Alex

Initial version, seems to be working, fixes all teh tickets about pickling translatable strings as well.

comment:6 Changed 5 years ago by mrts

Let me point out that there's an important difference in behaviour in of delayed loading in Alex's patch. Namely, as with Alex's, Malcolm's code installs the reference to delayed_loader() as a proxy for every real translation function. But when delayed_loader() runs, it substitutes the references to itself with the actual functions from the imported real translation module in Malcolm's version. So it will be called exactly once and function calls will be fast subsequently.

As of now, Alex's version retains the references so that all calls to translation functions actually pass through delayed_loader(). As

  • calls to translation functions are frequent and
  • imports inside a function make calling it rather slow compared with a simple proxy function call (which is somewhat unexpected -- one would assume that only the first call to import is expensive and others cheap once the module is loaded -- but only timeit will tell as they say :) )

I recommend retaining Malcolm's optimized version. Currying is fine though and makes the code more readable IMHO.

See discussion at http://botland.oebfare.com/logger/django-dev/2010/3/20/3/ .

comment:7 Changed 5 years ago by sjaensch

  • Cc sj@… added

Changed 5 years ago by Alex

Added some forgotten files (tests), also updated some comments.

Changed 5 years ago by Alex

Killed some, now dead, code.

comment:8 Changed 5 years ago by russellm

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [12866]) Fixed #12769, #12924 -- Corrected the pickling of curried and lazy objects, which was preventing queries with translated or related fields from being pickled. And lo, Alex Gaynor didst slayeth the dragon.

comment:13 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top