Opened 16 years ago

Closed 16 years ago

Last modified 12 years ago

#7813 closed (fixed)

Problem Pickling Querysets

Reported by: Ed Menendez <ed@…> Owned by: Malcolm Tredinnick
Component: Database layer (models, ORM) Version: dev
Severity: Keywords: queryset pickle select_related
Cc: 7944 Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I saw a previously closed ticket 7506 for a related (no pun intended) problem. It seems like when we use select_related suddenly the queryset can't be pickled. This used to work back in May sometime. Currently using SVN 7944.

The models is attached.

# This works
ld = LeagueDivision.objects.all()
cache.set('test2', ld)

# This doesn't work
ld = LeagueDivision.objects.select_related('league_conference').all()
cache.set('test2', ld)

Traceback (most recent call last):

File "<console>", line 1, in <module>
File "C:\Python25\lib\site-packages\django\core\cache\backends

\memcached.py", line 35, in set

self._cache.set(smart_str(key), value, timeout or

self.default_timeout)

File "C:\Projects\pyapps\memcache.py", line 437, in set

return self._set("set", key, val, time, min_compress_len)

File "C:\Projects\pyapps\memcache.py", line 609, in _set

store_info = self._val_to_store_info(val, min_compress_len, key)

File "C:\Projects\pyapps\memcache.py", line 581, in

_val_to_store_info

pickler.dump(val)

File "C:\Python25\lib\copy_reg.py", line 71, in _reduce_ex

raise TypeError, "can't pickle %s objects" % base.name

TypeError: can't pickle function objects

The dev environment is Windows, Python 2.5.1 using runserver built in web server with memcache (also on Windows) and Postgres 8.2 on Windows.

  • Ed

Attachments (2)

models.py (923 bytes ) - added by Ed Menendez <ed@…> 16 years ago.
Example model
clear_related_select_fields.diff (416 bytes ) - added by jbronn 16 years ago.

Download all attachments as: .zip

Change History (6)

by Ed Menendez <ed@…>, 16 years ago

Attachment: models.py added

Example model

by jbronn, 16 years ago

comment:1 by jbronn, 16 years ago

Component: UncategorizedDatabase wrapper
Has patch: set
Keywords: select_related added
Needs tests: set
Owner: changed from nobody to Malcolm Tredinnick

I encountered this bug this past week and forgot to file a ticket. The problem is that when select_related is used, one of the internal Query data structures still contains references to Fields (some of which can't be pickled because of curried functions). This patch blanks out the related_select_fields structure prior to pickling, which should be OK because it's repopulated every time pre_sql_setup is called in as_sql. Malcolm will tell us if I'm wrong :)

comment:2 by Simon Greenhill, 16 years ago

milestone: 1.0 beta
Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:3 by Malcolm Tredinnick, 16 years ago

Resolution: fixed
Status: newclosed

(In [8053]) Fixed #7813 -- Allow pickling of Query classes that use select_related().
Based on a patch from Justin Bronn.

The test in this patch most likely breaks on Oracle. That's another issue.

comment:4 by Jacob, 12 years ago

milestone: 1.0 beta

Milestone 1.0 beta deleted

Note: See TracTickets for help on using tickets.
Back to Top