Django

Code

Ticket #7813 (closed: fixed)

Opened 5 months ago

Last modified 4 months ago

Problem Pickling Querysets

Reported by: Ed Menendez <ed@menendez.com> Assigned to: mtredinnick
Milestone: 1.0 beta Component: Database layer (models, ORM)
Version: SVN Keywords: queryset pickle select_related
Cc: 7944 Triage Stage: Accepted
Has patch: 1 Needs documentation: 0
Needs tests: 1 Patch needs improvement: 1

Description

I saw a previously closed ticket 7506 for a related (no pun intended) problem. It seems like when we use select_related suddenly the queryset can't be pickled. This used to work back in May sometime. Currently using SVN 7944.

The models is attached.

# This works
ld = LeagueDivision.objects.all()
cache.set('test2', ld)

# This doesn't work
ld = LeagueDivision.objects.select_related('league_conference').all()
cache.set('test2', ld)

Traceback (most recent call last):

File "<console>", line 1, in <module> File "C:\Python25\lib\site-packages\django\core\cache\backends

\memcached.py", line 35, in set

self._cache.set(smart_str(key), value, timeout or

self.default_timeout)

File "C:\Projects\pyapps\memcache.py", line 437, in set

return self._set("set", key, val, time, min_compress_len)

File "C:\Projects\pyapps\memcache.py", line 609, in _set

store_info = self._val_to_store_info(val, min_compress_len, key)

File "C:\Projects\pyapps\memcache.py", line 581, in

_val_to_store_info

pickler.dump(val)

File "C:\Python25\lib\copy_reg.py", line 71, in _reduce_ex

raise TypeError?, "can't pickle %s objects" % base.name

TypeError?: can't pickle function objects

The dev environment is Windows, Python 2.5.1 using runserver built in web server with memcache (also on Windows) and Postgres 8.2 on Windows.

- Ed

Attachments

models.py (0.9 kB) - added by Ed Menendez <ed@menendez.com> on 07/18/08 11:58:51.
Example model
clear_related_select_fields.diff (416 bytes) - added by jbronn on 07/18/08 14:48:38.

Change History

07/18/08 11:58:51 changed by Ed Menendez <ed@menendez.com>

  • attachment models.py added.

Example model

07/18/08 14:48:38 changed by jbronn

  • attachment clear_related_select_fields.diff added.

07/18/08 14:50:31 changed by jbronn

  • needs_better_patch changed.
  • component changed from Uncategorized to Database wrapper.
  • needs_tests set to 1.
  • owner changed from nobody to mtredinnick.
  • keywords changed from queryset pickle to queryset pickle select_related.
  • needs_docs changed.
  • has_patch set to 1.

I encountered this bug this past week and forgot to file a ticket. The problem is that when select_related is used, one of the internal Query data structures still contains references to Fields (some of which can't be pickled because of curried functions). This patch blanks out the related_select_fields structure prior to pickling, which should be OK because it's repopulated every time pre_sql_setup is called in as_sql. Malcolm will tell us if I'm wrong :)

07/18/08 18:40:14 changed by Simon Greenhill

  • needs_better_patch set to 1.
  • stage changed from Unreviewed to Accepted.
  • milestone set to 1.0 beta.

07/22/08 03:23:21 changed by mtredinnick

  • status changed from new to closed.
  • resolution set to fixed.

(In [8053]) Fixed #7813 -- Allow pickling of Query classes that use select_related(). Based on a patch from Justin Bronn.

The test in this patch most likely breaks on Oracle. That's another issue.


Add/Change #7813 (Problem Pickling Querysets)




Change Properties
Action