Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#21102 closed Bug (fixed)

Cannot pickle/unpickle a QuerySet that prefetches related objects more than once (Pickling Idempotency)

Reported by: Minjong Chung Owned by: Anssi Kääriäinen <anssi.kaariainen@…>
Component: Database layer (models, ORM) Version: 1.5
Severity: Release blocker Keywords: ORM, pickle, cache
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

https://code.djangoproject.com/ticket/20157
https://code.djangoproject.com/ticket/20257
https://github.com/django/django/commit/bac187c0d8e829fb3ca2ca82965eabbcbcb6ddd5#django/db/models/query.py

The patch above introduced a new bug for pickling idempotency. Simply put, a QuerySet that prefetches related objects cannot be pickled and unpickled more than once. The second pickling attempt raises an exception.

With the follow models:

class Post(models.Model):
    post_date = models.DateTimeField(default=datetime.datetime.now)

class Material(models.Model):
    post = models.ForeignKey(Post, related_name='materials')

This code won't run smoothly:

post = Post.objects.create()
posts = Post.objects.all()
posts = posts.prefetch_related('materials')
posts = pickle.loads(pickle.dumps(posts))
posts = pickle.loads(pickle.dumps(posts))

Change History (5)

comment:1 by Minjong Chung <mjipeo@…>, 11 years ago

A new pull request was submitted.

https://github.com/django/django/pull/1627

comment:2 by Tim Graham, 11 years ago

Has patch: set
Triage Stage: UnreviewedReady for checkin

bmispelon confirmed this affects 1.5.x only.

comment:3 by Anssi Kääriäinen <anssi.kaariainen@…>, 11 years ago

Owner: set to Anssi Kääriäinen <anssi.kaariainen@…>
Resolution: fixed
Status: newclosed

In e66fe357b2324f984e91392286b3b0e6b5dd627e:

Fixed #21102 -- pickling a QuerySet with prefetches twice

Fixed the bug that a QuerySet that prefetches related objects cannot be
pickled and unpickled more than once (The second pickling attempt
raises an exception).

Added a new test for the queryset pickling idempotency.

The bug was introduced by
bac187c0d8e829fb3ca2ca82965eabbcbcb6ddd5.

comment:4 by Anssi Kääriäinen <anssi.kaariainen@…>, 11 years ago

In 74b91b3888383fca28dea00e0e1ffb5aecec7c0f:

Added tests for double-pickling a QuerySet

Refs #21102.

comment:5 by Anssi Kääriäinen <anssi.kaariainen@…>, 11 years ago

In 4c4954a3c133e824390296f6bb444622a8b25331:

[1.6.x] Added tests for double-pickling a QuerySet

Refs #21102.

Backpatch of 74b91b3888383fca28dea00e0e1ffb5aecec7c0f

Note: See TracTickets for help on using tickets.
Back to Top