Code

Opened 7 months ago

Closed 7 months ago

Last modified 7 months ago

#21102 closed Bug (fixed)

Cannot pickle/unpickle a QuerySet that prefetches related objects more than once (Pickling Idempotency)

Reported by: mjipeo Owned by: Anssi Kääriäinen <anssi.kaariainen@…>
Component: Database layer (models, ORM) Version: 1.5
Severity: Release blocker Keywords: ORM, pickle, cache
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

https://code.djangoproject.com/ticket/20157
https://code.djangoproject.com/ticket/20257
https://github.com/django/django/commit/bac187c0d8e829fb3ca2ca82965eabbcbcb6ddd5#django/db/models/query.py

The patch above introduced a new bug for pickling idempotency. Simply put, a QuerySet that prefetches related objects cannot be pickled and unpickled more than once. The second pickling attempt raises an exception.

With the follow models:

class Post(models.Model):
    post_date = models.DateTimeField(default=datetime.datetime.now)

class Material(models.Model):
    post = models.ForeignKey(Post, related_name='materials')

This code won't run smoothly:

post = Post.objects.create()
posts = Post.objects.all()
posts = posts.prefetch_related('materials')
posts = pickle.loads(pickle.dumps(posts))
posts = pickle.loads(pickle.dumps(posts))

Attachments (0)

Change History (5)

comment:1 Changed 7 months ago by Minjong Chung <mjipeo@…>

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

A new pull request was submitted.

https://github.com/django/django/pull/1627

comment:2 Changed 7 months ago by timo

  • Has patch set
  • Triage Stage changed from Unreviewed to Ready for checkin

bmispelon confirmed this affects 1.5.x only.

comment:3 Changed 7 months ago by Anssi Kääriäinen <anssi.kaariainen@…>

  • Owner set to Anssi Kääriäinen <anssi.kaariainen@…>
  • Resolution set to fixed
  • Status changed from new to closed

In e66fe357b2324f984e91392286b3b0e6b5dd627e:

Fixed #21102 -- pickling a QuerySet with prefetches twice

Fixed the bug that a QuerySet that prefetches related objects cannot be
pickled and unpickled more than once (The second pickling attempt
raises an exception).

Added a new test for the queryset pickling idempotency.

The bug was introduced by
bac187c0d8e829fb3ca2ca82965eabbcbcb6ddd5.

comment:4 Changed 7 months ago by Anssi Kääriäinen <anssi.kaariainen@…>

In 74b91b3888383fca28dea00e0e1ffb5aecec7c0f:

Added tests for double-pickling a QuerySet

Refs #21102.

comment:5 Changed 7 months ago by Anssi Kääriäinen <anssi.kaariainen@…>

In 4c4954a3c133e824390296f6bb444622a8b25331:

[1.6.x] Added tests for double-pickling a QuerySet

Refs #21102.

Backpatch of 74b91b3888383fca28dea00e0e1ffb5aecec7c0f

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.