#30727 closed Bug (fixed)
Pickling a QuerySet evaluates the querysets given to Subquery in annotate.
Reported by: | Andrew Brown | Owned by: | Andrew Brown |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
I wrote a test case for tests/queryset_pickle/tests.py
modeled after the test from bug #27499 which is very similar.
def test_pickle_subquery_queryset_not_evaluated(self): """ Verifies that querysets passed into Subquery expressions are not evaluated when pickled """ groups = Group.objects.annotate( has_event=models.Exists(Event.objects.filter(group_id=models.OuterRef('id'))) ) with self.assertNumQueries(0): pickle.loads(pickle.dumps(groups.query))
The Subquery class (which is the base for Exists
) only stores the underlying query object and throws the QuerySet away (as of this commit, although I don't think it worked before that). So in theory it shouldn't be pickling the QuerySet.
However, the QuerySet is still stored on the instance within the _constructor_args
attribute added by the @deconstructible
decorator on the BaseExpression
base class. So when the inner query object gets pickled, so does the QuerySet, which attempts to evaluate it. In this case, it gets the error "ValueError: This queryset contains a reference to an outer query and may only be used in a subquery." since the inner queryset is being evaluated independently when called from pickle: it calls QuerySet.__getstate__()
.
I'm not sure what the best solution is here. I made a patch that adds the following override to __getstate__
to the SubQuery class, which fixes the problem and passes all other Django tests on my machine. I'll submit a PR shortly, but welcome any better approaches since I'm not sure what else that may effect.
class Subquery(Expression): ... def __getstate__(self): obj_dict = super().__getstate__() obj_dict.pop('_constructor_args', None) return obj_dict
Change History (8)
comment:1 by , 5 years ago
Has patch: | set |
---|
comment:2 by , 5 years ago
Type: | Uncategorized → Bug |
---|
comment:3 by , 5 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
Summary: | Pickling a Query object evaluates querysets in Subquery expressions → Pickling a QuerySet evaluates the querysets given to Subquery in annotate. |
Triage Stage: | Unreviewed → Accepted |
Version: | 2.2 → master |
PR: https://github.com/django/django/pull/11707