Opened 3 years ago

Closed 3 years ago

Last modified 2 years ago

#21584 closed Uncategorized (invalid)

prefetch_related child queryset does not update on create

Reported by: Lucas Wiman Owned by: nobody
Component: Database layer (models, ORM) Version: 1.6
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


When a child foreign key relationship has been prefetched, calling
the .create method on the queryset does not update the queryset.
I've reproduced this bug in Django 1.5.4 and Django 1.6.

How to reproduce:

from django.db import models
class Parent(models.Model):
class Child(models.Model):
    parent = models.ForeignKey(Parent)

In the shell:

>>> p = Parent.objects.create()
>>> list(p.child_set.all())
>>> child = p.child_set.create()
>>> list(p.child_set.all())
[<Child: Child object>]
>>> p2 = Parent.objects.create()
>>> parents = Parent.objects.filter('child_set')
>>> [p2_prefetched] = parents
>>> list(p2_prefetched.child_set.all())
>>> p2_prefetched.child_set.create()
<Child: Child object>
>>> list(p2_prefetched.child_set.all())

The last expression should return a list with one child in it, but returns an empty list instead.

Change History (3)

comment:1 Changed 3 years ago by Luke Plant

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Resolution: invalid
Status: newclosed

This is not a bug, I'm afraid. The results of DB queries that have already been run are never updated automatically by the Django ORM, and that is by design. (There may be some small exceptions, such as when you update a FK object, and the FK ID on that object may be updated as well, but only those that can be done with no dependency tracking). When you specify 'prefetch_related', you are specifying this exact behaviour i.e. the 'child_set.all()' is not lazy, but prefetched in a single query.

Changing this would really require an identity mapper, and a very fundamental change to the way the Django ORM works.

comment:2 Changed 2 years ago by Lucas Wiman

This is not a bug, I'm afraid.

It's very unintuitive that the child_set won't keep track of what elements are in it when it is mutated by its own methods. This seems to break the "iterable" abstraction.

There may be some small exceptions, such as when you update a FK object, and the FK ID on that object may be updated as well, but only those that can be done with no dependency tracking

Isn't that precisely the behavior that's being described here? The foreign key relationship should know that a new entry was created, because the create method was called on the child_set itself. Create should invalidate the cache, because there's no way the cache can remain valid after create is called. Couldn't create set self._results_cache to None, and self.prefetch_done to False when create is called?

comment:3 Changed 2 years ago by Luke Plant

Sorry, that's just how the ORM works. Objects that represent collections do not keep track of their elements, because they represent querysets i.e. queries that may or may not yet have been evaluated, not actual collections of objects. If you have:

    my_objects = Foo.objects.all().filter(bar=1)
    list(my_objects) # evaluate query

then you will find that the 'update' has not affected anything in my_objects - either by changing the instances, or by removing them from the collection (since they no longer match the requirement bar=1).

In the same way, p.child_set does not keep track of elements that are referred to. When you call all(), it executes a query every time, (rather than tracking creates/adds/deletes etc.). If you have used prefetch_related, however, it never executes a query when you just do all() because it has been prefetched. This is exactly what prefetch_related is supposed to do - the all() will not return data to reflect what is in the DB at that moment in time, but what was in the DB when the query was first evaluated.

Note: See TracTickets for help on using tickets.
Back to Top