Opened 6 years ago

Last modified 3 years ago

#22757 assigned Cleanup/optimization

prefetch_related isn't as effecient as it could be with GenericForeignKey and proxy models

Reported by: gwahl@… Owned by: Paulo
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: prefetch_related
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


prefetch_related is able to prefetch across a GenericForiegnKey. However, it does more queries than necessary when using proxy models.

Our models:

from django.db import models
from django.contrib.contenttypes.generic import GenericForeignKey
from django.contrib.contenttypes.models import ContentType

class Parent(models.Model):
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    child = GenericForeignKey('content_type', 'object_id', for_concrete_model=False)

class Child(models.Model):

class ProxiedChild(Child):
    class Meta:
        proxy = True

Now create some instances

>>> child = Child.objects.create()
>>> proxied_child = ProxiedChild.objects.create()
>>> p1 = Parent.objects.create(child=child)
>>> p2 = Parent.objects.create(child=proxied_child)

And query using prefetch_related:

>>> Parent.objects.all().prefetch_related('child')
SELECT "foo_parent"."id", "foo_parent"."content_type_id", "foo_parent"."object_id" FROM "foo_parent" LIMIT 21

SELECT "foo_child"."id" FROM "foo_child" WHERE "foo_child"."id" IN (3)

SELECT "foo_child"."id" FROM "foo_child" WHERE "foo_child"."id" IN (2)

This is doing 3 queries instead of the expected 2. The two queries for the foo_child table should be combined to be "foo_child"."id" IN (2, 3)

Change History (2)

comment:1 Changed 6 years ago by Simon Charette

Component: contrib.contenttypesDatabase layer (models, ORM)
Keywords: prefetch_related added
Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization
Version: 1.6master

comment:2 Changed 3 years ago by Paulo

Owner: changed from nobody to Paulo
Status: newassigned
Note: See TracTickets for help on using tickets.
Back to Top