Opened 45 minutes ago

Last modified 9 minutes ago

#36764 new Bug

QuerySet.only() causes n+1 queries with reverse foreign key relation

Reported by: bernhard Owned by:
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: queryset, only, defered
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

To reproduce I tried adding the following test to tests.defer.tests.DeferTests:

def test_only_reverse_fk(self):
        qs = self.s1.primary_set.only("name")
        with self.assertNumQueries(1):
            for primary in qs:
                primary.name

This fails because it produces an additional query for every Primary instance to just fetch related_id:

test_only_reverse_fk (defer.tests.DeferTests.test_only_reverse_fk) failed:

    AssertionError('3 != 1 : 3 queries executed, 1 expected
    Captured  queries were:
        1. SELECT "defer_primary"."id", "defer_primary"."name" FROM "defer_primary" WHERE "defer_primary"."related_id" = 1
        2. SELECT "defer_primary"."id", "defer_primary"."related_id" FROM "defer_primary"  WHERE "defer_primary"."id" = 1 LIMIT 21
        3. SELECT "defer_primary"."id", "defer_primary"."related_id" FROM "defer_primary"  WHERE "defer_primary"."id" = 2 LIMIT 21
    ')

When replacing the queryset with qs = Primary.objects.filter(related_id=self.s1.pk).only("name") basically the same SQL is produced, but no additional queries.
I think the additional queries are quite an unexpected behaviour and it is unnecessary to retrieve related_id for every row. Somehow this is also a dangerous behaviour as it can cause n + 1 queries when actually trying to optimize your query.

If this behaviour should be the expected one it should at least be documented and tested somewhere (I don't think there are tests for only() when using relations like this somewhere).

Change History (1)

comment:1 by Clifford Gama, 10 minutes ago

Thanks for the report! FWIW, this works:

    def test_only_reverse_fk(self):
        qs = self.s1.primary_set.only("name", "related_id")
        with self.assertNumQueries(1):
            for primary in qs:
                primary.name

and so does .values("name").

Last edited 9 minutes ago by Clifford Gama (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top