Opened 3 weeks ago

Closed 2 weeks ago

Last modified 2 weeks ago

#37344 closed Bug (fixed)

FETCH_PEERS does not apply to instances loaded by select_related()

Reported by: Mykhailo Havelia Owned by: Mariano Ignacio Baragiola
Component: Database layer (models, ORM) Version: 6.1
Severity: Release blocker Keywords: fetch_mode, FETCH_PEERS, select_related, peers
Cc: Mykhailo Havelia, Adam Johnson, Varun Kasyap Pentamaraju Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Description

Combining select_related() with FETCH_PEERS silently disables peer batching one level below the join, turning an intended optimization into an N+1 and making the query count *worse* than not calling select_related() at all.

Given Species -> Genus -> Family:

# 3 queries: species, batch of genus, batch of family
for species in Species.objects.fetch_mode(FETCH_PEERS):
    species.genus.family.name

# 4 queries: the join, then ONE QUERY PER FAMILY
for species in Species.objects.select_related("genus").fetch_mode(FETCH_PEERS):
    species.genus.family.name

The second form emits:

SELECT ... FROM species INNER JOIN genus ON (species.genus_id = genus.id);
SELECT ... FROM family WHERE family.id = 1 LIMIT 21;
SELECT ... FROM family WHERE family.id = 2 LIMIT 21;
SELECT ... FROM family WHERE family.id = 3 LIMIT 21;

Was it intended to be this way? If so, I think it’s worth mentioning in the documentation.

Change History (11)

comment:1 by Mykhailo Havelia, 3 weeks ago

Cc: Mykhailo Havelia added

comment:2 by Sarah Boyce, 3 weeks ago

Cc: Adam Johnson added
Severity: Normal → Release blocker
Triage Stage: Unreviewed → Accepted
Type: Cleanup/optimization → Bug

Thank you for the report! Replicated and to me, this feels like a bug but I will let folks chime in as to whether this is expected behavior

  • tests/select_related/tests.py

    a b class SelectRelatedTests(TestCase):  
    281281            FETCH_PEERS,
    282282        )
    283283
     284    def test_fetch_peers_for_select_related_objects(self):
     285        species = Species.objects.select_related("genus__family").fetch_mode(
     286            FETCH_PEERS
     287        )
     288        with self.assertNumQueries(2):
     289            orders = [obj.genus.family.order.name for obj in species]
     290            self.assertEqual(
     291                orders,
     292                ["Diptera", "Primates", "Fabales", "Agaricales"],
     293            )
     294
    284295
    285296class SelectRelatedValidationTests(SimpleTestCase):

Refs #28586

comment:3 by Sarah Boyce, 3 weeks ago

Summary: Document that FETCH_PEERS does not apply to instances loaded by select_related() → FETCH_PEERS does not apply to instances loaded by select_related()

comment:4 by Varun Kasyap Pentamaraju, 3 weeks ago

Cc: Varun Kasyap Pentamaraju added

comment:5 by Vishy, 3 weeks ago

Owner: set to Vishy
Status: new → assigned

comment:6 by Mariano Ignacio Baragiola, 2 weeks ago

Owner: changed from Vishy to Mariano Ignacio Baragiola

comment:7 by Sarah Boyce, 2 weeks ago

Has patch: set

comment:8 by Sarah Boyce, 2 weeks ago

Patch needs improvement: set

comment:9 by Sarah Boyce, 2 weeks ago

Patch needs improvement: unset
Triage Stage: Accepted → Ready for checkin

comment:10 by Sarah Boyce <42296566+sarahboyce@…>, 2 weeks ago

Resolution: → fixed
Status: assigned → closed

In 6ade625:

Fixed #37344 -- Enabled FETCH_PEERS batching for instances loaded by select_related.

Thank you to Mykhailo Havelia for the report.

comment:11 by Sarah Boyce <42296566+sarahboyce@…>, 2 weeks ago

In 64c6e09:

[6.1.x] Fixed #37344 -- Enabled FETCH_PEERS batching for instances loaded by select_related.

Thank you to Mykhailo Havelia for the report.

Backport of 6ade6258480fba84a7e895b4b1e1716dfc954778 from main.

Note: See TracTickets for help on using tickets.
Back to Top