Opened 3 weeks ago

Closed 2 weeks ago

Last modified 2 weeks ago

#37300 closed Bug (fixed)

Custom Prefetch querysets on forward FK/O2O fields are no longer routed to the parent queryset's database

Reported by: Norbert Kwizera Owned by: Yassin Bahri
Component: Database layer (models, ORM) Version: 6.1
Severity: Release blocker Keywords:
Cc: Adam Johnson Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Jacob Walls)

In Django ≤ 6.0, when a Prefetch with a custom queryset (with no explicit .using()) was used to prefetch a forward ForeignKey or OneToOneField, get_prefetch_querysets() called queryset._add_hints(instance=instances[0]), so the default router's db_for_read() resolved to the parent instances' database via the instance hint. The custom prefetch queryset therefore ran against the same database as the outer queryset.

In 6.1, commit 821619aa8771ef211c4c4922001efdf914201ca3 (Refs #28586, "Simplified related descriptor get_queryset() methods") moved the hint into get_queryset(instance=...), which is only called when no custom queryset is passed. The custom-queryset path in ForwardManyToOneDescriptor.get_prefetch_querysets() and ReverseOneToOneDescriptor.get_prefetch_querysets() lost the _add_hints() call entirely, so a custom prefetch queryset now silently runs against "default" even when the outer queryset uses another alias. The reverse many-to-one and many-to-many manager paths still call _add_hints() for custom querysets, so those relation types are unaffected — making the forward FK/O2O behavior inconsistent with them as well as with 6.0.

Reproduction, with a "replica" alias configured identically to "default":

from django.db.models import Prefetch

book = Book.objects.using("replica").prefetch_related(
    Prefetch("author", queryset=Author.objects.only("id", "name")),
)[0]

book._state.db          # "replica" in both versions
book.author._state.db   # 6.0: "replica"  /  6.1: "default"
Without the custom queryset (.prefetch_related("author")), 6.1 still correctly returns "replica".

Beyond queries silently hitting the wrong database (e.g. a primary instead of a read replica), this breaks code that assigns prefetched objects onto instances from the outer database — the forward descriptor's set then raises:

ValueError: Cannot assign "<Author: ...>": the current database router prevents this relation.
because the two instances' _state.db values no longer match under the default router's allow_relation().

This isn't mentioned in the 6.1 release notes, and the commit message suggests the hint was intended to be set earlier, not dropped for custom querysets, so it appears unintentional. Restoring queryset._add_hints(instance=instances[0]) in the custom-queryset branches of both descriptors' get_prefetch_querysets() fixes it. Still present on main and stable/6.1.x as of this report.

Change History (10)

comment:1 by Norbert Kwizera, 3 weeks ago

Component: UncategorizedDatabase layer (models, ORM)

comment:2 by Clifford Gama, 3 weeks ago

Severity: NormalRelease blocker
Type: UncategorizedBug

comment:3 by Yassin Bahri, 3 weeks ago

Owner: set to Yassin Bahri
Status: newassigned

I reproduced this on current main and opened PR: https://github.com/django/django/pull/21828

The regression happens for both custom forward FK prefetch querysets and custom reverse O2O prefetch querysets. In both cases, the custom queryset no longer receives the parent instance hint, so routers can choose the wrong database.

The PR restores _add_hints(instance=instances[0]) in both descriptor paths and adds multi-database regression tests for Book.editor and User.userprofile.

Locally tested with:

python tests/runtests.py multiple_database --verbosity 1
Version 0, edited 3 weeks ago by Yassin Bahri (next)

comment:4 by Yassin Bahri, 3 weeks ago

Has patch: set

comment:5 by Yassin Bahri, 3 weeks ago

Triage Stage: UnreviewedAccepted

comment:6 by Jacob Walls, 2 weeks ago

Description: modified (diff)

comment:7 by Jacob Walls, 2 weeks ago

Triage Stage: AcceptedReady for checkin

comment:8 by Jacob Walls, 2 weeks ago

Cc: Adam Johnson added

comment:9 by Jacob Walls <jacobtylerwalls@…>, 2 weeks ago

Resolution: fixed
Status: assignedclosed

In 60a17fc:

Fixed #37300 -- Preserved parent instance hints on custom Prefetch querysets.

Regression in 821619aa8771ef211c4c4922001efdf914201ca3.

Thanks Norbert Kwizera for the report.

comment:10 by Jacob Walls <jacobtylerwalls@…>, 2 weeks ago

In 4b0185a5:

[6.1.x] Fixed #37300 -- Preserved parent instance hints on custom Prefetch querysets.

Regression in 821619aa8771ef211c4c4922001efdf914201ca3.

Thanks Norbert Kwizera for the report.

Backport of 60a17fccf4f2fa31b823b0f938005897f2f3ef16 from main.

Note: See TracTickets for help on using tickets.
Back to Top