Opened 2 years ago

Closed 2 years ago

#19578 closed New feature (wontfix)

lazy prefetch_related

Reported by: g00fy Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: prefetch_related
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I suggest, that prefetch_related should be lazy. What I mean is, it would only execute aditional query, when related model needs to be accessed.
Example:

# this would do just one query - prefetch_related is lazy
pizzas = Pizza.objects.all().prefetch_related('toppigns')
for pizza in pizzas:
    print pizza.name
# Proposal A

# now we want to access the related objects, 
# 1 aditional query (intermediate m2m table) is executed
# not 2 since, we only need the id of the related object (which is already in the m2m table)
# the 2nd query would result in a set of 'lazy' objects
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.id

# now 3rd query is made
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.name
# Proposal B


# 2nd query is made (with the current implementation - m2m table join)
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.id

Usecase:

let's consider logic like this:

pizzas = Pizza.objects.all()

print len(pizzas)
if pizzas[0].name == 'pepperoni':
    for pizza in pizzas:
        print [topping.name for topping in pizza.toppings.all()]
else:
    for pizza in pizzas:
        print pizza.name

in otherwords, this would be useful, if there is some complex logic that sometimes may require prefetch_related (and this requirement can be only checked after the query was made - so there is no way to do the checking & then doing prefetch_related).

Change History (1)

comment:1 Changed 2 years ago by akaariai

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to wontfix
  • Status changed from new to closed

I am going to close this.

There are two reasons for this:

  1. I don't see any easy way to achieve this. Implementing this is likely possible, but it is going to cost us in code complexity.
  2. Lazy prefetch_related might lead to situations where prefetch_related is littered just in case in get_query_set() of base manager. This has the side effect that if you access toppings.all() for one object, then you are going to do a prefetch for the whole queryset. In short, it will be harder to understand what queries are made and when which isn't a nice property to have in the ORM.

If there is an easy way to make this achievable in 3rd party app, then lets add that into core. But that is separate ticket's issue, and that ticket should be opened only if it is accompanied with an idea how to achieve the wanted result.

Note: See TracTickets for help on using tickets.
Back to Top