Code

Opened 6 years ago

Closed 5 years ago

Last modified 3 years ago

#6432 closed Uncategorized (wontfix)

select_related() should also follow ManyToMany fields

Reported by: Bastian Kleineidam <calvin@…> Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: m2m queryset-refactor perf
Cc: Triage Stage: Someday/Maybe
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The select_related() documentation states that it "follows" foreign-key relationships, but it does not so with ManyToMany fields. This is a humble request to also follow the ManyToMany relations.

This ticket might me best suited for the queryset-refactor branch.

Attachments (0)

Change History (8)

comment:1 Changed 6 years ago by anonymous

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

My hunch is that this simply isn't possible with SQL. Can you provide a regular SQL query that illustrates this working?

comment:2 Changed 6 years ago by Bastian Kleineidam <calvin@…>

  • Resolution set to wontfix
  • Status changed from new to closed
  • Triage Stage changed from Unreviewed to Someday/Maybe

Perhaps it is better to wait for the queryset-refactor branch to land on the trunk, then revisit this idea. I will close the ticket for now.

comment:3 Changed 6 years ago by mtredinnick

This is a very dangerous feature, by the way. Following multiple many-to-many relations is a recipe for disaster. If you accidentally follow two relations, the only way to do it in a single query (without using a union) will return n1*n2 results, where each relation has n1 and n2 results, respectively. Set n1 and n2 to 1000 each and notice the problem: you only want to retrieve 2000 rows, not one million rows.

If you have to do it as a union, you might as well just do two queries and merge the results in Python, since the extra overhead isn't that much and it keeps the Django core code much simpler.

So one day we might add support to select_related() for a single many-to-many relation at a time, but it will probably only be when somebody writes a patch to do so and after queryset-refactor has landed. Agree with closing for now, but this has cropped up before (although I can't find the dupe) and I just wanted to note the problem for anybody wanting to implement this.

comment:4 Changed 5 years ago by extropy

  • Resolution wontfix deleted
  • Status changed from closed to reopened

This totally makes sense if you manually pass relation names while calling select_related.

The correct way to implement this would be to execute two queries - one for selecting the main data, then another for selecting the related data.

For example:

Suppose we have these tables - posts, tags and authors. With each post having single author and many (overlapping) tags.

Doing a query Post.objects.filter(year >= 2009).select_related("tags","authors")

Perform the following SQL queries:

  • First query: get all posts and authors
  • Second query: get all tags that are bound to posts returned by first query.

As far as I know, currently the only alternative of getting the tags is to perform a query for each post.

comment:5 Changed 5 years ago by ubernostrum

  • Resolution set to wontfix
  • Status changed from reopened to closed

Please don't reopen a wontfix'd ticket without first building some consensus around it on the django-developers list.

comment:6 Changed 5 years ago by jdunck

  • Keywords perf added

comment:8 Changed 3 years ago by lukeplant

  • Easy pickings unset
  • Severity set to Normal
  • Type set to Uncategorized
  • UI/UX unset

See #16937, which adds a new feature, rather than trying to add this into select_related.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.