Opened 10 years ago

Closed 9 years ago

Last modified 7 years ago

#6432 closed Uncategorized (wontfix)

select_related() should also follow ManyToMany fields

Reported by: Bastian Kleineidam <calvin@…> Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: m2m queryset-refactor perf
Cc: Triage Stage: Someday/Maybe
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


The select_related() documentation states that it "follows" foreign-key relationships, but it does not so with ManyToMany fields. This is a humble request to also follow the ManyToMany relations.

This ticket might me best suited for the queryset-refactor branch.

Change History (8)

comment:1 Changed 10 years ago by anonymous

My hunch is that this simply isn't possible with SQL. Can you provide a regular SQL query that illustrates this working?

comment:2 Changed 10 years ago by Bastian Kleineidam <calvin@…>

Resolution: wontfix
Status: newclosed
Triage Stage: UnreviewedSomeday/Maybe

Perhaps it is better to wait for the queryset-refactor branch to land on the trunk, then revisit this idea. I will close the ticket for now.

comment:3 Changed 10 years ago by Malcolm Tredinnick

This is a very dangerous feature, by the way. Following multiple many-to-many relations is a recipe for disaster. If you accidentally follow two relations, the only way to do it in a single query (without using a union) will return n1*n2 results, where each relation has n1 and n2 results, respectively. Set n1 and n2 to 1000 each and notice the problem: you only want to retrieve 2000 rows, not one million rows.

If you have to do it as a union, you might as well just do two queries and merge the results in Python, since the extra overhead isn't that much and it keeps the Django core code much simpler.

So one day we might add support to select_related() for a single many-to-many relation at a time, but it will probably only be when somebody writes a patch to do so and after queryset-refactor has landed. Agree with closing for now, but this has cropped up before (although I can't find the dupe) and I just wanted to note the problem for anybody wanting to implement this.

comment:4 Changed 9 years ago by extropy

Resolution: wontfix
Status: closedreopened

This totally makes sense if you manually pass relation names while calling select_related.

The correct way to implement this would be to execute two queries - one for selecting the main data, then another for selecting the related data.

For example:

Suppose we have these tables - posts, tags and authors. With each post having single author and many (overlapping) tags.

Doing a query Post.objects.filter(year >= 2009).select_related("tags","authors")

Perform the following SQL queries:

  • First query: get all posts and authors
  • Second query: get all tags that are bound to posts returned by first query.

As far as I know, currently the only alternative of getting the tags is to perform a query for each post.

comment:5 Changed 9 years ago by James Bennett

Resolution: wontfix
Status: reopenedclosed

Please don't reopen a wontfix'd ticket without first building some consensus around it on the django-developers list.

comment:6 Changed 9 years ago by Jeremy Dunck

Keywords: perf added

comment:8 Changed 7 years ago by Luke Plant

Easy pickings: unset
Severity: Normal
Type: Uncategorized
UI/UX: unset

See #16937, which adds a new feature, rather than trying to add this into select_related.

Note: See TracTickets for help on using tickets.
Back to Top