Django

Code

Ticket #9554 (closed: wontfix)

Opened 8 months ago

Last modified 4 months ago

admin app's change list usage of select related - maximum depth

Reported by: gert Assigned to: nobody
Milestone: Component: Contrib apps
Version: 1.0 Keywords: select_related list_display list_select_related
Cc: Triage Stage: Unreviewed
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

In applications with deep object relationships "select related" often results in huge queries that can hit the database very hard. I.e. we have an accounting module that immediately adds 10 levels of related objects to your average "client" object.

I think don't think it makes sense the select related for more than 2 or 3 relationships deep. My suggestion is to add a "maximum depth" parameter to select related that defaults to something sensible like 3 or at least make it configurable.

Attachments

Change History

11/08/08 03:56:23 changed by ramiro

  • status changed from new to closed.
  • needs_better_patch changed.
  • resolution set to invalid.
  • needs_tests changed.
  • needs_docs changed.

What about a depth argument like implemented and documented at http://docs.djangoproject.com/en/dev/ref/models/querysets/#id4 ? (granted, perhaps it should be included in the signature in the section title)

12/16/08 02:08:09 changed by gert

  • status changed from closed to reopened.
  • resolution deleted.
  • component changed from Database layer (models, ORM) to Contrib apps.

I think maybe a better place to address this issue is in the admin application by just adding a depth=3 parameter to the two select_related instructions in admin/views/main.py

def get_query_set(self):
    ...

        # Use select_related() if one of the list_display options is a field
        # with a relationship.
        if self.list_select_related:
            qs = qs.select_related(depth=3)
        else:
            for field_name in self.list_display:
                try:
                    f = self.lookup_opts.get_field(field_name)
                except models.FieldDoesNotExist:
                    pass
                else:
                    if isinstance(f.rel, models.ManyToOneRel):
                        qs = qs.select_related(depth=3)
                        break


12/16/08 05:54:24 changed by ramiro

So, you were talking specifically about the admin contrib app?.

12/16/08 19:43:52 changed by mtredinnick

This doesn't sound like a clear problem description to me yet. Select_related already has a default maximum depth of 5, which already meets the definition of "sensible" for many cases. I don't see any great need to micromanage at that point by saying "perhaps 2 or 3 is better" -- we just pick a value and stick with it and 5 is the current value. We could be here all day debating 2 or 3 or 5 or 7.

So the real issue is why this particular problem report is not respecting the maximum depth of 5. If it is, then this becomes a proposal to change 5 to 3 and ... meh.

12/18/08 01:31:20 changed by gert

ramiro: yes, I only picked it up in the admin contrib app. At that stage I was unaware of the depth=X parameter. I think it adequately addresses the issue for custom applications, also I don't think internally select_related needs to drop the 5 down to 3.

BUT, for the admin application I think a tweak is required. Arguments for explicitly lowering select_related for admin to say 3 :) are;

1) Whatever value you take must hit the sweat spot for 90% of situations otherwise it will slow down rather than increase overall application speed. I can not even think of an example where I want to go down 5 levels. (object.level1.level2.level3.level4.level5)

2) Remember that the effect on the database increases exponentially, e.g.

3 levels 2 references per object => 32 = 9 DB tables

5 levels 2 references per object => 52 = 25 DB tables

5 levels 3 references per object => 53 = 125 DB tables

12/18/08 03:42:23 changed by ramiro

  • keywords changed from select related to select related select_related list_display list_select_related.
  • summary changed from select related - maximum depth to admin app's change list usage of select related - maximum depth.

02/22/09 11:52:12 changed by ramiro

#9849 (closed as a duplicate of this ticket) proposed being able to control the select related queries by being able to specify which relationships to follow.

02/26/09 16:07:51 changed by jacob

  • status changed from reopened to closed.
  • resolution set to wontfix.

Yeah, let's not micromanage the admin like this. You can just override queryset if you need advanced usage.

03/11/09 17:47:58 changed by ramiro

  • keywords changed from select related select_related list_display list_select_related to select_related list_display list_select_related.

Add/Change #9554 (admin app's change list usage of select related - maximum depth)




Change Properties
Action