Opened 5 years ago

Closed 5 months ago

#18729 closed Bug (fixed)

Admin changelist view defaults to `DISTINCT`, unusable on reasonably sized databases

Reported by: Henrique C. Alves Owned by: AlexMalek
Component: contrib.admin Version: master
Severity: Normal Keywords: admin sql distinct slow performance
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


The default behavior on the admin is not usable for reasonably sized tables (a couple thousand tuples is enough for >1s queries).

Right now, it always includes a DISTINCT clause if you add a M2M field on list_filter, even if you're not applying any filters. Use of DISTINCT is known to be painfully slow, and even more so without a WHERE clause, because it forces SQL databases into a full table scan. The behavior is not easily overridden because the logic is scattered on a couple ChangeList methods (

The only solution right now is not including a M2M field on list_filter and limiting functionality. It would be better if that behavior could be overridden (a use_distinct parameter on get_query_set?) or, optimally, moving this logic somewhere else. Ticket #15819 already includes a patch for that.

Change History (9)

comment:1 Changed 5 years ago by Travis Swicegood

Resolution: duplicate
Status: newclosed
Type: UncategorizedBug

Resolving this as a dupe #15819. Looks like that issue has all of the same points here and fixes this issue.

Feel free to re-open if I missed something in this ticket that's different than #15819.

comment:2 Changed 3 years ago by AlexMalek

Resolution: duplicate
Status: closednew

I think it is a different issue.
It is possible to alter the check that examines list_filter to determine if distinct() is needed based on if the field appears in the query/request but this does not address the deeper issues in #15819
For example in Django 1.5.2 I think the following changes solves the issue:

< def lookup_needs_distinct(opts, lookup_path, query_dict=None):
> def lookup_needs_distinct(opts, lookup_path):
<          not field.field.unique) and
<        (query_dict is None or (field_name in ",".join(query_dict.keys())))
<       ):
>          not field.field.unique)):

<                                                           field_path, lookup_params))
>                                                           field_path))
Last edited 20 months ago by Tim Graham (previous) (diff)

comment:3 Changed 3 years ago by AlexMalek

Owner: changed from nobody to AlexMalek
Status: newassigned

comment:4 Changed 3 years ago by AlexMalek

Has patch: set

comment:5 Changed 3 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

comment:6 Changed 3 years ago by Nick Sandford

Triage Stage: AcceptedReady for checkin

comment:7 Changed 3 years ago by Tim Graham

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

I left comments for improvement on the PR. Please uncheck "Patch needs improvement" when you update it, thanks.

comment:8 Changed 6 months ago by Tim Graham

Patch needs improvement: unset

comment:9 Changed 5 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 8f76673:

Fixed #18729 -- Made admin changelist not use distinct() if a list_filter doesn't require it.

Note: See TracTickets for help on using tickets.
Back to Top