Add ModelAdmin.list_prefetch_related

[Hidde Bultsma]

Currently, applying prefetch_related to a ModelAdmin's queryset is a relatively simple operation. Take the following example:

class CollectionAdmin(admin.ModelAdmin):
    list_display = ('name', 'item_count')
    inlines = (ItemInline,)

    def item_count(self, obj):
        return obj.items.count()

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        return qs.prefetch_related('items')

With an override of get_queryset we can apply prefetch_related.

While this massively reduces the query count in the list view, it also introduces an extra redundant query in the change view. Both the prefetch and the ItemInline from this example will essentially execute the same query:

This prefetch was really only needed in the list view to show the item count per collection. But because get_queryset is called from both the list and change view, the prefetch was also applied in both.

New option ModelAdmin.list_prefetch_related

With a list_prefetch_related option in the ModelAdmin, prefetches could be more specifically targeted to the list view, preventing undesired prefetches in the change view.

This option should allow a list of lookups or None as value; the possible parameters of prefetch_related. It should be possible to set the value dynamically using a get_list_prefetch_related hook.

The example could be reimplemented with this option to the following code:

class CollectionAdmin(admin.ModelAdmin):
    list_display = ('name', 'item_count')
    list_prefetch_related = ('items',)
    inlines = (ItemInline,)

    def item_count(self, obj):
        return obj.items.count()

[Hidde Bultsma]

​PR

[Carlton Gibson]

Hi Hidde Bultsma.

Thanks for the suggestion here. I'm going to say wontfix: ModelAdmin already has many options, we don't want to add more unless it's going to be for a core use-case.

Have you thought about using an ​annotation to provide the item_count value? This would save the second query entirely. It that's not fast enough for you I would advise examining request in get_queryset() to only conditionally apply the prefetch (or annotation).

[Hidde Bultsma]

Hi Carlton,

Thanks for your response, I haven't thought about using an annotation for the item count. That's indeed much better than doing a prefetch. I can think of different cases where instead a prefetch is needed in the changelist, such as displaying a list of items: "foo, bar, and item3".

The decision is understandable, the ModelAdmin API is already huge, and a list_prefetch_related option is maybe too specific.

So, unless I'm missing something, to do anything querysetwise just for the changelist, one needs to check the resolver_match property of the request:

def get_queryset(self, request):
    qs = super().get_queryset(request)
    if request.resolver_match.view_name.endswith('changelist'):
        return qs.prefetch_related('items')
    return qs

Maybe a get_list_queryset method could help with this, but that would also add to the ModelAdmin API.

[Simon Charette]

Maybe a get_list_queryset method could help with this, but that would also add to the ModelAdmin API.

In most cases using prefetch_related() even for the details queryset is not going to be hurtful.

Your CollectionAdmin is a good example as it requires fetching items anyway to display the inlines. Doing it through prefetch_related or not will still result in a query.

FWIW the get_list_queryset idea is already tracked in #10761.

[Hidde Bultsma]

Replying to Simon Charette:

Your CollectionAdmin is a good example as it requires fetching items anyway to display the inlines. Doing it through prefetch_related or not will still result in a query.

From what I have observed, applying prefetch_related results in an extra query besides the similar query from the inline. So it looks like prefetched data isn't used by inlines. See the following query debug output from the details view: (edited for readability)

SELECT "id", "collection_id", "name" FROM "djangotest_item" WHERE "collection_id" IN (1) ORDER BY "name" ASC;
SELECT "id", "collection_id", "name" FROM "djangotest_item" WHERE "collection_id" =   1  ORDER BY "name" ASC;

[Simon Charette]

Ah you're right. The non-reuse of prefetched relationship for inlines is tracked in #18597.