`filter_horizontal` duplicates entries in "Chosen" column after instance is added via in another field using the "plus" JS action

[devin13cox]

Hi there, I noticed a minor frontend bug related to the filter_horizontal tables in the admin console. Here is what I have:

There is a model called Transition with two fields, source and target:

source = models.ManyToManyField(State, related_name="transition_source")
target = models.ForeignKey(State, related_name="transition_target", on_delete=models.CASCADE)

For the admin console, we set filter_horizontal to include 'source'. See "First Screenshot" for an example of a given state of a Transition, with "test" selected as Source and "test2" selected as Target.

Now, if I add a new State by clicking the "+" Icon next to the Target field (let's call it test3), it will add it to both the Target AND the "Chosen Source" (see "Second Screenshot" for an example).

I will note that upon saving the Transition, it will not actually save "test3" to the Chosen Source, meaning that this is an issue only from a UI perspective. After any refresh or save, it will also no longer show in the Chosen Source column. Additionally, there is no issue going the other direction (if I add directly using the "+" next to Chosen Source, it will not appear to set the "Target" field as well, and it will save properly). Finally, if I were to select another available source (ex. "test2") and add it to the Chosen Source, it will successfully add "test2" and remove "test3" without needing a save or refresh.

I believe this is due to the admin conveniently updating all related fields to include the added object, and therefore displaying it as if it was selected temporarily until refreshed

Diving into this, I believe the issue is in this file: django/django/admin/static/admin/js/admin/RelatedObjectLookups.js

In this function: UpdateRelatedSelectOptions, on this query:

const selectsRelated = document.querySelectorAll(`[data-model-ref="${modelName}"] select:not(.admin-autocomplete)`);

My initial guess is that updating the query to not include the "Chosen" column in filter_horizontal fields would solve this issue (something like

const selectsRelated = document.querySelectorAll(`[data-model-ref="${modelName}"] div:not(.selector-chosen) > select:not(.admin-autocomplete)`);

Let me know if more context is needed to help reproduce what is going on here. Thanks!

[devin13cox]

First Screenshot

[devin13cox]

Second Screenshot

[Natalia Bidart]

Hello! Thank you for your report. Could you please post your admin.py definition as well as your models.py (reduced to this example)? So we can try to reproduce and triage accordingly. Thanks!

[devin13cox]

Replying to Natalia Bidart:

Hello! Thank you for your report. Could you please post your admin.py definition as well as your models.py (reduced to this example)? So we can try to reproduce and triage accordingly. Thanks!

Of course, here is the model (reduced to the example):

class Workflow(models.Model):
    initial = models.ForeignKey('workflow.State', blank=True, null=True, related_name="workflow_initial", on_delete=models.SET_NULL)

class Transition(models.Model):
    source = models.ManyToManyField(State, related_name="transition_source")
    target = models.ForeignKey(State, related_name="transition_target", on_delete=models.CASCADE)

class State(models.Model):
    label = models.CharField(max_length=255)

And the admin (also reduced):

class WorkflowAdmin(SortableAdminBase, admin.ModelAdmin):
    inlines = [TransitionInline, ]
    fieldsets = (
        (None, {
            'fields': ["initial"],
        }),
    )

class TransitionInline(SortableStackedInline, admin.StackedInline):

    model = Transition
    extra = 0
    filter_horizontal = ['source']

    fieldsets = (
        (' ', {
            'classes': ('collapse',),
            'fields': ('source', 'target')
        }),
    )

class StateAdmin(admin.ModelAdmin):
    fieldsets = (
            (None, {
                'fields': ("label",),
            }),
        )

As you can see, the model of interest is Transition and in the admin console it is represented through as an inline for another model called Workflow. I tried to remove any extra information to avoid confusion (part of this removal was my_order fields in case there is confusion as to why Sortable is being used).

While I don't think that the "Workflow" component (and Inline stacking) are relevant to this bug -- I kept them there to keep it as close to my current flow as possible. This should be able to be reproduced without the workflow element.

[Natalia Bidart]

Replying to devin13cox:

Of course, here is the model (reduced to the example):

Great! Thank you.

While I don't think that the "Workflow" component (and Inline stacking) are relevant to this bug -- I kept them there to keep it as close to my current flow as possible. This should be able to be reproduced without the workflow element.

Confirmed I was able to reproduce with a reduced example like this:

from django.db import models


class State(models.Model):
    label = models.CharField(max_length=255)

    def __str__(self):
        return self.label


class Transition(models.Model):
    source = models.ManyToManyField(State, related_name="transition_source")
    target = models.ForeignKey(State, related_name="transition_target", on_delete=models.CASCADE)

from django.contrib import admin

from .models import State, Transition


class TransitionAdmin(admin.ModelAdmin):
    filter_horizontal = ['source']


admin.site.register(State)
admin.site.register(Transition, TransitionAdmin)

Would you like to prepare a patch?

[Mariusz Felisiak]

Check out related tickets #34025 and #11803.

[David Sanders]

Replying to devin13cox:

My initial guess is that updating the query to not include the "Chosen" column in filter_horizontal fields would solve this issue (something like

const selectsRelated = document.querySelectorAll(`[data-model-ref="${modelName}"] div:not(.selector-chosen) > select:not(.admin-autocomplete)`);

That could work 👍 I'm just wondering though whether there's a better approach than to continuously blacklist ancestors as we come across these 🤔

[yokeshwaran1]

Hi David,
Im new to this repo, so I appreciate any guidance or feedback u can offer.

Regardin issue with filter_horizontal "Chosen" column. From the discussion, I think about a solution which will not continuously blacklisting ancestors

1. Adding a data attribute like data-context to select elements during its render phase in the Django admin templates. So each select will have a more specific context about its purpose (eg. source, target, chosen etc.).

2. Then modifying the query in RelatedObjectLookups.js to use this new data-context attribute, so our selector will become:

const selectsRelated = document.querySelectorAll(`[data-model-ref="${modelName}"][data-context="source"] select:not(.admin-autocomplete)`);

This approach will provide clear context for each select element which makes JavaScript logic more intentional. would this be a feasible, acceptable solution in ur opinion? I'm eager to hear ur thoughts & make any necessary adjustments. Once approach is approved, I'm interested in working on this ticket. Please let me know if that's possible.

Replying to David Sanders:

Replying to devin13cox:

My initial guess is that updating the query to not include the "Chosen" column in filter_horizontal fields would solve this issue (something like

const selectsRelated = document.querySelectorAll(`[data-model-ref="${modelName}"] div:not(.selector-chosen) > select:not(.admin-autocomplete)`);

That could work 👍 I'm just wondering though whether there's a better approach than to continuously blacklist ancestors as we come across these 🤔

[David Sanders]

Replying to yokeshwaran1:

I think about a solution which will not continuously blacklisting ancestors

Yep that's along the lines of what I was getting at 😊👍

The question is though which approach is clearer/easier to maintain. The benefit of whitelisting is we can remove not(.admin-autocomplete) too I think 🤔

Edit: Felix & Nesita will decide which is better; if you like submit a PR with a fix + some tests.

[yokeshwaran1]

Thanks, David. I’ll make the necessary changes and submit a PR with tests soon.

[yokeshwaran1]

Hi David,
I have opened a PR to address this issue: ​https://github.com/django/django/pull/17219

Replying to yokeshwaran1:

Thanks, David. I’ll make the necessary changes and submit a PR with tests soon.

[Natalia Bidart]

Setting the has patch flag.

[devin13cox]

Opened new PR on this ​https://github.com/django/django/pull/17897. Ran into small issue that still needs work, as described in the old PR. Will get it patched soon, but if anyone has direction on it I would appreciate feedback. Thanks!

[devin13cox]

Hello, just wanted to check in and see if there is anything else I need to do pending a review for this ticket. No rush on getting this in, I just wanted to make sure you guys aren't waiting on something from me. Thanks!

[Tim Graham]

No, the ticket flags are set correctly to put the ticket in the patch review queue. You can confirm this by looking at the yellow box below the ticket description ("According to the ticket's flags...").