Django admin many to many recursive field widget shows identical objects

[Darshan Singh Khalsa]

I have a scheduling app with a backend managed via Django Admin. This production app runs okay on a dedicated server using Ubuntu 14.04 LTS, Python2.7.3 with Django1.7.1 on a MySQL 5.5 database where the datatables evolved with the app through scores of code updates and migrations.

The Members table for managing people and their role includes two recursive fields that indicate whether a particular person has an employer (ForeignKey) or is associated with one or more customers (ManyToMany).

I subsequently discovered that when creating a brand new MySQL database, the Members table has flaws that render the Django Admin unuseable on that table. The Members table in its original model had two recursive fields that referred to fields within the "Member" table; current Django syntax requires recursive fields to be in the "self" table.

The problem is that when adding or editing a Member via Django Admin, the employer picklist and customers picklist fields display "Member object" for each row instead of the desired list of various company names.

Same problem in Python2.7.3 with Django1.7.1 and in Python3.4.3 with Django1.10.3.

I posted this problem 8 month ago on StackOverflow.com at ​https://stackoverflow.com/questions/40954339/django-admin-many-to-many-recursive-field-widget-shows-identical-objects but no one volunteered a solution. The code snippets below create a minimum database that illustrates the problem. I can provide more documentation and screenshots if desired.

# models.py
class Member(models.Model):
    first_name = models.CharField(max_length = 30)
    last_name = models.CharField(max_length = 30)

    role_choices = (
        ('Attendee', 'Attendee'),
        ('Customer', 'Customer'),
        ('Registrar', 'Registrar'),
          )

    now = timezone.now()

    role = models.CharField(choices=role_choices, max_length=50)
    created_date = models.DateTimeField(default=now)
    employer = models.ForeignKey('self', null=True, blank=True,
                     limit_choices_to={'role': "Customer"})
    customers = models.ManyToManyField('self', limit_choices_to= 
        {'role': "Customer"}, related_name="registrar_customers", 
        null=True, blank=True)
    company_name = models.CharField(max_length=100, null=True, blank=True)

    def __unicode__(self):
        if self.role == "Customer":
            name = self.company_name
        elif self.role == "Attendee":
            name = "{} - {}{}".format(self.employer.company_name, 
                self.first_name, self.last_name)
        else:
            name = self.first_name + " " + self.last_name

        return name

# admin.py
class MemberAdminForm(forms.ModelForm):
    class Meta:
        model = Member

    # Form override to force company names into picklists. This code section does not work.
        customers = forms.MultipleChoiceField(required=False)

        def __init__(self, *args, **kwargs):
            super(MemberAdminForm, self).__init__(*args, **kwargs)
            cust_choices = ((obj.id, obj.company_name) for obj 
                            in Member.objects.filter(role = 'Customer'))

            self.fields['employer'].choices = cust_choices
            self.fields['customers'].choices = cust_choices

        fields = ['first_name', 'last_name', 'role', 
                    'created_date', 'employer', 'customers', 'company_name']

class MemberAdmin(admin.ModelAdmin):
    list_display = ('first_name', 'last_name', 'role', 
            'created_date', 'employer', 'company_name')

    form = MemberAdminForm

admin.site.register(Member, MemberAdmin)

[Darius Azimi]

Can u provide some screenshots? Like to replicate this with python 3.x and django 1.1x.
I'm at django con today and can take a look.

[Darshan Singh Khalsa]

ScreenshotA of a live site showing Member table in Django Admin (Python 2.7.1) in Change member page showing Employer companies in field picklist

[Darshan Singh Khalsa]

ScreenshotB of a live site Member table in Django Admin showing attendees with Employer field correctly populated

[Darshan Singh Khalsa]

ScreenshotC of a Python3 Member table with customers and attendees. Note Employer field is not populated yet.

[Darshan Singh Khalsa]

ScreenshotD of a Python3 Member in Django Admin edit mode showing Employer picklist values as "Member object" instead of company names

[Darshan Singh Khalsa]

Thanks for your prompt feedback. I have attached 4 screenshots, two (A and B) from a live development site built in Django1.7.1 and two (C and D) from a test site built in Django1.10.3.

[Darius Azimi]

I am going to try to replicate this with py 3.x and django 1.x. Do you have ur code on github so that I can do a pull request?

[Darshan Singh Khalsa]

The small test app "memberobj" that I built in Django1.10.3 is not up on GitHub but I should be able to upload it within the next hour or so. Otherwise I think I could create a Zip file of the source code for the test app.

[Darius Azimi]

Just looking at the code, your using unicode with python 3?

In Python 3, you need to define a str method instead of a unicode method. There is a decorator python_2_unicode_compatible which helps you to write code which works in Python 2 and 3.

see below. There are example of this on django docs.

​https://docs.djangoproject.com/en/1.11/topics/python3/

Can u try it and provide an update.

from __future__ import unicode_literals
from django.utils.encoding import python_2_unicode_compatible

@python_2_unicode_compatible

[Darshan Singh Khalsa]

Thanks for your suggestion. I added the code snippets you suggested and put the @python_2_unicode_compatible decorator in front of each models.py occurrence of the def unicode(self): section. Running the Python3 test version of the live app with these changes resulted in no difference, "Member object" still appears in the employer and customer picklist fields. I thought perhaps I should migrate to see if the code edits would apply, but got many errors. Probably due to my not working with the test code for several months and having so many different versions while tinkering to overcome the "Member object" issue.

I internationalized my Python2 live app last year and had to overcome many tricky Unicode issues related to translation into multiple foreign languages and writing downloads of data files containing Unicode characters. Had to rewrite quite a bit of code to get the app running in Python3, so once I get everything working properly in Python3, there is no going back. This bug report's issue has been the main obstacle to moving to Python3.

I have a concern that using a Py2-Py3 compatible decorator as you recommend may drag me back towards Python2 and all of its Unicode inadequacies. I will have to read up on the decorator you recommended as well as read about str instead of Unicode in my Python3 models.py.

[Darius Azimi]

I'd recommend moving to python3 and upload a simplified version of your code to GitHub if that issue still exists. So for now I will close this issue if that is ok.

[Darshan Singh Khalsa]

Thank you, I got the simple model working with valid data in the recursive field picklists. Consider this ticket closed.