Related fields pointing to custom fields that store different values in the database than the python representation do not remember selections

[thenewguy]

I have uploaded a demonstration to github here: ​https://github.com/thenewguy/django_custom_primary_key_related_field_incompatibility Concisely, forms do not preserve related field selections on these custom fields.

If you have a custom field that returns a custom object from the database that is different from the value returned from to_python, related fields that point to forget the current selection when they are rendered by forms.

I encountered this attempting to use a custom field that maps signed integers to fixed length string representation of unsigned integers for use as identifiers.

[Tim Graham]

I spent some time looking at this, but ultimately, I couldn't determine whether this was a bug in Django or in your own code. It would be helpful if you could provide more details about where you think the problem lies.

By the way, there are ​plans to deprecate SubFieldBase. Your issue may still be valid, but I thought I'd mention it so you are aware.

[thenewguy]

Sorry for the delayed response. I didn't get a notice of activity for some reason.

I am not sure exactly where the bug is. This is from memory from awhile ago, but when the form is rendered the widget is passed the raw value of the relation (e.g. -200 instead of 1 for the int field in my demo). So when the choice widget compares the value -200 against possible choice values, it doesn't match because it is comparing the db value against the to_python value. The choices iterable that the widget checks for the value -200 in contains the to_python value that -200 was mapped to (i.e. 1).

I do not think the bug is in my code. It seems to work for everything except for this issue with forms.

Have I explained the problem well enough for you to reproduce/observe the symptoms?

[thenewguy]

I reopened because it was marked as (needsinfo) and I posted more information.

[Tim Graham]

I think we really need more effort on your part in suggesting a solution since the example is quite complex and very specialized. Once again, I spent over an hour looking at this and couldn't figure out the exact issue.

Something suspicious that seems related to the problem:

>>> a = BarInt.objects.get()
>>> a.pk
-15
>>> # a.pk is a shortcut for a.id_id
>>> a.id_id
-15
>>> a.id
<FooInt: 185>

>>> # how form data is generated
>>> model_to_dict(a)
{'id': -15}

>>> # how form choices are generated
>>> [str(a) for a in FooInt.objects.all()]
['185']

I could make the Id and Fk widgets work by adding this method to the model:

    def serializable_value(self, field_name):
        val = super(FooInt, self).serializable_value(field_name)
        if field_name == 'id':
            return val.db_value
        return val

but this may not be a good solution and I'm not sure if this will cause other problems (also m2m didn't work).

Also it would be helpful to modify your project so that it doesn't use SubFieldBase to verify the issue still exists without that (I suspect it does).

[thenewguy]

Hello Tim. I am happy to spend more time on this. I am covered up for a few weeks with deadlines. Can we reopen this issue?

[Tim Graham]

Yes, please reopen when you provide more details showing what the bug is then I'll be happy to take another look.

[Tim Graham]

This may be a duplicate or at least related to #17122.

[thenewguy]

So this has been fixed in DJ18 with the from_db_value() method. I was never able to get something working under DJ17. I've been able to make this work with DJ18, DJ19 and master.