Database converters are not run for related fields referencing related fields

[oyooyo]

Sample models.py:

from django.db import models

import uuid

class Base_Model(models.Model):
	id = models.UUIDField(primary_key = True, default=uuid.uuid4, editable=False)
	# id = models.AutoField(primary_key = True)
	prototype = models.ForeignKey('Prototype_Model', blank=True, null=True)
	def __str__(self):
		return str(self.id)

class Prototype_Model(Base_Model):
	pass

Sample admin.py:

from django.contrib import admin

from .models import Prototype_Model

class Base_Model_Admin(admin.ModelAdmin):
	list_display = ['id', 'prototype']

admin.site.register(Prototype_Model, Base_Model_Admin)

Steps to reproduce:

1. Create a Prototype_Model via the admin. Leave the "prototype" ForeignKey field empty.
2. Create another Prototype_Model via the admin. For the "prototype" ForeignKey field, choose the first created Prototype_Model from the dropdown
3. Switch to the Prototype_Model list view, it should show that the "prototype" ForeignKey field of the second Prototype_Model indeed references the first model
4. Click on this second Prototype_Model in order to access the change/edit model form and look at the dropdown for the "prototype" field. The expected behaviour is that the dropdown should have the referenced first model automatically selected, but instead, nothing is automatically selected.

If an AutoField is used instead of a UUIDField as the primary key "id" field (see commented out line in Base_Model), the behaviour is as expected.

[Tim Graham]

I can reproduce this at eb7fb565e6483637fc4ee251940b86db813485a0 on SQLite but not PostgreSQL -- I guess it's based on whether or not the database has a native UUID type. On SQLite, for example, the initial form data appears as {'prototype': '35bba53ceafc4df2b1abc110d5f51b2a'} but the UUID values in the form's <select> have dashes, so there's a mismatch. On PostgreSQL, the initial data is {'prototype': UUID('154e323f-cc57-4399-9023-58357ae9ce21')} and there's no issue.

[Wayne Merry]

A topic was posted to Django Developers regarding a UUID foreign key value representation when using sqllite3. See ​https://groups.google.com/forum/#!topic/django-developers/h46RAw67g_g

[Simon Charette]

I feel like this isn't a form level issue from a bit of debugging locally.

I'm not sure what's going on here but somehow the database converters are not considered for the foreign key. We've got tests for foreign key values but I feel like the convoluted inheritance chain here as something to do with the issue.

[Simon Charette]

Alright so the underlying issue is the fact the prototype foreign key point to Prototype_Model's primary key which is a OneToOneField.

Somehow the related field chain is not followed along to determine the non-related internal type and prevents the database converters from correctly being chosen.

​https://github.com/django/django/blob/084573c7156530047bec2c19e732423fa9d0ec13/django/db/backends/sqlite3/operations.py#L210-L220

It looks like most of the places that relies on get_internal_type could be affected by that. Here's one example that works for one level relationship chains but not for multiple ones

​https://github.com/django/django/blob/084573c7156530047bec2c19e732423fa9d0ec13/django/db/backends/oracle/operations.py#L557

I think the solution here is to add a new get_target_internal_type method on ForeignKey and along these lines

def get_target_internal_type(self):
    if isinstance(self.target, ForeignKey):
        return self.target.get_target_internal_type()
    return self.target.get_internal_type()

And audit the get_internal_type() usages to determine if they need to be adjusted to call this method or not.

[Wayne Merry]

A PR has been made, but only for a test case that isolates this problem. See ​https://github.com/django/django/pull/10538

[Simon Charette]

A minimal reproduction test as of 084573c7156530047bec2c19e732423fa9d0ec13 is

diff --git a/tests/model_fields/test_uuid.py b/tests/model_fields/test_uuid.py
index bc1c8d5bc0..6b6af3ea7e 100644
--- a/tests/model_fields/test_uuid.py
+++ b/tests/model_fields/test_uuid.py
@@ -170,8 +170,12 @@ class TestAsPrimaryKey(TestCase):
         self.assertEqual(r.uuid_fk, u2)

     def test_two_level_foreign_keys(self):
+        gc = UUIDGrandchild()
         # exercises ForeignKey.get_db_prep_value()
-        UUIDGrandchild().save()
+        gc.save()
+        gc.refresh_from_db()
+        self.assertIsInstance(gc.uuidchild_ptr_id, uuid.UUID)

[Simon Charette]

A bit more diffing revealed that foreign keys to foreign keys was overlooked in #24343.

​https://github.com/django/django/pull/10541

[Ramiro Morales]

In 5e3463f6bcec818431f0e1f4649d6a5bd944c459:

Fixed #27595 -- Made ForeignKey.get_col() follow target chains.

Previously, foreign relationships were followed only one level deep which
prevents foreign keys to foreign keys from being resolved appropriately.
This was causing issues such as improper database value conversion for
UUIDField on SQLite because the resolved expression's output field's
internal type wasn't correct. Added tests to make sure unlikely foreign
reference cycles don't cause recursion errors.

Refs #24343.

Thanks oyooyo for the report and Wayne Merry for the investigation.