Opened 2 years ago

Closed 3 weeks ago

Last modified 3 weeks ago

#27595 closed Bug (fixed)

Database converters are not run for related fields referencing related fields

Reported by: oyooyo Owned by:
Component: Database layer (models, ORM) Version: 1.10
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Sample models.py:

from django.db import models

import uuid

class Base_Model(models.Model):
	id = models.UUIDField(primary_key = True, default=uuid.uuid4, editable=False)
	# id = models.AutoField(primary_key = True)
	prototype = models.ForeignKey('Prototype_Model', blank=True, null=True)
	def __str__(self):
		return str(self.id)

class Prototype_Model(Base_Model):
	pass

Sample admin.py:

from django.contrib import admin

from .models import Prototype_Model

class Base_Model_Admin(admin.ModelAdmin):
	list_display = ['id', 'prototype']

admin.site.register(Prototype_Model, Base_Model_Admin)

Steps to reproduce:

  1. Create a Prototype_Model via the admin. Leave the "prototype" ForeignKey field empty.
  2. Create another Prototype_Model via the admin. For the "prototype" ForeignKey field, choose the first created Prototype_Model from the dropdown
  3. Switch to the Prototype_Model list view, it should show that the "prototype" ForeignKey field of the second Prototype_Model indeed references the first model
  4. Click on this second Prototype_Model in order to access the change/edit model form and look at the dropdown for the "prototype" field. The expected behaviour is that the dropdown should have the referenced first model automatically selected, but instead, nothing is automatically selected.

If an AutoField is used instead of a UUIDField as the primary key "id" field (see commented out line in Base_Model), the behaviour is as expected.

Change History (11)

comment:1 Changed 2 years ago by Tim Graham

Component: UncategorizedForms
Summary: In admin panel, ForeignKey to model subclass using UUID as primary key is not automatically "selected"ForeignKey to model subclass using UUID as primary key isn't populated in form's <select>
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

I can reproduce this at eb7fb565e6483637fc4ee251940b86db813485a0 on SQLite but not PostgreSQL -- I guess it's based on whether or not the database has a native UUID type. On SQLite, for example, the initial form data appears as {'prototype': '35bba53ceafc4df2b1abc110d5f51b2a'} but the UUID values in the form's <select> have dashes, so there's a mismatch. On PostgreSQL, the initial data is {'prototype': UUID('154e323f-cc57-4399-9023-58357ae9ce21')} and there's no issue.

comment:2 Changed 23 months ago by Sarthak Mehrish

Owner: changed from nobody to Sarthak Mehrish
Status: newassigned

comment:3 Changed 4 weeks ago by Wayne Merry

Owner: changed from Sarthak Mehrish to Wayne Merry

comment:4 Changed 4 weeks ago by Wayne Merry

A topic was posted to Django Developers regarding a UUID foreign key value representation when using sqllite3. See https://groups.google.com/forum/#!topic/django-developers/h46RAw67g_g

comment:5 Changed 4 weeks ago by Simon Charette

Component: FormsDatabase layer (models, ORM)

I feel like this isn't a form level issue from a bit of debugging locally.

I'm not sure what's going on here but somehow the database converters are not considered for the foreign key. We've got tests for foreign key values but I feel like the convoluted inheritance chain here as something to do with the issue.

comment:6 Changed 4 weeks ago by Simon Charette

Summary: ForeignKey to model subclass using UUID as primary key isn't populated in form's <select>Database converters are not run for related fields referencing related fields

Alright so the underlying issue is the fact the prototype foreign key point to Prototype_Model's primary key which is a OneToOneField.

Somehow the related field chain is not followed along to determine the non-related internal type and prevents the database converters from correctly being chosen.

https://github.com/django/django/blob/084573c7156530047bec2c19e732423fa9d0ec13/django/db/backends/sqlite3/operations.py#L210-L220

It looks like most of the places that relies on get_internal_type could be affected by that. Here's one example that works for one level relationship chains but not for multiple ones

https://github.com/django/django/blob/084573c7156530047bec2c19e732423fa9d0ec13/django/db/backends/oracle/operations.py#L557

I think the solution here is to add a new get_target_internal_type method on ForeignKey and along these lines

def get_target_internal_type(self):
    if isinstance(self.target, ForeignKey):
        return self.target.get_target_internal_type()
    return self.target.get_internal_type()

And audit the get_internal_type() usages to determine if they need to be adjusted to call this method or not.

comment:7 Changed 4 weeks ago by Wayne Merry

A PR has been made, but only for a test case that isolates this problem. See https://github.com/django/django/pull/10538

comment:8 Changed 4 weeks ago by Wayne Merry

Owner: Wayne Merry deleted
Status: assignednew

comment:9 Changed 4 weeks ago by Simon Charette

A minimal reproduction test as of 084573c7156530047bec2c19e732423fa9d0ec13 is

diff --git a/tests/model_fields/test_uuid.py b/tests/model_fields/test_uuid.py
index bc1c8d5bc0..6b6af3ea7e 100644
--- a/tests/model_fields/test_uuid.py
+++ b/tests/model_fields/test_uuid.py
@@ -170,8 +170,12 @@ class TestAsPrimaryKey(TestCase):
         self.assertEqual(r.uuid_fk, u2)

     def test_two_level_foreign_keys(self):
+        gc = UUIDGrandchild()
         # exercises ForeignKey.get_db_prep_value()
-        UUIDGrandchild().save()
+        gc.save()
+        gc.refresh_from_db()
+        self.assertIsInstance(gc.uuidchild_ptr_id, uuid.UUID)

comment:10 Changed 3 weeks ago by Simon Charette

Has patch: set

A bit more diffing revealed that foreign keys to foreign keys was overlooked in #24343.

https://github.com/django/django/pull/10541

Last edited 3 weeks ago by Simon Charette (previous) (diff)

comment:11 Changed 3 weeks ago by Ramiro Morales

Resolution: fixed
Status: newclosed

In 5e3463f6bcec818431f0e1f4649d6a5bd944c459:

Fixed #27595 -- Made ForeignKey.get_col() follow target chains.

Previously, foreign relationships were followed only one level deep which
prevents foreign keys to foreign keys from being resolved appropriately.
This was causing issues such as improper database value conversion for
UUIDField on SQLite because the resolved expression's output field's
internal type wasn't correct. Added tests to make sure unlikely foreign
reference cycles don't cause recursion errors.

Refs #24343.

Thanks oyooyo for the report and Wayne Merry for the investigation.

Last edited 3 weeks ago by Tim Graham (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top