Opened 4 months ago

Last modified 4 months ago

#28385 assigned Bug

deserializers ignore natural keys when primary key has a default value

Reported by: Daniel Knell Owned by: Irindu Indeera
Component: Core (Serialization) Version: 1.11
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: yes UI/UX: no

Description (last modified by Tim Graham)

def build_instance(Model, data, db):
    """
    Build a model instance.
    If the model instance doesn't have a primary key and the model supports
    natural keys, try to retrieve it from the database.
    """
    obj = Model(**data)
    if (obj.pk is None and hasattr(Model, 'natural_key') and
            hasattr(Model._default_manager, 'get_by_natural_key')):
        natural_key = obj.natural_key()
        try:
            obj.pk = Model._default_manager.db_manager(db).get_by_natural_key(*natural_key).pk
        except Model.DoesNotExist:
            pass
    return obj

(source on GitHub)

this causes loaddata to fail on second attempt when using natural keys and uuid primary keys as pk is set.

class FooManager(models.Manager):
    def get_by_natural_key(self, name):
        return self.get(name=name)

class Foo(DateTimeMixin, models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    name = models.CharField(max_length=100, unique=True)

    objects = FooManager()

    def natural_key(self):
        return (self.name,)

checking the primary key was not actually set seems to fix things:

def build_instance(Model, data, db):
    obj = Model(**data)
    pk = data.get(Model._meta.pk.name, None)
    
    if (pk is None and hasattr(Model, 'natural_key') and
            hasattr(Model._default_manager, 'get_by_natural_key')):
        natural_key = obj.natural_key()
        try:
            obj.pk = Model._default_manager.db_manager(db).get_by_natural_key(*natural_key).pk
        except Model.DoesNotExist:
            pass
    return obj

import django.core.serializers.base

django.core.serializers.base.build_instance = build_instance

Change History (9)

comment:1 Changed 4 months ago by Tim Graham

Description: modified (diff)
Summary: deserialisers ignore natural keys when primary key has a default valuedeserializers ignore natural keys when primary key has a default value
Triage Stage: UnreviewedAccepted

comment:2 Changed 4 months ago by Irindu Indeera

Owner: changed from nobody to Irindu Indeera
Status: newassigned

comment:3 Changed 4 months ago by Dmytryi Striletskyi

Hello,

I cannot reproduce this bug, i tried few ways, but got no error during loaddata.
May my understanding is not clear, but how do we want to get natural_key if Foo model's primary key exists?

From build_instance docstring:

If the model instance doesn't have a primary key and the model supports natural keys, try to retrieve it from the database.

Clarify bug's space for me, please. I wanna resolve it. Thanks.

comment:4 Changed 4 months ago by Daniel Knell

When a models primary key has default set, and a fixture is loaded that does not have a pk attribute, then the serialisation code will refuse to update the existing instance and instead try to create a new record in the db.

When the primary key attribute has a default value - such as models using UUID primary keys - the model will fill the pk attribute with that at instantiation time, and the check in build_instance will see that default value and refuse to goto the database to fetch the existing instance.

This is against the expectations set in the docs, that when the model/manager implement the natural keys interfaces and the primary key is omitted, the framework will update the existing instance when deserialising.

comment:5 Changed 4 months ago by Dmytryi Striletskyi

UPD: False solution.

Last edited 4 months ago by Dmytryi Striletskyi (previous) (diff)

comment:6 Changed 4 months ago by Irindu Indeera

Can you please list the correct steps to reproduce this bug ? or please tell us how you did?

Last edited 4 months ago by Irindu Indeera (previous) (diff)

comment:7 Changed 4 months ago by Daniel Knell

define the above models then try and import the following fixture using loaddata.

- model: myapp.Foo
  fields:
    name: foobar

if you try and import a second time it will create a duplicate instance instead of using the existing one.

the monkey patch i provided in the original post will prevent that and make it behave as expected.

Last edited 4 months ago by Daniel Knell (previous) (diff)

comment:8 Changed 4 months ago by Dmytryi Striletskyi

Has patch: set

comment:9 Changed 4 months ago by Markus Holtermann

Patch needs improvement: set
Note: See TracTickets for help on using tickets.
Back to Top