Opened 7 years ago

Closed 5 years ago

#28385 closed Bug (fixed)

deserializers ignore natural keys when primary key has a default value

Reported by: Daniel Knell Owned by: Hasan Ramezani
Component: Core (Serialization) Version: 1.11
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Tim Graham)

def build_instance(Model, data, db):
    """
    Build a model instance.
    If the model instance doesn't have a primary key and the model supports
    natural keys, try to retrieve it from the database.
    """
    obj = Model(**data)
    if (obj.pk is None and hasattr(Model, 'natural_key') and
            hasattr(Model._default_manager, 'get_by_natural_key')):
        natural_key = obj.natural_key()
        try:
            obj.pk = Model._default_manager.db_manager(db).get_by_natural_key(*natural_key).pk
        except Model.DoesNotExist:
            pass
    return obj

(source on GitHub)

this causes loaddata to fail on second attempt when using natural keys and uuid primary keys as pk is set.

class FooManager(models.Manager):
    def get_by_natural_key(self, name):
        return self.get(name=name)

class Foo(DateTimeMixin, models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    name = models.CharField(max_length=100, unique=True)

    objects = FooManager()

    def natural_key(self):
        return (self.name,)

checking the primary key was not actually set seems to fix things:

def build_instance(Model, data, db):
    obj = Model(**data)
    pk = data.get(Model._meta.pk.name, None)
    
    if (pk is None and hasattr(Model, 'natural_key') and
            hasattr(Model._default_manager, 'get_by_natural_key')):
        natural_key = obj.natural_key()
        try:
            obj.pk = Model._default_manager.db_manager(db).get_by_natural_key(*natural_key).pk
        except Model.DoesNotExist:
            pass
    return obj

import django.core.serializers.base

django.core.serializers.base.build_instance = build_instance

Change History (11)

comment:1 by Tim Graham, 7 years ago

Description: modified (diff)
Summary: deserialisers ignore natural keys when primary key has a default valuedeserializers ignore natural keys when primary key has a default value
Triage Stage: UnreviewedAccepted

comment:2 by Irindu Indeera , 7 years ago

Owner: changed from nobody to Irindu Indeera
Status: newassigned

comment:3 by Dmytryi Striletskyi, 7 years ago

Hello,

I cannot reproduce this bug, i tried few ways, but got no error during loaddata.
May my understanding is not clear, but how do we want to get natural_key if Foo model's primary key exists?

From build_instance docstring:

If the model instance doesn't have a primary key and the model supports natural keys, try to retrieve it from the database.

Clarify bug's space for me, please. I wanna resolve it. Thanks.

comment:4 by Daniel Knell, 7 years ago

When a models primary key has default set, and a fixture is loaded that does not have a pk attribute, then the serialisation code will refuse to update the existing instance and instead try to create a new record in the db.

When the primary key attribute has a default value - such as models using UUID primary keys - the model will fill the pk attribute with that at instantiation time, and the check in build_instance will see that default value and refuse to goto the database to fetch the existing instance.

This is against the expectations set in the docs, that when the model/manager implement the natural keys interfaces and the primary key is omitted, the framework will update the existing instance when deserialising.

comment:5 by Dmytryi Striletskyi, 7 years ago

UPD: False solution.

Last edited 7 years ago by Dmytryi Striletskyi (previous) (diff)

comment:6 by Irindu Indeera , 7 years ago

Can you please list the correct steps to reproduce this bug ? or please tell us how you did?

Last edited 7 years ago by Irindu Indeera (previous) (diff)

comment:7 by Daniel Knell, 7 years ago

define the above models then try and import the following fixture using loaddata.

- model: myapp.Foo
  fields:
    name: foobar

if you try and import a second time it will create a duplicate instance instead of using the existing one.

the monkey patch i provided in the original post will prevent that and make it behave as expected.

Last edited 7 years ago by Daniel Knell (previous) (diff)

comment:8 by Dmytryi Striletskyi, 7 years ago

Has patch: set

comment:9 by Markus Holtermann, 7 years ago

Patch needs improvement: set

comment:10 by Hasan Ramezani, 5 years ago

Owner: changed from Irindu Indeera to Hasan Ramezani
Patch needs improvement: unset

comment:11 by Tim Graham <timograham@…>, 5 years ago

Resolution: fixed
Status: assignedclosed

In dcd1025:

Fixed #28385 -- Fixed deserializing natural keys when primary key has a default value.

Co-Authored-By: Hasan Ramezani <hasan.r67@…>

Note: See TracTickets for help on using tickets.
Back to Top