Serialization of natural foreign key in migration scripts does not work

[Bowen Song]

The serializers.serialize(..., use_natural_foreign_keys=True) is not working in migration scripts, the ManyToManyField always have the value of the pk instead of the natural_key

An example to reproduce this bug:

models.py

# -*- coding: utf-8 -*-
from django.db import models


class Student(models.Model):
    student_id = models.CharField(max_length=10, unique=True)
    name = models.CharField(max_length=30)

    def __unicode__(self):
        return self.name

    def natural_key(self):
        return self.student_id


class Teacher(models.Model):
    name = models.CharField(max_length=30)
    students = models.ManyToManyField(Student)

    def __unicode__(self):
        return self.name

migrations/0002_example.py

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import migrations, models
from django.core import serializers


def example(apps, schema_editor):
    Teacher = apps.get_model("mytest", "Teacher")
    json_data = serializers.serialize('json', Teacher.objects.all(), use_natural_foreign_keys=True)
    print json_data


class Migration(migrations.Migration):

    dependencies = [
        ('mytest', '0001_initial'),
    ]

    operations = [
        migrations.RunPython(example, reverse_code=migrations.RunPython.noop)
    ]

I applied the initial migration, and inserted some data into those tables.

The dumpdata command output is:

$ python manage.py dumpdata mytest --natural-foreign 
[{"fields": {"student_id": "1234567890", "name": "Alice"}, "model": "mytest.student", "pk": 1}, {"fields": {"student_id": "9876543210", "name": "Bob"}, "model": "mytest.student", "pk": 2}, {"fields": {"students": ["1234567890", "9876543210"], "name": "Petter"}, "model": "mytest.teacher", "pk": 1}]

When running the migrate command, output is:

$ python manage.py migrate mytest 0002
Operations to perform:
  Target specific migration: 0002_example, from mytest
Running migrations:
  Rendering model states... DONE
  Applying mytest.0002_example...[{"fields": {"students": [1, 2], "name": "Petter"}, "model": "mytest.teacher", "pk": 1}]
 OK

Expected output is:

$ python manage.py migrate mytest 0002
Operations to perform:
  Target specific migration: 0002_example, from mytest
Running migrations:
  Rendering model states... DONE
  Applying mytest.0002_example...[{"fields": {"students": ["1234567890", "9876543210"], "name": "Petter"}, "model": "mytest.teacher", "pk": 1}]
 OK

[Tim Graham]

I think this is expected behavior because the natural_key() method isn't available in migrations. Per ​the docs, "Because it’s impossible to serialize arbitrary Python code, these historical models will not have any custom methods that you have defined."

[Tim Graham]

I thought about calling this out in the docs, but I'm not convinced that doing serialization in migrations is a common use case or something that we should promote (see #24778 for discussion).

[Bowen Song]

OK, my problem was because I'm using the elasticsearch (​https://www.elastic.co/products/elasticsearch), and elasticsearch uses JSON format.
In the normal use case, elasticsearch being updated by the post_save signal, but in the event of data migration, the post_save function is not being called. In that case, I have to do the elasticsearch update in the migration script, and this didn't work because the natural key issue, I got wrong (unexpected) data in the JSON.
If you are not going to fix it, would you able to provide any suggestions about how to do this in a correct way?

[Tim Graham]

Maybe you can define the natural_key() method in the migration and then add it to the model retrieved from the apps registry:

def natural_key(self):
    ...

Teacher = apps.get_model("mytest", "Teacher")
Teacher.natural_key = natural_key

[Bowen Song]

Replying to timgraham:

Maybe you can define the natural_key() method in the migration and then add it to the model retrieved from the apps registry:

def natural_key(self):
    ...

Teacher = apps.get_model("mytest", "Teacher")
Teacher.natural_key = natural_key

Thanks, this works perfect.