Document how to set a unique value for new fields using migrations

[Michael Barr]

This ticket questions two items: the models.UUIDField documentation, and the action of a callable default on a model field for migrations.

The current ​documentation for a models.UUIDField states to have a default set to default=uuid.uuid4.
Assuming I originally had the following model:

from django.db import models
class TestUUID(models.Model):
    name = models.CharField(max_length=15)

Then I run a makemigrations and a migrate to create the model.

I then run:

from testapp.models import TestUUID
TestUUID.objects.create(name='Test 1')
TestUUID.objects.create(name='Test 2')
TestUUID.objects.create(name='Test 3')
TestUUID.objects.create(name='Test 4')
TestUUID.objects.create(name='Test 5')

I decide to add a models.UUIDField to the model:

from django.db import models
import uuid
class TestUUID(models.Model):
    name = models.CharField(max_length=15)
    uuid = models.UUIDField(default=uuid.uuid4, unique=True)

I run makemigrations, which generates:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import uuid


class Migration(migrations.Migration):

    dependencies = [
        ('testapp', '0001_initial'),
    ]

    operations = [
        migrations.AddField(
            model_name='testuuid',
            name='uuid',
            field=models.UUIDField(default=uuid.uuid4, unique=True, max_length=32),
            preserve_default=True,
        ),
    ]

When I run migrate, I get the following error:

  Applying testapp.0002_testuuid_uuid...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/core/management/base.py", line 442, in execute
    output = self.handle(*args, **options)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/core/management/commands/migrate.py", line 193, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/migrations/executor.py", line 68, in migrate
    self.apply_migration(migration, fake=fake)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/migrations/executor.py", line 102, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/migrations/operations/fields.py", line 51, in database_forwards
    field,
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/sqlite3/schema.py", line 167, in add_field
    self._remake_table(model, create_fields=[field])
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/sqlite3/schema.py", line 135, in _remake_table
    self.quote_name(model._meta.db_table),
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/schema.py", line 100, in execute
    cursor.execute(sql, params)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/michaelbarr/.virtualenvs/django_18_venv/src/django/django/db/backends/sqlite3/base.py", line 518, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: column uuid is not unique

I have seen the response to Automaticly created migration calls callable for default only once, which states:

I wouldn't consider this a bug as the default value is nothing that is enforced on database level but inside Django. If you want to have random values per row, you need to add a ​​RunPython operation that updates the value per row.

This is exactly as designed; when creating columns databases only take a single argument for the DEFAULT clause, so we can only provide a single value, so we only call a callable once. If you want unique values per row, you'll need to populate them in RunPython as Markush2010 suggests.

The new UUID field does not allow to automatically generate a UUID on save also discusses this issue, talking of the possibility of auto_add and auto_add_now.

It is my personal belief that if a developer sets the model field's default value to be a callable, it is meant to be handled by Python, not the database. In my instance above, I am stating that I want a new uuid.uuid4 to be created as the default. This works GREAT for forms and new saves, but not for the migrations since they are all the same uuid.uuid4(), as the migrations casts the default value to the called value. From what I can tell, I am not the only person that is confused by the defaults. In Python, the defaults work as expected, but database migrations do not yield the same results.

What I am attempting to do (eventually) is create an abstract model which is reused in multiple applications as a convention. However, this model has existed for quite some time, so each application already has multiple models created. I feel that the models.UUIDField should consider correcting the documentation or making a note of it, or reconsider how migration defaults are handled as a whole. I wrote about this as a comment here regarding how much work it is to work around this issue. If that is to be the case, so be it, but please be sure to make this clear in the documentation.

[Markus Holtermann]

As stated in #23408, the handling in migrations was a design decision that I still consider to be the right way to do. The normal cases are non-unique columns that are added with a default value equal for every existing row.

What I do agree with, is explaining how to set a default value for a unique field:

• AlterField / CreateField with unique=True
• RunPython / RunSQL to propagate existing rows
• AlterField to default=uuid.uuid4

[Shai Berger]

(nitpicking) A more general recipe would be:

• AlterField / CreateField with unique=False, null=True
• RunPython / RunSQL to propagate existing rows
• AlterField to null=False, unique=True and in the case of UUID default=uuid.uuid4

[Michael Barr]

Excellent points. Another talking point that may be helpful is how one might handle inheritance with migrations (such as creating a base migration). Assuming we have an abstract model that can be extended, what would be the suggested standard to handle the migration for new/existing models? It is difficult to know which fields already exist on the models, and which fields don't. Thus, it appears that each migration for each app would have to be hand-coded.

Thank you for your consideration, responses, and clarification.

[Tim Graham]

​PR

[Tim Graham <timograham@…>]

In 1f9e44030e9c5300b97ef7b029f482c53a66f13b:

Fixed #23932 -- Added how-to on migrating unique fields.

[Loic Bistuer <loic.bistuer@…>]

In 564487601e34b9962ad16bc7e4e62c8b68928c84:

[1.8.x] Fixed #23932 -- Added how-to on migrating unique fields.

Backport of 1f9e44030e9c5300b97ef7b029f482c53a66f13b from master