Race condition like error in migration

[Martin Häcker]

Hi there,

I'm having a strange error in some migrations. Let me describe:

At first I had several migrations, one renamed a model process.ProcessDef to process.ProcessDefinition, then a later one would rename a field on that model.

        migrations.RenameModel(
            old_name='ProcInstance',
            new_name='ProcessInstance',
        ),
        migrations.RenameField(
            model_name='processinstance',
            old_name='status',
            new_name='runstatus',
        ),

This creates a stack trace that looks like this:

  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/core/management/base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 221, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/migrations/executor.py", line 147, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/migrations/operations/fields.py", line 275, in database_forwards
    to_model._meta.get_field(self.new_name),
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 484, in alter_field
    old_db_params, new_db_params, strict)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/backends/sqlite3/schema.py", line 200, in _alter_field
    self._remake_table(model, alter_fields=[(old_field, new_field)])
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/backends/sqlite3/schema.py", line 137, in _remake_table
    self.create_model(temp_model)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 232, in create_model
    definition, extra_params = self.column_sql(model, field)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 131, in column_sql
    db_params = field.db_parameters(connection=self.connection)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1989, in db_parameters
    return {"type": self.db_type(connection), "check": []}
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1980, in db_type
    rel_field = self.related_field
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1886, in related_field
    return self.foreign_related_fields[0]
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1620, in foreign_related_fields
    return tuple(rhs_field for lhs_field, rhs_field in self.related_fields)
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1607, in related_fields
    self._related_fields = self.resolve_related_fields()
  File "/Users/dwt/.virtualenvs/pycess/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1592, in resolve_related_fields
    raise ValueError('Related model %r cannot be resolved' % self.rel.to)
ValueError: Related model 'process.ProcessDef' cannot be resolved

That is, it seems as if the internal migration logic is missing the fact that it has just renamed a model, and is trying to look up a relation by it's old un-renamed name - and then failing.

I have tried to reduce this to one squashed migration, but I'm not sure I entirely succeeded, as the bug seems to be transient with this migration, i.e. I get it in about 80-90% of cases, and removing more from it which *should* be unrelated seems to bring this ratio down quite drastically.

So to sum it up, it feels like a race condition.

Anyway, here it is:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):

    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
    ]

    operations = [
        migrations.CreateModel(
            name='FieldDef',
            fields=[
                ('id', models.AutoField(primary_key=True, verbose_name='ID', serialize=False, auto_created=True)),
                ('name', models.CharField(max_length=200)),
                ('descript', models.CharField(max_length=200)),
                ('fieldhelp', models.CharField(max_length=200)),
                ('fieldtype', models.PositiveSmallIntegerField()),
                ('length', models.PositiveSmallIntegerField()),
                ('editable', models.NullBooleanField()),
                ('must', models.NullBooleanField()),
                ('type', models.PositiveSmallIntegerField()),
                ('parent', models.ForeignKey(to='process.FieldDef')),
            ],
        ),
        migrations.CreateModel(
            name='ProcessDef',
            fields=[
                ('id', models.AutoField(primary_key=True, verbose_name='ID', serialize=False, auto_created=True)),
                ('name', models.CharField(max_length=200)),
                ('descript', models.CharField(max_length=200)),
                ('status', models.PositiveSmallIntegerField()),
                ('version', models.PositiveSmallIntegerField()),
                ('refering', models.ForeignKey(to='process.ProcessDef')),
            ],
        ),
        migrations.CreateModel(
            name='ProcInstance',
            fields=[
                ('id', models.AutoField(primary_key=True, verbose_name='ID', serialize=False, auto_created=True)),
                ('starttime', models.DateTimeField()),
                ('stoptime', models.DateTimeField()),
                ('status', models.PositiveSmallIntegerField()),
                ('process', models.ForeignKey(to='process.ProcessDef')),
            ],
        ),
        migrations.AlterField(
            model_name='processdef',
            name='refering',
            field=models.ForeignKey(to='process.ProcessDef', null=True),
        ),
        migrations.AlterField(
            model_name='processdef',
            name='refering',
            field=models.ForeignKey(to='process.ProcessDef', blank=True, null=True),
        ),
        migrations.RenameModel(
            old_name='ProcessDef',
            new_name='ProcessDefinition',
        ),
        migrations.RenameModel(
            old_name='ProcInstance',
            new_name='ProcessInstance',
        ),
        migrations.RenameField(
            model_name='processinstance',
            old_name='status',
            new_name='runstatus',
        ),
    ]

You can see the project where this is from at ​https://github.com/pycess/pycess for reference. There are some old bugs in the bug database that look kind of similar (e.g. #22319) but not quite, which is why I am filing this new ticket.

[Tim Graham]

I couldn't reproduce the error using the provided migration. I ran python manage.py migrate and python manage.py migrate process zero several times in succession without an error. If you could provide a project we could download to reproduce the error, please reopen. Thanks!