SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

[ezaquarii]

SQLite table alteration uses rename and copy method. When running a database migration from Django command (via call_command('migrate')), database is left with references to <table_name>__old table that has been dropped after migration.

This happens only when running SQLite DB migrations from management command under transaction.
Running migrations via ./manage.py migrate does not cause any issues.

This is the demonstration:
​https://github.com/ezaquarii/django-sqlite-migration-bug

Steps to reproduce the bug

1. run ./bootstrap.sh
2. from import app.models import *
3. Bravo.objects.create()
4. SQL cannot be executed due to missing table main.app_alpha__old

• app/management/commands/bootstrap.py:9: run the migration
• alpha table is created
• bravo table is created; it uses foreign key to alpha
• another alpha migration is run, adding dummy field
• alpha table is renamed to alpha__old
• bravo foreign key is re-linked to alpha__old
• new table alpha is created and data from alpha__old is copied with applied alterations
• alpha__old is dropped
• bravo still references alpha__old - ForeignKey constraint is not updated and DB is left in unusable state

CREATE TABLE "app_bravo" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "dummy" varchar(10) NOT NULL, "alpha_id" integer NOT NULL REFERENCES "app_alpha__old" ("id") DEFERRABLE INITIALLY DEFERRED);

Tested with Django versions: 2.0.1 and 2.0.2

[ezaquarii]

Bug reproduction steps as shell script

[ezaquarii]

Database schema after running migrations - see references to temporary openvpn_serverold table

[Tim Graham]

Could you provide a more minimal project that reproduces the issue? In particular, the steps to reproduce shouldn't require installing a bunch of NodeJS packages. Thanks!

[ezaquarii]

Please check this repository:

​https://github.com/ezaquarii/django-sqlite-migration-bug

Please check out bootstrap.py

        with transaction.atomic():
            call_command('migrate')

I think this is the issue. It works ok without transaction.