Error when running test with MySQL database

[luccascorrea]

I am running into this error when I try to test an app using a MySQL database. I get an IntegrityError(1215, 'Cannot add foreign key constraint')" while creating the tables.

What is happenning is this: I have a model called SyncLog which has a foreign key to another model called UserEmployee. The error happens when this foreign key is added.

I checked the history of queries run by Django while creating the table and the results are below shown. What I can see is that Django starts creating the SyncLog table before the UserEmployee table and, because that table does not exist yet, the creation fails. I would suppose that there is a bug when determining the dependencies between models in order to determine the order the tables should be created.

CREATE DATABASE `test_compras`
set autocommit=0
set autocommit=1
SET SQL_AUTO_IS_NULL = 0
SHOW FULL TABLES
CREATE TABLE `django_migrations` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `app` varchar(255) NOT NULL, `name` varchar(255) NOT NULL, `applied` datetime(6) NOT NULL)
SELECT engine FROM information_schema.tables WHERE table_name = \django_migrations\
SELECT `django_migrations`.`app`, `django_migrations`.`name` FROM `django_migrations`
SHOW FULL TABLES
set autocommit=0
CREATE TABLE `e89_syncing_synclog` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `user_id` integer NOT NULL, `timestamp` datetime(6) NOT NULL, `identifier` varchar(100) NOT NULL)
SELECT engine FROM information_schema.tables WHERE table_name = \e89_syncing_synclog\
CREATE TABLE `e89_tools_keyvaluestore` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `key` varchar(255) NOT NULL UNIQUE, `value` longtext NOT NULL)
SELECT engine FROM information_schema.tables WHERE table_name = \e89_tools_keyvaluestore\
CREATE TABLE `e89_push_messaging_device` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `owner_id` integer NOT NULL, `registration_id` varchar(200) NOT NULL, `platform` varchar(30) NOT NULL, UNIQUE (`registration_id`, `platform`))
SELECT engine FROM information_schema.tables WHERE table_name = \e89_push_messaging_device\
CREATE TABLE `e89_beta_testing_betatester` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `email` varchar(255) NOT NULL UNIQUE, `name` varchar(255) NOT NULL, `platform` varchar(30) NULL, `udid` varchar(255) NULL, `phone` varchar(20) NOT NULL, `token` varchar(50) NOT NULL)
SELECT engine FROM information_schema.tables WHERE table_name = \e89_beta_testing_betatester\
CREATE TABLE `e89_beta_testing_iphonebetaapp` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `plist_url` varchar(300) NULL, `ipa` varchar(100) NOT NULL, `plist` varchar(100) NOT NULL)
SELECT engine FROM information_schema.tables WHERE table_name = \e89_beta_testing_iphonebetaapp\
ALTER TABLE `e89_syncing_synclog` ADD CONSTRAINT `e89_syncing_user_id_670069ad1cf9eaae_fk_accounts_useremployee_id` FOREIGN KEY (`user_id`) REFERENCES `accounts_useremployee` (`id`)
rollback
set autocommit=1

The stacktrace I get is pasted below:

IntegrityError: (1215, 'Cannot add foreign key constraint')
  File "raven/contrib/django/management/__init__.py", line 41, in new_execute
  File "django/core/management/base.py", line 444, in execute
  File "django/core/management/commands/migrate.py", line 179, in handle
  File "django/core/management/commands/migrate.py", line 317, in sync_apps
  File "django/db/backends/utils.py", line 64, in execute
  File "django/db/utils.py", line 97, in __exit__
  File "django/db/backends/utils.py", line 62, in execute
  File "django/db/backends/mysql/base.py", line 124, in execute
  File "MySQLdb/cursors.py", line 205, in execute
  File "MySQLdb/connections.py", line 36, in defaulterrorhandler

I ran the same test with Django 1.7.9 and it worked fine.

I am using Sentry to monitor exceptions and I have their full stack trace with variable values and all. I will not paste it here because it's too long and kind of hard to read but I will attach it as a separate file because I believe it could help identifying the cause of this bug.

[luccascorrea]

Json with the stacktrace generated by Sentry. It contains variable values at the moment the exception occurred.

[Tim Graham]

Could you create a sample project with your models and migrations so we can easily reproduce the issue?

[zetab]

I maybe have same problem here.

Jesses-MacBook-Pro:cloudft asg$ python manage.py test main -v 3
Creating test database for alias 'default' ('test_db')...
Got an error creating the test database: (1007, "Can't create database 'test_db'; database exists")
Type 'yes' if you would like to try deleting the test database 'test_db', or 'no' to cancel: yes
Destroying old test database 'default'...
Operations to perform:
  Synchronize unmigrated apps: authtoken, corsheaders, staticfiles, debug_toolbar, messages, raven.contrib.django, humanize, debug_panel
  Apply all migrations: admin, contenttypes, main, auth, sessions
Synchronizing apps without migrations:
Running pre-migrate handlers for application main
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application sessions
Running pre-migrate handlers for application corsheaders
Running pre-migrate handlers for application debug_toolbar
Running pre-migrate handlers for application raven.contrib.django
Running pre-migrate handlers for application authtoken
  Creating tables...
    Creating table corsheaders_corsmodel
    Creating table authtoken_token
    Running deferred SQL...
Traceback (most recent call last):
  File "manage.py", line 8, in <module>
    execute_from_command_line(sys.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/test.py", line 30, in run_from_argv
    super(Command, self).run_from_argv(argv)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/test.py", line 74, in execute
    super(Command, self).execute(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/test.py", line 90, in handle
    failures = test_runner.run_tests(test_labels)
  File "/Library/Python/2.7/site-packages/django/test/runner.py", line 210, in run_tests
    old_config = self.setup_databases()
  File "/Library/Python/2.7/site-packages/django/test/runner.py", line 166, in setup_databases
    **kwargs
  File "/Library/Python/2.7/site-packages/django/test/runner.py", line 370, in setup_databases
    serialize=connection.settings_dict.get("TEST", {}).get("SERIALIZE", True),
  File "/Library/Python/2.7/site-packages/django/db/backends/base/creation.py", line 368, in create_test_db
    test_flush=not keepdb,
  File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
    created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/Library/Python/2.7/site-packages/django/core/management/commands/migrate.py", line 317, in sync_apps
    cursor.execute(statement)
  File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Library/Python/2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/Library/Python/2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/Library/Python/2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/Library/Python/2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.IntegrityError: (1215, 'Cannot add foreign key constraint')

If i open mysql server and write SHOW ENGINE INNODB STATUS\G,

it will print

LATEST FOREIGN KEY ERROR
------------------------
2015-08-31 17:36:23 12af59000 Error in foreign key constraint of table test_db/#sql-7d4_a:
 FOREIGN KEY (`user_id`) REFERENCES `main_userprofile` (`id`):
Cannot resolve table name close to:
 (`id`)

makemigrations, migrate and runserver works fine. Tests is the only problem here.

[Shai Berger]

@zetab: According to your log, your problem is that an unmigrated app (authtoken) depends (has a FK to) a migrated app (main). This is a well-known no-no.

Restoring the "needsinfo" resolution, waiting for more info from the OP.

[Tim Graham]

Yes, please see the 1.8 release notes: "An app without migrations with a ForeignKey to an app with migrations may now result in a foreign key constraint error when migrating the database or running tests. In Django 1.7, this could fail silently and result in a missing constraint. To resolve the error, add migrations to the app without them."

[zetab]

Replying to shaib:

@zetab: According to your log, your problem is that an unmigrated app (authtoken) depends (has a FK to) a migrated app (main). This is a well-known no-no.

Restoring the "needsinfo" resolution, waiting for more info from the OP.

Oh, sorry for not knowing. I commented out that authtoken and now everything works. So the problem is in rest_framework

#'rest_framework.authtoken',