test_permission_register_order raises IntegrityError when tests are run and INNODB storage engine is used for MySQL

[Jim Dalton]

When running tests using the trunk version of Django (rev [14992]) I get the following error when using the INNODB storage engine for MySQL:

$ ./manage.py test
Creating test database for alias 'default'...
...............................................................................E..............................................................................................................................................................................................................................
======================================================================
ERROR: test_permission_register_order (django.contrib.auth.tests.permissions.TestAuthPermissions)
Test that the order of registered permissions doesn't break
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jsdalton/webs/testproject/src/django/django/contrib/auth/tests/permissions.py", line 32, in test_permission_register_order
    create_permissions(auth_models, [], verbosity=0)
  File "/Users/jsdalton/webs/testproject/src/django/django/contrib/auth/management/__init__.py", line 51, in create_permissions
    content_type=ctype
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/manager.py", line 138, in create
    return self.get_query_set().create(**kwargs)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/query.py", line 360, in create
    obj.save(force_insert=True, using=self.db)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/base.py", line 458, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/base.py", line 551, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/query.py", line 1430, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/sql/compiler.py", line 791, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/models/sql/compiler.py", line 735, in execute_sql
    cursor.execute(sql, params)
  File "/Users/jsdalton/webs/testproject/src/django/django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/Users/jsdalton/webs/testproject/lib/python2.6/site-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/Users/jsdalton/webs/testproject/lib/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_ere_testproj`.`auth_permission`, CONSTRAINT `content_type_id_refs_id_728de91f` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`))')

----------------------------------------------------------------------
Ran 302 tests in 4.206s

FAILED (errors=1)
Destroying test database for alias 'default'...

This error is on a bare test project, on Mac OS X running MySQL 5.1.40. I am setting the storage engine via the OPTIONS key on the DATABASES setting as follows:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'testproj',                      # Or path to database file if using sqlite3.
        'USER': ****                      # Not used with sqlite3.
        'PASSWORD': ****                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
        'OPTIONS': {"init_command": "SET storage_engine=INNODB"},
    }
}

The tests run fine with no errors when the init_command specified in OPTIONS above is not used.

This isn't a huge deal for me; I primarily use INNODB in my development environment because of its transaction support (using the default MyISAM makes the test suite run intolerably slow). I can easily disregard the error here, and I have not experienced any other related issues in development.

Note that the error message and behavior appears to be quite similar to what was reported in #9207; however, changing the order of content_types and auth in INSTALLED_APPS does not resolve it.

[Jim Dalton]

I added a patch which clears the ContentTypes query cache during setup for this test. This resolves the error that was causing the test to fail when I ran tests.

There is a note in the test's tearDown() method that reads:

    def tearDown(self):
        # These tests mess with content types, but content type lookups
        # are cached, so we need to make sure the effects of this test
        # are cleaned up.
        contenttypes_models.ContentType.objects.clear_cache()

I also had observed that if I ran this test in isolation, i.e.:

$ manage.py test auth.TestAuthPermissions

Then the test did not raise the error. Based on this observation and the note in the tearDown() method above, I reasoned that the error was likely being caused by the fact that though the ContentType record were all being deleted, they were still sticking around in the cache. So I added the same call to clear_cache() that appears in the tearDown() method.

The tests no longer raise the error and execute fine.

[Russell Keith-Magee]

Confirmed; although the test doesn't fail by itself. You need to run it with another test that modifies content types.

./runtests.py --settings=mysql_innodb auth.BackendTest.test_custom_perms auth.TestAuthPermissions.test_permission_register_order

This is a failure in the test suite, so it's a 1.3 blocker.

[Russell Keith-Magee]

#14975 has a very similar set of circumstances, with a different presentation.

[Russell Keith-Magee]

(In [15192]) Fixed #14975, #14925 -- Added some cache flushing to avoid some cross-test effects. Thanks to jsdalton and rpbarlow for the reports.

[Russell Keith-Magee]

(In [15193]) [1.2.X] Fixed #14975, #14925 -- Added some cache flushing to avoid some cross-test effects. Thanks to jsdalton and rpbarlow for the reports.

Backport of r15192 from trunk.

[Jacob]

Milestone 1.3 deleted