Multi-database tests fail in ContentType

[def@…]

I have a multi-database setup:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'wall2',
        'HOST': 'localhost',
        'PORT': '',
    },

    'slave': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'wall2',
        'HOST': 'localhost',
        'PORT': '',
        'TEST_MIRROR': 'default',
    },

    'station': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'station',
        'HOST': 'localhost',
        'PORT': '',
        'TEST_DEPENDENCIES': ['default',]
    },

    'catalog': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'catalog',
        'HOST': 'localhost',
        'PORT': '',
        'TEST_DEPENDENCIES': ['default', 'station']
    },
}

I have 2 routers, one handles the master/slave and one handles the 2 "data" database station and catalog. they run fine on production, but when we try to run our unit tests i get the following stack trace:

Traceback (most recent call last):
  File "./27.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/site-packages/south/management/commands/test.py", line 8, in handle
    super(Command, self).handle(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/django/core/management/commands/test.py", line 37, in handle
    failures = test_runner.run_tests(test_labels)
  File "/usr/lib/python2.7/site-packages/django/test/simple.py", line 396, in run_tests
    old_config = self.setup_databases()
  File "/usr/lib/python2.7/site-packages/django/test/simple.py", line 334, in setup_databases
    test_db_name = connection.creation.create_test_db(self.verbosity, autoclobber=not self.interactive)
  File "/usr/lib/python2.7/site-packages/django/db/backends/creation.py", line 357, in create_test_db
    load_initial_data=False)
  File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 166, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/usr/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 107, in handle_noargs
    emit_post_sync_signal(created_models, verbosity, interactive, db)
  File "/usr/lib/python2.7/site-packages/django/core/management/sql.py", line 182, in emit_post_sync_signal
    interactive=interactive, db=db)
  File "/usr/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 172, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/lib/python2.7/site-packages/django/contrib/contenttypes/management.py", line 25, in update_contenttypes
    ct.save()
  File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 458, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 551, in save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 195, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 1524, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 788, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "/usr/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 732, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.IntegrityError: (1062, "Duplicate entry 'tools-station' for key 'app_label'")

after some debugging i've found that the

    def update_contenttypes(app, created_models, verbosity=2, **kwargs):

method looks up the tools-station content type in the station database, doesn't find it, and then tries to write it to the default database, where it already exists.

by simply adding "using('default')" to the two ContentType.objects calls in the above method everything works fine. i'm not sure if all content types are meant to be in the 'default' database, but that seems to be the only way it can work at this point?

i'll attach a simple patch,

[Aymeric Augustin]

Either your database routers allow reading ContentType objects in the 'station' database (which they shouldn't), or they are bypassed somehow.

Could you post the code of your routers?

[Aymeric Augustin]

And, obviously, re-open the ticket - thanks :)

[def@…]

db router

[def@…]

last router in chain

[def@…]

i have added a test for ContentType and the first router returns 'None' and the final router returns slave on calls to db_for_read for ContentType.

[Bas Peschier]

Seems like this is not the only ticket about multi-db contenttype problems, could you test whether ticket #16281 solves your problem?

[Łukasz Rekucki]

Fixed formatting (please use preview!).

[Aymeric Augustin]

I tried and failed to reproduce this bug. We'll need the original reporter's feedback to determine if it's a duplicate of #16281.

While investigating, I stumbled upon another problem and I created #16353.

[anonymous]

well, i'm testing with 1.2.5, so i had to edit the patch as follows, but it didnt' work for me:

        #return self.model_class()._default_manager.using(self._state.db).get(**kwargs)
        return self.model_class()._default_manager.using(self.model_class().objects.db).get(**kwargs)

but still got the same error:

django.db.utils.IntegrityError: (1062, "Duplicate entry 'tools-station' for key 'app_label'")

[Aymeric Augustin]

If someone can provide a way to reproduce this on 1.3, please reopen the ticket.

Only critical fixes are backported to the 1.2 branch, and this problem is certainly not critical.