Django doesn't initialize two databases with the same name, port and host

[Cesar Canassa]

When you have two or more databases with the same name, port and host the test suite will only initialize the first one.

Steps to reproduce:

Create a settings file with the following database configuration and run the test suite:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    }
}

When the test suit runs Django will only initialize the 'default' database.

Creating test database for alias 'default'...
Ran 4 tests in 4.030s

Workaround

A temporary workaround for this issues is changing the second database NAME or put anything in the HOST or PORT. That would be enough to mark the two databases as distinct from the perspective of the test setup
mechanism.

After changing it, this is the test suite output:

Creating test database for alias 'default'...
Creating test database for alias 'log'...
Ran 4 tests in 0.076s

Note that the test time has decreased when both databases are initialized. This is due to Django failing to initialize the 'log' database on the first run which makes the test suite run without transitions enabled.

For more information, see the topic in the mailing list:
​http://groups.google.com/group/django-users/browse_thread/thread/6542c80f1b0e482a

[Russell Keith-Magee]

To clarify - the fact that Django doesn't initialize the database twice is by design (and a direct response to previous bugs where duplicate initialization could cause problems on teardown) The complication in this case is the use of an :memory: database.

[Ramiro Morales]

I'm tempted to won't fix this, at least with code.

A settings file for sqlite3 like the one the OP reports is using, i.e. with multiple database definitions whose (all?) the 'NAMES' are ':memory:' makes no sense because I don't think any real world project would work with such a setup.

If you need to have two (or more) in-memory sqlite3 DBs for testing then you have two choices:

• Explicitly use the 'TEST_NAME' DATABASES inner option that, as ​documented, is the right one to use in this situation:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    }
}

• Simply don't use 'NAME' nor 'TEST_NAME' at all (just like we do with the test_sqlite.py settings we ship with our test suite.)

I'd not call the above workarounds but rather the correct ways to achieve what you are after. No need to mess with bogus HOST or PORT values.

[masj@…]

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

[Ramiro Morales]

Please don't hijack an existing ticket to report a different issue. Open a new one instead.

Replying to masj@…:

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

[Jacob]

Following ramiro's comment, I'm marking this wontfix. Thanks!

[Ramiro Morales]

#17762 reported this again, namely a setup with sqlite3 where all the NAME keys in DATABASES have ':memory:' as value, was closed as duplicate of this ticket.

[Ramiro Morales]

I'm reopening this because what I suggested in comment:2 (explicitly use the 'TEST_NAME' DATABASES inner option with a ':memory:' value) doesn't work. Sorry for the confusion.

[Ramiro Morales]

Patch for the isue, includes tests

[Ramiro Morales]

In [17702]:

Fixed #16329 -- Fixed detection of transaction-handling capabilities when all test databases are sqlite3, in-memory.

Thanks canassa for the report and agriffis (#17762) and lrekucki (in #17758) for their contribution to the fix.

[Aymeric Augustin]

There's some interference / state leakage between the test introduced in r17702 and ProxyModelInheritanceTests.

The failure can be seen ​on the CI server.

It can be reproduced with: runtests.py --settings=test_sqlite proxy_model_inheritance test_runner

[Aymeric Augustin]

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.