Opened 13 years ago

Closed 12 years ago

Last modified 11 years ago

#16329 closed Bug (fixed)

Django doesn't initialize two databases with the same name, port and host

Reported by: Cesar Canassa Owned by: nobody
Component: Testing framework Version: 1.3
Severity: Normal Keywords:
Cc: aron@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When you have two or more databases with the same name, port and host the test suite will only initialize the first one.

Steps to reproduce:

Create a settings file with the following database configuration and run the test suite:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    }
}

When the test suit runs Django will only initialize the 'default' database.

Creating test database for alias 'default'...
Ran 4 tests in 4.030s

Workaround

A temporary workaround for this issues is changing the second database NAME or put anything in the HOST or PORT. That would be enough to mark the two databases as distinct from the perspective of the test setup
mechanism.

After changing it, this is the test suite output:

Creating test database for alias 'default'...
Creating test database for alias 'log'...
Ran 4 tests in 0.076s

Note that the test time has decreased when both databases are initialized. This is due to Django failing to initialize the 'log' database on the first run which makes the test suite run without transitions enabled.

For more information, see the topic in the mailing list:
http://groups.google.com/group/django-users/browse_thread/thread/6542c80f1b0e482a

Attachments (1)

16329.3.diff (3.4 KB ) - added by Ramiro Morales 12 years ago.
Patch for the isue, includes tests

Download all attachments as: .zip

Change History (14)

comment:1 by Russell Keith-Magee, 13 years ago

To clarify - the fact that Django doesn't initialize the database twice is by design (and a direct response to previous bugs where duplicate initialization could cause problems on teardown) The complication in this case is the use of an :memory: database.

comment:2 by Ramiro Morales, 13 years ago

Triage Stage: UnreviewedDesign decision needed

I'm tempted to won't fix this, at least with code.

A settings file for sqlite3 like the one the OP reports is using, i.e. with multiple database definitions whose (all?) the 'NAMES' are ':memory:' makes no sense because I don't think any real world project would work with such a setup.

If you need to have two (or more) in-memory sqlite3 DBs for testing then you have two choices:

  • Explicitly use the 'TEST_NAME' DATABASES inner option that, as documented, is the right one to use in this situation:
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    }
}
  • Simply don't use 'NAME' nor 'TEST_NAME' at all (just like we do with the test_sqlite.py settings we ship with our test suite.)

I'd not call the above workarounds but rather the correct ways to achieve what you are after. No need to mess with bogus HOST or PORT values.

Last edited 13 years ago by Ramiro Morales (previous) (diff)

comment:3 by masj@…, 13 years ago

Version: 1.3SVN

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

in reply to:  3 comment:4 by Ramiro Morales, 13 years ago

Version: SVN1.3

Please don't hijack an existing ticket to report a different issue. Open a new one instead.

Replying to masj@…:

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

comment:5 by Jacob, 13 years ago

Resolution: wontfix
Status: newclosed

Following ramiro's comment, I'm marking this wontfix. Thanks!

comment:6 by Ramiro Morales, 12 years ago

#17762 reported this again, namely a setup with sqlite3 where all the NAME keys in DATABASES have ':memory:' as value, was closed as duplicate of this ticket.

Last edited 12 years ago by Ramiro Morales (previous) (diff)

comment:7 by Ramiro Morales, 12 years ago

Resolution: wontfix
Status: closedreopened

I'm reopening this because what I suggested in comment:2 (explicitly use the 'TEST_NAME' DATABASES inner option with a ':memory:' value) doesn't work. Sorry for the confusion.

Last edited 11 years ago by Ramiro Morales (previous) (diff)

comment:8 by Aron Grififs, 12 years ago

Cc: aron@… added

by Ramiro Morales, 12 years ago

Attachment: 16329.3.diff added

Patch for the isue, includes tests

comment:9 by Ramiro Morales, 12 years ago

Has patch: set
Triage Stage: Design decision neededAccepted

comment:10 by Ramiro Morales, 12 years ago

Resolution: fixed
Status: reopenedclosed

In [17702]:

Fixed #16329 -- Fixed detection of transaction-handling capabilities when all test databases are sqlite3, in-memory.

Thanks canassa for the report and agriffis (#17762) and lrekucki (in #17758) for their contribution to the fix.

comment:11 by Aymeric Augustin, 12 years ago

Resolution: fixed
Severity: NormalRelease blocker
Status: closedreopened

There's some interference / state leakage between the test introduced in r17702 and ProxyModelInheritanceTests.

The failure can be seen on the CI server.

It can be reproduced with: runtests.py --settings=test_sqlite proxy_model_inheritance test_runner

comment:12 by Aymeric Augustin, 12 years ago

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.

comment:13 by Aymeric Augustin, 12 years ago

Resolution: fixed
Severity: Release blockerNormal
Status: reopenedclosed
Note: See TracTickets for help on using tickets.
Back to Top