Code

Opened 3 years ago

Closed 2 years ago

Last modified 17 months ago

#16329 closed Bug (fixed)

Django doesn't initialize two databases with the same name, port and host

Reported by: canassa Owned by: nobody
Component: Testing framework Version: 1.3
Severity: Normal Keywords:
Cc: aron@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When you have two or more databases with the same name, port and host the test suite will only initialize the first one.

Steps to reproduce:

Create a settings file with the following database configuration and run the test suite:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
    }
}

When the test suit runs Django will only initialize the 'default' database.

Creating test database for alias 'default'...
Ran 4 tests in 4.030s

Workaround

A temporary workaround for this issues is changing the second database NAME or put anything in the HOST or PORT. That would be enough to mark the two databases as distinct from the perspective of the test setup
mechanism.

After changing it, this is the test suite output:

Creating test database for alias 'default'...
Creating test database for alias 'log'...
Ran 4 tests in 0.076s

Note that the test time has decreased when both databases are initialized. This is due to Django failing to initialize the 'log' database on the first run which makes the test suite run without transitions enabled.

For more information, see the topic in the mailing list:
http://groups.google.com/group/django-users/browse_thread/thread/6542c80f1b0e482a

Attachments (1)

16329.3.diff (3.4 KB) - added by ramiro 2 years ago.
Patch for the isue, includes tests

Download all attachments as: .zip

Change History (14)

comment:1 Changed 3 years ago by russellm

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

To clarify - the fact that Django doesn't initialize the database twice is by design (and a direct response to previous bugs where duplicate initialization could cause problems on teardown) The complication in this case is the use of an :memory: database.

comment:2 Changed 3 years ago by ramiro

  • Triage Stage changed from Unreviewed to Design decision needed

I'm tempted to won't fix this, at least with code.

A settings file for sqlite3 databases whose (all?) the 'NAMES' are ':memory:' makes no sense because I don't think any real world project would work with such a setup.

If you need to have two (or more) in-memory sqlite3 DBs for testing then you have two choices:

  • Explicitly use the 'TEST_NAME' DATABASES inner option that, as documented, is the right one to use in this situation:
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    },

    'log': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ...,
        'TEST_NAME': ':memory:',
    }
}
  • Simply don't use 'NAME' nor 'TEST_NAME' at all (just like we do with the test_sqlite.py settings we ship with our test suite.)

I'd not call the above workarounds but rather the correct ways to achieve what you are after. No need to mess with bogus HOST or PORT values.

Version 1, edited 3 years ago by ramiro (previous) (next) (diff)

comment:3 follow-up: Changed 3 years ago by masj@…

  • Version changed from 1.3 to SVN

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

comment:4 in reply to: ↑ 3 Changed 3 years ago by ramiro

  • Version changed from SVN to 1.3

Please don't hijack an existing ticket to report a different issue. Open a new one instead.

Replying to masj@…:

I've faced the same problem while running a test on the latest SVN.

A slave database with a 'TEST_MIRROR':'default' will create a dummy connection.

What happens is that in django.test.testcases:

def connections_support_transactions():
    """
    Returns True if all connections support transactions.
    """
    return all(conn.features.supports_transactions for conn in connections.all())

The above code looks for features.supports_transactions in each connection. However, since the slave connection is a dummy connection that just mirrors default, .supports_transactions for the slave connection returns nothing.

This is causing tests to run without transaction support. Ideally what connections_support_transactions() should do is resolve the dummy connection to the correct database and then check the features of the real connection.

comment:5 Changed 3 years ago by jacob

  • Resolution set to wontfix
  • Status changed from new to closed

Following ramiro's comment, I'm marking this wontfix. Thanks!

comment:6 Changed 2 years ago by ramiro

#17762 reported this again, namely a setup with sqlite3 where all the NAME keys in DATABASES have ':memory:' as value, was closed as duplicate of this ticket.

Last edited 2 years ago by ramiro (previous) (diff)

comment:7 Changed 2 years ago by ramiro

  • Resolution wontfix deleted
  • Status changed from closed to reopened

I'm reopening this because what I suggested in comment:2 (explicitly use the 'TEST_NAME' DATABASES inner option with a ':memory:' value) doesn't work. Sorry for the confusion.

Last edited 17 months ago by ramiro (previous) (diff)

comment:8 Changed 2 years ago by agriffis

  • Cc aron@… added

Changed 2 years ago by ramiro

Patch for the isue, includes tests

comment:9 Changed 2 years ago by ramiro

  • Has patch set
  • Triage Stage changed from Design decision needed to Accepted

comment:10 Changed 2 years ago by ramiro

  • Resolution set to fixed
  • Status changed from reopened to closed

In [17702]:

Fixed #16329 -- Fixed detection of transaction-handling capabilities when all test databases are sqlite3, in-memory.

Thanks canassa for the report and agriffis (#17762) and lrekucki (in #17758) for their contribution to the fix.

comment:11 Changed 2 years ago by aaugustin

  • Resolution fixed deleted
  • Severity changed from Normal to Release blocker
  • Status changed from closed to reopened

There's some interference / state leakage between the test introduced in r17702 and ProxyModelInheritanceTests.

The failure can be seen on the CI server.

It can be reproduced with: runtests.py --settings=test_sqlite proxy_model_inheritance test_runner

comment:12 Changed 2 years ago by aaugustin

In [17712]:

Added cleanup code to the proxy_model_inheritance tests. Refs #12286, #16329.

comment:13 Changed 2 years ago by aaugustin

  • Resolution set to fixed
  • Severity changed from Release blocker to Normal
  • Status changed from reopened to closed

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.