Opened 6 years ago

Closed 8 months ago

#14296 closed Bug (needsinfo)

' test' failing for apps that access read-only databases

Reported by: kthhrv Owned by: nobody
Component: Testing framework Version: 1.2
Severity: Normal Keywords:
Cc: tomek@…, gregchapple1@…, Shai Berger Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


searching 'Testing framework' component i can't find a ticket for this issue so here it is.

I have an app that accesses two databases one of which is read-only, when i run ' test <app>' i get an insufficient privileges error while its trying to create/destroy the tables on the read-only database. This causes the test run to fail then and there.

all the models in the read-only app are marked managed=False.

we can't always have RW access to legacy databases.

is there a workaround for this issue? i'd love to start running tests on this app.


Change History (12)

comment:1 Changed 6 years ago by Luke Plant

What kind of workaround do you envisage?

If you are looking for read-write tests, the normal work-around for a read-only database is to create a separate, writeable, blank database and use that for your tests. You will need a different settings file, or a settings file that has some way of knowing to use a different DB when you are in test mode.

If you cannot create a new database, and cannot create any tables in your existing one, you would have to run on the live data. Assuming this database is being used for something, you are presumably not going to want to do any writes to the DB at all, which limits you to read-only tests on your actual live data. Is that what you are wanting? It is already possible - just use the TestCase and test runners provided by unittest rather than using the Django subclasses and, and your tests will connect to the normal database. You will miss out on some goodness from Django's TestCase, but not much, because most of it is to do with writing to databases. You can also instantiate your own Client for testing views, so you don't miss out on that.

comment:2 Changed 6 years ago by…

A simple hack that seems to work for this is to modify the action of setup_databases in django/test/

 def setup_databases(self, **kwargs):
        from django.db import connections
        old_names = []
        mirrors = []
        for alias in connections:
            connection = connections[alias]
+           # If the alias name is contained within this setting, 
+           # just skip round the loop.
+           # This causes all tests to occur against the aliased
+           # connection, and not create a test db.
+           try:
+               if alias in settings.RUN_TESTS_ON_LIVE_DB:
+                   continue
+           except:
+               pass

            # If the database is a test mirror, redirect it's connection
            # instead of creating a test database.
            if connection.settings_dict['TEST_MIRROR']:
                mirrors.append((alias, connection))
                mirror_alias = connection.settings_dict['TEST_MIRROR']
                connections._connections[alias] = connections[mirror_alias]

                old_names.append((connection, connection.settings_dict['NAME']))
                connection.creation.create_test_db(self.verbosity, autoclobber=not self.interactive)
        return old_names, mirrors

and then place the names of the connections inside the variable in your

    RUN_TESTS_ON_LIVE_DB = ['dbname',]

This seems to force the tests to run against the live database, write operations will simply fail due to permissions and one no longer needs complex alternative db setups.

Last edited 15 months ago by Tim Graham (previous) (diff)

comment:4 Changed 6 years ago by Russell Keith-Magee

Triage Stage: UnreviewedAccepted

This sort of thing makes my teeth itch, because it's *really* bad testing practice -- tests should be repeatable, and anything with live data, by definition, isn't guaranteed repeatable.

A more interesting, but related use case is when you don't have permission to create or destroy databases, but you do have access to a database that you can use for testing purposes. Since solving this use case essentially implies solving the use case for this ticket, I'll mark this accepted.

comment:5 Changed 6 years ago by Russell Keith-Magee

#14339 was a ticket requesting the alternate use case I described.

comment:6 Changed 6 years ago by Julien Phalip

Severity: Normal
Type: Bug

comment:7 Changed 5 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:8 Changed 5 years ago by Aymeric Augustin

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:9 Changed 3 years ago by Tomek Paczkowski

Cc: tomek@… added

comment:10 Changed 3 years ago by RK

What about if it were possible to specify in the database parameters that the test runner should create the test databases using a different database server config entirely? Eg:

    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'myproject',
        'HOST': 'maindb',
         # ... plus some other settings
    'readonly': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'myproject',
        'HOST': 'readonlydb',
        'TEST_DATABASE': 'default'
        # ... plus some other settings

This would trigger the test runner to create another test database on the "default" server, for use with anything during the test which tries to access the "readonly" database.

comment:11 Changed 3 years ago by Shai Berger

The Oracle backend supports a test parameter CREATE_DB that tells it not to create the (tablespace, in this context equivalent to database) for testing but rather use an existing one. It still tries to create tables, but I guess the managed=False setting would take care of that in this context.

I would like to make that parameter apply to all database backends.

comment:12 Changed 2 years ago by Greg Chapple

Cc: gregchapple1@… added

As of b7aa7c4ab4372d2b7994d252c8bc87f77dd217ae you can use the --keepdb parameter to use an existing database rather than creating a new one for testing. This sounds more or less like what shai is describing above. It will run any outstanding migrations, and flush the data, but ultimately the database will be preserved at the end of the run.

comment:13 Changed 8 months ago by Shai Berger

Cc: Shai Berger added
Resolution: needsinfo
Status: newclosed

6 years after a general workaround was suggested, and almost two years since the last comment, I think that anybody who wants improvement on this should specify exactly what they need and why.

In particular -- the combination of --keepdb and database routers disallowing migrations on the read-only database will stop any automatic schema changes, and using Luke's suggestions for test-cases should stop data changes as well.

Note: See TracTickets for help on using tickets.
Back to Top