Opened 2 years ago

Closed 18 months ago

#22583 closed New feature (fixed)

Allow per-database migrations in multidb configurations

Reported by: akaariai Owned by: nobody
Component: Migrations Version: master
Severity: Normal Keywords:
Cc: loic Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Currently a database router can be used to skip operations on given model. However, this approach doesn't work at all for RunPython or RunSQL commands - especially RunSQL has no notion of which models the SQL touches. For this reason there should be some way to specify which databases the migration should apply to. The simplest way seems to be addition of using kwarg to RunPython and RunSQL commands. For example:

operations = [
    RunSQL("create table foobar(id serial primary key);", using='default'),
    RunSQL("create table foobar(id integer primary key auto_increment) engine myisam;",
           using=('mysql_alias1', 'mysql_alias2'))

However the problem here is that this assumes the migrations know the used database aliases. This doesn't work for 3rd party app migrations. For these cases "for_vendor='mysql'/'postgresql'/..." argument would work better.

Still, there might be need to allow skipping whole apps for migations - say you have a 3rd party app you don't want to migrate on database other - my understanding is that one can only skip all model operations on given database, but there is no way to force skip of RunSQL and RunPython commands, too. If it would be possible to define that the app in whole shouldn't be migrated on database other, then there would be no problem when running migrate.

Change History (8)

comment:1 Changed 2 years ago by timo

  • Triage Stage changed from Unreviewed to Accepted
  • Type changed from Bug to New feature

comment:2 Changed 22 months ago by andrewgodwin

So, it's worth pointing out that RunPython can just switch on the value of schema_editor.connection if it wants to make decisions about the current database, so I'm not sure if I want to add extra logic to RunSQL? Especially a run-per-backend thing...

comment:3 Changed 22 months ago by andrewgodwin

  • Resolution set to wontfix
  • Status changed from new to closed

I'm going to WONTFIX this and suggest that people use RunPython instead to do any custom logic; they can either call RunSQL from inside that or, more easily, just use schema_editor.execute() if they wish to run raw SQL.

comment:4 Changed 20 months ago by lorden

I had this problem too and RunPython was not reliable enough. I created a package to run raw SQL on a specific connection similar to the example given in the ticket description: django-migrations-plus. Contributions are appreciated!

comment:5 Changed 18 months ago by loic

  • Has patch set
  • Resolution wontfix deleted
  • Status changed from closed to new

Reopening after discussing this on multiple occasions on IRC.

RunPython and RunSQL should query the database router like every other operations and hints can be used to assist the router in taking a decision. This is consistent with how db routing is performed in the other parts of the system (i.e. db_for_read and db_for_write):

class MyRouter(object):
    def allow_migrate(self, db, model, **hints):
        if 'target_db' in hints:
            return db == hints['target_db']
        return True

RunPython(forwards, hints={'target_db': 'default'})

This also allows completely opting out of migrations for any given db with:

class MyRouter(object):
    def allow_migrate(self, db, model, **hints):
        return db != 'excluded_db'


comment:6 Changed 18 months ago by loic

  • Cc loic added

comment:7 Changed 18 months ago by MarkusH

  • Triage Stage changed from Accepted to Ready for checkin

comment:8 Changed 18 months ago by Loic Bistuer <loic.bistuer@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 8f4877c89d0f28e289399fbb46357623a49e7eb0:

Fixed #22583 -- Allowed RunPython and RunSQL to provide hints to the db router.

Thanks Markus Holtermann and Tim Graham for the review.

Note: See TracTickets for help on using tickets.
Back to Top