Migrations that belong to apps that are always routed to different databases can be shortcut to be "faked" earlier

[Suzannah Cooper]

One usage of routers is to be able to say which database tables within a Django app should be routed to. Within the application I am working on, we have an implementation of allow_migrate which takes in the app_label and database and always returns False if it is for an app_label that should go to a different database.

I've been working within a large Django codebase, which has many migrations (that should have been squashed, but for whatever reason have not been squashed yet). We have been setting up another database. Django requires that all the migrations that are routed to the default database also be applied on this new database. Given that those old migrations should never be routed, we expected these to only be no-ops and performing an insert to show that the migration has been marked as completed.

When profiling locally with a sample size of ~3000 migrations that would always be routed elsewhere, we found that the executor.py::apply_migration function was taking up a large chunk of the time. For example, this took somewhere in the realm of half an hour to run all migrations. The migration.apply(state, schema_editor) is where the allow_migrate function was being checked within, another few function calls deep, within the migration database_forwards logic itself.

When we added a check for allow_migrate within the apply_migration function, skipping the migration.apply if allow_migrate returned False, we had a speedup so significant that it changed from the order of an hour or so to a minute.

Attaching some profiled logs screenshots for the individual functions. Can provide more upon request

[Suzannah Cooper]

Profiling using cProfile before - can see 3408 instances of calling apply_migration with cumulative time of 1000

[Suzannah Cooper]

Profiling using cProfile after - can see 3408 instances of calling apply_migration with cumulative time of ~0.66