Add setting to control installed PostgreSQL extensions.

[Markus Bertheau]

In #25388 a ​feature was introduced to disable migrations for test databases. Before that it was possible to disable migrations for all apps with MIGRATION_MODULES, mapping every app to None (​as explained here). Other code is used for example in pytest-django (​here and ​here) to disable migrations in other ways. All for the speed-up in creating a database "directly" (circumventing migrations) as opposed to creating it with the migration mechanism.

Also, ​here and ​here it is explained how extensions should be installed with a migration operation.

Both don't mix, because when disabling migrations, the extension doesn't get installed. Using features of the extension, for example column types, doesn't work.

As a side note: postgis functionality is provided through a separate database engine that provides its own DatabaseWrapper that takes care of installing the postgis extension, independent of the migration mechanism.

Maybe extensions should be configured in settings.DATABASES and then automatically installed by the postgresql engine.

[Mariusz Felisiak]

I'm not sure what you're proposing, TBH.

Before that it was possible to disable migrations for all apps with MIGRATION_MODULES, mapping every app to None.

It's still possible.

Both don't mix, because when disabling migrations, the extension doesn't get installed.

I don't see any issue in this, it's an expected behavior. We cannot treat CreateExtension() operations differently.

[Mariusz Felisiak]

Maybe extensions should be configured in settings.DATABASES and then automatically installed by the postgresql engine.

CreateExtension() is a normal database migration operation, we shouldn't treat it differentially. If you don't want to (or cannot) control creating extensions with migrations then you can create them directly in a database.

[Markus Bertheau]

felixxm, I'm sorry, I may have expressed myself not as clearly as I hoped and that maybe led to some misunderstandings:

Replying to felixxm:

I'm not sure what you're proposing, TBH. (EDIT: I missed the last sentence).

Before that it was possible to disable migrations for all apps with MIGRATION_MODULES, mapping every app to None.

It's still possible.

I meant to say that even before the MIGRATE setting feature, it was already possible with means built-in into django – MIGRATION_MODULES – to circumvent migrations. I didn't mean to imply that the MIGRATION_MODULES way doesn't work anymore. Just that the problem I'm describing existed even before the MIGRATE setting.

Both don't mix, because when disabling migrations, the extension doesn't get installed.

I don't see any issue in this, it's an expected behavior. We cannot treat CreateExtension() operations differently.

Right, that's why I didn't report a bug or an issue. Nowhere in the documentation does Django suggest that this should be working. I'm not proposing handling CreateExtension() differently.

Nevertheless what I described is a use-case that I feel is common but Django doesn't cover. That's why I reported it as a cleanup/optimization, not as a bug.

Replying to felixxm:

Maybe extensions should be configured in settings.DATABASES and then automatically installed by the postgresql engine.

CreateExtension() is a normal database migration operation, we shouldn't treat it differentially. If you don't want to (or cannot) control creating extensions with migrations then you can create them directly in a database.

I'm not proposing a specific solution to the problem or that CreateExtension() should be treated differently. I also know that I can do stuff manually. That's of course a bit harder when you need to find a place to hook into Django's database creation process during test runs.

I'm pointing out: Django has two ways to create the database structure:

1. With migrations. With all the benefits that has for gradually keeping production databases up-to-date with respect to their structure and data. That works well with PostgreSQL database extensions documented ​here.
2. Circumventing migrations. This has mainly a speed benefit and is commonly used in tests, where a database is created from scratch just for the test run. Speed is important here, as evidenced by a) the Django feature to circumvent migrations and create the final database structure directly based on the model definitions and b) the feature where a later test run can reuse an existing test database.

However, the second way is no longer possible when PostgreSQL database extensions are used. Django provides no mechanism that issues the CREATE EXTENSION command necessary to successfully create the model as described in models.py.

I guess what I'm asking is: Does the Django team consider this a problem worth solving in Django, or do they say "we don't want to support that use-case, don't bother coming up with a PR".

[Adam Johnson]

I strongly recommend you do not disable migrations during tests. Migrations are highly important code that affect your most vital asset - your data. I wrote a whole section in my book ​Speed Up Your Django Tests against disabling migrations during tests.

But... if you really want to continue on that route... You can run CREATE EXTENSION, or any arbitrary SQL, in a custom test runner's overridden setup_databases(). I have an example in my blog post ​| How to Add Database Modifications Beyond Migrations to Your Django Project .

You could also override the migrate command to run SQL before or after it does the migrations. See example 2 in my post ​How to Add Database Modifications Beyond Migrations to Your Django Project .

[Mariusz Felisiak]

.. or use MIGRATION_MODULES in tests settings and point to a module with single migration that will create extensions.

​Please follow triaging guidelines with regards to wontfix tickets.