Migration with django-database-view don't work with multiple databases.

[razielvamp666]

My configuration is:

python 3.7.1

Django==2.2.24
django-axes==5.16.0
django-bootstrap-modal-forms==1.3.1
django-celery-results==1.0.4
django-cors-headers==3.1.1
django-database-view==0.3.0
django-debug-toolbar==1.11
django-dynamic-db-router==0.3.0
django-ipware==3.0.7
django-redis-cache==3.0.0
django-redis-sessions==0.6.2
django-ses==1.0.3
django-storages==1.12.3
django-taggit==0.24.0
djangorestframework==3.12.4
pytest-django==4.4.0

I using Django with multiple databases and default DB is not set for security reasons (I prefer to get error if Django will fallback to default connection, because it means that current tenant DB is not used for some reasons). so default connection is empty ({}).

When I run python manage.py migrate --database=some_db_name

I've get error:

  Applying app_name.0154_migration_name...Traceback (most recent call last):
  File "manage.py", line 41, in <module>
    execute_from_command_line(sys.argv)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/dbview/helpers.py", line 21, in database_forwards
    self._create_standard_view(model, schema_editor)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/dbview/helpers.py", line 60, in _create_standard_view
    qs = str(model.view())
  File "/user/code/app_name/app_name/models/file_name.py", line 136, in view
    return str(qs.query)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/query.py", line 265, in __str__
    sql, params = self.sql_with_params()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/query.py", line 273, in sql_with_params
    return self.get_compiler(DEFAULT_DB_ALIAS).as_sql()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 474, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 54, in pre_sql_setup
    self.setup_query()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 45, in setup_query
    self.select, self.klass_info, self.annotation_col_map = self.get_select()
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 254, in get_select
    sql, params = self.compile(col, select_format=True)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/expressions.py", line 737, in as_sql
    return "%s.%s" % (qn(self.alias), qn(self.target.column)), []
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 396, in quote_name_unless_alias
    r = self.connection.ops.quote_name(name)
  File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/backends/dummy/base.py", line 20, in complain
    raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.

You can see that problem is here:

return self.get_compiler(DEFAULT_DB_ALIAS).as_sql()

DEFAULT_DB_ALIAS is default and it hardcoded.

Also DEFAULT_DB_ALIAS is used couple of times inside django/db/models/sql/query.py (please search by DEFAULT_DB_ALIAS in file to see details)

It should use CURRENT_DB as other parts of Django
Unfortunately I didn't check how migration determines current DB when --database=... is set so can't recommend exact solution, but I may see couple hints:

1. DB may be get from --database=... directly (But not sure if that is a correct solution).
2. DB may be get from [DatabaseRouter](​https://docs.djangoproject.com/en/2.2/topics/db/multi-db/#database-routers) as other parts of Django (It is looking like correct solution)
3. Here should be using option to be able set connection DB directly (for example, all functions inside django/db/transaction.py has using argument to be able set connection name directly)

Right now I using workaround and set from django.conf import settings; settings.DATABASES['default'] = current_db_taken_from_cmd if command is migrate.
Not all migrations fail. I don't know reason why it happens only with specific migrations, but I guess that it because this migration doing CreateView

Migration file is looking like:

# Generated by Django 2.2.10 on 2021-10-06 04:17
from dbview.helpers import CreateView

import common.enums.some_enum
from django.db import migrations, models
import django.db.models.deletion


class Migration(migrations.Migration):

    dependencies = [
        ('app_name', '0153_previous_migration'),
    ]

    operations = [
        migrations.CreateModel(
            name='SomeName',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('create_datetime', models.DateTimeField(auto_now_add=True, verbose_name='A')),
                ('user', models.CharField(default='admin', max_length=32, verbose_name='B')),
                ('update_datetime', models.DateTimeField(auto_now=True, verbose_name='C')),
                ...
            ],
            options={
                'db_table': 'table_name',
                'unique_together': {('other_table', 'table_name')},
            },
        ),
        CreateView(
            name='SomeNameView',
            fields=[
                ('table_name_1', models.OneToOneField(on_delete=django.db.models.deletion.PROTECT, primary_key=True, serialize=False, to='orm_django.ApplyingInformation')),
                ('table_name_2', models.IntegerField(verbose_name='A')),
                ('table_name_3', models.IntegerField(verbose_name='B')),
                ('table_name_3', models.IntegerField(verbose_name='C')),
                ...
            ],
            options={
                'db_table': 'table_view',
            },
        ),
        migrations.AddIndex(
            model_name='modelname',
            index=models.Index(fields=['field_A', 'field_B', ...], name='index_name'),
        ),
    ]

[Mariusz Felisiak]

Thanks for this report. However a lot of things happen outside of Django, you have many 3rd-party packages, a custom Operation in migrations, etc. The main issue is that you're using the query attribute to generate an SQL which is not recommended, documented, or supported. Moreover, this helper has issues with parameters (see #25705) and doesn't work when the default database is empty (see #25947). I'm not marking this as a duplicate of #25947 as fixing it will not make .query the recommended way to generate an SQL. I would use a raw SQL in your case.

[razielvamp666]

Replying to Mariusz Felisiak:

Thanks for this report. However a lot of things happen outside of Django, you have many 3rd-party packages, a custom Operation in migrations, etc. The main issue is that you're using the query attribute to generate an SQL which is not recommended, documented, or supported. Moreover, this helper has issues with parameters (see #25705) and doesn't work when the default database is empty (see #25947). I'm not marking this as a duplicate of #25947 as fixing it will not make .query the recommended way to generate an SQL. I would use a raw SQL in your case.

Thank you for answer.
Sorry, I didn't notice that was used little bit our code too. Not just standard libraries.

But, Is I will wrong if I say:

1. Django support multiple databases
2. django/db/models/sql/query.py not support multiple databases because it uses DEFAULT_DB_ALIAS and no matter how and when it run. Result will be always same for non default DB
3. django/db/models/sql/query.py is a standard part of Django
4. standard part of Django not support multiple databases
5. => Django not support multiple databases (at least for all features)

Isn't this a kind of bug?

[razielvamp666]

And thank you for point to #25947
Yes, the actual problem is the same.

6 years as accepted bug :)

[Mariusz Felisiak]

But, Is I will wrong if I say:

1. Django support multiple databases
2. django/db/models/sql/query.py not support multiple databases because it uses DEFAULT_DB_ALIAS and no matter how and when it run. Result will be always same for non default DB
3. django/db/models/sql/query.py is a standard part of Django
4. standard part of Django not support multiple databases
5. => Django not support multiple databases (at least for all features)

Isn't this a kind of bug?

Using DEFAULT_DB_ALIAS is not always against supporting multiple databases, sometimes is adequate. It's not always easy, feasible, or necessary to pass a current connection. You're making too strong assumptions. Also, as far as I'm concerned we don't claim that Django support interacting with multiple databases everywhere, see ​docs.

[Mariusz Felisiak]

Replying to razielvamp666:

6 years as accepted bug :)

Feel-free to prepare a patch.