Opened 5 months ago

Last modified 3 months ago

#31398 assigned Bug

multiple_database.tests.AuthTestCase doesn't flush the default database if transactions aren't supported.

Reported by: Tim Graham Owned by: Hassaan Ali Wattoo
Component: Core (Other) Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

On databases that don't support transactions, the database is flushed between test methods.

In Django's test suite, multiple_database.tests.AuthTestCase.test_auth_manager() is the first test to run in that class. It creates two Users: alice in the 'other' database and bob in the 'default' database.

The test class uses AuthRouter which has an allow_migrate() method that makes sure the auth app only appears in the 'other' db, however, the test database creation doesn't respect that router and so the table is created in both databases (thus no errors when creating the users as described above).

The problem comes during TransactionTestCase. _fixture_teardown() after the first test. It invokes the flush command which doesn't include any tables from the auth app because of the allow_migrate() method on the router. (The flush command calls django.core.management.sql.sql_flush which calls BaseDatabaseIntrospection.django_table_names() which calls router.get_migratable_models() which eventually invokes allow_migrate().) Since the auth_user table isn't flushed after the first test, an IntegrityError (violating the unique username constraint) is raised when the second test creates bob in the 'default' database.

Change History (7)

comment:1 Changed 5 months ago by felixxm

Summary: multiple_database.tests.AuthTestCase doesn't flush the default database if transactions aren't supportedmultiple_database.tests.AuthTestCase doesn't flush the default database if transactions aren't supported.
Triage Stage: UnreviewedAccepted

I didn't reproduce this issue, but the analysis sounds correct.

Last edited 5 months ago by felixxm (previous) (diff)

comment:2 Changed 4 months ago by Hassaan Ali Wattoo

Owner: changed from nobody to Hassaan Ali Wattoo
Status: newassigned

comment:3 Changed 4 months ago by Hassaan Ali Wattoo

I'm having trouble duplicating this issue. Running the multiple_database test suite for the latest code on master results in no failures.

.............................................................................
----------------------------------------------------------------------
Ran 77 tests in 0.366s

Could you give us more information about how you reproduced this issue? (version of Django, operating system, steps to reproduce the issue)? Thanks!

comment:4 Changed 4 months ago by Tim Graham

This issue only affects databases that don't support transactions. I used Cloud Spanner.

comment:5 Changed 4 months ago by Simon Charette

Hassaan, you could try running the suite with MySQL's MyISAM to reproduce locally.

comment:6 in reply to:  5 Changed 4 months ago by Hassaan Ali Wattoo

Replying to Simon Charette:

Hassaan, you could try running the suite with MySQL's MyISAM to reproduce locally.


Using MySQL 8.0.11 MyISAM, I ran into a separate issue. It seems that the following line creates a record in the default database.

1535. User.objects.create_user('alice', 'alice@example.com')

The behavior then is that the test case fails at the following assert

1546.  with self.assertRaises(User.DoesNotExist):
            User.objects.using('default').get(username='alice')

However, the second test case passes successfully, indicating that the database flushes correctly. Do you have any idea why this might happen?

Last edited 4 months ago by Hassaan Ali Wattoo (previous) (diff)

comment:7 Changed 3 months ago by Tim Graham

Hi Hassaan, I can't reproduce the issue you described.

After adding:

'OPTIONS': {
   'init_command': 'SET default_storage_engine=MYISAM',
}

to both databases in the test settings files, I can reproduce the issue I reported:

$ ./tests/runtests.py --settings=test_mysql --parallel=1 multiple_database.tests.AuthTestCase
Testing against Django installed in '/home/tim/code/django/django'
Creating test database for alias 'default'...
Creating test database for alias 'other'...
System check identified no issues (0 silenced).
.E
======================================================================
ERROR: test_dumpdata (multiple_database.tests.AuthTestCase)
dumpdata honors allow_migrate restrictions on the router
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.IntegrityError: (1062, "Duplicate entry 'bob' for key 'username'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/tests/multiple_database/tests.py", line 1564, in test_dumpdata
    User.objects.db_manager('default').create_user('bob', 'bob@example.com')
  File "/home/tim/code/django/django/contrib/auth/models.py", line 146, in create_user
    return self._create_user(username, email, password, **extra_fields)
  File "/home/tim/code/django/django/contrib/auth/models.py", line 140, in _create_user
    user.save(using=self._db)
  File "/home/tim/code/django/django/contrib/auth/base_user.py", line 66, in save
    super().save(*args, **kwargs)
  File "/home/tim/code/django/django/db/models/base.py", line 750, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/tim/code/django/django/db/models/base.py", line 788, in save_base
    force_update, using, update_fields,
  File "/home/tim/code/django/django/db/models/base.py", line 891, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/home/tim/code/django/django/db/models/base.py", line 931, in _do_insert
    using=using, raw=raw,
  File "/home/tim/code/django/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/tim/code/django/django/db/models/query.py", line 1248, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/home/tim/code/django/django/db/models/sql/compiler.py", line 1386, in execute_sql
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/tim/.virtualenvs/django37/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
django.db.utils.IntegrityError: (1062, "Duplicate entry 'bob' for key 'username'")
Note: See TracTickets for help on using tickets.
Back to Top