Test loader accesses MySQL before database is created.

[Ahmad A. Hussein]

Tests fail on the MySQL backend because the test runner accesses the database while loading tests prior to database creation. This happens in annotations.tests and aggregation.tests.

While the test runner loads the tests, it invokes their skipIf statements making calls to the database on [​https://github.com/django/django/blob/master/tests/aggregation/tests.py#L1208 L1208. This causes an exception because the database does not yet exist and the settings have not been configured.

Relevant traceback:

(parallelpr) c:\Users\ahmad\Desktop\Projects\DjangoGSOC\tests>runtests.py --parallel=1 --settings=test_mysql aggregation.tests
Testing against Django installed in 'c:\users\ahmad\desktop\projects\djangogsoc\django'
Traceback (most recent call last):
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 220, in ensure_connection
    self.connect()
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 201, in connect
    self.connection = self.get_new_connection(conn_params)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\mysql\base.py", line 234, in get_new_connection
    return Database.connect(**conn_params)
  File "C:\Users\ahmad\Envs\parallelpr\lib\site-packages\MySQLdb\__init__.py", line 84, in Connect
    return Connection(*args, **kwargs)
  File "C:\Users\ahmad\Envs\parallelpr\lib\site-packages\MySQLdb\connections.py", line 179, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (1049, "Unknown database 'main'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\ahmad\Desktop\Projects\DjangoGSOC\tests\runtests.py", line 574, in <module>
    failures = django_tests(
  File "C:\Users\ahmad\Desktop\Projects\DjangoGSOC\tests\runtests.py", line 314, in django_tests
    failures = test_runner.run_tests(
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\test\runner.py", line 754, in run_tests
    suite = self.build_suite(test_labels, extra_tests, process_setup, process_setup_args)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\test\runner.py", line 593, in build_suite
    tests = self.test_loader.loadTestsFromName(label)
  File "c:\users\ahmad\appdata\local\programs\python\python38\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\Users\ahmad\Desktop\Projects\DjangoGSOC\tests\aggregation\tests.py", line 21, in <module>
    class AggregateTestCase(TestCase):
  File "C:\Users\ahmad\Desktop\Projects\DjangoGSOC\tests\aggregation\tests.py", line 1208, in AggregateTestCase
    connection.vendor == 'mysql' and 'ONLY_FULL_GROUP_BY' in connection.sql_mode,
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\__init__.py", line 28, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\mysql\base.py", line 398, in sql_mode
    sql_mode = self.mysql_server_data['sql_mode']
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\mysql\base.py", line 359, in mysql_server_data
    with self.temporary_connection() as cursor:
  File "c:\users\ahmad\appdata\local\programs\python\python38\lib\contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 604, in temporary_connection
    with self.cursor() as cursor:
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 260, in cursor
    return self._cursor()
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 236, in _cursor
    self.ensure_connection()
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 220, in ensure_connection
    self.connect()
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 220, in ensure_connection
    self.connect()
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\base\base.py", line 201, in connect
    self.connection = self.get_new_connection(conn_params)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\utils\asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "c:\users\ahmad\desktop\projects\djangogsoc\django\db\backends\mysql\base.py", line 234, in get_new_connection
    return Database.connect(**conn_params)
  File "C:\Users\ahmad\Envs\parallelpr\lib\site-packages\MySQLdb\__init__.py", line 84, in Connect
    return Connection(*args, **kwargs)
  File "C:\Users\ahmad\Envs\parallelpr\lib\site-packages\MySQLdb\connections.py", line 179, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (1049, "Unknown database 'main'")

[Ahmad A. Hussein]

Removing ​this change fixes this issue on aggregation.tests

[Mariusz Felisiak]

Agreed, we should be able to fix this by moving checks to tests, e.g.

diff --git a/tests/aggregation/tests.py b/tests/aggregation/tests.py
index c738cbb27e..dfdf53a86e 100644
--- a/tests/aggregation/tests.py
+++ b/tests/aggregation/tests.py
@@ -1204,16 +1204,16 @@ class AggregateTestCase(TestCase):
         ])                                                                                                                                                                                     
                                                                                                                                                                                                
     @skipUnlessDBFeature('supports_subqueries_in_group_by')
-    @skipIf(
-        connection.vendor == 'mysql' and 'ONLY_FULL_GROUP_BY' in connection.sql_mode,
-        'GROUP BY optimization does not work properly when ONLY_FULL_GROUP_BY '
-        'mode is enabled on MySQL, see #31331.',
-    )
     def test_aggregation_subquery_annotation_multivalued(self):
         """
         Subquery annotations must be included in the GROUP BY if they use
         potentially multivalued relations (contain the LOOKUP_SEP).
         """
+        if connection.vendor == 'mysql' and 'ONLY_FULL_GROUP_BY' in connection.sql_mode:
+            self.skipTest(
+                'GROUP BY optimization does not work properly when '
+                'ONLY_FULL_GROUP_BY mode is enabled on MySQL, see #31331.'
+            )
         subquery_qs = Author.objects.filter(
             pk=OuterRef('pk'),
             book__name=OuterRef('book__name'),

Do you want to prepare a patch?

[Ahmad A. Hussein]

That was the approach I imagined as well. I was looking through the documentation for that.

I'll definitely prepare the patch.

[Ahmad A. Hussein]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 493b26b:

Fixed #31888 -- Avoided module-level MySQL queries in tests.