id summary reporter owner description type status component version severity resolution keywords cc stage has_patch needs_docs needs_tests needs_better_patch easy ui_ux 34083 Cloning test database fails with mariadb-server 8.x Stephen Finucane nobody "I'm attempting to run tests in parallel using MariaDB. When running tests, all access to the additional databases fail. Scrolling back to the top, I see the following error messages: {{{ ❯ tox -e py38-django41 -- patchwork.tests.api.test_user patchwork.tests.api.test_event py38-django41 run-test-pre: PYTHONHASHSEED='631429049' py38-django41 run-test: commands[0] | python /home/patchwork/patchwork/manage.py test --noinput --parallel -v 2 --timing -- patchwork.tests.api.test_user patchwork.tests.api.test_event Found 25 test(s). Creating test database for alias 'default' ('test_patchwork')... Operations to perform: Synchronize unmigrated apps: django_filters, humanize, messages, rest_framework, staticfiles Apply all migrations: admin, auth, authtoken, contenttypes, patchwork, sessions, sites Synchronizing apps without migrations: Creating tables... Running deferred SQL... Running migrations: Applying contenttypes.0001_initial... OK ... {skipped} ... Applying sites.0002_alter_domain_unique... OK Cloning test database for alias 'default' ('test_patchwork')... mysqldump: Couldn't execute 'SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM, '$.""number-of-buckets-specified""') FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table 'COLUMN_STATISTICS' in information_schema (1109) Cloning test database for alias 'default' ('test_patchwork')... mysqldump: Couldn't execute 'SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM, '$.""number-of-buckets-specified""') FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table 'COLUMN_STATISTICS' in information_schema (1109) System check identified no issues (0 silenced). ... }}} The same issue happens on multiple Django versions. With a bit of hacking on the code in the tox venv, I was able to inspect the command that Django is actually executing: {{{ Dumping! cmd = mysqldump --user=patchwork --host=localhost --routines --events test_patchwork }}} Executing this locally, I see the same issue: {{{ ❯ MYSQL_PWD=password mysqldump --user=patchwork --host=localhost --routines --events patchwork -- MySQL dump 10.13 Distrib 8.0.30, for Linux (x86_64) -- -- Host: localhost Database: patchwork -- ------------------------------------------------------ -- Server version 5.5.5-10.5.16-MariaDB /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!50503 SET NAMES utf8mb4 */; /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */; /*!40103 SET TIME_ZONE='+00:00' */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Table structure for table `auth_group` -- DROP TABLE IF EXISTS `auth_group`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!50503 SET character_set_client = utf8mb4 */; CREATE TABLE `auth_group` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(150) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `name` (`name`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Dumping data for table `auth_group` -- LOCK TABLES `auth_group` WRITE; /*!40000 ALTER TABLE `auth_group` DISABLE KEYS */; /*!40000 ALTER TABLE `auth_group` ENABLE KEYS */; UNLOCK TABLES; mysqldump: Couldn't execute 'SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM, '$.""number-of-buckets-specified""') FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = 'patchwork' AND TABLE_NAME = 'auth_group';': Unknown table 'COLUMN_STATISTICS' in information_schema (1109) ❯ echo $? 2 }}} I'm using Fedora 36 with the default packages: {{{ ❯ cat /etc/system-release Fedora release 36 (Thirty Six) ❯ sudo dnf list installed | grep mariadb mariadb-backup.x86_64 3:10.5.16-1.fc36 @updates mariadb-common.x86_64 3:10.5.16-1.fc36 @updates mariadb-connector-c.x86_64 3.2.7-1.fc36 @updates mariadb-connector-c-config.noarch 3.2.7-1.fc36 @updates mariadb-connector-c-devel.x86_64 3.2.7-1.fc36 @updates mariadb-connector-c-doc.noarch 3.2.7-1.fc36 @updates mariadb-cracklib-password-check.x86_64 3:10.5.16-1.fc36 @updates mariadb-devel.x86_64 3:10.5.16-1.fc36 @updates mariadb-errmsg.x86_64 3:10.5.16-1.fc36 @updates mariadb-gssapi-server.x86_64 3:10.5.16-1.fc36 @updates mariadb-server.x86_64 3:10.5.16-1.fc36 @updates mariadb-server-utils.x86_64 3:10.5.16-1.fc36 @updates ❯ sudo dnf list installed | grep mysql community-mysql.x86_64 8.0.30-2.fc36 @updates community-mysql-common.x86_64 8.0.30-2.fc36 @updates mysql-selinux.noarch 1.0.5-1.fc36 @updates }}} I had previously reported a similar issue using a combination of `mysql-client` with an older version of `mysql-server` or `mariadb-server`. This was reported as #33537. However, I'm now seeing the issue with the latest versions of both the server and client. As noted in the issue, I can work around this on the command line by adding the `--column-statistics=0` argument to `mysqldump`, however, there doesn't appear to be any mechanism to do this in Django. Alternatively, I can add the following to `my.cnf` (or equivalent): {{{ /etc/my.cnf.d/mysqldump.cnf [mysqldump] column-statistics=0 }}} There's a chance that someone has packaged something wrong in Fedora land and that I'm using an incompatible client and server combo (MySQL-derived client, MariaDB server). However, it's surprising that the tests proceeded to run despite the DB clone failing. As you can see above, `mysqldump` returned a non-zero exit code so there doesn't appear to be any reason Django couldn't have stopped immediately. I think it would make sense to fail fast at a minimum. If we wanted, we could also add the `--column-statistics=0` argument to our call to `mysqldump` (or provide a mechanism for users to do so), however, this isn't compatible with the 5.7.x version of `mysqldump` or the version provided by e.g. the `mariadb-client` package on Ubuntu) so this would have to be a version-specific check." Bug closed Database layer (models, ORM) 4.1 Normal duplicate Unreviewed 0 0 0 0 0 0