DatabaseError running tests with MySQL & utf8mb4 charset

[Greg Barker <fletch@…>]

Character set and collation set to utf8mb4:

$ cat /etc/mysql/my.cnf | grep utf8
character-set-server = utf8mb4
collation-server     = utf8mb4_general_ci

Create a new site

$ virtualenv test-env
$ cd test-env
$ source ./bin/activate
(temp-test) $ pip install django
(temp-test) $ pip install mysql-python
(temp-test) $ django-admin.py startproject mysite

Update settings.py to use MySQL and create your database

Then run the tests, hit the error

$ python mysite/manage.py test
Creating test database for alias 'default'...
DatabaseError: (1071, 'Specified key was too long; max key length is 767 bytes')

MySQL query log reveals it fails on this statement:

CREATE TABLE `auth_customuser` (
    `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
    `password` varchar(128) NOT NULL,
    `last_login` datetime NOT NULL,
    `email` varchar(255) NOT NULL UNIQUE,
    `is_active` bool NOT NULL,
    `is_admin` bool NOT NULL,
    `date_of_birth` date NOT NULL
)

Which originates from custom_user.py in django.contrib.auth.tests

class CustomUser(AbstractBaseUser):
    email = models.EmailField(verbose_name='email address', max_length=255, unique=True)
    is_active = models.BooleanField(default=True)
    is_admin = models.BooleanField(default=False)
    date_of_birth = models.DateField()

I think it should be like the User class, which does not specify the max_length, so it defaults the value to 75.

[Tim Graham]

Seems like a duplicate (or at least related) to #21196. I don't see how the proposed change would fix the issue - have you tested it?

[Greg Barker <fletch@…>]

Replying to timo:

Seems like a duplicate (or at least related) to #21196. I don't see how the proposed change would fix the issue - have you tested it?

Oops, didn't see your reply on the mailing list before I submitted this ticket.

After removing max_length from those two EmailFields in custom_user.py, I'm able to run the tests successfully:

$ python mysite/manage.py test
Creating test database for alias 'default'...
..................................................................................................................................................s......................................................................................................................................x.......................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 481 tests in 12.454s

OK (skipped=1, expected failures=1)
Destroying test database for alias 'default'...

[Ramiro Morales]

From ​http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html

Tip

To save space with UTF-8, use VARCHAR instead of CHAR. Otherwise, MySQL must reserve three (or four) bytes for each character in a CHAR CHARACTER SET utf8 (or utf8mb4) column because that is the maximum possible length.

Django CharFields map to SQL CHAR. That would mean that the maximum working max_length value in this particular case should be 191.

[Ramiro Morales]

I'd say this ticket and #21196 are duplicates of #18392.

[Tim Graham]

Ok, closing this as a duplicate of #18392. Made a note of this there.