mysql DatabaseSchemaEditor can't `quote_value` on byte strings

[Ryan Siemens]

Using the the django.db.backends.mysql backend with mysqlclient fails to render the schema migration with sqlmigrate when using a BinaryField with a binary string default value.

You can recreate this with a simple app using the following model or see the attached sample project (MD5 = 2be52a23a4001e90728d3b0f5276e087):

class TestModel(models.Model):
    ok = models.BinaryField(default=b'some-blob')
    # add this after the initial migration
    # not_ok = models.BinaryField(default=b'some-other-blob')

Then perform the following steps:

1. run initial manage.py makemigrations
2. uncomment the not_ok = models.BinaryField(default=b'some-other-blob')
3. run manage.py makemigrations again to generate the alter table migration
4. run manage.py sqlmigrate <app> 0002
5. assert the following TypeError is raised

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/commands/sqlmigrate.py", line 29, in execute
    return super().execute(*args, **options)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/core/management/commands/sqlmigrate.py", line 65, in handle
    sql_statements = loader.collect_sql(plan)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/migrations/loader.py", line 345, in collect_sql
    state = migration.apply(state, schema_editor, collect_sql=True)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/migrations/operations/fields.py", line 104, in database_forwards
    schema_editor.add_field(
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/backends/mysql/schema.py", line 89, in add_field
    super().add_field(model, field)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 487, in add_field
    self.execute(sql, params)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 137, in execute
    self.collected_sql.append((sql % tuple(map(self.quote_value, params))) + ending)
  File "/Users/rsiemens/sandbox/quote_value_bug/.venv/lib/python3.8/site-packages/django/db/backends/mysql/schema.py", line 55, in quote_value
    quoted = self.connection.connection.escape(value, self.connection.connection.encoders)
TypeError: bytes() argument 2 must be str, not dict

The error seems to be raised from the mysqlclient connection.escape, but interestingly this works if you alter the self.connection.connection.encoders right before escaping (​https://github.com/django/django/blob/main/django/db/backends/mysql/schema.py#L57) by doing self.connection.connection.encoders.pop(bytes). Looking at the default converters mysqlclient sets up (​https://github.com/PyMySQL/mysqlclient/blob/v2.0.1/MySQLdb/converters.py#L106-L139) I don't see a bytes one being added, so I'm guessing django adds it somewhere along the way.

Using:
Python 3.8.5
MySQL 8.0.23
Django==3.1.7
mysqlclient==2.0.3

[Ryan Siemens]

example app to recreate bug

[Mariusz Felisiak]

Thanks for this ticket, however that's an issue in ​mysqlclient (see also ​mysqlclient#306). I'm not sure but maybe they should use _bytes_literal(). Please create a new issue in mysqlclient and feel-free to ping me on it.

[Ryan Siemens]

Thanks, Mariusz. I went ahead and opened ​https://github.com/PyMySQL/mysqlclient/issues/489

[Mariusz Felisiak]

There seems to be no easy way to fix this in mysqlclient at least for now, see [discussion](​https://github.com/PyMySQL/mysqlclient/issues/489). I'm going to add a workaround in the MySQL backend.

[Mariusz Felisiak]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 3c75f1f:

Refs #32595 -- Added MySQL's SchemaEditor.quote_value() tests for values with Unicode chars.

[Mariusz Felisiak <felisiak.mariusz@…>]

In f6018c1e:

Fixed #32595 -- Fixed SchemaEditor.quote_value() crash with bytes.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 682eba53:

[3.2.x] Refs #32595 -- Added MySQL's SchemaEditor.quote_value() tests for values with Unicode chars.

Backport of 3c75f1f3cac7985e8a134fc1c33eb6e01639a04b from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In d67d48e:

[3.2.x] Fixed #32595 -- Fixed SchemaEditor.quote_value() crash with bytes.

Backport of f6018c1e63a04e0c12e2ca759e76e05ccf5e09de from main