id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
35483	ModelMultipleChoiceField with CharField key throws ValueError if data contains NUL (0x00) characters	Jennifer Richards	lotvall	"In a form using a ModelMultipleChoiceField with a model that has a CharField primary key, I'm getting server errors when a request submits input with a nul (0x00) character. This is resulting from a ValueError raised by the postgresql backend. It happens whether the data is submitted as a query string or in the request body.

I've tried adding an explicit ProhibitNullCharactersValidator to the field, but this fails because the field's clean method runs validators after evaluating whether the input is a member of the queryset.

A fairly minimal example that exhibits this is:

{{{
class MyModel(models.Model):
    slug = models.CharField(primary_key=True)

class MyForm(forms.Form):
    field = forms.ModelMultipleChoiceField(queryset=MyModel.objects.all())

def my_view(request):
    form = MyForm(data=request.GET)
    if form.is_valid():
        return HttpResponse(""yay"")
    return HttpResponse(""boo"", status=400)
}}}

With that running, the following triggers the error:
{{{
$ curl 'http://localhost:8000/my-view?field=hi%00'
}}}

The output from the Django dev server is
{{{
ERROR: django.request:241: Internal Server Error: /my-view
Traceback (most recent call last):
  File ""/home/dev/.local/lib/python3.9/site-packages/django/core/handlers/exception.py"", line 55, in inner
    response = get_response(request)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/core/handlers/base.py"", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File ""/workspace/ietf/doc/views_search.py"", line 98, in my_view
    if form.is_valid():
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/forms.py"", line 201, in is_valid
    return self.is_bound and not self.errors
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/forms.py"", line 196, in errors
    self.full_clean()
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/forms.py"", line 433, in full_clean
    self._clean_fields()
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/forms.py"", line 445, in _clean_fields
    value = field.clean(value)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/models.py"", line 1590, in clean
    qs = self._check_values(value)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/forms/models.py"", line 1623, in _check_values
    pks = {str(getattr(o, key)) for o in qs}
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/models/query.py"", line 398, in __iter__
    self._fetch_all()
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/models/query.py"", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/models/query.py"", line 91, in __iter__
    results = compiler.execute_sql(
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/models/sql/compiler.py"", line 1562, in execute_sql
    cursor.execute(sql, params)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/backends/utils.py"", line 102, in execute
    return super().execute(sql, params)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/backends/utils.py"", line 67, in execute
    return self._execute_with_wrappers(
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/backends/utils.py"", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File ""/home/dev/.local/lib/python3.9/site-packages/django/db/backends/utils.py"", line 89, in _execute
    return self.cursor.execute(sql, params)
ValueError: A string literal cannot contain NUL (0x00) characters.
[27/May/2024 08:02:37] ""GET /my-view?field=hi%00 HTTP/1.0"" 500 180131
}}}

As a workaround, replacing forms.ModelMultipleChoiceField with
{{{
class MyModelMultipleChoiceField(forms.ModelMultipleChoiceField):
    validate_no_nulls = validators.ProhibitNullCharactersValidator()
    def clean(self, value):
        for item in value:
            self.validate_no_nulls(item)
        return super().clean(value)
}}}
correctly handles the same input, with only 
{{{
[27/May/2024 08:04:22] ""GET /my-view?field=hi%00 HTTP/1.0"" 400 3
}}}
in the server log.

I'm seeing this with Django 4.2.13 using the postgresql backend with psycopg2 2.99 against PostgreSQL 14.6."	Bug	closed	Forms	4.2	Normal	fixed	ModelMultipleChoiceField null nul		Ready for checkin	1	0	0	0	0	0