Builtin csrf_failure() view uses wrong charset

[MrVichr]

When Django detects wrong CSRF token, it shows an error using view django.views.csrf.csrf_failure. That file ends with

    return HttpResponseForbidden(t.render(c), content_type="text/html;")

When the template (CSRF_FAILURE_TEMPLATE_NAME) is written using UTF-8, it is rendered incorrectly. I suggest changing that line to

    return HttpResponseForbidden(t.render(c), content_type="text/html;"+
                                              f" charset={settings.DEFAULT_CHARSET};")

or perhaps leaving out the content_type entirely.

Currently I'm using a workaround, by adding

<meta http-equiv="Content-type" content="text/html; charset=utf-8" />

to the template's HEAD, but it seems to me that the suggested fix is a better solution.

[Carlton Gibson]

Hi. Can you provide a working example of the issue please, and exactly what you mean by " it is rendered incorrectly".

Creating a test project with a UTF8 403_csrf.html works as expected, so I'm guessing there's something else going on.

Thanks.

[Claude Paroz]

I think I'd rather omit content_type when it is text/html (several other locations are affected, too). I'll try a patch to see what's the outcome in tests.

[Claude Paroz]

Here's the ​PR.

[Carlton Gibson]

Hey Claude,

Yes, I took it out in that one place, with no effect.

But, if there is an error, it would be good to see how that could come up.
(It should be a no-op right? Maybe some browser default charset if it's not specified issue... 🤔)

[Claude Paroz]

When the charset is missing from a response, the browser has to guess (either using a default, a user preference, or sniffing the content). We should avoid that. Read ​https://www.w3.org/International/articles/http-charset/index.en

[Carlton Gibson]

Good link!

It is very important to always label Web documents explicitly. HTTP 1.1 says that the default charset is ISO-8859-1. ...

So we can presume the doc is interpreted as ISO-8859-1, rather than UTF-8, and so an error (although exact example...)

>>> r = HttpResponse("Hello 🎉", content_type="text/html")
>>> r.charset
'utf-8'
>>> r.serialize_headers()
b'Content-Type: text/html'

>>> r = HttpResponse("Hello 🎉")
>>> r.serialize_headers()
b'Content-Type: text/html; charset=utf-8'

Claude's, of removing the half-specified content_type automatically picks up MrVichr's idea to add the explicit charset.

[Carlton Gibson <carlton@…>]

In 93803a1:

Fixed #33567 -- Avoided setting default text/html content type on responses.