HttpResponse._get_content shouldn't accept non-text types

[Aymeric Augustin]

Currently, the httpwrappers tests verify the behavior of constructs such as HttpResponse([1,1,2,4,8]). This doesn't make any sense to me.

HttpResponse must convert its content to bytes. But when a Content-Encoding is already set, it mustn't perform any encoding, so it can't go through a unicode representation.

Now, in Python 3, bytes(3) == \x00\x00\x00. So I don't see any way to implement the construct above in Python 3 without special-casing integers.

Rather than explicitly hacking a meaningless behavior, I'd like to remove these tests and require that HttpResponse be instantiated with a (byte)string or an iterable of (byte)strings.

I'm marking this as a release blocker because it causes a test failure under Python 3 (httpwrappers.HttpResponseTests.test_iter_content).

[Claude Paroz]

+1, I tried more than once to redesign HttpResponse and found that current tests make this hard, so I fully concur on this issue.

[Karen Tracey]

Note #16494 also dealt with HttpResponse behavior when handed non-string types, and was fixed by converting to string representation. The fact that someone hit that hit that problem makes me fear people may be using this behavior even if they don't realize it. Couldn't we convert non-text types to unicode and encode (using the encoding specified, if it has been specified)? Or at least continue to support the existing behavior for those cases where the app hasn't messed with the response encoding?

[Aymeric Augustin]

Karen, thanks for pointing me to that ticket.

I'll review that discussion and implement a consistent solution — it isn't hard.

[Aymeric Augustin]

I've just attached a patch that's consistent with the tickets I could find on this topic.

force_bytes might be a bit overkill here but it has the following advantage:

>>> bytes(10)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>> from django.utils.encoding import smart_bytes
>>> smart_bytes(10)
b'10'

Allowing ASCII encoding when a Content-Encoding is set is still correct with regard to #4969: Python 2 allowed itself to perform str <=> unicode conversions of ASCII data transparently, making that explicit doesn't hurt.

[Aymeric Augustin <aymeric.augustin@…>]

In [536b0303637c55c914784b57841e6d59c4968d5f]:

[py3] Supported integers in HttpResponse

Fixed #18764.

[Aymeric Augustin <aymeric.augustin@…>]

In [2892cb0ec4ebde4955f12c92183b7804558ca381]:

[py3] Fixed regression introduced in 536b030363.

Refs #18764.

Reverted 536b030363 and switched to a more explicit way of avoiding
calling bytes(<int>).

This definitely deserves a refactoring. Specifically, _get_content
should just return b.join(self). Unfortunately that's impossible
with the current tests.