The serialize_headers method of HttpResponse fails to handle latin1-compatible values

[Raphaël Barrois]

If a header value (passed as unicode) contains only ascii data, serialize_headers() doesn't fail
If it contains characters that can't be encoded in latin1, serialize_headers() doesn't fail
If it contains only characters valid in latin1, including some outside the ascii range, serialize_headers() fails with a UnicodeDecodeError.

The culprit seems to lie on line 132 of django/http/response.py: that line calls ('%s: %s' % (key, value)).encode('us-ascii'), but at this point, key is ascii bytes and value is a bytes array containing either latin1-encoded text or mime-encoded utf8 text.

Since we're using unicode_literals, Python tries value.decode('ascii'), which fails if, and only if, value contains latin1 characters outside the ascii range.

I have attached a patch containing both a test exhibiting the issue and a patch fixing it (no test failure).

Note: This is not a release blocker, since the serialize_headers() and its callers (serialize() and __str__()) aren't used while answering requests.

[Raphaël Barrois]

Test and fix for HttpResponse.serialize_headers() bug.

[Raphaël Barrois]

The test suite passes with the proposed patch:

Ran 5906 tests in 201.214s

OK (skipped=348, expected failures=11)

[Claude Paroz]

py3 compatible version

[Claude Paroz]

I've attached a slightly different patch, Python 3 compatible. Aymeric's review mandatory!

[Claude Paroz <claude@…>]

In a14f08723304be27e851c753a68c8200473a9ca1:

Fixed #21282 -- Made HttpResponse.serialize_headers accept latin-1

Thanks Raphaël Barrois for the report and the initial patch and
Aymeric Augustin for the review.

[Claude Paroz <claude@…>]

In b2f9c74ed1cd246022ab52d239eeb33f950dcc70:

[1.6.x] Fixed #21282 -- Made HttpResponse.serialize_headers accept latin-1

Thanks Raphaël Barrois for the report and the initial patch and
Aymeric Augustin for the review.

Backport of a14f087233 from master.