Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#21282 closed Bug (fixed)

The serialize_headers method of HttpResponse fails to handle latin1-compatible values

Reported by: xelnor Owned by: nobody
Component: HTTP handling Version: 1.4
Severity: Normal Keywords: http header encoding
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If a header value (passed as unicode) contains only ascii data, serialize_headers() doesn't fail
If it contains characters that can't be encoded in latin1, serialize_headers() doesn't fail
If it contains only characters valid in latin1, including some outside the ascii range, serialize_headers() fails with a UnicodeDecodeError.

The culprit seems to lie on line 132 of django/http/response.py: that line calls ('%s: %s' % (key, value)).encode('us-ascii'), but at this point, key is ascii bytes and value is a bytes array containing either latin1-encoded text or mime-encoded utf8 text.

Since we're using unicode_literals, Python tries value.decode('ascii'), which fails if, and only if, value contains latin1 characters outside the ascii range.

I have attached a patch containing both a test exhibiting the issue and a patch fixing it (no test failure).

Note: This is not a release blocker, since the serialize_headers() and its callers (serialize() and __str__()) aren't used while answering requests.

Attachments (2)

fix_ticket_21282_httpresponse_serialize.patch (1.5 KB) - added by xelnor 18 months ago.
Test and fix for HttpResponse.serialize_headers() bug.
21282-2.diff (1.9 KB) - added by claudep 18 months ago.
py3 compatible version

Download all attachments as: .zip

Change History (7)

Changed 18 months ago by xelnor

Test and fix for HttpResponse.serialize_headers() bug.

comment:1 Changed 18 months ago by xelnor

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

The test suite passes with the proposed patch:

Ran 5906 tests in 201.214s

OK (skipped=348, expected failures=11)

comment:2 Changed 18 months ago by claudep

  • Triage Stage changed from Unreviewed to Accepted

Changed 18 months ago by claudep

py3 compatible version

comment:3 Changed 18 months ago by claudep

I've attached a slightly different patch, Python 3 compatible. Aymeric's review mandatory!

comment:4 Changed 18 months ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In a14f08723304be27e851c753a68c8200473a9ca1:

Fixed #21282 -- Made HttpResponse.serialize_headers accept latin-1

Thanks Raphaël Barrois for the report and the initial patch and
Aymeric Augustin for the review.

comment:5 Changed 18 months ago by Claude Paroz <claude@…>

In b2f9c74ed1cd246022ab52d239eeb33f950dcc70:

[1.6.x] Fixed #21282 -- Made HttpResponse.serialize_headers accept latin-1

Thanks Raphaël Barrois for the report and the initial patch and
Aymeric Augustin for the review.

Backport of a14f087233 from master.

Note: See TracTickets for help on using tickets.
Back to Top