id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
29323	HTTPRequest QueryDict wrongly decodes binary encoded values in python2	Thomas Riccardi	nobody	"`application/x-www-form-urlencoded` bodies allow binary values: just percent-escape (most of) the bytes.

It doesn't work with python2 and django.

In python2, QueryDict returns an `unicode` string for values, which makes it impossible to represent binary data.
Worse: it returns an `unicode` string whose unicode code points are the raw expected bytes (caused by lax behavior of `str.decode('iso-8859-1')`) so it silently fails.

Example:
{{{#!python
q = QueryDict('foo=%00%7f%80%ff')
q['foo']
# actual: 
# u'\x00\x7f\x80\xff'
# expected:
# '\x00\x7f\x80\xff'
}}}

Relevant code:
https://github.com/django/django/blob/f89b11b879b83aa505dc8231da5f06ca4b1b062e/django/http/request.py#L397-L401

This was introduced by https://github.com/django/django/commit/fa02120d360387bebbbe735e86686bb4c7c43db2 while trying to accept broken user agents that send non-ascii as urlencoded raw values:
- the python 3 version seems fine: it tries to decode '''before''' calling the query string parser (from `urllib`)
- the python 2 version made a mistake: it decodes '''after''' calling the query string parser (and only on the value, not the key strangely)


I'm not sure how to fix this, as there are many locations where the key and value are re-encoded (`appendlist` does it too..)


I have not (yet) tested with python3."	Uncategorized	new	HTTP handling	1.11	Normal				Unreviewed	0	0	0	0	0	0