"Content-Transfer-Encoding: base64" not honored when uploading files

[gene@…]

When the client POST-s a file in base64 encoding, Django does not decode the file correctly.

How to Repeat

Upload a file to a Django view in an HTML form with method="POST" and enctype="multipart/form-data"; inside the MIME part for the file, encode the body in base64 and indicate the encoding in the Content-Transfer-Encoding: header.

Expected Result

The uploaded file, when examined on disk, is decoded (by Django).

Actual Result

The uploaded file, when examined on disk, is still base64-encoded.

Analysis

In module django.http.multipartparser, MultiPartParser.parse() iterates over each part as returned by a Parser object. The Parser object yields a header dictionary, which MultiPartParser.parse() captures in meta_data. parse() then fetches the Content-Transfer-Encoding header from meta_data into transfer_encoding.

transfer_encoding, as returned by meta_data.get(), is not simply a string such as 'base64' but really a 2-tuple (value, params). Also, the header value (the first element of the returned tuple) is not whitespace-trimmed.

As a result, if the part contained 'Content-Transfer-Encoding: base64', meta_data.get('Content-Transfer-Encoding') does not return just 'base64' but a 2-tuple (' base64', {}):

• The header has no parameters, hence an empty parameter dictionary;
• The value portion retains the leading space, right after the colon (:) in the raw header.

parse() needs the whitespace-trimmed version of the first element of the tuple; the attached patch converts the returned tuple into the string.

[Jannis Leidel]

In [16176]:

Fixed #15496 -- Corrected handling of base64 file upload encoding. Thanks, gene and Claude Paroz.

[anonymous]

My multipart post data has only one field that's base64 encoded. I applied this patch over the stock 1.3 release and found that it screwed up the other parameters, with their default encoding. Maybe I am doing something wrong. Here's my post request. Works great without the patch, I just have to unencode the chunks of the file in my filehandler rather than relying on the automatic behavior which, I agree, would be preferred.

Content-Type: multipart/form-data; boundary=RANDOM_STRING_BOUNDARY
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="csrfmiddlewaretoken"

1e9c92a33c9a70fc335f93f8f78e20cd
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="settings_group_id"

257
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="serial_number"

10001
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="image_file"; filename="2.tif"
Content-Type: application/octet-stream
Content-Transfer-Encoding: base64

[...base64 encoded data...]
--RANDOM_STRING_BOUNDARY--

[Claude Paroz]

I tried to extend the test to include non base-64 data, I double-checked the code which detects the encoding, and I don't see how other data may be affected by this bug's patch. Of course, I may be missing something. But please give us more details about how your other parameters are screwed up. What do you get in resulting request.POST?

[anonymous]

Thanks for looking into it. One of these days I'll get back to it and repeat my test. The above is my actual POST data... it should be easy enough for me to repeat the test when I get a free hour. (Ha! A free hour!) Thank you again for double-checking the patch.

[Aymeric Augustin]

This could be backported to 1.3 on the grounds that it's a data-loss bug.

[Aymeric Augustin]

In [17546]:

[1.3.x] Fixed #15496 -- Corrected handling of base64 file upload encoding. Backport of r16176 from trunk.