Opened 6 years ago

Closed 5 years ago

#15496 closed Bug (fixed)

"Content-Transfer-Encoding: base64" not honored when uploading files

Reported by: gene@… Owned by: nobody
Component: HTTP handling Version: master
Severity: Normal Keywords: base64 file upload
Cc: Tom Christie Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When the client POST-s a file in base64 encoding, Django does not decode the file correctly.

How to Repeat

Upload a file to a Django view in an HTML form with method="POST" and enctype="multipart/form-data"; inside the MIME part for the file, encode the body in base64 and indicate the encoding in the Content-Transfer-Encoding: header.

Expected Result

The uploaded file, when examined on disk, is decoded (by Django).

Actual Result

The uploaded file, when examined on disk, is still base64-encoded.

Analysis

In module django.http.multipartparser, MultiPartParser.parse() iterates over each part as returned by a Parser object. The Parser object yields a header dictionary, which MultiPartParser.parse() captures in meta_data. parse() then fetches the Content-Transfer-Encoding header from meta_data into transfer_encoding.

transfer_encoding, as returned by meta_data.get(), is not simply a string such as 'base64' but really a 2-tuple (value, params). Also, the header value (the first element of the returned tuple) is not whitespace-trimmed.

As a result, if the part contained 'Content-Transfer-Encoding: base64', meta_data.get('Content-Transfer-Encoding') does not return just 'base64' but a 2-tuple (' base64', {}):

  • The header has no parameters, hence an empty parameter dictionary;
  • The value portion retains the leading space, right after the colon (:) in the raw header.

parse() needs the whitespace-trimmed version of the first element of the tuple; the attached patch converts the returned tuple into the string.

Attachments (2)

django-base64-upload-patch.diff (667 bytes) - added by gene@… 6 years ago.
base64-upload-with-tests.diff (3.4 KB) - added by Claude Paroz 6 years ago.

Download all attachments as: .zip

Change History (13)

Changed 6 years ago by gene@…

comment:1 Changed 6 years ago by Russell Keith-Magee

Needs tests: set
Triage Stage: UnreviewedAccepted

comment:2 Changed 6 years ago by Tom Christie

Cc: Tom Christie added

comment:3 Changed 6 years ago by Łukasz Rekucki

Severity: Normal
Type: Bug

Changed 6 years ago by Claude Paroz

comment:4 Changed 6 years ago by Claude Paroz

Easy pickings: unset
Needs tests: unset
Version: 1.2SVN

comment:5 Changed 6 years ago by Jannis Leidel

Triage Stage: AcceptedReady for checkin

comment:6 Changed 6 years ago by Jannis Leidel

Resolution: fixed
Status: newclosed

In [16176]:

Fixed #15496 -- Corrected handling of base64 file upload encoding. Thanks, gene and Claude Paroz.

comment:7 Changed 5 years ago by anonymous

Resolution: fixed
Status: closedreopened
UI/UX: unset

My multipart post data has only one field that's base64 encoded. I applied this patch over the stock 1.3 release and found that it screwed up the other parameters, with their default encoding. Maybe I am doing something wrong. Here's my post request. Works great without the patch, I just have to unencode the chunks of the file in my filehandler rather than relying on the automatic behavior which, I agree, would be preferred.

Content-Type: multipart/form-data; boundary=RANDOM_STRING_BOUNDARY
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="csrfmiddlewaretoken"

1e9c92a33c9a70fc335f93f8f78e20cd
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="settings_group_id"

257
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="serial_number"

10001
--RANDOM_STRING_BOUNDARY
Content-Disposition: form-data; name="image_file"; filename="2.tif"
Content-Type: application/octet-stream
Content-Transfer-Encoding: base64

[...base64 encoded data...]
--RANDOM_STRING_BOUNDARY--

comment:8 Changed 5 years ago by Claude Paroz

Resolution: needsinfo
Status: reopenedclosed

I tried to extend the test to include non base-64 data, I double-checked the code which detects the encoding, and I don't see how other data may be affected by this bug's patch. Of course, I may be missing something. But please give us more details about how your other parameters are screwed up. What do you get in resulting request.POST?

comment:9 Changed 5 years ago by anonymous

Thanks for looking into it. One of these days I'll get back to it and repeat my test. The above is my actual POST data... it should be easy enough for me to repeat the test when I get a free hour. (Ha! A free hour!) Thank you again for double-checking the patch.

comment:10 Changed 5 years ago by Aymeric Augustin

Resolution: needsinfo
Status: closedreopened

This could be backported to 1.3 on the grounds that it's a data-loss bug.

comment:11 Changed 5 years ago by Aymeric Augustin

Resolution: fixed
Status: reopenedclosed

In [17546]:

[1.3.x] Fixed #15496 -- Corrected handling of base64 file upload encoding. Backport of r16176 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top