Multipart base64 file decoding of fails with large files when the encoded string contains newlines.

[jhobbs]

Large files are files larger than the chunk size in MultiPartParser.parse().

parse() tries to process base64 encoded files in chunks with lengths a multiple of 4. The base64 encoded string can contain newline characters, which aren't significant in base64 and should be ignored, but are counted toward's a chunk's length in parse(). This means when whitespace is stripped from the string, the count of base64 encoded characters may not be a multiple of 4, leading to an "Incorrect padding" error.

This pull request has a testcase for producing the failure and a proposed patch to fix the issue.

​https://github.com/django/django/pull/3151/files

Stacktrace:

Traceback (most recent call last):
  File "/home/jason/canonical/code/django/tests/file_uploads/tests.py", line 109, in test_big_base64_newlines_upload
    "Big data" * 68000, encode=base64.encodestring)
  File "/home/jason/canonical/code/django/tests/file_uploads/tests.py", line 96, in _test_base64_upload
    response = self.client.request(**r)
  File "/home/jason/canonical/code/django/django/test/client.py", line 443, in request
    six.reraise(*exc_info)
  File "/home/jason/canonical/code/django/django/core/handlers/base.py", line 121, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/jason/canonical/code/django/tests/file_uploads/views.py", line 103, in file_upload_echo_content
    r = dict((k, f.read().decode('utf-8')) for k, f in request.FILES.items())
  File "/home/jason/canonical/code/django/django/core/handlers/wsgi.py", line 152, in _get_files
    self._load_post_and_files()
  File "/home/jason/canonical/code/django/django/http/request.py", line 249, in _load_post_and_files
    self._post, self._files = self.parse_file_upload(self.META, data)
  File "/home/jason/canonical/code/django/django/http/request.py", line 214, in parse_file_upload
    return parser.parse()
  File "/home/jason/canonical/code/django/django/http/multipartparser.py", line 220, in parse
    six.reraise(MultiPartParserError, MultiPartParserError(msg), sys.exc_info()[2])
  File "/home/jason/canonical/code/django/django/http/multipartparser.py", line 216, in parse
    chunk = base64.b64decode(chunk)
  File "/usr/lib/python2.7/base64.py", line 76, in b64decode
    raise TypeError(msg)
MultiPartParserError: Could not decode base64 data: TypeError(Error('Incorrect padding',),)