Opened 62 minutes ago
#37397 new Bug
request.body does not exclude file upload data from DATA_UPLOAD_MAX_MEMORY_SIZE, contrary to docs
| Reported by: | marcapdev | Owned by: | |
|---|---|---|---|
| Component: | Documentation | Version: | 6.1 |
| Severity: | Normal | Keywords: | |
| Cc: | marcapdev | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Summary
The documentation for DATA_UPLOAD_MAX_MEMORY_SIZE states that the size check, when triggered via either request.body or request.POST, excludes file upload data (request.FILES) from the calculation. This is true for request.POST (via MultiPartParser), but is not true for request.body — that property compares the full Content-Length header (file bytes included) against the setting, with no file-exclusion logic at all, and has nothing to do with request.FILES.
Documentation text (verbatim, https://docs.djangoproject.com/en/6.1/ref/settings/#data-upload-max-memory-size)
The maximum size in bytes that a request body may be before a
SuspiciousOperation(RequestDataTooBig) is raised. The check is done when accessingrequest.bodyorrequest.POSTand is calculated against the total request size excluding any file upload data (request.FILES). You can set this toNoneto disable the check. Applications that are expected to receive unusually large form posts should tune this setting.
What the code actually does (tag 6.1.1, current stable)
request.body (django/http/request.py:399-442):
@property def body(self): if not hasattr(self, "_body"): if self._read_started: raise RawPostDataException( "You cannot access body after reading from request's data stream" ) # Make Content-Length fall back to 0 if malformed (e.g. ASGIRequest # comma-joins duplicate Content-Length headers). try: content_length = int(self.META.get("CONTENT_LENGTH") or 0) except (ValueError, TypeError): content_length = 0 # Limit the maximum request data size that will be handled # in-memory. Reject early when Content-Length is present and # already exceeds the limit, avoiding reading the body at all. self._check_data_too_big(content_length) # Content-Length can be absent or understated (e.g. # `Transfer-Encoding: chunked` on ASGI), so for seekable # streams (e.g. SpooledTemporaryFile on ASGI), check the actual # buffered size before reading it all into memory. if self._stream.seekable(): stream_size = self._stream.seek(0, os.SEEK_END) self._check_data_too_big(stream_size) self._stream.seek(0) try: self._body = self.read() except OSError as e: raise UnreadablePostError(*e.args) from e finally: self._stream.close() self._stream = BytesIO(self._body) return self._body def _check_data_too_big(self, length): if ( settings.DATA_UPLOAD_MAX_MEMORY_SIZE is not None and length > settings.DATA_UPLOAD_MAX_MEMORY_SIZE ): raise RequestDataTooBig( "Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE." )
request.POST (for multipart requests, via django/http/multipartparser.py:222-256):
The parser only accumulates bytes toward the limit for FIELD-type parts:
if item_type == FIELD: # Avoid reading more than DATA_UPLOAD_MAX_MEMORY_SIZE. if settings.DATA_UPLOAD_MAX_MEMORY_SIZE is not None: read_size = settings.DATA_UPLOAD_MAX_MEMORY_SIZE - num_bytes_read ... data = field_stream.read(size=read_size) num_bytes_read += len(data) ... if ( settings.DATA_UPLOAD_MAX_MEMORY_SIZE is not None and num_bytes_read > settings.DATA_UPLOAD_MAX_MEMORY_SIZE ): raise RequestDataTooBig( "Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE." ) elif item_type == FILE: # Avoid storing more than DATA_UPLOAD_MAX_NUMBER_FILES. ... # routed to upload handlers instead; num_bytes_read is never touched here
FILE-type parts are routed to upload handlers instead and never touch num_bytes_read / this check at all. So for request.POST, the documentation's claim is accurate — file content genuinely is excluded, and this is the code path that actually populates request.FILES.
Why this matters in practice
Code that accesses request.body for a reason unrelated to parsing form data (e.g. logging/backing up the raw body for error reporting, signature verification helpers, etc.) on a multipart file-upload request will unexpectedly raise RequestDataTooBig for files well under what a developer would reasonably expect from reading this documentation — since the documented behavior ("excludes file upload data (request.FILES)") simply does not apply to that code path. We hit this in production: a multipart request with a single ~5 MB file (default DATA_UPLOAD_MAX_MEMORY_SIZE of 2.5 MB) crashed with RequestDataTooBig the moment a piece of unrelated middleware touched request.body before the view accessed request.POST/request.FILES — despite the documented guarantee that file data doesn't count toward this limit.
Suggested fix
Clarify the documentation to note the distinction explicitly, e.g.:
The check is done when accessing
request.bodyorrequest.POST. Forrequest.POST(i.e. parsed multipart form data), the calculation excludes any file upload data (request.FILES). Forrequest.body, the check is performed against the full size of the request body, including any file upload data — sincerequest.bodymaterializes the entire request body in memory (or verifies its buffered size) regardless of its content, the size limit applies to the whole body in that case.
Reproduction
- Set
DATA_UPLOAD_MAX_MEMORY_SIZEto its default (2.5 MB, or any value < file size). - Add trivial middleware:
def process_request(self, request): request.body(or anything that accesses.bodybefore the view touches.POST/.FILES). - POST a multipart/form-data request with a single file field larger than the limit (and no other large fields) to any view.
- Observe
RequestDataTooBig: Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.is raised — even though the file is the only thing contributing meaningfully to the request size, and the documentation states file data is excluded from this check.