id,summary,reporter,owner,description,type,status,component,version,severity,resolution,keywords,cc,stage,has_patch,needs_docs,needs_tests,needs_better_patch,easy,ui_ux 7581,Middleware accessing HttpResponse.content breaks streaming HttpResponse objects.,Tai Lee,ccahoon,"{{{ConditionalGetMiddleware}}}, {{{GZipMiddleware}}}, {{{CommonMiddleware}}}, and {{{CsrfMiddleware}}} all access response.content directly or indirectly. This prevents a streaming response from being initiated until any generator passed to the {{{HttpResponse}}} as content is consumed, which can cause a timeout when trying to stream large dynamically generated content. I've already put together a patch based on the assumption that {{{HttpResponse._is_string}}} being {{{False}}} indicates content has been passed in as a dynamic generator and thus we shouldn't delay streaming a response to the browser while consuming the entire generator. The patch implements the following: * Allow middleware to assign a new generator to {{{HttpResponse.content}}} without consuming it (e.g. {{{GZipMiddleware}}}) * Compress chunks of content yielded by {{{HttpResponse._container}}} progressively in {{{GZipMiddleware}}} to allow streaming GZipped content * Only generate an ETag in {{{CommonMiddleware}}} from {{{HttpResponse.content}}} if {{{HttpResponse._is_string}}} is {{{True}}} * Only check that the length of {{{HttpResponse.content}}} is less than 200 in {{{GZipMiddleware}}} if {{{HttpResponse._is_string}}} is {{{True}}} * Only set the Content-Length header in {{{GZipMiddleware}}} and {{{ConditionalGetMiddleware}}} if {{{HttpResponse._is_string}}} is {{{True}}} With {{{CommonMiddleware}}} enabled by default and breaking the streaming response functionality if ETags are enabled in {{{settings.py}}}, I'd consider this a bug. It can be worked around by manually specifying a bogus ETag before returning the response, which doesn't seem ideal. With this patch, users still have the option of consuming a generator before passing it to the {{{HttpResponse}}} in order to enable ETag and Content-Length headers, and conditional GZipping when the content length is less than 200. With {{{CsrfMiddleware}}}, the generator is only consumed when the Content-Type header is {{{text/html}}} or {{{application/xhtml+xml}}}, which may be an acceptable compromise - no streaming response for HTML content if you choose to use {{{django.contrib.csrf}}}. This patch accesses {{{HttpResponse._is_string}}} and {{{HttpResponse._container}}} from external middleware classes. Perhaps these properties could be made public and/or renamed to be more logical in this context? ",New feature,closed,Core (Other),dev,Normal,fixed,stream HttpResponse Content-Length,real.human@… mmitar@… mindsocket,Accepted,1,0,0,0,0,0