Opened 5 years ago

Last modified 4 years ago

#30565 closed Bug

Close StreamingHttpResponse content immediately after iterating it — at Version 1

Reported by: Chris Jerdonek Owned by: nobody
Component: HTTP handling Version: dev
Severity: Normal Keywords: HttpResponse, streaming, StreamingHttpResponse
Cc: Johannes Maron Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Chris Jerdonek)

This ticket is to suggest doing for StreamingHttpResponse what #25725 did for HttpReponse, namely to close the underlying content iterator after it has been iterated over.

Currently, if creating a StreamingHttpResponse from a file-like object, it doesn't seem like there's an obvious way to close the underlying file after the file has been streamed. And as one of the comments in #25725 pointed out, trying to do this in StreamingHttpResponse.close() isn't a good solution because WSGI servers can't be relied upon to call close().

I believe an alternative, more reliable solution may be to call close() immediately after the iterator has been exhausted (if hasattr(value, 'close') is true, as #25725 does). This is essentially what #25725 did for non-streaming HttpResponse objects. Here is that code:

def content(self, value):
    # Consume iterators upon assignment to allow repeated iteration.
    if hasattr(value, '__iter__') and not isinstance(value, (bytes, str)):
        content = b''.join(self.make_bytes(chunk) for chunk in value)
        if hasattr(value, 'close'):
            try:
                value.close()
            except Exception:
                pass
    else:
        content = self.make_bytes(value)

(from
https://github.com/django/django/blob/1564e42ad397021093585147875a21dae1a3b3fc/django/http/response.py#L310-L319 )

In the streaming case, the content value argument could be wrapped something like so (inside StreamingHttpResponse._set_streaming_content(value)):

def iter_content():
    yield from value
    if hasattr(value, 'close'):
        try:
            value.close()
        except Exception:
            pass

new_value = iter_content()

Here is the current code for StreamingHttpResponse._set_streaming_content():

def _set_streaming_content(self, value):
    # Ensure we can never iterate on "value" more than once.
    self._iterator = iter(value)
    if hasattr(value, 'close'):
        self._closable_objects.append(value)

(from: https://github.com/django/django/blob/1564e42ad397021093585147875a21dae1a3b3fc/django/http/response.py#L376-L380)

Change History (1)

comment:1 by Chris Jerdonek, 5 years ago

Description: modified (diff)
Type: UncategorizedCleanup/optimization
Version: 2.2master
Note: See TracTickets for help on using tickets.
Back to Top