GZipMiddleware buffers streaming responses

[Adam Johnson]

Currently, GZipMiddleware, via compress_sequence(), buffers the entire response before sending it to the client. This can cause issues for clients that expect to receive data in chunks, such as those using Server-Sent Events (SSE) or WebSockets.

This issue was reported to me in the django-browser-reload project back in ​Issue #161 (2023), where a contributor fixed it with a workaround, and I didn't think to investigate. Now, while implementing ​django-http-compression, I have realized that it’s a proper bug that can be fixed by adding a call to zfile.flush(), as done in its ​PR #8.

To reproduce the issue, use the app below, which can be run with uv run --script. If you comment out GzipMiddleware and load the page in a browser, you will see the numbers incrementing every second. If you include GzipMiddleware, the page will never load. Adding the zfile.flush() call in compress_sequence() fixes the issue.

#!/usr/bin/env uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = [
#     "django",
# ]
# ///
from __future__ import annotations

import os
import sys
import time

from django.conf import settings
from django.core.wsgi import get_wsgi_application
from django.http import StreamingHttpResponse
from django.urls import path

settings.configure(
    # Dangerous: disable host header validation
    ALLOWED_HOSTS=["*"],
    # Use DEBUG=1 to enable debug mode
    DEBUG=(os.environ.get("DEBUG", "") == "1"),
    # Make this module the urlconf
    ROOT_URLCONF=__name__,
    # Only gzip middleware
    MIDDLEWARE=[
        "django.middleware.gzip.GZipMiddleware",
    ],
)


def clock(request):
    def stream():
        yield "<h1>Clock</h1>\n"
        count = 1
        while True:
            yield f"<p>{count}</p>\n"
            count += 1
            time.sleep(1)

    return StreamingHttpResponse(stream())


urlpatterns = [
    path("", clock),
]

app = get_wsgi_application()

if __name__ == "__main__":
    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

[Adam Johnson]

Dupe of #36293, will reply there.