GZipMiddleware drops content from async streaming responses

[Adam Johnson]

0bd2c0c9015b53c41394a1c0989afbfd94dc2830 (#33735) expanded GZipMiddleware to support async streaming responses. But it does so with a faulty gzip_wrapper() that compresses chunks as individual files, rather than as a continuous stream. As a result, only the first chunk is decompressible, and browsers drop the rest of the response, even as they stay connected and download all the data.

The solution is to use a streaming approach with GzipFile, as is already done for sync responses in compress_sequence().

Additionally, the sync approach currently starts by sending an empty chunk to flush the headers. I think that may be necessary for async responses too, since the first content chunk may take an arbitrary amount of time to be generated.

To reproduce the issue, use the app below, which can be run with uv run --script. If you comment out GzipMiddleware and load the page in a browser, you will see the numbers incrementing every second. If you include GzipMiddleware, only the header will appear, and the rest of the response will be dropped.

#!/usr/bin/env uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = [
#     "daphne",
#     "django",
# ]
# ///
from __future__ import annotations

import asyncio
import os
import sys

from django.conf import settings
from django.core.asgi import get_asgi_application
from django.http import StreamingHttpResponse
from django.urls import path

settings.configure(
    # Dangerous: disable host header validation
    ALLOWED_HOSTS=["*"],
    # Use DEBUG=1 to enable debug mode
    DEBUG=(os.environ.get("DEBUG", "") == "1"),
    # Make this module the urlconf
    ROOT_URLCONF=__name__,
    # Use Daphne for async runserver
    INSTALLED_APPS=[
        "daphne",
    ],
    ASGI_APPLICATION=f"{__name__}.app",
    # Only gzip middleware
    MIDDLEWARE=[
        "django.middleware.gzip.GZipMiddleware",
    ],
)


async def clock(request):
    async def stream():
        yield "<h1>Clock</h1>\n"
        count = 1
        while True:
            yield f"<p>{count}</p>\n"
            count += 1
            await asyncio.sleep(1)

    return StreamingHttpResponse(stream())


urlpatterns = [
    path("", clock),
]

app = get_asgi_application()

if __name__ == "__main__":
    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

[AdityaJai@…]

I’d like to work on this issue. Can you assign it to me?

[Natalia Bidart]

Replying to AdityaJai@…:

I’d like to work on this issue. Can you assign it to me?

This ticket already has an owner as per the ticket metadata so we should give the owner some time to work on it.

[Natalia Bidart]

Hey Adam, thank you for your ticket! My uneducated eye sees this as a dupe of #36293. What do you think? (Feel free to reopen if you disagree.)

[Adam Johnson]

I think you meant to close #36655, not this one.

[Natalia Bidart]

Replying to Adam Johnson:

I think you meant to close #36655, not this one.

Yes! Sorry I had the two opened at the same time and I mixed them up.

[Jacob Walls <jacobtylerwalls@…>]

In a0323a0c:

Fixed #36656 -- Avoided truncating async streaming responses in GZipMiddleware.

[Jacob Walls <jacobtylerwalls@…>]

In bb4fcf5f:

[6.0.x] Fixed #36656 -- Avoided truncating async streaming responses in GZipMiddleware.

Backport of a0323a0c44135c28134672e6e633e5f4a7a68d5d from main.