Opened 2 months ago

Last modified 2 weeks ago

#37319 assigned Bug

GZipMiddleware should skip responses without body.

Reported by: Vladimir Nani Owned by: Vladimir Nani
Component: HTTP handling Version: 6.1
Severity: Normal Keywords: GZipMiddleware
Cc: Vaibhav Pant Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

GZipMiddleware compresses responses with status 304 that are not allowed to have content body.

Negative effect:

  • streaming response with 304 status code and empty body is passed to GZipMiddleware
  • Middleware check doesnt apply for streaming response and proceeds with compression
  • In situation where connections are pooled, for example with Traefik, the body is not read (according to rfc 9112).
  • The empty body compressed is left as a garbage in the sockets buffer.
  • When the socket is read for the next time instead of starting with "HTTP" it reads "\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\xffaaaa"
  • The next unrelated request that reuses the socket fails with malformed http:
{"level":"debug","error":"net/http: HTTP/1.x transport connection broken: malformed HTTP version \"\\x1f\\x8b\\b\\b\\x00\\x00\\x00\\x00\\x00\\xffaaaaaaaaaaaaaaaa\\x00\\x03\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00HTTP/1.1\"","time":"2026-09-04T12:33:17+02:00","caller":"github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:117","message":"500 Internal Server Error"}

Proposal:
Check for 204 and 304 and do not compress these responses.

​https://www.rfc-editor.org/rfc/rfc9112.html#section-6.3-2.1

Change History (18)

comment:1 by Md. Saikat Islam, 2 months ago

You are right. The middleware currently handles 204, 304, and short responses under 200 bytes only in the non-streaming case. This is the guard in GZipMiddleware:

if not response.streaming and len(response.content) < 200:
    return response

For streaming responses, this guard does not apply, and the middleware compresses them anyway — which it should not do.

I tested this to verify for 204, 304, short response, and long response.

Non-streaming (test.py):

204 204 None b''
304 304 None b''
short 200 200 None b'short body'
long 200 200 gzip b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\x03aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\x00\xab\xa8\x18\x05#\r\x00\x00Q\xce\xb2\n\xf4\x01\x00\x00'

Streaming (test_streaming.py):

streaming 304 empty 304 gzip b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\xffaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00'
streaming 204 empty 204 gzip b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\xffaaaaaaa\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00'
streaming short 200 gzip b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\xffaa\x00*\xce\xc8/*QH\xcaO\xa9\x04\x00\x00\x00\xff\xff\x03\x009\xb2\xed\xc0\n\x00\x00\x00'
streaming 200 long 200 gzip b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x00\xffaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\x00JL\x1c\x05#\r\x00\x00\x00\x00\xff\xff\x03\x00\x11A\x92\x05\xf4\x01\x00\x00'

In the streaming case, even the 204 and 304 responses with empty content still get Content-Encoding: gzip and non-empty compressed bytes — gzip-compressing an empty sequence still produces gzip header/footer framing, so it's never truly empty. This matches the bug behavior described in the report.

I can provide the test files here if needed.

I would like to work on this issue.

comment:2 by Yassin Bahri, 2 months ago

I reproduced this on current Django main through the WSGI handler.

A StreamingHttpResponse([], status=304) passed through GZipMiddleware with Accept-Encoding: gzip is returned with Content-Encoding: gzip and a non-empty gzip payload. The same behavior occurs with status 204.

RFC 9112 section 6.3 states that responses with 1xx, 204, or 304 status codes cannot contain a message body:

​https://www.rfc-editor.org/rfc/rfc9112.html#section-6.3

The issue therefore appears valid and actionable. I suggest classifying it as:

  • Type: Bug
  • Component: HTTP handling
  • Severity: Normal
  • Triage stage: Accepted

I would also suggest narrowing the summary to:

GZipMiddleware compresses streaming 204 and 304 responses.

“Responses without body” is slightly ambiguous because an empty 200 response may legally be gzip encoded. The problem here is responses whose status code prohibits content.

Before implementing the fix, the complete scope should be considered. RFC 9112 also covers 1xx responses, while RFC 9110 prohibits content in 205 responses. HEAD responses have separate semantics and may still include representation metadata, so they should not automatically be treated identically.

This does not appear to be a Django 6.1 regression; the relevant middleware behavior is also present in older supported branches.

comment:3 by Md. Saikat Islam, 2 months ago

I will wait untill a maintainer set the triage stage to accepted. After that I will assign myself to this issue.

comment:6 by Vladimir Nani, 2 months ago

Component: Uncategorized → HTTP handling
Has patch: set
Type: Uncategorized → Bug
Version: 6.1

comment:7 by Md. Saikat Islam, 2 months ago

According to the triaging guidelines, another community member should review and reproduce the issue before it’s marked as “Accepted”. Reference: ​https://docs.djangoproject.com/en/dev/internals/contributing/triaging-tickets/

Since Yassin Bahri independently reproduced the issue, and I was also able to reproduce it, I’m marking this ticket as accepted.

comment:8 by Md. Saikat Islam, 2 months ago

Owner: set to Md. Saikat Islam
Status: new → assigned
Triage Stage: Unreviewed → Accepted
Version: → 6.1

comment:9 by Md. Saikat Islam, 6 weeks ago

Owner: Md. Saikat Islam removed
Status: assigned → new

comment:10 by OsmnvAslan, 6 weeks ago

Needs documentation: set
Patch needs improvement: set

comment:11 by Vaibhav Pant, 4 weeks ago

Owner: set to Vaibhav Pant
Status: new → assigned

in reply to:  5 comment:12 by Vaibhav Pant, 4 weeks ago

Replying to Vladimir Nani:

​https://github.com/django/django/commit/324434140cf30b4e1923f3bb251107afcdafc6db have a patch to fix this

Hi @vladimirnani, I’ve claimed the ticket on Trac, so the missing-owner check that closed this PR shouldn’t be a problem anymore. Would you like to reopen it and push the remaining review changes? I’ll wait a little while, and if it’s still closed after that, I’ll open a new PR based on yours and make sure you’re credited. Thanks for the work on this!

comment:13 by Vaibhav Pant, 4 weeks ago

Cc: Vaibhav Pant added
Owner: Vaibhav Pant removed
Status: assigned → new

comment:14 by Vimal Sahani, 3 weeks ago

Owner: set to Vimal Sahani
Status: new → assigned

in reply to:  14 comment:15 by Vaibhav Pant, 3 weeks ago

Replying to Vimal Sahani:

Hi, please check the existing comments before claiming an issue or opening a PR. I had already started working on this and was waiting for a response regarding the existing PR. It would have been better to leave a comment and allow some time for a response. Thanks for understanding!

comment:16 by Vladimir Nani, 2 weeks ago

Hey, could you please help me understand why cannot I merge my pr? It looks like the issue i've found is resolved in my pr.

in reply to:  16 comment:17 by Vaibhav Pant, 2 weeks ago

Replying to Vladimir Nani:

Hey, could you please help me understand why cannot I merge my pr? It looks like the issue i've found is resolved in my pr.

Hi Vladimir. The usual flow is: claim the ticket, open a PR, and set “Has patch” on the ticket. A maintainer then reviews it, suggests changes, and merges it once everything looks good, so contributors can’t merge their own PRs.

More details here: ​https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/submitting-patches/

Your PR was auto-closed because the ticket had no owner at that point. It’s also still flagged “Needs documentation” and “Patch needs improvement”, so the review comments, docs, and tests need to be sorted out first.

comment:18 by Vladimir Nani, 2 weeks ago

Needs documentation: unset
Owner: changed from Vimal Sahani to Vladimir Nani
Patch needs improvement: unset

Thanks Vaibhav for explanation, reopening the pr
​https://github.com/django/django/pull/22202

Last edited 2 weeks ago by Vladimir Nani (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top