static file serving fails with "Connection reset by peer" on HEAD requests

[Yehor Smoliakov]

In developer mode I ran this command in the console:

curl --head http://127.0.0.1:8000/media/blog/author/UqdQXbTuSjW_UbGCNMhFpA.jpg

And got this exception:

[20/Apr/2018 10:36:43] "HEAD /media/blog/author/UqdQXbTuSjW_UbGCNMhFpA.jpg HTTP/1.1" 500 59
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49750)
Traceback (most recent call last):
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 138, in run
    self.finish_response()
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 180, in finish_response
    self.write(data)
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 279, in write
    self._write(data)
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 453, in _write
    result = self.stdout.write(data)
  File "/usr/lib/python3.6/socketserver.py", line 775, in write
    self._sock.sendall(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 141, in run
    self.handle_error()
  File "/home/yehor/.local/share/virtualenvs/webapp-Dlno2ZDu/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 86, in handle_error
    super().handle_error()
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 368, in handle_error
    self.finish_response()
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 180, in finish_response
    self.write(data)
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 274, in write
    self.send_headers()
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 331, in send_headers
    if not self.origin_server or self.client_is_modern():
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 344, in client_is_modern
    return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/socketserver.py", line 639, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.6/socketserver.py", line 696, in __init__
    self.handle()
  File "/home/yehor/.local/share/virtualenvs/webapp-Dlno2ZDu/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 154, in handle
    handler.run(self.server.get_app())
  File "/usr/lib/python3.6/wsgiref/handlers.py", line 144, in run
    self.close()
  File "/usr/lib/python3.6/wsgiref/simple_server.py", line 35, in close
    self.status.split(' ',1)[0], self.bytes_sent
AttributeError: 'NoneType' object has no attribute 'split'
----------------------------------------

[Tim Graham]

I don't think this is a bug in Django -- the traceback comes from Python's wsgiref. Feel free to reopen if you can explain why Django is at fault.

[mandm]

Similar problem with HEAD requests after upgrading to Django 1.10.8 (1.8.x & 1.9.x work)

curl -I http://localhost:8000/ 

gives this error

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
    self.finish_response()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
    self.write(data)
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 217, in write
    self._write(data)
  File "/usr/lib/python2.7/socket.py", line 328, in write
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 307, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 104] Connection reset by peer
ERROR: 2018-06-16 11:53:28,639 - "HEAD / HTTP/1.1" 500 59

[Claude Paroz]

This happens when streaming a FileResponse through a wsgiref.FileWrapper instance, when the file is bigger than the streaming block size. curl is closing the connection when it receives the first chunk, and the error then happens when FileWrapper is writing the second chunk to the socket.
I cannot yet state about a possible culprit in that scenario.

[Tim Graham]

Tentatively accept for further investigation.

[Jannik Schürg]

I can not reproduce this. I tried Python 3.6.4 and 2.7.6 with Django 1.10.8 and 2.0 on Mac OS. The setup was a new project with media serving and a 50MB jpg file.

Could you provide an example project or more information in order to reproduce the error?

I got

[2018-11-07 05:07:39,626] - Broken pipe from ('127.0.0.1', 63603)

but no exception.

[Jannik Schürg]

I could reproduce it now a few times but not consistently, it is a timing issue maybe. Or my version of curl is more hesitant with TCP RST (leading to ConnectionResetError).

Here is what happens, I think:

1. Curl closes the connection by resetting it instead of closing it (RST vs. FIN).
2. An exception is thrown in wsgiref/socket.
3. wsgiref.handlers.BaseHandler.close() is called, which reset the state of the handler (in particular self.environ is set to None).
4. The error handler handle_error() is called. Here Django has overwritten the method and checks if is_broken_pipe_error() in which case nothing would happen.
5. But curl did reset instead of close, so we have not "broken pipe," but "connection reset". The default handler is called, which tries to use self.environ, exception.

A proper fix probably would be to modify core.servers.basehttp.ServerHandler by adding the method

def finish_response(self):
    try:
        if self.environ['REQUEST_METHOD'] == 'HEAD':
            self.finish_content()
        else:
            if not self.result_is_file() or not self.sendfile():
                for data in self.result:
                    self.write(data)
                self.finish_content()
    finally:
        self.close()

With this the server does no longer try to send the body for a HEAD request (as it should be).
I am sure there are cases where this does produce an incorrect HEAD response according to the HTTP standard, but at least no body is sent anymore.

Alternatively/additionally one might want to change the check with is_broken_pipe(). For example one could also check for connection reset, or if self.environ is not None, or if the connection was closed (overwrite close() method).

[Jacob Walls]

​PR

[Dave Johansen]

This might already be known, but the issue appears to be basically all HEAD requests in general and not just those related to serving static files. To demonstrate that, I added a simpler reproducer in #33115 and I tested the patch from the PR and it resolved the issue for me.

[Mariusz Felisiak]

Duplicate of #28054.