Exceptions in UploadHandler cause hang

[Karen Tracey]

I ran across this trying to recreate another problem. On Windows, if I set FILE_UPLOAD_TEMP_DIR to reside on a disk that is nearly out of space, and then use the django.core.files.uploadhandler.TemporaryFileUploadHandler to handle uploaded files, file uploads will hang. I changed the implementation of write in the Windows implementation of TemporaryFile in django/core/files/temp.py to be:

        def write(self, s):
            print 'in TemporaryFile(nt), about to call self.file.write(s)'
            x = self.file.write(s)
            print 'in TemporaryFile(nt), back from self.file.write(s)'
            return x

and then when I try the upload (using dev server) I see the first print, but not the second. It's not in a hard loop, the CPU is mostly idle, so it's like its waiting for free space. Clearing space on the disk doesn't wake it up, though.

[Karen Tracey]

Further investigation shows the write is not hanging, it is raising an exception. If I change the code to:

        def write(self, s):
            print 'in TemporaryFile(nt), about to call self.file.write(s)'
            try:
                x = self.file.write(s)
            except Exception, e:
                print 'Caught exception on self.file.write(s): %s' % str(e)
                raise
            print 'in TemporaryFile(nt), back from self.file.write(s)'
            return x

and try to upload to a disk with not enough free space, I see:

in TemporaryFile(nt), about to call self.file.write(s)
Caught exception on self.file.write(s): [Errno 28] No space left on device

in my dev server console output. No error is reflected anywhere I can see, there is no further console output and the browser page that initiated the upload is stuck at "Loading....".

Changing milestone to 1.0 since this appears to be a more far-reaching problem than I initially thought and someone should probably take a look at it sooner rather than later.

Also exceptions in upload handlers causing requests to hang has been reported here: ​http://groups.google.com/group/django-users/browse_thread/thread/b2fb9bf7fe1d47f9#

[Malcolm Tredinnick]

Karen, what's the full stack trace leading to that exception? traceback.print_stack() will be handy there.

[Karen Tracey]

Here it is:

in TemporaryFile(nt), about to call self.file.write(s)
Caught exception on self.file.write(s): [Errno 28] No space left on device
  File "d:\u\kmt\django\trunk\django\core\management\commands\runserver.py", line 54, in inner_run
    run(addr, int(port), handler)
  File "d:\u\kmt\django\trunk\django\core\servers\basehttp.py", line 665, in run
    httpd.serve_forever()
  File "d:\bin\Python2.5.2\lib\SocketServer.py", line 201, in serve_forever
    self.handle_request()
  File "d:\bin\Python2.5.2\lib\SocketServer.py", line 222, in handle_request
    self.process_request(request, client_address)
  File "d:\bin\Python2.5.2\lib\SocketServer.py", line 241, in process_request
    self.finish_request(request, client_address)
  File "d:\bin\Python2.5.2\lib\SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "d:\u\kmt\django\trunk\django\core\servers\basehttp.py", line 557, in __init__
    BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
  File "d:\bin\Python2.5.2\lib\SocketServer.py", line 522, in __init__
    self.handle()
  File "d:\u\kmt\django\trunk\django\core\servers\basehttp.py", line 602, in handle
    handler.run(self.server.get_app())
  File "d:\u\kmt\django\trunk\django\core\servers\basehttp.py", line 277, in run
    self.result = application(self.environ, self.start_response)
  File "d:\u\kmt\django\trunk\django\core\servers\basehttp.py", line 634, in __call__
    return self.application(environ, start_response)
  File "d:\u\kmt\django\trunk\django\core\handlers\wsgi.py", line 222, in __call__
    response = self.get_response(request)
  File "d:\u\kmt\django\trunk\django\core\handlers\base.py", line 86, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "d:\u\kmt\django\trunk\django\contrib\auth\decorators.py", line 67, in __call__
    return self.view_func(request, *args, **kwargs)
  File "D:\u\kmt\software\web\xword\..\xword\crossword\views.py", line 51, in upload_puzzle
    form = UploadPuzzleForm(request.POST, request.FILES)
  File "d:\u\kmt\django\trunk\django\core\handlers\wsgi.py", line 152, in _get_post
    self._load_post_and_files()
  File "d:\u\kmt\django\trunk\django\core\handlers\wsgi.py", line 130, in _load_post_and_files
    self._post, self._files = self.parse_file_upload(self.META, self.environ['wsgi.input'])
  File "d:\u\kmt\django\trunk\django\http\__init__.py", line 124, in parse_file_upload
    return parser.parse()
  File "d:\u\kmt\django\trunk\django\http\multipartparser.py", line 205, in parse
    counters[i])
  File "d:\u\kmt\django\trunk\django\core\files\uploadhandler.py", line 142, in receive_data_chunk
    self.file.write(raw_data)
  File "d:\u\kmt\django\trunk\django\core\files\uploadedfile.py", line 89, in write
    def write(self, s):             return self._file.write(s)
  File "d:\u\kmt\django\trunk\django\core\files\temp.py", line 60, in write
    traceback.print_stack()

[anonymous]

Karen mentioned it above that there are general problems with exceptions in FileUploadHandlers.
When I raise an Exception in my custom File UploadHandler, than the server gets stuck.

[vung]

request.POST is computed the first time it is accessed and cached in request._post; if request._post is missing, the request.POST accessor starts parsing POST data.

Parsing POST data means that a django.http.MultiPartParser is instantiated and starts reading from the socket.

If an exception is raised in the meantime, request._post is not set and any access to request.POST will trigger again the parsing.

This second parsing happens when the exception is handled, in django.core.handlers.base.BaseHandler.handle_uncaught_exception, which calls repr(request), etc.

A second MultiPartParser is instantiated and since the first parser already consumed some bytes this second one will never get the chance to read up to content_length, so it will block waiting for content -- the exact place is django.http.LimitBytes.read()

I made a small test which reproduces the problem (no solution, though)

[vung]

Since the error is caused by triggering again POST data parsing, it would make sense for the fix to try to prevent it somehow; this is what the attached patch does.

Unrelated, but maybe useful for someone who (like me, until now) doesn't know: if an error is elusive, set DEBUG_PROPAGATE_EXCEPTIONS to True in settings.py

[Jacob]

(In [8748]) Fixed #8622: accessing POST after a POST handling exception no longer throws the server into an infinite loop. Thanks to vung for tracking this one down and fixing it.

[Jacob]

Milestone 1.0 deleted