Add a PersistedTemporaryFileUploadHandler for file upload

[Adnane Guettaf]

Using the PersistedTemporaryFileUploadHandler upload handler, Django will write the uploaded file to a temporary file stored in your system's temporary directory, as the TemporaryFileUploadHandler already do. However, the file this time will not be suppressed after your first read, and will persist until you delete it.

PersistedTemporaryFileUploadHandler class inherit from the TemporaryFileUploadHandler class.

Use cases

• One use case that I personally faced is having many threads reading from the same uploaded file. When one of the threads reads the file content, the file is automatically deleted, and so became unavailable for the other threads.
• Another use case was find in a ​stack overflow question, where a user was looking for a way to control the life time of the uploaded file.

Implementation

PersistedTemporaryUploadedFile

class PersistedTemporaryUploadedFile(UploadedFile):
    """
    A file uploaded to a temporary location (i.e. stream-to-disk).
    The file does not get deleted after a first read from disk
    """

    def __init__(self, name, content_type, size, charset, content_type_extra=None):
        _, ext = os.path.splitext(name)
        file = tempfile.NamedTemporaryFile(
            suffix=".upload" + ext,
            dir=settings.FILE_UPLOAD_TEMP_DIR,
            delete=False,  # forbid the file from being deleted automatically
        )
        super().__init__(file, name, content_type, size, charset, content_type_extra)

    def temporary_file_path(self):
        """Return the full path of this file."""
        return self.file.name

    def close(self):
        try:
            return self.file.close()
        except FileNotFoundError:
            # The file was moved or deleted before the tempfile could unlink
            # it. Still sets self.file.close_called and calls
            # self.file.file.close() before the exception.
            pass

PersistedTemporaryFileUploadHandler

class PersistedTemporaryFileUploadHandler(TemporaryFileUploadHandler):
    """
    Upload handler that streams data into a temporary file.
    The file does not get deleted after a first read and should
    be closed after use.
    """

    def new_file(self, *args, **kwargs):
        """
        Create the file object to append to as data is coming in.
        """
        super().new_file(*args, **kwargs)
        self.file = PersistedTemporaryUploadedFile(
            self.file_name, self.content_type, 0, self.charset, self.content_type_extra
        )

The upload handler should after be defined in the setting: FILE_UPLOAD_HANDLERS setting: django.core.files.uploadhandler.PersistedTemporaryFileUploadHandler

Tests

def test_upload_temporary_file_handler(self):
        with tempfile.NamedTemporaryFile() as temp_file:
            temp_file.write(b"a")
            temp_file.seek(0)
            response = self.client.post("/temp_file/", {"file": temp_file})
            temp_path = response.json()["temp_path"]
            # File get deleted after first access
            self.assertIs(os.path.exists(temp_path), False)

def test_upload_persisted_temporary_file_handler(self):
        with tempfile.NamedTemporaryFile() as temp_file:
            temp_file.write(b"a")
            temp_file.seek(0)
            response = self.client.post(
                "/temp_file/persisted_file/", {"file": temp_file}
            )
            temp_path = response.json()["temp_path"]
            # File does not get deleted after first access
            self.assertIs(os.path.exists(temp_path), True)
            os.remove(temp_path)
            self.assertIs(os.path.exists(temp_path), False)

N.B

Even thought it may be easy to forbid the temporary file from being deleted, I think it's a necessary feature or class to have withing the framework as it's a behavior that I would naturally consider available in such a great framework.

[Mariusz Felisiak]

Please first start a discussion on the ​django-developers mailing list (or ​Django Forum), where you'll reach a wider audience and see what other think, and ​follow the triaging guidelines with regards to wontfix tickets.

As for me, it sounds like a third-party package is the best way to proceed so that people can try it out.