FieldFile assumes efficient Storage.size

[Peter Sagerson]

FieldFile's _get_size, like many of its methods, passes the operation on to its Storage object. However, in this case, assuming that a storage mechanism can easily discover the size of a file by name is rather rash. If the file is compressed, for instance, it may be necessary to decompress the entire file to determine the size. Remote storage could suffer from similar issues.

Fortunately, FieldFile has a File object provided by the storage. The Storage derivative is responsible for returning a valid File object, which means it must be able to correctly report its size. By asking the File object for its size directly, we provide opportunities for caching so that a size request followed by a read operation need not trigger two independent retrievals of the data.

[Peter Sagerson]

Change in FieldFile._get_size

[Maxim Ivanov]

I've run into problem related to this bug. If we try to access file size in save() method of model, we are unable to get size, at least in case of InMemoryUploadedFile, because default storage unable to find file on FS.

[Karen Tracey]

Replying to redbaron:

I've run into problem related to this bug. If we try to access file size in save() method of model, we are unable to get size, at least in case of InMemoryUploadedFile, because default storage unable to find file on FS.

This is actually the problem identified in #10300.

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Change Easy pickings from NULL to False.

[Jacob]

I think this was fixed as part of #10300. If that's not true, please reopen.

[Kevin Turner]

This is still present in Django 1.11.11; FieldFile.size usually delegates to Storage instead of allowing its File to report (and, at its discretion, cache) the size:

​https://github.com/django/django/blob/1.11.11/django/db/models/fields/files.py#L73

I don't understand what the if self._committed is doing in there. Why wouldn't it be appropriate to return file.size whether it's committed or not? That's the implementation suggested by the initial patch here.

[Collin Anderson]

Why wouldn't it be appropriate to return file.size whether it's committed or not? That's the implementation suggested by the initial patch here.

self.file could be None unless the file has been opened. But maybe that check should instead be for _file, like the open() method. It looks like the code path for using file.size was added for efficiency in the first place in 68a890e79f660484d05482902663b6168f0bd71e.