Opened 10 years ago

Closed 10 years ago

#22718 closed Bug (wontfix)

django.core.files.File doesn't handle certain file-like objects (e.g. zipfile.ZipExtFile)

Reported by: linkjuggler@… Owned by: M. Shafiq
Component: File uploads/storage Version: 1.6
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

django.core.files.File checks for presence of seek() method before calling in some places but not in others causing problems with file-like objects that do not implement seek (e.g. zipfile.ZipExtFile).

For example, given myapp.models with:

class ASimpleModel(models.Model):
    a_file_field=models.FileField(upload_to="docs") 

and a zipfile abc.zip containing a file with path "dir/file.txt", you get the following:

>>> from myapp.models import ASimpleModel
>>> from zipfile import ZipFile
>>> from django.core.files import File
>>> zip = ZipFile("abc.zip")
>>> zfile = File(zip.open("dir/file.txt")
>>> m = ASimpleModel()
>>> m.a_file_field.save("file.txt",zfile)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/django/db/models/fields/files.py", line 90, in save
    self._size = content.size
  File "/usr/lib/python2.7/dist-packages/django/core/files/base.py", line 45, in _get_size
    pos = self.file.tell()
UnsupportedOperation: seek
>>> 

Change History (5)

comment:1 by Aymeric Augustin, 10 years ago

Triage Stage: UnreviewedAccepted

comment:2 by M. Shafiq, 10 years ago

Owner: changed from nobody to M. Shafiq
Status: newassigned

comment:3 by M. Shafiq, 10 years ago

Digged down in this issue on EuroPython 2014 Django Sprint, and If I break the zfile = File(zip.open("dir/file.txt") as below

opened_zip_file = zip.open("dir/file.txt")
zfile = File(opened_zip_file)

and inspect on the opened_zip_file its an instance of class ZipExtFile(io.BufferedIOBase): http://hg.python.org/cpython/file/2.7/Lib/zipfile.py#l501 which being file-like object do not support seek interface on this file instance (also return False if you check opened_zip_file.seekable()).

1 - If we want to improve the python zipfile library to support this interface (if necessary?) the correct fix should be done by cpython.

2 - I don't see a practical example of this happening (to send file-like objects directly to django FileField) without being involved a user's custom logic, so I think this is an individual user's problem (not Django's) to solve it by using some trick / extracting zip file in /tmp directory of server or something like this. *I think some core django developer (if agreed?) please mark this ticket as invalid and close it.*

3 - Another peer sitting around suggested to develop a ZipFileField but I think I don't see any usage of it currently! (if someone need it please define some standard usage, interfaces, and examples and ask for a new feature ticket and I will appreciate if you place the link of the new ticket here if you do so! I might pick that one later).

comment:4 by anonymous, 10 years ago

For me (the issue reporter) the only issue was size() failing. So I solved it by wrapping the ZipFile in another class that copied the file_size attribute of ZipFile to the _size attribute of the wrapping class, and passed through the other file-like interface methods.

You could simply check for a "file_size" attribute in _get_size(), but I agree it's probably not worth it if ZipFile is the only class that causes the issue.

comment:5 by Tim Graham, 10 years ago

Resolution: wontfix
Status: assignedclosed

Closing based on last two comments.

Note: See TracTickets for help on using tickets.
Back to Top