Opened 9 years ago

Closed 4 years ago

#6543 closed Bug (duplicate)

serving files/iterated content with setting.USE_ETAGS = True

Reported by: pcicman Owned by: nobody
Component: Core (Other) Version: master
Severity: Normal Keywords: USE_ETAGS etag file zip md5 iterator
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

It seems there's a bug in django.core.servers.basehttp.FileWrapper. FileWraper should do seek(0) before new interation.
Problem comes only with using Etags and this line (in django.middleware.common.CommonMiddleware):

etag = md5.new(response.content).hexdigest()

this calls code in HttpResponse object:

def _get_content(self):
        if self.has_header('Content-Encoding'):
            return ''.join(self._container)
        return smart_str(''.join(self._container), self._charset)

and join iterates over FileWrapper object:

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
        raise StopIteration

but there's no return back to file begining. md5 Makes first iteration, and next iteration is called when response gets converted to string and set back to client.

Solution can probably be:

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
            self.filelike.seek(0)
        raise StopIteration

Better solution

Add some temporary cache variable to HttpResponse object, so content gets iterated only once (i think this can be much more faster solution, shorter exec time).
In this case there's no seek(0) required.

Attachments (1)

6543.diff (505 bytes) - added by Thomas Güttler 9 years ago.

Download all attachments as: .zip

Change History (8)

comment:1 Changed 9 years ago by pcicman

Wrong indentation used in solution :S

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
        self.filelike.seek(0)
        raise StopIteration

comment:2 Changed 9 years ago by Thomas Güttler

Has patch: set
Triage Stage: UnreviewedAccepted

I understand this problem. I wrote a patch which checks first, if
the filelike object has a seek method.

Is FileWrapper documented somewhere? It is the first time I heard of it.

Changed 9 years ago by Thomas Güttler

Attachment: 6543.diff added

comment:3 Changed 8 years ago by Jim Garrison

See bug #6527, which has a patch for iterating only once. In fact, it may make sense to mark this bug as a duplicate of that one.

comment:4 Changed 6 years ago by Julien Phalip

Type: Bug

comment:5 Changed 6 years ago by Julien Phalip

Easy pickings: unset
Needs tests: set
Severity: Normal

comment:6 Changed 5 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:7 Changed 4 years ago by Aymeric Augustin

Resolution: duplicate
Status: newclosed

This is a duplicate of #6027.

Note: See TracTickets for help on using tickets.
Back to Top