Opened 7 years ago

Closed 3 years ago

#6543 closed Bug (duplicate)

serving files/iterated content with setting.USE_ETAGS = True

Reported by: pcicman Owned by: nobody
Component: Core (Other) Version: master
Severity: Normal Keywords: USE_ETAGS etag file zip md5 iterator
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

It seems there's a bug in django.core.servers.basehttp.FileWrapper. FileWraper should do seek(0) before new interation.
Problem comes only with using Etags and this line (in django.middleware.common.CommonMiddleware):

etag = md5.new(response.content).hexdigest()

this calls code in HttpResponse object:

def _get_content(self):
        if self.has_header('Content-Encoding'):
            return ''.join(self._container)
        return smart_str(''.join(self._container), self._charset)

and join iterates over FileWrapper object:

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
        raise StopIteration

but there's no return back to file begining. md5 Makes first iteration, and next iteration is called when response gets converted to string and set back to client.

Solution can probably be:

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
            self.filelike.seek(0)
        raise StopIteration

Better solution

Add some temporary cache variable to HttpResponse object, so content gets iterated only once (i think this can be much more faster solution, shorter exec time).
In this case there's no seek(0) required.

Attachments (1)

6543.diff (505 bytes) - added by guettli 7 years ago.

Download all attachments as: .zip

Change History (8)

comment:1 Changed 7 years ago by pcicman

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Wrong indentation used in solution :S

def next(self):
        data = self.filelike.read(self.blksize)
        if data:
            return data
        self.filelike.seek(0)
        raise StopIteration

comment:2 Changed 7 years ago by guettli

  • Has patch set
  • Triage Stage changed from Unreviewed to Accepted

I understand this problem. I wrote a patch which checks first, if
the filelike object has a seek method.

Is FileWrapper documented somewhere? It is the first time I heard of it.

Changed 7 years ago by guettli

comment:3 Changed 7 years ago by garrison

See bug #6527, which has a patch for iterating only once. In fact, it may make sense to mark this bug as a duplicate of that one.

comment:4 Changed 4 years ago by julien

  • Type set to Bug

comment:5 Changed 4 years ago by julien

  • Easy pickings unset
  • Needs tests set
  • Severity set to Normal

comment:6 Changed 3 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:7 Changed 3 years ago by aaugustin

  • Resolution set to duplicate
  • Status changed from new to closed

This is a duplicate of #6027.

Note: See TracTickets for help on using tickets.
Back to Top