Django

Code

Ticket #9632 (new)

Opened 8 months ago

Last modified 4 months ago

File.chunks contains potentially expensive size operation

Reported by: psagers Assigned to: nobody
Milestone: Component: File uploads/storage
Version: 1.0 Keywords: file
Cc: Triage Stage: Design decision needed
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

django.core.files.base.File's chunk method uses the file's size to determine how many chunks to return. Retrieving the size of a file is very fast for simple files, but may be very expensive for compressed files and other storage mechanisms. And in this case, it's completely unnecessary. Replacing the size-based loop with a simple check for the end of the stream has fewer dependencies and avoids a potentially expensive operation.

To cite one practical example, if the file is stored as a compressed file on disk, the current implementation will result in either decompressing the entire file twice or caching the entire decompressed file in memory. Removing the size dependency avoids both.

Attachments

chunks.diff (0.7 kB) - added by psagers on 11/19/08 00:28:37.

Change History

11/19/08 00:28:37 changed by psagers

  • attachment chunks.diff added.

02/26/09 19:10:26 changed by jacob

  • needs_better_patch changed.
  • stage changed from Unreviewed to Design decision needed.
  • needs_tests changed.
  • needs_docs changed.

Add/Change #9632 (File.chunks contains potentially expensive size operation)




Change Properties
Action