Opened 6 years ago

Closed 5 years ago

Last modified 4 years ago

#11155 closed (duplicate)

Buffer size in get_image_dimensions too small

Reported by: seph Owned by: nobody
Component: File uploads/storage Version: master
Severity: Keywords: ImageField, get_image_dimensions, buffer
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The buffer size inside django/core/files/images.py, get_image_dimensions is currently set to 1024 bytes in trunk.
This wouldn't nescessarily mean many iterations if the image can be identified through PIL without loading the entire image. But using a 32 MB TIFF will result in excessive CPU usage due to the large amounts of iterations needed to reach 32 MB (somewhere over 30.000 iterations). This effectively means the ImageField which uses get_image_dimensions cannot operate under these conditions.

Increasing this buffer from 1K to 16K would mean a lot for general performance of the ImageField.

My current workaround is moneypatching the function to override the buffer size, but some kind of fix or setting would really be appreciated.

Attachments (1)

buffersize.diff (415 bytes) - added by seph 6 years ago.
Patch to set buffer size to 16k

Download all attachments as: .zip

Change History (6)

Changed 6 years ago by seph

Patch to set buffer size to 16k

comment:1 Changed 6 years ago by jacob

  • milestone changed from 1.1 to 1.2
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

Not a 1.1 issue.

Plus: why should you need to read the whole TIFF to get its dimensions? What sort of messed-up format does that?

comment:2 Changed 6 years ago by seph

Replying to jacob:

Plus: why should you need to read the whole TIFF to get its dimensions? What sort of messed-up format does that?

Yes, why would that be needed? Using ImageField has some notable effects on performance. Uploading an image of 32 mb locally to the dev server was practically impossible (waited for several minutes). It helped to switch to using PIL directly. I would caution anyone using ImageField as long as the performance is at this level. 1024 bytes of buffer just doesn't cut it when the entire image needs to be loaded. I can't imgine increasing the read-buffer would cause any issues as the entire image is loaded into ram anyway.

comment:3 Changed 6 years ago by kua

Just surfing around here looking for solutions, this appears to be the same issue as in #11158

The dimensions of a tiff file according to this:
http://www.compix.com/fileformattif.htm
occur in the first kilobyte

comment:4 Changed 5 years ago by ubernostrum

  • Resolution set to duplicate
  • Status changed from new to closed

This certainly looks like a duplicate of #11158; closing in favor of that ticket since it has a more recent patch.

comment:5 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top