Opened 7 years ago

Closed 7 years ago

#18154 closed Bug (fixed)

Too many open files.

Reported by: eltonplima@… Owned by: Gabe Jackson
Component: File uploads/storage Version: 1.4
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


I'm doing a migration between two applications in django, everything was going normal until I got a class that has a field of type FileField. The import function normally to a given time in the system raises the following exception:

Original exception was:

Traceback (most recent call last):
  File "./", line 10, in <module>
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/", line 443, in execute_from_command_line
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/", line 382, in execute
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/", line 196, in run_from_argv
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/", line 232, in execute
  File "/home/eltonplima/workspace/ci/ci/migracao/management/commands/", line 291, in handle
  File "/home/eltonplima/workspace/ci/ci/migracao/management/commands/", line 139, in protocolo_relatorio_arquivo
  File "/home/eltonplima/workspace/ci/ci/migracao/protocolo/protocolos/", line 168, in __unicode__
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/", line 46, in _get_file
  File "/usr/local/lib/python2.7/dist-packages/django/core/files/", line 33, in open
  File "/usr/local/lib/python2.7/dist-packages/django/core/files/", line 156, in _open
IOError: [Errno 24] Too many open files: u'/home/eltonplima/workspace/ci/ci/media/uploads/estado_a/municipio_a/cliente_a/2010/03_marco/arquivo.pdf'

That is, django is opening each file but is not closing, so I'm forced to close it manually, in theory django should not automatically close the file?

My code:

conversoes = ArquivoProtocolo.objects.using('protocolo').all()
if conversoes.count() > 0:
    for conversao in conversoes:
        conversao.arquivo.close()# This should really be necessary?

Change History (3)

comment:1 Changed 7 years ago by Gabe Jackson

Needs documentation: set
Owner: changed from nobody to Gabe Jackson
Triage Stage: UnreviewedAccepted

comment:2 Changed 7 years ago by Aymeric Augustin

Generally this isn't an issue because you don't open thousands of files in a query. The file objects can be garbage collected and the file descriptors closed as soon as the request is finished.

Django can't close the files automatically because it doesn't know when you're done with the file. Specifically we can't use the context manager.

So I suppose it's just a gotcha we should document.

comment:3 Changed 7 years ago by Chris Beaven <smileychris@…>

Resolution: fixed
Status: newclosed

In [ffa6d95f65363b7f4f9047ab11561880be29049a]:

Fixed #18154 -- Documentation on closing File objects and best practices

Note: See TracTickets for help on using tickets.
Back to Top