Opened 20 months ago

Closed 20 months ago

Last modified 20 months ago

#21122 closed Bug (fixed)

runtests.py fails while trying to clean up temp directory on Windows

Reported by: marfire Owned by: marfire
Component: Testing framework Version: master
Severity: Normal Keywords: windows
Cc: k@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

There are two unfortunate aspects to this. One is that most test runs end like this:

Traceback (most recent call last):
  File "runtests.py", line 374, in <module>
    options.failfast, args)
  File "runtests.py", line 220, in django_tests
    teardown(state)
  File "runtests.py", line 188, in teardown
    shutil.rmtree(six.text_type(TEMP_DIR))
  File "C:\Program Other\Python27\Lib\shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Program Other\Python27\Lib\shutil.py", line 250, in rmtree
    os.remove(fullname)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process:
u'c:\\users\\me\\appdata\\local\\temp\\django_ekeere\\tmprx3hfn'

The other is that the temp directory isn't removed. That's not a severe problem when it comes to temp files and directories, but it's not particularly nice either.

As indicated in the stack trace, the error is caused by the calls to os.remove() in shutil.rmtree() in runtests.teardown(). As indicated in the os.remove() documentation: "On Windows, attempting to remove a file that is in use causes an exception to be raised." This includes files opened by the current process.

The root cause of this, then, is that a number of tests open temporary files but never close them, thus preventing the cleanup process from working properly.

Change History (6)

comment:1 Changed 20 months ago by marfire

  • Cc k@… added
  • Keywords windows added
  • Needs documentation unset
  • Needs tests unset
  • Owner changed from nobody to marfire
  • Patch needs improvement unset
  • Status changed from new to assigned

I propose a twofold solution:

  1. Tracking down the tests that are not closing their open temp files, and making sure they close them.
  1. Catching exceptions generated by shutil.rmtree() and printing a helpful message instead of a panicky stack trace.

comment:2 Changed 20 months ago by timo

  • Triage Stage changed from Unreviewed to Accepted

Could be related to #19905 - "ResourceWarning in formtools tests".

comment:3 Changed 20 months ago by marfire

I couldn't reproduce the other ticket but I added a note there.

comment:5 Changed 20 months ago by Tim Graham <timograham@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In 978e1351a60a0d7b44d367877784109ac01ebc62:

Fixed #21122 -- Improved clean up of test temp directory on Windows

  • Fixed test that didn't close the files it opened
  • Caught and handled exception when temp directory cannot be removed

comment:6 Changed 20 months ago by Tim Graham <timograham@…>

In 02c7dbd2559de4fa09cf6c644a2434e440230c0d:

[1.6.x] Fixed #21122 -- Improved clean up of test temp directory on Windows

  • Fixed test that didn't close the files it opened
  • Caught and handled exception when temp directory cannot be removed

Backport of 978e1351a6 of master

Note: See TracTickets for help on using tickets.
Back to Top