id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
16082	Directory creation race condition in FileSystemStorage	Pi Delport	nobody	"My company recently encountered the following problem on a production server:

{{{
Traceback (most recent call last):

[...]

 File ""[...]"", line 191, in process_view
   storage.save(path, File([...]))

 File ""[...]/python2.6/site-packages/django/core/files/storage.py"", line 47, in save
   name = self._save(name, content)

 File ""[...]/python2.6/site-packages/django/core/files/storage.py"", line 146, in _save
   os.makedirs(directory)

 File ""[...]/python2.6/os.py"", line 157, in makedirs
   mkdir(name, mode)

OSError: [Errno 17] File exists: '[MEDIA_ROOT]/avatars/foo'
}}}

I tracked this down to a race condition in {{{FileSystemStorage._save}}}. When there is enough load, two concurrent processes can attempt to create the same intermediate directory, causing the second one to fail entirely.

To handle this robustly, {{{FileSystemStorage}}} should catch the {{{EEXIST}}} from {{{os.makedirs}}} and continue normally: I implemented this along with some tests, in the attached patch.

Note: For comparison, Python 3.2 added a keyword argument to {{{os.makedirs}}} to do this: see [http://bugs.python.org/issue9299 Issue 9299: os.makedirs(): Add a keyword argument to suppress ""File exists"" exception]."	Bug	closed	File uploads/storage	dev	Normal	fixed	FileSystemStorage directory creation race condition		Ready for checkin	1	0	0	0	0	0