Opened 8 months ago

Closed 8 months ago

Last modified 8 months ago

#29890 closed Bug (fixed)

FileSystemStorage._save() doesn't catch FileExistsError on concurrent os.mkdirs()

Reported by: mar77i Owned by: Vishvajit Pathak
Component: File uploads/storage Version: 2.1
Severity: Normal Keywords:
Cc: Vishvajit Pathak Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

My django tests (ran in --parallel) raised an exception which Django appears to attempt to be dealing with already:

Traceback (most recent call last):
  [...]
  File "[app dir]/models.py", line 56, in create_user
    return self._create_user(email, password, **extra_fields)
  File "[app dir]/models.py", line 50, in _create_user
    user.save(using=self._db)
  File "[app dir]/models.py", line 157, in save
    save=False
  File "[project dir]venv/lib/python3.5/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "[project dir]venv/lib/python3.5/site-packages/django/core/files/storage.py", line 49, in save
    return self._save(name, content)
  File "[project dir]venv/lib/python3.5/site-packages/django/core/files/storage.py", line 236, in _save
    os.makedirs(directory)
  File "/usr/lib/python3.5/os.py", line 241, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '[storage dir]'

Inspecting storage.py showes that os.makedirs() is guarded with a try...except block concerning FileNotFoundError, but apparently, FileExistsError appears to be what that exception handler was supposed to be catching.

Change History (6)

comment:1 Changed 8 months ago by Vishvajit Pathak

Cc: Vishvajit Pathak added
Owner: changed from nobody to Vishvajit Pathak
Status: newassigned

comment:2 Changed 8 months ago by Tim Graham

Summary: storage: catch the right exception on concurrent os.mkdirsFileSystemStorage._save() doesn't catch FileExistsError on concurrent os.mkdirs()
Triage Stage: UnreviewedAccepted

It looks like my mistake in 632c4ffd9cb1da273303bcd8005fff216506c795. We can add a test by mocking os.makedirs to raise FileExistsError.

comment:3 Changed 8 months ago by Srinivas Reddy Thatiparthy

Vishvajit ,

Are your working on a fix?

comment:4 Changed 8 months ago by Tim Graham

Has patch: set

I created a PR so we can get this fixed in tomorrow's bug fix release.

comment:5 Changed 8 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 98ef3829:

Fixed #29890 -- Fixed FileSystemStorage crash if concurrent saves try to create the same directory.

Regression in 632c4ffd9cb1da273303bcd8005fff216506c795.

comment:6 Changed 8 months ago by Tim Graham <timograham@…>

In cd7d6c8a:

[2.1.x] Fixed #29890 -- Fixed FileSystemStorage crash if concurrent saves try to create the same directory.

Regression in 632c4ffd9cb1da273303bcd8005fff216506c795.

Backport of 98ef3829e96ebc73d4d446f92465e671ff520d2b from master.

Note: See TracTickets for help on using tickets.
Back to Top