Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#28170 closed Bug (fixed)

File upload to a CIFS mount fails with Permissions Denied

Reported by: Derrick Jackson Owned by: Peter J. Farrell
Component: File uploads/storage Version: 1.11
Severity: Release blocker Keywords:
Cc: pjf@…, derrick.jackson73@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Tim Graham)

I upgraded from DJango 1.10.6 to 1.11 and can no longer use FileField to upload files larger than 2.5 MB. When an attempt is made to upload a larger file I get the error below occurs. Please note if I downgrade to 1.10.6 again the upload feature works just fine.

Django Version:	1.11
Exception Type:	PermissionError
Exception Value:	
[Errno 1] Operation not permitted
Exception Location:	/usr/lib/python3.5/shutil.py in copystat, line 195
Python Executable:	/usr/bin/python3
Python Version:	3.5.2

Environment:


Request Method: POST
Request URL: http://localhost:8000/orders/service/vpn/5/

Django Version: 1.11
Python Version: 3.5.2
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'impersonate',
 'stronghold',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'ajax_select',
 'django_object_actions',
 'ckeditor',
 'mail_templated',
 'administrative',
 'changes',
 'decommission',
 'ecm',
 'feedback',
 'file_manager',
 'gunicorn',
 'orders',
 'old_net_chg_req',
 'server_request',
 'security',
 'shared',
 'system',
 'utilities',
 'downtime',
 'debug_toolbar']
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'shared.middleware.CustomDowntimeMiddleware',
 'stronghold.middleware.LoginRequiredMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'shared.middleware.GlobalRequestMiddleware',
 'shared.middleware.UserTracebackMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'impersonate.middleware.ImpersonateMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'debug_toolbar.middleware.DebugToolbarMiddleware')



Traceback:

File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/vagrant/orders/views_services.py" in service_vpn
  538.                     service_order_item.save()

File "/vagrant/orders/models.py" in save
  1150.         super(ServiceOrder, self).save(*args, **kwargs)

File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py" in save
  806.                        force_update=force_update, update_fields=update_fields)

File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py" in save_base
  836.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py" in _save_table
  900.                       for f in non_pks]

File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py" in <listcomp>
  900.                       for f in non_pks]

File "/usr/local/lib/python3.5/dist-packages/django/db/models/fields/files.py" in pre_save
  297.             file.save(file.name, file.file, save=False)

File "/usr/local/lib/python3.5/dist-packages/django/db/models/fields/files.py" in save
  95.         self.name = self.storage.save(name, content, max_length=self.field.max_length)

File "/usr/local/lib/python3.5/dist-packages/django/core/files/storage.py" in save
  54.         return self._save(name, content)

File "/usr/local/lib/python3.5/dist-packages/django/core/files/storage.py" in _save
  338.                     file_move_safe(content.temporary_file_path(), full_path)

File "/usr/local/lib/python3.5/dist-packages/django/core/files/move.py" in file_move_safe
  70.     copystat(old_file_name, new_file_name)

File "/usr/lib/python3.5/shutil.py" in copystat
  195.         follow_symlinks=follow)

Exception Type: PermissionError at /orders/service/vpn/5/
Exception Value: [Errno 1] Operation not permitted

Change History (15)

comment:1 by Tim Graham, 7 years ago

Component: Core (Other)File uploads/storage
Description: modified (diff)

I'm not sure if there's much we can do with the report without a way to reproduce the problem. If you can bisect to determine the commit where the behavior changed, that would be useful.

comment:2 by Tim Graham, 7 years ago

Resolution: needsinfo
Status: newclosed

comment:3 by Peter J. Farrell, 7 years ago

Cc: pjf@… added

comment:4 by Derrick Jackson, 7 years ago

Tim,

After chatting with a colleague of mine we were able to determine that this commit f734e2d is where the issue I reported started. Specifically changing
file.save(file.name, file, save=False) to file.save(file.name, file.save, save=False) in files.py causes the copystat error.

Please let me know if you need additional details regarding this issue.
Thanks,
Derrick

comment:5 by Derrick Jackson, 7 years ago

Cc: derrick.jackson73@… added
Resolution: needsinfo
Status: closednew

meant to re-open this ticket once I added my comment.

comment:6 by Derrick Jackson, 7 years ago

Additional Details: In my case, the files are larger than 2.5 MB. The tmp directory is on a local linux file system and the final location is a mounted CIFS drive. This Stack Overflow post covers my exact situation.

Last edited 7 years ago by Derrick Jackson (previous) (diff)

comment:7 by Tim Graham, 7 years ago

Severity: NormalRelease blocker
Summary: FileField Upload Permissions DeniedFile upload to a CIFS mount fails with Permissions Denied
Triage Stage: UnreviewedAccepted

Thanks for the additional details. Can you offer a patch that fixes the issue for you? I guess a regression test might involve some mocking to simulate a CIFS mount.

comment:8 by Derrick Jackson, 7 years ago

Tim... I will see what I can do. The tough part will be simulating a CIFS mount. I know the easy route would be to wrap line 296 of django/db/models/fields/files.py with a try/except but I know there's more to it.

try:
    file.save(file.name, file.file, save=False)
except Exception as e:
    file.save(file.name, file, save=False)

I'll keep digging into it.

comment:9 by Derrick Jackson, 7 years ago

Has patch: set

Tim,

A co-worker (pj@…) and I put together a suggested patch for the reported issue. We created the following pull request: https://github.com/django/django/pull/8486

comment:10 by Tim Graham, 7 years ago

Needs tests: set

Looks okay at first glance. The next step is to add some tests, probably by mocking shutils.copystat().

comment:11 by Derrick Jackson, 7 years ago

Needs tests: unset

The requested modifications and tests have been made. Here is the corresponding Pull Request: PR

comment:12 by Mariusz Felisiak, 7 years ago

Patch needs improvement: set

comment:13 by Peter J. Farrell, 7 years ago

Owner: changed from nobody to Peter J. Farrell
Patch needs improvement: unset
Status: newassigned
Triage Stage: AcceptedReady for checkin

comment:14 by Tim Graham <timograham@…>, 7 years ago

Resolution: fixed
Status: assignedclosed

In 789c2901:

Fixed #28170 -- Fixed file_move_safe() crash when moving files to a CIFS mount.

comment:15 by Tim Graham <timograham@…>, 7 years ago

In 7250393:

[1.11.x] Fixed #28170 -- Fixed file_move_safe() crash when moving files to a CIFS mount.

Backport of 789c290150a0a5e7312e152df281dbcaf4ec174e from master

Note: See TracTickets for help on using tickets.
Back to Top