File upload permission denied error on large files

[James Miller]

Hi, if an uploaded file exceeds the DATA_UPLOAD_MAX_MEMORY_SIZE of 2.5 mb, django starts streaming the file data to a file in /tmp.

When finished it copies it over to the its required file path location, but despite the permissions being set correctly, a permission denied error occurs.

Request Method: 	POST
Request URL: 	http://127.0.0.1:8000/xxxx/upload/
Django Version: 	3.1.5
Exception Type: 	PermissionError
Exception Value: 	

[Errno 13] Permission denied: '/opt/django_forum/django_forum/media/uploads/users/xxxxx/Albert_Memorial_London_-_May_2008_vWcDQT6.jpg'

Exception Location: 	/usr/local/lib/python3.9/shutil.py, line 329, in _copyxattr
Python Executable: 	/usr/local/bin/python
Python Version: 	3.9.1
Python Path: 	

['/opt/django_forum/django_forum',
 '/opt/django_forum/django_forum',
 '/etc/opt/django_forum',
 '/opt/django_forum',
 '/usr/local/lib/python39.zip',
 '/usr/local/lib/python3.9',
 '/usr/local/lib/python3.9/lib-dynload',
 '/usr/local/lib/python3.9/site-packages']

I have tried setting the following settings:

FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o755
FILE_UPLOAD_PERMISSIONS = 0o644

Most disturbingly, the large files are created in their final position with 0755 permissions.

I am using a containerised workflow as opposed to a venv but I am not certain that that should cause any issues.

[Tim Graham]

I don't think there are enough details here to confirm that Django is at fault. Please debug the issue and if Django is at fault, reopen with details. Thanks!

[René Fleschenberg]

We got some more information on IRC. Django calls copystat() at ​https://github.com/django/django/blob/master/django/core/files/move.py#L71 which, in this specific case, raises PermissionError with EACCES (13). Django catches the exception, but reraises it unless errno is EPERM: ​https://github.com/django/django/blob/master/django/core/files/move.py#L76

I don't know what exactly causes the PermissionError here, or if it would be good to also swallow EACCES. James mentioned that SELinux might play a role here.

Looks like this needs more research, but I think we should keep the ticket open for now. Since the permissions end up being less restrictive than the FILE_UPLOAD_PERMISSIONS setting, this also has security implications.

[James Miller]

Ok, so I am investigating selinux contexts on files and directories in containers, and it seems I may have volume mounted the directories that I share with the host using incorrect selinux context flags. I used a lower case z (share context between containers) when I should have used an upper case Z (context is private to container).
I am going to recreate my container with the correct selinux flag on volume mounts and see if it fixes the error.

[James Miller]

So I recreated a pod using the more correct selinux context switch, but with no luck regarding this issue.

I think it is probably an issue with python itself. see:

​https://bugs.python.org/issue38893

[Mariusz Felisiak]

Closing as invalid, unless someone can prove it's Django's fault or there is anything we can improve in Django itself.