Opened 9 months ago

Closed 7 months ago

Last modified 3 months ago

#35326 closed Bug (fixed)

OverwritingStorageTests fail if a TemporaryUploadedFile is used

Reported by: bcail Owned by: bcail
Component: File uploads/storage Version: dev
Severity: Normal Keywords:
Cc: bcail, Tim Graham Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Ticket #28144 added the option of using custom flags on a storage object to allow overwriting files in storage. However, this doesn't seem to work for temporary uploaded files, since the alternate path is taken in the _save method.

Here is an example test that fails for me - it loops forever:

diff --git a/tests/file_storage/tests.py b/tests/file_storage/tests.py
index 420314573d..d404300708 100644
--- a/tests/file_storage/tests.py
+++ b/tests/file_storage/tests.py
@@ -648,6 +648,34 @@ class OverwritingStorageTests(FileStorageTests):
         finally:
             self.storage.delete(name)
 
+    def test_save_overwrite_behavior_temp_file(self):
+        """Saving to same file name twice overwrites the first file."""
+        name = "test.file"
+        self.assertFalse(self.storage.exists(name))
+        content_1 = b"content one"
+        content_2 = b"second content"
+        f_1 = TemporaryUploadedFile('tmp1', 'text/plain', 11, 'utf8')
+        f_1.write(content_1)
+        f_1.seek(0)
+        f_2 = TemporaryUploadedFile('tmp2', 'text/plain', 14, 'utf8')
+        f_2.write(content_2)
+        f_2.seek(0)
+        stored_name_1 = self.storage.save(name, f_1)
+        try:
+            self.assertEqual(stored_name_1, name)
+            self.assertTrue(self.storage.exists(name))
+            self.assertTrue(os.path.exists(os.path.join(self.temp_dir, name)))
+            with self.storage.open(name) as fp:
+                self.assertEqual(fp.read(), content_1)
+            stored_name_2 = self.storage.save(name, f_2)
+            self.assertEqual(stored_name_2, name)
+            self.assertTrue(self.storage.exists(name))
+            self.assertTrue(os.path.exists(os.path.join(self.temp_dir, name)))
+            with self.storage.open(name) as fp:
+                self.assertEqual(fp.read(), content_2)
+        finally:
+            self.storage.delete(name)

Change History (19)

comment:1 by alexialg05, 9 months ago

can i be assigned on this issue?

comment:2 by bcail, 9 months ago

@alexialg05, yes, you can assign yourself to this issue, but we should probably see if others confirm this as an issue. If it is confirmed as an issue, there may be design questions to figure out.

comment:3 by Natalia Bidart, 9 months ago

Cc: Tim Graham added
Triage Stage: UnreviewedAccepted

Hello bcail, thank you for your report!

This definitely seems like a valid issue, file_move_safe (from django.core.files.move) requires the allow_overwrite flag to be passed to perform overwrites. I don't have clarity on the possible fix, I agree that it requires a design decision. A forum post seeking advice would be the best next step, I think.

comment:4 by bcail, 9 months ago

Thanks, @Natalia. I opened a new forum post.

comment:5 by bcail, 9 months ago

I opened a draft PR with a possible direction we could go.

comment:6 by Natalia Bidart, 9 months ago

Has patch: set
Owner: changed from nobody to bcail
Status: newassigned

comment:7 by Sarah Boyce, 8 months ago

Patch needs improvement: set

comment:8 by bcail, 7 months ago

Patch needs improvement: unset

comment:9 by Sarah Boyce, 7 months ago

Patch needs improvement: set

comment:10 by bcail, 7 months ago

Patch needs improvement: unset

comment:11 by Sarah Boyce, 7 months ago

Patch needs improvement: set

comment:12 by bcail, 7 months ago

Patch needs improvement: unset

comment:13 by Sarah Boyce, 7 months ago

Triage Stage: AcceptedReady for checkin

comment:14 by Sarah Boyce <42296566+sarahboyce@…>, 7 months ago

Resolution: fixed
Status: assignedclosed

In 0b33a3a:

Fixed #35326 -- Added allow_overwrite parameter to FileSystemStorage.

comment:15 by Sarah Boyce <42296566+sarahboyce@…>, 7 months ago

In 480ccf90:

Refs #35326 -- Made cosmetic edits to 5.1 release notes.

comment:16 by Sarah Boyce <42296566+sarahboyce@…>, 5 months ago

In 8d6a20b:

Fixed #35604, Refs #35326 -- Made FileSystemStorage.exists() behaviour independent from allow_overwrite.

Partially reverts 0b33a3abc2ca7d68a24f6d0772bc2b9fa603744e.

Storage.exists(name) was documented to "return False if
the name is available for a new file." but return True if
the file exists. This is ambiguous in the overwrite file
case. It will now always return whether the file exists.

Thank you to Natalia Bidart and Josh Schneier for the
review.

comment:17 by Sarah Boyce <42296566+sarahboyce@…>, 5 months ago

In e42defb:

[5.1.x] Fixed #35604, Refs #35326 -- Made FileSystemStorage.exists() behaviour independent from allow_overwrite.

Partially reverts 0b33a3abc2ca7d68a24f6d0772bc2b9fa603744e.

Storage.exists(name) was documented to "return False if
the name is available for a new file." but return True if
the file exists. This is ambiguous in the overwrite file
case. It will now always return whether the file exists.

Thank you to Natalia Bidart and Josh Schneier for the
review.

Backport of 8d6a20b656ff3fa18e36954668a44a831c2f6ddd from main.

comment:18 by nessita <124304+nessita@…>, 3 months ago

In 47f18a72:

Refs #35326 -- Adjusted deprecation warning stacklevel in FileSystemStorage.OS_OPEN_FLAGS.

comment:19 by Natalia <124304+nessita@…>, 3 months ago

In 8f5d2c37:

[5.1.x] Refs #35326 -- Adjusted deprecation warning stacklevel in FileSystemStorage.OS_OPEN_FLAGS.

Backport of 47f18a722624527cc72eef44cfc9d1e07ea4b4e0 from main.

Note: See TracTickets for help on using tickets.
Back to Top