Opened 7 months ago

Closed 7 months ago

#35397 closed Uncategorized (invalid)

override_settings for STORAGES loses OPTIONS

Reported by: Will Giddens Owned by: nobody
Component: File uploads/storage Version: 4.2
Severity: Normal Keywords: override_settings, storages
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If you use override_settings to change STORAGES, any OPTIONS set on the default storage are ignored.

The Settings.__init__() checks if STORAGES is overwritten and sets the DEFAULT_FILE_STORAGE to the new value. StorageHandler.backends checks to see if DEFAULT_FILE_STORAGE is overridden. It proceeds to overwrite the STORAGES["default"] based on the DEFAULT_FILE_STORAGE setting and overrides the explicitly set OPTIONS.

It looks like STATICFILES_STORAGE is handled the same way.

Change History (7)

comment:1 by Will Giddens, 7 months ago

It looks like this is a regression from https://github.com/django/django/commit/6b965c600054f970bdf94017ecf2e0e6e0a4326b.

To me, it seems to make more sense to resolve all config elements in Settings instead of StorageHandler to avoid these edge cases.

comment:2 by Sarah Boyce, 7 months ago

Resolution: needsinfo
Status: newclosed

Can you share how you are using override_settings to change STORAGES and the original STORAGES setting? That OPTIONS set on the default storage are ignored when using override_settings to me sounds like expected behaviour

comment:3 by Will Giddens, 7 months ago

Sure, here's a simplified snippet.

I'm using django-storages to configure S3 storage, but I confirmed Django's built-in storage engines also ignore the OPTIONS. I can provide a test case with the built-in storage engines if it helps.

settings.py:

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            "bucket_name": "prod-bucket",
            "file_overwrite": False,
        },
    },
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
    },
}

The test:

@override_settings(
    STORAGES={
        "default": {
            "BACKEND": "storages.backends.s3.S3Storage",
            "OPTIONS": {
                "bucket_name": "test-bucket",
                "file_overwrite": False,
            },
        },
    },
)
class ExampleStorageTestCase(TestCase):
    def test_override_worked(self):
        # This fails with AssertionError: None != 'test-bucket'
        self.assertEqual(default_storage.bucket_name, "test-bucket")

comment:4 by Will Giddens, 7 months ago

Resolution: needsinfo
Status: closednew

comment:5 by Sarah Boyce, 7 months ago

Resolution: invalid
Status: newclosed

I can provide a test case with the built-in storage engines if it helps.

Please do as this is working for me on main:

  • tests/file_storage/tests.py

    diff --git a/tests/file_storage/tests.py b/tests/file_storage/tests.py
    index 420314573d..4e54968c11 100644
    a b class FileStorageTests(SimpleTestCase):  
    567567                settings["FILE_UPLOAD_DIRECTORY_PERMISSIONS"],
    568568            )
    569569
     570    @override_settings(
     571        STORAGES={
     572            DEFAULT_STORAGE_ALIAS: {
     573                "BACKEND": "django.core.files.storage.InMemoryStorage",
     574                "OPTIONS": {
     575                    "location": "explicit_location",
     576                    "base_url": "explicit_base_url/",
     577                    "file_permissions_mode": 0o666,
     578                    "directory_permissions_mode": 0o666,
     579                },
     580            },
     581        }
     582    )
     583    def test_override_settings(self):
     584        self.assertEqual(default_storage._location, "explicit_location")
     585        self.assertEqual(default_storage._base_url, "explicit_base_url/")
     586        self.assertEqual(default_storage._file_permissions_mode, 0o666)
     587        self.assertEqual(default_storage._directory_permissions_mode, 0o666)
     588
    570589    def test_file_methods_pathlib_path(self):
    571590        p = Path("test.file")
    572591        self.assertFalse(self.storage.exists(p))

comment:6 by Will Giddens, 7 months ago

Resolution: invalid
Status: closednew

It looks like your test is good- it works on main for me as well. Whenever the DEFAULT_FILE_STORAGE setting was removed this bug goes away.

Since this only affects stable/4.2.x and stable/5.0.x should we pursue a fix or let it be?

comment:7 by Mariusz Felisiak, 7 months ago

Resolution: invalid
Status: newclosed

Django 4.2 is in extended support so it no longer receives bugfixes (except security patches), and this issue doesn't qualify for a backport to the Django 5.0. Closing as "invalid".

Note: See TracTickets for help on using tickets.
Back to Top