Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#27658 closed Bug (fixed)

collectstatic overwrites newer files in remote storage

Reported by: Paolo Dente Owned by: nobody
Component: contrib.staticfiles Version: 1.10
Severity: Release blocker Keywords: staticfiles
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Paolo Dente)

The change to django/contrib/staticfiles/management/commands/collectstatic.py in commit 2cd2d188516475ddf256e6267cd82c495fb5c430 causes the command to overwrite all static files in remote storage (eg. S3), instead of only updatable ones.

A quick workaround is to override the command and restore the previous Command.delete_file() in the subclass, but I suppose this should be fixed.

Change History (11)

comment:1 Changed 3 years ago by Tim Graham

Easy pickings: unset

Could you be more specific about the steps to reproduce the issue (ideally without S3 involved since Django's test suite cannot rely on that)? If you could provide a test case for tests/staticfiles_tests that would be great.

comment:2 Changed 3 years ago by Rebkavets Vitali

This line Github link
full_path for remote storages such as S3 is always None
The clause was improperly refactored
It should look like

...
                    if (target_last_modified.replace(microsecond=0) >= source_last_modified.replace(microsecond=0) and
                            not full_path or full_path and not (self.symlink ^ os.path.islink(full_path))):
...
Last edited 3 years ago by Rebkavets Vitali (previous) (diff)

comment:3 Changed 3 years ago by Paolo Dente

Description: modified (diff)

comment:4 Changed 3 years ago by Paolo Dente

Description: modified (diff)

@Tim: as far as I can tell, any non-local storage should be affected so a possible test would require mocking Storage or the management command so that Command.local() is False; I can try and write one. On the other hand, it looks like @Vitali spotted the problem with the refactoring, so I don't know how to proceed from here.

comment:5 Changed 3 years ago by Claude Paroz

Triage Stage: UnreviewedAccepted

The next step is to write a test that demonstrates the current regression.

comment:6 Changed 3 years ago by Rebkavets Vitali

Has patch: set

comment:7 Changed 3 years ago by Claude Paroz

Needs tests: set

comment:8 Changed 3 years ago by Rebkavets Vitali

Here is a code snippet demonstrating the problem with the refactoring:

http://ideone.com/3kkY7N

comment:9 Changed 3 years ago by Tim Graham

Needs tests: unset
Severity: NormalRelease blocker

I added a test to the PR.

comment:10 Changed 3 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In c85831e4:

Fixed #27658 -- Prevented collectstatic from overwriting newer files in remote storages.

Thanks revimi for the initial patch.

comment:11 Changed 3 years ago by Tim Graham <timograham@…>

In da9b36c5:

[1.10.x] Fixed #27658 -- Prevented collectstatic from overwriting newer files in remote storages.

Thanks revimi for the initial patch.

Backport of c85831e4b7b5a7e4249df10327175b7251cb012d from master

Note: See TracTickets for help on using tickets.
Back to Top