id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
20620	CachedFileMixin.post_process breaks when cache size is exceeded	julians37@…	jcatalan	"Using staticfiles with {{{STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.CachedStaticFilesStorage'}}}, running collectstatic without the {{{--no-post-process}}} flag can break if the number of static files exceeds {{{settings.CACHES['staticfiles']['OPTIONS']['MAX_ENTRIES']}}} (i.e. 300 files in the default configuration), with an unhelpful error message:

{{{
ValueError: The file 'foo' could not be found with <django.contrib.staticfiles.storage.CachedStaticFilesStorage object at 0x94f654c>.
}}}

(Full stack trace below.)

The reason is that with more than {{{MAX_ENTRIES}}} files, some files might be evicted from the cache at some point before they are referenced by the post-processing code.

The workaround is to increase {{{MAX_ENTRIES}}} to a value larger than the number of static files.

I believe this can be reproduced fairly easily by setting {{{MAX_ENTRIES}}} to 1 and having a bunch of static files that reference each other (I think this bug only kicks in when {{{url_converter}}} has enough work to do, because that's where additional cache entries are created. So just dumping a bunch of empty files into the {{{static}}} directory won't do.)

It would be nice if (ideally) the post-processing code would use a different cache that never evicts items, or (less ideal) provide a more helpful error message when the limit is reached. Perhaps the easiest fix would be to guard the invocation of {{{self.cache.set}}} in {{{CachedFileMixin.url}}} to ensure that the cache still has capacity, but I'm not sure this is the correct or best fix for the issue.

{{{
stderr: Traceback (most recent call last):
  File ""manage.py"", line 10, in <module>
    execute_from_command_line(sys.argv)
  File ""/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py"", line 443, in execute_from_command_line
    utility.execute()
  File ""/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py"", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File ""/usr/local/lib/python2.7/dist-packages/django/core/management/base.py"", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File ""/usr/local/lib/python2.7/dist-packages/django/core/management/base.py"", line 232, in execute
    output = self.handle(*args, **options)
  File ""/usr/local/lib/python2.7/dist-packages/django/core/management/base.py"", line 371, in handle
    return self.handle_noargs(**options)
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py"", line 163, in handle_noargs
    collected = self.collect()
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py"", line 120, in collect
    for original_path, processed_path, processed in processor:
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/storage.py"", line 226, in post_process
    content = pattern.sub(converter, content)
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/storage.py"", line 167, in converter
    hashed_url = self.url(unquote(joined_result), force=True)
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/storage.py"", line 114, in url
    hashed_name = self.hashed_name(clean_name).replace('\\', '/')
  File ""/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/storage.py"", line 74, in hashed_name
    (clean_name, self))
ValueError: The file 'foo' could not be found with <django.contrib.staticfiles.storage.CachedStaticFilesStorage object at 0x94f654c>.
}}}
"	Bug	closed	contrib.staticfiles	1.5	Normal	worksforme		julians37@…	Accepted	0	0	0	0	0	0