Use hashes for the filenames in "filebased" backend of django.core.cache

[Brian Johnson]

This refactor of the filebased cache creates the cache files as hashes broken into subdirectories like "6f/1e/d002ab5595859014ebf0951522d9" as suggested in ​this django-dev thread.

This also fixes a problem with add() and has_key() with expired cache items. I've included regression tests for both of those problems.

[Jacob]

I like what you've done here with the caching of the entry counts, but it's a bit over-engineered -- why not just have self.entry_count be an integer and update it on insert/delete?

And actually, now that I think about it, that's sort of a second issue. So could you redo this patch to just have the hashing (and not the entry count caching) and open a seperate ticket for that?

[Also, Simon, as a point of order: if a patch contains the word "TODO", it can't really be ready for checkin :)]

[Brian Johnson]

I like what you've done here with the caching of the entry counts, but it's a bit over-engineered -- why not just have self.entry_count be an integer and update it on insert/delete?

The cache is persistent (since it's saved in files), so you need to persist the entry count as-well. Then there's also the issue of multiple processes writing to the cache (changing the count).

On the other hand the count is only used to match to max_entries, so it needs to be close but probably not super precise. Perhaps there's a way to estimate the number of entries without keeping a running count and without going through the expense of walking the whole cache. Then again, maybe I'm just over-estimating the expense of walking the tree.

And actually, now that I think about it, that's sort of a second issue. So could you redo this patch to just have the hashing (and not the entry count caching) and open a seperate ticket for that?

OK

[Brian Johnson]

New patch - removes refcounting stubs

[Jacob]

(In [6887]) Fixed #6099: the filebased cache backend now uses md5 hashes of keys instead of sanitized filenames. For good measure, keys are partitioned into subdirectories using the first few bits of the hash. Thanks, sherbang.