Allowing Callbacks/Handlers to be called on Cache entry Expiration

[Dylan Herman]

I read a suggestion about this here: ​http://stackoverflow.com/questions/20866460/django-cache-backend-how-to-implement-callback-when-cache-timeout .

I have written up an implementation of it with some tests for FileBasedCache (filebased.py) and want to add it to locmem.py too (not done yet).

Here are code snippets from filebased.py that have been tested:

The edits are in set and _is_expired

 def set(self, key, value, timeout=DEFAULT_TIMEOUT, version=None,handler=None):
        self._createdir()  # Cache dir can be deleted at any time.
        fname = self._key_to_file(key, version)
        self._cull()  # make some room if necessary
        fd, tmp_path = tempfile.mkstemp(dir=self._dir)
        renamed = False
        try:
            with io.open(fd, 'wb') as f:
                expiry = self.get_backend_timeout(timeout)
                f.write(pickle.dumps(expiry, pickle.HIGHEST_PROTOCOL))
                #check if handler exists
                if handler:
                  #write handler to pickle
                  f.write(pickle.dumps(handler,pickle.HIGHEST_PROTOCOL))
                f.write(zlib.compress(pickle.dumps(value, pickle.HIGHEST_PROTOCOL)))
            file_move_safe(tmp_path, fname, allow_overwrite=True)
            renamed = True
        finally:
            if not renamed:
                os.remove(tmp_path)


 def _is_expired(self, f):
        """
        Takes an open cache file and determines if it has expired,
        deletes the file if it is has passed its expiry time.
        """
        exp = pickle.load(f)
        if exp is not None and exp < time.time():
            #if there is a handler, call it
            try:
              handler = pickle.load(f)
              handler(exp)
            except pickle.UnpicklingError:
             #handler not added(no handler specified), so pickling error occurs
              pass
            f.close()  # On Windows a file has to be closed before deleting
            self._delete(f.name)
            return True
        return False

The handler has the following format

        class Handler(object):
              def __call__(self,exp):
                    ...

[Tim Graham]

I'm not sure about this feature. It won't work for memcached where cache entries may be evicted without Django's knowledge, correct?

[Dylan Herman]

Replying to timgraham:

I'm not sure about this feature. It won't work for memcached where cache entries may be evicted without Django's knowledge, correct?

I'm not sure about memcached, but I believe for the db cache it won't either. It seems to work nicely for filebased though, and possibly for locmem

[Tim Graham]

I think the feature is too specialized for inclusion in Django. You could ask on the DevelopersMailingList to see if anyone else feels it's an appropriate use of a cache, but I'm skeptical. There are probably better ways to solve the problem in the linked ticket than this idea. Also requiring file-based or local memory caching in production is a bit impractical.