Add get_or_set method to cache API

[Alex Gaynor]

Add a cache.get_or_set() method to Django's cache abstraction, this takes the same arguments as get() except the 2nd argument is a callable, simple implementation looks like:

def get_or_set(self, key, func, **kwargs):
    val = self.get(key)
    if val is None:
        val = func()
        self.set(key, val)
    return val

[Chris Petrilli]

There's a big race condition in here, unfortunately. If you're using caching (and you should be), and multiple threads access the cache at the same time, then you end up with them over-writing one another. Depending on how this is used, it could be a big issue, or not. If you're talking to memcached, then what you want is add/replace or append/prepend. If you really want to use this approach, then you'll need to use cas, which is "check and set".

I don't believe that these are exposed at this point, unfortunately.

[Russell Keith-Magee]

The race condition is a problem, but as Alex noted on IRC, we should be able to avoid this by doing an add() rather than a set().

[James Snow]

Is work still ongoing on this? If not, I have a patch, docs and tests waiting in the wings.

[James Snow]

Not sure how else to go about contacting jmeed. I've gone ahead and added my patch, hopefully I'm not stepping on any toes.

[Jannis Leidel]

Removing 1.3 milestone since we are way over the feature freeze.

[Preston Holmes]

patch applies to rev 16843
Docs are clear
tests look pretty thorough

[Alex Gaynor]

Moving back to accepted, there are a few issues: 1) in the docs, get_or_set() should show its return values, 2) get_or_set() should *only* take a callable second argument, it's too easy to get the parameters wrong otherwise, 3) it should handle when add() raises an error (which it will if the key is already set).

Thanks for working on this ticket!

[Jacob]

Milestone 1.4 deleted

[mumino]

I made a patch based on snow0x2d0's patch.
my proposal is a little different signature for get_or_set:

   cached_data, is_set = cache.get_or_set(key, default, timeout=None, version=None)

maybe we shouldn't force second parameter be callable. I prefer

cache.get_or_set(score, 0}

to

cache.get_or_set(score, lambda: 0}

[James Snow]

Wow, I completely missed the updates to this apparently, my apologizes.

I created two topic github branches:

​https://github.com/snow0x2d0/django/tree/ticket_12982
​https://github.com/snow0x2d0/django/commit/07d441763ce2fbae801cf8fe024aa8ab72e204e9

​https://github.com/snow0x2d0/django/tree/ticket_12982_b
​https://github.com/snow0x2d0/django/commit/5942333a7994e1a33e050d85da03dfa30ae22b12

They both address two of Alex's concerns. Documentation has been updated to show the return values and an additional condition has been added to catch when add fails if a key exists. From looking over the other backend code it looks like in most cases add() will return False if a key exists already (this is the case for both memcache backends). In the case of database backend an add() with an existing key simply functions like a set().

Personally I don't know that there is likely to be confusion over the order of arguments. The order of parameters is just like the get() function and also follows the function name (namely get, then set). That being the case the primary difference between the two branches above is that the ticket_12982_b branch the second parameter _may_ be callable and in the ticket_12982 branch the second argument _must_ be callable (and a typeerror is thrown if not).

Both branches have been rebased to the current master commit (as of about 6am CST 10-29-12) and all tests pass. I can submit a proper pull request for either once approved.

[Berker Peksag]

​https://github.com/django/django/pull/3609

[Berker Peksag]

I've just updated the PR.

[Tim Graham]

Looks good, pending a few cosmetic tweaks.

[Berker Peksag <berker.peksag@…>]

In 34fb9091:

Fixed #12982 -- Added a get_or_set() method to the BaseCache backend.