Code

Opened 6 years ago

Closed 3 years ago

#8907 closed (duplicate)

ORM-level Caching

Reported by: bradcater Owned by: anonymous
Component: Core (Cache system) Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: UI/UX:

Description

MOTIVATION:

  • Suppose the following is very expensive:

MyStuff.objects.filter(<conditions>)

We commonly use caching to avoid doing this operation often, which can
leave our code littered with the following:

    cached = cache.get(<cachekey>)
    if not cached:
        cached = MyStuff.objects.filter(<conditions>)
        cache.set(<cachekey>,cached,<time>)
    return cached

We seek to unify the Django ORM and caching in a natural way.

USE CASES:

  • To cache QuerySets, append .cache(<time>) to the end of the Django

ORM call. This will cache the given object for <time> seconds provided
that the cachekey is not already in the cache.

  • Example: MyStuff.objects.filter(extra_type=4).cache(3600)
  • This should return all MyStuff objects with an id greater than 10 and should store the result to the cache for 3600 seconds (1 hour).
  • To force caching (even if the cachekey already exists), provide the

force argument (e.g., .cache(<time>,force=True)).

  • Example: MyStuff.objects.filter(mass=20).cache(3600,force=True)
  • This should return all MyStuff objects with a mass of 20 and should cache the result for 3600 seconds even if it would over-write a previously-cached result.
  • To ensure that a cached value is not used, use .nocache() before any

specific call.

  • Example: MyStuff.objects.nocache().filter(user__id=7)
  • This should return all MyStuff objects with a User whose id is 7 and should not use a previously-cached value even if it exists.

NOTES:

  • Cachekeys are generated as the md5 hexdigest of the generated SQL.

Ultimately, this detail should not matter and is subject to change.

  • It is a subtle but salient point that you can call, e.g., MyStuff.objects.nocache().filter(contact__id=2).cache()

thus using cache() and nocache() together. It is desirable to do this
if you wish to get the latest results from the db (ignoring any data
currently in the cache) and caching them for later use.

DJANGO CHANGES:

  • .../django/db/models/query.py :
    • cache(self,time,force=False) added for QuerySet objects
    • nocache(self) added for QuerySet objects
    • _cachekey(self) added for QuerySet objects
    • _clone(self, klass=None, setup=False, kwargs) updated for QuerySet objects

Attachments (1)

query.py.diff (1.3 KB) - added by bradcater 6 years ago.
Diff adding ORM-level caching support.

Download all attachments as: .zip

Change History (7)

Changed 6 years ago by bradcater

Diff adding ORM-level caching support.

comment:1 follow-up: Changed 6 years ago by anonymous

  • Needs tests set
  • Owner changed from bradcater to anonymous
  • Status changed from new to assigned

comment:2 Changed 6 years ago by PhiR

patch is broken ? can't see from here

comment:3 Changed 6 years ago by kmtracey

Trac isn't displaying it properly, probably because it is a simple diff of a single file instead of an svn diff from the root of the source tree. You can still see if if you follow the link to "download in original format", it's just harder.

comment:4 in reply to: ↑ 1 Changed 5 years ago by rays

Replying to anonymous:

has there been any progress here? the diff is unviewable and now out of date. I'm really interested in what this does. Can the poster please attach a new diff or full file?

comment:5 Changed 5 years ago by anonymous

  • milestone post-1.0 deleted

Milestone post-1.0 deleted

comment:6 Changed 3 years ago by anonymous

  • Resolution set to duplicate
  • Status changed from assigned to closed

The outcome of #17 should resolve in a fix for this.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.