Django

Code

Changeset 2758

Show
Ignore:
Timestamp:
04/27/06 22:39:27 (2 years ago)
Author:
adrian
Message:

magic-removal: Greatly enhanced docs/cache.txt -- now using caching segment from Django book

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • django/branches/magic-removal/docs/cache.txt

    r2607 r2758  
    33======================== 
    44 
    5 So, you got slashdotted_. Now what? 
    6  
    7 Django's cache framework gives you three methods of caching dynamic pages in 
    8 memory or in a database. You can cache the output of specific views, you can 
    9 cache only the pieces that are difficult to produce, or you can cache your 
    10 entire site. 
    11  
    12 .. _slashdotted: http://en.wikipedia.org/wiki/Slashdot_effect 
     5A fundamental tradeoff in dynamic Web sites is, well, they're dynamic. Each 
     6time a user requests a page, the Web server makes all sorts of calculations -- 
     7from database queries to template rendering to business logic -- to create the 
     8page that your site's visitor sees. This is a lot more expensive, from a 
     9processing-overhead perspective, than your standard read-a-file-off-the-filesystem 
     10server arrangement. 
     11 
     12For most Web applications, this overhead isn't a big deal. Most Web 
     13applications aren't washingtonpost.com or slashdot.org; they're simply small- 
     14to medium-sized sites with so-so traffic. But for medium- to high-traffic 
     15sites, it's essential to cut as much overhead as possible. 
     16 
     17That's where caching comes in. 
     18 
     19To cache something is to save the result of an expensive calculation so that 
     20you don't have to perform the calculation next time. Here's some pseudocode 
     21explaining how this would work for a dynamically generated Web page: 
     22 
     23    given a URL, try finding that page in the cache 
     24    if the page is in the cache: 
     25        return the cached page 
     26    else: 
     27        generate the page 
     28        save the generated page in the cache (for next time) 
     29        return the generated page 
     30 
     31Django comes with a robust cache system that lets you save dynamic pages so 
     32they don't have to be calculated for each request. For convenience, Django 
     33offers different levels of cache granularity: You can cache the output of 
     34specific views, you can cache only the pieces that are difficult to produce, or 
     35you can cache your entire site. 
     36 
     37Django also works well with "upstream" caches, such as Squid 
     38(http://www.squid-cache.org/) and browser-based caches. These are the types of 
     39caches that you don't directly control but to which you can provide hints (via 
     40HTTP headers) about which parts of your site should be cached, and how. 
    1341 
    1442Setting up the cache 
    1543==================== 
    1644 
    17 The cache framework allows for different "backends" -- different methods of 
    18 caching data. There's a simple single-process memory cache (mostly useful as a 
    19 fallback) and a memcached_ backend (the fastest option, by far, if you've got 
    20 the RAM). 
    21  
    22 Before using the cache, you'll need to tell Django which cache backend you'd 
    23 like to use. Do this by setting the ``CACHE_BACKEND`` in your settings file. 
    24  
    25 The ``CACHE_BACKEND`` setting is a "fake" URI (really an unregistered scheme). 
    26 Examples: 
    27  
    28     ==============================  =========================================== 
    29     CACHE_BACKEND                   Explanation 
    30     ==============================  =========================================== 
    31     memcached://127.0.0.1:11211/    A memcached backend; the server is running 
    32                                     on localhost port 11211.  You can use 
    33                                     multiple memcached servers by separating 
    34                                     them with semicolons. 
    35  
    36                                     This backend requires the 
    37                                     `Python memcached bindings`_. 
    38  
    39     db://tablename/                 A database backend in a table named 
    40                                     "tablename". This table should be created 
    41                                     with "django-admin createcachetable". 
    42  
    43     file:///var/tmp/django_cache/   A file-based cache stored in the directory 
    44                                     /var/tmp/django_cache/. 
    45  
    46     simple:///                      A simple single-process memory cache; you 
    47                                     probably don't want to use this except for 
    48                                     testing. Note that this cache backend is 
    49                                     NOT thread-safe! 
    50  
    51     locmem:///                      A more sophisticated local memory cache; 
    52                                     this is multi-process- and thread-safe. 
    53  
    54     dummy:///                       Doesn't actually cache; just implements the 
    55                                     cache backend interface and doesn't do 
    56                                     anything. This is an easy way to turn off 
    57                                     caching for a test environment. 
    58     ==============================  =========================================== 
    59  
    60 All caches may take arguments -- they're given in query-string style.  Valid 
    61 arguments are: 
     45The cache system requires a small amount of setup. Namely, you have to tell it 
     46where your cached data should live -- whether in a database, on the filesystem 
     47or directly in memory. This is an important decision that affects your cache's 
     48performance; yes, some cache types are faster than others. 
     49 
     50Your cache preference goes in the ``CACHE_BACKEND`` setting in your settings 
     51file. Here's an explanation of all available values for CACHE_BACKEND. 
     52 
     53Memcached 
     54--------- 
     55 
     56By far the fastest, most efficient type of cache available to Django, Memcached 
     57is an entirely memory-based cache framework originally developed to handle high 
     58loads at LiveJournal.com and subsequently open-sourced by Danga Interactive. 
     59It's used by sites such as Slashdot and Wikipedia to reduce database access and 
     60dramatically increase site performance. 
     61 
     62Memcached is available for free at http://danga.com/memcached/ . It runs as a 
     63daemon and is allotted a specified amount of RAM. All it does is provide an 
     64interface -- a *super-lightning-fast* interface -- for adding, retrieving and 
     65deleting arbitrary data in the cache. All data is stored directly in memory, 
     66so there's no overhead of database or filesystem usage. 
     67 
     68After installing Memcached itself, you'll need to install the Memcached Python 
     69bindings. They're in a single Python module, memcache.py, available at 
     70ftp://ftp.tummy.com/pub/python-memcached/ . If that URL is no longer valid, 
     71just go to the Memcached Web site (http://www.danga.com/memcached/) and get the 
     72Python bindings from the "Client APIs" section. 
     73 
     74To use Memcached with Django, set ``CACHE_BACKEND`` to 
     75``memcached://ip:port/``, where ``ip`` is the IP address of the Memcached 
     76daemon and ``port`` is the port on which Memcached is running. 
     77 
     78In this example, Memcached is running on localhost (127.0.0.1) port 11211:: 
     79 
     80    CACHE_BACKEND = 'memcached://127.0.0.1:11211/' 
     81 
     82One excellent feature of Memcached is its ability to share cache over multiple 
     83servers. To take advantage of this feature, include all server addresses in 
     84``CACHE_BACKEND``, separated by semicolons. In this example, the cache is 
     85shared over Memcached instances running on IP address 172.19.26.240 and 
     86172.19.26.242, both on port 11211:: 
     87 
     88    CACHE_BACKEND = 'memcached://172.19.26.240:11211;172.19.26.242:11211/' 
     89 
     90Memory-based caching has one disadvantage: Because the cached data is stored in 
     91memory, the data will be lost if your server crashes. Clearly, memory isn't 
     92intended for permanent data storage, so don't rely on memory-based caching as 
     93your only data storage. Actually, none of the Django caching backends should be 
     94used for permanent storage -- they're all intended to be solutions for caching, 
     95not storage -- but we point this out here because memory-based caching is 
     96particularly temporary. 
     97 
     98Database caching 
     99---------------- 
     100 
     101To use a database table as your cache backend, first create a cache table in 
     102your database by running this command:: 
     103 
     104    python manage.py createcachetable [cache_table_name] 
     105 
     106...where ``[cache_table_name]`` is the name of the database table to create. 
     107(This name can be whatever you want, as long as it's a valid table name that's 
     108not already being used in your database.) This command creates a single table 
     109in your database that is in the proper format that Django's database-cache 
     110system expects. 
     111 
     112Once you've created that database table, set your ``CACHE_BACKEND`` setting to 
     113``"db://tablename/"``, where ``tablename`` is the name of the database table. 
     114In this example, the cache table's name is ``my_cache_table``: 
     115 
     116    CACHE_BACKEND = 'db://my_cache_table' 
     117 
     118Database caching works best if you've got a fast, well-indexed database server. 
     119 
     120Filesystem caching 
     121------------------ 
     122 
     123To store cached items on a filesystem, use the ``"file://"`` cache type for 
     124``CACHE_BACKEND``. For example, to store cached data in ``/var/tmp/django_cache``, 
     125use this setting:: 
     126 
     127    CACHE_BACKEND = 'file:///var/tmp/django_cache' 
     128 
     129Note that there are three forward slashes toward the beginning of that example. 
     130The first two are for ``file://``, and the third is the first character of the 
     131directory path, ``/var/tmp/django_cache``. 
     132 
     133The directory path should be absolute -- that is, it should start at the root 
     134of your filesystem. It doesn't matter whether you put a slash at the end of the 
     135setting. 
     136 
     137Make sure the directory pointed-to by this setting exists and is readable and 
     138writable by the system user under which your Web server runs. Continuing the 
     139above example, if your server runs as the user ``apache``, make sure the 
     140directory ``/var/tmp/django_cache`` exists and is readable and writable by the 
     141user ``apache``. 
     142 
     143Local-memory caching 
     144-------------------- 
     145 
     146If you want the speed advantages of in-memory caching but don't have the 
     147capability of running Memcached, consider the local-memory cache backend. This 
     148cache is multi-process and thread-safe. To use it, set ``CACHE_BACKEND`` to 
     149``"locmem:///"``. For example:: 
     150 
     151    CACHE_BACKEND = 'locmem:///' 
     152 
     153Simple caching (for development) 
     154-------------------------------- 
     155 
     156A simple, single-process memory cache is available as ``"simple:///"``. This 
     157merely saves cached data in-process, which means it should only be used in 
     158development or testing environments. For example:: 
     159 
     160    CACHE_BACKEND = 'simple:///' 
     161 
     162Dummy caching (for development) 
     163------------------------------- 
     164 
     165Finally, Django comes with a "dummy" cache that doesn't actually cache -- it 
     166just implements the cache interface without doing anything. 
     167 
     168This is useful if you have a production site that uses heavy-duty caching in 
     169various places but a development/test environment on which you don't want to 
     170cache. In that case, set ``CACHE_BACKEND`` to ``"dummy:///"`` in the settings 
     171file for your development environment. As a result, your development 
     172environment won't use caching and your production environment still will. 
     173 
     174CACHE_BACKEND arguments 
     175----------------------- 
     176 
     177All caches may take arguments. They're given in query-string style on the 
     178``CACHE_BACKEND`` setting. Valid arguments are: 
    62179 
    63180    timeout 
    …… 
    67184    max_entries 
    68185        For the simple and database backends, the maximum number of entries 
    69         allowed in the cache before it is cleaned. Defaults to 300. 
     186        allowed in the cache before it is cleaned. Defaults to 300. 
    70187 
    71188    cull_percentage 
    …… 
    78195        at the expense of more cache misses. 
    79196 
    80 For example:: 
     197In this example, ``timeout`` is set to ``60``:: 
    81198 
    82199    CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=60" 
     200 
     201In this example, ``timeout`` is ``30`` and ``max_entries`` is ``400``:: 
     202 
     203    CACHE_BACKEND = "memcached://127.0.0.1:11211/?timeout=30&max_entries=400" 
    83204 
    84205Invalid arguments are silently ignored, as are invalid values of known 
    85206arguments. 
    86207 
    87 .. _memcached: http://www.danga.com/memcached/ 
    88 .. _Python memcached bindings: ftp://ftp.tummy.com/pub/python-memcached/ 
    89  
    90208The per-site cache 
    91209================== 
    92210 
    93 Once the cache is set up, the simplest way to use the cache is to cache your 
     211Once the cache is set up, the simplest way to use caching is to cache your 
    94212entire site. Just add ``django.middleware.cache.CacheMiddleware`` to your 
    95213``MIDDLEWARE_CLASSES`` setting, as in this example:: 
    …… 
    160278API to store objects in the cache with any level of granularity you like. 
    161279 
    162 The cache API is simple:: 
    163  
    164     # The cache module exports a cache object that's automatically 
    165     # created from the CACHE_BACKEND setting. 
     280The cache API is simple. The cache module, ``django.core.cache``, exports a 
     281``cache`` object that's automatically created from the ``CACHE_BACKEND`` 
     282setting:: 
     283 
    166284    >>> from django.core.cache import cache 
    167285 
    168     # The basic interface is set(key, value, timeout_seconds) and get(key). 
     286The basic interface is ``set(key, value, timeout_seconds)`` and ``get(key)``:: 
     287 
    169288    >>> cache.set('my_key', 'hello, world!', 30) 
    170289    >>> cache.get('my_key') 
    171290    'hello, world!' 
    172291 
    173     # (Wait 30 seconds...) 
     292The ``timeout_seconds`` argument is optional and defaults to the ``timeout`` 
     293argument in the ``CACHE_BACKEND`` setting (explained above). 
     294 
     295If the object doesn't exist in the cache, ``cache.get()`` returns ``None``:: 
     296 
     297    >>> cache.get('some_other_key') 
     298    None 
     299 
     300    # Wait 30 seconds for 'my_key' to expire... 
     301 
    174302    >>> cache.get('my_key') 
    175303    None 
    176304 
    177     # get() can take a default argument. 
    178     >>> cache.get('my_key', 'has_expired') 
    179     'has_expired' 
    180  
    181     # There's also a get_many() interface that only hits the cache once. 
    182     # Also, note that the timeout argument is optional and defaults to what 
    183     # you've given in the settings file. 
     305get() can take a ``default`` argument:: 
     306 
     307    >>> cache.get('my_key', 'has expired') 
     308    'has expired' 
     309 
     310There's also a get_many() interface that only hits the cache once. get_many() 
     311returns a dictionary with all the keys you asked for that actually exist in the 
     312cache (and haven't expired):: 
     313 
    184314    >>> cache.set('a', 1) 
    185315    >>> cache.set('b', 2) 
    186316    >>> cache.set('c', 3) 
    187  
    188     # get_many() returns a dictionary with all the keys you asked for that 
    189     # actually exist in the cache (and haven't expired). 
    190317    >>> cache.get_many(['a', 'b', 'c']) 
    191318    {'a': 1, 'b': 2, 'c': 3} 
    192319 
    193     # There's also a way to delete keys explicitly. 
     320Finally, you can delete keys explicitly with ``delete()``. This is an easy way 
     321of clearing the cache for a particular object:: 
     322 
    194323    >>> cache.delete('a') 
    195324 
    …… 
    197326can be pickled safely, although keys must be strings. 
    198327 
    199 Controlling cache: Using Vary headers 
    200 ===================================== 
    201  
    202 The Django cache framework works with `HTTP Vary headers`_ to allow developers 
    203 to instruct caching mechanisms to differ their cache contents depending on 
    204 request HTTP headers. 
    205  
    206 Essentially, the ``Vary`` response HTTP header defines which request headers a 
    207 cache mechanism should take into account when building its cache key. 
     328Upstream caches 
     329=============== 
     330 
     331So far, this document has focused on caching your *own* data. But another type 
     332of caching is relevant to Web development, too: caching performed by "upstream" 
     333caches. These are systems that cache pages for users even before the request 
     334reaches your Web site. 
     335 
     336Here are a few examples of upstream caches: 
     337 
     338    * Your ISP may cache certain pages, so if you requested a page from 
     339      somedomain.com, your ISP would send you the page without having to access 
     340      somedomain.com directly. 
     341 
     342    * Your Django Web site may site behind a Squid Web proxy 
     343      (http://www.squid-cache.org/) that caches pages for performance. In this 
     344      case, each request first would be handled by Squid, and it'd only be 
     345      passed to your application if needed. 
     346 
     347    * Your Web browser caches pages, too. If a Web page sends out the right 
     348      headers, your browser will use the local (cached) copy for subsequent 
     349      requests to that page. 
     350 
     351Upstream caching is a nice efficiency boost, but there's a danger to it: 
     352Many Web pages' contents differ based on authentication and a host of other 
     353variables, and cache systems that blindly save pages based purely on URLs could 
     354expose incorrect or sensitive data to subsequent visitors to those pages. 
     355 
     356For example, say you operate a Web e-mail system, and the contents of the 
     357"inbox" page obviously depend on which user is logged in. If an ISP blindly 
     358cached your site, then the first user who logged in through that ISP would have 
     359his user-specific inbox page cached for subsequent visitors to the site. That's 
     360not cool. 
     361 
     362Fortunately, HTTP provides a solution to this problem: A set of HTTP headers 
     363exist to instruct caching mechanisms to differ their cache contents depending 
     364on designated variables, and to tell caching mechanisms not to cache particular 
     365pages. 
     366 
     367Using Vary headers 
     368================== 
     369 
     370One of these headers is ``Vary``. It defines which request headers a cache 
     371mechanism should take into account when building its cache key. For example, if 
     372the contents of a Web page depend on a user's language preference, the page is 
     373said to "vary on language." 
    208374 
    209375By default, Django's cache system creates its cache keys using the requested 
    …… 
    242408header (which may already exist) rather than setting it from scratch. 
    243409 
    244 Note that you can pass multiple headers to ``vary_on_headers()``:: 
     410You can pass multiple headers to ``vary_on_headers()``:: 
    245411 
    246412    @vary_on_headers('User-Agent', 'Cookie') 
    …… 
    262428sensitive. ``"User-Agent"`` is the same thing as ``"user-agent"``. 
    263429 
    264 You can also use a helper function, ``patch_vary_headers()``, directly:: 
     430You can also use a helper function, ``django.utils.cache.patch_vary_headers``, 
     431directly:: 
    265432 
    266433    from django.utils.cache import patch_vary_headers 
    …… 
    274441and a list/tuple of header names as its second argument. 
    275442 
    276 .. _`HTTP Vary headers`: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44 
     443For more on Vary headers, see the `official Vary spec`_. 
     444 
     445.. _`official Vary spec`: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44 
    277446 
    278447Controlling cache: Using other headers 
    …… 
    318487        ... 
    319488 
    320 Any valid ``Cache-Control`` directive is valid in ``cache_control()``. For a 
    321 full list, see the `Cache-Control spec`_. Just pass the directives as keyword 
    322 arguments to ``cache_control()``, substituting underscores for hyphens. For 
    323 directives that don't take an argument, set the argument to ``True``. 
    324  
    325 Examples: 
    326  
    327     * ``@cache_control(max_age=3600)`` turns into ``max-age=3600``. 
    328     * ``@cache_control(public=True)`` turns into ``public``. 
     489Any valid ``Cache-Control`` HTTP directive is valid in ``cache_control()``. 
     490Here's a full list: 
     491 
     492    * ``public=True`` 
     493    * ``private=True`` 
     494    * ``no_cache=True`` 
     495    * ``no_transform=True`` 
     496    * ``must_revalidate=True`` 
     497    * ``proxy_revalidate=True`` 
     498    * ``max_age=num_seconds`` 
     499    * ``s_maxage=num_seconds`` 
     500 
     501For explanation of Cache-Control HTTP directives, see the `Cache-Control spec`_. 
    329502 
    330503(Note that the caching middleware already sets the cache header's max-age with 
    …… 
    333506precedence, and the header values will be merged correctly.) 
    334507 
    335 Disabling HTTP caching for a particular view 
    336 ============================================ 
    337  
    338 If you want to use headers to disable HTTP caching altogether for a particular 
    339 view, use one of the two utility functions the come with Django: 
    340  
    341     * ``django.utils.cache.add_never_cache_headers`` takes a single 
    342       ``HttpResponse`` object as its argument and alters the response to adds 
    343       headers that ensure the response won't be cached by browsers or other 
    344       caches. 
    345     * ``django.views.decorators.never_cache`` is a view decorator that does the 
    346       same thing but can be applied to a view function for convenience. 
    347       Example:: 
    348  
    349         from django.views.decorators.cache import never_cache 
    350         @never_cache 
    351         def myview(request): 
    352             # ... 
    353  
    354 Note that these functions disable HTTP caching (as described in the 'Controlling 
    355 Cache' sections of this document) -- they do *not* disable performance caching 
    356 (as described in the first few sections of this document). 
    357  
    358508.. _`Cache-Control spec`: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9 
    359509