Django

Code

Changeset 810

Show
Ignore:
Timestamp:
10/08/05 19:55:08 (3 years ago)
Author:
adrian
Message:

Fixed #580 -- Added mega support for generating Vary headers, including some view decorators, and changed the CacheMiddleware? to account for the Vary header. Also added GZipMiddleware and ConditionalGetMiddleware?, which are no longer handled by CacheMiddleware? itself. Also updated the cache.txt and middleware.txt docs. Thanks to Hugo and Sune for the excellent patches

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • django/trunk/django/middleware/cache.py

    r786 r810  
     1import copy 
    12from django.conf import settings 
    23from django.core.cache import cache 
     4from django.utils.cache import get_cache_key, learn_cache_key, patch_response_headers 
    35from django.utils.httpwrappers import HttpResponseNotModified 
    4 from django.utils.text import compress_string 
    5 import datetime, md5 
    66 
    77class CacheMiddleware: 
    88    """ 
    99    Cache middleware. If this is enabled, each Django-powered page will be 
    10     cached for CACHE_MIDDLEWARE_SECONDS seconds. Cache is based on URLs. Pages 
    11     with GET or POST parameters are not cached. 
     10    cached for CACHE_MIDDLEWARE_SECONDS seconds. Cache is based on URLs. 
    1211 
    13     If the cache is shared across multiple sites using the same Django 
    14     installation, set the CACHE_MIDDLEWARE_KEY_PREFIX to the name of the site, 
    15     or some other string that is unique to this Django instance, to prevent key 
    16     collisions. 
     12    Only parameter-less GET or HEAD-requests with status code 200 are cached. 
    1713 
    18     This middleware will also make the following optimizations: 
     14    This middleware expects that a HEAD request is answered with a response 
     15    exactly like the corresponding GET request. 
    1916 
    20     * If the CACHE_MIDDLEWARE_GZIP setting is True, the content will be 
    21       gzipped. 
     17    When a hit occurs, a shallow copy of the original response object is 
     18    returned from process_request. 
    2219 
    23     * ETags will be added, using a simple MD5 hash of the page's content. 
     20    Pages will be cached based on the contents of the request headers 
     21    listed in the response's "Vary" header. This means that pages shouldn't 
     22    change their "Vary" header. 
     23 
     24    This middleware also sets ETag, Last-Modified, Expires and Cache-Control 
     25    headers on the response object. 
    2426    """ 
     27    def __init__(self, cache_timeout=None, key_prefix=None): 
     28        self.cache_timeout = cache_timeout 
     29        if cache_timeout is None: 
     30            self.cache_timeout = settings.CACHE_MIDDLEWARE_SECONDS 
     31        self.key_prefix = key_prefix 
     32        if key_prefix is None: 
     33            self.key_prefix = settings.CACHE_MIDDLEWARE_KEY_PREFIX 
     34 
    2535    def process_request(self, request): 
    26         """ 
    27         Checks whether the page is already cached. If it is, returns the cached 
    28         version. Also handles ETag stuff. 
    29         """ 
    30         if request.GET or request.POST: 
    31             request._cache_middleware_set_cache = False 
     36        "Checks whether the page is already cached and returns the cached version if available." 
     37        if not request.META['REQUEST_METHOD'] in ('GET', 'HEAD') or request.GET: 
     38            request._cache_update_cache = False 
    3239            return None # Don't bother checking the cache. 
    3340 
    34         accept_encoding = '' 
    35         if settings.CACHE_MIDDLEWARE_GZIP: 
    36             try: 
    37                 accept_encoding = request.META['HTTP_ACCEPT_ENCODING'] 
    38             except KeyError: 
    39                 pass 
    40         accepts_gzip = 'gzip' in accept_encoding 
    41         request._cache_middleware_accepts_gzip = accepts_gzip 
    42  
    43         # This uses the same cache_key as views.decorators.cache.cache_page, 
    44         # so the cache can be shared. 
    45         cache_key = 'views.decorators.cache.cache_page.%s.%s.%s' % \ 
    46             (settings.CACHE_MIDDLEWARE_KEY_PREFIX, request.path, accepts_gzip) 
    47         request._cache_middleware_key = cache_key 
     41        cache_key = get_cache_key(request, self.key_prefix) 
     42        if cache_key is None: 
     43            request._cache_update_cache = True 
     44            return None # No cache information available, need to rebuild. 
    4845 
    4946        response = cache.get(cache_key, None) 
    5047        if response is None: 
    51             request._cache_middleware_set_cache = True 
    52             return None 
    53         else: 
    54             request._cache_middleware_set_cache = False 
    55             # Logic is from http://simon.incutio.com/archive/2003/04/23/conditionalGet 
    56             try: 
    57                 if_none_match = request.META['HTTP_IF_NONE_MATCH'] 
    58             except KeyError: 
    59                 if_none_match = None 
    60             try: 
    61                 if_modified_since = request.META['HTTP_IF_MODIFIED_SINCE'] 
    62             except KeyError: 
    63                 if_modified_since = None 
    64             if if_none_match is None and if_modified_since is None: 
    65                 pass 
    66             elif if_none_match is not None and response['ETag'] != if_none_match: 
    67                 pass 
    68             elif if_modified_since is not None and response['Last-Modified'] != if_modified_since: 
    69                 pass 
    70             else: 
    71                 return HttpResponseNotModified() 
    72         return response 
     48            request._cache_update_cache = True 
     49            return None # No cache information available, need to rebuild. 
     50 
     51        request._cache_update_cache = False 
     52        return copy.copy(response) 
    7353 
    7454    def process_response(self, request, response): 
    75         """ 
    76         Sets the cache, if needed. 
    77         """ 
    78         if request._cache_middleware_set_cache: 
    79             content = response.get_content_as_string(settings.DEFAULT_CHARSET) 
    80             if request._cache_middleware_accepts_gzip: 
    81                 content = compress_string(content) 
    82                 response.content = content 
    83                 response['Content-Encoding'] = 'gzip' 
    84             response['ETag'] = md5.new(content).hexdigest() 
    85             response['Content-Length'] = '%d' % len(content) 
    86             response['Last-Modified'] = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT') 
    87             cache.set(request._cache_middleware_key, response, settings.CACHE_MIDDLEWARE_SECONDS) 
     55        "Sets the cache, if needed." 
     56        if not request._cache_update_cache: 
     57            # We don't need to update the cache, just return. 
     58            return response 
     59        if not request.META['REQUEST_METHOD'] == 'GET': 
     60            # This is a stronger requirement than above. It is needed 
     61            # because of interactions between this middleware and the 
     62            # HTTPMiddleware, which throws the body of a HEAD-request 
     63            # away before this middleware gets a chance to cache it. 
     64            return response 
     65        if not response.status_code == 200: 
     66            return response 
     67        patch_response_headers(response, self.cache_timeout) 
     68        cache_key = learn_cache_key(request, response, self.cache_timeout, self.key_prefix) 
     69        cache.set(cache_key, response, self.cache_timeout) 
    8870        return response 

  • django/trunk/django/middleware/sessions.py

    r669 r810  
    11from django.conf.settings import SESSION_COOKIE_NAME, SESSION_COOKIE_AGE, SESSION_COOKIE_DOMAIN 
    22from django.models.core import sessions 
     3from django.utils.cache import patch_vary_headers 
    34import datetime 
    45 
    …… 
    6263        # If request.session was modified, or if response.session was set, save 
    6364        # those changes and set a session cookie. 
     65        patch_vary_headers(response, ('Cookie',)) 
    6466        try: 
    6567            modified = request.session.modified 

  • django/trunk/django/views/decorators/cache.py

    r786 r810  
    1 from django.core.cache import cache 
    2 from django.utils.httpwrappers import HttpResponseNotModified 
    3 from django.utils.text import compress_string 
    4 from django.conf.settings import DEFAULT_CHARSET 
    5 import datetime, md5 
     1""" 
     2Decorator for views that tries getting the page from the cache and 
     3populates the cache if the page isn't in the cache yet. 
    64 
    7 def cache_page(view_func, cache_timeout, key_prefix=''): 
    8     """ 
    9     Decorator for views that tries getting the page from the cache and 
    10     populates the cache if the page isn't in the cache yet. Also takes care 
    11     of ETags and gzips the page if the client supports it. 
     5The cache is keyed by the URL and some data from the headers. Additionally 
     6there is the key prefix that is used to distinguish different cache areas 
     7in a multi-site setup. You could use the sites.get_current().domain, for 
     8example, as that is unique across a Django project. 
    129 
    13     The cache is keyed off of the page's URL plus the optional key_prefix 
    14     variable. Use key_prefix if your Django setup has multiple sites that 
    15     use cache; otherwise the cache for one site would affect the other. A good 
    16     example of key_prefix is to use sites.get_current().domain, because that's 
    17     unique across all Django instances on a particular server. 
    18     """ 
    19     def _check_cache(request, *args, **kwargs): 
    20         try: 
    21             accept_encoding = request.META['HTTP_ACCEPT_ENCODING'] 
    22         except KeyError: 
    23             accept_encoding = '' 
    24         accepts_gzip = 'gzip' in accept_encoding 
    25         cache_key = 'views.decorators.cache.cache_page.%s.%s.%s' % (key_prefix, request.path, accepts_gzip) 
    26         response = cache.get(cache_key, None) 
    27         if response is None: 
    28             response = view_func(request, *args, **kwargs) 
    29             content = response.get_content_as_string(DEFAULT_CHARSET) 
    30             if accepts_gzip: 
    31                 content = compress_string(content) 
    32                 response.content = content 
    33                 response['Content-Encoding'] = 'gzip' 
    34             response['ETag'] = md5.new(content).hexdigest() 
    35             response['Content-Length'] = '%d' % len(content) 
    36             response['Last-Modified'] = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT') 
    37             cache.set(cache_key, response, cache_timeout) 
    38         else: 
    39             # Logic is from http://simon.incutio.com/archive/2003/04/23/conditionalGet 
    40             try: 
    41                 if_none_match = request.META['HTTP_IF_NONE_MATCH'] 
    42             except KeyError: 
    43                 if_none_match = None 
    44             try: 
    45                 if_modified_since = request.META['HTTP_IF_MODIFIED_SINCE'] 
    46             except KeyError: 
    47                 if_modified_since = None 
    48             if if_none_match is None and if_modified_since is None: 
    49                 pass 
    50             elif if_none_match is not None and response['ETag'] != if_none_match: 
    51                 pass 
    52             elif if_modified_since is not None and response['Last-Modified'] != if_modified_since: 
    53                 pass 
    54             else: 
    55                 return HttpResponseNotModified() 
    56         return response 
    57     return _check_cache 
     10Additionally, all headers from the response's Vary header will be taken into 
     11account on caching -- just like the middleware does. 
     12""" 
     13 
     14from django.utils.decorators import decorator_from_middleware 
     15from django.middleware.cache import CacheMiddleware 
     16 
     17cache_page = decorator_from_middleware(CacheMiddleware) 

  • django/trunk/docs/cache.txt

    r705 r810  
    33======================== 
    44 
    5 So, you got slashdotted. Now what? 
     5So, you got slashdotted_. Now what? 
    66 
    77Django's cache framework gives you three methods of caching dynamic pages in 
    …… 
    1010entire site. 
    1111 
     12.. _slashdotted: http://en.wikipedia.org/wiki/Slashdot_effect 
     13 
    1214Setting up the cache 
    1315==================== 
    1416 
    15 The cache framework is split into a set of "backends" that provide different 
    16 methods of caching data. There's a simple single-process memory cache (mostly 
    17 useful as a fallback) and a memcached_ backend (the fastest option, by far, if 
    18 you've got the RAM). 
     17The cache framework allows for different "backends" -- different methods of 
     18caching data. There's a simple single-process memory cache (mostly useful as a 
     19fallback) and a memcached_ backend (the fastest option, by far, if you've got 
     20the RAM). 
    1921 
    2022Before using the cache, you'll need to tell Django which cache backend you'd 
    2123like to use. Do this by setting the ``CACHE_BACKEND`` in your settings file. 
    2224 
    23 The CACHE_BACKEND setting is a "fake" URI (really an unregistered scheme). 
     25The ``CACHE_BACKEND`` setting is a "fake" URI (really an unregistered scheme). 
    2426Examples: 
    2527 
    …… 
    4042                                    probably don't want to use this except for 
    4143                                    testing. Note that this cache backend is 
    42                                     NOT threadsafe! 
     44                                    NOT thread-safe! 
    4345 
    4446    locmem:///                      A more sophisticated local memory cache; 
    …… 
    7375arguments. 
    7476 
     77.. _memcached: http://www.danga.com/memcached/ 
     78 
    7579The per-site cache 
    7680================== 
    7781 
    78 Once the cache is set up, the simplest way to use the cache is to simply 
    79 cache your entire site. Just add ``django.middleware.cache.CacheMiddleware`` 
    80 to your ``MIDDLEWARE_CLASSES`` setting, as in this example:: 
     82Once the cache is set up, the simplest way to use the cache is to cache your 
     83entire site. Just add ``django.middleware.cache.CacheMiddleware`` to your 
     84``MIDDLEWARE_CLASSES`` setting, as in this example:: 
    8185 
    8286    MIDDLEWARE_CLASSES = ( 
    …… 
    8589    ) 
    8690 
    87 Make sure it's the first entry in ``MIDDLEWARE_CLASSES``. (The order of 
    88 ``MIDDLEWARE_CLASSES`` matters.) 
    89  
    90 Then, add the following three required settings: 
     91(The order of ``MIDDLEWARE_CLASSES`` matters. See "Order of MIDDLEWARE_CLASSES" 
     92below.) 
     93 
     94Then, add the following three required settings to your Django settings file: 
    9195 
    9296* ``CACHE_MIDDLEWARE_SECONDS`` -- The number of seconds each page should be 
    …… 
    103107  zipping, and the cache will hold more pages because each one is smaller. 
    104108 
    105 Pages with GET or POST parameters won't be cached. 
    106  
    107 The cache middleware also makes a few more optimizations: 
    108  
    109 * Sets and deals with ``ETag`` headers. 
    110 * Sets the ``Content-Length`` header. 
     109The cache middleware caches every page that doesn't have GET or POST 
     110parameters. Additionally, ``CacheMiddleware`` automatically sets a few headers 
     111in each ``HttpResponse``: 
     112 
    111113* Sets the ``Last-Modified`` header to the current date/time when a fresh 
    112114  (uncached) version of the page is requested. 
    113  
    114 It doesn't matter where in the middleware stack you put the cache middleware. 
     115* Sets the ``Expires`` header to the current date/time plus the defined 
     116  ``CACHE_MIDDLEWARE_SECONDS``. 
     117* Sets the ``Cache-Control`` header to give a max age for the page -- again, 
     118  from the ``CACHE_MIDDLEWARE_SECONDS`` setting. 
     119 
     120See the `middleware documentation`_ for more on middleware. 
     121 
     122.. _`middleware documentation`: http://www.djangoproject.com/documentation/middleware/ 
    115123 
    116124The per-page cache 
    …… 
    135143        ... 
    136144 
    137 This will cache the result of that view for 15 minutes. (The cache timeout is 
    138 in seconds.) 
     145``cache_page`` takes a single argument: the cache timeout, in seconds. In the 
     146above example, the result of the ``slashdot_this()`` view will be cached for 15 
     147minutes. 
    139148 
    140149The low-level cache API 
    141150======================= 
    142151 
    143 There are times, however, that caching an entire rendered page doesn't gain 
    144 you very much. The Django developers have found it's only necessary to cache a 
    145 list of object IDs from an intensive database query, for example. In cases like 
    146 these, you can use the cache API to store objects in the cache with any level 
    147 of granularity you like. 
     152Sometimes, however, caching an entire rendered page doesn't gain you very much. 
     153For example, you may find it's only necessary to cache the result of an 
     154intensive database. In cases like this, you can use the low-level cache API to 
     155store objects in the cache with any level of granularity you like. 
    148156 
    149157The cache API is simple:: 
    150158 
    151     # the cache module exports a cache object that's automatically 
    152     # created from the CACHE_BACKEND setting 
     159    # The cache module exports a cache object that's automatically 
     160    # created from the CACHE_BACKEND setting. 
    153161    >>> from django.core.cache import cache 
    154162 
    155     # The basic interface is set(key, value, timeout_seconds) and get(key) 
     163    # The basic interface is set(key, value, timeout_seconds) and get(key). 
    156164    >>> cache.set('my_key', 'hello, world!', 30) 
    157165    >>> cache.get('my_key') 
    …… 
    162170    None 
    163171 
    164     # get() can take a default argument 
     172    # get() can take a default argument. 
    165173    >>> cache.get('my_key', 'has_expired') 
    166174    'has_expired' 
    …… 
    184192can be pickled safely, although keys must be strings. 
    185193 
    186 .. _memcached: http://www.danga.com/memcached/ 
     194Controlling cache: Using Vary headers 
     195===================================== 
     196 
     197The Django cache framework works with `HTTP Vary headers`_ to allow developers 
     198to instruct caching mechanisms to differ their cache contents depending on 
     199request HTTP headers. 
     200 
     201Essentially, the ``Vary`` response HTTP header defines which request headers a 
     202cache mechanism should take into account when building its cache key. 
     203 
     204By default, Django's cache system creates its cache keys using the requested 
     205path -- e.g., ``"/stories/2005/jun/23/bank_robbed/"``. This means every request 
     206to that URL will use the same cached version, regardless of user-agent 
     207differences such as cookies or language preferences. 
     208 
     209That's where ``Vary`` comes in. 
     210 
     211If your Django-powered page outputs different content based on some difference 
     212in request headers -- such as a cookie, or language, or user-agent -- you'll 
     213need to use the ``Vary`` header to tell caching mechanisms that the page output 
     214depends on those things. 
     215 
     216To do this in Django, use the convenient ``vary_on_headers`` view decorator, 
     217like so:: 
     218 
     219    from django.views.decorators.vary import vary_on_headers 
     220 
     221    # Python 2.3 syntax. 
     222    def my_view(request): 
     223        ... 
     224    my_view = vary_on_headers(my_view, 'User-Agent') 
     225 
     226    # Python 2.4 decorator syntax. 
     227    @vary_on_headers('User-Agent') 
     228    def my_view(request): 
     229        ... 
     230 
     231In this case, a caching mechanism (such as Django's own cache middleware) will 
     232cache a separate version of the page for each unique user-agent. 
     233 
     234The advantage to using the ``vary_on_headers`` decorator rather than manually 
     235setting the ``Vary`` header (using something like 
     236``response['Vary'] = 'user-agent'``) is that the decorator adds to the ``Vary`` 
     237header (which may already exist) rather than setting it from scratch. 
     238 
     239Note that you can pass multiple headers to ``vary_on_headers()``: 
     240 
     241    @vary_on_headers('User-Agent', 'Cookie') 
     242    def my_view(request): 
     243        ... 
     244 
     245Because varying on cookie is such a common case, there's a ``vary_on_cookie`` 
     246decorator. These two views are equivalent:: 
     247 
     248    @vary_on_cookie 
     249    def my_view(request): 
     250        ... 
     251 
     252    @vary_on_headers('Cookie') 
     253    def my_view(request): 
     254        ... 
     255 
     256Also note that the headers you pass to ``vary_on_headers`` are not case 
     257sensitive. ``"User-Agent"`` is the same thing as ``"user-agent"``. 
     258 
     259You can also use a helper function, ``patch_vary_headers()``, directly:: 
     260 
     261    from django.utils.cache import patch_vary_headers 
     262    def my_view(request): 
     263        ... 
     264        response = render_to_response('template_name', context) 
     265        patch_vary_headers(response, ['Cookie']) 
     266        return response 
     267 
     268``patch_vary_headers`` takes an ``HttpResponse`` instance as its first argument 
     269and a list/tuple of header names as its second argument. 
     270 
     271.. _`HTTP Vary headers`: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.44 
     272 
     273Other optimizations 
     274=================== 
     275 
     276Django comes with a few other pieces of middleware that can help optimize your 
     277apps' performance: 
     278 
     279    * ``django.middleware.http.ConditionalGetMiddleware`` adds support for 
     280      conditional GET. This makes use of ``ETag`` and ``Last-Modified`` 
     281      headers. 
     282 
     283    * ``django.middleware.gzip.GZipMiddleware`` compresses content for browsers 
     284      that understand gzip compression (all modern browsers). 
     285 
     286Order of MIDDLEWARE_CLASSES 
     287=========================== 
     288 
     289If you use ``CacheMiddleware``, it's important to put it in the right place 
     290within the ``MIDDLEWARE_CLASSES`` setting, because the cache middleware needs 
     291to know which headers by which to vary the cache storage. Middleware always 
     292adds something the ``Vary`` response header when it can. 
     293 
     294Put the ``CacheMiddleware`` after any middlewares that might add something to 
     295the ``Vary`` header. The following middlewares do so: 
     296 
     297    * ``SessionMiddleware`` adds ``Cookie`` 
     298    * ``GzipMiddleware`` adds ``Accept-Encoding`` 

  • django/trunk/docs/middleware.txt

    r585 r810  
    8989    automatic documentation system. 
    9090 
     91``django.middleware.gzip.GZipMiddleware`` 
     92    Compresses content for browsers that understand gzip compression (all 
     93    modern browsers). 
     94 
     95``django.middleware.http.ConditionalGetMiddleware`` 
     96    Handles conditional GET operations. If the response has a ``ETag`` or 
     97    ``Last-Modified`` header, and the request has ``If-None-Match`` or 
     98    ``If-Modified-Since``, the response is replaced by an HttpNotModified. 
     99 
     100    Also removes the content from any response to a HEAD request and sets the 
     101    ``Date`` and ``Content-Length`` response-headers. 
     102 
    91103``django.middleware.sessions.SessionMiddleware`` 
    92104    Enables session support. See the `session documentation`_.