Django

Code

Ticket #6262 (assigned)

Opened 1 year ago

Last modified 1 month ago

Cache templates

Reported by: SmileyChris Assigned to: SmileyChris (accepted)
Milestone: Component: Template system
Version: SVN Keywords:
Cc: mocksoul@gmail.com, lemuelf@sectioneleven.org Triage Stage: Design decision needed
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

Templates are currently loaded and compiled from the filesystem on every get_template call. It seems like a useful feature to be able to cache these templates.

This enhancement, in my limited testing, can easily halve the time it takes to load a template.

For this to be of any real use, #5701 should be applied - otherwise templates with any stringfilter decorated filters won't be get cached.

Attachments

cache_templates.diff (3.3 kB) - added by SmileyChris on 12/21/07 15:19:21.
test_templates_cache.tar.gz (2.7 kB) - added by SmileyChris on 12/22/07 00:56:45.
a small app to test speed

Change History

12/21/07 15:19:21 changed by SmileyChris

  • attachment cache_templates.diff added.

12/21/07 15:23:55 changed by SmileyChris

  • needs_better_patch changed.
  • needs_tests changed.
  • needs_docs changed.

A couple of calls I've made in the patch:

  • Templates are only cached while DEBUG is False (because if you're testing, you don't want caching getting in the way of your latest templates).
  • CACHE_TEMPLATES defaults to 600. I originally had it set to 0 (i.e. off) but I thought that people may as well benefit from it. I'd be just as comfortable with it being off by default.

Oh, and there's a comment in the patch saying "(Python 2.3 can't pickle any filter decorated with stringfilter)", since my understanding is that this will work in >2.3 after of the #5701 patch is applied.

12/21/07 22:13:00 changed by mtredinnick

Can you please drop in some details of the performance testing you did for this. In the past, the reason template caching wasn't done is because for real world applications the speed up wasn't that significant, when measure in terms of total request time.

12/22/07 00:56:45 changed by SmileyChris

  • attachment test_templates_cache.tar.gz added.

a small app to test speed

12/22/07 01:01:22 changed by SmileyChris

Ok, I've attached an app which I added to the end of a smallish project I'm working on.

The tests use the test client to get the response, so I guess that's somewhat close to total request time. The matching url uses the direct_to_template view to display a simple template (which inherits a simple base template). The client runs 1000 gets uncached (to warm up the drive), then times 1000 cached vs 1000 uncached requests.

Granted, this is testing on a laptop with slower drive speeds than your average server, but still there is a notable difference between speeds (even with simplistic templates) 

WARMING UP TO TEST FOR CACHE TEMPLATE SPEEDS
CACHED TEMPLATE...   3.9408
UNCACHED TEMPLATE... 4.4971

12/22/07 01:03:59 changed by SmileyChris

(this is just testing with the default locmem cache, by the way - I'm not sure if other ones would be faster or slower)

(follow-up: ↓ 7 ) 12/24/07 10:03:06 changed by jim-django@dsdd..org

This seems to be just cacheing the body of the template file - why would this be any better than relying on the OS's underlying buffers? It might make more sense to check real-world profile data to see how much time is spent parsing, rather than reading, the templates.

02/11/08 15:17:21 changed by korpios

  • cc set to korpios@korpios.com.

(in reply to: ↑ 5 ) 02/15/08 08:26:23 changed by akaihola

Replying to jim-django@dsdd..org:

... how much time is spent parsing, rather than reading, the templates.

It's caching the result of get_template_from_string(), which as far as I can tell is the compiled template.

02/16/08 16:04:02 changed by Simon Greenhill <dev@simon.net.nz>

  • stage changed from Unreviewed to Design decision needed.

02/21/08 17:11:46 changed by SmileyChris

  • owner changed from nobody to SmileyChris.

04/04/08 07:27:30 changed by MikeH <mike@mugwuffin.com>

I've benchmarked our applications with and without this patch. With the patch we deliver about 25% more pages per second than without.

08/21/08 11:04:46 changed by korpios

  • cc deleted.

08/30/08 23:52:32 changed by Vadim Fint <mocksoul@gmail.com>

  • cc set to mocksoul@gmail.com.

09/24/08 19:56:36 changed by SmileyChris

  • status changed from new to assigned.

Related ticket: #9154

09/25/08 16:27:37 changed by anonymous

I've benchmarked this with an app that uses full view caching, and it seems to be the cause of a large performance DEcrease :( I'm not sure why, but without this patch I get 900page/second (using ab), with it almost halves.

09/25/08 19:56:37 changed by SmileyChris

Could be something to do with the caching backend you're using (since this patch uses Django's caching). I don't really see how it could slow it down that much though - after the initial template cache, it should be returning your full cached view.

I find it hard to see how a one time cache can cause an ongoing half in your views if they are fully cached.

09/26/08 02:02:13 changed by anonymous

We are using memcache as the backend. I haven't had time to check it out fully yet, but it seems like it's when 'cache' gets imported on line 24 there is some work going on which slows down the response. I'll investigate further and report back.

10/30/08 01:03:23 changed by lemuelf

  • cc changed from mocksoul@gmail.com to mocksoul@gmail.com, lemuelf@sectioneleven.org.

Add/Change #6262 (Cache templates)




Change Properties
Action