Added template cache in template loader

[novoya]

The following view module fails to make good reuse to the template
For each new request, all inherited/included templates will be recreated and compiled

tpl = loader.get_template('mytemplate.html')

def view_func1(request):
     '''
        Any inherited templates from tpl
        will created and recompiled again and again
     '''
     return HttpResponse(tpl.render(context)) 

Attached a very simple patch to solve the problem:
1.Simple template caching in the django.template.loader
2.Minimum change in django.template.loader_tags

[novoya]

templace cache patch

[ElliottM]

I'm confused, when was this fixed?

[James Bennett]

It wasn't fixed. Somebody needs to learn how to read the documentation on the ticket process to find out what "fixed" means (hint: not "some random person uploaded a patch").

[Malcolm Tredinnick]

This is almost certainly going to cause rednering bugs. The Python objects representating templates are not just the compiled form, they also (intentionally) contain rendering state. It also assumes that the same name will load the same template in every thread, which isn't true (a dynamic template loader, for example). So there are larger considerations that don't seem to have been addressed here.

Right now, this is probably the wrong approach to solving the perceived problem, but want to think about it some more.

[Luke Plant]

It also is a hit in terms of memory usage -- every single template used in the system is going to get stored in memory, if I read the patch correctly. This could be a much worse penalty than any benefit it gives. I read somewhere that Adrian and co had originally intended templates to be stored in some compiled form, but they realised later that it simply wasn't worth the overhead.

[novoya]

Replying to ubernostrum:

It wasn't fixed. Somebody needs to learn how to read the documentation on the ticket process to find out what "fixed" means (hint: not "some random person uploaded a patch").

Yes. Sorry guys. My mistake. I isn't fixed
Anyway, I didn't tested too much, but from my tests, my SIMPLE fix boosts performance considerably (for each refresh- 0 template reloads + compiles instead of 5 or more in a typical web app)
About memory usage concerns, it is very easy to limit the size of the cache in code.

[novoya]

Replying to mtredinnick:

This is almost certainly going to cause rednering bugs. The Python objects representating templates are not just the compiled form, they also (intentionally) contain rendering state. It also assumes that the same name will load the same template in every thread, which isn't true (a dynamic template loader, for example). So there are larger considerations that don't seem to have been addressed here.

Right now, this is probably the wrong approach to solving the perceived problem, but want to think about it some more.

How can template contain rendering state, if each time a template needed, it gets reloaded and recompiled ??

[Simon Litchfield]

Some benchmarking might be useful here. I could imagine the benefit being negligent on simple templates; but it could give a big plus for high traffic templates and/or large, complex templates (eg with lots of includes, multiple levels of inheritance, loading lots of tag libraries). Maybe it could be a setting, eg TEMPLATE_CACHE = 100, set zero by default. I have noticed before that after the database has been optimised and memcache used wherever possible, a unnecessarily high number of cycles are chewed up on template rendering.

[(none)]

Milestone post-1.0 deleted

[groks]

Fixes for cache key name, thread safety. Adds config, tests.

[groks]

Y'all, here's a new patch which fixes a couple of the concerns raised above:

Add TEMPLATE_CACHE config setting (default off), update the docs.
Don't cache templates when we're in TEMPLATE_DEBUG mode.
Use auxiliary template load dirs as part of cache key name.
Add some locking around the template cache.
Add some tests.

Not only is template caching faster, but it opens the door for further tuning. For example, we could walk the compiled template and coalesce adjacent TextNode's.

What else needs to be done to get this accepted?

[Ramiro Morales]

See also #9154, it contains a patch with a similar implementation and it seems to take in account problems found with the {{ block.super}} variable.

[mrts]

Duplicate of #9154.