Make it possible to specify a default template engine

[Artur Barseghyan]

This is my first django ticket here. Please, forgive for possible mistakes.

Issue:
I want to be able to use a custom template engine (same DjangoTemplates, but with different settings). Multiple template engines seems to be a great feature, but it becomes not that usable with big projects that depend on third party packages (such as django-cms, wagtail, etc) - you have to tell each view/render to use the default engine, otherwise you get "Several DjangoTemplates backends are configured. You must select one explicitly." ImproperlyConfigured exception.

Solution:
It would be really useful if it would be possible to mark one of the template engines as default and choose alternative engine where desired.

Changes to :meth:django.template.engine.get_default and engine configuration (TEMPLATES setting) would be really trivial. I could make a patch if approved.

Thank you (in any case)!

[Aymeric Augustin]

You shouldn't hit this error unless the third party apps rely on deprecated or private APIs.

Can you provide the full stack trace for the exception you're getting?

[Artur Barseghyan]

@Aymeric Augustin:

Sure!

---

My settings:

TEMPLATES = [
    # The default engine - a heavy one with a lot of context processors
    {
        'NAME': 'default',
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.i18n',
                'django.template.context_processors.request',
                'django.template.context_processors.media',
                'django.template.context_processors.static',
                'sekizai.context_processors.sekizai',
                'cms.context_processors.cms_settings',
            ],
            'debug': True,
        }
    },

    # Light engine - only very necessary things go here
    {
        'NAME': 'light',
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.i18n',
                'django.template.context_processors.request',
                'django.template.context_processors.media',
                'django.template.context_processors.static',
            ],
            'debug': True,
        },
    },
]

---

Environment:


Request Method: GET
Request URL: http://localhost:8000/de-at/suchen/

Django Version: 1.8.7
Python Version: 2.7.6
Installed Applications:
('djangocms_admin_style',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.redirects',
 'django.contrib.staticfiles',
 'django.contrib.sites',
 'django.contrib.sitemaps',
 'mptt',
 'treebeard',
 'menus',
 'sekizai',
 'easy_select2',
 'easy_thumbnails',
 'metatags',
 'ckeditor',
 'cmsplugin_filer_file',
 'cmsplugin_filer_folder',
 'cmsplugin_filer_link',
 'cmsplugin_filer_image',
 'cmsplugin_filer_teaser',
 'cmsplugin_filer_video',
 'cms',
 'hvad',
 'reversion',
 'compressor',
 'filer',
 'rosetta',
 'debug_toolbar')
Installed Middleware:
('django.middleware.cache.UpdateCacheMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.contrib.admindocs.middleware.XViewMiddleware',
 'django.middleware.common.CommonMiddleware',
 'cms.middleware.page.CurrentPageMiddleware',
 'cms.middleware.user.CurrentUserMiddleware',
 'cms.middleware.toolbar.ToolbarMiddleware',
 'cms.middleware.language.LanguageCookieMiddleware',
 'django.middleware.cache.FetchFromCacheMiddleware',
 'debug_toolbar.middleware.DebugToolbarMiddleware')


Traceback:
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  164.                 response = response.render()
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/response.py" in render
  158.             self.content = self.rendered_content
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/response.py" in rendered_content
  135.         content = template.render(context, self._request)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/backends/django.py" in render
  74.         return self.template.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  210.                     return self._render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/test/utils.py" in instrumented_test_render
  96.     return self.nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  135.         return compiled_parent._render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/test/utils.py" in instrumented_test_render
  96.     return self.nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/classytags/core.py" in render
  138.         return self.render_tag(context, **kwargs)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/sekizai/templatetags/sekizai_tags.py" in render_tag
  79.         rendered_contents = nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/classytags/core.py" in render
  138.         return self.render_tag(context, **kwargs)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/sekizai/templatetags/sekizai_tags.py" in render_tag
  79.         rendered_contents = nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/classytags/core.py" in render
  138.         return self.render_tag(context, **kwargs)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/templatetags/cms_tags.py" in render_tag
  681.         rendered_contents = nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  65.                 result = block.nodelist.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render
  905.                 bit = self.render_node(node, context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in render_node
  919.         return node.render(context)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/classytags/core.py" in render
  138.         return self.render_tag(context, **kwargs)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/templatetags/cms_tags.py" in render_tag
  308.             content = get_placeholder_content(context, request, page, name, inherit, nodelist)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/templatetags/cms_tags.py" in get_placeholder_content
  227.         placeholder = _get_placeholder(current_page, page, context, name)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/templatetags/cms_tags.py" in _get_placeholder
  191.     placeholders = page.rescan_placeholders().values()
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/models/pagemodel.py" in rescan_placeholders
  1378.         placeholders = get_placeholders(self.get_template())
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/utils/placeholder.py" in get_placeholders
  236.     placeholders = _scan_placeholders(_get_nodelist(compiled_template))
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/utils/placeholder.py" in _scan_placeholders
  203.             placeholders += _extend_nodelist(node)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/utils/placeholder.py" in _extend_nodelist
  260.     _extend_blocks(extend_node, blocks)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/utils/placeholder.py" in _extend_blocks
  288.     parent = extend_node.get_parent(get_context())
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/cms/utils/placeholder.py" in get_context
  40.         context.template = Template('')
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/base.py" in __init__
  188.             engine = Engine.get_default()
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/utils/lru_cache.py" in wrapper
  125.                     result = user_function(*args, **kwds)
File "/home/me/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/template/engine.py" in get_default
  83.                 "Several DjangoTemplates backends are configured. "

Exception Type: ImproperlyConfigured at /de-at/suchen/
Exception Value: Several DjangoTemplates backends are configured. You must select one explicitly.

[Aymeric Augustin]

Right, this situation is described in this part of the documentation: ​https://docs.djangoproject.com/en/1.10/ref/templates/api/#loading-a-template (the first two paragraphs).

I suppose the transition could be smoothed by adopting a convention for determining the default Django templates engine, perhaps by deciding it the one called "django" (or similar) when there are multiple engines configured.

Can you start by reporting this issue to django-cms? They're doing context.template = Template('') which, per the current documentation, isn't supported.

[Artur Barseghyan]

@Aymeric Augustin:

Thanks! I have already tracked it down and reported an issue.

• ​https://github.com/divio/django-cms/issues/5730

[Tim Graham]

Feel free to reopen if it turns out some change is needed in Django after all.

[Raphael Das Gupta]

django-crispy-forms ​seems to have the same problem when used in a Django project that has multiple template engines of the same backend configured. (I haven't yet tried with multiple engines of different backends.)

What are libraries supposed to use instead of Template('template content here')? By definition they cannot know what template engines the projects that the library will be used in have configured.

[Carl Meyer]

Given the current API, we are effectively asking apps like django-cms or crispy-forms (any third-party app that wants to load and render a Django template using the project's settings) to require their users to specify some setting (like DJANGO_CMS_TEMPLATE_ENGINE) if the project has more than one DjangoTemplate engine configured. This may be OK: it's explicit and gives the project fine-grained control per third-party app. On the downside, it's unlikely to be the obvious first-choice for the author of a third-party app; the obvious first choice is to just try instantiating Template directly, and even worse, this will probably appear to work fine until they think to test (or more likely, one of their users reports a bug) with multiple configured DTL engines.

IMO currently engine-less instantiation of Template is an attractive nuisance that appears easy and obvious and only breaks in edge cases, at which point the person who hits the problem (the project owner) has no reasonable way to fix it (without forking the third-party app).

I think we must do one of two things. We could a) deprecate and eventually remove engine-less instantiation of Template altogether, removing the attractive nuisance and requiring everyone to always be explicit from the start. Or we could b) add a "default DTL engine" marker, as suggested in this ticket, so that when a third-party app does the easy and naive thing (directly instantiate Template), a project owner with multiple DTL engines configured has some recourse to fix it without forking.

I think (b) may be the more practical solution, and kinder to our user base.

[Simon Charette]

I also think (b) is the most practical solution.

This problem also manifested itself in other project such as ​django-filter.

[Carlton Gibson]

As maintainer of both django-filter and crispy forms, I'd say I (too) am in favour of the b solution, i.e. adopting the proposal.

If we were writing new code, requiring an explicit template engine would be fine, but we're seeing code that's been working fine — quite literally — for years that's leading to the error here.

What's more it's only (≈) 1 user in a 100 that configures multiple template engines so the point about it being an untested and surprising edge-case is exactly right.

[Simon Charette]

Another related issue ​django-coverage-plugin issue.

[Carlton Gibson]

[​PR

Adds a failing test case for the issue and a naive fix.

Here we just take the first DTL engine returned. Users should employ collections.OrderedDict if they need to control this prior to Python 3.6.
Need to add a note to the docs to that effect if the patch is acceptable.

Alternative proposals:

Have a naming convention. e.g. pick engine named 'default'
Add a specific OPTION to the config dict.

This is something of an edge case, and so, (IMO) 'pick the first' should be sufficient.

[Raphael Das Gupta]

See ​https://github.com/django/django/pull/8000#issuecomment-282727217

[Raphael Das Gupta]

Patch ​looks good now.

[Tim Graham <timograham@…>]

In 6b3724fa:

Fixed #27359 -- Made Engine.get_default() return the first DjangoTemplates engine if multiple are defined.