{%extends%}-ing a file-based template object is broken by deprecation path indirection

[Julien Hartmann]

Hello,
I could not find it on the bug tracker, sorry if I just missed it.
I am using git version, commit id bd80fa6b0f7e5a0cc4ea26cedd56d0c4c4894420 on branch stable/1.8.x

Setup:

• load a template in a view using select_template
• insert the result of select_template into the context['foo'] of another template, that has {% extends foo %}.
• render that template with that context.

That is:

# as part of a class-based view
template_name = 'child_template.html'

def get(self, *args, **kwargs):
    context = self.get_context_data()
    context['base_template'] = select_template(['foo.html', 'bar.html'])
    return self.render_to_response(context)

If child_template.html contains a {% extends base_template %}, the rendering process will break while trying to access the template's nodelist:

======================================================================
ERROR: test_admin_change_form_title (hvad.tests.admin.NormalAdminTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/spectras/Projects/hvad/django-hvad/hvad/tests/admin.py", line 226, in test_admin_change_form_title
    response = self.client.get(url)
  File "/home/spectras/Projects/hvad/django/django/test/client.py", line 499, in get
    **extra)
  File "/home/spectras/Projects/hvad/django/django/test/client.py", line 302, in get
    return self.generic('GET', path, secure=secure, **r)
  File "/home/spectras/Projects/hvad/django/django/test/client.py", line 378, in generic
    return self.request(**r)
  File "/home/spectras/Projects/hvad/django/django/test/client.py", line 465, in request
    six.reraise(*exc_info)
  File "/home/spectras/Projects/hvad/django/django/utils/six.py", line 659, in reraise
    raise value
  File "/home/spectras/Projects/hvad/django/django/core/handlers/base.py", line 163, in get_response
    response = response.render()
  File "/home/spectras/Projects/hvad/django/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/home/spectras/Projects/hvad/django/django/template/response.py", line 135, in rendered_content
    content = template.render(context, self._request)
  File "/home/spectras/Projects/hvad/django/django/template/backends/django.py", line 83, in render
    return self.template.render(context)
  File "/home/spectras/Projects/hvad/django/django/template/base.py", line 211, in render
    return self._render(context)
  File "/home/spectras/Projects/hvad/django/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/spectras/Projects/hvad/django/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/spectras/Projects/hvad/django/django/template/debug.py", line 80, in render_node
    return node.render(context)
  File "/home/spectras/Projects/hvad/django/django/template/loader_tags.py", line 118, in render
    for node in compiled_parent.nodelist:
AttributeError: 'Template' object has no attribute 'nodelist'

It seems the ExtendsNode tries to access template.nodelist directly.

However, the Template object returned by select_template is a django.template.backends.django.Template, that encapsulates the actual template, so it can intercept render() and throw some deprecation warnings.

I guess it just misses a nodelist property that would forward to the actual template's nodelist.

I can build a proper example if absolutely needed, but it seemed simple enough that it would not be necessary.

[Aymeric Augustin]

This is covered by the following paragraphs of the release notes:

• ​https://docs.djangoproject.com/en/dev/releases/1.8/#rendering-templates-loaded-by-get-template-with-a-context
• ​https://docs.djangoproject.com/en/dev/ref/templates/upgrading/#get-template-upgrade-django-18

[Julien Hartmann]

You are right, it is covered in the documentation. The culprit code, however, lies in Django itself, as the {% extends %} tag has not been updated: it does not support the new backend-agnostic template objects and still expects a django.template.Template. However, there is no supported way to get one before the rendering stage.

According to the {% extends %} documentation, this should work:

from django.views.generic.base import TemplateView
from django.template.loader import get_template

class MyView(TemplateView):
    template_name = "mytemplate.html"
    def get_context_data(self, **kwargs):
        kwargs['parent_template'] = get_template('parent.html')
        return super(MyView, self).get_context_data(**kwargs)

{# mytemplate.html #}
{% extends parent_template %}

I guess there are two options:
1) clearly mark in the documentation that {% extends %} is incompatible with get_template
or
2) update {% extends %} so it handles agnostic templates correctly.

[Aymeric Augustin]

There's at least the possibility to improve the documentation.

If you control of configured template engines, you can do:

from django.template import engines
from django.views.generic.base import TemplateView


class MyView(TemplateView):
    template_name = "mytemplate.html"

    def get_context_data(self, **kwargs):
        kwargs['parent_template'] = engines['django'].get_template('parent.html')
        return super(MyView, self).get_context_data(**kwargs)

Perhaps we should include the same hack as in inclusion_tag:
​https://github.com/django/django/blob/4ea43ac9/django/template/base.py#L1269-L1270

I don't think option 2 would work. How could a Django template extend, say, a Genshi template?

[Julien Hartmann]

I do not control that, as I am working on a library module.
The user-code defines the parent template, and may override none to all of the templates as the developer sees fit. Therefore, all of the following may happen:

• they completely override the template (child template does not extend anything). Point is moot.
• they use django template engine and extend the parent template specified in the context.
• they use some other template engine that also supports inheritance, and still want to extend the parent template specified in the context.

For this reason I cannot know what engine is used, and I believe I should not know at this stage anyway.

For now, I resorted to the exact same hack you just linked, only using duck typing and less checks:

        if hasattr(context['base_template'], 'template'):
            context['base_template'] = context['base_template'].template

But it is not unthinkable to allow extending templates across languages. As long as they both have the concept of inheritance of course. I seriously doubt the utility would warrant the effort though.

Anyway, this is beyond this issue. The issue here is I have to either add a hard connection to a specific template engine in the view, forcing the developer to inherit and override it if they use another template engine; or not use inheritance at all. Both options are inconvenient to the user, and the very existence of this hack shows the inherent possibility to make it work cleanly.

To the very least, I believe {%extends%} should be able to receive the result of get_template and not choke on it when it is indeed a Django template.

That would have the added benefit of not breaking all the projects that fed {%extends%} some preloaded templates before version 1.8.

And of course, it would allow third-party modules to manipulate templates in an agnostic way, which I believe was the point of the new templating system.

[Aymeric Augustin <aymeric.augustin@…>]

In 47ee7b48adbcc0dafc3404034286c5fcbcd1cea6:

Fixed #24338 -- Accepted Template wrapper in {% extends %}.

Explicitly checking for django.template.Template subclasses is
preferrable to duck-typing because both the django.template.Template and
django.template.backends.django.Template have a render() method.

Thanks spectras for the report.

[Aymeric Augustin <aymeric.augustin@…>]

In 0f3eb8260b251a15c57f577fb33baad88b284942:

[1.8.x] Fixed #24338 -- Accepted Template wrapper in {% extends %}.

Explicitly checking for django.template.Template subclasses is
preferrable to duck-typing because both the django.template.Template and
django.template.backends.django.Template have a render() method.

Thanks spectras for the report.

Backport of 47ee7b48 from master