Template lexer fails on comment/verbatim tags without whitespace after {%

[Spoko Dev]

Since 1a9fba7a6c2a0acead19bbb45c25ceb0fdcca7c8 (#23424), _comment_end_re and _verbatim_end_re in django/template/base.py require whitespace after {%, so {%endcomment%} and {%endverbatim%} never close a raw block. _raw_start_re requires it too, so with debug=False a {%verbatim%} block is not lexed as raw text. Django 6.1.2 accepts these tags without the spaces.

On main (3d3d943b74):

from django.template import Context, Engine

Engine(debug=True).from_string("{%comment%}hidden{%endcomment%}visible").render(Context())
# TemplateSyntaxError: Unclosed tag on line 1: 'comment'. Looking for one of: endcomment.

Engine().from_string("{% comment %}hidden{%endcomment%}visible").render(Context())
# TemplateSyntaxError: Unclosed tag on line 1: 'comment'. Looking for one of: endcomment.

Engine().from_string("{%verbatim%}{{ x }}{%endverbatim%}").render(Context({"x": 1}))
# AttributeError: 'NoneType' object has no attribute 'engine'

On 6.1.2 these render visible, visible and {{ x }}. With debug=False, {%comment%}…{%endcomment%} alone still renders, but it fails as soon as the template also contains {% comment anywhere, even inside a {# #} comment.

Templates in the wild use this form, for example templates/releasetext.html in postgres/pgweb ({%comment%}…{%endcomment%}): on main, DebugLexer finds no endcomment token in it, while 6.1.2 finds one.

makemessages is affected as well: with {% comment %}note{%endcomment%} before {% translate "Welcome" %}, templatize() no longer outputs "Welcome", so the string is left out of the .po file without an error.

Allowing zero whitespace in the three patterns restores the 6.1.2 behaviour; I have a patch with tests and will open a PR. Open ​PR #21982 changes _comment_end_re and _verbatim_end_re to \s+endcomment\s+ / \s+endverbatim\s+, which keeps this problem and would also require a space before %}.

[Spoko Dev]

​PR