Opened 10 years ago

Last modified 7 months ago

#23356 new Cleanup/optimization

Unable to create template tag which behaves similar to {% verbatim %}

Reported by: Jacob Rief Owned by:
Component: Template system Version: dev
Severity: Normal Keywords: verbatim
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by shayneoneill)

Currently it is impossible to create custom template tags which behave similar to {% verbatim %} .
The reason is in Lexer.create_token(). There, the class member self.verbatim is set for template blocks in "verbatim" state. It is impossible to turn on that state from outside, ie. a template tag.

Fixing this, would be as simple as changing if block_content[:9] in ('verbatim', 'verbatim ') to if block_content.startswith('verbatim') or to if block_content[:9] in ('verbatim', 'verbatim ', 'verbatim_').
Then all template tags beginning with verbatim..., would start in "verbatim" state. I don't assume this will break any existing code; who starts the name of a templatetag with 'verbatim...' if not for that purpose?

Background information why this is useful:
Templates syntax for Django and AngularJS is very similar, and with some caveats it is possible to reuse a Django template for rendering in AngularJS. I therefore attempted to add a context sensitive variation of the verbatim tag to this app https://github.com/jrief/django-angular, but was hindered by this issue.

BTW: This part of the Django code did not change from 1.6 up to master.

Change History (14)

comment:2 by shayneoneill, 10 years ago

Description: modified (diff)

comment:3 by shayneoneill, 10 years ago

Description: modified (diff)

comment:5 by None, 9 years ago

I am creating template tag and need get code between {% mytag %}...{% endmytag %}.
I read code and saw that similar template tag "verbatim" is hardcode using in Lexer.
This is bad idea because nobody can't create template like "verbatim".
Please, improve this thing and make django template system more convenient and flexible.

comment:6 by None, 9 years ago

Has patch: unset
Type: UncategorizedCleanup/optimization

comment:7 by Jacob Rief, 9 years ago

BTW: In the meantime I found another solution for the described problem.

comment:8 by Tim Graham, 9 years ago

Triage Stage: UnreviewedAccepted

I'm open to a solution, although I'm not sure if the change proposed in the description is ideal.

comment:9 by Atul Bhouraskar, 3 years ago

Owner: changed from nobody to Atul Bhouraskar
Status: newassigned

The Lexer.create_token() method can be greatly simplified by removing the hard coded logic for self.verbatim and instead saving the full token_string in the Token for tags.

Currently the Lexer.create_token() logic returns tokens of type TokenType.TEXT if it encounters a {% verbatim %} tag. This change removes this logic, ie. tokens are created as normal, instead a simple Parser.parse_verbatim() method can be implemented to return verbatim text in the parsing stage. This can be called by any tag instead of calling Parser.parse().

I've created a quick pull request https://github.com/django/django/pull/14686 with the above implemented - all template tests pass.

This approach fully removes the 'special case' for the built in verbatim tag and allows creating custom tags that need similar behaviour.
The simplified Lexer.create_token() should now be easier to maintain and probably also faster than the existing code (though I haven't profiled this).

comment:10 by Atul Bhouraskar, 3 years ago

Has patch: set

comment:11 by Chris Jerdonek, 3 years ago

Currently the Lexer.create_token() logic returns tokens of type TokenType.TEXT if it encounters a {% verbatim %} tag. This change removes this logic, ie. tokens are created as normal,

It seems like making this change would make it harder to fix #23424 and might even be incompatible with fixing it. The reason is that changing things in this way would require tokens to be parsed before finding the end of the verbatim tag, whereas fixing that bug would require looking for the end of the verbatim tag before attempting to parse tokens in between. Thus, I would suggest fixing the bug in #23424 (and adding test cases) before reworking the implementation into a way that might be incompatible with fixing it.

comment:12 by Atul Bhouraskar, 3 years ago

Thanks Chris, I wasn't aware of #23424.

In fact that ticket would be much simpler to fix using this approach - I've updated my pull request to make a possible fix for #23424 possible (and added tests that expose the bug).

I've also created a proof of concept fix for #23424 based of this fix - https://github.com/django/django/pull/14786 - all tests pass in that one.

comment:13 by Chris Jerdonek, 3 years ago

In fact that ticket would be much simpler to fix using this approach

It seems more complicated to me, IMO, and a lot less efficient because you need to be re-parsing multiple times. With the other approach, it would just be one pass.

comment:14 by Chris Jerdonek, 3 years ago

If the use cases for this ticket were spelled out in more detail, I feel like it would be easier to discuss other ways of solving it. Currently, it's not clear what requirements a solution needs to satisfy. It seems like there should be a solution that preserves the "one pass" nature of parsing. For example, maybe there can be a way to register verbatim tags so that the list consulted in create_token() will no longer be hard-coded.

comment:15 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

I agree with Chris that re-parsing multiple times is not an acceptable solution. IMO we should take a step back and try to fix #23424 without considering PR14686, then we can rethink solution for this ticket.

comment:16 by Mariusz Felisiak, 7 months ago

Owner: Atul Bhouraskar removed
Status: assignednew
Note: See TracTickets for help on using tickets.
Back to Top