New html-util/filter: unescape

[Johan Bergström <bugs@…>]

This is bascially a shameless escape "rip". I had the need for it in a specific project; so here it is.
What it does is reverse the effect of escape (ie: &amp; => &).
There are plugs for filter and django.utils.html (precisely as escape).

No documentation as of yet - if you're willing to put this into trunk, i'll gladly contribute docs.

[Johan Bergström <bugs@…>]

Patch and test case for unescape

[anonymous]

Shouldn't unescaping be the responsibility of the view? The only valid place I can see unescaping is with legacy data or data from another source, neither of which I would put in a template. Unescaping in templates feels as dirty as magic_quotes_gpc.

[Johan Bergström <bugs@…>]

Hello anonymous; Your suggestion is to punt the filter but keep the util? That's the way I'm currently utilizing this function - but for consistency with escape (available both as util and filter, i personally see no greater harm in leaving this option open for whatever data that may turn up in a template.

[James Bennett]

This feels like something that might not be necessary with the autoescpaing proposals.

[Johan Bergström <bugs@…>]

Replying to ubernostrum:

This feels like something that might not be necessary with the autoescpaing proposals.

I'm not very current with the autoescaping branch, but how would one handle data that you know is escaped already (let's say you run a ping service where most data recieved is unescaped, but for some reason some of the data is escaped)? I can agree on the lacking necessity for this utility in a template; but this would help as a util.

The only reason i sent in a patch was becase you generally want to have 'reversible' functions avaliable - if this patch better suits in my 'own' branch, feel free to close the ticket!

[Malcolm Tredinnick]

I don't think this is an appropriate filter for core. The use-cases are a bit too marginal.

[anonymous]

I think this ticket should be re-opened.

Anytime a library gives you escaped HTML (eg. Django Rest Framework's auto-generated documentation) a tag like this is super-useful. Plus, it's just kind of strange that core Django has fifty different template tags/filters to escape/not-escape stuff, but 0 tags/filters to unescape stuff.

[anonymous]

P.S. Just to be clear, no Django doesn't actually have 50 different tags/filters; it has a bunch, and I was being hyperbolic

[Russell Keith-Magee]

The reason the unescape tag doesn't exist is because you shouldn't need it. If you're ever in a situation where you need to undo escaping, then you haven't been paying attention to where things have been encoded along the line. The fix isn't to allow code to be bounced back and forth between encoded and unencoded -- it's to make sure you're only ever encoding when you need to be.

[Aymeric Augustin]

In addition to Russell's answer, if a library hands you escaped HTML, the right place to unescape it is "right when you obtain it", not "in the template engine".

Always work with data in the canonical form, and not in an escaped representation for an arbitrary output format.

[anonymous]

Well, that'd be lovely ... if my library wasn't handling the view layer also (and therefore preventing me from unescaping the code in the view). However, if Django REST framework is just following a bad practice, it's certainly not reasonable to add a filter just for one bad case. I guess I'll just have to dig through the source for some underscored method to override or something ...

Thanks for the explanation on the re-close!

[Collin Anderson]

I just want to note that I would use an html_unescape template filter in pretty much every case where I use the striptags filter.

For example, I have a product.html_description field and I may want to auto-generate a meta description like so:

<meta name="description" content="{{ product.html_description|striptags|html_unescape|truncatechars:400 }}">

striptags removes tags, but I also want to replace — -> — and ' -> ' before running it through truncatechars. I especially don't want to truncate in the middle of an — or other html entity. The template engine will take care of auto-escaping the resulting text to html. (And it's fine if the html is longer than 400 chars as long as the text is <= 400 chars.)

I think some people use the insecure striptags|safe as a workaround to "fix" html entities showing up in output, whereas a striptags|html_unescape would solve the same problem and actually be much more secure.

Or maybe it would help to have an html_to_text filter that is basically just striptags|unescape_html. I assume most people who use striptags also want to html entities unescaped too. Again, it might help cut down on insecure striptags|safe usage.