Support for __html__ for Library interoperability

[Armin Ronacher]

The pylons developers proposed the __html__ special method to determine the HTML representation of strings which is currently implemented by Pylons for mako, Jinja2 and soon Genshi. Having support for that in Django would make it possible to use the autoescaping facilities of these template engines in Django.

The idea is that if an object implements __html__ which returns a string this is used as HTML representation (eg: on escaping). If the object is a str or unicode subclass and returns itself the object is a safe string type.

This Genshi ticket shows some details about it: ​http://genshi.edgewall.org/ticket/202

[David Cramer]

There is no reason this should not be in Django. I may write up a patch once I read up on how it's used everywhere. This could then easily replace the auto-escaping system which is in Django, for a better, standardized system.

[Malcolm Tredinnick]

This shouldn't be thought of as *replacement* for autoescaping, since that's not required, but as something that uses it for interoperability. It's a feature addition. It looks like the __html__ method will be something along the lines of

#!
def __html__(self):
    return mark_safe(conditional_escape(self))

Possibly with a force_unicode or force_str call at the end, since it's not clear whether this method is expected to return an object that can be stringified or a bytestring or anything basestring-derived.

There's also the flip side to this, which is adjusting conditional_escape to know how to handle objects with this special method name. That should only be two or three lines as well, by the looks of it.

Note, however, that there is at least one reason for not including it: it goes against Python's guidelines of not introducing methods with leading and trailing double-underscores. It's the one namespace that Python has really reserved for internal object-level methods since pretty much forever. We'll have to make a decision about whether breaking that rule is acceptable.

[Jeff Balogh]

implement html on SafeData for template engine interop

[Malcolm Tredinnick]

Where is the spec for how this __html__() method is meant to work and what API guarantees it offers? Adding a method just to support introducing security problems isn't a good idea. Is there *any* documentation for how it's supposed to work?

[Armin Ronacher]

How is that introducing security problems?

The spec is literally: "if an object has an html method it can be used to get a safe html representation of the object in unicode that is guaranteed to be convertible into an unicode string but might be a unicode string subclass". End of spec :)

[ivank]

html.diff updated for django trunk

[Unai Zalakain]

Pull request: ​https://github.com/django/django/pull/1744

[Unai Zalakain <unai@…>]

In af64429b991471b7a441e133b5b7d29070984f24:

Fixed #7261 -- support for html for library interoperability

The idea is that if an object implements html which returns a string this is
used as HTML representation (eg: on escaping). If the object is a str or unicode
subclass and returns itself the object is a safe string type.

This is an updated patch based on jbalogh and ivank patches.

[Alex Gaynor <alex.gaynor@…>]

In 694d7da6c53b9ad35381dccdf4f688771b0b932e:

Merge pull request #1744 from unaizalakain/ticket_7261

Fixed #7261 -- support for html for library interoperability

[Aymeric Augustin <aymeric.augustin@…>]

In 6d52f6f8e688b5c4e70be8352eb02c05fea60e85:

Fixed #23831 -- Supported strings escaped by third-party libs in Django.

Refs #7261 -- Made strings escaped by Django usable in third-party libs.

The changes in mark_safe and mark_for_escaping are straightforward. The
more tricky part is to handle correctly objects that implement html.

Historically escape() has escaped SafeData. Even if that doesn't seem a
good behavior, changing it would create security concerns. Therefore
support for html() was only added to conditional_escape() where this
concern doesn't exist.

Then using conditional_escape() instead of escape() in the Django
template engine makes it understand data escaped by other libraries.

Template filter |escape accounts for html() when it's available.
|force_escape forces the use of Django's HTML escaping implementation.

Here's why the change in render_value_in_context() is safe. Before Django
1.7 conditional_escape() was implemented as follows:

if isinstance(text, SafeData):

return text

else:

return escape(text)

render_value_in_context() never called escape() on SafeData. Therefore
replacing escape() with conditional_escape() doesn't change the
autoescaping logic as it was originally intended.

This change should be backported to Django 1.7 because it corrects a
feature added in Django 1.7.

Thanks mitsuhiko for the report.

[Aymeric Augustin <aymeric.augustin@…>]

In 3483682749577b4b5a8141a766489d5b460e30e9:

[1.7.x] Fixed #23831 -- Supported strings escaped by third-party libs in Django.

Refs #7261 -- Made strings escaped by Django usable in third-party libs.

The changes in mark_safe and mark_for_escaping are straightforward. The
more tricky part is to handle correctly objects that implement html.

Historically escape() has escaped SafeData. Even if that doesn't seem a
good behavior, changing it would create security concerns. Therefore
support for html() was only added to conditional_escape() where this
concern doesn't exist.

Then using conditional_escape() instead of escape() in the Django
template engine makes it understand data escaped by other libraries.

Template filter |escape accounts for html() when it's available.
|force_escape forces the use of Django's HTML escaping implementation.

Here's why the change in render_value_in_context() is safe. Before Django
1.7 conditional_escape() was implemented as follows:

if isinstance(text, SafeData):

return text

else:

return escape(text)

render_value_in_context() never called escape() on SafeData. Therefore
replacing escape() with conditional_escape() doesn't change the
autoescaping logic as it was originally intended.

This change should be backported to Django 1.7 because it corrects a
feature added in Django 1.7.

Thanks mitsuhiko for the report.

Backport of 6d52f6f from master.