Opened 7 years ago

Closed 23 months ago

Last modified 8 months ago

#7261 closed New feature (fixed)

Support for __html__ for Library interoperability

Reported by: mitsuhiko Owned by: unaizalakain
Component: Template system Version: master
Severity: Normal Keywords: __html__
Cc: miracle2k, oliver@…, clouserw, ivank, jdunck@… Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

The pylons developers proposed the __html__ special method to determine the HTML representation of strings which is currently implemented by Pylons for mako, Jinja2 and soon Genshi. Having support for that in Django would make it possible to use the autoescaping facilities of these template engines in Django.

The idea is that if an object implements __html__ which returns a string this is used as HTML representation (eg: on escaping). If the object is a str or unicode subclass and returns itself the object is a safe string type.

This Genshi ticket shows some details about it: http://genshi.edgewall.org/ticket/202

Attachments (2)

html.diff (2.4 KB) - added by jbalogh 6 years ago.
implement html on SafeData for template engine interop
7261-add-__html__.diff (1.9 KB) - added by ivank 4 years ago.
html.diff updated for django trunk

Download all attachments as: .zip

Change History (24)

comment:1 Changed 7 years ago by edgarsj

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

comment:2 Changed 7 years ago by dcramer

There is no reason this should not be in Django. I may write up a patch once I read up on how it's used everywhere. This could then easily replace the auto-escaping system which is in Django, for a better, standardized system.

comment:3 Changed 7 years ago by mtredinnick

This shouldn't be thought of as *replacement* for autoescaping, since that's not required, but as something that uses it for interoperability. It's a feature addition. It looks like the __html__ method will be something along the lines of

#!
def __html__(self):
    return mark_safe(conditional_escape(self))

Possibly with a force_unicode or force_str call at the end, since it's not clear whether this method is expected to return an object that can be stringified or a bytestring or anything basestring-derived.

There's also the flip side to this, which is adjusting conditional_escape to know how to handle objects with this special method name. That should only be two or three lines as well, by the looks of it.

Note, however, that there is at least one reason for not including it: it goes against Python's guidelines of not introducing methods with leading and trailing double-underscores. It's the one namespace that Python has really reserved for internal object-level methods since pretty much forever. We'll have to make a decision about whether breaking that rule is acceptable.

comment:4 Changed 7 years ago by miracle2k

  • Cc miracle2k added

comment:5 Changed 7 years ago by miracle2k

  • Summary changed from Support for __html__ for Library interobability to Support for __html__ for Library interoperability

comment:6 Changed 6 years ago by anonymous

  • Cc oliver@… added

Changed 6 years ago by jbalogh

implement html on SafeData for template engine interop

comment:7 Changed 6 years ago by jbalogh

  • Has patch set
  • Owner changed from nobody to jbalogh
  • Status changed from new to assigned

comment:8 Changed 6 years ago by clouserw

  • Cc clouserw added

comment:9 Changed 5 years ago by mtredinnick

Where is the spec for how this __html__() method is meant to work and what API guarantees it offers? Adding a method just to support introducing security problems isn't a good idea. Is there *any* documentation for how it's supposed to work?

comment:10 Changed 5 years ago by mitsuhiko

How is that introducing security problems?

The spec is literally: "if an object has an html method it can be used to get a safe html representation of the object in unicode that is guaranteed to be convertible into an unicode string but might be a unicode string subclass". End of spec :)

Changed 4 years ago by ivank

html.diff updated for django trunk

comment:11 Changed 4 years ago by ivank

  • Cc ivank added

comment:12 Changed 4 years ago by lukeplant

  • Severity set to Normal
  • Type set to New feature

comment:13 Changed 4 years ago by Alex

  • Easy pickings unset
  • Triage Stage changed from Design decision needed to Accepted
  • UI/UX unset

comment:14 Changed 2 years ago by Kamu

  • Needs documentation set
  • Needs tests set
  • Patch needs improvement set

comment:15 Changed 2 years ago by Kamu

  • Owner jbalogh deleted
  • Status changed from assigned to new

comment:16 Changed 23 months ago by unaizalakain

  • Owner set to unaizalakain
  • Status changed from new to assigned

comment:18 Changed 23 months ago by Unai Zalakain <unai@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In af64429b991471b7a441e133b5b7d29070984f24:

Fixed #7261 -- support for html for library interoperability

The idea is that if an object implements html which returns a string this is
used as HTML representation (eg: on escaping). If the object is a str or unicode
subclass and returns itself the object is a safe string type.

This is an updated patch based on jbalogh and ivank patches.

comment:19 Changed 23 months ago by Alex Gaynor <alex.gaynor@…>

In 694d7da6c53b9ad35381dccdf4f688771b0b932e:

Merge pull request #1744 from unaizalakain/ticket_7261

Fixed #7261 -- support for html for library interoperability

comment:20 Changed 23 months ago by jdunck

  • Cc jdunck@… added

comment:21 Changed 8 months ago by Aymeric Augustin <aymeric.augustin@…>

In 6d52f6f8e688b5c4e70be8352eb02c05fea60e85:

Fixed #23831 -- Supported strings escaped by third-party libs in Django.

Refs #7261 -- Made strings escaped by Django usable in third-party libs.

The changes in mark_safe and mark_for_escaping are straightforward. The
more tricky part is to handle correctly objects that implement html.

Historically escape() has escaped SafeData. Even if that doesn't seem a
good behavior, changing it would create security concerns. Therefore
support for html() was only added to conditional_escape() where this
concern doesn't exist.

Then using conditional_escape() instead of escape() in the Django
template engine makes it understand data escaped by other libraries.

Template filter |escape accounts for html() when it's available.
|force_escape forces the use of Django's HTML escaping implementation.

Here's why the change in render_value_in_context() is safe. Before Django
1.7 conditional_escape() was implemented as follows:

if isinstance(text, SafeData):

return text

else:

return escape(text)

render_value_in_context() never called escape() on SafeData. Therefore
replacing escape() with conditional_escape() doesn't change the
autoescaping logic as it was originally intended.

This change should be backported to Django 1.7 because it corrects a
feature added in Django 1.7.

Thanks mitsuhiko for the report.

comment:22 Changed 8 months ago by Aymeric Augustin <aymeric.augustin@…>

In 3483682749577b4b5a8141a766489d5b460e30e9:

[1.7.x] Fixed #23831 -- Supported strings escaped by third-party libs in Django.

Refs #7261 -- Made strings escaped by Django usable in third-party libs.

The changes in mark_safe and mark_for_escaping are straightforward. The
more tricky part is to handle correctly objects that implement html.

Historically escape() has escaped SafeData. Even if that doesn't seem a
good behavior, changing it would create security concerns. Therefore
support for html() was only added to conditional_escape() where this
concern doesn't exist.

Then using conditional_escape() instead of escape() in the Django
template engine makes it understand data escaped by other libraries.

Template filter |escape accounts for html() when it's available.
|force_escape forces the use of Django's HTML escaping implementation.

Here's why the change in render_value_in_context() is safe. Before Django
1.7 conditional_escape() was implemented as follows:

if isinstance(text, SafeData):

return text

else:

return escape(text)

render_value_in_context() never called escape() on SafeData. Therefore
replacing escape() with conditional_escape() doesn't change the
autoescaping logic as it was originally intended.

This change should be backported to Django 1.7 because it corrects a
feature added in Django 1.7.

Thanks mitsuhiko for the report.

Backport of 6d52f6f from master.

Note: See TracTickets for help on using tickets.
Back to Top