Code

Opened 8 years ago

Closed 6 years ago

Last modified 5 years ago

#2910 closed enhancement (fixed)

[patch] Add extension support to Markdown filter

Reported by: waylan@… Owned by: wayla
Component: Contrib apps Version: master
Severity: normal Keywords: markup markdown
Cc: waylan@…, gajon@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Starting with Markdown 1.6 (released Oct 12, 2006) including extentions has become much easier. It would be nice to easily utilize that functionality in Django. Therefore, I propose that my suggestion, which Yuri expounded upon, be included in the markup contrib app. (Note that I am leaving out the use of escape and linebreaks on import failure here - that is another issue for another ticket)

Usage:

  • Load extensions by listing them in quotes separated by comas:
{{ post.body|markdown:"imagelinks,footnotes" }}
  • To disable HTML, include "safe" as the first argument:
{{ post.body|markdown:"safe" }}
  • For no extensions (backword compatible) just do the same as before:
{{ post.body|markdown }}

Attachments (7)

markup.diff (421 bytes) - added by waylan@… 8 years ago.
diff for adding extention support to markdown
markup.2.diff (910 bytes) - added by waylan@… 8 years ago.
markup.diff in proper format
markup.3.diff (1.2 KB) - added by Waylan Limberg <waylan@…> 7 years ago.
Added check for older versions of Markdown, in which case the extensions are ignored silently.
2910.diff (1.9 KB) - added by ubernostrum 7 years ago.
Alternate version of other patches
2910-2.diff (1.9 KB) - added by alex@… 7 years ago.
fix for stdout+mod_wsgi problem, please review wrt. unicode changes
2910-3.diff (2.2 KB) - added by wayla 6 years ago.
Address older versions of markdown and unicode support.
2910-4.diff (2.1 KB) - added by wayla 6 years ago.
Patch updated to support the new auto-escaping

Download all attachments as: .zip

Change History (28)

Changed 8 years ago by waylan@…

diff for adding extention support to markdown

Changed 8 years ago by waylan@…

markup.diff in proper format

comment:1 Changed 7 years ago by waylan@…

  • Cc waylan@… added
  • Version set to SVN

comment:2 Changed 7 years ago by Waylan Limberg <waylan@…>

  • Keywords markup markdown added
  • Summary changed from Add extension support to Markdown filter to [patch] Add extension support to Markdown filter

comment:3 Changed 7 years ago by Simon G. <dev@…>

  • Triage Stage changed from Unreviewed to Design decision needed

Are the older versions of MarkDown going to have problems with these extensions?

Changed 7 years ago by Waylan Limberg <waylan@…>

Added check for older versions of Markdown, in which case the extensions are ignored silently.

comment:4 Changed 7 years ago by Waylan Limberg <waylan@…>

Just added a updated patch which checks for older versions of Markdown. Unfortunately, Markdown only identifies the version in the doc string, so is was easier to wrap the call in a try, except block. Perhaps it would be better to wait for the newer versions of Markdown to become more common and defer this for then.

Personally, I copied the contrib app to my project and made the changes there. Anyone else is able to do the same.

Changed 7 years ago by ubernostrum

Alternate version of other patches

comment:5 Changed 7 years ago by ubernostrum

Added a patch which cobbles together useful features from various patches:

  • Exception trapping for no Markdown.
  • Not using extensions on older versions of Markdown.
  • Running escape(linebreaks(value)) when no Markdown is available, to safely handle the input.

comment:6 follow-up: Changed 7 years ago by alex@…

just silently ignoring the "safe" extension seems to be a possible source of security problems...

comment:7 in reply to: ↑ 6 Changed 7 years ago by Waylan Limberg <waylan@…>

Replying to alex@gc-web.de:

just silently ignoring the "safe" extension seems to be a possible source of security problems...

I understand the concern, but allowing raw html in Markdown is generally considered a feature - which some will want/expect. Therefore, it should be left to each site developer to chose for him/herself whether markdown should use safe mode in that particular app according to the needs of that projects users. The above patch does that.

Regarding the fact that safe mode is ignored in older versions, that is no different that the current behavior. If I recall correctly, it was not possible to call safe mode that easily, or at all, in older versions of Markdown. If anything, this adds one more reason to apply this patch.

comment:8 Changed 7 years ago by sciyoshi@…

Is there a way we could do the extension loading manually by using extension.extendMarkdown(markdown)?
Currently, python-markdown's extension loading seems to require that packages be on sys.path.
I think it would be nicer if you could do something like

{{ content|markdown:"myproject.utils.markdown_extension" }}

A SETTINGS variable that specifies the default extensions to use would be cool too.

comment:9 Changed 7 years ago by Jorge Gajon <gajon@…>

  • Cc gajon@… added

comment:10 Changed 7 years ago by Waylan Limberg <waylan@…>

  • Owner changed from nobody to anonymous
  • Status changed from new to assigned

comment:11 Changed 7 years ago by wayla

  • Owner changed from anonymous to wayla
  • Status changed from assigned to new

Lets try this again, now that I'm logged in.

comment:12 Changed 7 years ago by alex@…

There is a problem with this patch in combination with mod_wsgi: when using the markdown filter without any arguments, arg is '', so extensions ends up being ['']. This causes markdown to print the following on stdout:

couldn't load extension (looking for mdx_ module)

when running under mod_wsgi this results in the following exception:

IOError: sys.stdout access restricted by mod_wsgi

because mod_wsgi disables writing to stdout. see http://code.google.com/p/modwsgi/wiki/DebuggingTechniques

It can be worked around by filtering out empty values: extensions = [e for e in arg.split(",") if e]

Changed 7 years ago by alex@…

fix for stdout+mod_wsgi problem, please review wrt. unicode changes

comment:13 Changed 7 years ago by jpd <jd@…>

To be expected of course, but this will fail with "markdown() takes exactly 1 non-keyword argument" if an older version of markdown is used (for instance 1.4, which is still the latest available in ubuntu feisty, debian etch, etc). Is it worthwhile to check markdown.version (an attr which doesn't exist in the older markdown.py) here and provide a more helpful error?

Changed 6 years ago by wayla

Address older versions of markdown and unicode support.

Changed 6 years ago by wayla

Patch updated to support the new auto-escaping

comment:14 Changed 6 years ago by wayla

I just updated this patch to support the recently added auto-escaping. I left off the fallback to linebreaks(escape(text)) when markdown fails to import. IMO auto-escaping should be taking care of that - if not, then textile & restructuredtext should be patched as well. That's also why I left off the linebreaks - while its nice to have, it should be applied to all three processors, not just markdown - and therefore isn't relevant to this patch IMO.

comment:15 Changed 6 years ago by jacob

  • Triage Stage changed from Design decision needed to Ready for checkin

comment:16 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [6834]) Fixed #2910 -- Added support for markdown extensions to the markdown markup
filter (try saying that ten times quickly!). Thanks, Waylan Limberg.

comment:17 Changed 6 years ago by anonymous

  • Resolution fixed deleted
  • Status changed from closed to reopened

I have problems using german umlauts with the markdown markup filter.

Using markdown.markdown(smart_unicode(value) instead of markdown.markdown(smart_str(value) in the returns solves the problem.

s = "öäüß"

print force_unicode(markdown.markdown(smart_str(s)))

MARKDOWN-CRITICAL: "UnicodeDecodeError: Markdown only accepts unicode or ascii input."

print force_unicode(markdown.markdown(smart_unicode(s)))

<p>öäüß</p>

comment:18 Changed 6 years ago by jokull@…

  • Resolution set to duplicate
  • Status changed from reopened to closed

comment:19 Changed 6 years ago by jokull@…

  • Resolution duplicate deleted
  • Status changed from closed to reopened
  • Triage Stage changed from Ready for checkin to Design decision needed

Oops. 6387 is a duplicate of this. Sorry.

comment:20 Changed 6 years ago by wayla

  • Resolution set to fixed
  • Status changed from reopened to closed
  • Triage Stage changed from Design decision needed to Ready for checkin

Replying to anonymous:

I have problems using german umlauts with the markdown markup filter.

Using markdown.markdown(smart_unicode(value) instead of markdown.markdown(smart_str(value) in the returns solves the problem.

This is a result of stricter unicode enforcement for the newly released python-markdown 1.7 and not related to the extension support at all. In fact, this would have failed before the patch for this ticket was committed. Therefore, I am restoring this ticket to it's closed/fixed status and have opened a new ticket #6577, which has then been marked a duplicate of #6387.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.