Markdown for Python v1.7 support needed in contrib.markup -- handling of unicode data changed

[Craig Ogg]

In v1.7rc1 Markdown moved to expecting unicode and always returning unicode. This causes the utf-8 encoding method that django.contrib.markup currently uses to generate incorrect encodings.

I am attaching a patch that changes the Markdown version and correctly passes unicode if the version is high enough.

[Craig Ogg]

​Link to mailing list announcement that describes changes to markdown. Relevant bit:

Additionally, the `encoding` argument has been removed from both `markdown` and
`Markdown`. Markdown expects unicode (or ascii) input and it is the
users responsibility to ensure that's what is provided. Therefore, all
output is in unicode. Only `markdownFromFile` accepts an encoding.

[Craig Ogg]

Support for Markdown for Python v1.7rc1 and later

[Waylan Limberg]

There is a duplicate of this at #6577 with some additional comments.

[anonymous]

The patch works fine. How can we get this implemented?

[jedie]

The patch didn't work for older version of markdown!
The version_info added in v1.6.2rc-2 previous versions has no version_info, see: ​http://python-markdown.svn.sourceforge.net/viewvc/python-markdown/markdown.py?r1=30&r2=31&pathrev=94

I will attached a new patch.

[Dmitri Fedortchenko <zeraien@…>]

Minor syntax error in previous patch, : missing from if statement. Corrected.

[Malcolm Tredinnick]

(In [7423]) Fixed #6387 -- Updated markdown filter to handle markdown-1.7 as well as
earlier versions. Patch from cogg, jedie and Dmitri Fedortchenko.

[italomaia]

This last change breaks markdown for me. Using ubuntu hardy, sqlite3 and python-markdown 1.6b1(repository).

return mark_safe(force_unicode(markdown.markdown(smart_str(value), extensions, safe_mode=safe_mode))) 

# unicode support only in markdown v1.7 or above. 
# version_info exist only in markdown v1.6.2rc-2 or above. 
if getattr(markdown, "version_info", None) < (1,7): 
    return mark_safe(force_unicode(markdown.markdown(smart_str(value), extensions, safe_mode=safe_mode))) 
else: 
    return mark_safe(markdown.markdown(force_unicode(value), extensions, safe_mode=safe_mode)) 

If i use

return mark_safe(markdown.markdown(force_unicode(value), extensions, safe_mode=safe_mode)) 

The error occors when i have a string with special characters on it, like ã or ç.

Without

if getattr(markdown, "version_info", None) < (1,7): 
    return mark_safe(force_unicode(markdown.markdown(smart_str(value), extensions, safe_mode=safe_mode))) 
else: 

it works great!

[Malcolm Tredinnick]

@italomaia: Please open a new ticket with the details of your failure (including an example of how to repeat the problem -- it's a little hard to work out what you're doing in this comment).

The main issue in this ticket has been fixed and you're seeing something that's a bit different: possibly due to a specific version, possibly due to the input you're passing... at the moment, it's hard to tell. But it will just get confusing if we mix up that report with the original issue here.

In the new report, please include what version_info reports for you version of markdown. It's a little annoying to have to support a beta release when the final has been out for ages (and subsequent releases fixed some problems in b1 -- I know because I sent in the patches to markdown). I wish Ubuntu would stop shipping stuff like that, or at least upgrade when the real version comes out. But if there's a way of distinguishing the versions, we'll try to handle it.

[Waylan Limberg]

@italomaia: It appears that the issue you are having is discussed in #5663. See my comment (9) on that ticket

[italomaia]

Thanks Wayla. That was pretty much what i meant. mtred, i'll send them a email asking for new markdown version. And, by the way, the patch in
ticket 5663 works for me(it was the same solution i did here, but, as it is there, is official! =D)
Sorry if i was not clear.