Opened 18 years ago

Closed 17 years ago

#2276 closed enhancement (duplicate)

[patch] make slugify filter to be aware of unicode

Reported by: nkeric Owned by: Adrian Holovaty
Component: Template system Version: dev
Severity: normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Currently the slugify filter only works with english alphanumeric characters, here is a simple patch that makes it to be able to "match the characters [0-9_] plus whatever is classified as alphanumeric in the Unicode character properties database".

here is the ref:

http://docs.python.org/lib/re-syntax.html

http://docs.python.org/lib/node115.html

Regards,

  • Eric

Attachments (6)

defaultfilters.py.diff (600 bytes ) - added by nkeric 18 years ago.
defaultfilters.py.2.diff (606 bytes ) - added by nkeric 18 years ago.
new diff with re.U replaced with re.UNICODE
validators.py.diff (603 bytes ) - added by nkeric 18 years ago.
patch the validator isSlug
defaultfilters.py.3.diff (740 bytes ) - added by nkeric 18 years ago.
output string should be utf-8 encoded as input string
validators.py.2.diff (1.0 KB ) - added by nkeric 18 years ago.
decode utf-8 encoded string for doing re search
defaultfilters.py.patch (770 bytes ) - added by Jonas von Poser 17 years ago.
Normallizes string

Download all attachments as: .zip

Change History (13)

by nkeric, 18 years ago

Attachment: defaultfilters.py.diff added

comment:1 by nkeric, 18 years ago

I tested it and it handles Chinese Characters properly (should work with other language's characters too), and all Chinese punctuations are removed as expected :)

by nkeric, 18 years ago

Attachment: defaultfilters.py.2.diff added

new diff with re.U replaced with re.UNICODE

by nkeric, 18 years ago

Attachment: validators.py.diff added

patch the validator isSlug

by nkeric, 18 years ago

Attachment: defaultfilters.py.3.diff added

output string should be utf-8 encoded as input string

by nkeric, 18 years ago

Attachment: validators.py.2.diff added

decode utf-8 encoded string for doing re search

comment:2 by Adrian Holovaty, 18 years ago

Resolution: duplicate
Status: newclosed

Closing in favor of #2489.

comment:3 by (none), 17 years ago

milestone: Version 1.0

Milestone Version 1.0 deleted

comment:4 by Jonas von Poser, 17 years ago

Resolution: duplicate
Status: closedreopened

I found this program: Slughifi is slugify with support for international characters.

http://amisphere.com/contrib/python-django/

http://amisphere.com/contrib/python-django/slughifi.py

comment:5 by Malcolm Tredinnick, 17 years ago

Resolution: duplicate
Status: reopenedclosed

Somebody brought that up on the mailing list at one point. Further investigation revealed is was under the GPL license (no license is mentioned in the file, which is even worse), so we cannot use it.

We have a different solution in the works over at #4365 which I'll resolve in the near future.

comment:6 by Jonas von Poser, 17 years ago

Resolution: duplicate
Status: closedreopened

By now, I found a solution.

Got from http://www.djangosnippets.org/snippets/98/ (_string_to_slug method) that points to
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/251871 (Aaron Bentley)

E.g:
It translates: Á È ï ô ü ñ
to:            A E i o u n

so it's very usefull for a lot of languages.

I add a patch.

by Jonas von Poser, 17 years ago

Attachment: defaultfilters.py.patch added

Normallizes string

comment:7 by Jonas von Poser, 17 years ago

Resolution: duplicate
Status: reopenedclosed

Closing in favor of #4365

Note: See TracTickets for help on using tickets.
Back to Top