Opened 10 years ago

Closed 8 years ago

#393 closed enhancement (fixed)

[patch] Filters don't take the str() value of a var

Reported by: Boffbowsh Owned by: adrian
Component: Core (Other) Version:
Severity: critical Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

If I have a category model and do {{category|upper}} it fails, even though i have a repr(). Added str() to a number of filters to fix this behaviour. Maybe would be better to add to the template definition what sort of var it should accept? If it only accepts strings, str() everything first within django.core.template
Also added basic tests to string filters that don't accept args

Index: /usr/local/django_src/django/core/defaultfilters.py
===================================================================
--- /usr/local/django_src/django/core/defaultfilters.py	(revision 544)
+++ /usr/local/django_src/django/core/defaultfilters.py	(working copy)
@@ -9,7 +9,7 @@
 
 def addslashes(value, _):
     "Adds slashes - useful for passing strings to JavaScript, for example."
-    return value.replace('"', '\\"').replace("'", "\\'")
+    return str(value).replace('"', '\\"').replace("'", "\\'")
 
 def capfirst(value, _):
     "Capitalizes the first character of the value"
@@ -19,7 +19,7 @@
 def fix_ampersands(value, _):
     "Replaces ampersands with ``&`` entities"
     from django.utils.html import fix_ampersands
-    return fix_ampersands(value)
+    return fix_ampersands(str(value))
 
 def floatformat(text, _):
     """
@@ -35,7 +35,7 @@
 def linenumbers(value, _):
     "Displays text with line numbers"
     from django.utils.html import escape
-    lines = value.split('\n')
+    lines = str(value).split('\n')
     # Find the maximum width of the line count, for use with zero padding string format command
     width = str(len(str(len(lines))))
     for i, line in enumerate(lines):
@@ -44,7 +44,7 @@
 
 def lower(value, _):
     "Converts a string into all lowercase"
-    return value.lower()
+    return str(value).lower()
 
 def make_list(value, _):
     """
@@ -55,7 +55,7 @@
 
 def slugify(value, _):
     "Converts to lowercase, removes non-alpha chars and converts spaces to hyphens"
-    value = re.sub('[^\w\s-]', '', value).strip().lower()
+    value = re.sub('[^\w\s-]', '', str(value)).strip().lower()
     return re.sub('\s+', '-', value)
 
 def stringformat(value, arg):
@@ -74,7 +74,7 @@
 
 def title(value, _):
     "Converts a string into titlecase"
-    return re.sub("([a-z])'([A-Z])", lambda m: m.group(0).lower(), value.title())
+    return re.sub("([a-z])'([A-Z])", lambda m: m.group(0).lower(), str(value).title())
 
 def truncatewords(value, arg):
     """
@@ -93,17 +93,17 @@
 
 def upper(value, _):
     "Converts a string into all uppercase"
-    return value.upper()
+    return str(value).upper()
 
 def urlencode(value, _):
     "Escapes a value for use in a URL"
     import urllib
-    return urllib.quote(value)
+    return urllib.quote(str(value))
 
 def urlize(value, _):
     "Converts URLs in plain text into clickable links"
     from django.utils.html import urlize
-    return urlize(value, nofollow=True)
+    return urlize(str(value), nofollow=True)
 
 def urlizetrunc(value, limit):
     """
@@ -116,7 +116,7 @@
 
 def wordcount(value, _):
     "Returns the number of words"
-    return len(value.split())
+    return len(str(value).split())
 
 def wordwrap(value, arg):
     """
@@ -163,11 +163,11 @@
 def linebreaks(value, _):
     "Converts newlines into <p> and <br />s"
     from django.utils.html import linebreaks
-    return linebreaks(value)
+    return linebreaks(str(value))
 
 def linebreaksbr(value, _):
     "Converts newlines into <br />s"
-    return value.replace('\n', '<br />')
+    return str(value).replace('\n', '<br />')
 
 def removetags(value, tags):
     "Removes a space separated list of [X]HTML tags from the output"
@@ -410,7 +410,7 @@
 def phone2numeric(value, _):
     "Takes a phone number and converts it in to its numerical equivalent"
     from django.utils.text import phone2numeric
-    return phone2numeric(value)
+    return phone2numeric(str(value))
 
 def pprint(value, _):
     "A wrapper around pprint.pprint -- for debugging, really"
Index: /usr/local/django_src/tests/othertests/templates.py
===================================================================
--- /usr/local/django_src/tests/othertests/templates.py	(revision 544)
+++ /usr/local/django_src/tests/othertests/templates.py	(working copy)
@@ -1,4 +1,4 @@
-from django.core import template, template_loader
+from django.core import template, template_loader, defaultfilters
 
 # Helper objects for template tests
 class SomeClass:
@@ -10,6 +10,9 @@
         
     def method2(self, o):
         return o
+	
+    def __repr__(self):
+	return 'DjAnGo'
 
 class OtherClass:
     def method(self):
@@ -166,6 +169,25 @@
     'exception04': ("{% extends 'inheritance17' %}{% block first %}{% echo 400 %}5678{% endblock %}", {}, template.TemplateSyntaxError),
 }
 
+STRING_FILTERS = (
+	'linebreaksbr',
+	'striptags',
+	'escape',
+	'linebreaks',
+	'urlize',
+	'fix_ampersands',
+	'title',
+	'capfirst',
+	'wordcount',
+	'linenumbers',
+	'urlencode',
+	'lower',
+	'upper',
+	'phone2numeric',
+	'addslashes',
+	'slugify'
+	)
+
 # This replaces the standard template_loader.
 def test_template_loader(template_name, template_dirs=None):
     try:
@@ -176,6 +198,8 @@
 def run_tests(verbosity=0, standalone=False):
     template_loader.load_template_source, old_template_loader = test_template_loader, template_loader.load_template_source
     failed_tests = []
+    for stringfilter in STRING_FILTERS:
+	    TEMPLATE_TESTS['filter_' + stringfilter] = ( '{{ var|%s }}' % stringfilter, {"var": SomeClass()}, str(defaultfilters.template.registered_filters[stringfilter][0](str( 'DjAnGo' ), ())) )
     tests = TEMPLATE_TESTS.items()
     tests.sort()
     for name, vals in tests:

Attachments (2)

filter-object.diff (7.4 KB) - added by Boffbowsh 10 years ago.
str_filters.patch (10.1 KB) - added by SmileyChris 8 years ago.
Shiny new patch, with tests and docs

Download all attachments as: .zip

Change History (22)

comment:1 Changed 10 years ago by anonymous

What about Unicode strings?

comment:2 Changed 10 years ago by Boffbowsh

Would repr() be better then? On unicode strings, this performs the equivalent of:
r = u"%s" % repr(u.encode('unicode-escape'))
which is the same format as u"string" literals

comment:3 Changed 10 years ago by eugene@…

Did you try something like this?

{{category.__repr__|upper}}

comment:4 Changed 10 years ago by jacob

That won't work -- templates don't allow access to variables starting with underscores. The initial fix is correct: filters that take strings should explicitly str()-ify the variables.

comment:5 Changed 10 years ago by Boffbowsh

Need someone with more python unicode experience (and test data) to comment on this

comment:6 Changed 10 years ago by Boffbowsh

Ok, real patch now that takes into account unicode too. Added an extra optional arg to register_filter called string_only, signifies that the filter only works on strings and that resolve_variable_with_filter should convert the input first. Below is a test script:

from django.core import template

class TestClass:
	def __repr__(self):
		return('dJaNgO')

r = TestClass()

s = '''String: {{s|capfirst}}
Unicode: {{u|capfirst}}
Class: {{c|capfirst}}'''
t = template.Template(s)
c = template.Context({'u': u'la Pe\xf1a', 's': 'pyTHON', 'c': r})
print t.render(c)

and the output:

$ python unicodefiltertest.py
String: PyTHON
Unicode: La Peña
Class: DJaNgO

Changed 10 years ago by Boffbowsh

comment:7 Changed 10 years ago by Boffbowsh

*bump*

comment:8 Changed 10 years ago by jacob

Please, don't bump tickets; that's not polite.

comment:9 Changed 10 years ago by Boffbowsh

Sorry, but the patch has been sitting there for ages now, any idea of when you plan to include it?

comment:10 Changed 10 years ago by Maniac <Maniac@…>

I've filed a ticket 924 with a patch that converts all str's on input to unicode. This will probably breal your patch, sorry!

However I beleive this approach with total unicodisation is cleaner since it isolates type checking in only one place. May be you could update your patch to assume that all strings you get are unicode's. Thoughts?

comment:11 Changed 9 years ago by anonymous

  • Component changed from Template system to Core framework
  • priority changed from normal to low
  • Severity changed from normal to critical
  • Type changed from defect to enhancement

comment:12 Changed 9 years ago by Boffbowsh

Removing self from CC

comment:13 Changed 9 years ago by Boffbowsh

  • Cc paul.bowsher@… removed

Removing self from CC

comment:14 Changed 8 years ago by Marc Fargas <telenieko@…>

  • Patch needs improvement set

This bug has been arond without activity for 5 months now, and the original report is one year old. Can someone confirm it's still valid?
Marking "Patch needs improvement" as the patch references no longer existing files.

comment:15 Changed 8 years ago by Ivan Sagalaev <Maniac@…>

May I suggest looking at #924 that kinda superseeds this one and is safer. With non-ascii data simple str()ing is a recipe for breakage since it will use us-ascii codec by default.

comment:16 Changed 8 years ago by Ivan Sagalaev <Maniac@…>

And by "looking" I really mean closing it also in favor of unicodification if that will happen before 1.0

Changed 8 years ago by SmileyChris

Shiny new patch, with tests and docs

comment:17 Changed 8 years ago by SmileyChris

  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Ready for checkin

My patch sticks with str for now. I don't want to get into the whole unicode debate, just fix the bug. My patch will make it much easier to upgrade to that future step however.

comment:18 Changed 8 years ago by Marc Fargas <telenieko@…>

I would rather set "Design decission needed" and let some core developer decide if the smart_string() function on the patch should support unicode before checkin (in favour of #2489) or if it's ok for now to be checked in.

But also, the bug should be fixed, so "Ready for chekin" is ok! But can I suggest opening a new ticket about adding unicode support to smart_string() when this one gets closed?

comment:19 Changed 8 years ago by SmileyChris

Will do Marc. Like you say, the bug should be fixed and if it's been sitting around this long, I'd prefer not to complicate the issue for now. I just wanted to also show some forward thinking :)

comment:20 Changed 8 years ago by jacob

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in [4558].

Note: See TracTickets for help on using tickets.
Back to Top