assertContains fails when there is unicode in the response content

[drakonen]

# views.py
#

from django.shortcuts import render_to_response

def index(request):
    return render_to_response('index.html')

#
# index.html

aaäaa

#
# tests.py

from django.test import TestCase

class TestViews(TestCase):
    def test_index(self):
        response = self.client.get('/')
        self.assertContains(response, u'aa')


#
# results in:

======================================================================
ERROR: test_index (stuff.tests.TestViews)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/johan/devel/tester/stuff/tests.py", line 6, in test_index
    self.assertContains(response, u'aa')
  File "/usr/lib/python2.5/site-packages/django/test/testcases.py", line 267, in assertContains
    real_count = response.content.count(text)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

----------------------------------------------------------------------

[trbs]

A little rational behind the patch;

Python raised the UnicodeDecodeError because it's asked to count a unicode string within a non-unicode string.
HttpResponse's content property uses smart_str to create content data, it would seem to make sense to do
the same for any text used in assertContains and assertNotContains.

I've added an updated patch to this ticket which uses smart_str(text, response._charset) which matches the HttpResponse
object behavior. Could also have used settings.DEFAULT_CHARSET but it seems with ticket #10190 this might change later on so response._charset seems a safer bet :)

[(none)]

Milestone post-1.0 deleted

[Eric Holscher]

This makes sense. Having the tests mimick what normal Django does is probably a good idea.

[Claude Paroz]

Been confronted to this today. What prevents this to be included in 1.1 ?

[Eric Holscher]

Patch of the test case I made that couldn't reproduce the above.

[Eric Holscher]

I couldn't reproduce this. I have attached a test case to Django that passes. If you can make one that fails, please attach it to the case and reopen the ticket.

[trbs]

The reason the test case did not fail was because the assertion is not unicode.

I've attached a test case that does fail, it uses some Japanese in the template and tries to match a Unicode string in assertContains. Also attached a slight improvement on the patch for testcases.py because before a failed assertion did not use the smart_str version of the text resulting in "AssertionError: <unprintable AssertionError object>"

With the new test case it fails as follows:

======================================================================
ERROR: test_unicode_contains (regressiontests.test_client_regress.models.AssertContainsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/trbs/django/trunk/tests/regressiontests/test_client_regress/models.py", line 71, in test_unicode_contains
    self.assertContains(r, '\xe5\xb3\xa0'.decode('utf-8'))
  File "/home/trbs/django/trunk/django/test/testcases.py", line 333, in assertContains
    real_count = response.content.count(text)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 2: ordinal not in range(128)

----------------------------------------------------------------------
Ran 48 tests in 4.005s

FAILED (errors=1)
Destroying test database...

Hope this clears things up, otherwise you can also drop me a line on irc :)

[Russell Keith-Magee]

(In [10415]) [1.0.X] Fixed #10183 -- Corrected the handling of unicode in assertContains and assertNotContains. Thanks to trbs for the patch.

Merge of r10414 from trunk.

[Jacob]

Milestone 1.1 deleted