Opened 7 years ago

Closed 7 years ago

Last modified 6 years ago

#25218 closed Uncategorized (wontfix)

python_2_unicode_compatible causes infinite recursion when super().__str__() is called

Reported by: Josh Crompton Owned by: nobody
Component: Utilities Version: 1.8
Severity: Normal Keywords:
Cc: tzanke@… Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have a class A which implements __str__() and is decorated with @python_2_unicode_compatible.

I have sub-class of A, B, and B 's implementation of __str__() calls super().__str__().

When I call b.__str__(), I get the following error: RuntimeError: maximum recursion depth exceeded while calling a Python object.

Partial traceback:

  File "/home/josh/Desktop/temp/djtest/local/lib/python2.7/site-packages/django/utils/encoding.py", line 42, in <lambda>
    klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
  File "recursion.py", line 27, in __str__
    super().__str__(),
  File "/home/josh/Desktop/temp/djtest/local/lib/python2.7/site-packages/django/utils/encoding.py", line 42, in <lambda>
    klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
  File "recursion.py", line 27, in __str__
    super().__str__(),

The docs (https://docs.djangoproject.com/en/dev/ref/utils/#django.utils.encoding.python_2_unicode_compatible) imply that the decorator should be applied to *any* class implementing __str__()

Attachments (1)

recursion.py (668 bytes) - added by Josh Crompton 7 years ago.

Download all attachments as: .zip

Change History (13)

Changed 7 years ago by Josh Crompton

Attachment: recursion.py added

comment:1 Changed 7 years ago by Tim Graham

The attached script prints "foo bar" for me on Python 3.4. Is it supposed to reproduce the infinite recursion?

comment:2 Changed 7 years ago by Josh Crompton

Apologies, I should have specified that this problem only occurs with Python 2 (I have tried only with 2.7.6). My understanding is that python_2_unicode_compatible is a noop in Python 3.

I also notice now that python_2_unicode_compatible is actually in the six library, so it might be more appropriate for me to make a ticket for that project than here.

comment:3 Changed 7 years ago by Tim Graham

Can you omit @python_2_unicode_compatible on the subclass?

comment:4 Changed 7 years ago by Josh Crompton

Unfortunately not, if you include a unicode character in Bar.extra and remove the decorator, you'll get something like:

Traceback (most recent call last):
  File "./recursion.py", line 34, in <module>
    print(b)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u1546' in position 4: ordinal not in range(128)

comment:5 Changed 7 years ago by Josh Crompton

I believe there's a related infinite-recursion problem on saving models in certain circumstances but I haven't had time to extract a useful example from the project in which I'm seeing it. At this point, I think it's a problem with the implementation of __str__ in django.db.models.base.Model.

comment:6 Changed 7 years ago by Aymeric Augustin

I don't understand how super().__str__() can work as super() without arguments is illegal in Python 2.

comment:7 Changed 7 years ago by Tim Graham

I think the future library does some monkeypatching to allow it to work, but it would be interesting to know if this issue can be reproduced without it.

comment:8 Changed 7 years ago by Simon Charette

I reproduce with the correct super() call on Python 2.7:

In [1]: from django.utils.six import python_2_unicode_compatible

In [2]: @python_2_unicode_compatible
class A(object):
    def __str__(self):
        return str('a')
   ...:     

In [3]: str(A())
Out[3]: 'a'

In [4]: unicode(A())
Out[4]: u'a'

In [5]: class B(A):
    def __str__(self):
        return str('b') + super(B, self).__str__()
   ...:     

In [6]: str(B())
Out[6]: 'ba'

In [7]: unicode(B())  # Notice how B.__str__() isn't called.
Out[7]: u'a'

In [8]: @python_2_unicode_compatible
class C(A):
    def __str__(self):
        return str('c') + super(C, self).__str__()
   ...:     

In [9]: str(C())
--------------------------------------------------------------------------
...
<ipython-input-8-efd21456ef94> in __str__(self)
      2 class C(A):
      3     def __str__(self):
----> 4         return str('c') + super(C, self).__str__()
      5 

django/django/utils/six.pyc in <lambda>(self)
    810                              klass.__name__)
    811         klass.__unicode__ = klass.__str__
--> 812         klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
    813     return klass
    814 

RuntimeError: maximum recursion depth exceeded in __subclasscheck__
In [10]: unicode(C())
--------------------------------------------------------------------------
...
<ipython-input-8-efd21456ef94> in __str__(self)
      2 class C(A):
      3     def __str__(self):
----> 4         return str('c') + super(C, self).__str__()
      5 

django/django/utils/six.pyc in <lambda>(self)
    810                              klass.__name__)
    811         klass.__unicode__ = klass.__str__
--> 812         klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
    813     return klass
    814 

RuntimeError: maximum recursion depth exceeded in __subclasscheck__

comment:9 in reply to:  7 Changed 7 years ago by Josh Crompton

Replying to timgraham:

I think the future library does some monkeypatching to allow it to work, but it would be interesting to know if this issue can be reproduced without it.

Correct, I should have mentioned that in the ticket, sorry. As @charettes demos above, it's reproducible with the normal Python2-style super call.

Last edited 7 years ago by Josh Crompton (previous) (diff)

comment:10 Changed 7 years ago by Aymeric Augustin

@python_2_unicode_compatible changes the implementation of __str__ and __unicode__ on Python 2. If you want to call the __str__ method you defined in the parent class, you have to call super(B, self).__unicode__().

The following works both on Python 2 and 3:

from __future__ import print_function, unicode_literals

from django.utils.six import PY3, python_2_unicode_compatible

@python_2_unicode_compatible
class A(object):
    def __str__(self):
        return 'a'

@python_2_unicode_compatible
class B(A):
    def __str__(self):
        text_method = '__str__' if PY3 else '__unicode__'
        return getattr(super(B, self), text_method)() + 'b'

print(str(B()))

This version is more readable:

from __future__ import print_function, unicode_literals

from django.utils.six import PY3, python_2_unicode_compatible

@python_2_unicode_compatible
class A(object):
    def text(self):
        return 'a'
    __str__ = text

@python_2_unicode_compatible
class B(A):
    def text(self):
        return super(B, self).text() + 'b'
    __str__ = text

print(str(B()))

This isn't obvious but I can't see a way around it. (I'm the original author of this decorator which was added to six later.)

comment:11 Changed 7 years ago by Tim Graham

Resolution: wontfix
Status: newclosed

comment:12 Changed 6 years ago by TZanke

Cc: tzanke@… added
Note: See TracTickets for help on using tickets.
Back to Top