Cleanups when no supported version of Django supports Python 2 anymore

[Aymeric Augustin]

Compatibility with Python 2 is currently being removed in master as part of the Django 2.0 development cycle: see #23919.

However third-party apps may want to remain compatible with all supported version of Django and the corresponding versions of Python, which includes Python 2.

To allow this, compatibility functions or modules aren't removed. They're still importable. They're usually just aliases or no-ops. Here's a list of things to deprecate and remove eventually:

• django.utils.six
• django.utils.lru_cache
• django.utils._os.abspathu, upath, npath
• django.utils.decorators.available_attrs
• django.utils.encoding.python_2_unicode_compatible
• django.utils.encoding.smart/force_text
• django.utils.http.urlquote/urlquote_plus/urlunquote/urlunquote_plus (​unmerged commit)
• django.utils.safestring.SafeBytes
• One of either ​django.utils.safestring.SafeString or django.utils.safestring.SafeText
• django.test.utils.str_prefix()
• django.utils.encoding.DjangoUnicodeDecodeError
• django.utils.functional.curry() (in favor of functools.partial()/partialmethod(); see 5b1c389603a353625ae1603ba345147356336afb)
• django.test.utils.patch_logger()

[Simon Charette]

Should we revert the removal of eb0b921c29ace8643a5a4cd136c433727c53dead in this case?

[Tim Graham]

Those methods don't work on Python 3 (AttributeError: 'dict' object has no attribute 'iterkeys') so I don't think they have value for the use case described in the ticket description. A third-party app should use six.iterkeys() as the old test did (which uses keys() etc. on Python 3), or am I missing something?

[Claude Paroz]

The question we discussed on IRC and GitHub with Tim is about deprecating or not deprecating those items.

I would be for a standard deprecation process as I don't see how this is different from any other deprecation. But I'm open to read about use cases which would push for a delayed deprecation process for items related to Python 2.

[Tim Graham]

Please don't work on this ticket -- the timetable for these removals isn't decided but it's not for some years. The "Someday/Maybe" "Triage Stage" of the ticket indicates that it's not appropriate to work on right now.

[Tim Graham <timograham@…>]

In 0a66aa15:

Refs #27308, #27753 -- Removed obsolete cookie test mixing bytes with str.

Python 3's SimpleCookie treats all values as strings. Passing a bytes
object coerces to the repr value.

[Tim Graham]

I think this was discussed on the mailing list or in IRC but I can't find the discussion so I'll state it here. The reason we don't want to deprecate this stuff now is because that would add lots of non-actionable warnings for Django libraries that want to support Python 2 and 3 until 2020.

[Simon Charette]

I just wanted to mention that in my opinion it would make more sense for SafeText to go away instead of SafeString to be consistent with the (smart|force)_text removal.

Given we're currently publicly documenting that SafeText is an alias for SafeString we should adjust the code and documentation for it to be other way around instead before we move forward with the SafeText removal or deprecation.

[GitHub <noreply@…>]

In 83c2bc52:

Refs #27753 -- Deprecated django.utils.http urllib aliases.

[Tim Graham]

​PR for some removals.

[Tim Graham]

​PR for favoringforce_str(), smart_str(), and SafeString over force_text(), smart_text(), and SafeText. It also deprecates the former functions. No deprecation for SafeText because it seems more trouble than it's worth right now (deprecation warnings in __init__() and __isinstance__()?).

[Tim Graham]

My PRs address everything in this ticket except DjangoUnicodeDecodeError -- I'm not sure about the rationale for removing it as it still may provide some value:
DjangoUnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte. You passed in b'\xff' (<class 'bytes'>) vs. UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte.

Could you explain your thinking, Jon? Maybe the mistake leading to this exception (#5640) isn't common enough in Python 3 to continue with the special handling?

[Jon Dufresne]

I can't recall exactly, but I suspect my thinking was along the lines of:

1. After dropping Python 2, force_text() & friends would eventually be removed from Django and replaced by stricter type contracts at the boundaries. This has proved to be time consuming and difficult to accomplish, so this isn't yet complete (but we're getting there).

2. DjangoUnicodeDecodeError looks like a close duplicate of UnicodeDecodeError. As you pointed out, it is not as it provides more information. Although, if the Python version is insufficient, perhaps effort could be spent improving that message.

So, given the above, I'm OK for it to stick around. No need to remove it.

Maybe the mistake leading to this exception (#5640) isn't common enough in Python 3 to continue with the special handling?

I suspect that is probably true, but I can't say for certain.

[Tim Graham <timograham@…>]

In 6d2ae49:

Refs #27753 -- Removed django.test.utils.patch_logger() and str_prefix().

[Tim Graham <timograham@…>]

In c679f357:

Refs #27753 -- Removed django.utils.lru_cache.

[Tim Graham <timograham@…>]

In d1f4b3c:

Refs #27753 -- Removed django.utils.decorators.available_attrs().

[Tim Graham <timograham@…>]

In efe28d3:

Refs #27753 -- Removed django.utils._os.abspathu(), upath(), and npath().

[Tim Graham <timograham@…>]

In 41384812:

Refs #27753 -- Removed django.utils.six.

[Tim Graham <timograham@…>]

In 9a750cb:

Refs #27753 -- Removed django.utils.decorators.ContextDecorator alias.

[Tim Graham <timograham@…>]

In 3004d705:

Refs #27753 -- Removed django.utils.functional.curry().

[Tim Graham <timograham@…>]

In f09b0f6:

Refs #27753 -- Removed django.utils.safestring.SafeBytes.

[Tim Graham <timograham@…>]

In 3bb6a439:

Refs #27753 -- Favored force/smart_str() over force/smart_text().

[Tim Graham <timograham@…>]

In d55e8829:

Refs #27753 -- Deprecated django.utils.encoding.force_text() and smart_text().

[Tim Graham <timograham@…>]

In 77d25dbd:

Refs #27753 -- Favored SafeString over SafeText.

[Mariusz Felisiak <felisiak.mariusz@…>]

In b915b9f:

Refs #27753 -- Deprecated django.utils.text.unescape_entities().

The function was undocumented and only required for compatibility with
Python 2.

Code should use Python's html.unescape() that was added in Python 3.4.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 88ed1c8d:

Refs #27753 -- Removed django.utils.http urllib aliases per deprecation timeline.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 810f037b:

Refs #27753 -- Removed django.utils.encoding.force_text() and smart_text() per deprecation timeline.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 157ab32f:

Refs #27753 -- Removed django.utils.text.unescape_entities() per deprecation timeline.

[GitHub <noreply@…>]

In a21a63cc:

Refs #27753 -- Removed unused django.utils.text._replace_entity() and _entity_re.

Unused since 157ab32f3446da7fa1f9d716509c290069a2a156.