Audit force_text invocations

[Claude Paroz]

django.utils.encoding.force_text is called many times in Django code for several use cases:

• force the evaluation of a lazy string
• convert bytes or a bytestring (PY2) to str
• convert an object to its string representation

A global audit is needed to clean that usage, also considering that Python 3 native strings are more wildly used.
I'm working on it currently.

[Claude Paroz <claude@…>]

In ccfd1295:

Refs #27795 -- Prevented SafeText from losing safe status on str()

This will allow to replace force_text() by str() in several places (as one of
the features of force_text is to keep the safe status).

[Claude Paroz <claude@…>]

In 2757209c:

Refs #27795 -- Removed force_text from templatize function

[Claude Paroz <claude@…>]

In 3a148f95:

Refs #27795 -- Removed force_text from the template layer

Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In 8346680e:

Refs #27795 -- Removed unneeded force_text calls

Thanks Tim Graham for the review.

[GitHub <noreply@…>]

In e5dce7b0:

Refs #27795 -- Removed unneeded force_text calls from the Oracle backend.

[Claude Paroz <claude@…>]

In 301de77:

Refs #27795 -- Replaced many force_text() with str()

Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In a8343fe7:

Refs #27795 -- Replaced force_text() with str() in feed generators

[Tim Graham <timograham@…>]

In b39aabc:

Refs #27795 -- Reworked get_or_create test erroneously mixing bytes and str.

As CharField.to_python() now always calls str(), assigning bytes to a
CharField is no longer correct usage. Doing so results in a warning:

django/db/models/fields/init__.py:1061: BytesWarning: str() on a bytes instance

Use a unique constraint violation to trigger the database error instead.

Warning introduced in 301de774c21d055e9e5a7073e5bffdb52bc71079.

[Claude Paroz <claude@…>]

In d842ada:

Refs #27795 -- Stopped converting integer format settings to str in JS/JSON i18n views

Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In f87256b1:

Refs #27795 -- Replaced force_text() with str() in contrib.messages

Thanks Tim Graham for the review.

[Claude Paroz <claude@…>]

In 45bbb9aa:

Refs #27795 -- Removed force_text() usage in serialization framework

[Tim Graham <timograham@…>]

In 6b2f8fb9:

Refs #27795 -- Replaced force_text() usage in django.core.management.

Use decode() since Popen.communicate() always returns bytes.

[Tim Graham <timograham@…>]

In 16c5a334:

Refs #27795 -- Replaced force_text/bytes() with decode()/encode() in password hashers.

[Carlton Gibson <carlton.gibson@…>]

In 6a039800:

Refs #27795 -- Replaced force_bytes() usage in django.core.cache.

[Tim Graham <timograham@…>]

In b38532c:

Refs #27795 -- Replaced force_bytes() usage in django.core.signing.

[GitHub <noreply@…>]

In fa75b2c:

Refs #27795 -- Removed force_bytes/text() usage in tests.

[Tim Graham <timograham@…>]

In 7d96f0c4:

Refs #27795 -- Replaced force_bytes() usage in django.http.

[Jon Dufresne <jon.dufresne@…>]

In 968329b:

Refs #27795 -- Replaced force_text() calls with str() in django.utils.html.

[Tim Graham <timograham@…>]

In 1f3852b:

Refs #27795 -- Removed force_bytes() usage in django/template/loaders/cached.py.

[Tim Graham <timograham@…>]

In 919d5981:

Refs #27795 -- Removed force_text() usage in django/test/testcases.py.

[Tim Graham <timograham@…>]

In b8a41a28:

Refs #27795 -- Removed force_bytes() in django.test.client where possible.

[Tim Graham <timograham@…>]

In 90f3ef3:

Refs #27795 -- Removed force_bytes()/smart_text() from model fields.

[Tim Graham <timograham@…>]

In ee184e1:

Refs #27795 -- Removed force_bytes() usage in django/utils/cache.py.

[Tim Graham <timograham@…>]

In cc79c7ee:

Refs #27795 -- Removed force_text() in Model._get_FIELD_display().

[Tim Graham <timograham@…>]

In 1d65ddd9:

Refs #27795 -- Removed force_bytes() usage in db/backends/utils.py.

[Tim Graham <timograham@…>]

In abeed587:

Refs #27795 -- Removed force_bytes() usage in db/backends/base/schema.py.

[Tim Graham <timograham@…>]

In bb81c22:

Refs #27795 -- Removed force_bytes() usage in utils/_os.py.

[Tim Graham <timograham@…>]

In 8d873503:

Refs #27795 -- Removed force_bytes() usage in contrib/auth/handlers/modwsgi.py.

[Tim Graham <timograham@…>]

In d4fc111c:

Refs #27795 -- Removed force_bytes() usage in admindocs.

Refs #12892 is probably obsolete.

[Carlton Gibson <carlton.gibson@…>]

In efd8a82e:

Refs #27795 -- Removed force_bytes() usage in MySQL backend.

The mysqlclient cursor attribute _last_executed is always stored as
bytes. Decode it.

TextField values are already type str. No need to decode.

[Carlton Gibson <carlton.gibson@…>]

In a47dd99:

[2.1.x] Refs #27795 -- Removed force_bytes() usage in MySQL backend.

The mysqlclient cursor attribute _last_executed is always stored as
bytes. Decode it.

TextField values are already type str. No need to decode.

Backport of efd8a82e268a82b3ad0be77bd5b4548c30bcb4d7 from master

[Carlton Gibson <carlton.gibson@…>]

In bdae19cf:

Refs #27795 -- Removed force_bytes() usage in sessions.

SessionBase.decode() is the inverse operation to SessionBase.encode().
As SessionBase.encode() always returns a string, SessionBase.decode()
should always be passed a string argument. Fixed the file backend, which
was the only backend still passing a bytestring.

[Carlton Gibson <carlton.gibson@…>]

In f3f31b0:

[2.1.x] Refs #27795 -- Removed force_bytes() usage in sessions.

SessionBase.decode() is the inverse operation to SessionBase.encode().
As SessionBase.encode() always returns a string, SessionBase.decode()
should always be passed a string argument. Fixed the file backend, which
was the only backend still passing a bytestring.

Backport of bdae19cf6395d6bfee80864d9e87c4aec241eceb from master

[Carlton Gibson <carlton.gibson@…>]

In 54eedddb:

Revert "[2.1.x] Refs #27795 -- Removed force_bytes() usage in MySQL backend."

This reverts commit a47dd99cc6883fe3e35895e8c5d4bb53996bd184.

[Carlton Gibson <carlton.gibson@…>]

In 5045f8af:

Revert "[2.1.x] Refs #27795 -- Removed force_bytes() usage in sessions."

This reverts commit f3f31b0fc26f772b198c9683a427f2f7fe208c37.

[Tim Graham <timograham@…>]

In 4ab071f4:

Refs #27795 -- Removed force_bytes() usage in contrib/staticfiles/storage.py.

[Tim Graham <timograham@…>]

In 2ba588e7:

Refs #27795 -- Removed force_text() usage in db/models/sql/query.py.

[Tim Graham <timograham@…>]

In c82893cb:

Refs #27795 -- Removed force_bytes() usage from django/utils/http.py.

django.utils.http.urlsafe_base64_encode() now returns a string, not a
bytestring. Since URLs are represented as strings,
urlsafe_base64_encode() should return a string. All uses immediately
decoded the bytestring to a string anyway.

As the inverse operation, urlsafe_base64_decode() accepts a string.

[Claude Paroz]

I think this is mostly done now.

[Carlton Gibson <carlton.gibson@…>]

In 1e6b9e2:

Refs #27795 -- Removed an unnecessary force_bytes() call in uri_to_iri().

The value returned from urllib.parse.quote() is always a string, so can
safely call .encode().