Python 3.12 compatibility

[Mariusz Felisiak]

Python 3.12 final is scheduled for October 2023 (see ​PEP 693). This is a tracking ticket for compatibility fixes for Django submitted in the meantime.

Django 4.2 will be the first version to support Python 3.12, because Django 4.1 will end the mainstream support in April 2023.

[David Sanders]

I'm getting errors on the following which was working for 3.11.0.

I'm on macos 12.6

tests main % python --version
Python 3.11.0
tests main % ./runtests.py queries.tests.Queries1Tests.test_avoid_infinite_loop_on_too_many_subqueries
Testing against Django installed in '/Users/dsanders/projects/django/django' with up to 10 processes
Found 1 test(s).
Creating test database for alias 'default'...
System check identified no issues (1 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.295s

OK
Destroying test database for alias 'default'...


… 8< … switch venvs


tests main % python --version
Python 3.12.0a1
tests main % ./runtests.py queries.tests.Queries1Tests.test_avoid_infinite_loop_on_too_many_subqueries
Testing against Django installed in '/Users/dsanders/projects/django/django' with up to 10 processes
Found 1 test(s).
Creating test database for alias 'default'...
System check identified no issues (1 silenced).
F
======================================================================
FAIL: test_avoid_infinite_loop_on_too_many_subqueries (queries.tests.Queries1Tests.test_avoid_infinite_loop_on_too_many_subqueries)
----------------------------------------------------------------------
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dsanders/projects/django/tests/queries/tests.py", line 550, in test_avoid_infinite_loop_on_too_many_subqueries
    with self.assertRaisesMessage(RecursionError, msg):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 155, in __exit__
    self.gen.throw(value)
  File "/Users/dsanders/projects/django/django/test/testcases.py", line 890, in _assert_raises_or_warns_cm
    self.assertIn(expected_message, str(getattr(cm, cm_attr)))
AssertionError: 'Maximum recursion depth exceeded: too many subqueries.' not found in 'maximum recursion depth exceeded'

----------------------------------------------------------------------
Ran 1 test in 0.207s

FAILED (failures=1)
Destroying test database for alias 'default'...

[bcail]

Another data point: SQLite and Postgres tests pass for me on Linux with Python 3.12 alpha 1.

[GitHub <noreply@…>]

In 67504ea5:

Refs #34118 -- Skipped not compatible requirements on daily builds for Python 3.12.

[GitHub <noreply@…>]

In 2848e5d:

Refs #34118 -- Used smtplib's context parameter instead of keyfile and certfile.

keyfile and certfile parameters were removed in Python 3.12, see
​https://github.com/python/cpython/commit/ef0e72b31d22f780d3a165d7d0471806061fe380

[GitHub <noreply@…>]

In 32d70b2:

Refs #34118 -- Adopted asgiref coroutine detection shims.

Thanks to Mariusz Felisiak for review.

[GitHub <noreply@…>]

In 829f4d1:

Refs #31546, Refs #34118 -- Corrected CommandTests.test_requires_system_checks_specific().

System checks are never called without skip_checks=False. Moreover,
called_once_with() is not a proper assertion and raise AttributeError
on Python 3.12.

[GitHub <noreply@…>]

In 38e63c9:

Refs #34118 -- Fixed CustomChoicesTests.test_uuid_unsupported on Python 3.12+.

​https://github.com/python/cpython/commit/2a4d8c0a9e88f45047da640ce5a92b304d2d39b1

[Mariusz Felisiak <felisiak.mariusz@…>]

In 32cfa73c:

[4.2.x] Refs #34118 -- Fixed CustomChoicesTests.test_uuid_unsupported on Python 3.12+.

​https://github.com/python/cpython/commit/2a4d8c0a9e88f45047da640ce5a92b304d2d39b1
Backport of 38e63c9e61152682f3ff982c85a73793ab6d3267 from main

[Sam James]

Hi folks,

​https://github.com/django/django/commit/38e63c9e61152682f3ff982c85a73793ab6d3267 was applied to fix Python 3.12, but cpython upstream have backported the fix for ​https://github.com/python/cpython/issues/102549 to Python 3.11 too.

The Django patch will need adjusting to probably just allow both output messages (or sniff the Python minor/patch version).

[Mariusz Felisiak]

Replying to thesamesam:

Hi folks,

​https://github.com/django/django/commit/38e63c9e61152682f3ff982c85a73793ab6d3267 was applied to fix Python 3.12, but cpython upstream have backported the fix for ​https://github.com/python/cpython/issues/102549 to Python 3.11 too.

The Django patch will need adjusting to probably just allow both output messages (or sniff the Python minor/patch version).

Thanks! It's unfortunate, will fix this.

[GitHub <noreply@…>]

In 2eb1f372:

Refs #34118 -- Fixed CustomChoicesTests.test_uuid_unsupported on Python 3.11.4+.

​https://github.com/python/cpython/commit/5342f5e713e0cc45b6f226d2d053a8cde1b4d68e

Follow up to 38e63c9e61152682f3ff982c85a73793ab6d3267.

[Mariusz Felisiak <felisiak.mariusz@…>]

In f6e0029:

[4.2.x] Refs #34118 -- Fixed CustomChoicesTests.test_uuid_unsupported on Python 3.11.4+.

​https://github.com/python/cpython/commit/5342f5e713e0cc45b6f226d2d053a8cde1b4d68e

Follow up to 38e63c9e61152682f3ff982c85a73793ab6d3267.
Backport of 2eb1f37260f0e0b71ef3a77eb5522d2bb68d6489 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In f55bcff9:

[4.1.x] Refs #34118 -- Fixed CustomChoicesTests.test_uuid_unsupported on Python 3.11.4+.

​https://github.com/python/cpython/commit/5342f5e713e0cc45b6f226d2d053a8cde1b4d68e

Follow up to 38e63c9e61152682f3ff982c85a73793ab6d3267.

Backport of 2eb1f37260f0e0b71ef3a77eb5522d2bb68d6489 from main

[GitHub <noreply@…>]

In 5a6d4d3b:

Refs #34118, Refs #34504 -- Added backward incompatibility note about EmailBackend.ssl_context.

Follow up to 2848e5d0ce5cf3c31fe87525536093b21d570f69.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 4f343a1:

[4.2.x] Refs #34118, Refs #34504 -- Added backward incompatibility note about EmailBackend.ssl_context.

Follow up to 2848e5d0ce5cf3c31fe87525536093b21d570f69.
Backport of 5a6d4d3bfde07daab9777545694beb014c832264 from main

[GitHub <noreply@…>]

In fc9c90d9:

Refs #34118 -- Fixed FunctionalTests.test_cached_property_reuse_different_names() on Python 3.12+.

Python 3.12+ no longer wraps exceptions in set_name, see
​https://github.com/python/cpython/commit/55c99d97e14618dfce41472dd4446f763b0da13f

[Mariusz Felisiak <felisiak.mariusz@…>]

In bf5249fc:

[4.2.x] Refs #34118 -- Fixed FunctionalTests.test_cached_property_reuse_different_names() on Python 3.12+.

Python 3.12+ no longer wraps exceptions in set_name, see
​https://github.com/python/cpython/commit/55c99d97e14618dfce41472dd4446f763b0da13f
Backport of fc9c90d9c4611d441d2617a7d74f622b1f520f7c from main

[GitHub <noreply@…>]

In da2f8e8:

Refs #34118 -- Improved sanitize_address() error message for tuple with empty strings.

[GitHub <noreply@…>]

In 62b83889:

Fixed #34731, Refs #34118 -- Replaced assertEquals() with assertEqual() in test_condition_with_func_and_lookup_outside_relation_name().

Signed-off-by: Michel Alexandre Salim <salimma@…>

[Mariusz Felisiak]

​https://www.python.org/downloads/release/python-3120/

[GitHub <noreply@…>]

In 20b7aac7:

Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.1+.

Python 3.12.1+ no longer includes skipped tests in the number of
running tests. Check out:

​https://github.com/python/cpython/issues/110890#issuecomment-1763458686
​https://github.com/python/cpython/pull/106588

[Mariusz Felisiak <felisiak.mariusz@…>]

In 448510a9:

[5.0.x] Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.1+.

Python 3.12.1+ no longer includes skipped tests in the number of
running tests. Check out:

​https://github.com/python/cpython/issues/110890#issuecomment-1763458686
​https://github.com/python/cpython/pull/106588
Backport of 20b7aac7ca60b0352d926340622e618bcbee54a8 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In a576ef9:

[4.2.x] Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.1+.

Python 3.12.1+ no longer includes skipped tests in the number of
running tests. Check out:

​https://github.com/python/cpython/issues/110890#issuecomment-1763458686
​https://github.com/python/cpython/pull/106588
Backport of 20b7aac7ca60b0352d926340622e618bcbee54a8 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In e2922b0:

Refs #34118 -- Avoided repeat coroutine checks in MiddlewareMixin.

[GitHub <noreply@…>]

In 19c4052f:

Refs #34118 -- Used Python 3.12 for GitHub actions.

[Mariusz Felisiak]

​PR

[GitHub <noreply@…>]

In b5750b9f:

[4.2.x] Refs #34118 -- Fixed stacklevel in complex_setting_changed on Python 3.12.

This fix is unnecessary in Django 5.0 since
e83a88566a71a2353cebc35992c110be0f8628af because signals no longer use
sync_to_async().

[Mariusz Felisiak <felisiak.mariusz@…>]

In f955c90:

Refs #34118 -- Added Python 3.12 to classifiers and tox.ini.

[Mariusz Felisiak <felisiak.mariusz@…>]

In ecfea05:

Refs #34118 -- Doc'd Python 3.12 compatibility in Django 4.2.x.

[Mariusz Felisiak <felisiak.mariusz@…>]

In c342b58:

[5.0.x] Refs #34118 -- Added Python 3.12 to classifiers and tox.ini.

Backport of f955c90488e1f32e6e38b3c956e937579d389fb2 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In f721fb93:

[5.0.x] Refs #34118 -- Doc'd Python 3.12 compatibility in Django 4.2.x.

Backport of ecfea054ee2b8ddfa027459ff8b6aecba05facf7 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In 233e2a9e:

[4.2.x] Refs #34118 -- Added Python 3.12 to classifiers and tox.ini.

Backport of f955c90488e1f32e6e38b3c956e937579d389fb2 from main.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9afeb6b9:

[4.2.x] Refs #34118 -- Doc'd Python 3.12 compatibility in Django 4.2.x.

Backport of ecfea054ee2b8ddfa027459ff8b6aecba05facf7 from main.

[Mariusz Felisiak <felisiak.mariusz@…>]

In bc8471f0:

Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.2+.

Python 3.12.2 bring back the skipped tests in the number of running
tests. Refs
​https://github.com/python/cpython/commit/0a737639dcd3b7181250f5d56694b192eaddeef0

[Mariusz Felisiak <felisiak.mariusz@…>]

In 1ba5afa:

[5.0.x] Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.2+.

Python 3.12.2 bring back the skipped tests in the number of running
tests. Refs
​https://github.com/python/cpython/commit/0a737639dcd3b7181250f5d56694b192eaddeef0

Backport of bc8471f0aac8f0c215b9471b594d159783bac19b from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In 86d8034:

[4.2.x] Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.2+.

Python 3.12.2 bring back the skipped tests in the number of running
tests. Refs
​https://github.com/python/cpython/commit/0a737639dcd3b7181250f5d56694b192eaddeef0

Backport of bc8471f0aac8f0c215b9471b594d159783bac19b from main

[Sarah Boyce <42296566+sarahboyce@…>]

In 338ec052:

Refs #35361 -- Added test for Email line length checks when dealing with surrogate pairs.

Refs #33173, #34118 and #34900.

[Sarah Boyce <42296566+sarahboyce@…>]

In 83ec18b1:

[5.0.x] Refs #34118 -- Used Python 3.12 for GitHub actions.