Improve free-threading performance

[Sam Gross]

I’m a CPython core developer working on free-threading. Would the Django team be interested in reviewing patches aimed at improving Django's performance on free-threaded Python (e.g., 3.14t)?

For example, there are a few scaling bottlenecks caused by creating temporary Field instances, such as in rel_db_type(). This can lead to contention on the shared creation_counter and also invalidate CPython’s attribute lookup cache. I expect most of the patches to be fairly small.

For context, I’m using Zulip to identify and investigate these issues, and I’m also working to fix the underlying problems in CPython upstream where possible.

[Natalia Bidart]

Hello Sam! Thank you for the ticket and for the suggestion to improve Django's performance on free-threaded Python.

I think this is a generous offer, and that Django would benefit from this work. Since Django is a community-driven project, the best place to discuss this proposal is on the ​Django Forum. Please consider opening a topic in the “Django Internals” category outlining your proposal. It would also be very helpful if you could share some details about the issues you have identified so far.

I'll make sure to subscribe to the forum post and monitor the replies. Once there is discussion and community interest on the forum, we can reopen and accept the ticket if there is clear agreement to proceed.

[Sam Gross]

Thank you Natalia! I posted here: ​https://forum.djangoproject.com/t/improve-free-threading-performance/44497

[Jacob Walls]

I agree with Natalia that Django will benefit from this work. Reception was positive on the forum. Adjusting the ​triage state to reflect the "long-term changes" aimed at here.

[Carlton Gibson]

​PR skipping some GC related tests with the GIL disabled, since the behaviour changed.

[Jacob Walls <jacobtylerwalls@…>]

In 5d489a5:

Refs #36983 -- Skipped GC specific tests when GIL disabled.

[Carlton Gibson]

​New PR adds GHA CI runs against Python 3.13t and 3.14t.

[Carlton Gibson]

Still small numbers of (maybe intermittent) test failures to investigate on PR branch.

[Carlton Gibson]

Fixed the test failures on the 3.13t and 3.14t runs, so should be good for a review.

[Simon Charette]

Left some comments on the PR.

I think we should consider explicitly not supporting/testing against 3.13t given it systematically immortalize locally scoped class objects which Django dynamically creates a ton of via type(name, bases, attrs) (e.g. think of formset/inline usage in the admin) and will result in memory leaks.

[Carlton Gibson]

I think we should consider explicitly not supporting/testing against 3.13t…

Agreed. Dropped it from the test matrix in the PR.

[Jacob Walls <jacobtylerwalls@…>]

In 726440a4:

Refs #36983 -- Added CI tests on free-threaded Python 3.13t and 3.14t.

[Jacob Walls <jacobtylerwalls@…>]

In cae38ec:

Refs #36983 -- Fixed race condition in ExceptionReporterTests under free-threading.