Opened 5 months ago

Closed 5 months ago

#34986 closed Bug (fixed)

Fix up support for PyPy 3.10

Reported by: Nick Pope Owned by: Nick Pope
Component: Core (Other) Version: dev
Severity: Normal Keywords: pypy
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Python 3.10 support was released in PyPy v7.3.12 in June 2023.

It was highlighted here that a number of tests are failing, and here and here that the changes in #31262 have introduced some large "regressions". It turns out that there is a bug in PyPy related to abstract base classes being used for isinstance() checks with structural pattern matching. This can be resolved by either waiting for a fix in PyPy or implementing a workaround by removing the pattern matching approach. Aside from that, however, there are also a number of other failures and crashes that still need to be addressed.

PyPy claims to be highly compatible with CPython and that it supports Django. Django claims to be compatible with PyPy. We need to determine whether it is possible to fix these issues or whether we need to remove this claim.

One of the issues is that currently there is no CI to ensure that PyPy support remains in a working state. An initial PR has been created to run a daily scheduled test run on SQLite only. Later it may make sense to enable triggering testing via assigning a label to a PR.

Change History (18)

comment:1 by Mariusz Felisiak, 5 months ago

Component: UncategorizedCore (Other)
Triage Stage: UnreviewedAccepted

comment:2 by Nick Pope, 5 months ago

Has patch: set

comment:3 by GitHub <noreply@…>, 5 months ago

In 0d407bc:

Refs #34986 -- Installed libmemcached-dev in PyPy 3.10 daily builds.

comment:4 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 174369a9:

Refs #34986 -- Avoided pickling error in DjangoUnicodeDecodeError.

By using the existing object reference instead of a custom one, pickling
related issues when running the test suite in parallel can be avoided.

comment:5 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 9baaf89:

Refs #34986 -- Added django.utils.version.PYPY.

comment:6 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 6089230:

Refs #34986 -- Fixed mocking in utils_tests.test_http.HttpDateProcessingTests.test_parsing_rfc850.

Mocking in the datetime module can be tricky. In CPython the datetime
C module is used, but PyPy uses a pure Python implementation. This
caused issues with the prior approach to mocking datetime.datetime.

See https://docs.python.org/3/library/unittest.mock-examples.html#partial-mocking

comment:7 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 051dbb53:

Refs #34986 -- Added PyPy support to django.utils.autoreload.get_child_arguments().

It seems that sys._xoptions is also supported on PyPy.

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In baf705f3:

Refs #34986 -- Fixed some test assertions for PyPy.

These failures were due to minor inconsistencies or implementation
differences between CPython and PyPy.

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 5f9e5c1:

Refs #34822, Refs #34986 -- Fixed migrations serializer support for functools.lru_cache().

It turns out that functools._lru_cache_wrapper is only a class when
CPython's _functools C module provides it, otherwise it is a function.
PyPy also provides it as a function.

comment:10 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 06c5cb12:

Refs #34986 -- Removed redundant CustomChoicesTests.test_timezone_unsupported().

This test relied on the behavior of subclassing datetime.timezone
which is not permitted by the C-extension version of CPython's
datetime module. This restriction isn't enforced by the pure
Python version, nor by PyPy.

See https://github.com/python/cpython/issues/112451

It's not critical, and doesn't test any Django behavior, so just
remove it.

comment:11 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 1c3614e3:

Refs #34986 -- Avoided implementation-specific unpickleable types.

The implementation of some core types differ between CPython and PyPy
and this may affect the way that pickling works such that errors are
raised in differing locations in the interpreter or not at all.

Use our own custom non-pickleable type instead to avoid these quirks.

comment:12 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In fb921638:

Refs #34986 -- Moved garbage_collect() helper to django.test.utils.

comment:13 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 66d58e77:

Refs #34986 -- Added PyPy to tox.ini.

comment:14 by Mariusz Felisiak, 5 months ago

Triage Stage: AcceptedReady for checkin

comment:15 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 958a590e:

Refs #34986 -- Used non-binary build of psycopg for PyPy.

comment:16 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In af6e7e3d:

Refs #34986 -- Fixed backends.postgresql.test_server_side_cursors.ServerSideCursorsPostgres tests for PyPy.

comment:17 by Mariusz Felisiak <felisiak.mariusz@…>, 5 months ago

In 2dca98f:

Refs #34986 -- Added PyPy 3.10 with PostgreSQL to daily builds.

comment:18 by Mariusz Felisiak, 5 months ago

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.
Back to Top