Opened 4 months ago

Closed 4 months ago

Last modified 3 months ago

#35361 closed Bug (fixed)

SafeMIMEText.set_payload() crash on Python 3.11.9

Reported by: Sarah Boyce Owned by: Natalia Bidart
Component: Core (Mail) Version: 4.2
Severity: Release blocker Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

A recent bug fix to Python gh-76511 has been back ported to Python 3.11 and 3.12.
See the release notes of Python 3.11.9 and the soon to be released Python 3.12.3.

As part of the work to prepare Django for Python 3.13, a fix for this has already been merged to main in b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8.

Believe this needs to be back ported to Django 4.2 and Django 5.0 to support the latest (or soon to be latest) Python 3.11 and Python 3.12 versions.

Confirmed when testing Django 4.2 against Python 3.11.9, we know have the following test failure:

======================================================================
ERROR [0.034s]: test_dont_base64_encode (tests.mail.tests.MailTests.test_dont_base64_encode)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tests/django/tests/mail/tests.py", line 902, in test_dont_base64_encode
    s = msg.message().as_string()
        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tests/django/django/core/mail/message.py", line 133, in as_string
    g.flatten(self, unixfrom=unixfrom, linesep=linesep)
  File "/usr/local/lib/python3.11/email/generator.py", line 115, in flatten
    self._write(msg)
  File "/usr/local/lib/python3.11/email/generator.py", line 180, in _write
    self._dispatch(msg)
  File "/usr/local/lib/python3.11/email/generator.py", line 217, in _dispatch
    meth(msg)
  File "/usr/local/lib/python3.11/email/generator.py", line 246, in _handle_text
    msg.set_payload(msg._payload, charset)
  File "/tests/django/django/core/mail/message.py", line 168, in set_payload
    has_long_lines = any(
                     ^^^^
  File "/tests/django/django/core/mail/message.py", line 169, in <genexpr>
    len(line.encode()) > RFC5322_EMAIL_LINE_LENGTH_LIMIT
        ^^^^^^^^^^^^^
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 28-33: surrogates not allowed

Change History (12)

comment:1 by Natalia Bidart, 4 months ago

Triage Stage: UnreviewedAccepted

comment:2 by Natalia Bidart, 4 months ago

Has patch: set
Owner: changed from nobody to Natalia Bidart
Status: newassigned

PR for main adding a test (to be backported along with the fix from b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8):
https://github.com/django/django/pull/18057/

comment:3 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In 338ec052:

Refs #35361 -- Added test for Email line length checks when dealing with surrogate pairs.

Refs #33173, #34118 and #34900.

comment:4 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

Resolution: fixed
Status: assignedclosed

In 42435fc5:

Fixed #35361 -- Added release notes for 4.2.12 and 5.0.5 for backport of b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8.

comment:5 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In 2e6ae1e9:

[5.0.x] Refs #34900, Refs #35361 -- Fixed SafeMIMEText.set_payload() crash on Python 3.13.

Payloads with surrogates are passed to the set_payload() since
https://github.com/python/cpython/commit/f97f25ef5dfcdfec0d9a359fd970abd139cf3428

Backport of b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8 from main.

comment:6 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In 23d605d3:

[5.0.x] Fixed #35361 -- Added release notes for 4.2.12 and 5.0.5 for backport of b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8.

Backport of 42435fc55cbf7c04c1389ee46cc50e2565b40e37 from main.

comment:7 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In 0d3ddcaf:

[4.2.x] Refs #34900, Refs #35361 -- Fixed SafeMIMEText.set_payload() crash on Python 3.13.

Payloads with surrogates are passed to the set_payload() since
https://github.com/python/cpython/commit/f97f25ef5dfcdfec0d9a359fd970abd139cf3428

Backport of b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8 from main.

comment:8 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In 27c32cc:

[4.2.x] Fixed #35361 -- Added release notes for 4.2.12 for backport of b231bcd19e57267ce1fc21d42d46f0b65fdcfcf8.

Backport of 42435fc55cbf7c04c1389ee46cc50e2565b40e37 from main.

comment:9 by Sarah Boyce <42296566+sarahboyce@…>, 3 months ago

In cd823778:

Refs #35361 -- Clarified release notes for 4.2.12 and 5.0.5.

comment:10 by Sarah Boyce <42296566+sarahboyce@…>, 3 months ago

In a1baaddb:

[5.0.x] Refs #35361 -- Clarified release notes for 4.2.12 and 5.0.5.

Backport of cd823778e66307b82469858cfd8d1aa75613b49a from main.

comment:11 by Sarah Boyce <42296566+sarahboyce@…>, 3 months ago

In 1d85b416:

[4.2.x] Refs #35361 -- Clarified release notes for 4.2.12.

Backport of cd823778e66307b82469858cfd8d1aa75613b49a from main.

comment:12 by Anders Kaseorg, 3 months ago

A disadvantage of the current May 6 schedule for the release with these Python 3.11.9 and 3.12.3 compatibility fixes is that Ubuntu 24.04 LTS will be releasing with Python 3.12.3 on April 25.

Note: See TracTickets for help on using tickets.
Back to Top