Long email address causes crash when generating a message

[Alexandru Chirila]

Trying to generate an email message that has a long recipient address and non-ASCII characters causes a crash.

How to reproduce:

EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()

Looking for an existing issue I have found #31784. It seems like the issue there was only fixed for long names and long addresses.

---

Version info:

• Django: 5.0.6
• Python: 3.12.3

---

Full log:

In [2]: from django.core.mail import EmailMessage

In [3]: EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()
---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
File .venv/lib/python3.12/site-packages/django/core/mail/message.py:64, in forbid_multi_line_headers(name, val, encoding)
     63 try:
---> 64     val.encode("ascii")
     65 except UnicodeEncodeError:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:267, in EmailMessage.message(self)
    265 msg["Subject"] = self.subject
    266 msg["From"] = self.extra_headers.get("From", self.from_email)
--> 267 self._set_list_header_if_not_empty(msg, "To", self.to)
    268 self._set_list_header_if_not_empty(msg, "Cc", self.cc)
    269 self._set_list_header_if_not_empty(msg, "Reply-To", self.reply_to)

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:432, in EmailMessage._set_list_header_if_not_empty(self, msg, header, values)
    430 except KeyError:
    431     value = ", ".join(str(v) for v in values)
--> 432 msg[header] = value

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:165, in SafeMIMEText.__setitem__(self, name, val)
    164 def __setitem__(self, name, val):
--> 165     name, val = forbid_multi_line_headers(name, val, self.encoding)
    166     MIMEText.__setitem__(self, name, val)

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:67, in forbid_multi_line_headers(name, val, encoding)
     65 except UnicodeEncodeError:
     66     if name.lower() in ADDRESS_HEADERS:
---> 67         val = ", ".join(
     68             sanitize_address(addr, encoding) for addr in getaddresses((val,))
     69         )
     70     else:
     71         val = Header(val, encoding).encode()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:68, in <genexpr>(.0)
     65 except UnicodeEncodeError:
     66     if name.lower() in ADDRESS_HEADERS:
     67         val = ", ".join(
---> 68             sanitize_address(addr, encoding) for addr in getaddresses((val,))
     69         )
     70     else:
     71         val = Header(val, encoding).encode()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:120, in sanitize_address(addr, encoding)
    117     localpart = Header(localpart, encoding).encode()
    118 domain = punycode(domain)
--> 120 parsed_address = Address(username=localpart, domain=domain)
    121 return formataddr((nm, parsed_address.addr_spec))

File ~/.pyenv/versions/3.12.3/lib/python3.12/email/headerregistry.py:33, in Address.__init__(self, display_name, username, domain, addr_spec)
     31 inputs = ''.join(filter(None, (display_name, username, domain, addr_spec)))
     32 if '\r' in inputs or '\n' in inputs:
---> 33     raise ValueError("invalid arguments; address parts cannot contain CR or LF")
     35 # This clause with its potential 'raise' may only happen when an
     36 # application program creates an Address object using an addr_spec
     37 # keyword.  The email library code itself must always supply username
     38 # and domain.
     39 if addr_spec is not None:

ValueError: invalid arguments; address parts cannot contain CR or LF

[Sarah Boyce]

#31784 discussed and went against the idea updating the line length to 998 and so we would need to revisit that discussion if we are to accept the current PR.
From what I see, the arguments in #31784 still stand and this is not the approach to resolve this issue.

[Mike Edmunds]

[This issue would also be solved—without altering line length—by upgrading django.core.mail from legacy email.message.Message (policy=compat32) to modern email.message.EmailMessage (policy=default), and letting the modern email package handle header sanitization and folding. Though that's a much larger scope than this individual bug.]

import email.message
msg = email.message.EmailMessage()
msg["To"] = "ţēśţ." * 6 + "@example.com"
print(msg.as_bytes().decode("ascii"))
# To: =?utf-8?b?xaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMu?=
#  =?utf-8?b?xaPEk8WbxaM=?=.@example.com

[Florian Apolloner]

Replying to Mike Edmunds:

[This issue would also be solved—without altering line length—by upgrading django.core.mail from legacy email.message.Message (policy=compat32) to modern email.message.EmailMessage (policy=default), and letting the modern email package handle header sanitization and folding. Though that's a much larger scope than this individual bug.]

Would be worth to investigate how much effort that would be. I'd be in favor of staying in line with Python.

[Mike Edmunds]

Would be worth to investigate how much effort [upgrading to Python modern email API] would be.

Discussion in progress: ​https://groups.google.com/g/django-developers/c/2zf9GQtjdIk

[Clinton Christian]

Replying to Mike Edmunds:

Would be worth to investigate how much effort [upgrading to Python modern email API] would be.

Discussion in progress: ​https://groups.google.com/g/django-developers/c/2zf9GQtjdIk

Thanks, I was unfamiliar with this thread, and #31784.

My takeaway:

We are updating the django.core.mail to take advantage of changes made to python's 3.6+ email module, while ensuring backwards compatibility (as much as possible).

Methods in question from django.core.email:
CachedDnsName
DNS_NAME
EmailMessage
EmailMultiAlternatives
SafeMIMEText
SafeMIMEMultipart
DEFAULT_ATTACHMENT_MIME_TYPE
make_msgid
BadHeaderError
forbid_multi_line_headers
get_connection
send_mail
send_mass_mail
mail_admins
mail_managers

Deprecations:
SafeMIMEText
SafeMIMEMultipart
forbid_multi_line_headers
BadHeaderError

There are several underscore methods in EmailMessage and EmailMultiAlternatives. These will need to be removed.
Python's 3.6+ email.message API changes do alter default behavior. It seems like we would need to override these new defaults in order to ensure backwards compatibility.

Regression tests will need to be introduced to ensure that we aren't making (unintended) breaking changes.

Are my assumptions accurate? Let me know if I missed anything.

Also, has a ticket already been created for this? If not I can create one, and reference it here, as the increased scope warrants a dedicated ticket.

[Mike Edmunds]

Also, has a ticket already been created for this?

I just posted to django-developers a few days ago. Waiting to create a ticket until we get positive/negative votes there.

Agreed that this ticket is not the best place for the larger scope proposal. I just added another post to the django-developers thread with more details on the proposed change. I'd prefer to keep the discussion over on django-developers for now, so that it's all in one place.

[Mike Edmunds]

I'm closing this ticket as wontfix. Here's why:

Django does not actually support email addresses with non-ASCII localparts (the "username" in "username@domain"). It may seem like it tries to, but the code added in #25986 generates an invalid, undeliverable address.

#35713 will raise an error for all attempts to use non-ASCII characters in a localpart (of any length), making this ticket obsolete.

(Correctly supporting non-ASCII email addresses is a new feature request in #35714.)