#35378 closed Bug (invalid)
Incorrect folding of long address headers with special characters when using 7bit Content-Transfer-Encoding in EmailMessage
Reported by: | andres | Owned by: | |
---|---|---|---|
Component: | Core (Mail) | Version: | dev |
Severity: | Normal | Keywords: | email, compat32 |
Cc: | Florian Apolloner, Joachim Jablon, Mike Edmunds | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | yes |
Easy pickings: | no | UI/UX: | no |
Description
Hi!
I'm getting errors when trying to send email messages when the "to" header has non-ASCII chars.
The problem seems to happen when all these conditions are true at the same time:
- the readable name part of the recipient is big (e.g. "A very long and big name for this recipient" <to@…>)
- there is at least one non-ASCII char
- there is at least one "special char" (only seems to happen with comma or parenthesis)
- the special char isn't close to or between no-ASCII chars
Code example:
import email from django.core.mail import EmailMessage recipient = '"A véry long name with non-ASCII char and, comma" <to@example.com>' msg = EmailMessage(from_email='from@example.com', to=[recipient]).message() msg.policy = email.policy.default.clone(cte_type='7bit') print(msg.as_bytes())
That prints a bytes string with:
\nTo: A =?utf-8?q?v=C3=A9ry?= long name with non-ASCII char and, comma\n <to@example.com>\n
It has an unprotected comma, and the message is rejected by ESPs.
Using default policy fix this problem, but possibly causes other bad consequences:
\nTo: =?utf-8?q?A_v=C3=A9ry_long_name_with_non-ASCII_char_and=2C_comma?=\n <to@example.com>\n
More details here: https://github.com/anymail/django-anymail/issues/369
Change History (12)
comment:1 by , 7 months ago
Cc: | added |
---|---|
Summary: | Bad UTF-8 "To" header encoding in EmailMessage → Incorrect folding of long address headers with special characters when using 7bit Content-Transfer-Encoding in EmailMessage |
Triage Stage: | Unreviewed → Accepted |
Version: | 5.0 → dev |
comment:2 by , 7 months ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:4 by , 6 months ago
Thanks, Lufafa Joshua! I manually reproduced your fix here, did a quick test and it seems to work!
comment:5 by , 6 months ago
Patch needs improvement: | set |
---|
comment:6 by , 6 months ago
Has patch: | unset |
---|
comment:7 by , 6 months ago
Owner: | removed |
---|---|
Status: | assigned → new |
comment:8 by , 5 months ago
[I'm responsible for the example cited above. Setting an existing message's policy
attribute like that isn't documented anywhere, so probably shouldn't be considered a valid test case. Apologies for that.]
I've come to the conclusion that any attempt to use modern email.policy.default to serialize an email.message.Message (which is constructed with legacy email.policy.compat32) is prone to errors, and probably just isn't supported by the Python email package. Additional information in https://github.com/anymail/django-anymail/issues/369#issuecomment-2184252097
I would suggest not trying to fix this particular issue within Django. Long term, there would be a lot of advantages to upgrading django.core.mail to use modern email.message.EmailMessage (and email.policy.default). But that's a separate issue, and I think needs to happen all at once; trying to mix legacy and modern email code seems problematic.
Here's an updated test, using only documented Python email package features (generator policy override). It has the same problem, but I think clarifies this is more of a Python email bug than a Django problem:
import email.generator import email.policy import io import django.core.mail to = '"Người nhận a very very long, name" <to@example.com>' mime_message = django.core.mail.EmailMessage(to=[to]).message() fp = io.BytesIO() g = email.generator.BytesGenerator( fp, policy=email.policy.default.clone(cte_type="7bit"), ) g.flatten(mime_message) print(fp.getvalue().decode("ascii")) # [... other headers ...] # To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1u?= a very very long, name <to@example.com>
comment:9 by , 5 months ago
Cc: | added |
---|
comment:10 by , 5 months ago
Keywords: | email compat32 added |
---|
comment:11 by , 4 months ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
Thank you for the update Mike, updated the ticket accordingly
Also following the discussion of https://groups.google.com/u/2/g/django-developers/c/2zf9GQtjdIk which is nice to see 👍
Thank you for the report!
I can replicate and the discussion in https://github.com/anymail/django-anymail/issues/369 is also very useful, thank you.
Agree that #31784 is also somewhat related.