Opened 8 years ago

Closed 8 years ago

Last modified 5 years ago

#7747 closed (fixed)

Long email subject headers have tab character inserted

Reported by: mark.allison@… Owned by: Rami Kassab
Component: Core (Mail) Version: master
Severity: Keywords: aug22sprint
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

When sending email from Django long subject headers have tab characters inserted. This is a problem with the underlying email stdlib which has had recent discussion at
http://bugs.python.org/issue1974
and been moved to
http://www.mail-archive.com/email-sig@python.org/msg00198.html

Attachments (3)

tabs_in_subject_line_fix.diff (458 bytes) - added by mark.allison@… 8 years ago.
Workaround for tab insertion in long email subject lines
tabs_in_subject_line_fix.2.diff (452 bytes) - added by mark.allison@… 8 years ago.
Updated, slightly cleaner, workaround for tabs in email subject headers
long_email_subject_wrapping_fix_and_test.diff (2.1 KB) - added by mark.allison@… 8 years ago.
updated patch for compatibility with latest code and added test

Download all attachments as: .zip

Change History (14)

Changed 8 years ago by mark.allison@…

Workaround for tab insertion in long email subject lines

comment:1 Changed 8 years ago by Sung-jin Hong

Has patch: set
milestone: 1.0 beta
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

Perhaps adding some comments about the workaround might help. :)

Changed 8 years ago by mark.allison@…

Updated, slightly cleaner, workaround for tabs in email subject headers

comment:2 Changed 8 years ago by mark.allison@…

The workaround is to always use the email Header class to create the subject header for an email. The current code only does this if the header is non-ascii, otherwise, a string is used. The patch addresses this case.

When using a string a long subject line is folded with tabs. In (recent versions?) of Outlook the tabs are removed resulting in abutted words. In (recent versions?) of Tbird the tabs are rendered giving "wide" spaces between words. When using the header class, however, folding is done with a space and you seem to get the subject line looking as you'd expect.

If you read the discussions referenced above it seems the email stdlib is a bit broken. This patch just takes advantage of, what I think is, the happy accident that using the Header class seems to give the expected results (or, at least, better results than having tabs inserted in your subject line).

comment:3 Changed 8 years ago by Sung-jin Hong

milestone: 1.0 beta1.0

Looks good to me.

comment:4 Changed 8 years ago by Russell Keith-Magee

Needs tests: set
Patch needs improvement: set

I'm afraid I don't see how this patch helps. forbid_multi_line_headers() is only invoked as part of SafeMIMEText and SafeMIMEMultipart. SafeMIMEText is used on the message and on each attachment; SafeMIMEMultipart is used on the message of a multipart email. The subject of an email is assigned to the EmailMessage verbatim on line 219, and then put in the to message header verbatim on line 244. As far as I can make out, neither of these wrappers are used during subject handling.

A regression test case would be very helpful (regressiontests/mail would be the obvious location).

Changed 8 years ago by mark.allison@…

updated patch for compatibility with latest code and added test

comment:5 Changed 8 years ago by mark.allison@…

Needs tests: unset

I updated that patch and added a test to regressionstest/mail to show the problem.

The message method of EmailMessage creates an instance of SafeMIMEText which overrides the default setitem of MIMEText to always invoke forbid_multi_line_headers on header assignment. So, at line 244 (247 patched), the assignment of the subject will cause forbid_multi_line_headers() to be called and the patch will ensure that an ascii subject header is returned as an instance of the Header class instead of a unicode string. This, in turn, ensures that a space, not a tab, is used as the continuation character if the subject line is long and has to be folded over several lines (this is what the added test checks for).

Prefering a space over a tab means that Outlook and Thunderbird will show a long email subject correctly.

comment:6 Changed 8 years ago by Rami Kassab

Owner: changed from nobody to Rami Kassab

comment:7 Changed 8 years ago by Rami Kassab

Keywords: aug22sprint added
Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

Wrote a quick test app to confirm this issue. Raw email headers were as follows:

Before Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how
	this turned out in my email application. For some reason,
	tabs are inserted to fold the subject but we want spaces instead....
	don't we now?!
...

After Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how this
 turned out in my email application. For some reason,
 tabs are inserted to fold the subject but we want spaces instead.... don't
 we now?!
...

Also ran the regression test in the patch, which function as expected. I think this is ready for checkin

comment:8 Changed 8 years ago by Rami Kassab

Resolution: worksforme
Status: newclosed

comment:9 Changed 8 years ago by Rami Kassab

Resolution: worksforme
Status: closedreopened

comment:10 Changed 8 years ago by Russell Keith-Magee

Resolution: fixed
Status: reopenedclosed

(In [8483]) Fixed #7747: Altered EmailMessage such that messages with long subject lines don't use tabs in their continutation sequence. Tabs in subjects cause problems with Outlook and Thunderbird. Thanks to Mark Allison <mark.allison@…> for the report and fix.

comment:11 Changed 5 years ago by Jacob

milestone: 1.0

Milestone 1.0 deleted

Note: See TracTickets for help on using tickets.
Back to Top