Opened 7 years ago

Closed 7 years ago

Last modified 4 years ago

#7747 closed (fixed)

Long email subject headers have tab character inserted

Reported by: mark.allison@… Owned by: ramikassab
Component: Core (Mail) Version: master
Severity: Keywords: aug22sprint
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

When sending email from Django long subject headers have tab characters inserted. This is a problem with the underlying email stdlib which has had recent discussion at
http://bugs.python.org/issue1974
and been moved to
http://www.mail-archive.com/email-sig@python.org/msg00198.html

Attachments (3)

tabs_in_subject_line_fix.diff (458 bytes) - added by mark.allison@… 7 years ago.
Workaround for tab insertion in long email subject lines
tabs_in_subject_line_fix.2.diff (452 bytes) - added by mark.allison@… 7 years ago.
Updated, slightly cleaner, workaround for tabs in email subject headers
long_email_subject_wrapping_fix_and_test.diff (2.1 KB) - added by mark.allison@… 7 years ago.
updated patch for compatibility with latest code and added test

Download all attachments as: .zip

Change History (14)

Changed 7 years ago by mark.allison@…

Workaround for tab insertion in long email subject lines

comment:1 Changed 7 years ago by serialx

  • Has patch set
  • milestone set to 1.0 beta
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

Perhaps adding some comments about the workaround might help. :)

Changed 7 years ago by mark.allison@…

Updated, slightly cleaner, workaround for tabs in email subject headers

comment:2 Changed 7 years ago by mark.allison@…

The workaround is to always use the email Header class to create the subject header for an email. The current code only does this if the header is non-ascii, otherwise, a string is used. The patch addresses this case.

When using a string a long subject line is folded with tabs. In (recent versions?) of Outlook the tabs are removed resulting in abutted words. In (recent versions?) of Tbird the tabs are rendered giving "wide" spaces between words. When using the header class, however, folding is done with a space and you seem to get the subject line looking as you'd expect.

If you read the discussions referenced above it seems the email stdlib is a bit broken. This patch just takes advantage of, what I think is, the happy accident that using the Header class seems to give the expected results (or, at least, better results than having tabs inserted in your subject line).

comment:3 Changed 7 years ago by serialx

  • milestone changed from 1.0 beta to 1.0

Looks good to me.

comment:4 Changed 7 years ago by russellm

  • Needs tests set
  • Patch needs improvement set

I'm afraid I don't see how this patch helps. forbid_multi_line_headers() is only invoked as part of SafeMIMEText and SafeMIMEMultipart. SafeMIMEText is used on the message and on each attachment; SafeMIMEMultipart is used on the message of a multipart email. The subject of an email is assigned to the EmailMessage verbatim on line 219, and then put in the to message header verbatim on line 244. As far as I can make out, neither of these wrappers are used during subject handling.

A regression test case would be very helpful (regressiontests/mail would be the obvious location).

Changed 7 years ago by mark.allison@…

updated patch for compatibility with latest code and added test

comment:5 Changed 7 years ago by mark.allison@…

  • Needs tests unset

I updated that patch and added a test to regressionstest/mail to show the problem.

The message method of EmailMessage creates an instance of SafeMIMEText which overrides the default setitem of MIMEText to always invoke forbid_multi_line_headers on header assignment. So, at line 244 (247 patched), the assignment of the subject will cause forbid_multi_line_headers() to be called and the patch will ensure that an ascii subject header is returned as an instance of the Header class instead of a unicode string. This, in turn, ensures that a space, not a tab, is used as the continuation character if the subject line is long and has to be folded over several lines (this is what the added test checks for).

Prefering a space over a tab means that Outlook and Thunderbird will show a long email subject correctly.

comment:6 Changed 7 years ago by ramikassab

  • Owner changed from nobody to ramikassab

comment:7 Changed 7 years ago by ramikassab

  • Keywords aug22sprint added
  • Patch needs improvement unset
  • Triage Stage changed from Accepted to Ready for checkin

Wrote a quick test app to confirm this issue. Raw email headers were as follows:

Before Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how
	this turned out in my email application. For some reason,
	tabs are inserted to fold the subject but we want spaces instead....
	don't we now?!
...

After Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how this
 turned out in my email application. For some reason,
 tabs are inserted to fold the subject but we want spaces instead.... don't
 we now?!
...

Also ran the regression test in the patch, which function as expected. I think this is ready for checkin

comment:8 Changed 7 years ago by ramikassab

  • Resolution set to worksforme
  • Status changed from new to closed

comment:9 Changed 7 years ago by ramikassab

  • Resolution worksforme deleted
  • Status changed from closed to reopened

comment:10 Changed 7 years ago by russellm

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [8483]) Fixed #7747: Altered EmailMessage such that messages with long subject lines don't use tabs in their continutation sequence. Tabs in subjects cause problems with Outlook and Thunderbird. Thanks to Mark Allison <mark.allison@…> for the report and fix.

comment:11 Changed 4 years ago by jacob

  • milestone 1.0 deleted

Milestone 1.0 deleted

Note: See TracTickets for help on using tickets.
Back to Top