Django

Code

Ticket #7747 (closed: fixed)

Opened 5 months ago

Last modified 3 months ago

Long email subject headers have tab character inserted

Reported by: mark.allison@maplecroft.com Assigned to: ramikassab
Milestone: 1.0 Component: django.core.mail
Version: SVN Keywords: aug22sprint
Cc: Triage Stage: Ready for checkin
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

When sending email from Django long subject headers have tab characters inserted. This is a problem with the underlying email stdlib which has had recent discussion at http://bugs.python.org/issue1974 and been moved to http://www.mail-archive.com/email-sig@python.org/msg00198.html

Attachments

tabs_in_subject_line_fix.diff (458 bytes) - added by mark.allison@maplecroft.com on 07/14/08 07:09:12.
Workaround for tab insertion in long email subject lines
tabs_in_subject_line_fix.2.diff (452 bytes) - added by mark.allison@maplecroft.com on 07/16/08 04:35:45.
Updated, slightly cleaner, workaround for tabs in email subject headers
long_email_subject_wrapping_fix_and_test.diff (2.1 kB) - added by mark.allison@maplecroft.com on 08/18/08 06:07:25.
updated patch for compatibility with latest code and added test

Change History

07/14/08 07:09:12 changed by mark.allison@maplecroft.com

  • attachment tabs_in_subject_line_fix.diff added.

Workaround for tab insertion in long email subject lines

07/15/08 21:10:21 changed by serialx

  • needs_better_patch changed.
  • needs_tests changed.
  • milestone set to 1.0 beta.
  • needs_docs changed.
  • has_patch set to 1.
  • stage changed from Unreviewed to Accepted.

Perhaps adding some comments about the workaround might help. :)

07/16/08 04:35:45 changed by mark.allison@maplecroft.com

  • attachment tabs_in_subject_line_fix.2.diff added.

Updated, slightly cleaner, workaround for tabs in email subject headers

07/16/08 05:00:36 changed by mark.allison@maplecroft.com

The workaround is to always use the email Header class to create the subject header for an email. The current code only does this if the header is non-ascii, otherwise, a string is used. The patch addresses this case.

When using a string a long subject line is folded with tabs. In (recent versions?) of Outlook the tabs are removed resulting in abutted words. In (recent versions?) of Tbird the tabs are rendered giving "wide" spaces between words. When using the header class, however, folding is done with a space and you seem to get the subject line looking as you'd expect.

If you read the discussions referenced above it seems the email stdlib is a bit broken. This patch just takes advantage of, what I think is, the happy accident that using the Header class seems to give the expected results (or, at least, better results than having tabs inserted in your subject line).

07/16/08 05:39:48 changed by serialx

  • milestone changed from 1.0 beta to 1.0.

Looks good to me.

08/15/08 22:31:10 changed by russellm

  • needs_better_patch set to 1.
  • needs_tests set to 1.

I'm afraid I don't see how this patch helps. forbid_multi_line_headers() is only invoked as part of SafeMIMEText and SafeMIMEMultipart. SafeMIMEText is used on the message and on each attachment; SafeMIMEMultipart is used on the message of a multipart email. The subject of an email is assigned to the EmailMessage? verbatim on line 219, and then put in the to message header verbatim on line 244. As far as I can make out, neither of these wrappers are used during subject handling.

A regression test case would be very helpful (regressiontests/mail would be the obvious location).

08/18/08 06:07:25 changed by mark.allison@maplecroft.com

  • attachment long_email_subject_wrapping_fix_and_test.diff added.

updated patch for compatibility with latest code and added test

08/18/08 06:29:38 changed by mark.allison@maplecroft.com

  • needs_tests deleted.

I updated that patch and added a test to regressionstest/mail to show the problem.

The message method of EmailMessage? creates an instance of SafeMIMEText which overrides the default setitem of MIMEText to always invoke forbid_multi_line_headers on header assignment. So, at line 244 (247 patched), the assignment of the subject will cause forbid_multi_line_headers() to be called and the patch will ensure that an ascii subject header is returned as an instance of the Header class instead of a unicode string. This, in turn, ensures that a space, not a tab, is used as the continuation character if the subject line is long and has to be folded over several lines (this is what the added test checks for).

Prefering a space over a tab means that Outlook and Thunderbird will show a long email subject correctly.

08/22/08 15:39:28 changed by ramikassab

  • owner changed from nobody to ramikassab.

08/22/08 16:25:52 changed by ramikassab

  • keywords set to aug22sprint.
  • needs_better_patch deleted.
  • stage changed from Accepted to Ready for checkin.

Wrote a quick test app to confirm this issue. Raw email headers were as follows:

Before Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how
	this turned out in my email application. For some reason,
	tabs are inserted to fold the subject but we want spaces instead....
	don't we now?!
...

After Patch:

...
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: This is a really long subject and I just wanted to test to see how this
 turned out in my email application. For some reason,
 tabs are inserted to fold the subject but we want spaces instead.... don't
 we now?!
...

Also ran the regression test in the patch, which function as expected. I think this is ready for checkin

08/22/08 16:33:42 changed by ramikassab

  • status changed from new to closed.
  • resolution set to worksforme.

08/22/08 16:34:48 changed by ramikassab

  • status changed from closed to reopened.
  • resolution deleted.

08/23/08 08:31:29 changed by russellm

  • status changed from reopened to closed.
  • resolution set to fixed.

(In [8483]) Fixed #7747: Altered EmailMessage? such that messages with long subject lines don't use tabs in their continutation sequence. Tabs in subjects cause problems with Outlook and Thunderbird. Thanks to Mark Allison <mark.allison@maplecroft.com> for the report and fix.


Add/Change #7747 (Long email subject headers have tab character inserted)




Change Properties
Action