Code

Opened 8 years ago

Closed 7 years ago

#1541 closed defect (fixed)

Add multipart message capability to django.core.mail

Reported by: bruce@… Owned by: mtredinnick
Component: Core (Mail) Version:
Severity: normal Keywords:
Cc: rushman@…, ross@…, mssnlayam@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: UI/UX:

Description

I wanted to be able to use the integrated mail functionality to send multipart messages. However, in Python 2.4.1, you cannot simply give send_mail a MIMEMultipart message. It errors, trying to encode. The encoding step is really not needed if you've already constructed a MIMEText or MIMEMultipart message. So, I went ahead and added that functionality by making a "SafeMIMEMultipart" class. If the message parameter is either SafeMIMEMultipart or SafeMIMEtext, it is accepted without re-encoding. Another way to do this would've been to put a "skip_reencode" parameter on the method, but I prefer more transparent solutions.

Eample usage:


msg = mail.SafeMIMEMultipart('alternative', charset="utf-8")
msg.attach(mail.SafeMIMEText("example text part", "text"))
msg.attach(mail.SafeMIMEText("<p>example <b>html</b> part", "html"))
mail.send_mail("example multipart mail", msg, "me@…", ["you@…",])

Attachments (8)

multipart_mail.diff (2.1 KB) - added by bruce@… 8 years ago.
mail.py patch
mail_attachment.diff (5.8 KB) - added by mssnlayam@… 7 years ago.
mail_attachment_as_str.diff (5.3 KB) - added by mssnlayam@… 7 years ago.
Similar to mail_attachment.diff, but accepts str objects instread of file objects
1541_multipart_unified.diff (3.5 KB) - added by sime <simon@…> 7 years ago.
unified #1541 and #3366 - needs improvement to handle other mimetypes
mail.diff (5.2 KB) - added by nick.lane.au@… 7 years ago.
Patch to add attachments to django.core.mail
mail.2.diff (5.6 KB) - added by nick.lane.au@… 7 years ago.
Slight change to patch, added docs
mail.3.diff (5.6 KB) - added by nick.lane.au@… 7 years ago.
Slight change to patch, added docs (last patch broken)
mail.4.diff (5.8 KB) - added by nick.lane.au@… 7 years ago.
Slight change to patch, added docs (last patch broken)

Download all attachments as: .zip

Change History (44)

Changed 8 years ago by bruce@…

mail.py patch

comment:1 Changed 8 years ago by bruce@…

Sorry, forgot to hit the "code block" button. Here's try two for that example code.

msg = mail.SafeMIMEMultipart('alternative', charset="utf-8") 
msg.attach(mail.SafeMIMEText("example text part", "text"))
msg.attach(mail.SafeMIMEText("<p>example <b>html</b> part", "html")) 
mail.send_mail("example multipart mail", msg, "me@example.com", ["you@example.com",])

comment:2 Changed 8 years ago by bruce@…

  • Summary changed from Add multipart message capability to django.core.mail to [patch] Add multipart message capability to django.core.mail

comment:3 Changed 8 years ago by rushman@…

Maybe someone can integrate this patch into magic-removal branch?

comment:4 Changed 8 years ago by Helga

  • Type changed from enhancement to defect

comment:5 Changed 8 years ago by test@…

  • Cc qhkpnvflz, nildwaq added
  • Keywords qhkpnvflz nildwaq added; smtp, MIMEMultipart removed
  • Summary changed from [patch] Add multipart message capability to django.core.mail to qhkpnvflz nildwaq

uctqpdosy lrcnde yudgorpah yjaqpcxv toujymqd njqpgsl xudpbf

comment:6 Changed 8 years ago by adrian

  • Cc qhkpnvflz, nildwaq removed
  • Keywords qhkpnvflz nildwaq removed
  • Summary changed from qhkpnvflz nildwaq to [patch] Add multipart message capability to django.core.mail
  • Version 0.91 deleted

comment:7 Changed 8 years ago by Sergey Kirillov <rushman@…>

  • Cc rushman@… added

comment:8 Changed 8 years ago by anonymous

  • Cc ross@… added

comment:9 Changed 8 years ago by jacob

  • Owner changed from adrian to jacob
  • Status changed from new to assigned

This is great, and I want to check it in. However, it needs docs added to mail.txt first.

comment:11 Changed 7 years ago by Sergey <rushman@…>

Does anybody knows when this patch will be integrated into trunk?

comment:12 Changed 7 years ago by adrian

  • Component changed from Core framework to django.core.mail

comment:13 Changed 7 years ago by mssnlayam@…

  • Cc mssnlayam@… added

I am attaching a patch that adds multipart message capability. This is slightly different from multipart_mail.diff, in that Django creates the individual mime parts and attaches them.

Example

    attachments = [
        ('image.png', open('/home/user/image.png', 'rb')),
        ('audio.mp3', open('/home/user/audio.mp3', 'rb')),
    ]
    send_mail(subject, body, sender, recipients, attachments=attachments)

Changed 7 years ago by mssnlayam@…

comment:14 Changed 7 years ago by mssnlayam@…

Forgot to add that send_mass_mail() might not work as expected, when attachments is reused for multiple datatuples.

Changed 7 years ago by mssnlayam@…

Similar to mail_attachment.diff, but accepts str objects instread of file objects

comment:15 Changed 7 years ago by russellm

#3307 has a competing version of this feature, along with some other details.

comment:16 Changed 7 years ago by Gary Wilson <gary.wilson@…>

  • Triage Stage changed from Unreviewed to Accepted

Replying to ubernostrum from #3307:

Hmm. I thought I'd searched on this previously and not found anything, but I guess I was wrong. #1541 has a safer implementation of the multipart stuff.

comment:17 follow-up: Changed 7 years ago by Gary Wilson <gary.wilson@…>

Can someone please test the patch. Also with adding all these new features to an email here and in #3307, it seems like the datatuple is becoming smelly. Perhaps some sort of EmailMessage class storing all these different data pieced as attributes?

comment:18 in reply to: ↑ 17 Changed 7 years ago by Gary Wilson <gary.wilson@…>

Replying to Gary Wilson <gary.wilson@gmail.com>:

Perhaps some sort of EmailMessage class storing all these different data pieced as attributes?

Created #3366.

comment:19 Changed 7 years ago by mssnlayam@…

I would like the third patch to be applied and this bug closed. We can start working on #3366, but that should not keep this bug waiting.

comment:20 Changed 7 years ago by Gary Wilson <gary.wilson@…>

  • Needs documentation set

Also, maybe the Python bug mentioned in #3104 should be added as a note in the documentation of this new feature.

comment:21 Changed 7 years ago by Gary Wilson <gary.wilson@…>

  • Needs documentation unset

Actually, I guess #3104 is only for multipart form uploads, sorry.

comment:22 Changed 7 years ago by Gary Wilson <gary.wilson@…>

  • Triage Stage changed from Accepted to Ready for checkin

Jacob mentioned that the only thing holding this back was docs, and docs are now here.

comment:23 follow-up: Changed 7 years ago by russellm

  • Triage Stage changed from Ready for checkin to Design decision needed

This ticket contains three competing implementations of the same feature, with another option on #3307. The version that was initially accepted by Jacob still isn't documented. There needs to be some public discussion on which option to adopt, and how that choice integrates with #3366.

comment:24 in reply to: ↑ 23 Changed 7 years ago by mssnlayam@…

Replying to russellm:

This ticket contains three competing implementations of the same feature, with another option on #3307. The version that was initially accepted by Jacob still isn't documented. There needs to be some public discussion on which option to adopt, and how that choice integrates with #3366.

Mails to the developers mailing list have not induced a discussion. What should be the plan of action if there is no or insufficient response?

comment:25 follow-up: Changed 7 years ago by bruce@…

I am the original author of this patch, which has now grown into a tangle. I'd be willing to consolidate the various patches, as I don't think they are really conflicting all that much.

The combined version would:

1) Use the idea from #3366, making an EmailMessage class, which is really just an extension of the "SafeMimeMultipart" I made in the original patch.

2) Add the bcc_list idea from #3307

3) Add the convenience method for adding a list of attachments at once.

How does that sound?

comment:26 Changed 7 years ago by russellm

@bruce

Sounds like a good approach to me. Show me the code! :-)

comment:27 in reply to: ↑ 25 ; follow-up: Changed 7 years ago by anonymous

Replying to bruce@coderseye.com:

How does that sound?

Sounds good. I have a few concerns/comments. I prefer an approach where I do not have to know what SafeMIMEMultipart is. It is not clear how the EmailMessage class will maintain backward compatibility. We will be spending a lot of time accepting an interface and implementation for EmailMessage, and have to disturb everyone when that patch is applied. Not many people are interested, this is not newforms ;) I'd recommend applying patch 3 now and after that, working on #3366.

comment:28 in reply to: ↑ 27 Changed 7 years ago by Gary Wilson <gary.wilson@…>

Replying to anonymous:

It is not clear how the EmailMessage class will maintain backward compatibility.

Well, it's not completely necessary to maintain backward compatibility, as it's not guaranteed until 1.0.

We will be spending a lot of time accepting an interface and implementation for EmailMessage, and have to disturb everyone when that patch is applied. Not many people are interested, this is not newforms ;) I'd recommend applying patch 3 now and after that, working on #3366.

The patch for this ticket maintains backwards compatibility since it allows for a variable length datatuple, but what happens when it's time to add the features from other tickets? The interface would probably have to be changed anyway. If the interface is fixed first (with EmailMessage in #3366?), then all these features from the various tickets could be added afterwards (separately or together) without breaking anything.

comment:29 Changed 7 years ago by sime <simon@…>

  • Needs documentation set
  • Needs tests set
  • Patch needs improvement set
  • Triage Stage changed from Design decision needed to Unreviewed

Well folks here is a quick mash of Bruce's #1541 and Gary Wilson's #3366 patch, against rev 5468. Plus a convenience parameter for sending HTML.

I think all we need now, is to make it handle images and other mime types transparently too. I haven't played with that stuff in python before and have no immediate need right now - anyone else keen?

Changed 7 years ago by sime <simon@…>

unified #1541 and #3366 - needs improvement to handle other mimetypes

comment:30 Changed 7 years ago by lukeplant

Using the new EmailMessage system, we can already implement attachments without much duplication and without patching the source. I've had to do it just now, and below is the code I came up with (based on the code in the patch). Only EmailWithAttachments.message() has any significant duplication of core code.

I'm not sure what further work on changing the EmailMessage class and send_mail function is planned. In case nothing further is done, or in case it takes a while, the code below may help other people. (NB, I haven't tested it massively, it works for my usage).

from django.core.mail import EmailMessage, SMTPConnection, SafeMIMEText, Header, BadHeaderError, formatdate, make_msgid
from email import Encoders
from email.MIMEBase import MIMEBase
from email.MIMEMultipart import MIMEMultipart
from django.conf import settings

"""
Utilities for sending email with attachments
"""

class SafeMIMEMultipart(MIMEMultipart): 
    def __setitem__(self, name, val): 
        "Forbids multi-line headers, to prevent header injection." 
        if '\n' in val or '\r' in val: 
            raise BadHeaderError, "Header values can't contain newlines (got %r for header %r)" % (val, name) 
        if name == "Subject": 
            val = Header(val, settings.DEFAULT_CHARSET) 
        MIMEMultipart.__setitem__(self, name, val) 
 
    def attachFile(self, filename, content, mimetype): 
        maintype, subtype = mimetype.split('/', 1) 
        msg = MIMEBase(maintype, subtype) 
        msg.set_payload(content) 
        Encoders.encode_base64(msg) 
        msg.add_header('Content-Disposition', 'attachment', filename=filename) 
        MIMEMultipart.attach(self, msg) 

class EmailWithAttachments(EmailMessage):
    def __init__(self, *args, **kwargs):
        attachments = kwargs.pop('attachments', None)
        super(EmailWithAttachments, self).__init__(*args, **kwargs)
        self.attachments = attachments

    def message(self):
        simple_msg = SafeMIMEText(self.body, 'plain', settings.DEFAULT_CHARSET)
        if self.attachments:
            # This is a multipart mail
            msg = SafeMIMEMultipart()
            # First the body
            msg.attach(simple_msg)
            # Then the various files to be attached.
            for (filename, content, mimetype) in self.attachments:
                msg.attachFile(filename, content, mimetype)
        else:
            msg = simple_msg
        
        msg['Subject'] = self.subject
        msg['From'] = self.from_email
        msg['To'] = ', '.join(self.to)
        msg['Date'] = formatdate()
        msg['Message-ID'] = make_msgid()
        if self.bcc:
            msg['Bcc'] = ', '.join(self.bcc)
        return msg

def send_mail_with_attachments(subject, message, from_email, recipient_list, 
                              fail_silently=False, auth_user=None, auth_password=None, attachments=None): 
    connection = SMTPConnection(username=auth_user, password=auth_password,
                                 fail_silently=fail_silently)
    return EmailWithAttachments(subject, message, from_email, recipient_list,
                                 connection=connection, attachments=attachments).send()

comment:31 Changed 7 years ago by sime <simon@…>

Looks good, just need to get this stuff into core. Multipart, file attachments, transparent HTML, and possibly even a convenience function for template/context merging.

Changed 7 years ago by nick.lane.au@…

Patch to add attachments to django.core.mail

comment:32 Changed 7 years ago by nick.lane.au@…

I have a similar patch, which I improved a bit using lukeplant's example, which allows you to add attachments to a EmailMessage object. I also added a render_to_mail shortcut:

def render_to_mail(*args, **kwargs):
    return EmailMessage(body=loader.render_to_string(*args, **kwargs))

So my typical usage is:

m = render_to_mail('emails/foo.html', {'var': 'bar'})
m.attach_file('files/some_file.pdf')
m.subject = 'Here is your file'
m.to = ['nick.lane.au@gmail.com']
m.send()

comment:33 Changed 7 years ago by SmileyChris

  • Triage Stage changed from Unreviewed to Design decision needed

Changed 7 years ago by nick.lane.au@…

Slight change to patch, added docs

Changed 7 years ago by nick.lane.au@…

Slight change to patch, added docs (last patch broken)

Changed 7 years ago by nick.lane.au@…

Slight change to patch, added docs (last patch broken)

comment:34 Changed 7 years ago by nick.lane.au@…

Sorry about the bad patches... I swore the last one would work. I'll have to wait a week till I'm back at home and can do a proper patch against SVN. I can also add in the tests I have and make sure they work against SVN too.

Feel free to remove mail.2.diff -> mail.4.diff.

comment:35 Changed 7 years ago by mtredinnick

  • Needs documentation unset
  • Owner changed from jacob to mtredinnick
  • Patch needs improvement unset
  • Status changed from assigned to new
  • Summary changed from [patch] Add multipart message capability to django.core.mail to Add multipart message capability to django.core.mail
  • Triage Stage changed from Design decision needed to Accepted

No need to worry about updating the patches any further, Nick. I'm partway through merging them into trunk. Using most of your design, with only a few small tweaks. Should have something committed in the next day or two.

comment:36 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [5547]) Fixed #1541 -- Added ability to create multipart email messages. Thanks, Nick
Lane.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.