#17471 closed New feature (fixed)
Add ability to use smtplib.SMTP_SSL connection when sending email
Reported by: | dje | Owned by: | nobody |
---|---|---|---|
Component: | Core (Mail) | Version: | dev |
Severity: | Normal | Keywords: | smtp sprints-django-ar |
Cc: | sorin, Simon Charette, senko | Triage Stage: | Accepted |
Has patch: | yes | Needs documentation: | yes |
Needs tests: | yes | Patch needs improvement: | no |
Easy pickings: | yes | UI/UX: | no |
Description
There was a ticket opened three years ago to report this issue and it was brushed off with no investigation: https://code.djangoproject.com/ticket/9575. I've attached my patch which completely resolved the problem on my local machine. The problem seems to arise when using TLS, with username/password and port 465. I didn't have any servers with a port other than 465 to test with, but the previous bug report mentioned changing the port worked for them.
To reproduce the issue, I'd recommend using Gmail's servers (since they are free):
EMAIL_HOST_USER = 'user'
EMAIL_HOST_PASSWORD = 'pass'
EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_PORT = 465
The servers I've tested it against are not just Gmail, so this is clearly not Google's issue. There may be an issue with the underlying Python SMTP library, but patching this in Django was the quickest way for me to get my project back up and running.
Attachments (5)
Change History (20)
by , 13 years ago
Attachment: | smtp_patch.txt added |
---|
comment:1 by , 13 years ago
Component: | Uncategorized → Core (Mail) |
---|---|
Patch needs improvement: | set |
Summary: | Using TLS and port 465 for SMTP email causing Django to hang sending. → Add ability to use smtplib.SMTP_SSL connection when sending email |
Type: | Bug → New feature |
Google's docs now say that port 465 is for SSL, while TLS/STARTTLS is 587 (they didn't used to be that specific as to which port was for what protocol). What Django implements, when EMAIL_USE_TLS is True, is starttls. So the proper port to use for Django speaking to GMail servers, today, given that Django implements TLS and not SSL, is 587, not 465.
The patch as presented can't be used in Django as is. It re-interprets Django's EMAIL_USE_TLS setting to mean "use an smtplib.SMTP_SSL
connection". Such a connection doesn't work for servers who implement TLS, not SSL. For example if you use the patched code against GMail's server port 587 you get an exception: SSLError: (1, '_ssl.c:503: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol'). So if we were to change the code as is done in the patch, suddenly everyone who had a working config talking to a TLS server would be broken.
It seems like what is needed to talk to an SSL-implementing server (like GMail port 465) is yet another option (sigh) to tell the email code to use an smtplib.SMTP_SSL
connection rather than doing the starttls thing. Note smtplib.SMTP_SSL
is new in Python 2.6 so we cannot support this new type of email security except when running under Python 2.6 (and higher).
Changing this to a feature request rather than a bug, and changing the summary to match. It's not a bug if things don't work correctly if they've been misconfigured, and telling Django to use tls against an email server who is expecting ssl and not tls is a misconfiguration. But what we are missing on the Djagno side is apparently any way to do the ssl connection instead of TLS, and if there are common servers that implement only SSL and not TLS then Django likely should support talking to them.
comment:2 by , 13 years ago
Thank you for the correction. I'll follow up with the providers I was having trouble with to see if their documentation is wrong. One such provider is Amazon's SES service, so they are becoming quite a common server (I personally am developing two applications using them). They indicate TLS, but it only works with the SSL connection.
comment:3 by , 13 years ago
Cc: | added |
---|---|
Has patch: | unset |
Type: | New feature → Bug |
So I've spent a while reading through the RFC specs on TLSv1 and discussing with the developers of a few of the providers I was having issues with and it turns out that this is actually a *bug* and not a new feature. TLSv1 has two operational modes: explicit and implicit when forming the connection/handshake. Django incorrectly assumes that all TLS connections are explicit connections. Explicit connections use STARTTLS and are generally accepted on port 587. Implicit connections (which may also fall back to SSL) are typically made over port 465 and should not use the STARTTLS command.
I'm changing this back to a bug as it is a misinterpretation of the spec and only half implemented.
comment:4 by , 13 years ago
Cc: | removed |
---|
comment:5 by , 13 years ago
Has patch: | set |
---|---|
Needs documentation: | set |
Needs tests: | set |
Triage Stage: | Unreviewed → Accepted |
I'll skip the bug vs. feature request debate; it doesn't really matter.
To sum up:
- Currently, Django supports only explicit connections (with STARTTLS);
- This ticket is about supporting implicit connections too;
- The change must be backwards compatible, which is likely to require yet another setting.
comment:6 by , 13 years ago
Type: | Bug → New feature |
---|
Yeah, sorry about switching it back to a bug. It probably should have been left as a feature. I've attached a patch that should maintain backwards compatibility. The only parts I was unsure about was adding use_ssl to the constructor and whether to call starttls if they set TLS and SSL to true. Technically, they are supposed to be mutually exclusive, which is how I coded it.
What's in the patch:
A new setting: EMAIL_USE_SSL
Updated documentation.
Added tests around USE_TLS and USE_SSL
by , 13 years ago
Attachment: | ssl_setting_patch.diff added |
---|
same as the same-named attachment with .txt extension, only with .diff as extension to allow for easy trac viewing
comment:7 by , 12 years ago
Cc: | added |
---|
comment:8 by , 12 years ago
I just send a pull request for this, this is the link https://github.com/django/django/pull/792
comment:9 by , 12 years ago
Keywords: | sprints-django-ar added |
---|
by , 12 years ago
Attachment: | patch_ticket_17471_b.txt added |
---|
comment:11 by , 12 years ago
During djangocon eu, did review of the patch and PR
please find a new rebase patch in attachment and a new PR
https://github.com/django/django/pull/1124
comment:12 by , 12 years ago
This is almost RFC, but on Python 3.2, I'm getting this output (but the test succeeds anyway):
.error: uncaptured python exception, closing channel <smtpd.SMTPChannel connected 127.0.0.1:47904 at 0x38db710> (<class 'UnicodeDecodeError'>:'utf-8' codec can't decode byte 0x99 in position 12: invalid start byte [/usr/lib/python3.2/asyncore.py|read|83] [/usr/lib/python3.2/asyncore.py|handle_read_event|449] [/usr/lib/python3.2/asynchat.py|handle_read|190] [/usr/lib/python3.2/smtpd.py|collect_incoming_data|278] [/home/claude/virtualenvs/djangogit32/lib/python3.2/encodings/utf_8.py|decode|16])
If this is reproducible on other systems, it would be nice if this could be silenced...
by , 11 years ago
Attachment: | patch_ticket_17471_c.txt added |
---|
Update to previous patch to suppress UnicodeDecodeError
comment:13 by , 11 years ago
Cc: | added |
---|---|
Patch needs improvement: | unset |
I've updated the previous patch to use a fake SMTPChannel which ignores the UnicodeDecodeError.
comment:14 by , 11 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
comment:15 by , 11 years ago
#13142 is a duplicate which contains some additional settings for the cert/key files.
svn diff patch