Support EMAIL_SSL_CERTFILE for private certificate authority

[dkaylor]

We have an SMTP server that is not signed by a public CA. Sending email with no SSL settings results in an "SSL: CERTIFICATE_VERIFY_FAILED" error.

If we set EMAIL_SSL_CERTFILE, we receive the same error. We do not have access to the key file to test with EMAIL_SSL_KEYFILE. Clients often do not have access to keys so this shouldn't be required.

Django is loading the cert files with load_cert_chain, but I believe load_verify_locations would be more appropriate:

​https://github.com/django/django/blob/main/django/core/mail/backends/smtp.py#L63

The examples in the Python docs use the former for servers and the latter for clients:

​https://docs.python.org/3/library/ssl.html

I wrote a simple test with load_cert_chain and it fails with the same SSL error:

ssl_context.load_cert_chain(cacert)

If I change to load_verify_locations it works

ssl_context.load_verify_locations(cacert)

[Mike Edmunds]

This makes sense to me, but Python's SSL/TLS is a little outside my expertise. It would be good to get Mariusz's input.

Django 4.2 changed to use ssl.create_default_context() if neither certfile nor keyfile is set. This enables certificate validation and hostname checking, and is a Python ​ssl security best practice.

I wonder if we shouldn't also be using ssl.create_default_context(capath=...) when an EMAIL_SSL_CERTFILE is provided, for exactly the same reasons? Followed by load_cert_chain() when necessary. (This would require a release note ​similar to the one in 4.2.)

See also ticket:34550 and ​this StackOverflow answer.

[Sarah Boyce]

​In the 4.2 release notes:

EmailBackend now verifies a hostname and certificates. If you need the previous behavior that is less restrictive and not recommended, subclass EmailBackend and override the ssl_context property.

Sounds like a duplicate of #34504

[Mike Edmunds]

I don't think this is a duplicate of #34504. That ticket wanted to disable hostname checking and certificate verification, which is indeed not recommended.

This ticket is trying to use a private certificate authority with hostname checking and certificate verification enabled. I'd think we'd want to encourage that when using a private CA. Django's SMTP EmailBackend makes that difficult right now, by requiring subclassing and overriding an undocumented property.

I would treat this as a feature request for Django's SMTP EmailBackend to support setting EMAIL_SSL_CERTIFICATE to a private CA or self-signed certificate, with all recommended security enabled.

If we don't want to do that, we should probably add some documentation along the lines of:

• EMAIL_SSL_CERTIFICATE is meant only for client authentication, and therefore must either include the private key or be used together with EMAIL_SSL_KEYFILE.
• To use a private certificate authority or self-signed certificate with your SMTP server, don't use Django's EMAIL_SSL_CERTIFICATE. Instead, add your private CA to your system's OpenSSL ca-path or set the SSL_CERT_FILE and/or SSL_CERT_DIR environment variables to point to it. (See Python's ​ssl.get_default_verify_paths().)

(Also, I might be misunderstanding, but it looks like when EMAIL_SSL_CERTIFICATE is set the ​SMTP backend creates a less-secure ​SSLContext with checking disabled. And I see we've actually ​documented that behavior.)

[Mike Edmunds]

(Also, since the docs for EMAIL_SSL_CERTIFICATE already say "certificate chain file," it seems reasonable to expect that could be a private CA bundle. Which would make this a bug.)

[Igor Scheller]

I added a ​PR to implement it as an additional feature (altho changing the current behaviour might be an option too)

[Igor Scheller]

As noted in the PR discussion, adding another option should be avoided in favour of the upcoming EMAIL_PROVIDERS setting, discussed in #35514.
The PR now reflects that change by allowing the setting to be set in the constructor and later from above providers config.

[Mike Edmunds]

This seems like a useful addition, given that:

• Internal private CAs are not all that exotic.
• Django's current documentation seems to suggest that EMAIL_SSL_CERTIFICATE can be set to a private CA bundle, but this doesn't actually work.
• Although the problem can be solved by subclassing smtp.EmailBackend to override ssl_context, that seems to be error prone. A lot of high-ranking solutions disable certificate checking entirely or introduce other security issues. (Another common recommendation is downgrading to Django 4.1.)

Question: am I understanding correctly that the proposed ssl_cafile option would also work to securely verify self-signed certs? (That seems like another semi-common Django email question that generates a lot of less-secure answers.)

[Igor Scheller]

Yes, it allows you to add a ​root-CA certificate / bundle of certificates which is used as the trust anchor, it can either be your own self-signed one or the one your organisation set up as a private CA.

[Mike Edmunds]

Since the last activity on this ticket:

• We've documented how to use self-signed certs and private CAs with Django's SMTP EmailBackend by updating the system CA bundle. (Added in ​Django 5.1 settings docs, expanded into its own ​section in 6.1.)
• We've implemented dictionary-based MAILERS configuration, which allows adding new EmailBackend options without needing new top-level settings.
• There's been some helpful discussion in the PR about why modifying the system CA bundle is not always possible or desirable and an SMTP EmailBackend option makes sense. (​https://github.com/django/django/pull/18456#issuecomment-2604487356 et seq.).

I think a reasonable next step would be updating the open PR for the Django 6.2dev codebase (taking advantage of MAILERS OPTIONS). Or if there's still controversy over this, bouncing it to the new-features process and consolidating all of the pros and cons there.