Opened 5 years ago

Closed 19 months ago

Last modified 19 months ago

#12422 closed Bug (fixed)

Django overrides the default email charset behaviour for utf-8

Reported by: simonb Owned by: ramiro
Component: Core (Mail) Version: master
Severity: Normal Keywords: charset utf-8
Cc: bnomis@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by aaugustin)

# Don't BASE64-encode UTF-8 messages so that we avoid unwanted attention from
# some spam filters.
Charset.add_charset('utf-8', Charset.SHORTEST, Charset.QP, 'utf-8')

The above bit of code in django.core.mail overrides the standard Python behaviour for utf-8 in the Python email module. It would be better to set the charset by calling set_charset() on the Message before sending if needed rather than a global override. This behaviour and assumption is not documented anywhere in Django as far as I can tell. I think this is a case of the framework being a bit too clever. Not everyone using Django is using the django email sending methods nor wants Django to (silently) change the system defaults.

For example, I use the standard Python email sending methods and was expecting the Python lib to behave as advertised but the encoding of my utf-8 messages kept coming out quoted-printable because of the override above.

Attachments (1)

12422-1.diff (6.0 KB) - added by claudep 2 years ago.
Do not change python global CHARSETS

Download all attachments as: .zip

Change History (16)

comment:1 Changed 5 years ago by russellm

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 5 years ago by simonb

  • Cc bnomis@… added

comment:3 Changed 4 years ago by mattmcc

  • Severity set to Normal
  • Type set to Bug

comment:4 Changed 3 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:5 Changed 3 years ago by aaugustin

  • Easy pickings unset

Change Easy pickings from NULL to False.

comment:6 Changed 3 years ago by aaugustin

  • Description modified (diff)

comment:7 Changed 3 years ago by aaugustin

This also caused me a lot of trouble when I was writing a patch for #18916.

Changed 2 years ago by claudep

Do not change python global CHARSETS

comment:8 Changed 2 years ago by claudep

  • Has patch set

Patch attached. Tests changes are only the different order of headers, nothing significant.

comment:9 Changed 21 months ago by anonymous

Tests pass. I made a PR to make it easier to merge into master. https://github.com/django/django/pull/1376/files

comment:10 Changed 21 months ago by susan

  • Owner changed from nobody to susan
  • Status changed from new to assigned

comment:11 Changed 20 months ago by ptone

  • Patch needs improvement set
  • Version changed from 1.1 to master

Unfortunately the email tests have multiple failures under Python3 with this patch applied - I did not investigate the causes.

comment:12 Changed 20 months ago by ramiro

  • Owner changed from susan to ramiro

Actually, the proposed patch tests fail with Python 3.3. They pass for 3.2.x (and 2.7..x).

The pre-3.3.0 Python commit where things break is:

changeset:   75893:9ceac471bd8c
user:        R David Murray <rdmurray@bitdance.com>
date:        Thu Mar 22 22:40:44 2012 -0400
summary:     #14380: Make actual default match docs, fix __init__ order.

See:

Thank you very much to all individuals that have worked on this so far. I have a fix ready, will open a new PR soon and post a link here so you can review it.

Last edited 20 months ago by ramiro (previous) (diff)

comment:14 Changed 19 months ago by Ramiro Morales <cramm0@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In ececbe77ff573707d8f25084018e66ee07f820fd:

Fixed #12422 -- Don't override global email charset behavior for utf-8.

Thanks simonb for the report, Claude Paroz and Susan Tan for their work
on a fix.

comment:15 Changed 19 months ago by Andrew Godwin <andrew@…>

In 8e571e5f8f21d87ab5a5462730289c755b8022d3:

Fixed #12422 -- Don't override global email charset behavior for utf-8.

Thanks simonb for the report, Claude Paroz and Susan Tan for their work
on a fix.

Note: See TracTickets for help on using tickets.
Back to Top