Make EmailValidator and URLValidator IDNA 2008 compliant

[j-bernard]

This ticket is the second of a list of tickets aiming at bringing Email Address Internationalization (EAI) compliance to Django by supporting International Domain Name (IDN) with regards to the latest standard (IDNA 2008) and fixing some processing on internationalized domains or email addresses.
Previous ticket: #33967

Domain validation is not fully compliant with IDNA 2008 (either in EmailValidator or UrlValidator) as defined in ​RFC5891 section-4.2

A domain name cannot be validated properly with a regex, therefore, an IDN validation should be performed with an appropriate library.

The current validation ignores IDNA errors. Instead, IDNA should be used for domain validation and the regex validation should be skipped for domains as it may lack some specific rules and then end up with invalid domains being accepted.

Moreover, the current validation is made by performing a conversion to A-Label with the Python encodings.idna module which implements a deprecated standard (IDNA 2003).

This conversion should be made IDNA 2008 compliant. The most used Python IDNA 2008 package is ​idna, which is among the most downloaded Python packages according to PyPI (4th as for the current month) and referred in the ​official Python documentation.

[Mariusz Felisiak]

Thanks for this ticket, however adding a new dependency is always controversial and it isn't a light decision so a strong consensus on the mailing list is required. Please first start a discussion on the DevelopersMailingList, where you'll reach a wider audience and see what other think, and ​follow the guidelines with regards to requesting features.

Personally, I don't think it's worth complexity. My initial response would be similar to the Python's, i.e. "If you need the IDNA 2008 standard from RFC 5891 and RFC 5895, use a third-party validator".

[j-bernard]

Thanks, I started the discussion on the ​mailing list

[Mike Edmunds]

If this gets reconsidered in the future, it will need to address potential security issues in changing how django.core.mail encodes recipient domains. As of July 2024, using IDNA 2003 for sending email (not IDNA 2008) still seems to be the correct choice—or at least, matches what Gmail and Microsoft's Outlook.com do. Details in ​https://github.com/django/django/pull/16276#issuecomment-2227512278.

[Mike Edmunds]

I've reopened the EmailValidator part of this with more details as #36014. The URLValidator part became moot in Django 1.8, and the (no longer active) call to punycode() was removed in #36007.